# Tips for Gentoo in a production environment

## Klavs

Hi guys,

I just read a post about Gentoo in a production environment and wanted to give you my idea of how that could best be achieved - following my own experience as sysadmin for many years  :Smile: 

If I ran Gentoo on many production servers, I would have a compile server - that only functioned as a compile and test environment (you could use an UML/chroot/vserver if short on spare servers  :Smile: 

and that should be set to ALWAYS build packages.

I would then NEVER install anything but packages built on that box on my servers.

This way you ensure that every server is alike - and this gives great stability - and you could f.ex. use stuff like sisuite.org or cfengine to maintain your servers and add File IDS easily.

Only difference would be that your compile server would have every server prog you use on it - and your prod servers most likely won't.

Or perhaps you should spend the few extra MB's on having every production program needed on every box - and then f.ex. nfs-mount the bigger data-files (or if space is no problem - mirror them on every box too).

This way, you could easily startup a service on another box - should you get a hardware problem. It could even be done automagically by your surveilance system (such as BigBrother, Nagios etc.)  :Wink: 

I have a cool idea of how to do that - with UML or vservers/jail(on BSD) over NFS - imagine the rest - or just ask and I'll elaborate  :Smile: 

Also I would strongly consider disabling GCC and friends on my production servers - for several reasons:

1) then you won't accidently compile something on a prod-server.

2) if you should get rooted - some (how many?) rootkits compile their tools on the server (because there are so many different hosts/libraries) - and they won't work without gcc and friends  :Smile: 

also consider removing wget/ncftp etc. - and perhaps restricting ftp/http outbound connections to your server-package repository (iptables - and firewall in front of the prod servers) this way a hacker/rootkit can't just download packages/binaries they need if they root your box.

I'm sure there's many other tips I haven't listed here - but as you can see - having a compile/install environment and then just "mirror" the changes can have many advantages  :Smile: 

----------

## Sven Vermeulen

Our compilation server doesn't have everything installed. Don't forget you can ask portage to compile and package something without really installing it (there is a difference between "emerge -b" and "emerge -B").

However due to the non-existance of only-security-updates and the rather fast and unstable release-cycles (not in respect to the stability of the packages but the conservatism of the development and release-cycle), and of course the full trust of rsync-servers (no validity-checking), we're currently only deploying gentoo in development; production stays redhat.

----------

## psp

I've setup custom profiles for my servers with static package versions. This helps conform the servers (just in case).

----------

## EvilTwinSkippy

I'm just getting started with Gentoo, but I've been running a cluster of RedHat machines for about 3 years now. I'm in the process of migrating our RedHat servers to Gentoo. We have a lot custom software, and have had some very bad experiences with RPM. (One of my file servers must have 4 different builds of samba on it by now...)

<p>

Shell scripts, shell scripts, shell scripts. If you find yourself doing a task more than 3 times, write a script for it. Doubly so if you need to apply the same change to all your machines at once. It makes your work repeatable. You will also find that you make fewer mistakes when you spell out the entire thought process before commencing. I personally use a combination of TCL and Bash. TCL has hooks into my MySQL database, and I use it to sythesize a few of my configuration files. 

<p>

Keep a log book. I have a little website I use to tie together my department. I have a little section with a logbook. Every time someone mods a file, tweaks the system, or finds something wierd, it goes into the log book. I don't have a large department, it's just me and an assistant. But being able to go back in time to pair up the report of a problem to the changes makes life a lot easier. This log could work equally well on paper, or in a text file on the network.

<p>

My final tip is to remember that every data center is different. A lot of the decision making process is less technical, and more political or just plan making choices. I generally find that going with what I know best keeps everything running smoothly. For instance, I use MySQL and TCL all over the place on the network where others would use Perl or LDAP. The answer at the end has to be that you understand it and that you can keep it running.

----------

