Although I've run BSD-based production servers for 15 years or so, I find that I tend to get rusty since they basically just sit there doing their thing until there's a hardware failure. Being the proactive fellow that I am, I tend to fire hardware before it can quit, so I decided to take the weekend and build a FreeBSD 6.1-RELEASE server to replace one that had been running 4.9-RELEASE for years. This server do Although I’ve run BSD-based production servers for 15 years or so, I find that I tend to get rusty since they basically just sit there doing their thing until there’s a hardware failure. Being the proactive fellow that I am, I tend to fire hardware before it can quit, so I decided to take the weekend and build a FreeBSD 6.1-RELEASE server to replace one that had been running 4.9-RELEASE for years. This server does just about everything, from handling a massive mail volume and the associated filters and virus scanning duties, to mailing lists served via mailman, to hosting over 80 domains for both DNS, mail and Web hosting. This upgrade would be major indeed, upgrading to PHP5, MySQL5, Apache 2.0, and on and on. Also, the disk in this server is standard ATA/133 PATA drives using software RAID. What follows are brief notes on my migration, some FreeBSD basics, and things I wish I’d known at the time.Installation- Boot from the bootonly ISO, standard install, construct your partitions on one of the drives (ad0), set the MBR, basically all the defaults, and select the Developer package set. Let the installer do its thing, but don’t bother installing any specific ports yet. When it’s all over, set up a local user, root password, timezone, and the like. Then, before rebooting, configure the RAID.RAID1 with gmirror-Before rebooting the box, type Alt-F4 and get to a shell. Type sysctl kern.geom.debugflags=16 to remove the mount checks, and then gmirror label -v -b round-robin gm0 /dev/ad0 will set the mirror up on ad0. Now, echo geom_mirror_load="YES" > /boot/loader.confto instruct the bootloader to head for the mirror, and now, edit /etc/fstab, replacing “/dev/ad0” with “/dev/mirror/gm0” to mount the RAID device instead of the raw device on boot. If all is well, reboot. Following the initial boot, assume root and enter gmirror insert gm0 /dev/ad2 which will make /dev/ad2 part of the gm0 mirror. gmirror status will show you the resync status and tell you when the array has completed the rebuild. Also, gstat will show you how hard the mirror’s working, and which disks are in use, measured in usecs. If you’re anything like me, one of the first things you’ll do is install bonnie from ports and test the mirror’s I/O. I found a particularly nasty IRQ problem this way, which resulted in 5.5MB/s writes to the mirror. Fixing that brought the performance into the 26MB/s write, 80MB/s read territory since reads are striped from each disk. Quite nice.Installing portsAfter the first boot, cvsup you ports tree and src trees, then install portupgrade. I found very little change to the src tree, but plenty of ports updates. I dislike the prompting in the installer to install ports directly from there — I’d much rather do it following the first boot, though portupgrade makes life lots easier. The FreeBSD KernelI wanted to run pf as the firewall, since it’s the slickest firewall available on any OS. To do this, cd /usr/src/sys/i386/conf and cp GENERIC HOSTNAME, substituting the system’s hostname for HOSTNAME. Use the SMP kernel file if it’s a multi-CPU system. Add these lines to the file: device pf device pflog device pfsync options ALTQ and recompile the kernel with make buildkernel from the /usr/src directory. Install the kernel with make installkernel, and reboot.pf I really really like pf. The tables structure and configuration file syntax bring happiness to my heart, as does the use of variables within the config, as seen in this example: ext_if="fxp0" loop="lo0" table <smtpblock> { 10.0.0.0/8 } tcp_services = "{ 25, 53, 20, 21, 22, 80, 443, 110, 143, 993, 995 }" udp_services = "{ 53 }" block all pass quick on $loop all block drop in quick on $ext_if inet proto tcp from <smtpblock> to ($ext_if) port 25 pass in on $ext_if inet proto tcp from any to ($ext_if) port $tcp_services flags S/SA keep state pass in on $ext_if inet proto udp from any to ($ext_if) port $udp_services pass out quick on $ext_if inet proto { tcp, udp, icmp } all keep state antispoof for $loop antispoof for $ext_if That’s it, a full configuration with a table (no NAT). Make sure that you have pf_enable="YES" pf_rules="/etc/pf.conf" pf_flags="" in /etc/rc.conf, and you’re all set. Some handy pf commands are pfctl -s rulesShow the current rules pfctl -saShow the current rules, connection tables, and statistics pfctl -vvsaShow the current rules, connection tables, and statistics, with extra verbosity pfctl -vvsTableShow the currently defined table statistics pfctl -t smtpblock -T showShow the entries in table smtpblock. pfctl -t smtpblock -T add -f /tmp/bl Add a list of IPs from file /tmp/bl to the table pfctl -s rulesShow the current rules There’s much more to pf, and to this build, but it’ll have to wait for another entry. I hope to have time to detail more of the migration, including gotchas encountered when moving between versions of common services, system-level changes and so on.Suffice it to say, the server was rebuilt and put in place in a few hours’ time, and I’m sitting back enjoying the knowledge that aside from minor patches, I won’t have to touch it again for another few years.