Thursday, June 07, 2007

My FreeBSD Adventure With Thebeast

Warning: not for the faint of heart. This is a pretty long chronicle, full of technical jargon, but it might prove useful/entertaining to Linux/BSD users.

A while ago I bought a computer from the surplus store. There were a couple of things that were noticeable about this computer: the first was that when I turned it on, the hard drives inside were exceptionally loud, and the second was its identification:


Behold, the mark of thebeast!

The hard drive noise turned out to be an old SCSI hard drive that had been hooked up to the power supply, but as there is no SCSI support, not to the motherboard. I took it out, obviously, but because of its roar, and the markings on the case, I decided that this computer's name would be known on the network as thebeast (not to be eschatologically confused with the Antichrist).

Another beast-like quality of this computer is its chimera-like nature: there are four different operating systems installed in a quad-boot configuration. An interesting thing about the hard drive that came with this computer is that it's a 20GB hard drive, but Windows thinks it's a 7.8GB hard drive. There apparently are some contradictory drive geometry settings programmed onto the drive. I suspect that this was done in order to keep the same standard part number and specs, while upgrading to newer, more often-produced hardware: it's been a while since they've made 8GB desktop hard drives in quantity.

This of course means that if ever Windows tries to install on the drive, it totally screws up the partition table. I initially tried to install it on a 4 GB partition at the beginning of the disk, but it insisted on writing its own. "Fine," I said, and I let it write its own table. This worked, but as soon as I tried to partition the rest of the disk on the same table, Windows wouldn't boot anymore. What I ended up doing was taking a 3GB hard drive that I had around and installing Windows 2000 Pro on that. Of course, to do this, I had to set it as the primary drive, install it, and then switch it back to the secondary drive (so that I could use Grub, a boot loader, to pick an operating system at boot time). Windows doesn't play nice with other operating systems. It thinks it owns the disk. To get it to boot off of the second disk, you even have to trick it into thinking that its disk is really first. Here's my grub.conf file:
default 0
fallback 1
timeout 15
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Gentoo Linux 2.6.20-r8
root (hd0,0)
kernel /boot/gentoo/kernel-linux-2.6.20-gentoo-r6 root=/dev/hda6 video=i810fb:accel
savedefault

title Gentoo Linux 2.6.20-r8 (rescue)
root (hd0,0)
kernel /boot/gentoo/kernel-linux-2.6.20-gentoo-r8 root=/dev/hda6 init=/bin/bb

title Gentoo/FreeBSD 6.2
root (hd0,1,d)
kernel /boot/loader
savedefault

title FreeBSD 6.2
root (hd0,2,d)
kernel /boot/loader
savedefault

title Windows 2000 Professional
# Make Windows think its disk is first
map (hd1) (hd0)
map (hd0) (hd1)
rootnoverify (hd1,0)
makeactive
chainloader +1
savedefault
As you may be able to tell, when the system boots from the BIOS, it gets a menu screen with the following choices:
  • Gentoo Linux 2.6.20-r8
  • Gentoo Linux 2.6.20-r8 (rescue)
  • Gentoo/FreeBSD 6.2
  • FreeBSD 6.2
  • Windows 2000 Professional
Whichever option is chosen is the operating system that will boot. Each of the operating systems sits on its own partition, but they can all see each other's files, with the exception of Windows, which can't properly read the partition table/disk geometry of the first hard drive. If not for that limitation, I would be able to see the Ext2 and Ext3 partitions (in this case Linux's /boot and /home) after installing a driver in Windows. I typically use ReiserFS for the other Linux partitions (/, /usr, and /var), for which there might be a read-only Windows driver, and FreeBSD uses the UFS file system, which as far as I know has no Windows driver.

Getting FreeBSD and Linux to see Windows' NTFS partition was pretty easy. In Linux the line in /etc/fstab looks like this:
/dev/hdb1       /mnt/win2k      ntfs            ro,nosuid,nls=utf8      0 0 # ~3.0GiB
and in FreeBSD it looks like this:
/dev/ad1s1      /mnt/win2k      ntfs            ro                      0 0 # ~3.0GiB
They're essentially the same. Linux uses "hd" (for an IDE hard drive), "b" meaning the second drive (the first drive is "hda"), and "1" denoting the first (and in this case, only) partition on that drive. FreeBSD identifies devices by the name of the driver used to access them in stead of their function as in Linux. "ad" refers to the disk driver, "1" refers to the second disk (ad0 is the first disk), "s" is for 'slice' which is the BSD word for a what Linux and Windows call primary partitions, and "1" refers to the first slice. This is, by the way, distinct from the way that Grub refers to hard drives. Grub calls your first hard drive (hd0), and its first partition is (hd0,0). If that partition has sub-partitions, as BSD or Solaris UFS slices are wont to have, then your root slice on a second partition on a third disk would be: (hd2,1,a).
The first partition of the first disk:
Linux: /dev/hda1
Grub: (hd0,0)
BSD: /dev/ad0s0

The first sub-partition in the first slice of the first disk:
Grub: (hd0,0,a)
BSD: /dev/ad0s0a
Simple, right? It took a fair amount of searching and sifting to find all that out.

Getting FreeBSD to recognize Linux partitions is as easy as identifying the partitions and filesystem:
# Gentoo Linux
/dev/ad0s6 /mnt/linux reiserfs rw,noatime,nodev 0 0
/dev/ad0s1 /mnt/linux/boot ext2fs rw,noatime,nodev 0 0
/dev/ad0s7 /mnt/linux/usr reiserfs ro,noatime,nodev 0 0
/dev/ad0s8 /mnt/linux/var reiserfs ro,noatime,nodev 0 0
/dev/ad0s9 /mnt/linux/home ext3 rw,noatime,nodev 0 0
Getting Linux to see my Gentoo/FreeBSD partition was equally straight-forward, since there were no sub-partitions, though I did have to specify the ufstype attribute:
/dev/hda2       /mnt/gfbsd      ufs             ro,ufstype=ufs2,nosuid,nodev    0 0 # 3586MiB
Getting Linux to see UFS sub-partitions was a bit trickier. Doing an "fdisk -l" only showed the main slice (/dev/hda3), not the sub-partitions, but entering "cat /proc/partitions" revealed the device names of the missing sub-partitions. (I suppose ls /dev/hd*) would have revealed the names, too). It was a bit tricky to figure out which device went with which. The extended partition with the Linux partitions on it began at hda5, and where those left off was where the UFS sub-partitions began, but they weren't in the order in which they were created. UFS apparently assigned the first sub-partition I had created within this slice (which was a swap partition, shared with the Gentoo/FreeBSD install) as ad0s3b, reserving the "a" slot for my / partition, and making the /var and /usr partitions "d" and "e" respectively. Therefore, this is how I mapped my drives from Linux:
# First Hard Drive
/dev/hda1 /boot ext2 noauto,noatime 1 2 # 16MiB
/dev/hda2 /mnt/gfbsd ufs ro,ufstype=ufs2,nosuid,nodev 0 0 # 3586MiB
#/dev/hda3 is the freebsd slice: hda10,11,12,13 # 7680MiB
#/dev/hda4 is the extended partition: hda5,6,7,8,9
/dev/hda5 none swap sw 0 0 # 1024MiB
/dev/hda6 / reiserfs noatime,user_xattr 0 1 # 256MiB
/dev/hda7 /usr reiserfs noatime,user_xattr 0 1 # 3586MiB
/dev/hda8 /var reiserfs noatime,user_xattr 0 1 # 256MiB
/dev/hda9 /home ext3 noatime,user_xattr 0 1 # ~3.8GB
/dev/hda10 /mnt/freebsd ufs ro,ufstype=ufs2,nosuid,nodev 0 0 # 7680MiB
#/dev/hda11 /mnt/freebsd is the FreeBSD swap partition # 512MiB
/dev/hda12 /mnt/freebsd/var ufs ro,ufstype=ufs2,nosuid,nodev 0 0 # 1024MiB
/dev/hda13 /mnt/freebsd/usr ufs ro,ufstype=ufs2,nosuid,nodev 0 0 # ~5.3GB

# Second Hard Drive
/dev/hdb1 /mnt/win2k ntfs ro,nosuid,nls=utf8 0 0 # ~3.0GB
After I had Windows and Linux up and running (Gentoo is my favorite flavor of Linux, so that's what I used), I set my sights on FreeBSD. This was done for a number of reasons:
  1. With a chimera machine called thebeast, one must, must have a BSD install. I mean, have you seen their 'Beastie the BSD daemon' mascot? BSD is the essence of geeky hotness, and with that "E280666" sticker on the box, who could resist? It was a sign.
  2. Operating systems are a hobby of mine, and I've never worked with a BSD before. Caleb had mentioned that I should look at BSD, because Mac OS X is based on it.
  3. I hear Gentoo is a very BSD-like Linux, so it wouldn't be totally foreign, and there's even a project to get Gentoo's Portage on BSD.
I had once before tried to install FreeBSD on this computer, but I kept running into problems during the install. Once I finally had it installed, I had no idea how to get X working, let alone any actual user software. This time, I decided I would try the Gentoo/FreeBSD approach. I chose FreeBSD over NetBSD and OpenBSD because there seems to be a better community and support around FreeBSD. Along with that, (and probably having to do with that) it was the most supported (read 'least experimental') in the Gentoo-Alt *BSD project, and that was my stepping stone of choice from Linux.

Installing Gentoo/FreeBSD was as simple as following the provided online instructions. They were written by Gentoo/Linux developers who mess with FreeBSD, not FreeBSD developers, which means I could follow them more naturally: they basically gloss over the similarities, while slowing down to explain the differences.

I went with the LiveCD option for installing, since I had no existing installation: I downloaded and burned a FreeSBIE CD, and followed the guide, deviating where necessary. My biggest BSD-related installation issue was getting the right Ethernet driver configured in /etc/conf.d/net. In Linux, you don't care what the driver is called, you just configure it as net.eth0, but in BSD, I had to know that I was using the xl driver, so my init script was called net.xl0.

Also, I ran into trouble with the /etc/conf.d/net config file. In Linux, I had this:
dhcpcd_eth0="-t 15 -N"
dhcp_eth0=( "nontp nonis" )
config_eth0=( "192.168.5.16/24 brd 192.168.5.255" )
routes_eth0=( "default gateway 192.168.5.1" )
fallback_eth0=( "dhcp" )
but in FreeBSD, I needed to remove the parentheses, and I also discovered that I should use "via" in stead of "gw" or "gateway" to specify the default route to the Internet.
dhcpcd_xl0="-t 15 -N"
dhcp_xl0=" 'nontp nonis' "
config_xl0=" '192.168.5.16/24 broadcast 192.168.5.255' "
routes_xl0=" 'default via 192.168.5.1' "
fallback_xl0=" 'dhcp' "
One thing that was nice about doing Gentoo/FreeBSD before the actual FreeBSD install was that in Gentoo/FreeBSD, the init script layout is the same, and you install all of your userland applications through portage, which with I'm very familiar, and so I was tweaking use flags, configuring daemons and services, and using package.keywords without so much as batting an eye.

Configuring and compiling the kernel was relatively straight-forward, FreeBSD's handbook is an excellent resource, and I used it extensively later on in the actual installation.

When I got to /etc/fstab, I was accustomed to having the /tmp directory (as well as some others) on its own virtual memory disk, so I looked up how to do that. Here's the line from /etc/fstab:
In Linux:
tmpfs   /tmp            tmpfs   size=512M,nr_inodes=1M,mode=1777,noexec 0 0
In BSD:
md      /tmp            mfs     rw,-s100m               0 0
Unfortunatley, the Gentoo/FreeBSD project doesn't enable you to easily install just anything. I was able to install X with gnome-light, and firefox, as well as a few other things, but the majority of ebuilds in the portage tree aren't ready for installation on FreeBSD out of the box. To get a real FreeBSD installation, I was going to have to do it the right way. One program that I annoyingly could not install was fortune-mod. An earlier installation of Gentoo/FreeBSD had successfully installed with it, but then the ebuild had been updated to a new version (and all older versions removed), and I was left without a viable fortune program. A tragedy, I know. Anyway, onward.

Installing FreeBSD
I burned the FreeBSD CD ISOs that I had downloaded via Bittorrent, pulled up the handbook, and dove in. This time I was able to follow the handbook much better than the last time I had done the installation, and thanks to the copious amount of poking around the system I had done during my Gentoo/FreeBSD install, I was up and running in no time. I customized the kernel to exclude hardware I didn't have, and also to build my sound driver. To build it, I added the following to my kernel config file:
# Sound card support customized to this machine
device sound
device snd_ich
and the following line to the /boot/loader.conf file to have the module automatically loaded:
snd_ich_load="YES"              # Intel ICH
Then I ran into a bit of a problem. In my previous installations, my mouse had worked just fine. I had been successfully using a 2-port Iogear KVM switch between a small Linux box (minix) and thebeast, but I had since switched to using a 4-port LinksKey KVM (with audio support) between four different boxes, and the mouse wouldn't work in BSD. It worked in Windows and Linux just fine, but in FreeBSD, no matter what kernel I used, and even if I used the FreeSBIE LiveCD, the mouse pointer would be there on the screen in X, but it wouldn't move or click. What was more, I discovered that the mouse device was absent from /dev. It was quite frustrating to finally have a fancy graphical environment up and running on BSD, and not be able to use it, except by keyboard shortcuts and tabbing.

I searched the forums online to see if others had had similar problems with the FreeBSD psm driver. I found quite a few people who had issues with touchpads, and a few who had trouble with KVMs. I found solutions to the touchpad issues by adjusting the flags passed into the driver from the kernel configuration device.hints file, but those flags were specific to touchpad initialization and waking up from a laptop's suspended state. No one that I could find had fixed the KVM issue in a way that was helpful for me. What worked for most people was recompiling the kernel with
options       KBD_RESETDELAY=200
options KBD_MAXWAIT=5
adjusted to higher values, but this didn't help me. What I found to be the most help was the psm man page and the dmesg output. Adjusting the KBD_RESETDELAY didn't help at all, even at ridiculously high values, because I found that the mouse was failing to reset, or at least not properly responding, no matter how long the driver waited for a response. Increasing the logging threshold using the kernel following kernel config options:
options       PSM_DEBUG=2
options KBDIO_DEBUG=2
...yielded the following in dmesg:
psm0: current command byte:0065
psm0: failed to reset the aux device.
This combined with the psm man page led me to a flag in /boot/device.hints that disables the initialization attempt at boot time:
hint.psm.0.flags="0x400"
This gave me the following in dmesg (and /var/log/messages) with PSM_DEBUG still at 2:
psm0: current command byte:0065
psm0: found IntelliMouse Explorer
psm0: flags 0x400 irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons
psm0: config:00000400, flags:00000008, packet size:4
psm0: syncmask:08, syncbits:00
...but I still had no mouse movement in X, whereas before, I had been able to use this mouse just fine. A few hardware options presented themselves: I could use a serial mouse that I have stuffed in a drawer in my room; I could use an adapter to make the PS/2 keyboard and mouse plug into a USB port in stead, and thus use a different driver, or at least a different initialization routine; I could get an extra mouse, or a different mouse, but that defeats the purpose. This should work. It occurred to me finally to specify the mouse device directly in xorg.conf; in stead of /dev/sysmouse, to use /dev/psm0:
Section "InputDevice"
Identifier "Mouse0"
Driver "mouse"
Option "Protocol" "auto"
# Option "Device" "/dev/sysmouse"
Option "Device" "/dev/psm0"
Option "ZAxisMapping" "4 5 6 7"
Option "Buttons" "6"
EndSection
And it worked. Yay!

I installed ports (FreeBSD's package management system), and thus far I've only installed packages using the "pkg_add -r name" method, although I do hear you can install directly from source if you so desire. In my experience, it's been pretty slick with just binary packages, and I'm content for now. I'm not nearly as adept at ports' intricacies as I am with Gentoo's portage.

Here's what I ended up doing with the graphical interfaces:
  • Since Gentoo/FreeBSD doesn't work with KDE yet, and Gnome is minimal, I went with Xfce4, which is an excellent desktop environment (although I'm using GDM as the login manager: Gentoo has an excellent gdm-themes package, which I unmasked with the x86 keyword in order to install). I'm also using Xfce4 on a 200MHz Pentium-MMX machine that I'm tinkering with (it's called crunchy, and I paid a whopping $5 for it). The Gentoo/FreeBSD partition is also relatively small, so I don't think I'm ever going to install a full Gnome on it, and certainly not KDE.
  • My usual desktop environment for Linux has been Gnome. I also use KDE occasionally, but I prefer Gnome, so in order to be consistent, the Linux install uses GDM and Gnome by default.
  • I installed GDM, Gnome, Fluxbox, Xfce, and some others, but I decided that my FreeBSD system would use KDM for graphical login and KDE as the default desktop.

1 comment: