Downtime

Athena was down all day while I was at work. Everything was working fine last night. The upgrade had gone as planned. Just to be sure there wouldn't be any surprises along the road, and to make sure I was using the latest compiles of everything, I decided to reboot before I went to work. I logged in, su'd, typed "shutdown -r now" and then went off and brushed my teeth and got dressed. When I came back, the login screen had come up. I switched over to tty1 to see if any of the services had screamed at me: none had, so I switched back, logged in, opened up firefox, and typed "http://localhost/" into the address bar. My (boring) directory listing page came up, indicating that Apache was running a-okay, and so I turned off the projector and headed to work. When I got there and tried to log in, however, I got no response from athena.

I tried accessing it through a web browser: nothing. Both ssh and apache weren't responding, so I figured either the whole box had gone down (i.e., kernel panic) or there was some sort of connection issue: perhaps the router wasn't port forwarding, but there was no way I could tell until after work, so I logged on to blogger and saved the post I'd put up directing people to download some files off of athena as a draft (so it would disappear), and went on with my work. When I got home, athena was running just fine, but when I tried to pull up external web sites, I got nothing: so it was a connection issue.

It wasn't the router: my laptop was doing fine, and the port forwarding was still set up. I looked at my init scripts and their accompanying config files: nothing seemed amiss. I restarted the ethernet interfaces and, viola, my network connections were restored: everything worked perfectly. That still didn't satisfy me: I didn't want athena to always start up without a net connection, so I rebooted her again, and the same thing happened. Then I remembered something I had read somewhere (probably the weekly newsletter) about the init scripts changing with the 2006.1 profiles. There was probably a dependency issue somewhere that was messing me up because my ethernet connections were loading too soon, but I had to have parallel startup because of my local DNS cache software.

I decided to look deeper into the issue, and so I pulled up the wiki page for dnscache, and searched the text for "parallel." Nothing. That was odd. I'm absolutely positive that this was the page that told me that I needed to set RC_PARALLEL_STARTUP="yes" in /etc/conf.d/rc. Could it be that the requirements (and the page) had changed? "Parallel" was one of my search terms to find the wiki page, so I went back to my Google tab and opened the cached version of the page: lo and behold, there it was, so someone had very recently changed the page to delete that instruction. I fired up vi, and changed the value to "no" and rebooted. Worked like a charm, only charms don't actually work, whereas this did. So, it wasn't gcc, and it wasn't my config files getting overwritten, it was some change they had incorporated into the new 2006.1 profile.

From the newsletter:

Some highlights from the release include the AMD64, HPPA, x86, PowerPC, and 64-bit PowerPC with a 32-bit userland releases being built with version 4.1 of the GNU Compiler Collection (GCC). AMD64, PowerPC, and x86 also feature version 2.4 of the GNU C library (glibc), while all architectures use baselayout 1.12.1, which features many improved startup scripts.

Improved. Indeed. Well, at least they warned me.