A Blog by Tim Bishop

We’ve finally done another libstatgrab release. It’s been the best part of 8 months since the last release. Given the length of time you might be mistaken for thinking we’ve made lots of changes, but we haven’t. All this release really includes is some mostly untested Windows support, and handful of bugfixes.

I guess the problem is that we’ve hit a bit of a brick wall. Adding more features is now quite tricky; we’ve done all the common ones that make sense across multiple platforms. Adding more platforms is hard since any new ones would be the slightly more obscure operating systems (otherwise we would have done them already). Add to that our lack of enthusiasm and interest for making any radical changes and you get very little progress.

I suppose it at least works in its current state, so as long as we fix any bugs we find we’re probably keeping people happy.

So recently my router decided it didn’t want to whir its fans anymore and consequently gave up on life. It’s a dual CPU machine and both CPU fans had managed to wedge. After fixing them and getting things running again I heard klunking noises coming from the front of the case; one of the disks in the mirror had failed. I rapidly copied everything off the remaining disk, but didn’t have a spare to hand. Next morning the remaining disk went too. I wasn’t having much luck really, but on the positive side I did have a full backup.

After a day or so of fiddling with hardware I got something that resembled a working machine; I’d gone through a stack of various old disks by this point, most of which were dead. For a while I’d been pondering a fresh install for the machine, so this was the perfect opportunity. I decided to think about what I wanted it to do - this is what I came up with.

1. Obviously needs ADSL connection (via rather old, but working, USB modem)
2. I’d quite like a VPN connection to work for various (but not all) work servers
3. IPv6 routing both internally and out to the world
4. Internal NIC with my private and public address ranges
5. A second internal NIC for my wireless network
6. A better firewall setup (I decided on PF in the end)

Rather predictably I decided to do all this with FreeBSD. Nothing exciting about the install, other than I used gmirror this time. I’m still trying to find the best RAID solution on FreeBSD. So far I think gmirror has impressed me most compared to ataraid and gvinum.

So most of the things I wanted the router to do are things it did before. The new things were the VPN, IPv6 and PF. Those are what I’ll write about.

Setting up the VPN was straightforward. I installed the net/pptpclient port, bunged the sample config and my credentials in /etc/ppp/ppp.conf, and knocked up a quick RC script (let me know if you’d like a copy). I also added specific entries to ppp.conf for the hosts I wanted to route over the VPN, rather than letting it route whole subnets.

Something worth noting about ppp is the -unitN flag. Using this you can make sure ppp always uses the same numbered tun device. For example, my VPN connection has -unit1 ensuring it is always tun1. This makes firewall configuration a bit more manageable.

I’ve also knocked up a slightly better RC script for starting the ADSL connection (compared to the one provided with net/pppoa) that checks the line is up before returning. This allows subsequent startup scripts to be pretty much guaranteed access to the Internet. Again, let me know if you’d like a copy.

The next task was getting the IPv6 connection going. I decided to use the BT IPv6 Tunnel Broker service. In retrospect this might not have been the best choice; it’s been down for the last few days. I’ll let you know how I decide to proceed with that, but I’m reluctant to change because I’ll get a whole new address range. Getting this set up was pleasantly simple, particularly when compared with my past experiences trying to set up an IPv6 tunnel. Upon registering I was allocated an IP range and given a FreeBSD-compatible script to bring the link up. I decided to set things up more permanently using the excellent guide on the FreeBSD Diary website and the details from the broker’s script.

Surprisingly with the relevant tunneling, routing, and advertisments going setting up clients was a doddle. On my FreeBSD desktop machine I turned on ipv6_enable in rc.conf and it sprang to life (after a reboot). Even on our Windows systems it was as simple as running “ipv6 install”.

This finally left PF. Now that I’ve finished setting it up I can happily say it seems much nicer than IPFW, but I won’t pretend the journey was easy. It took a while to get my head around the differences, the main one being last-match versus first-match rules. I still need to figure out some of the ALTQ stuff though; my last attempt left me throttling internal traffic to 0.5Mb/s

I’ve always been fascinated by computer generated art, in particular the landscape work. Over the past few years they seem to be getting so good that it’s often hard to tell the difference between them and real scenery photographs.

The main site I visit is digitalblasphemy.com, and I pay the small subscription fee to get access to the latest images. Today I found the user gallery - not sure how I missed it in the past! I was quite shocked to see such high quality images. I don’t know what I was expecting, but it wasn’t what I found!

One artist in the gallery stood out to me, and he’s Juergen Eilts over at shiftedreality.com. He also maintains a blog, but doesn’t seem to keep it updated on a regular basis. It’d be nice to have a feed of his work.

It’s great to have guys like these generating such good work; there’s no hope of me ever creating anything like it. Without them I’d have nothing to put on my desktop!

I recently found a neat little tool whilst looking for applications that link against libstatgrab. It’s called bwm-ng and is written by a guy called Volker Gropp. The tool itself isn’t anything revolutionary (it’s influenced by the original bwm tool), it’s just a handy way of displaying current bandwidth usage across multiple interfaces.

This screenshot shows bwm-ng in action on my FreeBSD router.

It has a bunch of input methods to make it more portable, including libstatgrab which in theory might make it work on Windows. The default output method is the curses interface as shown in the above screenshot, but it’ll also do various textual formats including HTML.

The bwm-ng website gives links to a whole bunch of pre-packaged builds for various Linux distributions, and I’ve recently added it to the FreeBSD ports collection. Building from source is trivial too. 

I’m always on the lookout for handy little tools like this that just give you the raw facts in a simple and easily digested format. And it’s even better when they make use of libstatgrab

Over the past year or so the space usage on our research and web filesystems has pretty much doubled to the point where we’re dangerously close to running out of space. There’s currently about 1TiB of filestore available of which less than 10% remains unused.

Teaching filestore, however, has barely grown at all during the last year. I attribute this primarily to quota control, but also to the regular turnover of undergraduate students.

Fortunately we saw this problem arising quite a while ago, so we’ve had time to purchase new storage and infrastructure that should alleviate this problem and make it easier for us to expand the storage availability in the future.

Our new system consists of a pair of Sun StorEDGE 3511 arrays attached by fibre channel to our existing Veritas cluster. We’ll use VxFS for the filesystems, which could lead to some interesting new technologies like filesystem checkpointing; we could have a mount point of /yesterday to allow users to retrieve their files as they were at some point during the previous day, thereby reducing the need for us to do tape restores. VxFS also works quite happily with large filesystems, unlike Solaris UFS. The only problem we’ve found is that VxFS doesn’t support hard linking directories, but that’s not something we commonly, if ever, want to do. We also initially had problems integrating VxFS with the Solaris quota system over NFS, but we soon fixed that the “fun” way

Currently the research and teaching servers have locally attached filestore, which means if we have a hardware failure in one of the main servers we’re unable to get at user filestore from any other systems (without moving cables). The new solution provides NFS mounts of the filestore directly to each of the servers, which will allow files to be accessed via secondary machines should one of the main servers die. This is all part of our long term plan to increase the resilience of our systems.

One other interesting point to note is the use of the Solaris automounter to individually mount user home directories. Soon there’ll be mounts a bit like this all over the place:

resfs.cs:/home/cur/tdb 1.5T 54G 1.4T 4% /home/cur/tdb

Which will make things much more interesting!

At some point towards the end of last week I had the idea of writing a blog. I spent a while looking around for a decent online blogging tool, and, as I suspect most people do, I landed on blogger.com. It looked fairly swish, and had plenty of useful features. Except one. Where is the support for categories? So, moving on, and with the advice of some friends, I ended up at wordpress.org. It ticked all the boxes and is available in the FreeBSD ports collection.

This is where things went downhill. Step one, I figured, would be to get the existing packages on my server updated, since I’d quite like to use PHP5 for running Wordpress, and I noticed it also needed MySQL. This didn’t take too long, despite there being a new version of perl to install. The problems came when I tried to build MySQL; it wouldn’t even build. After a while I got it to build and install, only to discover PHP behaving oddly as well; it would hang when Wordpress used it’s mail() function.

I asked a few fellow FreeBSD developers, but didn’t get much sympathy since I was at that point running FreeBSD 5.2.1, a long since outdated version.

So on Sunday I started the task of upgrading to RELENG_5 (see my other post for more). The build process was fairly straightforward, thankfully. Then I noticed the serial line to my server was down, so I figured it would best to wait until Monday to do the actual install.

On Monday the install went fine, gvinum started without any problems, and the machine booted up with no unusual errors. There were a few tense moments during the actual reboots, but a lot of relief afterwards. I was surprised to find my PHP problems, and a long standing issue with mutt, had gone away before I’d even started the rebuilds.
The rest of Monday, and half of Tuesday was spent rebuilding all the packages on the system. By the end of Tuesday I was ready to start setting up the blog.

So, I make that the best part of 5 days to set up a blog. Surely that’s got to be a record?

Now to think about which theme to use…

I’ve been putting off upgrading my remotely hosted server from FreeBSD 5.2.1 for a while now, but after I started getting random problems I decided I had to move forward.

To start with there were a few reasons putting me off doing the upgrade:

1. From version 5.3 of FreeBSD vinum was pretty much broken, so I’d need to switch to gvinum.
2. The threading library changed from libc_r to libpthread.
3. A handful of key libraries in /lib and /usr/lib had their versions bumped.

The first problem is pretty straightforward to work around. Changing “vinum_load” to “gvinum_load” in /boot/loader.conf, and changing “vinum” to “gvinum” in /etc/fstab was all it took. To be on the safe side I did a fsck of the filesystems after rebooting in to 5.5.

The other two problems can be dealt with after a reboot by rebuilding all the applications on the machine. This is as simple as running “portupgrade -af”, but I chose to do it in chunks so I could get key things up and running quickly. One trick I used was to touch a file in each directory under /var/db/pkg, which would get removed when the package was upgraded. This allowed me to easily see what I still needed to do.

Most of this information was gathered from looking at mailing list archives and most importantly from reading /usr/src/UPDATING.

So, what problems had I been having before this that forced me to do the upgrade?

1. When piping a message to a command in mutt the pipe would never close. So, for example, piping to cat would display the message and then just hang.
2. PHP would hang when applications used the PHP mail() function; this forked sendmail and piped the message to it. I suspect this is the same problem as above.
3. Various things, most noticeably MySQL 5, would not compile. The problem was the recent libtool upgrade; libtool filters out -lc_r linker arguments.

After doing the upgrade these problems went away. The whole procedure was far less painful than I had been expecting.

Of course, the next challenge is upgrading to RELENG_6. But maybe I’ll leave that for another couple of years…

So, yet another blog. Given that there are more bloggers than blog readers one might have thought another blog would be largely pointless. They’re probably right.

However, my intention is not to write an uninteresting blog about what I saw on television last night, what colour the sky is on my planet, or whether I prefer my toast with or without jam. Nor do I intend to waffle at great length about my political opinions; that would simply be mean.

This begs the question, what am I going to write about? Well, I’ve split the blog in to a couple of categories. Firstly the category called Work, which will contain details about interesting technical stuff I’m doing in my job. That’ll probably be mostly of interest to people at my institution.

The next category is Computers. If you know me at all then you’ll be fully aware that I spend an awful lot of time messing around with them, so it makes sense that I’ll have a fair amount to say. I guess I’ll mainly be writing about cool stuff I’ve found or set up, development projects I’m working on, or gripes I have about other pieces of software. I’ve specifically made a sub-category for FreeBSD, an operating system for which I am a developer.

Thanks for reading!