Google Bookmarks

I’ve known about Google Bookmarks for a while, but until recently couldn’t really see how they’d be useful to me. A single set of bookmarks on the web is great, but if you have to go to a webpage to find them it rapidly becomes too much effort. Compare this to a single click within your browser’s menu.

My Internet Explorer using friends have pointed out that it’s now integrated in to the Google Toolbar, but at the moment it’s only the IE version that has it. The new version for Firefox has many of the newer features in the IE version, but unfortunately not the bookmark functionality.

After giving up on waiting for Google to integrate this functionality in to their toolbar I decided to have a look for a Firefox extension to do it instead. Quite why I hadn’t done this before I don’t know. On the first page of search results I found this one.

It’s working pretty well so far. Sometimes it seems to be a bit fussy about the title of pages, and it’s not the best at automatic updating, but it does do the job pretty well. I’ve dropped the extension on to my menubar too, so it fits in quite nicely. It also works quite happily under all my different Firefox installations (Windows, FreeBSD, and Solaris).

This handily means I don’t need the Google Toolbar. Firefox already has a perfectly good search box, and this extension gives me the bookmark features. Sorted!


A Virtual Universe

I’ve been playing a game called Entropia Universe (previously Project Entropia) recently, and I’m finding it quite addictive. It’s much like other MMORPGs at a first glance, but it boasts a “Real Cash Economy”. This means that there’s a direct exchange rate (fixed to the US dollar) and you can exchange money to and from the game currency just like you can with any other currency in the real world.

Within the game itself you learn skills, take on professions, and if you’re lucky you can even make a real profit. My experience so far, though, suggests that making real money is highly unlikely; but if you get good enough you might just about be able to fund yourself within the game – so no loss of real money either.

It’s worth giving the game a go. It’s a free (but large – approx 1Gb) download and it’s free to get an account. You can also get started in the game at no expense by “sweating” animals, although it gets tedious after a while.

I’m still fairly new, so I’ve only deposited a bit of real money to help progress me along a bit. I’ve just bought my first gun, and I’m now running around killing the animals instead of just sweating them. I have a reasonable amount of confidence in my ability not to get carried away and soak too much money in to this game 🙂

There’s a wealth of information on the game website, on wikipedia, and on the forums. There’s not much point in me repeating it all, so go take a look for yourselves. Maybe I’ll see you in the game? 😉


NFS+IPsec Performance

We’ve recently moved to having our filestore NFS exported from a cluster. This provides almost complete resilience from hardware failures, and moves us away from depending on individual end-user systems with locally attached filestore.

Given the inherent insecurities with NFS we opted to use IPsec authentication (but not encryption) between the hosts involved. The NFS server only accepts connections from a list of hosts, and we know those hosts are who they say they are by relying on the IPsec authentication. We’ve also made it use privileged ports to ensure local users don’t try any spoofing 🙂

The trade-off here appears to be latency. I’ve done some completely unscientific tests that involved shovelling UDP data at a fixed rate between two machines. These are the “jitter” figures they produced:

  • 0.10ms – direct
  • 0.30ms – via router
  • 0.70ms – via router with IPsec

Bear in mind that those figures might not bear any relation to the latencies involved with NFS packets, but it should give an idea of the relative delays added by routing and IPsec.

We could, to some extent, reduce those figures by replacing hardware. Quicker routers would undoubtedly remove some of the routing latency, and quicker machines could perform the IPsec calculations faster. But this probably isn’t the cheapest solution.

The first test I want to try is adding a private network between the NFS server and NFS client, with no routing involved. Seeing as it’s private we can reasonably trust that people won’t be able to spoof packets on that network and remove the IPsec authentication. In theory, these differences could signficantly reduce the latencies involved.

We’ll continue to monitor this for a while first, though. We need to keep an eye on loading on the NFS server, network usage, and so on. But, at the moment, it seems likely the problems are in the network part of NFS communication process.


Erm, whoops?

I’d finally finished migrating everything off the old myrtle disk arrays, so I was feeling quite pleased. I’d just unplugged the last array from myrtle and plugged it in to the test machine for wiping. Then I tried to log in to the machine room SunRay, but strangely it didn’t work.

I checked the console logs for myrtle and was surprised to see it counting “12%… 13%… 14%”. I glanced up and saw my colleagues attempting to come in to the machine room and tell me something, but for some reason were unable to open the door. Scrolling back over the console logs I saw what it was up to:

panic[cpu2]/thread=2a100105d40: md: Panic due to lack of DiskSuite state database replicas. Fewer than 50% of the total were available, so panic to ensure data integrity.

That made immediate sense to me, and I gave myself a bit of a kick. The RAID system we use for internal disks, DiskSuite (actually Volume Manager now, but it seems they haven’t updated this error message), has state databases stored on every disk. On myrtle we had 6 – two on the internal disks, and one on each of the four disk arrays. You need at least 50% for things to work.

A week or so ago I removed the first pair of arrays without any problems. At that point we had 4 out of 6 databases. Today I removed the last 2 giving us only 2 remaining, which is less than 50%, and the machine dutifully paniced itself.

Fixing it was made tricky by the fact that it could no longer mount the root filesystem because the RAID wouldn’t start. Thankfully the arrays were still to hand, so I just plugged them back in. After booting I removed the databases from the arrays, and added an additional one on each of the internal disks – this gives us 4 in total, 2 on each disk, which is what we normally do.

I also used the handy opportunity to mount the new filestore directly on /home and /proj, rather than using symlinks.

I’ll end this post with a bit of a rant. I can understand why the system won’t boot with less than 50% of the state databases – it has no way of knowing if they represent the correct state of things. But, what I don’t understand is why it needs to panic the system when it has less than 50%. It knows the remaining ones are valid because they’re currently in use. In fact panicing just makes it harder for the sysadmin to deal with the problem. Or am I missing something?


A new web site

The Computer Science department got it’s new website (more of a re-skin, actually) yesterday, so I decided it was about time to update my staff page.

I’ve brought it in-line with the new look of the main website, although it doesn’t completely follow the standard templates. It’s also XHTML 1.1 valid, which makes the pedantic side of me feel much better.

There’s not much else to say – go take a look (unless you’re already there), and let me know what you think.


Strange kerberos problems

A few days ago one of our users reported that they couldn’t change their password. The error coming out of the passwd command was confusing in itself – it said ‘bad old password’, or similar, which turns out to be a bug in our wrapper script.

After some investigation we discovered that neither kadmin or kpasswd worked:

tdb [~] % kadmin -p tdb/admin
Enter Password:
kadmin: Operation failed for unspecified reason while
initializing kadmin interface
tdb [~] % kpasswd
kpasswd: Changing password for tdb.
Old password:
kpasswd: Cannot establish a session with the Kerberos
administrative server for realm CS.UKC.AC.UK.
Operation failed for unspecified reason.

The completely unhelpful bit there is the “failed for unspecified reason” error message. How are you meant to even begin debugging that? After a couple of hours digging I logged the call with Sun.

It turns out that there is a known bug:

Document ID:6410919
Title:Patch 112908-24 will cause the kadmin -p kws/admin to exit with a error message

The solution presented was to remove patch 112908-24. This time I’m willing to do that, but from past experience I’d like to see them actually fix the problem rather than just back it out. Or, at the very least, remove the patch from cluster patches. Otherwise in 6 months time I’m left staring at the same problem.

What I’ve found most interesting in all this is that it took the best part of a month for anyone to notice passwords couldn’t be changed 🙂


Car Washing

I’ve been using the Flash Car Wash to wash my car for the past year. It’s a fantastic device, and has really good results. Not only does it leave my car shining without blotchy water marks, but it removes the need to use a bucket at all.

There’s only one flaw with it though – it requires a hosepipe. Here in Kent we’ve had a hosepipe ban since last summer (which I was unaware of until recently), so using the hose is tricky. I could have pleaded ignorance, but they sent a letter not long ago. I also live on a main road, so it’s hard to do it sneakily 🙂

So yesterday I’m sitting inside wondering how I’m going to rinse the car off before and after washing it with a bucket of soapy water. It was a horrible day too, absolutely pouring with rain. Then it occured to me that the rain was probably the answer – it wouldn’t contain lime would it? I’d also only just had a shower, so my hair was already wet.

Out I went to an already soaking wet car and started cleaning it. Sure enough, the rain washed the soap off nicely, albeit making me rather wet in the process.

Today I went out to check the results and it was pretty impressive – no horrible marks to be seen anywhere… apart from the bits of grass left from me strimming the front lawn 🙂


Ever increasing petrol prices

I’ve done a bit of driving back and forth across the country recently, and I’ve seen quite a few places go over the £1 per litre mark. Admittedly they were the more expensive places anyway such as motorway service areas.

Fortunately I’m not a heavy user of petrol, but it still frustrates me that something like 70% of the cost is government tax. Frankly it’s an outrageous amount to charge.

So I was quite pleased to see this article written by the Fools over at The Motley Fool. It’s well worth a read.

The most interesting bit for me was the bit about saving money buying petrol – the rest of the article is mostly common sense. I headed over to and discovered that the best price for unleaded in my area is 94.9p. After registering it turns out the Esso garage I generally use is charging this price. That’s a good start!

The next interesting thing was the Pipeline Card which appears to be a loyalty card that intends to get it’s members discounts on fuel prices. It’ll be very interesting to see if that takes off.

Save money on fuel with a Free Pipeline Card

There’s a couple of useful tips there, but I guess the best solution is to vote for whoever (if anyone!) says they’ll reduce fuel tax in the next election 🙂


The end of the T3 saga

So after copying everyone off the limping T3 arrays I arranged for a Sun engineer to return to site to fix it properly. Sun Dispatch had a bit of a moan because I’d had the parts for too long, but they realised it’d make most sense to keep the parts on site rather than collect them and then send them back to me 🙂

After replacing a loop card in the secondary unit the problems mostly went away. I guess this makes sense since it was the second unit we were having trouble connecting to. The engineer also replaced the primary controller which fixed the remaining minor problems. Finally we had a working array.

I’ve now wiped the disks, updated the firmware (not really sure why I did that!), and cleared the NVRAM. They’re all ready to pass on to someone else now.

So, just the failing cluster node and broken KDC to deal with… 🙁