Archive for the ‘Computing’ Category

Extracting album art from iTunes

Wednesday, December 17th, 2008 in Computing

My music collection is primarily played using SqueezeCenter. But, I also have iTunes available for syncing to my iPod which I use in the car. One thing iTunes does fairly well is fetching album artwork, and I’ve been looking for a way of getting this in to SqueezeCenter without embedding the images in the MP3 files.

I found this script (there’s a download link towards the bottom) which does this job perfectly. It runs through my library and saves out all the iTunes artwork in a single file per album called cover.jpg. It’s written in javascript, so it’s not at all pretty. You just run it and wait until the dialogue appears saying it’s finished.

One gotcha I found was that I had to do a complete rescan of SqueezeCenter to make it notice that the album artwork had appeared. It was sporadic before that. Caching maybe?

That’s one problem I can tick off with a nice easy solution. :-)

  • Share/Bookmark

Exhange 2007 OOF (Out Of Office) and Free/Busy Availability, fixed

Tuesday, November 18th, 2008 in Computing

I’ve spent the past couple of weeks on and off trying to figure out why the OOF (Out Of Office) and Free/Busy availability information were broken in our Outlook 2007 clients. They generated errors saying the service was not available. They worked fine in OWA.

After some digging I thought I’d try some of the EWS URLs manually in a web browser. I’d tried some of the others and although they produced odd things, they at least did something. The EWS URLs just produced a 404.

Looking in IIS there were clearly files in the EWS directory with the correct names, so something else must have been missing. I started a comparison with another reference system. Eventually I noticed a difference in the web.config file located in C:\Program Files\Microsoft\Exchange Server\ClientAccess\exchweb\ews.

The copy on our production system contained entries like this:

<codeBase version=”0.0.0.0″ href=”file:///%ExchangeInstallDir%bin\Microsoft.Exchange.Common.IL.dll” />

Whilst our reference system contained entries like this:

<codeBase version=”0.0.0.0″ href=”file:///C:\Program Files\Microsoft\Exchange Server\bin\Microsoft.Exchange.Common.IL.dll” />

As far as I could tell %ExchangeInstallDir% was not set on either system. Further, looking at the autodiscover and OWA web.config files I noticed they did it the same way as the reference system.

So I ducked over to a Unix system, did a proper comparison of the production file versus the reference one and determined that the only difference was the expansion of the variable. So I simply dropped the reference one in to place on the production system and restarted IIS.

And it worked!

I’m not sure how we got in to that state, but I’m pleased it’s sorted out. I’m not particularly getting on with this Exchange stuff, and peculiar issues like this really don’t help. Given we installed both the production and reference systems in the same way I can’t understand how this happened.

Now to fix the remaining issues… :-(

Update: Now that I know what the issue was I can google for it. I found this post (better formatted here for non-IE users) which shows that the issue occured with the accidently released update for Exchange a while back. We got this update in the small window in which it was available and later uninstalled it. Looks like that caused the problem.

  • Share/Bookmark

FreeBSD filesystem snapshots

Monday, November 10th, 2008 in Computing, FreeBSD

I’ve been looking at filesystem snapshots on FreeBSD lately and I have to say I wasn’t too impressed. Yes, the functionality is there, but the interface sucks. For UFS you have to create a snapshot, which takes a while, and which appears as a file within the filesystem you’re snapshotting. Then you have to use an md device to mount it. ZFS is easier, but different. What we need is a simple and standard interface to both.

Fortunately I discovered Ralf Engelschall’s snapshot management utilities. He’s written a couple of tools that make creating and managing snapshots really easy.

Using cron one can schedule creation of hourly, daily and weekly snapshots for filesystems. You can specify that you want 3 hourly snapshots, 2 daily and 1 weekly for a given filesystem. The system takes care of everything else. And by using amd (the FreeBSD automounter) these snapshots can automatically be made available through a given mount point. It couldn’t get any easier!

Well, actually, it could get slightly easier. Currently these tools are available from Ralf’s site or from the FreeBSD ports collection. It strikes me that something as useful and fundamental as this should really make its way in to the base system.

Update: Whilst this worked well in testing, once I applied it to my server with a large (approx, but not over, 2TB) filesystem it hung the machine completely, even on the console. I haven’t investigated further yet.

Update 2: Well, I have to say I’m disappointed. Ralf’s scripts worked great, but the snapshotting of large UFS filesystems in FreeBSD is as good as broken. When it takes hours to create a snapshot, locking out the filesystem (and maybe even the machine) for hours, it might as well not be there. There seems to be an attitude of “just accept it”, which I’m not impressed about either. Roll on ZFS… (yes – I know it’s there now, but I’d like it to mature just a little more :-) ).

  • Share/Bookmark

FreeBSD with Netgear WG311T

Saturday, November 8th, 2008 in Computing, FreeBSD

A few days I wrote about my new Soekris net5501 router. In that post I mentioned that the only thing left to sort out was the wireless card. It turned out to be simpler to do than I thought.

I decided to go for a Netgear WG311T. It’s a 802.11b/g PCI card that’s compatible with FreeBSD through the Atheros chipset and ath driver, and it fits in the net5501 just fine. As expected I had to remove the net5501 board from the case to attach the card, but that only involved undoing a handful of screws.

Getting it working on FreeBSD was trivial. I added the following lines to my kernel configuration (they’re already there in GENERIC, I believe, but I built my own kernel because of the net5501):

device wlan
device wlan_ccmp
device wlan_scan_ap
device wlan_scan_sta
device wlan_xauth
device ath
device ath_hal
device ath_rate_sample

Then it was a simple case of initialising the card in /etc/rc.conf:

ifconfig_ath0="inet 1.2.3.4 netmask 255.255.255.0 ssid myssid mode 11g mediaopt hostap"
ipv6_ifconfig_ath0="1:2:3::4 prefixlen 64"

And I also added ath0 to rtadvd_interfaces and dhcpd_interfaces.

With that done the final step was to configure hostapd through /etc/hostapd.conf:

interface=ath0
debug=1
ctrl_interface=/var/run/hostapd
ctrl_interface_group=wheel
ssid=myssid
country_code=GB
wpa=2
wpa_passphrase=my passphrase
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP

This enables WPA2 using AES (rather than TKIP).

Connecting clients was no problem. My laptop and my squeezeboxes all connected fine.

One concern I had doing this was whether a PCI wireless card would have the same signal strength as a purpose access point. I seem to be getting the same signal power from this new setup as I did from my old 3com 802.11b access point. What does that tell me? I’m not sure. I would have expected technology to have come on a bit over the years. Maybe it has, but the fact that the card is lower powered balances it out? Regardless, it covers my whole house, so it’s not an issue.

So now I’m done. I’ve switched off my last piece of old equipment. My power draw has dropped significantly, the noise levels have decreased, and I’m a happy geek with a cupboard full of shiny new stuff. :-)

  • Share/Bookmark

Current Cost Electricity Meter

Wednesday, November 5th, 2008 in Computing, General

It seems to be the “in thing” to do at the moment; get a Current Cost electricity meter and produce pretty graphs. I couldn’t resist, so I picked one up with a USB cable to connect it to my server.

The system itself is trivial to install. There’s a box with a clamp that attaches to your mains supply (your side of the house meter), and a display which can be placed anywhere in your house (within the wireless range of the two units). Turn it on and it just works. I adjusted the electricity prices, but it’s not clear how accurate that’ll be given the multiple tiers of pricing we have.

So even without connecting it to a PC it’s a pretty useful device. Although I am developing a habit of running around looking for what’s caused the usage to jump up. Hopefully that’ll pass :-)

Connecting it to my FreeBSD server took a bit of effort. It needed the ucom module, but (I think) because I had ugen built in to the kernel it was using that instead. A kernel rebuild to include both fixed it. I also got some strange issues connecting to the device. On the first connect I got the expected XML output, but on the second connect I got messed up output. Turns out not to happen when I use my script to parse the data, so I don’t think I’ll worry about it.

I did the graphing using rrdtool. I’d like to take the credit for doing that, but I just stole all of Paul’s work. Thanks Paul :-)

The excitement has gone now, but I’m sure over time the data will prove to be interesting and useful.

  • Share/Bookmark

A new router (Soekris, Draytek and NanoBSD)

Wednesday, November 5th, 2008 in Computing, FreeBSD

A few months back I wrote about building a new server. It turned out to be more complicated than I thought, but 5 months on it’s still working well. Over the last few weeks I’ve been working on my next project – replacing my router.

The old router was an old dual CPU Pentium 3 machine with a couple of small SCSI hard disks in it. It was a full tower case and took up a lot of room and made a lot of noise. And, surprisingly, the power consumption was pretty similar to the server I recently built (which has way more in it). It even still had the original Speedtouch USB modem that BT once gave to me. So it had to go.

I spent quite some time deliberating the way forward. I could have gone for a domestic router like most other people do, but I’m a geek, and I like the flexibility of doing it myself. But at the same time, a small sized unit, with low power requirements, and no noise, is what I wanted. The solution came in the form of a Soekris net5501.

I went for the net5501-70 which has a 500Mhz CPU and 512MB of RAM. Not a lot by today’s standards, but more than sufficient for what I needed. And incidently, it’s quicker than the old router. I got the model with a case, and got the mounting brackets for a hard drive (which although I don’t intend to use at this stage, it was cheaper to get it now than later). I also purchased a pair of 4GB SanDisk Extreme III Compact Flash cards to run the thing from. It’s worth noting that Soekris recommend SanDisk CF cards, and they’re peanuts at play.com.

The next point to consider was how to connect to the ADSL line. I could have stuck with the USB modem, but the drivers were aging, I wasn’t sure if it was the cause of the odd disconnections and failures to reconnect that I’d been getting. I looked at internal ADSL cards, but it seemed to be a bit of a gamble as to how they worked and if I’d be able to get the right drivers. In the end I settled on the Draytek Vigor 110.

The Vigor 110 is basically just a PPPoA to PPPoE bridge. PPPoE isn’t widely used in the UK, but is in other parts of the world, so the support in FreeBSD was good (via ppp and the ng_pppoe module). It worked perfectly. It really was just a case of plugging it in and pointing ppp at it – no configuration required! And, just like the USB modem, it gives the router IP directly to the PC, so there’s no messing around to get that working either.

Longer term I plan to fill the net5501′s PCI slot with a wireless card, but I haven’t decided which to go for yet. This would turn the unit in to my wifi access point as well, but for now I’ll just stick with the separate one. I’d welcome advice on cards that are supported by FreeBSD.

So, that’s hardware all sorted. Next came the software. If you’ve been following my other posts you’ll know I’m a big fan of FreeBSD, so it’s pretty clear what route I was going to take here. But given the use of CF cards I had to approach things differently. I also wanted to be able to power the system off without causing any filesystem problems, so this required the card to be mounted read-only.

NanoBSD to the rescue! NanoBSD is a script that builds an image containing FreeBSD that can be written directly to a CF card (or anything else, really). It’s customisable, and I wrote a few bits to pull down the packages I wanted, and to make some configuration tweaks. It has a read-only root filesystem on the card, and uses memory-backed filesystems for /var and /etc. Config is stored in a separate partition on the disk and is copied to the memory-backed /etc during the boot process. But the best bit is the way it handles upgrades.

Upgrades are neatly done by having two root filesystems on the card. When you’re running off one you’re free to upgrade the other. NanoBSD generates two images; one for the entire card, and one that can be written to a single root filesystem. It also provides a script to write the image to the card and update the boot loader to boot from the right partition. So upgrading is as simple as re-running the NanoBSD script, writing the new image to the “other” partition, and rebooting. It can all be done live, and the only downtime is the time taken for a reboot (which is under a minute).

Of course, to use NanoBSD you need another system to do the builds on. Fortunately I’ve got a nice beefy server that can handle the job (although it took a few hacks to build the i386 image on an amd64 system). I’ve also got a nice Tinderbox setup which I already use for testing ports and which provided a nice supply of up-to-date packages.

So I’m happy, at last. Apart from my wifi access point I’ve managed to replace all my aging, power hungry, noisy equipment with nice new stuff. I guess I’ll be doing it all again in a few years :D .

  • Share/Bookmark

Dovecot is a neat piece of software

Friday, October 17th, 2008 in Computing, Work

For many years now (since before I started working at the University) we’ve been using University of Washington’s IMAP and POP daemons. They worked well, and (through an old bit of unsupported code) also allowed our MH users to access their email.

As time went on people wanted to do more than UW’s software could offer. Things like nested folders, and server-side caching. That’s when we started running Courier IMAP in parallel. This worked, but required users to use a non-standard set of port numbers.

At the time we looked at Dovecot, but it was fairly new, and we were unsure about trusting it with all our users’ email. That was a few years ago, so I decided this week to take another look. This was mainly driven by demand for faster IMAP access to the Maildir folders served by Courier IMAP.

My first impressions were good. I read through much of the stuff on the Dovecot wiki, and I kept thinking and saying to my colleagues “wow, that’s really neat”. Dovecot came across as a well thought out and well structured program, with a vast amount of useful tips and configuration ideas on their website. The level of customisation was good too, right down to allowing you to write a shell script to put in-line and tweak configuration to meet your exact needs.

After a few days of fiddling around I’ve managed to get a setup working that can replace both of our ageing Courier IMAP and UW IMAP installations. It should be a fairly seemless transition for our users, but I’m sure it won’t be that simple in practice. I’ve written a shell script that automatically detects at runtime where a user’s mail might be and sets the configuration accordingly. The script also allows the user to override the mail location and turn on debugging options.

And then there’s the performance issues. One of my colleagues has been having issues with the speed of Courier IMAP, and so far he’s impressed with Dovecot. The main gain here was the ability to store indexes in a separate location. Our mail is stored on an NFS server which becomes a performance bottleneck when using Maildir. Dovecot works around this by storing indexes and caches on a local disk making response times better.

Finally, there’s support. I hit a couple of issues getting things set up so I made use of the Dovecot mailing list. The response times in both cases were brilliant, and in both cases I got an answer to my problem straight away (maybe I asked common or stupid questions? :-) ).

So Dovecot comes highly recommended from me. Give it a try!

(And what about the MH users? Thankfully most have moved on to other things like Maildir & Thunderbird.)

  • Share/Bookmark

“Disc quota exceeded”

Friday, September 12th, 2008 in Computing, Work

Today we saw a strange problem on our Solaris hosts that NFS mount VxFS filestore from our Veritas cluster. The users were seeing “Disc quota exceeded” messages, whilst the quota command wasn’t showing they’d hit their limit. After some digging on the cluster node we found the following error message:

Sep 12 11:04:33 bes vxfs: [ID 702911 kern.warning]
        WARNING: msgcnt 10 mesg 089: V-2-89:
        quotas on /cluster/ResFS invalid;
        disk usage for group id 2805 exceeds 2147483646 sectors

Ah-ha! Group quota! We hadn’t even set group quotas, but it appears the system tracks the usage anyway when you mount with -o quota. Some googling turned up the following document:

http://seer.entsupport.symantec.com/docs/277535.htm

So it turns out there’s a 1TB maximum limit when using quotas. Since we weren’t using group quotas the simple option was to disable them:

vxquotaoff -gv /cluster/ResFS

Then I edited the Mount resource and changed the quota mount option to usrquota.

This only alleviates the problem for a while. Eventually someone will need to use 1TB of storage for themselves, but hopefully that’s a little way off yet. Maybe we’ll be using ZFS by then anyway :-)

  • Share/Bookmark

“Any idea WTF is going on?”

Wednesday, July 16th, 2008 in Computing, Work

“Any idea WTF is going on?” is what I read on my phone as I stumbled out of bed this morning. It was from one of my colleagues who, for some reason I can’t understand, seems to like getting in to work at a ridiculous hour in the morning.

Still half asleep I plodded through to my desk and sat down at my computer. I tried to check my email but nothing was responding. Then I saw the message “NFS server resfs.fs.cs not responding”… and woke up rather quickly. This meant either our network was shafted, or more likely, the cluster had blown up again.

I discovered one of the cluster nodes was offline and marked as failed, and the service group that manages our filestore was also marked as failed. That was odd, but it had happened before. I dug a bit further and found a screenful of SCSI errors. This was bad – something must have gone wrong with the storage.

Next I checked the arrays. The first one I checked had numerous errors on it; failed disks, missing disks, and drive not ready messages. I can’t stress enough how important this data is – it holds files, email and shared areas for all the staff and researchers in our department, and I really didn’t want to explain to them that we’d lost it all (well, we do have backups…). I nervously moved on to check the second array – they were mirrored, so as long as one was OK we’d be fine – and I was delighted to find no error messages.

So, now I knew that the likely cause of the problems was an array failing. It turned out later on to be the controller in this array, which was a good thing because Sun managed to send the wrong disks anyway. The next steps were to get the array fixed and to get things back online. I asked my colleague, who was already in the office, to disconnect the fibres from the failed array (to keep it completely out of the loop whilst it was fixed) and get on to Sun to fix it. Whilst he did that I, still at home, not dressed, and without breakfast, got on with getting things back online.

This, in theory, should have been the easy part. We had a mirrored setup so the plan was to just bring the volume back online with only half of the mirror. No problem, I thought. Except when it wouldn’t come online. When the initial problem had occured the cluster software (VCS) had failed to unmount the disk from the node it was on. It had decided that it needed to do this to bring it online on another node (little did it know that it wouldn’t work on any other node either), so as a last resort it asked the machine to panic. This is something akin to asking it to commit suicide. It duely did it’s job, but in the process left the disks in an odd state.

When I tried to mount these disks on one of the other nodes I got errors from the volume manager telling me a split brain had occured (this happens when a live cluster splits in two, but neither half can see the other). I knew that wasn’t the case, so I tried to force the mount. That failed with write errors. After a lot of head scratching I realised it was probably the I/O fencing stopping this node from accessing the disk. Whilst frustrating, it was nice to see the software behaving as it should – in a real split brain situation this is exactly what you want.

A while later I figured out how to clear the SCSI3 reservations on the disks (-o clearreserve option to vxdg import). This was nearly enough. Another issue with the split brain was that the configuration data stored on the disks didn’t quite match (I’m not 100% sure why, but I believe the node that paniced hadn’t managed to consistently update the metadata). After dumping the configuration it was clear that they were identical, bar a revision number, so by using -o selectcp we were able to get the diskgroup imported.

vxdg -fC -o clearreserve -o selectcp=1128804183.107.qetesh \
    import ResFS

Success! The diskgroup was online. From here it was just a case of waiting for fsck to confirm everything looked OK and then unleashing VCS to bring the service group back online.

By this point Sun had sent out an engineer and parts to fix the other array (we get a good service from them, thankfully). That’s currently resyncing its disks, which will take a day or two. Once that’s done we’ll hook it back in to the fibre fabric and bring things back online. It’ll take just as long again to resync the data, but all I have to do is sit and watch :-)

Finally, after hours of investigation I finally found out the cause of all the problems. We’ve just ordered a newer, bigger array. The old ones are just jealous.

(And a quick thanks to Pete for his help in debugging things this morning :-) )

  • Share/Bookmark

FreeBSD stuff

Wednesday, July 16th, 2008 in Computing, FreeBSD

I’ve done a bit of work on my FreeBSD ports lately. Firstly, after building my new server, I got round to upgrading from SlimServer to SqueezeCenter. This also meant sorting out ports for all the plugins I use. This didn’t take too long, and you can find them all over here. So far I’m liking SqueezeCenter, and I’d highly recommend it (and a SqueezeBox, of course).

I also maintain a port for a suite of software called KRoC. KRoC is written and maintained where I work, so apart from making it available to FreeBSD users I also have an interest in supporting the work done by our department. I’ve been waiting some time for a 1.5.x release of KRoC, but I finally got impatient. I automated the production of snapshots from their stable branch, and updated the port to build from that. I also run a FreeBSD 7 machine in their buildbot system to further test KRoC on my favourite operating system :-)

And in other FreeBSD news, I cast my vote in the FreeBSD Core elections. It’s hard to know who to vote for, but I gave their statements a good read and made a decision. Good luck to them all!

  • Share/Bookmark