Tinderbox setup
The machine I’m using is currently being used to test port updates. It has a bunch of jails for the -STABLE branches and a copy of the ports tree that I make changes to when testing port updates. I decided to use this machine for my package builder but this meant keeping things separated. So for package building I have the following set up in Tinderbox:
- A jail for 8.2-RELEASE (RELENG_8_2).
- A jail for 9.0-RELEASE (RELENG_9_0), when that gets branched.
- A separate ports tree that I can keep pristine and update automatically without affecting my other ports work.
I won’t document how to do that. The Tinderbox README covers it in plenty of detail.
Index generation
If you’re just doing a pristine ports tree, with no OPTIONS or other environment tweaks, and you don’t care that the INDEX file may be slightly newer than your package set, you don’t need to do this. I have some OPTIONS set and I wanted the INDEX to exactly match the versions of the available packages, so I’m building my own INDEX file.
I checked the Tinderbox archives for the best way to do this. Others seem to be doing it using a hook on the ports tree update. The problem with this is that you need to do some extra work to make sure any OPTIONS or environment changes are included, and if you’re doing it for multiple OS versions you’ll need to cover that too (otherwise it’ll build the INDEX according to your Tinderbox host’s OS version).
The solution I came up with was to make a small custom port. It builds INDEX and installs it to /usr/local. I build this inside each build I’m using for my package building and the result is a package containing the INDEX file that fits my criteria (OPTIONS, environment, and matches my ports tree exactly).
Here’s the port’s Makefile. The symlink line is only needed because Puppet, which I use, looks for INDEX.bz2 rather than INDEX-8.bz2.
PORTNAME= makeindex
PORTVERSION= 0
CATEGORIES= ports-mgmt
MASTER_SITES= # none
DISTFILES= # none
MAINTAINER= tdb@FreeBSD.org
COMMENT= Generate INDEX file
USE_PERL5= yes # make index requires perl
PLIST_FILES= ${INDEXFILE}.bz2 INDEX.bz2
do-build:
cd ${PORTSDIR} && make index INDEXDIR=${WRKDIR} -DINDEX_PRISTINE
bzip2 -9 ${WRKDIR}/${INDEXFILE}
do-install:
${INSTALL_DATA} ${WRKDIR}/${INDEXFILE}.bz2 ${PREFIX}
ln -s ${INDEXFILE}.bz2 ${PREFIX}/INDEX.bz2
.include <bsd.port.mk>
Package builds
The next step is to tie the INDEX generation together with updating the ports tree and building packages. It’s a pretty simple process; update the ports tree, generate and install the new INDEX file and then build any new packages. Below is the script I use to do this, and here are a few useful notes:
$TB/tdb/autobuildportsis a list of ports that I want to build, one per line, in the format “category/portname”.$TB/tdb/makeindexis the port discussed in the previous section.- I use the
-norebuildflag totinderbuildto ensure I don’t rebuild the leaf ports unless necessary. - The last step after the
forloop is mostly so I can check what it’s done, and isn’t necessary for things to work.
#!/bin/sh TB=/u1/tinderbox PT=FreeBSD_auto portlist=$TB/tdb/autobuildports PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:$PATH export PATH $TB/scripts/tc updatePortsTree -p $PT for b in `ls $TB/builds | grep $PT`; do rsync -rlpvc --delete --force --exclude=CVS/ \ $TB/tdb/makeindex/. \ $TB/portstrees/$PT/ports/ports-mgmt/makeindex $TB/scripts/tc addPort \ -b $b -d ports-mgmt/makeindex $TB/scripts/tc tinderbuild \ -b $b -nullfs ports-mgmt/makeindex cd $TB/packages/$b && tar -zxvf All/makeindex-0.tbz INDEX\* for p in `cat $portlist`; do echo "===> $p on $b" $TB/scripts/tc addPort \ -b $b -d $p $TB/scripts/tc tinderbuild \ -b $b \ -nullfs -norebuild \ $p done cd $TB/packages/$b/All && ls > $TB/packages/$b/All.new echo "New packages:" comm -1 -3 $TB/packages/$b/All.last $TB/packages/$b/All.new mv $TB/packages/$b/All.new $TB/packages/$b/All.last done
I run this script on a daily basis from cron.
Portmaster setup
The final step is installing these packages. I could do this by hand using pkg_add, but I prefer to use Portmaster. It’ll handle upgrades too. I use the following config in my portmaster.rc file which sets all the useful options for working with binary packages.
Portmaster will pull the INDEX file automatically as required. I picked /var/db/portmaster as the temporary area to put packages in, but you could use another place if /var is space limited.
# Do not create temporary backup packages before pkg_delete (-B) NO_BACKUP=Bopt # Only install packages (-PP or --packages-only) PM_PACKAGES=only # Use the INDEX file instead of /usr/ports (--index-only) PM_INDEX=pm_index PM_INDEX_ONLY=pm_index_only # Delete packages after they are installed (--delete-packages) PM_DELETE_PACKAGES=pm_delete_packages # Local paths PACKAGES=/var/db/portmaster INDEXDIR=/var/db/portmaster MASTER_SITE_INDEX=http://my.tinderbox.server/packages/8.2-RELEASE-FreeBSD/ PACKAGESITE=http://my.tinderbox.server/packages/8.2-RELEASE-FreeBSD/
So that’s it. I can now run portmaster category/port to install a new port or portmaster -a to upgrade everything and I’ll get the latest packages built using my custom options.
My final point is that this is all still a little fresh. I only just wrote it and I haven’t been using it long. So there’s undoubtedly something I’ve missed. You’ve been warned!
.
Broken pingbacks – what’s to blame?
Tuesday, March 24th, 2009 in ComputingFor a while I’ve been noticing a lack of pingbacks from people linking to me. At first I put it down to people just not linking to me, but then after discussions with a colleague today I realised I didn’t have that annoying problem of them happening when I link to my own posts. I tested further and realised they weren’t working at all.
I debugged the issue using the good old “print debug messages all over the place” method – crude but effective. I started in xmlrpc.php and discovered that it was stopping at this line:
This meant it was getting as far as trying to post the comment. It was then that I had a lightbulb moment. What would be stopping comments being posted? My captcha to stop comment spam (has it really been three years since I set that up?), that’s what!
My colleague has been using a numerical captcha-style plugin called Math Comment Spam Protection (MCSP) instead. I did a quick comparison between it and the SecureImage plugin and fairly quickly found the difference. This is from the MCSP plugin:
By comparison, the SecureImage plugin only checked if the user is registered:
The fix looks simple. But, given the lack of updates for SecureImage I decided it was time to move on. So now I have the MCSP plugin instead, and pingbacks are once again working.
I just have one request to make. Can anyone who’s linked to me in the last three years please try again?
Tags: captcha, comment spam, Math Comment Spam Protection, mcsp, pingback, SecureImage, spam
Posted in Computing | No Comments »