{"id":410,"date":"2011-09-23T17:43:58","date_gmt":"2011-09-23T17:43:58","guid":{"rendered":"https:\/\/www.bishnet.net\/tim\/blog\/?p=410"},"modified":"2011-09-23T21:27:06","modified_gmt":"2011-09-23T21:27:06","slug":"maildirarc-a-maildir-archiving-tool","status":"publish","type":"post","link":"https:\/\/www.bishnet.net\/tim\/blog\/2011\/09\/23\/maildirarc-a-maildir-archiving-tool\/","title":{"rendered":"Maildirarc – a Maildir archiving tool"},"content":{"rendered":"

I keep my email in Maildir folders. It works well on the whole for every-day email, but it doesn’t work so well for large email archives (mainly because Unix systems don’t tend to cope well with folders containing a very large number of files). My system of archiving had been to simply copy messages older than a given number of days to a different Maildir folder that I use for my archives.<\/p>\n

The problem was mainly backups. The backup tool I use (Tarsnap<\/a> – which is brilliant by the way!) was taking ages to crawl over the archive folders. In addition, the folders were taking up a lot of space on disk and compressing many small files isn’t easy without making a tar file, or similar.<\/p>\n

So I decided the best plan was to archive the messages to Mbox files. They’d compress well (in the end I just used a compressed ZFS filesystem), be backup friendly (because they’d rarely, if ever, change), and be quick to read from disk (it’s easier to read a large file than many little ones).<\/p>\n

It can’t be hard, right? Isn’t an Mbox file approximately this?<\/p>\n

cat Maildir\/cur\/* > mboxfile<\/code><\/p>\n

Well, it turned out to be more effort than that. First you need to create the \"From \"<\/code> separator line, which requires the sender and delivery date. These can be found by parsing the headers, but it’s surprising how many broken emails there were in my archives.<\/p>\n

Next you need to decide what Mbox format to use. I thought there was only one! You can either escape \"From \"<\/code> lines in the body, or you can add a Content-Length<\/code> header, or do both.<\/p>\n

After far more effort than I originally intended I came up with Maildirarc. It’s an extended version of my original shell script that just copied messages from one Maildir folder to another. I wrote it in Perl and decided to have a play with Git and Github for version control. You can see the results here:<\/p>\n

https:\/\/github.com\/tdb\/maildirarc<\/a><\/p>\n

The end result turned out to be slightly more than just an archival tool. It can also be used to do Maildir to Mbox conversions, which might be useful to other people.<\/p>\n

If you decide to give it a go please feel free to let me know how you get on by posting a comment below. If you have any ideas or changes you can fork it on Github and send me a pull request.<\/p>\n","protected":false},"excerpt":{"rendered":"

I keep my email in Maildir folders. It works well on the whole for every-day email, but it doesn’t work so well for large email archives (mainly because Unix systems don’t tend to cope well with folders containing a very large number of files). My system of archiving had been to simply copy messages older than a given number of …<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[124,130,126,127,128,129,123,125],"class_list":["post-410","post","type-post","status-publish","format-standard","hentry","category-computing","tag-maildir","tag-maildir-archive","tag-maildir-archiver","tag-maildir-archiving","tag-maildir-to-mbox","tag-maildir2mbox","tag-maildirarc","tag-mbox"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t