Filtering mail into IMAP folders


I've spent another large chunk of my time on something I've wanted for a long time: keeping all my mail in one IMAP account. The idea is that I want all mail i receive to enter that one IMAP account and be automatically sorted server-side into the correct folders.

Sounds as something which enough other people want, so there should be enough tools to help me. Setting up an IMAP server is quite easy, dovecot does a good job. The quest for good software to download my mail and filter it into the correct folders is slightly harder.

I quickly found two programs for downloading mail: Fetchmail and getmail. Fetchmail requires a seperate MTA to actually deliver the mails, while getmail can nicely write into the maildirs I want to use. So on with getmail it is.

Downloading mail works just fine with getmail, sorting them into the various folders requires seperate programs. The biggest problem is on what criteria I want to filter my mail. I have a large number of mail addresses enabled on my domain, and I usually want to sort depending on the mail address of my domain the mail is meant for. The mail daemon for my domain however forwards all mail to one catch-all address which it then nicely puts in all usable header fields for the receiver. So the actual receiver of the mail is unusable.

On the other end of the forwarding pipeline there are other issues. Quite often mail is forwarded to various mail addresses before it reaches me. For example a server sends errors to it's local root address, which is set to forward to a general server administrators address, which then forwards to me (and others). This means that the original receiver is not usable either.

My problem should be clear by now :) I found that there is some header I can use to find the mailaddresses I want to filter on: the Received header. It contains information about all hops the message passed on it's way to me. The downside is that the mail address is within loads of other information somewhere halfway the body of this header.

The simple way to fix this was to write a program which would extract the useful mail address and put it by itself in a separate header which getmail can then easily filter on. So i set up getmail to filter the messages through that program and than abuse the multidrop system to read that header. Problems. The multidrop header is expected to exist when the message is downloaded from the server, while I only add it in the filter. Solution failed.

For more advanced filtering, getmail refers to maildrop. It's a nice program with a perl-like syntax to write scripts in which determine where your mail goes. It can put the mails into my maildir just fine, so another win. I configured getmail to use maildrop to deliver the mails and started to write a mailfilter file for maildrop.

The syntax is actually harder than it should be. I very often make mistakes which basically mean that no mail is being processed anymore. So i thought: why not write an easy maintainable script to filter the mail addresses myself? Same idea as I had with getmail: put the destination in the headers in the script an let maildrop figure out where the maildir for that destination is. This works, but it seems that mail headers are pretty free-form and parsing them well is probably something that would take me days to do correctly.

So that's where I am currently. It will all work eventually, but in the end it will have cost me days to set it up and migrating everything will take quite some time as well. Let's see if I want to put that much time in it still, or am I missing an obvious, much simpler approach to my problem?

Comments (1)

Posted by Nan on 18 Jan 2015 at 05:41

WOW just what I was looking for. Came here by earching for seawrch

Post your own comment
Email: (optional)
Website: (optional)