Things n' Stuff Thoughts about the universe in general

I’m not really proud of the fact, but my work place has decided to migrate the mail server (after several tests of open source platforms) to Microsoft Exchange. For that we need to copy the current mail boxes of users from the server being phased out to the new MS-Exchange mail server.

In previous migrations we’ve used the excellent imapsync program which is written in perl and connects using IMAP to two servers and copies messages and folders from one server to another. Nonetheless, trying to get imapsync to work with the beast that is MS-Exchange has proved to be daunting. This is not really a rant (although you can read it as thus – I won’t mind at all 😉 ), but trying to get the thing to work I searched the web high and low and found precious little information about migrating to MS-Exchange¹, so this is basically a summary of my recollection of the process in hopes that others will find this information useful².

To make a long story short, the main problem is IMAP flags – A standard feature in the IMAP spec, MS-Exchange doesn’t really support them, as discussed in the project page for something called imap2exchange (In this project the author appears to be using MS-Exchange web services, which is not what I had in mind, but he explains IMAP flags and the MS-Exchange problem with them quite clearly). Strictly speaking, any message that has more then just the completely obvious flags (specifically, Seen only) is rejected with an error message:

BAD Protocol Error: "Specified set of flags is not valid".


I’ve seen some people offer a solution to just use an older version of imapsync (possibly because it does not attempt to sync flags), but I have a better one –

imapsync offers an undocumented switch called --regexflag which allows you to run a regular expression on the message flags before storing the message to the target server. I use this feature to remove almost all the flags that MS-Exchange doesn’t like. So my command line for imapsync look like this:


imapsync --fast --subscribe --noauthmd5 --skipsize --syncinternaldates --host1 source.mail.server --host2 target.mail.server --password1 'source-secret' --password2 'target-secret' --skipheader '^Content-Type' --user1 source-user --user2 target-user --regexflag 's/receipt-handled//' --regexflag 's/$Forwarded Forwarded//' --regexflag 's/\Answered//' --exclude Junk


The first regexflag substitution gets rid of the Zimbra private “receipt has been handled for this message” flag (yes, the source server is Zimbra). The second one gets read of the commonly used “message has been forwarded” flag (and a private Zimbra copy of the flag), while the third gets rid of the very standard “message has been replied to” flag which MS-Exchange doesn’t understand either.

MS-Exchange additionally doesn’t understand any junk mail flags, to in my scenario I simply do not copy the “Junk” folder under the assumption that it contains only SPAM that users do not want anyway. Another problem is the standard Deleted flag which MS-Exchange doesn’t like either, and under the same logic as the junk problem I’m going to assume that users do not want to keep deleted messages and simply let imapsync fail on those emails (the author of the imap2exchange project thinks that we should move such messages to the MS-Exchange deleted items folder, and he of course is right, but I’m too lazy to do that).

The main problems with losing flags, is that some are very important – I’m not aware of any one that the Answered flag is very important to, but there are other useful flags such as “mark for followup” which are rather important.

And if this was not trouble enough, these are not the only problems. Another problem with the migration is that MS-Exchange fails to store messages which are too long, and imapsync reports:

NO This message is too big and can not be appended.

By default MS-Exchange rejects messages that are over 10MB in size. 10MB is a default size for other mail agents as well, for example postfix, and its a good idea to increase this immediately after you install your mail server. After you change the appropriate setting in the MS-Exchange server configuration, it should be alright.

We haven’t finished the migration yet, but considering the above caveats, it looking OK for now. Of course, living with an MS-Exchange mail server is a whole new ball game, and about that – at a different time 🙂

1. Hopefully that means that not a lot of people are doing it ? [↩]
2. And in higher hopes that others will not need this information 😉 [↩]

This entry was posted on Tuesday, June 3rd, 2008 at 17:18 and is filed under Tech. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.