Idealog: So you want to host your own Linux mail server…

Categories

UPDATE: There is now a HowTo on setting up SpamAssassin to train across all the mailboxes on your server here.

UPDATE 2: Do not sign up with RedwoodVirtual as a hosting company right now. There customer service sucks. See this post.

For the past couple of years, I hosted my personal domain through a Windows hosting company, CrystalTech.com. They provided mail services via SmarterTools’ SmarterMail 2.0.

CrystalTech’s SmarterMail implementation provided 250 mail accounts, a slowish Web-mail interface, IMAP or POP mail client access, virus scanning, junk mail filtering and content filtering for about $16.95 per month (incl. Web hosting for the domain) and $5 per month per 50 MB of space.

Several events led to me to consider a change. First, I’ve always had trouble using POP with Microsoft Outlook from multiple computers. The read/unread state of mail was never synchronized across systems. Every once in a while, Outlook would download my entire mailbox as new messages – creating duplicates for every message that required an add-in tool to remove. Eventually, I switched to IMAP to avoid these problems.

Unfortunately, IMAP and Microsoft Outlook don’t work very well. Microsoft spends most of its time making Outlook work well with Exchange Server. As I wrote in Microsoft’s Sacred Cash Cow, my Windows XP system at work was crashing regularly at work. Once I stopped using Microsoft Outlook 2003 and replaced it with Mozilla Thunderbird – my Windows XP system stopped crashing.

Once my colleague, Kellan, set up a Linux Debian server to host our Groundspring Seattle office's email, Thunderbird and IMAP worked amazingly well. Unfortunately, CrystalTech’s SmarterTools doesn’t work very well under IMAP with Thunderbird. Furthermore, as I added accounts for other family members, the disk space limits we’re starting to add up.

I wanted an unlimited depository for my email. I wanted more control. I wanted the potential to create a more searchable index for my email over time.

So, Kellan suggested I consider hosting my email on my own Linux Debian server. He pointed me to RedwoodVirtual. RedwoodVirtual provides virtual Linux machine hosting for as little as $100 per year (prepaid). Essentially, they provide you a shared server that appears to be a standalone Linux installation e.g. you have root access, unfiltered access to the Internet, etc. Most importantly, RedwoodVirtual provides 2 GB of hard disk space as part of its base package. The promise of unlimited mail storage was at hand.

The only problem it seemed is that I wasn’t a Linux system administrator. Yet, with some early guidance from Kellan and some helpful instructional scripts he’s gathered from around the Web, I had my mail server up and running in a few hours – working almost exactly the way I wanted it within a few days.

The configuration I’ve set up is:
Linux Debian + Postfix + Courier + Apache + SquirrelMail + Amavis + SpamAssassin

I haven’t yet finished my Clam Anti-virus configuration.

My primary concerns at this point are data robustness. RedwoodVirtual does not provide tape backup. They only have a high reliability RAID drive. So, it will be important for me to backup my mail to my local system. I can have Thunderbird do this on my desktop Mac by downloading the contents of IMAP folders. And/or, I can set up a script to regularly rsync my maildir folders to my home hard drive.

Some of you may ask why I don't use GMail more - they can't host my domain - but they do offer the opportunity to archive 1 GB of mail. I don't like the GMail Web interface for primary work. And, it turns out there are legal privacy distinctions for mail hosted on a shared Internet server e.g. Yahoo, Hotmail or GMail and mail hosted on your own server. I'm presuming my virtual machine at RedwoodVirtual is considered my private email as it is hosted on my own domain - though I could be wrong on this. Anyway, according to Jason Schultz of the Electronic Frontier Foundation (I interviewed him for my recent article, Citizen Microsoft), if John Ashcroft wants my GMail - he only need write a memo to get mail older than three months. Yet, if John Ashcroft wants email on my personal server, he must obtain a search warrant signed by a Judge.

I was able to move my mail hosting off of a Windows 2003 SmarterMail 2.0 platform and go entirely open source for less money, with nearly unlimited mail storage. I hope this helps others make similar moves toward better technology that provides more control over their email. There are new possibilities open to me now for managing my mail that weren't available with a closed Microsoft applications platform.

It's a credit to the progress of open source software that I'm able to install my own free mail server within a few hours. Although, there is still work to do to make this easier and more well documented.

I was most impressed that it only took three minutes under Debian to install Apache and SquirrelMail before I was off and running with Web mail. Kick ass!

Continue to the installation guide..

I also picked up a few O'Reilly books to help get me up to speed with Linux...

I couldn't find any Debian books at Barnes & Noble but here are two as well:

Comments

Nice work - the recommendation to use sudo in your setup instruction is unusual but excellent - most folks just say "su root" but having trackability of admins actions (if for no other reason so you have a log of what the heck you did!) is a "good" thing.

BTW, it looks like you write about a combination of technology and politics ... so you might enjoy the "Vote for HULK" site at http://www.komar.org/cgi-bin/halloween_webcam

Posted by: hulkster | Oct 17, 2004 at 08:55 AM

i to setup my own *nix server but used FreeBSD, Qmail, Qmail-Scanner, Spamassassin, clamav. Its been up over 200 days w/o a reboot. It just runs, I agree with IMAP and Outlook but Outlook Express and IMAP work fast however OE is a bit of a security hole! Good luck.

Posted by: Steve | Oct 17, 2004 at 08:06 PM

Hosting your mail on RedwoodVirtual's server is the same as storing it on GMail servers. The hosting company owns the server; you are leasing it, or have contracted its use. Depending on the company's privacy policy, your data may not be more private than GMail, or it might be.

Anyway, another great option for running your own personal mail/web/whatever server is to find a cheap colocation deal-- you bring your own server hardware to the facility and plug it in to their internet connection. They provide the network, the server is completely your responsability. I use MV Communications' (http://www.mv.com) inexpensive "personal co-location pool" service -- all the servers in the pool share a certain allotment of bandwidth. You can spike in bandwidth usage, but if you use more than your fair share for a sustained period you have to get out and get a more serious colocation plan (or limit your bandwidth).

I just carried my battered old PC into a corner of their server room, and plugged it in next to everyone else's battered old PC. However, if it goes down and won't reboot, or I screw it up horribly, I'll have to drive up there and fix it. But I have complete control and ownership over it. And they're really nice people :)

Posted by: reed | Oct 18, 2004 at 09:55 AM

I'm just going to throw in my recommendation for Cyrus IMAPd if you ever need to take your mail "up the scale". It has a (deserved) reputation for awful or absent documentation and difficult authentication set-up, but once running it's rock solid and incredibly fast.

Cyrus supports POP3 and IMAP, though it really shines with IMAP. In particular, it keeps good mailbox indexes and can keep full-text indexes for server-side IMAP SEARCH. I can search 30,000 message mailboxes in a second or two.

If you're looking for something a little ... friendlier, dovecot is becoming very popular for a simple, solid, and reliable mail service that's a little less long in the tooth than Courier.

Posted by: Craig Ringer | Oct 19, 2004 at 11:15 AM

To clarify - Cyrus 2.1.x and 2.2.x are excellent. I'd advise staying away from 1.x.

On Debian, the best way to get an up-to-date Cyrus seems to be to use Sarge or compile from source.

Posted by: Craig Ringer | Oct 19, 2004 at 11:21 AM