ntang (ntang) wrote,

Sendmail: 1, Postfix: 0. (Exim: -5.)

We've been doing some testing to see which mailer performs better as a mass-mailer, since we typically send over half a million emails to our members every day, things like reminders that they've gotten new notes (sort of like email), that they've gotten new dating matches, etc. (It's all legitimate site-news stuff and opt-out-able, if you're wondering, and with nearly 10 million users, half a million a day isn't that much.)

So here's the interesting thing we're finding. I predicted it, but everyone else was rooting for postfix.

Well, so far, the sendmail server is able to send at least 25% more (and closer to 50% more) emails in the same time period as the postfix server - and either one blows the exim servers out of the water. (We've been running exim for a while, and I finally got sick of it and decided to move on to a better mailer.)

Here are some samples, taken over the same time period:

batch one:
sendmail: 4919 (this was running for less than half the time of the others)
postfix: 11000
exim1: 3000
exim2: 1250
exim3: 3250

batch two:
sendmail: 17000
postfix: 12204
exim1: 3250
exim2: 2500
exim3: 3000

batch three:
sendmail: 131741
postfix: 86500
exim1: 22248
exim2: 18749
exim3: 17997

Addendum: It's worth noting that the sendmail and postfix boxes are both Redhat 7.2, with a 2.4 kernel, and the exim boxes are all Redhat 6.2 with a 2.2 kernel. The kernel undoubtedly contributes to the performance difference. The sendmail and postfix boxes also have faster cpu's, but they also have slightly slower disks, so I'm not sure how much of a difference that'll make considering how IO-intensive mailing is. The exim boxes have spool directories running reiserfs; the sendmail and postfix boxes are using the standard ext3fs, w/ data=journal as the only mount option.

In short, it's not entirely a fair test, there are many, many variables. Exim would undoubtedly perform better on a 2.4 kernel, and I'd be willing to bet ext3fs is faster than reiserfs, and the faster cpu's aren't going to hurt even if I don't think they'd help much. I also think, though, that even with everything else being equal, that the exim boxes would still be the slowest of the bunch.

I'm tempted to build out 2 more mail servers; one new one (with the same hardware specs and the 2.4 kernel) running exim and one new one running qmail, just so we can say we really covered the bases on this test. Also, more tuning needs to be done on the postfix and sendmail servers, both are using mostly out-of-the-box configs and could undoubtedly be extensively tuned. All of the machines also have just a single hard disk, so are definitely IO-limited there.

For the final mail servers, I think I'll get boxes with dual (or perhaps even more) high-rpm scsi disks instead of the cheap single-ide disks the current ones have, and we can lay out the disks differently, tune them with hdparm, make sure that the kernel is tweaked for mailing (both filesystem and network), and then tune the hell out of the mailers themselves. Should be fun, actually. The goal is to have them send 1m mails per day per box, so that means we need to be sending 12 mails per second per box to achieve that goal. That should be cake. If we can get them sending 15.5 mails per second, then we can even scale the cluster down from 5 boxes to 4 boxes, which'll save a bit of money and would be nice just for the sake of doing it.

Update 2: I take that back. The server exim3 is running w/ redhat 7.2 and a 2.4 kernel and ext3fs. Ok, exim just plain sucks.
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded