I had the admin apply some changes to our mailservers, and we sent another batch just now. With the same OS-level configuration applied to the exim boxes (we upgraded the RPM's to the latest ones, including the kernel; we remounted the mail spool with data=journal, and made a few other tweaks) the exim box actually outperformed the postfix box! Sendmail was, still, the fastest.
The latest results:
sendmail: 9500 mails sent (throughput of 6.76 mails/sec.)
postfix: 6000 (4.16/sec.)
exim1: 1750 (1.25/sec.)
exim2: 750 (0.53/sec.)
exim3: 6651 (4.68/sec.)
We've got 2 more batches going out tonight; I'll post updates as those go out. All I can say is: yowza! Suddenly this competition is getting interesting. It looks like Exim might actually be a lot better than I thought. I'm also impressed at what an incredibly drastic difference a few minutes of system tuning can make - it's one of those things you're always told, but I've rarely seen an effect this dramatic.
How the test works:
We have 5 mailservers. 3 are old boxes, running exim. Of those boxes, 2 are running Redhat 6.2, and 1 is running Redhat 7.2. The 2 new boxes are running Redhat 7.2, and one is Sendmail, one is Postfix. We have a database w/ a table of recipients, and a fairly static mass mailing (other than username and a few other things that are inserted into the text, it's all static). The mails are pulled in bunches, 250 per bunch, and each server grabs a bunch, sends it, then grabs the next bunch. When the mail server contacts the db and finds the table empty, it then considers itself done and fires off a mail to us summarizing its results. The script is a perl script and uses Net::SMTP sending with 127.0.0.1 (i.e., itself) as its server, sending each email individually.
The 7.2 boxes are configured identically, from an OS level: they have the latest vendor RPM's for everything including the kernel. They have a single IDE disk internally, all of the partitions are ext3, and their mail-spool directory is mounted w/ data=journal. They are all configured to log through syslog, and are logging asynchronously. In addition, they've had a single kernel tweak: their file-max settings are bumped to 65336 to make sure they don't run out of filehandles. They're running the latest stable versions of their respective mailers (exim, sendmail, postfix), all built from source, and all running mostly untuned configs (at least not tuned for high-performance).