MS performance as a gateway comments

[Alan]

This is just some of the things I have experienced over the last year
related to performance. FWIW, maybe it will help someone down the line...

I have been running MS/SA for about a year now, and have been struggling
with performance issues since day one. Over this time, many of you have
greatly helped me in my tweaking, including:

tempfs for working dir
seperate scsi drive for queue partition
reiserfs for queue fs
dual pent 4 with 2 gig ram
minimal logging
smaller batch size (20)

In my configuration, I have 4 smtp servers receiving email from the outside
world. Each of these 4 servers route the mail they accept directly to my
MS/SA server for spam/virus/exploit filtering. Everything that is passed by
MS/SA is then finally sent on to a final smtp server where the inboxes
reside. So you can see that my MS/SA server acts as a gateway; it accepts
no email directly from the outside world, and it has no inboxes on it.
About 300,000 emails per day enter the MS/SA server, and its filtering out
about 72% of that volume, passing the rest on to my users.

In my original install, I was using sendmail as the MTA. During much of the
day, so many MTAs were running that server load would hover around 14-20.
During quieter times, it would drop to maybe 5-7, never below. If I
configured sendmail to limit MTAs, I would have backups on the 4 incoming
smtp servers as they could not deliver their queues to the MS/SA server
fast enough, and/or there would be backups in the delivery queue of the
MS/SA server itself as there were not enough MTAs available for final
delivery to the inbox server. If I ran enough MTAs to handle incoming and
outgoing traffic, load would be so high that MS became the bottle neck.

Bayes tests were out of the question due to load, and various network tests
(razor2, surbl etc.) often had to be disabled when back logs were bad. It
was not uncommon each day to have 20,000 messages waiting to be processed.
During daily heavy load, the time from when an email hit my first server
until it was dropped into the user's inbox would be 4-7 hours.

A couple of months ago I replaced sendmail as the MTA with exim. I really
liked exim's configuration flexability and features, and was quickly able
to get some neat tweaks running. I found that each exim MTA seemed to use
less resources than each sendmail MTA had used, and I was able to keep
network tests running much more often. It was an improvement, but loads
were still usually above 8 and total delivery time was still poor,
sometimes as high as 2 hours during the day.

Now I have replaced exim with zmailer, and all I can say is WOW! First,
zmailer reuses the same tcp/ip connection to my final inbox server for as
long as there are messages in the delivery queue to be delivered. This
means there is never more than a single MTA running to deliver messages.
Secondly, the zmailer smtpserver process that receives incoming messages
from my 4 smtp servers, along with the zmailer smtp process that delivers
filtered messages to my inbox server all use minimal system resources.

Since switching to zmailer as my MTA, system load is averaging under 4, and
very few peaks above 5, and they are very short. Pre-filtered queue size
has yet to go over 100 during peak hours, as opposed to previous queue
sizes of over 10,000. Delivery queue size is likewise low, typically under
50. Delivery time from initial reception to user inbox is now down to 3
minutes or so.

So, if you are struggling with system load and bottlenecks as I have been,
and have made all the hardware tweaks you can think of, I HIGHLY recommend
you try zmailer as your MTA.

------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).