Performance Enhancements

[Julian Field]

At 08:17 10/01/2003, you wrote:
>I suspect that this load is cause because milter runs the mail through
>SA as it arrives and rejects is before it even enters the server and MS
>runs SA on mail in batches. Maybe it is just too much for any server to
>have multiple instances of SA running at the same time?

SA is quite memory-hungry. You can always try reducing the number of child
processes to, say, 2 and see if that actually performs better than 5. My
guess at 5 was made on the basis of using all the RAM in a 512MB machine
with some sample test messages. I would not be at all surprised if it
wasn't the right figure for other systems.

Keep an eye on the overall load, as some systems (SPARC systems come to
mind) have a very high context-switching overhead, so the more
CPU-intensive jobs it is switching between, the slower it goes.

>On Fri, 10 Jan 2003, Scott Adkins wrote:
> > Okay, since we turned on MailScanner with Spam Assassin and Sophos, we
> > have definitely seen high load come out of the server... It looks like
> > the culprit is Spam Assassin.  SA seems to take about 5 times as much
> > CPU to process mail as what Sophos does (which is backwards from what
> > I was expecting, actually).
> >
> > The system we are running this on is a pretty decent system.   It is a
> > two member Compaq Alpha Tru64 5.1a cluster.  One member is an ES40 with
> > 4 EV6.7 667Mhz CPU's, 4GB CPU cache and 8GB memory.  The other member
> > is an Alpha 4100 with 4 EV5.6 600Mhz CPU's, 4GB CPU cache and 6GB memory.
> > The first member is more than twice as powerful as the second member is.
> >
> > This is our primary email system, and we regularly see 400-500k worth of
> > emails go through the system on a daily basis.  We support well over 60k
> > users and typically have 1500+ concurrent IMAP/POP users logged onto the
> > system.  The system performs great under these conditions...
> >
> > The idea was to run MailScanner and mail queue processing on one machine,
> > and our Cyrus IMAP and IMSP servers, as well as everything else on the
> > other machine.  We still saw high loads coming from the MailScanner stuff.
> > In fact, MailScanner literally drover our second member into the ground
> > (poor thing).
> >
> > I am interested in what other large sites have done to optimize the
> > processing of spam and virus scanning.  I currently run with 20 MailScanner
> > processes, since we have 4 CPU's.  From what I can tell, it pulls in 100
> > messages at a time to process in a large batch and then sends them on their
> > way.  Doing it this way shows that disk IO gets slammed, and when it does
> > recover, the CPU gets slammed, and then it starts all over again.  I am
> > thinking that maybe processing smaller chunks of emails might even out the
> > load a little and maybe make things run a bit better.
> >
> > Another thought is with Spam Assassin.  I know it has the capability to run
> > in daemon mode (spamd).  Does MailScanner even support this?  Does running
> > spamd in daemon mode give you any performance advantage at all?
> >
> > Anyways, I thought I would check to see what other people are doing...
> >
> > Thanks!
> > Scott
> >
>
>
>--
>This message has been scanned for viruses and
>dangerous content by MailScanner, and is
>believed to be clean.

--
Julian Field
www.MailScanner.info
MailScanner thanks transtec Computers for their support