Performance Enhancements

[Remco Barendse]

You are right in your suspicions about SpamAssassin. I have encountered
the exact same problem on several boxes, when I disable SA there is almost
no load on the server, when I enable SA to load goes through the roof!

A friend of mine only wanted spam scanning and not virusscanning and first
tried MS/SA but backed away due to extremely high load figures. He is now
using SA with sendmail's milter option with much better load figures.
They have a relatively large volume server (hosting provider).

I suspect that this load is cause because milter runs the mail through
SA as it arrives and rejects is before it even enters the server and MS
runs SA on mail in batches. Maybe it is just too much for any server to
have multiple instances of SA running at the same time?

On Fri, 10 Jan 2003, Scott Adkins wrote:

> Okay, since we turned on MailScanner with Spam Assassin and Sophos, we
> have definitely seen high load come out of the server... It looks like
> the culprit is Spam Assassin.  SA seems to take about 5 times as much
> CPU to process mail as what Sophos does (which is backwards from what
> I was expecting, actually).
>
> The system we are running this on is a pretty decent system.   It is a
> two member Compaq Alpha Tru64 5.1a cluster.  One member is an ES40 with
> 4 EV6.7 667Mhz CPU's, 4GB CPU cache and 8GB memory.  The other member
> is an Alpha 4100 with 4 EV5.6 600Mhz CPU's, 4GB CPU cache and 6GB memory.
> The first member is more than twice as powerful as the second member is.
>
> This is our primary email system, and we regularly see 400-500k worth of
> emails go through the system on a daily basis.  We support well over 60k
> users and typically have 1500+ concurrent IMAP/POP users logged onto the
> system.  The system performs great under these conditions...
>
> The idea was to run MailScanner and mail queue processing on one machine,
> and our Cyrus IMAP and IMSP servers, as well as everything else on the
> other machine.  We still saw high loads coming from the MailScanner stuff.
> In fact, MailScanner literally drover our second member into the ground
> (poor thing).
>
> I am interested in what other large sites have done to optimize the
> processing of spam and virus scanning.  I currently run with 20 MailScanner
> processes, since we have 4 CPU's.  From what I can tell, it pulls in 100
> messages at a time to process in a large batch and then sends them on their
> way.  Doing it this way shows that disk IO gets slammed, and when it does
> recover, the CPU gets slammed, and then it starts all over again.  I am
> thinking that maybe processing smaller chunks of emails might even out the
> load a little and maybe make things run a bit better.
>
> Another thought is with Spam Assassin.  I know it has the capability to run
> in daemon mode (spamd).  Does MailScanner even support this?  Does running
> spamd in daemon mode give you any performance advantage at all?
>
> Anyways, I thought I would check to see what other people are doing...
>
> Thanks!
> Scott
>


--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.