mailscanner with heavy load

Steve Freegard steve.freegard at fsl.com
Mon Feb 2 14:30:00 GMT 2009


Hi Paulo,

Paulo Roncon wrote:
> Steve : 60msg/sec = 60msg*60s*60m*24h= 5184000!!!

Yeah - sorry about that - noticed that after I posted.

> So 5 childs/1RAM/1CPU => 20childs/4RAM/4CPU, right???

Yes.

> But i have RAM to spare... can I increase the number of childs?

The 5 children per 1Gb RAM/CPU isn't set in stone - it a balancing act
to put as much load of the various components (CPU + CPU+Mem+Disk IO)
without causing the machine to thrash or become bound by one component
(e.g. CPU or Disk), so you *might* be able to increase the number of
children - you'll need to measure the batch times to find out what the
optimal values are given your typical loads.

To do this you'll need to fully load the box (you could do this with
test messages - but they should be representative of the actual mail
stream) and to do that you'll need to make sure that all children are
busy; so if you have a batch size of 30 (the default) you'll need (Batch
Size * (Number of Children + Batch Size)) = messages (10 children/30
batch = 330) in the mqueue.in directory (or hold directory if you use
Postfix) and then start MailScanner (with Log Speed=yes) and then
calculate the avg. scan time per message based on the log output.  Start
increasing the number of children and see the effect of the avg. scan
time - as soon as the scan times start to show a downward trend; you
have found the limit (be sure to leave a 'fudge' factor of around -2
children to allow for variances in your message types).

You'll want to follow all the usual tuning advice for MailScanner e.g.
/var/spool/MailScanner/incoming on tmpfs, so I tend to find that having
surplus memory improves performance with MailScanner particularly if you
have large rulesets and that most installations are typically IO bound;
so your disk controller and RAID configuration are important (RAID 0+1
on /var will likely give the best performance).

Speaking from experience - if you allow your users to make white/black
list entries, then you'll probably want to use CustomFunction to read
the values from an indexed datasource (something supports row-level
locking) as rulesets are compiled at child start-up and large rulesets
cause MailScanner to take a long time to reload (remember: MailScanner
restarts each child every 2 hours by default - a several minute start-up
time will cause a significant backlog at 60msgs/sec) and will also cause
each child to use up a *lot* of extra memory.

Cheers,
Steve.

-- 
Steve Freegard
Development Director
Fort Systems Ltd.


More information about the MailScanner mailing list