Minimum hardware capacity for 35k e-mail scans/day

[Julian Field]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Are you actually seeing mail queues getting big, particularly in 
exim.in/input?
Just having a high load-average figure is not a good sign that the 
machine is struggling, you need to look to see if it is keeping up with 
the incoming mail. What size do the batches get to be? Best thing to 
look for is
    tail -f /var/log/maillog | grep 'New Batch'
and see how many it is scanning at a time. If this is setting less than 
about 4 or 5, and not walking up to 30 in a batch, then that really is 
getting overloaded. Ignore the "messages waiting" figure, that includes 
messages that are in the process of being received from remote SMTP 
servers belonging to other people.

Quad-core Xeon boxes with 8Gb RAM are pretty cheap these days. I just 
bought one with mirrored 500GB SATA disks and a hardware RAID 
controller, along with a quad-core CPU and Gb RAM for under 1500 quid 
(i.e. about US $2500 or less) from Dell.

Let us know how big your batches get.

Jules.

Edward Prendergast wrote:
> Hi,
>
> We are reaching capacity with our current hardware setup. Our current
> machine is a Dual Xeon 2.40GHz + 2g RAM workhorse which runs all services
> required for web hosting (MySQL, Apache, Exim + Courier + MailScanner +
> MailWatch + SpamAssassin + ClamAV, named) but are looking to separate this
> out as the machine can no longer handle the load inflicted by all these
> services.
>
> By 5pm we've usually scanned 35,000 mails, 10% of that legitimate. We reject
> at MTA based on greet-pause and invalid recipients. We have tried to reduce
> the amount of users with catch-alls to the absolute minimum. We went on a
> Mail optimisation overhaul a few months ago during which time MailScanner's
> working area was switched to tempfs and we started using compiled rules. 
>
> We archive mails for 7 days and keep 20 days worth of mail. 
>
> The only bottleneck on our current setup appears to be CPU - vmstat looks
> like this:
> procs -----------memory---------- ---swap-- -----io---- --system--
> ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id
> wa
>  2  3 311500 801316  37144  97396    1    0     3     1    0     0 15  5 70
> 10
>
> With bayes on, the load average hikes up to around 10, 11 or higher
> depending on time of day. Without bayes the load usually sits around 5/6. 
>
> At present the hardware budget is fairly constricted so I'm looking to find
> out what minimum level of hardware would be required to run:
>
> MailScanner 
> MTA in gateway mode (forwarding scanned mail to existing server)
> MailWatch (MySQL DB stored elsewhere)
> SpamAssasin, compiled rules, bayes
>
> Any advice from the vast pool of MailScanner experience would be greatly
> appreciated.
>
> Regards,
> Edward
>
>
> ************
> The information in this email is confidential and may be legally privileged.
> It is intended solely for the addressee. Access to this email by anyone else
> is unauthorised. If you are not the intended recipient, any action taken or
> omitted to be taken in reliance on it, any form of reproduction,
> dissemination, copying, disclosure, modification, distribution and/or
> publication of this E-mail message is strictly prohibited and may be
> unlawful. If you have received this E-mail message in error, please notify
> us immediately. Please also destroy and delete the message from your
> computer.
> ************
>
>   

Jules

- -- 
Julian Field MEng CITP
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

MailScanner customisation, or any advanced system administration help?
Contact me at Jules at Jules.FM

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
For all your IT requirements visit www.transtec.co.uk


-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.7.0 (Build 867)
Comment: Use Thunderbird's Enigmail add-on to verify this message
Charset: ISO-8859-1

wj8DBQFHO0AREfZZRxQVtlQRAuFHAJ9jstNeTu3A8mQH1AGsTvHLobjgLACg794e
85oLnKgg0KmeQLIzjVCK6Yg=
=fG2Z
-----END PGP SIGNATURE-----

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.