Performance numbers for a DELL R710

[Steve Freegard]

Zaeem Arshad wrote:
> 
> 
> On Wed, Jun 3, 2009 at 12:47 AM, Martin Hepworth <maxsec at gmail.com
> <mailto:maxsec at gmail.com>> wrote:
> 
>     Zaeem
> 
>     nice.
> 
>     depends on the tests you run (RBLs etc) and the size of the emails.
> 
>     look in the wiki for performance and tuning on both MailScanner and
>     Spamassassin.
> 
> 
> 
> I have a test box with the same specs but 8 Gigs of RAM. My performance
> tuning so far has been
> 
> - tmpfs for message scanning
> - DNS caching server on the same box
> - Lower timeouts on Postfix (another instance handling retries)
> - Compiled rules on SA
> - DCC, Razor
> - Clamd
> - Asyncrhonous logging
> 
> Filesystem underneath is XFS and average mailsize is around 60KB. I have
> pretty much made all the changes suggested and currently the server is
> handling around 45000 emails/hour. Do you think increasing the number of
> MailScanner children might help? What other performance tweaks can I
> have? I am looking to scale the system to handle at least 65
> emails/second with antivirus, antispam scanning and RBL checks.
> 

With 8 cores and 8Gb RAM or greater then you can start with Max Children
= 40 and measure the performance, then steadily increase the children by
5 until you start to see the performance worsen.  I personally find the
batch size to be largely irrelevant and usually stick to batches of 30,
50 or 100 (the larger the batch size; the greater the delivery latency
of any given message).

65 message/sec is 5,616,000 messages per day; with volume like that you
will *have* to purchase licensed feeds of Spamhaus, URIBL and/or SURBL
as you will exceed their fair usage limits even if the usage is
non-commercial.    There is also a big advantage when running the lists
locally as it decreases your scan times considerably due to the
decreased lookup latency.

Other performance tuning I would recommend:

- Don't run DCC, Razor or Pyzor (unless you run a local mirror); the
network latency is simply too high and will drastically reduce the
number of message you can handle.
- Disable all unnecessary RBLs in SpamAssassin, the rfc-ignorant lists
in particular as pretty worthless so cut those out at minimum; I would
personally nuke any RBL that you don't run a local mirror of.
- Mount your /var partition with noatime.
- Use sa-compile and enable the Rule2XS SA plug-in.
- Put Bayes in MySQL with InnoDB tables and use the persistent DBI plug-in.
- Disable Bayes opportunistic expiry and run it manually in a cronjob
once per day.

Finally - enable the 'Log Speed = Yes' option in MailScanner.conf and
watch your total batch times.  To reach 65 messages/sec you'll need to
keep the average time per message (e.g. total batch time/number of
message) to under 1.625 to achieve your goal with 40 MailScanner
children.  If you can get away with running > 40 children then the
average scan time can increase accordingly.

Cheers,
Steve.