Extracting those last drops of performance

[Steve Freegard]

On 05/05/10 18:52, Zaeem Arshad wrote:
> On Wed, May 5, 2010 at 5:10 PM, Steve Freegard<steve.freegard at fsl.com>  wrote:
>> On 04/05/10 17:44, Zaeem Arshad wrote:
>>>
>>> I am using caching dns, ram disk for processing, clamd, compiled
>>> spamassassin rules and optimized scheduler and filesystems in this
>>> server. Any ideas on how I can tweak the above settings to gain more
>>> MailScanner performance?
>>
>> Set 'Log Speed = yes' in MailScanner.conf; then plot the average time spent
>> per message.  This is the key metric when tuning; it should be<  10 seconds
>> and preferably 2-3 secs.
>>
>
> I have already set that but I am not plotting them today. Is there any
> perl script out there or should I set on writing one? Here is a
> snippet of yesterday's MS performance under heavy load.
>
> May  4 18:58:05  MailScanner[14995]: Batch completed at 35840 bytes
> per second (11183886 / 312)
> May  4 18:58:05  MailScanner[14995]: Batch (50 messages) processed in
> 312.04 seconds
> May  4 18:58:20  MailScanner[14287]: Batch completed at 7375 bytes per
> second (2866715 / 388)
> May  4 18:58:20  MailScanner[14287]: Batch (50 messages) processed in
> 388.70 seconds
> May  4 18:58:22  MailScanner[16953]: Batch completed at 14607 bytes
> per second (4765445 / 326)
> May  4 18:58:22  MailScanner[16953]: Batch (50 messages) processed in
> 326.23 seconds
> May  4 18:58:39  MailScanner[14596]: Batch completed at 40698 bytes
> per second (13424209 / 329)
> May  4 18:58:39  MailScanner[14596]: Batch (50 messages) processed in
> 329.84 seconds
> May  4 18:58:39  MailScanner[14892]: Batch completed at 15236 bytes
> per second (5905691 / 387)
> May  4 18:58:39  MailScanner[14892]: Batch (50 messages) processed in
> 387.61 seconds
> May  4 18:58:45  MailScanner[22446]: Batch completed at 11210 bytes
> per second (3952639 / 352)
> May  4 18:58:45  MailScanner[22446]: Batch (50 messages) processed in
> 352.59 seconds
> May  4 18:58:53  MailScanner[22253]: Batch completed at 34643 bytes
> per second (12635074 / 364)
> May  4 18:58:53  MailScanner[22253]: Batch (50 messages) processed in
> 364.72 seconds
> May  4 18:59:06  MailScanner[16612]: Batch completed at 34091 bytes
> per second (12036761 / 353)
> May  4 18:59:06  MailScanner[16612]: Batch (50 messages) processed in
> 353.07 seconds
> May  4 18:59:17  MailScanner[12768]: Batch completed at 30726 bytes
> per second (11356957 / 369)
> May  4 18:59:17  MailScanner[12768]: Batch (50 messages) processed in
> 369.61 seconds
> May  4 18:59:28  MailScanner[13267]: Batch completed at 31812 bytes
> per second (11076141 / 348)
> May  4 18:59:28  MailScanner[13267]: Batch (50 messages) processed in
> 348.17 seconds
> May  4 18:59:30  MailScanner[15241]: Batch completed at 71356 bytes
> per second (23231603 / 325)
> May  4 18:59:30  MailScanner[15241]: Batch (50 messages) processed in
> 325.57 seconds
> May  4 18:59:56  MailScanner[15478]: Batch completed at 12213 bytes
> per second (4247264 / 347)
> May  4 18:59:56  MailScanner[15478]: Batch (50 messages) processed in
> 347.75 seconds
>

Average scan time is 6-7 seconds based on those numbers.  That's a 
little high but not bad.

>>   Are you using
>> Bayes?  If so - using which storage method?
>
> Yes I am using file based bayes stored on tmpfs. I do take regular backups.
>

Try MySQL + InnoDB with the Mail::SpamAssassin::Bayes::MySQL driver - 
according to the benchmarks it's by far the fastest storage method and 
it won't suffer from the locking issues that you'll see with file-based 
bayes.

Regards,
Steve.