Extracting those last drops of performance

[Zaeem Arshad]

On Wed, May 5, 2010 at 11:18 PM, Steve Freegard <steve.freegard at fsl.com> wrote:
> On 05/05/10 18:52, Zaeem Arshad wrote:
>>
>> On Wed, May 5, 2010 at 5:10 PM, Steve Freegard<steve.freegard at fsl.com>
>>  wrote:
>>>
>>> On 04/05/10 17:44, Zaeem Arshad wrote:
>>>>
>>>> I am using caching dns, ram disk for processing, clamd, compiled
>>>> spamassassin rules and optimized scheduler and filesystems in this
>>>> server. Any ideas on how I can tweak the above settings to gain more
>>>> MailScanner performance?
>>>
>>> Set 'Log Speed = yes' in MailScanner.conf; then plot the average time
>>> spent
>>> per message.  This is the key metric when tuning; it should be<  10
>>> seconds
>>> and preferably 2-3 secs.
>>>
>>
>> I have already set that but I am not plotting them today. Is there any
>> perl script out there or should I set on writing one? Here is a
>> snippet of yesterday's MS performance under heavy load.
>>
>> May  4 18:58:05  MailScanner[14995]: Batch completed at 35840 bytes
>> per second (11183886 / 312)
>> May  4 18:58:05  MailScanner[14995]: Batch (50 messages) processed in
>> 312.04 seconds
>> May  4 18:58:20  MailScanner[14287]: Batch completed at 7375 bytes per
>> second (2866715 / 388)
>> May  4 18:58:20  MailScanner[14287]: Batch (50 messages) processed in
>> 388.70 seconds
>> May  4 18:58:22  MailScanner[16953]: Batch completed at 14607 bytes
>> per second (4765445 / 326)
>> May  4 18:58:22  MailScanner[16953]: Batch (50 messages) processed in
>> 326.23 seconds
>> May  4 18:58:39  MailScanner[14596]: Batch completed at 40698 bytes
>> per second (13424209 / 329)
>> May  4 18:58:39  MailScanner[14596]: Batch (50 messages) processed in
>> 329.84 seconds
>> May  4 18:58:39  MailScanner[14892]: Batch completed at 15236 bytes
>> per second (5905691 / 387)
>> May  4 18:58:39  MailScanner[14892]: Batch (50 messages) processed in
>> 387.61 seconds
>> May  4 18:58:45  MailScanner[22446]: Batch completed at 11210 bytes
>> per second (3952639 / 352)
>> May  4 18:58:45  MailScanner[22446]: Batch (50 messages) processed in
>> 352.59 seconds
>> May  4 18:58:53  MailScanner[22253]: Batch completed at 34643 bytes
>> per second (12635074 / 364)
>> May  4 18:58:53  MailScanner[22253]: Batch (50 messages) processed in
>> 364.72 seconds
>> May  4 18:59:06  MailScanner[16612]: Batch completed at 34091 bytes
>> per second (12036761 / 353)
>> May  4 18:59:06  MailScanner[16612]: Batch (50 messages) processed in
>> 353.07 seconds
>> May  4 18:59:17  MailScanner[12768]: Batch completed at 30726 bytes
>> per second (11356957 / 369)
>> May  4 18:59:17  MailScanner[12768]: Batch (50 messages) processed in
>> 369.61 seconds
>> May  4 18:59:28  MailScanner[13267]: Batch completed at 31812 bytes
>> per second (11076141 / 348)
>> May  4 18:59:28  MailScanner[13267]: Batch (50 messages) processed in
>> 348.17 seconds
>> May  4 18:59:30  MailScanner[15241]: Batch completed at 71356 bytes
>> per second (23231603 / 325)
>> May  4 18:59:30  MailScanner[15241]: Batch (50 messages) processed in
>> 325.57 seconds
>> May  4 18:59:56  MailScanner[15478]: Batch completed at 12213 bytes
>> per second (4247264 / 347)
>> May  4 18:59:56  MailScanner[15478]: Batch (50 messages) processed in
>> 347.75 seconds
>>
>
> Average scan time is 6-7 seconds based on those numbers.  That's a little
> high but not bad.
>
>>>  Are you using
>>> Bayes?  If so - using which storage method?
>>
>> Yes I am using file based bayes stored on tmpfs. I do take regular
>> backups.
>>
>
> Try MySQL + InnoDB with the Mail::SpamAssassin::Bayes::MySQL driver -
> according to the benchmarks it's by far the fastest storage method and it
> won't suffer from the locking issues that you'll see with file-based bayes.
>

Thanks Steve. I will definitely test this out.