New MS install is slow to an extreme

[DAve]

Julian Field wrote:
> 
> 
> DAve wrote:
>> After much plugging away, double checking, triple checking and herding 
>> of cats I *think* we may out of the woods. I won't know until tomorrow 
>> AM when traffic picks up again. Here are my findings so far.
>>
>> MailScanner 4.67.6
>> ClamAV 0.92.1
>> SpamAssassin 3.2.4
>>
>> Virus Scanners = [clamav | clamavmodule] - There appears to be no real 
>> gain in running clamavmodule, some speed increase but not enough to be 
>> noticed. I have clamavmodule configured just to save some memory.
>>
>> ClamAV Full Message Scan = yes - That is a killer, it seems to really 
>> increase processing time. I have it now set to no, and I have removed 
>> my MSRBL sigs.
>>
>> Incoming Work Dir = tmpfs (mdmfs in FreeBSD) - Surprisingly little 
>> difference. I left it on a memory file system for now.
>>
>> mailscanner.cf -> skip_rbl_checks 1 - Oddly does not do what it 
>> claims. SA is still doing rbl checks. I commented out the DNSEval 
>> plugin in v320.pre file and was rewarded with errors for my effort. 
>> Not certain what the correct method of disabling rbl checks in SA is 
>> now. Peter Farrow found a message where this has been seen already. 
>> http://markmail.org/message/xzqi5fmrbj3tfgg2
>>
>> MailScanner batch size - With version 4.54.6 MS processed 10 messages 
>> per batch and kept up just fine. With version 4.67.6 it will grab 30+ 
>> messages which takes longer to process. Increasing MS children has no 
>> effect. More children working slower doesn't process more mail for me. 
>> I don't see where I can configure this.
> "Max Children =" in MailScanner.conf. If you were using 
> upgrade_MailScanner_conf to upgrade your MailScanner.conf file then this 
> setting would not have been changed between versions. Do you really copy 
> over all your settings by hand into the new MailScanner.conf file? Wow! 
> That must take *hours*.
> 

Nope, I can modify Max Children, my question is can I control how large 
a batch size each child will process? Previously if I had 500 messages 
waiting each child would pick up 10 messages, now they will each pick up 
30 messages. This is clearly evident in my MRTG graphs where I show over 
  the last four months I never had a batch over 10, yesterday I had 
batch sizes of 30 for several hours.

I do use upgrade_MailScanner_conf, and works a treat ;^)

DAve

>>
>> I am currently seeing processing times of .8 to 20 seconds per 
>> message, generally around the 2 to 4 seconds mark. This is for batches 
>> of 1 to 10 messages. I was seeing as much as 800 seconds for a batch 
>> size of 30 messages this morning. So there has been improvement.
>>
>> I am compiling my SA rules and I run my RBLs in the MTA (hence why I 
>> do not want rbl checking in SA).
>>
>> Overall, my previous install of MS 4.54.6, Clam .92, and SA 3.1.9 
>> would run rings around this install. I am seriously contemplating 
>> rolling back but I am uncertain if I have the original tarball for 
>> Julian's Clam+SA package.
>>
>> I believe my issue is configuration of MS or SA at this point. I am 
>> open to suggestions.
>>
>> Thanks for the help.
>>
>> DAve
>>
> 
> Jules
> 


-- 
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.