New MS install is slow to an extreme

[Julian Field]

DAve wrote:
> Julian Field wrote:
>>
>>
>> DAve wrote:
>>> After much plugging away, double checking, triple checking and 
>>> herding of cats I *think* we may out of the woods. I won't know 
>>> until tomorrow AM when traffic picks up again. Here are my findings 
>>> so far.
>>>
>>> MailScanner 4.67.6
>>> ClamAV 0.92.1
>>> SpamAssassin 3.2.4
>>>
>>> Virus Scanners = [clamav | clamavmodule] - There appears to be no 
>>> real gain in running clamavmodule, some speed increase but not 
>>> enough to be noticed. I have clamavmodule configured just to save 
>>> some memory.
>>>
>>> ClamAV Full Message Scan = yes - That is a killer, it seems to 
>>> really increase processing time. I have it now set to no, and I have 
>>> removed my MSRBL sigs.
>>>
>>> Incoming Work Dir = tmpfs (mdmfs in FreeBSD) - Surprisingly little 
>>> difference. I left it on a memory file system for now.
>>>
>>> mailscanner.cf -> skip_rbl_checks 1 - Oddly does not do what it 
>>> claims. SA is still doing rbl checks. I commented out the DNSEval 
>>> plugin in v320.pre file and was rewarded with errors for my effort. 
>>> Not certain what the correct method of disabling rbl checks in SA is 
>>> now. Peter Farrow found a message where this has been seen already. 
>>> http://markmail.org/message/xzqi5fmrbj3tfgg2
>>>
>>> MailScanner batch size - With version 4.54.6 MS processed 10 
>>> messages per batch and kept up just fine. With version 4.67.6 it 
>>> will grab 30+ messages which takes longer to process. Increasing MS 
>>> children has no effect. More children working slower doesn't process 
>>> more mail for me. I don't see where I can configure this.
>> "Max Children =" in MailScanner.conf. If you were using 
>> upgrade_MailScanner_conf to upgrade your MailScanner.conf file then 
>> this setting would not have been changed between versions. Do you 
>> really copy over all your settings by hand into the new 
>> MailScanner.conf file? Wow! That must take *hours*.
>>
>
> Nope, I can modify Max Children, my question is can I control how 
> large a batch size each child will process? Previously if I had 500 
> messages waiting each child would pick up 10 messages, now they will 
> each pick up 30 messages. This is clearly evident in my MRTG graphs 
> where I show over  the last four months I never had a batch over 10, 
> yesterday I had batch sizes of 30 for several hours.
Max Unscanned Bytes Per Scan = 100m
Max Unsafe Bytes Per Scan = 50m
Max Unscanned Messages Per Scan = 30
Max Unsafe Messages Per Scan = 30

>
> I do use upgrade_MailScanner_conf, and works a treat ;^)
Phew! You had me worried for a moment there :-)

>
> DAve
>
>>>
>>> I am currently seeing processing times of .8 to 20 seconds per 
>>> message, generally around the 2 to 4 seconds mark. This is for 
>>> batches of 1 to 10 messages. I was seeing as much as 800 seconds for 
>>> a batch size of 30 messages this morning. So there has been 
>>> improvement.
>>>
>>> I am compiling my SA rules and I run my RBLs in the MTA (hence why I 
>>> do not want rbl checking in SA).
>>>
>>> Overall, my previous install of MS 4.54.6, Clam .92, and SA 3.1.9 
>>> would run rings around this install. I am seriously contemplating 
>>> rolling back but I am uncertain if I have the original tarball for 
>>> Julian's Clam+SA package.
>>>
>>> I believe my issue is configuration of MS or SA at this point. I am 
>>> open to suggestions.
>>>
>>> Thanks for the help.
>>>
>>> DAve
>>>
>>
>> Jules
>>
>
>

Jules

-- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

Need help customising MailScanner?
Contact me!
Need help fixing or optimising your systems?
Contact me!
Need help getting you started solving new requirements from your boss?
Contact me!

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.