Slow MailScanner

[Jay Chandler]

Jay Chandler wrote:
> sandrews at andrewscompanies.com wrote:
>> Bayes dbs are typically in ./root or /etc/MailScanner/bayes
>> -----Original Message-----
>> From: mailscanner-bounces at lists.mailscanner.info
>> [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Jay
>> Chandler
>> Sent: Tuesday, February 13, 2007 12:38 PM
>> To: MailScanner discussion
>> Subject: Re: Slow MailScanner
>>
>> DAve wrote:
>>  
>>> Jay Chandler wrote:
>>>    
>>>> I have two servers.
>>>>
>>>> Here's one:
>>>>
>>>> aconcagua#  tail -f /var/log/maillog |grep rocessed Feb 13 08:39:58 
>>>> aconcagua MailScanner[83401]: Batch (1 message) processed in 6.66 
>>>> seconds Feb 13 08:39:58 aconcagua MailScanner[83008]: Batch (1 
>>>> message) processed in 10.06 seconds Feb 13 08:40:00 aconcagua 
>>>> MailScanner[83989]: Batch (1 message) processed in 6.11 seconds Feb 
>>>> 13 08:40:03 aconcagua MailScanner[83677]: Batch (1 message) processed
>>>>       
>>
>>  
>>>> in 6.84 seconds Feb 13 08:40:04 aconcagua MailScanner[82475]: Batch 
>>>> (1 message) processed in 6.70 seconds Feb 13 08:40:05 aconcagua 
>>>> MailScanner[82359]: Batch (1 message) processed in 8.74 seconds Feb 
>>>> 13 08:40:06 aconcagua MailScanner[83301]: Batch (2 messages) 
>>>> processed in 12.81 seconds Feb 13 08:40:07 aconcagua 
>>>> MailScanner[82879]: Batch (1 message) processed in 7.75 seconds Feb 
>>>> 13 08:40:09 aconcagua MailScanner[82035]: Batch (1 message) processed
>>>>       
>>
>>  
>>>> in 6.53 seconds Feb 13 08:40:11 aconcagua MailScanner[83989]: Batch 
>>>> (1 message) processed in 6.41 seconds Feb 13 08:40:11 aconcagua 
>>>> MailScanner[84046]: Batch (1 message) processed in 6.84 seconds Feb 
>>>> 13 08:40:12 aconcagua MailScanner[83301]: Batch (1 message) processed
>>>>       
>>
>>  
>>>> in 6.56 seconds
>>>>
>>>> Here's the other:
>>>>
>>>>      
>>>>> spacecowboy# tail -f /var/log/maillog |grep rocessed Feb 13 08:38:57
>>>>>         
>>
>>  
>>>>> spacecowboy MailScanner[54541]: Batch (9 messages) processed in 
>>>>> 252.21 seconds Feb 13 08:39:12 spacecowboy MailScanner[49475]: Batch
>>>>>         
>>
>>  
>>>>> (2 messages) processed in 61.60 seconds Feb 13 08:39:12 
>>>>> spacecowboy MailScanner[53408]: Batch (4 messages) processed in 
>>>>> 86.83 seconds Feb 13 08:39:14 spacecowboy MailScanner[53430]: 
>>>>> Batch (2 messages) processed in 31.38 seconds Feb 13 08:39:17 
>>>>> spacecowboy MailScanner[54987]: Batch (8 messages) processed in 
>>>>> 166.69 seconds Feb 13 08:39:18 spacecowboy MailScanner[53490]: 
>>>>> Batch (19 messages) processed in 531.03 seconds Feb 13 08:39:21 
>>>>> spacecowboy MailScanner[53398]: Batch (14 messages) processed in 
>>>>> 384.67 seconds Feb 13 08:39:30 spacecowboy MailScanner[53412]: 
>>>>> Batch (8 messages) processed in 97.58 seconds Feb 13 08:39:32 
>>>>> spacecowboy MailScanner[54123]: Batch (2 messages) processed in 
>>>>> 62.52 seconds Feb 13 08:39:38 spacecowboy MailScanner[53430]: 
>>>>> Batch (1 message) processed in 24.16 seconds Feb 13 08:39:39 
>>>>> spacecowboy MailScanner[55686]: Batch (30 messages) processed in 
>>>>> 647.57 seconds Feb 13 08:39:48 spacecowboy MailScanner[56780]: 
>>>>> Batch (5 messages) processed in 68.93 seconds
>>>>>         
>>>> Any idea what would be causing this?  Same configuration, same MX 
>>>> priority.  The one with delays has built quite the queue backlog.
>>>>
>>>>       
>>> Can both machines resolve DNS with the same speed?
>>> Do you have a caching name server on both machines?
>>> Is the Bays DB the same size on both machines?
>>>
>>>
>>>     
>> I've been wrestling with this a bit.
>>
>> A few questions:
>>
>> 1. How do I set up a caching nameserver?  Can someone throw me a link?
>>
>> 2. I've searched high and low, but I can't find the bayes DB 
>> location.  I never explicitly set it up, but it's apparently running...
>>
>> Output of mailscanner --lint below:
>>
>> spacecowboy# mailscanner --lint
>> Read 759 hostnames from the phishing whitelist Checking version
>> numbers...
>> Version number in MailScanner.conf (4.58.9) is correct.
>> MailScanner setting GID to  (125)
>> MailScanner setting UID to  (125)
>>
>> Checking for SpamAssassin errors (if you use it)...
>> Using SpamAssassin results cache
>> Connected to SpamAssassin cache database
>> pyzor: check failed: internal error
>> SpamAssassin reported no errors.
>> Using locktype = flock
>> MailScanner.conf says "Virus Scanners = auto"
>> Found these virus scanners installed: bitdefender, clamavmodule
>>
>>
>>
>> -- 
>> Jay Chandler
>> Network Administrator, Chapman University
>> 714.628.7249 / chandler at chapman.edu
>> Today's Excuse: Processes running slowly due to weak power supply
>> -- 
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>>   
> Nope, neither location (nor in /usr/local/etc/MailScanner/bayes, since 
> this is from the FreeBSD ports tree).
>
> So, getting desperate to clear the backlog, I ran sa-learn --clear on 
> the troubled box, and went to get myself a cup of coffee from the 
> break room.
> On the plus side, I now have coffee.  On the downside, it's still 
> taking upwards of 20 seconds per message.
>
> The nameserver config is the same.  The hardware SHOULD be good-- 
> these boxes are identical, and a month old.
>
Very interesting.  A restart of the box, and the queue is gone, and load 
times are reasonable.  I suspect there's something stealing all the RAM 
after a few days-- possibly MailScanner.  I'll have to investigate this 
the next time it happens.

Thanks to all who helped-- I'm still debating the merits of a caching 
nameserver.

Also-- would there be any benefit to setting up bayes in a SQL 
environment to share between the two servers?

-- 
Jay Chandler
Network Administrator, Chapman University
714.628.7249 / chandler at chapman.edu
Today's Excuse: Processes running slowly due to weak power supply