MailScanner/Spamassassin slow after version upgrade

Mon Jul 30 19:35:24 IST 2007

Here are the differences between the old and new conf file:

# diff MailScanner.conf old/MailScanner.conf.07262007 
< Zip Attachments = no
< Attachments Zip Filename = MessageAttachments.zip
< Attachments Min Total Size To Zip = 100k
< Attachment Extensions Not To Zip = .zip .rar .gz .tgz .jpg .jpeg .mpg
.mpe .mpeg .mp3 .rpm .htm .html .eml
< Monitors for ClamAV Updates = /usr/local/share/clamav/*.inc/*
/usr/local/share/clamav/*.cvd
> Monitors for ClamAV Updates = /usr/local/share/clamav/*.cvd
< Clamd Port = 3310
< Clamd Socket = /tmp/clamd
< Clamd Lock File = # /var/lock/subsys/clamd
< Clamd Use Threads = no
< Known Web Bug Servers = msgtag.com
< Signature Image Filename = %report-dir%/sig.jpg
< Signature Image <img> Filename = signature.jpg
< Attach Image To Signature = no
< Attach Image To HTML Message Only = yes
< Spam List = # spamhaus-ZEN # You can un-comment this to enable them
> Spam List = SBL+XBL
< Ignore Spam Whitelist If Recipients Exceed = 40
> Ignore Spam Whitelist If Recipients Exceed = 75
< Add Watermark = yes
< Check Watermarks = yes
< Treat Invalid Watermarks as Spam = spam
< Watermark Secret = %org-name%-Secretfrog
< Watermark Lifetime = 604800
< Watermark Header = X-%org-name%-MailScanner-Watermark:
< Max SpamAssassin Size = 100k
> Max SpamAssassin Size = 90k
< Max SpamAssassin Timeouts = 10
> Max SpamAssassin Timeouts = 20
< SpamAssassin Rule Actions =
< SpamAssassin Temporary Dir =
/var/spool/MailScanner/incoming/SpamAssassin-Temp
< MailScanner Version Number = 4.62.6
> MailScanner Version Number = 4.59.4

I could downgrade, but I am reticent to do so.

What I may do is work on another mail server and upgrade in pieces.  

1)  Upgrade MS, but not SA.
2)  turn off watermarking at first
3)  leave rule sets the same, then implement changes one by one
4)  sa-compile after the other tests
5)  upgrade SA

-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info
[mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Julian
Field
Sent: Monday, July 30, 2007 1:10 PM
To: MailScanner discussion
Subject: Re: MailScanner/Spamassassin slow after version upgrade

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Also, what new settings did it add when you did the 
upgrade_MailScanner_conf?
Are any of those options on when you would prefer them to be off?
The new code should only ever look up new config options, and not 
execute new code if you don't have them switched on.

Also, how easy is it for you to downgrade again? 
upgrade_MailScanner_conf will happily downgrade as well as upgrade.

Gottschalk, David wrote:
> How many children are you running?
>  
> Are you running a caching DNS server for SA lookups?
>  
> what does 'MailScanner --debug -debug-sa' look like?
>  
> One thing that made a difference for me was uping my children from 5 
> to 10. Also lowering my "Max Unsafe Messages Per Scan" to 10, and my 
> "Max Unscanned Messages Per Scan" to 10 helped speed things up as
well.
>  
> Hope that helps.
>  
> David Gottschalk
>  
>
------------------------------------------------------------------------
> *From:* mailscanner-bounces at lists.mailscanner.info 
> [mailto:mailscanner-bounces at lists.mailscanner.info] *On Behalf Of 
> *donald.dawson at bakerbotts.com
> *Sent:* Monday, July 30, 2007 1:44 PM
> *To:* mailscanner at lists.mailscanner.info
> *Subject:* MailScanner/Spamassassin slow after version upgrade
>
> The time it takes to process an email has doubled since Friday 
> (7/26/07) when I upgraded to MS 4.62.6 and SA 3.2.2.  It was around 
> 3.5 seconds, and now it is averaging 7 seconds.
>
> We use DCC and razor2 - no pyzor.
>
> Jul 30 12:36:35 houmx05 MailScanner[21682]: Batch (30 messages) 
> processed in 187.50 seconds
> Jul 30 12:37:06 houmx05 MailScanner[21662]: Batch (16 messages) 
> processed in 105.59 seconds
> Jul 30 12:37:24 houmx05 MailScanner[21727]: Batch (30 messages) 
> processed in 196.20 seconds
> Jul 30 12:37:28 houmx05 MailScanner[21776]: Batch (30 messages) 
> processed in 211.89 seconds
> Jul 30 12:37:38 houmx05 MailScanner[21793]: Batch (30 messages) 
> processed in 223.64 seconds
>
> sar -u shows idle CPU time:
> 12:00:01 AM   CPU    %user    %nice  %system  %iowait    %idle
> Average:          all     10.49      0.00      4.99      1.00
83.51
>
> # uptime is normal
>  12:38:30 up 5 days, 25 min,  1 user,  load average: 0.81, 0.79, 0.83
>
> I have implemented sa-compile using re2c expecting to get a 
> performance boost.
>
> 'MailScanner --lint' doesn't show any errors, except noting that we 
> have clamav processing turned off (output attached).
>
> <<ms.txt>>
> I have included output from spamassassin's lint command.
>
> /etc/mail/spamassassin contents:
> bakerbotts.cf   - custom local rule file
> Botnet.cf
> Botnet.pm
> init.pre
> init.pre.pre-v310
> KAM.cf
> local.cf
> mailscanner.cf
> pdfinfo.cf
> sare-sa-update-channels.txt
> sa-update-keys
> v310.pre
> v312.pre
> v320.pre
>
> contents of sare-sa-update-channels.txt used by update_spamassassin:
>
> updates.spamassassin.org
> 70_sare_adult.cf.sare.sa-update.dostech.net
> 70_sare_bayes_poison_nxm.cf.sare.sa-update.dostech.net
> 70_sare_evilnum0.cf.sare.sa-update.dostech.net
> 70_sare_genlsubj0.cf.sare.sa-update.dostech.net
> 70_sare_genlsubj1.cf.sare.sa-update.dostech.net
> 70_sare_genlsubj2.cf.sare.sa-update.dostech.net
> 70_sare_header.cf.sare.sa-update.dostech.net
> 70_sare_highrisk.cf.sare.sa-update.dostech.net
> 70_sare_html.cf.sare.sa-update.dostech.net
> 70_sare_obfu.cf.sare.sa-update.dostech.net
> 70_sare_oem.cf.sare.sa-update.dostech.net
> 70_sare_random.cf.sare.sa-update.dostech.net
> 70_sare_specific.cf.sare.sa-update.dostech.net
> 70_sare_spoof.cf.sare.sa-update.dostech.net
> 70_sare_stocks.cf.sare.sa-update.dostech.net
> 70_sare_unsub.cf.sare.sa-update.dostech.net
> 70_sare_uri0.cf.sare.sa-update.dostech.net
> 70_sare_uri1.cf.sare.sa-update.dostech.net
> 70_sare_uri2.cf.sare.sa-update.dostech.net
> 70_sare_whitelist.cf.sare.sa-update.dostech.net
> 70_sare_whitelist_rcvd.cf.sare.sa-update.dostech.net
> 70_sare_whitelist_spf.cf.sare.sa-update.dostech.net
> 72_sare_bml_post25x.cf.sare.sa-update.dostech.net
> 72_sare_redirect_post3.0.0.cf.sare.sa-update.dostech.net
> 88_FVGT_headers.cf.sare.sa-update.dostech.net
> 99_FVGT_Tripwire.cf.sare.sa-update.dostech.net
> 99_sare_fraud_post25x.cf.sare.sa-update.dostech.net
>
> Any ideas would be greatly appreciated.
>
> Thanks,
> Donald
>
> Donald Dawson
> Security Administrator
> Baker Botts L.L.P.
> 713-229-2183
>

Jules

- -- 
Julian Field MEng CITP
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

MailScanner customisation, or any advanced system administration help?
Contact me at Jules at Jules.FM

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
For all your IT requirements visit www.transtec.co.uk

-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.6.2 (Build 2014)
Charset: ISO-8859-1

wj8DBQFGrilqEfZZRxQVtlQRAi2RAJ4280SEQjRWFol//o6+uCgUoYCqVQCggoQe
+N11/ZhGl/e6VHtCnWRsHYY=
=rDAY
-----END PGP SIGNATURE-----

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
For all your IT requirements visit www.transtec.co.uk

-- 
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website! 