Batch times ... seems like its the Online test(how to debug which)

Wed Mar 5 20:27:56 GMT 2008

Hi,

I think I have some problems on my MailScanner box ....

As we speak the queue is filling up and there are ATM 900 messages in
queue, started 19.30, about 2 hours ago ...

So ... are my system to small or do we need some tweaking ... ?

First question ...
What are the best way to get a list of what RBL's SA are using and
maybe test if some of them are down, not online any more, I'm beeing
black listed and maybe get a timeout ? As I
think we have a bit too high scan times cause of this ... maybe its
something else ..... :-( But its not easy to figure out what is taking
all the time ... or maybe just to many RBL ( I'm only using them inside SA )

As a manually scan dont take more than ... well, very short time ...
dont know really how to debug it .... MS says about 8 sec per mail ...

spam02# spamassassin --progress -t -L < 65FB580C30C.D8DAA
Content analysis details:   (7.8 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 4.5 BAYES_99               BODY: Bayesian spam probability is 99 to 100%
                            [score: 1.0000]
 1.5 DATE_IN_PAST_12_24     Date: is 12 to 24 hours before Received: date
 1.7 FUZZY_ERECT            BODY: Attempt to obfuscate words in spam
 0.0 HTML_MESSAGE           BODY: HTML included in message
 0.1 RDNS_NONE              Delivered to trusted network by a host with no rDNS
100% Completed 38479.85 msgs/sec in 00m00s

vs

spam02# spamassassin --progress -t -L < 65FB580C30C.D8DAA

Content analysis details:   (22.1 points, 5.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 2.0 RCVD_IN_UCE_PFSM_1     RBL: Received via a relay in UCE_PFSM_1
                            [210.113.42.102 listed in dnsbl-1.uceprotect.net]
 2.0 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
              [Blocked - see <http://www.spamcop.net/bl.shtml?210.113.42.102>]
 3.0 RCVD_IN_XBL            RBL: Received via a relay in Spamhaus XBL
                            [210.113.42.102 listed in zen.spamhaus.org]
 1.5 RCVD_IN_PSBL           RBL: Received via a relay in PSBL
                            [210.113.42.102 listed in psbl.surriel.com]
 1.0 RCVD_IN_LASHBACK       RBL: lashback
                            [210.113.42.102 listed in ubl.unsubscore.com]
 4.5 BAYES_99               BODY: Bayesian spam probability is 99 to 100%
                            [score: 1.0000]
 1.0 DATE_IN_PAST_12_24     Date: is 12 to 24 hours before Received: date
 0.8 FUZZY_ERECT            BODY: Attempt to obfuscate words in spam
 0.0 HTML_MESSAGE           BODY: HTML included in message
 1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
                            above 50%
                            [cf: 100]
 0.5 RAZOR2_CHECK           Listed in Razor2 (http://razor.sf.net/)
 1.5 RAZOR2_CF_RANGE_E4_51_100 Razor2 gives engine 4 confidence level
                            above 50%
                            [cf: 100]
 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
                            [cf: 100]
 2.2 DCC_CHECK              Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
 0.0 DIGEST_MULTIPLE        Message hits more than one network digest check
 0.1 RDNS_NONE              Delivered to trusted network by a host with no rDNS

100% Completed 38479.85 msgs/sec in 00m00s

>From a few days ago:
And from the maillog:
Feb 28 15:09:54 spam02 MailScanner[16804]: Batch (20 messages)
processed in 124.66 seconds
Feb 28 15:09:55 spam02 MailScanner[17104]: Batch (8 messages)
processed in 52.43 seconds
Feb 28 15:09:57 spam02 MailScanner[16838]: Batch (7 messages)
processed in 40.10 seconds
Feb 28 15:10:07 spam02 MailScanner[16830]: Batch (3 messages)
processed in 26.82 seconds
Feb 28 15:10:11 spam02 MailScanner[16935]: Batch (2 messages)
processed in 9.52 seconds
Feb 28 15:10:13 spam02 MailScanner[16901]: Batch (1 message) processed
in 8.35 seconds
Feb 28 15:10:37 spam02 MailScanner[16935]: Batch (2 messages)
processed in 17.79 seconds

>From today:
Mar  5 21:19:25 spam02 MailScanner[88165]: Batch (20 messages)
processed in 149.17 seconds
Mar  5 21:19:27 spam02 MailScanner[88368]: Batch (20 messages)
processed in 182.13 seconds
Mar  5 21:20:01 spam02 MailScanner[88268]: Batch (20 messages)
processed in 159.18 seconds
Mar  5 21:20:05 spam02 MailScanner[88418]: Batch (20 messages)
processed in 171.26 seconds
Mar  5 21:20:09 spam02 MailScanner[88212]: Batch (20 messages)
processed in 180.10 seconds
Mar  5 21:20:34 spam02 MailScanner[88286]: Batch (20 messages)
processed in 153.17 seconds

so ... how can I debug this better ... I'm total lost ... with the
above testing ... with and without external ( -L, --local
         Local tests only (no online tests) )

The local are way faster ... so ... a good way to figure out how to
test the online sources I'm using ... most of them are default in the
SA install ... but I have also added some ...

Any hint etc are most welcome ...

System:
FreeBSD 7.0-BETA1 ( Will be upgraded soon )
Dell PowerEdge 860
CPU	Dual Core Intel Xeon 3060 2.40 Ghz, 4MB L2 cache, 1066 Fsb
Ram	2GB DDR2 667MHz(2x1GB Dual ranked DIMM's
Harddrives	2 stk 146GB in raid 1

MS settings:
Max Children = 10
Max Unscanned Bytes Per Scan = 100m
Max Unsafe Bytes Per Scan = 50m
Max Unscanned Messages Per Scan = 20
Max Unsafe Messages Per Scan = 20

Dont know what more options you need ... if, just ask ....

best regards
Mikael Syska