Massive queue buildup

Chandler, Jay chandler at chapman.edu
Tue Nov 14 17:12:12 GMT 2006


Actually, I can do one better than that-- I've got Mailscanner MRTG
installed.

Right now it's right around 1700 bytes a second, and it's been pretty
consistently there. 


--
Jay Chandler
Network Administrator, Chapman University
714.628.7249 / chandler at chapman.edu
Today's Excuse: We are Microsoft.  What you are experiencing is not a
problem; it is an undocumented feature. 

-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info
[mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Brent
Addis
Sent: Monday, November 13, 2006 5:35 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup

Thats sounds like the one you want.
 
Check your syslog, it should show you your batch processing speed
 
IE:
 
Nov 14 14:30:10 mx3 MailScanner[5500]: Spam Checks completed at 826
bytes per second Nov 14 14:30:10 mx3 MailScanner[5500]: Virus and
Content Scanning: Starting Nov 14 14:30:13 mx3 MailScanner[5500]: Virus
Scanning completed at 4633 bytes per second Nov 14 14:30:13 mx3
MailScanner[5500]: Uninfected: Delivered 1 messages Nov 14 14:30:13 mx3
MailScanner[5500]: Virus Processing completed at 692908 bytes per second
Nov 14 14:30:13 mx3 MailScanner[5500]: Batch completed at 700 bytes per
second Nov 14 14:30:13 mx3 MailScanner[5500]: Batch (1 message)
processed in 9.86 seconds Nov 14 14:30:13 mx3 MailScanner[5500]: Logging
message 1Gjn7U-0000rl-44 to SQL Nov 14 14:30:13 mx3 MailScanner[5500]:
"Always Looked Up Last" took 0.00 seconds
 
Do you have the spamassassin cache enabled?

 

________________________________

From: mailscanner-bounces at lists.mailscanner.info on behalf of Chandler,
Jay
Sent: Tue 11/14/2006 2:15 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup



I don't show any match for anything approaching that, other than "log
speed," which is set to yes for MRTG.




--
Jay Chandler
Network Administrator, Chapman University
714.628.7249 / chandler at chapman.edu
Today's Excuse: not approved by the FCC

-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info
[mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Brent
Addis
Sent: Monday, November 13, 2006 4:55 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup

ok, what about batch processing speed? Try enabling that within
MailScanner.conf

________________________________

From: mailscanner-bounces at lists.mailscanner.info on behalf of Chandler,
Jay
Sent: Tue 11/14/2006 1:52 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup



brewer# time spamassassin -C
/usr/local/etc/mail/spamassassin/mailscanner.conf --lint
3.092u 0.386s 0:09.71 35.7%     10+37348k 0+0io 0pf+0w
brewer# time spamassassin -C /usr/local/etc/mail/spamassassin/ --lint
4.135u 0.318s 0:04.97 89.3%     10+48865k 0+0io 0pf+0w
brewer#

Looks good from here.

--
Jay Chandler
Network Administrator, Chapman University
714.628.7249 / chandler at chapman.edu
Today's Excuse: terrorist activities

-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info
[mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Brent
Addis
Sent: Monday, November 13, 2006 4:13 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup

3 seconds?

Not bad!

However, are you sure that includes the mailscanner config? (It might be
in /etc/spamassassin or /etc/mail/spamassassin)

if not, include the config file in your lint (I think its either -C or
-P /path/to/spam.assassin.conf)

________________________________

From: mailscanner-bounces at lists.mailscanner.info on behalf of Chandler,
Jay
Sent: Tue 11/14/2006 1:09 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup



Time spamassassin -D --lint takes three point seven seconds-- I don't
think that's where the holdup is occuring.

Our DNS server is local and on the same netblock as the mailserver in
question.

We have a few RBLs at connecttime, but they seem to be holding up well.

It's a bit of a stumper...


--
Jay Chandler
Network Administrator, Chapman University
714.628.7249 / chandler at chapman.edu
Today's Excuse: your keyboard's space bar is generating spurious
keycodes.

-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info
[mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Brent
Addis
Sent: Monday, November 13, 2006 3:48 PM
To: MailScanner discussion
Subject: RE: Massive queue buildup

if you want to only check spamassassin (generally the slowest part with
lots of rules) "time spamassassin -D --lint" is a good start. as long as
your mailscanner.cf is in a reasonable place it should pick it up. If
you keep an eye on where it seems to take a while, it should help you
track down the problem.

Or, there is an option within mailscanner to turn on batch process time.
I forget the name however its in the last quarter of the config
somewhere.

are you running bayes? If you are, and its standard db files on the
server, I would recommend migrating these to an sql server somewhere.
This helped scan times a fair bit too. Another upside is this also means
you can have multiple servers using the same db.

With RBL's, don't forget spamassassin also does RBL checking so make
sure your not doing twice the lookups you need to.

100k a day should be fine on the hardware you have.



________________________________

From: mailscanner-bounces at lists.mailscanner.info on behalf of Jay
Chandler
Sent: Tue 11/14/2006 12:39 PM
To: MailScanner discussion
Subject: Re: Massive queue buildup



On Nov 13, 2006, at 2:42 PM, Brent Addis wrote:


        Check your not running one of those massive blacklists from
SARE.

        I was running one for a while while testing and a similar thing
happened. removing it dropped my average scan time from ~2 1/2 minutes
to 11 seconds per message.


Where does one determine how long the average scan time is?


        Other ideas:

        - Check your dns servers are capable of standing up to the
amount of dns requests you are making. Running something like nscd
locally is a good idea.


I suspect they are, but I'll verify this.


        - Are you running very many RBL's within mailscanner? Try
disabling these and see if it helps.


Three or four-- nothing insane.


        - Are you running any type of recipient verification? (as in,
checking that the person being sent the mail actually exists). If not,
try turning it on. I am unsure what it is called within postfix as I
don't use it.


We are.  Messages to undefined users fault to a 5xx error.


        - Check out
http://wiki.mailscanner.info/doku.php?id=maq:index#optimization_tips
which has a few others as well.


Thanks!


        How much mail are you handling a day? I have a couple of single
cpu 3.0 ghz machines comfortably handling many thousands of messages a
day.



Right now, about 100K a day.


Thanks for the help!



--
Jay Chandler
Network Administrator, Chapman University
714-628-7249 / chandler at chapman.edu
"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
does quite what I want.  I wish Christopher Robin was here." -- Peter Da
Silva in a.s.r.


--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!


--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner

Before posting, read http://wiki.mailscanner.info/posting

Support MailScanner development - buy the book off the website!




More information about the MailScanner mailing list