many spamassassin timeouts

Mon Feb 2 14:44:34 GMT 2004

As I understand it, SpamAssassin opportunistically rebuilds the database
and expires old tokens. In some cases, SpamAssassin times out (as
configured in MailScanner) before the rebuilding completes. Ultimately,
this leads to more timeouts and an accumulation of *.lock and *.expiry
files. You may also see a bayes_toks.new file. It's not really a
SpamAssassin or MailScanner issue, but more of a timing issue
(presumably on slower systems).

I've been closely monitoring my database and rebuilding it manually
(with the --force-expire option). I also increased my SpamAssassin time
out, but I've still had the same problems (although not as frequently).
As mentioned below, this has been an issue for others in the list and
Julian added some code that will generate the rebuild for us. I'm
planning to upgrade in a few days.

Fortunately, it's not really an urgent problem (as it doesn't corrupt my
bayes database), just more of an inconvenience. 

Nathan

-----Original Message-----
From: Mickey Everts [mailto:mickey-ml at GREENGLOW.ORG] 
Sent: Sunday, February 01, 2004 8:03 PM
To: MAILSCANNER at JISCMAIL.AC.UK
Subject: Re: many spamassassin timeouts

Damn...there were a ton of *lock* and *expire* files and I deleted them
all.
I'll give it a couple days to see if the problem is really solved, but
it
sounds likely.  Thanks again for the tip!

I haven't looked in that directory for ages since I didn't even realize
the
locking issue existed and every time I looked in the past, it just had
the
typical files:

auto-whitelist
bayes_journal
bayes_seen
bayes_toks

I just found the "Bayesian shenanigans" thread but it sounds like people
haven't exactly gotten to the bottom of this issue yet.  It sounds like
the
general opinion is it is some issue with spamassassin itself...right?

Mickey

-----Original Message-----
From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
Behalf
Of Nathan Johanson
Sent: Sunday, February 01, 2004 2:22 PM
To: MAILSCANNER at JISCMAIL.AC.UK
Subject: Re: many spamassassin timeouts

Make sure you aren't havin Bayes locking issues. My timeouts were
attributable to this more than once. Check /var/spool/spamassassin (or
wherever your Baye's database resides) for extra bayes lock files and
delete them (you may also need to delete the *.expiry files). Try
running a manual rebuild of the database like so:

sa-learn -D -p /etc/MailScanner/spam.assassin.prefs.conf --rebuild
--force-expire

If this is the cause of the problem, consider taking advantage of the
bayes rebuild options available in the latest release of MailScanner (or
run the command regularly via cron).

Nathan

-----Original Message-----
From: Stephen Swaney [mailto:steve.swaney at FSL.COM] 
Sent: Saturday, January 31, 2004 12:07 PM
To: MAILSCANNER at JISCMAIL.AC.UK
Subject: Re: many spamassassin timeouts

> -----Original Message-----
> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf Of Mickey Everts
> Sent: Saturday, January 31, 2004 2:54 PM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: many spamassassin timeouts
>
> Here is something very weird I just noticed in trying to track this
down.
> Here is just a small sample of my logs, but notice the time outs
happen
> almost exactly every ten minutes? I am running mailscanner-4.25-14.
>
[SKS]
Do you have an event that is slowing down you network every 10 minutes.
Try a sniffer and see.

This is the typical cause for SpamAssassin timeouts.

Steve

Stephen Swaney
President
Fortress Systems Ltd.
Steve.Swaney at FSL.com

> Jan 31 05:48:41 defender MailScanner[18146]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 05:59:05 defender MailScanner[17813]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 06:09:02 defender MailScanner[17717]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 06:19:03 defender MailScanner[18146]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 06:29:41 defender MailScanner[17795]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 06:39:26 defender MailScanner[17813]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 06:50:14 defender MailScanner[17717]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:00:05 defender MailScanner[18146]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:10:43 defender MailScanner[17795]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:20:32 defender MailScanner[17795]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:30:42 defender MailScanner[17784]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:40:45 defender MailScanner[17717]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 07:50:53 defender MailScanner[18146]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 08:00:48 defender MailScanner[17717]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 08:11:01 defender MailScanner[17795]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
> Jan 31 08:20:59 defender MailScanner[17717]: SpamAssassin timed out
and
> was
> killed, consecutive failure 1 of 10
>
> Mickey
>
> -----Original Message-----
> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf
> Of Julian Field
> Sent: Saturday, January 31, 2004 6:37 AM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: many spamassassin timeouts
>
> At 21:17 30/01/2004, you wrote:
> >I turned on Debug = yes and Debug SpamAssassin = yes but I am not
seeing
> >output similar to below in maillog.  Should I be looking elsewhere
else?
> I
> >am trying to track down the source of some spamassassin timeouts I
have
> been
> >having.  Ideally I need to log the equivalent of "spamassassin -D"
for a
> few
> >hours.
>
> Those 2 options will cause "check_mailscanner" to log all the SA
output to
> the terminal. It will process 1 batch of messages and then quit.
> I have been getting a lot of Razor timeouts recently, and have
currently
> disabled it. You can do this by adding
> use_razor2 0
> to your spam.assassin.prefs.conf and restarting MailScanner.
>
>
>
> >Thanks!
> >
> >Mickey
> >
> >-----Original Message-----
> >From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf
> >Of Piet Bos
> >Sent: Monday, January 26, 2004 3:02 AM
> >To: MAILSCANNER at JISCMAIL.AC.UK
> >Subject: Re: many spamassassin timeouts
> >
> >a part of the debug output.
> >I find the  0 behind Net::DNS resolver unavailable rather curious
> >do you agree?
> >
> >grtz Piet
> >
> >debug: running raw-body-text per-line regexp tests; score so far=4.3
> >debug: running uri tests; score so far=4.3
> >debug: uri tests: Done uriRE
> >debug: running full-text regexp tests; score so far=4.3
> >debug: Razor2 is not available
> >debug: DCC is not available: dccproc not found
> >debug: Razor1 is not available
> >debug: Pyzor is not available: pyzor not found
> >debug: is Net::DNS::Resolver unavailable? 0
> >debug: trying (3) gwdg.de...
> >debug: looking up MX for 'gwdg.de'
> >debug: MX for 'gwdg.de' exists? 1
> >debug: MX lookup of gwdg.de succeeded => Dns available (set
dns_available
> to
> >hardcode)
> >debug: is DNS available? 1
> >debug: running meta tests; score so far=5.3
> >----- Original Message -----
> >From: "Julian Field" <mailscanner at ECS.SOTON.AC.UK>
> >To: <MAILSCANNER at JISCMAIL.AC.UK>
> >Sent: Monday, January 26, 2004 9:39 AM
> >Subject: Re: many spamassassin timeouts
> >
> >
> > > Run with Debug = yes and Debug SpamAssassin = yes, and see where
the
> > > slow-down is.
> > >
> > > At 08:33 26/01/2004, you wrote:
> > > >Experiencing many spamassassin timeouts lately.
> > > >Is there a valid reason for that?
> > > >I'm using version 4.26-1 starting
> > > >my settings in MailScanner.conf are:
> > > >SpamAssassin Timeout = 40
> > > >Max SpamAssassin Timeouts = 50
> > > >
> > > >Any suggestions?
> > > >brgds Piet
> > >
> > > --
> > > Julian Field
> > > www.MailScanner.info
> > > MailScanner thanks transtec Computers for their support
> > >
> > > PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>
> --
> Julian Field
> www.MailScanner.info
> Professional Support Services at www.MailScanner.biz
> MailScanner thanks transtec Computers for their support
> PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
> Fortress Systems Ltd.
> www.fsl.com
>

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Fortress Systems Ltd.
www.fsl.com