many spamassassin timeouts

[Julian Field]

The code to do this for you currently doesn't work on Solaris, but it does
appear to work fine on Linux. It's a locking semantics problem which I
haven't got to the bottom of yet.

See the
Re: 4.26.7, bayes rebuild, confused. -- 4.26.8
thread for more discussion on this.

At 14:44 02/02/2004, you wrote:
>As I understand it, SpamAssassin opportunistically rebuilds the database
>and expires old tokens. In some cases, SpamAssassin times out (as
>configured in MailScanner) before the rebuilding completes. Ultimately,
>this leads to more timeouts and an accumulation of *.lock and *.expiry
>files. You may also see a bayes_toks.new file. It's not really a
>SpamAssassin or MailScanner issue, but more of a timing issue
>(presumably on slower systems).
>
>I've been closely monitoring my database and rebuilding it manually
>(with the --force-expire option). I also increased my SpamAssassin time
>out, but I've still had the same problems (although not as frequently).
>As mentioned below, this has been an issue for others in the list and
>Julian added some code that will generate the rebuild for us. I'm
>planning to upgrade in a few days.
>
>Fortunately, it's not really an urgent problem (as it doesn't corrupt my
>bayes database), just more of an inconvenience.
>
>Nathan
>
>
>-----Original Message-----
>From: Mickey Everts [mailto:mickey-ml at GREENGLOW.ORG]
>Sent: Sunday, February 01, 2004 8:03 PM
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: many spamassassin timeouts
>
>
>Damn...there were a ton of *lock* and *expire* files and I deleted them
>all.
>I'll give it a couple days to see if the problem is really solved, but
>it
>sounds likely.  Thanks again for the tip!
>
>I haven't looked in that directory for ages since I didn't even realize
>the
>locking issue existed and every time I looked in the past, it just had
>the
>typical files:
>
>auto-whitelist
>bayes_journal
>bayes_seen
>bayes_toks
>
>I just found the "Bayesian shenanigans" thread but it sounds like people
>haven't exactly gotten to the bottom of this issue yet.  It sounds like
>the
>general opinion is it is some issue with spamassassin itself...right?
>
>Mickey
>
>-----Original Message-----
>From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
>Behalf
>Of Nathan Johanson
>Sent: Sunday, February 01, 2004 2:22 PM
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: many spamassassin timeouts
>
>Make sure you aren't havin Bayes locking issues. My timeouts were
>attributable to this more than once. Check /var/spool/spamassassin (or
>wherever your Baye's database resides) for extra bayes lock files and
>delete them (you may also need to delete the *.expiry files). Try
>running a manual rebuild of the database like so:
>
>sa-learn -D -p /etc/MailScanner/spam.assassin.prefs.conf --rebuild
>--force-expire
>
>If this is the cause of the problem, consider taking advantage of the
>bayes rebuild options available in the latest release of MailScanner (or
>run the command regularly via cron).
>
>Nathan
>
>
>
>-----Original Message-----
>From: Stephen Swaney [mailto:steve.swaney at FSL.COM]
>Sent: Saturday, January 31, 2004 12:07 PM
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: many spamassassin timeouts
>
>
> > -----Original Message-----
> > From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> > Behalf Of Mickey Everts
> > Sent: Saturday, January 31, 2004 2:54 PM
> > To: MAILSCANNER at JISCMAIL.AC.UK
> > Subject: Re: many spamassassin timeouts
> >
> > Here is something very weird I just noticed in trying to track this
>down.
> > Here is just a small sample of my logs, but notice the time outs
>happen
> > almost exactly every ten minutes? I am running mailscanner-4.25-14.
> >
>[SKS]
>Do you have an event that is slowing down you network every 10 minutes.
>Try a sniffer and see.
>
>This is the typical cause for SpamAssassin timeouts.
>
>Steve
>
>Stephen Swaney
>President
>Fortress Systems Ltd.
>Steve.Swaney at FSL.com
>
>
> > Jan 31 05:48:41 defender MailScanner[18146]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 05:59:05 defender MailScanner[17813]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 06:09:02 defender MailScanner[17717]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 06:19:03 defender MailScanner[18146]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 06:29:41 defender MailScanner[17795]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 06:39:26 defender MailScanner[17813]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 06:50:14 defender MailScanner[17717]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:00:05 defender MailScanner[18146]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:10:43 defender MailScanner[17795]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:20:32 defender MailScanner[17795]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:30:42 defender MailScanner[17784]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:40:45 defender MailScanner[17717]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 07:50:53 defender MailScanner[18146]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 08:00:48 defender MailScanner[17717]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 08:11:01 defender MailScanner[17795]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> > Jan 31 08:20:59 defender MailScanner[17717]: SpamAssassin timed out
>and
> > was
> > killed, consecutive failure 1 of 10
> >
> > Mickey
> >
> > -----Original Message-----
> > From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> > Behalf
> > Of Julian Field
> > Sent: Saturday, January 31, 2004 6:37 AM
> > To: MAILSCANNER at JISCMAIL.AC.UK
> > Subject: Re: many spamassassin timeouts
> >
> > At 21:17 30/01/2004, you wrote:
> > >I turned on Debug = yes and Debug SpamAssassin = yes but I am not
>seeing
> > >output similar to below in maillog.  Should I be looking elsewhere
>else?
> > I
> > >am trying to track down the source of some spamassassin timeouts I
>have
> > been
> > >having.  Ideally I need to log the equivalent of "spamassassin -D"
>for a
> > few
> > >hours.
> >
> > Those 2 options will cause "check_mailscanner" to log all the SA
>output to
> > the terminal. It will process 1 batch of messages and then quit.
> > I have been getting a lot of Razor timeouts recently, and have
>currently
> > disabled it. You can do this by adding
> > use_razor2 0
> > to your spam.assassin.prefs.conf and restarting MailScanner.
> >
> >
> >
> > >Thanks!
> > >
> > >Mickey
> > >
> > >-----Original Message-----
> > >From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> > Behalf
> > >Of Piet Bos
> > >Sent: Monday, January 26, 2004 3:02 AM
> > >To: MAILSCANNER at JISCMAIL.AC.UK
> > >Subject: Re: many spamassassin timeouts
> > >
> > >a part of the debug output.
> > >I find the  0 behind Net::DNS resolver unavailable rather curious
> > >do you agree?
> > >
> > >grtz Piet
> > >
> > >debug: running raw-body-text per-line regexp tests; score so far=4.3
> > >debug: running uri tests; score so far=4.3
> > >debug: uri tests: Done uriRE
> > >debug: running full-text regexp tests; score so far=4.3
> > >debug: Razor2 is not available
> > >debug: DCC is not available: dccproc not found
> > >debug: Razor1 is not available
> > >debug: Pyzor is not available: pyzor not found
> > >debug: is Net::DNS::Resolver unavailable? 0
> > >debug: trying (3) gwdg.de...
> > >debug: looking up MX for 'gwdg.de'
> > >debug: MX for 'gwdg.de' exists? 1
> > >debug: MX lookup of gwdg.de succeeded => Dns available (set
>dns_available
> > to
> > >hardcode)
> > >debug: is DNS available? 1
> > >debug: running meta tests; score so far=5.3
> > >----- Original Message -----
> > >From: "Julian Field" <mailscanner at ECS.SOTON.AC.UK>
> > >To: <MAILSCANNER at JISCMAIL.AC.UK>
> > >Sent: Monday, January 26, 2004 9:39 AM
> > >Subject: Re: many spamassassin timeouts
> > >
> > >
> > > > Run with Debug = yes and Debug SpamAssassin = yes, and see where
>the
> > > > slow-down is.
> > > >
> > > > At 08:33 26/01/2004, you wrote:
> > > > >Experiencing many spamassassin timeouts lately.
> > > > >Is there a valid reason for that?
> > > > >I'm using version 4.26-1 starting
> > > > >my settings in MailScanner.conf are:
> > > > >SpamAssassin Timeout = 40
> > > > >Max SpamAssassin Timeouts = 50
> > > > >
> > > > >Any suggestions?
> > > > >brgds Piet
> > > >
> > > > --
> > > > Julian Field
> > > > www.MailScanner.info
> > > > MailScanner thanks transtec Computers for their support
> > > >
> > > > PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
> >
> > --
> > Julian Field
> > www.MailScanner.info
> > Professional Support Services at www.MailScanner.biz
> > MailScanner thanks transtec Computers for their support
> > PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
> >
> > --
> > This message has been scanned for viruses and
> > dangerous content by MailScanner, and is
> > believed to be clean.
> >
> > Fortress Systems Ltd.
> > www.fsl.com
> >
>
>
>
>--
>This message has been scanned for viruses and
>dangerous content by MailScanner, and is
>believed to be clean.
>
>Fortress Systems Ltd.
>www.fsl.com

--
Julian Field
www.MailScanner.info
MailScanner thanks transtec Computers for their support

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654