Super Slow Queue Processing

[Richard Frovarp]

Peter Russell wrote:
> Hi there, we have a MailScanner system that has had a lot (any) 
> attention for 6 months or so and all of a sudden it is unable to 
> process the messages in the queue faster than they are building up.
>
> Doing an sa -D -t < on any of the high scoring spam messages i have is 
> very quick (less than 5sec). there was some error about hash from 
> sa-learn --force-expire, a google revealed naught so i disabled bayes 
> for now. but it maks no difference.
>
> But changing Use Spamassassin to no means the queue is processed 
> lightening quick. which seems weird because SA appears to run quite 
> quick.
>
> The messages do eventually seem to get processed but it takes quite a 
> while (upto 30min) and the queue seems to be growing muh quicker than 
> processing.
>
> MS debug takes AGES.
>
> I cant seem to find anything useful or interesting in the logs, i have 
> attached the output of the MS debug (yes i need to update clamav).
>
> Appreciate ANY tips on how to diagnose (or hopefully fix) this. :)
>
> Many thanks
> Pete

Do your lints work? Have you ran sa-update? Are you swapping?

My two guesses would be thrashing or a slow DNS lookup or timeout. If 
you are doing your testing on a machine that processed the message or 
has access to the same DNS, the result would be cached. You can run 
MailScanner --debug --debug-sa to see the SA debug output on a batch. 
You might be able to get a feel for where SA is hanging in the process 
on incoming messages and it might give you an idea. It's possible you 
have been blocked from SpamHaus or are using a service that has been 
shut down (hence the sa-update question).