Solved? Re: Still stuck in queue, version 4.52.2

[Jim Holland]

On Sun, 9 Apr 2006, Julian Field wrote:

> > On Thu, 6 Apr 2006, Max Kipness wrote:
> >
> >   
> >> I've since upgraded to version 4.52.2, and I'm getting better
> >> performance (probably less getting stuck in the queue), yet yesterday
> >> there was one message that got processed over 6000 times!
> >>
> >> Here is a sample of one that is stuck right now. It's been processed 512
> >> times. Any clue to what else I can do to remedy this issue?
> >>     
> >  
> > I wish I knew the cause of this problem.  I regularly come across this
> > issue, but fortunately at long intervals (a couple of months or more
> > between each occurrence) with all the versions of MailScanner that I have
> > used (currently 4.50.10-1 - just about to install 4.52.2).  When I come
> > across stuck mail I generally find that the whole of the associated batch
> > of up to 30 messages tend to have the same problem of being endlessly
> > reprocessed.  My fix is to remove the first message of the batch from
> > mqueue.in and then try to process the rest of the batch.  If that fails
> > then I remove the next one, and so on until I have identified the problem
> > message.  I then return the remaining messages to the queue and finally
> > convert the d and q files of the problem message to a standard RFC822
> > message file, scan it with clamscan, and if it OK I then move the d and q
> > files to mqueue to bypass MailScanner.  It works, but I would like to get
> > to the bottom of the problem.
> >
> > In several such cases I noticed that the message contained a zip file 
> > together with another file.  In almost all cases the message was over 500 
> > KB in size (but as we regularly handle messages of up to 1.5 MB that is 
> > not in itself a particular problem).  On other occasions it was just a 
> > large pps file.
> >
> > I never see any specific error message in the maillog file (I was using
> > sendmail 8.13.1 before the upgrade to 8.13.6) - it reports that a
> > message has been processed by MailScanner but there is no corresponding 
> > delivery notice.  All the problem mail has been incoming to our users.
> >   
> I haven't been around for a while, so haven't seen this one.
> Please can you send me (off-list) the df and qf files (in a zip file) 
> along with a copy of your MailScanner.conf file (preferably without the 
> comments) so I can see your setup.
> 
> I hope I can reproduce the problem. The snag often is that I can't 
> reproduce the problem.
> 
> What I would also like you to do is, when you are tracking down the 
> errant message, shutdown MailScanner and then do
>     MailScanner --debug
> and note down any error messages that appear (except the EOCD signature 
> warnings). This may well help me locate the problem for you.
> 
> If I can't reproduce the problem on my system, but you have got a 
> message that reliably makes the problem appear, then remote access to 
> your system would enable me to track it down and get it fixed once and 
> for all.

Sorry for the late response to this.

I have been experimenting with some of the problem messages previously
archived and am not getting consistent results - sometimes they fail,
sometimes they don't.  Nothing shows up in debug mode because then they
don't fail.  However I have seen some clues that may lead to an
explanation:

	The messages mostly involved not just moderately large zip files,
	but highly compressible files - eg a 600 KB message
	containing a zip file that expanded to 3.5 MB.

	In another case a message contained a dat file, which I didn't
	originally realise could also be expanded, again to over 3 MB.

	There appears to be an association between processing failure
	and the following error in the maillog file:

		MailScanner[5811]: Commercial scanner clamav timed out!
		MailScanner[5811]: Virus Scanning: Denial Of Service attack detected!

I am now running:

	Red Hat 7.1
	sendmail 8.13.6 (configured to accept max message size of 1.5 MB)
	MailScanner 4.52.2
	ClamAV 0.88.1
with:
	500 MHz AMD-K6 CPU
	256 MB RAM

I assume now that the problem arises when the virus scanning (which seems 
very slow on this machine - a minimum of 30 seconds if scanning a single 
message) takes too long for the batch.  This could also explain why the 
problem sometimes clears itself - if the load level is low then the 
scanning can be done more quickly.

I have now changed the default setting for the following in 
MailScanner.conf from 300 to 600 seconds and hope it helps:

	Virus Scanner Timeout = 600

May I suggest for your next update that the error messages listed when the
virus scanner times out also include the SMTP id of the message that
causes the problem?  That would make the identification of the problem far
simpler as there would then be a direct association between the problem 
message and the relevant error message when grepping the log.  At the 
moment it is hard to notice the warning without a line-by-line scrutiny of 
the log file.

Regards

Jim Holland
System Administrator
MANGO - Zimbabwe's non-profit e-mail service