Old messages no being processed
MailScanner at ecs.soton.ac.uk
Wed Sep 28 20:36:53 IST 2005
[ The following text is in the "ISO-8859-1" character set. ]
[ Your display is set for the "US-ASCII" character set. ]
[ Some characters may be displayed incorrectly. ]
-----BEGIN PGP SIGNED MESSAGE-----
Terran Wright wrote:
>My problem now is that it seems that only new mails are being processed and
>the older mails in the queue are being neglected. The queue went down to
>less than 2000 mails and has now gone back up to > 7000 mails. I've
>implemented spamhaus, spamcop and various other rbls at the MTA level
>temporarily to help clear up the queue however this is not helping.
What you are seeing is a side-effect of the failure of the "Emergency
Queue-Clearing Mode" that MailScanner will switch into when the queue
gets very large.
Normally, MailScanner processes messages in strict date order, oldest
first. In order to do this, for every batch it has to read in the entire
directory and sort it by date. On some filesystem designs this can take
a very long time. So when the queue gets bigger than the "Max Normal
Queue Size", MailScanner switches into a mode where it just processes
the first messages it finds in the queue, regardless of how old they
are. On many systems, this is a lot faster as it doesn't have to read in
the whole dir or do any sorting.
If you think about it, you will see that if you process the first 30 and
delete them from the incoming queue, that leaves 30 blank slots that
will be filled by the next 30 messages to arrive in the queue. So if you
are only just keeping up, you will end up continuously processing the
newest messages and not clearing the backlog very much. This is actually
exactly what most people want as it enables MailScanner to continue to
process new mail very quickly, giving the appearance of it staying
responsive to your users. When your incoming mail load drops (when
everyone goes home or to sleep or whatever your users do), it will slog
through the old backlog and sort itself out.
If you look at your system when it has a high incoming load in this
state, it won't appear to be processing old mail very much, if at all.
So you can either leave it to get on with it, as it will deliver the old
messages when it can. Or else you can raise the "Max Normal Queue Size"
number a lot to force it to stay in the normal date-order processing
mode. But that will add a large delay to all new mail as well as the old
stuff currently in the queue. So it will actually appear to be worse to
your users than just letting it get on with it.
It's actually quite a bit more clever about it than my explanation
above, especially when and if it decides to switch back to normal
oldest-first processing. It's not a trivial solution, as you need some
hysterisis in it to avoid a positive feedback oscillating system. And
who would have thought that an email system involved concepts explained
in detail in 2nd-year degree level mathematics. Feedback theory, hmmm.....
Sorry that's quite so long an explanation, but the background and
rationale for it is quite a complex behaviour model.
I suspect that most other systems out there don't quite go to such
Hope that helps explain what's happening,
Buy the MailScanner book at www.MailScanner.info/store
Professional Support Services at www.MailScanner.biz
MailScanner thanks transtec Computers for their support
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.0.2 (Build 2424)
-----END PGP SIGNATURE-----
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the Wiki (http://wiki.mailscanner.info/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
Support MailScanner development - buy the book off the website!
More information about the MailScanner