Max Normal Queue Size

[Julian Field]

At 13:41 06/04/2004, you wrote:
>I had a large amount of email hit my server last night about 4000 items in
>the queue, it didnt seem to be clearing very quickly, however I noticed
>this in the config file :-
>
># If more messages are found in the queue than this, then switch to an
># "accelerated" mode of processing messages. This will cause it to stop
># scanning messages in strict date order, but in the order it finds them
># in the queue. If your queue is bigger than this size a lot of the time,
># then some messages could be greatly delayed. So treat this option as
># "in emergency only".
>Max Normal Queue Size = 800
>
>What actually happens when it goes into this "accelerated" mode ?  does it
>only deliver the newer messages in the queue, and wait until it is free to
>send the older stuff, as it did not seem to be clearing the backlog
>however when I changed the value to 4000 the queue dissapeared in about an
>hour or so and seems to have been okay since then.

In accelerated mode, it abandons the rule of always processing mail in
strict order of the time of arrival of the message. Instead it just uses
the first n messages it finds in the queue directory (when n is the maximum
batch size). This saves reading the whole directory and then pulling out
files that are (effectively) randomly scattered throughout the directory
structure. Any filesystem (most of them, except XFS) use simple lists for
storing directories, and so traversing the entire directory to open a
particular named file is actually an expensive operation for large directories.

By just processing the first n in the structure, you start clearing the
queue enormously faster.

Once the queue size exceeds the "max normal queue size" setting, it
switches into accelerated mode. It stays in accelerated mode while the
queue is still too big, and also after that for a further 40 message
batches in an attempt to clear out the  whole queue and not just to get it
just under the threshold size.

In my own tests, and in tests on customer sites, this approach has helped a
lot and I have never seen it cause a problem before.
--
Julian Field
www.MailScanner.info
Professional Support Services at www.MailScanner.biz
MailScanner thanks transtec Computers for their support
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654