Antistarving patch

[Mariano Absatz]

Hi,

I noticed that, when you exceed 'Max Normal Queue Size', probably because
there's more mail coming in than MailScanner can handle at one time, AND you
are using multiple incoming queue directories (that is 'Incoming Queue Dir'
has wildcards or is a file with a list of directories), the 'bottom' incoming
queue directories tend to starve.

That is, suppose you have:
Incoming Queue Dir = /var/spool/mqueue.in/*
Max Normal Queue Size = 800

And you have 26 subdirectories in /var/spool/mqueue.in/ named 'A' thru 'Z'.

If at one point you have some 2600 files evenly spread across these
directories (that is, 100 in each) and your mail server keep filling them up
evenly spread, MailScanner will be in 'emergency mode' and it won't order the
incoming files to pick the older messages first.

That is _good_ since, stating & sorting thousands of files in a crisis is A
Very Bad Idea (TM).

The bad thing is that MailScanner will _always_ pick first the files from the
_first_ directories (in the order it got them when reading the configuration
file).

If the order was alphabetical, you'll notice that your 'A' subdirectory will
(during the crisis) _always_ be smaller than the other ones... and messages
in the 'X', 'Y', 'Z' subdirectories will probably starve until the crisis
pass.

I wrote a little patch that forces MailScanner to choose the directories in a
random order every time it starts processing a batch IF it is in 'emergency
mode'.

I only tested with ZMailer, but the change is so small, simple and stupid
that there is no reason it won't work with Sendmail, Exim, Postfix or Qmail,
so I wrote the patch for every MTA MailScanner supports.

The patch simply changes this line:
    foreach $queuedirname (@queuedirnames) {

With these:
    my @aux_queuedirnames=@queuedirnames;
    while( defined($queuedirname=splice(@aux_queuedirnames,
        ($UnsortedBatchesLeft<=0 ? 0 :int(rand(@aux_queuedirnames))),1))) {

when MailScanner is in 'emergency mode', the variable $UnsortedBatchesLeft is
greater than 0 so, only when that happens, instead of picking the directory
names in the order they are, the loop picks them randomly.

I left the patch in http://baby.com.ar/MailScanner/NoStarveQueue/ the
filename is MailScanner-4.31.6-NoStarveQueue.patch and it will apply silently
to version 4.31.6 of MailScanner, but it shouldn't have any problems if you
try to patch older (4.2x, 4.3x) versions.

I also left the patched 4.31.6 source files in
http://baby.com.ar/MailScanner/NoStarveQueue/

Please let me know if you have problems with these patch.

I'll try to package all my latest patches together before this weekend, so
Julian can see them all together.


--
Mariano Absatz
El Baby
----------------------------------------------------------
Always remember you're unique, just like everyone else.

-------------------------- MailScanner list ----------------------
To leave, send    leave mailscanner    to jiscmail at jiscmail.ac.uk
Before posting, please see the Most Asked Questions at
http://www.mailscanner.biz/maq/     and the archives at
http://www.jiscmail.ac.uk/lists/mailscanner.html