Antistarving patch

[Julian Field]

I have put this in for all MTAs except Qmail. Can someone mail the guys at
opencomputing for me to get it into the qmail code. I don't want to fork
their code at all.

At 16:26 29/06/2004, you wrote:
>Hi,
>
>I noticed that, when you exceed 'Max Normal Queue Size', probably because
>there's more mail coming in than MailScanner can handle at one time, AND you
>are using multiple incoming queue directories (that is 'Incoming Queue Dir'
>has wildcards or is a file with a list of directories), the 'bottom' incoming
>queue directories tend to starve.
>
>That is, suppose you have:
>Incoming Queue Dir = /var/spool/mqueue.in/*
>Max Normal Queue Size = 800
>
>And you have 26 subdirectories in /var/spool/mqueue.in/ named 'A' thru 'Z'.
>
>If at one point you have some 2600 files evenly spread across these
>directories (that is, 100 in each) and your mail server keep filling them up
>evenly spread, MailScanner will be in 'emergency mode' and it won't order the
>incoming files to pick the older messages first.
>
>That is _good_ since, stating & sorting thousands of files in a crisis is A
>Very Bad Idea (TM).
>
>The bad thing is that MailScanner will _always_ pick first the files from the
>_first_ directories (in the order it got them when reading the configuration
>file).
>
>If the order was alphabetical, you'll notice that your 'A' subdirectory will
>(during the crisis) _always_ be smaller than the other ones... and messages
>in the 'X', 'Y', 'Z' subdirectories will probably starve until the crisis
>pass.
>
>I wrote a little patch that forces MailScanner to choose the directories in a
>random order every time it starts processing a batch IF it is in 'emergency
>mode'.
>
>I only tested with ZMailer, but the change is so small, simple and stupid
>that there is no reason it won't work with Sendmail, Exim, Postfix or Qmail,
>so I wrote the patch for every MTA MailScanner supports.
>
>The patch simply changes this line:
>     foreach $queuedirname (@queuedirnames) {
>
>With these:
>     my @aux_queuedirnames=@queuedirnames;
>     while( defined($queuedirname=splice(@aux_queuedirnames,
>         ($UnsortedBatchesLeft<=0 ? 0 :int(rand(@aux_queuedirnames))),1))) {
>
>when MailScanner is in 'emergency mode', the variable $UnsortedBatchesLeft is
>greater than 0 so, only when that happens, instead of picking the directory
>names in the order they are, the loop picks them randomly.
>
>I left the patch in http://baby.com.ar/MailScanner/NoStarveQueue/ the
>filename is MailScanner-4.31.6-NoStarveQueue.patch and it will apply silently
>to version 4.31.6 of MailScanner, but it shouldn't have any problems if you
>try to patch older (4.2x, 4.3x) versions.
>
>I also left the patched 4.31.6 source files in
>http://baby.com.ar/MailScanner/NoStarveQueue/
>
>Please let me know if you have problems with these patch.
>
>I'll try to package all my latest patches together before this weekend, so
>Julian can see them all together.
>
>
>--
>Mariano Absatz
>El Baby
>----------------------------------------------------------
>Always remember you're unique, just like everyone else.
>
>-------------------------- MailScanner list ----------------------
>To leave, send    leave mailscanner    to jiscmail at jiscmail.ac.uk
>Before posting, please see the Most Asked Questions at
>http://www.mailscanner.biz/maq/     and the archives at
>http://www.jiscmail.ac.uk/lists/mailscanner.html

--
Julian Field
www.MailScanner.info
Professional Support Services at www.MailScanner.biz
MailScanner thanks transtec Computers for their support
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654

-------------------------- MailScanner list ----------------------
To leave, send    leave mailscanner    to jiscmail at jiscmail.ac.uk
Before posting, please see the Most Asked Questions at
http://www.mailscanner.biz/maq/     and the archives at
http://www.jiscmail.ac.uk/lists/mailscanner.html