Max Children and List Server Lagging

[David Lee]

On Thu, 16 Nov 2006, Jim Coates wrote:

> [...]
> However, I've noticed now that I can see the # of children MailScanner has
> running that it seems to be ignoring my Max Children setting.
>
> I have Max Children set to "5", but recently we were doing a large mailing
> (we house a few mailing lists) and I noticed that it said there were 20+
> SendMail procs and 11 MailScanner children.

What follows is from (possibly faulty) memory, from a non-expert.  I think
it is correct, but I may be wrong.

Looking at just the MS processes (not sendmail):

I seem to recall that this behaviour is actually to be expected.  Let's
represent the "Max Children" as "N".  There will always be a parent
process, which will start up the "N" children each of which lives for
several hours.  So even on a quiet system there will be "N+1" MailScanner
processes.  When incoming emails are discovered, it seems that one of the
children itself spawns a child (so, overall, a grandchild) to handle it,
for a short time.

So on a very busy system, if each child itself has a child there could be
up to "2N+1" MS processes.

In your case N=5.  The 11 you observe is this "2N+1".

On a reasonable system you would expect to exceed "N+1" (6 or more on
yours).  But if it is regularly close to "2N+1" (close to 11 on yours)
then you probably do have problems.

Julian recommends setting "Max Children" (i.e. our "N") to 5 per CPU on
the box.  So your system it is probably about right as a starting point.

BUT...

> I've read here in the past that too many children will slow a box down if it
> doesn't have the memory and CPU to handle it, which I think is exactly the
> case going on with me.  We were sending out a large mailing today and it is
> taking an extraordinary amount of time (going on 14 hours of trying to send
> out a total of 25,000 emails).

CPU:  I understand that an apparently high load average might not be a
problem, if your "Max Children" is right.  (Which yours seems to be.)

Memory:  Things are memory hungry.  The more memory you have, the better.
In "MailScanner.conf" near the "Max Children" setting is a comment about
MS needing about 20MB per MS process.  My own view is that this is way too
small (the comment is probably old and in need of revision) especially if
SpamAssassin is being run.

> We do run grey listing, but our IP is white listed for the milter-greylist
> and in the white list within MailScanner, so I'm not sure what is causing
> the deep delays or the extra children showing from MailScanner.

I recently tried milter-greylist.  For us, that was a real CPU hog, and I
had to back out of it.  (I understand the development version will have
a major efficiency improvement.)

[Extra MS children: as discussed above: to be expected, up to "2N+1".]


Another thing to investigate: How well is your DNS resolution performing?

We recently had a misconfiguration so that every DNS attempt was made to a
local (127.0.0.1) server, but there wasn't a local server!  Long timeouts.
As soon as I made this consistent, things improved dramatically.

Some folk advise running a caching DNS server within your MS box.  That's
nice if you can do it, but far important is to check that the first
"/etc/resolv.conf" entry has a well-running server.



Summary:

1. "Max Children": Set to 5 is OK.  Seeing up to 11 MS processes is OK.

2. Memory: The more the merrier.  You probably want at least 1GB.

3. Check that your DNS resolution is OK.  In particular that all the
servers, and especially the first, listed in "/etc/resolv.conf" are
running well.


Hope something there is of help.

-- 

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:                                           Durham University     :
:  http://www.dur.ac.uk/t.d.lee/            South Road            :
:                                           Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :