Mailscanner mqueuein trouble

[Logan Shaw]

On Wed, 19 Jul 2006, Marcos Sobral wrote:
> Checks.
> Virus Scan with clamav
> Attach Filename check
> SpamAssassin
>
> I forward the spam to another acount to do the spam-digest.
>
> There are some things that maybe could help me with the tunning of 
> MailScanner, like the Number of mails in each process, now that I´m working 
> with a tmpfs maybe I could rise the Max Messages Per Scan values.

I'm very doubtful that your tmpfs is gaining you any significant
performance increase.  Unless your users are sending and
receiving nothing but large mail messages, the I/O to read
and write the messages to and from disk is very unlikely to
be the bottleneck.

Mail scanning is just not a very I/O intensive operation.
Instead, with something like MailScanner and SpamAssassin,
usually the scarce resources are CPU time and memory.

> And what about the Childs, is it good to have a good number of childs? do 
> they allow me to process more mail?

Here is the way I understand it:

The way to achieve maximum throughput (without disabling tests
or changing what you're testing) is to, as much as possible,
make sure the CPU or CPUs are not sitting idle while there
is work to be done.  If the processes were cpu-bound, then
one child per CPU would be enough.  But, for example, if a
child process is checking a message against a DNS blacklist,
the child process will block while it is waiting to get a reply
back from the DNS server.  If there is only one child per CPU,
the CPU will sit idle while the child is blocked waiting on
the network.  So, you need to increase the number of children
to the point where there is always one (or more) runnable.
(A similar situation occurs if a child blocks on a disk wait,
but with a modern kernel and modern hardware, in practice this
is pretty rare, since sendmail will write the data to disk and
it will remain in cache for MailScanner to grab instantly with
no physical I/O necessary.)

Balanced against this is the fact that increasing the children
increases the memory used that is being actively used by the
system.  If you reach the point where physical RAM isn't enough
and the system starts paging, then performance becomes terrible.

A whole different approach to coping with blocked processes
is to reduce the external factors that cause them to block.
Do you have a dedicated caching-only DNS server set up on
the machines runnin MailScanner, or at least on a machine
(that isn't overloaded) on the same network so that they can
access it quickly?  The longer it takes to get replies to DNS
queries, the longer the children will block.  Also, if your
mail volume is high enough, it may be good to get local copies
of some blacklist databases.  I don't run a high-volume server,
so I haven't looked into that and can't give specifics.

> How can I know which amount is the right 
> one for my servers?.

By testing!  Set it to a reasonable starting value, like 5
children per CPU.  Then watch the system under heavy load when
the input queue has lots of messages in it.  Run "top" and check
the CPU usage.  Is the CPU idle percentage significant (like
over 5% or 10%)?  If so, you could maybe benefit by increasing
the number of children.  If not, then adding children won't
make that much of a difference, as a general rule.

   - Logan