Minimum hardware capacity for 35k e-mail scans/day

[Glenn Steen]

On 15/11/2007, Edward Prendergast <edward.prendergast at netring.co.uk> wrote:
> > Any advice from the vast pool of MailScanner experience would be greatly
> > appreciated.
>
> >> Are you queues actually building up or are you just concerned with the
> >> load average?
>
> In reply to Greg and Julian:
>
> The queue seems to go up with the load. By the time the load is around 10
> the queue starts to climb without any sign of going down. It usually reaches
> the 300 mark within half an hour to an hour, but has gone up as far as
> 3,000. When I turn bayes off the load goes down and mail queues start
> becoming manageable again.
>
> Our users largely use POP3, with a few using IMAP through squirrelmail.
> Their mailboxes usually tend to contain around 300 messages.
>
> Here's how vmstat looks now with bayes enabled:
> # vmstat 10 5
> procs -----------memory---------- ---swap-- -----io---- --system--
> ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id
> wa
>  0  5 377144 682732  11324  90488    1    0     4     1    0     0 15  5 70
> 10
>  1 12 377056 723572  12468  94280   47    2  2130   725 1475   795 12  4 45
> 39
>  1  4 376984 668068  15100 101712   53   15  2215   778 1486   759  7  3 50
> 40
>  1  5 377016 704180  13952 108476   56    4  2127  1596 1500   997 11  4 39
> 46

About the only really useful vmstat info is that you have a small swap
activity. si and so should be 0. So the timetested advice "Add more
RAM" holds true for you.
One can guess that a fair amount of I/O is going on, just plain
waiting... Hence the idle values... Investigating that might be
useful. Guess 1) would be bayes (which might be related to locking or
disk IO speed), guess 2) would be network (most likely DNS (caching
only DNS?) or a dead RBL or somesuch).
Oh, I see Steve message. Good thing to try. And add some more RAM...
Or split of some of the workload to another box;-).

> It looks to me like there might be some thrashing going on.

Thrashing is usually an audible phenomenon... Have you been listening
to your HDDs lately?:-).
Some swap activity, yes, but thrashing... Well, perhaps:).

> At this level, the queue looks like this:
> Nov 15 14:32:36 server6 MailScanner[8106]: New Batch: Found 2697 messages
> waiting
> Nov 15 14:32:36 server6 MailScanner[8106]: New Batch: Scanning 10 messages,
> 38032 bytes

Cheers
-- 
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se