New server request

[DAve]

Richard Frovarp wrote:
> DAve wrote:
>> Richard Frovarp wrote:
>>> DAve wrote:
>>>> Currently we get hit with 200k to 300k connections a day that hit an 
>>>> RBL. We see 15k to 25k pipeline attempts. We spam scan almost 50% of 
>>>> our mail and we Virus scan everything that comes in. We process 4gb 
>>>> of mail a day on two servers, total around 50k to 65k message we 
>>>> actually deliver. We process 16,908 whitelist and 14,348 blacklist 
>>>> entries from MailWatch.
>>>>
>>>> Mail delivery for our clients *INCLUDES* outbound scanning and 
>>>> filtering through my smtp servers (different hardware) and coming 
>>>> back in through my MailScanner servers.
>>>>
>>>> I can get that done in 5 minutes round trip time for a message. 90% 
>>>> of that time is spent in the MS server, queues, waiting for pickup, 
>>>> etc. I think that is pretty darned good.
>>>>
>>>> That is apparently not good enough. Every month or so I get told 
>>>> that mail delivery in incredibly slow and I need to look at the 
>>>> servers. I do, and every message I check takes around five minutes.
>>>>
>>>> I need a recommendation for the root'n toot'nist, rockem sockem, 
>>>> nuklear powered, rocket fuel fed servers money can buy. I want to 
>>>> push a batch of 30 messages through a full featured install of SA, 
>>>> Clamav, and local rulesets in less than 5 seconds. Tops. When my 
>>>> sales director hits send in his outlook, I want the message to 
>>>> deliver so fast his laptop jumps from his desk.
>>>>
>>>> I think I need striped SAS disks with 15k spindles, four CPUs, and 
>>>> 16gb of ram. I am open to realistic suggestions, though humor is 
>>>> still welcome. I intend to submit a quote this week.
>>>>
>>>> Thanks,
>>>>
>>>> DAve
>>>>
>>>
>>> I've got an old 2.66 GHz dual Xeon with 2 GB of RAM that pushes 
>>> through mail relatively well. Standard RAID 1 SCSI disks. Right now 
>>> it's doing batches of 2 in about 15 seconds. It handles about 4 GB of 
>>> of traffic and scans about 46 K a day. I would expect a dual quad 
>>> core with the requisite amount of RAM would be plenty. Network tests 
>>> take a while anyway, and there isn't much you can do to speed that 
>>> up. I am running greylist, greet pause, valid user lookup, and 
>>> blacklists in sendmail to reduce the load. I also have two other 
>>> machines that see similar load.
>>>
>>
>> Not much different that the servers we currently run. We do not run 
>> RAID at the moment. Except I have two servers were you have one. 
>> Batches of 2 take about 6 seconds, in the evening. During peak hours I 
>> get batches of 10 that require anywhere from 60 to 190 seconds. I can 
>> go from 7 messages waiting to 300 messages waiting in the blink of an 
>> eye. Though left to it's own, MS will chew through them just fine.
>>
>> We also run greylisting (with client's whitelisted), greetpause (with 
>> our own network whitelisted), RBL (in MTA), caching DNS, and 
>> milter-ahead to the pop toasters.
>>
>> DAve
>>
>>
> Actually I have 3 public facing and 1 internal MailScanner boxes. Lower 
> your batch sizes. How many of those 300 are really waiting? If you are 
> doing batches  of max of 10 with 10 children, that's 100 messages being 
> processed at the moment. If you have max batch sizes of 30, that's all 
> 300 being processed.
> 
> Assuming that other aspects aren't affecting load, the batch performance 
> would seem to be better with smaller numbers of messages. You may want 
> to try lowering the batch sizes. Sometimes less is more.

I thought so too, it seems at least with our mail, more children 
processing smaller batches is faster than large batches and fewer 
children. I currently run 15 children and batch size of 10 and doing well.

DAve

-- 
In 50 years, our descendants will look back on the early years
of the internet, and much like we now look back on men with
rockets on their back and feathers glued to their arms, marvel
that we had the intelligence to wipe the drool from our chins.