performance impact of growing rules files

[Ken A]

Ramprasad wrote:
> Hi,
>   We scan mails for quiet a large number of domains ( around 1.5k
> domains). The scanning happens on multiple identically configured MS
> +postfix+SA  linux boxes behind load balancers
> 
>    For every domain that is added there will be entries in
> spamcheck.rules spamaction.rules etc. Besides the domains will have
> their own whitelists and blacklists which go into whitelist/blacklist
> rules files. Already these have more than 10000 lines  each
> 
> I am not sure how this architecture will scale. Additional hardware is
> not a problem , but the solution must scale
> 
>  Assume I have 10x more domains and traffic next year .. will there be a
> performance hit because Mailscanner has to read such huge rules files.
> What will be a 100% scalable architecture 

You certainly don't want a million rules stuffed into RAM every time 
MailScanner starts up! If the largest number of rules you want 
MailScanner to work with is 10,000 rules, then figure out how many rules 
your average domain has, and divide up your MS boxes into groups based 
on that. Then set the MX for domains [a-c] to MX1, domains [d-f] to MX2 
and so on... Then have your load balancers handle which group of boxes 
those MX's map requests to. This way you don't have an excessive number 
of rules on any one group of MS boxes.

Ken
Pacific.Net

> Thanks
> Ram
> 
> 
> 
>