MailScanner Deficiency: Multi-Ruleset Processing per Email Recipient

[Glenn Steen]

Can only agree with Martin and Alex, there is no way around either
splitting mails per recipient (very feasible), or som major rework of both
the MailScanner and mailWatch code (very infeasible).
But I also have to agree that the increase in hardware seem quite
excessive... i suppose you arrived at that figure by analysing the number
of recipients per mail (and frequency of multi-recipient emails)? Well, the
number isnät everything:-)
Provided you use the normal caching-dns-thingy and also use "Cache
SpamAssassin Results = yes", the actual processing time and resource use
will be minimized (not to mention that the normal batch-processing style of
MailScanner will ... help...:-).
Introducing a "splitting MX" between the internet and your regular
MailScanner hosts should be rather simple, as well as adjusting which
Received: lines your MailScanner hosts should ignore (since they otherwise
will perceive all messages as originating from the "splitting MX" host)...
So why not try that, with the gear you have ATM, and see where that leads
you? Depending on what mailstore hosts you eventually deliver to, the
storage impact should be minimal or even non-existant, since even
M-Sexchange has abandioned "single store" since ... way back... so every
recipient would eventually have their own copy in their own mailbox
anyway;-).

As Alex says, we know nothing about your actual mail volume, but my money
is on there being much less of a problem than you think, even if you do
have ... serious traffic... (more than a few thousand mails/hour). the
likeliest problem point/bottleneck is likely your MailWatch database so...
keep an eye on that one, make sure you run it as InnoDB etc.

Cheers!
-- 
-- Glenn


On 11 July 2014 15:49, Martin Hepworth <maxsec at gmail.com> wrote:

> Might want to also consider having a more flexible approach as Alex had
> mentioned.
> Will also help with some of the hardware requirements as you can also
> reject non-valid recipients at MTA as well as splitting the emails up, so
> the core MailScanner farm has less to do.
>
> --
> Martin Hepworth, CISSP
> Oxford, UK
>
>
> On 11 July 2014 09:51, Sam Gelbart <samg at synaq.com> wrote:
>
>> Hi All,
>>
>> We at SYNAQ use and have used Mailscanner for many years. As an Email
>> Hygiene provider MailScanner has served us very well.
>> However, as we have grown (very rapidly in the past 6 months, to many
>> more customer domains) we have noticed some deficiencies in MailScanner.
>>
>> Below is a brief description covering our problem areas:
>>
>> Overview
>> The issue has arisen due to SYNAQ's ever growing client base and the fact
>> that we're provisioning more and more customers (and email domains) on our
>> hygiene platform, and that more than one of these customer
>> recipients/domains (and their applicable rulesets) are being addressed in
>> the same email.
>>
>> Problem 1
>> 1) abc.co.za and xyz.co.za are both provisioned on our platform.
>> 2) abc.co.za has quarantining of SPAM configured, while xyz.co.za does
>> not.
>> 3) Mailscanner accepts the message for processing but "chooses"
>> user at abc.co.za and abc.co.za as the Message's "to_address" and
>> "to_domain".
>> 4) MailScanner determines that the message is SPAM and because it has
>> "chosen" @abc.co.za as the email domain it deletes the message as the
>> configured spam action for @abc.coz.a is to delete.
>> 5) However the rule for xyz.co.za is to store/quarantine spam. This does
>> not happen because of the actions above and data is also never logged via
>> MailWatch.
>> 6) The example above is a based on very simple scenario, and as you are
>> aware this applies to many more complex rulesets (size, File Type etc)
>> across the system.
>>
>> Problem 2
>> 1) abc.co.za and xyz.co.za are both provisioned on our platform.
>> 2) A third party emails both user at abc.co.za and user at xyz.co.za in a
>> single email message.
>> 3) Mailscanner accepts the message for processing but "chooses"
>> user at abc.co.za and abc.co.za as the Message's "to_address" and
>> "to_domain".
>> 4) When the message is processed, the MailWatch.pm script receives a
>> message object for SQL logging with data only for user at abc.co.za and
>> abc.co.za; xyz.co.za is never logged.
>>
>> Finally we have considered splitting incoming messages by recipient at an
>> MTA level to address this problem, but our calculations show that it would
>> require 3.5x more hardware to process this increased mail load. So for us a
>> MailsScanner solution is ideal.
>>
>> Based on the above, could you tell me if there is anything that can be
>> done from a MailScanner community point of view to help develop MailScanner
>> functionality to address these issues?
>> We'd be very happy to give a nice donation for a fix or patch.
>>
>> Also if the community has any ideas on other ways we can remedy this
>> problem we welcome your feedback.
>>
>> Thanks and regards,
>>
>> Sam Gelbart
>> SYNAQ
>>
>>
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>
>> Before posting, read http://wiki.mailscanner.info/posting
>>
>> Support MailScanner development - buy the book off the website!
>>
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>


-- 
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20140805/ba2f40a1/attachment.html