sendmail message splitting defeats bandwidth savings?

[Furnish, Trever G]

> -----Original Message-----
> From: Raymond Dijkxhoorn [mailto:raymond at PROLOCATION.NET]
> Sent: Monday, November 03, 2003 10:14 AM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: sendmail message splitting defeats bandwidth savings?
>
>
> Hi!
>
> > Hopefully I'm just incorrect in my current understanding of
> what happens
> > when a message is split by sendmail (please correct me if
> so), but this is
> > how I think things change when queue groups are used:
> >
> > Without queue group message splitting:
> > 1. One message comes in meant for many recipients at the
> same domain.
> > 2. Sendmail writes one queue file pair.
> > 3. MailScanner scans and re-queues that message.
> > 4. Sendmail delivers the message, sending it ONLY ONCE over
> the wire to the
> > next MX.
>
> Step 4 is unclear to me, if people have custom rules, and the one user
> doenst want spam tagging and one user does, how will that be
> combined into
> one message? In my eyes, you cant.

Agreed, but there is a huge difference between splitting only those messages
that need special handling (my suggested approach) and splitting ALL
messages (the current approach).  In fact the difference for a site handling
a significant amount of email is likely to be *many* orders of magnitude.

> > 3. MailScanner scans and re-queues all of the (now many) messages.
> > 4. Sendmail delivers the messages, one copy per recipient,
> resulting in the
> > original message being sent MANY TIMES over the wire to the next MX.
>
> Even if MS will do this trick i think mailvolume increase is
> the only way
> to accomplish this approach.

Yes, but again, the question is *how much* it ought to increase.  The
current approach has the potential to increase it drastically, with most of
the increase being completely unrelated to productive mail filtering.

Here's a more detailed example.  Suppose I want to whitelist mail to
bob at foo.com but not to tom at foo.com or harry at foo.com.  A single message comes
in bound for all of them:

To: bob at foo.com, tom at foo.com, harry at foo.com

With queue group message splitting, it becomes three messages:

To: bob at foo.com
To: tom at foo.com
To: harry at foo.com

With MailScanner doing the message splitting on its own, there would only be
a need to for two messages:

To: bob at foo.com
To: tom at foo.com, harry at foo.com

Now suppose a message comes in for harry at foo.com and tom at foo.com.  Even
though there's no need for message splitting at all, sendmail queue groups
would split the message in two, doubling its traffic.

I believe in most environments, the percentage of messages that actually
need special whitelisting will be FAR outweighed by the percentage of
messages that would just pass through MS untouched.

And in my case, unfortunately, email messages may be 150MB in size,
frequently sent to several local addresses.  Put those addresses behind a
192k/s link and multiplying the message-related bandwidth use becomes a big
problem.