sendmail message splitting defeats bandwidth savings?

[Furnish, Trever G]

Well, I had thought of suggesting re-submitting the message after deciding
simply that it needed to be split, but that would remove the ability to
check based on sender IP, right?

On the other hand, if we just do whatever needs to be "done" to the message
and then submit it for outbound delivery, then all we have to do is deal
with queue formats, right?

So maybe we could just use a different queue for outbound messages that have
been split.  No second pass through MS, just delivery via a modified
mechanism.  For example, how about delivery through a third sendmail, this
one listening on a different port and just accepting submissions and sending
them immediately?

Ie we'd end up with these sendmail parents:
1. Port 25 listener, queueing into mqueue.in
2. Normal delivery queue runner, watching mqueue
3. Split-message listener/runner, listening on port 1024 and performing
immediate delivery, queuing into mqueue.split only if needed.

Then again, if you were going that route, then would there still be a
significant advantage in keeping both this new delivery method and the
"traditional" queue file moving method?  (Probably - it seems much faster.
But maybe this would make the postfix author a bit more agreeable too. ;-) )

I have the painful apprehension that I may be typing more quicly than I'm
thinking. ;-)

-t.


> -----Original Message-----
> From: Julian Field [mailto:mailscanner at ECS.SOTON.AC.UK]
> Sent: Monday, November 03, 2003 12:53 PM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: sendmail message splitting defeats bandwidth savings?
>
>
> I would need a *very* reliable SMTP client, for which Net::SMTP (or
> whatever it's called) might be a good candidate.
> But it's definitely a cool idea, it's even independent of the
> MTA which I
> definitely like.
>
> What I would probably have to do is split every message that
> has multiple
> recipients, it is hard to work out if any of the rules would produce
> different results for each recipient of the message. Not
> impossible, but
> not easy. I would need to collect the results of every config
> variable for
> each message in a batch, and then look through all the
> results to find ones
> that aren't all the same. Then I would need to duplicate the
> message back
> into loopback once for each recipient and throw away the
> original (as I
> probably wouldn't be able to tell at that point which result
> I should be
> using).
>
> That's got me thinking...
> (and there I was planning a nice quiet evening watching the telly :-)
>
> At 17:28 03/11/2003, you wrote:
> >Maybe an idea...
> >
> >Have MailScanner split and resubmit the message, as needed
> by the rule
> >sets, to sendmail on the loopback interface before spam or virus
> >checking happens. Then sendmail can control its own queue numbers and
> >the message will be single recipient per message for the rest of the
> >checks.
> >
> >Thoughts?
> >
> >Brent Strignano
> >System Administrator
> >Granite Capital Holdings
> >Sidney NY USA
> >
> >
> >
> >-----Original Message-----
> >From: Julian Field [mailto:mailscanner at ECS.SOTON.AC.UK]
> >Sent: Monday, November 03, 2003 11:50 AM
> >To: MAILSCANNER at JISCMAIL.AC.UK
> >Subject: Re: sendmail message splitting defeats bandwidth savings?
> >
> >
> >At 16:04 03/11/2003, you wrote:
> > >On Mon, 3 Nov 2003 10:00:19 -0500, you wrote:
> > >
> > > >By contrast what I'd prefer MS to do is: if a message
> comes in bound
> > > >for multiple recipients and only a few of those
> recipients should be
> > > >handled specially (whitelisted), create separate copies of the
> > > >message for those recipients, queuing the files into mqueue by
> > > >generating its own IDs.
> > >
> > >MS should be kept up to date with changes in de qf and df files of
> > >sendmail. And it should be able to distinguish between the
> different
> > >sets of changes.
> >
> >One of the main reasons I haven't done this before is that reading
> >message filenames is a lot easier than creating new ones.
> For example,
> >Sendmail has changed its format at least once that I can immediately
> >think of, and it is non-trivial to work out (given an empty queue at
> >startup) which format of filename I should use.
> >
> >When only sendmail is creating them, it's easy, I just use whatever
> >filenames it supplies. But if I want to create unique new
> ones, then how
> >do I work out what to call them? I need to keep strictly to
> its naming
> >scheme so that if the sendmail folks tighten up their queue filename
> >checking code, everything keeps working. So it's not good enough to
> >"just do something that works", I have to get it 100% correct.
> >
> >I also have to guarantee that any new filename I create won't be
> >possibly re-used later by the MTA. For example...
> >
> >The queues all start off empty, for simplicity.
> >
> >A message 1111 comes in, with 2 recipients with different
> rules, so it
> >needs to be split. 2222 is a legal name for this MTA, and is
> not in use
> >right now. So MailScanner creates 2 output messages 1111 and
> 2222. Then
> >the MTA receives another message, which it decides to call
> 2222 (which
> >isn't in use in the incoming queue, so I can't stop it doing it).
> >MailScanner processes that and tries to create another
> message 2222 in
> >the outgoing queue, which clashes with the earlier one.
> >
> >Consider what happens when 2222 has been in the outgoing queue for
> >nearly a week, and is still waiting to be delivered. How do
> I stop the
> >incoming MTA creating a queue file with a name that hasn't
> been used in
> >the past week/month/year?
> >
> >It can't be done.
> >
> >I welcome comments to the contrary... :-)
> >--
> >Julian Field
> >www.MailScanner.info
> >MailScanner thanks transtec Computers for their support
> >
> >PGP footprint: EE81 D763 3DB0 0BFD E1DC  7222 11F6 5947 1415 B654
>
> --
> Julian Field
> www.MailScanner.info
> Professional Support Services at www.MailScanner.biz
> MailScanner thanks transtec Computers for their support
> PGP footprint: EE81 D763 3DB0 0BFD E1DC  7222 11F6 5947 1415 B654
>