Still getting duplicate mails!

[Neil Robst]

I've tried the approach below and my users are still reporting getting
mulitple mail messages though apparently only where there are mulitple
recipients on the mail message (including them!).

The method described below does appear to be working according to my
maillogs... I can see the postfix/cleanup daemon hold the messages and
then MailScanner does it's bit and drops the messages in postfix's
incoming queue.

I've never seen any entires in the log file that suggest that mail is
being duplicated here... it's very strange. Can anyone suggest anything
else I can do?

Thanks for all your help so far,
Neil

On Tue, 2004-01-20 at 11:51, Drew Marshall wrote:
> As I understand Postfix doesn't use much in the way of file locking. It
> doesn't need to. In standard form a message is dropped into the onward
> directory and the next process is called using a 1b message and so mail
> makes it's way through the MTA. MailScanner upsets it by trying to grab
> the file from the deferred directory for processing. Now the deferred
> directory is used by Postfix as the place where mail is put when delivery
> fails, pending re-try (Keeps the active queues down) and every so often
> (As set in master.cf) the queue runner process goes to the deffered queue
> and inspects the messages for any that are due for retry. If the time
> stamp has expired it picks up the message and trys to deliver it. Through
> all of this there is not a need for much in the way of locking as what is
> going to touch that file? Postfix (As far as Postfix is concerned!) and
> Postfix knows what it's doing (We hope :-) ) If MailScanner and Postfix
> queue runner should happen to try to take the same message, you get the
> 'still being delivered' message in the logs and up pops a duplicated mail!
>
> Easy way round it, use the hold queue. This is designed to only have
> messages dropped in it for leter inspection by the postmaster and so the
> queue runner doesn't ever re-inspect this directory. Ideal for
> MailScanner, message gets dropped (MS knows how to tell when it's
> complete), picks up the new message, does it's bit and puts it back in the
> incoming queue for Postfix to deal with in it's usual efficient manner.
>
> I haven't had a single duplicate since putting this in place.
>
> Drew
>
> Neil Robst said:
> > And you think this resolves the duplicate mail problem?
> >
> > I'm unsure how it differs (apart from only having one postfix daemon
> > running) from using /var/spool/postfix.in/deferred and
> > /var/spool/postfix/incoming...?
> >
> > However, as I've just had a report from my users saying that upgrading
> > to the 4.26-4 beta hasn't worked, I'm willing to try anything :-)
> >
> >
> > On Tue, 2004-01-20 at 10:20, Drew Marshall wrote:
> >> Neil
> >>
> >> What I have done is below, as suggested by Peter Bates and forwarded to
> >> me
> >> from this list.
> >>
> >> > I'm using MS with Postfix in a slightly 'non-standard' way, but which
> >> is
> >> working fine for 13-15K messages we deal with (actually it might be
> >> more, I never bothered counting our outgoing email!)...
> >> > I'm using a 'header_check' like so:
> >> > In main.cf -
> >> > header_checks = pcre:/etc/postfix/header_checks
> >> > In header_checks -
> >> > /^Received:.*by .*\.your.domain.tld \(Postfix\)/ HOLD
> >> > This puts the incoming mail in the 'hold' queue, and then
> >> > I have in MailScanner.conf -
> >> > Incoming Queue Dir = /var/spool/postfix/hold
> >> > Outgoing Queue Dir = /var/spool/postfix/incoming
> >>
> >> With this, you will need to stop postfix.in and uncomment the smtp line
> >> in
> >> master.cf (Basically revert your set up to a non-MailScanner set up (It
> >> may be easier if Postfix.in runs chrooted and postfix doesn't to just
> >> alter postfix.in to become just postfix, what ever your mileage!)). Stop
> >> all instances and restart just postfix and you now have one postfix
> >> instance with MailScanner.
> >>
> >> Works great!
> >>
> >> Drew
> >> --
> >>
> >>
> >> Neil Robst said:
> >> > Drew,
> >> >
> >> > Can you explain a bit more about how you've configured postfix,
> >> please?
> >> I'm using the suggested setup of two postfix instances - the first runs
> >> everything in a chroot jail and smtp, local and virtual and deferred.
> >> Mailscanner then picks everything out the deferred queue, does it's
> >> stuff and drops it back into the incoming queue of the second postfix
> >> instance. Seems to be working well, but you said you'd changed postfix
> >> to bypass the duplicate problems...
> >> >
> >> > Regards,
> >> > Neil
> >> >
> >> > On Tue, 2004-01-20 at 09:06, Drew Marshall wrote:
> >> >> I've been running it now since the weekend without problem. I would
> >> suggest that although marked as a beta and potentialy unstable, it's
> >> about
> >> >> as unstable as the production releases :-) The new patches seem to be
> >> working well.
> >> >> I have to admit, I changed my Postfix set up to by pass the duplicate
> >> problems and haven't changed it back. I now use a rule in Postfix to
> >> hold
> >> >> all incoming mail, let MS collect from the hold queue (The queue
> >> runner
> >> doesn't ever run in there) and drop back into the incoming queue for
> >> delivery. It just means that I only have to ever run just one Postfix
> >> instance. I only ever use SMTP connection so don't have to worry about
> >> direct queue injection by passing MailScanner.
> >> >> Drew
> >> >> Neil Robst said:
> >> >> > Yes... fingers crossed!
> >> >> >
> >> >> > Any other issues known with the 4.26-4 beta currently? What's the
> >> general feeling in the community of it's stability, etc?
> >> >> >
> >> >> > On Mon, 2004-01-19 at 22:07, Drew Marshall wrote:
> >> >> >> Just for my 2p, my server doesn't have a high load but I suffered
> >> duplicate mail. My old set up on Slackware didn't suffer, the new on
> >> Gentoo did :-(  . I'm not quite sure why but it seemed that the
> >> >> Postfix
> >> >> >> queue runner and MailScanner got in each others way with the
> >> result
> >> >> that
> >> >> >> MS picked up incomplete messages.
> >> >> >>
> >> >> >> <fingers crossed> Any way that's all in the past now </fingers
> >> >> crossed>
> >> >> >>
> >> >> >> Drew
> >> >> >>
> >> >> >> Neil Robst wrote:
> >> >> >>
> >> >> >> >Hi all,
> >> >> >> >
> >> >> >> >Just applied the 4.26-4 beta of MailScanner to my mail server,
> >> >> though
> >> >> >> I've
> >> >> >> >been unable to replicate the problem with the duplicate mails
> >> either
> >> >> >> before
> >> >> >> >or after (as expected) the upgrade. Do you know any details about
> >> that -whether it only manifested itself when there were lots of
> >> >> >> recepients
> >> >> >> >on the message or a high load on the server or what?
> >> >> >> >
> >> >> >> >Regards,
> >> >> >> >Neil
> >> >> >> >
> >> >> >> >
> >> >> >> >--
> >> >> >> >This message has been scanned for viruses and
> >> >> >> >dangerous content by MailScanner, and is
> >> >> >> >believed to be clean.
> >> >> >> >
> >> >> >> >
> >> >> >>
> >> >> >> --
> >> >> >> In line with our policy, this message has
> >> >> >> been scanned for viruses and dangerous
> >> >> >> content by MailScanner, and is believed to be clean.
> >> >> >> www.themarshalls.co.uk/policy
> >> >> >
> >> >> --
> >> >> In line with our policy, this message has
> >> >> been scanned for viruses and dangerous
> >> >> content by MailScanner, and is believed to be clean.
> >> >> www.themarshalls.co.uk/policy
> >> >
> >>
> >>
> >>
> >>
> >> --
> >> In line with our policy, this message has
> >> been scanned for viruses and dangerous
> >> content by MailScanner, and is believed to be clean.
> >> www.themarshalls.co.uk/policy
> >
>
>
> --
> In line with our policy, this message has
> been scanned for viruses and dangerous
> content by MailScanner, and is believed to be clean.
> www.themarshalls.co.uk/policy