Random messages stuck in mqueue
Corey McFadden
cmcfadden01 at gmail.com
Wed Aug 1 23:01:06 IST 2007
Julian,
Thanks for the clarification on the Delivery Method behavior. What you're
saying makes a lot of sense and seems the optimal way to deliver the best
results under normal circumstances.
It's starting to look like it may be a DNS issue after all. The server has
a local DNS resolver bound to its IP address and configured via
/etc/resolv.conf. All DNS tests performed in the shell were very clean. It
seems though that sendmail (for whatever reason and despite its lack of
presence in resolv.conf) is attempting (on occasion) to talk to localhost.
About an hour ago I modified the bind configuration to listen on everything
and it seems like things may be improving.
Thanks for all the responses on this guys. I'll monitor this box for a few
hours and see what happens.
-Corey
On 8/1/07, Julian Field <MailScanner at ecs.soton.ac.uk> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
>
> Corey McFadden wrote:
> >
> > It does seem as though the problem begins when MailScanner disposes of
> > a message. Maybe someone could clarify the process whereby Sendmail
> > picks up the message and decides to deliver it (locally or relay or
> > whatever.)
> See my explanation below. The normal setting to use is "Delivery Method
> = batch". Though for debugging MailScanner, I run no outgoing queue
> runner process at all, and have "Delivery Method = queue". The result is
> that processed messages are placed in the outgoing mqueue and no attempt
> to deliver them is made at all. I can then look at the messages, check
> whether they are correct and that the development code is working, and
> then just delete them from the mqueue by hand. I don't want to ever
> actually deliver these messages, all the information I want is in the
> df+qf files.
> >
> > It doesn't look like a DNS issue. We've got a local resolver (that
> > has a PTR record for the machine's IP) and a backup resolver. I've
> > done a number of DNS tests and haven't been able to produce an error
> > there.
> It's usually fairly easy to test DNS speed problems with a few "dig"
> commands for some random domains (most English words .com exist, so
> picking a random word is usually pretty reliable, and probably won't be
> in your DNS cache already).
> >
> > On other systems everything happens so quickly that it's hard to see
> > what happens but does it resemble this:
> > - MailScanner finds message in mqueue.in <http://mqueue.in>
> > - MailScanner processes message and dumps qf/df into mqueue
> > - Sendmail daemon periodically scans mqueue
> >
> > or:
> > - MailScanner finds message in mqueue.in <http://mqueue.in>
> > - MailScanner processes message and spawns a sendmail process for
> > delivery
> It always put the qf/df in the mqueue first. It doesn't directly pass
> the message text to sendmail, that would involve an extra copy of the
> message being written down a pipe which slows things down. I try quite
> hard to make MailScanner as fast and efficient as I can. Which is why I
> have a full MailScanner+SpamAssassin+ClamAV setup on one box in my
> office which can process 2.2 million messages per day, on 1 server.
> > What are the implications of the "Delivery Method" queue vs. batch
> > option on the above?
> The explanation below is described for sendmail, but the same basic
> design applies to all the other supported MTAs as well. They all work in
> pretty much the same way, more or less.
>
> With "Delivery Method = queue" the processed message(s) is placed in the
> mqueue. That's it. You will then have a sendmail queue runner regularly
> attempting delivery of everything in the queue that's due for a delivery
> attempt. So if the outgoing sendmail process is running (ie the queue
> runner) then delivery will be delayed until the next regular queue run.
> You can start and stop this bit with "service MailScanner startout" and
> "service MailScanner stopout".
>
> With "Delivery Method = batch" the processed message(s) are placed in
> the mqueue. Then a "sendmail -qI....." command is then executed, with
> "....." set to the IDs of the message files just placed in the mqueue.
> This tells sendmail to immediately make 1 delivery attempt of each
> message. If this succeeds, the net result is instant message delivery.
> If it fails, then it is left to be retried by the outgoing sendmail
> queue runner process, as described above. In cases where the batch is
> large, measures are taken to ensure that the "sendmail -qI....." command
> is not too long for the operating system to be able to handle. When
> there are multiple mqueues in use, a separate "sendmail -qI....."
> command is issued for the messages in each outgoing queue, so that
> delivery of all messages is attempted, regardless of the number of
> outgoing queues they are spread across. This is needed as the mqueue
> directory can be specified by a ruleset or Custom Function, allowing you
> to use different queue runner parameters for each of several queues
> (e.g. a fast queue runner for local or small messages, with a slower one
> for remote or large messages, for example).
>
> Hopefully that explains what happens in full detail.
>
> >
> > Thanks for the input!
> >
> > -Corey
> >
> >
> >
> >
> > On 8/1/07, * Scott Silva* <ssilva at sgvwater.com
> > <mailto:ssilva at sgvwater.com>> wrote:
> >
> > Corey McFadden spake the following on 7/31/2007 9:46 PM:
> > >
> > > Guys,
> > >
> > > We're experiencing an issue on one of our boxes where messages
> > (a good
> > > percentage) are hung in /var/spool/mqueue after being processed by
> > > MailScanner. Even those to be delivered locally will sit for
> > > (sometimes) hours until they're finally delivered.
> > >
> > > I've never seen this behavior before and something is definitely
> > awry.
> > >
> > > Here's the environment:
> > >
> > > This is CentOS release 4.5 (Final)
> > > This is Perl version 5.008008 (5.8.8)
> > >
> > > This is MailScanner version 4.61.7
> > >
> > > Executing 'sendmail -q -v' WILL process the stuck messages but
> > can take
> > > quite a long time to run because it will try to deliver everything
> > > queued in a linear fashion.
> > >
> > Since it is in mqueue and not mqueue.in <http://mqueue.in>,
> > mailscanner is pretty much out of the
> > loop. Check if you have a FQDN that resolves properly as this will
> > choke
> > sendmail quickly. Also try a local caching nameserver, as sendmail
> > might be
> > taking a long time to resolve delivery addresses.
> >
> > --
> >
> > MailScanner is like deodorant...
> > You hope everybody uses it, and
> > you notice quickly if they don't!!!!
> >
> > --
> > MailScanner mailing list
> > mailscanner at lists.mailscanner.info
> > <mailto:mailscanner at lists.mailscanner.info>
> > http://lists.mailscanner.info/mailman/listinfo/mailscanner
> >
> > Before posting, read http://wiki.mailscanner.info/posting
> > <http://wiki.mailscanner.info/posting>
> >
> > Support MailScanner development - buy the book off the website!
> >
> >
>
> Jules
>
> - --
> Julian Field MEng CITP
> www.MailScanner.info
> Buy the MailScanner book at www.MailScanner.info/store
>
> MailScanner customisation, or any advanced system administration help?
> Contact me at Jules at Jules.FM
>
> PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
> For all your IT requirements visit www.transtec.co.uk
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: PGP Desktop 9.6.2 (Build 2014)
> Charset: ISO-8859-1
>
> wj8DBQFGsNm8EfZZRxQVtlQRAuAWAJ9bWzKisXCEcEeJGZ5TrFdMcBFAKgCcCMjh
> gmylU66oi8Mi4/pxd+5q+wg=
> =Sru3
> -----END PGP SIGNATURE-----
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
> For all your IT requirements visit www.transtec.co.uk
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20070801/379bf9b1/attachment.html
More information about the MailScanner
mailing list