Invalid Queue Files with Postfix

[Brad Beckenhauer]

>>> On 8/17/2007 at 3:39 PM, in message
<223f97700708171339h2115ce1fu1af220aa23462924 at mail.gmail.com>, "Glenn
Steen"
<glenn.steen at gmail.com> wrote:
> On 17/08/07, Brad Beckenhauer <bbecken at aafp.org> wrote:
>> >>> On 8/17/2007 at 9:34 AM, in message
>> <46C5B203.5020108 at ecs.soton.ac.uk>, Julian
>> Field <MailScanner at ecs.soton.ac.uk> wrote:
>> > What versions of MailScanner and Postfix are you using?
>> > If you are using a very recent Postfix, particularly if you are
using
>>
>> > milters at all, you need to be running a pretty modern
MailScanner.
>>
>> Currently running:
>> postfix-2.1.5-5   upgrading to postfix-2.3.x in September 2007
>> MailScanner 4.60.8  also upgrading in September.
>> Centos 4.5
>> At some point I'll move to sendmail when my work project load
permits.
>>
>> Postfix has been running on 2.1.5 since June 2006, and I normally
keep
>> MailScanner up to date.
>> It's pretty normal to find several (2-20) of these invalid queue
files
>> daily, so I check daily to make sure the queues keep clean.
> 
> Um, no ... or perhaps, depending in your normal volume that might be
> normal.... But really, invalid queue files aren't really ...
normal:-)

Normal, as in "I expect to find invalid queue files in the system", not
that they are supposed to be there. :-)
Perhaps I should have said, on an average day, MailScanner reports
finding between 2-20 of these invalid queue files.  Which I manually
clean out using postsuper -d queue_file.

> 
> Being slightly intoxicated (sorry Hugo), I don't quite remember if
> there was any changes to my milter changes ..... after 4.60.8, but
you
> might benefit from an upgrade.

And I'm ready to upgrade, I see many new features in the latest code I
want to try.

> The key issue is the use of milters, but I do seem to recall there
> being some changes that affected all (since milters aren't an option
> for that old a version of postfix).... where I didn't check for the
> use of p records/milters before doing a spin through. Then again,
the
> spin should only adversely affect performance, not function.
> So then the question becomes: how large a volume do you have?

Normal volume total is around 200,000 email/day with recent spikes up
to 900,000 email/day as this is the busy time of year for us.
Figure about 91% of the total volume is rejected/spam.
I have Vispan stats if you want them.

That would be a nice feature request for MailScanner, something to
provide consistant volume benchmarks across all supported MailScanner
platforms.  
Something like:
MailScanner -stats
or a configuration setting to enable benchmarking with the
understanding that there would be system overhead associated with the
additional process.

> 
>> > Brad Beckenhauer wrote:
>> >> I'm running a Postfix system using the Hold method and recently
>> >> experienced a huge backlog due to "invalid queue files". 
Eventually
>> the
>> >> hold directory got enough invalid queue files that MailScanner
>> switch to
>> >> "accelerated" mode of processing messages.
>> >>
>> >> sample from the maillog:
>> >> Aug 17 08:37:48 mx2 MailScanner[26605]: New Batch: Found invalid
>> queue
>> >> files: B1FB93EE101
>> >> Postcat of this file is below.
>> >>
>> >> Finding what's causing these invalid queue files and eliminating
>> them
>> >> is the first issue.
>> >>
>> >> Would a feature in MailScanner that takes action when invalid
queue
>> >> files are found be useful to others?
>> >> Such a feature could:
>> >> Detect invalid queue files (which it already does) and
>> >> a) Move the invalid queue file somewhere safe and notify the
admin
>> >> b) postsuper -d queue_file to get rid of it.
>> >>
>> >> Or how about a courtesy email to the email admin when
MailScanner
>> >> switches to "accelerated mode".
>> >>
>> >> postcat B1FB93EE101
>> >>
>> >> *** ENVELOPE RECORDS B1FB93EE101 ***
>> >> message_size:             489             336               1
>>
>> >>     0
>> >> message_arrival_time: Thu Aug 16 09:22:55 2007
>> >> sender: Lucien.Hatch8400 at hot.ee 
>> >> named_attribute: client_name=62.43.179.119.dyn.user.ono.com
>> >> named_attribute: client_address=62.43.179.119
>> >> named_attribute:
>> >> message_origin=62.43.179.119.dyn.user.ono.com[62.43.179.119]
>> >> named_attribute: helo_name=62.43.179.119.dyn.user.ono.com
>> >> named_attribute: protocol_name=SMTP
>> >> original_recipient: tnolte at stfm.org 
>> >> recipient: tnolte at stfm.org 
>> >> *** MESSAGE CONTENTS B1FB93EE101 ***
>> >> Received: from 62.43.179.119.dyn.user.ono.com
>> >> (62.43.179.119.dyn.user.ono.com [62.43.179.119])
>> >>         by mx1.aafp.org (Postfix) with SMTP id B1FB93EE101
>> >>         for <tnolte at stfm.org>; Thu, 16 Aug 2007 09:22:55 -0500
>> (CDT)
>> >> Received: from martinique.hotbox.com (unknown [111.13.212.156])
>> >>            by galleryplanet.com with SMTP id 5[10
>> >> Message-Id: <20070816142255.B1FB93EE101 at mx1.aafp.org>
>> >> Date: Thu, 16 Aug 2007 09:22:55 -0500 (CDT)
>> >> From: Lucien.Hatch8400 at hot.ee 
>> >> To: undisclosed-recipients:;
>> >> *** HEADER EXTRACTED B1FB93EE101 ***
>> >> *** MESSAGE FILE END B1FB93EE101 ***
>> >>
>> >>
>> >>
>> >>
>> >
>> > Jules
> 
> Cheers