Invalid postfix queue files (UNCLASSIFIED)

[Glenn Steen]

On 12/06/07, Kash, Howard (Civ, ARL/CISD) <hmkash at arl.army.mil> wrote:
> Classification:  UNCLASSIFIED
> Caveats: NONE
>
>
> On 09/06/07, Glenn Steen <glenn.steen at gmail.com> wrote:
> > On 08/06/07, Kash, Howard (Civ, ARL/CISD) <hmkash at arl.army.mil> wrote:
> > >
> > > After upgrading to MS 4.60.8, MailScanner has started reporting "New
> Batch:
> > > Found invalid queue files: <list of queue names>".  Each of the
> queue files
> > > appears to have a truncated message contents section and 90% of them
> end
> > > with "To: undisclosed-recipients:;".  There's a total of about 30 of
> them
> > > since I upgraded on June 4.  Anyone else seeing this?  I also
> upgraded
> > > postfix from 2.3.9 to 2.3.11 at the same time, but figured I'd start
> here
> > > first since the postfix group will blame MailScanner anyway...
> > >
> > >
> > > Thanks,
> > > Howard
> > Hi Howard,
> >
> > Do you employ any milter(s)?
> > If not, the only changes that could possibly affect PF in that version
> > of MS would be the spin-through of the body.... and any error in that
> > would be.... more fatal:-)
> > As is, I drop any mails hitting the end of the queue file from the
> > batch, in that code segment. That way, it'll be picked up by the next
> > one running through hold, hopefully more completely written than
> > before. The only other way to break out of the loop is by finding the
> > X record after the body... So any error should have rather disastrous
> > (in a more prominent way:-) effects.
> > None of the p record changes actually change how things are stored
> > into the message object... apart from handling the p records (jumping
> > to where they point) and w records (just ignoring them, they signify
> > deleted records) I just let Jules code copy everything as before.
> >
> > Hm, one would want to get a look at both the queue files _before_ and
> > _after_ MS. Probably a tad too much to wish for:-).
> > Could you send us one of them? Or at least the postcat'd result?
> >
> > Cheers
>
> No, don't use milters.  No errors in logs regarding p records.  The only
> error is:
>
> Jun 11 09:40:41 mailserver MailScanner[6069]: New Batch: Found invalid
> queue files:0B06F50582D E1AB5505840 42F50505842 248EB505887 2EAC7505897
> 1575450589E 818BB5058AC C8D135058BB 0FC8C5058E1 09C92505914
>
>
> Here's a postcat of one of the invalid queue files:
>
> *** ENVELOPE RECORDS 09C92505914 ***
> message_size:             495             623               1
>      0
> message_arrival_time: Mon Jun 11 06:21:56 2007
> create_time: Mon Jun 11 06:22:02 2007
> named_attribute: rewrite_context=remote
> sender: wemihopzxds at yahoo.com
> named_attribute: log_client_name=cm34.omega17.maxonline.com.sg
> named_attribute: log_client_address=218.186.17.34
> named_attribute:
> log_message_origin=cm34.omega17.maxonline.com.sg[218.186.17.34]
> named_attribute: log_helo_name=cm34.omega17.maxonline.com.sg
> named_attribute: log_protocol_name=SMTP
> named_attribute: client_name=cm34.omega17.maxonline.com.sg
> named_attribute: reverse_client_name=cm34.omega17.maxonline.com.sg
> named_attribute: client_address=218.186.17.34
> named_attribute: helo_name=cm34.omega17.maxonline.com.sg
> named_attribute: client_address_type=2
> named_attribute: dsn_orig_rcpt=rfc822;user at arl.mil
> original_recipient: user at arl.mil
> recipient: user at arl.mil
> *** MESSAGE CONTENTS 09C92505914 ***
> Received: from cm34.omega17.maxonline.com.sg
> (cm34.omega17.maxonline.com.sg [218.186.17.34])
>          by mailserver.arl.army.mil (Postfix) with SMTP id 09C92505914
>          for <user at arl.mil>; Mon, 11 Jun 2007 06:21:56 -0400 (EDT)
> Received: from crm9.yahoo.com (localhost.localdomain [127.0.0.1])
>          by crm5.yahoo.com (Postfix) with ESMTP id 1[6
> Message-Id: <20070611102202.09C92505914 at mailserver.arl.army.mil>
> Date: Mon, 11 Jun 2007 06:21:56 -0400 (EDT)
> From: wemihopzxds at yahoo.com
> To: undisclosed-recipients:;
> *** HEADER EXTRACTED 09C92505914 ***
> *** MESSAGE FILE END 09C92505914 ***
>
>
>
> All logs pertaining to this message:
>
> Jun 11 06:22:02 mailserver postfix/smtpd[17535]: 09C92505914:
> client=cm34.omega17.maxonline.com.sg[218.186.17.34]
> Jun 11 06:22:03 mailserver postfix/cleanup[18172]: 09C92505914: hold:
> header Received: from cm34.omega17.maxonline.com.sg
> (cm34.omega17.maxonline.com.sg [218.186.17.34])??by
> mailserver.arl.army.mil (Postfix) with SMTP id 09C92505914??for
> <user at arl.mil>; Mon, 11 Jun 2007 06:21:56 - from
> cm34.omega17.maxonline.com.sg[218.186.17.34];
> from=<wemihopzxds at yahoo.com> to=<user at arl.mil> proto=SMTP
> helo=<cm34.omega17.maxonline.com.sg>
> Jun 11 06:22:03 mailserver postfix/cleanup[18172]: 09C92505914:
> message-id=<20070611102202.09C92505914 at mailserver.arl.army.mil>

Hi Howard, and thanks for getting back.

I'll be looking hard at this, time permitting, over the next few days.
Ie also gotten a good report (very detailed) from Nerijus Baliunas
along the same lines, but... I can't really see how any MailScanner
code could be responsible for damaging anything in hold (might be me
needing new glasses:-) simply due to the fact that we only _read_ from
hold (well, eventually we unlink that message, after successful
requeue, of course).

But I will make an effort to ascertain whether this really is due to
the code added to 4.60.8 or not. Who knows, perhaps I'm doing
something stupid when droping the message from the batch (rightly)...
But I rather doubt it:-)

You can help further by giving more complete log snippets/examples
(from connect until it pops up as invalid).

Cheers
-- 
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se