Mailscanner generated duplicate message

[Julian Field]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Glenn Steen wrote:
> On 08/02/2008, Julian Field <MailScanner at ecs.soton.ac.uk> wrote:
>   
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>>
>> Glenn Steen wrote:
>>     
>>> On 08/02/2008, Glenn Steen <glenn.steen at gmail.com> wrote:
>>>
>>>       
>>>> On 08/02/2008, Glenn Steen <glenn.steen at gmail.com> wrote:
>>>>
>>>>         
>>>>> On 07/02/2008, Cedric Devillers <cde at alunys.com> wrote:
>>>>>
>>>>>           
>>>>>> Hello,
>>>>>>
>>>>>> I'm trying to revive this thread from the last month because we are
>>>>>> observing the exact same behavior on one of our servers.
>>>>>>
>>>>>>             
>>>>> Thanks for doing that, and for providing some more info.
>>>>>
>>>>>
>>>>>           
>>>>>> So to remember the facts :
>>>>>>
>>>>>> - We are using mailscanner with postfix, and duplicated messages are
>>>>>> generated by mailscanner.
>>>>>>
>>>>>> - This system is the only one where we are observing this behavior. It
>>>>>> have a little particularity : it mainly act as a mail relay, but
>>>>>> sometimes many mails are generated by the server itself (a script) and
>>>>>> injected in postfix queues via sendmail command. We can always reproduce
>>>>>> some duplicated messages with this script.
>>>>>>
>>>>>> - MailScanner is configured (by ruleset) to bypass scanning for thoses
>>>>>> messages, but they are still entering the mailscanner logic (postix ->
>>>>>> hold queue -> mailscanner (no scan) -> active queue).
>>>>>>
>>>>>>             
>>>>> What does the ruleset look like? I'm sure it doesn't matter, but ...
>>>>> just out of curiosity:-)...
>>>>>
>>>>>
>>>>>           
>>>>>> - Mailwatch is running on this server, and for each duplicates we see
>>>>>> entries with null size body (2, 3, 4, sometimes 5) then at last a final
>>>>>> entry with the full body. Note that the recipient see the full body on
>>>>>> every duplicate.
>>>>>>
>>>>>> It looks like a locking problem, because all duplicates are with the
>>>>>> same postfix queue ID and different entropy part (ID.xxxx, ID.yyyy,
>>>>>> ID.zzzz, etc). Can it be possible that a mailscanner child "fail" to
>>>>>> lock some queue file when message is marked not to be scanned by
>>>>>> mailscanner ?
>>>>>>
>>>>>>             
>>>>> Yes, this seems plausible... Could you provide some log examples? Just
>>>>> to see that it really is separate children reading the same queue
>>>>> file...
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I will not be very helpfull to debug perl code, but i can provide any
>>>>>> needed logs to help finding the origin of the problem.
>>>>>>
>>>>>>             
>>>>> I'll see what I can do, but... I think this isn't "my" code snippets,
>>>>> but a thing that might have been present for a while... And I have a
>>>>> serious lack of time to spend on this ATM (worse than last time,
>>>>> before Xmas)... So no promises:-).
>>>>>
>>>>>
>>>>>           
>>>>>> This is really a serious problem in this particular installation. But i
>>>>>> must say that we have dozens of other servers that are running
>>>>>> mailscanner/postfix, and we are very happy about thems :)
>>>>>>
>>>>>>             
>>>>> Does it help if you DO scan with MS, but skip things at the next
>>>>> level, for example:
>>>>> Scan Messages = yes
>>>>> Use SpamAssassin = no
>>>>> Dangerous Content Scanning = no
>>>>> ... and possibly a few more (do them with a ruleset, of course:-)?
>>>>>
>>>>>
>>>>>           
>>>> BTW, do you have any milters enabled in Postfix? What version of Postfix?
>>>>
>>>> Cheers
>>>>
>>>>         
>>> I think we need Jules on this one, not only feeble lil' me:-).
>>> AFAICS, the locking/unlocking is handled _exactly_ the same regardless
>>> of the scanmail setting... But then, this is a rather complex bit of
>>> code, where the "execution path" isn't always as straightforward as it
>>> seems... Jules, could you spare a moment or two? Just to look at what
>>> could possibly be wrong with the message->scanmail = 0 scenario?
>>>
>>>
>>>       
>> Can you *briefly* explain what the problem is, what the symptoms are and
>> where you think the problem might lie? This is a very long thread.... :-)
>>
>> Jules
>>
>>     
> In short:
> When using Postfix and setting Scan Messages = no (with a rulset, for
> some....), duplicates are "generated" by several MailScanner children
> picking up and delivering the same message.
Is the whole message being delivered multiple times, or are the 
duplicates truncated at all?

P.S. Sorry for top-posting on this thread a few minutes ago :-(

Jules

- -- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store

MailScanner customisation, or any advanced system administration help?
Contact me at Jules at Jules.FM

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
PGP public key: http://www.jules.fm/julesfm.asc


-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.8.0 (Build 2158)
Comment: Use Thunderbird Enigmail to verify this message
Charset: ISO-8859-1

wj8DBQFHs3nZEfZZRxQVtlQRAgtmAKDyt+y+fafkRvZQURVQajXKUBPCEACglEOV
N3ZN77/lKwzizAeWVhpbGkQ=
=3CSz
-----END PGP SIGNATURE-----

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.