"Banned Content" question - Lock/Unlock does not fix problem
Quentin Campbell
Q.G.Campbell at NEWCASTLE.AC.UK
Thu Jan 13 11:32:59 GMT 2005
Julian
The mods to subs "Lock" and "Unlock" have not fixed the problem.
It appears to be also present on a 4.37.7-1 system. As this has the same
Lock/Unlock code that should not be a surprise.
I will now allow web bugs to see if the corruption stops.
Quentin
---
PHONE: +44 191 222 8209 Information Systems and Services (ISS),
University of Newcastle,
Newcastle upon Tyne,
FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
------------------------------------------------------------------------
"Any opinion expressed above is mine. The University can get its own."
>-----Original Message-----
>From: MailScanner mailing list
>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Quentin Campbell
>Sent: 13 January 2005 08:50
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: "Banned Content" question - possibly a Web Bug
>code problem
>
>Julian
>
>You had already given me a new SMDiskStore.pm module, dated 16 December
>to try. The locking code in this differs from the new code you want me
>to try as follows:
>
>< #JKF MailScanner::Lock::unlockclose($this->{indhandle});
>< close($this->{indhandle});
>---
>> # Now we lock the df file as well, we must unlock it too.
>> MailScanner::Lock::unlockclose($this->{indhandle});
>> #close($this->{indhandle});
>
>I have made the change as above and will let you know what happens. I
>also note that the new code is in the MailScanner-4.37.7-1
>SMDiskStore.pm which I was planning to move to anyway.
>
>I have not touched the "Allow WebBugs = disarm" setting which I assume
>is an essential part of the test of the changes to SMDiskStore.pm.
>
>Quentin
>---
>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
> University of Newcastle,
> Newcastle upon Tyne,
>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>---------------------------------------------------------------
>---------
>"Any opinion expressed above is mine. The University can get
>its own."
>
>>-----Original Message-----
>>From: MailScanner mailing list
>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>Sent: 12 January 2005 16:02
>>To: MAILSCANNER at JISCMAIL.AC.UK
>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>code problem
>>
>>In which case try editing SMDiskStore.pm and replace the sub Lock and
>>sub Unlock with this code:
>>
>># Open and lock the message
>>sub Lock {
>> my $this = shift;
>>
>> #print STDERR "About to lock " . $this->{hpath} . " and " .
>> # $this->{dpath} . "\n";
>> MailScanner::Lock::openlock($this->{inhhandle}, '+<' .
>>$this->{hpath},
>>'w', 'quiet')
>> or return undef;
>> #print STDERR "Got hlock\n";
>>
>> # If locking the dfile fails, then must close and unlock the
>>qffile too
>> # 14/12/2004 Try putting this back in for now.
>> unless (MailScanner::Lock::openlock($this->{indhandle},
>> '+<' . $this->{dpath}, 'w', 'quiet')) {
>> #JKF 14/12/2004 open($this->{indhandle}, '+<' .
>>$this->{dpath})) {
>> MailScanner::Lock::unlockclose($this->{inhhandle});
>> return undef;
>> }
>> #print STDERR "Got dlock\n";
>> return undef unless $this->{inhhandle} && $this->{indhandle};
>> return 1;
>>}
>>
>>
>># Close and unlock the message
>>sub Unlock {
>> my $this = shift;
>>
>> # Now we lock the df file as well, we must unlock it too.
>> MailScanner::Lock::unlockclose($this->{indhandle});
>> #close($this->{indhandle});
>> MailScanner::Lock::unlockclose($this->{inhhandle});
>>}
>>
>>
>>
>>Quentin Campbell wrote:
>>
>>>Julian
>>>
>>>The version of MailScanner on which I have seen the problem
>>is 4.35.10.
>>>
>>>Quentin
>>>---
>>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>>> University of Newcastle,
>>> Newcastle upon Tyne,
>>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>>--------------------------------------------------------------
>>----------
>>>"Any opinion expressed above is mine. The University can get
>its own."
>>>
>>>
>>>
>>>>-----Original Message-----
>>>>From: MailScanner mailing list
>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>>>Sent: 12 January 2005 15:30
>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>>code problem
>>>>
>>>>What version of MailScanner are you using? I slightly improved the
>>>>locking code (took out an "improvement" I made a long time
>>ago which I
>>>>only made after lots of people requested it) in 4.37. It now
>>locks the
>>>>df as well as the qf, which slows down delivery slightly in some
>>>>situations, but appears to be more reliable than just
>locking the qf.
>>>>
>>>>Quentin Campbell wrote:
>>>>
>>>>
>>>>
>>>>>>-----Original Message-----
>>>>>>From: MailScanner mailing list
>>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Mike
>>>>>>Sent: 12 January 2005 11:53
>>>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>>>Subject: Re: "Banned Content" question - a related problem
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>From: MailScanner mailing list
>>>>>>>
>>>>>>>
>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On
>>>>
>>>>
>>>>>>>Behalf Of Quentin Campbell
>>>>>>>
>>>>>>>All the systems are now up2date as far as RH AS 3 patches are
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>concerned.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>All the systems use the Sendmail that comes with these
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>system; the last
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>time they were updated this was Sendmail 8.12.11. I use
>>the default
>>>>>>>locking in MailScanner.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>I also had this problem on sendmail 8.12.10. After changing
>>>>>>the locking to posix, the problem was gone. So, although the
>>>>>>docs state that the locking problem occurs only from 8.13 on,
>>>>>>it seems that also some 8.12 versions are affected. Please set
>>>>>>the locking mechanism to "posix" and see if it solves
>your problem.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>I will do this as a last resort. There are four reasons why
>>I want to
>>>>>investigate other things first. In particular I want to capture a
>>>>>message before then after it has gone through MailSanner and got
>>>>>corrupted:
>>>>>
>>>>>1. Locking works OK on RH AS 3 systems with an up-to-date kernel.
>>>>>2. The symptoms we are seeing do not appear to be repeatable so far
>>>>>which makes conclusive testing difficult.
>>>>>3. I have looked for other evidence of locking problems but
>>>>>
>>>>>
>>>>cannot find
>>>>
>>>>
>>>>>any. For example I can show that all messages tagged as spam by
>>>>>MailScanner have been tagged once only. If there is a
>>locking problem
>>>>>you will see the same message (ie. same Sendmail QID) being
>>tagged as
>>>>>spam more than once by two or more MS processes.
>>>>>4. The problem appears related to the Web Bug check. I will
>>>>>
>>>>>
>>>>switch that
>>>>
>>>>
>>>>>off first. See below for more details of this.
>>>>>
>>>>>Having looked further at the problem it appears to be
>>related to MIME
>>>>>multipart/alternative messages having all or part of the HTML part
>>>>>corrupted. The text part is not being affected.
>>>>>
>>>>>In all of the cases the logs show that MailScanner has
>>"disarmed" the
>>>>>HTML content. Since I only "disarm" Web Bugs it appears
>>that there may
>>>>>be a bug in the Web Bugs code that causes an intermittent
>>>>>
>>>>>
>>>>problem. This
>>>>
>>>>
>>>>>suspicion is reinforced by the observation that the problem
>>appears to
>>>>>have started when I enabled the Web Bug check late last
>year. I will
>>>>>first of all try "Allow WebBugs = yes" and see what happens.
>>>>>
>>>>>Quentin
>>>>>
>>>>>------------------------ MailScanner list ------------------------
>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>'leave mailscanner' in the body of the email.
>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>
>>>>>Support MailScanner development - buy the book off the website!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>--
>>>>Julian Field
>>>>www.MailScanner.info
>>>>Buy the MailScanner book at www.MailScanner.info/store
>>>>
>>>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>>>
>>>>------------------------ MailScanner list ------------------------
>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>'leave mailscanner' in the body of the email.
>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>
>>>>Support MailScanner development - buy the book off the website!
>>>>
>>>>
>>>>
>>>>
>>>
>>>------------------------ MailScanner list ------------------------
>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>'leave mailscanner' in the body of the email.
>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>
>>>Support MailScanner development - buy the book off the website!
>>>
>>>
>>>
>>
>>--
>>Julian Field
>>www.MailScanner.info
>>Buy the MailScanner book at www.MailScanner.info/store
>>
>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>
>>------------------------ MailScanner list ------------------------
>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>'leave mailscanner' in the body of the email.
>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>
>>Support MailScanner development - buy the book off the website!
>>
>>
>
>------------------------ MailScanner list ------------------------
>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>'leave mailscanner' in the body of the email.
>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>
>Support MailScanner development - buy the book off the website!
>
>
------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
Support MailScanner development - buy the book off the website!
More information about the MailScanner
mailing list