"Banned Content" question - still getting "Content Check"
messages
Quentin Campbell
Q.G.Campbell at NEWCASTLE.AC.UK
Thu Jan 13 14:26:38 GMT 2005
Julian
I have applied your SMDiskStore.pm changes as well as set "Allow WebBugs
= yes".
However we are still getting corrupted HTML in multipart/alternative
message. These are all accompanied with "Content Checks: Detected and
will disarm HTML message in ..." messages in the Sendmail log. This is
happening on both 4.35.10 and 4.37.7 systems.
Unfortunately the problem is INTERMITTENT and it cannot be repeated by
sending the same message again to the same recipient. :-(
The "Content Checks:" message is misleading since I am no longer using
the "disarm" content action anywhere, either in MailScanner.conf or in
the MailScanner rules files. Where I specify an action to deal with HTML
content it only uses "striphtml".
Why then is MailScanner telling me it is "disarming" HTML when I have
not asked it to?
This is really getting frustrating and more users are complaining. :-(
PS I note that I applied two patches from you late last year to the
4.35.10 system;
these were to SMDiskStore.pm and SweepContent.pm.
Quentin
---
PHONE: +44 191 222 8209 Information Systems and Services (ISS),
University of Newcastle,
Newcastle upon Tyne,
FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
------------------------------------------------------------------------
"Any opinion expressed above is mine. The University can get its own."
>-----Original Message-----
>From: MailScanner mailing list
>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Quentin Campbell
>Sent: 13 January 2005 11:33
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: "Banned Content" question - Lock/Unlock does not
>fix problem
>
>Julian
>
>The mods to subs "Lock" and "Unlock" have not fixed the problem.
>
>It appears to be also present on a 4.37.7-1 system. As this
>has the same
>Lock/Unlock code that should not be a surprise.
>
>I will now allow web bugs to see if the corruption stops.
>
>Quentin
>---
>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
> University of Newcastle,
> Newcastle upon Tyne,
>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>---------------------------------------------------------------
>---------
>"Any opinion expressed above is mine. The University can get
>its own."
>
>>-----Original Message-----
>>From: MailScanner mailing list
>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Quentin Campbell
>>Sent: 13 January 2005 08:50
>>To: MAILSCANNER at JISCMAIL.AC.UK
>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>code problem
>>
>>Julian
>>
>>You had already given me a new SMDiskStore.pm module, dated
>16 December
>>to try. The locking code in this differs from the new code you want me
>>to try as follows:
>>
>>< #JKF MailScanner::Lock::unlockclose($this->{indhandle});
>>< close($this->{indhandle});
>>---
>>> # Now we lock the df file as well, we must unlock it too.
>>> MailScanner::Lock::unlockclose($this->{indhandle});
>>> #close($this->{indhandle});
>>
>>I have made the change as above and will let you know what happens. I
>>also note that the new code is in the MailScanner-4.37.7-1
>>SMDiskStore.pm which I was planning to move to anyway.
>>
>>I have not touched the "Allow WebBugs = disarm" setting which I assume
>>is an essential part of the test of the changes to SMDiskStore.pm.
>>
>>Quentin
>>---
>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>> University of Newcastle,
>> Newcastle upon Tyne,
>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>---------------------------------------------------------------
>>---------
>>"Any opinion expressed above is mine. The University can get
>>its own."
>>
>>>-----Original Message-----
>>>From: MailScanner mailing list
>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>>Sent: 12 January 2005 16:02
>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>code problem
>>>
>>>In which case try editing SMDiskStore.pm and replace the sub Lock and
>>>sub Unlock with this code:
>>>
>>># Open and lock the message
>>>sub Lock {
>>> my $this = shift;
>>>
>>> #print STDERR "About to lock " . $this->{hpath} . " and " .
>>> # $this->{dpath} . "\n";
>>> MailScanner::Lock::openlock($this->{inhhandle}, '+<' .
>>>$this->{hpath},
>>>'w', 'quiet')
>>> or return undef;
>>> #print STDERR "Got hlock\n";
>>>
>>> # If locking the dfile fails, then must close and unlock the
>>>qffile too
>>> # 14/12/2004 Try putting this back in for now.
>>> unless (MailScanner::Lock::openlock($this->{indhandle},
>>> '+<' . $this->{dpath}, 'w', 'quiet')) {
>>> #JKF 14/12/2004 open($this->{indhandle}, '+<' .
>>>$this->{dpath})) {
>>> MailScanner::Lock::unlockclose($this->{inhhandle});
>>> return undef;
>>> }
>>> #print STDERR "Got dlock\n";
>>> return undef unless $this->{inhhandle} && $this->{indhandle};
>>> return 1;
>>>}
>>>
>>>
>>># Close and unlock the message
>>>sub Unlock {
>>> my $this = shift;
>>>
>>> # Now we lock the df file as well, we must unlock it too.
>>> MailScanner::Lock::unlockclose($this->{indhandle});
>>> #close($this->{indhandle});
>>> MailScanner::Lock::unlockclose($this->{inhhandle});
>>>}
>>>
>>>
>>>
>>>Quentin Campbell wrote:
>>>
>>>>Julian
>>>>
>>>>The version of MailScanner on which I have seen the problem
>>>is 4.35.10.
>>>>
>>>>Quentin
>>>>---
>>>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>>>> University of Newcastle,
>>>> Newcastle upon Tyne,
>>>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>>>--------------------------------------------------------------
>>>----------
>>>>"Any opinion expressed above is mine. The University can get
>>its own."
>>>>
>>>>
>>>>
>>>>>-----Original Message-----
>>>>>From: MailScanner mailing list
>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>>>>Sent: 12 January 2005 15:30
>>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>>>code problem
>>>>>
>>>>>What version of MailScanner are you using? I slightly improved the
>>>>>locking code (took out an "improvement" I made a long time
>>>ago which I
>>>>>only made after lots of people requested it) in 4.37. It now
>>>locks the
>>>>>df as well as the qf, which slows down delivery slightly in some
>>>>>situations, but appears to be more reliable than just
>>locking the qf.
>>>>>
>>>>>Quentin Campbell wrote:
>>>>>
>>>>>
>>>>>
>>>>>>>-----Original Message-----
>>>>>>>From: MailScanner mailing list
>>>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Mike
>>>>>>>Sent: 12 January 2005 11:53
>>>>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>>>>Subject: Re: "Banned Content" question - a related problem
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>From: MailScanner mailing list
>>>>>>>>
>>>>>>>>
>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On
>>>>>
>>>>>
>>>>>>>>Behalf Of Quentin Campbell
>>>>>>>>
>>>>>>>>All the systems are now up2date as far as RH AS 3 patches are
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>concerned.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>All the systems use the Sendmail that comes with these
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>system; the last
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>time they were updated this was Sendmail 8.12.11. I use
>>>the default
>>>>>>>>locking in MailScanner.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>I also had this problem on sendmail 8.12.10. After changing
>>>>>>>the locking to posix, the problem was gone. So, although the
>>>>>>>docs state that the locking problem occurs only from 8.13 on,
>>>>>>>it seems that also some 8.12 versions are affected. Please set
>>>>>>>the locking mechanism to "posix" and see if it solves
>>your problem.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>I will do this as a last resort. There are four reasons why
>>>I want to
>>>>>>investigate other things first. In particular I want to capture a
>>>>>>message before then after it has gone through MailSanner and got
>>>>>>corrupted:
>>>>>>
>>>>>>1. Locking works OK on RH AS 3 systems with an up-to-date kernel.
>>>>>>2. The symptoms we are seeing do not appear to be
>repeatable so far
>>>>>>which makes conclusive testing difficult.
>>>>>>3. I have looked for other evidence of locking problems but
>>>>>>
>>>>>>
>>>>>cannot find
>>>>>
>>>>>
>>>>>>any. For example I can show that all messages tagged as spam by
>>>>>>MailScanner have been tagged once only. If there is a
>>>locking problem
>>>>>>you will see the same message (ie. same Sendmail QID) being
>>>tagged as
>>>>>>spam more than once by two or more MS processes.
>>>>>>4. The problem appears related to the Web Bug check. I will
>>>>>>
>>>>>>
>>>>>switch that
>>>>>
>>>>>
>>>>>>off first. See below for more details of this.
>>>>>>
>>>>>>Having looked further at the problem it appears to be
>>>related to MIME
>>>>>>multipart/alternative messages having all or part of the HTML part
>>>>>>corrupted. The text part is not being affected.
>>>>>>
>>>>>>In all of the cases the logs show that MailScanner has
>>>"disarmed" the
>>>>>>HTML content. Since I only "disarm" Web Bugs it appears
>>>that there may
>>>>>>be a bug in the Web Bugs code that causes an intermittent
>>>>>>
>>>>>>
>>>>>problem. This
>>>>>
>>>>>
>>>>>>suspicion is reinforced by the observation that the problem
>>>appears to
>>>>>>have started when I enabled the Web Bug check late last
>>year. I will
>>>>>>first of all try "Allow WebBugs = yes" and see what happens.
>>>>>>
>>>>>>Quentin
>>>>>>
>>>>>>------------------------ MailScanner list ------------------------
>>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>>'leave mailscanner' in the body of the email.
>>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>>
>>>>>>Support MailScanner development - buy the book off the website!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>--
>>>>>Julian Field
>>>>>www.MailScanner.info
>>>>>Buy the MailScanner book at www.MailScanner.info/store
>>>>>
>>>>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>>>>
>>>>>------------------------ MailScanner list ------------------------
>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>'leave mailscanner' in the body of the email.
>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>
>>>>>Support MailScanner development - buy the book off the website!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>------------------------ MailScanner list ------------------------
>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>'leave mailscanner' in the body of the email.
>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>
>>>>Support MailScanner development - buy the book off the website!
>>>>
>>>>
>>>>
>>>
>>>--
>>>Julian Field
>>>www.MailScanner.info
>>>Buy the MailScanner book at www.MailScanner.info/store
>>>
>>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>>
>>>------------------------ MailScanner list ------------------------
>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>'leave mailscanner' in the body of the email.
>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>
>>>Support MailScanner development - buy the book off the website!
>>>
>>>
>>
>>------------------------ MailScanner list ------------------------
>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>'leave mailscanner' in the body of the email.
>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>
>>Support MailScanner development - buy the book off the website!
>>
>>
>
>------------------------ MailScanner list ------------------------
>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>'leave mailscanner' in the body of the email.
>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>
>Support MailScanner development - buy the book off the website!
>
>
------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
Support MailScanner development - buy the book off the website!
More information about the MailScanner
mailing list