"Banned Content" question - still getting "Content Check"
messages
Julian Field
MailScanner at ecs.soton.ac.uk
Thu Jan 13 14:54:03 GMT 2005
[ The following text is in the "ISO-8859-1" character set. ]
[ Your display is set for the "US-ASCII" character set. ]
[ Some characters may be displayed incorrectly. ]
A minor bugfix to Message.pm which probably won't make any difference,
but is still worth a try.
@@ -3687,6 +3687,13 @@
#print STDERR "Tags to convert are " . $this->{tagstoconvert} . "\n";
# Set the disarm booleans for this message
+ $DisarmFormTag = 0;
+ $DisarmScriptTag = 0;
+ $DisarmCodebaseTag = 0;
+ $DisarmCodebaseTag = 0;
+ $DisarmIframeTag = 0;
+ $DisarmWebBug = 0;
+ $DisarmPhishing = 0;
$DisarmFormTag = 1 if $this->{tagstoconvert} =~ /form/i;
$DisarmScriptTag = 1 if $this->{tagstoconvert} =~ /script/i;
$DisarmCodebaseTag = 1 if $this->{tagstoconvert} =~ /codebase/i;
I'm afraid I can't think of much else. All the other variables are
initialised, and these few just maintain state for the particular
message, they don't hold any message content.
Quentin Campbell wrote:
>Julian
>
>I have applied your SMDiskStore.pm changes as well as set "Allow WebBugs
>= yes".
>
>However we are still getting corrupted HTML in multipart/alternative
>message. These are all accompanied with "Content Checks: Detected and
>will disarm HTML message in ..." messages in the Sendmail log. This is
>happening on both 4.35.10 and 4.37.7 systems.
>
>Unfortunately the problem is INTERMITTENT and it cannot be repeated by
>sending the same message again to the same recipient. :-(
>
>The "Content Checks:" message is misleading since I am no longer using
>the "disarm" content action anywhere, either in MailScanner.conf or in
>the MailScanner rules files. Where I specify an action to deal with HTML
>content it only uses "striphtml".
>
>Why then is MailScanner telling me it is "disarming" HTML when I have
>not asked it to?
>
>This is really getting frustrating and more users are complaining. :-(
>
>PS I note that I applied two patches from you late last year to the
>4.35.10 system;
> these were to SMDiskStore.pm and SweepContent.pm.
>
>Quentin
>---
>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
> University of Newcastle,
> Newcastle upon Tyne,
>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>------------------------------------------------------------------------
>"Any opinion expressed above is mine. The University can get its own."
>
>
>
>>-----Original Message-----
>>From: MailScanner mailing list
>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Quentin Campbell
>>Sent: 13 January 2005 11:33
>>To: MAILSCANNER at JISCMAIL.AC.UK
>>Subject: Re: "Banned Content" question - Lock/Unlock does not
>>fix problem
>>
>>Julian
>>
>>The mods to subs "Lock" and "Unlock" have not fixed the problem.
>>
>>It appears to be also present on a 4.37.7-1 system. As this
>>has the same
>>Lock/Unlock code that should not be a surprise.
>>
>>I will now allow web bugs to see if the corruption stops.
>>
>>Quentin
>>---
>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>> University of Newcastle,
>> Newcastle upon Tyne,
>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>---------------------------------------------------------------
>>---------
>>"Any opinion expressed above is mine. The University can get
>>its own."
>>
>>
>>
>>>-----Original Message-----
>>>From: MailScanner mailing list
>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Quentin Campbell
>>>Sent: 13 January 2005 08:50
>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>code problem
>>>
>>>Julian
>>>
>>>You had already given me a new SMDiskStore.pm module, dated
>>>
>>>
>>16 December
>>
>>
>>>to try. The locking code in this differs from the new code you want me
>>>to try as follows:
>>>
>>>< #JKF MailScanner::Lock::unlockclose($this->{indhandle});
>>>< close($this->{indhandle});
>>>---
>>>
>>>
>>>> # Now we lock the df file as well, we must unlock it too.
>>>> MailScanner::Lock::unlockclose($this->{indhandle});
>>>> #close($this->{indhandle});
>>>>
>>>>
>>>I have made the change as above and will let you know what happens. I
>>>also note that the new code is in the MailScanner-4.37.7-1
>>>SMDiskStore.pm which I was planning to move to anyway.
>>>
>>>I have not touched the "Allow WebBugs = disarm" setting which I assume
>>>is an essential part of the test of the changes to SMDiskStore.pm.
>>>
>>>Quentin
>>>---
>>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>>> University of Newcastle,
>>> Newcastle upon Tyne,
>>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>>---------------------------------------------------------------
>>>---------
>>>"Any opinion expressed above is mine. The University can get
>>>its own."
>>>
>>>
>>>
>>>>-----Original Message-----
>>>>From: MailScanner mailing list
>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>>>Sent: 12 January 2005 16:02
>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>>code problem
>>>>
>>>>In which case try editing SMDiskStore.pm and replace the sub Lock and
>>>>sub Unlock with this code:
>>>>
>>>># Open and lock the message
>>>>sub Lock {
>>>> my $this = shift;
>>>>
>>>> #print STDERR "About to lock " . $this->{hpath} . " and " .
>>>> # $this->{dpath} . "\n";
>>>> MailScanner::Lock::openlock($this->{inhhandle}, '+<' .
>>>>$this->{hpath},
>>>>'w', 'quiet')
>>>> or return undef;
>>>> #print STDERR "Got hlock\n";
>>>>
>>>> # If locking the dfile fails, then must close and unlock the
>>>>qffile too
>>>> # 14/12/2004 Try putting this back in for now.
>>>> unless (MailScanner::Lock::openlock($this->{indhandle},
>>>> '+<' . $this->{dpath}, 'w', 'quiet')) {
>>>> #JKF 14/12/2004 open($this->{indhandle}, '+<' .
>>>>$this->{dpath})) {
>>>> MailScanner::Lock::unlockclose($this->{inhhandle});
>>>> return undef;
>>>> }
>>>> #print STDERR "Got dlock\n";
>>>> return undef unless $this->{inhhandle} && $this->{indhandle};
>>>> return 1;
>>>>}
>>>>
>>>>
>>>># Close and unlock the message
>>>>sub Unlock {
>>>> my $this = shift;
>>>>
>>>> # Now we lock the df file as well, we must unlock it too.
>>>> MailScanner::Lock::unlockclose($this->{indhandle});
>>>> #close($this->{indhandle});
>>>> MailScanner::Lock::unlockclose($this->{inhhandle});
>>>>}
>>>>
>>>>
>>>>
>>>>Quentin Campbell wrote:
>>>>
>>>>
>>>>
>>>>>Julian
>>>>>
>>>>>The version of MailScanner on which I have seen the problem
>>>>>
>>>>>
>>>>is 4.35.10.
>>>>
>>>>
>>>>>Quentin
>>>>>---
>>>>>PHONE: +44 191 222 8209 Information Systems and Services (ISS),
>>>>> University of Newcastle,
>>>>> Newcastle upon Tyne,
>>>>>FAX: +44 191 222 8765 United Kingdom, NE1 7RU.
>>>>>--------------------------------------------------------------
>>>>>
>>>>>
>>>>----------
>>>>
>>>>
>>>>>"Any opinion expressed above is mine. The University can get
>>>>>
>>>>>
>>>its own."
>>>
>>>
>>>>>
>>>>>
>>>>>
>>>>>>-----Original Message-----
>>>>>>From: MailScanner mailing list
>>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Julian Field
>>>>>>Sent: 12 January 2005 15:30
>>>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>>>Subject: Re: "Banned Content" question - possibly a Web Bug
>>>>>>code problem
>>>>>>
>>>>>>What version of MailScanner are you using? I slightly improved the
>>>>>>locking code (took out an "improvement" I made a long time
>>>>>>
>>>>>>
>>>>ago which I
>>>>
>>>>
>>>>>>only made after lots of people requested it) in 4.37. It now
>>>>>>
>>>>>>
>>>>locks the
>>>>
>>>>
>>>>>>df as well as the qf, which slows down delivery slightly in some
>>>>>>situations, but appears to be more reliable than just
>>>>>>
>>>>>>
>>>locking the qf.
>>>
>>>
>>>>>>Quentin Campbell wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>-----Original Message-----
>>>>>>>>From: MailScanner mailing list
>>>>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf Of Mike
>>>>>>>>Sent: 12 January 2005 11:53
>>>>>>>>To: MAILSCANNER at JISCMAIL.AC.UK
>>>>>>>>Subject: Re: "Banned Content" question - a related problem
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>From: MailScanner mailing list
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>[mailto:MAILSCANNER at JISCMAIL.AC.UK] On
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>Behalf Of Quentin Campbell
>>>>>>>>>
>>>>>>>>>All the systems are now up2date as far as RH AS 3 patches are
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>concerned.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>All the systems use the Sendmail that comes with these
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>system; the last
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>time they were updated this was Sendmail 8.12.11. I use
>>>>>>>>>
>>>>>>>>>
>>>>the default
>>>>
>>>>
>>>>>>>>>locking in MailScanner.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>I also had this problem on sendmail 8.12.10. After changing
>>>>>>>>the locking to posix, the problem was gone. So, although the
>>>>>>>>docs state that the locking problem occurs only from 8.13 on,
>>>>>>>>it seems that also some 8.12 versions are affected. Please set
>>>>>>>>the locking mechanism to "posix" and see if it solves
>>>>>>>>
>>>>>>>>
>>>your problem.
>>>
>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>I will do this as a last resort. There are four reasons why
>>>>>>>
>>>>>>>
>>>>I want to
>>>>
>>>>
>>>>>>>investigate other things first. In particular I want to capture a
>>>>>>>message before then after it has gone through MailSanner and got
>>>>>>>corrupted:
>>>>>>>
>>>>>>>1. Locking works OK on RH AS 3 systems with an up-to-date kernel.
>>>>>>>2. The symptoms we are seeing do not appear to be
>>>>>>>
>>>>>>>
>>repeatable so far
>>
>>
>>>>>>>which makes conclusive testing difficult.
>>>>>>>3. I have looked for other evidence of locking problems but
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>cannot find
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>any. For example I can show that all messages tagged as spam by
>>>>>>>MailScanner have been tagged once only. If there is a
>>>>>>>
>>>>>>>
>>>>locking problem
>>>>
>>>>
>>>>>>>you will see the same message (ie. same Sendmail QID) being
>>>>>>>
>>>>>>>
>>>>tagged as
>>>>
>>>>
>>>>>>>spam more than once by two or more MS processes.
>>>>>>>4. The problem appears related to the Web Bug check. I will
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>switch that
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>off first. See below for more details of this.
>>>>>>>
>>>>>>>Having looked further at the problem it appears to be
>>>>>>>
>>>>>>>
>>>>related to MIME
>>>>
>>>>
>>>>>>>multipart/alternative messages having all or part of the HTML part
>>>>>>>corrupted. The text part is not being affected.
>>>>>>>
>>>>>>>In all of the cases the logs show that MailScanner has
>>>>>>>
>>>>>>>
>>>>"disarmed" the
>>>>
>>>>
>>>>>>>HTML content. Since I only "disarm" Web Bugs it appears
>>>>>>>
>>>>>>>
>>>>that there may
>>>>
>>>>
>>>>>>>be a bug in the Web Bugs code that causes an intermittent
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>problem. This
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>suspicion is reinforced by the observation that the problem
>>>>>>>
>>>>>>>
>>>>appears to
>>>>
>>>>
>>>>>>>have started when I enabled the Web Bug check late last
>>>>>>>
>>>>>>>
>>>year. I will
>>>
>>>
>>>>>>>first of all try "Allow WebBugs = yes" and see what happens.
>>>>>>>
>>>>>>>Quentin
>>>>>>>
>>>>>>>------------------------ MailScanner list ------------------------
>>>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>>>'leave mailscanner' in the body of the email.
>>>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>>>
>>>>>>>Support MailScanner development - buy the book off the website!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>--
>>>>>>Julian Field
>>>>>>www.MailScanner.info
>>>>>>Buy the MailScanner book at www.MailScanner.info/store
>>>>>>
>>>>>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>>>>>
>>>>>>------------------------ MailScanner list ------------------------
>>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>>'leave mailscanner' in the body of the email.
>>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>>
>>>>>>Support MailScanner development - buy the book off the website!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>------------------------ MailScanner list ------------------------
>>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>>'leave mailscanner' in the body of the email.
>>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>>
>>>>>Support MailScanner development - buy the book off the website!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>--
>>>>Julian Field
>>>>www.MailScanner.info
>>>>Buy the MailScanner book at www.MailScanner.info/store
>>>>
>>>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>>>
>>>>------------------------ MailScanner list ------------------------
>>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>>'leave mailscanner' in the body of the email.
>>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>>
>>>>Support MailScanner development - buy the book off the website!
>>>>
>>>>
>>>>
>>>>
>>>------------------------ MailScanner list ------------------------
>>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>>'leave mailscanner' in the body of the email.
>>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>>
>>>Support MailScanner development - buy the book off the website!
>>>
>>>
>>>
>>>
>>------------------------ MailScanner list ------------------------
>>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>>'leave mailscanner' in the body of the email.
>>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>>
>>Support MailScanner development - buy the book off the website!
>>
>>
>>
>>
>
>------------------------ MailScanner list ------------------------
>To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
>'leave mailscanner' in the body of the email.
>Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
>the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>
>Support MailScanner development - buy the book off the website!
>
>
>
--
Julian Field
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store
PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
Support MailScanner development - buy the book off the website!
More information about the MailScanner
mailing list