Current Cambridge patchset [LONG]

Tony Finch dot at DOTAT.AT
Fri Jul 25 14:53:06 IST 2003


I've made a number of modifications to MailScanner in order to suit
it better to our requirements at Cambridge. I think most of them are
generally useful, so I'm posting them here for your interest (since
Julian is still on holiday). The changes are as follows (by file)...

EXAMPLES, README, Config.pm:

        I've added a new kind of rule: a "soft" rule, which gives
        you a bit more flexibility in the behaviour you can get
        using rulesets. We want all incoming email to be scanned,
        except for certain domains which have opted out of our
        central scanning service because they want to run their
        own. This is fine, but what do you do if a message is sent
        to two addresses, one opted out, one not? We've decided to
        err on the side of safety, and scan. However rulesets are
        first-match-wins, so have to be ordered from the specific
        to the general, which means that it would err on the side
        of the opt-out. By making the opted-out rule soft, other
        addresses on the message can be tested against later rules
        to give us the behaviour we want.

        I've added a special-case match for bounces (messages with
        an empty return path), equivalent to /^$/ but more readable.
        Default rules are handled slightly differently: as before
        they match bounces, but * and *@* only match addresses which
        contain an @ which does not include bounces. Default rules
        must now be last in the ruleset, or be soft.

EximDiskStore.pm, PFDiskStore.pm, SMDiskStore.pm, ZMSiskStore.pm,
SA.pm, Message.pm:

        Instead of not scanning large messages, SpamAssassin scans
        only the first part of messages up to the Max SpamAssassin Size
        configuration.

        SA.pm now tells Message.pm if it has not scanned the message
        for some reason, so that Mesasge.pm can put the Unscanned Header
        Value in the Spam Header instead of leaving it blank if you are
        Always Including the SpamAssassin Report.

        I've also changed the SpamAssassin report so that the required=N
        clause is not added if N is zero.  This is because we don't
        filter spam centrally, and MailScanner's threshold in the Spam
        Header was confusing users because it had nothing to do with the
        threshold they had configured on their own filters.  I've added
        a Log Non Spam option so that the SpamAssassin results of every
        message can be logged.

        I've also changed the From: header of messages carrying
        disinfected attachments to be the same as the from address of
        the original message, to encourage users to follow our policy
        of chasing up virus problems with the infected person rather
        than with us, and for better bounce and autoresponse behaviour.

ConfigDefs.pm MessageBatch.pm Exim.pm SweepContent.pm SweepOther.pm:

        I've changed MessageBatch.pm to log the number of High Scoring
        messages as well as regular spam.  This is for stats accumulation,
        especially if the Required SpamAssassin Score is zero.

        MessageBatch.pm also has a hand in ensuring that "Not Scanned"
        is put in the Spam Header if the message isn't scanned.

        Log HTML Form Tags and Log Object Codebase Tags options, to go
        with the existing Log IFrame Tags option. I've changed these
        options to work even when the corresponding tags are permitted,
        so that you can gauge how much pain would be caused by disallowing
        them.

        There's now a Log Message IDs option to make it easier to track
        what MailScanner has done to each message it handles. I've also
        changed a few existing log messages to include the message-ID.
        I think the coverage of this option is still a bit patchy; in
        particular the other MTA-specific modules need support.

Tony.
--
f.a.n.finch  <dot at dotat.at>  http://dotat.at/
SHANNON: WEST 5 OR 6 DECREASING 4. RAIN THEN SHOWERS. MODERATE OR POOR
BECOMING GOOD.



--- etc/rules/EXAMPLES  25 Mar 2003 18:32:39 -0000      1.1.1.3
+++ etc/rules/EXAMPLES  24 Jul 2003 18:13:02 -0000      1.2
@@ -55,5 +55,13 @@
    To:         @def.com                postmaster at me.com bill at def.com
    FromOrTo:   default                 postmaster at me.com

+6. Virus scan all but a few domains; if a message is to more than one
+   domain, err on the side of scanning
+
+   Set "Virus Scanning = /opt/MailScanner/etc/rules/virus.scanning.rules".
+   SoftTo:     opt-out-1.com   no
+   SoftTo:     opt-out-2.com   no
+   To:         default         yes
+
 I will add more ideas as I think of them.
 All suggestions for clever tricks are most welcome!

--- etc/rules/README    3 Mar 2003 04:34:23 -0000       1.1.1.3
+++ etc/rules/README    24 Jul 2003 18:13:02 -0000      1.2
@@ -31,6 +31,13 @@
    This specifies the whether the rule should be matched against
    the sender's address (or IP address), or the recipient's address.

+   The direction can also include the word "soft". Usually the value
+   of a ruleset is the value of the first rule that matches. If the
+   rule is soft, then MailScanner continues to look for a rule that
+   matches one of the remaining addresses (excluding the one that
+   matched the soft rule). If only soft rules match, then the value
+   of the ruleset is the value of the last soft rule that matched.
+
 2. The pattern describes what messages should match this rule.
    Some examples are:
        user at sub.domain.com     # Individual address
@@ -45,8 +52,9 @@
                                  # Perl regular expression
        /^192\.168\.1[4567]\./  # Any SMTP client IP address in the networks
                                # 192.168.14 - 192.168.17
-       *@*                     # Default value
        default                 # Default value
+       bounce                  # bounce messages
+       *@*                     # non-bounce messages
    You should be able to do just about anything with that.

 3. The result value is what you could have put in the entry in the main

--- lib/perl5/MailScanner/Config.pm     4 Jul 2003 18:08:27 -0000       1.1.1.7
+++ lib/perl5/MailScanner/Config.pm     24 Jul 2003 18:01:21 -0000      1.11
@@ -62,7 +63,7 @@
 my(%StaticScalars, %ScannerCmds, %SpamLists);
 my(%KeywordCategory, %FilenameRules, %FiletypeRules);
 my(%LanguageStrings);
-my(%RuleScalars, %Defaults, $DefaultRegexp);
+my(%RuleScalars, %Defaults);
 my(%CustomFunctions);
 my(%PercentVars); # For putting substituted variables in all settings

@@ -74,10 +75,6 @@

 %CustomFunctions = (); # These are names of user-written functions

-# This is what the RuleToRegexp function produces when given
-# either "*@*" or "default".
-$DefaultRegexp = '^.*\@.*\.?$';
-
 # Need to read in a filename/ruleset whose value is the location of
 # filename.rules.conf files. Check every rule of every ruleset in
 # turn, stopping with the result of the first rule that matches.
@@ -127,7 +124,8 @@

   my($category, $rulelist, $rule);
   my($direction, $iporaddr, $regexp, $value);
-  my($to, %matches, @addresses, $misses);
+  my($to, %matches, $misses, $return);
+  my(%rcpts, $from, $clientip);

   $category = $KeywordCategory{$name};
   $rulelist = $RuleScalars{$name};
@@ -142,6 +140,12 @@
     # It's a first-match rule
     #

+    $from = $msg->{from};
+    %rcpts = map { $_ => '' } @{$msg->{to}};
+    $clientip = $msg->{clientip};
+
+    $return = $Defaults{$name};
+
     #print STDERR "$name first-match rule\n";

     # If there is no ruleset either, then return the default
@@ -155,26 +159,44 @@
         # It's a text address-based rule
         if ($direction =~ /b/) {
           # Match against all the To addresses and the From address
+          # (not just the ones that are left after matching soft rules).
           $misses = 0;
           $misses++ unless $msg->{from} =~ /$regexp/i;
           foreach $to (@{$msg->{to}}) {
             $misses++,last unless $to =~ /$regexp/i;
           }
-          return $value if $misses == 0;
+          #return $value if $misses == 0;
+          if ($misses == 0) {
+            return $value unless $direction =~ /s/;
+            $return = $value;
+            undef $from;
+            %rcpts = ();
+          }
         } else {
           # Match against any of From and/or To addresses
           if ($direction =~ /f/) {
             # Match against the From address
             #print STDERR "From " . $msg->{from} . " against $regexp\n";
-            return $value if $msg->{from} =~ /$regexp/i;
+            #return $value if $msg->{from} =~ /$regexp/i;
+            if (defined $from and $from =~ /$regexp/i) {
+              return $value unless $direction =~ /s/;
+              $return = $value;
+              undef $from;
+            }
             #print STDERR "Miss\n";
           }
           if ($direction =~ /t/) {
             # Match against every To address
-            foreach $to (@{$msg->{to}}) {
+            #foreach $to (@{$msg->{to}}) {
+            foreach $to (keys %rcpts) {
               #print STDERR "To " . $to . " against $regexp\n";
               #print STDERR "Resulting value would be $value\n";
-              return $value if $to =~ /$regexp/i;
+              #return $value if $to =~ /$regexp/i;
+              if ($to =~ /$regexp/i) {
+                return $value unless $direction =~ /s/;
+                $return = $value;
+                delete $rcpts{$to};
+              }
               #print STDERR "Miss\n";
             }
           }
@@ -185,20 +207,20 @@
         if ($direction =~ /f/) {
           # Match against the SMTP Client IP address
           #print STDERR "Matching IP " . $msg->{clientip} . " against $regexp\n";
-          return $value if $msg->{clientip} =~ /$regexp/;
-        }
-        if ($direction =~ /[tb]/) {
-          # Don't know the target IP address
-          MailScanner::Log::WarnLog("Config Error: Cannot match against " .
-            " destination IP address when resolving configuration option " .
-            " \"%s\"", $name);
+          #return $value if $msg->{clientip} =~ /$regexp/;
+          if (defined $clientip and $clientip =~ /$regexp/i) {
+            return $value unless $direction =~ /s/;
+            $return = $value;
+            undef $clientip;
+          }
         }
       }
     }
     # Nothing matched, so return the default value
     #print STDERR "Nothing matched, so returning default value: " .
     #             $Defaults{$name} . "\n";
-    return $Defaults{$name};
+    #return $Defaults{$name};
+    return $return;

   } else {

@@ -1165,32 +1189,25 @@
     return ('t',$rule);
   }

-  # If it is "default" or "*", then make it *@*
-  if ($rule eq 'default' || $rule eq '*') {
-    $rule = '*@*';
-  }
-  # If it doesn't contain @
-  if ($rule !~ /@/) {
-    if ($rule =~ /^\*/) {
-      # If it starts with *, then make it *@*.domain.com
-      $rule = '*@' . $rule;
-    } else {
-      # If it doesn't contain a *, then make it *@domain.com
-      $rule = '*@' . $rule;
-    }
+  if ($rule eq 'default') {
+    # match anything
+    $rule = '*';
+  } elsif ($rule eq 'bounce') {
+    # bounces have an empty return path
+    $rule = '';
+  } elsif ($rule !~ /\@/) {
+    # If it has no @ then match any local part in the domain
+    $rule = '*@' . $rule;
   }
   # Prepend * if leading @
   $rule = '*' . $rule if $rule =~ /^\@/;
   # Append  * if traiing @
   $rule = $rule . '*' if $rule =~ /\@$/;

-  # Now it's got an @ sign and something both sides of it
   # Change . into \., @ into \@, * into .*
   $rule =~ s/\@/\\@/g;
   $rule =~ s/\./\\./g;
   $rule =~ s/\*/.*/g;
-  # and tack on the optional "." at the end
-  $rule .= '\.?';
   # and tack on the start+end anchors
   $rule = '^' . $rule . '$';
   ('t',$rule);
@@ -1250,6 +1267,9 @@

     # Syntax check and shorten fromto
     $fromto = '';
+    # fanf: soft rules may be overriden by later rules that match a
+    # different address (i.e. one not matched by the soft rule)
+    $fromto = 's' if $direction =~ /soft/i;
     if ($direction =~ /and/i) {
       $fromto = 'b'; # b = both from AND to at the same time
     } else {
@@ -1267,6 +1287,17 @@
     # Pass it the keyword so it can set the default value if there is one.
     ($ruletype, $regexp) = RuleToRegexp($rule);

+    if ($fromto =~ /[tb]/ and lc $rule eq 'bounce') {
+      MailScanner::Log::WarnLog("Config Error: Bounces cannot match " .
+        "against destination addresses in line %d of ruleset %s",
+                                $linecounter, $rulesfilename);
+    }
+    if ($fromto =~ /[tb]/ and $ruletype eq 'd') {
+      MailScanner::Log::WarnLog("Config Error: Cannot match against " .
+        "destination IP address in line %d of ruleset %s",
+        $linecounter, $rulesfilename);
+    }
+
     # Syntax check the value
     #print STDERR "Config: $keyword has rule value " . $File{$keyword} .
     #             " = " . $values{$value} . "\n";
@@ -1281,13 +1312,15 @@
     if (defined $internalvalue || $rulesettype eq 'other') {
       # Update the default value if this is it
       #print STDERR "Ruleset: Is \"$regexp\" the default rule?\n";
-      if ($regexp eq $DefaultRegexp) {
+      # fanf: this is now acheived by the user putting the default
+      # rule last or by making it soft
+      #if ($regexp eq $DefaultRegexp) {
         # Don't store it in the main ruleset as it will always match,
         # whereas we want it to be used only if nothing else matches.
         #print STDERR "Ruleset: Storing Defaults($keyword) = $internalvalue\n";
-        $Defaults{$keyword} = $internalvalue;
-        next;
-      }
+        #$Defaults{$keyword} = $internalvalue;
+        #next;
+      #}
       # It is a valid value, so use it and store it
       my $record = join("\0", $fromto, $ruletype, $regexp, $internalvalue);
       push @{$RuleScalars{$keyword}}, $record;


--- lib/perl5/MailScanner/EximDiskStore.pm      13 May 2003 17:32:25 -0000      1.1.1.5
+++ lib/perl5/MailScanner/EximDiskStore.pm      21 Jul 2003 18:05:01 -0000      1.14
@@ -320,8 +321,8 @@
 # Passed a ref to the array.
 sub ReadBody {
   my $this = shift;
-  my($body) = @_;
-
+  my($body,$max) = @_;
+  my $size = 0;
   my $dh = $this->{indhandle};

   seek($dh, 0, 0); # Rewind the file
@@ -329,10 +330,11 @@
   my $line = <$dh>;
   # FIXME: check that id is correct here

-  while(<$dh>) {
+  while(defined <$dh> and $size < $max) {
     # End of line characters are already there, so don't add them
     #push @{$body}, $_ . "\n";
     push @{$body}, $_;
+    $size += length $_;
   }
 }


--- lib/perl5/MailScanner/PFDiskStore.pm        4 Jul 2003 18:08:28 -0000       1.1.1.3
+++ lib/perl5/MailScanner/PFDiskStore.pm        21 Jul 2003 18:05:01 -0000      1.2
@@ -381,14 +382,16 @@
 # Passed a ref to the array.
 sub ReadBody {
   my $this = shift;
-  my($body) = @_;
+  my($body,$max) = @_;
+  my $size = 0;

   my $b= Body->new( $this->{hdpath} );
   if ($b) {
     $b->Start();
     my $line;
-    while(defined($line = $b->Next())) {
+    while(defined($line = $b->Next()) and $size < $max) {
       push @{$body}, $line . "\n";
+      $size += length $line + 1;
     }
     $b->Done();
   }

--- lib/perl5/MailScanner/SMDiskStore.pm        4 Jul 2003 18:08:28 -0000       1.1.1.7
+++ lib/perl5/MailScanner/SMDiskStore.pm        21 Jul 2003 18:05:01 -0000      1.14
@@ -285,15 +286,17 @@
 # Passed a ref to the array.
 sub ReadBody {
   my $this = shift;
-  my($body) = @_;
-  my($dh) = $this->{indhandle};
+  my($body,$max) = @_;
+  my $size = 0;
+  my $dh = $this->{indhandle};

   seek($dh, 0, 0); # Rewind the file

-  while(<$dh>) {
+  while(defined <$dh> and $size < $max) {
     # End of line characters are already there, so don't add them
     #push @{$body}, $_ . "\n";
     push @{$body}, $_;
+    $size += length $_;
   }
 }


--- lib/perl5/MailScanner/ZMDiskStore.pm        4 Jul 2003 18:08:28 -0000       1.1.1.3
+++ lib/perl5/MailScanner/ZMDiskStore.pm        21 Jul 2003 18:05:01 -0000      1.2
@@ -291,13 +292,15 @@
 #REVISO LEOH
 sub ReadBody {
   my $this = shift;
-  my($body) = @_;
+  my($body,$max) = @_;
+  my $size = 0;

   my $b= Body->new( $this->{hdpath} );
   $b->Start();
   my $line;
-  while( $line= $b->Next() ) {
+  while(defined($line = $b->Next()) and $size < $max) {
     push @{$body}, $line;
+    $size += length $line;
   }
   $b->Done();
 }


--- lib/perl5/MailScanner/SA.pm 4 Jul 2003 18:08:28 -0000       1.1.1.9
+++ lib/perl5/MailScanner/SA.pm 21 Jul 2003 18:05:01 -0000      1.19
@@ -160,10 +161,7 @@
 # Do the SpamAssassin checks on the passed in message
 sub Checks {
   my $message = shift;
-
-  my($dfhandle);
-  my($dfilename, $dfile, $dsize, @WholeMessage, $SAResult, $SAHitList);
-  my($HighScoring, $SAScore);
+  my(@WholeMessage, $SAResult, $SAHitList, $HighScoring, $SAScore);

   # Bail out and fake a miss if too many consecutive SA checks failed
   my $maxfailures = MailScanner::Config::Value('maxspamassassintimeouts');
@@ -174,7 +172,7 @@
   # attempts, then disable it completely.
   if ($maxfailures>0) {
     if ($safailures>=2*$maxfailures) {
-      return (0,0,
+      return (-1,0,
         sprintf(MailScanner::Config::LanguageValue($message,'sadisabled'),
         2*$maxfailures), 0);
     } elsif ($safailures>$maxfailures) {
@@ -195,9 +193,13 @@
        # LEOH 26/03/2003 We do not always have dpath file, so we ask to
   #                 the store module the size
   # $dsize = (stat($message->{store}{dpath}))[7];
-  $dsize = $message->{store}->dsize();
-  return (0,0, MailScanner::Config::LanguageValue($message,'satoolarge'), 0)
-    if $dsize > MailScanner::Config::Value('maxspamassassinsize');
+  #$dsize = $message->{store}->dsize();
+  #$dmax = MailScanner::Config::Value('maxspamassassinsize');
+  #return (-1,0, MailScanner::Config::LanguageValue($message,'satoolarge'), 0)
+  #  if $dsize > MailScanner::Config::Value('maxspamassassinsize');
+  #MailScanner::Log::InfoLog("Truncating large message %s for SpamAssassin",
+  #                          $message->{id})
+  #  if $dsize > $dmax and MailScanner::Config::Value('logspam');

   # Construct the array of lines of the header and body of the message
   # JKF 30/1/2002 Don't chop off the line endings. Thanks to Andreas Piper
@@ -208,10 +210,11 @@
   #}
   @WholeMessage = $global::MS->{mta}->OriginalMsgHeaders($message, "\n");
   #print STDERR "Headers are : " . join(', ', @WholeMessage) . "\n";
-  return (0,0, MailScanner::Config::LanguageValue($message, 'sanoheaders'), 0)
+  return (-1,0, MailScanner::Config::LanguageValue($message, 'sanoheaders'), 0)
     unless @WholeMessage;
   push(@WholeMessage, "\n");
-  $message->{store}->ReadBody(\@WholeMessage);
+  $message->{store}->ReadBody(\@WholeMessage,
+                        MailScanner::Config::Value('maxspamassassinsize'));

   #print STDERR "Whole message is this:\n";
   #print STDERR "----------------------\n";
@@ -319,9 +322,9 @@
   # Construct the hit-list including the score we got.
   $SAReqHits = MailScanner::Config::Value('reqspamassassinscore',$Message)+0.0;
   $SAHitList = MailScanner::Config::LanguageValue($Message, 'score') . '=' .
-               ($SAHits+0.0) . ', ' .
+               ($SAHits+0.0) . ($SAReqHits ? ', ' .
                MailScanner::Config::LanguageValue($Message, 'required') .' ' .
-               $SAReqHits . ($SAHitList?", $SAHitList":'');
+               $SAReqHits : '') . ($SAHitList?", $SAHitList":'');

   # Note to self: I only close the KID in the parent, not in the child.



--- lib/perl5/MailScanner/Message.pm    4 Jul 2003 18:08:28 -0000       1.1.1.7
+++ lib/perl5/MailScanner/Message.pm    8 Jul 2003 16:49:49 -0000       1.17
@@ -81,7 +82,7 @@
 # $isrblspam            set by IsSpam
 # $ishigh               set by IsSpam
 # $sascore             set by IsSpam
-# $spamreport           set by IsSpam
+# $spamreport           set by MessageBatch::SpamChecks and IsSpam
 # $deleted             set by delivery functions
 # $headerspath          set by WriterHeaderFile # file is read-only
 # $cantparse           set by Explode
@@ -257,6 +258,7 @@
   my $RBLsaysspam   = 0;
   my $rblcounter    = 0;
   my $LogSpam = MailScanner::Config::Value('logspam');
+  my $LogNonSpam = MailScanner::Config::Value('lognonspam');
   my $LocalSpamText = MailScanner::Config::LanguageValue($this, 'spam');

   # Construct a pretty list of all the unique domain names for logging
@@ -269,12 +271,11 @@
   # $spamwhitelisted      set by IsSpam
   # $isspam               set by IsSpam
   # $ishigh               set by IsSpam
-  # $spamreport           set by IsSpam
+  # $spamreport           set by MessageBatch::SpamChecks and updated here

   $this->{spamwhitelisted} = 0;
   $this->{isspam} = 0;
   $this->{ishigh} = 0;
-  $this->{spamreport} = "";
   $this->{sascore} = 0;

   ## If it's a blacklisted address, don't bother doing any checks at all
@@ -366,10 +367,17 @@
   my $SAHighScoring = 0;
   my $saheader = "";
   my $sascore  = 0;
+  my $scanned  = 1;
   ($SAsaysspam, $SAHighScoring, $saheader, $sascore)
     = MailScanner::SA::Checks($this);
   $this->{sascore} = $sascore; # Save the actual figure for use later...

+  # not scanned after all?
+  if ($SAsaysspam < 0) {
+      $scanned = 0;
+      $SAsaysspam = 0;
+  }
+
   # Fix the return values
   $SAsaysspam = 0 unless $saheader;    # Solve bug with empty SAreports
   $saheader =~ s/\s+$//g if $saheader; # Solve bug with trailing space
@@ -385,8 +393,11 @@
     $this->{issaspam} = $SAsaysspam;
   }

-  # If it's spam...
-  if ($this->{isspam}) {
+  # it didn't scan the message after all?
+  if (not $scanned) {
+      $spamheader = MailScanner::Config::Value('unscannedheader', $this)
+         . ", $saheader";
+  } elsif ($this->{isspam}) {
     #print STDERR "It is spam\nInclude SA = $includesaheader\n";
     #print STDERR "SAHeader = $saheader\n";
     $spamheader = $rblspamheader;
@@ -421,7 +432,7 @@
   }

   # Do the spam logging here so we can log high-scoring spam too
-  if ($LogSpam && $this->{isspam}) {
+  if (($LogSpam && $this->{isspam}) || $LogNonSpam) {
     my $ReportText = $spamheader;
     $ReportText =~ s/\s+/ /sg;
     MailScanner::Log::InfoLog("Message %s from %s (%s) to %s is %s",
@@ -2098,7 +2109,7 @@

   # Create the top-level MIME entity, just the headers
   $top = MIME::Entity->build(Type       => 'multipart/mixed',
-                             From       => "MailScanner <$localpostmaster>",
+                             From       => $from,
                              To         => $to,
                              Subject    => $newsubject,
                              'X-Mailer' => 'MailScanner',


--- lib/perl5/MailScanner/ConfigDefs.pl 4 Jul 2003 18:08:27 -0000       1.1.1.7
+++ lib/perl5/MailScanner/ConfigDefs.pl 10 Jul 2003 15:00:41 -0000      1.12
@@ -85,6 +86,8 @@
 lastlookup                     = alwayslookeduplast
 listsascores                    = includescoresinspamassassinreport
 logfacility                    = syslogfacility
+logformtags                    = loghtmlformtags
+logobjecttags                  = logobjectcodebasetags
 maxdirtybytes                  = maxunsafebytesperscan
 maxdirtymessages               = maxunsafemessagesperscan
 maxmessagesize                 = maximummessagesize
@@ -140,6 +143,8 @@
 debugspamassassin      0       no      0       yes     1
 deliverinbackground    1       no      0       yes     1
 logspam                        1       no      0       yes     1
+lognonspam             0       no      0       yes     1
+logmessageids          0       no      0       yes     1
 expandtnef             1       no      0       yes     1
 showscanner            0       no      0       yes     1
 spamassassinautowhitelist 1    no      0       yes     1
@@ -223,6 +228,8 @@
 ListSAScores           0       no      0       yes     1
 #LoadSpamAssassin      0       no      0       yes     1
 LogIFrameTags          0       no      0       yes     1
+LogFormTags            0       no      0       yes     1
+LogObjectTags          0       no      0       yes     1
 LogPermittedFilenames  0       no      0       yes     1
 LogPermittedFiletypes  0       no      0       yes     1
 MultipleHeaders                append  append  append  replace replace add     add


--- lib/perl5/MailScanner/MessageBatch.pm       4 Jul 2003 18:08:28 -0000       1.1.1.6
+++ lib/perl5/MailScanner/MessageBatch.pm       22 Jul 2003 09:45:04 -0000      1.11
@@ -140,6 +141,7 @@

   my($id, $message);
   my $counter = 0;
+  my $highcount = 0;

   #print STDERR "Starting spam checks\n";

@@ -148,10 +150,16 @@

   while(($id, $message) = each %{$this->{messages}}) {
     next if $message->{deleted};
+
+    # set default value in case we always add a spam report header
+    $message->{spamreport} =
+       MailScanner::Config::Value('unscannedheader', $this);
+
     next unless MailScanner::Config::Value('spamchecks', $message);

     #print STDERR "Spam checks for $id\n";
     $counter += $message->IsSpam();
+    $highcount += $message->{ishigh};

     if (!MailScanner::Config::Value('spamdetail', $message)) {
       $message->{spamreport} = MailScanner::Config::LanguageValue($message,
@@ -161,6 +169,8 @@
   }
   MailScanner::Log::InfoLog("Spam Checks: Found $counter spam messages")
     if $counter>0;
+  MailScanner::Log::InfoLog("Spam Checks: Found $highcount high-scoring spam messages")
+    if $highcount>0;
   #print STDERR "$counter messages were spam\n";
 }

@@ -231,9 +241,13 @@
       # or the HTML stripping.
       if ($message->{bodymodified}) {
         $message->DeliverModifiedBody('unscannedheader');
+        MailScanner::Log::InfoLog("Delivered modified message $id")
+             if MailScanner::Config::Value('logmessageids');
       } else {
         $OutQ = MailScanner::Config::Value('outqueuedir', $message);
         $message->DeliverUnscanned($OutQ);
+        MailScanner::Log::InfoLog("Delivered unscanned message $id")
+             if MailScanner::Config::Value('logmessageids');
       }
       $message->{deleted} = 1; # This marks it for purging from disk
       push @messages, $message;
@@ -274,12 +288,12 @@
   MailScanner::Log::InfoLog("Virus Scanning: Found %d viruses", $viruses+0)
     if defined $viruses && $viruses>0;

-  #MailScanner::Log::InfoLog("Other Checks: Starting");
+  MailScanner::Log::InfoLog("Other Checks: Starting");
   my $others  = MailScanner::SweepOther::ScanBatch($this, 'scan');
   MailScanner::Log::InfoLog("Other Checks: Found %d problems", $others+0)
     if defined $others && $others>0;

-  #MailScanner::Log::InfoLog("Content Checks: Starting");
+  MailScanner::Log::InfoLog("Content Checks: Starting");
   my $content  = MailScanner::SweepContent::ScanBatch($this, 'scan');
   MailScanner::Log::InfoLog("Content Checks: Found %d problems", $content+0)
     if defined $content && $content>0;
@@ -465,6 +479,8 @@
     next if $message->{infected};
     #print STDERR "Delivering uninfected message $id\n";
     $message->DeliverUninfected();
+    MailScanner::Log::InfoLog("Delivered uninfected message $id")
+           if MailScanner::Config::Value('logmessageids');
     $message->{deleted} = 1;
     push @messages, $message;
   }
@@ -531,6 +547,11 @@
       $message->DeliverCleaned();
       #print STDERR "Deleting silent-infected message " . $message->{id} . "\n";
       push @messages, $message;
+      MailScanner::Log::InfoLog("Delivering message $id with silent virus")
+          if MailScanner::Config::Value('logmessageids');
+    } else {
+      MailScanner::Log::InfoLog("DISCARDED message $id with silent virus")
+          if MailScanner::Config::Value('logmessageids');
     }
     $message->{deleted} = 1;
     $message->{stillwarn} = 1;


--- lib/perl5/MailScanner/Exim.pm       4 Jul 2003 18:08:28 -0000       1.1.1.8
+++ lib/perl5/MailScanner/Exim.pm       8 Jul 2003 16:25:12 -0000       1.20
@@ -1320,6 +1321,8 @@
        $HitLimit4 = 1
          if $DirtyBytes>=$MaxDirtyB;
        $newmessage->WriteHeaderFile(); # Write the file of headers
+       MailScanner::Log::InfoLog("New Message: $id to be scanned")
+               if MailScanner::Config::Value('logmessageids');
       } else {
        $newmessage->NeedsScanning(0);
        $CleanMsgs++;
@@ -1329,6 +1332,8 @@
        $HitLimit2 = 1
          if $CleanBytes>=$MaxCleanB;
        $newmessage->WriteHeaderFile(); # Write the file of headers
+       MailScanner::Log::InfoLog("New Message: $id to be forwarded")
+               if MailScanner::Config::Value('logmessageids');
       }
     }



--- lib/perl5/MailScanner/SweepContent.pm       4 Jul 2003 18:08:28 -0000       1.1.1.4
+++ lib/perl5/MailScanner/SweepContent.pm       22 Jul 2003 13:56:28 -0000      1.7
@@ -75,7 +76,7 @@
   # Can play with the MIME headers of a message using $mime.

   my($id,$message,$ent,$partialcount,$allowiframes,$allowobjects,$allowforms);
-  my($stripdangerous, $counter, $stripcounter);
+  my($logiframes,$logobjects,$logforms,$stripdangerous,$counter,$stripcounter);
   $counter = 0;
   $stripcounter = 0; # No. of messages we need to strip HTML from

@@ -114,7 +115,7 @@
     }

     # Search for Microsoft-specific attacks
-    # Disallow both by default. Allow them only if all addresses agree.
+    # Disallow by default. Allow them only if all addresses agree.
     $allowiframes = 0;
     $allowobjects = 0;
     $allowforms   = 0;
@@ -124,10 +125,15 @@
       if MailScanner::Config::Value('allowobjecttags', $message) =~ /^[1\s]+$/;
     $allowforms   = 1
       if MailScanner::Config::Value('allowformtags', $message) =~ /^[1\s]+$/;
+    # Log if any address requires that.
+    $logiframes  = MailScanner::Config::Value('logiframetags', $message);
+    $logobjects   = MailScanner::Config::Value('logobjecttags', $message);
+    $logforms      = MailScanner::Config::Value('logformtags', $message);
     $stripdangerous = MailScanner::Config::Value('stripdangeroustags',$message);
     # Shortcut the check completely if they want to allow everything
-    # and are not converting nasty tags to text
-    if (!($allowiframes && $allowforms && $allowobjects && !$stripdangerous) &&
+    # and are not converting nasty tags to text or logging
+    if (!($allowiframes && $allowforms && $allowobjects && !$stripdangerous
+         && !$logiframes && !$logobjects && !$logforms) &&
         FindHTMLExploits($message, $id, $ent, $allowiframes,
                          $allowobjects, $allowforms, $stripdangerous)) {
       $counter++;
@@ -356,10 +362,10 @@
       }
     }
     if ($formfound) {
-      ## Log the <IFrame>
-      #MailScanner::Log::InfoLog("HTML Form tag found in message %s from %s",
-      #  $id, $message->{from})
-      #  if MailScanner::Config::Value('logformtags', $message);
+      # Log the <Form>
+      MailScanner::Log::InfoLog("HTML Form tag found in message %s from %s",
+        $id, $message->{from})
+        if MailScanner::Config::Value('logformtags', $message);
       # Mark the message
       if ($allowforms) {
         if ($stripdangerous) {
@@ -377,6 +383,10 @@
       }
     }
     if ($codebasefound) {
+      # Log the object codebase
+      MailScanner::Log::InfoLog("HTML Object codebase tag found ".
+        "in message %s from %s", $id, $message->{from})
+        if MailScanner::Config::Value('logobjecttags', $message);
       if ($allowobjects) {
         if ($stripdangerous) {
           $message->{needsstripping} = 1;


--- lib/perl5/MailScanner/SweepOther.pm 4 Jul 2003 18:08:28 -0000       1.1.1.6
+++ lib/perl5/MailScanner/SweepOther.pm 4 Jul 2003 19:13:31 -0000       1.6
@@ -195,19 +196,20 @@
           $MatchFound = 1;
           if ($allowdeny eq 'deny') {
             # It's a rejection rule, so log the error.
-            MailScanner::Log::InfoLog("Filename Checks: %s (%s)",
-                                      $logtext, $attach);
+            MailScanner::Log::InfoLog("Filename Checks: %s (%s %s)",
+                                      $logtext, $id, $attach);
             $message->{namereports}{$safename} .= "$usertext ($safename)\n";
             $message->{nametypes}{$safename}   .= "f";
             $counter++;
             $message->{nameinfected}++;
           } else {
-            MailScanner::Log::InfoLog("Filename Checks: Allowing %s", $safename)
+            MailScanner::Log::InfoLog("Filename Checks: Allowing %s %s",
+                                     $id, $safename)
               if $LogNames;
           }
         }
-        MailScanner::Log::InfoLog("Filename Checks: Allowing %s " .
-                                  "(no rule matched)", $safename)
+        MailScanner::Log::InfoLog("Filename Checks: Allowing %s %s " .
+                                  "(no rule matched)", $id, $safename)
           if $LogNames && !$MatchFound;
       }
     }



More information about the MailScanner mailing list