MailScanner causes SpamAssassin rules to firing inconsistently

[Wolfgang Baudler]

> Wolfgang,
>
> Would you do me a favor and test this PR in your setup?
>
> https://github.com/MailScanner/v4/pull/42/files
>

I tried this version of the patch, and yes it seems to fix the issue in
our setup.

There might be other MTA implementations other then sendmail that might
need the same fix, though (There is EximDiskStore.pm, PFDiskStore.pm 
QMDiskStore.pm, SMDiskStore.pm, ZMDiskStore.pm). I have only looked at 
SMDiskStore.pm.

Wolfgang

> On Mon, Nov 16, 2015 at 3:53 PM, Wolfgang Baudler <wbaudler at gb.nrao.edu>
> wrote:
>
>> > On 11/05/2015 11:05 AM, Wolfgang Baudler wrote:
>> >>
>> >> no difference in log messages, except the senders domain and address
>> of
>> >> course.
>> >>
>> >> internal log example:
>> >> Nov  5 13:50:58 io MailScanner[24033]: Message tA5IopES005503 from
>> >> 192.33.116.115 (wbaudler at gb.nrao.edu) to gb.nrao.edu is not spam,
>> >> SpamAssassin (score=-199.008, required 5, autolearn=disabled,
>> >> TEST_RULE_AA
>> >> 1.00, NRAO_HEADER_PRESENT -100.00, TVD_SPACE_RATIO 0.00,
>> >> T_RP_MATCHES_RCVD
>> >> -0.01, USER_IN_WHITELIST -100.00)
>> >>
>> >> external log example:
>> >> Nov  5 13:55:47 io MailScanner[24004]: Message tA5ItQmr006622 from
>> >> 98.138.229.70 (wbaudler at yahoo.com) to gb.nrao.edu is not spam,
>> >>  SpamAssassin (score=0.902, required 5, autolearn=disabled,
>> >> DKIM_ADSP_CUSTOM_MED 0.00, DKIM_SIGNED 0.10, FREEMAIL_FROM 0.00,
>> >> LOCAL_ID_JAVAMAIL 1.00, NML_ADSP_CUSTOM_MED 1.20, RCVD_IN_DNSWL_LOW
>> >> -0.70, RCVD_IN_MSPIKE_H3 -0.70, SPF_PASS -0.00, T_DKIM_INVALID 0.01,
>> >> T_RP_MATCHES_RCVD -0.01)
>> >>
>> >> The TEST_RULE_AA test result is missing in the external example. The
>> >> message sent was completely identical.
>> >
>> >
>> > At this point I am at a loss unless your "Max SpamAssassin Size"
>> setting
>> > and your test message size are such that the extra headers from the
>> > remote source push the test string out of range. This seems highly
>> > unlikely.
>> >
>> > It seems this might be a spamassassin bug triggered by something in
>> the
>> > message headers from the remote servers, but this seems unlikely too.
>> >
>> > --
>> > Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
>> > San Francisco Bay Area, California    better use your sense - B. Dylan
>> >
>> >
>>
>> After doing some extended chasing I have an update on this issue.
>>
>> It seems that the firing or non-firing of body rules depends on the MUA
>> used to send the message. In particular on the fact that some MUA add an
>> empty line (0x0a newline) at the end of the body when
>> sending and some do not.
>>
>> Those that add the extra line with an newline will fire body rules
>> correctly if processed through Mailscanner, those that do not have the
>> extra line will not fire.
>>
>> Some particular real spam messages seem to consistently lack this empty
>> line and thus get not tagged correctly.
>>
>> I have not figured out exactly where this missing newline throws
>> MailScanner off, but I was able to implement a crude fix by modifying
>> the
>> loop of the ReadBody function in SMDiskStore.pm like this (we are using
>> sendmail with MailScanner):
>>
>>   while(defined($line = <$dh>) && $size<$max) {
>>     push @{$body}, $line;
>>     $size += length($line);
>>     #print STDERR "Line read2 is ****" . $line . "****\n";
>>   }
>>   $lastlineread = $line;
>>   push @{$body}, "\n";
>>
>> Only the last line was added, which pushes an unconditional newline at
>> the
>> end of the body just read. After that modification all body rules fire
>> correctly as expected.
>>
>> Hopefully someone more familiar with the MailScanner code can come up
>> with
>> a proper patch to fix this issue?
>>
>> Wolfgang
>>
>>
>>
>>
>> --
>> MailScanner mailing list
>> mailscanner at lists.mailscanner.info
>> http://lists.mailscanner.info/listinfo/mailscanner
>>
>>
>
>
> --
> Shawn Iverson
> Director of Technology
> Rush County Schools
> 765-932-3901 x271
> iversons at rushville.k12.in.us
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/listinfo/mailscanner
>
>