MailScanner causes SpamAssassin rules to firing inconsistently

[Wolfgang Baudler]

> On 11/05/2015 11:05 AM, Wolfgang Baudler wrote:
>>
>> no difference in log messages, except the senders domain and address of
>> course.
>>
>> internal log example:
>> Nov  5 13:50:58 io MailScanner[24033]: Message tA5IopES005503 from
>> 192.33.116.115 (wbaudler at gb.nrao.edu) to gb.nrao.edu is not spam,
>> SpamAssassin (score=-199.008, required 5, autolearn=disabled,
>> TEST_RULE_AA
>> 1.00, NRAO_HEADER_PRESENT -100.00, TVD_SPACE_RATIO 0.00,
>> T_RP_MATCHES_RCVD
>> -0.01, USER_IN_WHITELIST -100.00)
>>
>> external log example:
>> Nov  5 13:55:47 io MailScanner[24004]: Message tA5ItQmr006622 from
>> 98.138.229.70 (wbaudler at yahoo.com) to gb.nrao.edu is not spam,
>>  SpamAssassin (score=0.902, required 5, autolearn=disabled,
>> DKIM_ADSP_CUSTOM_MED 0.00, DKIM_SIGNED 0.10, FREEMAIL_FROM 0.00,
>> LOCAL_ID_JAVAMAIL 1.00, NML_ADSP_CUSTOM_MED 1.20, RCVD_IN_DNSWL_LOW
>> -0.70, RCVD_IN_MSPIKE_H3 -0.70, SPF_PASS -0.00, T_DKIM_INVALID 0.01,
>> T_RP_MATCHES_RCVD -0.01)
>>
>> The TEST_RULE_AA test result is missing in the external example. The
>> message sent was completely identical.
>
>
> At this point I am at a loss unless your "Max SpamAssassin Size" setting
> and your test message size are such that the extra headers from the
> remote source push the test string out of range. This seems highly
> unlikely.
>
> It seems this might be a spamassassin bug triggered by something in the
> message headers from the remote servers, but this seems unlikely too.
>
> --
> Mark Sapiro <mark at msapiro.net>        The highway is for gamblers,
> San Francisco Bay Area, California    better use your sense - B. Dylan
>
>

After doing some extended chasing I have an update on this issue.

It seems that the firing or non-firing of body rules depends on the MUA
used to send the message. In particular on the fact that some MUA add an
empty line (0x0a newline) at the end of the body when
sending and some do not.

Those that add the extra line with an newline will fire body rules
correctly if processed through Mailscanner, those that do not have the
extra line will not fire.

Some particular real spam messages seem to consistently lack this empty
line and thus get not tagged correctly.

I have not figured out exactly where this missing newline throws
MailScanner off, but I was able to implement a crude fix by modifying the
loop of the ReadBody function in SMDiskStore.pm like this (we are using
sendmail with MailScanner):

  while(defined($line = <$dh>) && $size<$max) {
    push @{$body}, $line;
    $size += length($line);
    #print STDERR "Line read2 is ****" . $line . "****\n";
  }
  $lastlineread = $line;
  push @{$body}, "\n";

Only the last line was added, which pushes an unconditional newline at the
end of the body just read. After that modification all body rules fire
correctly as expected.

Hopefully someone more familiar with the MailScanner code can come up with
a proper patch to fix this issue?

Wolfgang