Certain Spamassassin rules do not seem to be firing all of the time

Fri Jun 14 19:33:33 IST 2013

http://pastebin.com/VQs2FSxK

I also tried disabling caching with SpamAssassin in my Mailscanner.conf today. I don't think it made a difference..   I don't have many examples today, it seems as if this specific spammer is only sending out a few today.

The above example just came in within the last 15 minutes.  It did manage to get classified as Spam, but when I compare what rules it hit on going through MailScanner/Spamassassin vs using the above text and scanning with -test-mode, some of the rules are not hitting when going through MailScanner/Spamassassin.

The rules it hits on for me through Mailscanner:

X-MailScanner-SpamCheck: spam, SpamAssassin (score=9.3, required 6.5,
                BAYES_50 0.80, LOTS_OF_MONEY 0.00, RAZOR2_CHECK 8.50,
                RP_MATCHES_RCVD -0.00)

The rules it hits on according to spamassassin -test-mode:

Content analysis details:   (28.1 hits, 6.5 required)
 6.6 BAYES_99               BODY: Bayes spam probability is 99 to 100%
                            [score: 1.0000]
 6.5 URIBL_JP_SURBL         Contains an URL listed in the JP SURBL blocklist
                            [URIs: eldmil.com]
 5.0 URIBL_DBL_SPAM         Contains an URL listed in the DBL blocklist
                            [URIs: eldmil.com]
  10 URIBL_BLACK            Contains an URL listed in the URIBL blacklist
                            [URIs: eldmil.com]
-0.0 RP_MATCHES_RCVD        Envelope sender domain matches handover relay domain
 8.5 RAZOR2_CHECK           Listed in Razor2 (http://razor.sf.net/)
 0.0 LOTS_OF_MONEY          Huge... sums of money
-8.4 AWL                    AWL: From: address is in the auto white-list

------ End of SpamAssassin results, Original message follows --------

Thanks for your assistance.

Brian

BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / (312) 577-8045 f / (312) 577-4490
brian.duncan at kattenlaw.com / www.kattenlaw.com

From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Friday, June 14, 2013 12:15 PM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time

Very odd can u pastebin the raw email and drop the pastebin link so we can run it over our systems to compare

On Friday, 14 June 2013, Duncan, Brian M. wrote:
Looks like deleting the spamassassin cache made no difference.

This morning I received another spam that made it through.

This is what it scored when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
                required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,
                RP_MATCHES_RCVD -0.00)

I moved it over to my mailscanner/spamassassin box within 30 seconds of receiving it and this is what it scored on my Mailscanner box from the command line doing spamassassin -test-mode < message.txt:

Content analysis details:   (14.6 hits, 6.5 required)
-0.0 RP_MATCHES_RCVD        Envelope sender domain matches handover relay domain
 3.0 BAYES_60               BODY: Bayes spam probability is 60 to 80%
                            [score: 0.6460]
 2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
                            above 50%
                            [cf: 100]
 8.5 RAZOR2_CHECK           Listed in Razor2 (http://razor.sf.net/)
 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
                            [cf: 100]
 0.0 LOTS_OF_MONEY          Huge... sums of money
 0.1 FROM_12LTRDOM          From a 12-letter domain

------ End of SpamAssassin results, Original message follows --------

The really odd thing, is if I take the body and subject from the spam message above and send it through a hotmail account I have (which I white list, which is why that shows in the below results), this is what it scores when passed through Mailscanner/Spamassassin:

X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached,
                score=20.146, required 6.5, autolearn=spam, AWL -13.90,
                BAYES_50 0.80, FREEMAIL_ENVFROM_END_DIGIT 0.25, FREEMAIL_FROM 0.00,
                HTML_MESSAGE 0.00, LOTS_OF_MONEY 0.00, RAZOR2_CF_RANGE_51_100 0.50,
                RAZOR2_CF_RANGE_E8_51_100 2.50, RAZOR2_CHECK 8.50,
                RCVD_IN_DNSWL_NONE -0.00, RP_MATCHES_RCVD -0.00, SPF_PASS -0.00,
                URIBL_BLACK 10.00, URIBL_DBL_SPAM 5.00, URIBL_JP_SURBL 6.50)

This makes no sense to me, it's almost like this specific Spammer has figured out a way to get Mailscanner to stop scanning portions of its message.

I am going to turn off caching of spamassassin results next in my mailscanner conf to see if that has any impact.

If anyone has any ideas please let me know.

Brian

BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / (312) 577-8045 f / (312) 577-4490
brian.duncan at kattenlaw.com<javascript:_e(%7b%7d,%20'cvml',%20'brian.duncan at kattenlaw.com');> / www.kattenlaw.com<http://www.kattenlaw.com>

From: mailscanner-bounces at lists.mailscanner.info<javascript:_e(%7b%7d,%20'cvml',%20'mailscanner-bounces at lists.mailscanner.info');> [mailto:mailscanner-bounces at lists.mailscanner.info<javascript:_e(%7b%7d,%20'cvml',%20'mailscanner-bounces at lists.mailscanner.info');>] On Behalf Of Duncan, Brian M.
Sent: Thursday, June 13, 2013 7:35 PM
To: MailScanner discussion
Subject: RE: Certain Spamassassin rules do not seem to be firing all of the time

Thanks for the suggestions Martin.

I don't have any specific user that I run as:

Run As User =

So I assume it is running as root? My tests with -test-mode were run as root.. I do have the .spamassassin dir in root that has bayes db's that are the ones that get updated, and I did confirm there was nothing there causing problems.

I took your advice and started by stopping Mailscanner and deleting the cache and any orphaned files in the directories, hopefully that will have a positive impact.

I assume it must be something odd like that,  these messages started coming through last week.  I have to believe if all my rules were not firing since I built that box a year or so ago I would have noticed this sooner.

One thing I noticed after taking other messages that failed due to body checks that actually wind up tagged as Spam, most seem to have more rules that fire off when I run them locally as root with -test-mode then what they have in my mail client after they have come through.

I do see hits on messages for rules that ONLY exist in some of the rules in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory.  So I know it is accessing those files, just not all of them for some reason at certain times..

I just took a message that made it through today for me:

X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,

                required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD -0.00)

When I check this message on my MailScanner box with Spamassassin as root I get:

Content analysis details:   (30.1 hits,

--
--
Martin Hepworth, CISSP
Oxford, UK

===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law.  If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or 
distribution of this information may be subject to legal restriction or sanction.  Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original 
message without making any copies.
===========================================================
NOTIFICATION:  Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/5a315a0a/attachment-0001.html 