Certain Spamassassin rules do not seem to be firing all of the time
Duncan, Brian M.
brian.duncan at kattenlaw.com
Fri Jun 14 14:23:49 IST 2013
Looks like deleting the spamassassin cache made no difference.
This morning I received another spam that made it through.
This is what it scored when passed through Mailscanner/Spamassassin:
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,
RP_MATCHES_RCVD -0.00)
I moved it over to my mailscanner/spamassassin box within 30 seconds of receiving it and this is what it scored on my Mailscanner box from the command line doing spamassassin -test-mode < message.txt:
Content analysis details: (14.6 hits, 6.5 required)
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
3.0 BAYES_60 BODY: Bayes spam probability is 60 to 80%
[score: 0.6460]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money
0.1 FROM_12LTRDOM From a 12-letter domain
------ End of SpamAssassin results, Original message follows --------
The really odd thing, is if I take the body and subject from the spam message above and send it through a hotmail account I have (which I white list, which is why that shows in the below results), this is what it scores when passed through Mailscanner/Spamassassin:
X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached,
score=20.146, required 6.5, autolearn=spam, AWL -13.90,
BAYES_50 0.80, FREEMAIL_ENVFROM_END_DIGIT 0.25, FREEMAIL_FROM 0.00,
HTML_MESSAGE 0.00, LOTS_OF_MONEY 0.00, RAZOR2_CF_RANGE_51_100 0.50,
RAZOR2_CF_RANGE_E8_51_100 2.50, RAZOR2_CHECK 8.50,
RCVD_IN_DNSWL_NONE -0.00, RP_MATCHES_RCVD -0.00, SPF_PASS -0.00,
URIBL_BLACK 10.00, URIBL_DBL_SPAM 5.00, URIBL_JP_SURBL 6.50)
This makes no sense to me, it's almost like this specific Spammer has figured out a way to get Mailscanner to stop scanning portions of its message.
I am going to turn off caching of spamassassin results next in my mailscanner conf to see if that has any impact.
If anyone has any ideas please let me know.
Brian
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / (312) 577-8045 f / (312) 577-4490
brian.duncan at kattenlaw.com / www.kattenlaw.com
From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Duncan, Brian M.
Sent: Thursday, June 13, 2013 7:35 PM
To: MailScanner discussion
Subject: RE: Certain Spamassassin rules do not seem to be firing all of the time
Thanks for the suggestions Martin.
I don't have any specific user that I run as:
Run As User =
So I assume it is running as root? My tests with -test-mode were run as root.. I do have the .spamassassin dir in root that has bayes db's that are the ones that get updated, and I did confirm there was nothing there causing problems.
I took your advice and started by stopping Mailscanner and deleting the cache and any orphaned files in the directories, hopefully that will have a positive impact.
I assume it must be something odd like that, these messages started coming through last week. I have to believe if all my rules were not firing since I built that box a year or so ago I would have noticed this sooner.
One thing I noticed after taking other messages that failed due to body checks that actually wind up tagged as Spam, most seem to have more rules that fire off when I run them locally as root with -test-mode then what they have in my mail client after they have come through.
I do see hits on messages for rules that ONLY exist in some of the rules in the /var/lib/spamassassin/3.003001/updates_spamassassin_org directory. So I know it is accessing those files, just not all of them for some reason at certain times..
I just took a message that made it through today for me:
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,
required 6.5, BAYES_50 0.80, DIET_1 0.00, RP_MATCHES_RCVD -0.00)
When I check this message on my MailScanner box with Spamassassin as root I get:
Content analysis details: (30.1 hits, 6.5 required)
6.5 URIBL_JP_SURBL Contains an URL listed in the JP SURBL blocklist
[URIs: nthjus.com]
0.0 DIET_1 BODY: Lose Weight Spam
1.3 RCVD_IN_RP_RNBL RBL: Relay in RNBL,
https://senderscore.org/blacklistlookup/
[64.191.19.228 listed in bl.score.senderscore.com]
10 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: nthjus.com]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5001]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
It seems to be all the rules that don't fire are the ones where it would actually be looking something up, right? Through DNS?
BRIAN M. DUNCAN
Data Security Administrator
Katten Muchin Rosenman LLP
525 W. Monroe Street / Chicago, IL 60661-3693
p / (312) 577-8045 f / (312) 577-4490
brian.duncan at kattenlaw.com / www.kattenlaw.com
From: mailscanner-bounces at lists.mailscanner.info [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf Of Martin Hepworth
Sent: Thursday, June 13, 2013 1:51 AM
To: MailScanner discussion
Subject: Re: Certain Spamassassin rules do not seem to be firing all of the time
Are you running the tests against the same user MailScanner runs as to make sure any .spamassassin directory settings arent overriding
in both headers you're getting spamassassin cache hits which is a mailscanner option. You might want to stop MailScanner, delete the spamassassin cache file ans retry. Could be the cache file has got corrupt somehow.
martin
--
Martin Hepworth, CISSP
Oxford, UK
On 12 June 2013 22:05, Duncan, Brian M. <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>> wrote:
spamassassin-3.3.1-3.el5.rf
mailscanner-4.83.5-1
Looking for some help here, it looks like sometimes Mailscanner is causing SpamAssassin to not use some rules. (Not exactly sure on this I assume it is Mailscanner based on the behavior I am seeing)
I receive the message and it is not tagged as Spam and winds up in my inbox. The headers show on this example:
X-MailScanner-SpamCheck: not spam, SpamAssassin (cached, score=0.8,required 6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00,RP_MATCHES_RCVD -0.00)
I then take that message and drag it into a separate mailbox I had setup on our Exchange server, then pull it down to my Sendmail/Mailscanner/SpamAssassin box through imap in rfc822 format.
I then run the same message through Spamassassin with -test-mode locally from my mail server I get different scoring on, it looks like I am missing some of the checks because now it defiantly shows as Spam:
Content analysis details: (17.3 hits, 6.5 required)
5.0 URIBL_DBL_SPAM Contains an URL listed in the DBL blocklist
[URIs: eelefs.net<http://eelefs.net>]
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60%
[score: 0.5050]
2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level
above 50%
[cf: 100]
8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
[cf: 100]
0.0 LOTS_OF_MONEY Huge... sums of money
------ End of SpamAssassin results, Original message follows --------
So I was wondering if it had to with my MailScanner.conf having this line: SpamAssassin Local State Dir = # /var/lib/spamassassin
But based on my debug of MailScanner, it does not matter if the # is present or not, MailScanner seems to think it knows where all the rules are. The below output is with SpamAssassin Local State Dir = /var/lib/spamassassin
In Debugging mode, not forking...
Trying to setlogsock(unix)
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: adding facilities: all
15:54:01 Jun 12 15:54:01.475 [32352] dbg: logger: logging level is DBG
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: SpamAssassin version 3.3.1
15:54:01 Jun 12 15:54:01.475 [32352] dbg: generic: Perl 5.008008, PREFIX=/usr, DEF_RULES_DIR=/usr/share/spamassassin, LOCAL_RULES_DIR=/etc/mail/spamassassin, LOCAL_STATE_DIR=/var/lib/spam
assassin
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: timing enabled
15:54:01 Jun 12 15:54:01.475 [32352] dbg: config: score set 0 chosen.
15:54:01 Jun 12 15:54:01.477 [32352] dbg: util: running in taint mode? no
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: is Net::DNS::Resolver available? yes
15:54:01 Jun 12 15:54:01.480 [32352] dbg: dns: Net::DNS version: 0.65
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/init.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v310.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v312.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v320.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: read file /etc/mail/spamassassin/v330.pre
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for sys rules pre files
15:54:01 Jun 12 15:54:01.481 [32352] dbg: config: using "/var/lib/spamassassin/3.003001" for default rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /var/lib/spamassassin/3.003001/updates_spamassassin_org.cf<http://updates_spamassassin_org.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: using "/etc/mail/spamassassin" for site rules dir
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_evilnum1.cf<http://70_sare_evilnum1.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/70_sare_unsub.cf<http://70_sare_unsub.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/chickenpox.cf<http://chickenpox.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/local.cf<http://local.cf>
15:54:01 Jun 12 15:54:01.482 [32352] dbg: config: read file /etc/mail/spamassassin/mailscanner.cf<http://mailscanner.cf>
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: using "/root/.spamassassin/user_prefs" for user prefs file
15:54:01 Jun 12 15:54:01.483 [32352] dbg: config: read file /root/.spamassassin/user_prefs
15:54:01 Jun 12 15:54:01.484 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
15:54:01 Jun 12 15:54:01.488 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
15:54:01 Jun 12 15:54:01.491 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
15:54:01 Jun 12 15:54:01.494 [32352] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
15:54:01 Jun 12 15:54:01.496 [32352] dbg: pyzor: network tests on, attempting Pyzor
The odd thing here to me, is if I search my maillog for some of the hits from above, Like URIBL_DBL_SPAM, I am seeing many hits on this.. It just seems to be skipping some of the rules for a certain messages. I looked through
Anyone have any ideas where I can start to figure this one out? I checked my rules, but since some of the rules are firing I assumed it can't have anything to do with that..
Here is the complete output from the message I give as an example from above: (minus the spammy body)
Received: from CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) by
CHI-US-CAS-1B.us.kmz.com<http://CHI-US-CAS-1B.us.kmz.com> (10.125.15.2) with Microsoft SMTP Server (TLS) id
14.3.123.3; Wed, 12 Jun 2013 15:44:04 -0500
Received: from chi-us-vwall-01.us.kmz.com<http://chi-us-vwall-01.us.kmz.com> (10.18.16.181) by
CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com> (10.18.17.28) with Microsoft SMTP Server id
14.3.123.3; Wed, 12 Jun 2013 15:44:03 -0500
Received: from venus.kattenlaw.com<http://venus.kattenlaw.com> ([10.18.3.33]) by us.kmz.com<http://us.kmz.com>
([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id 844d8c4f001d4ac4
; Wed, 12 Jun 2013 15:44:01 -0500
Received: from smtp1.eelefs.net<http://smtp1.eelefs.net> (smtp1.eelefs.net<http://smtp1.eelefs.net> [66.197.143.105]) by
venus.kattenlaw.com<http://venus.kattenlaw.com> (8.13.8/8.13.4) with ESMTP id r5CKi0H8028960 for
<brian.duncan at kmzr.com<mailto:brian.duncan at kmzr.com>>; Wed, 12 Jun 2013 15:44:03 -0500
From: 2013 Models <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
To: "Duncan, Brian M." <brian.duncan at kattenlaw.com<mailto:brian.duncan at kattenlaw.com>>
Subject: *Reduction Information* 2013's for thousands less
Thread-Topic: *Reduction Information* 2013's for thousands less
Thread-Index: AQHOZ62T+0z+e2LgwkiBidggfWeC0A==
Date: Wed, 12 Jun 2013 15:43:58 -0500
Message-ID: <29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net<mailto:29295056e3e7741908e644022e5f0220 at smtp1.eelefs.net>>
Reply-To: "Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>" <Jorge.Mendoza at eelefs.net<mailto:Jorge.Mendoza at eelefs.net>>
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Anonymous
X-MS-Exchange-Organization-AuthSource: CHI-US-HT-01.us.kmz.com<http://CHI-US-HT-01.us.kmz.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-mailscanner-from: jorgemendoza at smtp1.eelefs.net<mailto:jorgemendoza at smtp1.eelefs.net>
x-mailscanner-spamcheck: not spam, SpamAssassin (cached, score=0.8, required
6.5, BAYES_50 0.80, LOTS_OF_MONEY 0.00, RP_MATCHES_RCVD -0.00)
x-kattenlaw-mailscanner-information:
x-mailscanner-spam: no
x-kattenlaw-mailscanner-id: r5CKi0H8028960
x-tm-imss-message-id: <844d8c4f001d4ac4 at us.kmz.com<mailto:844d8c4f001d4ac4 at us.kmz.com>>
x-kattenlaw: NS
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com<mailto:8737EB66163E6F4DA060748F2D862AD0 at kattenlaw.com>>
MIME-Version: 1.0
Thanks for any help.
===========================================================
CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue
Service, any tax advice contained herein is not intended or written to be used and cannot be used
by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.
===========================================================
CONFIDENTIALITY NOTICE:
This electronic mail message and any attached files contain information intended for the exclusive
use of the individual or entity to whom it is addressed and may contain information that is
proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you
are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or
distribution of this information may be subject to legal restriction or sanction. Please notify
the sender, by electronic mail or telephone, of any unintended recipients and delete the original
message without making any copies.
===========================================================
NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has
elected to be governed by the Illinois Uniform Partnership Act (1997).
===========================================================
--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/mailman/listinfo/mailscanner
Before posting, read http://wiki.mailscanner.info/posting
Support MailScanner development - buy the book off the website!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130614/413d766f/attachment.html
More information about the MailScanner
mailing list