Certain Spamassassin rules do not seem to be firing all of the time
Martin Hepworth
maxsec at gmail.com
Tue Jun 18 16:09:22 IST 2013
really odd, seems to be suffering with network based rules, not just the
URI ones but razor as well.
Personally I always put all the RBL checks into SA rather than letting
MailScanner do it by itself. that way no 1 rbl can false postive and email
and the RBL just add to the overall score.
What MTA are you running? and is there a .spamassassin directory in root's
home dir?
--
Martin Hepworth, CISSP
Oxford, UK
On 15 June 2013 21:08, Duncan, Brian M. <brian.duncan at kattenlaw.com> wrote:
> Thanks for the recommendations Martin.****
>
> ** **
>
> The way I have it setup in Mailscanner is if the sending mail server is on
> a RBL, (If it on at least 1 of the 4 RBLS we use with MailScanner) It
> becomes high scoring spam and is tagged and moved on, it does not get
> scanned by Spamassassin then.****
>
> ** **
>
> The one thing I never considered before was if Spamassassin is scanning
> the same sending mail server IP for being listed when it does not get
> caught by MailScanner as being on any of the 4 RB:’s I use. Not that it
> is causing my problem now, but that it is not very efficient if it is doing
> it again. (I would guess 80% of my mail never gets scanned by SpamAssassin
> each day because the sending mail gateway is blacklisted and it is marked
> as Spam and moves on)****
>
> ** **
>
> When you say: “ I normally remove most of the RBL's from being scanned in
> Spamassassin by giving most of them a zero score (see 50_scores.cf<http://spamassassin.apache.org/dist/rules/50_scores.cf>in the DNSEval section).”
> ****
>
> ** **
>
> I don’t think I follow, are you saying Spamassassin is scanning the
> sending mail host again against the RBL’s? So by giving them a zero score
> you are avoiding the double effort? This section has nothing to do with the
> URL/URI scanning that is happening? I assumed the rules that I have that
> are NOT hitting when it goes through MailScanner/Spamassassin have all been
> based on the URI/URL’s in the body of the message. ****
>
> ** **
>
> ** **
>
> When I take these specific Spam messages that make it into my inbox, I am
> noticing they never hit on the same URIBL hits I get when I move the
> message locally to the box, if I take one of these URI based RBL checking
> rules like for example URIBL_BLACK, I have never seen that rule hit on ANY
> of these ones making it into my inbox. If I search my maillog from
> yesterday for every message that wound up being scanned by Spamassassin, I
> see that there were 1014 times that rule is listed on detected Spam. ****
>
> ** **
>
> Last night I first tried setting up a caching bind server local to the
> box. Made no difference.****
>
> ** **
>
> I tried upgrading to MailScanner 4.84.5-3 after and updating to
> SpamAssassin 3.3.2-1 to see if that would make a difference, I even looked
> at the Perl modules that come with MailScanner, one of them was
> perl-Net-DNS-0.65-2, I was running****
>
> perl-Net-DNS-0.65-1, was hoping that had something to do with this so I
> updated to .65-2 of that perl modules.. the rest all seemed to be the same
> version that comes with 4.84.5-3 (I was running 4.83.5-1 before I updated)
> ****
>
> ** **
>
> I went over all my configs for both MailScanner and SpamAssassin, nothing
> seems wrong or set to low that would create the situation I am seeing. I
> did find I had the pyzor plugin loading in SpamAssassin but no exe, so I
> just disabled pyzor and verified in the –debug-sa that everything looks
> fine.****
>
> ** **
>
> I waited and sure enough it happened again today. We get less mail on the
> weekends so it took awhile waiting..****
>
> ** **
>
> I have posted my MailScanner –debug-sa to pastebin if anyone can take a
> look and give me a recommendation of where to look next. I am almost out
> of things to try.****
>
> ** **
>
> Here is one –debug-sa:****
>
> ** **
>
> http://pastebin.com/C2XPs7D2****
>
> ** **
>
> Then I kept running with –debug-sa till it caught one with a DNS based
> rule like URIBL_* rules.****
>
> ** **
>
> This one hits on those URIBL rules that are DNS based and it looks like
> everything is OK as far as I can tell.. This is really the first time I
> have tried to debug a debug log from MailScanner/Spamassassin before..****
>
> ** **
>
> http://pastebin.com/iWMnJqf3****
>
> ** **
>
> ** **
>
> Thanks****
>
> ** **
>
> ** **
>
> ** **
>
> BRIAN M. DUNCAN
> Data Security Administrator
> Katten Muchin Rosenman LLP
> 525 W. Monroe Street / Chicago, IL 60661-3693
> p / (312) 577-8045 f / (312) 577-4490
> brian.duncan at kattenlaw.com / www.kattenlaw.com
> ****
>
> ** **
>
> *From:* mailscanner-bounces at lists.mailscanner.info [mailto:
> mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
> *Sent:* Saturday, June 15, 2013 8:46 AM
> *To:* MailScanner discussion
> *Subject:* Re: Certain Spamassassin rules do not seem to be firing all of
> the time****
>
> ** **
>
> Ok, you really need to put a local DNS server on the MailScanner box,
> doesn't matter if the DNS resolver is next to the server in the switch
> port, DNS is actually quite heavy on network traffic and hitting this all
> the time makes a huge difference. It can forward to the current machine,
> but the time this saves is actually quite noticable.****
>
> that three seconds for the pass across seems very quick to me, esp as it's
> got all the DNS requests to process.I normally remove most of the RBL's
> from being scanned in Spamassassin by giving most of them a zero score (see
> 50_scores.cf <http://spamassassin.apache.org/dist/rules/50_scores.cf> in
> the DNSEval section). also make sure you're updating sa rules regularly. In
> fact its almost as if you've got skip-rbl-checks set to 1 in a spamassassin
> .cf or mailscanner,conf file somewhere.****
>
> I'd double check all the setup to make sure everythings OK, as it's really
> odd that you're getting DNS based hits in test mode but not in test mode.
> Check the MailScanner.conf setttings and any site MailScanner.conf, and
> also get rid of any .spamassassin dirs esp if there's anything in root's
> home dir (so i presume ther MTA is sendmail?) to make sure that isnt
> overriding any settings. Check you've got one MailScanner.conf and not
> multiple ones, sometimes some distributions put the active file in
> 'non-standard' places.
>
> ****
>
>
> ****
>
> --
> Martin Hepworth, CISSP
> Oxford, UK****
>
> ** **
>
> On 15 June 2013 03:22, Duncan, Brian M. <brian.duncan at kattenlaw.com>
> wrote:****
>
> Thanks, yes I noticed that, they all do seem to be the DNS rules. I do
> have a caching DNS server but it is on the local network. I will try and
> see if the behavior changes at all by running one locally on the box itself.
> ****
>
> ****
>
> When you say “that youre not timing out the network checks in sa too
> quickly” I have not changed anything in the defaults of Mailscanner or
> included any directives that would lower whatever time limits are set by
> default.****
>
> ****
>
> I took a look at the last example I put on pastebin, and it looks like it
> took 3 seconds to go from my Mailscanner box to my next gateway. ****
>
> ****
>
> Received: from venus.kattenlaw.com ([10.18.3.33]) by us.kmz.com****
>
> ([10.18.16.181]) with ESMTP (TREND IMSS SMTP Service 7.1) id
> 8e3c2381002025b2****
>
> ; Fri, 14 Jun 2013 14:01:09 -0500****
>
> Received: from a.loselit.net (a.loselit.net [66.96.254.156]) by****
>
> venus.kattenlaw.com (8.13.8/8.13.4) with ESMTP id r5EJ13oK014449
> for****
>
> <brian.duncan at kmzr.com>; Fri, 14 Jun 2013 14:01:06 -0500****
>
> ****
>
> I am assuming the 3 seconds going from my incoming mail server Venus, to
> the next hop in my environment includes the time it took for the Spammer to
> send me the message.****
>
> ****
>
> I also don’t see anything in my maillogs related to Spam Assassin timing
> out for anything.. I recall many years ago when we used to run systems with
> much less CPU power (10+) seeing Spam Assassin time outs.****
>
> ****
>
> Which BTW, at the peak of activity today the lowest idle %idle was 91.00
> and that is because I turned off caching of SpamAssassin in Mailscanner to
> see if that had any impact.****
>
> ****
>
> I also looked at the local caching DNS server that is on the same switch
> as this box, and it was peaking at like 30 Kilobytes per second on UDP 53
> requests from anything that uses it locally according to iptraf.****
>
> ****
>
> It also seems to be these messages from the same Spammer, as I said before
> if I take any of these message bodies and send them in myself I seem to get
> the DNS Spam Assassin hits then. ****
>
> ****
>
> Really odd one..****
>
> ****
>
> Thanks for your help****
>
> ****
>
> ****
>
> ****
>
> BRIAN M. DUNCAN
> Data Security Administrator
> Katten Muchin Rosenman LLP
> 525 W. Monroe Street / Chicago, IL 60661-3693
> p / (312) 577-8045 f / (312) 577-4490
> brian.duncan at kattenlaw.com / www.kattenlaw.com
> ****
>
> ****
>
> *From:* mailscanner-bounces at lists.mailscanner.info [mailto:
> mailscanner-bounces at lists.mailscanner.info] *On Behalf Of *Martin Hepworth
> *Sent:* Friday, June 14, 2013 4:16 PM
> *To:* MailScanner discussion
> *Subject:* Certain Spamassassin rules do not seem to be firing all of the
> time****
>
> ****
>
> Hmm most if the extra rules youre hitting are dns based****
>
> I'd check youre running a local caching dns server on the scanning box and
> that youre not timing out the network checks in sa too quickly****
>
> ****
>
> Martin
>
> On Friday, 14 June 2013, Duncan, Brian M. wrote:****
>
> Here is one more that just came in to me and was not tagged as Spam:****
>
> ****
>
> http://pastebin.com/w8SJk660****
>
> ****
>
> ****
>
> Mailscanner/Spamassassin results:****
>
> ****
>
> X-MailScanner-SpamCheck: not spam, SpamAssassin (score=2.999, required 6.5,
> ****
>
> BAYES_60 3.00, RP_MATCHES_RCVD -0.00)****
>
> ****
>
> ****
>
> --test-mode results:****
>
> ****
>
> Content analysis details: (10.5 hits, 6.5 required)****
>
> 6.6 BAYES_99 BODY: Bayes spam probability is 99 to 100%****
>
> [score: 1.0000]****
>
> -0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay
> domain****
>
> 2.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8 confidence level****
>
> above 50%****
>
> [cf: 100]****
>
> 8.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)****
>
> 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%****
>
> [cf: 100]****
>
> -7.5 AWL AWL: From: address is in the auto white-list**
> **
>
> ****
>
> ------ End of SpamAssassin results, Original message follows --------****
>
> ****
>
> ===========================================================****
>
> CIRCULAR 230 DISCLOSURE: Pursuant to Regulations Governing Practice Before the Internal Revenue****
>
> Service, any tax advice contained herein is not intended or written to be used and cannot be used****
>
> by a taxpayer for the purpose of avoiding tax penalties that may be imposed on the taxpayer.****
>
> ===========================================================****
>
> CONFIDENTIALITY NOTICE:****
>
> This electronic mail message and any attached files contain information intended for the exclusive****
>
> use of the individual or entity to whom it is addressed and may contain information that is****
>
> proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you****
>
> are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or ****
>
> distribution of this information may be subject to legal restriction or sanction. Please notify****
>
> the sender, by electronic mail or telephone, of any unintended recipients and delete the original ****
>
> message without making any copies.****
>
> ===========================================================****
>
> NOTIFICATION: Katten Muchin Rosenman LLP is an Illinois limited liability partnership that has****
>
> elected to be governed by the Illinois Uniform Partnership Act (1997).****
>
> ===========================================================****
>
>
>
> --
> --
> Martin Hepworth, CISSP
> Oxford, UK****
>
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!****
>
> ** **
>
> --
> MailScanner mailing list
> mailscanner at lists.mailscanner.info
> http://lists.mailscanner.info/mailman/listinfo/mailscanner
>
> Before posting, read http://wiki.mailscanner.info/posting
>
> Support MailScanner development - buy the book off the website!
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20130618/31a09fe7/attachment.html
More information about the MailScanner
mailing list