SA, MS, RBL problem

Wed Jan 24 17:46:28 CET 2007

GENTS!

I have setup rbldnsd daemon, and am rsyncing down from NJABL...

I have made the local DNS cachers forward queries to rbldnsd. That can 
be tested via dig @dnscacheserver 2.0.0.127.dnsbl.njabl.org... i see 
that query get forwarded to rbldnsd, and i see the query in the log so i 
know 100 percent that rsync is working, dns forwarding is working 
correctly from bind to rbldnsd host

so thats out of the question....   MS machine is checking for DNS only 
against that dns cache server... so any queries for njabl should in 
theory forward to my rbldnsd box

now, just so you know, no queries come to rbldnsd box, nothing in the 
logs there

1169650318 172.30.35.65 192.120.70.217.dnsbl.njabl.org A IN: NOERROR/1/323
1169650345 172.30.35.64 192.120.70.217.dnsbl.njabl.org A IN: NOERROR/1/323
1169650365 172.30.35.64 19.120.70.217.dnsbl.njabl.org A IN: NXDOMAIN/0/92
1169650378 172.30.35.64 18.120.70.217.dnsbl.njabl.org A IN: NXDOMAIN/0/92
1169650385 172.30.35.65 11.120.70.217.dnsbl.njabl.org A IN: NXDOMAIN/0/92
1169650407 172.30.35.65 101.192.247.63.dnsbl.njabl.org A IN: NXDOMAIN/0/93

now, what you see there is the log from rbldnsd

regardless if it finds a record, or doesnt find one...  it STILL logs it
so i know for a fact that those queries are not hitting that box

this will show you what happens when I test SA

cat message.test
spamassassin -D < fix.pl > /dev/null

Can someone please help me setup NJABL properly?  I am invoking SA via 
MS... as you can see below, SA does seem to check NJABL...

THANKS!

./message.test
[15101] dbg: logger: adding facilities: all
[15101] dbg: logger: logging level is DBG
[15101] dbg: generic: SpamAssassin version 3.1.7
[15101] dbg: config: score set 0 chosen.
[15101] dbg: util: running in taint mode? yes
[15101] dbg: util: taint mode: deleting unsafe environment variables, 
resetting PATH
[15101] dbg: util: PATH included '/usr/kerberos/sbin', keeping
[15101] dbg: util: PATH included '/usr/kerberos/bin', keeping
[15101] dbg: util: PATH included '/usr/local/bin', keeping
[15101] dbg: util: PATH included '/bin', keeping
[15101] dbg: util: PATH included '/usr/bin', keeping
[15101] dbg: util: PATH included '/usr/X11R6/bin', keeping
[15101] dbg: util: PATH included '/home/vboulytchev/bin', which doesn't 
exist, dropping
[15101] dbg: util: PATH included '/usr/sbin', keeping
[15101] dbg: util: PATH included '/sbin', keeping
[15101] dbg: util: PATH included '/usr/local/apache/bin', which doesn't 
exist, dropping
[15101] dbg: util: PATH included '/usr/local/apache/rsawebagent', which 
doesn't exist, dropping
[15101] dbg: util: final PATH set to: 
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/sbin:/sbin
[15101] dbg: message: ---- MIME PARSER START ----
[15101] dbg: message: main message type: text/plain
[15101] dbg: message: parsing normal part
[15101] dbg: message: added part, type: text/plain
[15101] dbg: message: ---- MIME PARSER END ----
[15101] dbg: dns: is Net::DNS::Resolver available? yes
[15101] dbg: dns: Net::DNS version: 0.59
[15101] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
[15101] dbg: config: read file /etc/mail/spamassassin/init.pre
[15101] dbg: config: read file /etc/mail/spamassassin/v310.pre
[15101] dbg: config: read file /etc/mail/spamassassin/v312.pre
[15101] dbg: config: using "/usr/share/spamassassin" for sys rules pre files
[15101] dbg: config: using "/usr/share/spamassassin" for default rules dir
[15101] dbg: config: read file /usr/share/spamassassin/10_misc.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_advance_fee.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_anti_ratware.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_body_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_compensate.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_drugs.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_head_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_html_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_meta_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_net_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_phrases.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_porn.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_ratware.cf
[15101] dbg: config: read file /usr/share/spamassassin/20_uri_tests.cf
[15101] dbg: config: read file /usr/share/spamassassin/23_bayes.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_accessdb.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_antivirus.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_body_tests_es.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_body_tests_pl.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_dcc.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_dkim.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_domainkeys.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_hashcash.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_pyzor.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_razor2.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_replace.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_spf.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_textcat.cf
[15101] dbg: config: read file /usr/share/spamassassin/25_uribl.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_de.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_fr.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_it.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_nl.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_pl.cf
[15101] dbg: config: read file /usr/share/spamassassin/30_text_pt_br.cf
[15101] dbg: config: read file /usr/share/spamassassin/50_scores.cf
[15101] dbg: config: read file /usr/share/spamassassin/60_awl.cf
[15101] dbg: config: read file /usr/share/spamassassin/60_whitelist.cf
[15101] dbg: config: read file /usr/share/spamassassin/60_whitelist_dk.cf
[15101] dbg: config: read file /usr/share/spamassassin/60_whitelist_dkim.cf
[15101] dbg: config: read file /usr/share/spamassassin/60_whitelist_spf.cf
[15101] dbg: config: read file 
/usr/share/spamassassin/60_whitelist_subject.cf
[15101] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[15101] dbg: config: read file /etc/mail/spamassassin/local.cf
[15101] dbg: config: using "/home/vboulytchev/.spamassassin" for user 
state dir
[15101] dbg: config: using "/home/vboulytchev/.spamassassin/user_prefs" 
for user prefs file
[15101] dbg: config: read file /home/vboulytchev/.spamassassin/user_prefs
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xa660cc0)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::Hashcash=HASH(0xa6ab670)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::SPF=HASH(0xa6cfd94)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::DCC from @INC
[15101] dbg: dcc: network tests on, registering DCC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::DCC=HASH(0xa6ae24c)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
[15101] dbg: pyzor: network tests on, attempting Pyzor
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::Pyzor=HASH(0xa7298a8)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::Razor2 from @INC
[15101] dbg: razor2: razor2 is available, version 2.82
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::Razor2=HASH(0xa731e9c)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC
[15101] dbg: reporter: network tests on, attempting SpamCop
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::SpamCop=HASH(0xac0224c)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::AWL from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::AWL=HASH(0xab6618c)
[15101] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::AutoLearnThreshold from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0xac34d04)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::TextCat from @INC
[15101] dbg: textcat: loading languages file...
[15101] dbg: textcat: loaded 73 language models
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::TextCat=HASH(0xac4e080)
[15101] dbg: plugin: loading 
Mail::SpamAssassin::Plugin::WhiteListSubject from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0xaec0ac8)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from 
@INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0xaec69e4)
[15101] dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags 
from @INC
[15101] dbg: plugin: registered 
Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0xaecff78)
[15101] dbg: config: adding redirector regex: 
/^http:\/\/chkpt\.zdnet\.com\/chkpt\/\w+\/(.*)$/i
[15101] dbg: config: adding redirector regex: 
/^http:\/\/www(?:\d+)?\.nate\.com\/r\/\w+\/(.*)$/i
[15101] dbg: config: adding redirector regex: 
/^http:\/\/.+\.gov\/(?:.*\/)?externalLink\.jhtml\?.*url=(.*?)(?:&.*)?$/i
[15101] dbg: config: adding redirector regex: 
/^http:\/\/redir\.internet\.com\/.+?\/.+?\/(.*)$/i
[15101] dbg: config: adding redirector regex: 
/^http:\/\/(?:.*?\.)?adtech\.de\/.*(?:;|\|)link=(.*?)(?:;|$)/i
[15101] dbg: config: adding redirector regex: 
m'^http.*?/redirect\.php\?.*(?<=[?&])goto=(.*?)(?:$|[&#])'i
[15101] dbg: config: adding redirector regex: 
m'^https?:/*(?:[^/]+\.)?emf\d\.com/r\.cfm.*?&r=(.*)'i
[15101] dbg: config: adding redirector regex: 
m'/(?:index.php)?\?.*(?<=[?&])URL=(.*?)(?:$|[&#])'i
[15101] dbg: config: adding redirector regex: 
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/url\?.*?(?<=[?&])q=(.*?)(?:$|[&#])'i
[15101] dbg: config: adding redirector regex: 
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\s])site:(.*?)(?:$|%20|[\s+&#])'i
[15101] dbg: config: adding redirector regex: 
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\s])(?:"|%22)(.*?)(?:$|%22|["\s+&#])'i
[15101] dbg: config: adding redirector regex: 
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/translate\?.*?(?<=[?&])u=(.*?)(?:$|[&#])'i
[15101] dbg: plugin: 
Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0xaecff78) implements 
'finish_parsing_end'
[15101] dbg: replacetags: replacing tags
[15101] dbg: replacetags: done replacing tags
[15101] dbg: bayes: using username: mailscanner
[15101] dbg: bayes: database connection established
[15101] dbg: bayes: found bayes db version 3
[15101] dbg: bayes: Using userid: 1
[15101] dbg: bayes: not available for scanning, only 0 spam(s) in bayes 
DB < 200
[15101] dbg: config: score set 1 chosen.
[15101] dbg: bayes: database connection established
[15101] dbg: bayes: found bayes db version 3
[15101] dbg: bayes: Using userid: 1
[15101] dbg: bayes: not available for scanning, only 0 spam(s) in bayes 
DB < 200
[15101] dbg: dns: dns_available set to yes in config file, skipping test
[15101] dbg: metadata: X-Spam-Relays-Trusted:
[15101] dbg: metadata: X-Spam-Relays-Untrusted:
[15101] dbg: metadata: X-Spam-Relays-Internal:
[15101] dbg: metadata: X-Spam-Relays-External:
[15101] dbg: plugin: Mail::SpamAssassin::Plugin::TextCat=HASH(0xac4e080) 
implements 'extract_metadata'
[15101] dbg: message: ---- MIME PARSER START ----
[15101] dbg: message: main message type: text/plain
[15101] dbg: message: parsing normal part
[15101] dbg: message: added part, type: text/plain
[15101] dbg: message: ---- MIME PARSER END ----
[15101] dbg: message: no encoding detected
[15101] dbg: textcat: message too short for language analysis
[15101] dbg: textcat: X-Languages: "", X-Languages-Length: 188
[15101] dbg: plugin: 
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xa660cc0) implements 
'parsed_metadata'
[15101] dbg: uridnsbl: domains to query:
[15101] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set 
sblxbl-lastexternal
[15101] dbg: dns: checking RBL sa-accredit.habeas.com., set 
habeas-firsttrusted
[15101] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl
[15101] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
[15101] dbg: dns: checking RBL combined.njabl.org., set njabl-lastexternal
[15101] dbg: dns: checking RBL combined.njabl.org., set njabl
[15101] dbg: dns: checking RBL 
combined-HIB.dnsiplists.completewhois.com., set whois
[15101] dbg: dns: checking RBL list.dsbl.org., set dsbl-lastexternal
[15101] dbg: dns: checking RBL bl.spamcop.net., set spamcop
[15101] dbg: dns: checking RBL sa-trusted.bondedsender.org., set 
bsp-firsttrusted
[15101] dbg: dns: checking RBL 
combined-HIB.dnsiplists.completewhois.com., set whois-lastexternal
[15101] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-lastexternal
[15101] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs
[15101] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted
[15101] dbg: check: running tests for priority: 0
[15101] dbg: rules: running header regexp tests; score so far=0
[15101] dbg: spf: no suitable relay for spf use found, skipping SPF-helo 
check
[15101] dbg: eval: all '*From' addrs:
[15101] dbg: eval: all '*To' addrs:
[15101] dbg: spf: no suitable relay for spf use found, skipping SPF check
[15101] dbg: rules: ran eval rule NO_RELAYS ======> got hit
[15101] dbg: rules: ran eval rule __ENV_AND_HDR_FROM_MATCH ======> got hit
[15101] dbg: spf: cannot get Envelope-From, cannot use SPF
[15101] dbg: spf: def_spf_whitelist_from: could not find useable 
envelope sender
[15101] dbg: rules: ran eval rule __UNUSABLE_MSGID ======> got hit
[15101] dbg: rules: ran eval rule MISSING_HEADERS ======> got hit
[15101] dbg: spf: spf_whitelist_from: could not find useable envelope sender
[15101] dbg: rules: running body-text per-line regexp tests; score so 
far=0.188
[15101] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "o"
[15101] dbg: uri: running uri tests; score so far=0.188
[15101] dbg: bayes: database connection established
[15101] dbg: bayes: found bayes db version 3
[15101] dbg: bayes: Using userid: 1
[15101] dbg: bayes: not available for scanning, only 0 spam(s) in bayes 
DB < 200
[15101] dbg: bayes: not scoring message, returning undef
[15101] dbg: rules: running raw-body-text per-line regexp tests; score 
so far=0.188
[15101] dbg: rules: running full-text regexp tests; score so far=0.188
[15101] dbg: info: entering helper-app run mode
[15101] dbg: info: leaving helper-app run mode
[15101] dbg: razor2: part=0 engine=4 contested=0 confidence=0
[15101] dbg: razor2: results: spam? 0
[15101] dbg: razor2: results: engine 8, highest cf score: 0
[15101] dbg: razor2: results: engine 4, highest cf score: 0
[15101] dbg: pyzor: pyzor is available: /usr/bin/pyzor
[15101] dbg: info: entering helper-app run mode
[15101] dbg: pyzor: opening pipe: /usr/bin/pyzor check < 
/tmp/.spamassassin15101RuRka7tmp
[15102] dbg: util: setuid: ruid=0 euid=0
[15101] dbg: pyzor: [15102] finished: exit=0x0100
[15101] dbg: pyzor: got response: 66.250.40.33:24441 TimeoutError:
[15101] dbg: info: leaving helper-app run mode
[15101] dbg: pyzor: failure to parse response "66.250.40.33:24441 
TimeoutError: "
[15101] dbg: dcc: dccifd is not available: no r/w dccifd socket found
[15101] dbg: dcc: dccproc is available: /usr/local/bin/dccproc
[15101] dbg: info: entering helper-app run mode
[15101] dbg: dcc: opening pipe: /usr/local/bin/dccproc -H -x 0 < 
/tmp/.spamassassin15101RuRka7tmp
[15103] dbg: util: setuid: ruid=0 euid=0
[15101] dbg: dcc: got response: missing SMTP header lines; fatal error
[15101] dbg: info: leaving helper-app run mode
[15101] dbg: dcc: check failed: no X-DCC returned (did you create a map 
file?): missing SMTP header lines; fatal error
[15101] dbg: plugin: 
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xa660cc0) implements 'check_tick'
[15101] dbg: check: running tests for priority: 500
[15101] dbg: plugin: 
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0xa660cc0) implements 
'check_post_dnsbl'
[15101] dbg: rules: running meta tests; score so far=0.188
[15101] dbg: rules: running header regexp tests; score so far=1.666
[15101] dbg: rules: running body-text per-line regexp tests; score so 
far=1.666
[15101] dbg: uri: running uri tests; score so far=1.666
[15101] dbg: rules: running raw-body-text per-line regexp tests; score 
so far=1.666
[15101] dbg: rules: running full-text regexp tests; score so far=1.666
[15101] dbg: check: running tests for priority: 1000
[15101] dbg: rules: running meta tests; score so far=1.666
[15101] dbg: rules: running header regexp tests; score so far=1.666
[15101] dbg: rules: running body-text per-line regexp tests; score so 
far=1.666
[15101] dbg: uri: running uri tests; score so far=1.666
[15101] dbg: rules: running raw-body-text per-line regexp tests; score 
so far=1.666
[15101] dbg: rules: running full-text regexp tests; score so far=1.666
[15101] dbg: plugin: 
Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0xac34d04) 
implements 'autolearn_discriminator'
[15101] dbg: learn: auto-learn: currently using scoreset 1
[15101] dbg: learn: auto-learn: message score: 1.666, computed score for 
autolearn: 1.668
[15101] dbg: learn: auto-learn? ham=0.1, spam=12, body-points=0, 
head-points=0.189, learned-points=0
[15101] dbg: learn: auto-learn? no: inside auto-learn thresholds, not 
considered ham or spam
[15101] dbg: check: is spam? score=1.666 required=5
[15101] dbg: check: 
tests=MISSING_HEADERS,MISSING_SUBJECT,NO_RECEIVED,NO_RELAYS,TO_CC_NONE
[15101] dbg: check: 
subtests=__ENV_AND_HDR_FROM_MATCH,__NONEMPTY_BODY,__UNUSABLE_MSGID

Vasiliy Boulytchev
vasiliy at linuxspecial.com