Problem with MS on OpenBSD 4.1

Christoph Schneeberger cschnee at box.telemedia.ch
Wed Jul 4 09:08:23 IST 2007 

Glenn Steen wrote:
> On 03/07/07, Christoph Schneeberger <cschnee at box.telemedia.ch> wrote:
>> Hi Julian,
>>
>> Thanks for your reply.
>>
>> Julian Field wrote:
>> ..
>>
>> > >When I run MS and SA in debug mode I get an error at line 832 in
>> > >Mailscanner which is the following line:
>> >
>> > >$batch->Explode();
>> >
>> > >The error I get is
>> >
>> > ><loads of debug deleted>
>> > >[24820] dbg: locker: safe_unlock: unlocked
>> > /root/.spamassassin/bayes.mutex
>> > >[24820] dbg: learn: initializing learner
>> > >Ignore errors about failing to find EOCD signature
>> >
>> >
>> > That line gives a hint.
>> >
>> > >format error: can't find EOCD signature
>> > > at /opt/MailScanner/bin/MailScanner line 832
>> >
>> >
>> > So you can ignore that.
>> >
>> > >Stopping now as you are debugging me.
>> > > Done.
>> >
>> >
>> > It has run to completion normally. It hasn't bombed out on an error at
>> > all. It has done exactly what it is supposed to do in Debug mode:
>> > process 1 batch of messages and then exit.
>>
>>
>> Ok thanks, i was thinking that too, but somebody on irc told me i need
>> to get rid of this line832 error and that would solve my problem of not
>> having any detailed Spamassassin result headers at all.
>>
>> So could you give me any direction or hints where I could further search
>> to get that problem of not having detailed results in the header solved,
>> since thats the only problem I really have.
>>
>> Or asked else: Is anybody on this list running a current MailScanner on
>> OpenBSD 4.1 successfully and do you have any hints for me where too
>> look ?
>>
>> Thanks a lot and best regards,
>> Christoph
>>
> OpenBSD isn]t exactly unheard of, but it certainly isn\t one of the
> more used OSes.... But this might not be anything specific to your
> OS... Call me dull, but did you run a
> MailScanner --debug --debug-sa
> ... with something obvious, like a GTUBE, on queue?
>
> Cheers

Thanks for your reply, I had to google GTUBE before I knew what you
meant ;-)

I have run with debug and debug-sa (from MailScanner.conf but I guess
that is the same result) and people on irc told me the output i pasted
looks good, but I'll happily provide it at the end of this mail.

I have sent a mail (only body with copy-paste) from my inbox that is
spam through the MS in question and it scored it with 18 and flagged the
Subject properly with {Spam?}, BUT only the Spamscore and the Subject
Flag are here, no details on which tests how many score was given, just
a result.

The reason I want OpenBSD is the spamd/pf combo which is quite unique
for greylisting and since 4.1 even more use- and powerful. Also I have
used OpenBSD since 2.5 and I am quite satisfied with its robustness and
safety. Another reason is that I am a sendmail veteran because there was
nothing else really serious at the time i needed my first mailhub, so I
read different sendmails books and am quite comfortable with it. I've
never found my way into other MTAs and since most Linux are now shipping
Postfix, Exim or whatever I am sticking with OpenBSD for Mailhubs,
Mailgates etc.

So here is the output of
./bin/MailScanner --debug --debug-sa 2>&1 | tee /tmp/log
---

Currently you are using no virus scanners.
This is probably not what you want.

In your /opt/MailScanner/etc/MailScanner.conf file, set
    Virus Scanners = clamav
Then download
   
http://www.sng.ecs.soton.ac.uk/mailscanner/files/4/install-Clam-SA.tar.gz
Unpack it, "cd" into the directory and run ./install.sh

In Debugging mode, not forking...
SpamAssassin temp dir = /var/spool/MailScanner/incoming/SpamAssassin-Temp
[14467] dbg: logger: adding facilities: all
[14467] dbg: logger: logging level is DBG
[14467] dbg: generic: SpamAssassin version 3.1.8
[14467] dbg: config: score set 0 chosen.
[14467] dbg: util: running in taint mode? no
[14467] dbg: message: ---- MIME PARSER START ----
[14467] dbg: message: main message type: text/plain
[14467] dbg: message: parsing normal part
[14467] dbg: message: added part, type: text/plain
[14467] dbg: message: ---- MIME PARSER END ----
[14467] dbg: dns: is Net::DNS::Resolver available? yes
[14467] dbg: dns: Net::DNS version: 0.59
[14467] dbg: ignore: test message to precompile patterns and load modules
[14467] dbg: config: using "/etc/mail/spamassassin" for site rules pre files
[14467] dbg: config: read file /etc/mail/spamassassin/init.pre
[14467] dbg: config: read file /etc/mail/spamassassin/v310.pre
[14467] dbg: config: read file /etc/mail/spamassassin/v312.pre
[14467] dbg: config: using "/usr/local/share/spamassassin" for sys rules
pre files
[14467] dbg: config: using "/usr/local/share/spamassassin" for default
rules dir
[14467] dbg: config: read file /usr/local/share/spamassassin/10_misc.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_advance_fee.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_anti_ratware.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_body_tests.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_compensate.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_dnsbl_tests.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_drugs.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_fake_helo_tests.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_head_tests.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_html_tests.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/20_meta_tests.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_net_tests.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_phrases.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_porn.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_ratware.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/20_uri_tests.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/23_bayes.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_accessdb.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_antivirus.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/25_body_tests_es.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/25_body_tests_pl.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_dcc.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_dkim.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/25_domainkeys.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_hashcash.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_pyzor.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_razor2.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_replace.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_spf.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_textcat.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/25_uribl.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/30_text_de.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/30_text_fr.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/30_text_it.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/30_text_nl.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/30_text_pl.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/30_text_pt_br.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/50_scores.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/60_awl.cf
[14467] dbg: config: read file /usr/local/share/spamassassin/60_whitelist.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/60_whitelist_dk.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/60_whitelist_dkim.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/60_whitelist_spf.cf
[14467] dbg: config: read file
/usr/local/share/spamassassin/60_whitelist_subject.cf
[14467] dbg: config: using "/etc/mail/spamassassin" for site rules dir
[14467] dbg: config: read file /etc/mail/spamassassin/local.cf
[14467] dbg: config: read file /etc/mail/spamassassin/mailscanner.cf
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x4a1cbc80)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::Hashcash=HASH(0x4390e3a0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::SPF=HASH(0x4c4dc6a0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
[14467] dbg: pyzor: network tests on, attempting Pyzor
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::Pyzor=HASH(0x4b99feb0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::Razor2 from @INC
[14467] dbg: razor2: razor2 is available, version 2.82
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::Razor2=HASH(0x46e71a90)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC
[14467] dbg: reporter: network tests on, attempting SpamCop
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::SpamCop=HASH(0x4ac42bb0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::AWL from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::AWL=HASH(0x441d61c0)
[14467] dbg: plugin: loading
Mail::SpamAssassin::Plugin::AutoLearnThreshold from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::AutoLearnThreshold=HASH(0x47fce710)
[14467] dbg: plugin: loading
Mail::SpamAssassin::Plugin::WhiteListSubject from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::WhiteListSubject=HASH(0x500524c0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from
@INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::MIMEHeader=HASH(0x4caf65a0)
[14467] dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags
from @INC
[14467] dbg: plugin: registered
Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x487f0b70)
[14467] dbg: config: adding redirector regex:
/^http:\/\/chkpt\.zdnet\.com\/chkpt\/\w+\/(.*)$/i
[14467] dbg: config: adding redirector regex:
/^http:\/\/www(?:\d+)?\.nate\.com\/r\/\w+\/(.*)$/i
[14467] dbg: config: adding redirector regex:
/^http:\/\/.+\.gov\/(?:.*\/)?externalLink\.jhtml\?.*url=(.*?)(?:&.*)?$/i
[14467] dbg: config: adding redirector regex:
/^http:\/\/redir\.internet\.com\/.+?\/.+?\/(.*)$/i
[14467] dbg: config: adding redirector regex:
/^http:\/\/(?:.*?\.)?adtech\.de\/.*(?:;|\|)link=(.*?)(?:;|$)/i
[14467] dbg: config: adding redirector regex:
m'^http.*?/redirect\.php\?.*(?<=[?&])goto=(.*?)(?:$|[&#])'i
[14467] dbg: config: adding redirector regex:
m'^https?:/*(?:[^/]+\.)?emf\d\.com/r\.cfm.*?&r=(.*)'i
[14467] dbg: config: adding redirector regex:
m'/(?:index.php)?\?.*(?<=[?&])URL=(.*?)(?:$|[&#])'i
[14467] dbg: config: adding redirector regex:
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/url\?.*?(?<=[?&])q=(.*?)(?:$|[&#])'i
[14467] dbg: config: adding redirector regex:
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\s])site:(.*?)(?:$|%20|[\s+&#])'i
[14467] dbg: config: adding redirector regex:
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/search\?.*?(?<=[?&])q=[^&]*?(?<=%20|..[=+\s])(?:"|%22)(.*?)(?:$|%22|["\s+&#])'i
[14467] dbg: config: adding redirector regex:
m'^http:/*(?:\w+\.)?google(?:\.\w{2,3}){1,2}/translate\?.*?(?<=[?&])u=(.*?)(?:$|[&#])'i
[14467] dbg: plugin:
Mail::SpamAssassin::Plugin::ReplaceTags=HASH(0x487f0b70) implements
'finish_parsing_end'
[14467] dbg: replacetags: replacing tags
[14467] dbg: replacetags: done replacing tags
[14467] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks
[14467] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen
[14467] dbg: bayes: found bayes db version 3
[14467] dbg: bayes: DB journal sync: last sync: 0
[14467] dbg: bayes: not available for scanning, only 0 spam(s) in bayes
DB < 200
[14467] dbg: bayes: untie-ing
[14467] dbg: bayes: untie-ing db_toks
[14467] dbg: bayes: untie-ing db_seen
[14467] dbg: config: score set 1 chosen.
[14467] dbg: message: ---- MIME PARSER START ----
[14467] dbg: message: main message type: text/plain
[14467] dbg: message: parsing normal part
[14467] dbg: message: added part, type: text/plain
[14467] dbg: message: ---- MIME PARSER END ----
[14467] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_toks
[14467] dbg: bayes: tie-ing to DB file R/O /root/.spamassassin/bayes_seen
[14467] dbg: bayes: found bayes db version 3
[14467] dbg: bayes: DB journal sync: last sync: 0
[14467] dbg: bayes: not available for scanning, only 0 spam(s) in bayes
DB < 200
[14467] dbg: bayes: untie-ing
[14467] dbg: bayes: untie-ing db_toks
[14467] dbg: bayes: untie-ing db_seen
[14467] dbg: dns: dns_available set to yes in config file, skipping test
[14467] dbg: metadata: X-Spam-Relays-Trusted:
[14467] dbg: metadata: X-Spam-Relays-Untrusted:
[14467] dbg: metadata: X-Spam-Relays-Internal:
[14467] dbg: metadata: X-Spam-Relays-External:
[14467] dbg: message: no encoding detected
[14467] dbg: plugin:
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x4a1cbc80) implements
'parsed_metadata'
[14467] dbg: uridnsbl: domains to query:
[14467] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set
sblxbl-lastexternal
[14467] dbg: dns: checking RBL sa-accredit.habeas.com., set
habeas-firsttrusted
[14467] dbg: dns: checking RBL sbl-xbl.spamhaus.org., set sblxbl
[14467] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
[14467] dbg: dns: checking RBL combined.njabl.org., set njabl-lastexternal
[14467] dbg: dns: checking RBL combined.njabl.org., set njabl
[14467] dbg: dns: checking RBL
combined-HIB.dnsiplists.completewhois.com., set whois
[14467] dbg: dns: checking RBL list.dsbl.org., set dsbl-lastexternal
[14467] dbg: dns: checking RBL bl.spamcop.net., set spamcop
[14467] dbg: dns: checking RBL sa-trusted.bondedsender.org., set
bsp-firsttrusted
[14467] dbg: dns: checking RBL
combined-HIB.dnsiplists.completewhois.com., set whois-lastexternal
[14467] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-lastexternal
[14467] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs
[14467] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted
[14467] dbg: check: running tests for priority: 0
[14467] dbg: rules: running header regexp tests; score so far=0
[14467] dbg: rules: ran header rule __HAS_MSGID ======> got hit: "<"
[14467] dbg: rules: ran header rule __MSGID_OK_DIGITS ======> got hit:
"1183536114"
[14467] dbg: rules: ran header rule __SANE_MSGID ======> got hit:
"<1183536114.8107 at spamassassin_spamd_init>
[14467] dbg: rules: "
[14467] dbg: rules: ran header rule NO_REAL_NAME ======> got hit:
"ignore at compiling.spamassassin.taint.org
[14467] dbg: rules: "
[14467] dbg: rules: ran header rule __MSGID_OK_HOST ======> got hit:
"@spamassassin_spamd_init>"
[14467] dbg: spf: no suitable relay for spf use found, skipping SPF-helo
check
[14467] dbg: eval: all '*From' addrs:
ignore at compiling.spamassassin.taint.org
[14467] dbg: eval: all '*To' addrs:
[14467] dbg: spf: no suitable relay for spf use found, skipping SPF check
[14467] dbg: rules: ran eval rule NO_RELAYS ======> got hit
[14467] dbg: spf: cannot get Envelope-From, cannot use SPF
[14467] dbg: spf: def_spf_whitelist_from: could not find useable
envelope sender
[14467] dbg: rules: ran eval rule __UNUSABLE_MSGID ======> got hit
[14467] dbg: rules: ran eval rule MISSING_HEADERS ======> got hit
[14467] dbg: spf: spf_whitelist_from: could not find useable envelope sender
[14467] dbg: rules: running body-text per-line regexp tests; score so
far=0.738
[14467] dbg: rules: ran body rule __NONEMPTY_BODY ======> got hit: "I"
[14467] dbg: uri: running uri tests; score so far=0.738
[14467] dbg: rules: running raw-body-text per-line regexp tests; score
so far=0.738
[14467] dbg: rules: running full-text regexp tests; score so far=0.738
[14467] dbg: info: entering helper-app run mode
[14467] dbg: info: leaving helper-app run mode
[14467] dbg: razor2: part=0 engine=4 contested=0 confidence=0
[14467] dbg: razor2: results: spam? 0
[14467] dbg: razor2: results: engine 8, highest cf score: 0
[14467] dbg: razor2: results: engine 4, highest cf score: 0
[14467] dbg: util: current PATH is: /sbin:/bin:/usr/sbin:/usr/bin
[14467] dbg: pyzor: pyzor is not available: no pyzor executable found
[14467] dbg: pyzor: no pyzor found, disabling Pyzor
[14467] dbg: plugin:
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x4a1cbc80) implements
'check_tick'
[14467] dbg: check: running tests for priority: 500
[14467] dbg: plugin:
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x4a1cbc80) implements
'check_post_dnsbl'
[14467] dbg: rules: running meta tests; score so far=0.738
[14467] info: rules: meta test DIGEST_MULTIPLE has undefined dependency
'DCC_CHECK'
[14467] dbg: rules: running header regexp tests; score so far=2.216
[14467] dbg: rules: running body-text per-line regexp tests; score so
far=2.216
[14467] dbg: uri: running uri tests; score so far=2.216
[14467] dbg: rules: running raw-body-text per-line regexp tests; score
so far=2.216
[14467] dbg: rules: running full-text regexp tests; score so far=2.216
[14467] dbg: check: running tests for priority: 1000
[14467] dbg: rules: running meta tests; score so far=2.216
[14467] dbg: rules: running header regexp tests; score so far=2.216
[14467] dbg: rules: running body-text per-line regexp tests; score so
far=2.216
[14467] dbg: uri: running uri tests; score so far=2.216
[14467] dbg: rules: running raw-body-text per-line regexp tests; score
so far=2.216
[14467] dbg: rules: running full-text regexp tests; score so far=2.216
[14467] dbg: check: is spam? score=2.216 required=5
[14467] dbg: check:
tests=MISSING_HEADERS,MISSING_SUBJECT,NO_REAL_NAME,NO_RECEIVED,NO_RELAYS,TO_CC_NONE
[14467] dbg: check:
subtests=__HAS_MSGID,__MSGID_OK_DIGITS,__MSGID_OK_HOST,__NONEMPTY_BODY,__SANE_MSGID,__UNUSABLE_MSGID
Ignore errors about failing to find EOCD signature
format error: can't find EOCD signature
 at ./bin/MailScanner line 832
Stopping now as you are debugging me.
---

Please not I have disabled clamav for the moment to debug this without
having to care about possible problems with the Antivirus, however the
result was the same (no spamassassin details) when having the antivirus
set to 'clamav' or 'clamd'.

Also I have installed clamav and SA from local ports in OpenBSD 4.1, but
maybe I should use the provided package from the MS site ? Would that be
worth a try ?

Thanks for any hints or tips in advance.


Cheers,
Christoph

-- 
     ---------------------------------------------------+
    / Christoph Schneeberger    /  SCS TeleMedia AG     |
   / GIAC GSEC                 / Liestalerstrasse 47    |
  / cschnee at telemedia.ch      / info at telemedia.ch       |
 / 4419 Lupsingen            / http://www.telemedia.ch  |
/ tel +41 61 915 9155       / fax +41 61 911 0714       |
--------------------------------------------------------+ 

This e-mail is confidential and may be privileged. It may 
be read, copied and used only by the addressee. If you 
have received it in error, please contact us immediately.


          "Quis custodiet ipsos custodes?"

More information about the MailScanner mailing list