Mailscanner - Spamassassin Issue
Toby
toby at rosecott.net
Fri Jun 8 07:19:19 UTC 2018
Hi,
After a bit more digging i have done an ExamDiff on the two logs
produced by a successful scan and a failed scan. There appears to be a
block of differences, they continue the same until the failed one gives
a "finished parsing", at which point the successfull seems to give a
much more detailed report. I have highlighted the differences and
removed the vast amount of differences leading up to "Finished parsing"
(lots of fix path, add tld list, using file statements).
the good scan give: -
Good one
dbg: logger: adding facilities: all
dbg: logger: logging level is DBG
dbg: generic: SpamAssassin version 3.4.1
dbg: generic: Perl 5.020002, PREFIX=/usr/local,
DEF_RULES_DIR=/usr/local/share/spamassassin,
LOCAL_RULES_DIR=/etc/mail/spamassassin,
LOCAL_STATE_DIR=/var/lib/spamassassin
dbg: config: timing enabled
dbg: config: score set 0 chosen.
dbg: util: running in taint mode? yes
dbg: util: taint mode: deleting unsafe environment variables, resetting
PATH
dbg: util: PATH included '/usr/local/sbin', keeping
dbg: util: PATH included '/usr/local/bin', keeping
dbg: util: PATH included '/usr/sbin', keeping
dbg: util: PATH included '/usr/bin', keeping
dbg: util: PATH included '/sbin', keeping
dbg: util: PATH included '/bin', keeping
dbg: util: final PATH set to:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
dbg: archive-iterator: _set_default_message_selection_opts After:
Scanprob[1 , want_date[0 , cache[0 , from_regex[^From \S+
?(\S\S\S \S\S\S .\d .\d:\d\d:\d\d \d{4}|.\d-\d\d-\d{4}_\d\d:\d\d:\d\d_)
dbg: config: using /etc/mail/spamassassin for site rules pre files
dbg: config: read file /etc/mail/spamassassin/init.pre
dbg: config: read file /etc/mail/spamassassin/v310.pre
dbg: config: read file /etc/mail/spamassassin/v312.pre
dbg: config: read file /etc/mail/spamassassin/v320.pre
dbg: config: read file /etc/mail/spamassassin/v330.pre
dbg: config: read file /etc/mail/spamassassin/v340.pre
dbg: config: read file /etc/mail/spamassassin/v341.pre
dbg: config: using /var/lib/spamassassin/3.004001 for sys rules pre
files
dbg: config: using /var/lib/spamassassin/3.004001 for default rules dir
dbg: config: read file
/var/lib/spamassassin/3.004001/updates_spamassassin_org.cf
dbg: config: using /etc/mail/spamassassin for site rules dir
****The successful scan gives loads of information here, that does not
exist in the failed scan *****
dbg: config: finish parsing
The failed one gives
Bad One
dbg: logger: adding facilities: all
dbg: logger: logging level is DBG
dbg: generic: SpamAssassin version 3.4.1
dbg: generic: Perl 5.020002, PREFIX=/usr/local,
DEF_RULES_DIR=/usr/local/share/spamassassin,
LOCAL_RULES_DIR=/etc/mail/spamassassin,
LOCAL_STATE_DIR=/var/lib/spamassassin
dbg: config: timing enabled
dbg: config: score set 0 chosen.
dbg: util: running in taint mode? yes
dbg: util: taint mode: deleting unsafe environment variables, resetting
PATH
dbg: util: PATH included '/usr/local/sbin', keeping
dbg: util: PATH included '/usr/local/bin', keeping
dbg: util: PATH included '/usr/sbin', keeping
dbg: util: PATH included '/usr/bin', keeping
dbg: util: PATH included '/sbin', keeping
dbg: util: PATH included '/bin', keeping
dbg: util: final PATH set to:
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
dbg: archive-iterator: _set_default_message_selection_opts After:
Scanprob[1 , want_date[0 , cache[0 , from_regex[^From \S+
?(\S\S\S \S\S\S .\d .\d:\d\d:\d\d \d{4}|.\d-\d\d-\d{4}_\d\d:\d\d:\d\d_)
dbg: config: using /etc/mail/spamassassin for site rules pre files
dbg: config: read file /etc/mail/spamassassin/init.pre
dbg: config: read file /etc/mail/spamassassin/v310.pre
dbg: config: read file /etc/mail/spamassassin/v312.pre
dbg: config: read file /etc/mail/spamassassin/v320.pre
dbg: config: read file /etc/mail/spamassassin/v330.pre
dbg: config: read file /etc/mail/spamassassin/v340.pre
dbg: config: read file /etc/mail/spamassassin/v341.pre
dbg: config: using /etc/MailScanner/spamassassin.conf for sys rules pre
files
dbg: config: read file /etc/MailScanner/spamassassin.conf
dbg: config: using /etc/MailScanner/spamassassin.conf for default rules
dir
dbg: config: read file /etc/MailScanner/spamassassin.conf
dbg: config: using /etc/mail/spamassassin for site rules dir
dbg: config: read file /etc/mail/spamassassin/MailScanner.cf
dbg: config: read file /etc/mail/spamassassin/local.cf
dbg: config: using /var/spool/postfix/.spamassassin for user state dir
dbg: config: using /var/spool/postfix/.spamassassin/user_prefs for user
prefs file
dbg: config: read file /var/spool/postfix/.spamassassin/user_prefs
dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::Pyzor from @INC
dbg: pyzor: network tests on, attempting Pyzor
dbg: plugin: loading Mail::SpamAssassin::Plugin::Razor2 from @INC
dbg: razor2: razor2 is available, version 2.84
dbg: plugin: loading Mail::SpamAssassin::Plugin::SpamCop from @INC
dbg: reporter: network tests on, attempting SpamCop
dbg: plugin: loading Mail::SpamAssassin::Plugin::AutoLearnThreshold
from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::WhiteListSubject from
@INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEHeader from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::ReplaceTags from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::DKIM from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::Check from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::HTTPSMismatch from
@INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::URIDetail from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::Bayes from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::BodyEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::DNSEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::HTMLEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::HeaderEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::MIMEEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::RelayEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::URIEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::WLBLEval from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::VBounce from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::ImageInfo from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::FreeMail from @INC
dbg: plugin: loading Mail::SpamAssassin::Plugin::AskDNS from @INC
*****there is a huge chunk missing here****
dbg: config: finish parsing
------ Original Message ------
From: "Shawn Iverson" <iversons at rushville.k12.in.us>
To: "MailScanner Discussion" <mailscanner at lists.mailscanner.info>
Sent: 07/06/2018 22:23:21
Subject: Re: Re[2]: Mailscanner - Spamassassin Issue
>Toby,
>
>What happens if you run this, which is equivalent to your successful
>command as root, but as postfix?
>
>sudo -u postfix spamassassin -D -t Testy.eml
>
>Which, if it doesn't work, means that your postfix user cannot access
>things that spamassassin needs to perform a successful scan.
>
>
>
>
>
>On Thu, Jun 7, 2018 at 3:24 PM, Toby <toby at rosecott.net> wrote:
>>Hi,
>>
>>So after playing with the suggestion below, and changing the command a
>>little bit i now definatley have two different behaviours.
>>
>>Running the commands :-
>>1. spamassassin -D -t Testy.eml
>>
>>consistantly gives me a full scan and result set and ends (before
>>reporting the score) with :-
>>
>>Jun 7 20:14:41.072 [21537] dbg: check: is spam? score=8.997
>>required=5
>>Jun 7 20:14:41.072 [21537] dbg: check:
>>tests=BAYES_99,BAYES_999,DIGEST_MULTIPLE,HTML_MESSAGE,PYZOR_CHECK,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RDNS_NONE,T_REMOTE_IMAGE
>>Jun 7 20:14:41.073 [21537] dbg: check:
>>subtests=__ANY_TEXT_ATTACH,__ANY_TEXT_ATTACH_DOC,__BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE,__CAN_HELP,__CT,__CTYPE_HAS_BOUNDARY,__CTYPE_MULTIPART_ALT,__CTYPE_MULTIPART_ANY,__DKIM_DEPENDABLE,__DOS_DIRECT_TO_MX,__DOS_HAS_ANY_URI,__DOS_RCVD_MON,__DOS_SINGLE_EXT_RELAY,__HAS_ANY_URI,__HAS_DATE,__HAS_FROM,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HTML_LINK_IMAGE,__KHOP_NO_FULL_NAME,__LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH,__LOCAL_PP_NONPPURL,__MIME_HTML,__MIME_VERSION,__MISSING_REF,__MISSING_REPLY,__MSGID_OK_HOST,__NONEMPTY_BODY,__RDNS_NONE,__REMOTE_IMAGE,__SANE_MSGID,__TAG_EXISTS_BODY,__TAG_EXISTS_HEAD,__TAG_EXISTS_HTML,__TAG_EXISTS_META,__TOCC_EXISTS,__TVD_MIME_ATT_TP
>>Jun 7 20:14:41.075 [21537] dbg: timing: total 46298 ms - init: 5474
>>(11.8%), parse: 6 (0.0%), extract_message_metadata: 137 (0.3%),
>>get_uri_detail_list: 27 (0.1%), tests_pri_-1000: 130 (0.3%),
>>compile_gen: 713 (1.5%), compile_eval: 138 (0.3%), tests_pri_-950: 22
>>(0.0%), tests_pri_-900: 24 (0.1%), tests_pri_-400: 36459 (78.7%),
>>check_bayes: 36436 (78.7%), b_tokenize: 29 (0.1%), b_tok_get_all: 11
>>(0.0%), b_comp_prob: 9 (0.0%), b_tok_touch_all: 0.51 (0.0%), b_finish:
>>36381 (78.6%), expire_bayes: 36225 (78.2%), tests_pri_0: 3647 (7.9%),
>>check_spf: 161 (0.3%), dkim_load_modules: 106 (0.2%),
>>check_dkim_signature: 2.5 (0.0%), check_dkim_adsp: 126 (0.3%),
>>poll_dns_idle: 0.80 (0.0%), check_razor2: 1416 (3.1%), check_pyzor:
>>284 (0.6%), tests_pri_500: 331 (0.7%)
>>
>>Running the commands
>>1. sudo su postfix -p -c "spamassassin -D -t -C
>>/etc/MailScanner/spamassassin.conf < Testy.eml" - in the dbg
>>statements for this command, there are some warnings about permission
>>2. sudo -u postfix spamassassin -D -t -C
>>/etc/MailScanner/spamassassin.conf Testy.eml
>>3. spamassassin -D -t -C /etc/MailScanner/spamassassin.conf
>>Testy.eml
>>Seems to start spamassassin, but it finishes way way sooner than the
>>successful command, it stops very early on in the dbg, and gives a
>>message saying "(no Report template found)
>>
>>the above 3 commands all end up with
>>Jun 7 20:23:20.745 [21665] dbg: check: is spam? score=0 required=5
>>Jun 7 20:23:20.746 [21665] dbg: check: tests=
>>Jun 7 20:23:20.746 [21665] dbg: check: subtests=
>>Jun 7 20:23:20.747 [21665] dbg: timing: total 796 ms - init: 696
>>(87.5%), parse: 6 (0.7%), extract_message_metadata: 62 (7.8%),
>>get_uri_detail_list: 24 (3.0%), tests_pri_0: 21 (2.6%), compile_gen:
>>4.3 (0.5%)
>>
>>There are plenty of other difference betweent the two reports.
>>
>>Still very little idea of what i am doing with this, so help is still
>>very much appreciated!
>>
>>regards
>>
>>Toby
>>
>>------ Original Message ------
>>From: "Toby Widdows" <toby at rosecott.net>
>>To: "MailScanner Discussion" <mailscanner at lists.mailscanner.info
>><mailto:mailscanner at lists.mailscanner.info>>
>>Sent: 06/06/2018 23:15:04
>>Subject: Re: Mailscanner - Spamassassin Issue
>>
>>>Ahh never thought of that, will give it a try
>>>
>>>Regards
>>>
>>>
>>>Toby Widdows
>>>
>>>
>>>
>>>On Wed, Jun 6, 2018 at 11:14 PM +0100, "Shawn Iverson"
>>><iversons at rushville.k12.in.us> wrote:
>>>
>>>>Toby,
>>>>
>>>>Are you executing spamassassin in the same way that MailScanner
>>>>calls spamassassin?
>>>>
>>>>In other words...assuming MailScanner is running as
>>>>postfix...something like this....
>>>>
>>>>sudo su postfix -p -c "spamassassin -t -d -C
>>>>/etc/MailScanner/spamassassin.conf < somespamemail"
>>>>
>>>>This may shed some insight.
>>>>
>>>>On Wed, Jun 6, 2018 at 2:47 PM, Toby <toby at rosecott.net> wrote:
>>>>>Hi,
>>>>>
>>>>>I have just set up MailScanner in a PI3 running Jessie, I have
>>>>>postfix as a mail server.
>>>>>
>>>>>I'm not sure MailScanner is calling or allowing Spamassassin to run
>>>>>properly.
>>>>>
>>>>>I can see from the X headers that stuff is happening, but it is not
>>>>>catching emails that are blatantly spam. i get a lot of scores
>>>>>below 3.
>>>>>
>>>>>It passes the eicar, and gtube tests.
>>>>>
>>>>>However i picked an email that passed through mailscanner with a
>>>>>low score of 3, saved it, and then ran spamassassin -D <filename>
>>>>>and this process marked it as a 9!
>>>>>
>>>>>I think, but am not sure, that MailScanner is not triggering
>>>>>Spamassassin or Spamassassin when triggered by MailScanner is
>>>>>either not completing or not doing anything.
>>>>>
>>>>>I am completely new to this, and have no idea where to look, or
>>>>>what to do.
>>>>>
>>>>>Any help greatly appreciated
>>>>>
>>>>>Regards
>>>>>
>>>>>Toby
>>>>>
>>>>>
>>>>>
>>>>>--
>>>>>MailScanner mailing list
>>>>>mailscanner at lists.mailscanner.info
>>>>><mailto:mailscanner at lists.mailscanner.info>
>>>>>http://lists.mailscanner.info/mailman/listinfo/mailscanner
>>>>><http://lists.mailscanner.info/mailman/listinfo/mailscanner>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>--
>>>>Shawn Iverson, CETL
>>>>Director of Technology
>>>>Rush County Schools
>>>>765-932-3901 x1171
>>>>iversons at rushville.k12.in.us
>>>>
>>>>
>>
>>
>>
>>--
>>MailScanner mailing list
>>mailscanner at lists.mailscanner.info
>><mailto:mailscanner at lists.mailscanner.info>
>>http://lists.mailscanner.info/mailman/listinfo/mailscanner
>><http://lists.mailscanner.info/mailman/listinfo/mailscanner>
>>
>>
>
>
>
>--
>Shawn Iverson, CETL
>Director of Technology
>Rush County Schools
>765-932-3901 x1171
>iversons at rushville.k12.in.us
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mailscanner.info/pipermail/mailscanner/attachments/20180608/96726e02/attachment-0001.html>
More information about the MailScanner
mailing list