More details in the logs

Mon Mar 1 14:54:08 GMT 2004

I have patched Message.pm to provide all the "To:" information as
well as the subject in the logs. it would produce output such as:

Mar  1 08:26:35 west MailScanner[17879]: Message 1AxnR4-0006Rt-5n
from 66.148.140.2 (sender at domain.com) to ourdomain.com is spam,
SpamAssassin (score=5.978, required 5, BODY_8BITS 1.50,
HTML_70_80 1.50, HTML_COMMENT_SAVED_URL 0.82,
HTML_FONTCOLOR_UNKNOWN 0.10, HTML_FONT_FACE_BAD 0.20,
HTML_MESSAGE 0.00, HTML_TAG_BALANCE_BODY 0.26,
HTML_TAG_BALANCE_TABLE 0.20, J_CHICKENPOX_110 0.30,
J_CHICKENPOX_210 0.30, J_CHICKENPOX_33 0.30, b_OBFU_QnoU 0.50
Report Len is 323)
:someone at ourdomain.com;someoneelse at ourdomain.com :
FWNew ESP contact details.

The spam report is truncated to 500 chars if over 500 (I have
seen chickenpox/tripwire combos produce lines over 1000) and
original length is show at the end of the report (ex: Len
is/truncated from 323) and the "To" and Subject info is separated
by colons with the multiple recipients being separated by
semi-colons. I have a script that parses the output above into a
HTML email in table form so it makes for easy reading.

the line is:
date time host MailScanner log tag message I remote host
(sender_domain) "to domain" spam tag SpamAssassin report TO(s)
and subject

If you want to try the patch (applies cleanly to vers from at
least 4.23-5 through 4.27-7)
I have attached it. I did the patch because I can generally look
at the to, subject and report and tell if it's really spam or a
false positive without bothering to look at the actual message
text. The patch includes full comments so if someone sees a
cleaner way to do it please feel free to change it.

Rick

> -----Original Message-----
> From: MailScanner mailing list
> [mailto:MAILSCANNER at JISCMAIL.AC.UK]On
> Behalf Of Jim Holland
> Sent: Monday, March 01, 2004 8:25 AM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: More details in the logs
>
>
> Hi
>
> On Mon, 1 Mar 2004, Patrik Bäckström wrote:
>
> > We use MailScanner for several customers/domains
> (currently version
> > 4.25-14) and we would like to gather statistics per
> customer on how
> > many mails scanned (that i can get from postfix),
> how many rejected
> > and why and so on.
> >
> > Currently, it only tells us that something has been
> blocked and why,
> > but not from or, more importat, to who the mail was sent.
>
> I think this is an important requirement.  Unlike with
> worms, it is not
> possible to be 100% certain that a particular message
> is spam.  I would
> like to use a very agressive spam blocklist - eg
> dnsbl.net.au.  However if
> spam is quarantined without a notice to either sender
> or recipient it is
> quite possible that genuine mail will be lost.  The
> use of the "notify"
> option is not really an option, as I would not like to
> receive a separate
> notification for each of the 150 spam messages per day
> that people
> normally try to send me.  Before using MailScanner we
> could simply analyse
> the sendmail maillog file for details of recipients
> whose mail had been
> blocked.  Sadly, I now see that in a significant
> number of cases where
> spam is blocked there is no longer a sendmail entry
> indicating who it was
> going to be delivered to (see more details appended),
> and the MailScanner
> Spam Actions entry does not indicate the recipient either.
>
> What we are doing now is to run a nightly script that
> analyses the headers
> of all quarantined spam for recipients, and also
> checks the maillog file
> for recipients that might be listed there for the same
> quarantined
> messages.  We then send a summary to our users that
> lists details of all
> quarantined mail.  I think the concept of a daily
> archival notice is a
> good compromise between sending no notices at all and
> sending a separate
> notice for each message.
>
> Another way of handling this issue would be to write
> the MailScanner
> notification messages to a separate log file instead
> of delivering them to
> the recipients.  That log file could then be analysed
> separately.  However
> there is currently no option for sending the
> notifications anywhere other
> than to the recipient.
>
> Regards
>
> Jim Holland
> System Administrator
> MANGO - Zimbabwe's non-profit e-mail service
>
> Logging of blocked spam
>
> Normally the sendmail maillog file will have the
> following entries:
>
>         sendmail from= line with details of sender
>         sendmail to= line indicating recipient, stat=queued
>         MailScanner RBL checks: details of why message
> is blocked
>         MailScanner Message line, eg:
>                 Message i21D03F24046 from 213.120.110.92
>                 (manmeet at liquidstorms.com) to mango.zw
> is spam, spamhaus-XBL
>         MailScanner Spam Actions . . . actions are store
>
> For reasons I don't understand, the second (or more,
> if there are multiple
> recipients) sendmail line is not always present, so
> there is no consistent
> log info about the recipient(s).  If the MailScanner
> Message line could
> include the details of the recipients in it then it
> would be possible to
> meet the requirements of Patrik for statistics, and
> also use it for
> purposes of user notifications.
>
> A more advanced option might be for MailScanner to
> provide a proper daily
> archival notification facility rather than the current
> per message
> notification which is really unworkable given the huge
> volume of spam.
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Message.patch
Type: application/octet-stream
Size: 2313 bytes
Desc: not available
Url : http://lists.mailscanner.info/pipermail/mailscanner/attachments/20040301/ca56addc/Message.obj