Log analyzer

[John Rudd]

On Jun 25, 2004, at 2:15 AM, Steve Freegard wrote:

> John,
>
>>
>> What I'd like to see in a log analyzer is something that will tell me,
>> for each message:
>>
>> 1) when was it received, and then when did MailScanner pick it up?
>> 2) did it have any kind of dangerous
>> content/bad-filenames/bad-filetypes
>> 3) did it have a virus, and if the log knows, which one? and
>> if it did,
>> was it deleted as a silent virus?
>> 4) what was its spam assassin score (even if it wasn't marked
>> as spam)?
>> 5) did it trip any DNSBL's?
>> 6) what spam actions were applied to it?
>> 7) if it was submitted to the main/outgoing queue, when was that? (or,
>> if not, is it still in the processing pipeline, or is
>> something wrong?)
>> 8) when was it finally delivered/relayed/etc.?  Or is it still in the
>> queue?
>>
>> So, then I can run a report which will tell me, with absolute
>> certainty,
>> exactly what happened to each and every message.  And, from
>> that, I can
>> perhaps do a grep (or something) that will look for messages that had
>> certain characteristics, or determine my average spam score (which I
>> can't do now, because MS only reports messages that were marked as
>> spam), or see that "the reason this message never arrived is
>> because it
>> contained a virus" or something.  Or, tell me "W messages in,
>> X messages
>> delivered/relayed, Y messages still processing or in the mqueue, Z
>> messages missing." and then tell me _which_ messages are missing (so I
>> can inform the sender and maybe the original recipients).
>>
>> Right now, from looking at the logs, it seems like sometimes "messages
>> just disappear".  For the most part, it appears that (on our sendmail
>> machines) this is only happeneing when it's supposed to (silent
>> viruses), but I can't actually verify that.  With our CommuniGate Pro
>> systems, we did lose some messages, and the lack of "When did
>> mailscanner pick up this exact message?" and "did it delete it or
>> eventually send it back?" type log entries made it very difficult to
>> figure just which thing was dropping the ball (I suspect it was the
>> script that MS was invoking as Sendmail2 that was the problem, but,
>> again, I don't actually know for sure).
>>
>
> MailWatch does almost everything you list above as I had exactly the
> same
> requirements which is why I wrote it in the first place.  See
> http://www.sourceforge.net/projects/mailwatch.
>
> Hope this helps.
>

Oh... hm.  Except for the php and mysql parts, yeah :-}

I'll have to think more about it though.  Maybe it wouldn't be such a
bad thing to run it that way.  It just wasn't the way I was thinking of
running it (I was thinking of basic perl script that runs against
syslog output only (no database) and spits out a textual report).

When you say "does almost everything", which part(s) does it not do?

(and how did you go about determining when viruses were being deleted?
or do you still deliver silent viruses?  I'm thinking I might start
doing that.)

-------------------------- MailScanner list ----------------------
To leave, send    leave mailscanner    to jiscmail at jiscmail.ac.uk
Before posting, please see the Most Asked Questions at
http://www.mailscanner.biz/maq/     and the archives at
http://www.jiscmail.ac.uk/lists/mailscanner.html