Log Analysis

[Scott Silva]

    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "US-ASCII" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Pentland G. spake the following on 12/7/2005 2:36 AM:
> Apologies for not having time to read the entire thread...
> 
> I have cron jobs that read yesterday's log file "maillog.0.gz" and email
> the results from a series of perl scripts of the form...
> 
> <snip>
> #!/usr/bin/perl
> 
> # Reset counters
> $TotalMails   = 0;
> $Totalin      = 0;
> $TotalViruses = 0;
> $TotalSpam    = 0;
> $DeliveredSpam = 0;
> $DeletedSpam = 0;
> $DeletedViruses = 0;
> $UserUnknown = 0;
> $satimeout = 0;
> $MsgsDeletedAtSix         = 0;
> $MsgsDeletedAtEight       = 0;
> $MsgsDeletedAtTwentyEight = 0;
> $MsgsNotDeletedBySpamdump =0;
> $UnderFour               = 0;
> $FourToEight             = 0;
> $EightToTwelve           = 0;
> $TwelveToSixteen         = 0;
> $SixteenToTwenty         = 0;
> $TwentytoTwentyFour      = 0;
> $TwentyFourtoTwentyEight = 0;
> $OverTwentyEight         = 0;
> $TotalBelowEight         = 0;
> $TotalAboveEight         = 0;
> $TotalBelowSix           = 0;
> $TotalSixToEight         = 0;
> $TotalEightToTwentyEight = 0;
> $TotalAboveTwentyEight   = 0;
> $rfcerrors               = 0;
> 
> while($_=<STDIN>) {
>   chomp;
>   if (/sendmail/) {
>     $TotalMails += $1 if /nrcpts=(\d+),/;
>     $UserUnknown += 1 if /User does not exist at this site/i;
>     $rfcerrors += 1 if /address does not conform to RFC 2821 syntax/;
>     next;
>   }
>   if (/mailscanner/i) {
>     $TotalViruses += $1 if /Virus Scanning: Found (\d+) viruses/i;
>     $TotalSpam    += $1 if /Spam Checks: Found (\d+) spam messages/i;
>     $DeliveredSpam += 1 if /actions are deliver/i;
>     $DeletedSpam  += 1 if /actions are delete/i;
>     $DeletedSpam  += 1 if /Returning action delete/i;
>     $DeletedViruses += 1 if /Viruses marked as silent/;
>     $satimeout += 1 if /SpamAssassin (timed out)/;
>     if (/Returning value (\d+) for (\w+)/) {
>         $MsgsDeletedAtSix += 1 if $1 == 6;
>         $MsgsDeletedAtEight += 1 if $1 == 8;
>         $MsgsDeletedAtTwentyEight += 1 if $1 == 28;
>         $MsgsNotDeletedBySpamdump += 1 if $1 == 600;
>     }
> 
>   $TotalBelowSix            += 1 if /score\=(\d+)/ && $1 < 6;
>   $TotalSixToEight          += 1 if /score\=(\d+)/ && 6 <= $1 && $1 < 8;
>   $TotalEightToTwentyEight  += 1 if /score\=(\d+)/ && 8 <= $1 && $1 <
> 28;
>   $TotalAboveTwentyEight    += 1 if /score\=(\d+)/ && 28 <= $1;
>   }
> }
> 
> $Totalin = $MsgsDeletedAtSix + $MsgsDeletedAtEight +
> $MsgsDeletedAtTwentyEight + $MsgsNotDeletedBySpamdump;
> 
> print "Main Totals\n\n";
> print "Total Mails
> $TotalMails\n";
> print " Total inbound from the outside world
> $Totalin\n";
> print "Total Viruses
> $TotalViruses\n";
> print "Total Spam
> $TotalSpam\n";
> print "Rejected mail for unknown recipients
> $UserUnknown\n";
> print "Rejected as not to RFC 2821 syntax
> $rfcerrors\n";
> print "Auto-Deleted Spam
> $DeletedSpam\n";
> print "Silently Deleted Viruses
> $DeletedViruses\n";
> print "SpamAssassin Timeouts
> $satimeout\n";
> print "\n";
> print "Number of lookups for users that have set SpamDump at 6
> $MsgsDeletedAtSix\n";
> print "Number of lookups for users that have set SpamDump at 8
> $MsgsDeletedAtEight\n";
> print "Number of lookups for users that have set SpamDump at 28
> $MsgsDeletedAtTwentyEight\n";
> print "Number of lookups for users that have set SpamDump off
> $MsgsNotDeletedBySpamdump\n";
> print "\n\n";
> print "SpamAssassin Score Distribution\n\n";
> print "Messages Below 6            $TotalBelowSix\n";
> print "Messages of 6 -> Below 8    $TotalSixToEight\n";
> print "Messages of 8 -> Below 28   $TotalEightToTwentyEight\n";
> print "Messages 28 or Above        $TotalAboveTwentyEight\n";
> </snip>
> 
> This is one of 7 scripts and the resultant emails are collated by
> management into their spreadsheets, everyone seems happy with type of
> setup although I am trying to get enough money to buy a database server
> (or steal some space on the corporate one) to run Mailwatch:-)
> 
> Hope that is of some use,
> 
> Gary
> 
You don't have to buy a database server. The load that MySQL adds to an
existing mail server is minimal, and you can keep the database size
manageable by setting how many days of info to keep.
If you absolutely "need" a separate DB server, go look in the bin for
the systems that were "too slow for Windows XP" and throw a free linux
distro on it. An old PentiumII with 256 Megs of ram and half a dozen
Gigs of hard drive should be more than enough to keep up.


-- 

/-----------------------\           |~~\_____/~~\__  |
| MailScanner; The best |___________ \N1____====== )-+
| protection on the net!|                   ~~~|/~~  |
\-----------------------/                      ()

------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the Wiki (http://wiki.mailscanner.info/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).

Support MailScanner development - buy the book off the website!