Idea for a handy new feature

[Julian Field]

At 16:29 08/11/2003, you wrote:
>Hi All,
>
>I've just thought of a new feature for MailScanner that I'd personally find
>really useful.
>
>In light of the recent problems with Razor, Pyzor, DCC and the RBLs due to
>DDOS attacks and the like, it is difficult to tell sometimes if
>MailScanner's throughput of messages is as great as it could be due to these
>outside influences.
>
>A number of people have asked before if it is possible to give a time to
>process for each message through the various virus/spam/rbl checks - which
>is a problem as MailScanner batches the messages together.
>
>I think I've come up with a way to achieve this relativly easily that will
>provide stats for MailScanner-MRTG, Mailstats and MailWatch, and provide a
>some benchmarks that can be used by a MailScanner admin to tune his/her
>hardware and settings for maximum performance.
>
>Here's my theory:
>
>- MailScanner picks up message(s) from mqueue.in
>maillog: New Batch: Scanning 7 messages, 30160 bytes
>
>Variables to store:
>  batch[Number of messages]
>  batch[Total bytes]
>  batch[Start] = timestamp
>
>- MailScanner goes through the RBL, Spam, Virus, Attachment and Message
>Processing stages
>
>For each process stage record the variables:
>  <<process_name>>[Start] = timestamp
>  <<process_name>>[Finish] = timestamp
>
>- Once processing is finished, and just prior to moving the messages to
>mqueue
>
>Store:
>  batch[Finish] = timestamp
>
>Calculate:
>  batch[Finish] - batch[Start] = batch[Total Time]
>  batch[Total Time] / batch[Total Bytes] = batch[Bytes Per Second]
>
>  For each process:
>  <<process_name>>[Finish] - <<process_name>>[Start] = <<process_name>>[Total
>Time]
>  <<process_name>>[Total Time] / batch[Total Bytes] = <<process_name>>[Bytes
>Per Second]
>
>Then for each message in the batch (using the $message object) store:
>   $message->{total_time} = batch[Bytes Per Second] * $message->{size}
>   For each process:
>   $message->{<<process_name>>_time} = <<process_name>>[Bytes Per Second] *
>$message->{size}

What do you want to do with this? Just store it for MailWatch?

>- Move messages to mqueue for delivery
>maillog: Uninfected: Delievered 7 messages, 30160 bytes [Total tt.ttt at nn.nnn
>bps, RBL tt.ttt at n.nnn bps, SA tt.ttt at nn.nnn bps, Virus tt.ttt at nn.nnn bps,
>Content tt.ttt at nn.nnn bps] (tt = time, nn = bytes per second)

I would do the times and speeds as a separate log entry. The calculations
aren't quite as simple as you suggest, due to things like the time taken to
reconstruct modified messages from their attachments and things like that.
There are all sorts of "overheads" like that to be considered. So it might
be good to add an "overheads" time to the log output, otherwise the figures
won't add up. There will be plenty of rounding errors anyway, which won't help.


>Thoughts??
>
>Kind regards,
>Steve.
>
>--
>This email and any files transmitted with it are confidential and
>intended solely for the use of the individual or entity to whom they
>are addressed. If you have received this email in error please notify
>the sender and delete the message from your mailbox.
>
>This footnote also confirms that this email message has been swept by
>MailScanner (www.mailscanner.info) for the presence of computer viruses.

--
Julian Field
www.MailScanner.info
Professional Support Services at www.MailScanner.biz
MailScanner thanks transtec Computers for their support
PGP footprint: EE81 D763 3DB0 0BFD E1DC  7222 11F6 5947 1415 B654