Benchmarking

[Julian Field]

At 22:09 17/09/2002, you wrote:
>On Tue, 2002-09-17 at 15:35, Thomas DuVally wrote:
> > Hi all,
> >         Anyone know of a good way to benchmark a mailserver?  We are
> getting
> > ready to implement Sendmail/Mailscanner/SpamAssassin and would like to
> > have some numbers that tell us how it's goind to perform before we put
> > it in.
> >         We think we know how we can load test it: Make it a relay just
> for the
> > ORBS and MAPS domains and submit it for testing.  Once it gets posted we
> > can watch the spam start flying.  But we want something a little more
> > controlled.
> >
>I don't know that I've ever looked for a specific application to do load
>tests, but I can tell you how I do it. I've got a collection of qf/df
>pairs that I gathered from traffic through a real server. There's some
>15K messages (~350Mb) in my sample and approximately half is spam. I use
>a perl script to feed the messages into MailScanners Incoming Queue Dir
>at what ever rate I want. MailScanner does its thing and drops the
>result into the Outgoing Queue DIR. The queue dirs aren't known to
>sendmail, so it remains out of the loop and all I see is the performance
>of MailScanner.
>
>I don't see any point in including Sendmail in the test since there are
>so many unknowns in mail traffic. It's easy enough to watch the load and
>how many messages are being processed per unit of time when you have
>control of the message input rate. And by working with a fixed set of
>messages you have predictable behaviour of MailScanner over the set.
>With a live mail feed the content and percentage of spam varies greatly.

Seconded. What I have done is capture (using the Archive Mail feature)
20,000 real messages coming in to our department. I go through all the qf
files, and replace the sender and recipients with "anonymous". All our mail
servers have an alias called "anonymous" whose value is "/dev/null". This
way I'm protected from embarrassment should any of the messages leak out of
my test server by mistake.

I switch off the incoming and outgoing sendmails, and set the delivery mode
in MS to "queue" so it doesn't make any delivery attempts. The only things
that should then leak are all the sender reports, and the messages
containing disinfected documents (with macro viruses removed). These all
end up at /dev/null when they first hit any of my other mail servers.

I then plonk 20,000 messages in /var/spool/mqueue.in and time how long it
takes for /var/spool/mqueue.in to become empty. Works a treat.
--
Julian Field                Teaching Systems Manager
jkf at ecs.soton.ac.uk         Dept. of Electronics & Computer Science
Tel. 023 8059 2817          University of Southampton
                             Southampton SO17 1BJ