Batch sizes?

[Tim Bishop]

On Wed, May 21, 2003 at 12:00:12PM +0100, Julian Field wrote:
> At 22:39 20/05/2003, you wrote:
> >What do people think is a sensible batch size?
> >
> >I notice the default is 100 - but my feeling is this is a little too
> >high if you're doing spamassassin and RBL checks. My system took a fair
> >while to process that many in one go this morning - they also got mostly
> >taken by the first MailScanner process, leaving the others mostly idle.
> >
> >So is it better to do doing small but quick batches? or is it better
> >to do larger but longer batches? The latter results in quite a delay on
> >the mail... whilst at least the former gives a reassuring trickle :-)
>
> Remember that the batch size is also limited by the number of messages in
> the queue. So if MailScanner is keeping up with your mail traffic, the
> batches will be very small. It does not sit around waiting for the batch to
> fill up before doing anything.

Sure. I was specifically thinking about when a huge burst comes through
at once.

I'm testing this on my laptop, and pulling mail to it from my work
server. So every morning there's a huge burst - which helps simulate a
sudden load on a real mail server.

Before pulling mail this morning I dropped the batch size right down to
10, and it seemed to do a must better job. So when I go live on my main
mail server I suspect I'll opt for a size around 20-30, rather than 100.

Cheers,
Tim.