Message Batches

[Desai, Jason]

> At 17:35 15/05/2003, you wrote:
> >It got me thinking - would it not be more efficient for each
> child to work
> >to the batch size using the following:  total messages in
> queue/max number
> >of children = batch size per child - which would clear the queue more
> >efficiently???
>
> At the moment, any child takes as many messages as it can up
> to the limits
> set in the batch size parameters.
> Counting the total number of messages that *can* be scanned
> is actually
> quite an expensive operation. At the moment it only has to be
> done once as
> messages are date-sorted then put in the batch. The
> file-locking operation
> only has to be done once.
>
> Changing to the method you suggest could only be done by
> adding them all to
> the batch and then by removing all the excess ones you don't
> want. Which
> means you need to remember what order they were added.
> Possible, but messy.

(I don't know if this can be done but ...) what about some sort of
communication with each of the mailscanner processes to see how many
messasges they are currently processing?  Or a global variable that keeps
track of the current number of messages being processed for all sub
processes?  Then the routine would be something like:

Sort all of the messages (queue files)
Get the number of messages the other processes are working on
Available msgs = all msgs - msgs being processed

Now you can do some algorithm on the number of available messages.

Just a thought, and maybe it's not possible.

Jason