Maximum Processing Attempts

[David Lee]

On Tue, 28 Apr 2009, Kai Schaetzl wrote:

> Julian Field wrote on Mon, 27 Apr 2009 22:40:16 +0100:
>
>> Should I leave it switched on by default, so heavily loaded sites can
>> switch it off to gain that extra bit of performance, at the cost of a
>> potential reliability hit?
>
>> From the past discussion I think it is mainly those heavy-traffic sites
> that would benefit from it. I think we had exactly two folks mentioning a
> problem with occasional messages getting not processed somehow. Both with
> high-volume sites. So, the majority does not need this feature to be on.
> On the other hand, exactly those people who need it will also be hit most
> by the performance hit (as small as it may be).
> Unfortunately none of them participated in the recent discussion.

Kai, the feature isn't primarily about "occasional messages getting not 
processed somehow".  Rather it is about the severe knock-on effects that 
can have.

(Reminder: If there is some characteristic with a number of emails that 
causes perl/MS to crash, then that causes the whole "batch", including 
many innocent emails, not to get processed.  Subsequent runs keep tripping 
over the same thing.  If there are a few more such rogue emails than there 
are MS children, then almost all the innocent email gets held up 
indefinitely.  Such rogue emails are typically spam, and so there tend to 
be many instances of it.  Yes, the problem is rare.  But when it hits, it 
can be very severe: all email blocked; massive inbound queue build-up; 
load average through the roof.)

We've been running it in production ever since Julian first put it into a 
beta.  (I was honour-bound to do so; I had suggested it!)  I haven't 
noticed a performance impact.


My vote is to leave it on.

Email sys.admins have varying ability.  When this problem hits, even 
experienced sys.admins struggle.  (Been there; more than once.)  The less 
experienced would struggle even more.  The default of this option should 
be biased in favour of the inexperienced sys.admin.  So the default (I 
suggest) should be "on".  (A sys.admin. who is experienced enough to know 
they don't need it is, by definition, experienced enough to find their way 
to switching it off.)

Hope that helps.

-- 

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:  UNIX Team Leader                         Durham University     :
:                                           South Road            :
:  http://www.dur.ac.uk/t.d.lee/            Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :