MS/perl segfaults

David Lee t.d.lee at durham.ac.uk
Mon Jan 19 10:53:54 GMT 2009


On Sat, 17 Jan 2009, Julian Field wrote:

> On 17/1/09 15:14, shuttlebox wrote:
>> On Sat, Jan 17, 2009 at 3:03 PM, Julian Field
>> <MailScanner at ecs.soton.ac.uk>  wrote:
>> 
>>> Re-visiting this issue.
>>> Is it still a problem?
>>> Is it worth attempting to solve?

Speaking as the original requester: yes please.  The problem is indeed, 
quite rare.  But when it hits its effects can be severe.

Reminder/refresher: the problem is perl crashes from MS or subcomponents, 
typically caused by a new type of data, or malformed data, in incoming 
emails.  As such an event occurs, live, it can act as a DOS against the MS 
installation and against its own users.  The proposal is to identify such 
DOS, as it happens, and mitigate its effects, live.

And for a big site, such an event happening (say) at the start of an 
extended unattended (lights out) period such as Christmas/New-Year can be 
unfortunate.  (We were relatively lucky with our most recent such DOS: it 
was an ordinary weekend; even so the Monday (and Tuesday) were worrying.)


Shuttlebox asks:

>> Do we need a database? Couldn't you just stat the queue files to see
>> how old they are and get the same result?
>> 
>> To me, the queue dir is like a database, and the queue files are like
>> records in the database. You have to put timestamps into the database
>> but the files already have that. There's no records to remove when the
>> message has been delivered because the files will be gone.
>> 
>> If I'm not missing something it seems unnecessarily complex with a 
>> database..?

Good question, shuttlebox.  Thanks.

I originally proposed a database simply as a concept to help us think 
through the principles.  But a bit of lateral thinking like that, 
regarding the implementation, seems fine to me, so long as it doesn't 
unduly compromise that implementation.

Julian continues:

> Good idea, but what happens when older queue files are put in the queue? 
> Such as when you suspend MailScanner but leave the incoming sendmail 
> working when working on MailScanner but want to leave incoming sendmail 
> working? You need a timestamp that is touched by MailScanner but not by 
> the message being written into the queue. Can't use the last-accessed 
> timestamp as that will be touched by MailScanner reading it anyway.

Indeed.  It was was such very reasons that I proposed a database concept. 
Using (mis-using?) existing features in an implementation could lead us 
into trouble in some situations.  (And the whole point of this proposal is 
to eradicate one source of trouble, not replace it with another.)

> And the timestamp we use needs to be implementable regardless of the MTA 
> in use. Is the last-modified timestamp used by any of them? We also need 
> to be able to tell if it hasn't been touched yet, maybe 
> last-modified==created ? Again, does this work in every MTA?

Indeed.  Are we in danger of leading ourselves into trouble?

> I entirely agree it would be a very neat solution, but only if we can 
> make it work in all MTAs.

OK.  So how to implement?  Suggest:

1. Our thinking should continue to be database-like for a robust design.

2. Shuttlebox has led to consider whether the incoming queue files can 
somehow be their own database.  A good idea.  I like it.

3. Central to the database is clear understanding and use of timestamps. 
We ought to be very wary of compromising on this principle.

4. Using UNIX create/access timestamps on files could be a major source of 
such compromise.  (All MTAs?  Dodgy power supplies?  Machine outage and 
fix with mail still in inbound Q?  Etc.)

5. MS already has strong mechanisms for adding its own headers (although I 
realise that is on the outbound rather than inbound side).  So the 
database principles of insertion, (updating?) and locking may already be 
in place in MS.


So suppose we continue to model this using "timestamp in a database" 
thinking, but actually store, read and process those timestamps in the 
inbound file itself.  I realise that this implementation detail will be 
MTA-specific, but I think that might slot cleanly into MS's existing 
MTA-specific code.  (Julian?)

Hope that helps.


-- 

:  David Lee                                I.T. Service          :
:  Senior Systems Programmer                Computer Centre       :
:  UNIX Team Leader                         Durham University     :
:                                           South Road            :
:  http://www.dur.ac.uk/t.d.lee/            Durham DH1 3LE        :
:  Phone: +44 191 334 2752                  U.K.                  :


More information about the MailScanner mailing list