Inode Preservation Possible?

[Leonardo Helman]

Hi, another zmailer user...

nice to see you. 

I wrote (copy/paste/hack) the ZMailer and it's store module

We use a custom log to follow the mails in the differen parts
of the program.

Don't forget that the inodes are constantly reused.


This is how we deal with that (I think we are happy with our solution
there)

First, we build a custom message identifier
with a md5, (that's a patch to Julian MS)
Then we have a custom log hooked 
in "Always Looked Up Last"

That way, when we want to follow a single mail, we
look first for it's md5, and our log, 

We have almost all the data that's usually needed.

If we need more, with some data in that same log,
we could find the conversation in the smtpserver/router/scheduler
part (we also have some custom log in the router)


That logging/md5 part didn't get to the trunk, but it's a trivial one


Saludos
Leonardo Helman
Pert Consultores SRL
Argentina


On Fri, 2008-05-09 at 10:42 -0700, Neal Morgan wrote:
> Julian Field wrote:
> > Neal Morgan wrote:
> > > Hi Julian and list!
> > >
> > > I've been using MailScanner with ZMailer successfully for some time.
> > > One thing I didn't like was the fact that there was a "disconnect"
> in
> > > the logs (syslog, mail) between the message ID when received by
> SMTP,
> > > handed off to MS, taken again by router/scheduler.  This made it
> very
> > > difficult to diagnose problems with individual messages.
> > >
> > > I'll try to keep the background info short.  Zmailer derives a
> message
> > > ID from the inode and create date.  If I could keep the message in
> the
> > > same/original file, the logging would stay correct from ZM to MS and
> > > back.
> > >
> > > I managed to build on the work of others (Carlos?), added some of my
> own
> > > scripts to preserve the original inode through copying and cat'ing.
> It
> > > works but it's kind of ugly.
> > >
> > > So, the question is, would it be possible to make MailScanner do
> this
> > > for me?  In other words, don't delete the incoming file, instead
> copy it
> > > to its work area, perform whatever changes are necessary, then put
> the
> > > altered content BACK into the original file so the inode would be
> > > preserved?
> > And what happens when the power dies or the OS crashes when you are 
> > half-way through over-writing the content of the original file? I
> don't 
> > like this *at all*, sorry.
> > 
> > >    ...finally, and equally important, use a "mv" to put in the
> > > outgoing folder rather than copying.
> > >   
> > I can only mv if I'm not modifying the file.
> > 
> > MailScanner never "owns" a message, as a power outage or crash could 
> > cause you to lose a message.
> > 
> > Jules
> > 
> > 
> 
> Thanks for your feedback.
> 
> That's a valid concern.  My ugly hack addresses this by keeping the
> original and sending a copy to MS.  The original is only touched when I
> have a valid match in output from MS.  At this point, the contents of
> the original are replaced with the contents of the file from MS.  If
> there were a problem I still have the output from MS, so I haven't lost
> anything.
> 
> What I'm trying to accomplish here is to have a complete audit trail in
> syslog as the message moves through the various processes.  It's a real
> pain in the neck to troubleshoot if the ID changes when the messages
> enters MS and is returned.  
> 
> Below is a sample of what I've managed to accomplish with my hack.  (If
> you know the message of interest was "S1049805AbYEIRIA" when received by
> smtpserver, you can grep for that in syslog).  
> 
> I'm just wondering if there is a better way to accomplish this:
> 
> 
> 10:08:00 smtpserver[21196]: S1049805AbYEIRIA: (9813c) accepted from
> s.myserver.org/54957
> 10:08:15 MailScanner[13776]: ZMMS: (S1049805AbYEIRIA) ID=1049805
> OutQueueDir=/var/spool/postoffice/router/D
> 10:08:15 MailScanner[13776]: ZMMS: (S1049805AbYEIRIA) Creating rename
> map: D/1065521 back to D/1049805
> 10:08:27 router[21264]: ZMMS: (S1049805AbYEIRIA) D/1049805 has been
> scanned and is now being routed
> 10:08:27 router[3594]: S1049805AbYEIRIA:
> from=<someaddress at myserver.org>, <snip>
> 10:08:27 smtp[14101]: S1049805AbYEIRIA: to=<someotheraddress
> someaddress at myserver.org>, delay=<snip>
> 10:08:27 scheduler[3579]: S1049805AbYEIRIA: complete (total 1
> recipients, 0 failed)
> 
> 
> With the default MS/ZM setup, the "S1049805AbYEIRIA" ID would have
> changed at least once, and there would be no way to draw a conclusion
> about what its new ID is without manually scanning the order of syslog
> recs and trying to interpolate.  (That ID is derived from inode and
> create date.  A new file back from MS generates a different ID.)
> 
> Thanks again for your feedback.  If you don't have any other suggestions
> I guess I'll just keep my hack...
> 
> 
> Thanks,
> 
> Neal Morgan
>