Why does MS rename postfix queue IDs?

[Glenn Steen]

On 02/04/06, Mike Jakubik <mikej at rogers.com> wrote:
> Glenn Steen wrote:
> > On 02/04/06, Glenn Steen <glenn.steen at gmail.com> wrote:
> >
> >> On 02/04/06, Mike Jakubik <mikej at rogers.com> wrote:
> >>
> >>> So, as the topic says, why does MS rename postfix queue IDs? Whats is
> >>> the reason for this?
> >>>
> >>> --
> >>> Apr  2 15:34:01 fbsd postfix/smtpd[18878]: 1EE3E2B2036:
> >>> client=localhost[127.0.0.1]
> >>> Apr  2 15:34:01 fbsd postfix/cleanup[18879]: 1EE3E2B2036: hold: header
> >>> Received:
> >>> ...
> >>> Apr  2 15:34:04 fbsd MailScanner[17694]: Requeue: 1EE3E2B2036.F1395 to
> >>> F39462B2043
> >>> --
> >>>
> >>> Why add the .##### to the ID? Also, is it really necessary to change the
> >>> ID when re queuing the message?
> >>>
> >> This is a bit of a FAQ it seems, for the postfix implementation... I
> >> noticed that with MW and PF, since PF _will reuse queue IDs_, that I
> >> got a rather disturbing amount of duplicates in my database....
> >> (Could've been any database logging too, or even a script calculating
> >> things based on the queue ID. Any such system was bound to have a fair
> >> amount of errors, particularly if you employ a "less than simplistic
> >> partitioning scheme", since the amount of continuous i-node
> >> consumption will play a role too. I had var on its own partition, so
> >> got hit pretty bad) ... I badgered first Steve for a fix, then
> >> Jules... Who was gracious enough to oblige.
> >>
> >> As mentioned, the whole problem is that the queue ID will be reused,
> >> since it is calculated from the i-node and the present microsecond...
> >> Sounds rather random, but simply isn't "random enough" (as Jules
> >> comment in the code goes:).... Even in some rather common "standard
> >> setups" you _will_ be bit by this.
> >>
> >> Jules solution (to manage some extra randomness, tagged on behind a
> >> very "scriptabe"/"ignorable" <dot><five hex digits> is purely
> >> briliant. And no, it should stay, no matter what;-).
> >>
> >>
> > (Replying to myself.... Sigh:-)
> > About the requeueing bit, that is necessary, yes. "man postsuper"
> > tells a lot about the "hoary" details of how PF really works:-).
> >
>
> Thats for the detailed explanation. In this case i agree with you,
> things should stay the same. Do you think it is safe to assume that a
> logged msg id in a db will not be duplicated, say over a span of 3
> years? I think one should probably still refer to records by record id,
> not msg id, just to be safe...
>
I haven't "done the math" for that long a time-period. Remember that
the likelihood of "ID reuse" is dependant not only on the time period
(3 years), but also on the frequency  (meaning amount of messages
handled)... And on how you've partitioned things.
In my case it would be safe for that time-period, yes, but fortunately
I don't need to handle more than three months, so ... I'm
"super-safe":-). Without the fix, I had several duplicates/day,
seriously confusing things ... particularily in the quarantine
view.... So for me this is an essential fix.

>From the message POV, record id is meaningless. Sure, that makes the
duplicates "non-duplicates" from a DB POV, but they don't really help
with the messages (where you often don't have anything more than the
message ID or queue ID to start with, if that), so ... yes and
no:-):-).

--
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se