More encoded subject woes

[Nick Smith]

Hi,

MS 4.54-2 / Postfix 2.10

I've got more trouble with encoded subject headers being "mishandled"
from a recipient's point of view. The issue occurs when, for whatever
reason, MIME-Tools is unable to decode an encoded subject properly -
this example is UTF-8, but I don't know if it may affect other
encoding types too

=?UTF-8?B?5oOF5aCx6YCj57Wh56WoIC0gVVNHcumVt+WQiOitsOW+heOBoSA=?==?
UTF-8?B?LSDnrKzvvJTvvJjlm57lhajml6XmnKwgLSDlsZXnpLrkvJrjga7lh7rlsZU=?=

If you feed that string to MIME::WordDecoder::unmime it returns:

????? - USGr????? - ??????? - ??????

I have absolutely no idea why this happens - whether it's a bug or
expected behaviour on the part of MIME-Tools, but I assume that each
question mark represents a multi-byte (Japanese in this case)
character that it was not possible to decode

Drop the same string into an Outlook message and send it via SMTP
(making sure that it bypasses MailScanner), and when it arrives it
should show a bunch of Japanese characters. The recipients are
understandably not happy that the subject of their email when it shows
up has been replaced by a bunch of question marks

I've worked around this problem with a patch against Postfix.pm
(attached), but I'm less than comfortable with it. Basically what it
does is to unmime into a temporary holding string instead of the
$message structure and then take a look at the results of its
handiwork. If it sees more than an arbitrary number of consecutive ?'s
(I picked more than 3 as a reasonable number), it assumes that the
unmime was unsuccessful and allows the original encoded subject to
pass. Otherwise it assumes decode success and fills the
message->{subject} structure with the unmime result

The first problem is that the ???? test is far from foolproof -
there's loads of scope for false +ves and false -ves. The second
problem is I'm not sure what issues this might cause if MS has to
alter the subject later. I'm not altering any subjects at all so it
wouldn't be a worry on my system but...

Clearly I'm working with Postfix here, but this affects other MTA's
too. Equally clearly the proper answer is to figure out what's up with
MIME-Tools, but I'm afraid that's way beyond my capabilities :(

Thoughts appreciated

Thanks

Nick
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ms-4.54-2-skiputf8.patch
Type: application/octet-stream
Size: 1403 bytes
Desc: not available
Url : http://lists.mailscanner.info/pipermail/mailscanner/attachments/20060519/4bef7f90/ms-4.54-2-skiputf8.obj