Corrupt pdf files - a cautionary tale.

[Quentin Campbell]

> -----Original Message-----
> From: Julian Field [mailto:mailscanner at ECS.SOTON.AC.UK] 
> Sent: 29 July 2003 12:31
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: Corrupt pdf files, any advice.
> 
[snip]
> Outlook XP always appears to use Base64, so I suspect the 
> problem may just 
> exist in Exchange 5.5 and/or Outlook 97. Don't know about 
> Outlook 2000.
> 
> Whether Acrobat Reader (on some platforms) will continue to 
> be able to use 
> the damaged file is another matter entirely, something over 
> which I have no 
> control.
> 
> All I can suggest is you request people using the particular 
> troublesome 
> versions always zip their PDF files to stop Outlook destroying them.

These PDF files will continue to be a problem for sites that are similar
to ourselves. BEWARE if your site shares the following characteristics:

(1) Uses Sophos as your A-V tool; (2) have upgraded or are about to
upgrade from an oldish version of MailScanner that does not have the
"Allowed Sophos Error Messages =" option in the configuration file, and
(3) receive and pass on lots of PDF files.

We were running MS 4.10-1 for many months and were processing scores of
PDF file attachments each day without any problems. Then two weeks ago I
upgraded to MS 4.22-5 and at that point most PDF file attachments
started to be rejected.

The common symptom was that Sophos reported the attachment as being
"corrupt". Sophos itself had not changed recently so the problem lay
either with MS 4.22-5 or the way in which I had configured it. After
some discussion with Julian it became clear that it was the latter. 

The fault lay with my ignoring a fairly new configuration file option
which is specific to Sophos: "Allowed Sophos Error Messages =". In fact
the potential problem with this option is highlighted by Julian in the
comments that prefix this option in the configuration file - I should
perhaps have read more carefully.

That said the default value for this new option is "unsafe" in
situations like ours because it has immediate and damaging operational
consequences that are not as apparent as they should be from Julian's
warning. The option's default value is "safe" from a security point of
view and can be ignored by sites that do not use Sophos.   

By default the value of this option is null which means that if Sophos
detects a corrupt attachment (which it thus cannot scan) then MS will
"fail safely" and flag the attachment as possibly containing a virus.

After the upgrade to MS 4.22-5 I had left this option with its default
value with the result that all the dodgy PDF files that Sophos and MS
4.10-1 had been quietly ignoring were suddenly being flagged as possible
viruses.

The fix was to set "Allowed Sophos Error Messages = corrupt" in the
configuration file which means that Sophos will simply ignore any
"damaged" attachment that generates the Sophos error string "(corrupt)".


Many of these PDF files had been received from other sites before I
upgraded MS so people were not aware of a problem with them since they
viewed OK. It was only later when they tried to send them on to other
people (after MS was upgraded) that the damage became apparent. 

It is no good of course zipping them once they are damaged but zipping
them before first mailing them would have prevented the problem, as
Julian suggests. However we have no control over the many sites that
send us PDF files so it is likely that this will be a continuing problem
for sites like ourselves for a long time to come.

Unfortunately the fix to this problem has the consequence that any
corrupt file, PDF or non-PDF, that Sophos cannot scan will potentially
be delivered. However since we also run with a second A-V product
(McAfee) and have the usual MS filename extension and file type blocks
in place I hope we will be reasonably protected.

Quentin
---
PHONE: +44 191 222 8209    Computing Service, University of Newcastle
FAX:   +44 191 222 8765    Newcastle upon Tyne, United Kingdom, NE1 7RU.
------------------------------------------------------------------------
"Any opinion expressed above is mine. The University can get its own."