Make sure im doing this right

[Quentin Campbell]

>-----Original Message-----
>From: Julian Field [mailto:mailscanner at ECS.SOTON.AC.UK] 
>Sent: 17 March 2004 15:08
>To: MAILSCANNER at JISCMAIL.AC.UK
>Subject: Re: Make sure im doing this right
>
>
>At 15:05 17/03/2004, you wrote:
>>Our Mailscanner / SpamAssassin / Bayes / Big Evil List / ChickenPox /
>>SpamCop.Net setup has been doing so well I have never gotten a false
>>positve (although I get false negatives every so often).
>>
>>We have the Spam score set to 6 and high scoring spam set to 10.  We
>>deliver spam and delete high scoring spam.
>>
>>Ive been thinking about lowering the high scoring spam to 8 
>and lowering
>>spam to 5.  Lowering the spam level to 5 would basically wipe 
>out all my
>>false negatives.
>>
>>My question is this:  What is the "normal" settings.  Is 5 
>and 8 for high
>>scoring spam too low?
>
>I use 6 on the basis that a few false negatives is better than 1 false
>positive. I have 1800 users with that setting, and don't hear any
reports
>of false positives. People don't bother checking their spam folders any
>more. I don't use the high score as I need to deliver 
>everything to leave my users in control, which makes them a lot
happier.

Reducing the false positive rate to zero is the "Holy Grail" of spam
filtering. If that is achieved then you can safely auto delete _all_
tagged messages at your mail gateways.

However reducing the false positive rate towards zero almost inevitably
means that the false negative rate (untagged spam) will increase. The
consequences are not as bad as they seem. 

Assume that in achieving a false positive rate of zero your false
negative rate jumps from 3% to 5% (of all incoming spam). But because
you can now safely auto delete all tagged messages at your mail
gateways, you have reduced the amount of spam delivered to user
mailboxes by 95%. That is a very real benefit. However the 5% that does
reach users will not be tagged of course. But that small amount is
manageable.

There are circumstances where requiring recipients to set up personal
mail filters to deal with tagged messages is not desirable or is
difficult to implement throughout an organisation. 

Outlook is a particular problem. It has easy to use filter rules (via
the "Rules Wizard") but unless it is left running all the time you
cannot permanently delete messages. If you switch Outlook off overnight
or while you are on holiday then your "Delete" folder can fill up
pushing you over quota; you may lose mail as a consequence.

We currently deliver tagged messages and leave it to the user to decide
what to do with them. However after two years of this our users are now
pushing for us to delete as much spam as possible at our mail gateways.
Part of the reason for this is that more than >55% of all our incoming
mail is spam. As we receive 1,000,000+ messages per week many users here
have to filter a lot of tagged messages. 

We have just pushed our spam threshold up to 6 to reduce the false
positive rate. We plan to move soon to a position where we auto delete
at the mail gateways all tagged spam with a score >10 (High Scoring Spam
threshold). Users will thus receive far less tagged mail that they have
to deal with.

Quentin
---
PHONE: +44 191 222 8209    Information Systems and Services (ISS),
                           University of Newcastle,
                           Newcastle upon Tyne,
FAX:   +44 191 222 8765    United Kingdom, NE1 7RU.
------------------------------------------------------------------------
"Any opinion expressed above is mine. The University can get its own."