should I change score for Bayes_90?

[Matt Kettler]

At 01:41 PM 10/21/2003, Mark Nienberg wrote:
>Also, I don't see why you would have higher scores for lower bayes
>probablilities, the
>way your third column does.

One of these day's I'll have to enter this stuff into the FAQ on the SA site..

Scores that don't meet your "common sense" expectations not an uncommon
phenomenon in SA...

If the rules were scored completely on their own, you'd wind up with
increasing scores for all the bayes categories, but the scoresets for SA
aren't calculated on their own, their calculated as a complete set. Thus,
the rule scores are aren't based on the performance of a rule by itself,
but based on the combinations of other rules they interact with.. this
makes the scoring VERY nonlinear, which at first glance seems wrong, but
when you sit down and start studying the patterns that exist in real email,
that's really how things wind up working.

Also in general, never underestimate the ability of human behaviors to fail
to fit into any simple mathematical model.

In this case, I suspect there's a fair amount of "spammish" non-spam (ie:
crude jokes) which don't go very high on the bayes scale, but wind up in
the 80 or 90 ballpark and also wind up matching some of the static rules..
in order to avoid excessive false positives, the GA will tend to hack down
the score of one or more of the rules, but will try to do so without
causing excessive false negatives.. it looks like these "upper-mid" range
bayes scores were the best candidate for sacrificing score to correct these.

This same kind of non-linear behavior can also be seen in older versions of
SA which had the "spam phrases" ruleset. In that system, the higher the
spam phrases score, the more spam phrases exist in the email. You'd expect
this to have a nice, constantly increasing correlation with spam score..
but of course reality proves otherwise, and you wind up with weird dips in
the curve which are the results of that ruleset interacting with a whole
pile of other rules.