BarricadeMX experiences

[Steve Freegard]

Richard Frovarp wrote:
> Kai Schaetzl wrote:
>> Richard Frovarp wrote on Wed, 25 Jul 2007 11:32:06 -0500:
>>
>>  
>>> The OP did say he was running sbl+xbl at the mta
>>>     
>>
>> Yes, but that seems to be the only "protection" for the MTA. Looking 
>> at our figures Spamhaus rejections (although the single most source of 
>> rejections) account for only 20% of our rejections after greylisting 
>> (not sure if rejections occur before or after greylisting). For 
>> instance I reject almost as much because of bogus HELOs. Which is also 
>> part of BarricadeMX.
>>
>> Kai
>>
>>   
> 
>  From my testing it goes:
> 
> greet pause
> rbls
> greylist
> bad user
> 
> I'm guessing that the bogus HELOs would be around the rbl time. 
> greylisting doesn't reject until the rcpt to, and it does it before a 
> valid user check is done against LDAP.


All this sort of stuff will vary massively over each site as there are 
lots of variables (e.g. number of domains, average age of the domain, 
type of user, user habits etc. etc.) that govern the type of spam each 
site will get and thus the types of rejections that are possible.  So 
what works well for one site won't necessarily work well for the other.

While a couple of people (Res mainly) have noted that you can use 
milters, Exim routers or Postfix policy daemons to achieve some of the 
common stuff to get rid of spam at the MTA level - the reason we chose 
to develop BarricadeMX as an SMTP proxy was because none of these 
methods gave us enough control over what we wanted to be able to do. 
For example - the milter API does not allow you to instruct Sendmail to 
rate limit it's command reponses or give feedback on how many other 
concurrent connections a given IP address has etc. it also suffers from 
the fact all the communications have to go via a socket etc.

Here are some typical stats from a running BarricadeMX system, not all 
of the percentages relate to rejections, some are informational only and 
some are not enabled - but it gives an idea of the amount of tests that 
are carried out.

214-2.0.0 smtpf/1.0.146 (runtime)
214-2.0.0 start-time=Wed, 25 Jul 2007 00:03:08 -0400
214-2.0.0 age=125422
214-2.0.0 active-connections=725
214-2.0.0 high-connections=1382 (100.00%)
214-2.0.0 high-connections-per-second=22 (100.00%)
214-2.0.0 high-session-time=4624 (100.00%)
214-2.0.0 total-KB=7153778 (100.00%)
214-2.0.0 CLIENTS=1237908 (100.00%)
214-2.0.0 dropped=1049469 (84.78%)
214-2.0.0 data-354=91549 (7.40%)
214-2.0.0 client-io-error=164995 (13.33%)
214-2.0.0 client-timeout=12933 (1.04%)
214-2.0.0 server-io-error=55353 (4.47%)
214-2.0.0 admin-commands=2 (0.00%)
214-2.0.0 auth-pass=0 (0.00%)
214-2.0.0 auth-fail=0 (0.00%)
214-2.0.0 bogus-helo=857 (0.07%)
214-2.0.0 concurrent=587 (0.05%)
214-2.0.0 connect-bl=82395 (6.66%)
214-2.0.0 connect-lan=0 (0.00%)
214-2.0.0 connect-localhost=5 (0.00%)
214-2.0.0 connect-relay=5 (0.00%)
214-2.0.0 connect-wl=961 (0.08%)
214-2.0.0 dns-bl=416763 (33.67%)
214-2.0.0 dns-gl=24269 (1.96%)
214-2.0.0 dns-wl=0 (0.00%)
214-2.0.0 ehlo-no-helo=29930 (2.42%)
214-2.0.0 helo-claims-us=0 (0.00%)
214-2.0.0 helo-ip-mismatch=33414 (2.70%)
214-2.0.0 helo-schizophrenic=7282 (0.59%)
214-2.0.0 idle-retest-timer=87 (0.01%)
214-2.0.0 rate-client=40199 (3.25%)
214-2.0.0 rate-throttle=8105 (0.65%)
214-2.0.0 client-ip-in-ptr=0 (0.00%)
214-2.0.0 client-ptr-required=311762 (25.18%)
214-2.0.0 client-ptr-required-error=18088 (1.46%)
214-2.0.0 rfc2821-strict-helo=12541 (1.01%)
214-2.0.0 smtp-command-non-ascii=3055 (0.25%)
214-2.0.0 smtp-command-pause=89673 (7.24%)
214-2.0.0 smtp-drop-after=1152 (0.09%)
214-2.0.0 smtp-drop-unknown=452 (0.04%)
214-2.0.0 smtp-enable-esmtp=350765 (28.34%)
214-2.0.0 smtp-greet-pause=195045 (15.76%)
214-2.0.0 smtp-reject-delay=0 (0.00%)
214-2.0.0 uri-bl-helo=1658 (0.13%)
214-2.0.0 uri-bl-ptr=7906 (0.64%)
214-2.0.0 SENDERS=671843 (100.00%)
214-2.0.0 null-sender=68425 (10.18%)
214-2.0.0 call-back-cache=0 (0.00%)
214-2.0.0 call-back-made=0 (0.00%)
214-2.0.0 cli-envelope=0 (0.00%)
214-2.0.0 client-is-mx=64122 (9.54%)
214-2.0.0 grey-continue=6468 (0.96%)
214-2.0.0 grey-tempfail=54954 (8.18%)
214-2.0.0 mail-bl=129 (0.02%)
214-2.0.0 mail-wl=300 (0.04%)
214-2.0.0 mail-parse=1238 (0.18%)
214-2.0.0 require-sender-mx=530 (0.08%)
214-2.0.0 require-sender-mx-error=1138 (0.17%)
214-2.0.0 siq-query-cache=0 (0.00%)
214-2.0.0 siq-query-made=0 (0.00%)
214-2.0.0 siq-score-reject=0 (0.00%)
214-2.0.0 siq-score-tag=0 (0.00%)
214-2.0.0 spf-pass=16970 (2.53%)
214-2.0.0 spf-fail=2678 (0.40%)
214-2.0.0 spf-none=176221 (26.23%)
214-2.0.0 spf-neutral=3591 (0.53%)
214-2.0.0 spf-softfail=8241 (1.23%)
214-2.0.0 spf-perm-error=555 (0.08%)
214-2.0.0 spf-temp-error=7835 (1.17%)
214-2.0.0 uri-bl-mail=19467 (2.90%)
214-2.0.0 RECIPIENTS=201118 (100.00%)
214-2.0.0 rcpt-reject=51545 (25.63%)
214-2.0.0 one-rcpt-per-null=9 (0.00%)
214-2.0.0 rcpt-bl=0 (0.00%)
214-2.0.0 rcpt-wl=49 (0.02%)
214-2.0.0 rcpt-parse=4 (0.00%)
214-2.0.0 MESSAGES=95646 (100.00%)
214-2.0.0 msg-accept=81757 (85.48%)
214-2.0.0 msg-discard=0 (0.00%)
214-2.0.0 msg-drop=331 (0.35%)
214-2.0.0 msg-reject=13497 (14.11%)
214-2.0.0 dsn-sent=216 (0.23%)
214-2.0.0 7bit-headers=0 (0.00%)
214-2.0.0 cli-content=0 (0.00%)
214-2.0.0 infected=0 (0.00%)
214-2.0.0 junk-mail=0 (0.00%)
214-2.0.0 line-length=0 (0.00%)
214-2.0.0 message-limit=0 (0.00%)
214-2.0.0 message-size=0 (0.00%)
214-2.0.0 ret-pass=0 (0.00%)
214-2.0.0 ret-fail=0 (0.00%)
214-2.0.0 ret-ttl=0 (0.00%)
214-2.0.0 strict-dot=0 (0.00%)
214-2.0.0 uri-bl=13475 (14.09%)
214-2.0.0 uri-max-limit=0 (0.00%)
214-2.0.0 uri-max-test=3685 (3.85%)
214 2.0.0 End.

I can also tell you that no one who tries this will get the same results 
- there are simply too many factors.  What I can tell you is that no MTA 
  or MTA plug-in can do some of these tests as they simply are not able 
to (the only thing that might come close is qpsmtpd) and it will 
significantly reduce the amount of messages input to your MTA and to 
MailScanner to allow it to scale better on the same amount of hardware 
as that was it's original design goal.

Kind regards,
Steve.