[Fwd: Some highlights about BarricadeMX]
steve.freegard at fsl.com
Thu Jun 14 20:48:47 IST 2007
Cross posting Anthony's reply to Rob as Anthony is not a member of this
-------- Original Message --------
Subject: Some highlights about BarricadeMX
Date: Thu, 14 Jun 2007 21:16:02 +0200
From: Anthony Howe <achowe at snert.com>
To: Steve Freegard <steve.freegard at fsl.com>
CC: barricademx at snertsoft.info
Steve, please forward my reply to the MailScanner list. I'm posting to
the BarricadeMX list as it might be of some interest there too.
Rob Poe (from the MailScanner list) wrote:
> So, I understand that Fort and Snert have partnered up to stop spam
> at the MTA ...... and obviously it will be a proprietary method, but
> inquiring minds still want to know .. Anyone have any kind of idea on
> the techniques?
The usual suspects. BarricadeMX blends many of the SnertSoft milters
into one comprehensive package.
a) Enhanced grey-listing which applies two optimisations (one to address
mail server pools sharing a common queue ie. gmail.com; the other, once
a message from a host passes greylisting messages from that host/domain
for all subsequent messages will pass, ie. pay the grey-list delay ONCE
b) Enhanced Message-ID as Email Watermark, which can be used to filter
backscatter and skip content filtering if a message is identified as
reply to previously sent mail.
MailScanner had a patch applied to take advantage of EMEW. If combined
with dns-gl it would be possible to by-pass both pre and post-DATA
filtering. Of course we have dns-bl and dns-wl support too.
c) Some other interesting tests are "client is MX" to softens PTR
requirements and IP-in-PTR ie. looks like dynamic IP tests.
d) Another interesting combination is multiline welcome banner combined
with 500ms or 1s command-pause will catch out
e) The multicast/unicast cache deals with the problem of sharing
call-ahead, grey-listing, and other global data between MXes. It is fast
f) Client Rate Throttling, Global Rate Throttle, & Client Concurrency.
Nothing particularly spectacular there, sendmail already has this and
ours is similar, though we apply ours in the accept() loop.
g) URLBL application to HELO and MAIL FROM:. Not a new idea, but not
widely used either.
h) Tar-pitting of negatives SMTP responses. Handy for dictionary like
probing during a SMTP session.
i) EHLO no HELO and schizophrenic HELO checks. By disabling ESMTP you
catch some spamware that assume EHLO and fail to fall back on HELO when
rejected. Also if they EHLO/HELO with one name, but fail to EHLO/HELO
with the same name on the next or subsequent attempts you can reject
too. Note that RFC 2821 HELO/EHLO is equivalent to RSET once the first
HELO is accepted.
There is more. So much I forget what is novel and what has been done
before. You have to read the documentation to get see for yourself.
> I have 2 clients. One is a trucking firm. They get a lot of trade
> industry magazines (which SpamAssassin hates) and a lot of people in
> the trucking industry who think it fine that they have a 400x400
> inline .gif with their signature and other misc. stuff, in an email
> with a "Hey give me a call if you want me to service your loads" ...
> yeah SA hates that too..
BarricadeMX can do some real-time content filtering such as clamav,
spamassassin, and uribl tests in which it will reject or discard.
However, it is limited by the proxy design in that it cannot tag nor
quarantine messages based on content (we don't use temp. or queue
files). milter-spamc, MailScanner, or similar is still required for that
level of functionality. So things like inline images and their like will
require tweaking of SpamAssassin as with MailScanner.
> The other is a law firm. They want Artificial Intelligence for their
> spam filter. Everything that ISN'T SPAM gets delivered and
> everything that IS SPAM doesn't -- no errors.
No anti-spam technique is perfect. Expectations of such are unrealistic.
We just don't have enough good AI for such yet. (I'll note it as an
enhancement request though; insert Spock's brain.)
Depending on your paranoia and the set of BarricadeMX options you apply,
you can tone down the filtering or make it really strict. The FSL web
user interface for BarricadeMX provides three default levels of
strength, after which you can customise.
Regardless of the options you apply, a MailScanner or SpamAssassin
installation will still benefit as the machine will see less system
load. In your example, you turn on only those tests you are happy with
and then let a second level filter like MailScanner have its turn.
BarricadeMX was originally intended as a pre-filter for MailScanner to
reduce system load by catching the low hanging fruit. What we found with
BarricadeMX in beta testing, was it would catch the low hanging fruit on
really tall trees :-)
> Of course, if you go too strict on filtering you can lose emails, and
> if you go too lose you end up with horse sex in your inbox ... how
> does this product compare to current methods?
> Greylisting has helped tremendously, but there are still some
> seriously BRAIN DEAD admins running some seriously BRAIN DEAD email
> apps that don't ever retry.. So for the two above clients, it didn't
> work so well.
There is no suitable solution for brain dead brokeness that fails to
follow RFC 2821 recommended mail delievery strategies. If a customer
chooses to use broken software and the developer won't fix it, then all
you can do is white list them if you like them enough.
> (oh, and why do people make 10 outbound mail gateways, so a message
> that retries goes between the 10 different hosts, getting a greylist
> deny every time, but te delivery retry is > 1 minute) ???
We can fix ignorance, we can't fix stupidity. Really short retry times
less than 5 minutes are IMHO are just stupid and the result of pointy
clickity type mentalities performing system administration tasks in
which they are not competent in. However, rate throttling and the
multicast/unicast cache does help some aspects of this problem.
Another feature of BarricadeMX are the runtime and hourly stats (version
1.1 will have rolling 60 minutes stats too). Below is an example from a
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220-XXX.XXX.XXX ESMTP Welcome to smtpf
220 Copyright 2006, 2007 by SnertSoft. All rights reserved.
214-2.0.0 smtpf/1.0.146 (runtime)
214-2.0.0 start-time=Thu, 14 Jun 2007 11:57:12 -0400
214-2.0.0 high-connections=23 (100.00%)
214-2.0.0 high-connections-per-second=5 (100.00%)
214-2.0.0 high-session-time=350 (100.00%)
214-2.0.0 total-KB=973 (100.00%)
214-2.0.0 CLIENTS=5724 (100.00%)
214-2.0.0 dropped=4422 (77.25%)
214-2.0.0 data-354=76 (1.33%)
214-2.0.0 client-io-error=1076 (18.80%)
214-2.0.0 client-timeout=11 (0.19%)
214-2.0.0 server-io-error=113 (1.97%)
214-2.0.0 admin-commands=75 (1.31%)
214-2.0.0 auth-pass=0 (0.00%)
214-2.0.0 auth-fail=0 (0.00%)
214-2.0.0 bogus-helo=2 (0.03%)
214-2.0.0 concurrent=0 (0.00%)
214-2.0.0 connect-bl=12 (0.21%)
214-2.0.0 connect-lan=0 (0.00%)
214-2.0.0 connect-localhost=76 (1.33%)
214-2.0.0 connect-relay=76 (1.33%)
214-2.0.0 connect-wl=0 (0.00%)
214-2.0.0 dns-bl=2750 (48.04%)
214-2.0.0 dns-gl=38 (0.66%)
214-2.0.0 dns-wl=0 (0.00%)
214-2.0.0 ehlo-no-helo=1508 (26.35%)
214-2.0.0 helo-claims-us=0 (0.00%)
214-2.0.0 helo-ip-mismatch=110 (1.92%)
214-2.0.0 helo-schizophrenic=7 (0.12%)
214-2.0.0 idle-retest-timer=0 (0.00%)
214-2.0.0 rate-client=51 (0.89%)
214-2.0.0 rate-throttle=0 (0.00%)
214-2.0.0 client-ip-in-ptr=0 (0.00%)
214-2.0.0 client-ptr-required=873 (15.25%)
214-2.0.0 client-ptr-required-error=60 (1.05%)
214-2.0.0 rfc2821-strict-helo=30 (0.52%)
214-2.0.0 smtp-command-non-ascii=0 (0.00%)
214-2.0.0 smtp-command-pause=51 (0.89%)
214-2.0.0 smtp-drop-after=0 (0.00%)
214-2.0.0 smtp-drop-unknown=6 (0.10%)
214-2.0.0 smtp-enable-esmtp=2813 (49.14%)
214-2.0.0 smtp-greet-pause=65 (1.14%)
214-2.0.0 smtp-reject-delay=0 (0.00%)
214-2.0.0 uri-bl-helo=34 (0.59%)
214-2.0.0 uri-bl-ptr=206 (3.60%)
214-2.0.0 SENDERS=3411 (100.00%)
214-2.0.0 null-sender=18 (0.53%)
214-2.0.0 call-back-cache=0 (0.00%)
214-2.0.0 call-back-made=0 (0.00%)
214-2.0.0 cli-envelope=0 (0.00%)
214-2.0.0 client-is-mx=130 (3.81%)
214-2.0.0 grey-continue=40 (1.17%)
214-2.0.0 grey-tempfail=171 (5.01%)
214-2.0.0 mail-bl=1 (0.03%)
214-2.0.0 mail-wl=0 (0.00%)
214-2.0.0 mail-parse=4 (0.12%)
214-2.0.0 require-sender-mx=0 (0.00%)
214-2.0.0 require-sender-mx-error=0 (0.00%)
214-2.0.0 siq-query-cache=0 (0.00%)
214-2.0.0 siq-query-made=0 (0.00%)
214-2.0.0 siq-score-reject=0 (0.00%)
214-2.0.0 siq-score-tag=0 (0.00%)
214-2.0.0 spf-pass=97 (2.84%)
214-2.0.0 spf-fail=9 (0.26%)
214-2.0.0 spf-none=220 (6.45%)
214-2.0.0 spf-neutral=1 (0.03%)
214-2.0.0 spf-softfail=7 (0.21%)
214-2.0.0 spf-perm-error=0 (0.00%)
214-2.0.0 spf-temp-error=0 (0.00%)
214-2.0.0 uri-bl-mail=64 (1.88%)
214-2.0.0 RECIPIENTS=257 (100.00%)
214-2.0.0 rcpt-reject=9 (3.50%)
214-2.0.0 one-rcpt-per-null=0 (0.00%)
214-2.0.0 rcpt-bl=0 (0.00%)
214-2.0.0 rcpt-wl=0 (0.00%)
214-2.0.0 rcpt-parse=0 (0.00%)
214-2.0.0 MESSAGES=80 (100.00%)
214-2.0.0 msg-accept=60 (75.00%)
214-2.0.0 msg-discard=0 (0.00%)
214-2.0.0 msg-drop=0 (0.00%)
214-2.0.0 msg-reject=20 (25.00%)
214-2.0.0 dsn-sent=0 (0.00%)
214-2.0.0 7bit-headers=0 (0.00%)
214-2.0.0 cli-content=0 (0.00%)
214-2.0.0 infected=0 (0.00%)
214-2.0.0 junk-mail=0 (0.00%)
214-2.0.0 line-length=0 (0.00%)
214-2.0.0 message-limit=0 (0.00%)
214-2.0.0 message-size=0 (0.00%)
214-2.0.0 ret-pass=0 (0.00%)
214-2.0.0 ret-fail=11 (13.75%)
214-2.0.0 ret-ttl=0 (0.00%)
214-2.0.0 strict-dot=0 (0.00%)
214-2.0.0 uri-bl=9 (11.25%)
214-2.0.0 uri-max-limit=0 (0.00%)
214-2.0.0 uri-max-test=4 (5.00%)
214 2.0.0 End.
Anthony C Howe Skype: SirWumpus SnertSoft
+33 6 11 89 73 78 ICQ: 7116561 Sendmail Milter Solutions
More information about the MailScanner