how to bock mailservers that have only an ip address

[Jim Holland]

On Mon, 1 May 2006, John Rudd wrote:

> >> 2) I'm not sure if require_rdns can be made to exempt those in certain
> >> IP blocks, those who have done SMTP-AUTH, etc ... but mimedefang can
> >> (that's why I do the check in filter_sender instead of filter_relay).
> >
> > You can control whitelisting in your access file.
> 
> Which doesn't answer the part about SMTP-AUTH.  Which was the important 
> part, really.
> 
> >> 3) mimedefang lets you do LOTS of other checks, is incredibly 
> >> flexible,
> >> and you modify behaviors in perl instead of sendmail cf expressions 
> >> ...
> >> I much prefer perl to sendmail cf files.  It can check for recipients 
> >> on
> >> remote relays (similar to milter-ahead), HELO/EHLO verification, and
> >> even attachment filename/filetype checks, anti-virus checks, and
> >> SpamAssassin.
> >
> > Having had a look at mimedefang it looks as if it is an alternative
> > package to MailScanner, rather than complementary, and hence not at all
> > compatible with it.
> 
> It is both alternative and complementary.  Depending on how you use it.
> 
> Yes, you can do virus, filename, filetype, and spam checks in
> mimedefang.  Or not.  You can vary your actions for each check, too.  
> Just like in mailscanner.
> 
> You can also do checks during SMTP time, unlike mailscanner.  And you
> can do checks during the connection based upon the relay, the helo/ehlo
> string, the sender, sender-options, and recipients.  For example, I can
> reject all connections from hosts whose IP address appears in their
> hostname.  Or reject all connections from hosts whose forward and
> reverse DNS don't match.  I can't _easily_ do those with mailscanner.  
> And I definitely can't do them during the SMTP session.

Sadly, since starting to use require_rdns I have found that there are so
many systems whose PTR records point to invalid hostnames that I am
seriously thinking of just limiting its use to blocking those systems with
no PTR records at all.  The manual workload in whitelisting all the
genuine systems with apparently forged addresses is very discouraging.
 
> Last, I can do my virus scanning in mimedefang, and thus reduce the
> amount of anti-spam work that mailscanner does by eliminating the
> viruses first.  Something several people have wanted to do in
> mailscanner, but that Julian has said would require too much of a
> re-write to accomplish any time soon.

Point taken.  However I do appreciate the much more satisfactory highly
configurable silent virus approach taken by MailScanner than the
recommended mimedefang approach: "we believe that on balance, it's better
to bounce a virus than to silently discard it. It's almost never a good
idea to hide a problem".
 
> > All incoming mail would get parsed by mimedefang,
> > with all attachments being extracted etc, only to have the same mail 
> > then
> > being reprocessed and split again by MailScanner.  This looks like an
> > incredible waste of resources,
> 
> AFAIK, it's a small drop in the bucket compared to running SpamAssassin.
> 
> Though, mimedefang does have an option for turning off various checks..  
> so I've asked for a way to turn off MIME decoding (and thus turning off
> "filter" and "filter_multipart", but leaving "filter_begin" and
> "filter_end" on, as I don't think those need MIME decoding in order to
> run).
> 
> > especially if it is just to make use of a
> > couple of mimedefang features such as milter-ahead, HELO/EHLO 
> > verification
> > or blocking of servers with no PTR records.  Carrying out filename/
> > filetype checks, anti-virus checks, and running SpamAssassin through
> > mimedefang while also using MailScanner is clearly pointless as
> > MailScanner does it all so much better.
> 
> Who said anything about doing filename/filetype checks, anti-virus 
> checks, and running spam assassin in _both_?
> 
> IMO, if I was going to do, or recommend, a hybrid approach, I'd do:
> 
> 0) greet_pause and spamhaus sbl & xbl in sendmail
> 
> 1) relay, helo, sender, recipient checks, and clamd anti-virus in 
> mimedefang
> 
> 2) spam assassin, filename, filetype, phishing, and sophos anti-virus 
> in mailscanner (or some other scanner besides clamav, to have a safe 
> 2nd layer of anti-virus scanning)
> 
> 0 & 1 reduce the number of messages you're feeding through Spam 
> Assassin, which is probably going to be your single biggest resource 
> hog (even if you are MIME decoding every message in both places).  I 
> slightly like the filename and filetype checks in mailscanner better 
> than in mimedefang, and if you did find a way to turn off mime decoding 
> in mimedefang, you'd have to do it in mailscanner instead anyway.

Thanks for this very interesting set of suggestions.
 
> Though, honestly, I prefer to do it all in mimedefang these days.  I 
> have grown to be annoyed by the dual queue approach, and I like to 
> reject as many messages as possible during the SMTP transaction.

I agree with this in principle and do adopt this approach.  I wonder
however if anyone has done any research into one aspect of the behaviour
of some spammers: I suspect that when a spam message is rejected directly,
they then pass the message off to a different server, using a round-robin
approach and making multiple delivery attempts from different locations in
the hope that eventually one of them will get through.  If this is indeed
a significant element in spamming behaviour then the overall traffic load
might be less by simply accepting spam and then quarantining it than by
trying to reject it at MTA level.
 
> > I see the benefits of plugins that work directly with sendmail for
> > additional MTA-related functionality, but using mimedefang filters for
> > that purpose together with MailScanner seems incredible overkill.
> >
> >> Last week I posted a URL to my mimedefang-filter (which is where all 
> >> of
> >> your site-specific perl code goes).  I'm about to re-do it, though
> >> (clean up the code some, move some code around to sub-routines so it's
> >> easier to customize certain details, have different versions for 
> >> people
> >> who may want to skip virus/attachment/anti-spam checks if they're 
> >> doing
> >> that somewhere else, etc.).  I'll post about the update in a couple
> >> days.
> >
> > Do you have an integrated solution that ensures that MailScanner 
> > continues
> > to do what it does best while disabling all the MailScanner-type 
> > features
> > in mimedefang so as to avoid the enormous performance hit that will 
> > happen
> > if all messages are MIME-decoded twice?
> 
> With the exception of actually turning off the mimedecoding?  yes.
> 
> Just have filter, filter_multipart, and filter_end automatically accept 
> all messages without even looking at them (ie.  immediately do:
> 
> return action_accept();
> 
> in filter and filter_multipart, and just immediately return in 
> filter_end).

> If you're not going to do the virus checking or any header manipulation
> in filter_begin, you can do the same thing there too.  Then just do the
> things you want in filter_relay, filter_helo, filter_sender, and
> filter_recipient.  Though, I do all of the filter_relay and filter_helo
> stuff in filter_sender, so that I can exempt based on SMTP-AUTH.
> 
> > Surely the logical approach is to
> > write milters that work directly with sendmail (as with milter-ahead)
> > instead of filters that work with mimedefang which then hooks into
> > sendmail?
> 
> uh... what do you think mimedefang is?  it's a milter.  It is exactly a 
> "milter that works directly with sendmail".  It's just more flexible 
> than milter-ahead, that's all.
 
Thanks for all your explanations.  I will give it a go, but foresee much
work in getting to terms with all the options involved, how to manage the
additional blacklisting/whitelisting etc, fine tuning of parameters . . .

Regards

Jim Holland
System Administrator
MANGO - Zimbabwe's non-profit e-mail service