Mailscanner milter to reject high score spam at MTA level
Shawn Iverson
iversons at rushville.k12.in.us
Tue Aug 14 14:56:59 UTC 2018
Dear MailScanner users:
I am officially working on creating a lightweight milter for MailScanner.
This milter will not provide MTA protocol rejection for postfix, due to the
severe performance penalty it would cause. All mail will be intercepted,
accepted, and silently dropped from the postfix queue and placed in a
MailScanner queue.
I have a working prototype, and it is processing mail! It is in need of
heavy refactoring and some bug squashing.
Currently it attempts to create a postfix formatted queue file (very ugly,
who thought up this file format???!!!). I may instead create a new Milter
Processor for MailScanner that reduces the overhead of doing this and can
read the incoming email in a simple line-by-line format. This may also
increase performance overall and reduce all the conversions happening.
The other side of the coin is what to do when MailScanner is done
processing mail. Currently, it generates a postfix queue file and drops it
into postfix incoming directory. It should not do this but instead drop
the message into postfix using native postfix tools. That will be the next
part I tackle as part of the Milter Processor.
Why am I doing this? I want to place MailScanner back in a good standing
with Postfix folks (at least when the milter + postfix method is in use).
I have no plans of removing the old method but rather provide a more
supported path for postfix users.
Wish me luck. I could be heard across the neighborhood when MailScanner
processed an email from the Milter for the first time! :D
On Sat, Aug 11, 2018 at 9:58 AM David Jones <djones at ena.com> wrote:
> On 08/11/2018 08:52 AM, Shawn Iverson wrote:
> > David,
> >
> > I agree that this is true, and part of my lack of motivation to do it.
> > One reason I wanted it as an option was to reconcile the ongoing
> > conflict with the postfix community and return MailScanner to good
> > standing to this community. Weitze has been very stern about
> > MailScanner directly tapping the postfix queues.
> >
> > Perhaps an alternative option would be to create a fast MailScanner
> > milter that behaves more like the HOLD queue. Basically just a milter
> > that immediately fires back accept to postfix and places all the
> > messages in a MailScanner HOLD queue as opposed to a postfix HOLD
> > queue. Doing so would maintain speed, simplicity, and be more compliant
> > with postfix. The code would also be very simple.
> >
> > Then, as you say, if you need MTA level functionality for SA, use other
> > software and methods.
> >
> >
>
> This light MS milter would make a lot of sense based on your goal to get
> compliant with Postfix and back "in" with the Postfix community. +1
>
> >
> > On Sat, Aug 11, 2018 at 9:39 AM David Jones <djones at ena.com
> > <mailto:djones at ena.com>> wrote:
> >
> > On 08/11/2018 08:15 AM, Shawn Iverson wrote:
> > > I have been planning for a MailScanner milter for quite some
> > time. I
> > > have been specifically studying rpamd's milter source for this
> > purpose.
> > > Alas, lack of time and lack of money are always an issue, and I
> > put a
> > > lot of hours in my day job. As Jerry would say, I like to eat
> > and have
> > > a roof over my head :D
> > >
> > > If I do find the time to build a milter, performance will
> > definitely be
> > > impacted. The reason is that postfix will have to keep each
> session
> > > open for the duration of scanning, and each MailScanner child
> > would have
> > > to issue a callback to postfix after scanning the spam so that
> > postfix
> > > can responds to the connection appropriately (i.e. reject or
> > accept).
> > > This will slow down mail processing considerably. If I do this,
> > I am
> > > going to keep the HOLD queue around, so you would have to choose
> > between
> > > speed or MTA level rejection functionality.
> > >
> > >
> > >
> >
> > My gut tells me that this is going to be so slow, that it's not
> > going to
> > be worth the time to put into it. If you want to reject at MTA time,
> > throw in amavis-new or spamd (not rspamd) using the same
> SpamAsssassin
> > rules and Bayes DB to get most of the same features as MailScanner
> > during the SMTP conversation. Then the mail that gets through can be
> > filtered by MailScanner for it's extra features that make it unique.
> >
> > I understand there are different local legal requirements around the
> > world that if email is accepted at MTA time then it has to be passed
> on
> > to the end user's mailbox. If you are located in one of these
> > countries, then this would be more of an issue. But since I am in a
> > country that doesn't have this legal requirement, I do block email
> > post-MTA by MailScanner.
> >
> > The majority of my spam is blocked at the MTA level already by highly
> > tuned RBLs and postscreen's RBL weighting which is very, very good.
> > Only a small percentage of spam that is zero-hour or from compromised
> > accounts makes it to MailScanner.
> >
> > I highly recommend the Invaluement RBL. It's very accurate -- only
> > 1 or
> > 2 false positives over 5+ the years. This RBL is very cost effective
> > and has allowed me to disable all Spamhaus RBL checks in SpamAssassin
> > saving thousands of dollars a year. (We have too high a volume to
> stay
> > under the free usage limits of Spamhaus so we were having to pay for
> > the
> > RBL feed.)
> >
> > >
> > >
> > >
> > > On Tue, Aug 7, 2018 at 10:52 AM David Jones via MailScanner
> > > <mailscanner at lists.mailscanner.info
> > <mailto:mailscanner at lists.mailscanner.info>
> > > <mailto:mailscanner at lists.mailscanner.info
> > <mailto:mailscanner at lists.mailscanner.info>>> wrote:
> > >
> > > On 08/07/2018 05:03 AM, info at schroeffu.ch
> > <mailto:info at schroeffu.ch> <mailto:info at schroeffu.ch
> > <mailto:info at schroeffu.ch>>
> > > wrote:
> > > >
> > > > Hi Mailscanner friends,
> > > >
> > > > is there any progress to make MailScanner usable as a
> > postfix milter?
> > > > The most biggest problem I have is, SPAM is not possible to
> > > reject when
> > > > reaching a high score at MTA level. For my understanding,
> > connect
> > > via
> > > > milter instead of queue ^HOLD would be the solution.
> > > >
> > > > For the next decade we are still using MailScanner instead
> > of others
> > > > like Rspamd, because MailScanner is like a mail suite for
> mail
> > > security,
> > > > but if there will never be the possibility to reject at
> > MTA level
> > > the
> > > > high score spam, we will also change in 1-3 years while
> > replacing
> > > the OS
> > > > beyond.
> > > >
> > >
> > > One of MailScanner's strongest features is it's batch mode
> > processing
> > > that will allow it to handle a very high volume of mail
> > flow. I doubt
> > > that MailScanner will ever be changed to run as a milter for
> this
> > > reason.
> > >
> > > I tried rspamd and found it wasn't as good as the author
> > claims so no
> > > reason to try to use that as a milter. It also wasn't as
> > fast as it
> > > claims. I could not send high volumes of mail through it
> > like I could
> > > with MailScanner.
> > >
> > > If you want to block high scoring spam at the MTA level, I
> > suggest
> > > using
> > > amavis or spamd with the same SA rulesets as MailScanner.
> > This will
> > > get
> > > you most of the power of MailScanner's blocking at the MTA.
> > >
> > > https://wiki.apache.org/spamassassin/IntegratedInMta
> > >
> > > If you you use postscreen and postwhite at the Postfix MTA
> > level, you
> > > can block most of the obvious spam with a tuned list of
> > RBLs. See the
> > > SA users mailing list over the past year for details on this
> > from me
> > > and
> > > a few others.
> > >
> > > I suggest setting up a quick test VM with iRedmail to get a
> good
> > > example
> > > of how to do TLS and amavis integration well with Postfix.
> > >
> > > --
> > > David Jones
> > >
> > >
> > > --
> > > MailScanner mailing list
> > > mailscanner at lists.mailscanner.info
> > <mailto:mailscanner at lists.mailscanner.info>
> > > <mailto:mailscanner at lists.mailscanner.info
> > <mailto:mailscanner at lists.mailscanner.info>>
> > > http://lists.mailscanner.info/mailman/listinfo/mailscanner
> > >
> > >
> > >
> > > --
> > > Shawn Iverson, CETL
> > > Director of Technology
> > > Rush County Schools
> > > 765-932-3901 x1171
> > > iversons at rushville.k12.in.us
> > <mailto:iversons at rushville.k12.in.us>
> > <mailto:iversons at rushville.k12.in.us
> > <mailto:iversons at rushville.k12.in.us>>
> > >
> > >
> >
> > --
> > David Jones
> >
> >
> >
> > --
> > Shawn Iverson, CETL
> > Director of Technology
> > Rush County Schools
> > 765-932-3901 x1171
> > iversons at rushville.k12.in.us <mailto:iversons at rushville.k12.in.us>
> >
> >
>
>
> --
> David Jones
>
--
Shawn Iverson, CETL
Director of Technology
Rush County Schools
765-932-3901 x1171
iversons at rushville.k12.in.us
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mailscanner.info/pipermail/mailscanner/attachments/20180814/1134da86/attachment.html>
More information about the MailScanner
mailing list