whitelist_to getting exploited

[Furnish, Trever G]

Thanks for your comments, Glenn.  Some responses below.

> -----Original Message-----
> From: mailscanner-bounces at lists.mailscanner.info 
> [mailto:mailscanner-bounces at lists.mailscanner.info] On Behalf 
> Of Glenn Steen
> Sent: Saturday, December 30, 2006 6:39 AM
> To: MailScanner discussion
> Subject: Re: whitelist_to getting exploited
> 
> Hi Trever,
> 
> Just a few odd comments below...
> 
> > > Of Ramprasad
> > > Sent: Friday, December 29, 2006 5:22 AM
> > > To: MailScanner discussion
> > > Subject: Re: whitelist_to getting exploited
> > >
> > > On Fri, 2006-12-29 at 19:34 +1000, Res wrote:
> > > > On Fri, 29 Dec 2006, Ramprasad wrote:
> > > But user-1 wants all mails including spam  , not others
> > >
> > > For eg If I want to allow abuse at mydomain to get all mail without 
> > > check someone sends a mail To:the_top_man at domain,abuse at domain
> > >
> > > Then this mail would bypass spam checks and reach 
> > > the_top_man at domain 
> > > Obviously this would be a concern to everyone , how are you folks 
> > > getting over this issue
> >
> > Mailscanner can't split one message into several and treat them 
> > differently based on recipient.  Doing so would risk queue filename 
> > conflicts.
 
> This should be possible to handle....:-).

Sure -- anything's possible.  And I suppose I never actually
read that this is the reason MS doesn't do the splitting --
I just assumed that's the reason.

> > There are some definite caveats to consider though:
> >         - you'll use more bandwidth, since you're
> >           delivering multiple copies of a message where
> >           before you only delivered one.  This may or may
> >           not be significant for you.
> 
> With gateway systems (which is a very common setup, after all, of
> MailScanner) this is generally not a concern, since you will 
> have a very much more capable LAN/"internal WAN" link than 
> "internet-facing" link.

Good point.  In my case it's only significant because the
'internal WAN' links are much smaller than the WAN link, AND
because we don't impose any size limit on incoming messages
(because we lack a suitable replacement mechanism to give
to users for receiving large files. :-( ).

> >         - you'll increase the number of rows in your
> >           mailwatch tables, if you're using mailwatch.
> >                 - However, mailwatch 1.x is 'broken' in that
> >                   it only records one recipient per message
> >                   anyway, so while you're increasing the load
> >                   a bit, you also may be saving yourself a
> >                   different headache later.
> 
> Both these are true, and if I understood how Steve intends to 
> handle these for multiple recipient mails in 2.0 (fixing the 
> broken behaviour of 1.x) the first point will continue to be 
> a real concern for sites with large amounts of messages... 
> Splitting will likely make it one of your jobs to keep on top 
> of daily. Sigh. One more ...:-). But if one has a low volume 
> setup, it doesn't matter that much.

In my case the mailwatch bug mentioned above was enough of a problem
that I had to either fix it myself or replace the whole system
(including mailscanner) with some other tool.  Management dictated 1.)
use of quarantine, 2.) allowing users to release their own messages, and
3.) total lack of authentication, unless it was tied into Active
Directory.  I took a cue from Steve's notes for 2.0 and created a
separate table for relating message IDs to message recipients, then
changed all of the queries on the pages that I was interested in so that
they use the new table for queries that need to list all messages to a
given recipient.  That also improves performance, since the new table is
a small fraction of the maillog table -- nightly reports were taking
many hours, but now they take only a few minutes.

However, that kind of change also "breaks" the reporting interface,
among other things, so I have two mailwatch installs -- one that is just
a stock 1.0 install for the most part, and one that is heavily modified
to present a very stripped down interface for individual users and to
use the new table for improved performance and accurate lists of the
messages to their addresses.

My needs might also be a lot different from everyone else's -- I need to
keep around ten days worth of messages in the database and have it be
responsive enough to let users browse around quickly and to generate
nightly reports in only a few minutes.  We get about 180k messages per
day, so that's almost two million messages in the database at all times.
And for the web interface, I needed the users to be able to see a very
stripped down version that shows only their own messages without any
authentication at all -- they get a link and a report in email each
morning and can only view their own quarantined messages.  No searching,
reporting, or authenticating.  The only "authentication" is receipt of
the URL via email -- if you received the nightly report, then you have a
URL that will let you into the online version of that report for viewing
and releasing individual messages.

The changes I made are pretty straightforward, but they're also very
specific to my company's needs, enough so that I don't believe they'd be
useful for anyone else.  And I wouldn't want to distract from mw2.0.
And I anticipate much personal pain at some point in the future when I
decide to try to port the functional changes over to a mw2.0 install.


> >         - you'll increase the number of log entries -- this
> >           is probably insignificant.
> Agreed.
> 
> >         - you'll increase the mailscanner processing load,
> >           since e.g. one message may become five messages.
> 
> The worst "hog" in MS is SA, and with the SpamAssassin result 
> cache feature on, you really take the sting out of this one. 
> True, you'll likely see a bit of load from AV scanners etc, 
> but SA should yield only the cache fingerprint "cost" and 
> nothing more.

Good point -- I hadn't considered that!

> > I used to split all inbound messages.  I wish I still 
> could, but in my 
> > case I started bumping against the limits of my hardware 
> and opted to 
> > gain some performance by turning off the splitting.
> 
> Do you by any chance run BDC still? It can "hurt" things 
> bad... Or do you have a lot of BLs in MS? That could well be 
> "hurtfull too, depending on what limit you encounter... Or 
> was it the MW bit you mention? Hopefully 2.0 will make a lot 
> of difference there:-)

Did you mean to write DCC, not "BDC"?  I'm not familiar with "BDC" in a
mailscanner context.  If you meant DCC, I don't run Pyzor or DCC.  I do
use Razor, but didn't have enough confidence in the others to use them.

I don't use BLs in MailScanner -- ideally those would be at the MTA
level, but I prefer to be able to weight different BLs differently, so I
only use most BLs at the SA level.  The only BL I currently use at the
MTA level is SBL+XBL (which has been nothing short of amazingly
effective).

Regarding MW contributing to the load, it was really only a heavy
resource user when I was browsing the web interface.  The changes I made
seem to have helped out tremendously there.

> Anyway, as said, just a few random comment from a mind 
> definitely still on holiday leave:-) Best Regards & Happy New Year!
> --
> -- Glenn
> email: glenn < dot > steen < at > gmail < dot > com
> work: glenn < dot > steen < at > ap1 < dot > se

Thanks for your thoughts, and happy new year to you too!

--
Trever