Mailscanner and Sendmail Null Client

[John Rudd]

On Jul 22, 2004, at 9:03 PM, William Burns wrote:

> John Rudd wrote:
>
>> A) use libwrap on the scan array machines to only accept connections
>> from the primaries
>>
>> B) have the primaries in their relay domain (so they accept all
>> messages, no matter what the to/from might be, and thus the primaries
>> are responsible for being set up to not do promiscusious relaying)
>>
>>
> Aren't A and B redundant?

No, they're complimentary.  libwrap says "don't let any servers besides
these ones connect to me".  The relaying says "let the ones I allow to
connect then submit any combination of to/from addresses".

>> I'm trying to think of ways to set up a dedicated 'scanning array'
>> using
>> mailscanner.  What I envision is this:
>>
>> 1) If the primary email servers receive email that didn't come from
>> the
>> scan array, then they send that message to the scan array (this is
>> pretty trivial to set up under CommuniGate Pro).
 >
> Ok... If your "primary" side is really that trivial, then it sounds
> like
> you're done.
> I assume that your communigate config will still only accept mail for
> legit users in your domain.
> ..and that it will send mail to the scanner(s) even if it would
> otherwise have been delivered locally.

Correct.  And it will also send mail to the scanners if it would not
have been delivered locally (ie. outbound to other domains).  That way
we don't send out viruses, either.

>> 2) the scan array is a load balanced group of machines running
>> sendmail
>> + mailscanner.  When they're done scanning messages, they send all
>> messages back to the primary email servers (no matter what their
>> actual
>> destination might be).
>>
> So... The primaries have to relay for the scanners too.
> Unusual, but it should work.
>
> Just curious... Why bother?
> The scanner machines could send mail directly to end destinations just
> as easily as your "primary" server(s).

It would seem to be at least rude to have them send out messages when
they're not willing to accept them (due to the libwrap config).  That
makes them seem like clients instead of MTAs, and I wouldn't expect
anyone to want to receive mail from a machine that behaves more like a
client than an MTA.

Plus, I'd prefer to have them be transparent to the outside world
(except for the fingerprints they leave in the Received headers).  The
outside world should only directly interact with the primaries.

(though, currently, my mailscanner machines are in fact fully
functional sendmail machines ... they're the legacy mail servers for
our central campus service, and the CGP machines are our "new service",
but in the long run I want to make the sendmail/mailscanner machines
into an array of dedicated scanning hosts)

> Are you trying to avoid having to look at sendmail logs on the scanner
> machines? It's the only reason I can think of to send outbound mail
> back
> to a primary. Any postmaster/bounce messages will then be generated by
> the primaries, so the mail logs of errors should all be on the
> primaries.

Not the logs, the queues.  I don't want to have to do a lot of queue
management on the scanner machines, I don't want their queues backing
up when some remote host isn't behaving, etc.  I want all of that
management done on the primaries.  (as for the logs, I actually prefer
sendmail logs, in some ways, to CGP logs)

> How many different primaries will there be?
> If you've got multiple primaries hosting multiple domains, you'll need
> a
> mailertable on your scanners too.

Well, multiple primaries, multiple domains, but as far as the outside
world is concerned, it looks like one primary (it's a dynamic cluster,
4 machines that act like one machine ... 2 up front that manage
connections and 2 in the back that, essentially, manage data ... if a
front end and/or back end goes down, the service stays up, or if I
decide to take something off line to do maintenance, etc.)

> By the way, if that's your intent, then I don't think you'll be able to
> get your scanners to forward mail back to the "correct" smart-host.
> (The
> one that originally sent the scanner that piece of mail) So that's
> going
> to ruin the "simplicity" of your mail logs on the primaries, since
> outbound mail could leave through any primary, instead of the one
> primary that your end-user sends his outbound mail to.

They'll send it to the name of the cluster, which is a single name (and
thus, to the scanners, a single machine).  So they don't have to do
anything intelligent in that respect.

> And.. Unlike having mailscanner on the same machine as your primary,
> each  message will have two message IDs on your primary ... one for
> inbound and one for outbound mail.

Actually 4.  So that's already a problem, and one for which I'm writing
some scripts for handling the tracking/accounting (as well as
integrating in the message id's on the mailscanner hosts ... that's why
I was recently harping about how mailscanner doesn't do enough
syslog'ing).

The Communigate Pro cluster receives a message, gives it an ID.  It
then hands it off to the sendmail machines.

The sendmail machines receive it, give it an ID.  Mailscanner crunches
on it, and then sends it back to one of the CommuniGate Pro front ends.

The CGP front end gives it a new (third) ID, and if it's a local
recipient, it sends it to a back end machine for storage.

The CGP back end gives it a _fourth_ ID, and then stores it.

(the 1st and 4th happen no matter what, so it's already a problem I'm
dealing with, and since our legacy machines are still the actual
"@ucsc.edu" machines, I deal with 2, 3, and 4 for any mail coming in
from off campus ... so you might then say "why not keep that situation"
-- because that makes both my job, and the jobs of my users, more
complicated, and that's something we're trying to eliminate)

> ----------
> I wonder... How hard would it be to have mailscanner style sendmail-in
> and sendmail-out queue directories on your primaries, but to automate
> file transfer from the sendmail-in queue, to a sendmail-in queue on a
> dedicated mailscanner machine, and back from the sendmail-out queue
> directory.

That wouldn't be significantly different from my previous arrangement.

I had scripts that translated from CGP queue format to
mailscanner/sendmail queue format, and dropped messages into
mailscanner's incoming directory.  Then mailscanner would invoke a
second script (in place of its "sendmail2" config) which would
translate back.  The problem:  messages sometimes disappeared.  Since
mailscanner doesn't tell you when a silent virus is deleted, and I
couldn't see any obvious problems in my scripts, I decided that
continuing to attack the problem in that method would require a ton of
more work than I had the time to do.

Thus the "primary cluster and scanner array" solution.

Honestly, if I could have resolved that problem quickly, I would have
kept that.  Being able to get rid of my sendmail machines completely
would be the best solution.  I probably wont go back to those scripts,
though.  Instead, I've been working on making library .pm files for
mailscanner that will directly read and write in CGP queue format ...
but that's going to take me a while (more because I've got so little
time to work on it, than anything else).

-------------------------- MailScanner list ----------------------
To leave, send    leave mailscanner    to jiscmail at jiscmail.ac.uk
Before posting, please see the Most Asked Questions at
http://www.mailscanner.biz/maq/     and the archives at
http://www.jiscmail.ac.uk/lists/mailscanner.html