MailScanner and CommuniGate Pro (Was: Re: Load Balancing + File Locking)

Fri Oct 1 19:04:07 IST 2004

<x-flowed>
Vasiliy,

I'm working on a problem like this, but not the same.  (and, you might
remember that I'm the person who originally wrote ms2cgp and cgp2ms,
but then abandoned them; why?  because the tracking mechanisms
available me, both through CGP and MS, left me with a lot of doubt in
my ability to guarantee the ability to track every message without
losing any of them in the logs, so I have taken a new direcction).

What I'm doing now is working on a CGP "execute" script which takes a
message from CGP, then, using the SMTP perl module, sends it to my load
balancer's SMTP port (you could just as easily use a round-robin
approach, or any of a few other techniques), behind which sits an array
of standard sendmail+mailscanner+sophos systems.

It just so happens that the sendmail+mailscanner systems are also the
MX servers for our domain, but that's not a requirement.  The
arrangement could just as easily be a private array of systems that
aren't available to outside submitters (protected from those submitters
by tcpwrappers and/or sendmail access rules and/or network topology).

So, there is no "triple scanning" because the load balancer makes sure
the message goes to only one mailscanner system.  That also takes care
of the multiple paths problem, and whether or not mailscanner is doing
sequential or non-sequential processing (because it doesn't matter:
mailscanner can do whatever it wants, without competing with other
mailscanner processes).  And there's no need for file locking issues
(because there's no shared incoming directory).

I'll try to put up a web page with my current effort later today.

On Oct 1, 2004, at 10:35 AM, Boulytchev, Vasiliy wrote:

> Thoughts and setup  (mx is out of the questions, sorry, network is
> different
> here)
>
> A = Mail Server (CommunigatePro)
> S1 = Mailscanner node 1
> S2 = Mailscanner node 2
> S3 = Mailscanner node 3
>
> On A, we have all of our domains (hundreds).  I will not go into
> detail of
> how much redundancy this thing has, and why we cant simply point to the
> scanners with mx records, but here is my idea.  On A, have 2 NFS-shared
> folders (certainly on a private network and gig nics).  One Incoming,
> which
> is handled by cgp2ms and the other is Submitted, handled by ms2cgp.
> Now,
> say these three nodes keep grabbing batches of mail from these mount
> points,
> process them, and spit things out in the final Submitted location for
> Cgate
> to push the mail out.  Mailscanner should not have any issues with
> handling
> the mounted directories, the only thing I am worried about is file
> locking
> and fragmentation.
>         So the question is, how do I prevent (or is it a big deal),
> that
> lets say... There are 300 messages in the Incoming directory at any
> moment.
> Mailscanners pick that queue up, crunch on it on each node, and spit
> it out
> in the Submitted directory.  Now we start to ask these questions:
>         1.)     Am I tripple scanning each message and not getting any
> performance back.
>         2.)     As of right not, I think there are 2 ways MS goes
> through
> the Incoming directory, sequentially and non-sequentially (if there
> are more
> than x amount of messages).  How do I tell MS to share the load!
>         3.)     Maybe modify cgp2ms to spit things out into different
> Incoming directories, therefore walking right around the file locking
> problem.
>
> IDEAS?   Who else is doing this?
>
>
> Vasiliy Boulytchev
> Colorado Information Technologies, Inc.
> http://www.coinfotech.com
>
> -----Original Message-----
> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf
> Of Martin Hepworth
> Sent: Friday, October 01, 2004 10:59 AM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: Load Balancing + File Locking
>
>
> Vasiliv
>
> that's what's achieved by round-robin DNS.
>
> Or do you a central file store for all queues, quarantine dir etc. You
> could
> always have a third machine with the file stores on it that's mounted
> by all
> the other hosts, but you still have the issus of the fileserver going
> down...
>
> --
> Martin Hepworth
> Snr Systems Administrator
> Solid State Logic
> Tel: +44 (0)1865 842300
>
>
> Boulytchev, Vasiliy wrote:
>> :))))))))))))))   Brilliant! :)))))))))))))))))
>>
>> This is definitely a nice way to use round robin DNS... To your
>> advantage
>> even.  Sweet.
>>
>> Unfortunately this don't help us :(.   We must load balance this
>> sucker.
> If
>> we will not have loadbalancing, we shall achieve the same with running
>> sendmail on each mailscanner box, and vuala..... But I really want a
>> Mailscanner Cluster!  I want redundancy!
>>
>>
>> Vasiliy Boulytchev
>> Colorado Information Technologies, Inc.
>> http://www.coinfotech.com
>>
>> -----Original Message-----
>> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf
>> Of Denis Beauchemin
>> Sent: Thursday, September 30, 2004 2:56 PM
>> To: MAILSCANNER at JISCMAIL.AC.UK
>> Subject: Re: Load Balancing + File Locking
>>
>>
>> Boulytchev, Vasiliy wrote:
>>
>>
>>> Ladies and Gents,
>>>       The loads on the mail server require us to start moving
>>> Mailscanner with Spam and Virus filtering to other machines.
>>> Questions:
>>>       1.)     How does MS do the locking on files in the submitted
>>> directory (after cgp2ms)?  If NFS3 is used to mount that shared
>>> folder
>>> on say 3 servers designated to only scanning mail, how do we avoid
> run-ons?
>>>       2.)     Idea: if the local procs keep track of locking, then
>>> we can
>>> toss all that into mysql for example? :)))  Just thinking of ways to
>>> add scalability.
>>>
>>>
>>>
>>
>>
>> Vasiliy,
>>
>> We have 2 MS servers for incoming mail and they don't share any spool
>> directory.  They are separate servers and the load balancing is done
> through
>> the MX records.  Our 2 servers were not identical (one was about
>> twice as
>> fast as the other) and we had the following:
>> # host -t mx usherbrooke.ca
>> usherbrooke.ca mail is handled by 10 smtpe1.usherbrooke.ca.
>> usherbrooke.ca mail is handled by 10 smtpe2.usherbrooke.ca.
>> usherbrooke.ca mail is handled by 10 smtpe3.usherbrooke.ca.
>>
>> Here smtpe1 and smtpe3 are the same machine (the fastest one) with
> different
>> IP.  It received close to 2/3 of all mail.
>>
>> I just replaced the slowest one (smtpe2) by a machine identical to the
>> fastest one (smtpe1) so I asked our DNS guys to drop the smtpe3 name.
>>
>> Denis
>>
>
> **********************************************************************
>
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
>
> This footnote confirms that this email message has been swept
> for the presence of computer viruses and is believed to be clean.
>
> **********************************************************************
>
> ------------------------ MailScanner list ------------------------
> To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
> 'leave mailscanner' in the body of the email.
> Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
> the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>
> ------------------------ MailScanner list ------------------------
> To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
> 'leave mailscanner' in the body of the email.
> Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
> the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).

------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
</x-flowed>