MailScanner and CommuniGate Pro (Was: Re: Load Balancing + File Locking)

Fri Oct 1 19:25:57 IST 2004

John,  without your help with cgp2ms and ms2cgp last year, we would have
been completely stuck, thank you!  Too many questions arise when reading
your email.  What you are doing is extremely interesting to us, as well as
anyone else running into one server limitations.  Please post a website
regarding your efforts.  This should really be looked at by the Mailscanner
team.

THANKS GUYS!

Vasiliy Boulytchev
Colorado Information Technologies, Inc.
http://www.coinfotech.com

-----Original Message-----
From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On Behalf
Of John Rudd
Sent: Friday, October 01, 2004 12:04 PM
To: MAILSCANNER at JISCMAIL.AC.UK
Subject: MailScanner and CommuniGate Pro (Was: Re: Load Balancing + File
Locking)

Vasiliy,

I'm working on a problem like this, but not the same.  (and, you might
remember that I'm the person who originally wrote ms2cgp and cgp2ms, but
then abandoned them; why?  because the tracking mechanisms available me,
both through CGP and MS, left me with a lot of doubt in my ability to
guarantee the ability to track every message without losing any of them in
the logs, so I have taken a new direcction).

What I'm doing now is working on a CGP "execute" script which takes a
message from CGP, then, using the SMTP perl module, sends it to my load
balancer's SMTP port (you could just as easily use a round-robin approach,
or any of a few other techniques), behind which sits an array of standard
sendmail+mailscanner+sophos systems.

It just so happens that the sendmail+mailscanner systems are also the MX
servers for our domain, but that's not a requirement.  The arrangement could
just as easily be a private array of systems that aren't available to
outside submitters (protected from those submitters by tcpwrappers and/or
sendmail access rules and/or network topology).

So, there is no "triple scanning" because the load balancer makes sure the
message goes to only one mailscanner system.  That also takes care of the
multiple paths problem, and whether or not mailscanner is doing sequential
or non-sequential processing (because it doesn't matter:
mailscanner can do whatever it wants, without competing with other
mailscanner processes).  And there's no need for file locking issues
(because there's no shared incoming directory).

I'll try to put up a web page with my current effort later today.

On Oct 1, 2004, at 10:35 AM, Boulytchev, Vasiliy wrote:

> Thoughts and setup  (mx is out of the questions, sorry, network is
> different
> here)
>
> A = Mail Server (CommunigatePro)
> S1 = Mailscanner node 1
> S2 = Mailscanner node 2
> S3 = Mailscanner node 3
>
> On A, we have all of our domains (hundreds).  I will not go into
> detail of how much redundancy this thing has, and why we cant simply
> point to the scanners with mx records, but here is my idea.  On A,
> have 2 NFS-shared folders (certainly on a private network and gig
> nics).  One Incoming, which is handled by cgp2ms and the other is
> Submitted, handled by ms2cgp.
> Now,
> say these three nodes keep grabbing batches of mail from these mount
> points, process them, and spit things out in the final Submitted
> location for Cgate to push the mail out.  Mailscanner should not have
> any issues with handling the mounted directories, the only thing I am
> worried about is file locking and fragmentation.
>         So the question is, how do I prevent (or is it a big deal),
> that lets say... There are 300 messages in the Incoming directory at
> any moment.
> Mailscanners pick that queue up, crunch on it on each node, and spit
> it out in the Submitted directory.  Now we start to ask these
> questions:
>         1.)     Am I tripple scanning each message and not getting any
> performance back.
>         2.)     As of right not, I think there are 2 ways MS goes
> through
> the Incoming directory, sequentially and non-sequentially (if there
> are more than x amount of messages).  How do I tell MS to share the
> load!
>         3.)     Maybe modify cgp2ms to spit things out into different
> Incoming directories, therefore walking right around the file locking
> problem.
>
> IDEAS?   Who else is doing this?
>
>
> Vasiliy Boulytchev
> Colorado Information Technologies, Inc.
> http://www.coinfotech.com
>
> -----Original Message-----
> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf Of Martin Hepworth
> Sent: Friday, October 01, 2004 10:59 AM
> To: MAILSCANNER at JISCMAIL.AC.UK
> Subject: Re: Load Balancing + File Locking
>
>
> Vasiliv
>
> that's what's achieved by round-robin DNS.
>
> Or do you a central file store for all queues, quarantine dir etc. You
> could always have a third machine with the file stores on it that's
> mounted by all the other hosts, but you still have the issus of the
> fileserver going down...
>
> --
> Martin Hepworth
> Snr Systems Administrator
> Solid State Logic
> Tel: +44 (0)1865 842300
>
>
> Boulytchev, Vasiliy wrote:
>> :))))))))))))))   Brilliant! :)))))))))))))))))
>>
>> This is definitely a nice way to use round robin DNS... To your
>> advantage even.  Sweet.
>>
>> Unfortunately this don't help us :(.   We must load balance this
>> sucker.
> If
>> we will not have loadbalancing, we shall achieve the same with
>> running sendmail on each mailscanner box, and vuala..... But I really
>> want a Mailscanner Cluster!  I want redundancy!
>>
>>
>> Vasiliy Boulytchev
>> Colorado Information Technologies, Inc.
>> http://www.coinfotech.com
>>
>> -----Original Message-----
>> From: MailScanner mailing list [mailto:MAILSCANNER at JISCMAIL.AC.UK] On
> Behalf
>> Of Denis Beauchemin
>> Sent: Thursday, September 30, 2004 2:56 PM
>> To: MAILSCANNER at JISCMAIL.AC.UK
>> Subject: Re: Load Balancing + File Locking
>>
>>
>> Boulytchev, Vasiliy wrote:
>>
>>
>>> Ladies and Gents,
>>>       The loads on the mail server require us to start moving
>>> Mailscanner with Spam and Virus filtering to other machines.
>>> Questions:
>>>       1.)     How does MS do the locking on files in the submitted
>>> directory (after cgp2ms)?  If NFS3 is used to mount that shared
>>> folder on say 3 servers designated to only scanning mail, how do we
>>> avoid
> run-ons?
>>>       2.)     Idea: if the local procs keep track of locking, then
>>> we can
>>> toss all that into mysql for example? :)))  Just thinking of ways to
>>> add scalability.
>>>
>>>
>>>
>>
>>
>> Vasiliy,
>>
>> We have 2 MS servers for incoming mail and they don't share any spool
>> directory.  They are separate servers and the load balancing is done
> through
>> the MX records.  Our 2 servers were not identical (one was about
>> twice as fast as the other) and we had the following:
>> # host -t mx usherbrooke.ca
>> usherbrooke.ca mail is handled by 10 smtpe1.usherbrooke.ca.
>> usherbrooke.ca mail is handled by 10 smtpe2.usherbrooke.ca.
>> usherbrooke.ca mail is handled by 10 smtpe3.usherbrooke.ca.
>>
>> Here smtpe1 and smtpe3 are the same machine (the fastest one) with
> different
>> IP.  It received close to 2/3 of all mail.
>>
>> I just replaced the slowest one (smtpe2) by a machine identical to
>> the fastest one (smtpe1) so I asked our DNS guys to drop the smtpe3 name.
>>
>> Denis
>>
>
> **********************************************************************
>
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please notify
> the system manager.
>
> This footnote confirms that this email message has been swept for the
> presence of computer viruses and is believed to be clean.
>
> **********************************************************************
>
> ------------------------ MailScanner list ------------------------ To
> unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
> 'leave mailscanner' in the body of the email.
> Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and the
> archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).
>
> ------------------------ MailScanner list ------------------------ To
> unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
> 'leave mailscanner' in the body of the email.
> Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and the
> archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).

------------------------ MailScanner list ------------------------ To
unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and the
archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).

------------------------ MailScanner list ------------------------
To unsubscribe, email jiscmail at jiscmail.ac.uk with the words:
'leave mailscanner' in the body of the email.
Before posting, read the MAQ (http://www.mailscanner.biz/maq/) and
the archives (http://www.jiscmail.ac.uk/lists/mailscanner.html).

    [ Part 2, Application/X-PKCS7-SIGNATURE  5.9KB. ]
    [ Unable to print this part. ]