Load sharing techniques

[Julian Field]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 04/06/2009 15:38, Joey Casas wrote:
> I am new to this list but have been working with MailScanner for a few years now. With the recent (re)uptick in spam volumes, and increased rules to cope with new techniques, we are having general performance "problems" with our current hardware. Mainly, increased queue size and associated delays during peak times.
>
> Currently I have a basic round robin setup between three MailScanners and two external IPs that forward to all servers. I might be getting more "mediocre" hardware soon and I _can_ simply add them to the NAT pool. However, the servers are all of different capabilities and generally I have one or two servers that accumulate larger queues than the others. Average mails per second are very similar (MS1 .69, MS2 .74, MS4 .75).
>
> I also have some pretty fast RAID 5 disk that I can NFS or iSCSI mount - which leads me to my question. Has anyone done a "shared queue" setup where multiple servers look to the same mqueue to scan and what problems could be foreseen? I imagine maybe file locking, header accuracy, etc...
>    
Your biggest problem is file locking. It's got to be perfect for it to 
work, MTAs aren't normally written with shared filestore in mind, and so 
use locking mechanisms that are not supported on shared filesystems such 
as NFS or SMB. I didn't think iSCSI was a filesharing protocol, just a 
way of implementing SCSI over IP, so I don't quite see how that's 
relevant here.

I had 2 slower servers and 2 faster servers. I set up the MX records so 
that the
@ IN MX 10 slower.server
@ IN MX 5 faster.server
In that way, all the "real" mail used the MX priority number and hence 
went to the faster server which had the shorter queues and so on.
The slower server would never get any real mail as the faster one was 
always available, and it didn't matter if the slower one got a bit 
behind on processing its spam at times.
Each MX record pointed to an A record with 2 IP addresses, so that the 
slower.server name was actually 2 physical servers and the faster.server 
name was the other 2 physical servers.

That worked very well for me.

You don't need to use NAT to achieve any of this.

Hope that is of some use to you.

Jules

- -- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store
Follow me at twitter.com/JulesFM

MailScanner customisation, or any advanced system administration help?
Contact me at Jules at Jules.FM

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
PGP public key: http://www.jules.fm/julesfm.asc


-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.10.0 (Build 500)
Comment: Use PGP or Thunderbird Enigmail to verify this message
Charset: KOI8-R

wj8DBQFKJ/buEfZZRxQVtlQRAg93AKDQgxJr7UaQ6guCygrUBBU0LwejuwCg7Q76
uhv5TTpMSAzh7jkqIDcfqyA=
=hrUx
-----END PGP SIGNATURE-----

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.