Performance

[Brent Addis]

Are you sure you're only processing 7,500 (ish) messages per day? Your hardware configuration is exactly the same as my dev box, and can plow 25k messages through without too much issue. 70 seconds for a batch of 10 messages is fairly average, its not overly high. At that rate you should be able to do way more than 7,500 messages. However that huge backlog is fairly odd. Check your not using any large spamassasin rulesets (blacklist.cf comes to mind, its huge and slows things down a lot). Did all that mail come in at once or is it a trickle throughout the day? Doing things such as mounting the mailscanner spooldir in ram rather than on disk can speed things up a bit too. Is this happening on one machine, or multiple machines?


-----Original Message-----
From: mailscanner-bounces at lists.mailscanner.info on behalf of John Schmerold
Sent: Wed 1/31/2007 7:12 PM
To: mailscanner at lists.mailscanner.info
Subject: Performance
 
We're seeing significant backlogs, mail is taking 2-6 hours to get thru 
the Postfix/Mailscanner gauntlet we've setup. What's everyone else 
seeing in terms of mail processing time?

I've looked at the home page & WIKI, so, I'm guessing I am missing 
something or there are new techniques not yet published on the 
mailscanner.info

Some of my statistics are as follows:
Server config: 2.8GHz P4, 2GB DDR2, Maxtor SATA HDD
Mail volume: approx 7,500 messages per day
Misc: We have set the noatime flag on spool and log partitions & use a 
local DNS caching nameserver.

MS Configuration:
[root at mx1 ~]# cat /etc/MailScanner/MailScanner.conf
# See http://www.mailscanner.info/MailScanner.conf.index.html for all 
options & defaults
%etc-dir% = /etc/MailScanner
%mcp-dir% = /etc/MailScanner/mcp
%org-long-name% = Schmerold
%org-name% = Schmerold
%report-dir% = /etc/MailScanner/reports/en
%rules-dir% = /etc/MailScanner/rules
%web-site% = www.schmerold.com

Always Include SpamAssassin Report = yes
Archive Mail = /etc/MailScanner/rules/archive.rules
High Scoring Spam Actions = store
High SpamAssassin Score = 7
Incoming Queue Dir = /var/spool/postfix/hold
Incoming Work Dir = /var/spool/MailScanner/incoming
Language Strings = /etc/MailScanner/reports/en/languages.conf
MTA = postfix
Outgoing Queue Dir = /var/spool/postfix/incoming
Required SpamAssassin Score = 4
Restart Every = 7200
Run As Group = postfix
Run As User = postfix
Sign Clean Messages = no
SpamAssassin Site Rules Dir = /etc/mail/spamassassin

Log Speed = yes
Max Children = 2
Max Unscanned Messages Per Scan = 10
Max Unsafe Messages Per Scan = 10
Spam List =
Virus Scanners = f-prot
[root at mx1 ~]#

PostFix Configuration:
[root at mx1 ~]# postconf -n
canonical_maps = hash:/etc/postfix/canonical
config_directory = /etc/postfix
disable_vrfy_command = yes
hash_queue_names = ""
header_checks = regexp:/etc/postfix/header_checks
masquerade_exceptions = root
message_size_limit = 51200000
mydomain = schmerold.com
myhostname = mx1.schmerold.com
mynetworks = 127.0.0.0/8 65.16.251.208/29
relay_domains = katy.com katy.net katycomputer.com  schmerold.com
smtpd_data_restrictions = reject_unauth_pipelining, permit
smtpd_helo_required = yes
smtpd_recipient_restrictions = reject_invalid_hostname 
reject_non_fqdn_hostname reject_non_fqdn_sender  
reject_non_fqdn_recipient  reject_unknown_sender_domain 
permit_mynetworks reject_unauth_destination check_sender_access 
hash:/etc/postfix/whitelist reject_rbl_client cbl.abuseat.org 
reject_rbl_client zen.spamhaus.org permit
smtpd_sender_restrictions = hash:/etc/postfix/access
transport_maps = hash:/etc/postfix/transport
virtual_alias_domains = hash:/etc/postfix/virtual
virtual_alias_maps = hash:/etc/postfix/virtual
[root at mx1 ~]#


MS Log:
[root at mx1 ~]# cat /var/log/messages | grep "Jan 30 23:40"
Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: 4F51A4B4468.A8F46 to 
389AB894965
Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: A8330894942.93836 to 
A6D8289500D
Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: 368088943F4.C0B33 to 
20327894942
Jan 30 23:40:03 mx1 MailScanner[24752]: Uninfected: Delivered 7 messages
Jan 30 23:40:03 mx1 MailScanner[24752]: Batch completed at 128844 bytes 
per second (8272398 / 64)
Jan 30 23:40:03 mx1 MailScanner[24752]: Batch (10 messages) processed in 
64.20 seconds
Jan 30 23:40:03 mx1 MailScanner[24752]: New Batch: Found 7981 messages 
waiting
Jan 30 23:40:03 mx1 MailScanner[24752]: New Batch: Scanning 10 messages, 
169939 bytes
Jan 30 23:40:03 mx1 MailScanner[24752]: Expired 11 records from the 
SpamAssassin cache
Jan 30 23:40:04 mx1 named[2116]: lame server resolving 
'mail.voltech-auto.com' (in 'voltech-auto.com'?): 216.53.199.57#53
Jan 30 23:40:08 mx1 named[2116]: lame server resolving 
'21.36.70.194.in-addr.arpa' (in '36.70.194.in-addr.arpa'?): 194.70.36.12#53
Jan 30 23:40:42 mx1 MailScanner[24762]: Spam Checks: Found 5 spam messages
Jan 30 23:40:42 mx1 MailScanner[24762]: Spam Checks completed at 1227 
bytes per second
Jan 30 23:40:42 mx1 MailScanner[24762]: Virus and Content Scanning: Starting
Jan 30 23:40:43 mx1 MailScanner[24762]: Virus Scanning completed at 
156861 bytes per second
Jan 30 23:40:43 mx1 MailScanner[24762]: Found phishing fraud from 
www.google.com claiming to be www.chase.com in 6BE8F895371.5D53A
Jan 30 23:40:43 mx1 MailScanner[24762]: Content Checks: Detected and 
have disarmed web bug tags in HTML message in 6BE8F895371.5D53A from 
www-data at balancetechnology.com
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 3B29B894E55.CEBEA to 
6535E894D8C
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 6BE8F895371.5D53A to 
DB04E894E55
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 73748895A57.5ABB7 to 
0597D895371
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 937E689448D.77EDA to 
0CB4B8953AD
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 754F789466A.8DA78 to 
AC1D989448D
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: D5177894E67.3DEEA to 
A879089466A
Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: A3E798940E3.B4BEB to 
80A7B894E67
Jan 30 23:40:43 mx1 MailScanner[24762]: Uninfected: Delivered 7 messages
Jan 30 23:40:43 mx1 MailScanner[24762]: Virus Processing completed at 
650569 bytes per second
Jan 30 23:40:43 mx1 MailScanner[24762]: Batch completed at 1215 bytes 
per second (86123 / 70)
Jan 30 23:40:43 mx1 MailScanner[24762]: Batch (10 messages) processed in 
70.85 seconds
Jan 30 23:40:43 mx1 MailScanner[24762]: New Batch: Found 7993 messages 
waiting
Jan 30 23:40:43 mx1 MailScanner[24762]: New Batch: Scanning 10 messages, 
160591 bytes
[root at mx1 ~]#

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/ms-tnef
Size: 5544 bytes
Desc: not available
Url : http://lists.mailscanner.info/pipermail/mailscanner/attachments/20070131/0abeab5b/attachment.bin