Postfix vs MailScanner : Slow Incoming Queue

R Wahyudi rwahyudi at gmail.com
Thu Nov 8 22:56:18 GMT 2007 

Thanks for the reply  guys  .. I have scheduled postfix / mailscanner 
update  for today ..
Should be able to get the graph by tomorrow.

If it was the receiving ends, the queue will build up in the active / 
deferred queue and not in the incoming queue.
We have a couple of server on the receiving end running qmail and has no 
filter whatsoever.
They are load balanced using alteon and mailbox storage is mounted from 
HP storagework 8100 SAN via 2 Gb Fiber.
The receiving end capable of accepting a total of ~10,000 mails per 
minute. ( thats the highest i've seen) .

--- More details on the Antispam servers ---

We have 8 Antispam server, and for each server we receive around 400 
email per minutes and rejecting 300 at SMTP level.

RRD Graph :
SMTP connect vs Reject : 
http://img139.imageshack.us/img139/5412/vscan4smtpwu4.png
Rejected message at SMTP level : 
http://img150.imageshack.us/img150/309/vscan4blockedru9.png
Accepted Message : 
http://img145.imageshack.us/img145/7687/vscan4acceptedji0.png
Postfix Queue Level : 
http://img134.imageshack.us/img134/5902/vscan4queuepb1.png


2 filter on header check on postfix :
/^content-(type|disposition):.*name*=.*\.(pif|scr|vbs|com|cmd)"?[[:space:]]*(;|$)/ 
REJECT We cannot accept file with .$2 attachments
/^Received:/ HOLD

Queue Manger process ID is stable and has been the same for longer than 
24 hours.
-- Nov  5 12:21:58 vscan4 postfix/qmgr[13055]: 7A3D760869A: removed
-- Nov  6 10:37:03 vscan4 postfix/qmgr[13055]: A750B608734: removed

======== postconf -n  ===========
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
anvil_rate_time_unit = 15m
bounce_size_limit = 1024
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
debug_peer_level = 1
disable_vrfy_command = no
header_checks = regexp:/etc/postfix/maps /header_checks
html_directory = /usr/share/doc/postfix- 2.3.2-documentation/html
in_flow_delay = 0
inet_interfaces = all
local_recipient_maps =
local_transport = error:local mail delivery is disabled
mail_owner = postfix
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
message_size_limit = 41457280
mydestination =
mydomain =
myhostname = vscan4
mynetworks = xxxxxxxxxxxxxxxxxx
myorigin = $mydomain
newaliases_path = /usr/bin/newaliases.postfix
queue_directory = /var/spool/postfix
readme_directory = /usr/share/doc/postfix-2.3.2-documentation/readme
receive_override_options = no_address_mappings
relay_domains = mysql:/etc/postfix/relay_domains.cf
relay_recipient_maps = proxy:mysql:/etc/postfix/recipients- verification.cf
sample_directory = /usr/share/doc/postfix- 2.2.10/samples
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtpd_banner = $myhostname [vscan v0.1]
smtpd_client_connection_count_limit = 1
smtpd_client_connection_rate_limit = 15
smtpd_client_event_limit_exceptions = someip
smtpd_client_recipient_rate _limit = 50
smtpd_error_sleep_time = 12
smtpd_hard_error_limit = 6
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks,        
reject_invalid_hostname,        reject_non_fqdn_hostname,
smtpd_recipient_limit = 50
smtpd_recipient_overshoot_limit = 1
smtpd_recipient_restrictions = permit_mynetworks,      
reject_unauth_pipelining,      
reject_unauth_destination,      
reject_multi_recipient_bounce,       
reject_unlisted_recipient,  
check_client_access cidr:/etc/postfix/maps/exceptions_clients,
reject_rbl_client hotkey.rbl,
reject_rbl_client zen.spamhaus.org, 
check_policy_service inet: 127.0.0.1:10031
smtpd_sender_restrictions = permit_mynetworks,  check_client_access 
hash:/etc/postfix/maps/sender _blacklist,    check_sender_access 
hash:/etc/postfix/maps/from_blacklist,        reject_non_fqdn_sender,    
reject_unknown_sender_domain
smtpd_soft_error_limit = 1
smtpd_timeout = 20s
soft_bounce = no
transport_maps = hash:/etc/postfix/maps/transport
unknown_local_recipient_reject _code = 550


QSHAPE :

incoming :
                                        T   5 10 20 40 80 160 320 640 
1280 1280+
                                TOTAL 150 150  0  0  0  0   0   0   0    
0     0

Active :
                                         T  5 10 20 40 80 160 320 640 
1280 1280+
                                  TOTAL  0  0  0



EXAMPLE LOG FILE :

Nov  8 09:36:59 vscan4 postfix/smtpd[24127]: C953660802E: 
client=mail.creativefactory.com.au[203.89.197.34]
Nov  8 09:37:00 vscan4 postfix/cleanup[25817]: C953660802E: hold: header 
Received: from mail.creativefactory.com.au (mail.creativefactory.com.au 
[203.89.197.34])??by vscan4.hotkey.net.au (Postfix) with ESMTP id 
C953660802E??for <xxxxx at xxxxxxx.com>; Thu,  8 Nov 2007 0 from 
mail.creativefactory.com.au[203.89.197.34]; 
from=<news at discountnewcars.com.au> to=<xxx at xxxxxx.com> proto=ESMTP 
helo=<mail.xxxxxxxx.com.au>
Nov  8 09:37:00 vscan4 postfix/cleanup[25817]: C953660802E: 
message-id=<13290289.1194474906537.JavaMail.SYSTEM at dnc01>
Nov  8 09:37:03 vscan4 MailScanner[14160]: Message C953660802E.EFA0E 
from 203.89.197.34 (news at discountnewcars.com.au) to xxxxx.com is too big 
for spamchecks (118081 > 80000 bytes)
Nov  8 09:37:07 vscan4 MailScanner[14160]: Requeue: C953660802E.EFA0E to 
4141C608637
Nov  8 09:37:07 vscan4 MailScanner[14160]: Logging message 
C953660802E.EFA0E to SQL
Nov  8 09:37:07 vscan4 MailScanner[14438]: C953660802E.EFA0E: Logged to 
MailWatch SQL
[root at vscan4 src]# grep 4141C608637 /var/log/maillog
Nov  8 09:37:07 vscan4 MailScanner[14160]: Requeue: C953660802E.EFA0E to 
4141C608637
Nov  8 09:44:58 vscan4 postfix/qmgr[13055]: 4141C608637: 
from=<news at discountnewcars.com.au>, size=117724, nrcpt=1 (queue active)
Nov  8 09:44:58 vscan4 postfix/smtp[29892]: 4141C608637: 
to=<xxxx at xxxxxxx.com>, relay=mailroute.xxxxx.net.au[xxx.xx.7.8]:25, 
delay=479, delays=298/0.01/0.01/0.11, dsn=2.0.0, status=sent (250 ok 
1194475318 qp 7468)
Nov  8 09:51:58 vscan4 postfix/qmgr[13055]: 4141C608637: removed

As you can see .. there are long delay after mailscanner requeue the mail.
This is just one example and i've seen queue sitting longer that 10 mins ..


Lets see if postfix+mailscanner update fix this issue.

Regards,
Rianto Wahyudi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mailscanner.info/pipermail/mailscanner/attachments/20071109/b89df9ef/attachment.html

More information about the MailScanner mailing list