Performance

[Glenn Steen]

On 31/01/07, John Schmerold <john at katy.com> wrote:
> We're seeing significant backlogs, mail is taking 2-6 hours to get thru
> the Postfix/Mailscanner gauntlet we've setup. What's everyone else
> seeing in terms of mail processing time?
>
> I've looked at the home page & WIKI, so, I'm guessing I am missing
> something or there are new techniques not yet published on the
> mailscanner.info
>
> Some of my statistics are as follows:
> Server config: 2.8GHz P4, 2GB DDR2, Maxtor SATA HDD
> Mail volume: approx 7,500 messages per day
> Misc: We have set the noatime flag on spool and log partitions & use a
> local DNS caching nameserver.

This should be able to cope well...
(snip)

> PostFix Configuration:
> [root at mx1 ~]# postconf -n
> canonical_maps = hash:/etc/postfix/canonical
> config_directory = /etc/postfix
> disable_vrfy_command = yes
> hash_queue_names = ""
> header_checks = regexp:/etc/postfix/header_checks
> masquerade_exceptions = root
> message_size_limit = 51200000
> mydomain = schmerold.com
> myhostname = mx1.schmerold.com
> mynetworks = 127.0.0.0/8 65.16.251.208/29
> relay_domains = katy.com katy.net katycomputer.com  schmerold.com
Why is there no "companion" relay_recipient_maps? You should reject
unknown recipients.

> smtpd_data_restrictions = reject_unauth_pipelining, permit
> smtpd_helo_required = yes
Here you should perhaps have a
smtpd_helo_restrictions = permit_mynetworks, check_helo_access
hash:/etc/postfix/deny_domain_spoof
Where the deny_domain_spoof is simply an access file detailing the
domains and IP addresses you relay for like "katy.com REJECT". Will be
perfectly safe to use.

> smtpd_recipient_restrictions = reject_invalid_hostname
> reject_non_fqdn_hostname reject_non_fqdn_sender
> reject_non_fqdn_recipient  reject_unknown_sender_domain
> permit_mynetworks reject_unauth_destination check_sender_access
> hash:/etc/postfix/whitelist reject_rbl_client cbl.abuseat.org
> reject_rbl_client zen.spamhaus.org permit
> smtpd_sender_restrictions = hash:/etc/postfix/access
> transport_maps = hash:/etc/postfix/transport
> virtual_alias_domains = hash:/etc/postfix/virtual
> virtual_alias_maps = hash:/etc/postfix/virtual
> [root at mx1 ~]#
>
>
> MS Log:
> [root at mx1 ~]# cat /var/log/messages | grep "Jan 30 23:40"
> Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: 4F51A4B4468.A8F46 to
> 389AB894965
> Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: A8330894942.93836 to
> A6D8289500D
> Jan 30 23:40:03 mx1 MailScanner[24752]: Requeue: 368088943F4.C0B33 to
> 20327894942
> Jan 30 23:40:03 mx1 MailScanner[24752]: Uninfected: Delivered 7 messages
> Jan 30 23:40:03 mx1 MailScanner[24752]: Batch completed at 128844 bytes
> per second (8272398 / 64)
> Jan 30 23:40:03 mx1 MailScanner[24752]: Batch (10 messages) processed in
> 64.20 seconds
> Jan 30 23:40:03 mx1 MailScanner[24752]: New Batch: Found 7981 messages
> waiting
> Jan 30 23:40:03 mx1 MailScanner[24752]: New Batch: Scanning 10 messages,
> 169939 bytes
> Jan 30 23:40:03 mx1 MailScanner[24752]: Expired 11 records from the
> SpamAssassin cache
> Jan 30 23:40:04 mx1 named[2116]: lame server resolving
> 'mail.voltech-auto.com' (in 'voltech-auto.com'?): 216.53.199.57#53
> Jan 30 23:40:08 mx1 named[2116]: lame server resolving
> '21.36.70.194.in-addr.arpa' (in '36.70.194.in-addr.arpa'?): 194.70.36.12#53
> Jan 30 23:40:42 mx1 MailScanner[24762]: Spam Checks: Found 5 spam messages
> Jan 30 23:40:42 mx1 MailScanner[24762]: Spam Checks completed at 1227
> bytes per second
> Jan 30 23:40:42 mx1 MailScanner[24762]: Virus and Content Scanning: Starting
> Jan 30 23:40:43 mx1 MailScanner[24762]: Virus Scanning completed at
> 156861 bytes per second
> Jan 30 23:40:43 mx1 MailScanner[24762]: Found phishing fraud from
> www.google.com claiming to be www.chase.com in 6BE8F895371.5D53A
> Jan 30 23:40:43 mx1 MailScanner[24762]: Content Checks: Detected and
> have disarmed web bug tags in HTML message in 6BE8F895371.5D53A from
> www-data at balancetechnology.com
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 3B29B894E55.CEBEA to
> 6535E894D8C
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 6BE8F895371.5D53A to
> DB04E894E55
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 73748895A57.5ABB7 to
> 0597D895371
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 937E689448D.77EDA to
> 0CB4B8953AD
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: 754F789466A.8DA78 to
> AC1D989448D
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: D5177894E67.3DEEA to
> A879089466A
> Jan 30 23:40:43 mx1 MailScanner[24762]: Requeue: A3E798940E3.B4BEB to
> 80A7B894E67
> Jan 30 23:40:43 mx1 MailScanner[24762]: Uninfected: Delivered 7 messages
> Jan 30 23:40:43 mx1 MailScanner[24762]: Virus Processing completed at
> 650569 bytes per second
> Jan 30 23:40:43 mx1 MailScanner[24762]: Batch completed at 1215 bytes
> per second (86123 / 70)
> Jan 30 23:40:43 mx1 MailScanner[24762]: Batch (10 messages) processed in
> 70.85 seconds
> Jan 30 23:40:43 mx1 MailScanner[24762]: New Batch: Found 7993 messages
> waiting
> Jan 30 23:40:43 mx1 MailScanner[24762]: New Batch: Scanning 10 messages,
> 160591 bytes
> [root at mx1 ~]#

In this snippet of log we see a lot of requeueing, but no actual
deliveries. Are we to assume that they happen as expected? Most
messages seem to be on HOLD, so ... that is probably nothing.

What does pflogsumm (or similar tool) have to say about the last day
or so? Did all these messages just "plonk" in at approximately the
same time?
Approximately 10 messages/minute would make for somewhere around 14K
messages/day, which isn't that good, but not horrendous either... and
it should be able to keep up, unless you have extremely bursty
traffic.
Are all these adressed to the domains in question?
If you run a message through SA, to get the network tests, do you see
any ... noticeable ... lag anywhere? Any of the digest checks perhaps?

-- 
-- Glenn
email: glenn < dot > steen < at > gmail < dot > com
work: glenn < dot > steen < at > ap1 < dot > se