mailscanner (Solaris 2.6) Could not open file

[Chang Kai Cheong]

Hi all,

We have used MailScanner 4.20.3 installed on our Solaris 2.6 system (with
Sophos and Spamassassin 2.55).  This combination has been running fine for
nearly a year but we recently encountered a problem like this:

Feb  9 10:32:51 host MailScanner[21163]: Could not open file
>/var_spool/MailScanner/incoming/21163/i192WVVt005047.header: Resource temporarily
unavailable
Feb  9 10:32:51 host MailScanner[21163]: Cannot create + lock headers
file /var_spool/MailScanner/incoming/21163/i192WVVt005047.header,
....
Feb  9 10:33:58 host MailScanner[20958]: Could not open file
>/var_spool/MailScanner/incoming/20958/i192WVVt005047.header: Resource temporarily
unavailable
Feb  9 10:33:58 host MailScanner[20958]: Cannot create + lock headers
file /var_spool/MailScanner/incoming/20958/i192WVVt005047.header,

and the child died out one by one.  We have searched through the mailing
list and take the recommended actions:
- lower the number of child processes
- lower the max. messages per scan
- add additional resources (CPU and memory), now we should have around 20%
utilization left in terms of CPU
- add ulimit lines in check_mailscanner script:
ulimit -n 2048
ulimit -Hn 2048
ulimit -s 32678
ulimit -Hs 32678
ulimit -v 1048576
ulimit -Hv 1048576
ulimit -d 3932152
ulimit -Hd 3932152

However, we still get into the same problem.  When the above problem is
encountered, the child mailscanner processes die one by one and repeated
with the same error message.  We have to re-create a new mqueue.in
directory and gradually move back the queued files/messages in batches for
delivery.  Moving the queued files to fast would result in the same
problem.  I had performed some sar captured on this morning problem but
could only spot a sudden increase in slock/s:

10:27:17  atch/s  pgin/s ppgin/s  pflt/s  vflt/s slock/s
10:30:18  603.25  664.34 3030.24 19383.92 15327.84    0.00
10:33:18  664.29  782.39 3767.35 18025.67 14332.42    6.97
10:36:18  616.73  810.95 4190.90 10806.34 9065.96   20.91
10:39:18  570.98  643.07 3199.66 3298.72 3412.87  238.33
10:42:18  564.98  718.91 3597.47 4662.66 4904.46   17.60
10:45:18  864.03  779.65 3624.08 6613.88 6722.09    0.00
10:48:18  985.85  839.64 4035.54 11045.76 9830.23    0.00

Average   695.73  748.42 3635.03 10548.22 9085.19   40.54

Does anyone has similar experience to solve the problem?

Thanks in advance.
KC Chang