Mailscanner process mail too slow

Wed Dec 24 23:41:30 GMT 2014

El 24/12/14 a las 06:40, Steve Freegard escibió:
> On 24/12/14 02:48, Carlos R Laguna wrote:
>> i am not sure if the change to clamd work or not.
> The speed difference between the two is huge:
>
> [root at mail1-ec2 ~]# clamdscan -c /etc/clamd.d/scan.conf  /tmp/test.eml
> /tmp/test.eml: OK
>
> ----------- SCAN SUMMARY -----------
> Infected files: 0
> Time: 0.016 sec (0 m 0 s)
>
> .vs.
>
> [root at mail1-ec2 ~]# clamscan /tmp/test.eml
> /tmp/test.eml: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 4687566
> Engine version: 0.98.4
> Scanned directories: 0
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 18.914 sec (0 m 18 s)
>
> So 18.9 seconds .vs. 0.016 seconds.
>
> That would allow MailScanner to scan only 3000 messages per hour (with
> 15 children: (3600/18)*15 = 3000 that 200 messages per child) whereas
> with clamd that would be 3,375,000 messages.   However that doesn't take
> into consideration the overhead of unpacking the messages, SpamAssassin
> scans (usually 3 seconds each minimum) and all the other stuff that
> MailScanner does, but you get the idea; the difference between the two
> is massive.
>
> MailScanner --lint should tell you if clamd is working or not as it
> should report that it found 'clamd' and detect the Eicar test:
>
> [root at mta41 ~]# MailScanner --lint
> Trying to setlogsock(unix)
> ConfigSQL configuration loaded with serial 594, next check in 900 seconds
> Reading ruleset allowexternal for keyword allowexternal
> <snip>
> Checking version numbers...
> Version number in MailScanner.conf (4.77.10) is correct.
>
> Checking for SpamAssassin errors (if you use it)...
> Using SpamAssassin results cache
> Connected to SpamAssassin cache database
> SpamAssassin reported no errors.
> I have found clamd f-prot-6 scanners installed, and will use them all by
> default.
> Connected to Processing Attempts Database
> Created Processing Attempts Database successfully
> There are 0 messages in the Processing Attempts Database
> Using locktype = posix
> MailScanner.conf says "Virus Scanners = auto"
> Found these virus scanners installed: clamd, f-prot-6
> ===========================================================================
> Filename Checks: Blocked Filename Detected (1 eicar.com)
> Other Checks: Found 1 problems
> Virus and Content Scanning: Starting
> Clamd::INFECTED:: Eicar-Test-Signature :: ./1/
> Virus Scanning: Clamd found 1 infections
> Scanning: -
>    at /usr/lib/MailScanner/MailScanner/SweepViruses.pm line 2338
> Virus Scanning: F-Prot6 found 1 infections
> Infected message 1 came from 10.1.1.1
> Infected message 1.message->eicar.com came from
> Virus Scanning: Found 2 viruses
> ===========================================================================
>
> If any of your virus scanners (clamd,f-prot-6)
> are not listed there, you should check that they are installed correctly
> and that MailScanner is finding them correctly via its virus.scanners.conf.
>
> Regards,
> Steve.
Hi, and thank for your reply

However the mail queue persist to over 13k, system load ~0.30 memory 
consumption 60% of 4Gb almost no cpu over 25%. A litle more information, 
this deploy has 5 server two mailscanner using baruwa in cluster, the 
mysql server is a percona mariadb cluster 3 there is almost 2million 
messages in the db and another 1million in archive the sa_bayes database 
is also in this cluster. At this point no sure if there is a botleneck 
impacting mailscanner process. Thanks for all yours answers. and Regards

MailScanner --lint
Trying to setlogsock(unix)

Reading configuration file /opt/MailScanner/etc/MailScanner.conf
Reading configuration file /opt/MailScanner/etc/conf.d/baruwa.conf
Read 876 hostnames from the phishing whitelist
Read 5890 hostnames from the phishing blacklists
Config: calling custom init function BaruwaLowScore
Baruwa: Populating spam score settings
Baruwa: Read 20 spam score settings
Config: calling custom init function BaruwaShouldScan
Baruwa: Starting scanning settings
Baruwa: Read 21 settings
Config: calling custom init function BaruwaBlacklist
Baruwa: Starting blacklists
Baruwa: Read 0 blacklist items
Baruwa: Ip blocks blacklisted:
Config: calling custom init function BaruwaSQL
Baruwa: Starting SQL logger
Config: calling custom init function BaruwaHighScore
Baruwa: Populating high spam score settings
Baruwa: Read 20 high spam score settings
Config: calling custom init function BaruwaWhitelist
Baruwa: Starting whitelists
Baruwa: Read 1 whitelist items
Baruwa: Ip blocks whitelisted:

Checking version numbers...
Version number in MailScanner.conf (4.84.6) is correct.

Your envelope_sender_header in spam.assassin.prefs.conf is correct.
MailScanner setting GID to  (105)
MailScanner setting UID to  (104)

Checking for SpamAssassin errors (if you use it)...
Using SpamAssassin results cache
Connected to SpamAssassin cache database
SpamAssassin reported no errors.
Connected to Processing Attempts Database
Created Processing Attempts Database successfully
There are 326 messages in the Processing Attempts Database
Using locktype = flock
MailScanner.conf says "Virus Scanners = clamd"
Found these virus scanners installed: clamd
===========================================================================
Filename Checks: Windows/DOS Executable (1 eicar.com)
Other Checks: Found 1 problems
Virus and Content Scanning: Starting
Clamd::INFECTED::Eicar-Test-Signature :: ./1/
Clamd::INFECTED:: Eicar-Test-Signature :: ./1/eicar.com
Virus Scanning: Clamd found 2 infections
Infected message 1 came from 10.1.1.1
Virus Scanning: Found 2 viruses
===========================================================================
Virus Scanner test reports:
Clamd said "eicar.com was infected: Eicar-Test-Signature"

If any of your virus scanners (clamd)
are not listed there, you should check that they are installed correctly
and that MailScanner is finding them correctly via its virus.scanners.conf.
Config: calling custom end function BaruwaLowScore
Baruwa: Shutting down spam score settings
Config: calling custom end function BaruwaShouldScan
Baruwa: Shutting down scanning settings
Config: calling custom end function BaruwaBlacklist
Baruwa: Shutting down blacklists
Config: calling custom end function BaruwaSQL
Baruwa: Shutting down SQL logger
Config: calling custom end function BaruwaHighScore
Baruwa: Shutting down high spam score settings
Config: calling custom end function BaruwaWhitelist
Baruwa: Shutting down whitelists

MailScanner.conf

http://paste.desdelinux.net/5098
 ________________________________________________________________
 XII Edicion del Evento Nacional de Informatica para Jovenes. INFOCLUB.
 Abril. 2015. Ver www.jovenclub.cu
 ________________________________________________________________

-- 
Este mensaje ha sido analizado por MailScanner
en busca de virus y otros contenidos peligrosos,
y se considera que está limpio.