MailScanner ANNOUNCE: Stable 4.77 released

[Julian Field]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 01/06/2009 16:41, Steve Campbell wrote:
>
>
> Julian Field wrote:
>> Morning all!
>>
>> (If you are not already following me on twitter, then feel free to do 
>> so, I announce all new features and so on there to help you keep up 
>> to date: twitter.com/JulesFM)
>>
>> I have just released a new stable version of MailScanner, 4.77.
>>
>> The most important new features this month include:
>>
>> - You can finally use hostname, domain name, partial domain names 
>> including wildcards and Perl regular expressions to make a ruleset 
>> line apply to the name of the host the message came from, instead of 
>> having to just use the numerical IP address. You just put in ruleset 
>> lines that look like those below:
>>     From: host:mail.mydomain.com yes
>>     From: host:mail*.mydomain.com yes
>>     From: host:mydomain.com yes
>>     From: host:julianfield.* yes
>>     From: host:/(de|dk|es)$/ yes
>>   As you can see from the examples above, you have to put in the 
>> keyword "host:" at the start of the hostname, domain name, or regular 
>> expression. Regular expressions must be surrounded by "/" characters.
>>   Note that using this feature will require one extra DNS hostname 
>> lookup per message (but only if you use this feature), so there is a 
>> small performance hit.
>>   It is documented more fully in the etc/rules/README and 
>> etc/rules/EXAMPLES files.
>>
>> - MailScanner can now *unzip* small zip files and other archives. We 
>> have systems that mail us zipped files automatically, and we wanted 
>> to save the step of unzipping each attachment to get the small log 
>> file inside. This feature is supported by some new configuration 
>> settings:
>>   Unzip Maximum Files Per Archive = 4
>>   Unzip Maximum File Size = 50k
>>   Unzip Filenames = *.txt *.ini *.log *.csv
>>   Unzip MimeType = text/plain
>>
>> - The "Read IP Address From Received Header" setting has been 
>> extended so it can now take a number as well as just "yes" or "no". 
>> This is so you can choose the IP address from the n-th "Received:" 
>> header in the message, which fetchmail users will find useful.
>>
>> Download it as usual from
>>                 www.mailscanner.info
>>
>> Here is the full ChangeLog for this month:
>> * New Features and Improvements *
>> 1 Can now automatically unzip small zip files and other archives. This
>>   is very useful if you have some service automatically mailing you log
>>   files, which zips up the logfiles to save space. It will unpack 
>> them if
>>   there only a few of them, they are fairly small and they match a list
>>   of filename patterns.
>>   Unzip Maximum Files Per Archive = 4
>>   Unzip Maximum File Size = 50k
>>   Unzip Filenames = *.txt *.ini *.log *.csv
>>   Unzip MimeType = text/plain
>> 1 Hourly cron job about messages being processed only sends a message if
>>   'Send Notices = yes' is set in MailScanner.conf.
>> 1 "Read IP Address From Received Header" has been extended, so it will
>>   now take a number instead of yes or no. "yes"=1 and "no"=0. If it 
>> is set
>>   to "yes" or a number, then the SMTP client IP address is taken from 
>> the
>>   "Received:" header. For example, setting it to 2 will cause the IP
>>   address to be taken from the 2nd Received: header.
>>   Users of BarricadeMX might want to set this to 2, to get the real SMTP
>>   client IP address from the 2nd Received: header, and not the 127.0.0.1
>>   address that BarricadeMX put in the headers.
>>   Users of fetchmail might want to set this to 1 or 2 to skip over the
>>   127.0.0.1 address which will be inserted by fetchmail.
>> 5 Set up Antiword to always return UTF-8 characters and use that in the
>>   attachment it creates.
>> 6 Removed co.dk from country.domains.conf as it's not an official 2nd 
>> level
>>   domain.
>> 6-2 Upgraded DBD-SQLite to 1.25 to avoid RedHat 4 build problems.
>> 6-3 Improved detection of some x86_64 systems.
>> 6-4 Corrected DBD-SQLite packaging error.
>> 7 Improved --lint checking of "Processing Attempts Database" and 
>> improved
>>   logging related to that database. Also improved documentation about 
>> the
>>   two SQLite databases in MailScanner.conf.
>> 8 Implemented a new type of line in rulesets. When you specify a "From:"
>>   rule, you can use a syntax like "host:hostname.domain.com" to use the
>>   SMTP client's hostname instead of the numerical IP address. This can
>>   also be partial hostnames or domain names, such as "host:domain.com"
>>   or include wildcards anywhere, such as "host:mail*.dom*ain.com", or
>>   even Perl regular expressions such as "host:/(de|dk)$/". This goes
>>   where the numerical IP address would go in the rule, after the "From:"
>>   and before the value to return.
>>   Note that these are slightly slower than using the IP address as they
>>   involve a DNS lookup (maximum of once per message), but that value
>>   should be in your DNS cache as other things will have already had to
>>   look it up anyway.
>>   They are described in more detail in the etc/rules/README and
>>   etc/rules/EXAMPLES files.
>>
>> * Fixes *
>> 3 Fixed problem where Unzip functions would not be found. Set default 
>> to off.
>> 4 Fixed issue with Postfix not scanning some messages in 4.77.3.
>> 5 Fixed issue with Postfix scanning too many messages in 4.77.4. :-)
>> 6 Fixed issue with extra character on the front of files created by 
>> antiword.
>> 7 Fixed UTF-8 character in Perl source code in Esets output parser.
>> 7 Fixed issue with encapsulating messages containing silent 
>> whole-message
>>   infections.
>>
>> Jules
>>
>
> Confusion here on what this is doing.
>
> Is this to say that by putting "host:" in front of an address, MS will 
> now verify the IP of that host/domain and if the sending IP doesn't 
> match the host/domain IP, the rule(set) is disregarded and normal 
> processing continues? Sort of like SPF?
You can use "host:" for addresses without a PTR record at all, or when 
the DNS lookup fails.
If you want to match spoofed addresses, use "host:_SPOOFED_". You might 
want to do this in your "is.definitely.spam.rules" for example (if you 
have one).

So if you put in a normal "host:domain.com" (or whatever) rule, it won't 
match if the IP doesn't match its forward+reverse records. So if you say 
"host:google.com" you can be pretty sure that the mail really is coming 
from an IP which has a *.google.com hostname and not just some random 
spammer's host which has a PTR record saying it is blahblah.google.com.

This doesn't really relate to SPF at all, as far as I can see.

I hope that makes it a bit clearer.

Feel free to continue quizzing me if it doesn't explain it to you.

It knows the real IP address of the SMTP client (or whatever address you 
choose if you use the "Read IP Address From Received Header" 
configuration option) that initiated the conversation with the 
MailScanner server. It looks up the hostname of that IP address from the 
DNS PTR record. This gives it the hostname from the "reverse" record. It 
then looks up the IP addresses of that hostname. This gives it a list of 
IP addresses from the "forward" records. This list of IP addresses must 
include the IP address it started with.

If that double-check works, then the "host:mail.google.com" lookup will 
match if it was the IP address of "mail.google.com" that you started with.

If that double-check fails, then only "host:_SPOOFED_" will match.

If the first PTR lookup failed or produced no hostname at all, then only 
"host:" will match.

Jules

- -- 
Julian Field MEng CITP CEng
www.MailScanner.info
Buy the MailScanner book at www.MailScanner.info/store
Follow me at twitter.com/JulesFM

MailScanner customisation, or any advanced system administration help?
Contact me at Jules at Jules.FM

PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
PGP public key: http://www.jules.fm/julesfm.asc


-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.10.0 (Build 500)
Comment: Use PGP or Thunderbird Enigmail to verify this message
Charset: ISO-8859-1

wj8DBQFKJAhGEfZZRxQVtlQRAqb+AKC02Zq0uX5dCMNnCelWV/P6EGnOhACg54Sb
y1b9OVSVpDBZMdbb0gPY3aM=
=fDU9
-----END PGP SIGNATURE-----

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.