IPBlock question - Part 2
Matthew Baker
m at WHERES.CO.UK
Thu Jun 10 14:57:44 IST 2004
Hi,
Thanks for that. I've used a simple script that prints out the
Berkeley db file and can I see it is removing them ok. Doh!
#!/usr/bin/perl -w
use warnings ;
use strict ;
use DB_File ;
use vars qw( %h $k $v $DB ) ;
$DB=$ARGV[0];
tie %h, "DB_File", "$DB", O_RDWR, 0644, $DB_HASH
or die "Cannot open file $DB: $!\n";
# print the contents of the file
if (%h) {
print $DB . " {\n";
while (($k, $v) = each %h)
{ print "\t$k -> $v,\n" }
print "\t}\n";
}
#EOF#
usage: script_name db_filename.db
Second question is:
I have a fair number of users who send through us using SMTP AUTH. Is
there any stage of MailScanners processing that can tell if a message
has been delivered locally using authentication? It would be nice if the
IPBlock could do a check that says if the sender authenticated then use
a higher blocking threshold. I was going to write an external script
that monitored the mail log a watched for authenticated senders and
added them to the IPBlock.conf. But it may be better to do it within MS
and the IPBlock.conf could get quite messy.
Many thanks,
Matt
Jeff A. Earickson wrote:
> Hi,
> Try the attached perl script for analysis of the contents of your
> IPBlock list.
>
> Jeff Earickson
> Colby College
>
> On Fri, 4 Jun 2004, Julian Field wrote:
>
>
>>Date: Fri, 4 Jun 2004 16:06:38 +0100
>>From: Julian Field <mailscanner at ECS.SOTON.AC.UK>
>>Reply-To: MailScanner mailing list <MAILSCANNER at JISCMAIL.AC.UK>
>>To: MAILSCANNER at JISCMAIL.AC.UK
>>Subject: Re: IPBlock question
>>
>>Strings is not a reliable way to read the contents of a DB file.
>>
>>At 15:09 04/06/2004, you wrote:
>>
>>>Hi All,
>>>
>>> I have set up the module so entries are being added to the access.db.
>>>This works fine. However the Cronjob perl script copied off the end of
>>>CustomConfig.pm which runs hourly does not seem to remove old entries
>>>properly.
>>>
>>>What happens is that if I run strings /etc/mail/access.db I get the
>>>entries for hostnames and IPs added. Then if I run the IPBlock cleaner
>>>and run the strings command again it will sometimes not remove the entry
>>
>>>from the access.db and sometimes remove part of the string. E.g. will
>>
>>>remove the last segment up to the last '.' of an IP or hostname. It does
>>>however seem to remove them from the IPBlock.db file and log that lines
>>>were removed from the access file.
>>>
>>>Obvious things I have tried are:
>>> 1. Making sure the $Refusal line matches in the CustomConfig.pm
>>>and the
>>>cron script.
>>> 2. I have reduced the $OneHour to 1 so it should in theory remove
>>>entries one second old (just for testing but I have waited an hour and
>>>tested too).
>>> 3. I have rewritten the $Refusal so there are no special chars (:// in
>>>a URL was taken out).
>>> 4. Ran a separate file to the main access.db for testing.
>>>
>>>I'm running:
>>> Fedora Core 1
>>> MailScanner 4.31.6-1 rpm
>>> sendmail-8.12.11 (built from src not rpm)
>>> perl 5.8.3-16 rpm
>>>
>>>Incidentally it was doing this in 4.29-7 before I upgraded earlier today.
>>>
>>>Thanks in advance.
>>>
>>>Matt
>>>--
>>>
>>>[root at mail-gw mail]# strings IPBaccess.db
>>>"550 Site blocked by MailScanner due to excessive email see
>>>www.sovision.com"
>>>194.105.69.87
>>>"550 Site blocked by MailScanner due to excessive email see
>>>www.sovision.com"
>>>barney.gwork.org
>>>[root at mail-gw mail]# /usr/local/sbin/IPB
>>>IPBedit.pl IPBlock-clean.pl
>>>[root at mail-gw mail]# /usr/local/sbin/IPBlock-clean.pl
>>>[root at mail-gw mail]# Jun 4 14:41:29 mail-gw IPBlock[2551]: Deleted 2
>>>entries from sendmail access database
>>>
>>>[root at mail-gw mail]# strings IPBaccess.db
>>>"550 Site blocked by MailScanner due to excessive email see
>>>www.sovision.com"
>>>194.105.69.87
>>>"550 Site blocked by MailScanner due to excessive email see
>>>www.sovision.com"
>>>barney.gwork.org
>>>
>>>-------------------------- MailScanner list ----------------------
>>>To leave, send leave mailscanner to jiscmail at jiscmail.ac.uk
>>>Before posting, please see the Most Asked Questions at
>>>http://www.mailscanner.biz/maq/ and the archives at
>>>http://www.jiscmail.ac.uk/lists/mailscanner.html
>>
>>--
>>Julian Field
>>www.MailScanner.info
>>MailScanner thanks transtec Computers for their support
>>
>>PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654
>>
>>-------------------------- MailScanner list ----------------------
>>To leave, send leave mailscanner to jiscmail at jiscmail.ac.uk
>>Before posting, please see the Most Asked Questions at
>>http://www.mailscanner.biz/maq/ and the archives at
>>http://www.jiscmail.ac.uk/lists/mailscanner.html
>>
>
>
> -------------------------- MailScanner list ----------------------
> To leave, send leave mailscanner to jiscmail at jiscmail.ac.uk
> Before posting, please see the Most Asked Questions at
> http://www.mailscanner.biz/maq/ and the archives at
> http://www.jiscmail.ac.uk/lists/mailscanner.html
>
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/perl -I/usr/lib/MailScanner
>
> #
> # MailScanner - SMTP E-Mail Virus Scanner
> # Copyright (C) 2002 Julian Field
> #
> # This program is free software; you can redistribute it and/or modify
> # it under the terms of the GNU General Public License as published by
> # the Free Software Foundation; either version 2 of the License, or
> # (at your option) any later version.
> #
> # This program is distributed in the hope that it will be useful,
> # but WITHOUT ANY WARRANTY; without even the implied warranty of
> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> # GNU General Public License for more details.
> #
> # You should have received a copy of the GNU General Public License
> # along with this program; if not, write to the Free Software
> # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> #
> # The author, Julian Field, can be contacted by email at
> # Jules at JulianField.net
> # or by paper mail at
> # Julian Field
> # Dept of Electronics & Computer Science
> # University of Southampton
> # Southampton
> # SO17 1BJ
> # United Kingdom
> #
>
> use Net::CIDR;
> use FileHandle;
> use Fcntl qw(:DEFAULT :flock);
> use Getopt::Std;
> BEGIN { @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File) }
> use AnyDBM_File;
>
> #use strict 'vars';
> #use strict 'refs';
> #no strict 'subs'; # Allow bare words for parameter %'s
>
> #---subroutine prototypes
> sub sort_criterion();
> sub parseargs();
>
> my $OneHour = 3600; # seconds
> my $WhitelistFile= '/etc/MailScanner/IPBlock.conf';
> my $LockFile = '/var/spool/MailScanner/IPBlock.lock';
> my $BlockDB = '/var/spool/MailScanner/IPBlock.db';
> my $AccessDB = '/etc/mail/db/access.db';
> my $Refusal = '"550 Site blocked by MailScanner due to excessive email"';
>
> #---parse command line to get number of lines to print
> #---default is all lines
> parseargs();
>
> #---read in the IPBlock.conf file
> my $LimitsH = new FileHandle;
> $LimitsH->open($WhitelistFile) or die;
> $counter = 0;
> while(<$LimitsH>) {
> chomp;
> s/#.*$//;
> s/^\s*//g;
> s/\s*$//g;
> next if /^$/;
> ($cidr, $limit) = split;
> $cidr =~ s/\s//g;
> $limit = 0 unless defined $limit;
>
> my @cidrlist = undef;
> if ($cidr =~ /-/) {
> # It looks like 152.78.67.0-152.78.69.255
> @cidrlist = Net::CIDR::range2cidr($cidr);
> } elsif ($cidr =~ /\//) {
> # It looks like 152.78.0.0/16 or 152.78/16 or 152.78/255.255.0.0
> my($network, $bits, $count);
> ($network, $bits) = split(/\//, $cidr);
> $network =~ s/\.$//; # Delete any trailing dot
> $count = split(/\./, $network);
> $network .= '.0' x (4-$count); # Fill out the CIDR for Net::CIDR
> # 152.78 now looks like 152.78.0.0
> if ($bits =~ /\./) {
> # It's like 152.78.0.0/255.255.0.0
> push @cidrlist, Net::CIDR::addrandmask2cidr($network, $bits);
> } else {
> # It's like 152.78.0.0/16
> push @cidrlist, "$network/$bits";
> }
> } elsif ($cidr =~ /default/i) {
> # It is the default value used when nothing else matches
> $DefaultMaxMessagesPerHour = $limit;
> } else {
> # Must just be an IP address or look like 152.78 or 152.78.
> $cidr =~ s/\.$//; # Delete any trailing dot
> my $count = split(/\./, $cidr);
> $cidr .= '.0' x (4-$count);
> push @cidrlist, "$cidr/" . ($count*8);
> }
>
> # Build the map from CIDR to message limit
> foreach (@cidrlist) {
> next unless $_;
> #print STDERR "IPBlock: adding $_\n";
> $CIDRtoLimit{$_} = $limit;
> push @CIDRlist, $_;
> }
> $counter++;
> }
> close($LimitsH);
>
> #
> # Lock out everything else for the whole of this script
> #
> my $LockFileH = new FileHandle;
> openlock($LockFileH, ">$LockFile");
>
> #
> # Find all the entries to be deleted from the BlockDB file.
> #
>
> #Bind to BlockDB
> my(%BlockDB, %AccessDB);
> tie %BlockDB, "AnyDBM_File", $BlockDB, O_RDWR, 0644
> or BailOut("Failed to open $BlockDB, it may not exist yet, $!");
>
> tie %AccessDB, "AnyDBM_File", $AccessDB, O_RDWR, 0644
> or BailOut("Failed to open $AccessDB, have you got the path wrong? $!");
>
> # Read and print IPBlock DB
> my $now = time;
> my(@ips, $ip, $value, $hostname, $count, $time, $donealready, $flag);
> my $countrec = 0;
> my $countdel = 0;
> my $countblk = 0;
> print "MailScanner IP Blocking Summary\n";
> printf("S %15s: %s %s\n","IP Number","mesgs/limit","Hostname");
> print "--------------------------------------------------------------\n";
> @ips = sort { sort_criterion() } (keys %BlockDB);
> foreach $ip (@ips)
> {
> ($hostname, $count, $time, $donealready) = split(/,/, $BlockDB{$ip});
> # Is it more than an hour old, or has time_t wrapped (happens in year 2036)
> #print STDERR "Examining record for $ip, $count, $time\n";
> $countrec++;
> $flag = " ";
> if ($time>$now || $now>=$time+$OneHour) {
> $flag = "-";
> $countdel++;
> }
> if ($AccessDB{$ip} eq $Refusal) {
> $flag = "*";
> $countblk++;
> }
>
> #---look for CIDR and print if less than print limit
> if($countrec <= $opt_n)
> {
> #---find the limit per CIDR rule for this IP
> my($cidrkey, $foundcidr, $foundit, $limit);
> $foundit = 0;
> foreach $cidrkey (@CIDRlist) {
> #print STDERR "Looking for $ip in $cidrkey\n";
> if (Net::CIDR::cidrlookup($ip, $cidrkey)) {
> #print STDERR "Found it\n";
> $foundit = 1;
> $foundcidr = $cidrkey;
> last;
> }
> }
> # If we didn't find it, use the default value
> $limit = $foundit ? $CIDRtoLimit{$foundcidr} : $DefaultMaxMessagesPerHour;
> #print STDERR "Limit of $foundcidr is $limit\n";
> $fraction = sprintf("%d/%d",$count,$limit);
>
> printf("%1s %15s: %11s %s\n",$flag,$ip,$fraction,$hostname);
> }
> }
> print "--------------------------------------------------------------\n";
> printf("%5d DB records in IPBlock database\n",$countrec);
> printf("%5d DB records scheduled for deletion, next cron job (-)\n",$countdel);
> printf("%5d DB records listed in sendmail access db file (*)\n",$countblk);
>
> # Unlock and close the DB file
> untie %BlockDB;
> untie %AccessDB;
>
> unlockclose($LockFileH);
>
> exit 0;
>
> sub openlock {
> my($fh, $fn) = @_;
>
> if (open($fh, $fn)) {
> flock($fh, LOCK_EX) or die;
> } else {
> die "Died opening $fn, $!";
> }
> }
>
> sub unlockclose {
> my($fh) = @_;
>
> flock($fh, LOCK_UN);
> close ($fh);
> }
>
> sub BailOut {
> warn "@_, $!";
> exit 1;
> }
>
> sub parseargs()
> #---parse the command line arguments
> {
> $opt_n = 999999;
>
> getopts("n:");
>
> if($#ARGV ne -1)
> {
> print "Usage is: $0 [-n number]\n";
> print "\t-n is the number of lines to print out\n";
> exit 1;
> }
> }
>
> sub sort_criterion()
> #---sorts according to message count, biggest to smallest
> {
> my($counta, $countb);
> (undef, $counta, undef, undef) = split /,/, $BlockDB{$a};
> (undef, $countb, undef, undef) = split /,/, $BlockDB{$b};
> return($countb <=> $counta);
> }
-------------------------- MailScanner list ----------------------
To leave, send leave mailscanner to jiscmail at jiscmail.ac.uk
Before posting, please see the Most Asked Questions at
http://www.mailscanner.biz/maq/ and the archives at
http://www.jiscmail.ac.uk/lists/mailscanner.html
More information about the MailScanner
mailing list