NO! Dont go! ms2cgp problem
Vasiliy Boulytchev
vasiliy at linuxspecial.com
Wed Jan 17 17:35:16 CET 2007
John,
First of all, THANK YOU SO MUCH for helping us out! I have been
using your cgp2ms and ms2cgp scripts for years flawlessly.
Now that you decide to leave us, I get a problem with ms2cgp on a
new install...
It seems the MS is not giving the correct $job value to the ms2cgp
script.... syslog:
--
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: job =
-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: qf =
/etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: df =
/etc/MailScanner/shared/queues/ms1/out/df-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: msg =
/etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Job
-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out writing to
/etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Open input
/etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
failed, dying
--
Now, if I call ms2cgp from commandline via this script:
#!/usr/bin/perl -w
opendir(DIR, ".");
@files = grep(/qf/,readdir(DIR));
closedir(DIR);
foreach $file (@files) {
substr($file, 0, 1) = "";
substr($file, 0, 1) = "";
system ("/usr/local/etc/ms2cgp $file");
}
The batch processes just fine, syslog:
--
Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: job = 19659
Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: qf =
/etc/MailScanner/shared/queues/ms1/out/qf19659
Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: df =
/etc/MailScanner/shared/queues/ms1/out/df19659
Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: msg =
/etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: Job 19659 writing to
/etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
--
So it seems to me, MS is not passing the correct $job value...
furthermore, why is it making it -OQueueDirectory!?!?!?
What do you guys think?
THANK YOU VERY MUCH!!! My cgp2ms and ms2cgp scripts are attached.
Vasiliy Boulytchev
vasiliy at linuxspecial.com
John Rudd wrote:
>
>
> Just wanted to thank everyone at this list, starting with Julian, but
> also several of the subscribers who given a lot of information and
> input over the years I've been here. At some points, this was an
> incredibly valuable and informative mailing list for me.
>
> And, what has changed is not the list. The list, and the vast
> majority of its subscribers, are still great. What changed were my
> anti-spam/anti-virus needs. I'm moving to solutions that operate
> entirely during the SMTP session. As a result, I decommissioned my
> last mailscanner system 2 weeks ago.
>
> After a short waiting period to be sure I wasn't going to need to roll
> back, or have any other mailscanner related questions, I'm going to be
> unsubscribing from the list later today.
>
> I wish all of you the best, and thank you for a great set of software,
> a great source of information resources, and a great source of
> conversation over the last 4 or 5 years.
-------------- next part --------------
#!/usr/bin/perl
# cgp2ms - part of a MailScanner to CommuniGate Pro gateway
# Copyright (C) 2003 The Regents of the University of California
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
# The Author, John Rudd, can be by email at
# jrudd at ucsc.edu
#
# cgp2ms -f file Actually process the given file
# or
# cgp2ms -c Just check the queue sizes and print result
#
# This program is used by CommuniGate Pro as an "execute" filter. It takes
# in arguments that define where the CommuniGate Pro (CGP) queue file lives
# and uses the queue file to create a sendmail formated queue file (qf) and
# data file (df) pair. These two files are then inserted into the
# MailScanner (MS) incoming queue directory. When MS is done, it will use a
# program which is a companion to this one, ms2cgp, # to give the message
# back to CGP.
#
# Modified 10/2004 by Vasiliy Boulytchev and Randy Lindsey
# Added load balancing and sped up processing for larger volume
# 12/19/2004 Randy Lindsey
# Rather than read queuedirs for qf* files across remote shares,
# this version reads a set of files created on each system containing the queue count
#
# To take a host offline, just comment out its line in hostlist
%hostlist = (
# IA => '/var/CommuniGate/MailScanner/Incoming',
# TX2 => '/usr/local/mailscanner.tx2',
# AZ => '/usr/local/mailscanner.az',
# TMP => '/usr/local/mailscanner.tmp',
# CA => '/usr/local/mailscanner.ak/mailscanner.in',
MS1 => '/var/CommuniGate/MS.Status/queues/ms1/in',
# MS2 => '/usr/local/mailscanner.ak/mailscanner.ms2.in',
# MS3 => '/usr/local/mailscanner.ak/mailscanner.ms3.in',
# MS4 => '/usr/local/mailscanner.ak/mailscanner.ms4.in',
# MS5 => '/usr/local/mailscanner.ak/mailscanner.ms5.in',
# MS6 => '/usr/local/mailscanner.ak/mailscanner.ms6.in',
# MS7 => '/usr/local/mailscanner.ak/mailscanner.ms7.in',
# MS8 => '/usr/local/mailscanner.ak/mailscanner.ms8.in',
# MS9 => '/usr/local/mailscanner.ak/mailscanner.ms9.in',
);
# hostspeed - larger numbers are faster
%hostspeed = (
# IA => 1,
# TX2 => 4,
# AZ => 0.6,
# CA => 9,
# TMP => 100,
MS1 => 2,
MS2 => 2,
MS3 => 2,
MS4 => 15,
MS5 => 15,
MS6 => 30,
MS7 => 30,
MS8 => 30,
MS9 => 30,
);
sub check_one { #$queuedirname
open DH, "</var/CommuniGate/MS.Status/queues/count/$_[0]" or return 999999;
read(DH, $MsgsInQueue, 10);
close DH;
chomp $MsgsInQueue;
return $MsgsInQueue;
}
sub check_queues {
$totfiles = 0;
$totpoints = 0;
foreach my $hostid (keys %hostlist) {
$hostfiles{$hostid} = check_one(lc($hostid));
$totfiles += $hostfiles{$hostid};
# Assign points based on relative system speed
# We scale it up by 737 to reduce rounding errors with small numbers
if ($hostspeed{$hostid}) {
$hostpoints{$hostid} = ($hostfiles{$hostid} * 737) / $hostspeed{$hostid};
}
else {
$hostpoints{$hostid} = $hostfiles{$hostid};
}
}
@hostsort = sort { $hostpoints{$a} <=> $hostpoints{$b} } keys %hostpoints;
printf ("MailScanner to %s - Tot:%2u ",
$hostsort[0], $totfiles); # displays in CommuniGate logs
foreach my $hostid (keys %hostlist) {
printf ("%s:%2u/%2u ", $hostid, $hostfiles{$hostid}, $hostpoints{$hostid});
}
print ("\n");
return $hostlist{$hostsort[0]};
}
sub copy_stdin {
# Extra headers added by prior rules
my $hdrline;
while (defined($hdrline = <STDIN>)) {
print QF "H$hdrline";
print " $hdrline "; # display in logs
}
}
my ($i, $file, $from, $rcpt, @tempv, @rcpt, @argv, $inbody, $inhdr, $inmeta);
$checkonly = 0;
for ($i = 0; $i <= $#ARGV; $i++) { # parse in the arguments
if (defined($ARGV[$i])) {
if ($ARGV[$i] eq "-f") { # the location of the cgp queue file
$i++;
$file = $ARGV[$i];
}
elsif ($ARGV[$i] eq "-c") { # just check queue sizes and print result
$checkonly = 1;
}
else { # left over from debugging, these
push (@argv, $ARGV[$i]); # aren't actually used
}
}
}
if (! $file) {
$checkonly = 1; # default if no parameters passed
}
$qdir = check_queues(); # load balance between servers
if ($checkonly) {
exit(0);
}
open (JOB, "<$file");
open (QF, ">$qdir/msqf$$"); # minimize file moving by writing to final dir but
open (DF, ">$qdir/msdf$$"); # under a different name to prevent Mailscanner taking it
print QF "V4\n"; # I don't think MS actually uses these
print QF "T" . time . "\n"; # qf lines, but it helps the qf file
print QF "K0\n"; # be a little more authentic
print QF "N0\n"; #
print QF "P150900\n"; #
$inmeta = 1; # when reading the job file, are we still in the meta data?
$inhdr = 0; # when reading the job file, are we in the rfc822 headers?
$inbody = 0; # when reading the job file, are we in the rfc822 body/data?
$rec = 0; # have we read the most recent Received header yet?
# the most recent (first listed) Received header will
# tell us the information we'll use in the qf file's $_
# which is "which host relayed this message to us"
while (defined ($line = <JOB>)) {
chomp $line;
if ($line eq "") { # blank lines demark meta, headers, and body sections
if ($inmeta) {
$inmeta = 0; $inhdr = 1;
}
elsif ($inhdr) {
$inhdr = 0; $inbody = 1;
copy_stdin;
if ( !($rec) ) { # no received header = from localhost
$rec = 1;
print QF "\$_localhost [127.0.0.1]\n"; # print relay host
print QF "S$from\n"; # print sender
foreach $to (@rcpt) { # print the recipient list
print QF "RPFD:$to\n";
}
}
}
}
elsif ($inmeta) {
if ($line =~ /^P/) {
@tempv = split(/ /, $line);
$from = $tempv[7];
}
elsif ($line =~ /^R/) {
@tempv = split(/ /, $line);
$rcpt = $tempv[7];
push (@rcpt, $rcpt);
}
}
elsif ($inhdr) {
if ( (!($rec)) && ($line =~ /^Received:/) ) { # get relay host
$rec = 1;
if ($line =~ /^Received: from (.*) \((.* )?(\[.*\]).*/) {
if (defined $2) {
$h = $2;
$h =~ s/\s*$//;
}
else {
$h = $1;
}
$a = $3;
print QF "\$_$h $a\n"; # print relay host
}
else {
$line =~ /^Received: from (\[.*\]).*/;
$a = $1;
print QF "\$_$a\n"; # print relay host
}
print QF "S$from\n"; # print sender
foreach $to (@rcpt) { # print the recipient list
print QF "RPFD:$to\n";
}
print QF "H$line\n"; # then print the Received header
}
elsif ($line !~ /^\s/) { # get a header
print QF "H$line\n";
}
else { # get the rest of a multi-line header
print QF "$line\n";
}
}
if ($inbody) {
print DF "$line\n";
}
}
print QF ".\n"; # bat book 23.9.19, qf file should end in a "^\.$" line.
close (DF);
close (QF);
close(JOB);
# Rename the df file first, as Mailscanner looks for the qf and might interfere
# Note that this used to mv the files from /tmp to qdir, but a fraction of the time
# this failed due to race conditions with Mailscanner
rename ("$qdir/msdf$$", "$qdir/df$$");
rename ("$qdir/msqf$$", "$qdir/qf$$");
exit(0);
-------------- next part --------------
#!/usr/bin/perl
# ms2cgp - part of a MailScanner to CommuniGate Pro gateway
# Copyright (C) 2003 The Regents of the University of California
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#
# The Author, John Rudd, can be by email at
# jrudd at ucsc.edu
#
#
# ms2cgp -qI(id) ...
#
# This program acts as Mailscanner's outgoing "Sendmail2" program, taking
# an argument for each message Mailscanner (MS) has finished processing,
# and using that argument to find the pair of sendmail mqueue files that
# make up the message. This program then re-combines them into an rfc822
# file and submits them back to Communigate Pro (CGP) via its Submitted directory
#
# Modified 11/4/04 by Randy Lindsey to skip "sendmail" program and just
# write directly to Submitted in RFC822 format.
my $ServerPrefix = "MS1"; # make this unique per server
my $QDIR = "/etc/MailScanner/shared/queues/ms1/out"; # where MS sticks outgoing msgs.
my $SubDir = "/etc/MailScanner/mail.cluster/submx00"; # CG's Submitted directory
# my $Archive = "/extra/archivems2cgp/ia";
my ($job, $from, $rcpt, $df, $qf, $msg, $line);
use Sys::Syslog;
openlog('ms2cgp', 'pid', 'user');
foreach $job (@ARGV) {
$job =~ s/^-qI//;
$qf = $QDIR . "/qf" . $job; # the sendmail formatted queue file
$df = $QDIR . "/df" . $job; # the sendmail formatted data file
$msg = $SubDir . "/" . $ServerPrefix . ".ms2cgp.$job.$$.tmp"; # the tempfile we'll give to CGP
$rcpt = "";
# syslog('info', "Job $job copying to archive $Archive");
# system("/bin/cp -f $qf $Archive/qf$job");
# system("/bin/cp -f $df $Archive/df$job");
syslog ('info', "job = $job");
closelog();
syslog ('info', "qf = $qf");
closelog();
syslog ('info', "df = $df");
closelog();
syslog ('info', "msg = $msg");
closelog();
syslog('info', "Job $job writing to $msg");
if (! open (QF, "<$qf")) {
syslog('info', "Open input $qf failed, dying");
closelog();
die "Open input $qf failed!";
}
if (-e $msg) {
syslog('info', "Output file exists $msg so failed, dying");
closelog();
die "Output file exists $msg";
}
if (! open (MSG, ">$msg")) {
syslog('info', "Open output $msg failed, dying");
closelog();
die "Open output $msg failed!";
}
while (defined ($line = <QF>) ) {
chomp $line;
last if ($line =~ /^\./); # Bat book 23.9.19
if ($line =~ /^R/) { # This is needed for Bcc per CGate Pipe specs (see help file)
$line =~ s/^R[A-Z]*:/Envelope-To: /; # change sendmail RPFD to Envelope-To
print MSG "$line\n";
}
elsif ($line =~ /^H/) { # get the headers and put them in the msg
$line =~ s/^H//;
print MSG "$line\n";
}
elsif ($line =~ /^\s/) { # these should only be on line-wrapped
print MSG "$line\n"; # headers, so put them in the msg
}
# there's no "else" because we don't care about the other lines
}
print MSG "\n"; # put in a blank line to make sure there's one between the
# headers and the data
close (QF);
close (MSG);
# append the sendmail data file to the cgp message
if (system ("/bin/cat $df >> $msg") == 0) {
if (rename ($msg, "$msg.sub")) {
system ("/bin/rm $df $qf");
} else {
syslog('info', "rename $msg $msg.sub failed");
}
} else {
syslog('info', "cat $df to $msg failed");
}
}
closelog();
exit(0);
More information about the MailScanner
mailing list