NO! Dont go! ms2cgp problem
Vasiliy Boulytchev
vasiliy at linuxspecial.com
Wed Jan 17 18:00:15 CET 2007
Thank you very much for looking into. I think the problem is somewhere
in MS itself, since I am able to call your scripts manually, and get the
proper results.
THANKS!
Vasiliy Boulytchev
vasiliy at linuxspecial.com
John Rudd wrote:
>
>
> I haven't looked at those scripts in years...
>
> If I get a chance, I'll look at it, but I'm a little busy at the
> moment (some on-going file server issues, and a web server that's down).
>
>
> John
>
>
>
> Vasiliy Boulytchev wrote:
>> John,
>> First of all, THANK YOU SO MUCH for helping us out! I have been
>> using your cgp2ms and ms2cgp scripts for years flawlessly.
>> Now that you decide to leave us, I get a problem with ms2cgp on a
>> new install...
>>
>> It seems the MS is not giving the correct $job value to the ms2cgp
>> script.... syslog:
>>
>> --
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: job =
>> -OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: qf =
>> /etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>>
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: df =
>> /etc/MailScanner/shared/queues/ms1/out/df-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>>
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: msg =
>> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
>>
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Job
>> -OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out writing to
>> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
>>
>> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Open input
>> /etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>> failed, dying
>> --
>>
>> Now, if I call ms2cgp from commandline via this script:
>> #!/usr/bin/perl -w
>>
>> opendir(DIR, ".");
>> @files = grep(/qf/,readdir(DIR));
>> closedir(DIR);
>>
>> foreach $file (@files) {
>> substr($file, 0, 1) = "";
>> substr($file, 0, 1) = "";
>> system ("/usr/local/etc/ms2cgp $file");
>> }
>>
>> The batch processes just fine, syslog:
>>
>> --
>> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: job = 19659
>> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: qf =
>> /etc/MailScanner/shared/queues/ms1/out/qf19659
>> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: df =
>> /etc/MailScanner/shared/queues/ms1/out/df19659
>> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: msg =
>> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
>> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: Job 19659 writing to
>> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
>> --
>>
>> So it seems to me, MS is not passing the correct $job value...
>> furthermore, why is it making it -OQueueDirectory!?!?!?
>>
>> What do you guys think?
>>
>> THANK YOU VERY MUCH!!! My cgp2ms and ms2cgp scripts are attached.
>>
>> Vasiliy Boulytchev
>> vasiliy at linuxspecial.com
>>
>>
>>
>> John Rudd wrote:
>>>
>>>
>>> Just wanted to thank everyone at this list, starting with Julian,
>>> but also several of the subscribers who given a lot of information
>>> and input over the years I've been here. At some points, this was
>>> an incredibly valuable and informative mailing list for me.
>>>
>>> And, what has changed is not the list. The list, and the vast
>>> majority of its subscribers, are still great. What changed were my
>>> anti-spam/anti-virus needs. I'm moving to solutions that operate
>>> entirely during the SMTP session. As a result, I decommissioned my
>>> last mailscanner system 2 weeks ago.
>>>
>>> After a short waiting period to be sure I wasn't going to need to
>>> roll back, or have any other mailscanner related questions, I'm
>>> going to be unsubscribing from the list later today.
>>>
>>> I wish all of you the best, and thank you for a great set of
>>> software, a great source of information resources, and a great
>>> source of conversation over the last 4 or 5 years.
>>
>> ------------------------------------------------------------------------
>>
>> #!/usr/bin/perl
>>
>> # cgp2ms - part of a MailScanner to CommuniGate Pro gateway
>> # Copyright (C) 2003 The Regents of the University of California
>> #
>> # This program is free software; you can redistribute it and/or modify
>> # it under the terms of the GNU General Public License as published by
>> # the Free Software Foundation; either version 2 of the License, or
>> # (at your option) any later version.
>> #
>> # This program is distributed in the hope that it will be useful,
>> # but WITHOUT ANY WARRANTY; without even the implied warranty of
>> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> # GNU General Public License for more details.
>> #
>> # You should have received a copy of the GNU General Public License
>> # along with this program; if not, write to the Free Software #
>> Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
>> USA
>> #
>> # The Author, John Rudd, can be by email at
>> # jrudd at ucsc.edu
>> #
>> # cgp2ms -f file Actually process the given file
>> # or
>> # cgp2ms -c Just check the queue sizes and print result #
>> # This program is used by CommuniGate Pro as an "execute" filter. It
>> takes
>> # in arguments that define where the CommuniGate Pro (CGP) queue file
>> lives
>> # and uses the queue file to create a sendmail formated queue file
>> (qf) and
>> # data file (df) pair. These two files are then inserted into the
>> # MailScanner (MS) incoming queue directory. When MS is done, it
>> will use a
>> # program which is a companion to this one, ms2cgp, # to give the
>> message
>> # back to CGP.
>> #
>> # Modified 10/2004 by Vasiliy Boulytchev and Randy Lindsey
>> # Added load balancing and sped up processing for larger volume
>> # 12/19/2004 Randy Lindsey
>> # Rather than read queuedirs for qf* files across remote shares,
>> # this version reads a set of files created on each system
>> containing the queue count
>> # # To take a host offline, just comment out its line in hostlist
>> %hostlist = (
>> # IA => '/var/CommuniGate/MailScanner/Incoming',
>> # TX2 => '/usr/local/mailscanner.tx2',
>> # AZ => '/usr/local/mailscanner.az',
>> # TMP => '/usr/local/mailscanner.tmp',
>> # CA => '/usr/local/mailscanner.ak/mailscanner.in',
>> MS1 => '/var/CommuniGate/MS.Status/queues/ms1/in',
>> # MS2 => '/usr/local/mailscanner.ak/mailscanner.ms2.in',
>> # MS3 => '/usr/local/mailscanner.ak/mailscanner.ms3.in',
>> # MS4 => '/usr/local/mailscanner.ak/mailscanner.ms4.in',
>> # MS5 => '/usr/local/mailscanner.ak/mailscanner.ms5.in',
>> # MS6 => '/usr/local/mailscanner.ak/mailscanner.ms6.in',
>> # MS7 => '/usr/local/mailscanner.ak/mailscanner.ms7.in',
>> # MS8 => '/usr/local/mailscanner.ak/mailscanner.ms8.in',
>> # MS9 => '/usr/local/mailscanner.ak/mailscanner.ms9.in',
>> );
>>
>> # hostspeed - larger numbers are faster
>> %hostspeed = (
>> # IA => 1,
>> # TX2 => 4,
>> # AZ => 0.6,
>> # CA => 9,
>> # TMP => 100,
>> MS1 => 2,
>> MS2 => 2,
>> MS3 => 2,
>> MS4 => 15,
>> MS5 => 15,
>> MS6 => 30,
>> MS7 => 30,
>> MS8 => 30,
>> MS9 => 30,
>> );
>>
>> sub check_one { #$queuedirname
>> open DH, "</var/CommuniGate/MS.Status/queues/count/$_[0]" or
>> return 999999;
>> read(DH, $MsgsInQueue, 10);
>> close DH;
>> chomp $MsgsInQueue;
>> return $MsgsInQueue;
>> }
>>
>> sub check_queues {
>>
>> $totfiles = 0;
>> $totpoints = 0;
>> foreach my $hostid (keys %hostlist) {
>> $hostfiles{$hostid} = check_one(lc($hostid));
>> $totfiles += $hostfiles{$hostid};
>> # Assign points based on relative system speed
>> # We scale it up by 737 to reduce rounding errors with
>> small numbers
>> if ($hostspeed{$hostid}) {
>> $hostpoints{$hostid} = ($hostfiles{$hostid} * 737) /
>> $hostspeed{$hostid};
>> }
>> else {
>> $hostpoints{$hostid} = $hostfiles{$hostid};
>> }
>> }
>>
>> @hostsort = sort { $hostpoints{$a} <=> $hostpoints{$b} } keys
>> %hostpoints;
>> printf ("MailScanner to %s - Tot:%2u ",
>> $hostsort[0], $totfiles); # displays in CommuniGate logs
>> foreach my $hostid (keys %hostlist) {
>> printf ("%s:%2u/%2u ", $hostid, $hostfiles{$hostid},
>> $hostpoints{$hostid});
>> }
>> print ("\n");
>> return $hostlist{$hostsort[0]};
>> }
>>
>> sub copy_stdin {
>> # Extra headers added by prior rules
>> my $hdrline;
>> while (defined($hdrline = <STDIN>)) {
>> print QF "H$hdrline";
>> print " $hdrline "; # display in logs
>> }
>> }
>>
>> my ($i, $file, $from, $rcpt, @tempv, @rcpt, @argv, $inbody, $inhdr,
>> $inmeta);
>>
>> $checkonly = 0;
>> for ($i = 0; $i <= $#ARGV; $i++) { # parse in the arguments
>> if (defined($ARGV[$i])) {
>> if ($ARGV[$i] eq "-f") { # the location of the cgp queue file
>> $i++;
>> $file = $ARGV[$i];
>> }
>> elsif ($ARGV[$i] eq "-c") { # just check queue sizes and
>> print result
>> $checkonly = 1;
>> }
>> else { # left over from debugging, these
>> push (@argv, $ARGV[$i]); # aren't actually used
>> }
>> }
>> }
>> if (! $file) {
>> $checkonly = 1; # default if no parameters passed
>> }
>>
>> $qdir = check_queues(); # load balance between servers
>> if ($checkonly) {
>> exit(0);
>> }
>>
>> open (JOB, "<$file");
>> open (QF, ">$qdir/msqf$$"); # minimize file moving by writing to
>> final dir but
>> open (DF, ">$qdir/msdf$$"); # under a different name to prevent
>> Mailscanner taking it
>>
>> print QF "V4\n"; # I don't think MS actually uses these
>> print QF "T" . time . "\n"; # qf lines, but it helps the qf file
>> print QF "K0\n"; # be a little more authentic
>> print QF "N0\n"; #
>> print QF "P150900\n"; #
>> $inmeta = 1; # when reading the job file, are we still in the meta
>> data?
>> $inhdr = 0; # when reading the job file, are we in the rfc822 headers?
>> $inbody = 0; # when reading the job file, are we in the rfc822
>> body/data?
>> $rec = 0; # have we read the most recent Received header yet?
>> # the most recent (first listed) Received header will
>> # tell us the information we'll use in the qf file's $_
>> # which is "which host relayed this message to us"
>>
>> while (defined ($line = <JOB>)) {
>> chomp $line;
>> if ($line eq "") { # blank lines demark meta, headers, and body
>> sections
>> if ($inmeta) {
>> $inmeta = 0; $inhdr = 1;
>> }
>> elsif ($inhdr) {
>> $inhdr = 0; $inbody = 1;
>> copy_stdin;
>> if ( !($rec) ) { # no received header = from localhost
>> $rec = 1;
>> print QF "\$_localhost [127.0.0.1]\n"; # print relay host
>> print QF "S$from\n"; # print sender
>> foreach $to (@rcpt) { # print the recipient list
>> print QF "RPFD:$to\n";
>> }
>> }
>> }
>> }
>> elsif ($inmeta) {
>> if ($line =~ /^P/) {
>> @tempv = split(/ /, $line);
>> $from = $tempv[7];
>> }
>> elsif ($line =~ /^R/) {
>> @tempv = split(/ /, $line);
>> $rcpt = $tempv[7];
>> push (@rcpt, $rcpt);
>> }
>> }
>> elsif ($inhdr) {
>> if ( (!($rec)) && ($line =~ /^Received:/) ) { # get relay host
>> $rec = 1;
>> if ($line =~ /^Received: from (.*) \((.* )?(\[.*\]).*/) {
>> if (defined $2) {
>> $h = $2;
>> $h =~ s/\s*$//;
>> }
>> else {
>> $h = $1;
>> }
>> $a = $3;
>> print QF "\$_$h $a\n"; # print relay host
>> }
>> else {
>> $line =~ /^Received: from (\[.*\]).*/;
>> $a = $1;
>> print QF "\$_$a\n"; # print relay host
>> }
>> print QF "S$from\n"; # print sender
>> foreach $to (@rcpt) { # print the recipient list
>> print QF "RPFD:$to\n";
>> }
>> print QF "H$line\n"; # then print the Received header
>> }
>> elsif ($line !~ /^\s/) { # get a header
>> print QF "H$line\n";
>> }
>> else { # get the rest of a multi-line header
>> print QF "$line\n";
>> }
>> }
>> if ($inbody) {
>> print DF "$line\n";
>> }
>> }
>>
>> print QF ".\n"; # bat book 23.9.19, qf file should end in a "^\.$" line.
>>
>> close (DF);
>> close (QF);
>> close(JOB);
>>
>> # Rename the df file first, as Mailscanner looks for the qf and might
>> interfere
>> # Note that this used to mv the files from /tmp to qdir, but a
>> fraction of the time
>> # this failed due to race conditions with Mailscanner
>> rename ("$qdir/msdf$$", "$qdir/df$$");
>> rename ("$qdir/msqf$$", "$qdir/qf$$");
>>
>> exit(0);
>>
>>
>> ------------------------------------------------------------------------
>>
>> #!/usr/bin/perl
>>
>> # ms2cgp - part of a MailScanner to CommuniGate Pro gateway
>> # Copyright (C) 2003 The Regents of the University of California
>> #
>> # This program is free software; you can redistribute it and/or modify
>> # it under the terms of the GNU General Public License as published by
>> # the Free Software Foundation; either version 2 of the License, or
>> # (at your option) any later version.
>> #
>> # This program is distributed in the hope that it will be useful,
>> # but WITHOUT ANY WARRANTY; without even the implied warranty of
>> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> # GNU General Public License for more details.
>> #
>> # You should have received a copy of the GNU General Public License
>> # along with this program; if not, write to the Free Software
>> # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA
>> 02111-1307 USA
>> #
>> # The Author, John Rudd, can be by email at
>> # jrudd at ucsc.edu
>> #
>> #
>> # ms2cgp -qI(id) ...
>> #
>> # This program acts as Mailscanner's outgoing "Sendmail2" program,
>> taking
>> # an argument for each message Mailscanner (MS) has finished processing,
>> # and using that argument to find the pair of sendmail mqueue files that
>> # make up the message. This program then re-combines them into an
>> rfc822
>> # file and submits them back to Communigate Pro (CGP) via its
>> Submitted directory
>> #
>> # Modified 11/4/04 by Randy Lindsey to skip "sendmail" program and just
>> # write directly to Submitted in RFC822 format.
>>
>> my $ServerPrefix = "MS1"; # make this unique per server
>> my $QDIR = "/etc/MailScanner/shared/queues/ms1/out"; #
>> where MS sticks outgoing msgs.
>> my $SubDir = "/etc/MailScanner/mail.cluster/submx00"; # CG's
>> Submitted directory
>> # my $Archive = "/extra/archivems2cgp/ia";
>>
>> my ($job, $from, $rcpt, $df, $qf, $msg, $line);
>>
>> use Sys::Syslog;
>> openlog('ms2cgp', 'pid', 'user');
>>
>> foreach $job (@ARGV) { $job =~ s/^-qI//;
>> $qf = $QDIR . "/qf" . $job; # the sendmail formatted queue file
>> $df = $QDIR . "/df" . $job; # the sendmail formatted data file
>> $msg = $SubDir . "/" . $ServerPrefix . ".ms2cgp.$job.$$.tmp"; #
>> the tempfile we'll give to CGP
>> $rcpt = "";
>>
>> # syslog('info', "Job $job copying to archive $Archive");
>> # system("/bin/cp -f $qf $Archive/qf$job");
>> # system("/bin/cp -f $df $Archive/df$job");
>>
>> syslog ('info', "job = $job");
>> closelog();
>> syslog ('info', "qf = $qf");
>> closelog();
>> syslog ('info', "df = $df");
>> closelog();
>> syslog ('info', "msg = $msg");
>> closelog();
>>
>>
>>
>>
>>
>> syslog('info', "Job $job writing to $msg");
>> if (! open (QF, "<$qf")) {
>> syslog('info', "Open input $qf failed, dying");
>> closelog();
>> die "Open input $qf failed!";
>> }
>> if (-e $msg) {
>> syslog('info', "Output file exists $msg so failed, dying");
>> closelog();
>> die "Output file exists $msg";
>> }
>> if (! open (MSG, ">$msg")) {
>> syslog('info', "Open output $msg failed, dying");
>> closelog();
>> die "Open output $msg failed!";
>> }
>>
>> while (defined ($line = <QF>) ) {
>> chomp $line;
>>
>> last if ($line =~ /^\./); # Bat book 23.9.19
>>
>> if ($line =~ /^R/) { # This is needed for Bcc per CGate
>> Pipe specs (see help file)
>> $line =~ s/^R[A-Z]*:/Envelope-To: /; # change sendmail RPFD
>> to Envelope-To
>> print MSG "$line\n";
>> }
>> elsif ($line =~ /^H/) { # get the headers and put them in
>> the msg
>> $line =~ s/^H//;
>> print MSG "$line\n";
>> }
>> elsif ($line =~ /^\s/) { # these should only be on line-wrapped
>> print MSG "$line\n"; # headers, so put them in the msg
>> }
>> # there's no "else" because we don't care about the other lines
>> }
>> print MSG "\n"; # put in a blank line to make sure there's one
>> between the
>> # headers and the data
>>
>> close (QF);
>> close (MSG);
>>
>> # append the sendmail data file to the cgp message
>> if (system ("/bin/cat $df >> $msg") == 0) {
>> if (rename ($msg, "$msg.sub")) {
>> system ("/bin/rm $df $qf");
>> } else {
>> syslog('info', "rename $msg $msg.sub failed");
>> }
>> } else {
>> syslog('info', "cat $df to $msg failed");
>> }
>> }
>> closelog();
>> exit(0);
>>
More information about the MailScanner
mailing list