NO! Dont go! ms2cgp problem
Vasiliy Boulytchev
vasiliy at linuxspecial.com
Wed Jan 17 18:10:46 CET 2007
OK, a bit more information...
cd /usr
grep -R QueueDirectory *
lib/MailScanner/MailScanner/CustomConfig.pm:# sendmail -q30m
-OQueueDirectory=/var/spool/mqueue.spam
lib/MailScanner/MailScanner/CustomConfig.pm:# sendmail -q60m
-OQueueDirectory=/var/spool/mqueue.highspam
lib/MailScanner/MailScanner/Sendmail.pm: $args = "
-OQueueDirectory=$outqdir " if $outqdir;
src/mailscanner/src/MailScanner-install-4.57.6/perl-tar/MailScanner-4.57.6/lib/MailScanner/CustomConfig.pm:#
sendmail -q30m -OQueueDirectory=/var/spool/mqueue.spam
src/mailscanner/src/MailScanner-install-4.57.6/perl-tar/MailScanner-4.57.6/lib/MailScanner/CustomConfig.pm:#
sendmail -q60m -OQueueDirectory=/var/spool/mqueue.highspam
src/mailscanner/src/MailScanner-install-4.57.6/perl-tar/MailScanner-4.57.6/lib/MailScanner/Sendmail.pm:
$args = " -OQueueDirectory=$outqdir " if $outqdir;
src/mailscanner/src/MailScanner-install-4.57.6/perl-tar/MailScanner-4.57.6/INSTALL.OpenBSD:
sendmail_flags="-bd -OPrivacyOptions=noetrn -ODeliveryMode=queueonly
-OQueueDirectory=/var/spool/mqueue.in"
Any suggestions? :)
Again, I really dont need sendmail, I dont know why its everywhere.
probably my misunderstanding....
Vasiliy Boulytchev
vasiliy at linuxspecial.com
Vasiliy Boulytchev wrote:
> John,
> First of all, THANK YOU SO MUCH for helping us out! I have been
> using your cgp2ms and ms2cgp scripts for years flawlessly.
> Now that you decide to leave us, I get a problem with ms2cgp on a
> new install...
>
> It seems the MS is not giving the correct $job value to the ms2cgp
> script.... syslog:
>
> --
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: job =
> -OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: qf =
> /etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: df =
> /etc/MailScanner/shared/queues/ms1/out/df-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
>
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: msg =
> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
>
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Job
> -OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out writing to
> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out.19781.tmp
>
> Jan 17 11:02:20 mailscanner1 ms2cgp[19781]: Open input
> /etc/MailScanner/shared/queues/ms1/out/qf-OQueueDirectory=/etc/MailScanner/shared/queues/ms1/out
> failed, dying
> --
>
> Now, if I call ms2cgp from commandline via this script:
> #!/usr/bin/perl -w
>
> opendir(DIR, ".");
> @files = grep(/qf/,readdir(DIR));
> closedir(DIR);
>
> foreach $file (@files) {
> substr($file, 0, 1) = "";
> substr($file, 0, 1) = "";
> system ("/usr/local/etc/ms2cgp $file");
> }
>
> The batch processes just fine, syslog:
>
> --
> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: job = 19659
> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: qf =
> /etc/MailScanner/shared/queues/ms1/out/qf19659
> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: df =
> /etc/MailScanner/shared/queues/ms1/out/df19659
> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: msg =
> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
> Jan 17 11:07:53 mailscanner1 ms2cgp[19859]: Job 19659 writing to
> /etc/MailScanner/mail.cluster/submx00/MS1.ms2cgp.19659.19859.tmp
> --
>
> So it seems to me, MS is not passing the correct $job value...
> furthermore, why is it making it -OQueueDirectory!?!?!?
>
> What do you guys think?
>
> THANK YOU VERY MUCH!!! My cgp2ms and ms2cgp scripts are attached.
>
> Vasiliy Boulytchev
> vasiliy at linuxspecial.com
>
>
>
> John Rudd wrote:
>>
>>
>> Just wanted to thank everyone at this list, starting with Julian, but
>> also several of the subscribers who given a lot of information and
>> input over the years I've been here. At some points, this was an
>> incredibly valuable and informative mailing list for me.
>>
>> And, what has changed is not the list. The list, and the vast
>> majority of its subscribers, are still great. What changed were my
>> anti-spam/anti-virus needs. I'm moving to solutions that operate
>> entirely during the SMTP session. As a result, I decommissioned my
>> last mailscanner system 2 weeks ago.
>>
>> After a short waiting period to be sure I wasn't going to need to
>> roll back, or have any other mailscanner related questions, I'm going
>> to be unsubscribing from the list later today.
>>
>> I wish all of you the best, and thank you for a great set of
>> software, a great source of information resources, and a great source
>> of conversation over the last 4 or 5 years.
> ------------------------------------------------------------------------
>
> #!/usr/bin/perl
>
> # cgp2ms - part of a MailScanner to CommuniGate Pro gateway
> # Copyright (C) 2003 The Regents of the University of California
> #
> # This program is free software; you can redistribute it and/or modify
> # it under the terms of the GNU General Public License as published by
> # the Free Software Foundation; either version 2 of the License, or
> # (at your option) any later version.
> #
> # This program is distributed in the hope that it will be useful,
> # but WITHOUT ANY WARRANTY; without even the implied warranty of
> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> # GNU General Public License for more details.
> #
> # You should have received a copy of the GNU General Public License
> # along with this program; if not, write to the Free Software
> # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> #
> # The Author, John Rudd, can be by email at
> # jrudd at ucsc.edu
> #
> # cgp2ms -f file Actually process the given file
> # or
> # cgp2ms -c Just check the queue sizes and print result
> #
> # This program is used by CommuniGate Pro as an "execute" filter. It takes
> # in arguments that define where the CommuniGate Pro (CGP) queue file lives
> # and uses the queue file to create a sendmail formated queue file (qf) and
> # data file (df) pair. These two files are then inserted into the
> # MailScanner (MS) incoming queue directory. When MS is done, it will use a
> # program which is a companion to this one, ms2cgp, # to give the message
> # back to CGP.
> #
> # Modified 10/2004 by Vasiliy Boulytchev and Randy Lindsey
> # Added load balancing and sped up processing for larger volume
> # 12/19/2004 Randy Lindsey
> # Rather than read queuedirs for qf* files across remote shares,
> # this version reads a set of files created on each system containing the queue count
> #
> # To take a host offline, just comment out its line in hostlist
> %hostlist = (
> # IA => '/var/CommuniGate/MailScanner/Incoming',
> # TX2 => '/usr/local/mailscanner.tx2',
> # AZ => '/usr/local/mailscanner.az',
> # TMP => '/usr/local/mailscanner.tmp',
> # CA => '/usr/local/mailscanner.ak/mailscanner.in',
> MS1 => '/var/CommuniGate/MS.Status/queues/ms1/in',
> # MS2 => '/usr/local/mailscanner.ak/mailscanner.ms2.in',
> # MS3 => '/usr/local/mailscanner.ak/mailscanner.ms3.in',
> # MS4 => '/usr/local/mailscanner.ak/mailscanner.ms4.in',
> # MS5 => '/usr/local/mailscanner.ak/mailscanner.ms5.in',
> # MS6 => '/usr/local/mailscanner.ak/mailscanner.ms6.in',
> # MS7 => '/usr/local/mailscanner.ak/mailscanner.ms7.in',
> # MS8 => '/usr/local/mailscanner.ak/mailscanner.ms8.in',
> # MS9 => '/usr/local/mailscanner.ak/mailscanner.ms9.in',
> );
>
> # hostspeed - larger numbers are faster
> %hostspeed = (
> # IA => 1,
> # TX2 => 4,
> # AZ => 0.6,
> # CA => 9,
> # TMP => 100,
> MS1 => 2,
> MS2 => 2,
> MS3 => 2,
> MS4 => 15,
> MS5 => 15,
> MS6 => 30,
> MS7 => 30,
> MS8 => 30,
> MS9 => 30,
> );
>
> sub check_one { #$queuedirname
> open DH, "</var/CommuniGate/MS.Status/queues/count/$_[0]" or return 999999;
> read(DH, $MsgsInQueue, 10);
> close DH;
> chomp $MsgsInQueue;
> return $MsgsInQueue;
> }
>
> sub check_queues {
>
> $totfiles = 0;
> $totpoints = 0;
> foreach my $hostid (keys %hostlist) {
> $hostfiles{$hostid} = check_one(lc($hostid));
> $totfiles += $hostfiles{$hostid};
> # Assign points based on relative system speed
> # We scale it up by 737 to reduce rounding errors with small numbers
> if ($hostspeed{$hostid}) {
> $hostpoints{$hostid} = ($hostfiles{$hostid} * 737) / $hostspeed{$hostid};
> }
> else {
> $hostpoints{$hostid} = $hostfiles{$hostid};
> }
> }
>
> @hostsort = sort { $hostpoints{$a} <=> $hostpoints{$b} } keys %hostpoints;
> printf ("MailScanner to %s - Tot:%2u ",
> $hostsort[0], $totfiles); # displays in CommuniGate logs
> foreach my $hostid (keys %hostlist) {
> printf ("%s:%2u/%2u ", $hostid, $hostfiles{$hostid}, $hostpoints{$hostid});
> }
> print ("\n");
> return $hostlist{$hostsort[0]};
> }
>
> sub copy_stdin {
> # Extra headers added by prior rules
> my $hdrline;
> while (defined($hdrline = <STDIN>)) {
> print QF "H$hdrline";
> print " $hdrline "; # display in logs
> }
> }
>
> my ($i, $file, $from, $rcpt, @tempv, @rcpt, @argv, $inbody, $inhdr, $inmeta);
>
> $checkonly = 0;
> for ($i = 0; $i <= $#ARGV; $i++) { # parse in the arguments
> if (defined($ARGV[$i])) {
> if ($ARGV[$i] eq "-f") { # the location of the cgp queue file
> $i++;
> $file = $ARGV[$i];
> }
> elsif ($ARGV[$i] eq "-c") { # just check queue sizes and print result
> $checkonly = 1;
> }
> else { # left over from debugging, these
> push (@argv, $ARGV[$i]); # aren't actually used
> }
> }
> }
> if (! $file) {
> $checkonly = 1; # default if no parameters passed
> }
>
> $qdir = check_queues(); # load balance between servers
> if ($checkonly) {
> exit(0);
> }
>
> open (JOB, "<$file");
> open (QF, ">$qdir/msqf$$"); # minimize file moving by writing to final dir but
> open (DF, ">$qdir/msdf$$"); # under a different name to prevent Mailscanner taking it
>
> print QF "V4\n"; # I don't think MS actually uses these
> print QF "T" . time . "\n"; # qf lines, but it helps the qf file
> print QF "K0\n"; # be a little more authentic
> print QF "N0\n"; #
> print QF "P150900\n"; #
>
> $inmeta = 1; # when reading the job file, are we still in the meta data?
> $inhdr = 0; # when reading the job file, are we in the rfc822 headers?
> $inbody = 0; # when reading the job file, are we in the rfc822 body/data?
> $rec = 0; # have we read the most recent Received header yet?
> # the most recent (first listed) Received header will
> # tell us the information we'll use in the qf file's $_
> # which is "which host relayed this message to us"
>
> while (defined ($line = <JOB>)) {
> chomp $line;
> if ($line eq "") { # blank lines demark meta, headers, and body sections
> if ($inmeta) {
> $inmeta = 0; $inhdr = 1;
> }
> elsif ($inhdr) {
> $inhdr = 0; $inbody = 1;
> copy_stdin;
> if ( !($rec) ) { # no received header = from localhost
> $rec = 1;
> print QF "\$_localhost [127.0.0.1]\n"; # print relay host
> print QF "S$from\n"; # print sender
> foreach $to (@rcpt) { # print the recipient list
> print QF "RPFD:$to\n";
> }
> }
> }
> }
> elsif ($inmeta) {
> if ($line =~ /^P/) {
> @tempv = split(/ /, $line);
> $from = $tempv[7];
> }
> elsif ($line =~ /^R/) {
> @tempv = split(/ /, $line);
> $rcpt = $tempv[7];
> push (@rcpt, $rcpt);
> }
> }
> elsif ($inhdr) {
> if ( (!($rec)) && ($line =~ /^Received:/) ) { # get relay host
> $rec = 1;
> if ($line =~ /^Received: from (.*) \((.* )?(\[.*\]).*/) {
> if (defined $2) {
> $h = $2;
> $h =~ s/\s*$//;
> }
> else {
> $h = $1;
> }
> $a = $3;
> print QF "\$_$h $a\n"; # print relay host
> }
> else {
> $line =~ /^Received: from (\[.*\]).*/;
> $a = $1;
> print QF "\$_$a\n"; # print relay host
> }
> print QF "S$from\n"; # print sender
> foreach $to (@rcpt) { # print the recipient list
> print QF "RPFD:$to\n";
> }
> print QF "H$line\n"; # then print the Received header
> }
> elsif ($line !~ /^\s/) { # get a header
> print QF "H$line\n";
> }
> else { # get the rest of a multi-line header
> print QF "$line\n";
> }
> }
> if ($inbody) {
> print DF "$line\n";
> }
> }
>
> print QF ".\n"; # bat book 23.9.19, qf file should end in a "^\.$" line.
>
> close (DF);
> close (QF);
> close(JOB);
>
> # Rename the df file first, as Mailscanner looks for the qf and might interfere
> # Note that this used to mv the files from /tmp to qdir, but a fraction of the time
> # this failed due to race conditions with Mailscanner
> rename ("$qdir/msdf$$", "$qdir/df$$");
> rename ("$qdir/msqf$$", "$qdir/qf$$");
>
> exit(0);
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/perl
>
> # ms2cgp - part of a MailScanner to CommuniGate Pro gateway
> # Copyright (C) 2003 The Regents of the University of California
> #
> # This program is free software; you can redistribute it and/or modify
> # it under the terms of the GNU General Public License as published by
> # the Free Software Foundation; either version 2 of the License, or
> # (at your option) any later version.
> #
> # This program is distributed in the hope that it will be useful,
> # but WITHOUT ANY WARRANTY; without even the implied warranty of
> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> # GNU General Public License for more details.
> #
> # You should have received a copy of the GNU General Public License
> # along with this program; if not, write to the Free Software
> # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> #
> # The Author, John Rudd, can be by email at
> # jrudd at ucsc.edu
> #
> #
> # ms2cgp -qI(id) ...
> #
> # This program acts as Mailscanner's outgoing "Sendmail2" program, taking
> # an argument for each message Mailscanner (MS) has finished processing,
> # and using that argument to find the pair of sendmail mqueue files that
> # make up the message. This program then re-combines them into an rfc822
> # file and submits them back to Communigate Pro (CGP) via its Submitted directory
> #
> # Modified 11/4/04 by Randy Lindsey to skip "sendmail" program and just
> # write directly to Submitted in RFC822 format.
>
> my $ServerPrefix = "MS1"; # make this unique per server
> my $QDIR = "/etc/MailScanner/shared/queues/ms1/out"; # where MS sticks outgoing msgs.
> my $SubDir = "/etc/MailScanner/mail.cluster/submx00"; # CG's Submitted directory
> # my $Archive = "/extra/archivems2cgp/ia";
>
> my ($job, $from, $rcpt, $df, $qf, $msg, $line);
>
> use Sys::Syslog;
> openlog('ms2cgp', 'pid', 'user');
>
> foreach $job (@ARGV) {
> $job =~ s/^-qI//;
> $qf = $QDIR . "/qf" . $job; # the sendmail formatted queue file
> $df = $QDIR . "/df" . $job; # the sendmail formatted data file
> $msg = $SubDir . "/" . $ServerPrefix . ".ms2cgp.$job.$$.tmp"; # the tempfile we'll give to CGP
> $rcpt = "";
>
> # syslog('info', "Job $job copying to archive $Archive");
> # system("/bin/cp -f $qf $Archive/qf$job");
> # system("/bin/cp -f $df $Archive/df$job");
>
> syslog ('info', "job = $job");
> closelog();
> syslog ('info', "qf = $qf");
> closelog();
> syslog ('info', "df = $df");
> closelog();
> syslog ('info', "msg = $msg");
> closelog();
>
>
>
>
>
> syslog('info', "Job $job writing to $msg");
> if (! open (QF, "<$qf")) {
> syslog('info', "Open input $qf failed, dying");
> closelog();
> die "Open input $qf failed!";
> }
> if (-e $msg) {
> syslog('info', "Output file exists $msg so failed, dying");
> closelog();
> die "Output file exists $msg";
> }
> if (! open (MSG, ">$msg")) {
> syslog('info', "Open output $msg failed, dying");
> closelog();
> die "Open output $msg failed!";
> }
>
> while (defined ($line = <QF>) ) {
> chomp $line;
>
> last if ($line =~ /^\./); # Bat book 23.9.19
>
> if ($line =~ /^R/) { # This is needed for Bcc per CGate Pipe specs (see help file)
> $line =~ s/^R[A-Z]*:/Envelope-To: /; # change sendmail RPFD to Envelope-To
> print MSG "$line\n";
> }
> elsif ($line =~ /^H/) { # get the headers and put them in the msg
> $line =~ s/^H//;
> print MSG "$line\n";
> }
> elsif ($line =~ /^\s/) { # these should only be on line-wrapped
> print MSG "$line\n"; # headers, so put them in the msg
> }
> # there's no "else" because we don't care about the other lines
> }
>
> print MSG "\n"; # put in a blank line to make sure there's one between the
> # headers and the data
>
> close (QF);
> close (MSG);
>
> # append the sendmail data file to the cgp message
> if (system ("/bin/cat $df >> $msg") == 0) {
> if (rename ($msg, "$msg.sub")) {
> system ("/bin/rm $df $qf");
> } else {
> syslog('info', "rename $msg $msg.sub failed");
> }
> } else {
> syslog('info', "cat $df to $msg failed");
> }
> }
> closelog();
> exit(0);
>
>
More information about the MailScanner
mailing list