Any one else notice...

Mon Jun 3 13:29:05 IST 2002

ill test :)

----- Original Message -----
From: "Julian Field" <jkf at ECS.SOTON.AC.UK>
To: <MAILSCANNER at JISCMAIL.AC.UK>
Sent: Saturday, June 01, 2002 3:20 PM
Subject: Re: Any one else notice...


> I've spent the whole day staring at code today...
> and I *think* I have found the problem. What changed between the 2
versions
> Kelly first reported was that the code was doing 2 extra "-f" checks on
> each message. It's quite possible that some OS's don't cache directory
> metadata (i.e. the data about the files in the dir) very cleverly when
that
> data is constantly changing, as it does when I am processing the queue.
>
> So I've optimised out those "-f" checks, and have also been through the
> whole of the rest of the code optimising all the "-f" (file existence)
> checks and all the "-d" (directory existence) checks.
>
> So if I'm right, it should now run faster than it ever did :-)
>
> All I need now is a willing tester who has got a nice heavily loaded mail
> server.
> Kelly perhaps?
> If Kelly can't then any other offers for testing would be very much
> appreciated.
>
> Another thing I've been working on is a much-improved RPM. It will now
only
> install any needed Perl modules if you don't have a recent-enough version
> installed already. And if you find that the TNEF expander isn't coping
with
> a lot of your Outlook attachments, you now have the option to use a Perl
> module TNEF expander instead of the external binary one.
>
> I think there's one or two other things as well, but that's enough to keep
> you happy for now :-)
>
> Jules.
>
> At 21:18 31/05/2002, you wrote:
> >unfortunatly were already running a name server on the same machine, and
> >have 512megs ram in it. Its only a p3 500, but it was keeping up with the
> >load great before. I have my sendmail set to do rbl checks, i have spam
> >assassin's user_prefs set with skip_rbl_checks 1 and ignore_rbl_checks 1
and
> >commented the rbl checks out in mailscanner (this is how i was running
> >already) Something changed in the code in the last release or so which
> >changed some timing. Julian mentioned to me that spam checking is now
done
> >on seprate forks (seprate processes) and hes not sure if thats where the
> >bottleneck is. Although i did some testing with spam checks = no and it
> >improved a little but not as much as it should. In version 3.13-2 witrh
spam
> >checks off i can process 2000 messages in like 5 minutes, or less. with
the
> >new version it took well over 15 minutes.
> >
> >Just some more information for you guys.
> >thanks.
> >
> >----- Original Message -----
> >From: "Jeff A. Earickson" <jaearick at COLBY.EDU>
> >To: <MAILSCANNER at JISCMAIL.AC.UK>
> >Sent: Friday, May 31, 2002 3:49 PM
> >Subject: Re: Any one else notice...
> >
> >
> > > Hi,
> > >
> > >    I'll weigh in on this thread too.  Last Monday, I moved our mail
> >service
> > > to a Sun E220R, 2 cpus, running Solaris 8.  I'm running mailscanner
with
> > > spamassassin turned on.  I have all of the "Spam List" options in
> > > mailscanner.conf commented out.  I have Spamcop assigned a non-zero
score
> > > in spamassassin, so I hope/think SA is using spamcop (I'm not sure
yet).
> > > We use RBL+ in sendmail, and we subscribe to it transfer mode, so RBL+
> > > mail gets rejected before getting to mailscanner.  I have the delivery
> > > mode in mailscanner set to "queue".  We use Sophos.  We process
roughly
> > > 20K messages a day.  Result:  the system works great, no slowdowns, no
> > > clogged queues, nothing but bliss.  Julian should be knighted, IMHO.
> > >
> > >    Most of what I see in this thread sounds like DNS slowdowns.
Here's
> > > my advice:
> > >
> > >   * run a modern version bind on your mail server, at least in caching
> >mode,
> > >     to handle the DNS lookups for you.  If you use RBL+ or other
> >zone-transfer
> > >     mode DNS blocklists, do the zone transfers to the mail server, so
> > >     DNS queries never leave the box for RBL+.  You will probably have
to
> > >     pay money to get zone transfers.  As a part of running bind on
your
> >mail
> > >     server, make sure /etc/resolv.conf is configured so that the first
> >entry
> > >     is the external interface (not loopback) of the mail server.  Here
is
> >my
> > >     resolv.conf for my server, emerald:
> > >
> > > domain colby.edu
> > > nameserver 137.146.210.52   # emerald, this host, not loopback -- for
RBL+
> > > nameserver 137.146.210.46   # opal
> > > nameserver 137.146.210.45   # ruby
> > > nameserver 139.140.1.1      # polar.bowdoin.edu
> > > nameserver 204.70.128.1     # ns.cw.net
> > >
> > >     Any DNS lookup on emerald goes to the local cache first, then
other
> > >     local machines, then remotely.
> > >
> > > *   Have lots of memory in the machine for named to use.  Named is
using
> > >     about 140 MB of resident memory on my machine right now.  If you
are
> > >     using bind 9.X (you should be) and have a multi-cpu machine, let
bind
> > >     run threads on all cpus.
> > >
> > > *   If you are doing DNS spam-blocking, do it in sendmail.  Reject the
> > >     stuff before it gets to mailscanner or spamassassin.
> > >
> > > *   Comment out some the "Spam List" lookups in mailscanner and see if
> > >     that helps.  Fewer DNS lookups, especially to a remote site that
is
> > >     overloaded (like relays.ordb.org perhaps), may be a bottleneck.
> >Likewise,
> > >     check the config for SA and try to control DNS lookups there too.
> > >
> > > *   If you are running Solaris, shut off nscd!!  This code is a real
> > >     DNS bottleneck for a system doing beaucoup lookups.  When we first
> > >     moved our Apache web server to Sun, we were getting glacial
response
> > >     times to webpage requests.  I found a technote at the Apache site
> >about
> > >     nscd problems.  Turned it off, and things ran fast after that.
> > >     The same advice applies to other Solaris apps doing massive DNS,
> > >     and nscd could appear on other versions of UNIX.  Let bind do the
> > >     work instead.
> > >
> > > Of course our students are gone right now.  The system could blow up
when
> > > they return next Fall.
> > >
> > > ** Jeff A. Earickson, Ph.D                         PHONE: 207-872-3659
> > > ** Senior UNIX Sysadmin, Information Technology    EMAIL:
> >jaearick at colby.edu
> > > ** Colby College, 4214 Mayflower Hill,               FAX: 207-872-3076
> > > ** Waterville ME, 04901-8842
> >
> --------------------------------------------------------------------------
> >--
> > >
>
> --
> Julian Field                Teaching Systems Manager
> jkf at ecs.soton.ac.uk         Dept. of Electronics & Computer Science
> Tel. 023 8059 2817          University of Southampton
>                              Southampton SO17 1BJ
>