Real time logging to sql database

[Kearney, Rob]

Just fyi,

This topic came up several times in the list. You might want to research the
archives.

We modified the perl code to log each message in real time to the SQL
database.
Our message rate was about 65,000 message per day (.5-1 per sec). We
averaged about 25-30% cpu on a Compaq DL360g2 1.4Ghz PIII Xeon, and 1Gb RAM.
We used spamassassin with Razor2/pyzor/dcc, no bayes. We did not virus scan.

-rob

-----Original Message-----
From: David H. [mailto:dh at UPTIME.AT]
Sent: Monday, December 22, 2003 4:09 PM
To: MAILSCANNER at JISCMAIL.AC.UK
Subject: Re: Real time logging to sql database


-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

Paul Hamilton wrote:

> Hi all,
Hello.
>
> Is there anybody using real time logging for Mailscanner to a
> sql database per message, rather than generating temporary
> files and flushing every restart?
>
Just some general thought. I wonder how secure this is and what kind of
performance hit one has to take. Theorethically, if I wish to do this
"securely" each log message has to be its own transaction. Thus a
singular INSERT results ina full transaction within the database. Now,
when I imagine I have a fiarly busy MailServer with, let's say, 0.5 or 1
message per second, that might nag on the database a whole lot when I
log _directly_ into the Daemon.

Maybe it is indeed faster to simply write out SQL INSERT statements into
a file, to the disc and sync this file into the database every 15 or 30
minutes. This could be done in a single transaction then.

But then again, I am by far no Database guru.

- -d

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (Darwin)

iD8DBQE/5112PMoaMn4kKR4RA9mbAJ4rOPYzjnGcL9kus9eT1poJ/ZSwUQCeN57j
9L4MUB1vnC6x/zh6nRxBtFs=
=tSih
-----END PGP SIGNATURE-----