Duplicated messages

Wed Jul 29 14:12:29 UTC 2015

How small is that drive ? Pretty bad planning on someone's part .. Then to be reliant on a single MX for mail flow.

Either way .. Dump all entries from that day. Did I read it grew 10gb in a day ?

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 29, 2015, at 9:08 AM, Mark Adams > wrote:

It's not 1000 entries, its about that many (some id's with a huge amount more) per message ID. The Mailscanner DB has over 9,000,000 entries for that day. I was planning on looping through and deleting all the offending entries for the 300+ emails that seemed to get stuck.

The mailscanner DB is currently 14G (maillog.MYD)

On 29 July 2015 at 15:04, Jeremy McSpadden <jeremy at fluxlabs.net<mailto:jeremy at fluxlabs.net>> wrote:
1000 entries from a sql DB may be about 5mb of space.
I would do a little more digging on the / partition to see what's eating space. Probably better off dropping some older archive dir

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 29, 2015, at 9:00 AM, Mark Adams > wrote:

I agree with you completely, however that doesn't help my immediate situation. Can you provide advice on deleting from the mailscanner DB? is there any other tables I need to remove the offending ID's entries from?

Regards,
Mark

On 29 July 2015 at 14:34, Jeremy McSpadden <jeremy at fluxlabs.net<mailto:jeremy at fluxlabs.net>> wrote:
It's 2015 .. That shouldn't be an excuse. It's like 10 cents per 100gb of drive... Upgrade

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 29, 2015, at 8:27 AM, Mark Adams > wrote:

Hi Jeremy,

No I haven't yet - I'm short on space on my root partition still because of the large mysql DB so I want to clean that up first. Can you advise how best to do this? Is it safe enough to do delete from maillog where id='XXX';  for all the ID's with the dupes? is there any other tables that need to be cleared?

Regards,
Mark

On 29 July 2015 at 14:16, Jeremy McSpadden <jeremy at fluxlabs.net<mailto:jeremy at fluxlabs.net>> wrote:
Log won't show taint issues. Setup log rotation.

Have you enabled debug in mailscanner config like I stated yesterday ?

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 29, 2015, at 8:13 AM, Mark Adams > wrote:

Hi all,

So I have resolved getting the missing mails delivered from the quarantine. The main problem stopping this from being easy from the command line was the fact that "Quarantine Whole Messages As Queue Files = no" was set, whilst the MTA in use is exim. I've changed that setting to "yes" now...

I've read that if its postfix you can just send that "message" file back to the queue, I guess the headers are kept with the message in the quarantine with postfix. With exim they seem to be split between the database and the message file.

I first put the message ID's in to a file "idlist.txt" that had been put in to the quarantine with the "Other Bad Content Detected" error (every single email after a certain time on that day), then pulled the header from the db and combined them with the following simple loop;

-------
#!/bin/bash
for msgid in `cat idlist.txt`;
do
    /usr/bin/mysql -u root --password=XXXXX -N -e "select headers from maillog where id='$msgid' limit 1 \G;" mailscanner | grep -v "* 1. row *" >> with-headers/$msgid &&
    /bin/cat 20150724/$msgid/message >> with-headers/$msgid
done
-------

now I'm sending them out slowly (every 30 secs) with another simple loop...

-------
#!/bin/bash
for msgs in with-headers/*;
do
    cat $msgs | exim -ti
    mv $msgs with-headers-processed/
    sleep 30
done
-------

So at least the missing mail is now going to users.. but I'm no closer to knowing exactly why this happened in the first place. Jeremy mentioned a known "taint" issue? Can anyone elaborate on that?

I've also found now that Archive is enabled, and is set to "Archive Mail = " which I guess just defaults to the quarantine dir, as they seem to go to the "nonspam" folder in there (interestingly in a exim usable format!!) That couldn't have anything to do with the loop that appears to have killed my mailcleaner DB? I wouldn't think so as this has been running for years like this and not had this issue before but thought it worth mentioning.

Any other theories or places to check for clues? unfortunately the mail.log of the day got removed by the first person looking at the issue to try to free up space as it was over 4GB.

Regards,
Mark

On 28 July 2015 at 17:07, Mark Adams > wrote:
Hi Jerry,

If you wanted to pull a bunch of items from the quarantine from the command line and re-process them through Mailcleaner, how would you do that?

Regards,
Mark

On 28 July 2015 at 17:00, Jerry Benton <jerry.benton at mailborder.com<mailto:jerry.benton at mailborder.com>> wrote:
I am not sure on what parameters Mailwatch calls and logs “other bad content”. The MailScanner setting is "Notify Senders of Other Blocked Content”. Mailwatch could be calling a trigger of a spam RBL “other blocked content” for all we know. You are going to have to follow the below suggestion and enable debug or see if you can get an idea from /var/log/maillog.

-
Jerry Benton
www.mailborder.com<http://www.mailborder.com>

On Jul 28, 2015, at 11:49 AM, Mark Adams > wrote:

Of course, apologies - I'm using Mailwatch. Any advice on how to most efficiently pull things out of quarantine via command-line? (note they are stored as "message" rather than queue items, that would be too easy..)

I don't have Archive enabled, everything has gone in to the quarantine because of this "Other Bad Content Detected"

On 28 July 2015 at 16:43, Jerry Benton <jerry.benton at mailborder.com<mailto:jerry.benton at mailborder.com>> wrote:
By the way, there is no web interface in the MailScanner package. There are 3rd party products of course (I created one myself) but those questions would need to be directed to those support forums or mailing lists.

-
Jerry Benton
www.mailborder.com<http://www.mailborder.com/>

On Jul 28, 2015, at 11:34 AM, Mark Adams > wrote:

How do I try send them through again? At the moment they are just "message" in the quarantine, and if I try open them through the web interface it times out, I guess because its trying to open each one of the dupes?

"Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/html/mailscanner/functions.php on line 1022"

On 28 July 2015 at 16:31, Jeremy McSpadden <jeremy at fluxlabs.net<mailto:jeremy at fluxlabs.net>> wrote:
Yup. Turn on debug and watch it pass through. Last time I saw these it was a taint issue .. Which I am assuming has been fixed by now.

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 28, 2015, at 10:20 AM, Mark Adams > wrote:

Hi Jeremy,

Are you saying that something in these messages is crashing Mailscanner? Everything seems to be OK right now, but all 70 of the emails (all different types and from different servers) are now in the quarantine because of "Other Bad Content Detected" with the report "MailScanner: Message attempted to kill MailScanner". It seems it succeeded...

On 28 July 2015 at 15:59, Jeremy McSpadden <jeremy at fluxlabs.net<mailto:jeremy at fluxlabs.net>> wrote:
It's probably looping/crashing mailscanner. Drop MS into debug mode and watch logs.

--
Jeremy McSpadden | Flux Labs
Local - 850-250-5590x501<tel:850-250-5590;501> | Mobile - 850-890-2543<tel:850-890-2543>
Fax - 850-254-2955<tel:850-254-2955> | Toll Free - 877-699-FLUX<tel:877-699-FLUX>
Web - http://www.fluxlabs.net<http://www.fluxlabs.net/>

On Jul 28, 2015, at 9:54 AM, Mark Adams > wrote:

An update to this, the "2 or 4" duplicates showing in the exim log look like they are actually just separate deliveries to other addresses, so not duplicates. In 1 example there is a single email with 2 recipients (2 entries in exim log) that has over 1500+ entries in the mailcleaner DB. It looks like this email hasn't been delivered to the recipient at all either.

On 28 July 2015 at 15:14, Mark Adams > wrote:
Hi All,

If anyone could provide advice that would be great. Running Debian Wheezy Mailscanner 4.79.11-2.2

Our incoming dir filled up just before the weekend so we didn't see the issue for a couple of days. Normally we would just shut down mailcleaner and delete the dir then start it up again and all would be ok. However on this occasion, the root partition also become full because of the mysql DB (it got to 14G in 2 days..).

For some reason everything started duplicating. I can see lots of incoming messages in the exim logs with duplication (2 or 4 of what looks like the same email) but in the mailscanner database there is hundreds of each email listed (apparently there was over 9 million messages delivered on 1 day compared with the server average of about 1500!)

It seems like some sort of loop, but afaik nothing specific was changed in the config apart from the fact incoming became full. Space has been cleared on the root partition and incoming, and everything appears to be running as normal right now.

Any advice on debugging this would be much appreciated, also, how best should I clear out the DB of all the dupes?

Thanks!

--
MailScanner mailing list
mailscanner at lists.mailscanner.info<mailto:mailscanner at lists.mailscanner.info>
http://lists.mailscanner.info/listinfo/mailscanner

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mailscanner.info/pipermail/mailscanner/attachments/20150729/9e0ca257/attachment-0001.html>