Duplicated messages

Wed Jul 29 14:08:21 UTC 2015

It's not 1000 entries, its about that many (some id's with a huge amount
more) per message ID. The Mailscanner DB has over 9,000,000 entries for
that day. I was planning on looping through and deleting all the offending
entries for the 300+ emails that seemed to get stuck.

The mailscanner DB is currently 14G (maillog.MYD)

On 29 July 2015 at 15:04, Jeremy McSpadden <jeremy at fluxlabs.net> wrote:

>  1000 entries from a sql DB may be about 5mb of space.
> I would do a little more digging on the / partition to see what's eating
> space. Probably better off dropping some older archive dir
>
>  --
> Jeremy McSpadden | Flux Labs
> Local - 850-250-5590x501 <850-250-5590;501> | Mobile - 850-890-2543
> Fax - 850-254-2955 | Toll Free - 877-699-FLUX
> Web - http://www.fluxlabs.net
>
>
> On Jul 29, 2015, at 9:00 AM, Mark Adams <mark at workshopit.co.uk> wrote:
>
>   I agree with you completely, however that doesn't help my immediate
> situation. Can you provide advice on deleting from the mailscanner DB? is
> there any other tables I need to remove the offending ID's entries from?
>
> Regards,
> Mark
>
> On 29 July 2015 at 14:34, Jeremy McSpadden <jeremy at fluxlabs.net> wrote:
>
>>  It's 2015 .. That shouldn't be an excuse. It's like 10 cents per 100gb
>> of drive... Upgrade
>>
>>  --
>> Jeremy McSpadden | Flux Labs
>> Local - 850-250-5590x501 <850-250-5590;501> | Mobile - 850-890-2543
>> Fax - 850-254-2955 | Toll Free - 877-699-FLUX
>> Web - http://www.fluxlabs.net
>>
>>
>> On Jul 29, 2015, at 8:27 AM, Mark Adams <mark at workshopit.co.uk> wrote:
>>
>>   Hi Jeremy,
>>
>>  No I haven't yet - I'm short on space on my root partition still
>> because of the large mysql DB so I want to clean that up first. Can you
>> advise how best to do this? Is it safe enough to do delete from maillog
>> where id='XXX';  for all the ID's with the dupes? is there any other tables
>> that need to be cleared?
>>
>>  Regards,
>> Mark
>>
>> On 29 July 2015 at 14:16, Jeremy McSpadden <jeremy at fluxlabs.net> wrote:
>>
>>>  Log won't show taint issues. Setup log rotation.
>>>
>>>  Have you enabled debug in mailscanner config like I stated yesterday ?
>>>
>>>  --
>>> Jeremy McSpadden | Flux Labs
>>> Local - 850-250-5590x501 <850-250-5590;501> | Mobile - 850-890-2543
>>> Fax - 850-254-2955 | Toll Free - 877-699-FLUX
>>> Web - http://www.fluxlabs.net
>>>
>>>
>>> On Jul 29, 2015, at 8:13 AM, Mark Adams <mark at workshopit.co.uk> wrote:
>>>
>>>   Hi all,
>>>
>>>  So I have resolved getting the missing mails delivered from the
>>> quarantine. The main problem stopping this from being easy from the command
>>> line was the fact that "Quarantine Whole Messages As Queue Files = no" was
>>> set, whilst the MTA in use is exim. I've changed that setting to "yes"
>>> now...
>>>
>>>  I've read that if its postfix you can just send that "message" file
>>> back to the queue, I guess the headers are kept with the message in the
>>> quarantine with postfix. With exim they seem to be split between the
>>> database and the message file.
>>>
>>>  I first put the message ID's in to a file "idlist.txt" that had been
>>> put in to the quarantine with the "Other Bad Content Detected" error (every
>>> single email after a certain time on that day), then pulled the header from
>>> the db and combined them with the following simple loop;
>>>
>>>  -------
>>> #!/bin/bash
>>>  for msgid in `cat idlist.txt`;
>>> do
>>>     /usr/bin/mysql -u root --password=XXXXX -N -e "select headers from
>>> maillog where id='$msgid' limit 1 \G;" mailscanner | grep -v "* 1. row *"
>>> >> with-headers/$msgid &&
>>>     /bin/cat 20150724/$msgid/message >> with-headers/$msgid
>>> done
>>>  -------
>>>
>>>  now I'm sending them out slowly (every 30 secs) with another simple
>>> loop...
>>>
>>>  -------
>>>  #!/bin/bash
>>> for msgs in with-headers/*;
>>> do
>>>     cat $msgs | exim -ti
>>>     mv $msgs with-headers-processed/
>>>     sleep 30
>>> done
>>>  -------
>>>
>>>  So at least the missing mail is now going to users.. but I'm no closer
>>> to knowing exactly why this happened in the first place. Jeremy mentioned a
>>> known "taint" issue? Can anyone elaborate on that?
>>>
>>>  I've also found now that Archive is enabled, and is set to "Archive
>>> Mail = " which I guess just defaults to the quarantine dir, as they seem to
>>> go to the "nonspam" folder in there (interestingly in a exim usable
>>> format!!) That couldn't have anything to do with the loop that appears to
>>> have killed my mailcleaner DB? I wouldn't think so as this has been running
>>> for years like this and not had this issue before but thought it worth
>>> mentioning.
>>>
>>>  Any other theories or places to check for clues? unfortunately the
>>> mail.log of the day got removed by the first person looking at the issue to
>>> try to free up space as it was over 4GB.
>>>
>>>  Regards,
>>> Mark
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 28 July 2015 at 17:07, Mark Adams <mark at workshopit.co.uk> wrote:
>>>
>>>> Hi Jerry,
>>>>
>>>>  If you wanted to pull a bunch of items from the quarantine from the
>>>> command line and re-process them through Mailcleaner, how would you do that?
>>>>
>>>>  Regards,
>>>> Mark
>>>>
>>>> On 28 July 2015 at 17:00, Jerry Benton <jerry.benton at mailborder.com>
>>>> wrote:
>>>>
>>>>> I am not sure on what parameters Mailwatch calls and logs “other bad
>>>>> content”.  The MailScanner setting is "Notify Senders of Other Blocked
>>>>> Content”. Mailwatch could be calling a trigger of a spam RBL “other blocked
>>>>> content” for all we know. You are going to have to follow the below
>>>>> suggestion and enable debug or see if you can get an idea from
>>>>> /var/log/maillog.
>>>>>
>>>>> -
>>>>>  Jerry Benton
>>>>>  www.mailborder.com
>>>>>
>>>>>
>>>>>
>>>>>   On Jul 28, 2015, at 11:49 AM, Mark Adams <mark at workshopit.co.uk>
>>>>> wrote:
>>>>>
>>>>>  Of course, apologies - I'm using Mailwatch. Any advice on how to
>>>>> most efficiently pull things out of quarantine via command-line? (note they
>>>>> are stored as "message" rather than queue items, that would be too easy..)
>>>>>
>>>>>  I don't have Archive enabled, everything has gone in to the
>>>>> quarantine because of this "Other Bad Content Detected"
>>>>>
>>>>> On 28 July 2015 at 16:43, Jerry Benton <jerry.benton at mailborder.com>
>>>>> wrote:
>>>>>
>>>>>> By the way, there is no web interface in the MailScanner package.
>>>>>> There are 3rd party products of course (I created one myself) but those
>>>>>> questions would need to be directed to those support forums or mailing
>>>>>> lists.
>>>>>>
>>>>>> -
>>>>>>  Jerry Benton
>>>>>>  www.mailborder.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>   On Jul 28, 2015, at 11:34 AM, Mark Adams <mark at workshopit.co.uk>
>>>>>> wrote:
>>>>>>
>>>>>>  How do I try send them through again? At the moment they are just
>>>>>> "message" in the quarantine, and if I try open them through the web
>>>>>> interface it times out, I guess because its trying to open each one of the
>>>>>> dupes?
>>>>>>
>>>>>>  "Fatal error: Maximum execution time of 30 seconds exceeded in
>>>>>> /var/www/html/mailscanner/functions.php on line 1022"
>>>>>>
>>>>>> On 28 July 2015 at 16:31, Jeremy McSpadden <jeremy at fluxlabs.net>
>>>>>> wrote:
>>>>>>
>>>>>>>  Yup. Turn on debug and watch it pass through. Last time I saw
>>>>>>> these it was a taint issue .. Which I am assuming has been fixed by now.
>>>>>>>
>>>>>>>  --
>>>>>>> Jeremy McSpadden | Flux Labs
>>>>>>> Local - 850-250-5590x501 <850-250-5590;501> | Mobile - 850-890-2543
>>>>>>> Fax - 850-254-2955 | Toll Free - 877-699-FLUX
>>>>>>> Web - http://www.fluxlabs.net
>>>>>>>
>>>>>>>
>>>>>>> On Jul 28, 2015, at 10:20 AM, Mark Adams <mark at workshopit.co.uk>
>>>>>>> wrote:
>>>>>>>
>>>>>>>   Hi Jeremy,
>>>>>>>
>>>>>>>  Are you saying that something in these messages is crashing
>>>>>>> Mailscanner? Everything seems to be OK right now, but all 70 of the emails
>>>>>>> (all different types and from different servers) are now in the quarantine
>>>>>>> because of "Other Bad Content Detected" with the report "MailScanner:
>>>>>>> Message attempted to kill MailScanner". It seems it succeeded...
>>>>>>>
>>>>>>> On 28 July 2015 at 15:59, Jeremy McSpadden <jeremy at fluxlabs.net>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>  It's probably looping/crashing mailscanner. Drop MS into debug
>>>>>>>> mode and watch logs.
>>>>>>>>
>>>>>>>>  --
>>>>>>>> Jeremy McSpadden | Flux Labs
>>>>>>>> Local - 850-250-5590x501 <850-250-5590;501> | Mobile - 850-890-2543
>>>>>>>>
>>>>>>>> Fax - 850-254-2955 | Toll Free - 877-699-FLUX
>>>>>>>> Web - http://www.fluxlabs.net
>>>>>>>>
>>>>>>>>
>>>>>>>> On Jul 28, 2015, at 9:54 AM, Mark Adams <mark at workshopit.co.uk>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>   An update to this, the "2 or 4" duplicates showing in the exim
>>>>>>>> log look like they are actually just separate deliveries to other
>>>>>>>> addresses, so not duplicates. In 1 example there is a single email with 2
>>>>>>>> recipients (2 entries in exim log) that has over 1500+ entries in the
>>>>>>>> mailcleaner DB. It looks like this email hasn't been delivered to the
>>>>>>>> recipient at all either.
>>>>>>>>
>>>>>>>> On 28 July 2015 at 15:14, Mark Adams <mark at workshopit.co.uk> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>>  If anyone could provide advice that would be great. Running
>>>>>>>>> Debian Wheezy Mailscanner 4.79.11-2.2
>>>>>>>>>
>>>>>>>>>  Our incoming dir filled up just before the weekend so we didn't
>>>>>>>>> see the issue for a couple of days. Normally we would just shut down
>>>>>>>>> mailcleaner and delete the dir then start it up again and all would be ok.
>>>>>>>>> However on this occasion, the root partition also become full because of
>>>>>>>>> the mysql DB (it got to 14G in 2 days..).
>>>>>>>>>
>>>>>>>>>  For some reason everything started duplicating. I can see lots
>>>>>>>>> of incoming messages in the exim logs with duplication (2 or 4 of what
>>>>>>>>> looks like the same email) but in the mailscanner database there is
>>>>>>>>> hundreds of each email listed (apparently there was over 9 million messages
>>>>>>>>> delivered on 1 day compared with the server average of about 1500!)
>>>>>>>>>
>>>>>>>>>  It seems like some sort of loop, but afaik nothing specific was
>>>>>>>>> changed in the config apart from the fact incoming became full. Space has
>>>>>>>>> been cleared on the root partition and incoming, and everything appears to
>>>>>>>>> be running as normal right now.
>>>>>>>>>
>>>>>>>>>  Any advice on debugging this would be much appreciated, also,
>>>>>>>>> how best should I clear out the DB of all the dupes?
>>>>>>>>>
>>>>>>>>>  Thanks!
>>>>>>>>>
>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mailscanner.info/pipermail/mailscanner/attachments/20150729/d3100932/attachment.html>