<p>Short prose, since from phone:<br>
Old problem variant (gmane old ml archive for sa timeouts, bayes expiry etc). I set that t/o even higher... It's there in ms to detect hangs, and is by default way to low.</p>
<p>Cheers!</p>
<div class="gmail_quote">Den 16 sep 2011 17:07 skrev "Joolee" <<a href="mailto:mailscanner@joolee.nl">mailscanner@joolee.nl</a>>:<br type="attribution">> I've found out what the problem is. It wasn't because MailScanner doesn't<br>
> run the message through Spamassassin when there is an attachment error<br>> because it actually does. (Maybe someone can still add an option to skip<br>> extra checks when a file name rule is hit for Glenn Steen that thinks of it<br>
> as "not a problem, it's a feature... And a much needed one at that!" )<br>> <br>> <br>> <br>> The problem is that Mailscanner throws a timeout when Spamassassin is run<br>> and sets the score to 0.0 Now the mail is recognized as having "Bad Content"<br>
> but because the spam score is 0, the mail gets cleaned, a warning is added<br>> and the mail is forwarded to the recipient.<br>> <br>> <br>> <br>> As for the Spamassassin timeout, I think this is caused by the headers that<br>
> identify one of the attachments in the mails.<br>> <br>> This is:<br>> <br>>><br>>><br>>> ------=_NextPart_000_0006_01CC51AC.63F30F00<br>>><br>>> Content-Type: ;<br>>><br>
>> name="report_1609.pdf.zip"<br>>><br>>> Content-Transfer-Encoding: base64<br>>><br>>> Content-Disposition: attachment;<br>>><br>>> filename="report_1609.pdf.zip"<br>
>><br>> <br>> <br>> <br>> <br>> I think that because of the empty "Content-Type" header, the attachment is<br>> decoded and used for byasian learning. This takes somewhere between 90 and<br>
> 200 seconds, exceeding the timeout configured in MailScanner (I already<br>> changed that to 150 seconds but a batch of 25 mails can now effectively stop<br>> message processing for more than an hour and some messages get through)<br>
> <br>> <br>> <br>> I've come to this conclusion because when running a manual Spamassassin scan<br>> on a message, the following lines are very time consuming:<br>> <br>>> Sep 16 15:07:12.279 [8264] dbg: bayes: Using userid: 1 0.0004<br>
>><br>>> Sep 16 15:08:48.746 [8264] dbg: bayes: seen<br>>> (bf76e190b8121487c91051758a402dd20b18eaa6@sa_generated) put<br>>> 96.46636<br>>><br>> While that only takes +-4 ms for other mails.<br>
> <br>> <br>> <br>> When I run sa-learn manually, the timeout is seen in the following lines:<br>> <br>>> Sep 16 15:34:12.786 [18308] dbg: message: decoding base64<br>>><br>>> Forgot tokens from 1 message(s) (1 message(s) examined)<br>
>><br>>> Sep 16 15:35:49.764 [18308] dbg: plugin:<br>>> Mail::SpamAssassin::Plugin::Bayes=HASH(0x3891ba0) implements<br>>> 'learner_close', priority 0<br>>><br>> <br>> <br>> I'll file a bug report for Spamassassin. In the meantime, I'll just set the<br>
> timeout to 300 seconds and keep an eye on the Mailscnner queue with collectd<br>> or disable autolearning altogether.<br>> <br>> <br>> On 2 September 2011 14:58, Rick Cooper <<a href="mailto:rcooper@dwford.com">rcooper@dwford.com</a>> wrote:<br>
> <br>>> **<br>>><br>>><br>>> ------------------------------<br>>> *From:* <a href="mailto:mailscanner-bounces@lists.mailscanner.info">mailscanner-bounces@lists.mailscanner.info</a> [mailto:<br>
>> <a href="mailto:mailscanner-bounces@lists.mailscanner.info">mailscanner-bounces@lists.mailscanner.info</a>] *On Behalf Of *Joolee<br>>> *Sent:* Friday, September 02, 2011 6:20 AM<br>>><br>>> *To:* MailScanner discussion<br>
>> *Subject:* Re: MS Doesn't completely block spam with faulty attachments<br>>><br>>> A feature that i would like to be able to disable ;)<br>>><br>>> "Why would you want to spend precious resources on a meaningless check,<br>
>> when you already decided to stop the offending attachment?!"<br>>> To inform my paying user why the contract he's been waiting for was<br>>> blocked.<br>>><br>>> I think I already made quite clear why it's not an option for me to<br>
>> completely block them. I can't see why other users can't be bothered by it,<br>>> maybe they just accept that they can't solve it? (Not my way of handling<br>>> problems)<br>>> [Rick Cooper]<br>
>><br>>><br>>> Seems like you need to modify your multiple extension rules to include<br>>> dangerous extensions and ignore the rest. for instance a rule like<br>>> /\.(exe|com|bat|vbs)\..+$/<br>
>><br>>> would allow "something.good.doc.pdf" but would catch<br>>> "something.bad.doc.exe.pdf". Of course you would want (exe|vbs|com|bat) to<br>>> include extensions that you feel should be blocked in the multiple extension<br>
>> rule. I had to change mine long ago because there are a *lot* of people who<br>>> create files names like "something.good.09.01.2011.doc". The default rules<br>>> are there for out of the box functionality but you can modify them as<br>
>> required for your given situation and clearly you need to pass multiple<br>>> extensions that are not likely to be malware. With MailScanner you can<br>>> generally solve any issues without accepting the default rules, or asking<br>
>> for something else to be added either. There has been discussion in the past<br>>> regarding being able to define the order in which the processing events take<br>>> place but this would require a HUGE change in the core of MailScanner and<br>
>> Julian does have a job that puts food on the table. Unless MailScanner<br>>> evolves into a programming team or group that is not likely to ever happen.<br>>><br>>><br>>> On 1 September 2011 23:07, Glenn Steen <<a href="mailto:glenn.steen@gmail.com">glenn.steen@gmail.com</a>> wrote:<br>
>><br>>>> That's not a problem, it's a feature... And a much needed one at that!<br>>>> Why would you want to spend precious resources on a meaningless check,<br>>>> when you already decided to stop the offending attachment?!<br>
>>> Don't deliver it at all, if it bothers you;-)<br>>>><br>>>> Cheers<br>>>> --<br>>>> -- Glenn<br>>>> Den 1 sep 2011 19:12 skrev "Joolee" <<a href="mailto:mailscanner@joolee.nl">mailscanner@joolee.nl</a>>:<br>
>>><br>>>> > The problem with the current spam is that they're blocked for containing<br>>>> exe<br>>>> > files, not double file extensions (Although they woul've hit that one if<br>
>>> > exe's were not clocked.)<br>>>> ><br>>>> > Only quick temporary solution is to disable all file-name validation<br>>>> because<br>>>> > this can occur with more than just exe files and double extensions. This<br>
>>> is<br>>>> > no final solution though.<br>>>> ><br>>>> > On 1 September 2011 18:40, Kevin Miller <<a href="mailto:Kevin_Miller@ci.juneau.ak.us">Kevin_Miller@ci.juneau.ak.us</a><br>
>>> >wrote:<br>>>> ><br>>>> >> **<br>>>> >> Easiest thing to do in that case is to comment out the line in<br>>>> >> filename.rules.conf that disallows double extensions. The message will<br>
>>> be<br>>>> >> accepted as normal and go through the additional tests (is it an<br>>>> executable,<br>>>> >> is it a virus, is it spam, etc.)<br>>>> >><br>>>> >><br>
>>> >> ...Kevin<br>>>> >> --<br>>>> >> Kevin Miller Registered Linux User No: 307357<br>>>> >> CBJ MIS Dept. Network Systems Admin., Mail Admin.<br>>>> >> 155 South Seward Street ph: (907) 586-0242<br>
>>> >> Juneau, Alaska 99801 fax: (907 586-4500<br>>>> >><br>>>> >><br>>>> >> ------------------------------<br>>>> >> *From:* <a href="mailto:mailscanner-bounces@lists.mailscanner.info">mailscanner-bounces@lists.mailscanner.info</a> [mailto:<br>
>>> >> <a href="mailto:mailscanner-bounces@lists.mailscanner.info">mailscanner-bounces@lists.mailscanner.info</a>] *On Behalf Of *Joolee<br>>>> >> *Sent:* Thursday, September 01, 2011 7:32 AM<br>
>>> >> *To:* MailScanner discussion<br>>>> >> *Subject:* Re: MS Doesn't completely block spam with faulty attachments<br>>>> >><br>>>> >> I agree that it isn't a good idea to notify the sender of a spam or<br>
>>> virus<br>>>> >> message I'm not planning to do that, I know the troubles of<br>>>> backscatter.<br>>>> >><br>>>> >> What I've configured is that if a user sends a completely normal<br>
>>> >> (non-virus, non-spam) E-mail but with, for instance, a file named<br>>>> >> "CurriculumVitae.doc.pdf" (default output for a lot of PDF printers).<br>>>> The<br>>>> >> server sends out a warning to sender and the original message stripped<br>
>>> of<br>>>> >> it's attachment to the recipient of the message. Notifying the sender<br>>>> is not<br>>>> >> strictly necessary but if this is only done for such non-virus,<br>
>>> non-spam<br>>>> >> message, it isn't a problem either.<br>>>> >><br>>>> >> The situation that bugs me is when some spam message with a file named<br>>>> >> "CurriculumVitae.doc.pdf" is received. The message hits the filename<br>
>>> rule<br>>>> >> and* isn't processed any further to check if its a spam message*.<br>>>> Because<br>>>> >> it isn't processed any further, the warning messages are send out to<br>
>>> both<br>>>> >> sender and original recipient.<br>>>> >><br>>>> >> As I stated before, I can disable the sender notification. What I can't<br>>>> do<br>
>>> >> is tell my customers (the recipients) that such wrongly named files,<br>>>> most<br>>>> >> containing important documents, are silently discarded. Sending spam to<br>>>> my<br>
>>> >> customers that could have been recognized isn't an option either.<br>>>> >><br>>>> >> The simplest solution, I think, would be to *continue processing* the<br>>>> >> message after a file name rule is hit, decide if the E-mail is HAM and<br>
>>> in<br>>>> >> that case, send out the notifications. If the E-mail is spam, silently<br>>>> >> discard it.<br>>>> >> It would add a bit of load to the server but stopping spam is what it's<br>
>>> all<br>>>> >> about, isn't it? :P<br>>>> >><br>>>> >> On 1 September 2011 16:34, Julian Field <<a href="mailto:MailScanner@ecs.soton.ac.uk">MailScanner@ecs.soton.ac.uk</a><br>
>>> >wrote:<br>>>> >><br>>>> >>> He's probably switched on some "Notify Senders" options. Bad idea :-(<br>>>> >>><br>>>> >>><br>
>>> >>> On 01/09/2011 12:32, Martin Hepworth wrote:<br>>>> >>><br>>>> >>>> what version of MS?<br>>>> >>>><br>>>> >>>> I never inform the sender of junk as you end up with fake messages<br>
>>> sent<br>>>> >>>> out.<br>>>> >>>><br>>>> >>>> --<br>>>> >>>> Martin Hepworth<br>>>> >>>> Oxford, UK<br>
>>> >>>><br>>>> >>>><br>>>> >>>> On 1 September 2011 08:17, Joolee <<a href="mailto:mailscanner@joolee.nl">mailscanner@joolee.nl</a> <mailto:<br>>>> >>>> <a href="mailto:mailscanner@joolee.nl">mailscanner@joolee.nl</a>>**> wrote:<br>
>>> >>>><br>>>> >>>> Hallo Everybody,<br>>>> >>>><br>>>> >>>> I've experienced a small flood of virus E-mails. These E-mails<br>>>> >>>> (subj.: "ACH Payment *random number* Canceled") contain<br>
>>> >>>> attachments named like: "report_082011-65.pdf.exe"<br>>>> >>>> They obviously get blocked by the "no executables" and "No double<br>>>> >>>> file extensions" rules. The problem is that after blocking them,<br>
>>> >>>> an automated E-mail is send to the original recipient and the<br>>>> >>>> (faked) sender of the message, informing them of the blocked<br>>>> >>>> attachment.<br>
>>> >>>><br>>>> >>>> Had the E-mails been processed further, they would've probably hit<br>>>> >>>> the virusscanner (not tested) or spamassassin (gives a score of 27<br>
>>> >>>> when tested) and the E-mail would've silently been discarded as a<br>>>> >>>> virus / spam / phishing.<br>>>> >>>><br>>>> >>>> Is it possible to let the MailScanner continue it's processing<br>
>>> >>>> when hitting the file name rules and / or running the filename<br>>>> >>>> rule at a later time?<br>>>> >>>> --<br>>>> >>>> MailScanner mailing list<br>
>>> >>>> mailscanner@lists.mailscanner.**info<<br>>>> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a>><br>>>> >>>> <mailto:<a href="mailto:mailscanner@lists.">mailscanner@lists.</a>**<a href="http://mailscanner.info">mailscanner.info</a><<br>
>>> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a>>><br>>>> >>>><br>>>> >>>><br>>>> >>>> <a href="http://lists.mailscanner.info/**mailman/listinfo/mailscanner">http://lists.mailscanner.info/**mailman/listinfo/mailscanner</a><<br>
>>> <a href="http://lists.mailscanner.info/mailman/listinfo/mailscanner">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a>><br>>>> >>>><br>>>> >>>> Before posting, read <a href="http://wiki.mailscanner.info/**posting">http://wiki.mailscanner.info/**posting</a><<br>
>>> <a href="http://wiki.mailscanner.info/posting">http://wiki.mailscanner.info/posting</a>><br>>>><br>>>> >>>><br>>>> >>>> Support MailScanner development - buy the book off the website!<br>
>>> >>>><br>>>> >>>><br>>>> >>>><br>>>> >>>><br>>>> >>>><br>>>> >>>> Jules<br>>>> >>>><br>
>>> >>>> --<br>>>> >>>> Julian Field MEng CITP CEng<br>>>> >>>> <a href="http://www.MailScanner.info">www.MailScanner.info</a><br>>>> >>>><br>
>>> >>>> Buy the MailScanner book at <a href="http://www.MailScanner.info/store">www.MailScanner.info/store</a><br>>>> >>>> Need help customising MailScanner? Contact me!<br>>>> >>>><br>
>>> >>>> PGP footprint: EE81 D763 3DB0 0BFD E1DC 7222 11F6 5947 1415 B654<br>>>> >>>> Follow me at <a href="http://twitter.com/JulesFM">twitter.com/JulesFM</a><br>>>> >>>><br>
>>> >>>> 'It's okay to live without all the answers' - Charlie Eppes, 2011<br>>>> >>>> 'All programs have a desire to be useful' - Tron, 1982<br>>>> >>>><br>
>>> >>><br>>>> >>> --<br>>>> >>> This message has been scanned for viruses and<br>>>> >>> dangerous content by MailScanner, and is<br>>>> >>> believed to be clean.<br>
>>> >>><br>>>> >>> --<br>>>> >>> MailScanner mailing list<br>>>> >>> mailscanner@lists.mailscanner.**info <<br>>>> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a>><br>
>>> >>> <a href="http://lists.mailscanner.info/**mailman/listinfo/mailscanner">http://lists.mailscanner.info/**mailman/listinfo/mailscanner</a><<br>>>> <a href="http://lists.mailscanner.info/mailman/listinfo/mailscanner">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a>><br>
>>> >>><br>>>> >>> Before posting, read <a href="http://wiki.mailscanner.info/**posting">http://wiki.mailscanner.info/**posting</a><<br>>>> <a href="http://wiki.mailscanner.info/posting">http://wiki.mailscanner.info/posting</a>><br>
>>><br>>>> >>><br>>>> >>> Support MailScanner development - buy the book off the website!<br>>>> >>><br>>>> >><br>>>> >><br>>>> >> --<br>
>>> >> MailScanner mailing list<br>>>> >> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a><br>>>> >> <a href="http://lists.mailscanner.info/mailman/listinfo/mailscanner">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a><br>
>>> >><br>>>> >> Before posting, read <a href="http://wiki.mailscanner.info/posting">http://wiki.mailscanner.info/posting</a><br>>>> >><br>>>> >> Support MailScanner development - buy the book off the website!<br>
>>> >><br>>>> >><br>>>><br>>>> --<br>>>> MailScanner mailing list<br>>>> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a><br>
>>> <a href="http://lists.mailscanner.info/mailman/listinfo/mailscanner">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a><br>>>><br>>>> Before posting, read <a href="http://wiki.mailscanner.info/posting">http://wiki.mailscanner.info/posting</a><br>
>>><br>>>> Support MailScanner development - buy the book off the website!<br>>>><br>>>><br>>><br>>> --<br>>> MailScanner mailing list<br>>> <a href="mailto:mailscanner@lists.mailscanner.info">mailscanner@lists.mailscanner.info</a><br>
>> <a href="http://lists.mailscanner.info/mailman/listinfo/mailscanner">http://lists.mailscanner.info/mailman/listinfo/mailscanner</a><br>>><br>>> Before posting, read <a href="http://wiki.mailscanner.info/posting">http://wiki.mailscanner.info/posting</a><br>
>><br>>> Support MailScanner development - buy the book off the website!<br>>><br>>><br></div>