From gpapamichelakis at gmail.com Mon Sep 9 07:08:13 2019 From: gpapamichelakis at gmail.com (George Papamichelakis) Date: Mon, 9 Sep 2019 10:08:13 +0300 Subject: Subject when releasing message Message-ID: Hi list, when releasing a message from quarantine the original? message gets send as an attachment with the subject Message released from Quarantine. Is it possible to change this behavior and keep the original subject somehow ? (or add eg FW: original subject) ? Users gets confused and they sometimes don't understand that this message contains? the original message in it Thanks GP From gpapamichelakis at gmail.com Tue Sep 10 07:24:03 2019 From: gpapamichelakis at gmail.com (George Papamichelakis) Date: Tue, 10 Sep 2019 10:24:03 +0300 Subject: Subject when releasing message In-Reply-To: References: Message-ID: <5926b8a5-874d-5f70-a27f-3f3086618bc8@gmail.com> Sorry for wrong list posting , I just realized that this is a mailwatch problem !! Thanks GP On 9/9/19 10:08 AM, George Papamichelakis wrote: > Hi list, > > when releasing a message from quarantine the original? message gets > send as an attachment > > with the subject Message released from Quarantine. Is it possible to > change this behavior and > > keep the original subject somehow ? (or add eg FW: original subject) ? > > Users gets confused and they sometimes don't understand that this > message contains? the original > > message in it > > > Thanks > > GP > From th3penguinwhisperer at gmail.com Sun Sep 22 21:05:19 2019 From: th3penguinwhisperer at gmail.com (PenguinWhispererThe .) Date: Sun, 22 Sep 2019 23:05:19 +0200 Subject: Found x messages in the Processing Attempts Database Message-ID: Hi all, After an update of my mailserver I had a permission issue. I've corrected it but while troubleshooting I also noticed this message: "Found 20 messages in the Processing Attempts Database". Googling this doesn't bring any useful results. (matching messages but the counter is at 0 so not what I'm looking for) My mail seems to be delivered again however I was wondering if this is a problem. Where can I find these messages? Are these generally "suspicious" emails (trojan/...)? I've also seen a message about a mail that made mailscanner crash to much. I've deleted that message after short investigation. But that was only one message that was being mentioned. What's would be the best way to investigate these mails? Where can I find them and ideally see a reason why these are in the processing db? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at msapiro.net Sun Sep 22 22:17:41 2019 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 22 Sep 2019 15:17:41 -0700 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: Message-ID: On 9/22/19 2:05 PM, PenguinWhispererThe . wrote: > Hi all, > > After an update of my mailserver I had a permission issue. > I've corrected it but while troubleshooting I also noticed this message: > "Found 20 messages in the Processing Attempts Database". ...> What's would be the best way to investigate these mails? Where can I > find them and ideally see a reason why these are in the processing db? Start with the command MailScanner --processing The database location is configured in MailScanner as Processing Attempts Database (default /var/spool/MailScanner/incoming/Processing.db). It's a SQLite database, you can also examine it with sqlite3. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From maxsec at gmail.com Mon Sep 23 07:56:07 2019 From: maxsec at gmail.com (Martin Hepworth) Date: Mon, 23 Sep 2019 08:56:07 +0100 Subject: Julian Field Message-ID: For those who've been around the Mailscanner project a while, you'll know Jules as the originator of MailScanner. He's been in hospital for the past couple of weeks and I was able to visit him yesterday afternoon. Basically he's got "non-alcohol-related liver scarring from all its previous stress over 30 years of medical stuff". Again many of the longer timers here will know of Jule's struggles with his health. If anyone feels they'd like to send a card of encouragement, please send to his work address and folks will take them down to hospital or home depending where he is. Jules Field Electronics and Computer Science University of Southampton Southampton SO17 1BJ United Kingdom -- Martin Hepworth, CISSP Oxford, UK -------------- next part -------------- An HTML attachment was scrubbed... URL: From th3penguinwhisperer at gmail.com Mon Sep 23 14:25:56 2019 From: th3penguinwhisperer at gmail.com (PenguinWhispererThe .) Date: Mon, 23 Sep 2019 16:25:56 +0200 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: Message-ID: Thanks for the response Mark. Greatly appreciated. I've used the MailScanner --processing command and what's odd is that the amount of messages in the output does not add up to the number mentioned in the log. Currently being processed: Number of messages: 3 Tries Message Next Try At ===== ======= =========== 2 t6R14i8f014786 Mon Jul 27 03:14:54 2015 2 t5717xY9040021 Sun Jun 7 03:16:17 2015 2 t3M128WX002854 Wed Apr 22 03:09:29 2015 Archive: Number of messages: 7 Tries Message Last Tried ===== ======= ========== 6 707E8385511.A0119 Fri Sep 20 15:56:03 2019 6 E3B8438551E.A2CAC Fri Sep 20 14:31:53 2019 6 t54196qc031526 Thu Jun 4 03:36:12 2015 6 t4F1B25j048321 Fri May 15 03:37:05 2015 6 t4816L1V017190 Fri May 8 03:31:16 2015 6 t4518k5s060942 Tue May 5 03:35:53 2015 6 t3U15eUr056464 Thu Apr 30 03:33:55 2015 It seems most of these messages are old but 2 of them are recent. What's the difference here between the files messages starting with 't' and those without it? The messages from 2015 will probably be irrelevant. But I might want to look into the onces from a few days ago. Thanks in advance. On Mon, 23 Sep 2019 at 00:17, Mark Sapiro wrote: > On 9/22/19 2:05 PM, PenguinWhispererThe . wrote: > > Hi all, > > > > After an update of my mailserver I had a permission issue. > > I've corrected it but while troubleshooting I also noticed this message: > > "Found 20 messages in the Processing Attempts Database". > ...> What's would be the best way to investigate these mails? Where can I > > find them and ideally see a reason why these are in the processing db? > > > Start with the command > > MailScanner --processing > > The database location is configured in MailScanner as Processing > Attempts Database (default > /var/spool/MailScanner/incoming/Processing.db). It's a SQLite database, > you can also examine it with sqlite3. > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > > -- > MailScanner mailing list > mailscanner at lists.mailscanner.info > http://lists.mailscanner.info/mailman/listinfo/mailscanner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From th3penguinwhisperer at gmail.com Mon Sep 23 14:30:27 2019 From: th3penguinwhisperer at gmail.com (PenguinWhispererThe .) Date: Mon, 23 Sep 2019 16:30:27 +0200 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: Message-ID: I forgot to mention: The log message mentions 29 messages are in the processing attempt database. On Mon, 23 Sep 2019 at 16:25, PenguinWhispererThe . < th3penguinwhisperer at gmail.com> wrote: > Thanks for the response Mark. Greatly appreciated. > > I've used the MailScanner --processing command and what's odd is that the > amount of messages in the output does not add up to the number mentioned in > the log. > > Currently being processed: > > Number of messages: 3 > Tries Message Next Try At > ===== ======= =========== > 2 t6R14i8f014786 Mon Jul 27 03:14:54 2015 > 2 t5717xY9040021 Sun Jun 7 03:16:17 2015 > 2 t3M128WX002854 Wed Apr 22 03:09:29 2015 > > > Archive: > > Number of messages: 7 > Tries Message Last Tried > ===== ======= ========== > 6 707E8385511.A0119 Fri Sep 20 15:56:03 2019 > 6 E3B8438551E.A2CAC Fri Sep 20 14:31:53 2019 > 6 t54196qc031526 Thu Jun 4 03:36:12 2015 > 6 t4F1B25j048321 Fri May 15 03:37:05 2015 > 6 t4816L1V017190 Fri May 8 03:31:16 2015 > 6 t4518k5s060942 Tue May 5 03:35:53 2015 > 6 t3U15eUr056464 Thu Apr 30 03:33:55 2015 > > > It seems most of these messages are old but 2 of them are recent. What's > the difference here between the files messages starting with 't' and those > without it? > The messages from 2015 will probably be irrelevant. But I might want to > look into the onces from a few days ago. > > Thanks in advance. > > > On Mon, 23 Sep 2019 at 00:17, Mark Sapiro wrote: > >> On 9/22/19 2:05 PM, PenguinWhispererThe . wrote: >> > Hi all, >> > >> > After an update of my mailserver I had a permission issue. >> > I've corrected it but while troubleshooting I also noticed this message: >> > "Found 20 messages in the Processing Attempts Database". >> ...> What's would be the best way to investigate these mails? Where can I >> > find them and ideally see a reason why these are in the processing db? >> >> >> Start with the command >> >> MailScanner --processing >> >> The database location is configured in MailScanner as Processing >> Attempts Database (default >> /var/spool/MailScanner/incoming/Processing.db). It's a SQLite database, >> you can also examine it with sqlite3. >> >> -- >> Mark Sapiro The highway is for gamblers, >> San Francisco Bay Area, California better use your sense - B. Dylan >> >> >> -- >> MailScanner mailing list >> mailscanner at lists.mailscanner.info >> http://lists.mailscanner.info/mailman/listinfo/mailscanner >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From th3penguinwhisperer at gmail.com Wed Sep 25 18:44:32 2019 From: th3penguinwhisperer at gmail.com (PenguinWhispererThe .) Date: Wed, 25 Sep 2019 20:44:32 +0200 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: Message-ID: Sorry to bother. Mark, do you have perhaps other ideas or a response to what I've sent earlier? Anyone else can chime in on this? Or should I dive into the sqlite db or in the code to find out what's going on? I'm not really interested in the old messages from 2015 but I wonder what messages are causing the discrepancy in the logs. Thanks! On Mon, 23 Sep 2019 at 16:30, PenguinWhispererThe . < th3penguinwhisperer at gmail.com> wrote: > I forgot to mention: > > The log message mentions 29 messages are in the processing attempt > database. > > On Mon, 23 Sep 2019 at 16:25, PenguinWhispererThe . < > th3penguinwhisperer at gmail.com> wrote: > >> Thanks for the response Mark. Greatly appreciated. >> >> I've used the MailScanner --processing command and what's odd is that the >> amount of messages in the output does not add up to the number mentioned in >> the log. >> >> Currently being processed: >> >> Number of messages: 3 >> Tries Message Next Try At >> ===== ======= =========== >> 2 t6R14i8f014786 Mon Jul 27 03:14:54 2015 >> 2 t5717xY9040021 Sun Jun 7 03:16:17 2015 >> 2 t3M128WX002854 Wed Apr 22 03:09:29 2015 >> >> >> Archive: >> >> Number of messages: 7 >> Tries Message Last Tried >> ===== ======= ========== >> 6 707E8385511.A0119 Fri Sep 20 15:56:03 2019 >> 6 E3B8438551E.A2CAC Fri Sep 20 14:31:53 2019 >> 6 t54196qc031526 Thu Jun 4 03:36:12 2015 >> 6 t4F1B25j048321 Fri May 15 03:37:05 2015 >> 6 t4816L1V017190 Fri May 8 03:31:16 2015 >> 6 t4518k5s060942 Tue May 5 03:35:53 2015 >> 6 t3U15eUr056464 Thu Apr 30 03:33:55 2015 >> >> >> It seems most of these messages are old but 2 of them are recent. What's >> the difference here between the files messages starting with 't' and those >> without it? >> The messages from 2015 will probably be irrelevant. But I might want to >> look into the onces from a few days ago. >> >> Thanks in advance. >> >> >> On Mon, 23 Sep 2019 at 00:17, Mark Sapiro wrote: >> >>> On 9/22/19 2:05 PM, PenguinWhispererThe . wrote: >>> > Hi all, >>> > >>> > After an update of my mailserver I had a permission issue. >>> > I've corrected it but while troubleshooting I also noticed this >>> message: >>> > "Found 20 messages in the Processing Attempts Database". >>> ...> What's would be the best way to investigate these mails? Where can I >>> > find them and ideally see a reason why these are in the processing db? >>> >>> >>> Start with the command >>> >>> MailScanner --processing >>> >>> The database location is configured in MailScanner as Processing >>> Attempts Database (default >>> /var/spool/MailScanner/incoming/Processing.db). It's a SQLite database, >>> you can also examine it with sqlite3. >>> >>> -- >>> Mark Sapiro The highway is for gamblers, >>> San Francisco Bay Area, California better use your sense - B. Dylan >>> >>> >>> -- >>> MailScanner mailing list >>> mailscanner at lists.mailscanner.info >>> http://lists.mailscanner.info/mailman/listinfo/mailscanner >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at msapiro.net Wed Sep 25 21:19:00 2019 From: mark at msapiro.net (Mark Sapiro) Date: Wed, 25 Sep 2019 14:19:00 -0700 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: Message-ID: <0b9ab8da-b3e2-c2e6-0806-a068ad6a9044@msapiro.net> On 9/25/19 11:44 AM, PenguinWhispererThe . wrote: > > I've used the MailScanner --processing command and what's odd is > that the amount of messages in the output does not add up to the > number mentioned in the log. The number of messages in the processing database is dynamic. Messages are continually added and removed as they are processed. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From th3penguinwhisperer at gmail.com Thu Sep 26 19:45:42 2019 From: th3penguinwhisperer at gmail.com (PenguinWhispererThe .) Date: Thu, 26 Sep 2019 21:45:42 +0200 Subject: Found x messages in the Processing Attempts Database In-Reply-To: <0b9ab8da-b3e2-c2e6-0806-a068ad6a9044@msapiro.net> References: <0b9ab8da-b3e2-c2e6-0806-a068ad6a9044@msapiro.net> Message-ID: What's odd, although you mention this table is dynamic, I see the same entries being in the processing table for days now (even after restarting mailscanner again). sqlite> select * from processing; sAG1Zwmx014496|1|1416101917 sAI21lCL048525|1|1416276373 sAKESsEI061854|1|1416493861 sAL8iGJv065844|1|1416559666 sAP200rp009088|1|1416881004 sAP3FvgT009543|1|1416885645 sB57KW1j077564|1|1417764217 sB8BHQpJ098939|1|1418037788 sBC23AJO030564|1|1418350119 sBH23A1i063616|1|1418781917 sBI200Sg068899|1|1418868275 t2J22LZI009097|1|1426730686 t2S331Uu068277|1|1427512130 t3H12Hb5011899|1|1429232748 t3M128WX002854|2|1429664969 t3R126rV097720|1|1430096883 t4Q4qEWM099846|1|1432616194 t4S16p8x092225|1|1432775494 t5319C2P088537|1|1433293928 t5619Hcj004247|1|1433553112 t5717xY9040021|2|1433639777 t5B4hNKH095310|1|1433997939 t5I6eSQR022704|1|1434609981 t5M1HEZL012536|1|1434936137 t6J1P7LT079641|1|1437269327 t6K1Hep2095660|1|1437355245 t6R14i8f014786|2|1437959694 t843arsW033210|1|1441338030 t8614UXF088483|1|1441501659 Since I first mailed to this mailing list there's still 29 messages in the processing database. When I try to find files that correspond with the first column I find none (perhaps I'm searching for them in the wrong way). Not sure what the difference is between the ones starting with s and t. Is it possible some of these messages are still in the database but no longer exist and so never get removed from it? The last column doesn't seem to change. The column is called 'nexttime' from the schema. Not clear what it actually represent (doesn't seem to be seconds since EPOCH). CREATE TABLE processing (id TEXT, count INT, nexttime INT); On Wed, 25 Sep 2019 at 23:19, Mark Sapiro wrote: > On 9/25/19 11:44 AM, PenguinWhispererThe . wrote: > > > > I've used the MailScanner --processing command and what's odd is > > that the amount of messages in the output does not add up to the > > number mentioned in the log. > > > The number of messages in the processing database is dynamic. Messages > are continually added and removed as they are processed. > > > -- > Mark Sapiro The highway is for gamblers, > San Francisco Bay Area, California better use your sense - B. Dylan > > > -- > MailScanner mailing list > mailscanner at lists.mailscanner.info > http://lists.mailscanner.info/mailman/listinfo/mailscanner > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at msapiro.net Thu Sep 26 21:11:44 2019 From: mark at msapiro.net (Mark Sapiro) Date: Thu, 26 Sep 2019 14:11:44 -0700 Subject: Found x messages in the Processing Attempts Database In-Reply-To: References: <0b9ab8da-b3e2-c2e6-0806-a068ad6a9044@msapiro.net> Message-ID: <4813a613-ab9b-488a-237e-09b5f7c256da@msapiro.net> On 9/26/19 12:45 PM, PenguinWhispererThe . wrote: > What's odd, although you mention this table is dynamic, I see the same > entries being in the processing table for days now (even after > restarting mailscanner again). > > sqlite> select * from processing; > sAG1Zwmx014496|1|1416101917 > sAI21lCL048525|1|1416276373 > sAKESsEI061854|1|1416493861 > sAL8iGJv065844|1|1416559666 > sAP200rp009088|1|1416881004 > sAP3FvgT009543|1|1416885645 > sB57KW1j077564|1|1417764217 > sB8BHQpJ098939|1|1418037788 > sBC23AJO030564|1|1418350119 > sBH23A1i063616|1|1418781917 > sBI200Sg068899|1|1418868275 > t2J22LZI009097|1|1426730686 > t2S331Uu068277|1|1427512130 > t3H12Hb5011899|1|1429232748 > t3M128WX002854|2|1429664969 > t3R126rV097720|1|1430096883 > t4Q4qEWM099846|1|1432616194 > t4S16p8x092225|1|1432775494 > t5319C2P088537|1|1433293928 > t5619Hcj004247|1|1433553112 > t5717xY9040021|2|1433639777 > t5B4hNKH095310|1|1433997939 > t5I6eSQR022704|1|1434609981 > t5M1HEZL012536|1|1434936137 > t6J1P7LT079641|1|1437269327 > t6K1Hep2095660|1|1437355245 > t6R14i8f014786|2|1437959694 > t843arsW033210|1|1441338030 > t8614UXF088483|1|1441501659 > > Since I first mailed to this mailing list there's still 29 messages in > the processing database. Those are all ancient. The three columns are respectively the MTA queue ID, the number of tries and a timestamp. The timestamp is seconds since the epoch and is the time that MailScanner will next retry the message. As I said, they are ancient - the most recent one is Sat Sep 5 18:07:39 2015. When messages are retried, the the number of tries is incremented, but when it reaches the configured Maximum Processing Attempts (default 6) the message won't be further retrieed. > When I try to find files that correspond with the first column I find > none (perhaps I'm searching for them in the wrong way). Not sure what > the difference is between the ones starting with s and t. Is it possible > some of these messages are still in the database but no longer exist and > so never get removed from it? As I said, the first column is an MTA queue id with the addition of the s or t prefix (I don't know what that means), but these entries are all so old, that the messages are probably no longer queued. I suggest you just remove the Processing Attempts Database. As it says in MailScanner.conf > # This is the location of the database file used to track the number of > # times any message has been attempted. > # To clear out the database, just delete the file, MailScanner will re- > # create it automatically when it starts. > Processing Attempts Database = /var/spool/MailScanner/incoming/Processing.db This will clear these old entries and then you can focus on any new ones. -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From ricky.boone at gmail.com Fri Sep 27 19:23:48 2019 From: ricky.boone at gmail.com (Ricky Boone) Date: Fri, 27 Sep 2019 15:23:48 -0400 Subject: HTML base tags used for phishing, spam, etc. Message-ID: Before creating an issue in the Github project, I thought I'd start here to see what others thought. I'm seeing a number of reported phishing and spam messages come to me where the bad actor is utilizing the HTML base tag. I can see where there may be some legitimate use cases for the base tag in an email, however this seems to be a way for a bad actor to obfuscate their links, preventing automatic analysis from considering the full URL, and only seeing the base tag and "relative" URLs separately. I don't know if outright blocking the base tag is the right approach, but I see that there have been other discussions in the past on other sites about the topic. For example: https://www.avanan.com/resources/basestriker-vulnerability-office-365 So my thought is, perhaps an option could be added to block or rewrite the href value of the base tag, and have a whitelist of URLs that would be ignored. Another thought would be to somehow combine the base href and subsequent relative hrefs together when being evaluated by MailScannner and SpamAssassin, but that seems a bit more cumbersome. Thoughts? -------------- next part -------------- An HTML attachment was scrubbed... URL: