Phishing Server Change

[Kevin Miller]

Mine were in cron.daily, so that's good.  I replaced them with the new script so they should now use compressed curl.

I was thinking on ways to lighten the load.  Perhaps you could auto-publish a TXT record in DNS with a serial number in it that indicated the version of phishing conf files.  Something like 
  PhishingVersion IN TXT "20170803-1037"
to indicate the publishing date/time of the current file.

The download script could do an lookup on the DNS record, write it to a file, and the next time the script runs, compare the historical file to the current DNS result.  Or it could be imbedded as a comment in the conf file, then parsed and compared to the DNS lookup.  If the local file is up to date, then bail.  If not, download.  Either way, a DNS lookup is much lighter weight than a full download.

Just a thought.

Your work is appreciated...

...Kevin
--
Kevin Miller
Network/email Administrator, CBJ MIS Dept.
155 South Seward Street
Juneau, Alaska 99801
Phone: (907) 586-0242, Fax: (907) 586-4588 Registered Linux User No: 307357


-----Original Message-----
From: MailScanner [mailto:mailscanner-bounces+kevin.miller=juneau.org at lists.mailscanner.info] On Behalf Of Jerry Benton
Sent: Wednesday, August 02, 2017 9:03 PM
To: 'MailScanner Discussion'
Subject: Phishing Server Change

Please use the updated version of the phishing updates script from http://phishing.mailscanner.info/

I am trying to reduce bandwidth and the new script uses curl as the primary method, which support gzip by default. The fallback is wget, which uses more bandwidth. Most servers are still using the older update script version that uses wget as the primary method. (You cannot just tell wget to use gzip because the downloaded file ends up staying compressed and thus useless.) 

The phishing server does about 550GB per month in transfers. Using wget the bad phishing file transfer size is 280081 and with curl it is 119027. That is almost half the transfer size. I would like to try to get that 550GB number down. I don't want to have to pull out the big stick and start blocking wget user agents, so please help me out here. 

As a side note, there is no point in updating the phishing files every hour.
They get updated six times per day. If you are running every hour, please change your cron settings to something less taxing. Again, I don't want to pull out the big stick and start throttling IPs that are updating too often.


--
Jerry Benton
www.mailborder.com
+1   (843) 800-8605
+44 (020) 3883-8605





--
MailScanner mailing list
mailscanner at lists.mailscanner.info
http://lists.mailscanner.info/mailman/listinfo/mailscanner