New service - the Team Cymru Malware Hash Registry!

[Ken A]

Steve Freegard wrote:
> --[ UxBoD ]-- wrote:
>> Steve, understood, but if a local persistant cache was generated then 
>> the number of upstream look ups for the same hash would reduce ? would 
>> this not also reduce the load on the upstream servers ? sorry if I am 
>> being stupid but it kinda makes sense to me :( isn't that what AV sigs 
>> are for ?
> 
> All the records from the Malware Hash Registry have a TTL of 86400 
> seconds (24 hours), so that means that if you're looking up the same 
> hash within 24 hours - it will come from your local cache provided it 
> hasn't been purged to reclaim space.
> 
> If you maintain a local cache - you really don't save a lot of lookups 
> to the upstream since the vast majority of lookups are going to be 
> negative lookups (e.g. NXDOMAIN).
> 
> DNS was designed with caching in mind; and it works just fine for the 
> purposes it was designed for - adding a second cache is almost always a 
> bad idea and will introduce lag and incorrect results along with space 
> bloat.
> 
> If you're going to argue a local cache for these hashes - why not argue 
> for local caching for DNS BL or URI BL lookups as well?  The reason we 
> don't do local caching for these is exactly the same as why we shouldn't 
> do it here - the data is fluid - there's no guarantee that a positive or 
> negative lookup now yield the same result the next time you look at the 
> data.

Also consider DNS caching of NXDOMAIN responses. These are for much less 
time, but a local cache does significantly decrease DNS traffic 
upstream, but this depends a lot on your spam. Not only BL data is 
fluid, so are spam patterns. One approach may work better on most days 
than the other. I've not done that research. Just my two cents. :-)

Ken

> 
> Regards,
> Steve.


-- 
Ken Anderson
Pacific.Net