Maintaining a Publicly Available Blacklist - Mechanisms and Principles 89
badger.foo writes "When you publicly assert that somebody sent spam, you need to ensure that your data is accurate. Your process needs to be simple and verifiable, and to compensate for any errors, you want your process to be transparent to the public with clear points of contact and line of responsibility. Here are some pointers from the operator of the bsdly.net greytrap-based blacklist."
Blacklists are evilu even for spam filtering (Score:0, Insightful)
You end up losing mail and who is it for someone else to filter what I can and can't see. There is a delete button for a reason. Use it.
Re:Greylist instead (Score:5, Insightful)
If you ran an open relay you were on the right end of a blacklisting.
Re:Greylist instead (Score:2, Insightful)
... and all mails you get will be delayed by an hour or more, pretty unacceptable when you get an urgent complaint that something is down. And even in not work-related matters, making people wait for no reason is rude.
There are many spam fighting techniques without such flaws. And other than gmail, server admins are generally smart enough to handle failures properly (ie, with instant notification that something went wrong).
Re:Greylist instead (Score:4, Insightful)
... and all mails you get will be delayed by an hour or more, pretty unacceptable when you get an urgent complaint that something is down. And even in not work-related matters, making people wait for no reason is rude.
Simple solution: Use a whitelist first. If the email is from some on your family/friend/co-worker/customer list, or someone you have corresponded with in the past, then you see it immediately. Anyone else can wait.
Re:Using a blacklist ... (Score:4, Insightful)
I don't disagree with your premise. I work in a health based organization and the SPAM and "dirty word" lexicons block legit e-mails. I've also found that for receiving e-mails SPF and most other common sense checks block too much legit mail. God forbid businesses configure their hosts / gateways correctly. And don't get me started on third party mailer services. It makes an impossible job more impossibler.
Re:Greylist instead (Score:5, Insightful)
If it is so urgent, pick up the phone.
Not realistically achievable (Score:4, Insightful)
. Your process needs to be simple and verifiable,
The process can't be simple because spammers are endlessly creative with how they try to get past the filters. And if it was verifiable, that would mean published -- and once published, becomes useless. Spammers can simply test their latest creation against your filter, and now you effectively have given them a way to bypass your entire process, making it worthless.
and to compensate for any errors, you want your process to be transparent to the public
The administrative process can be transparent, but the technical process, as outlined above, cannot.
with clear points of contact and line of responsibility.
The problem here is; how do you tell the liars from the rest? Responsibility is fine, clear points of contact are fine, but what's the criterion for delineating between 'spam' and 'marketing'? How about between 'spam' and 'opt-in' that the user no longer wants? How about between... you get the idea. There is some grey here, and odds are good you're going to find someone doing something with a legitimate and ethical reason, that by all appearances... isn't. And then you're going to make a decision based on those appearances (because what else can you go on?) and then you're going to burn a bridge down.
These problems can't be solved with a handwave and a post on an internet forum.
wrong tech. (Score:4, Insightful)
Better solution: Stop trying to force email to be a reliable and concurrent source of information. It has never been reliable nor has it ever been concurrent protocol. Check the default settings for sending email - try every hour for up to 5 days before giving up. Wait one day before sending a trouble report.
That email now generally DOES deliver results in almost real time is no excuse to think it will ALWAYS deliver in real time. If your communication either critical and/or time sensitive, then email is the wrong tool to use.
Ever heard of "FastFlux" botnet design? (Score:1, Insightful)
Botnets generally don't use IP addresses, but host-domain names instead: Why? For the purposes of "fastflux" botnet construction
So - what's that? Well, put it THIS way:
The "infamous they" (law enforcement or other authories online etc.) take 1 out?
Well, no big deal!
Just "jump" to another node on your botnet in some 'enslaved' system(s) you have in it! This is done @ the botnet C&C (command & control) server master level.
(Which of course, your botnet's infestors on clientrigs in it also has the ability to 'serve up' your bogus 'site(s)' from it & ANY ONE OF THEM...).
* Doing it THAT way's is a LOT tougher to "take out" than hardcoded IP addresses is why...
(Which as you yourself noted, are fairly EASY to blacklist out, & from a LOT higher levels than ISP's even)...
You MAY want to read more, here -> http://en.wikipedia.org/wiki/Fast_flux [wikipedia.org]
I've been building my list since 1997, & see what gets used MOSTLY from 15 or so reputable sources for my data (and the rest comes from security articles from sources such as threatpost or sophos, among others).
Now - THAT bugged me to NO end, as to WHY they used host-domain names instead of IP addresses mostly, but once I got wind of that about a decade++ ago? It made sense...
APK
P.S.=> That answer anything for you? I hope so... & it's also WHY I use what I wrote here -> http://yro.slashdot.org/comments.pl?sid=3647643&cid=43447983 [slashdot.org] in custom hosts files (which work against bogus adbanners, maliciously coded sites/servers, or hosts-domains serving up the same or malwares even, and yes, spammers/phishers too)...
... apk
milter-greylist (Score:2, Insightful)