Zero Errors? Spamhaus Flubs Causing Domain Deletions 170
Since 2005 I've been running a proxy mailing list where users sign up to receive new proxy sites by email. (Proxy sites are sites for getting around Internet blocking software; most proxy sites that you can find through Google are already blocked by major blocking programs, which is why you would sign up to receive new ones by email, to use them until they get blocked as well.) In all that time, we've followed what are considered best practices for email newsletters: every new subscriber is sent a confirmation message by email, and they have to reply to that message, confirming that they really want to subscribe to the emails, before being added to the list. This practice, known as "verified-opt-in," is considered the gold standard for responsible emailing, since it ensures that everyone on your list actually wants to get your emails. (It also ensures that if you accuse an email publisher of spamming because you received their unwanted emails, they can't say, "Oh, one of your friends must have added you" — since if they're using verified-opt-in like they're supposed to, your friends can't add you.) I'm front-loading a lot of information here, although if you saw the words "Spamhaus errors" in the title, you may recognize the technique of literary foreshadowing being employed.
Despite conforming to verified-opt-in standards, the proxy emails have at times been blocked by spam filters used by Hotmail, Gmail, Yahoo Mail, AOL Mail, and various other systems. However, last month was the first time that an incorrect blacklisting caused the domains themselves to be disabled, so that the sites disappeared from the Internet entirely.
On September 17th I registered 10 new .info domains through NameCheap, set up new proxy sites at each of those domains, and mailed each site to 1/10th of our proxy mailing list. (Sending new sites only to a subset of the list makes it harder for blocking software companies to join the list and find all new sites as soon as they're released.) All seemed to be going well until October 2, when subscribers started telling me that they were getting "host not found" errors when trying to reach the sites. I tried the sites myself, found that they were indeed inaccessible, and spent about an hour testing for various problems with DNS servers and domain record settings, before logging in to NameCheap and seeing a message next to each of the new domains saying "domain locked due to illegal activity; please email legal@enom.com." (NameCheap being a reseller for the domain registrar eNom.)
So I sent eNom an email and followed up with a phone call to see if they could speed things up, since complaints kept pouring in from users that the sites were unreachable. eNom said that the domains had actually been suspended by Afilias, the company that handles all .info domain registrations no matter who you buy the domain from, and eNom was in the process of talking with Afilias. So I called Afilias myself to ask about getting the domains unlocked, but they refused to talk to me and said that they could only respond to inquiries from eNom. This, of course, is ridiculous — if someone notifies you that you or your company has made a error, you can investigate the issue no matter who brings it to your attention — and especially in cases where you're literally accusing someone of unspecified "illegal activity," you should bend over backwards to respond to any indication that you might have made a mistake. But they refused to do anything, so I waited for a response back from eNom.
A day and a half ticked by, with emails continuing to come in from our users wondering why the domains had disappeared, until finally eNom forwarded me a response from Afilias saying that two of my ten domains ("drybook.info" and "rootface.info") had been blacklisted by the UK-based organization Spamhaus on their Domain Block List. Spamhaus operates several different alleged "spam" blacklists, and claims that the DBL is a list of domains found in spam messages. The DBL FAQ says that it is "built predominantly using automated spamtraps and email flow monitoring" and "has many checks to prevent legitimate domains being listed," even going so far as to call it a "zero false-positive" list.
Even though only two of the ten domains that I had registered that day had been blacklisted by Spamhaus, Afilias had responded by disabling the entire group of ten domains that I had bought at the same time.
Now here's where I caught a bit of a break: It turns out I was able to get the domains instantly removed from the DBL by entering them in a form on the Spamhaus site and clicking a button, which took me to a page saying:
DBL removal successful
The domain was successfully removed from the DBL. Please allow 30 minutes for servers around the world to update their data. Please note that the domain will be re-listed if malicious activity is detected in the future.
Although, even this easy part of the process didn't inspire much confidence. Not that I wanted Spamhaus to make it harder for me to de-list by domain names, of course, but if you really think your blacklist is 100% accurate, why would you let anyone get any domain removed at any time just by submitting it in a form? In fact, this would seem to give an advantage to spammers over regular website owners — because a spammer, who knows about blacklists and would find it worthwhile to game the system in his favor, would be more likely to know about the Spamhaus DBL and the form for getting their domains de-listed. Whereas for a regular non-spamming website owner, it would take far more time to find out that their domains had been de-activated, that the de-activation had occurred because of an incorrect Spamhaus listing, etc.
Once the listing had been removed, I emailed eNom, who emailed Afilias, who eventually re-activated the domains after a few more hours. But the traffic never returned to the levels that it had been at before the domains were deleted, as most of our users had apparently concluded that the sites had been blocked or taken offline.
Spamhaus did not respond to requests for comment on this story. In fact, Spamhaus does not give you a way to contact them if you have been wrongly blacklisted — their "contacts" page redirects you to the "Blocklist Removal Center" if your domain is blocked, but that only leads you to the automated removal tools, not a way to contact the organization. I did email their "Press Office" email address, on the grounds that I was writing an article for Slashdot in addition to being a wrongly blacklisted domain owner, but didn't get an answer.
So I have no idea what will happen with the next group of domains that I send out to our proxy list. If Spamhaus signed up one of their "spamtrap" email addresses to our mailing list, then presumably any domain mentioned in a message sent to that email, will get automatically blacklisted (even though of course since they signed up the email address to our mailing list, that means it's not spam). If that happens, the entire next batch of domains might get disabled by Afilias as well.
Meanwhile, Spamhaus continues to claim that the DBL is a "zero false-positive" list. I don't know how many other false positives are on the list or how many domains have been abruptly disabled as a result, but if it's this easy to get incorrectly blacklisted, my money is not on "zero."
Spamhaus DBL IS network abuse (Score:5, Insightful)
Spamhaus DBL is poorly run and full of spite listings and other garbage. Zero false positives? They mean zero legit entries. Spamhaus has become what it set out to oppose, and it's time they were exposed for what they are today. A disgrace to the anti-spam, anti-abuse community.
No illegal activity? (Score:5, Insightful)
He claims that no illegal activity was taking place, but if he's running proxies that are open to the public via a mailing list, doesn't it seem entirely likely that a spammer may be making use of his mailing list to get more proxies that can be used for their operations? And, if so, isn't it entirely likely that that's exactly what got him blacklisted in the first place?
What evidence is there that his proxies weren't being used by others for illegal activities? Seems like he conveniently skirted that point in his entire write up.
Its not Spamhaus, its your registrar (Score:5, Insightful)
Your registrar sucks, its nothing to do with Spamhaus.
Re:no sympathy (Score:5, Insightful)
You should consider this a wake-up call. It's time to switch from mass-email to a web page with RSS.
If people really want your newsletter, they'll come to you.
...it's a proxy list.
how long do you think those sites would stay off chinas webfilters ?
a proxy list you can't get to is rather useless.
Spamhaus and RBL = evil (Score:5, Insightful)
It was a good idea in the beginning; Getting network and system administrators to share their stories of problems on the frontier. And for awhile, it was good. But as these services developed, they decided to start automating the process. And that's when the problems started. As an example, let's say all spammers use open relays. The logic here then is to test for open relays and block any that are found. Spam problem solved! Except it doesn't look at the reverse case: Namely, that not all open relays are used by spammers. In fact, it could be the case that the vast majority of open relays are perfectly harmless and have a legitimate reason for existing.
Now I'm not trying to discuss open relays from a technical standpoint, or the arguments for or against them -- what I'm trying to show is the logic problem in assuming that just because when 'A' is often found next to 'B', that means that 'B' is often found next to 'A'. That's the crux of the problem with the RBL and Spamhaus -- it's a logic fail of epic proportions.
Automation is attractive because it can catch things faster and with greater accuracy than humans can. But humans are better at making judgement calls, looking at the evidence, and problem resolution with other humans. Spamhaus and the RBL fail here because they implimented the automation and then because of their perceived success, they decided Automation Was God and made appealing the decision of its robot overlords increasingly difficult if not impossible. And that's when Spamhaus and the RBL became evil: The process stopped being overseen by humans, started to assume everyone was an evil spammer, and that the solution in every case was to follow the De Facto Anti-Spammer Laws as laid down by its robotic overlords. "Fix your open relay!" became the reply, instead of checking to see whether said open relay had actually sent any spam, or whether there was a good reason for its existance (again: No debates about open relays please! It's just the example!).
Of course, spammers got smarter and started coming up with more sophisticated methods of injecting their crap... which led to more complex robots, and as each new counter-measure was rolled out, the reply to hapless admins caught in the motorized wheels o spammy justice was "It's your problem, not ours!" My advice to system and network admins these days is to not use spamhaus or the RBL, or if you must, make sure your mailboxes and such are setup similar to how gmail and many exchange servers are: Have a separate spam folder, and give the user the option to whitelist anything your filters catch. Ultimately, you're providing a service to them... you have no duty or obligation to anyone else. Make sure they can use what you've given them.
Re:Sounds like (Score:4, Insightful)
Secondly, how sure are you somebody didn't forward your email to their own not-so-double-opt-in list which got reported as spam.
2/10 domains were blacklisted by Spamhaus, which means 2/10ths of his e-mail list might be contaminated.
It shouldn't be too much of a hassle to subdivide those users and flush out the one(s) which are causing the problem,
Ideally, you'd notify Afilias ahead of time so that they don't blacklist your honeypot domain(s).
People forget, or stop caring ... (Score:5, Insightful)
Sooner or later people forget they signed up, stop giving a damn, or otherwise get tired of what you're sending.
If they can't figure out how to get out of it (because, really, who is going to respond to something they think is spam to make it stop), they'll flag you as spam.
Or, something automated comes along and decides that whatever you're sending is spam.
As long as it stops coming when people get tired of it ... they really don't give a crap about what happens to you.
Re:Spamhaus and RBL = evil (Score:2, Insightful)
"My advice to system and network admins these days is to not use spamhaus or the RBL, or if you must, make sure your mailboxes and such are setup similar to how gmail and many exchange servers are: Have a separate spam folder, and give the user the option to whitelist anything your filters catch. "
you, sir, must have unlimited network resources. With spam taking up +90% [1] of internet traffic, you just rolled over and admitted that you
weren't as skilled as the opposition and let them sap your resources. I was hoping for better advice.
---
[1] http://skeptics.stackexchange.com/questions/2175/what-percentage-of-total-internet-traffic-is-spam
Re:Sounds like (Score:5, Insightful)
If that's what happened, it sounds like a DOS attack waiting to happen.How long do you suppose it will be before someone sets up an operation to spam your competition's websites to get them plonked.
Re:No illegal activity? (Score:2, Insightful)
That he runs peacefire isnt necessarily a mark in his favor. The idea that people have a right to circumvent filtering on computers they do not own is about as equally shady as whats being discussed here.
Theres "fighting for an ideal", and theres "going over the edge".
I have to challenge this (Score:5, Insightful)
Like the subject says, I have to challenge the claim that Spamhaus is wrong (full disclosure: I've been professionally involved in email and web security for more than a decade, but am not, and have never been, affiliated with Spamhaus. I do, however, hold them in high regard).
First of all, when I went to those domains, what was the first thing that caught my eye? "Get a green card" ads for usagc.org. I'm not specifically accusing usagc.org of spamming, but these sorts of businesses are most typically advertised by spam. I'm sure you've seen some.
Next, those sites are open proxies (by design). Anyone can create a URL like this: http://rootface.info/ojgnl.php?ZlQc9TMpAmsr3onaDWV0g=t1wn6QmM0TaAEo7rD%2F%2Bm%2Fy%2B365U2AwdnE4VH60DF8%2BU%3D [rootface.info] (nothing dangerous, it goes to cnn.com, but of course, you shouldn't trust me) and send it out in spam advertizing whatever they want.
Finally, you do not appear to state anywhere in your article that Spamhaus said your proxy mailing list was the source of the spam complaints (although they would not tell you if it was), and I doubt that it was. The most likely scenario is that someone abused your proxies to send spam, and since running an open proxy (regardless of noble motive) makes you complicit in that abuse, Spamhaus listed those domains.
Whether the registry's actions were justified or correct is a separate consideration. Maybe they were, maybe they weren't, but you are claiming without evidence that Spamhaus made a mistake. I'm pretty confident they didn't, for the reasons outlined above.