Email Servers Will Choke, Says Spamhaus

Rub3X writes, "The legal battle between antispam organization Spamhaus and e360 Insight is heating up. Spamhaus has a user base of around 650 million, and its lists block some fifty billion spam emails per day, according to the project's CEO Steve Linford. Spamhaus CIO Richard Cox says the immediate issue is that if the domain is suspended, the torrent of bulk mail hitting the world's mail servers would cause many of them to fail. More than 90% of of all email is now spam, Cox says, and he doubts that servers worldwide would be able to handle a ten-fold increase in traffic." Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable. The article paraphrases CIO Cox as saying that the service will continue "even if there is a short-term degradation."

I say let the spam come

It would be interesting if all email server admins suddenly opened the flood gates for a day or two. Maybe then the general population will gain a better appreciate of the scale of the matter.

I still think they 3360 guys just look and smell like spammers. That spamhaus aggrees just adds to this conclusion. Here's what seems to amount to the spam histroy of the "plantiff". [spamhaus.org]

Re:I say let the spam come

It would be interesting if all email server admins suddenly opened the flood gates for a day or two. Maybe then the general population will gain a better appreciate of the scale of the matter.

I think most internet users still remember what it was like before spam filtering became common. Wait a few more years. Then users will take the filtering for granted.

-matthew

Re:I say let the spam come

Most users probably don't remember the rate of spam before filtering was common for a number of reasons:

1. The rise in internet usage [internetworldstats.com] since the year 2000 indicates, at best, only 1/3rd of the internet population could remember the rate of spam before filtering was common.
2. The rise of email usage indicates a large population of the people who were connected pre-filtering weren't using email.
3. The current volume of spam per person is at least triple what it was pre-filtering.

Most of us who were using the internet before spam filtering became so common have not seen what today's volume of spam would look like unfiltered. Assuming spam per person has tripled, anyone who was getting 20 spam per day pre-filtering would be looking at 60 spam per day now.

It would be a much deserved wake up call if spam filter companies were to shut down operations for a few days. It's obvious that the bodies overseeing this case think of Spamhaus as little more than a novelty. I think Spamhaus needs to send a crystal clear message, and perhaps the most effective way to do that would be to show the world how green the other side of the fence really is.

Re:I say let the spam come

It would be interesting if all email server admins suddenly opened the flood gates for a day or two. Maybe then the general population will gain a better appreciate of the scale of the matter.

Which is why I'm surprised Spamhaus doesn't just "simulate" what life would be like without them... before we're without them. Dispense with the predictions of how much spam will increase and what fate will befall the servers. Just shut off your service for a bit and wait for everyone to offer you their firstborn. Enron did it with California's electricity and it worked like a charm.

Re:I say let the spam come

I'm surprised Spamhaus doesn't just "simulate" what life would be like without them

It's easy to explain why they don't do this. They know that only clueless email admins rely only on an RBL for Spam control. Only the "Spamhaus faithful" would get clobbered with the extra Spam and they would have to switch to a different method or lose their jobs. This would be a sure way to kill off your customer base by proving empiracally why a single point of failure in Spam detection is a bad idea.

I've seen as much bad behavior from the RBL maintainers as I have from the spammers, so I only use an RBL as a final check to hold email that is on an RBL but otherwise passes through the filter. The (very few) held emails are almost always legitimate. The only reason I even bother to hold them is to keep an eye on what's going on and kill the final few Spam emails. The system I use for my employer has an almost perfect rate of rejection. Most of our users get fewer than 10 Spam messages a year! I get a lot of questions from co-workers about how to deal with Spam in their personal accounts because we do such a great job of dealing with it in their work accounts.

I know the Spamhous fanboys will take offense at this post. My only comment is that you are free to use an RBL as your only Spam control if you wish, just as I am free to use what I consider to be better methods. Good luck to you if Spamhous ever goes dark for any reason -- you're gonna need it.

Re:I say let the spam come

I have.

Here is the result:

Spamhaus gives only further sub-5% improvement on top of greylisting with a positive feedback loop at delivery/user report level. With relay level content filtering feeding into the feedback loop that will be down to under 3%. Greylisting on its own does 90%+.

The CPU cost of greylisting is not that much higher compared to DNS blacklists (and on a large site you can dynamically gate greylists into a local DNS greylist zone for distribution). In fact it is less if you form temporary firewall reject lists from your greylisting database.

So the answer is: technically Spamhaus is full of shit and the floodgates will not open. On most well managed sites it will be just another day. A bit more SPAM, but not a lot. At most it will make admins tune feedback loops into grey/black lists a bit better.

Move along people, nothing to see here. Spamhaus should stop dragging the rest of the internet into the stupid internet governance battle which is not for them to fight in the first place. I already commented on their position on this issue in past Slashdot posts on it.

Spamhaus should stop talking BS and move their operations to the same domain as their legal country of residence.

Re:I say let the spam come

Of course they are spammers. If tiny gray guys in overcoats, fake moustaches, and dark sunglasses ask permission (in a squeaky voice) to shut down the mouse trap factory, what do you think is going on?

Buggy post

Meanwhile the rest of the planet will treat an unenforcable court order from this judge about as seriously as they would a court order from the judge in this case [bbc.co.uk].

GP was missing the link above.

Re:I say let the spam come

Actually, the problem (if you read the lawyers who've written on this) is that originally they _did_ go to court.

IIRC they asked the original (state, district ?) court to move the case to federal.

_Then_ they didn't turn up at the federal court because they _then_ decided they didn't accept its jurisdiction.

Re:I say let the spam come

You're leaving out the part where their solicitors requested the venue change without instructions. AFAIK Spamhaus dismissed them and are taking them to court for creating this whole fucking mess in the first place.

Re:I say let the spam come

So, who has standing to file a complaint against this spammer?

He lied on the jurisdiction issue, and if that takes Spamhaus off the network, then millions of us suffer economic damage from the result of his perjury.

Anyone in Illinois want to register a class action against the son of a bitch?

-jcr

Re:I say let the spam come

Ummmm, they didn't go to court and they have not accepted anything, Spamhaus are demonstrating their view that the court does not have jurisdiction, Spamhaus seem to have a clue what they are talking about but the judge isn't listening since they refused to recognise the court by showing up. And if push really did come to shove then Spamhaus would probably just "reboot the company" in a different country.

I hope it's like you say, because in the media it came more like this:

Spammer: I'll sue you!

Spamhaus: Sue me!

Spammer: I sue you and I sued you! Your domain is goin' away!

Spamhaus: Oh no we give up, omg world prepare for e-mailmageddon! Fair well, fair well!

ICANN: We can't take your domain, Spamhaus.

Spamhaus: Oh what tragedy is before us, pitty us and you and... ICANN, you can't? Hmmm (damn it)

Spammer: I continue suing you and will win anyway!

Spamhaus: Oh no, world see how unfair the world is prepare for spamornado, spamunami, we're all doomed! Oh I pitty my sad fate! Oooh... Noo! Oh oh...

Random Observer: Dude stop making ass of yourself, you need neither the domain, neither you're the only solution for filtering spam out there. Take it like a man and maybe start respecting the court.

Spamhaus: Shut up observer, you're interrupting my dramatic routine.

Two lists needed

Maybe some legal problems could be avoided by having two lists. One, a list of spammers. The second list is people who are not spammers (cough) who have threatened or engaged in legal action to be removed from the first list. In other words a list of plaintiffs in court cases. Mail server admins could choose whether to use one list or both for blocking mail.

kdawson at it again.

Here's the dnscache (part of the djbdns family [wikipedia.org]) solution: /service/dnscache/root/servers# cat spamhaus.org
216.168.28.44
204.69.234.1
204.74.101.1
204.152.184.186
#

No need to HUP -- once the file is created and filled with those IPs, it'll pick them up automatically. You can easily install dnscache with the other tools on your mail servers for 0 interuption of service.

Cheers.

Interesting legal argument.

I'm starting to wonder about the sanity of Spamhaus' lawyers -- or if they really have lawyers at all. So far their arguments seem to have been

1. This case is at the wrong court, it should go to a federal court instead.
2. (to the federal court) We agreed that you had jurisdiction over this, but we're going to pretend that we didn't say that.
3. What? You've decided that we broke the law? Well, you shouldn't punish us because we're really nice people.

While I do not doubt Spamhaus' credentials as really nice people, this is hardly relevant to the case in question.

Suggestion to spamhaus

Why don't spamhaus just remove the e360 adresses from their regular spam lists and add them to a new list named "addresses no longer blacklisted becuase we were sued and ordered to remove them"?

That list would then serve as a perfect permanent black list for all sysadmins who happen to think that people who sue spam lists might not be the kind of people who send worthwhile emails.

I would actually recommend even higher priority to that list in the spamassassin config file than spamhaus' regular blacklists :)...

Spamhaus have their problems

Most of the comments I've read so far seem to be in favour of Spamhaus, and while I agree that they do some good work, they are not all good. Specifically, they seem over keen to blacklist address ranges without providing any proof, and very reluctant to unblock these.

I work for an ISP providing dedicated server hosting & colocation. Recently a couple of our customers contacted us saying that they had appeared on the Spamhaus blacklist, and were consequently having trouble sending e-mails. They claimed that they had not involved in any spamming activities, and that this listing was therefore incorrect. We found out that Spamhaus had blacklisted a range of our IP addresses (specifically a /27 subnet), and their explanation was that we were hosting someone from their ROKSO list.

While it was indeed true that we were hosting a server for this person, Spamhaus had a) blocked an address range larger than the IP addresses involved with this spammer, and b) would not offer any proof that the spammer had been using the server we host for him to involve in any spamming activities. When we contacted them, they refused to unblock this range unless we suspended the account of this spammer (again without providing any proof of activities conducted from our network that would breach our TOS), even though they acknowledged that the range they were blocking involved innocent customers. For us to suspend him at the request of Spamhaus would have been US breaking our contract with him, as there was no indication that he had violated our AUP (which DOES prohibit involvement with spam).

When we refused to break our contract with our customer at the request of a third party (perfectly acceptable position imho!), Spamhaus said that if they blocked any of our customers in future, they would blacklist our entire network (which is a considerable amount of addresses). This is unacceptable in my view, they are essentially trying to hold us to ransom without providing any proof of activities. When talking with some other ISPs, we heard of similar stories. In one case, the ISP concerned suspended the spammer's account and contacted Spamhaus to have their blacklist removed, and were told that "due to under-staffing, Spamhaus would not be able to remove the blacklist entry for a couple of days. however, if they would like to make a donation to spamhaus, they would remove the entry much sooner".

To reiterate my earlier point, Spamhaus does provide a valuable service, there's not much doubt of this. But they way in which they are organised leaves a lot to be desired!

Questioning the Math/Assumptions

More than 90% of of all email is now spam
Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable

I think the math is a lot more complicated than this implies. Here's how I'd work it:

• P = % Spam (% of all sent mail)
• S(T) = Total Mail Sent
• S(S) = Spam Sent
• S(N) = Non-Spam Sent
• E(T) = Overall Filter Efficiency (% spam detected, Spamhaus + All Other Filters)
• E(S) = Spamhaus Filter Efficiency (% spam detected, Spamhaus Only)
• E(O) = Other Filter Efficiency (% spam detected, All Other Filters w/o Spamhaus)
• F(T) = Overall Type II Error Rate (% false positive, Spamhaus + All Other Filters)
• F(S) = Spamhaus Type II Error Rate (% false positive, Spamhaus Only)
• F(O) = Other Type II Error Rate (% false positive, All Other Filters w/o Spamhaus)
• R(T) = Total Mail Received
• R(S) = Spam Received
• R(N) = Non-Spam Received

We're interested in R(T) and what happens to it with and without Spamhaus. (Assuming we're still interested at all, since math sometimes does that ...).

With Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(T)] + S(N) x [ 1-F(T)]
• R(T) = P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)]

Without Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(O)] + S(N) x [ 1-F(O)]
• R(T) = P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)]

The difference, expressed as a ratio of (Without Spamhaus - With Spamhaus)/(With Spamhaus), is

[ P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)] ] - [ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

Divided By

[ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

The assumptions yielding either the ten-fold or the four-fold increase seem to be that E(O)=0, and of course that false positives don't matter. Even with these assumptions, the math in the OP is a bit fuzzy to me:

• E(O) = 0
• E(T) = E(S)
• F(O) = 0
• F(T) = 0 [i.e., F(S) = 0 as well]
  yields (reducing above ratio):
• [ P x S(T) + [ (1-P) x S(T) ] - [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
  
  Divided By
  
  [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
• Which Reduces To:
  
  P x E(T) / [ 1 - [ P x E(T) ] ]

The ten-fold increase seems to be predicated upon both P=.9 and E(S)=E(T)=1. However, even if that were true, the increase would actually be nine-fold (.9/.1).

The four-fold increase seems to be predicated upon P=.9 and E(S)=E(T)=.75. However, this would yield about a two-fold increase of

[.9 x .75] / [ 1 - (.9 x .75) ] = 27/13 = 2.08 (approx.)

Factoring in false positives might actually make the Without Spamhaus scenario more dire, but clearly it would be less dire if we assume that E(O) is not zero. A better approximation would use the marginal efficiency of Spamhaus. Even with a generous assumption that Spamhaus catches an additional third of all spams sent (vs. all others without Spamhaus, and ignoring false positives), the overall increase in R(T) looks less than 50% to me (.3/.7, or approximately 43%).

Re:Someone please tell me they have an alternative

Dude,
I am so ready to walk away from cars. I just need someone to point me to a workable replacement.
I am so ready to walk away from television. I just need someone to point me to a workable replacement.
I am so ready to walk away from radio. I just need someone to point me to a workable replacement.
I am so ready to walk away from life. I just need someone to point me to a workable replacement.
I am so ready to walk away from my legs. I just need someone to point me to a workable replacement.

Re:Someone please tell me they have an alternative

I am so ready to walk away from cars. I just need someone to point me to a workable replacement.

Here. [wikipedia.org]

I am so ready to walk away from television. I just need someone to point me to a workable replacement.

Here. [wikipedia.org]

I am so ready to walk away from radio. I just need someone to point me to a workable replacement.

Here. [wikipedia.org]

I am so ready to walk away from life. I just need someone to point me to a workable replacement.

Here. [wikipedia.org]

I am so ready to walk away from my legs. I just need someone to point me to a workable replacement.

You can't walk away from your legs. Not with the same legs, at least.

Re:Someone please tell me they have an alternative

You can't walk away from your legs. Not with the same legs, at least.

Here. [wikipedia.org]

Re:Someone please tell me they have an alternative

It's because you have no friends.

Re:Someone please tell me they have an alternative

Your company advocates a

(x) technical ( ) legislative (x) market-based ( ) vigilante

approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

( ) Spammers can easily use it to harvest email addresses
( ) Mailing lists and other legitimate email uses would be affected
( ) No one will be able to find the guy or collect the money
( ) It is defenseless against brute force attacks
( ) It will stop spam for two weeks and then we'll be stuck with it
(x) Users of email will not put up with it
( ) Microsoft will not put up with it
() The police will not put up with it
( ) Requires too much cooperation from spammers
(x) Requires immediate total cooperation from everybody at once
( ) Many email users cannot afford to lose business or alienate potential employers
( ) Spammers don't care about invalid addresses in their lists
(x) Anyone could anonymously destroy anyone else's career or business

Specifically, your plan fails to account for

( ) Laws expressly prohibiting it
(x) Lack of centrally controlling authority for email
( ) Open relays in foreign countries
( ) Ease of searching tiny alphanumeric address space of all email addresses
(x) Asshats
(x) Jurisdictional problems
(x) Unpopularity of weird new taxes
(x) Public reluctance to accept weird new forms of money
(x) Huge existing software investment in SMTP
( ) Susceptibility of protocols other than SMTP to attack
( ) Willingness of users to install OS patches received by email
(x) Armies of worm riddled broadband-connected Windows boxes
( ) Eternal arms race involved in all filtering approaches
( ) Extreme profitability of spam
( ) Joe jobs and/or identity theft
( ) Technically illiterate politicians
(x) Extreme stupidity on the part of people who do business with spammers
( ) Extreme stupidity on the part of people who do business with Microsoft
( ) Extreme stupidity on the part of people who do business with Yahoo
(x) Dishonesty on the part of spammers themselves
(x) Bandwidth costs that are unaffected by client filtering
(x) Outlook

and the following philosophical objections may also apply:

(x) Ideas similar to yours are easy to come up with, yet none have ever been shown practical
( ) Any scheme based on opt-out is unacceptable
( ) SMTP headers should not be the subject of legislation
( ) Blacklists suck
( ) Whitelists suck
( ) We should be able to talk about Viagra without being censored
( ) Countermeasures should not involve wire fraud or credit card fraud
(x) Countermeasures should not involve sabotage of public networks
( ) Countermeasures must work if phased in gradually
(x) Sending email should be free
(x) Why should we have to trust you and your servers?
( ) Incompatiblity with open source or open source licenses
( ) Feel-good measures do nothing to solve the problem
( ) Temporary/one-time email addresses are cumbersome
( ) I don't want the government reading my email
(x) Killing them that way is not slow and painful enough

Furthermore, this is what I think about you:

(x) Sorry dude, but I don't think it would work.
( ) This is a stupid idea, and you're a stupid company for suggesting it.
( ) Nice try, assh0le! I'm going to find out where you live and burn your house down!

Re:Someone please tell me they have an alternative

Easy. We just need to set up a protocol where an ISP is charged $0.01 per email sent. That will kill the spammers without having any real effect on people sending email.

Actually, the problem is not this simple. Spammers today send their emails from millions of hacked computers worldwide. They will just continue to do so, and these charges will drop on the clueless users whose computers are used to send the emails.

As long as computer security is as bad as it is today, there just is no easy solution to spam. All hyper-clever ideas about encrypted network id:s, black and whitelists, hashcash, etc, are just temporary solutions --- they only serve to drive the spammer to more intensly use the fact that a hacked computer also gives access to an online identity.

Re:Use the UK server name!

...it's not subject to US law

It's sad how this statement is becoming more and more associated with freedom nowadays.