Fighting the Hydra -- A Spam Warrior's Tale 333
Selanit writes "Salon has an interesting article about the battle against spam from the viewpoint of Suresh Ramasubramanian, a sysadmin working in Hong Kong. His most interesting complaint concerns the fragmentation of anti-spam forces: not only does he have to deal with spammers, but also with anti-spammers who assume because his company is Chinese that he isn't doing anything about spam. Hmm ... decentralized opponents striking from the shadows against quarreling allies. Does this sound familiar to anyone else?"
Re:75 million? (Score:2, Informative)
Re:Translation please... (Score:2, Informative)
It should be Sturmbannfuehrer.
Sturm -> storm
bann is a shortened form of banner, which is the same in english
fuehrer -> leader
--> storm banner leader
bahn is either course or a train running on the course/rail. I'd translate Sturmbahnfuehrer as storm train leader
Re:Translation please... (Score:2, Informative)
Whitelisting is unethical (Score:5, Informative)
And how do you feel about making all innocent senders of mail do extra work, while spammers simply ignore it and move on?
I simply cannot justify that, based on the redistribution of workload and increased aggravation - you send me a bounce message, I consider your email address invalid whether that bounce is "500 address unrouteable" (a valid, understandable error) *or* "500 I Don't Like You" - which I consider frankly offensive.
Go back to SpamAssassin, get 2.50 or better, which includes Bayesian analysis as well as all the above. Or just shove a Bayesian filter in the way after SA; here, I have outright regexp-based rejection and SA in exiscan, followed by bogofilter in procmail - very few spams get past the first hurdle (From: headers snarfed from Usenet) and those that do are caught either by SA and/or bogofilter.
This way happiness lies.
The bounce problem (Score:5, Informative)
If 50% of all mail in the US is spam, then the other 50% must be the bounces for all that undeliverable mail!
I run a mail gateway for a medium sized company, and although not on the scale of a large ISP, I see many of the same problems. Dealing with spam on a gateway level is quite different from dealing with a single personal mailbox. And spam flooding has gotten much worse in the last few months. Getting over a 1000 messages in under a minute can really start to tax your infrastructure. Actually from my own observations, I'd say that at least 75% of all mail is spam, and 80% of that is undeliverable.
Of course one of the big problems as Ramasubramanian points out is that spammers are getting very sophisticated at impersonating other entities. This results in a large number of bounces being directed back to the wrong guy. So not only are you getting spammed, but you are also indirectly spamming the poor guy who is being impersonated with your flood of bounces. And the bounces also cause other problems because it tends to fill up your outbound mail spools, as well as making the required postmaster account near useless sometimes.
One thing I've learned is that a mail administrator must be very careful about constructing blacklists and filters. I use sendmail [sendmail.org] and make heavy use of it's milter [milter.org] programatic filter interface. It's amazing how being able to analyze the mail at the protocol level (such as the HELO command) helps identify impersonated mail that can't just be done by only looking at mail headers or the message body. It is also possible to help correlate large volumes of nearly identical inbound mail from a large number of different servers, as well as correlate them with large number of undeliverable outbounds. I'm also very careful to check whois an other registrar databases before adding blacklist entries, to help prevent blacklisting the wrong guy. But I do admit that for a few of the most audacious flood attacks, I actually have to resort to iptables [netfilter.org] firewall blocks to stop it even before sendmail sees it. I really dislike having to disobey the SMTP standards, but spam floods are IMHO just as destructive as worms and viruses!
The thing I fear most as a mail administrator is not the inbound spam, but that some spammer may start impersonating my company! We'd start getting placed on blacklists and blocked, plus we'd start getting flooded with all those bounce messages (probably an order of magnitude more than direct spam). How can one possibly protect against that?
Re:Whitelist "black holes" (Score:4, Informative)
*shudders when thinking at the vacation-wars*
China is old news for spam. (Score:4, Informative)
The spammer [slashdot.org]
I knew has moved to the Philippines. Supposedly it's the next big shelter for
these roadhogs. China has a lot of business interest in the US so they are doing what they can do eliminate the problem.
Roughly speaking... (Score:4, Informative)
/me shudders
Cheers,
Ethelred
Re:Welcome to the life of a helpdesk worker. (Score:3, Informative)
I'm probably not seeing the full picture, because I preemptively block inbound SMTP from netspace that doesn't terminate spammers. The biggest chunks are 4.0.0.0/8 (open DSL proxies from Genuity/Verizon/LVLT depending on who's bankrupt this week), 12.0.0.0/8 (ditto in AT&T space), and 24.0.0.0/8 (ditto, but with cablemodems) and 200.0.0.0/6 (all of LACNIC and a decent chunk of Asia.)
That in mind...
"So what's it like at Rackspace?" :-)
Re:Perhaps we should let the dam break (Score:3, Informative)
Suresh (Score:1, Informative)
Re:Roughly speaking... (Score:2, Informative)
There were indeed some Orwellian aspects to the way SS positions were named, along with the entire Nazi regime. (The SS was originally "marketed" to the German people as some sort of overgrown Boy Scout troop for grown-ups, almost like a charity -- you should see some of the early propaganda posters. Really chilling when you know what the real meaning of the SS was.)
The Nazis succeeded in perverting the German language to their ends in many ways that are hard for non-German speakers to appreciate -- even today many words that sound innocuous in English have ominous overtones in German because of the way the Nazis (mis-)used them. "Sturmbahn" is a pretty innocuous word -- "storm path", just like in a weather report. But many Germans would be reminded of SS officers. "Fuehrer", which just means "leader", is obviously also corrupted. "Ueberfremdung", which originally meant something like "estrangement", now has xenophobic overtones thanks to the way the Nazis used the word in their propaganda.
Nowadays anyone who wants to "defend" the German language by keeping out English expressions, like the French do now, is usually derided as neo-Nazi (or at least suspiciously nationalistic). Which is why attempts to introduce "German" expressions for Internet ("Zwischennetz") or e-mail ("E-Post"), for example, have largely failed miserably. To be proud to speak "pure" German smacks of being rather right-wing, thanks to the Nazis and their obsession with pure German-ness (if there ever was such a thing).
No, it isn't quite like Newspeak (not NewsSpeak), where "unneeded" words are banned in order to prevent independent thought, but it was in many ways a similar process -- warp a language to suit your own ends. I don't know if Orwell was aware of the Nazi perversion of the German language (he was certainly aware of the regime's other tactics, many of which are reflected in "1984", along with those of Stalin's regime), but there are interesting parallels between the two ideas.
Cheers,
Ethelred