Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
Spam The Internet Your Rights Online

Armoring Spam Against Anti-Spam Filters 511

moggyf points to a BBC article about how spam can be successfully tweaked to slip past current filtering methods, excerpting "To finding out how to beat the filters Mr Graham-Cumming sent himself the same message 10,000 times but to each one added a fixed number of random words. When a message got through he trained an 'evil' filter that helped to tune the perfect collection of additional words." iluvspam adds "It's an interview with POPFile author John Graham-Cumming that summarizes his talk at the recent MIT Spam Conference. You can still listen to the technical details here (choose the Afternoon 1 session, he starts about 75 minutes in)."
This discussion has been archived. No new comments can be posted.

Armoring Spam Against Anti-Spam Filters

Comments Filter:
  • by bluelip ( 123578 ) on Wednesday February 04, 2004 @11:19AM (#8179637) Homepage Journal
    SO the ultimate spam protection mechanism would be an infinite number of monkeys type my list of words to associate w/ spam. :)
    • by AllUsernamesAreGone ( 688381 ) on Wednesday February 04, 2004 @11:34AM (#8179783)
      We better watch out for slashdot comments appearing in spam now.. ;)
    • by Jonas the Bold ( 701271 ) on Wednesday February 04, 2004 @11:36AM (#8179799)
      You kids and your monkeys

      In my day we didn't have monkeys. We had to filter spam by hand. And we liked it!

      You kids and your infinite monkeys... Shakespear wouldn't have used monkeys were he alive today. He would have rolled up his sleaves and written hamlet the right way!

      Damn kids..
    • by letxa2000 ( 215841 ) on Wednesday February 04, 2004 @12:00PM (#8180009)
      I'm not sure I understand why they think this is a problem with Bayesian filtering. Basically, they're saying that if a spammer sends you the same message thousands of times but inserts a few slightly different words each time, and if the thousands of messages get through the Bayesian filter to the user, and if the user doesn't disable HTML bugs on his email client, then we have a problem...?

      First, if the spammer sends thousands of copies of the same message and just changes the "extra words" that he is testing, it will take very little time for Bayesian to adapt to the rest of the message. Suddenly, the rest of the message that previously contained non-spammy words will be considered very spammy and will overwhelm the "extra words" that each message contains. Each time the message is caught as spam, the probability that any future tests get through--regardless of the "extra words"--will be reduced even further.

      Second, as the article said, it's a lot of work on the part of the spammer. They'd have to send out thousands of messages to each target to "sniff them out" and most of those wouldn't even be effective since most of them would be caught by filters and those few that got through very few would load the HTML bugs to identify themselves.

      Finally, it assumes that those that are using Bayesian filters are filtering their email but leaving their security (inasmuch as HTML bugs) wide open. While there may be some people that use Bayesian and leave HTML bugs active, it has to be a small minority.

      In short, it seems to me they've "found" a way to get around Bayesian that won't work, so to speak. I just don't see the problem.... ??

      • by Sique ( 173459 ) on Wednesday February 04, 2004 @12:12PM (#8180104) Homepage
        Second, as the article said, it's a lot of work on the part of the spammer. They'd have to send out thousands of messages to each target to "sniff them out" and most of those wouldn't even be effective since most of them would be caught by filters and those few that got through very few would load the HTML bugs to identify themselves.

        This is exactly the point. Most of the spam examples will die out because they have an ineffective collection of non spam words. But a few will survive and you now can train an own Bayesian filter which collects the versions of spam that generated webbug hits. After a while some words will shine prominently in your Bayesian filter database for being very effective at slipping through Bayesian spam filters.

        Basicly you a fighting the dote with itself. And yes. You can automate the process. Just take your everyday spam (penis enlargement, unsecured credit, Nigerian business opportunities...), take a dictionary and then randomly mix dictionary words into your spam messages and send them out to your email database. Create a website to get the webbug hits and associate every spam message with a hash of the random dictionary words to identify successful sets of anti spam words.
        • Or I could just sell the spammer a list of the words from 300,000 message Bayesian database that are 1% probability tokens.

          $50,000 gets you the whole 300,000 message Bayesian database.

          lindsayleeds _at_ comcast.net

          Pay up spammers.

      • by khasim ( 1285 ) <brandioch.conner@gmail.com> on Wednesday February 04, 2004 @12:35PM (#8180330)
        He managed to, randomly, find words that were high in _HIS_ "ham" list.

        He could have saved himself a lot of time and trouble and just looked in that file.

        And that file will be different for EVERY installation. So the words he found ("Berkshire", "Marriott", "wireless", "touch" and "comment") would NOT get spam past MY filter.

        So, the spammers have to keep (and update) a word list for EVERY PERSON on their lists.

        Which means that, with an incredible amount of effort, the spammers will be able to get spam to the people least likely to purchase a product from a spammer.

        There is no problem.
        • by WuphonsReach ( 684551 ) on Wednesday February 04, 2004 @02:02PM (#8181084)
          So, the spammers have to keep (and update) a word list for EVERY PERSON on their lists.

          That's one of the strengths of pushing bayesian filtering to as close to the final recipient as possible. Millions of customized bayesian scoring databases are much more difficult to defeat then a single centralized database. Bayesian databases are pretty much maintenance free, as long as the junk/not-junk/might-be user-interface is intuitive and makes life as easy for the user as possible.

          There is some value in putting the bayesian filtering at a workgroup level, where it helps that there's a bit of shared knowledge and everyone in the group pretty much agrees on their personal definition of what is/isn't spam. However, once you get past around 10-25 people, I'd say that bayesian is going to start becoming ineffective due to either over-zealous users, or overly-broad ham/spam classifications.

          What I'd be interested in is a bayesian that works both on the individual level and the workgroup level. With some sort of flag/switch/setting that tells the engine how much to consider the workgroup database as opposed to my personal database. This would be useful when adding a new member to the group, initially they'd rely heavily on the groups opinion as to what is ham/spam, but as time goes on it would adapt to their choices (as well as the group database slowly adapting to everyone elses).
        • He posted his "free-pass" words on the net.

          Never mind that his last name is "Cumming".
      • by FireBreathingDog ( 559649 ) on Wednesday February 04, 2004 @02:13PM (#8181191)
        It's much easier than that to defeat Bayesian filtering. Ever \/\/0|\|D3R why you're getting so much spam with obfuscated words? Or why you're getting so much spam where the text content is contained primarily in images rather than plaintext? Those things bypass Bayesian filters, that's why!

        Bayesian filters rely on words. That means it is dependent upon word breaks and certain spellings. Well, spammers have been avoiding word breaks (either by removing spaces or introducing unnecessary ones) and obvious "spam words" by mangling the word or introducing "1337"-type spelling.

        And Bayesian filters can't parse graphics, so a lot of spammers are careful to put words likely to trigger spam filters into graphics.

        BTW, this article [brain-terminal.com] explains why there will never be a filtering-based solution to solving spam until SMTP itself is made more secure.

        • Re:infinite monkeys (Score:5, Informative)

          by Jeremi ( 14640 ) on Wednesday February 04, 2004 @03:37PM (#8181958) Homepage
          Ever \/\/0|\|D3R why you're getting so much spam with obfuscated words?


          Nope, because my Bayesian filter works just as well for 0bfu5c4t3d words as it does for properly spelled ones. They are all just sequences of letters, and anything that is deliberately misspelled is going to become identified as spammy very quickly.


          Or why you're getting so much spam where the text content is contained primarily in images rather than plaintext?


          Nope, because I have images turned off by default in my mail viewer. If a stranger wants me to read his email, he'll need to send it as plain text, because (as you point out) HTML email with images is used as a spam vector and little else.


          BTW, this article explains why there will never be a filtering-based solution to solving spam until SMTP itself is made more secure.


          Funny, my Bayesian filter is working fine at this very moment. Who should I believe, your article or my own eyes?


          Jeremy

          • Nope, because my Bayesian filter works just as well for 0bfu5c4t3d words as it does for properly spelled ones. They are all just sequences of letters, and anything that is deliberately misspelled is going to become identified as spammy very quickly.

            The problem with obfuscated words is that there is a pretty sizable set of permutations for any given word. If one obfuscated variant ends up in your spam word list, that doesn't take care of the thousands of other obfuscated versions of the exact same word.

  • The bad news for spammers is that this flaw in filtering systems is not easy to exploit and can be combated. The cat and mouse game .. Find the "ham".
    But how do you combat someone that essentially has your "ham"?
    • How about going after the people who own the links in the body of the spam?
      Although it may be difficult to discover where the spam came originated, it's pretty clear where it wants you to go (probably the person who commisioned the spam in the first place.)
      • How about going after the people who own the links in the body of the spam?

        You are starting with a heretical premise that government, or rather, the large corporations which pull the strings, have the same objective as the end user (the end of spam). Of course, it could be stopped (by cracking down hard on those contracting the spammers). But it is much more useful for them if the "war on spam" goes on and on, while the measures with side-effects (on your wallet, your freedom and your privacy) are gradua
    • When a message got through he trained an "evil" filter that helped to tune the perfect collection of additional words. Soon he had generated a short list of words that, if added to a spam message, would guarantee its safe passage into his inbox.

      "The actual words it found were a total surprise," said Mr Graham-Cumming.

      The list included words such as "Berkshire", "Marriott", "wireless", "touch" and "comment". Including just one of these words convinced Mr Graham-Cumming's real spam filter that a message w

    • The best solution I have found so far is to have your own domain and generate specific email addresses for specific types of communications. You keep your actual ISP email address totally secret and don't give it to anybody except your domain registrar. You then generate an address for your best friends and aquaintances you can trust and keep it separate from everything else so you don't have to change it but once every few years if that. You have a specific Shopping and Registration address you kill and re
  • Ok fuck it (Score:5, Funny)

    by tomstdenis ( 446163 ) <tomstdenis@gma[ ]com ['il.' in gap]> on Wednesday February 04, 2004 @11:21AM (#8179652) Homepage
    I will pay 1000$ to anyone who seeks out and beats the living daylights out of a spammer. With as many pics on the web as possible for posterity.

    Screw these filters and shit. Start creaming spammers worldwide and they'll think twice about it.

    Tom
    • Re:Ok fuck it (Score:3, Informative)

      by swb ( 14022 )
      You do realize you've just comitted a pretty serious Federal crime, don't you? I know you're kidding or just emoting the same frustration many others, myself included, feel about the willful disregard spammers seem to have for many things.

      But you might've wanted to add a smiley...
      • Re:Ok fuck it (Score:3, Interesting)

        by cperciva ( 102828 )
        You do realize you've just comitted a pretty serious Federal crime, don't you?

        He hasn't, actually -- those laws don't apply extraterritorially, and Tom's in Canada.
        • by nigelc ( 528573 ) on Wednesday February 04, 2004 @11:43AM (#8179868) Homepage
          Ahh, an international terrorist proposing an attack. We should be invading Canada any day now...
        • Re:Ok fuck it (Score:3, Interesting)

          by Gaijin42 ( 317411 )
          Well, since this is an international forum, he has an out. But if it could be shown that he was soliciting someone to do that crime in the US, even if he did the solicitation from Canada, it would still be a crime in the US.

          At a minimum, he would be arrested if he came to the states. However, if someone actually went through with the crime, I'm sure Canada would be willing to extradite him. Canada doesn't want maniacs running around free, anymore than the US does.
          • Re:Ok fuck it (Score:3, Interesting)

            No true jury of his peers would convict him, since chances are they're sick of spam too! :)
          • by FreeUser ( 11483 ) on Wednesday February 04, 2004 @12:00PM (#8180001)
            At a minimum, he would be arrested if he came to the states. However, if someone actually went through with the crime, I'm sure Canada would be willing to extradite him. Canada doesn't want maniacs running around free, anymore than the US does.

            That assumes that beating the shit out of a SPAMmer is a "maniacal" act. I would argue that it is a perfectly rational course of action, and indeed a public service.

            Canada's Finlandization by the US might compell it to hand the guy over anyway, but certainly not for fear of having maniacs run loose (unless you count our troups poised on their border to enforce US Political Correctness Bush Style abroad). :-)

            [ Disclaimer required by Our Surveillence State: the preceding was a joke (c.f. humor). ]
    • by AdamD1 ( 221690 ) <adam@brainrubLAPLACE.com minus math_god> on Wednesday February 04, 2004 @11:54AM (#8179964) Homepage
      Is that illegal? After all he's not 'threatening' the spammer, he's merely presenting an offer he was pretty sure this guy was asking to receive. And besides: He can certainly "opt-out" at any time by choosing not to spam... ;)
    • by Anonymous Coward
      I will pay 1000$ to anyone who seeks out and beats the living daylights out of a spammer.

      Dear Slashdot,

      I am seeking volunteers to join me in a business oppurtunity which has recently come to my attention. Please volunteer if you meet the following three qualifications:

      1) Willing to send 1 spam email.
      2) Willing to have ass beaten.
      3) Want $250.

      If you said yes to all three of the above, please contact me. :D

      P.S. For those who consider #1 to be unethical, consider #2 your punishment.
    • I will pay 1000$ to anyone who seeks out and beats the living daylights out of a spammer. With as many pics on the web as possible for posterity.

      How about putting that $1K towards a legal use and offer it as a bounty to anybody who tracks down a spammer, sues him, and gets him thrown in jail and/or bankrupts him (via court imposed fines)? It may not have the same immediate satisfaction that you were originally seeking, but it's far more legal and I think you could find plenty of people here on Slashdot

  • by rmohr02 ( 208447 ) <mohr.42NO@SPAMosu.edu> on Wednesday February 04, 2004 @11:21AM (#8179656)
    POPFile [sf.net], maintained by John Graham-Cumming, is the best spam filter I've used. There may be small flaws with the fundamental concept of Bayesian filters, but POPFile still blocks all my spam.
  • by bc90021 ( 43730 ) * <bc90021 AT bc90021 DOT net> on Wednesday February 04, 2004 @11:22AM (#8179667) Homepage
    It's unfortunate that spam must be lucrative enough that one man will send himself the same message 10,000 times and train an evil filter! We need to get people to stop buying products advertised through spam (granted, easier said than done), as in the end, it's the financial incentive that makes a spammer spam. :(
    • by andih8u ( 639841 ) on Wednesday February 04, 2004 @11:32AM (#8179759)
      We need to get people to stop buying products advertised through spam

      As you alluded to, it'd be easier to teach fish to fly. The internet essentially carries with it a stupid-user tax. Worms, virii, spam, et al are the by-products of stupidity, but as with most taxes, it just something that you have to deal with.
      • by kent_eh ( 543303 ) on Wednesday February 04, 2004 @11:51AM (#8179937)
        One thing we can do is to make the spammers==virus_writers connection every time anyone asks us about (or even mentions) virusses.

        Aren't we the ones our friend(s) and co-workers ask about computer stuff?

        I have taken this a step further and contacted a few "computer journalists" locally and suggested that they make the spam/virus connection the next time they are writing about the latest virus. It's natural to answer the question 'where do these virusses come from' when talking about the latest scource of the internet.
      • by duck_prime ( 585628 ) on Wednesday February 04, 2004 @01:57PM (#8181034)
        ... The internet essentially carries with it a stupid-user tax. Worms, virii [sic, heh], spam, et al are the by-products of stupidity, but as with most taxes, it is just something that you have to deal with.
        With respect to spam, let's take a step back. Obviously somebody out there is gleefully munching handfuls of Viagra and (ahem) "enhancement" pills to psych himself up to (ahem) r0x0r his wife until her weight-loss pills kick in.

        It is silly to assume that all these people are just morons. After all, Viagra is proven to work, it is a legitimate product of sorts. The internet is there for hefty short limp (ahem ahem) non-digerati as well as for propeller heads, God bless 'em.

        It seems to me that spam is the runaway bastard-child of something which actually is good and useful -- that is, targeted marketing to the willing. Don't throw out the baby with the bathwater. There is a huge legitimate market out there, just begging to be flee^wmarketed.

        The anti-spam people are fighting against the Invisible Hand. Good luck.
    • by kris_lang ( 466170 ) on Wednesday February 04, 2004 @11:55AM (#8179967)
      Yes, it's dedication to research. He sent himself the 10k messages to see if he could outwit his own Bayesian filtering of spam messages. He effectively deduced that if the incoming message can be similar enough to items that have been specifically marked non-spam by the end-user of the Bayesian-spam-filter, it will be not be marked as spam.

      There's a cunning recursiveness to this which is at that fine line between clever and stupid. The difficulty is, as he also deduces, that each person's Bayesian rules for spam vs. nonspam are unique and will require many attempt in order to infer the pass-through words that will create a false negative and allow the spam to come through. The one step that people are missing is that if the evil spammer wishes to work on spamming a domain (both in the internet sense and in the "domain of expertise/specialization" sense) she can tailor the pass through words to the market. If she's sending spam to Intel or AMD corporate addresses, then lithography might be the magic word; if she's spamming Xilinx, the fpga will route through the Bayesian filter; if she's spamming Dave Barry, then debenture and fish falling from the sky might help spam make it through, Natalie may or may not make it through a /.'ers filter, actually usually including slashdot in the subject or as the name usually will make it through a slashdotter's filter. And the ease of this lies in that tailoring the open sesame words to a market will probably open the doors to all of the e-mail recipients at a domain, particularly is the spam filtering is done at the mail-server level and not at the end-user level. Thus rather than having to send 10k messages to a single user to crack open the spam doors, sending those 10k messages to multiple users at a domain and analysing which ones get through will effectively open the floodgates for all of the users at that internet domain. And using the concept of a priori probability distributions makes the hunt for these sesame words {[tm] /me :) } easier by limiting the dictionary to be searched to the keywords of the field/domain about to be spammed. That is what makes this dangerous.

      The counterattack from the corportate mail-server will be to look for these similarly unique messages being sent to multiple users.
  • Tch tch... (Score:5, Insightful)

    by supersam ( 466783 ) on Wednesday February 04, 2004 @11:22AM (#8179671) Homepage
    Didn't they know something as simple as...

    "Make it idiot-proof, and someone will make a better idiot"

    • Re:Tch tch... (Score:3, Interesting)

      by interiot ( 50685 )
      Well, that's not necessarily ALWAYS true... for instance, most crypto is at least heavily mathematics based, and therefore is much easier to analyze from a purely theoretical standpoint how much CPU is required to break. And in some cases (eg. DES) a lot of theoretical work HAS gone into them to identify weaknesses and analyze exactly how much CPU is required to break a given key length.

      Just that certain technical protections are of the nature that it's not a "I try some random protection, the idiots an

  • The only way (Score:4, Informative)

    by GuyinVA ( 707456 ) on Wednesday February 04, 2004 @11:22AM (#8179674)
    As technology gets more complicated, so does the spam. The only way to protect yourself is to not give out your address. Period. Heck, I don't even give my work e-mail address to my parents.
  • Great (Score:3, Interesting)

    by Polkyb ( 732262 ) on Wednesday February 04, 2004 @11:23AM (#8179684)

    I don't mind him trying to defeat the filters, if it comes up with a method of improving them, but the BBC should be shot for including the words that made it through

    Guess which words all tomorrows SPAM will contain...

    • Re:Great (Score:5, Funny)

      by stevesliva ( 648202 ) on Wednesday February 04, 2004 @11:30AM (#8179739) Journal
      Guess which words all tomorrows SPAM will contain...
      Touch my wireless Berkshire Marriot?
    • by Eevee ( 535658 ) on Wednesday February 04, 2004 @11:36AM (#8179803)

      In the article, it points out those words listed are good for getting past his filter. If you don't normally have mail that uses those words, then your filter will still catch it as spam.

      Now, if you do deal with the Berkshire Marriott frequently, asking them for comments on your wireless setup, then yes you're up the creek.

    • Those words are Mr Graham-Cumming's "magic" words. The article says you'd need to repeat the process for a particular individual to generate an equivalent list for them or, at best for the spammers, run the process against a pool of interconnected individuals, employees at the same company for example, to generate an organisation-wide list. My popfile probably wouldn't automatically let Berkshire or Marriott through, since I don't have sufficient ham that contains those words.
  • by Channard ( 693317 ) on Wednesday February 04, 2004 @11:24AM (#8179687) Journal
    Mozilla's filtering catches most spam for me, but some gets through. However, the only one that actually fooled me was quite a sneaky one - headed RE: Question from E-Bayer or whatever the actual subject is where you E-Bay something. Given that I sell on E-Bay, the spammers must have taken a gamble that enough people would read the subject and deem it worth looking at.
    • I have received piles of these recently. The names, item, item number, and amount change randomly, but it is always structured like a legitimate eBay message. I'm nervous about adding them to my bayesian filtering because I don't want to miss any eBay messages. I, too, sell a lot on eBay...
      • What, exactly, is wrong with the `make it computationally expensive to send email` solution Microsoft and others have proposed?
      • .. it would have to rely on the randomness of the sender's email, which is a giveaway when you actually look at the sender. It's as jumbled as the sender's email for most spam emails. The catch is, as the above poster mentions, missing an E-Bay mail isn't something that's particularly desirable. And I don't think Mozilla's filter could work effectively enough - baysian as it is - on just the jumbled 'from' address.
  • by Anonymous Coward on Wednesday February 04, 2004 @11:25AM (#8179694)
    I hate to see mainstream media coverage of this practice. I have started to get a lot of these spams lately.

    Typlically they include a large image at the top which is the entire intended content of the image and then a bunch of dictionary words at the bottom. It's basically impossible to filter these out unless you filter out ALL HTML e-mail because they don't contain any typical spam text.
  • my spam filter (Score:5, Insightful)

    by SkArcher ( 676201 ) on Wednesday February 04, 2004 @11:26AM (#8179706) Journal
    if Message header = "type = text/html" then send to "Spam"

    It works a treat :)

    The other trick I have found useful is the CamelCase nature of my name - spammers tend to mail me either as skarcher or SKARCHER, and both trip filters on my mailbox.
  • by Anonymous Coward
    All spammers have to do is read this analysis [mapilab.com] of the filter, then included the weighted non-spam strings, while avoiding the spam weighted strings. Pretty simple to blow past their filter.
  • by shrubya ( 570356 ) on Wednesday February 04, 2004 @11:27AM (#8179714) Homepage Journal
    ...if his surname weren't Cumming. At least his first name isn't Richard.
  • One word: WHITELIST. (Score:2, Informative)

    by jamehec ( 703164 )
    If you've whitelisted your email, that crap won't get through if you're not on the whitelist. That goes regardless of your Subject line. Same story if you do challenge/response, for that matter. Or you can munge, as I do.

    I still say spamming needs to be a felony, though.
    • I think whitelists end up discouraging quite a few legitimate users as well as spammers. I've received emails from people asking questions about this or that, I hit reply, and get shot back a message saying that I have to ask their permission to send them an email, even though I'm replying to them. I dunno if they're not setting up their whitelist properly to automatically add any address they send mail to, but I'm not going to hassle with writing out a reply to them, then having to go back a few minutes
  • by Faust7 ( 314817 ) on Wednesday February 04, 2004 @11:30AM (#8179741) Homepage
    Armoring Spam Against Anti-Spam Filters

    That description sounds too noble for an activity like this. More appropriate headlines would be Making Spam Slick as Owlshit or Infusing Spam with Satanic Strength.
  • Educate the people (Score:2, Interesting)

    by Theresa1 ( 748664 )
    When I was on holiday in tunisia, we were bothered quite a lot by trinket salesmen, who would not take no for an answer. Initially we had a lot of difficulty getting rid of them because my kids kept wanting me to buy the trinkets. <praying hands> plleeeese !!!!!!!! can we have one ? </praying hands>. Eventually even my kids got fed up with them, and a united front defeted them. Anyway my popint is, eventually the whole world will wise up and just ignore spam. There will bne no incentive for co
  • by Kidbro ( 80868 ) on Wednesday February 04, 2004 @11:31AM (#8179748)
    This would, for most slashdotters, be nothing to worry about. For those of you who didn't RTFA, the entire attack is limited by this particular little gem of info:

    He had to send himself thousands of copies of the same message each one holding an encoded chunk of HTML that reported back to him when it got past the filter.

    The concept is that the spammer has to find words that are so common in a person's ham that including them in spam would fool the filter. However, as those words are unique to each person, a lot (thousands or more) of spam must be sent to test the filter. The problem for the spammer is to figure out which spam actually got through (in order to identify the important words) - something s/he's not able to do for users with a decent email client...

    I still feel quite confident that SpamBayes will keep my inbox free from spam.
  • Why bother? (Score:2, Interesting)

    by nakedbonzai ( 618338 )
    I am still perplexed as of why a spammers wants to bypass someone's spam filter. Obviously, the person will simply delete any spam that gets through. They won't read it, they won't buy the product in question! Well, that's the case for me at least. I'd imagine the .001% of people who do respond to spam have no intention of ever using a spam filter.
  • Bogofilter does a really good job set as a filter rule in sylpheed-claws. Very few of those 'random valid word' type spams evade the filter, but every now and then one does.

    No problem. Just drag that sucker into the spam folder and the next hourly cron job learns about it. I've never seen it miss a repeat spam and false positives are extremely rare.
  • by musikit ( 716987 ) on Wednesday February 04, 2004 @11:35AM (#8179795)
    1. don't sign up on any page that requires you email address to verify *cough*like this one [slashdot.org] *cough*

    2. don't use free email services hotmail etc.
    3. don't use AOL
    4. don't let anyone have your address that forwards messages like "cute bunny pic" or "funny anti-geek joke" etc.
    5. don't post your email anywhere.
    6. don't sign up for majordomo lists.
    • by djrogers ( 153854 ) on Wednesday February 04, 2004 @12:33PM (#8180311)
      • 1) Register a domain (come on, they're cheap now)
      • 2) Get an email address from your ISP or other provider (yahoo, fastmail.fm etc) that is complex and convoluted - no names or words
      • 3) set up mail redirection with Zoneedit, redirection.net etc. with a catchall to your new mailbox.
      • 4) Use a different email address every time you must sign up for anything (ie amazon.com@newdomain.com)
      • 5) Filter on sent to headers at first sign of compromised id, or if the volume for a particular id gets too heavy and you're tired of client side filtering, set a specific redirection for it to sample@sample.com (do a whois on sample.com if you're curious).
      • 6) Enjoy the same spam free mailbox I've had for 2 years...
      Also helpful is to change your reply-to address every few months and give your friends different addresses based on how clueful they are
  • Line Noise (Score:5, Informative)

    by 4of12 ( 97621 ) on Wednesday February 04, 2004 @11:36AM (#8179801) Homepage Journal

    A previous story [slashdot.org] talked about the noise level of spam increasing.

    And a very entertaining NYT article [com.com] that is in the process of expiring.

    The upshot is that spam is being forced to look more and more like line noise. It will probably become less and less effective as the message has to submerge to the point where people can't recognize it.

  • Duh (Score:5, Informative)

    by Ricin ( 236107 ) on Wednesday February 04, 2004 @11:39AM (#8179832)
    Of course I can break my own Bayesian filtering.

    What matters is that while one person's spam might be very similar to another person's spam, their ham isn't. At best, it would require a semi-personal approach to sneak in spam. That's why you need to continually train your filter in the first place. Rinse and repeat, that's what it's all about.

    What's being described is not really a flaw, but rather a saturation point at which it's time to retrain your filter and perhaps even start over with a new database. The old one gets too much 'noise' after some time.

    They do point out one thing, be it from the spammers POV: Bayesian filtering is a continuous process and not and end to all solution. It requires fresh input and gets less effective if you keep old crud around for too long and if you train it too much on virtually the same spam/ham.

    It's still a much better solution than blacklists.
    • In the analog world many times if noise in a system is a repeating wave (hum in an audio line), it can be duplicated, inverted and added to the original to eliminate the noise and leave the signal.

      Apply this to a mail server. Hold all mail for about 5 minutes (from outside only). Compare them all. Look for matches of more than 50%. Cancel the matches out and filter the incomming for the same. This nails lots of the worms and spam by rejecting the common mode noise. Most spammers create a message and ma
  • Why is everyone surprised that every technique designed to eliminate spam can be fought? It's obvious that this is going to happen.

    The question should be: how do we live in a world where 99.9(n)% of email is spam? When the virus writers and zombie masters and spysters start using their communications infrastructure for its intended goal of delivering advertising?

    It's inevitable, and no amount of spam filtering will avoid it.

    Here's a prediction I made maybe 6 months ago on Slashdot: we're going to start seeing viruses that modify real outgoing emails to include their advertising messages. (And no Outlook jokes, thanks...) How does one filter spam when real emails are also infected?
  • by DocSnyder ( 10755 ) on Wednesday February 04, 2004 @11:40AM (#8179842)
    What they can't hide is the spamvertised target, as they want their victims to click onto a link and order something. Now you can resolve a link's IP address and check it against some common DNSBL blacklists (most spamvertised hosts are listed on SBL, SPEWS or chinanet.blackholes.us), or extract its domain and test it against some RHSBL or manual lists.

    What is more, if you multiply Bayesian or "word list" spam scores with results obtained with other methods, spammers may put "non-spammy" words into their spams as they like, but they only score their crap up instead of down.

  • by Jerf ( 17166 ) on Wednesday February 04, 2004 @11:45AM (#8179885) Journal
    Well, I may not have made it into the BBC but my attack is much more effective and much, much harder to defend against: Bayes Attack Report [jerf.org].

    It even counters the "personalization" quality of Bayes filters by finding the "common core" of personalization that we all share.

    Fortunately, spammers continue to be too stupid to understand this attack. Last time I posted this on Slashdot I got joe jobbed [jerf.org], because apparently it's easier to do that then to actually figure out what I was talking about.

    In summary, I wouldn't worry about your Bayes filters for a while: While they are attackable, spammers are too stupid to understand the attacks. (My article has been posted for over a year.) Thank goodness, sort of. (This will eventually be a temporary situation... but I see no particular evidence that the breakthrough will happen anytime soon.)
  • by The I Shing ( 700142 ) * on Wednesday February 04, 2004 @11:48AM (#8179907) Journal
    I've said this before, but I'll say it again. I really don't understand why all this even happens.

    When I'm going through the webmail access to my spam-bait accounts (the ones that are listed on my websites that I don't bother retrieving with my POP email client anymore because of hundreds of spams a day to each), if I'm fooled into opening one up, most likely because of it having a subject header that might be someone legitimate, the moment I see that the message body says anything spammy I immediately click the Delete button. I imagine everyone else in the world is doing the same thing.

    It's gotten to the point where the preoccupation of spamming is just to get past filters, the result of which is that the message is grumblingly deleted by the irritated recipient. Who out there is saying, "Oh, look, this message got past all my spam filters and contains a lot of jumbled, garbled nonsense text alongside a plug for herbal penis enlarging pills. This must be legitimate. Now, where's my credit card,"? Do the spammers think that we're all clones of Dilbert's pointy-haired manager?

    Spamming is not only irritating, it's pointless. Who is paying these people to spam us? Are people actually buying penis enlarging pills and patches, herbal viagra, mortgage refinancing, credit repair kits, or any of that stuff? Enough to put millions of dollars a month into the hands of career spammers?

    I'm hopelessly at sea in this matter.
    • It all depends upon where the blocking is taking place. Clearly some people are responding to spams, so there appears to be some incentive for the spammers to get their message through.

      Obviously, if an individual has gone to some trouble to set up spam filters, then she doesn't want to be bothered and the spam is pointless. However, the vast bulk of these filters are set up by the ISPs, and there's some value to the spammer to get through them to the idiot on the other side who apparently might actually

    • Here's the simple solution. Simply have your friends send you mail with "hot viagra teen sex mortgage" in the subject. Since all the spam is getting past the filters into the inbox, all of your real mail will be waiting for you in your junk mail folder
    • Spamming is not only irritating, it's pointless. Who is paying these people to spam us? Are people actually buying penis enlarging pills and patches, herbal viagra, mortgage refinancing, credit repair kits, or any of that stuff? Enough to put millions of dollars a month into the hands of career spammers?


      SHH!! If people paying for these things start looking carefully to see if they actually get a return on their investment, all sort of lunacy may follow:
      - Companies may start asking: Let's see, I spend $1
    • by tbmaddux ( 145207 ) * on Wednesday February 04, 2004 @12:56PM (#8180502) Homepage Journal
      Are people actually buying penis enlarging pills and patches, herbal viagra, mortgage refinancing, credit repair kits, or any of that stuff?
      Let me take a moment to tell you my sad story. I was in desperate need of penis enlargement, and so I did start ordering those pills. But they proved hard to swallow, and the patches were itchy, and I had an allergic reaction to the herbs in the herbal viagra. Unfortunately, I bought so much of this stuff that I had to refinance my home, and the bank wouldn't approve my loan because of all the penis purchases on my credit cards. So as a desperate last measure, I ordered some credit repair kits, but that didn't work either!

      Fortunately, this story has a happy ending! As I wrote this message, some polite people in West Africa contacted me and I think they are going to get me out of this financial mess.

  • by PixelCat ( 58491 ) on Wednesday February 04, 2004 @11:58AM (#8179989)
    What he's doing is a brute-force attempt to find words with--for himself--a high ham probability. I don't see how this is necessarily going to be an effective general-purpose technique. If you need to start bombarding people with thousands of messages to find the good words you're just going to drive more people into using filters--and this will almost certainly coerce ISPs into doing more filtering as well. Plus, you've got to deal with the issue of keeping data on all those users to find out which words are good for them. This would require you to tailor your spam to each individual user, which probably is going to increase the cost to the spammer (at least in terms of disk storage and time, anyway) and, as Graham-cumming implemented it, is going to fail utterly for anyone who isn't viewing mail as HTML, anyway.
  • I am building my own (Score:3, Interesting)

    by Tablizer ( 95088 ) on Wednesday February 04, 2004 @01:33PM (#8180834) Journal
    Any spam filter used by more than a few thousand people will be disected and and used to make filter-proof spam by the spammers. I am sure Bayesian has lots of holes if you work hard enough to find them. Bayesian depends on constistency in patterns. If spammers ruin that consistency, they won't work.

    Just the other day I found one spam that used a white font to put in legitamate-sounding text that would not visually show up on the screen. The spam text was a mix of graphics and pieces of real text. Thus, the word "penis" might start out with "pen" and end with a graphic for "is". Bayesian might start looking for the word "pen" after a while, but by that time the spammers will have a new trick up their sleeve. For example, if it looks for white fonts, then spammers might start using slightly off-white fonts, or black fonts on a black background. The combinations are probably endless.

    Thus, by making my own, my gizmo is not the target of spammers. They don't know about my filter nor care.

    The only alternative I can see is filter vendors constantly changing their algorithms every month or so, which would probably get expensive and risky. It is not like virus checking software that mostly just adds to their database and only tweak the algorithm a bit once every few years; it is like having to completely rewrite the virus filtering algorithms, not just the data.

    Ultimately, I think some sort of monetary postage system is the only effective solution. ISP and backbone makers will only have an incentive to track down spammers if they lose money on anonymous or forged spammers. This will make mass spamming far less lucrative.

    Either that, people will eventually find out the hard way that penis enlargers don't work and stop wanting to refinance their house. (I wonder if I can refinance all those expensive penis enlargers that I bought?)
  • easily combatable (Score:3, Insightful)

    by CAIMLAS ( 41445 ) on Wednesday February 04, 2004 @07:59PM (#8184948)
    This is easily defeated by an intelligent spellcheck built into antispam filters. It'd be able to recognize things such as commonly misspelled words, PGP/GPG keys, and file signatures, but would then create a rating based on number or percentage of non-words.

    It could then mark it with a spam rating and be combined with spamassassin or such.

    plus, wouldn't the spamassassin logic be able to say, "hey, we're getting a lot of non-word stuff - our filters tell us it's spam" and defeat this spam already?

"If it ain't broke, don't fix it." - Bert Lantz

Working...