Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Spam Your Rights Online

DSPAM v3.0 RC1 Spam Filter Released 182

Posted by simoniker from the egg-and-beans dept.
Nuclear Elephant writes "DSPAM v3.0 RC1 is now available for download, with a stable release scheduled for June 13. DSPAM has appeared on Slashdot and in Wired News in the past for its high levels of accurate spam filtering. v3.0 is the product of three solid months of work. Some of the highlights include a very sleek redesigned interface, PostgreSQL support, many mathematical enhancements, and support for many of Gary Robinson's algorithms (such as Chi-Square, Geometric Mean Test, and Robinson's technique for combining P-Values)."
This discussion has been archived. No new comments can be posted.

DSPAM v3.0 RC1 Spam Filter Released More

DSPAM v3.0 RC1 Spam Filter Released

Comments Filter:

  • How is this a YRO? (Score:3, Insightful)

    by magefile ( 776388 ) on Monday May 31, 2004 @02:16PM (#9297404)
    I don't get it.

  • Good filter (Score:5, Informative)

    by vegetasaiyajin ( 701824 ) on Monday May 31, 2004 @02:18PM (#9297416)
    I am using this filter and after some training it is very effective. Especially useful is the inoculation feature, which you can use to register a spam only address to spam sending sites so that it trains faster.
    • You could also try using Mailinator.com [mailinator.com] to use as a throwaway, although that's not really the intent you had, was it?
    • I wouldn't need to register a particular spam-only address; I get huge amounts of spam to made-up usernames at my domain. I used to get mail sent to any local-part @membled.com, but that became unmanageable about a year ago.

  • Another one for the arms race... (Score:5, Interesting)

    by Anonymous Coward on Monday May 31, 2004 @02:19PM (#9297420)
    How much more complex will spam filters have to get to gobble up all the CPU on the mailserver or mail client machine?

    I'm all for throwing technology at the problem, but I hope people still realise that having a complex (and effective) spam filter does not take away the millions of megabits of traffic wasted on UCE when it's in transit.

    • Re:Another one for the arms race... (Score:5, Insightful)

      by AntiOrganic ( 650691 ) on Monday May 31, 2004 @02:29PM (#9297477) Homepage
      I'm all for throwing technology at the problem, but I hope people still realise that having a complex (and effective) spam filter does not take away the millions of megabits of traffic wasted on UCE when it's in transit.
      If people stop receiving spam, and therefore the morons among us stop giving money to spammers by buying their crap, and thus remove all semblances of profits obtained through spamming, there won't really be much incentive to spam anymore, will there?

      • Re:Another one for the arms race... (Score:4, Insightful)

        by Senior Frac ( 110715 ) on Monday May 31, 2004 @02:39PM (#9297537) Homepage

        If people stop receiving spam, and therefore the morons among us stop giving money to spammers by buying their crap, and thus remove all semblances of profits obtained through spamming, there won't really be much incentive to spam anymore, will there?

        Boy, that's a losing battle you propose. The spammer only needs one sucker out of 10 million to stay in business (since he steals his advertising costs). Yet, the defending network must educate all 10 million not to buy from spammers, an impossible task.

        • He didn't say we educate 10 million not to buy from spam, he said we prevent those 10 million from getting the spam and having a chance to buy.
      • Problem is many organisations aren't allowed not to deliver email which is probably spam. So they have to deliver it, probably only tagged (with an extra X-Header or some phrase added to the subject). I.e. people still receive spam they just have the possibility to trash it easier - which stupid people, i.e. people buying things advertised in spam emails, won't do anyway.
        Sucks, eh? :-/
      • If people stop receiving spam, and therefore the morons among us stop giving money to spammers by buying their crap, and thus remove all semblances of profits obtained through spamming, there won't really be much incentive to spam anymore, will there?

        Right, because we all know that people with a no-longer-relevant business model are quite happy to give it up and move on to something else.

    • How much more complex will spam filters have to get to gobble up all the CPU on the mailserver or mail client machine?

      It already is. At 500+ users and 200 pieces of junk mail a day, that is already more mail than there are seconds in the same period. Would you think the new spam filters use less than 1 cpu second per mail? I hope you have a bad-ass mainframe for your companys spam filtering...

    • Re:Another one for the arms race... (Score:5, Funny)

      by WormholeFiend ( 674934 ) on Monday May 31, 2004 @02:33PM (#9297498)
      what I really like about this arms race is that news stories about "how bad spam is" are becoming a regular feature in numerous media outlets...

      what that means is that the opinion of the silent majority is being moved toward "angry mob" status, which, I believe will lead to the downfall of the Spam Kings.

      so if anyone is interested, I'm planning on opening an online store specializing in torches and pitchforks...

      • Re:Another one for the arms race... (Score:5, Funny)

        by Kwil ( 53679 ) on Monday May 31, 2004 @03:04PM (#9297659)
        Hey, you should send out some email about this service, I bet people would love the chance to buy in.

        Why, I think I know a place where you can send email to up to 2 million addresses for only...
      • I always wondered why angry mobs wait until nighttime. If they just rampaged during the day, they wouldn't need the torches, and could carry machetes instead.

      • what that means is that the opinion of the silent majority is being moved toward "angry mob" status, which, I believe will lead to the downfall of the Spam Kings.

        Yeah, those lousy spam kings, filling my inbox with their... ooh! a cheap way to make my penis larger! Where's my credit card?

      • what that means is that the opinion of the silent majority is being moved toward "angry mob" status, which, I believe will lead to the downfall of the Spam Kings.

        Spam is too deeply involved with big business, either as direct suppliers (premium cost network connectivity, credit card services), or tangental involvement (list selling). Besides, the "silent majority" hated telemarking for *how long* before we got the fairly limp no-call list?

        If the government actually had an interest in stopping spamm
        • I'm pretty convinced that the Feds have been told it's a non-priority, partly due to Ashcroft's terrorism paranoia, and partly because big business wants to keep spam as an option.

          Or, may be, it is just because spammers never hired hitmen to kill anyone? That they are not believed to have ever tried to bribe a judge, or kidnap a prosecutor's child?

          Using RICO laws against them may be just as inappropriate as some of the publicized (mis)applications of the PATRIOT act...

    • but I hope people still realise that having a complex (and effective) spam filter does not take away the millions of megabits of traffic
      Hence WPBL [pc9.org], which uses sightings by statistical filters (like DSPAM) from multiple sites to build a real-time blocklist based on consensus sightings. Once the IP is on the blocklist, you don't waste bandwidth accepting mail from them.
  • Why would I need this?
    • My copy of t-bird (0.6) its spam filter seems to suck more and more lately (perhaps its just the spammers are getting better at bypassing the filters). I just switched to server side spam filtering (just adding a tag to the subject), and then I key off that in t-bird.
      • I find that the spam letters that do get through T-Bird's junk mail filter are the ones padded with random strings of letters. My guess is that T-Bird is able to identify the spam words (eg: debt consolidation, enlargement) but the mispelled words (eg: peni5) are unknown to T-bird. So T-Bird makes the conservative decision not to mark the e-mail as spam. I figure a simple filter criteria that requires the correct spellings for at least half the words in the body (for unknown senders) should get rid of this problem. Anyone care to enlighten me if such a rule is in T-bird or is in the works? At the very least, this will have the side effect of encouraging people to at least spellcheck their e-mails before sending. :)
        • So T-Bird makes the conservative decision not to mark the e-mail as spam.

          T-Bird makes the mistake of making spam/ham a binary decision. I really wish it would work more like SpamBayes which has a trinary system (spam / unsure / ham). That works well because the stuff it tags as spam is almost always spam, and the false positives usually end up in the unsure pile. The "unsure" pile is also usually 1/10th the size of the "spam" pile, so it takes a lot less time to verify before tagging all of the "unsu
    • Simple, if what you have works for you then you probably don't need this.

    • Re:client-side versus server-side anti-spam (Score:5, Interesting)

      by ubiquitin ( 28396 ) * on Monday May 31, 2004 @02:34PM (#9297508) Homepage Journal
      When you run your own mail server, or administrate a mail server for a large number of people, server-side anti-spam filters and countermeasures start making a lot more sense. Do the math on a company with 100 employees (at $25/hr) who check mail twice a day and spend 5 minutes each time hassling with anti-spam measures in client-side mail apps. In this scenario, a seamless anti-spam solution is worth conservatively $400 per day, or $100k/year not counting bandwidth savings. There are definitely cases when client-side filtering makes sense, but if you can handle it at the server, email-based business methods scale better.

      • Unless you have false positives and such. (Score:4, Interesting)

        by khasim ( 1285 ) <brandioch.conner@gmail.com> on Monday May 31, 2004 @03:13PM (#9297708)
        I'm the one running the spam filter (SpamAssassin) at work. Overall, it has been VERY popular with everyone else. They don't receive the most obnoxious sex spams any more.

        On the other hand, there are a few false positives that reduce the overall savings in your post. I auto-delete anything about 10 and flag anything above 5.

        But the end users still have to look through the flagged stuff to see if there are any false positives. Then they drop them into the false positive folder. The users also have to identify all the missed spam and drop that into the spam folder.

        It's still work for them so the costs aren't as clear as in your post. But the non-tangible benefits are also important.

        I think we're at the point of dimishing returns on simple scanning processes. I think we need to look at actively seeding the spammer's lists with false names and tuning the spam filters with those.
        • SpamAssassin is a good start. If you're really wanting to reduce false positives, consider bringing
          dspam [nuclearelephant.com] into the mix. "DSPAM presently peaks at 99.985% accuracy, which is ten times more accurate than a human being and is presently being used on implementations as large as 125,000+ mailboxes." bogofilter [sf.net] is another advanced project in the same functional space.
          • Just wait till the spammers start training their spam generators through dspam.

            Furthermore: some people/sites just write messages that look like spam.

            A legit sender could say this: "Activate your registration now - click here [url link]". I mean what else do you want them to say without wasting bandwidth?

            A spammer could send nearly the same message.

            So you'd probably have to blacklist/whitelist the urls they link to.

            I think having decoy email accounts to identify spam could be a useful tool.

    • Wonderful, if you just want to stop seeing the spam. I, however, would enjoy not having to pay for it's delivery. This is the ostrich method of spam fighting.

  • But will it? (Score:5, Funny)

    by ajiva ( 156759 ) on Monday May 31, 2004 @02:21PM (#9297432)
    But will it find out who sent the SPAM and hurl them into the Sun? Until I get this feature, I don't think it'll be perfect :)

    • Re:But will it? (Score:4, Funny)

      by dheltzel ( 558802 ) on Monday May 31, 2004 @03:26PM (#9297773)
      I want a button on my mail client called "Retaliate", that will hunt down the sender, use various cracking techniques to take over their system, send back a copy of all their personal data, and subtly corrupt any email addresses it finds in any files or databases on the system. Optionally, it would locate some illegal content off the internet and copy it all over the filesystem, then send the IP address and other identifying info to the appropriate government agency.

      That would make it fun to get Spam!
      • I ^H^H a guy I know used to retaliate, stopped for a while when the spammers built up their defenses, and then tried it again last week against some spams which started leaking thru his filters.

        They are wide open again, brothers, because apparently no one else is dossing them anymore either and they have let down their guard.

        I would guess that they lost money when they overprotected their forms against that type of "response," which made too many legit buyers say fuck it instead of filling out some bossy
      • Retaliation against spammers, the eternal dilemma, the rights and wrongs of sinking to their level, fighting abuse with abuse, all that... But what it boils down to is a simple choice:

        Trident or Polaris?

        • True, and like nukes, it might have some deterrent value as well. I wonder if they might respect a black list a little more if they thought retaliation was a viable option for the recipient.

      • Wouldn't that be a rather mild reaction?

        No casteration by soldering-iron to ensure severe pain and cronic lack of reproductive skills?

        *shocked at the current /.-modesty*

  • Innovation - Statistical Hybrid Filter (Score:4, Insightful)

    by ospirata ( 565063 ) on Monday May 31, 2004 @02:22PM (#9297437)
    DSPAM has a strong focus on providing better data to already existing algorithms (Bayesian, Chi-Square, etcetera) Combination algorithms work inherently well, but depend on the quality of data. Some of the approaches deployed in DSPAM towards this goal include Chained Tokens, Inoculation Groups, Classification Groups, advanced de-obfuscation techniques, and a new noise reduction algorithm called Bayesian Noise Reduction. The goal is to incorporate processing algorithms that can withstand the long haul of ever increasing message complexity. So far we're doing a great job.

    The idea of combining more than one anti-spam heuristic is not new. But one thing that cant be denied is that all methods are just complementar to Bayesian analysis, that can reach up to 95% precision by itself. Chi-Square, itself, can reach up to 85% precision
    • Friend, you need to take a look at the specs on CRM114 at crm114.sourceforge.net. While the interface and initial setup are fairly painful for people who don't build their own email setups, various folks are publishing that they get over 99.9% correct detection of both spam and non-spam. That's far better than any other single filter out there.

  • Yay, we fixed spam! (Score:5, Insightful)

    by Anonymous Coward on Monday May 31, 2004 @02:24PM (#9297445)
    Look! We came out with this great filter so nobody else gets spam! This solves the problem of spam once and for all! Even though spam is still clogging our networks and wasting bandwidth, this filter will solve all of our problems.

    With all the time spent on making spam filters, why don't we spend that time working out a new protocol for email transfers, one that would not be able to spoofed, or spend that time installing server side programs that put a small time delay between messages as well as bandwidth restrictions for all outgoing mail?
    • While I agree that this is not a cure all for spam, the bottom line is that it will make a difference. Spam only happens because people buy the crap they're offering. Stop the cash and spam will stop as a result, or evolve into huge robots that come to your house and steal gas from your car... gas stealing bastards, wilson I know you were behind this !!! Was this post informative, funny, offtopic or all three? I leave that up to you to ponder... :)

    • Re:Yay, we fixed spam! (Score:5, Interesting)

      by Senior Frac ( 110715 ) on Monday May 31, 2004 @02:50PM (#9297584) Homepage

      With all the time spent on making spam filters, why don't we spend that time working out a new protocol for email transfers, one that would not be able to spoofed,

      Because there's nothing wrong with SMTP. SMTP already has extensions to allow authentication but it still requires a central authority to say "He is Senior Frac, we verify it." No one will trust such an authority even if it was scalable enough. If you think spam is caused by a lack of authentication, you're sadly misinformed. The cause is a lack of responsibility by the sending networks to enforce proper behavior of their users.

      or spend that time installing server side programs that put a small time delay between messages as well as bandwidth restrictions for all outgoing mail?

      These technologies exist. Unfortunately, most that install them stop monitoring them. Such work is considered a resource hog which the ISP would much rather spend on signing up new customers. Bandwidth restrictions on a customer who is running their own MTA makes things much more complex and much less scalable.

    • I appreciate what you're saying but the problem isn't with technology. It's with politics. Most spammers are breaking numerous laws, not the least of which are felony computer tampering laws when they use ghost proxies, and the larger problem is that the authorities don't seem that interested in going after them.

      Content-based filtering is a waste of time. The only exception to that would be to write a spam filter that sends a message to your local Attorney General trying to educate him on the illegal acti

      • Obligatory (Score:3, Insightful)

        by jonfelder ( 669529 )
        Your post advocates a

        ( ) technical (*) legislative ( ) market-based ( ) vigilante ( ) lack of an

        approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

        ( ) Spammers can easily use it to harvest email addresses
        ( ) Mailing lists and other legitimate email uses would be affected
        (*) No one will be able to find the guy o
    • Exactly! This is merely more duct tape to solve a leaking pipe problem. Look what happened with telemarketers! It became an epidemic, people got ticked off, and the FCC created the "Do not call list". Sure a few evil bastards found ways around it, but nevertheless, the amount of calls went down drastically. The same principles that kept telemarketers in business are the same that keep SPAM in business. There are a few morons that actually buy stuff from spam. I believe legislation is actually needed.

      • Look what happened with telemarketers! It became an epidemic, people got ticked off, and the FCC created the "Do not call list".

        I still get calls coming through every once in a while. Others like to waste the telemarketers' time, chatting them up then saying "hang on a minute, let me get my credit card" and then just putting the phone down and going off and doing something else, checking back in a half hour to see if they've hung up yet.

        Seinfeld's response was great as well: "Okay, give me your hom

        • Perhaps a whitelist where everything not on the whitelist goes into the "Junk Suspects" box, combined with a Bayesian filter? I don't have the answer, but there's gotta be one.

          That is essentially what I do. I have instructed Spamassassin to whitelist friends, family, work, and some mailing lists. This has the additional effect of those mails being autotrained as ham. I also have a training folder that is like your Suspects box. Anything with a positive score winds up there. Every once in a great whi

      • Obligatory (Score:5, Funny)

        by jonfelder ( 669529 ) on Monday May 31, 2004 @04:37PM (#9298151)
        Your post advocates a

        ( ) technical ( ) legislative ( ) market-based (*) vigilante ( ) lack of an

        approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

        ( ) Spammers can easily use it to harvest email addresses
        ( ) Mailing lists and other legitimate email uses would be affected
        (*) No one will be able to find the guy or collect the money
        ( ) It is defenseless against brute force attacks
        ( ) It will stop spam for two weeks and then we'll be stuck with it
        ( ) Users of email will not put up with it
        ( ) Microsoft will not put up with it
        (*) The police will not put up with it
        (*) Requires too much cooperation from spammers
        (*) Requires immediate total cooperation from everybody at once
        ( ) Many email users cannot afford to lose business or alienate potential employers
        ( ) Spammers don't care about invalid addresses in their lists
        ( ) Anyone could anonymously destroy anyone else's career or business

        Specifically, your plan fails to account for

        (*) Laws expressly prohibiting it
        ( ) Lack of centrally controlling authority for email
        (*) Open relays in foreign countries
        ( ) Ease of searching tiny alphanumeric address space of all email addresses
        (*) Asshats
        (*) Jurisdictional problems
        ( ) Unpopularity of weird new taxes
        ( ) Public reluctance to accept weird new forms of money
        ( ) Huge existing software investment in SMTP
        ( ) Susceptibility of protocols other than SMTP to attack
        ( ) Willingness of users to install OS patches received by email
        ( ) Armies of worm riddled broadband-connected Windows boxes
        (*) Eternal arms race involved in all filtering approaches
        (*) Extreme profitability of spam
        (*) Joe jobs and/or identity theft
        ( ) Technically illiterate politicians
        (*) Extreme stupidity on the part of people who do business with spammers
        (*) Dishonesty on the part of spammers themselves
        ( ) Bandwidth costs that are unaffected by client filtering
        ( ) Outlook

        and the following philosophical objections may also apply:

        (*) Ideas similar to yours are easy to come up with, yet none have ever
        been shown practical
        ( ) Any scheme based on opt-out is unacceptable
        ( ) SMTP headers should not be the subject of legislation
        ( ) Blacklists suck
        ( ) Whitelists suck
        ( ) No-lists suck
        ( ) We should be able to talk about Viagra without being censored
        ( ) Countermeasures should not involve wire fraud or credit card fraud
        (*) Countermeasures should not involve sabotage of public networks
        (*) Countermeasures must work if phased in gradually
        ( ) Sending email should be free
        (*) Why should we have to trust you and your servers?
        ( ) Incompatiblity with open source or open source licenses
        ( ) Feel-good measures do nothing to solve the problem
        ( ) Temporary/one-time email addresses are cumbersome
        ( ) I don't want the government reading my email
        ( ) Killing them that way is not slow and painful enough

        Furthermore, this is what I think about you:

        (*) Sorry dude, but I don't think it would work.
        ( ) This is a stupid idea, and you're a stupid person for suggesting it.
        ( ) Nice try, assh0le! I'm going to find out where you live and burn your
        house down!
        • Hahahah! If I may rebute:

          Tracability of the spammer always comes down to money. Money can easily be traced. Not every spammer would be nailed...just the big dogs.

          Making a company responsible for their marketing avenues would easily stop spamming.

          This would be stricly legal with no countermeasures involved from the tech community. We would provide answers and avenues to help the law track down some email messages.

    • Obligatory (Score:4, Insightful)

      by jonfelder ( 669529 ) on Monday May 31, 2004 @04:16PM (#9298037)
      Your post advocates a

      (*) technical ( ) legislative ( ) market-based ( ) vigilante ( ) lack of an

      approach to fighting spam. Your idea will not work. Here is why it won't work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

      ( ) Spammers can easily use it to harvest email addresses
      (*) Mailing lists and other legitimate email uses would be affected
      ( ) No one will be able to find the guy or collect the money
      ( ) It is defenseless against brute force attacks
      ( ) It will stop spam for two weeks and then we'll be stuck with it
      (*) Users of email will not put up with it
      (*) Microsoft will not put up with it
      ( ) The police will not put up with it
      ( ) Requires too much cooperation from spammers
      (*) Requires immediate total cooperation from everybody at once
      (*) Many email users cannot afford to lose business or alienate potential employers
      ( ) Spammers don't care about invalid addresses in their lists
      ( ) Anyone could anonymously destroy anyone else's career or business

      Specifically, your plan fails to account for

      ( ) Laws expressly prohibiting it
      (*) Lack of centrally controlling authority for email
      ( ) Open relays in foreign countries
      ( ) Ease of searching tiny alphanumeric address space of all email addresses
      ( ) Asshats
      ( ) Jurisdictional problems
      ( ) Unpopularity of weird new taxes
      ( ) Public reluctance to accept weird new forms of money
      (*) Huge existing software investment in SMTP
      (*) Susceptibility of protocols other than SMTP to attack
      ( ) Willingness of users to install OS patches received by email
      ( ) Armies of worm riddled broadband-connected Windows boxes
      ( ) Eternal arms race involved in all filtering approaches
      ( ) Extreme profitability of spam
      ( ) Joe jobs and/or identity theft
      ( ) Technically illiterate politicians
      ( ) Extreme stupidity on the part of people who do business with spammers
      ( ) Dishonesty on the part of spammers themselves
      ( ) Bandwidth costs that are unaffected by client filtering
      ( ) Outlook

      and the following philosophical objections may also apply:

      (*) Ideas similar to yours are easy to come up with, yet none have ever
      been shown practical
      ( ) Any scheme based on opt-out is unacceptable
      ( ) SMTP headers should not be the subject of legislation
      ( ) Blacklists suck
      ( ) Whitelists suck
      ( ) No-lists suck
      ( ) We should be able to talk about Viagra without being censored
      ( ) Countermeasures should not involve wire fraud or credit card fraud
      ( ) Countermeasures should not involve sabotage of public networks
      (*) Countermeasures must work if phased in gradually
      ( ) Sending email should be free
      (*) Why should we have to trust you and your servers?
      ( ) Incompatiblity with open source or open source licenses
      ( ) Feel-good measures do nothing to solve the problem
      ( ) Temporary/one-time email addresses are cumbersome
      ( ) I don't want the government reading my email
      ( ) Killing them that way is not slow and painful enough

      Furthermore, this is what I think about you:

      (*) Sorry dude, but I don't think it would work.
      ( ) This is a stupid idea, and you're a stupid person for suggesting it.
      ( ) Nice try, assh0le! I'm going to find out where you live and burn your
      house down!

  • Spam is in our culture to stay (Score:4, Insightful)

    by Jotaigna ( 749859 ) <jotaigna@yahoo.com> on Monday May 31, 2004 @02:24PM (#9297446) Homepage Journal
    unless mail sending protocol is redesigned(for example,in a way you have to have your fingerprints recognized when you type it) we will have to face the fact SPAM will be in our daily news. Soon slashdot will put an article where the best 3 spam filters are compared, like a normal review.

  • hmmm (Score:2, Informative)

    by FS1 ( 636716 )
    Been looking for a new spam filter, hope this one does the trick. I tend to have alot of false positives with most spam filters i have tried. I would rather have a few spam slip through rather than having to weed through all my spam just because it may have blocked a real email.

  • Is it easy to setup? (Score:4, Insightful)

    by Trashman ( 3003 ) on Monday May 31, 2004 @02:27PM (#9297468)
    I tried to setup spamassasin a couple of months back and I found it to be too much of a hassle to setup. Could someone who used both spamassasin and dspam comment on easy or difficult it is to setup dspam?
    • Well, using Gentoo Linux and evolution you don't really need to do too much in the way of configuring... I just emerged the package and added a piped filter rule to evolution. Unfortunately, it didn't seem very usable to me... no easy way to train it from within evolution, and it was taking like one to three seconds per message to process, which is kind of frustrating when your account tends to receive 80+ spams a day. (I know, that's still fairly minor, but that gives me like a 100:1 spam to real mail ra
      • While I agree with you that it does lack integration with Evolution (I have a similar setup to yours), regarding the time it takes to process each message - you can add '-local' to your SA commandline and it will speed things up considerably. As far as training it, I set up a cron job to have it read and learn from my Spam and Inbox nightly. Not the most elegant solution, but it works okay.

      • Re:Is it easy to setup? (Score:4, Insightful)

        by petabyte ( 238821 ) on Monday May 31, 2004 @03:02PM (#9297645)
        Are you piping it to spamassassin or spamc? If you have that much email it might make sense to run the spamd server (which is basically just spamassassin running all the time so you don't have to wait for it to start) and pipe the message to spamc to do its magic (the filter works the same). My advice is if you are really getting that much email, use spamd.

        Also it is posible to train spamassassin in evolution fairly easily. All you have to do are change two of the labels in evolution to "Ham" and "Spam". Then write 2 filter rules, 1 that says if its labeled "Ham" pipe it to sa-learn --ham; and another for "Spam" that does sa-learn --spam. Then you just change the label on the email you want to be spam, and apply filters to the message. There's a site on the web that has screenshots to go along with this but I can't find it at the moment.
    • Easy is a relative term, but I think it's safe to say that you found spamassasin a hassle, you will not have an easy time with DSPAM.

      Like most good server-side software, it requires a moderately good understanding of it's general operation and at least a passing familiarity with its command line arguments and such. Having a handle on how to make your MTA do whatever you want, and the willingness to do some reading of faqs, mailing lists etc doesn't hurt either.

      In short, it's does take some mucking aro

  • It takes a lot to train (Score:5, Informative)

    by smartin ( 942 ) on Monday May 31, 2004 @02:33PM (#9297502)
    Warning, it seems to be designed more for high volume use than individual sites. I've fed dspam almost 3000 spams and it is still only catching 80%, does seem to be getting better though.
    • Otherwise your weights will be all wrong.

      Equal parts ham and spam will yield good spam catching. RTFAQ [nuclearelephant.com].

  • Content based filters and spam (Score:3, Interesting)

    by Senior Frac ( 110715 ) on Monday May 31, 2004 @02:35PM (#9297518) Homepage

    I have not actually used DSPAM, but have just read the specs.

    Yawn. Yet another, albeit well designed, content-based filter. While content-based filters are a valuable tool, let's not forget that the spam problem is one of anti-social behavior and consent and has nothing to do with content. Using content as a factor in deciding what is spam or not spam will always be flawed. Even if you tweak your favorite filter from 99% to 99.9%, the spammers can just up the ante by sending more. Scaling up costs them little on an individual basis. It saddens me to see really brilliant people put great amounts of work into a project whose underlying premise is flawed.

  • a True AntiSpam measure (Score:4, Insightful)

    by CodeTRap ( 176342 ) on Monday May 31, 2004 @02:49PM (#9297579) Homepage
    would be to publicly humiliate/boycott the companies that use the spammers services. Like drug dealers, as long as there is a market, the spammers will be around. Remove the demand, and the suppliers will eventually move onto selling something else.

    If you can't kill the leeches because the water is too murky, then boil off the pond!
    • Oh great, and we all know how well the war-on-drugs is going..

      Oh ps, incase you haven't noticed those companies who push their products thru spam-vertising , arnt really well thought off, or well known to begin with (I for one never recognised any brand names for the verbal viagra or penis enlargers they sell, i'd hope the same goes for everyone!)
        • verbal viagra

        Maybe my spam-filter hooked that one, but I can't ever recall seeing advertisement for verbal viagra!

        Karma be damned: Please post, I can't wait!

    • Damn, I've been boycott Nigerian scammers for years and I still get those in my inbox. :( Obviously not the best solution, is it? (it's not like well-established businesses are spamming anyway).
    • Nothing will eliminate all spam. Period.

      That said, there are certain courses of action that would be quite effective against certain types of spam. For example, consider the dozen or so mortgage applications that arrive each day with specific promises along the lines of "$200,000 mortgage for $350 regardless of credit". If I were to reply some loan officer somewhere would presumably call me back. If said officer were required by law to give me $200,000 at $350/month you can believe that word would qui

    • With drugs, there is a huge demand on the part of the consumer. People will pay great amounts of money to get drugs. That gives big incentives for suppliers. No matter how hard it is to get the drugs in and how much they end up costing, you can almost be assured that someone will buy them.

      Not the case with SPAM. There is, in fact, basically zero consumer demand for SPAM. I have never met a person that demands they get e-mail advertisments and would pay to do so. In factm everyone I know (tech savvy or not)

  • So how does this help me reduce the ... (Score:4, Insightful)

    by Skapare ( 16644 ) on Monday May 31, 2004 @02:52PM (#9297596) Homepage

    So how does this help me reduce the amount of bandwidth and server resources used by spammers who continue to try sending spam to me and my users?

  • I wanted to try DSPAM some time ago, but I stopped as soon as I read that DSPAM puts an ID string in every mail it processes. In the mail body, that is. I have no problems with a program that adds headers, but it should leave the message body alone.

    Does DSPAM do that now? Can't find anything about it...

    • DSPAM ID (Score:4, Informative)

      by XanC ( 644172 ) on Monday May 31, 2004 @03:20PM (#9297739)
      DSPAM uses the ID string because people send corrections by forwarding mail to a certain address. Other filters require you to move mail to a Spam folder, but that requires a fairly specific configuration (you must use IMAP, filtering mail gateways are difficult, etc).

      You can configure DSPAM to not use the ID, but this requires users to "bounce" the incorrect e-mails instead of forwarding them (as forwarding strips the headers).

      Is the ID really that inconvenient?

      • For one, I can imagine it to cause problems with PGP-signed mails... Or just be unreadable at all for some reason. You don't have problems like that when you just insert a little header.
      • So how long will it be before spammers start inserting bogus DSPAM ID strings in an attempt to break the system?

        I understand the need for the ID, and have no qualms about it being used to identify my usage, but it does seem like a potential point of attack for "the other side."

        -ch
    • You can now set DSPAM to add headers with signatures etc instead of a tag in the body.

      The only thing to note is that users forwarding mail back to DSPAM for training must include the X-DSPAM headers. Apparently, some email clients do not do this by default.

  • K9 (Score:2, Informative)

    by Anonymous Coward
    An excellent spam filter for Windows is K9 found here. [keir.net]
  • I've been running DSpam for several months now and have found it works much better than Spam Assassin at catching spam. Furthermore, unlike SA, I have yet to get any false positives.

    My only problem is DSpam was not easy to set up with Postfix, at least for me since I'm not an experienced mail administrator. While I now have it mostly working, I have not been able to get the alias accounts working so I can forward missed spams for automatic learning.

    I look forward to upgrading to DSpam 3.0 when it is ful

  • Other excellent filters (Score:3, Interesting)

    by bigberk ( 547360 ) <bigberk@users.pc9.org> on Monday May 31, 2004 @03:47PM (#9297862)
    Others I've had direct experience with are spamprobe [sourceforge.net], spambayes [sourceforge.net], and CRM114 [sourceforge.net].

    My best experience has been with spamprobe, because it compiles as a standalone app, is very fast (at one point I was filtering over 10,000 emails a day on a Pentium 200 MHz) and is completely command-line oriented, best for scripting/custom mail systems. Colleagues of mine who use CRM114 are very happy with it, but I got discouraged by its large database files. I'm now experimenting with spambayes, the only difficulty so far being installing the python/bsddb environment.

  • The problem (Score:3, Interesting)

    by IGnatius T Foobar ( 4328 ) on Monday May 31, 2004 @04:02PM (#9297940) Homepage Journal
    ...is that spammers have access to the anti-spam tools.

    They have access to DSPAM. They have access to SpamAssassin. They have access to the Bayesian filters found in Mozilla and other products.

    When crafting their spams, they run them through these tools, and they keep obfuscating their spams until they get one through. Once they've got it perfect, they send a hundred million copies out to the world, and whammo! Your mo.rt-gage has been ap.prov/ed, and your v1ag---ra is ordered!
    • That hasn't been my experience, in nearly a year of using DSPAM. If the technique worked with DSPAM (as it does for SpamAssassin), I'd be seeing a lot more of the junk in my inbox. I'm not.

      I'm actually surprised on those one or two occasions per month when I find spam in my inbox.

  • Problems with DSPAM (Score:4, Informative)

    by gonz ( 13914 ) on Monday May 31, 2004 @04:28PM (#9298093)
    I've been using DSPAM for about three months. A few criticisms:

    First, by default DSPAM wants to run as the "root" user and usurp delivery of e-mails. (With Exim, they actually want it to recursively reinvoke the mail server for actual delivery!) It took quite a bit of configuring to get it to work like SpamAssassin from procmail.

    This software is somewhat buggy, so running DSPAM as root would also introduce security concerns. For example, I'm using 2.10.6 because the 3.0.0 compiled and installed with no problems, but failed to classify anything. (Even with several hours of gdb tracing I was unable to determine why). Another bug is that if I run the "--falsepositive" on an e-mail that's lacking the "!DSPAM" signatures, the message should be ignored, but apparently this is not the case because the statistics counters are incremented.

    From the FAQ:
    "Q. Does DSPAM support whitelists?
    A. DSPAM doesn't have a whitelist manager, rather whitelisting is an automatic function of DSPAM's Bayesian filtering mechanism."

    This is crazy -- the whole point of whitelists is for when the Bayesian filtering fails! And DSPAM does fail. Twice now I've had to reset my database because the classifications were wrong and training wasn't helping. All I can say is I'm glad I've got procmail to rescue the important e-mails.

    I think one source of my problems was that the default training mode ("train on everything") causes incorrect learning when you fail to report a false positive. This was a big problem for me, since I get around 700-800 spams/day. While false negatives are easily caught, the false positives go unnoticed unless I happen to wonder why someone never responded, and invest some time to search my spam folders. (I'm still trying to figure out exactly how to deal with this problem. E.g. maybe I could have it challenge the sender with Turing Test or something.)

    I will say that DSPAM's basic technology is quite good. It's just that the software still has a "prototype" feel, and I'd caution you to do some experiments before unleashing it on your users. (For example, there's no manpage, and there isn't even a command-line option to print out the current version number!)

    -Gonz

  • As time goes on DSPAM (and SpamAssassin for that matter) become more and more sophisticated, incorporating more complex algorithms. What I also find striking is that many of these algorithms appear to be compute intensive. These spam filters seem to be designed for server side ISP level email filtering. I would expect that a computer would have to be dedicated to running this anti-spam software.

    Also, as a number of posters have noted, configuring these spam filters takes some effort and education on

  • Coincidence? (Score:3, Interesting)

    by StormReaver ( 59959 ) on Monday May 31, 2004 @05:52PM (#9298749)
    Since this is a spam subject, this is at least partly relevant:

    I am a Direcway subscriber, and I was accustomed (angry, but accustomed) to receiving about 15-20 spams per day for as long as I can remember.

    Slashdot ran a story within the last 6 months (I don't remember which one exactly) about the FBI raiding one or two of the largest spammers and confiscating their setup.

    Almost to the day that the raid was to have occurred, all spam to my inbox instantly stopped. I haven't gotten a single spam message since the about the same time as the second raid.

    It seems to me that those guys may have been the sole sources of all the spam going through Direcway to my account. Are there any other Direcway subscribers here that had the same experience, was the whole thing just an extraordinary coincidence, or did Direcway find the holy grail of anti-spam?

    As far as I can tell, all my regular email is getting through and going out. No email that I knew was coming has yet failed to arrive, so any filtering at Direcway's servers, if such a tactic is being employed, is doing a great job.
    • It seems to me that those guys may have been the sole sources of all the spam going through Direcway to my account. Are there any other Direcway subscribers here that had the same experience, was the whole thing just an extraordinary coincidence, or did Direcway find the holy grail of anti-spam?

      I would agree that those guys were probably the sole sources of your spam, but it has nothing to do with DirecWay, and you won't see any pattern comparing to other DirecWay subscribers. Spammers don't care what IS
  • For the anti SPAM system that returns it to sender, prints it out and shoves it up his ass.

    Now that will, by god, cut down on SPAM.

Slashdot Top Deals

Good day to avoid cops. Crawl to work.

Close