Stories
Slash Boxes
Comments

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Copyright Tool Scans Web For Violations

Posted by Zonk on Tue Dec 19, 2006 11:31 AM
from the he-knows-when-you've-been-bad-or-good dept.
The Wall Street Journal is reporting on a tech start-up that proposes to offer the ultimate in assurance for content owners. Attributor Corporation is going to offer clients the ability to scan the web for their own intellectual property. The article touches on previous use of techniques like DRM and in-house staff searches, and the limited usefulness of both. They specifically cite the pending legal actions against companies like YouTube, and wonder about what their attitude will be towards initiatives like this. From the article: "Attributor analyzes the content of clients, who could range from individuals to big media companies, using a technique known as 'digital fingerprinting,' which determines unique and identifying characteristics of content. It uses these digital fingerprints to search its index of the Web for the content. The company claims to be able to spot a customer's content based on the appearance of as little as a few sentences of text or a few seconds of audio or video. It will provide customers with alerts and a dashboard of identified uses of their content on the Web and the context in which it is used. The content owners can then try to negotiate revenue from whoever is using it or request that it be taken down. In some cases, they may decide the content is being used fairly or to acceptable promotional ends. Attributor plans to help automate the interaction between content owners and those using their content on the Web, though it declines to specify how."
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • Wager (Score:4, Insightful)

    by Baricom (763970) on Tuesday December 19 2006, @11:33AM (#17300868)
    Anybody care to place a friendly wager that they're not going to honor robots.txt?
    • Raise. (Score:4, Funny)

      by Tackhead (54550) on Tuesday December 19 2006, @11:44AM (#17301006)
      > Anybody care to place a friendly wager that they're not going to honor robots.txt?

      127.0.0.1: $ cat robots.txt
      # robots.txt for 127.0.0.1
      # This file is copyright 2006 by me.
      User-agent: AttributorCorporationDMCABot
      Disallow: *

      And if they do honor robots.txt, I'll be able to sue the fuckers for infringing on my copyright, because they must have read it in order to honor it.

      • Good luck with that.

        Unless you also sell a few companies and put together a few billion as a stake to hand over to attorneys I suspect you'll fare as poorly as everyone else does.

      • Re: (Score:2, Insightful)

        You know, I've actually had a thought along those lines in trying to explain to untechnologically savvy individuals why Digital Rights laws are screwed up and that handling digital content on the web is a grey area. Consider the following.

        Most web sites have a copyright statement on them some where (even this one!). Technically speaking, if I go to that web site, my browser copies the page along with all it's media content and caches it. Since many of those sites do not have a terms of service posted
      • Reading something does not violate its copyright. If they distribute copies of robots.txt you might have a case of some sort.
        • Reading something does not violate its copyright. If they distribute copies of robots.txt you might have a case of some sort.

          how can you read it on the web then without having made a copy of it somewhere on your computer... you've pulled in a copy of it using your browser, there is now a copy of it in ram and also maybe in the cache... so you've made at least two unauthorised copies.

      • Re:Raise. (Score:5, Funny)

        by Mayhem178 (920970) on Tuesday December 19 2006, @12:12PM (#17301304)
        127.0.0.1: $ cat robots.txt
        # robots.txt for 127.0.0.1
        # This file is copyright 2006 by me.
        User-agent: AttributorCorporationDMCABot
        Disallow: *


        Hahaha! You screwed up! I have your IP address now! I will send 127.0.0.1 to every company that uses the sniffer and tell them the person at that IP is an evil, evil person who exploits innocent people for their own profit and power!
      • Re:Raise. (Score:4, Interesting)

        by FooAtWFU (699187) on Tuesday December 19 2006, @12:32PM (#17301496) Homepage
        You joke, of course, of course, but there are tools out there to detect when a bot is abusing your site and not following robots.txt. The usual technique is to hide a few links in your page, and also have these links blocked by robots.txt. When a user visits the link, they're banned from viewing the site. (Sometimes, a CAPTCHA-like utility for unblocking yourself is presented along with the 403 page, in the event that a particularly curious user manages to find the link and activate it manually.)
        • Re: (Score:3, Insightful)

          True, but there's a way around that as well. Any robot service worth its weight in fiber has more than one IP, and can have multiple subnets. Best way is to dump robots.txt links to a separate subnet, have it check later in the day. If the IP gets banned, it can check by trying to access the main page, see if it starts getting errors. It can then mark "booby-trap" sites on a list, and route around either the specific triggers or actually honor the robots.txt.

          You have to have more links than they have IP
    • Another company "Cyveillance" already does this for major corporations and the government. I've used htaccess rules to disallow all from their assigned netblocks after they racked up almost 20,000 hits to my personal site in one day. As you mentioned, they didn't follow robots.txt and attempted to index parts of my site that are password protected as well as content names that did not exist (music and videos and such), all the while identifying their bot as a variant of IE.

      Here's how to block two subnets
      • Re: (Score:3, Informative)

        There's an easier way. You can hand mod_access netblocks and more [apache.org]. This method will avoid eating cycles with mod_rewrite. If you can put it in your conf instead of .htaccess, you'll save even more time/processing. Just put it in for your doc root. From my httpd.conf:

        <Directory "/var/www/htdocs/">
        # BRYN'S DENIALS
        # allresearch.com
        deny from 209.73.228.160/28
        # branddimensions.com user-agent: BDFetch
        deny from 204.92.59.0/24
        # cyveillance.com
        deny from 63.148.99.224/27
        deny from 65.118.41.192/27
        # www.mar

      • by FooAtWFU (699187) on Tuesday December 19 2006, @11:57AM (#17301150) Homepage
        You're absolutely right that "if you don't want it on the public Web, don't put it there in the first place" -- but there are still times when you have a legitimate reason that you don't want a page indexed, downloaded, or otherwise visited by a robot. Dynamically generated content is one example reason; sometimes certain pages can be a big drain on your website, and you'd prefer not to have every spider in the world hitting them up every few minutes.

        Let's take a fun legitimate site like, oh... Wikipedia [wikipedia.org]:

        # Folks get annoyed when VfD discussions end up the number 1 google hit for
        # their name. See bugzilla bug #4776
        # en:
        Disallow: /wiki/Wikipedia:Articles_for_deletion/
        Disallow: /wiki/Wikipedia%3AArticles_for_deletion/
        Disallow : /wiki/Wikipedia:Votes_for_deletion/
        Disallow: /wiki/Wikipedia%3AVotes_for_deletion/
        Disallow: /wiki/Wikipedia:Pages_for_deletion/
        Disallow: /wiki/Wikipedia%3APages_for_deletion/
        Disallow: /wiki/Wikipedia:Miscellany_for_deletion/
        Disallow : /wiki/Wikipedia%3AMiscellany_for_deletion/
        Disall ow: /wiki/Wikipedia:Miscellaneous_deletion/
        Disallow: /wiki/Wikipedia%3AMiscellaneous_deletion/
        Disallo w: /wiki/Wikipedia:Copyright_problems
        Disallow: /wiki/Wikipedia%3ACopyright_problems
        (They also disallow certain specially generated pages like Special:Random, and any of the pages which actually let you edit the site).

        Let's see, what are some other sites? Ooh. Take a look at Slashdot's robots.txt [slashdot.org]! (disallows a variety of fun pages.) Microsoft's? [microsoft.com] How about whitehouse.gov [whitehouse.gov]? Google [google.com]?

        • by mandelbr0t (1015855) on Tuesday December 19 2006, @01:32PM (#17302496) Journal

          Dynamically generated content is one example reason; sometimes certain pages can be a big drain on your website

          And dynamic content is, of course, the answer. If I'm going to put up copyrighted content in the future, I'd use one of a dozen schemes that regenerate the download link on a per-session basis. Obviously they're not going to honour robots.txt, but why are your links readable by such a basic spider? You need to:

          1. Disallow anonymous downloads. You need to be logged onto the site to download anything, torrent or otherwise
          2. Use a CAPTCHA to prevent spiders from signing up for said accounts
          3. Use the session id to generate unique download links on a per-session basis
          4. Change the key on your BitTorrent tracker every 12-24 hours. This will require that a downloader get the latest torrent from the original website (which requires login), reducing the impact of a leaked torrent
          5. Compress and possibly encrypt the content so that it's less obvious what it is

          Anyone who follows the above steps (and most sites already do most or all of this) won't be found by the spider. Period.

          The only thing I can think of that this product would be useful for is to find people who have blatantly copied my website, but I'm sure you could find those people equally easily with Google.

          mandelbr0t

  • by LiquidCoooled (634315) on Tuesday December 19 2006, @11:33AM (#17300870) Homepage Journal
    Can't they just use google or torrent sites?
    If users can find items they want, presumably the copyright holders could use the same methods...
    • And the opposite situation shows why this tool is a waste of time.

      Imagine a tool where you could reliably return accurate and search results for images and video. Does this exist yet? No, as one who searches the web daily for pics and video for my own sordid uses, let me assure you that it most certainly does not yet exist.

      And what an horrific waste to have such a tool - if it works - for policing content for copyright violations. Bearing in mind also that such "violations" are no such thing in some
      • As always, and tell your family and friends, only buy music directly from the artist or secondhand. It's the only way to win.

        or else make it yourself... but then again you've got to pay the nickel for the bl00dy sheet music or tabs... and they don't half try to rip you off there as well... it's that or write your own... and then try and stop them from ripping you off...

  • buh (Score:5, Insightful)

    by lucky130 (267588) on Tuesday December 19 2006, @11:36AM (#17300910)
    "as little as a few sentences of text or a few seconds of audio or video"

    Like quotations in a paper, or video snippets in an educational presentation?
      • Re:buh (Score:5, Insightful)

        by NeutronCowboy (896098) on Tuesday December 19 2006, @12:38PM (#17301574)
        You're assuming anyone is going to manually verify any of the results. From my experience with people using monitoring software (especially non-techies who are simply consumers of the technology, but who provided the money for it), the vast majority of them are simply going to call their lawyers when they see the dashboard light up. I see vast letter writing campaigns come from this, with little actual infringing being prosecuted.

        This is a scary product. Not so much because of the technology behind it, but because of how it is going to be implemented and (ab)used.
  • Seems like spam obfuscation techniques will be useful against this sort of scan, too, if someone really wanted to infringe on copyright.
  • The Wall Street Journal is reporting on a tech start-up that proposes to offer the ultimate in assurance for content owners.
    This almost had me going until the second half of the sentence. When has anyone ever offered any product as the "ultimate" anything that ultimately proved to actually ultimately be the ultimate whatever it was?
  • by TheWoozle (984500) on Tuesday December 19 2006, @11:42AM (#17300994)
    Doesn't this merely serve to point out the absurdity of "Intellectual Property"?
    • TheWoozle,

      Today's world of copy protection is voluntary. You have the right to produce content that people want and to waive copyright on it. That's your free choice. Are you doing that? If not, then why not?
  • Yeah (Score:4, Interesting)

    by Hijacked Public (999535) on Tuesday December 19 2006, @11:45AM (#17301020)
    FTFA:

    If it works, it's a fantastic invention


    Its purpose aside, yes, it would be a fantastic thing to be able to scan the entire web and reliably identify the context and content of any specific media file type. Video, audio, image, etc. Particularly if it could identify purposely obfuscated content.

    I'm in what is almost certainly a tiny minority of Slashdotters in that I actually create copyrightable material rather than only consume it. I'm again in the minority in that I think copyrights are a good thing and again in the minority in that I can separate out the purpose of copyrights and the evil actions of the legal arms of **AA companies.

    Regardless, while scanning the internet for improperly used material sounds great on paper this will probably end up being as effective as finding water with a divining rod. The current tactic of locking down things at the hardware and OS levels will get more support from the media companies, not that they seem all that good at choosing tactics when the internet is involved.

    • Re:Yeah (Score:4, Insightful)

      by jedidiah (1196) on Tuesday December 19 2006, @11:59AM (#17301166) Homepage
      There's a wide gulf between copyright being a good idea in concept and being sensibly implemented in it's current form.

      Not everyone that creates content thinks that draconian enforcement attempts are a good idea, or even in the best interests of those that create content.

      If your work can't survive in the marketplace, which includes the prospect of everyone on the planet getting to use it for free, then perhaps you should get some sort of more conventional day job.

      The difference between a game that sells 50K and one that sells 5 Million has nothing to do with DRM.
    • Re:Yeah (Score:4, Interesting)

      by AdamKG (1004604) <slashdot@nOspAM.adamgomaa.com> on Tuesday December 19 2006, @12:17PM (#17301346) Homepage
      and again in the minority in that I can separate out the purpose of copyrights and the evil actions of the legal arms of **AA companies.
      Let's make one thing clear: the RIAA/MPAA lawsuits are not, in any way, shape, or form, an abuse, negative side of, misapplication or malicious use of Copyrights. They fulfill the role of Copyrights in the first place; they are the logical end result of a system that says citizens are allowed to distribute ideas (or expressions of ideas), then stop any further distribution of them.

      The **AA lawsuits are ridiculous, yes. But the ridiculous part is not the litigation itself, it's the laws on which the lawsuits are brought under.
    • Re: (Score:3, Interesting)

      I'm a patent attorney and no stranger to IP. Having said that, any IP law is, or at least should be, a balance to on the one hand freedom to operate (both for IP users and for IP creators) and on the other hand a means for compensation for IP creators. For patents, that balance is not there for patents on software. Also for patents, at least they last for 20 years max. For copyright, that balance is not there. And I'm curious to hear whether you think it is a good thing that whatever you create is still und
  • roughly equal to the entire volume of the publically available internet..

    think about it, to do what they say, they have to request ALL the data they can lay their hands on,
    and then chuck it.. and for comparative purposes, they'll have to do it again.

    so Sony hires 'jfm copyright trackers'
    and microsoft hires 'sco copyright trackers'
    and mgm hires yo momma

    and each of these 'ip owners' representatives have to scour the entire net, bit by byte by megabyte, for their clients.

    holy crap! think about the potential
  • by Weaselmancer (533834) on Tuesday December 19 2006, @11:46AM (#17301030)

    Attributor plans to help automate the interaction between content owners and those using their content on the Web, though it declines to specify how.

    And apparently being written by underpants gnomes.

  • by PingSpike (947548) on Tuesday December 19 2006, @11:46AM (#17301032)
    Great, now all the torrent sites will require captcha verification too! ;P

    Actually, can they even scan torrents without downloading the entire file? And whats to stop everyone from just blocking them from accessing their websites? Are they going to go in covertly, pretending to be actual users? I can see every legit website blocking their access as well, why pay for bandwidth to supply that?

    Sure, youtube can be more efficiently attacked...but youtube has been dancing in front of the cannons since its inception, we all knew it was going to get shot eventually.
  • This must be really essential bussiness software. It has a Dashboard! Wanna bet the next version is SOA enabled?
  • search by hash? (Score:4, Interesting)

    by straponego (521991) on Tuesday December 19 2006, @11:47AM (#17301040)
    Does Google allow searching by md5sum or equivalent? I'm sure they have the capability. While not as impressive as what this company claims, it'd also be more reliable for unaltered media files.

    But it looks like the real "innovation" these guys are pushing toward is fully automated filing of lawsuits. I think that was in Accelerando, which is fantastic, and which you can download it free. [accelerando.org]

    • Re:search by hash? (Score:4, Informative)

      by Johann Lau (1040920) on Tuesday December 19 2006, @12:45PM (#17301650) Homepage Journal
      "Unaltered media files" are the exception, not the rule. Changing even a bit of metadata (stripping exif from an image, changing an mp3 tag) would change the checksum, not to mention things like putting things into an archive, resizing images, (re)recompressing music.

      But yeah, it might make sense for Google to become "aware" of unique content and variations of it.. but I doubt they'd ever use that openly for (aiding in) hunting down copyright infringement, simply for PR reasons.
      • Re: (Score:3, Interesting)

        Hm, what about computing checksum of the actual media contents? For example, compute checksum only for sound data in MP3 or image data in image files, ignore all other data/metadata. Usualy media files are containers for smaller objects or data streams... Resampled or modified contents would not be detected though.
  • After all they just copied http://copyscape.com/ [copyscape.com] 's idea.

    • That's the first thing that came to mind when I saw the article. It's been around for years. I've used a it a few times and was amazed to find one of my random website texts in other peoples's work (It was properly cited so I don't complain).
  • Why the fuck does everyone want to be paid for every little thing these days? Sure, wholesale piracy is one thing. I disagree with the idea that people should be trading movies and music online with no restrictions at all. If you want an album, buy it. If you want software that costs something, buy it or learn to use free/open software. If you want to see a movie, pay to watch it in the theater or rent the DVD when it comes out. But, where this all falls apart is when someone quotes someone else onlin
    • If the industry had their way, rap music would have never happened

      I don't understand... your post seems to imply this is a Bad Thing?
    • If you want an album, buy it. If you want software that costs something, buy it or learn to use free/open software.

      So where's the free/open alternative to an album?

      Or... someone uses a popular song as the music bed in their Youtube video and the entire video clip is only 25 seconds long

      A ringtone is 25 seconds long, as that's how long it takes for the call to be routed to voice mail.

      or the quality is so poor that no one in their right mind would consider keeping it as something to put on their iPod.

      Over a mobile phone's ringer, quality matters little.

      Whatever happened to the concept of fair use and encouraging people to build upon the works of others?

      Sonny Bono happened [pineight.com].

  • As long as it respects basic internet rules of conduct (including respecting robots.txt), then this is ethically neutral.

    It all depends on how it's used. Many companies would prefer to avoid coypyright infringing material, and will take it down if the existence is pointed out to them. Many companies will simply be asking others to remove material which clearly and flagrantly breaches their copyright. This is perfectly reasonable behaviour.
  • Of course, "a few sentences of text or a few seconds of video" most likely are being used within legal fair use boundaries. So what's going to happen is that the corporate law firm will grab this program, then send out auto-takedown notices without a human being (to the extent anyone working in the legal department meets that criteria) ever looking to see if the use is even arguably a violation of copyright. Then you'll get the backlash where at least one such auto-generated letter makes its way to someone
  • by Anonymous Coward
    This may be much less helpful than its promoters claim.

    First of all, what's the their probability of a false alarm? Even if they false alarm fairly infrequently, the vast amount of content on the Web means they could easily have a flood of false alarms, in addition to whatever actual copies are found. The user of the system is then going to have to have human beings sift through that flood to identify what's A) really a copy, B) whether that copy is infringing or not, and C) if so, is it worth taking actio
  • Ok, it's supposed to be unlawful to access copyrighted information on the Internet without the copyright holder's permission, right? I mean, that's the gist of the *AA's arguments right -- we hold the rights, you can't access this material unless we say so. So if the tool has to access the information to determine the copyright, wouldn't it be violating that principle? Nitpicking I know, but an interesting thought. They'd have to get dispensation from the *AAs to do it, wouldn't they?

  • by Anonymous Coward

    ...then do not put it to the Internet.

    In fact, burn it to a DVD and lock it up to a safe, and never talk about it. That way nobody else will ever have access to your "intellectual property".

  • Proactive firewalls (IDS) properly configured should shut the "scan" down relatively quickly, no? Besides, if the service is provided by a specific location (IP block), then IP blocking is trivial.

    On another note, so now they are going to throw more traffic over the Internet? :P
  • This is the tool Micros - um, I mean - SCO has been waiting for. They can now just scan all those millions of Linux Servers on the intraweb and see their copyrighted code right there in the open....

    ...or maybe not.

  • Like there's any copyright infringement on The Interweb. I don't see how a whole book could fit in those tubes...
  • by bcrowell (177657) on Tuesday December 19 2006, @04:20PM (#17305144) Homepage

    I've experienced this from both sides.

    I have a bunch of my books on the web, and every once in a while I do a search on some text from my own books to see who else is mirroring them. The books happen to be copylefted (dual-licensed GFDL/CC-BY-SA), but I'd like to know who's mirroring them, and check whether they're violating the license. A lot of people just seem to be hoarding the PDF files on their university servers, maybe because they're afraid my web site will disappear; that's flattering. One guy was selling them on CDs on e-bay, violating my license (claimed they were PD, didn't propagate the license). Another guy translated them to html, with lots of errors, changed the license to a more restrictive one, and put his own ads up; he fixed the licensing violation when I complained, and in a way it was a good thing, because it motivated me to make my own html versions (which are now bringing me a significant amount of money from adsense every month). One kind of annoying thing about mirroring is that the people who are mirroring never bother to update their mirrors, but in general I just figure there's no such thing as bad publicity :-)

    From the other side, I once received an e-mail from a museum in the UK that was complaining that I was using a 17th century oil painting of Isaac Newton. I guess they own the original, and they may also have been the ones who did the scan that I found in a google image search, but under U.S. law (Bridgeman Art Library, Ltd. v. Corel Corp.), a realistic reproduction of a PD two-dimensional art work is not copyrightable. What really surprised me was that they came across it at all, because at that time I think my book was only in PDF format, and hadn't been indexed by google because the file size was too big.

    The whole thing doesn't seem negative to me in general. It makes just as much sense as people doing a vanity search in Google before they apply for a job, or authors watching their amazon.com sales rankings obsessively. I guess the most obvious potential for abuse would be if they send a nastygram to your webhost, and your webhost is a low-end one that figures it's not worth their time to keep your account, so they just shut off your account.

    • Re: (Score:3, Interesting)

      Pretty sure this is a dupe, or so closely related to an earlier story as to not matter.

      It's not a dupe. (Unless you count anything that appears on Digg first to be a dupe.) However, it's also not the first story of its kind. About a gazillion companies have formed with the exact same business plan (save for the "hotness" at the time being digital music) and about a gazillion of those companies have failed to develop software that catches anything but the most obvious infractions.

      Every so often, some RIAA/MP

    • Re:Dupe (Score:4, Interesting)

      by Maximum Prophet (716608) on Tuesday December 19 2006, @11:53AM (#17301106)
      Since copyright lasts a long time and doesn't depend on being defended like trademark, there will be some allowances "for promotional reasons" like this:
      1. Leak copywritten material in easy to copy format to places where it will be copied
      2. Watch viral marketing campaign take over
      3. Profit
      4. Wait 'til revenue falls
      5. Find infringers using new scan tools
      6. Sue them
      7. Profit more!!!
    • The editors could run this tool just on /. to check for dupes!