Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Microsoft Patents Your Rights Online

Microsoft Seeks Patent For "Search By Sketch" 104

Posted by samzenpus from the composite-search dept.
theodp writes "So, how does one search for images that aren't tagged with keywords? Google does offer its sometimes-spotty search by image, but what if you don't have an image handy that looks like what you're searching for? Microsoft, reports GeekWire, offers a solution that's 'a little like playing Pictionary with a search engine — drawing a sketch and seeing if the algorithm can return pictures that match it.' That's the concept behind Microsoft Research's patent-pending 'MindFinder' project, which has already been incorporated into a Windows Phone app called Sketch Match. A patent application made public Thursday notes that touch computing makes sketching easier than ever, making one wonder if we'll be 'giving Bing the finger' with Windows 8!"
This discussion has been archived. No new comments can be posted.

Microsoft Seeks Patent For "Search By Sketch" More

Microsoft Seeks Patent For "Search By Sketch"

Comments Filter:

  • Testing would be interesting (Score:5, Funny)

    by Rogerborg ( 306625 ) on Monday March 05, 2012 @08:08AM (#39246473) Homepage
    Could they make it work for the 5% of sketches that aren't boobs or dongs?

    • Re:Testing would be interesting (Score:5, Funny)

      by mwvdlee ( 775178 ) on Monday March 05, 2012 @08:12AM (#39246511) Homepage

      They'll just return the same nude picture on every search and claim 95% accuracy.

    • Go find out for yourself! Here's some prior art [systemone.at]. It's not very good (it doesn't understand the concept of edges like MS's stuff does), but unfortunately the article doesn't have a link to the actual patent to tell if it's specific enough... the link that should go to the patent goes to the app on the Windows Phone site. (Certainly, the algorithm is impressive, though.)

    • Could they make it work for the 5% of sketches that aren't boobs or dongs?

      This feature already exists in Digikam, from KDE:
      http://digikam.org/ [digikam.org]

      Interestingly, the feature works fine for things that aren't boobs or dongs, because boobs need circles and dongs need straight lines. It fails on normal geometric shapes, but works great on more abstract drawings.

      • Yes, and it works quite well I noticed.

        However, this patent will most likely include "in the cloud" and/or other buzzwords.

    • Comment removed based on user account deletion
  • ...don't most smartphones have cameras already? Wouldn't it be faster and easier for the user to just take a photo of whatever he/she wants to identify or search? (Of course, if they don't have a photo of the object available at the time, then this idea would make more sense.)

    • Re:Interesting concept, but... (Score:5, Informative)

      by zill ( 1690130 ) on Monday March 05, 2012 @08:21AM (#39246583)
      Sounds like you're talking about Google Goggles.
      • No, Google Goggles does nothing like this. Google Goggles (and Google search-by-image) is, from the experiments I've done, instance-based image retrieval. That is, it can match objects with exactly the same shape (given a picture of the Eiffel Tower, it will return other images of the Eiffel Tower). However, given a drawing, even a good one, the contour shapes won't match quite well enough, and the algorithm will return garbage. The same can be said for 'deformable objects' like dogs and people.

        In fa
  • Microsoft, once more, is one the very large corporations that still think they can base their business, in the 21st century, on patents. I wonder how long it is ( yet ) going to taken before corporations of this size realize that such practice is old-fashioned ?

  • Already Done (Score:5, Informative)

    by Anonymous Coward on Monday March 05, 2012 @08:11AM (#39246499)

    Already done here for LaTex symbols: http://detexify.kirelabs.org/classify.html

    Please please please don't approve the patent. I haven't read it, but according to the summary there's nothing new about it.

    • I agree... it really is just an image search anyways, it's just much looser in its algorithm

    • 3 minutes and we are up to 4 systems that show prior art already, do the Patent office actually do any searches ...?

      oh sorry this is "...on a mobile device" so they don't count ..?

    • Re:Already Done (Score:4, Insightful)

      by Miros ( 734652 ) on Monday March 05, 2012 @09:13AM (#39247045)
      You're citing prior art for the concept, which is not the same thing as prior art for what is actually being patented. There are 1000s of patents for "mousetraps." The fact that a mousetrap had been invented by someone first did not mean that other people could not invent other kinds of mousetraps. Just because people have built sketch based search technology before does not mean that Microsoft (or anyone else for that matter) can't invent an improved version of sketch based image search and then patent that method. That appears to be what they have done here.

  • My first thought (Score:3, Funny)

    by subreality ( 157447 ) on Monday March 05, 2012 @08:15AM (#39246539)

    Oh please [sketch sketch] let there be porn like this [scribble]

  • Already exists (Score:1)

    by Anonymous Coward

    That functionality was integrated a few years back into digikam.
    How can those corporations always get away with patenting stuff under prior art?

    • That functionality was integrated a few years back into digikam.
      How can those corporations always get away with patenting stuff under prior art?

      There are two ways.

      1. Simply ignore prior art and claim everything as your own invention.

      2. Make a trivial change and then claim that and bamboozle the patent office into believing you have therefore invented the wheel.

      • Re:Already exists (Score:5, Insightful)

        by Stewie241 ( 1035724 ) on Monday March 05, 2012 @09:25AM (#39247189)

        or
        3. Recognize that there is legitimate room within patents for doing something that has already been done but doing it in a non obvious, novel way.

        I'm not saying this is the case for this particular patent, but just pointing out that in my understanding patents aren't always about doing a certain thing but rather doing a certain thing in a certain way. Obviously this limits the scope of the patent to that one certain way.

        So, to put it in slashdot terms (i.e. a car analogy), pretty much every car sold today has shock absorbers, and the standard way of doing this is to use metal coil springs. If somebody were to put time and research into improving the design of shock absorbers and came up with a design that used marshmallows instead of metal coil springs, I would consider that novel and I think you would agree that it is non-obvious. To me, if we have the patent system that we have, that is at least a legitimate use of it. You aren't patenting shock absorbers, you are patenting shock absorbers that use marshmallows to absorb the shock of the impact.

        • Nerds! You made me unmod :(

          A shock absorber is a piston in a fluid column. It's force is dependent on the velocity through the working fluid.
          A spring on the other hand produces a force proportional to its displacement not its velocity.

          Confusion could arise because modern cars don't use separate units but instead use a strut assembly which is a shock + spring.

          • I think i this case his analogy would be replacing the fluid in the shocks/struts with marshmallows since the springs mainly is what maintains ride hight.

          • Nerds! You made me unmod :(

            A shock absorber is a piston in a fluid column. It's force is dependent on the velocity through the working fluid.
            A spring on the other hand produces a force proportional to its displacement not its velocity.

            Confusion could arise because modern cars don't use separate units but instead use a strut assembly which is a shock + spring.

            Nerds! You made me unmod :(

            A shock absorber is a piston in a fluid column. It's force is dependent on the velocity through the working fluid.
            A spring on the other hand produces a force proportional to its displacement not its velocity.

            Confusion could arise because modern cars don't use separate units but instead use a strut assembly which is a shock + spring.

            That's not the only possible kind of shock absorber. For instance, one could make magnetic shock absorbers that dissipate energy by moving a powerful magnet by a metal plate. Or you could have electromechanical shock absorbers that sense motion and acceleration and use fast-acting linear motors to actively absorb the shock. The energy could be captured and used to recharge the batteries in your car.

        • or
          3. Recognize that there is legitimate room within patents for doing something that has already been done but doing it in a non obvious, novel way.

          I'm not saying this is the case for this particular patent, but just pointing out that in my understanding patents aren't always about doing a certain thing but rather doing a certain thing in a certain way. Obviously this limits the scope of the patent to that one certain way.

          So, to put it in slashdot terms (i.e. a car analogy), pretty much every car sold today has shock absorbers, and the standard way of doing this is to use metal coil springs. If somebody were to put time and research into improving the design of shock absorbers and came up with a design that used marshmallows instead of metal coil springs, I would consider that novel and I think you would agree that it is non-obvious. To me, if we have the patent system that we have, that is at least a legitimate use of it. You aren't patenting shock absorbers, you are patenting shock absorbers that use marshmallows to absorb the shock of the impact.

          Most patents are about doing something a certain way. They ALL should be. But many patents and even more so claims of infringement involve unjustified extension of the patented invention to anything that's resembles its function at a much higher level.

          And now the patent office is allowing people to patent *ideas* that have never been reduced to practice, so that the invention that's patented can't actually be demonstrated until somebody else invents an enabling technology that doesn't exist yet. This onl

      • Re: (Score:2)

        by FreeUser ( 11483 )

        1. Simply ignore prior art and claim everything as your own invention.

        2. Make a trivial change and then claim that and bamboozle the patent office into believing you have therefore invented the wheel.

        You left out:

        3. Simply pay the US Patent Office Bribe, I mean "Fee". Patent approved, no questions asked.

        (And quietly, from one patent lawyer at the USPTO to another, at Corporation X, or defending defendent Y: go forth and make money!. We win, the only losers are the corporations who filed, and the innovato

  • And several other "search by sketch" technologies I've seen in the last decade or so...
    What's the big claim in the patent that differentiates it?

    • Re: (Score:2, Insightful)

      by DrSkwid ( 118965 )

      it used to be that they added "on the internet", now they add "on a mobile phone", looks like this one adds "with fingers on a mobile phone"

      • it used to be that they added "on the internet", now they add "on a mobile phone", looks like this one adds "with fingers on a mobile phone"

        Common misunderstanding, but no. This came up because many patent applications have dependent claims that may say things like "2. The method of claim 1, wherein the network is the Internet" but that's not independently claiming the Internet... Or even claiming that novelty is due to 'the internet'. Rather, it's a doctrine called claim differentiation: dependent claims are subsets of the independent claims from which they depend from. Like, if Claim 1 is a large box on a Venn diagram, then Claim 2 would be

  • Hopefully sketchers will be more accurate than hummers or whistlers -- a la Midomi.com. But I doubt it -- can't wait to see if this survives beta.

  • I remember seeing a research paper over 12 years ago describing exactly this concept.

  • Patent the concept or specific algorithm? (Score:5, Insightful)

    by rollingcalf ( 605357 ) on Monday March 05, 2012 @08:24AM (#39246605)

    I oppose software patents in general, but if this patent covers their specific algorithm I don't really have a problem with that.

    The trouble is that regardless of whatever algorithm they describe in the patent, the patent can be used to sue others who use a different algorithm to implement the same concept.

    So they can get a patent on an ineffective algorithm for image search, wait for somebody else to create a better algorithm that is actually effective, then sue the implementor of the better algorithm. The patent effectively covers the concept, not the algorithm, hindering innovation by preventing others from implementing their own different algorithms for the same concept.

    • The problem is that you can't actually patent an algorithm, there are laws that prevent that; but that does stop sleazy lawyers from working around that problem, and what that gives us is overly broad patents. If we let people patent algorithms, we might actually be better off.

    • I oppose software patents in general, but if this patent covers their specific algorithm I don't really have a problem with that.

      The trouble is that regardless of whatever algorithm they describe in the patent, the patent can be used to sue others who use a different algorithm to implement the same concept.

      So they can get a patent on an ineffective algorithm for image search, wait for somebody else to create a better algorithm that is actually effective, then sue the implementor of the better algorithm. The patent effectively covers the concept, not the algorithm, hindering innovation by preventing others from implementing their own different algorithms for the same concept.

      Not really... As others have pointed out, the general concept was already done by ImgSeek. So, Microsoft's patent will have to be narrower than the entire concept if they want it to be granted - they'll have to patent a specific implementation. That leaves others free to make different implementations, too. Additionally, if someone does use Microsoft's implementation but makes it better, more efficient, more effective, that someone can get a patent on that improvement, and Microsoft would have to pay them f

      • That's fine in theory, but the reality is that many do get sued and lose for implementing a similar concept using a different algorithm.

        • That's fine in theory, but the reality is that many do get sued and lose for implementing a similar concept using a different algorithm.

          Agreed... but that points to the problem being in the litigation stage, not in the patent application stage.

        • Re: (Score:2)

          by gl4ss ( 559668 )

          just implement this 1:1 . and put a "on a mobile device" at the end of your patent application. I guess there's going to be already 100 such patent applications by tonight evening though..

  • The source article links to a page on the windows phone marketplace where it claims to be linking to details about the actual patent. Does anyone have a link to that?

  • Rtriever (Score:5, Informative)

    by cyrano.mac ( 916276 ) on Monday March 05, 2012 @08:36AM (#39246681)
    Rtriever has been doing exactly the same thing since 2006. http://labs.systemone.at/retrievr/ [systemone.at]

  • I've been trying to find a plastic organize case to fit into an archery case I've just finished and am _not_ having any luck finding an organizer which is ~4" x 10.5" x 1.5"....

  • Comment removed (Score:4, Informative)

    by account_deleted ( 4530225 ) on Monday March 05, 2012 @08:47AM (#39246775)
    Comment removed based on user account deletion

    • Yeah, and I did a short presentation on it for a grad class in '94 (one of those classes which is everybody doing a short presentation on a new topic each week). Information Retrieval was a geeky/esoteric topic a the time.

        The idea certainly isn't new.

  • Patent will be invalid - It's been done before (Score:5, Informative)

    by Etylowy ( 1283284 ) on Monday March 05, 2012 @08:47AM (#39246779)

    Patent is invalid if there is prior art. Well, imgSeek has had that function since at least 2008 (can't find changelog for desktop version - it might have been there already in 2006).

    • Patent is invalid if there is prior art. Well, imgSeek has had that function since at least 2008 (can't find changelog for desktop version - it might have been there already in 2006).

      Patent is invalid if there is prior art that teaches or suggests each and every element of the claims. Microsoft's claimed implementation may be very different from imgSeek's implementation, and as long as Microsoft isn't claiming something broad like "searching via sketching," then imgSeek's implementation may not anticipate the claims.

      For example, if I was a Toyota engineer and invented the hybrid gas/electric transmission that they use, I could get a patent on that, and it wouldn't necessarily be antic

      • I couldn't claim "a transmission" generally, because that would be anticipated by any existing transmission, but I could start adding in additional new elements of a gas/electric hybrid transmission that aren't in existing transmissions.

        sure, whatever wasn't in the 1906 Auto-mixte, or H. Piper's 1905 patent...

      • I did in fact read the claims in patent application and a lot of them look like the wavelet based method used by imgSeek. I seriously doubt much will be left off this patent in result.

    • Yes, but...

      Someone has to file an objection, which takes time and money.

      Or they bring up prior art when being sued over the patent, which takes LOTS of time and money.

      As usual, no justice without money in this country.

  • prior art (Score:1)

    by Anonymous Coward

    http://detexify.kirelabs.org/classify.html

  • some weird application that appears while ago where you could draw a "shape" and name it - i.e. "Car" and then somewhere else on teh canvas you'd do "sun", "cloud" etc....

    Once you'd done it it'd attempt to search and find images to make up your "master" image.

  • Probably not as simple as it seems (Score:4, Informative)

    by Miros ( 734652 ) on Monday March 05, 2012 @09:04AM (#39246949)
    This is probably not as simple as Microsoft trying to patent something which has already been done before. You may want to check out some of the actual research they have done here: http://research.microsoft.com/en-us/projects/mindfinder/ [microsoft.com] I seriously doubt that they are not familiar with all of the prior art examples that have been brought up here, and they would not have spent the money on a patent if they did not think they had improved on the existing methods sufficiently for it to stand up to even basic scrutiny.

    • Re: (Score:2)

      by Miros ( 734652 )

      Retrieving images to match with a hand-drawn sketch query is a highly desired feature, especially with the popularity of devices with touch screens. Although query-bysketch has been extensively studied since 1990s, it is still very challenging to build a real-time sketch-based image search engine on a large-scale database due to the lack of effective and efcient matching/indexing solutions. The explosive growth of web images and the phenomenal success of search techniques have encouraged us to revisit this problem and target at solving the problem of web-scale sketchbased image retrieval. In this work, a novel index structure and the corresponding raw contour-based matching algorithm are proposed to calculate the similarity between a sketch query and natural images, and make sketch-based image retrieval scalable to millions of images. The proposed solution simultaneously considers storage cost, retrieval accuracy, and efciency, based on which we have developed a real-time sketch-based image search engine by indexing more than 2 million images. Extensive experiments on various retrieval tasks (basic shape search, specic image search, and similar image search) show better accuracy and efciency than state-of-the-art methods

      http://research.microsoft.com/pubs/149199/0630.pdf [microsoft.com]

      • Assuming they get this patent granted, if and when it goes to court they're going to need something better than it "shows better accuracy and efficiency than state-of-the-art methods". If it were that easy patents would be worthless as you could make a few changes on anything and be home free. Samsung: "our slide to unlock has a shorter distance to travel and a bigger slide :)". Of course, maybe the insanity is to such a level that it really is that easy but if that were the case I'd use your logic and s

        • Re: (Score:2)

          by Miros ( 734652 )
          My point is that they are patenting something which may not have been done before. The ends may be similar but the means are different, and the means are what matter for patents - as they should. The mechanism is specifically designed to maintain an incentive to come up with new and better ways of achieving the same purpose.
          • Based on my observations of how patent law is applied in the realm of software these days I think you're confusing theory with reality. As things are now software patents are enforced on the end not the means. Of course legal arguments can get very nebulous almost immediately. A good lawyer could ask the court, "what is the 'end' of what we're talking about? The end is to get a search result. Therefore the patent is invalid as you are patenting means." And on and on. Just look at the shenanigans betw

            • Re: (Score:2)

              by Miros ( 734652 )
              I don't have any greater faith in Microsoft research than I have in any other organization (including tiny start-ups who also choose to file patents). One of the protections is litigation. Many silly patents that actually make it to that step end up defeated because greater amounts of resources are put into finding prior art that specifically conflicts with the method which was patented (and litigated over). I would not be shocked or troubled if someone immediately posted valid prior art, but it is a lit

  • Now all you guys claiming prior art, please, send it to the USPO so they can at least ignore it publically instead of what they usually do, ignore by default. Even better, contact the people in this article from Slate "Stamping out patent trolls" [slate.com] and they, Article One Partners [articleonepartners.com],will take it from there. It is one thing for us to think the King has no clothes, or whisper it behind "closed doors" to friends, it is another to shout it out in public or to proper authorities so either action is taken, or you dis

  • There are already dozens if not hundreds of patents in this same area.
    https://www.google.com/search?q=sketch+image+search&btnG=Search+Patents&tbm=pts&tbo=1&hl=en [google.com]

  • Supposedly this patent is from 2010..

    but according to this article [skipperkongen.dk] "[Search by sketch] was mentioned in years 2007, 2010 and sometime in the late 90s"

    if that is the case then there should be enough prior art to invalidate the patent for this concept

  • Everyone's commenting on whether this is patentable or not, but I'd be interested in discussing whether it is useful or not. Outside of some very narrow use cases (recognizing letters/symbols or maybe chemical structures?), will people actually find this useful? When we know the name of something, it seems easier to type that in (or speak it) instead of trying to sketch it. When we don't know the name... hmm, well it still seems easier to type in related words instead of trying to draw a sketch. :-\
  • This patent application is rubbish. I implemented this myself nine years ago as a portfolio piece while I was interviewing at Google. And I did it based on a 1992 Siggraph paper.

  • These silly clowns. I expect this article will quickly fill with examples of prior art. Here's another one:

    Looks at digiKam.
    Looks at patent application.
    Looks back at digiKam.
    Folds arms.

  • They had a more nuanced idea, the ability to take a scientific formula scribbled somewhere and understand/translate it, but essentially it was the same. They basically substituted f of x for a bicycle.
  • I wrote a search-by-sketch app in 2003 as a portfolio piece when I was interviewing with Google. And I based it on a 1992 Siggraph paper. So whatever Microsoft has patented, it had better be more specific than "search by sketch".

Slashdot Top Deals

A Fortran compiler is the hobgoblin of little minis.

Close