Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?
Google Privacy

Yes, Google Does De-List Pages; But When? 133

Frequent Slashdot contributor Bennett Haselton writes "Google finds itself inserting a disclaimer once again above some offensive search results. But the disclaimer still leads many to believe (incorrectly) that Google doesn't tamper with search results even in cases of 'harmful' or 'offensive' material. We know that Google has in fact de-listed some pages at the request of offended parties. What is their real policy on the issue?" Read on for Bennet's essay.

In 2004, when Google users discovered that the top search result for the word "Jew" was the anti-semitic site Jew Watch, Google ran a disclaimer in the space usually reserved for ads, explaining that their results only reflected the reality of link counts on the Web, and that they did not endorse any Web sites which appeared at the top of their listings. Now the disclaimer has been dusted off again, as the top result on Google Images for "Michelle Obama" is a picture of a monkey's face with Michelle's hairdo. (Ironically, it looks as if the original image would have fallen out of the rankings, if it hadn't been for a follow-up blog post about the controversy, which itself now comes up as the first result.)

I first heard about the controversy from Dennis Prager's column in which he takes a New York Times columnist to task, because the columnist complained about "racially offensive images of the first couple" that come up in Google searches. Prager was unable to find any examples from Googling "first couple" or "Michelle and Barack Obama pictures," so he concluded that the NYT columnist "wildly exaggerated, if not made up" his claims. I tried Google Image searches for "first couple," "Barack Obama," and some other terms, and I couldn't find anything controversial either. However, it only took 10 seconds to enter "first couple google images controversy" on the regular Google Web search and find multiple blog posts explaining what all the fuss was about. Back to Google 101 for Dennis.

Many of the blog posts refer to Google's disclaimer about not tampering with search results. Those on one side are urging Google to make an exception and "fix" the results, while others sagely observe that Google just reflects reality, it doesn't create it.

All of this punditry is starting from a premise that's wrong. Google has actually removed pages from their search results — not because the pages were illegal or because the webmasters were search engine spamming, but because of the page's "offensive" content. In the "Chester's Guide" incident, a councilman in Chester, England discovered that one of the search results for "chester guide" was a satirical page titled "Chester's guide to picking up little girls." Although the page itself was obviously just someone's idea of sick humor, a Chester city councilman (who admitted that he hadn't looked at the page, saying that the title told him everything he needed to know) urged Google to remove the page from their index. Google at first refused, but later manually blacklisted the page to prevent it from appearing in their search results.

Whether or not you think this was the right decision, probably depends on what you think is the purpose of Google. If Google's purpose is to return the most useful results, then it made sense to remove the link, as Danny Sullivan of Search Engine Watch argued at the time, since it almost certainly was not a useful result for people searching for "Chester Guide." On the other hand, if the primary purpose of Google is to reflect the reality of what pages on the Web feature certain words most prominently (combined with all the other factors that Google weighs, of course), then the results shouldn't be altered.

But more people should at least realize that it happened. The Google disclaimer doesn't precisely say that they never blacklist pages or modify search results ("Google reserves the right to address such requests individually"), but it seems to give most people the impression that that's the case. According to that crudest of Googling techniques for which novice searchers are so frequently lampooned, there appear to be about 400 times as many stories on the Web about the Google "Jew Watch" controversy (where Google stood their ground) as there are stores about the "Chester's Guide" incident (where Google caved).

And Google-number-three Matt Cutts posted on his blog back in March explaining why Google does not remove "offensive" pages from search results; over a hundred comments followed, debating the pros and cons of the position, but none of them mentioned the Chester incident or any other case where Google actually had removed pages except as a result of a court order. One isolated comment from "Anonymous" said:

This is not quite true. I know of at least one web site that was de-listed for containing illegal content and/or promoting illegal activity.

which may or may not have been a reference to the Chester Guide incident. And that was it.

Is this a lot of hay to be making over something that happened years ago? Well, for one thing, I doubt if it happened just once. Consider that the Chester Guide incident involved a public declaration of outrage by a city council, and a public statement from Google, and still hardly anyone knows that it ever happened. If other incidents occurred without those high-profile elements, it would be even harder to discover them now. We'll probably never know how many such incidents took place, unless someone sues Google (maybe the owner of a blacklisted website, or maybe the victim of a RipOffReport hatchet job wondering why that site hadn't been blacklisted long ago), subpoenas Google for a list of cases where pages were de-indexed, and publishes the list if it's not sealed by a court order.

But whether it was one time or a handful, consider that political candidates like Arnold Schwarzenegger and Al Franken got asked during their campaigns about things they did 20 years earlier, and it's fair to ask a candidate about their past, because it's the same person standing in front of you now. Why did you do that? Have you stopped? Why?

And in the big scheme of things, Google is probably more powerful than a single US senator or the governor of California. So, can't we ask? What are their real rules about page removal? Have those rules changed since the Chester's Guide controversy? Can they even tell us what their rules are, or do they consider it a trade secret?

It is well known, of course, that Google censors some results in their search engines branded for different markets like China and even in liberal democracies like Germany. But nobody would call that a slippery slope towards censorship in the US version of Google, because the censorship in the Chinese and German versions is done at the behest of the governments there. On the other hand, Google does admit that they will de-index pages which include credit card numbers or social security numbers (which are all too easy to find on the Web). This might not seem like a controversial position, but even this act of voluntary self-censorship may be dipping their toe in the water further than it seems. Most people do consider their credit card information more private than their home address. But surely there are people like J.D. Salinger who less about the privacy of their credit card number (which is easily changeable) than their home address (which isn't). If someone finds Salinger's address and posts it on the Web, should Salinger be able to demand that Google de-index the page? Why should Google cater to the majority who want to keep their credit card number secret, but not to the minority who care more about keeping their address secret? Another commenter on Matt Cutts's blog post asked:

"hi. I have a question. My mom 'googled' herself and it shows some of her medical problems. She wants/needs these pages removed from search engines."

Again, why shouldn't that be considered at least as private as a credit card number?

And finally, even Google's decision to display an "offensive results" disclaimer, for some results but not for others, raises the same "Where do you draw the line?" questions as the issue of page removal. The Michelle Obama monkey picture gets a disclaimer. But search for 'george w bush' and the first row includes a photoshopped (I think!) image of Bush flipping off the press. Does that warrant a disclaimer as well? (Maybe that's considered less unfair because, even though the picture is fake, it does depict something that actually happened.) The first image result for "bristol palin" is a photo of her engaged in underage drinking — a real photo, but probably unfair to call it the single most relevant photo of her on the Web.

So while Google might consider credit cards and social security numbers and search engine spam to be on one side of a "bright line," and everything else is served up without alteration, I think the line is blurrier than that, for at least those three reasons: (a) credit cards and SSNs are less private than some other that things that Google serves up anyway; (b) Google has unambiguously removed some content that fell outside that bright line, as in the Chester's guide incident, and (c) they make other "slippery slope" judgment calls about search results all the time (as in the question of when to show the disclaimer). So I hope that Google someday comes out with a more complete answer to the question. What is their real policy on what they will remove? The Chester's guide incident — would they do that sort of thing if the same situation came up today, or have their rules changed? If they want to go really deep, then is there a general set of principles from which their rules follow — explaining why, for example, they treat credit card numbers as more private than sensitive medical information? (Google did not respond to my request for comment, either through official channels or the unofficial back channels of friends who work there.)

I hope Google gives an answer some day. Even just to say, "It's a classified internal policy and that's all we're going to tell you." But once and for all, the answer is not "Google doesn't remove content just because it's 'offensive' or 'harmful.'"

Meanwhile, a modest suggestion about the disclaimer displayed above the search results: Put it where people will actually see it, in a separate line below the ads, but above the search results. Right now the link to the disclaimer is displayed as one of three ads across the top, and people don't look at the ads. But hey, people do buy ads, so if you push the disclaimer down a bit where people will read it, you also free up space for 50% more ad revenue!

This discussion has been archived. No new comments can be posted.

Yes, Google Does De-List Pages; But When?

Comments Filter:
  • Dear Sir, (Score:4, Insightful)

    by u38cg ( 607297 ) <calum@callingthetune.co.uk> on Friday December 18, 2009 @02:08PM (#30490274) Homepage
    Am I alone in thinking that whoever Bennett is, I have no interest in his vague ramblings?
    • by Hognoxious ( 631665 ) on Friday December 18, 2009 @02:10PM (#30490318) Homepage Journal

      Well I didn't read it, did you?

      • by u38cg ( 607297 )
        I've read several of his punditisations in the past. The leopard does not change his spots.
      • Re: (Score:2, Informative)

        by Anonymous Coward

        Wow a DRTFA. That's gotta be a /. first...

    • Re:Dear Sir, (Score:4, Interesting)

      by clang_jangle ( 975789 ) on Friday December 18, 2009 @02:11PM (#30490334) Journal

      Am I alone in thinking that whoever Bennett is, I have no interest in his vague ramblings?

      Certainly you are not alone. AFAICT he's a self-appointed pundit who's in love with his own rather murky ideas. No doubt it's all about the ad sense dollars.

      • Re:Dear Sir, (Score:5, Insightful)

        by religious freak ( 1005821 ) on Friday December 18, 2009 @02:58PM (#30491136)
        I think he had some interesting points, but it could've been summed up in a couple paragraphs.
        • Would you?
      • who's in love with his own rather murky ideas. No doubt it's all about the ad sense dollars.

        You already said that when you said:

        AFAICT he's a self-appointed pundit

      • Am I alone in thinking that whoever Bennett is, I have no interest in his vague ramblings?

        Certainly you are not alone. AFAICT he's a self-appointed pundit who's in love with his own rather murky ideas. No doubt it's all about the ad sense dollars.

        I read that he likes to pick up little girls, but Google blacklisted that information.

    • Re: (Score:2, Flamebait)

      by swanzilla ( 1458281 )
      30+ links? I don't know if his inflated self worth offends me more than his tainted view of the average /.er, or visa versa.
    • Re:Dear Sir, (Score:5, Informative)

      by MaerD ( 954222 ) on Friday December 18, 2009 @02:52PM (#30491022)
      Bennett is the founder of Peacefire.org. He's been involved in things against spammers, censorware, etc.
      And I agree, some of his rants do ramble a bit long, but they tend to make me think about something I may not have otherwise.

      For more information, try the wikipedia page on him [wikipedia.org] and or Peacefire [wikipedia.org]
    • "Manually altering the eigenvectors of a link matrix, is like cutting warm butter..."

      "I'll be back, Bennett!"

    • by Sloppy ( 14984 )

      No, you're not alone, but you (plural) are missing out.

    • And whom may I ask was forcing you to read it? Over hear at my terminal, there is nobody with a gun to my head, and so I am able to choose which articles I find interesting, and which I'd rather ignore. I recommend this strategy, it saves time not only in not reading articles I don't want to read, but also in not having to make comments about how I didn't want to read the article I read.
    • Did he just say Google is more powerful than the Governor of Califoria? The Governor of California can call up the national guard. I admit I haven't checked recently, but I don't recall there being a "click here to deploy National Guard troops to this address" button on Google Maps.
    • Dunno who he is but I didn't think the essay was overly vague or rambling. Certainly I found it more interesting than most of the articles I see on Slashdot these days...

    • Am I alone in thinking that whoever Bennett is, I have no interest in his vague ramblings?

      It does not matter if you are alone in having no interest in read this, what matters is if I am alone in finding it interesting reading. Nothing can appeal to everyone the best you can hope for is to appeal to some, maybe even a majority but not necessarily.

      Most people are able to simply read what interests them and ignore what doesn't, why cant you? Must every article on slashdot go through your own personal approval?

  • by Bragador ( 1036480 ) on Friday December 18, 2009 @02:13PM (#30490376)
    If each IP adress can give a mark to each web page based on if they think the result is relevant and useful enough, then that should filter the "problems". On the other side... bye bye anonymity!
    • If each IP adress can give a mark to each web page based on if they think the result is relevant and useful enough, then that should filter the "problems". On the other side... bye bye anonymity!

      What could possibly go wrong?

      • Re: (Score:3, Interesting)

        by Bragador ( 1036480 )
        Many things, but Slashdot, Ddiggs and the like seem to be doing pretty well, no?
        • Many things, but Slashdot, Ddiggs and the like seem to be doing pretty well, no?

          Slashdot is not (very) politically sensitive. As soon as you have content that is and a mechanism that can be manipulated by individuals you will find you have attracted the attention of people like the government of China and their masses of individuals that they PAY to astroturf.

    • by gedrin ( 1423917 )
      Now, I had mod points, and considered using them to show a point, but it seemed in apropriate. I'm not sure if my pesonal restraint disproves the point. Still, I think without the ability to restrain a user's modding ability (like M2) you could get swarm modding on sites directed by interest groups. I think it probably has all the issues that /. modding has, but far too many users to implement an M2. Additionally, there's the possibility of owners of large numbers of IP's simply using their block vote t
      • Would that really be bad? At first glance, yes. On the other hand, if Microsoft has tons of IP adresses compared to "Microsoft Sucks", then the first link would be the official website. Not a bad deal uh?

        Also, the search results would be based on links first and foremost, but with a rating you could get a better idea of what to select. I'm using http://www.ixquick.com/ [ixquick.com] and the first results are rated based on how many other search engines have these results in their top 10. Ratings can help and those by us

        • Simply to add to my ideas... People could then filter searches by number of links, popularity (if tracked) and also by ratings. That would give different choices to explore the web.
        • Re: (Score:2, Insightful)

          by gedrin ( 1423917 )
          The Microsoft example is benign, but suppose all searches for news on scandal X redirected to poltical supporters (or opponents). It just seems like a mechanism that allows for exploitation by power blocks.
    • Logged in users already have this option for their own search results. It's unclear how personal choices affect search results for everyone else.

  • Neutral Party (Score:4, Insightful)

    by Reason58 ( 775044 ) on Friday December 18, 2009 @02:15PM (#30490400)
    It seems to me like it would be in Google's interest to remain neutral regarding search results. Now that Google has started censoring sites at their discretion I would think this forces them to take responsibility for all the results they provide. They are no longer simply a neutral party providing indexed results.
    • by maxume ( 22995 )

      They are responsible (in some sense, that word invokes both community and legal standards), no matter what, clearly they are doing whatever they can to maintain the appearance of neutrality while also trying to avoid controversy that is of no benefit to them.

      • Is the maker of a cork board responsible for whatever bulletins are placed on it? Why should it be different for Google (if they were neutral)?
        • by maxume ( 22995 )

          That analogy is insufficient, they are the maker of the cork board and the owner of the wall that it is mounted on.

          • That analogy is insufficient, they are the maker of the cork board and the owner of the wall that it is mounted on.

            To carry that analogy further, why would the wall owner be responsible for what I post on the board? Can I go out and put a flyer on a telephone poll saying Obama is a homosexual martian and then sue AT&T for libel?

            • by maxume ( 22995 )

              Say Mr. Baker owns a store and puts up a cork board, and Little Billy puts up a note that Sally is a whore. At a minimum, people are going to think Mr. Baker is an asshole if he doesn't take down the note.

              I'm not saying that Google is necessarily legally liable for their results, just that the results come up on pages with "Google" logos on them, and people are going to associate the results with Google no matter what Google does.

            • To carry that analogy further, why would the wall owner be responsible for what I post on the board? Can I go out and put a flyer on a telephone poll saying Obama is a homosexual martian and then sue AT&T for libel?

              You can (it's in a public place). The owner of the pole (or anyone else for that matter) could take it down if they felt sufficiently motivated to do so. Back to the cork board, though - if the board is say, in a business, then the business owner may have an interest in removing inflammato

            • Re: (Score:1, Insightful)

              by Anonymous Coward

              If you were to post child porn on the board, and the board owner said "well, I didn't post it, so it's not my problem if I just leave it up there", do you think that owner would end up in jail?

              I do.

    • Re:Neutral Party (Score:5, Insightful)

      by Interoperable ( 1651953 ) on Friday December 18, 2009 @02:54PM (#30491056)

      You make a couple of good points. I completely agree that the slippery slope that Google has to tread carefully on is not one of censorship, but one of liability. Many people seem to think that Google has a moral imperative to objectively reflect the "reality" of the web. They don't. Anyone who feels that Google does or should act in way that is not in their best interest is going to be disappointed.

      Google does what they must in each country to remain the dominant search engine. That means abide by local censorship laws, bow to public opinion and avoid becoming liable for search results. Google will always do what will funnel the most money into their coffers but so far they have been very clever to recognize that neutrality and openness can accomplish that goal very well. The debate of "should Google censor results?" hinges on only one criterion: profitability. That goal, in turn, depends on what will preserve the largest possible ad revenue while mitigating liability.

      Google is not a public service, it is a publicly traded corporation.

      • Re: (Score:3, Insightful)

        by SirWinston ( 54399 )

        > Google is not a public service, it is a publicly traded corporation.

        And that, in a nutshell, is what's wrong with this country. Publicly traded corporations can and will maximize short to medium-term profit at any and all costs, including costs to the health and well being of the country, its citizens, and their own long-term viability. In corporate persons, moral, ethical, and sometimes legal, obligations are trumped by fiduciary ones in ways which natural persons could never so compromise.


        • by Toonol ( 1057698 )
          Google is not a public service, it is a publicly traded corporation.

          And that, in a nutshell, is what's wrong with this country.

          That's the reason Google and all other decent search engines exist in the first place. If the state created a search engine as a 'public service' Google alternative... people would still flock to the ones ran by businesses. They're going to be better.
          • That's the reason Google and all other decent search engines exist in the first place. If the state created a search engine as a 'public service' Google alternative... people would still flock to the ones ran by businesses. They're going to be better.

            Google existed, and was wildly popular, for years before it went public.

        • by yuhong ( 1378501 )

          And that, in a nutshell, is what's wrong with this country.

          I would not go that far. But I have a slashdot submission coming about the problems of "shareholder value" and "agency theory". And yes Google did address that back when it IPOed.

          Google is not a public service, it is a publicly traded corporation.

          Actually Google is a publicly traded corporation that makes money by providing a public service.

        • Is this post really supporting the notion that web search should be a public utility controlled by the gov't rather than a corporation? I really hope not...

    • Re:Neutral Party (Score:5, Insightful)

      by ajs ( 35943 ) <ajs@nOspam.ajs.com> on Friday December 18, 2009 @03:13PM (#30491356) Homepage Journal

      "Now that Google has started censoring sites at their discretion"

      Nope, I don't read anything above which in any way suggests that it's at their discretion. The only example that might imply that we have too little detail to know for sure (the local government official that got Google to delist a page, which Google initially refused, but then complied... implying that there's an intermediate conversation we're not privy to).

      This all seems to run the usual route: when compelled to remove information by law, or when certain information presents an obvious legal and financial liability to Google (e.g. exposing credit card numbers), they delist pages as technical means of identification allow, as a matter of compliance.

      This is exactly what Google and every other search engine have been doing since the dawn of Web search, and it's the only reasonably correct solution.

    • by Sloppy ( 14984 )

      It seems to me like it would be in Google's interest to remain neutral regarding search results

      No, because if they did that, no one would ever use Google. When you search for "foo" you want a page about foo, not the page that says the word "foo" the greatest number of times. They have to analyze for true relevance and utility.

      But that can't be done objectively. When programmers create heuristics, they're really (in a roundabout way) stating personal opinions, about what works "best." And "look it up in

    • Google shouldn't 'blacklist' pages, that's for sure. No altering the reality - if the page contains a keyword, it should be listed, period.

      Now, as for -where- the page should be listed, this is sole discretion of Google. They are the authors of the ranking algorithm and they can tamper with it as they see fit. And manually reducing rank of a site down to oblivion seems like a very reasonable option. So if I want to find Chester's guide to picking little girls, I'll have to either dig through all the guides

    • by CAIMLAS ( 41445 )

      They do this by 'tweaking' their indexes to preference one thing over another thing in a results page, anyway. Granted, it's not as obvious or as overt, but it still happens. Many times, it seems they're doing it to de-emphasize "unpopular" data. People tend to not want to think about it, or even acknowledge it, though.

  • by Anonymous Coward
    Is there a firm algorithm for how Google ranks relevant pages, or is that a proprietary black box? Because if it is, I don't understand the problem - we are already unsure what they're doing behind the curtain, so who cares if they follow their usual algorithm for a page or treat it specially because someone finds it offensive? Incidentally, it'd be nice if Google kept their inner workings a mystery so we didn't have companies devoted entirely to increasing websites' rankings for more page views.
  • by bugs2squash ( 1132591 ) on Friday December 18, 2009 @02:20PM (#30490512)
    is just one step prior to suing them if someone obtained your credit card info via google and used it to rip you off.
    • Agreed. I think though, the resulting searched info would be from a web admins carelessness. A log file, or non-validated input variables for example.

      As such, liability is easily passed to that web admin.

      The exception might be if google tries "mini brute force" in its searches.

      Someone setting a honey pot for google...there is a risk for them.
  • Time for a new format of robots.txt:

    User-agent: *
    Allow: /
    Conditions: only_when_delist_possible
    Remarks: please_leave_your_name_when_done

  • It's not google that is hosting the questionable content so why should anyone sue google for it? The problem is though, if you would sue the hoster or webmaster instead you might run into problems of websites being hosted in certain countries with almost no jurisdiction on this area or countries that simply don't give a damn.

  • No wonder I couldn't find Lloyd C. Blankfein SS on Google...I guess I'll have to try finding it via Bing

  • by girlintraining ( 1395911 ) on Friday December 18, 2009 @02:46PM (#30490928)

    What is their real policy on the issue?

    I thought it was obvious simply because they're a publicly-traded company: Protect their own asses first. If Google could be subjected to substantially negative press, delist the site. Rationalizations come later in the form of policies, laws, rules, and procedures.

  • Obama and Muhammad (Score:3, Interesting)

    by Shivetya ( 243324 ) on Friday December 18, 2009 @02:47PM (#30490944) Homepage Journal

    I'm pretty much convinced they delist more than the author suspects. Many court orders have gone out in cases where the resolution was sealed and I would expect those related to internet postings could be buried the same way. Of course nothing stops the listing of material faster than leaving it out for all to see and having aggrieved parties (direct or indirect) going after hosting sites if not the actual people who generated the offensive content. Still much of this has to do with what side of you political spectrum you are in.

    As in I find it amusing authors examples of questionable photos/links about people associated with conservatives while using the most obnoxious example in regards to the current Administration. I am quite many can remember the all similarly racist and hate based pictures of Condoleeza Rice or Colin Powell. Yet where was the outrage? I guess its OK if one side improperly credited with doing the most for minorities in turn is most likely to turn a blind eye to those minorities if they leave the "plantation".

    Nah, deification of elected officials is dangerous and now I bet any picture which distorts Obama or members of his families appearance is automatically sacrilegious.

    • Re: (Score:2, Informative)

      by asylumx ( 881307 )
      Do you really not recall the googlebombs around "Miserable Failure" or "Weapons of Mass Destruction" or are you just ignoring them because they invalidate your point?

      deification of elected officials is dangerous

      What are you doing in their bathroom?

      • deification (d-f-kshn, d-) n. 1. a. The act or process of deifying. b. The condition of being deified. 2. One that embodies the qualities of a god. deification != defecation.
    • by Toonol ( 1057698 )
      Nah, deification of elected officials is dangerous and now I bet any picture which distorts Obama or members of his families appearance is automatically sacrilegious.

      The article writer, I think, was even-handed; mentioning a specific instance about Michelle Obama, and contrasting it with different handling of images of the 'opposing' political spectrum. That's reasonable. Now, the way Google and others reacted to the actual pictures isn't reasonable... how Michelle Obama's caricature was handled is a
  • by TomXP411 ( 860000 ) on Friday December 18, 2009 @02:57PM (#30491112)
    I'm waiting for the Google Labs option that automatically filters out the "direct download" sites that don't actually offer any added value, things like "freewareseeker.com" and "findyourdownload.net". You can drop individual search results, but where's the "never show me this domain or any other domain from this company ever again" button?
    • by Chrutil ( 732561 ) on Friday December 18, 2009 @03:11PM (#30491324)
      and experts-exchange please. I would pay hard cash to have them removed from any future search result.
      • by TomXP411 ( 860000 ) on Friday December 18, 2009 @03:17PM (#30491408)
        Please subscribe so I can show you the answer. :)
      • by danomac ( 1032160 ) on Friday December 18, 2009 @03:25PM (#30491516)
        I second this. Although recently I found if you scroll to the bottom of the page the answers from experts-exchange are there. Given the replies I've seen because of that I'm glad I didn't give them any money. Bug on their website maybe?
        • by base3 ( 539820 )
          Thirded. I don't know why they haven't been removed for cloaking.
          • Re: (Score:3, Interesting)

            by Joebert ( 946227 )
            I've been wondering the same thing. Despite what the anonymous coward replied with about it not being cloaking, it is in their case.

            Try entering from the index page of experts-exchange and making your way through to any topic on the site. There's no scroll down and get the answer gimmick if you do it that way. However, if you copy that URL into a Google search box and then click the link in the SERP so that when you go to the same exact URL but having Google as the referer, you get that scroll down and s
        • by colesw ( 951825 )
          Not a bug, they do this to get indexed on Google. If they didn't do that they would be providing different results to users and Google, which is a no no for Google.

          Once I realized this, I've found you can actually find some good answers on the site.
      • and you will see the real search results on the experts-exchange page

        this situation has evolved over time i noticed

        experts-exchange used to hide the real results a few years ago

        then they seemed to be delisted from google results

        then they came back to google results, and all you had to do was scroll all the way down to the bottom of the page to see the genuine text you searched for and the entire unobfuscated threaded discussion

        so there's a story behind that. what i don't know, but i would guess google saw t

      • I installed Greasemonkey just to get rid of experts-exchange. This script does it: http://userscripts.org/scripts/show/1898 [userscripts.org]
      • there's a firefox-plugin called "customizegoogle" that lets you regex-filter search results, among other nifty things like stream search results.

    • by base3 ( 539820 )
      OptimizeGoogle [optimizegoogle.com] does this for Firefox, at least on a domain by domain basis. It would be nice if Google would do it. Another thing I've found useful is to minus out the spammy search terms (e.g. -"direct download" -"full version" -keygen).
    • Yes, please!

      I do like that you can remove a site from results which I've used to remove those god awful rose india sites that pop-up in Java results. But then you slightly tweak your search and the site is back in that result. I'd love to just wipe sites completely from all searches.
    • by Bigbutt ( 65939 )

      Or can automatically filter out the ExpertSexchange site from my technical queries (yes, I can -expertsexchange tyvm).


      • You have to wonder: I'm betting this is the single most requested (realistic) feature. I wonder why they haven't implemented it yet. It'd be simple to do. Simply don't return results on specific domains (or wildcards), and optionally look up the domain's owner and refuse to return results from any other domain owned by that organization.
  • by fast turtle ( 1118037 ) on Friday December 18, 2009 @03:05PM (#30491226) Journal

    The first point that needs to be thought about is the U.S Privacy laws regarding Health/Medical Records. There is absolutely no reason for any pages from those two topics to be in the search results, particularly as Google is a United States Corporation. Means they can be sued/fined heavily under HIPPIA for violations.

    Another is the censorship issue in general. I'll agree that I don't like the Idea of them Caving in to China's demands but only the People of China have any say in their governments decision unless you are willing to declare war and attempt to enforce those requirements upon them by force of arms.

    In regards to the Chester Guide, I'm open to debate on whether the page should have been removed from the index or simply gotten the disclaimer? It's important to note that Censorship of any kind is the beginning of a very slippery slope and who's to say that Google hasn't already started the long slide into irrevelency by caving in to both China and Germany's demands and that's the bigger issue. Google has stated that they want to make all known information available but if they're censoring pages at the request of governments, who's to say they aren't censoring pages that governments have not requested? On the China and German Censorhip issues, keep in mind that the censorship only applies within the country that asked for it. Outside still gets access to it. This means the information is only censored on a regional level instead of worldwide as happened with the "Chester Guide".

  • No blacklists needed (Score:3, Interesting)

    by FrankSchwab ( 675585 ) on Friday December 18, 2009 @03:07PM (#30491260) Journal

    Google is a business. It is giving users a service (useful search results), and selling your eyeballs to advertisers (customers). I have no problem with that.

    If I searched for "Chester's Guide" because I was planning a trip to England and got a link to (even in-jest) pedophilia, that's not a search result that I would be looking for - it's a failure for Google's search engine. Frankly, if I were Google, I would want people to tell me when they think my search results weren't working well, so I could update my algorithms to serve the users better so I could get more money from the customers.

    This doesn't need to involve blacklists - all it requires is Google rejiggering it's algorithms to move more relevant links higher in the returned results, and less relevant links lower. They must do that on a regular basis anyway - heck they already (claim to) do it in cases of detected SEO abuse. Now, if its the case that a book on Pedophilia is more relevant given the search terms than a guide to a city in England, not only is Western Civilization in serious jeopardy, a certain city in England has its own issues of irrelevance. /frank

  • The main difference here is that listing people's social security numbers, credit card numbers, etc. make Google a one-stop source for identity theft on a large scale. Indeed, there's almost no other use for these types of personally identifying numbers. On the other hand, millions of people do want their addresses published in order to conduct business or maintain correspondence, and millions of other people want to find those same addresses for perfectly legitimate reasons. Since Google can't economica
  • tl;dr (Score:3, Funny)

    by baKanale ( 830108 ) on Friday December 18, 2009 @03:19PM (#30491442)

    What the hell? This is Slashdot! If I wanted to read an article then I would have gone somewhere else!

  • I think that at least the latter half of this post is missing one huge piece of information. How can you be sure that it is Google that is defining public/private data. A lot of organizations are required to follow federal standards which in themselves define what is public and private. Social Security Numbers and Credit Card Numbers are always at the top of every one of those lists. For reference look at PCI DSS (http://en.wikipedia.org/wiki/PCI_DSS), FERPA (http://www.ed.gov/policy/gen/guid/fpco/ferpa/in
  • "Google probably de-lists anybody who doesn't offer to install a google toolbar or other similar junk(ad)ware like Earth, or Chrome." Maybe he did, but who could tell in that mess? I get as little as five results on searching for common errors messages. Google is not a search engine in the way most people think it is. Guess I gotta send out my own droids if I want it done right.

  • Uhh... why? I thought that google was this great big evil company that made me lots of money. But this thing just pointed makes me want to sell my share in google.
  • Why Jew? (Score:1, Flamebait)

    And not kike? Or for that matter, nigger, faggot and chink. None of these hold that warning and yet they're more likely to produce "distributing" results.
  • Google may not be evil, but they /are/ a corporation, which means they are primarily concerned with their own persistence. If you actually want an index engine that "just reflects reality, [and doesn't] create it," I would suggest using a more open search plugin, such as majestic [majesticseo.com]

    And BTW, kdawson, well written article. I'd like more of these.

  • by Restil ( 31903 ) on Friday December 18, 2009 @05:31PM (#30493520) Homepage

    First off, home addresses and phone numbers never used to be a private matter. Everyone was always in the phone book, student directories with phone numbers AND addresses were passed out to all the students every year. IF someone had an unlisted number, it seemed to be noteworthy for some reason. Of course, I'm talking 20+ years ago. Now, people seem to be much more cautious about having their home addresses and phone numbers listed. Of course, now that you can be targeted for prank calls by anyone on the internet... perhaps hiding this information seems to make more sense.

    As for medical information, how did that end up on the searchable internet to begin with? Hospitals don't tend to create public webpages detailing the medical conditions of their patients, complete with real names. About the only way news of her extreme toenail fungus would end up on the internet is if she were blogging about it.... or telling friends about it, who in turn feel the need to discuss it in front of a world audience.

    As the post made clear, if you want something to disappear, the quickest way to do so is to STOP TALKING ABOUT IT. Nothing stirs up popularity in the age of the internet more quickly than someone complaining about, and then posting a link to, offensive content.

    Also, while Google can pretty much do whatever they want as far as delisting or rank adjusting, it's not in their best interests to censor information just because it's mildly offensive to someone, as it provides precedent and opens them up to potential lawsuits when they don't... or do... Common carrier defense and all that. However, in the
    age of pedophile witch-hunts, they can pretty safely de-link something of that nature without getting anyone too upset about it. Nobody is going to mount a strong opposition to the removal of that type of material, and anyone who supported it has no fight once it has been removed, so nobody talks about it. No talking, no linking, and therefore no Googling.


  • by snowwrestler ( 896305 ) on Friday December 18, 2009 @05:33PM (#30493538)

    Google may have delisted a few pages. I wonder if it happens in situations where they could have been concerned about (or maybe faced) legal action. Note that the Chester example is a person in the UK, about whom were written harmful things that are untrue. I believe the UK has libel laws that strongly favor the complainant.

    In many cases what Google has done is updated their algorithm. This is not the same as delisting, as the content is still findable. For instance it was not long before the monkey image of Michelle Obama was no longer on the first page for a GIS of "Michelle Obama." However if you searched "Michelle Obama monkey," it was the very first result. From the point of view of Google, this is probably an improvement to their product. IIRC when they defused the "miserable failure" Googlebomb of George W. Bush, many Googlebombs were shuffled out of the top spots as well.

    Google says their mission is to organize the world's information and make it findable. My guess is that they are firmly on the side of "search represents general relevance" rather than "search reflects online popularity at that moment in time." I think people too easily fall into thinking about how Google works, rather than what its ideal results should be. If I opened a history book 30 years from now and looked up "Michelle Obama" in the index, it would not make sense for that monkey image to be the illustration.

  • Who was Aramark's mob connected CEO during the 1996 Summer Olympics in Atlanta. In spite of Aramark taking out back cover ads in Time Magazine that summer there is no trace on the internet of him ever being associated with Aramark or the Olympics. That's one example of Google's mysterious delisting practices and The Way Back Machine's as well.
  • Gee, I always assumed Google simply ranked 1st page search results depending on who paid them the most, with people that COST them money simply disappearing.

    They do otherwise?

  • by efalk ( 935211 ) on Friday December 18, 2009 @06:56PM (#30494494)
    You can go to Chilling Effects and read up on many, many cases of Google censorship. The bottom line is that Google gets something like 1000 C&D notices per week, and they just can't afford to fight them, so in most cases, Google just immediately rolls over and complies.
  • Alternative? (Score:3, Interesting)

    by dissy ( 172727 ) on Friday December 18, 2009 @06:58PM (#30494520)

    I am curious why the submitter of this article did not include a link to his own search engine, that works as well as Google does but does not abide by any laws and actively breaks them as he suggests search engines should do.

    I'd definitely use it for the few hours it was in operation before the owner was hauled through court and the servers confiscated...

Our business in life is not to succeed but to continue to fail in high spirits. -- Robert Louis Stevenson