Google May Limit Free News Access 236
You know how, if you want to read a paywalled newspaper article, you can just paste its title into Google News and get a free pass? Those days may be coming to an end. Reader Captian Spazzz writes: "It looks like Google may be bowing to pressure from folks like News Corp.'s Rupert Murdoch. What I don't understand is what prevents the websites themselves from enforcing some limit. Why make Google do it?" (Danny Sullivan explains how they could do that.) "Newspaper publishers will now be able to set a limit on the number of free news articles people can read through Google, the company has announced. The concession follows claims from some media companies that the search engine is profiting from online news pages. Publishers will join a First Click Free programme that will prevent web surfers from having unrestricted access. Users who click on more than five articles in a day may be routed to payment or registration pages."
Re:What is going one here? (Score:5, Informative)
So it sounds like (maybe?) the news sites have a policy that says that clickthroughs from Google don't have to be routed through their access control. Why? Is this something Google requires newspapers to do in order to do display links to them on Google News?
It's something Google requires any websites to do to be linked at all. If you present different information to Googlebot than to normal users and Google finds out about it, you get kicked out of the Google index. So you have to choose between:
[a] Letting users see the story for free
[b] Showing Google the same login screen as everyone else
[c] Being kicked out of the Google index entirely
It sounds like Murdoch and co have threatened to take path [b], and Google have made concessions.
Re:Frist Psot! (Score:3, Informative)
Luckily, BBC News is run on the British TV Licence and can't - by power of it's charter - put adverts or start charging for anything.
Re:Censorship (Score:2, Informative)
The Wall Street Journal is doing fine with a paywall, so it may take some convincing.
What really pisses me right off about paywalled... (Score:5, Informative)
Google search, Google scholar etc always turns up paywalled articles outside of the news industry. In particular, research articles. On clicking through your are greeted by a screen to pay for the article, and the keywords that were searched for are not in the summary/abstract presented or even available to see. In effect Google has given me a "hit" on my search then led me to a place where not even the search terms are present... Google crawler has access to it but I do not.
ieeecomputersociety.org, springerlink.com, sciencedirect.com (anything but direct)... the list goes on.
Ok, you might say that they hold all the serious research papers - you might even be right, in some cases. I even understand that maybe just maybe, if I am really desperate, then I might actually want to search for paywalled articles and am prepared to pay the extra information access tax of $20-$40 a for every article. However what google is now doing is wasting their bandwidth and more importantly to me, completely wasting my time by including paywalled articles in top positions of all my search requests. Furthermore, Google does it by default.
I have written to their support, posted on their forums -please Google - if you are listening - MAKE PAYWALLED SITES AN OPTION in my preferences and set it OFF by default. If you think about where it leads: the quality of the future of all our search requests is at stake. Now Google is planning to add News to this time wasting highly annoying practice - and I want to be opted out by default, I am begging you!
Re:What really pisses me right off about paywalled (Score:3, Informative)
A question: If their crawler has unfettered access to the content then how are they supposed to know that it is a paywalled site?
They already seem to have identified the culprits of poor search hits. If you select their "shopping sites" as an option then you get majority paywalled articles - so they must have already done the work to identify paywalled articles.
Re:May (Not) Work (Score:2, Informative)
Woops, lost the links:
Doing something like this (showing different content by user-agent) is against Google's terms-of-service [google.com] and can cause your site to be removed from the index.
Google has said [blogspot.com] that each provider must figure out by itself how to implement this free view limit based on referrer. /quote
Re:What really pisses me right off about paywalled (Score:4, Informative)
1) get Firefox
2) get User Agent Switcher extension
3) Set you user agent to googlebot's UA
4) See what google sees
Re:What is going one here? (Score:4, Informative)
If you present different information to Googlebot than to normal users and Google finds out about it, you get kicked out of the Google index
False. Springer, the academic publisher, has dozens of paywalled journals that routinely return hits on Google that lead to pages that have none of the search terms and whose contents are inaccessible. Nor is there any metadata in those pages that would justify the hit, and I'm damned sure their pagerank isn't due to having many other high quality pages pointing at their requests for $29.95 for PDF download. The only way this is happening is if the GoogleBot is seeing something that ordinary users can't.
There is some non-obvious game being played here between Google and the newspapers, and I don't know what it is, but it doesn't smell good. This is "public policy theatre" we're watching here, which plays the same role as "security theatre": it distracts people from the real issues and makes them feel like their freedom is being taken away for a reason (yeah, ok, I'll take my tinfoil hat off now...)
Re:What really pisses me right off about paywalled (Score:5, Informative)
A fee is NOT a tax and misusing words ala 1984 just makes your arguments less credible.
No offense, but I suspect that your not aware [google.com] of the issues involved [taxpayeraccess.org]. We have already payed for the vast majority of the research articles indirectly through taxation. Outside the US this is even more true. Considering this then yes, my statement is correct that the fee is nothing more than an extra tax on top of what we have already paid for. But it is besides the point anyway - we are supposed to be talking about News, and with news I don't need nor want paywalled sites in my searches be default - they are not giving me what I searched for so why should it be included in my search results, why is Google complacent and happy to keep wasting time, bandwidth of its users?