Follow Slashdot stories on Twitter

 



Forgot your password?
typodupeerror
×

Censorship By Glut 391

Frequent Slashdot contributor Bennett Haselton writes "A 2006 paper by Matthew Salganik, Peter Dodds and Duncan Watts, about the patterns that users follow in choosing and recommending songs to each other on a music download site, may be the key to understanding the most effective form of "censorship" that still exists in mostly-free countries like the US It also explains why your great ideas haven't made you famous, while lower-wattage bulbs always seem to find a platform to spout off their ideas (and you can keep your smart remarks to yourself)." Read on for the rest of Bennett's take on why the effects of peer ratings on a music download site go a long way towards explaining how good ideas can effectively be "censored" even in a country with no formal political censorship.


In a country where you're free to say almost anything in the political arena, I think the only real censorship of good ideas is what you could call "censorship by glut". If you had a brilliant, absolutely airtight argument that we should do something -- indict President Bush (or Barack Obama), or send foreign investment to Chechnya, or let kids vote -- but you weren't an established writer or well-known blogger, how much of a chance do you think your argument would have against the glut of Web rants and other pieces of writing out there? Especially if your argument required people to read it and think about it for at least an hour? Perhaps your situation could be compared to that of a brilliantly talented band submitting a song for Matthew Salganik's experiment.

What Salganik and his co-authors did was recruit users through advertisements on Bolt.com (skewing toward a teen demographic) to sign up for a free music download site. Users would be able to listen to full-length songs and then decide whether or not to download the song for free. Some users were randomly divided into eight artificial "worlds" in which, while a user was listening to a song, they could see the number of times that the song had been downloaded by other users in the same world -- but only by other users within their own world, not counting the downloads by users in other worlds. The test was to see whether certain songs could become popular in some worlds while languishing in others, despite the fact that all groups consisted of randomly assigned populations that all had equal access to the same songs. The experiment also attempted to measure the "merit" of individual songs by assigning some users to an "independent" group, where they could listen to songs and choose whether to download them, but without seeing the number of times the song had been downloaded by anyone else; the merit of the song was defined as the number of times that users in the independent group decided to download the song after listening to it. Experimenters looked at whether the merit of the song had any effect on the popularity levels it achieved in the eight other "worlds".

The authors summed it up: "In general, the 'best' songs never do very badly, and the 'worst' songs never do extremely well, but almost any other result is possible." They also noted that in the "social influence" worlds where users could see each others' downloads, increasing download numbers had a snowball effect that widened the difference between the successful songs and the unsuccessful: "We found that all eight social influence worlds exhibit greater inequality -- meaning popular songs are more popular and unpopular songs are less popular -- than the world in which individuals make decisions independently." Figures 3(A) and 3(C) in the paper show that the relationship between a song's merit and its success in any given world -- while not completely random -- is tenuous. And if you're a talented musician and you want to get really depressed about your prospects of hitting the big time, Figures 3(B) and 3(D) show the relationship between a song's measured merit and its actual number of sales in the real world. (Although those graphs may cheer you up if you're a struggling musician who hasn't made it big yet -- maybe it's not you, it's just the roll of the dice.)

As the Richard Thaler and Cass Sunstein put it in their all-around fascinating book Nudge , where I first read about the Salganik study:

In many domains people are tempted to think, after the fact, that an outcome was entirely predictable, and that the success of a musician, an actor, an author, or a politician was inevitable in light of his or her skills and characteristics. Beware of that temptation. Small interventions and even coincidences, at a key stage, can produce large variations in the outcome. Today's hot singer is probably indistinguishable from dozens and even hundreds of equally talented performers whose names you've never heard. We can go further. Most of today's governors are hard to distinguish from dozens or even hundreds of politicians whose candidacies badly fizzled.

Is the blogosphere, or the "marketplace of ideas" in general, any different? If a random sample of bloggers were rated based on some independent measure of merit -- for example, independent ratings from a random sampling of blog readers, who were looking at the bloggers' writing samples for the first time, analogous to users in Salganik's "independent" world -- and then correlate that with the bloggers' traffic or some other measure of success, it's not hard to imagine the results would be similar to those of the 8-worlds experiment: the best often rise to the top, the very worst rarely do, but success in the vast middle would be close to random. In fact, while music listeners would have no logical reason to like a song just because others did, users in the blogosphere and other public forums would have several rational reasons to cluster around writers who are already popular: (1) errors are more likely to have been spotted and pointed out by someone else; (2) as an extension of that, others are more likely to have provided comments and other value-added content; (3) if you are the first person to spot an error, it's more important on a popular blog to point out the error and stop the misinformation from spreading, than on a minor blog that nobody has ever heard of. So the "snowball effect" of popularity in the blogosphere would be even more pronounced.

Then why do so many people believe in what Thaler and Sunstein call the "inevitability" of success based on merit, in domains like music, politics, and writing? I think it's because the belief is what scientists call an unfalsifiable one -- if the "best" acts are assumed to be the ones that end up on the top of the pile, then the marketplace has always sorted the "best" content to the top, by definition. Since the definition is circular, the premise could never be disproved by any amount of counter-evidence -- even if an act that used to be popular, suddenly falls under the radar, that could be seen as "proof" that they lost whatever magic touch they used to have, not as evidence of the arbitrariness of the market! The only disproof would be an artificial experiment like Salganik's, showing that once you get beyond a certain threshold of quality, commercial success has little relationship to independently measured merit -- but such experiments, which in Salganik's case required the cooperation of over 14,000 users, don't come along very often. And as long as most people don't realize how arbitrary the existing marketplaces are, there isn't enough demand to justify building a system that could work better -- indeed, to even justify asking the question of whether a system could be designed that would work better.

And that, I think, is how "censorship by glut" really works. It's not just the sheer amount of written content that censors small voices -- if you happen to know about a particular writer that you consider a fount of wisdom, then the existence of a billion other Web pages won't stop you from reading that writer's content. And it's not as if there aren't plenty of people who realize that success can be highly arbitrary. The problem is that as long as most people assume that the existing marketplace of ideas does a good job of sorting the best content to the top, then they'll be more inclined to stay with the most popular news sites and blogs, and even the minority who know that it's largely a lottery, will have no effective way of finding the best content among everything else, so they'll end up sticking with the most popular sites as well. Worse, as a secondary effect, most people with something useful to contribute won't even bother, if they don't already have a large built-in audience. I know plenty of people who could write insightful essays about social and technological issues, essays that would give most readers a new perspective such that they would definitely say afterwards: "That was worth my time to read it." But it wouldn't be worth it to the writers, because they know that their content isn't going to get magically sorted into its deserved place in the hierarchy.

(My own favorite blog that nobody's ever heard of is Seth Finkelstein's InfoThought, which is usually logical and insightful and is only about 25% of the time about how "nobody ever reads this blog, so what's the point". His Guardian columns are also good and usually don't have that subtext, perhaps because it's considered impolite to use a newspaper's column-inches (column-centimeters?) to complain that you have no voice.)

So can this problem be avoided, or is inequality and arbitrariness just a permanent part of the marketplace for content and ideas? You could create an artificial world that would sort user-submitted content according to some other algorithm -- and even if it didn't give good writers the fame that they theoretically deserved in the larger world, it might still provide them with enough of an audience within the artificial universe, to make it worth their time to keep writing. One option would be to use Salganik's "independence" world model, where users would read content without being able to see the ratings that other people had given to it, or without even seeing recommendations from similarly-minded friends within the system. The trouble is that without any information about what other readers liked, without any starting point to sort good content from bad content, it may not be worth the reader's time to read through all the dreck to find the occasional buried treasure. I believe about as strongly as a person can believe, that the existing marketplace for content is far from meritocratic, for example that there are probably thousands of songs on iTunes that I've never heard of but would nonetheless love -- but even I don't spend time listening to the 30-second clips of random songs on iTunes, because it takes too long to find the stuff I would like.

But I submit there is a solution -- a variant of an argument that I've suggested for stopping cheating on Digg, or building Wikia search into a meritocratic search engine, or helping the best writers rise to the top on Google Knol. The solution is sorting based on ratings from a random sample of users. The remainder of this speculation will be very theoretical, and will at times seem like a Rube-Goldberg approach to what should be a simple problem. But at each juncture, the complications to the algorithm are motivated by an argument that anything simpler would not work. At many points along the way, it will be tempting to throw up one's hands and say, "Why go to all this trouble, the existing system works well enough." But this statement is hard to quantify with any actual evidence -- unless you're just using the circular definition above, that whatever rises to the top is automatically the "best".

For music listeners, the gist of the algorithm is: When an artist submits a new song in the alt-rock category for example, the song is distributed to a random sample of 20 users who have indicated an interest in that genre. If the average rating from those users is high enough, the song gets recommended to all of the site's users who are interested in alt-rock. If the average rating is not high enough, then the artist receives a notification, perhaps with a list of comments from the listeners suggesting what to improve. As long as the initial random sample of users is large enough that the average rating is indicative of what the rest of the site's alt-rock fans would think, the good content will get to be enjoyed by all of the site's alt-rock customers, while the bad content would fizzle after only wasting the time of 20 people. If it turns out that a random selection of 20 users are typically too lazy to rate the songs that are submitted to them, you could even make artists submit $10 to have their songs rated by the focus group, and pay each of the 20 raters $0.50 each for their trouble. Artists can't withhold payment as revenge for a bad rating, so the average ratings should still be proportional to the song's actual quality.

At this point, you might object that this system suffers from the same unfalsifiable, circular reasoning as the belief that the marketplace rewards the "best" content, if the best content is the content that wins in the marketplace. If I define the "best" content to be the content that gets the highest average score in a random focus group, then of course this algorithm sorts the best content to the top, because that's how "best" was defined! But this system does actually have a non-trivial property: If you implement the system in multiple separate "worlds" (similar to those that Salganik created), then provided your focus groups are large enough to provide representative random samples, the same content should rise to the top in each of the worlds, unlike the results in Salganik's experiment.

This actually wouldn't be the case if the initial focus groups were not big enough -- then random variations in a few voters' opinions could cause many songs to succeed in one world and fail in another. So it's a non-trivial property that is not automatically true, and would not be true if you made an error in designing the system, like making the focus groups too small. But the larger the size of the random sample, the smaller the variance in the expected value of the average of their ratings, and the greater agreement you would expect between the results from different worlds.

As Salganik pointed out to me, this system does under-reward songs that might require repeated listenings over time to gain an appreciation of their qualities. But even this, strictly speaking, can be modeled in exchange for cash -- I'll pay 20 users $2 each if they listen to my song once today, once in three days, and once again a week after that (the site could stream the song to them to provide at least some likelihood that the users weren't cheating). This assumes some things, such as that repeated exposure has the same growing-on-you effect even if the exposure is forced -- but in the real world, songs often grow on you from repeated listenings that are "forced" anyway, if they're played in the doctor's office or on the radio when you don't bother to change the channel. And this might be more complicated than necessary -- often when a song grows on you, it at least interests you enough the first time you hear it, that you'd give it a positive rating on the first listen, which is all that the site requires for the song's success.

However, if you try to adapt this trick to a meritocracy for written content, you run into different problems. With a song, if you poll a random sample of users, the odds are very small that anyone being polled will be a vested interest in the success of the song, like one of the band members or one of the song's producers (assuming the population of users is large enough, and the song's producers have not been able to create a huge number of "sockpuppet" accounts to manipulate the voting). So you can assume the ratings will be free of any prior bias. But with a political post, for example, if you write a pro-Bush or anti-Bush essay, it's quite likely that among a random sample of users, there will be people who are biased to vote up (or vote down) any post that has anything good to say about the President. The essays voted to the top may not be the best-written ones, but simply the ones that pander to the most popularly held opinions.

But if the "best" essays are not the ones that receive the highest percentage of positive votes, even when polling a random sample of independent users -- which I was advocating as the gold standard for measuring merit -- then how do you define what makes the "best" essays, anyway? There are many possible answers, but I suggest: A necessary condition for being among the "best" essays would be to convince the most people of something that they didn't believe before, without resorting to tricks such as blatantly fabricating statistics or attributing made-up quotes. This is not a sufficient condition for merit -- maybe the point of view that you're convincing people of, is still wrong -- but I submit that if you're not at least changing some people's minds, then there's no point. An essay that changes a lot of people's minds in a random focus group, is usually worth reading, if only to see why it has that effect.

Unfortunately, this doesn't suggest a better way to poll users about the merit of an essay, because if you ask users, "Were you a Bush supporter before reading this essay?" and "Were you a Bush supporter afterwards?", Bush supporters are eventually going to figure out that the way to give the essay a high score on the mind-changing scale, would be to (falsely) say that they were not a Bush supporter before reading the essay, but they were one afterwards. So you'd still end up rewarding the essays that reinforce pre-existing opinions instead of the ones that change people's minds.

From here the counter-measures and counter-counter-measures get increasingly complicated. For each category of essays that a user wants to rate, such as Bush opinion pieces, you could require new users to enter their current opinion: either pro-Bush or anti-Bush. Then if they were asked to rate a pro-Bush essay, they would only be able to vote that the essay "changed their minds" by switching their registered opinion from "anti-Bush" to "pro-Bush". But Bush supporters could sign up initially as anti-Bush, just in the hopes of being part of a random focus group so they could cast their mind-changing vote for a Bush essay by changing their registration to "pro-Bush"! However, each user would only be able to do that once -- or do you allow users, after they've switched from anti-Bush to pro-Bush, to "reload" by spontaneously switching back to anti-Bush for no reason at all, so they're all set to cast a mind-changing vote for the next pro-Bush essay? Or would they only be allowed to switch back to anti-Bush, by casting a mind-changing vote as part of a random focus group for an anti-Bush essay -- thus giving a boost to an anti-Bush screed, as part of the price they pay for the next vote they cast for a pro-Bush piece? Then users could still game the system, by switching to "anti-Bush" when casting a vote for a very poorly written anti-Bush essay that they don't think anybody else will vote for anyway, and then switching back to "pro-Bush" only for the good essays that have a shot, hoping that their votes will coalesce around the decently-written pro-Bush essays and push them to the front page...

Am I over-thinking this? I submit this is an area where there's been too much under-thinking. Haven't we all been tempted to believe that the marketplace of ideas -- not to mention bands, blog posts, and business ventures -- efficiently sorts content to the place in the hierarchy of rewards that it deserves, without having any real evidence for this, except the circular definition of "quality" as being proportional to success? And the more people believe this, the more that marginalized voices will effectively be censored, even when they have something brilliant to contribute. We should at least think about ways that we could do better. Or else, prove logically that it can't be done (a logical proof can only approximate the real world, but it could show that such a pure meritocracy would be very improbable, or wouldn't work well). However I think the ideas above make it seem unlikely that a meritocracy is logically impossible. Maybe they're a step in the right direction. Maybe someone else's ideas would be better. The important thing is that a meritocratic algorithm be judged by something other than a circular definition, which simply decrees by fiat that the winning content is the best.

This discussion has been archived. No new comments can be posted.

Censorship By Glut

Comments Filter:
  • by oldspewey ( 1303305 ) on Monday December 01, 2008 @01:17PM (#25948195)
    I attribute the popularity of people like Ann Coulter - and networks like Fox News - to the fact there is a huge segment of the population that doesn't watch TV or log onto the internet to become informed ... they seek out information that validates their already-existing view of the world. Actual facts and truth might require a painful rewiring of preconceived notions.
    • Re: (Score:3, Insightful)

      by Anonymous Coward

      ... they seek out information that validates their already-existing view of the world. Actual facts and truth might require a painful rewiring of preconceived notions.

      The exact same thing can truthfully be said of those on the left of the political spectrum.

      • Re: (Score:3, Informative)

        by oldspewey ( 1303305 )

        The exact same thing can truthfully be said of those on the left of the political spectrum.

        Absolutely. There are people all across the political spectrum who are (or who seek) ideologues ... just as there are people all across the political spectrum who are open to opposing ideas and enjoy rational debate. My comment was in response to TFS which references Ann Coulter.

        • Re: (Score:3, Insightful)

          by ShieldW0lf ( 601553 )

          If you consider "the best" to be "everyone can appreciate this without effort", you'll get it from such a system.

          If you have to develop the capacity to appreciate a thing, you'll never find it from such a system.

          So, this methodology is a great way to find banal, tepid elevator music that challenges no one. If that's what you're into.

        • Re: (Score:3, Insightful)

          by Moryath ( 553296 )

          But with a political post, for example, if you write a pro-Bush or anti-Bush essay, it's quite likely that among a random sample of users, there will be people who are biased to vote up (or vote down) any post that has anything good to say about the President. The essays voted to the top may not be the best-written ones, but simply the ones that pander to the most popularly held opinions.

          Right in the article.

          And yet the post submitter scuttlemonkey and whoever approved it, decided that an undeserved cheap-

      • by compro01 ( 777531 ) on Monday December 01, 2008 @01:30PM (#25948459)

        The exact same thing can be truthfully said of a member in any position of the political spectrum, left, right, up, down, front, and back.

      • by Bill, Shooter of Bul ( 629286 ) on Monday December 01, 2008 @01:32PM (#25948479) Journal
        Yeah, its really sad when people have been through grad school and been declared Doctors of philosophy react to new information in the same manor as those who have not completed high school. We really need to do a better job teaching people how to think critically. I think Math and science does a pretty good job (well at least the physical sciences), but people are turned off by the science and/or compartmentalize the skill as only pertaining to science.
      • Re: (Score:2, Offtopic)

        by brian0918 ( 638904 )

        The exact same thing can truthfully be said of those on the left of the political spectrum.

        My, how the political spectrum have been skewed. Today's "right" is just as far to the left as the "left". They are both in favor of political and economic pragmatism, rejecting principled support of individual rights, in favor of increased welfare statism, all in an effort to buy votes from people who mistake their voting machines for slot machines.

        • Maybe if the voting machines occasionally spewed out some cash, you might get better voter turnout.
          • Re: (Score:3, Insightful)

            by brian0918 ( 638904 )

            Maybe if the voting machines occasionally spewed out some cash, you might get better voter turnout.

            Bah, the government is too smart for that! They'll promise you cash in exchange for votes - cash that they'll get from you in the first place - but they'll never actually give you that money, and certainly not so often as to "occasionally" happen. So I guess my analogy is not accurate - you're more likely to get money back from a slot machine than a voting machine.

      • Re: (Score:2, Interesting)

        by hey! ( 33014 )

        Yes. But we don't have our own network that feeds us back our viewpoint all day long. We scarcely have any print media left for that matter. It's not a profitable viewpoint.

        I suppose MSNBC might be a counter example of somebody trying to grab a distinct market segment off of Fox, and there is some legitimacy to that. But after all they took Olbermann and Matthews off their live event anchoring because they'd be perceived as biased. I think that was a good decision, but it is not something Fox would e

      • The exact same thing can truthfully be said of those on the left of the political spectrum.

        Facts don't seem to be essential to the Slashdot post or nod-up, either.

      • See the sig, that said, idiots can be found in all shapes, colours and persuasions.

      • Re: (Score:3, Interesting)

        by fugue ( 4373 )

        There is a strong correlation between those in scientific fields and those with certain political persuasions. There is also a strong tendency for science to weed out people who seek out information solely to validate already-existing views, rather than being open to absorbing a variety of pieces of information and reaching the best-supported conclusion.

        If you take a group of people who can be shown to be better than average at incorporating new information and re-evaluating preconceived notions, and dem

    • Re: (Score:2, Insightful)

      See Also: Huffington Post, Daily Kos
    • Re: (Score:2, Insightful)

      by Kamokazi ( 1080091 )

      Fox News is popular because it's watched by people who don't watch TV?

      If that's your example of 'actual facts and truth', I would think being rewired to think so illogically would be quite painful indeed.

      Seriously though, if idiots like you would quit all the insults and political stereotyping, this country would be a lot better for it.

      • Re: (Score:3, Informative)

        Fox News is popular because it's watched by people who don't watch TV?

        You might want to re-read OP's point:

        ...that doesn't watch TV or log onto the internet to become informed.

        In other words, they only want entertainment and/or reinforcement of previously held beliefs, rather than tuning in in order to actually learn something. The same is true of both left and right, as OP has already said.

        If you spent less time trying to get offended by political views that others haven't expressed, you'd probably be a much happier person.

        • by vlm ( 69642 )

          In other words, they only want entertainment and/or reinforcement of previously held beliefs, rather than tuning in in order to actually learn something.

          Why would anyone who wants to learn something, use a media that is either preaching to the choir or mindless infotainment?

          It's a circular downward spiral not a simple unidirectional cause and effect.

      • by clem ( 5683 )

        Give the parent post another read. He didn't state that Fox News is watched by a segment of the population that doesn't watch TV. He said that Fox News is watched by a segment of the population that seeks out programming that reinforces their own world view.

    • Re: (Score:3, Funny)

      by Cajun Hell ( 725246 )
      Oops, I thought the reference to low-wattage bulbs was the author praising Coulter's efficiency and greenness.
    • Re: (Score:3, Interesting)

      by Garwulf ( 708651 )

      That's not exactly uncommon, and it happens in just about every field where somebody can have an opinion. On here, the place where it tends to stand out for me is in the copyright debate - but then again, I've been a professional writer for around ten years, and a small press publisher now for two - I know most copyright issues like the back of my hand as an insider.

      There are people who don't like having their preconceptions challenged, even when there's ample evidence against them. One of my favorite mom

    • _Or_ that could just be you looking for excuses to validate your world view by looking down on people who don't believe or value what you believe and value.

      Your post seems to imply that you validate most of what you know against imperial data as opposed to other peoples opinions that you trust. If you claim that to be true I can conclude only one of two possibilities:
      Either you are a truly exceptionally talented and intelligent individual.
      or you are lying , to others, and possibly yourself.

      As for myself I

    • Re: (Score:3, Insightful)

      by TerranFury ( 726743 )

      they seek out information that validates their already-existing view of the world

      Like Slashdot readers? (Obviously I'm guilty too.)

  • by Wandering Wombat ( 531833 ) <mightyjalapeno.gmail@com> on Monday December 01, 2008 @01:19PM (#25948229) Homepage Journal
    Basically, you do what you think others want you to do. This... this is not news.

    However, it's good to see it being properly analyzed. I'll need at least an hour to think about this.
    • Re: (Score:2, Insightful)

      by Anonymous Coward

      The interesting thing about mob mentality is that you can game the system.

      Let's say you want to get ahead in some social group you belong to. If you're clever about it, you can listen to what people are saying, gossiping about, etc, and formulate theories about what individuals in the group want from others. Then you can carefully script your interactions with them so they perceive you, at least subconsciously, to be "good" (whatever their measure of "good" may be). I've always called this my "cultural cam

    • I'll need at least an hour to think about this.

      Given the size of the summary, I think I would need at least a few hours to read the article itself. Maybe this is intended as an example on how ideas can be submerged in a glut of information?

    • People has been suppressing alternative views since the beginning of time. Heck, even animals will eject non-conformists from the herd.
    • "Censorship" (for lack of a better word) is occuring not due to the innane mob network effect of the masses, but is the fault of the ranking algorithm.
      Come up with a better algorithm and merit will be more accurately and "fairly" distributed. Of course, there are a lot of related stories out there, something like the Netflix competition [slashdot.org] may produce a better algorithm, although it may end up being too damn complicated.
      I see this more as a math/engineering story; you can complain about the behavior of mobs, o

  • *thinks back to story yesterday*

  • NO SHIT (Score:4, Insightful)

    by larry bagina ( 561269 ) on Monday December 01, 2008 @01:25PM (#25948369) Journal

    slashdot moderation much?

  • Fascinating (Score:2, Funny)

    by Anonymous Coward

    It would appear that people are actually sheeps. ...

    Wait, how is that news?

    • At least sheep follow a leader with better cognitive ability then they (usually a dog).

      Humans tend to keep each other in line. Also, Many humans would rather believe in a lie than face an uncomfortable truth. Just think of how many scientific achievements have been ridiculed by the masses only to be seen as obvious to everyone years later!

    • What of those that want to be the bellwether? Let sheeple (iTunes users, perhaps) sign up to be trendsetters. You don't even have to pay them cash, but give them a free download after they've rated, say, ten new songs (and perhaps they get to keep the songs they've rated).

      An earlier commenter mentioned that the "blog" version of this might be like slashdot moderation, but even that can show the preconception bias. I'm not sure I have a good answer for this. In real psych labs, they can track how long you l

  • by ScentCone ( 795499 ) on Monday December 01, 2008 @01:36PM (#25948577)
    censorship
    a: the institution, system, or practice of censoring
    b: the actions or practices of censors ; especially : censorial control exercised repressively


    Which is not the same thing as people going with the flow, and acting like the rather lazy pack/herd animals that hundreds of millions of years of evolution has wired up.

    Having a great idea that you express below the Signal-to-Noise threshold is not the same as being censored.
    • by Aladrin ( 926209 )

      I think his whole point is that there's no real difference between the 2. There's certainly a dictionary difference, and nobody is arguing against that. But in the end, both systems see to it you aren't heard if you don't have the favor of the ruling powers.

      Personally, I've never been one to long for the entire world to know my name, so this isn't really hurting me at all. In fact, I think that most people would be better off if they didn't have world-wide reknown. They can't handle it.

      • Re: (Score:3, Insightful)

        by brian0918 ( 638904 )

        I think his whole point is that there's no real difference between the 2.

        One is a violation of individual rights, the other is not. Nobody forces you to accept the Slashdot rating system. Set your threshold to -1 and read every comment. You're sure to find gems rated 0 or 1. If, however, /. editors start deleting your comments, then you've got censorship. Even then, though, individual rights are not being violated, because they are allowed to delete comments on their own site. It would be foolish of them to try it, but they'd be justified in doing so. Only when a force-backed en

        • I think the point is that the end result is the same: an idea goes unheard. That one is morally repugnant and the other is not is a separate issue.

          • I think the point is that the end result is the same: an idea goes unheard. That one is morally repugnant and the other is not is a separate issue.

            Your ideas also go unheard if you don't tell them to anyone, but that ISN'T censorship. Censorship has very specific deffinitions, and this doesnt fit any of them.

            • Your ideas also go unheard if you don't tell them to anyone

              It's not my fault if people can't read my mind! It's The Man keeping me down, silencing anyone who tries to create a mind-reading device, thus preventing my brilliant idea from ever being realized.

              Unfortunately, my only brilliant idea is for a mind-reading device, so by the time someone actually succeeds in developing such a device to read my mind and know my brilliant idea, it ceases to be a brilliant idea.

              A cat in a box comes to mind.

      • Re: (Score:3, Insightful)

        by deraj123 ( 1225722 )
        Except there's still the difference between not being allowed to say something, and everyone ignoring what you have to say. I'd say that's a HUGE difference. Is there a difference in the effect of what you say has on society? Probably not. But that doesn't change the fact that everyone ignoring you is drastically different from censorship.
      • Does not imply the right to be heard. That's the difference. Censorship is when the government says "No, you can't say that." It is when they restrict you from being able to express what you want to express.

        However, just because you want to express something, doesn't mean anyone is required to listen. If people wish to ignore you, they are free to do so. To have it any other way would be to infringe on their rights. If you tell me I have to listen to someone, and especially if you tell me I have to agree wi

    • The definition of censorship above depends on the definition of a censor, the 4th definition below satisfies the use in the article.

      Webster's Revised Unabridged Dictionary (1913)

      1 definition(s) found

      Censor \Cen"sor\, n. [L. censor, fr. censere to value, tax.]
      1. (Antiq.) One of two magistrates of Rome who took a
      register of the number and property of citizens, and who
      also exercised the office of inspect

    • by pikine ( 771084 )

      It's not censorship but an editorial issue. I don't think it's the lack of meritocracy. The uncertainty in crowd rating is just an amplification that even professional critics can disagree randomly once the subject reaches a certain quality level.

      The only minor fault I would possibly find if I were nitpicking in the current rating system is that it typically requires a basis of comparison to a pool of existing works. However, a subject can be so unfamiliar to the audience, for example a new genre of music o

    • by ceoyoyo ( 59147 ) on Monday December 01, 2008 @02:30PM (#25949587)

      It's saying a lot about how people take freedom of speech for granted when someone writes a paper about how unpopularity is a form of censorship.

      Censorship is not when people don't want to read or hear about your idea. Censorship is when people can't read or hear about your idea because someone intentionally prevents it.

      A crappy blog not getting many pageviews is not censorship. Men in black knocking down your door and hauling you away so you stop writing your crappy blog is.

  • Maybe I misread it (Score:4, Informative)

    by Erwos ( 553607 ) on Monday December 01, 2008 @01:37PM (#25948585)

    But it seems like the real problem he's trying to solve is that current ranking algorithms don't take into effect the fact that "users" are not one segment, but rather composed of different segments with differing political, religious, sexual, ethnic, etc. tastes. That is to say, Digg's algorithms are very good if you match a stereotypical Digg profile. If you weren't, well, it wasn't so amazing.

    However, this is _hardly_ an unexplored area, and I would further submit that _Amazon_ is surprisingly good at this kind of thing. By analyzing what random samples of users bought (or, in other cases, ranked up or down), they're able to make (IMHO) often-insightful recommendations about what else you should buy. I've had thoughts about how you could make a site that would kick Digg's ass and probably be more valuable to advertisers using tagging, ranking, and some statistics, too.

  • by Doc Ruby ( 173196 ) on Monday December 01, 2008 @01:39PM (#25948617) Homepage Journal

    The failure of a group of people to communicate well does not constitute "censorship". Censorship is when someone or something selects communications for suppression. But when a room is too noisy for someone to be heard, that's not censorship. Unless a person or a group of people arranges for rooms to be noisy, with the plan to drown out some people.

    If the "censorship" is selective only of arbitrary communications, not according to content or meaning, but only according to signal strength or random chance, that's not "censorship". It should be fixed, but calling it "censorship" just makes it harder to deal with actual censorship.

    We have loads and loads of actual censorship, especially on the Internet. We should care about stopping censorship. So we shouldn't just call any failure to communicate "censorship", which makes it harder to communicate about censorship or the other interference, and therefore harder to fix either.

    • by betterunixthanunix ( 980855 ) on Monday December 01, 2008 @03:59PM (#25951131)
      It is a different form of censorship. Case-in-point: when the US government doesn't want news cameras filming caskets coming back from Iraq, they flood the news with irrelevant details about every battle fought in every small town in the middle east, so that Iraq reporters are too busy reporting on those stories to report on the number of dead soldiers on the US side. Nobody is physically stopping the media from showing those caskets, they are just giving them apparently "juicier" stories that they either have to take or be the only news network that is not reporting the story.
  • by rolfwind ( 528248 ) on Monday December 01, 2008 @01:44PM (#25948731)

    and the US, and watched the evening news - you definitely get a feel that the evening news in America is censored. This is not so much because the hide stories, but just the lack of airtime for most anything worthwhile, while fluff (Arnette's cat gets in a tree and rescued by firefighter, college sports) dominates. International events don't tend to be covered at all, unless it is really grand or some type of American involvement (1000 people die, including 12 American, etc).

    Now, I don't think this is a grand conspiracy, but it does have a dumbing down effect - I don't know if it came about because of viewer demand or a few program managers dictating what gets broadcast and other stations imitating them. In the evening news in Canada, UK, France, Germany (countries I personally traveled to) - there is definitely more awareness of what is going in internationally (or even nationally).

    • by Solandri ( 704621 ) on Monday December 01, 2008 @02:34PM (#25949675)

      The U.S. is so large and economically dominant that it's very easy to live here without ever knowing what's going on in the outside world at large. Very few things that happen abroad have a noticeable effect here. The U.S. has a GDP of $14 trillion, but imports and exports account for just $3.1 trillion, a 4.5 to 1 ratio. (Please note that GDP is gross domestic product and calculates exports minus imports, not the sum of imports and exports.)

      It's very different elsewhere. Most countries are small or have significant economic ties to their neighbors. Canada has a GDP of $1.3 trillion, with imports and exports accounting for $850 billion in trade - a 1.5 to 1 ratio. Germany has a $2.6 trillion GDP vs. $2.1 trillion in foreign trade - a 1.2 to 1 ratio. So international news and events have a much greater impact on their citizens' everyday lives.

      That said, I do agree the news broadcasts here are pretty pathetic. It seems the news stations cater to what people want to watch, instead of what's important. In terms of marketability, it would seem the fluff piece about Annette's cat in the tree gets better ratings than coverage about some terrorist attack in Mumbai. It's the only explanation I can think of for the existence of such shows as Jerry Springer.

      • Re: (Score:3, Interesting)

        I hadn't ever thought about it that way: if the world pretty much revolves around you, you're not really self-centered.

        However:
        >It seems the news stations cater to what people want to watch, instead of what's important.

        To the news broadcasters, what people want to watch *is* what's important.
        The problem is that that's self-amplifying. If a lot of people wanted to watch educational, world-centric news, they'd provide that, and because it's available, more people would start to watch it. And, indeed, tha

  • by girlintraining ( 1395911 ) on Monday December 01, 2008 @01:44PM (#25948739)

    "Popular people get noticed. Unpopular people don't. Sorry if you're in the second group."

  • Comment removed (Score:5, Insightful)

    by account_deleted ( 4530225 ) on Monday December 01, 2008 @01:47PM (#25948795)
    Comment removed based on user account deletion
  • Wow, when will we ever get this? The lower wattage bulb comment is obviously flamebait. We get dupe's (redundant) stories all the time. This is a feature /. needs. I want to browse the frontpage with my story threshold at 2. Yeah, I know we can pick catagories, but that's not what I'm looking for.
  • Simple solution:

    Rank the users also.

    That is, someone that has an account for 4 years and been "Pro-Bush" all that time carries full voting rights of say 100 shares. But when you first sign up or switch votes to anti-bush, suddenly you go back to having '1 share', doubling to 2 in a month, 4 in 3 months, 8 in 6 months, 12 in a year, 24 in 18 months, 48 in 2 years, 100 in 4 years.

    But with regards to the general idea, Amazon/Netflix already beats your base idea.

    That is, they use a bayesian probability fo

    • It wasn't done because they thought it made a good ranking system or business model. It was a psychology study about social influence and whether or not songs have "objective" quality. Get a clue.
  • It's the best example of network effect [wikipedia.org] in action. This is also why we have a long way before "the year of linux desktop".
  • Social Proof (Score:5, Insightful)

    by Missing_dc ( 1074809 ) on Monday December 01, 2008 @01:54PM (#25948939)

    My friends and I call this "Social Proof"

    in a nutshell:

    Partially due to our fast paced society(or perhaps amplified by it) people cannot take the time to learn about and judge things or people for themselves, so they use social indicators to determine worth.

    For example, seeing a well dressed well groomed individual vs their unkempt shabbily dressed twin. People tend to assume much better things about the well dressed twin simply by manner of his appearance.

    Another example, if you go out for drinks with an attractive coworker or friend of the opposite sex and the two of you are seen laughing and joking and having fun, your social value is increased in the eyes of the onlookers, they figure if this other person has taken the time to form a positive opinion of you, then you must have some desirable qualities, and they will be more receptive of your attention. This seems to be a mostly subconscious effect.

    Before I got married, I used to have several hot chicks that I would go party with, knowing that being out with a hot chick made it easier to pick up other chicks.

    People tend to be sheep.

  • I don't know whose bright idea it was to put a link to Ann Coulter in a slashdot article, but if there were any kind of moderation of the top-level, the article should be moderated "Troll".

    (If it were about slashdot comments, would there have been a link to goatse?)

    • by Toonol ( 1057698 )
      And referenced incorrectly, as well. You may feel Ann Coulter is evil, is deceptive, is ugly... whatever. But if you think she's a "dim bulb", you're wrong. She is far from dumb. Viciously intelligent would be a better description.

      That explains some of the other questions raised, by the way. Most governors are exceptional people. They have incredible talents and skills. They just aren't talented and skilled in the areas that we want them to be (leadership, integrity, common sense). They're talent
  • hilarious (Score:4, Insightful)

    by circletimessquare ( 444983 ) <circletimessquare.gmail@com> on Monday December 01, 2008 @01:58PM (#25949023) Homepage Journal

    what is posed as a philosophical breakthrough is simply nothing more than not understandning the goddamn meaning of the word "censorship"

    a high noise to signal ratio is not the same thing as censorship

    that's some pretty fruitless philosophical gymnastics there son

  • We can go further. Most of today's governors are hard to distinguish from dozens or even hundreds of politicians whose candidacies badly fizzled.

    Sarah Palin explained!

  • Anne Coulter just made a buttload of ad money.
    Thanks slashdot.

  • by macraig ( 621737 ) <mark.a.craigNO@SPAMgmail.com> on Monday December 01, 2008 @02:05PM (#25949167)

    The Slashdot comments system is a spot-on example of what Haselton describes: if one doesn't manage First Post or relatively close to it, the likelihood that your insightful/informative/funny comment will be widely read and modded-up decreases proportionately. People just don't have the time or stamina to read hundreds of comments, normally; they read just the first few dozen "visible" (highly rated) ones and then quit. If in fact that is the case, then being late to the party means that the quality of your comment is irrelevant because it will be drowned-out by the flood that preceded it. Really it's the people who are able to jump in and suck on the Firehose that get most of the attention here. I've been frustrated by this quantitative factor - what Haselton calls the "glut" - for a long time.

  • The solution the author presents is not entirely unlike an idea I've had on my own, but applied to a completely different realm: moderation of internet forums. Many people have noticed that a site tends to coalesce toward a particular "group think" as it goes along (Slashdot hates copyright; every political blog is either left- or right-leaning; etc.)

    My idea goes in two stages: in stage one, a new user can only indicate whether they agree or disagree with a comment. Once the system can, by comparison wit

  • by rodentia ( 102779 ) on Monday December 01, 2008 @02:13PM (#25949295)
    This is an exceptionally pernicious metaphor. We do often prefer one idea to another, but a market does not exist. One idea is right and one is wrong and the choice is usually a false one. Choosing what others have chosen is a CYA tactic and not a way to conduct one's intellectual life. This result demonstrates man the social animal impeding man the rational animal.

    These are not new problems and are not limited to democracies of taste or meritocratic capitalism. One of the more interesting results was the *new band* question. Participant is asked if they had heard of these four new bands, one of which was spurious. The profile of recognition was statistically identical to that for the three real, but little known, new bands. Respondents need to be seen as knowing, whether they have actual knowledge or not. This makes clear that musical taste as a function of personal identity formation and not music appreciation. The big labels have know this for years: it doesn't matter who you front as long as you flood the airwaves and hype the sucker.

    That said, there are a handful of people in all times and places who do not consider themselves tied to their peer's taste; who strive to think for themselves. They usually have unique access to actual ideas. They are often shunned by their peers because they call into question the intellectual shorthand everyone else contents themselves with. They are either crackpots or geniuses, sometimes both. One thing they never are is boring.
  • Rate the raters (Score:4, Interesting)

    by Squiffy ( 242681 ) on Monday December 01, 2008 @02:16PM (#25949355) Homepage

    In answer to the question about how to rate blog essays, I suggest that we need to rate the raters. How do we do that? I think a system can be built into the threads of discussion in response to an essay. If people rate your comment highly, it increases your standing as a rater, and your ratings figure more strongly into the rating metric. But people can't just rate. They must also supply, in the form of a comment, their reasoning behind the rating, which opens their comment and rating to responding comments and ratings, and so on. If people read and understand the terms of comment submission so they know that the point of the site is to rate the quality of reasoning, not the flavor of ideology, the system should correct itself.

    Then again, this system assumes that people will behave rationally, which is dubious, as any economist or divorce lawyer will tell you.

    • 1) by the same mechanism described in the article, raters will be randomly rated highly or overlooked.
      2) highly rated raters will age and slowly lose touch with what is relevant. Their rater rating will fail to reflect this. You see this with current arbiters of popular culture.

  • Rating algorithms (Score:4, Interesting)

    by bendodge ( 998616 ) <bendodgeNO@SPAMbsgprogrammers.com> on Monday December 01, 2008 @02:21PM (#25949443) Homepage Journal

    While I admit I haven't spent nearly as much time thinking or writing about this as Haselton has (he seems to have a great deal of time), I do like this paragraph particularly:

    And that, I think, is how "censorship by glut" really works. It's not just the sheer amount of written content that censors small voices -- if you happen to know about a particular writer that you consider a fount of wisdom, then the existence of a billion other Web pages won't stop you from reading that writer's content. And it's not as if there aren't plenty of people who realize that success can be highly arbitrary. The problem is that as long as most people assume that the existing marketplace of ideas does a good job of sorting the best content to the top, then they'll be more inclined to stay with the most popular news sites and blogs, and even the minority who know that it's largely a lottery, will have no effective way of finding the best content among everything else, so they'll end up sticking with the most popular sites as well. Worse, as a secondary effect, most people with something useful to contribute won't even bother, if they don't already have a large built-in audience. I know plenty of people who could write insightful essays about social and technological issues, essays that would give most readers a new perspective such that they would definitely say afterwards: "That was worth my time to read it." But it wouldn't be worth it to the writers, because they know that their content isn't going to get magically sorted into its deserved place in the hierarchy.

    I agree that there seems to be a lot of mob mentality and snowballing in Internet writing, but I think there are some external factors that are left out of his analysis. I think that the large chunk of people who 'can't be bothered' to contribute don't contribute because they have a personally successful life. I know it's gross stereotyping, but it seems as though the bulk of people who spend their time spouting ideals (Communists, OSS giants, pop stars, Obama) have done little to none of what society considers real work. These people have far more free time than personally ambitious, hardworking people who pursue personal success instead of a career in changing the world. Thus, these people who have too much time on their hands distort the written contents of the net. (Please keep in mind that this is a draft of a 5-minute theory, so it's sure to have some holes.)

    As far as remedies go, I think rating algorithms need to be much more sophisticated. For example, 5-star scales could calculate the rating based on the mean of the mode star and its two neighbors' frequencies.

    For simplicity, let's assume one person clicked 1, two people clicked 2, three people clicked 3, etc. This method would discard stars 1-3 and calculate a display rating of 4.5, instead of a simple mean of 3.6. By totally discarding far-out ratings, we might be able to keep ratings from all gravitating to the middle. This is another 5-minute theory, and I'm not a math whiz, so I'm sure there's a better/simpler way to implement a deviation scheme like this, but it's a thought.

    Hmm, for added fun, try taking ALL ratings in a database and adjusting them all on a curve! But that's liable to guzzle server resources...

  • You are only allowed to rate a song if you haven't looked at the current rating for the song. Ratings are only available after a threshold. Unfortunately, this means that you would be subject to listening to a lot of crap, if you want your opinion heard. Or, in Slashdot terms, you are only allowed to moderate responses in a story if you browse with moderation hidden.
  • Slashdot is a clear example of the Salganik effect. Watch as some random post gets modded +5 insightful, not because its any more insightful than the next post, but because some moderator gave it +1 and it just snowballed from there.
  • by Tekfactory ( 937086 ) on Monday December 01, 2008 @02:55PM (#25950097) Homepage

    Excuse me if this seems to ramble.

    I once read an article about Rating systems, ones that were resistant to gaming the system, unlike eBay's rating system. The system in questions rated things positively that you rated highly and negatively those that you didn't. Over time it tended to only show you things that were rated highly by people who rated things similarly to you.

    This leads to clustering of people with similar viewpoints, but lessens the effect of sockpuppets, trolls and griefers. They would have to be rated positively by enough people consistently in the same cluster to game the system.

    I wasn't looking at this for something like eBay, but rather an MMO. I also wonder sometimes about a Firefox plugin, but I digress.

    I'd like to further refine this system based on my experience with Amazon's recommendations. I and some of my friends have noticed if we buy a very new or niche publication we will get wierd and uneven recommendations off that purchase until enough people buy the book to smooth out the recommendations.

    Unfortunately Amazon only has "I own this" and "not interested" as responses. It doesn't have enough dimensions, and doesn't factor in reviews at all. I buy one video in a series, and I get recommendations for that series, and other series that are similar. When I say I am not interested in that other series episodes, say season 1 I still get recommendations for the rest of the series. I would like to be able to 'deny all' but I can't. If I wanted to tell it not to recommend horror movies, I can't.

    Likewise if I saw something online I didn't want to read, and I consistently didn't like, I'd filter it, not one blog post, but the author.

    Ok, so here I am ideally, rating things and filtering things, until I am at last, as the parent writes in my own "world" suddenly CNN, BBC, and Joe blogger have an equal voice because they are narrowcasting straight to my own little insular 'bubble' on the internet.

    To which I'd like to add we need more control, and more dimensions on this filtering thing if it's really supposed to work. I'd love to mod up stories 'Thought-Provoking', or 'I want my 5 minutes back'.

    I think there is an answer out there, and I think it has something to do with self-organizing systems.

  • by danwesnor ( 896499 ) on Monday December 01, 2008 @02:56PM (#25950115)

    and you can keep your smart remarks to yourself

    You've obviously never been to Slashdot before.

  • Intriguing... (Score:3, Interesting)

    by crmarvin42 ( 652893 ) on Monday December 01, 2008 @03:23PM (#25950583)
    I wonder how many /. readers think of themselves as being a member of the "Merit" group instead of a member of the "Social" group because they (mistakenly?) believe that they aren't effected by hit counters since they don't consciously pay attention to them.

    Taking it one step further, I wonder how many of the group above use that as personal validation that their opinions are "Correct" and everyone elses are "Wrong".
  • by quixote9 ( 999874 ) on Monday December 01, 2008 @03:38PM (#25950815) Homepage
    No, seriously. Haselton wants to tap into a wisdom-of-crowds effect to find the good stuff we're missing. So long as opinions are independent, many people do converge on the right answer more often than few people.

    In the good old days, getting enough people to see or hear a piece of creative work was a logistical nightmare. Using the kind of "peer review" he's talking about would have been impossible, even though it's a really promising approach. But the web could make it easy. It's the same kind of quantum shift, with equally huge ramifications, as the way the printing press made ideas accessible to many more people than before.

    Facilitating good ideas and making them visible pretty much defines a civilization. Finding a way to get good ideas known is about as non-trivial as it gets. Because even though developed countries have grown rather good at the facilitating part, we're still wasting 99% of our good people at the visibility end.

    You may have noticed by now that good people are hard to find. It'd be like climbing out of the Middle Ages if we stopped wasting 99% of them.
  • by clone53421 ( 1310749 ) on Monday December 01, 2008 @03:39PM (#25950841) Journal

    tl/dr.

    But in all seriousness...

    the only real censorship of good ideas is what you could call "censorship by glut". If you had a brilliant, absolutely airtight argument that we should do something -- indict President Bush (or Barack Obama), or send foreign investment to Chechnya, or let kids vote -- but you weren't an established writer or well-known blogger, how much of a chance do you think your argument would have against the glut of Web rants and other pieces of writing out there? Especially if your argument required people to read it and think about it for at least an hour?

    This is slashdot, for crying out loud! ... the land of armchair generals, amateur lawyers, and anonymous cowards!

Pause for storage relocation.

Working...