Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
Your Rights Online

Valve Finally Takes On Steam User Review Score Manipulation (eurogamer.net) 85

Valve is taking a step to stop developers from artificially inflating review scores on Steam. From a EuroGamer report: Valve just changed the way Steam user reviews work -- and it's certainly set the cat among the pigeons. In May, Valve updated Steam so that it highlighted recent reviews on games. The thinking behind this change was sound: it wanted to better show the current state of a game, many of which evolve quickly as developers issue updates. Now, though, Valve is changing the default review score that shows up at the top of each product page -- the one developers and potential customers put so much stock in -- so that it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
This discussion has been archived. No new comments can be posted.

Valve Finally Takes On Steam User Review Score Manipulation

Comments Filter:
  • ... it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.

    Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.

    • The other issue arises when developers decide to screw over the original backers. Release the bare minimum to get their kickstarter payments, then do a 180 on all their promises (because who cares about the original backers, they already got their money from those guys. Time to find a new audience).
      • On the contrary, this allows a game to be evaluated on its own merits.

        If the devs promised all of heaven and earth to kickstarter, but only deliver a decent game ... original backers are going to slam it. See _No Man's Sky._ It's not a terrible game, but it over-promised and under-delivered. But it's not a bad game. Average to middling. Definitely better than the 35% it currently holds.

        That's a bit of an extreme example, but the concept holds true for any game.

        • Except No Man's Sky wasn't kickstarted.
        • by lgw ( 121541 )

          No Man's Sky is a terrible $60 game. It's a tech demo for an important aspect of a game. As a stand-alone product, it's a neat little $10-20 indie effort that you'd wait to buy on sale for $5.

          (Of course, as a console game it's much worse, since it crashes frequently - all the downsides of PC gaming - but you can't change the graphics settings to something reasonable - so all the downsides of console gaming too.)

        • If the devs promised all of heaven and earth to kickstarter, but only deliver a decent game ... original backers are going to slam it.

          I'd argue that is information relevant to the review though because those same devs may be continuing to claim they will deliver amazing new features etc. in the future and so those contemplating a purchase should know that they have a history of not delivering.

          The best solution is to come up with what you think is a better algorithm and then display BOTH results. This way people can judge for themselves which score is most useful in a particular circumstance. In fact this would give you some idea which

        • I love No Man's Sky. I haven't played it at all, but it's responsible for this [youtube.com], which is probably more amusement than I've gotten out of some games I've actually bought.
    • by jiriw ( 444695 )

      Not only 3rd-party bundles... I guess this also goes for buying a games in a retail store and registering the CD key on Steam... hmmm...

      • How do you know if the game on Steam, matches (exactly) what you got on the CD? However, if you download and run the binaries from Steam (and not the CD), then you may have a point.

        I think Valve should also use the "play-time" as a metric for weighting scores as well as the source of installation.

        • by Anonymous Coward

          How do you know if the game on Steam, matches (exactly) what you got on the CD?

          What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it.

          Which my review of will no longer be counted apparently.

          • by Nidi62 ( 1525137 )

            How do you know if the game on Steam, matches (exactly) what you got on the CD?

            What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it. Which my review of will no longer be counted apparently.

            I bought Fallout 4 and Far Cry Primal from Amazon on Prime Day and got actual CDs in the mail. Surprisingly, the CD versions were slightly cheaper than the digital download versions. I just had to wait 2-3 days to get them.

          • by lgw ( 121541 )

            What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it.

            Which my review of will no longer be counted apparently.

            People still buy PC games in boxes in the 21st century? I'm surprised every time I hear this. You get nothing of value in the box any more, except maybe with some indie games. Oldschool games, sure, CDs are useful for games GOG doesn't have yet. Just find the patch online somewhere and you can run the game, at least once you escape Config Hell. But recent AAA games? None of that shit runs without a server playing nice. Once EA (or whoever) turns off the servers, you have a coaster anyway. Just buy

            • by Gr8Apes ( 679165 )
              At this point, I pretty much only buy GOG games with an occasional Steam game thrown in by accident more than anything else.
        • by jiriw ( 444695 )

          Actually, yes, that's exactly what I did.... twice.
          The Fallout 4 Pip-boy edition, which comes with CDs and steam key, I installed by Steam download (because my internet connection is faster than my optical drive) and just a few days ago I read something about being able to register Games for Windows with Steam. So I took my dusty Supreme Commander with Forged Alliance bundled CD's, saw a weird looking (for Steam-key, weird looking) CD code on one of its booklets and Io and Behold, it registered this time. I

        • by aliquis ( 678370 )

          Because they are connected and you'll get updates through Steam and so on, It will be the same.

    • one of my initial thoughts as well.

      I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.

      A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ra

      • one of my initial thoughts as well.

        I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.

        A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ratings over time), then the cost of these ratings inflation services will balloon.

        That suggests an idea that they should be doing already, with data they already have access to: rather than providing a single rating score (or even two with "recent" and "overall"), provide a graph of average rating vs. time played. If the average score among people who've played it less than 20 minutes is 4 stars, but the average score among people who've played it two hours is 2 stars, that's a lot more indicative of rating inflation and what the real game is like... Conversely, if the average score among short-term users is low, but the score shoots up among people who stick with it, that may indicate a difficult learning curve that most people give up on, or may indicate that it's a niche title only for users really into that genre, etc., etc. Either way, it would be very useful information to have.

      • It wouldn't help much. There are companies that sell positive reviews, and probably have a tool to fool the system into thinking the game they are playing is active (or just ask the developer to add a startup configuration app for the game). They will have active accounts (although you could flag them as having a large steam library of product-key activated games).

        • Steam monitors achievements as well.

          While it is probably trivial for the developer to add an inexpensive way to trip achievement activations based on time the game is running, that would produce a predictable pattern of achievement activations, which can be detected and enforced against.

    • Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.

      Ok... They are NOT stopping people from playing those games OR using steam keys. They aren't even stopping those people from posting user reviews. They are just taking those specific reviews out of the total score shown for the game. That's it.

    • by Ogive17 ( 691899 )
      If I read and understood everything correctly, it does not throw out these other reviews it simply does not count those towards the average score given to a game. So you can go in and review the game for others to read, it just won't get totaled with the others.

      Looking at the big picture, how often do you think legit, 3rd party review sources would dramatically influence an average score anyway? If the average Steam user thinks a game is great, chances are the average person that gets their key from som
      • Indie games and Humble Bundles. There are plenty of Indie games that don't have shit for reviews until they are included in a bundle because they simply got lost in a crowd. With these changes even if the majority says its a really fun game their opinion won't count if it was part of a Humble Bundle which is just bullshit, especially when a lot of indie games get shit on at release because of bugs that may have been fixed ages ago so those old reviews no longer reflect the current game.

        If they want to do th

        • by Ogive17 ( 691899 )
          And how often are those bundle's created?

          I'm not arguing it's a perfect system, just trying to put into perspective how rare this new system could negatively impact an otherwise good title.
          • Uhhhh a couple times a month? I have something like 60+ indie games and I have ONLY bought about 1 in 5 indie HBs, hell the one they have RIGHT NOW is a indie Gamemaker Pro bundle as you can see here [humblebundle.com] so I'd say its a hell of a lot.
    • Re: (Score:2, Interesting)

      by Anonymous Coward

      Good. Fuck humble bundle for selling out and selling steam keys instead of DRM-free games.

      I got burned on the recent Sierra bundle, because why the fuck would old-school games NOT be DRM-free? I didn't even look for the steam/DRM-free icons, because there was no reason for me to assume anything other than DRM-free. I already HAVE all of them DRM-free, FFS!

      I immediately realised after I had paid that they were only selling keys and not games, and demanded my money back.

      • by Anonymous Coward

        I stopped buying from Humble for this reason as well. I don't know that I'm quite so angry about it, bu I'm certainly not going to give money to them when I can get the same service directly from Steam.

    • by Anonymous Coward

      I'd chime in and note that I have over 900 games and absolutely none of them have been bought directly through Steam. Now, some have been added to my account through the Steam API (while that was available), but that's a small minority of my games. Having said that, I've also never given a review for a game so the net effect might be nothing for me in the long term.

      As for another post about basically word-of-mouth, after-bundle, purchased-through-steam games, that's a great way to get better reviews if a

    • Isn't that a good thing though? If you didn't by the game on Steam, why should you be able to contribute to the rating on Steam? Amazon does the same thing, it's called a verified purchase. To allow anything else is opening up the system for abuse.

      • Isn't that a good thing though? If you didn't by the game on Steam, why should you be able to contribute to the rating on Steam? Amazon does the same thing, it's called a verified purchase. To allow anything else is opening up the system for abuse.

        It's still a verified purchase... You get a Steam key on Humble (and other stores) that you then have to redeem at Steam, download the game from Steam, launch the game via Steam, etc. Steam sure as hell knows you have the game.

    • by vux984 ( 928602 )

      , one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores.

      This is not an unintended effect per se; because it should have NO effect.

      Why, for example, would people who buy steam keys on greenmangaming or desura or whatever review games statistically differently from people who buy them on steam directly.

      I can't really imagine any reason why people who bought a title on storeA would systematically rate a game higher or lower than if they bought it on storeB. It is exactly the same game, and they get it on steam either way so its even the same platform. So yes, even

      • by Anonymous Coward

        As to the problem of getting hidden gems more exposure... that's a tough problem, but reducing the amount of turds floating to the top on the backs of developer fraud and humblebundle pranksters isn't going to hurt.

        I've been thinking about this one. Right now, every time I click through Steam's suggested game list, it includes 3 indie games "because they are popular" that on every detail I read on the Steam page presented to me is not worth the hard drive space to install, and absolutely not worth the $9.99 listed price. Nearly all of the big-name "because they are popular" entries do not interest me at all, but the indie games that get pushed at me are quite definitely not "hidden gems."

        If the absolute indie-trash

    • by Anonymous Coward

      Policies intended to prevent "gaming the system" always have the side effect of shutting out a few legitimate cases. They are good policies if those cases are small and the gain is large.

      They can add up though. One example would be the collection of laws that regulate asset allocation following a divorce. Each provision had good motivation, but they add up to a tremendous bias in favor of the woman, leaving men with long-lasting, retirement-destroying, quality-of-life destroying financial obligations. T

    • It would've been smarter for them to have built in a seller code into the product key system so they could know more clearly where the key came from. Then, if some reviewer are suspected as illegitimate, they can trace the key's source, without necessarily having to block all Humble Bundle purchasers' reviews from counting.
    • As long as the gaming population which purchases from Steam is not statistically different from the gaming population which purchases from other sources, banning the latter group won't change the review statistics they collect. All it does is eliminate a sampling population that Valve has no quality control over.

      It's like taking samples to test for air quality. The samples you collect and the samples other people collect from the same location will yield the same results. But if you know and trust you
    • by aliquis ( 678370 )

      Actually using Steam I didn't noticed that reviews didn't mattered.

      I did notice that you could change to get only Steam purchases or only keys or both and the same for languages (which isn't even based on language but the origin of the writer / language of the client / OS, I get very few reviews from Swedes though sometimes written in English if I go with the default (though my Windows and Steam is set to English so maybe just if I'm not logged in on the web or something.)

    • This might be intentional on the part of Steam to discourage publishers - especially smaller publishers - from selling their products via HumbleBundle, Bundlestars, CDKeys or any other venue since Valve gets no (or much less) revenue from sales made outside of Steam. These other markets are important to smaller developers, who want to make their game as widely available as possible, but if it costs them recognition on Steam - the Walmart of digital PC game distribution - then they might think twice about se

  • Am very, very bored of the "Look for the lowercase L" copypasta spam everywhere, or ASCII tanks or whatever the current fad will be. Being copy/paste it is surely easy to detect and autoremove. This spam posts star ratings too, which will contribute to the overall standing of the game even though it's just gibberish postings.
  • Even if I purchased the game off HumbleBundle/gmg/g2a; I'm still getting a better deal than purchasing off Steam, so that would make me less likely to complain over the quality of the game compared to a full priced copy from steam.

    • That review would still be far more relevant than an astroturfing campaign.

      • by ChoGGi ( 522069 )

        It would, but it seems likely Steam has no way to differentiate between the two (well for now).

        • by Mashiki ( 184564 )

          The problem is mainly greenlight/indie games that are doing this. So really all steam has decided to do is use a nuke where a small hammer would work.

  • I play almost all indie games and, yes, I do buy through the humble bundle. I like early access and seeing what a game will become. I have ran into four or five really bad purchases where the reviews showed them as stellar, but the game was absolute garbage. This is a real problem for steam users and while I think the implementation might be a bit off (might make more sense to give a heavier weight to purchased copies rather than discount gift keys altogether because indie bundle and all), the idea that
  • No knowledge of this steam or the other thing mentioned. Is it critical to life?

You scratch my tape, and I'll scratch yours.

Working...