Valve Finally Takes On Steam User Review Score Manipulation (eurogamer.net) 85
Valve is taking a step to stop developers from artificially inflating review scores on Steam. From a EuroGamer report: Valve just changed the way Steam user reviews work -- and it's certainly set the cat among the pigeons. In May, Valve updated Steam so that it highlighted recent reviews on games. The thinking behind this change was sound: it wanted to better show the current state of a game, many of which evolve quickly as developers issue updates. Now, though, Valve is changing the default review score that shows up at the top of each product page -- the one developers and potential customers put so much stock in -- so that it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
Also kicks out scores from third party purchasers (Score:5, Insightful)
... it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.
Re: (Score:2)
Re: Also kicks out scores from third party purchas (Score:2)
On the contrary, this allows a game to be evaluated on its own merits.
If the devs promised all of heaven and earth to kickstarter, but only deliver a decent game ... original backers are going to slam it. See _No Man's Sky._ It's not a terrible game, but it over-promised and under-delivered. But it's not a bad game. Average to middling. Definitely better than the 35% it currently holds.
That's a bit of an extreme example, but the concept holds true for any game.
Re: (Score:2)
Re: (Score:2)
No Man's Sky is a terrible $60 game. It's a tech demo for an important aspect of a game. As a stand-alone product, it's a neat little $10-20 indie effort that you'd wait to buy on sale for $5.
(Of course, as a console game it's much worse, since it crashes frequently - all the downsides of PC gaming - but you can't change the graphics settings to something reasonable - so all the downsides of console gaming too.)
Provide all Info, Don't Restrict It (Score:3)
If the devs promised all of heaven and earth to kickstarter, but only deliver a decent game ... original backers are going to slam it.
I'd argue that is information relevant to the review though because those same devs may be continuing to claim they will deliver amazing new features etc. in the future and so those contemplating a purchase should know that they have a history of not delivering.
The best solution is to come up with what you think is a better algorithm and then display BOTH results. This way people can judge for themselves which score is most useful in a particular circumstance. In fact this would give you some idea which
Re: (Score:2)
Re: (Score:2)
Sorry, how is that relevant to a review? When you review a game, you're not reviewing what "might become available later"....
You are missing the point of a review. The idea is to provide information that is useful to a potential purchaser of the game. Developers regularly promise that new features are in the pipeline or will be coming for a "small" additional fee as DLC because that information may encourage people to purchase the game. Hence it is useful if a 'review' includes information to help a purchaser evaluate the trustworthiness of such claims.
Having the review cover not just the game but those who made it is often v
Re: (Score:3)
Re: (Score:2)
Not only 3rd-party bundles... I guess this also goes for buying a games in a retail store and registering the CD key on Steam... hmmm...
Re: Also kicks out scores from third party purchas (Score:1)
How do you know if the game on Steam, matches (exactly) what you got on the CD? However, if you download and run the binaries from Steam (and not the CD), then you may have a point.
I think Valve should also use the "play-time" as a metric for weighting scores as well as the source of installation.
Re: (Score:1)
How do you know if the game on Steam, matches (exactly) what you got on the CD?
What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it.
Which my review of will no longer be counted apparently.
Re: (Score:2)
How do you know if the game on Steam, matches (exactly) what you got on the CD?
What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it. Which my review of will no longer be counted apparently.
I bought Fallout 4 and Far Cry Primal from Amazon on Prime Day and got actual CDs in the mail. Surprisingly, the CD versions were slightly cheaper than the digital download versions. I just had to wait 2-3 days to get them.
Re: (Score:2)
What CD? The only thing included in my retail copy of Deus Ex: Mankind Divided was a piece of paper with a Steam key on it.
Which my review of will no longer be counted apparently.
People still buy PC games in boxes in the 21st century? I'm surprised every time I hear this. You get nothing of value in the box any more, except maybe with some indie games. Oldschool games, sure, CDs are useful for games GOG doesn't have yet. Just find the patch online somewhere and you can run the game, at least once you escape Config Hell. But recent AAA games? None of that shit runs without a server playing nice. Once EA (or whoever) turns off the servers, you have a coaster anyway. Just buy
Re: (Score:1)
Re: (Score:2)
Actually, yes, that's exactly what I did.... twice.
The Fallout 4 Pip-boy edition, which comes with CDs and steam key, I installed by Steam download (because my internet connection is faster than my optical drive) and just a few days ago I read something about being able to register Games for Windows with Steam. So I took my dusty Supreme Commander with Forged Alliance bundled CD's, saw a weird looking (for Steam-key, weird looking) CD code on one of its booklets and Io and Behold, it registered this time. I
Re: (Score:1)
Because they are connected and you'll get updates through Steam and so on, It will be the same.
Re: (Score:2)
one of my initial thoughts as well.
I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.
A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ra
Re:Also kicks out scores from third party purchase (Score:5, Interesting)
one of my initial thoughts as well.
I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.
A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ratings over time), then the cost of these ratings inflation services will balloon.
That suggests an idea that they should be doing already, with data they already have access to: rather than providing a single rating score (or even two with "recent" and "overall"), provide a graph of average rating vs. time played. If the average score among people who've played it less than 20 minutes is 4 stars, but the average score among people who've played it two hours is 2 stars, that's a lot more indicative of rating inflation and what the real game is like... Conversely, if the average score among short-term users is low, but the score shoots up among people who stick with it, that may indicate a difficult learning curve that most people give up on, or may indicate that it's a niche title only for users really into that genre, etc., etc. Either way, it would be very useful information to have.
Re: (Score:2)
It wouldn't help much. There are companies that sell positive reviews, and probably have a tool to fool the system into thinking the game they are playing is active (or just ask the developer to add a startup configuration app for the game). They will have active accounts (although you could flag them as having a large steam library of product-key activated games).
Re: (Score:2)
Steam monitors achievements as well.
While it is probably trivial for the developer to add an inexpensive way to trip achievement activations based on time the game is running, that would produce a predictable pattern of achievement activations, which can be detected and enforced against.
Re: (Score:1)
Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.
Ok... They are NOT stopping people from playing those games OR using steam keys. They aren't even stopping those people from posting user reviews. They are just taking those specific reviews out of the total score shown for the game. That's it.
Re: (Score:2)
Looking at the big picture, how often do you think legit, 3rd party review sources would dramatically influence an average score anyway? If the average Steam user thinks a game is great, chances are the average person that gets their key from som
Re: (Score:3)
Re: (Score:2)
I'm not arguing it's a perfect system, just trying to put into perspective how rare this new system could negatively impact an otherwise good title.
Re: (Score:2)
Re: (Score:2, Interesting)
Good. Fuck humble bundle for selling out and selling steam keys instead of DRM-free games.
I got burned on the recent Sierra bundle, because why the fuck would old-school games NOT be DRM-free? I didn't even look for the steam/DRM-free icons, because there was no reason for me to assume anything other than DRM-free. I already HAVE all of them DRM-free, FFS!
I immediately realised after I had paid that they were only selling keys and not games, and demanded my money back.
Re: (Score:1)
I stopped buying from Humble for this reason as well. I don't know that I'm quite so angry about it, bu I'm certainly not going to give money to them when I can get the same service directly from Steam.
Re: (Score:1)
I'd chime in and note that I have over 900 games and absolutely none of them have been bought directly through Steam. Now, some have been added to my account through the Steam API (while that was available), but that's a small minority of my games. Having said that, I've also never given a review for a game so the net effect might be nothing for me in the long term.
As for another post about basically word-of-mouth, after-bundle, purchased-through-steam games, that's a great way to get better reviews if a
Re: (Score:2)
Isn't that a good thing though? If you didn't by the game on Steam, why should you be able to contribute to the rating on Steam? Amazon does the same thing, it's called a verified purchase. To allow anything else is opening up the system for abuse.
Re: (Score:2)
Isn't that a good thing though? If you didn't by the game on Steam, why should you be able to contribute to the rating on Steam? Amazon does the same thing, it's called a verified purchase. To allow anything else is opening up the system for abuse.
It's still a verified purchase... You get a Steam key on Humble (and other stores) that you then have to redeem at Steam, download the game from Steam, launch the game via Steam, etc. Steam sure as hell knows you have the game.
Re: (Score:2)
But you didn't buy it from the Steam store.... right?
Re: (Score:2)
, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores.
This is not an unintended effect per se; because it should have NO effect.
Why, for example, would people who buy steam keys on greenmangaming or desura or whatever review games statistically differently from people who buy them on steam directly.
I can't really imagine any reason why people who bought a title on storeA would systematically rate a game higher or lower than if they bought it on storeB. It is exactly the same game, and they get it on steam either way so its even the same platform. So yes, even
Re: (Score:1)
As to the problem of getting hidden gems more exposure... that's a tough problem, but reducing the amount of turds floating to the top on the backs of developer fraud and humblebundle pranksters isn't going to hurt.
I've been thinking about this one. Right now, every time I click through Steam's suggested game list, it includes 3 indie games "because they are popular" that on every detail I read on the Steam page presented to me is not worth the hard drive space to install, and absolutely not worth the $9.99 listed price. Nearly all of the big-name "because they are popular" entries do not interest me at all, but the indie games that get pushed at me are quite definitely not "hidden gems."
If the absolute indie-trash
Re: (Score:1)
Policies intended to prevent "gaming the system" always have the side effect of shutting out a few legitimate cases. They are good policies if those cases are small and the gain is large.
They can add up though. One example would be the collection of laws that regulate asset allocation following a divorce. Each provision had good motivation, but they add up to a tremendous bias in favor of the woman, leaving men with long-lasting, retirement-destroying, quality-of-life destroying financial obligations. T
Re: (Score:2)
That won't change the review scores (Score:2)
It's like taking samples to test for air quality. The samples you collect and the samples other people collect from the same location will yield the same results. But if you know and trust you
Re: (Score:1)
Actually using Steam I didn't noticed that reviews didn't mattered.
I did notice that you could change to get only Steam purchases or only keys or both and the same for languages (which isn't even based on language but the origin of the writer / language of the client / OS, I get very few reviews from Swedes though sometimes written in English if I go with the default (though my Windows and Steam is set to English so maybe just if I'm not logged in on the web or something.)
Re: (Score:2)
This might be intentional on the part of Steam to discourage publishers - especially smaller publishers - from selling their products via HumbleBundle, Bundlestars, CDKeys or any other venue since Valve gets no (or much less) revenue from sales made outside of Steam. These other markets are important to smaller developers, who want to make their game as widely available as possible, but if it costs them recognition on Steam - the Walmart of digital PC game distribution - then they might think twice about se
Re: (Score:2)
Perhaps, but it really depends on the number of false reviews, as opposed to people who got the items through those sources. A developer with a lot of keys to pass out may well trump those who obtained their keys in those other ways.
You might think, however, that Steam might have a way to tell where they keys come from and it would make sense to block sources of keys more specifically. Of course, since that would be an obvious solution, they probably have no way to do that or they would have. It might ma
Anything to address spam? (Score:2)
Re: (Score:2)
Re: (Score:2)
I bet you are a real blast at parties... /eyeroll
SubjectsSomethingSomething (Score:1)
Even if I purchased the game off HumbleBundle/gmg/g2a; I'm still getting a better deal than purchasing off Steam, so that would make me less likely to complain over the quality of the game compared to a full priced copy from steam.
Re: (Score:2)
That review would still be far more relevant than an astroturfing campaign.
Re: (Score:1)
It would, but it seems likely Steam has no way to differentiate between the two (well for now).
Re: (Score:2)
The problem is mainly greenlight/indie games that are doing this. So really all steam has decided to do is use a nuke where a small hammer would work.
Re: (Score:1)
You can also just buy the thing, and then return it if it sucks. Steam gives you two hours of play, used within two weeks of purchase, to be guaranteed a grant on a return request.
Indie Gold (Score:1)
What is steam? I thought it was gaseous water! (Score:1)
No knowledge of this steam or the other thing mentioned. Is it critical to life?