Steam News

User Reviews Revisited
16 MAR @ 5:47AM	- TOMB
Some time ago we made some changes to how we presented the User Reviews for games, and their resulting Review Score. We talked about those changes in this blog post. As we describe in that post, we want to ensure that players who've played a game can voice their opinions about why other people should or shouldn't buy the game, and that our summary of those opinions into a single Review Score should represent the likelihood that a future purchaser will be happy with their purchase.

Since that post, we've continued to listen to feedback from both players and developers. It's clear to us that players value reviews highly, and want us to ensure they're accurate and trustworthy. Developers understand that they're valuable to players, but want to feel like they're being treated fairly. We've also spent a bunch of time building analysis tools to help us better understand what's happening in the reviews across all titles on Steam. With that feedback and data in hand, we think we're ready to make another change.

That change can be described easily: we're going to identify off-topic review bombs, and remove them from the Review Score.

But while easy to say, it raises a bunch of questions, so let's dig into the details. First, what do we mean by an off-topic review bomb? As we defined back in our original post, a review bomb is where players post a large number of reviews in a short period of time, aimed at lowering the Review Score of a game. We define an off-topic review bomb as one where the focus of those reviews is on a topic that we consider unrelated to the likelihood that future purchasers will be happy if they buy the game, and hence not something that should be added to the Review Score.

Obviously, there's a grey area here, because there's a wide range of things that players care about. So how will we identify these off-topic review bombs? The first step is a tool we've built that identifies any anomalous review activity on all games on Steam in as close to real-time as possible. It doesn't know why a given game is receiving anomalous review activity, and it doesn't even try to figure that out. Instead, it notifies a team of people at Valve, who'll then go and investigate. We've already run our tool across the entire history of reviews on Steam, identifying many reasons why games have seen periods of anomalous review activity, and off-topic review bombs appear to only be a small number of them.

Once our team has identified that the anomalous activity is an off-topic review bomb, we'll mark the time period it encompasses and notify the developer. The reviews within that time period will then be removed from the Review Score calculation. As before, the reviews themselves are left untouched - if you want to dig into them to see if they're relevant to you, you'll still be able to do so. To help you do that, we've made it clear when you're looking at a store page where we've removed some reviews by default, and we've further improved the UI around anomalous review periods.