The Impact of Game Reviews on Game Development

[mindcycle]

At the Triangle Game Conference last Wednesday, Julianne Greer, editor of Escapist magazine, moderated a panel discussion titled ?Teaching to the Test: The Impact of Reviews on Game Development?. To explain the title, K12 teachers tend to teach what is on end-of-grade tests to the exclusion of almost everything else. The panel considered how much game development studios and publishers create games to meet the ?test? of reviews.

Their answer to the main question was ?definitely not,? though they do pay attention to what individual game fans say on forums and email. The only exceptions would be a sequel, especially if another studio did the previous game, or a licensed property, where reviews of past games for that license can give clues to what needs to be changed or added.

Benito saw fan opinion as more "pure from the heart" than the reviews, which led into a discussion of whether reviews are influenced by manufacturers. This can be overt, through junkets or other ?bribes?, or through the influence of advertising money. (Consumer Reports magazine refuses to accept advertising to avoid any appearance of influence by manufacturers.) While the panelists had heard of this kind of shady dealing, only one knew of it happening (from his days at Microprose); however, Greer stated that Escapist magazine had received such influence offers (which they rejected).

How much do reviews affect sales? I was surprised that no one cited any survey, as surely someone has investigated this question; panelists speculated that reviews have a strong influence on hard core players, but virtually no influence on casual (e.g., Wii) games, as those are impulse buys. Greer showed slides from research company EEDAR showing that certain categories of games (RPGs, Music & Rhythm, Sports) received consistently higher aggregate review scores than others, with some lagging far behind (Arcade, Skill & Chance, General Ent (such as Wii Fit), and Narrative). We have no way to know whether this is a bias from reviewers or an actual difference in game quality, though I?d suspect it?s because most reviewers are hardcore players.

Benito looks at the Wii as a "critique-proof platform". Another panelist joked that if you put the word ?Party? in a Wii title, it will automatically sell at least 200,000 copies as parents want their children to be playing ?party? games.

Read the rest of the article here: http://www.gamasutra.com/blogs...n_Game_Development.php

 

[Zenoth]

Usually reviews will unfortunately often influence the gamers themselves more than the developers.

 

[Barfo]

Wasn't it the other way around? developers and publishers paying to get good reviews?

 

[shortylickens]

If they had proper demos I think think this would be less of an issue.
I honestly believe demos are so rare these days because publishers know they are putting out crap and they dont want to hurt their sales.

 

[mindcycle]

Originally posted by: shortylickens
I honestly believe demos are so rare these days because publishers know they are putting out crap and they dont want to hurt their sales.

It's either that or they don't want to spend the time and resources developing a demo.

 

[ZzZGuy]

I no longer trust reviews from the likes of Gamespot, IGN or any similar groups. After seeing games get 100% yet faults clearly stated, the numbers are meaningless. Then there is the possibilities of biased reviews and reviews by people who have no taste for games I like.

The reviews I rely on are comments from people who have bought and played the game who seem to share my views, this site and youtube in game footage. And from what the OP posted, seems the consider anyone who knows anything about computers and looks up a game before they buy it as "hardcore", which in itself is a meaningless term.

 

[minmaster]

i take gamespot's reviews with a grain of salt. few times i noticed a questionable game gets good reviews from them and the next thing i see is that same game have a prominent AD placement on gamespot's site. how can you trust them? e.g. scarface.

 

[JoshGuru7]

The issue here is that gaming is a medium that is perhaps more subjective to the end user than a medium such as Movies.

• Bugs and glitches often vary by hardware setup. The game that you love to death may be the game that crashed many times on me and I end up hating, or vice versa. Your OS matters, your video card and drivers matter, even your soundcard and sound drivers can cause frequent crashes depending on version.

• For PC gaming, the processing power can have a very direct impact on enjoyment. An FPS played at 60 FPS will be far more enjoyable than the same FPS played at 15 FPS, and being able to turn on high quality textures, lighting, and other effects can have a big difference when it comes to realism and how immersive a game like F.E.A.R. is. People are trying to play modern games based on everything from i7/GTX 295/12GB setups to p4/7300GT/512MB setups and all manners of resolutions.

• Skill level and difficulty curves have a lot to do with enjoyment. Everybody has different skill/experience levels with gaming and most of the time you don't know what difficulty level to pick coming into a game until you've played it through once. There's also a different level of enjoyment that comes from mastering a game with a steep difficulty curve (UT) versus being able to instantly excel at a game with a shallow difficulty curve (Q3:RA).

• Many game reviewers don't actually finish the games they review. Everybody who reviews movies sees the credits, but I can't begin to list how many reviews I've seen that looked like the reviewer played the first two hours and then wrote the review. Things that really differentiate ok games from awesome games (actual replayability, multiplayer, user content) are speculated at in reviews only. I read a review for UT3 which mentioned that they wished the CTF included the vehicles - the reviewer clearly hadn't bothered to try all of the multiplayer modes or they would have noticed the vehicle CTF mode.

In short, seeing an 85/100 review for a game on website X means absolutely nothing to me. Some of my favorite games received middling reviews and there are games with 90+ metacritic scores that I completely hated. Take World in Conflict: Soviet Assault which received a 4.5 from GameSpot and I thought was completely perfect. The reason for their low score was partly that the new content was interspersed with the old content, but that wasn't a problem for me because I actually liked the original game and had a blast playing through the whole thing again with the additional soviet missions interspersed.

There is also a very inconsistent and useless method in which gaming sites deal with the concept of value. In one case they will grade $20 and $50 games the same way and pan the $20 game for not including more features. In others they will give the $50 game perfect marks for including tons of features that I don't actually care about when I actually prefer the $20 game because it cuts out all the pointless junk and presents the fundamental gameplay very well.

Lastly, how important the individual components (gameplay, graphics, story, sound, realism, imagination) of a game are is a very subjective concept. Lots of people were willing to overlook the stunningly terrible main plot in Fallout 3, but it completely ruined the game for me. Similarly I can tolerate terrible graphics if the gameplay is great but for many people they won't touch a game unless the graphics look amazing. There's no reason to think that my gaming preferences would have anything in common with random internet game reviewer Bob or even that we would experience similar bugs or crashes.

 

[Edge1]

Thanks to the OP for the post/link. Interesting. The Wii comments I totally believe, as a Wii-owning parent. I check reviews of course, but my kids are less hard to please. They like the Avatar, the game I bought got weak reviews, yet they love it.

Totally different for PC games. I used to use GameSpot/IGN as the gold standards, but now gravitate to metacritic, which contains reviews from all over which contribute to a final score, along with player reviews (I ALWAYS take those with a grain of salt).

Generally they get it right, but there do seem to be some exceptions, especially lately.

Surprisingly a lot of PC owners don't check reviews, let alone system specs. They walk into Best Buy, see all the pretty exciting boxes and think "how could this NOT be good?", then go home and try to play the game on integrated graphics.

 

[mindcycle]

Originally posted by: JoshGuru7
In short, seeing an 85/100 review for a game on website X means absolutely nothing to me. Some of my favorite games received middling reviews and there are games with 90+ metacritic scores that I completely hated.

I agree. I really loved Two Worlds, but it got pretty hammered by a good majority of review sites. I often rely on demo's to get a feel for a game, or comments from people who often play the genre. I also sometimes take a chance on a game if I think it looks cool, regardless of reviews. Typically, a few weeks after a badly reviewed game is released it will drop in price, and that's when i'll pick it up if I want to check it out. I didn't do that for Two Worlds however, I pre-ordered it, but ended up really liking it.

 

[mindcycle]

Originally posted by: Edge1
Thanks to the OP for the post/link. Interesting.

n/p

Surprisingly a lot of PC owners don't check reviews, let alone system specs. They walk into Best Buy, see all the pretty exciting boxes and think "how could this NOT be good?", then go home and try to play the game on integrated graphics.

I'm sure this still happens quite frequently, yes. Although the average Joe is moving more towards consoles now, so it's not as much of a concern for the industry as a whole. I'd say a good majority of "hardcore" PC gamers are also system builders, or at least have general knowledge of video cards, processors, and the like. The clueless ones will always exist though..

 

[BladeVenom]

If publishers let reviews influence their games, they would stop releasing buggy games.