Some crude, semi-quantitative analyses on game length, price and quality

There are always debates about what kinds of factors should be considered when reviewing a game. Should game length be taken into consideration? Is it better to play a ONE 30-hour game or THREE 10-hour games? Should the price of the game affect the review score? Should there even be a score at all?

In my mind, the right answer depends on what kind of review we are talking about. If we are discussing just the creative or artistic merits of a game, or some interesting ways on how technology is designed and implemented, then the price is not important, as Adam Sessler explains in this great video.

But if, however, the purpose of the review is to provide some sort of practical consumer information, say for a kid with \$60 to spend over a few weeks of holidays, then the kid would need to judge the price of the game against its length. But these days many gamers actually prefer short games, or are less tolerant of fillers and grinding. Game length recently has been a topic of discussion lately, for example at PBS/Game show, and further continued by Polygon. So these need to be factored out. And of course, one wants to know whether the game is any good in the first place.

Let’s try to figure this out by attempting to quantify these things fairly, using information extracted from 30 games in my Steam library.

To start, let’s take a naive assumption that the figure of merit would be the number of hours per dollar. This is naive especially if we are aware that game length does not equal quality; We will gradually refine the quantification as we progress in this article. Calling the ratio of hours per dollar as $r$, we have

$r=\frac{h}{p}$.

which is the length of the game in hours ($h$) divided by the price of the game ($p$).

As a preliminary test of the formula, I took a look at my games on the Steam library, which records my hours spent and also the amount I paid for the game. Now if all goes well, the games with the highest $r$ should be the best value-for-money games. Doing the calculations, I got this:

Table 1: (Click to enlarge) Hours per dollar at purchase price

Apparently the game which got me the most bang for the buck was Sleeping Dogs, because I happened to buy it during a Steam sale at a heavy discount. I played through the main story twice which contributes to a total 66 hours of gameplay.

This list is not entirely fair because the games here were purchased at varying discount rates, while some were bought at full price. To quantify or rank the games against each other, it would be best to take the original, non-discounted prices. Ideally we should take the price at launch, since prices drop over time. But it’s too hard to track down the launch prices of all these games. Anyway I rarely buy games at launch, since my PC is usually behind on the specs. Instead I looked up the current price of each game (as of 5 Jan 2015, the time of writing, where the Holiday sale just ended and Daily Deals involve none of these games) and recalculated: (click to enlarge)

Table 2: (Click to enlarge) Hours per dollar at current price at 5 Jan 2015

Now, Brothers: A Tale of Two Sons, Spec Ops: The Line and Gone Home are the best gaming experiences I’ve ever had. I certainly didn’t regret buying these games and I really enjoyed playing them. So why are they so low on the list while Assassin’s Creed 2 and Borderlands 2 claiming the top two spots? I actually didn’t enjoy Borderlands 2 because of the grinding, and the enemies takes forever to kill. Other players might enjoy the shooting and the looting, but I found that clicking on an enemy for 5 minutes until its health drops to zero to be really boring. No. It appears that Borderlands 2 was No. 2 because I spent 57 hours on it. And out of that 57 hours, there was probably just some fraction of it where I had fun with the funny story and quirky characters. Most of it was just grinding and Borderlands needs to be penalized for that.

On the surface, the length of a game might be important because it reflects the amount of content. But a long game could possibly mean lots of grinding. Sometimes I found myself slogging for hours to level up or collect enough items to get to the “fun” part of the game. There should be some distinction which removes the “grinding hours” from the total gameplay hours, leaving behind some pure “fun hours” which would be more accurate.

So here comes the subjective part of my analysis. I am going to quantify the amount of “fun” I had in the game. Quantifying “fun” sounds funny but it’s essentially a review score without taking price and length into account. Also, instead of a single numerical score, I’ll introduce two quantities: The entertainment factor $f$ and a grinding factor $g$. The entertainment factor is basically my judgment on the gameplay mechanics, story/writing, and aesthetics; in short, the general quality of the game (normalized to maximum $f=10$). The grinding factor is my estimation on how much the game is padded. I consider a quest that makes you search a level to find 6 items instead 1 to be padding (*cough*Borderlandsweaponparts*cough*). In Assassin’s Creed you waste a lot of time collecting feathers and opening boxes. these might be fun to completionist gamers but certainly not for me. So these games will have a grinding factor less than 1, which reduces its overall score. On the other hand games like The Walking Dead are purely story-driven and has almost zero grinding, as are many indie games. So they get a grinding factor of 1. I will also divide the hours by the number of playthroughs, $c$, to standardize game length as the number of hours per main campaign. Taking all these into this account, I shall define the subjective value $v$ as:

$v=\frac{hfg}{pc}$

Assigning $f$ and $g$ to my games based solely on my personal opinions, I got this: (click to enlarge)

Table 3: (Click to enlarge) Subjective value

My top 5 games now are Assassin’s Creed 2, Sleeping Dogs, Dust: An Elysian Tail, Portal 2 and Arkham Asylum. I agree with the list a little more now. But generally the higher ranking games are the ‘commercial blockbusters’. Many of them are AAA games which I had fun for the most part, but some games, which hold special places in my heart like The Swapper and Spec Ops: The Line shouldn’t be that low on the list. Even though I assigned the “fun” and grinding factor myself, for some reason I still think The Swapper should be higher — why?

I wondered about this for a while, and the best answer I can put into words is this: Because of artistic merit. Sometimes a game doesn’t look too good, or is a little buggy, but has a real gut-punch ending which you will remember for a very long time. Some games have stories that made you cry. Or there are games that is really creative in making gameplay mechanics that are integral to the story (like The Swapper), or has something happened in the game that inspires you.

So, moving into even more arbitrary/subjective scoring, I shall assign another new quantity, called artistic merit $a$ (normalized arbitrarily to maximum $a=10$). I give a game high artistic merit if it is fulfilling or enriching in some way, or has imparted something which stayed with me long after I have finished the game, or that I remember some narrative aspects which guide me in my own fiction writing.

This so-called “artistic merit”, instead of multiplying all the other quantities, should be added as a new term instead:

$v=\frac{hfg}{pc}+a$.

The reason of adding to $\frac{hfg}{pc}$ instead of multiplying is that the artistic merit is something that transcends the other parts of the game. Thus it should not be affected by price, length, or surface quality. Brothers: Tale of Two Sons is really short, but is considered one of the best games because of the emotional bonds you form with the characters. Also historical landmarks like Portal should have a higher score than, say, Assassin’s Creed III. Now assigning “artistic merit” while keeping a straight face, the result is: (click to enlarge)

Table 4: (Click to enlarge) Subjective value with “Art” factor

Therefore according to my final results, my top 30 games are:

1. The Walking Dead Season 1
2. Portal 1
3. The Walking Dead Season 2
4. Brothers: A Tale of Two Sons
5. Assassin’s Creed 2
6. Gone Home
7. Bastion
8. The Swapper
9. Portal 2
10. Sleeping Dogs
11. Spec Ops: The Line
12. Dust: An Elysian Tail
13. Mass Effect 1
14. Contrast
15. Tomb Raider
16. The Stanley Parable
17. Far Cry 3: Blood Dragon
18. Arkham Asylum
19. Braid
20. Assassin’s Creed Brotherhood
21. Dishonored
22. Arkham Origins
23. Mark of the Ninja
24. Borderlands 2
25. Borderlands 1
26. Deus Ex: Human Revolution
27. Assassin’s Creed 1
28. Middle Earth: Shadow of Mordor
29. Assassin’s Creed 3
30. Hitman: Absolution

This is the list I agree with most, based on my personal tastes, with the price and game length taken into account. There are still a few things I found out of place, for example Tomb Raider should be higher (I really loved that game) and Bastion sits higher than I would have liked. The bottom 3, while getting good reviews, are indeed my 3 most disappointing games.

Looking at the table, it appears that generally the “Art” factor is the dominant factor for the games’ position on the list, but there are many exceptions. For example, Assassin’s Creed 2 takes fourth place despite having “Art” factor of 4, because it doesn’t have a particularly memorable story (something about the Pope?), but I did have lots of fun exploring 15th century Florence and Venice. Same for Sleeping Dogs. I didn’t like the story as much as, say, Mass Effect, but I had so much more fun driving around Hong Kong and fighting thugs.

The quantitative reason why the “Art” factor was so dominant was the arbitrary maximum score of 10. I chose 10 because it was in the same order of magnitude as the values in Table 3.

Of course, like all other review or rankings, my list was very subjective and was created based on what I look for in a game. If you are not interested in story or artistic merit then maybe $a$ is not relevant to your formula. Or if you’re really rich then the price is irrelevant.

At the end of the day, all of this could be bullshit and you might say a perfect review system probably does not exist. If anything, I learned a lot from my gaming habits and tastes by trying to articulate the criteria and seeing if the resulting list matches my subjective gut feeling. My over-reliance on “Art” factor tells me I’m chasing for some emotional or experiential high in video games. The end credits song at Portal made me pick up the guitar which I have not touched in years to learn it. I got a bit teary-eyed at the end of Brothers.

It is also clear that I prefer third-person shooters over first-person, especially with Far Cry and Dishonored placed below average on my list. I seem to be hating on Borderlands 2 too much because I’m not a fan of the shoot-and-loot RPG aspects of it, but it is actually a testament of how much ENJOYED the writing and humor by Anthony Burch. So much so that I played through a genre which I didn’t like just to see the writing and characters. Tiny Tina is one of my favorite video game characters of all time.

The fact that Middle Earth: Shadow of Mordor, Deus Ex: Human Revolution are near the bottom (even if “Art” isn’t taken into account) despite very good reviews elsewhere shows that I get bored easily playing the same mechanic over and over again.

There are some possible improvements that could be used to further refine the quantification:

1. The distinction between “Entertainment” and “Art” is fuzzy. When assigning “Entertainment” to the games, I mainly considered the presentation and gameplay mechanics. It appears I really like the combat system in Sleeping Dogs, Dust and Arkham Asylum. (Arkham Origins didn’t feel as good as Asylum, but I may have been to harsh in giving it a 4.) Mass Effect was amazing but I hated the Mako sections of the game.
2. The maximum score for “Art”: Because this is a separate term from $v=\frac{hfg}{pc}$, it is important to choose an appropriate scale when adding two quantities to get a meaningful result. I let 10 be the maximum “Art” score, but I’m not sure if that is the right one. I’m not too sure about this one.
3. Grinding factor: This is again subjective because different people might enjoy grinding differently. However a more accurate determination might be possible for games which tell you the completion percentage in the main menu. Perhaps starting a playthrough that only finishes the main story (plus sidequests, maybe) without finding all the hidden secrets/and collectibles. Then the completion percentage tells you the grinding factor. Or, to suit for more personal tastes, just play the game like you’d normally play it and check the percentage completion once you felt you’ve played enough. I remember stopping Arkham Asylum at about 70% completion, so I assigned 0.7 grinding factor to it. (I did not collect all the Riddler trophies, and I barely unlocked any challenge maps and predator modes.)

Whatever the case may be, experienced gamers who want to make informed decisions in choosing his/her games should know the styles and tastes of their favorite reviewers. As far as I can tell, it is the best way to make wise choices in choosing our games.