(Part of Loaded Dice.)
In August 2012 I fetched the BoardGameGeek dataset again, so that I could make comparisons with the 2011 dataset. A summary of my findings:
We'll give a game from 2011 a worse rating now, than we did when it was new. We'll give a 2010 game a worse rating than we did in 2011. This "cult-of-the-new effect" is detectable in games published as far back as 1997.
Despite that, the ratings we give out are inflating over time. Once a game is 15 years old, inflation cancels out the fading of the cult-of-the-new effect, and a game's rating starts going back up.
Despite that, we still rate newer games higher than older games.
All that below, plus lists of games that proved to be underrated or overrated.
Newly added games
There were 7328 games present in the 2013 dataset but not in the 2012 dataset. Only about half of these (3520) were "new" games; that is, games with a publication date of 2011 or later.
893 newly added games had no publication date. The remainder spanned the past thousand years of gaming. The earliest game added to BGG in 2011 was Four Seasons Chess, first known from a book dating from 1283.
Here's a graph of games added to BGG between July 2011 and August 2011, grouped by the year of original publication:
232 games had been removed from the dataset. My spot checks
indicated that most of these had been duplicates.
Since so many games added in 2012 were published much earlier, I wanted to see how many of those games were non-English games that were finally being added to the America-centric BoardGameGeek. The BGG API provides information about how much language you need to know to play a game, but it doesn't say what language(s) a game is actually published in. As a very rough approximation, I considered a game to be "English" if it has a name that 1) contains only characters found in ASCII, and 2) doesn't contain any common German words like "Das" or "Eine".
By this criteria 94.8% of the games in the 2011 dataset have an "English" name, compared to only 89.4% of the games added since the 2011 dataset. Non-English games are indeed more prevalent in the new dataset, but the vast majority of games still meet the "English" criteria, and of course there are tons of non-English games, like Qui Croque Quoi?, whose names seen "English" by this rough measure.
Only 224 of the newly added games are both obviously non-English,
and published before 2011.
The cult of the new
My main goal in this update was to see how peoples' opinions changed between 2011 and 2012. For a while now, I've predicted that the ratings of newer games go down over time, as the excitement wears off.
In my first attempt to detect this phenomenon, I took 2011's opinion of every year between 1980 to 2012. That is, I took the mean of all the ratings for games published in that year, so long as the game had 6 or more votes in the 2011 dataset. Then I took 2012's opinion of every year between 1980 and 2012, using the same criteria: the game had to have at least 6 votes in the 2012 dataset. (Six votes seemed like a good cutoff point because the median game from 2011 got six new votes in the 2012 dataset.)
People in 2011 had opinions about games that wouldn't be published until 2012, for a number of reasons. They'd been beta testers; they'd played the game at a con; they'd played a print-and-play version; or they hadn't played the game at all, they just had a really good feeling about it. In retrospect, these feelings were way off. The cult-of-the-new effect added three-quarters of a BGG star to the ratings of games that would be released in 2012.
Althouh much smaller, the effect is also present for games published in 2011 (0.3) and 2010 (0.17). It's visible all the way back to 2005 (0.03). The more recently a game was published, the more likely it is that its 2012 rating is worse than its 2011 rating.
As distinctive as that graph is, it's got one problem: the ratings in 2012 incorporate the ratings from 2011. The games are rated lower now than they were, but they'd be lower still if we only considered ratings from the second dataset. To isolate the "cult of the new" effect, we need to graph the average change in rating.
Here's a specific example: the game Say Anything Family Edition. In July 2011, this game had 10 votes and its rating was 7.49. In August 2012 it had 619 votes and its rating was 6.79. The difference between these ratings is due to a factor I'm going to call the cult-of-the-new effect, or CotN effect.
The simplest measurement of the CotN effect is -0.79 BGG stars, the difference between the 2011 rating and the 2012 rating. But that makes the effect look smaller than it is, because the 35 ratings that give the game 6.79 in August 2012 include the ten ratings that gave it 7.49 back in July 2011. I want to factor out those ten ratings so I don't count them twice.
Fortunately we don't need to know how each of those people voted. We can assume that current rating of 7.49 (ratingtotal) comes from ten people (votesa) giving the game a 7.49 (ratinga) back in the first dataset, and 25 people (votestotal-votesa) giving the game... some lower mystery rating, which we'll call ratingb. The equation looks like this:
(votesa * ratinga) + (votesb * ratingb) = (votestotal*ratingtotal)
We know all of these numbers except for ratingb, so we can solve for it:
(10 * 7.49) + (25 * ratingb) = 35*6.79
74.9 + (25 * ratingb) = 237.65
25 * ratingb = 162.75
ratingb = 6.51
So the current rating of 6.79 comes from ten people in the old dataset who gave the game an average of 7.49, and then 25 people in the new dataset who the game an average of 6.51. The difference between these two ratings is the CotN effect on this game between July 2011 and August 2012: -0.98 stars, much larger than -0.70 stars.
Here's a graph of the average CotN effect seen between 07/2011 and 08/2012. I took every game that had at least six ratings in the 2011 dataset, and had acquired at least six more ratings by the 2012 dataset. I calculated the cult-of-the-new effect for each of those games, and grouped the games by the year they were published. This graph shows the mean CotN effect over the period 07/2011-08/2012, for every publication year between 1980 and 2012.
On average, a rating of a 2011 game given today will be 0.66 stars
lower than a rating of the same game given in July 2011, due solely
to the excitement wearing off. As you go back in time, the 2012
cult-of-the-new effect diminishes. For games published before 1997,
the effect becomes erratic and, strangely enough, positive. The graph
breaks down completely around 1960, because not many games released
before 1960 got six new ratings in 2012.
Why do older games benefit from a positive CotN effect? My original idea was that good older games get reissued and judged according to modern standards. But all sorts of crappy games benefit from this effect. The CotN effect for Pogs is +1.57 stars, taking its rating from 3.33 to a still-awful 3.80. Monopoly: Pokémon didn't get reissued as far as I know, but it got 19 new ratings that took it from 5.07 to 5.25 (CotN effect: +1.47 stars).
I believe BoardGameGeek as a whole suffers from ratings inflation. In July 2011, the mean game rating on BoardGameGeek was 5.80 (median 6.00, std 1.56). In August 2012, the mean game rating is 5.82 (median 6.00, std 1.58). The mean went up but the median didn't change. Someone who liked a game in 2012 gives it a slightly higher rating than they would have in 2011.
As of last year, the mean of all the individual ratings on BGG was 6.82 (median=6.95, std=0.88). As of this year, the mean rating had gone up to 6.85 (median=6.98. std=0.87). By reusing the ratingb calculation above, I was able to estimate the mean rating handed out in 2012 as 7.00 (median=7.1, std=0.87).
I believe the cult-of-the-new effect is just a countervailing force against a general tendency to rate games more highly over time. 2012's rating of a 2011 game is lower than 2011's rating of the same game, but it's still a lot higher than the site average. A few 8 ratings are swamped by a ton of 7 ratings.
According to this theory, the cult-of-the-new phenomenon should
decrease a game's rating, year after year, until it's about fifteen
years old. After that, if the game is still being played, ratings
inflation will take over and gradually increase its rating.
Formerly Underrated Games
I made a list of the twenty games with the largest positive CotN effect between my datasets. These are mostly newer games that were highly underrated as of July 2011. As more people played them, their ratings went up—instead of down, as you'd expect for a newer game.
As before, I only considered games that had at least six ratings as of July 2011. But this time, I focused on well-known games by looking only at games that got at least 200 new ratings between July 2011 and August 2012.
|Game||Released in||Votes as of 201107||201107 rating||Votes from 201107-201208||201208 rating||CotN effect|
|1989: Dawn of Freedom||2012||21||7.50||282||8.44||1.00|
|Toc Toc Woodman||2008||61||5.94||386||6.55||0.71|
|Arkham Horror: The Curse of the Dark Pharaoh Expansion (Revised Edition)||2011||31||7.73||247||8.03||0.34|
|The Struggle for Catan||2011||112||6.29||359||6.52||0.31|
|Evolution: The Origin of Species||2010||62||6.40||249||6.64||0.30|
|Agricola: The Goodies Expansion||2010||227||7.26||245||7.36||0.21|
|The Castles of Burgundy||2011||658||7.83||2337||7.99||0.20|
|Glory to Rome||2005||3114||7.51||2065||7.56||0.14|
|Eat Poop You Cat||1990||582||7.54||200||7.57||0.14|
|Dice Town Extension||2011||35||7.46||221||7.57||0.12|
Note that "underrated" means underrated in 2011 relative to the current rating. "Toc Toc Woodman" had a huge swing, but that just means it went from being considered mediocre to being considered somewhat above average. The swing for "1989" may have to do with component quality: back in 2011, the game was a print-and-play game.
We don't yet know where these ratings will stabilize. Relatively
old games like "Antiquity" are still seeing significant year-by-year
changes in their ratings. Any one of these games might still be
underrated, or might now be overrated.
Formerly overrated games
Similarly, I made a table of the twenty games with the highest negative CotN effects. They were the recipients of hype which has since subsided. Again, I only considered games with more than 200 new ratings between July 2011 and August 2012.
|Game||Released in||Votes as of 201107||201107 rating||Votes from 201107-201208||201208 rating||CotN effect|
|War of the Ring Collector's Edition||2010||631||8.86||238||8.41||-1.63|
|JAB: Realtime Boxing||2011||22||8.18||478||6.85||-1.40|
|The Ares Project||2011||13||8.63||280||7.38||-1.30|
|The Lord of the Rings: The Card Game - The Hunt for Gollum||2011||35||8.27||509||7.35||-0.99|
|Strike of the Eagle||2011||11||9.08||239||8.18||-0.94|
|Dungeons & Dragons: Conquest of Nerath Board Game||2011||166||7.70||408||7.21||-0.69|
|Battleship Galaxies: The Saturn Offensive Game Set||2011||123||7.65||535||7.10||-0.67|
|Thunderstone: Doomgate Legion||2010||549||8.23||392||7.95||-0.66|
|Conflict of Heroes: Storms of Steel! Kursk 1943||2009||744||8.20||270||8.03||-0.64|
|A Few Acres of Snow||2011||90||8.35||2478||7.75||-0.62|
Again, these games were overrated in 2011 relative to their
current rating. "War of the Ring Collector's Edition" saw a
relatively huge drop in its rating as the novelty wore off, but
it's still significantly above average.
Here are twenty old games (published in 2010 or earlier) that became significantly more popular between my first dataset and my second. I only looked at games that had at least one rating in the 2011 dataset, and at least fifty ratings in the 2012 dataset. These are the breakout hits of late 2011/early 2012.
|Game||Year published||201107 ratings||201108 ratings||New rating percentage||Average rating|
|My Little Pony Hide & Seek||2005||1||85||98.82%||4.87|
|Cards Against Humanity||2009||38||574||93.38%||7.54|
|BattleCON: War of Indines||2010||14||171||91.81%||8.19|
|The Enigma of Leonardo||2007||5||59||91.53%||5.15|
|Sun of York||2005||6||61||90.16%||6.79|
|Rory's Story Cubes: Actions||2007||9||89||89.89%||6.92|
|Die Heisse Schlacht am Kalten Buffet||1974||22||180||87.78%||6.32|
|Toc Toc Woodman||2008||61||447||86.35%||6.55|
|Jungle Speed: Rabbids||2008||10||64||84.38%||7.19|
|Confusion: Espionage and Deception in the Cold War||1992||96||572||83.22%||7.46|
|Animal Upon Animal: Balancing Bridge||2010||20||103||80.58%||7.13|
|Evolution: The Origin of Species||2010||62||311||80.06%||6.64|
This document (source) is part of Crummy, the webspace of Leonard Richardson (contact information). It was last modified on Tuesday, July 23 2013, 14:05:23 Nowhere Standard Time and last built on Friday, December 13 2013, 09:00:07 Nowhere Standard Time.