Loaded Dice 2012 Update

(Part of Loaded Dice.)

• Introduction
• Newly added games
• English?
• The cult of the new
• Rating inflation
• Formerly underrated games
• Formerly overrated games
• Late bloomers

In August 2012 I fetched the BoardGameGeek dataset again, so that I could make comparisons with the 2011 dataset. A summary of my findings:

We'll give a game from 2011 a worse rating now, than we did when it was new. We'll give a 2010 game a worse rating than we did in 2011. This "cult-of-the-new effect" is detectable in games published as far back as 1997.

Despite that, the ratings we give out are inflating over time. Once a game is 15 years old, inflation cancels out the fading of the cult-of-the-new effect, and a game's rating starts going back up.

Despite that, we still rate newer games higher than older games.

All that below, plus lists of games that proved to be underrated or overrated.

Newly added games

There were 7328 games present in the 2013 dataset but not in the 2012 dataset. Only about half of these (3520) were "new" games; that is, games with a publication date of 2011 or later.

893 newly added games had no publication date. The remainder spanned the past thousand years of gaming. The earliest game added to BGG in 2011 was Four Seasons Chess, first known from a book dating from 1283.

Here's a graph of games added to BGG between July 2011 and August 2011, grouped by the year of original publication:

232 games had been removed from the dataset. My spot checks indicated that most of these had been duplicates.

English?

Since so many games added in 2012 were published much earlier, I wanted to see how many of those games were non-English games that were finally being added to the America-centric BoardGameGeek. The BGG API provides information about how much language you need to know to play a game, but it doesn't say what language(s) a game is actually published in. As a very rough approximation, I considered a game to be "English" if it has a name that 1) contains only characters found in ASCII, and 2) doesn't contain any common German words like "Das" or "Eine".

By this criteria 94.8% of the games in the 2011 dataset have an "English" name, compared to only 89.4% of the games added since the 2011 dataset. Non-English games are indeed more prevalent in the new dataset, but the vast majority of games still meet the "English" criteria, and of course there are tons of non-English games, like Qui Croque Quoi?, whose names seen "English" by this rough measure.

Only 224 of the newly added games are both obviously non-English, and published before 2011.

The cult of the new

My main goal in this update was to see how peoples' opinions changed between 2011 and 2012. For a while now, I've predicted that the ratings of newer games go down over time, as the excitement wears off.

In my first attempt to detect this phenomenon, I took 2011's opinion of every year between 1980 to 2012. That is, I took the mean of all the ratings for games published in that year, so long as the game had 6 or more votes in the 2011 dataset. Then I took 2012's opinion of every year between 1980 and 2012, using the same criteria: the game had to have at least 6 votes in the 2012 dataset. (Six votes seemed like a good cutoff point because the median game from 2011 got six new votes in the 2012 dataset.)

People in 2011 had opinions about games that wouldn't be published until 2012, for a number of reasons. They'd been beta testers; they'd played the game at a con; they'd played a print-and-play version; or they hadn't played the game at all, they just had a really good feeling about it. In retrospect, these feelings were way off. The cult-of-the-new effect added three-quarters of a BGG star to the ratings of games that would be released in 2012.

Althouh much smaller, the effect is also present for games published in 2011 (0.3) and 2010 (0.17). It's visible all the way back to 2005 (0.03). The more recently a game was published, the more likely it is that its 2012 rating is worse than its 2011 rating.

As distinctive as that graph is, it's got one problem: the ratings in 2012 incorporate the ratings from 2011. The games are rated lower now than they were, but they'd be lower still if we only considered ratings from the second dataset. To isolate the "cult of the new" effect, we need to graph the average change in rating.

Here's a specific example: the game Say Anything Family Edition. In July 2011, this game had 10 votes and its rating was 7.49. In August 2012 it had 619 votes and its rating was 6.79. The difference between these ratings is due to a factor I'm going to call the cult-of-the-new effect, or CotN effect.

The simplest measurement of the CotN effect is -0.79 BGG stars, the difference between the 2011 rating and the 2012 rating. But that makes the effect look smaller than it is, because the 35 ratings that give the game 6.79 in August 2012 include the ten ratings that gave it 7.49 back in July 2011. I want to factor out those ten ratings so I don't count them twice.

Fortunately we don't need to know how each of those people voted. We can assume that current rating of 7.49 (rating_total) comes from ten people (votes_a) giving the game a 7.49 (rating_a) back in the first dataset, and 25 people (votes_total-votes_a) giving the game... some lower mystery rating, which we'll call rating_b. The equation looks like this:

(votes_a * rating_a) + (votes_b * rating_b) = (votes_total*rating_total)

We know all of these numbers except for rating_b, so we can solve for it:

(10 * 7.49) + (25 * rating_b) = 35*6.79
74.9 + (25 * rating_b) = 237.65
25 * rating_b = 162.75
rating_b = 6.51

So the current rating of 6.79 comes from ten people in the old dataset who gave the game an average of 7.49, and then 25 people in the new dataset who the game an average of 6.51. The difference between these two ratings is the CotN effect on this game between July 2011 and August 2012: -0.98 stars, much larger than -0.70 stars.

Here's a graph of the average CotN effect seen between 07/2011 and 08/2012. I took every game that had at least six ratings in the 2011 dataset, and had acquired at least six more ratings by the 2012 dataset. I calculated the cult-of-the-new effect for each of those games, and grouped the games by the year they were published. This graph shows the mean CotN effect over the period 07/2011-08/2012, for every publication year between 1980 and 2012.

On average, a rating of a 2011 game given today will be 0.66 stars lower than a rating of the same game given in July 2011, due solely to the excitement wearing off. As you go back in time, the 2012 cult-of-the-new effect diminishes. For games published before 1997, the effect becomes erratic and, strangely enough, positive. The graph breaks down completely around 1960, because not many games released before 1960 got six new ratings in 2012.

Rating inflation

Why do older games benefit from a positive CotN effect? My original idea was that good older games get reissued and judged according to modern standards. But all sorts of crappy games benefit from this effect. The CotN effect for Pogs is +1.57 stars, taking its rating from 3.33 to a still-awful 3.80. Monopoly: Pokémon didn't get reissued as far as I know, but it got 19 new ratings that took it from 5.07 to 5.25 (CotN effect: +1.47 stars).

I believe BoardGameGeek as a whole suffers from ratings inflation. In July 2011, the mean game rating on BoardGameGeek was 5.80 (median 6.00, std 1.56). In August 2012, the mean game rating is 5.82 (median 6.00, std 1.58). The mean went up but the median didn't change. Someone who liked a game in 2012 gives it a slightly higher rating than they would have in 2011.

As of last year, the mean of all the individual ratings on BGG was 6.82 (median=6.95, std=0.88). As of this year, the mean rating had gone up to 6.85 (median=6.98. std=0.87). By reusing the rating_b calculation above, I was able to estimate the mean rating handed out in 2012 as 7.00 (median=7.1, std=0.87).

I believe the cult-of-the-new effect is just a countervailing force against a general tendency to rate games more highly over time. 2012's rating of a 2011 game is lower than 2011's rating of the same game, but it's still a lot higher than the site average. A few 8 ratings are swamped by a ton of 7 ratings.

According to this theory, the cult-of-the-new phenomenon should decrease a game's rating, year after year, until it's about fifteen years old. After that, if the game is still being played, ratings inflation will take over and gradually increase its rating.

Formerly Underrated Games

I made a list of the twenty games with the largest positive CotN effect between my datasets. These are mostly newer games that were highly underrated as of July 2011. As more people played them, their ratings went up—instead of down, as you'd expect for a newer game.

As before, I only considered games that had at least six ratings as of July 2011. But this time, I focused on well-known games by looking only at games that got at least 200 new ratings between July 2011 and August 2012.

Game	Released in	Votes as of 201107	201107 rating	Votes from 201107-201208	201208 rating	CotN effect
1989: Dawn of Freedom	2012	21	7.50	282	8.44	1.00
Toc Toc Woodman	2008	61	5.94	386	6.55	0.71
Dungeon Petz	2011	11	7.18	1716	7.68	0.50
Arkham Horror: The Curse of the Dark Pharaoh Expansion (Revised Edition)	2011	31	7.73	247	8.03	0.34
The Struggle for Catan	2011	112	6.29	359	6.52	0.31
Eminent Domain	2011	85	6.97	2289	7.27	0.31
Evolution: The Origin of Species	2010	62	6.40	249	6.64	0.30
Arcana	2009	479	6.25	348	6.36	0.24
Antiquity	2004	1023	7.86	358	7.92	0.24
Agricola: The Goodies Expansion	2010	227	7.26	245	7.36	0.21
The Castles of Burgundy	2011	658	7.83	2337	7.99	0.20
Get Bit!	2007	218	6.21	425	6.32	0.16
The Resistance	2009	1829	7.44	3366	7.54	0.16
Eclipse	2011	44	8.22	4248	8.36	0.15
Glory to Rome	2005	3114	7.51	2065	7.56	0.14
Eat Poop You Cat	1990	582	7.54	200	7.57	0.14
Maori	2009	555	6.60	260	6.64	0.13
Summoner Wars	2009	1556	7.66	646	7.70	0.12
Dice Town Extension	2011	35	7.46	221	7.57	0.12
Black Friday	2010	388	6.55	214	6.59	0.12

Note that "underrated" means underrated in 2011 relative to the current rating. "Toc Toc Woodman" had a huge swing, but that just means it went from being considered mediocre to being considered somewhat above average. The swing for "1989" may have to do with component quality: back in 2011, the game was a print-and-play game.

We don't yet know where these ratings will stabilize. Relatively old games like "Antiquity" are still seeing significant year-by-year changes in their ratings. Any one of these games might still be underrated, or might now be overrated.

Formerly overrated games

Similarly, I made a table of the twenty games with the highest negative CotN effects. They were the recipients of hype which has since subsided. Again, I only considered games with more than 200 new ratings between July 2011 and August 2012.

Game	Released in	Votes as of 201107	201107 rating	Votes from 201107-201208	201208 rating	CotN effect
War of the Ring Collector's Edition	2010	631	8.86	238	8.41	-1.63
JAB: Realtime Boxing	2011	22	8.18	478	6.85	-1.40
Heroica: Fortaan	2011	14	7.46	251	6.21	-1.32
The Ares Project	2011	13	8.63	280	7.38	-1.30
Belfort	2011	10	8.80	1093	7.54	-1.27
Drum Roll	2011	15	8.26	550	7.22	-1.07
The Lord of the Rings: The Card Game - The Hunt for Gollum	2011	35	8.27	509	7.35	-0.99
Strike of the Eagle	2011	11	9.08	239	8.18	-0.94
Munchkin Zombies	2011	67	7.62	215	7.00	-0.81
Yomi	2011	926	7.90	594	7.59	-0.78
Quarriors!	2011	50	7.84	4429	7.12	-0.74
Dungeon Run	2011	30	7.47	683	6.78	-0.72
Thunderstone: Dragonspire	2011	601	8.21	986	7.77	-0.71
Dungeons & Dragons: Conquest of Nerath Board Game	2011	166	7.70	408	7.21	-0.69
Battleship Galaxies: The Saturn Offensive Game Set	2011	123	7.65	535	7.10	-0.67
Dungeon Raiders	2011	34	7.34	241	6.76	-0.66
Thunderstone: Doomgate Legion	2010	549	8.23	392	7.95	-0.66
Conflict of Heroes: Storms of Steel! Kursk 1943	2009	744	8.20	270	8.03	-0.64
A Few Acres of Snow	2011	90	8.35	2478	7.75	-0.62
Puzzle Strike	2010	627	7.36	412	7.11	-0.62

Again, these games were overrated in 2011 relative to their current rating. "War of the Ring Collector's Edition" saw a relatively huge drop in its rating as the novelty wore off, but it's still significantly above average.

Late bloomers

Here are twenty old games (published in 2010 or earlier) that became significantly more popular between my first dataset and my second. I only looked at games that had at least one rating in the 2011 dataset, and at least fifty ratings in the 2012 dataset. These are the breakout hits of late 2011/early 2012.

Game	Year published	201107 ratings	201108 ratings	New rating percentage	Average rating
My Little Pony Hide & Seek	2005	1	85	98.82%	4.87
Big Five	2010	1	84	98.81%	6.35
Hanabi	2010	5	345	98.55%	7.48
Cards Against Humanity	2009	38	574	93.38%	7.54
BattleCON: War of Indines	2010	14	171	91.81%	8.19
The Enigma of Leonardo	2007	5	59	91.53%	5.15
Timeline: Inventions	2010	58	679	91.46%	6.84
Caveman Curling	2010	16	166	90.36%	6.80
Shoot Out	2002	5	51	90.20%	5.43
Sun of York	2005	6	61	90.16%	6.79
Rory's Story Cubes: Actions	2007	9	89	89.89%	6.92
Die Heisse Schlacht am Kalten Buffet	1974	22	180	87.78%	6.32
Toc Toc Woodman	2008	61	447	86.35%	6.55
Prêt-à-Porter	2010	74	479	84.55%	7.51
Jungle Speed: Rabbids	2008	10	64	84.38%	7.19
Confusion: Espionage and Deception in the Cold War	1992	96	572	83.22%	7.46
Perplexus	2001	9	52	82.69%	7.52
Reverse Charades	2010	11	58	81.03%	6.82
Animal Upon Animal: Balancing Bridge	2010	20	103	80.58%	7.13
Evolution: The Origin of Species	2010	62	311	80.06%	6.64