News You Can Bruise for 2016 January 10 (entry 0)

Sun Jan 10 2016 08:36 Minecraft Archive Project: The 201512 Capture: On December 27th I started the third capture for the Minecraft Archive Project. Previous captures ran in February 2015 and March 2014. This time I collected about 420 gigabytes of material.

Here's the breakdown by what I believe the new files to be:

Type	Number of files	Collective size
Maps	33112	320 GB
Maps (MCPE)	1552	2 GB
Resource packs	2137	30 GB
Resource packs (MCPE)	176	172 MB
Mods	6082	10 GB
Mods (MCPE)	1839	1 GB
Screenshots	335651	57 GB
Skins	31064	132 MB
Server records	25923	361 MB
Blog posts	6562	129 MB

This time I think I was able to archive about 60-65% of the maps I saw, compared to 73% in the last capture. Even so, we ended up with 33k new maps in this capture versus 22k in the last one--and I didn't even get the adf.ly maps this time! (Nor will I--it's a huge pain and I'm sick of it.) 2012 was the single biggest year for custom Minecraft maps, and there was a downward trend visible in 2013 and 2014, but it looks like 2015 was really huge.

Couple new features in this capture: I started keeping track of blog posts and server records from Planet Minecraft. Server records are especially important because they usually feature screenshots, and in twenty years those screenshots will be the only record of what those servers looked like.

I've completely given up on the idea of archiving public servers--it's still theoretically possible but it's a full-time job for two developers, so I'd need to get a grant or some volunteer interest from the modding comunity. In fact, a few months ago the multiuser server I played Minecraft on went down, and I don't know whether my stuff is still around. That's life! Gonna archive the screenshots.

The full dataset is now about 2.4 terabytes. I bought a new drive to store the archive and set it up with XFS, and it does seem to improve the performance when iterating over the file set.

As always I'm putting a copy of the data on a server at NYPL Labs, and I recently gave Jason Scott a drive that contained the first two captures, so he can do whatever Jason thing he wants with the data. I don't have any plans to make this archive public, or even to re-run the Minecraft Geologic Survey on the new data. My maximum supportable commitment is spending some time once a year to shepherd these scripts through saving a representative sample of this artform.

I'm going to leave everything else to the future when the archive becomes valuable to other people. I am doing exploratory work for adding a third site to the archive, but that's all I'll say about that for now.