News You Can Bruise for 2016 January 22 (entry 0)

(3) Fri Jan 22 2016 23:07 The Minecraft (And Other Games) Archive Project: As suggested in the previous Minecraft Archive Project post, I have now completed a capture of the CurseForge family of sites. They host a lot of Minecraft stuff I hadn't downloaded before, including the popular Feed the Beast series of modpacks, lots of other modpacks, mods, and a ton of Bukkit plugins (not really sure what those are or how they differ from mods TBH).

CurseForge also has sites for Terraria and Kerbal Space Program, as well as many other games I haven't heard of or don't care about. I paid $30 for a premium membership and grabbed it all, downloading about 500 gigabytes of images and binaries. This doubles the size of the 201512 capture (though it probably introduces a lot of duplicates).

Here are the spoils, ordered by game:

Game	What	Capture Size (GB)
Firefall	Add-ons	<1
Kerbal Space Program	Mods	23
Kerbal Space Program	Shareables	1.8
Minecraft	Bukkit plugins	19
Minecraft	Customization	<1
Minecraft	Modpacks (Feed the Beast)	15
Minecraft	Modpacks (Other)	87
Minecraft	Mods	33
Minecraft	Resource Packs	80
Minecraft	Worlds	45
Rift	Add-ons	7.5
Runes of Magic	Add-ons	1.8
Skyrim	Mods	6.4
Starcraft 2	Assets	4.7
Starcraft 2	Maps	46
Terraria	Maps	4.8
The Elder Scrolls Online	Add-ons	<1
The Secret World	Mods	<1
Wildstar	Add-ons	1.7
World of Tanks	Mods	40
World of Tanks	Skins	12
World of Warcraft	Addons	48

Here's the really cool part: CurseForge projects frequently link to Git repositories. I cloned every one I could find. I ended up with 5000 Minecraft/Bukkit repositories totalling 47 gigs, 103 Kerbal Space Program repositories totalling 6 gigs, and a couple hundred megabytes here and there for the other games. That's over 50 gigs of game-mod source code, which I predict will be a lot more useful to the future than a bunch of JAR files.

These numbers are gloriously huge and there are two reasons. 1. this is the first capture I've done of CurseForge, and possibly the only full capture I will ever do. So I got stuff dating back several years. 2. CurseForge keeps a full history of your uploaded files, not just the most recent version (which is typically what you'd find on Planet Minecraft or the Minecraft forum). Some of the World of Warcraft add-ons have hundreds of releases! I guess because they have to be re-released for every client update. And it doesn't take many releases for a 100MB Minecraft mod pack to start becoming huge.

Anyway, as always it's good to be done with a project like this, so I can work on other stuff, like all the short stories I owe people.

Comments:

Posted by BigBlueHat at Mon Jan 25 2016 20:19

Have you seen IPFS.io? If not, we should talk. Even if you have, we should talk... it's been awhile. :)

Posted by dbt at Tue Jan 26 2016 03:14

Mods are client/server extensions, bukkit plugins are server-side only extensions so they work on a vanilla/unmodded client.

Posted by Leonard at Tue Jan 26 2016 08:55

Thanks!