<M <Y
Y> M>

[Comments] (3) Frances Daily: We're out of the gate with the first Crummy project of the new year. I spent Christmas in Utah, and while I was there my sister Susanna foisted on me my mother's old Franklin day planner. She's already typed up my mother's Franklin diary from the 80s and 90s, a diary very similar to the weblog she kept from 2001 until she died in 2006, so all that was left of the day planner was an enormous block of calendars.

It's a very detailed calendar that will be useful for family history purposes but quite boring otherwise. Except, at the beginning of each month there was an "index" page on which my mother had written a brief summary of every day of that month. There were index pages from January 1987 to November 1990, with a few sporadic months from later on.

The cumulative effect of these daily summaries was incredibly powerful. This was a time of major upheaval in my family, leading up to (but not ending with) my father's death in 1992. The monthly summaries show my mother trying to keep it together while studying for a graduate degree and raising three kids pretty much on her own. It's inspiring and a bit horrifying.

I've started a Twitter account, @FrancesDaily, which is using Sycorax to reprint the daily summaries 25 years after my mother wrote them. (Here's the RSS feed.) The summary for each day will go up at 4 PM Pacific time. It's a little spotty in January, but once it picks up she doesn't miss many days for the next three years.

I don't know what the effect of the summaries will be when experienced in real time. Probably when I read one I'm going to mentally compare my day against the day Frances had 25 years earlier, and you might want to do the same.

This is a much lower-bandwidth project than Roy's Postcards, but where my father's writing almost never showed any emotion, these summaries pack a lot into just a couple words. Susanna read the index and said "Mom was really hard on herself." So please take entries like "wasted day" as accurate depictions of my mother's mental state, but not necessarily of reality.

A note about the names. I did the thing you frequently see in old journals that have passed through the hands of the journal-writer's descendants, and replaced most of the names with initials. For instance, the January 18 summary, "Mario's Eagle", became "M's Eagle". The Mario mentioned is Mario Canton, one of my dad's Boy Scouts and later a family friend. I initialized most of the names because you don't have the context to know who all these people are, and I think giving each entry an explanation significantly longer than the entry itself would ruin the effect.

In a few cases, making peoples' names into initials was also necessary to protect their privacy. I left a few names intact: mostly my mother's close relatives like her aunt LeJeune ("Jeuney"), her sister Anne, my father, and of course me and my sisters.

Again, that Twitter account is @FrancesDaily. Here's the RSS feed.

[Comments] (1) CG Author Commentary #5: "The Stars My Screensaver": Yeah, you know it's getting serious now. The microblog archive is up, I'm feeling good and it's time for some commentary:

And there's the commentary. Stay glued to the proverbial set for chapter 7, when Ariel will say, "Well, her hardware's Chinese..."

Image credits: Gisela Giardino, The United States Department of State, and the East German postal service.

<- Last week Next week ->

GeekDad Reviews Constellation Games: Wired's Jonathan Liu got an advance copy and calls it "a perfect blend of aliens, video game geekiness, and modern social media." Other quotes relevant to my interests include "absolutely loved it — it might not be the best book I read in 2011, but it’s certainly in the top 10."

C'mon, folks, do I have to draw you a picture? Because I can't draw very well. I use words instead.

Dear Santa: Dinosaurs:

My niece's Christmas stocking project.

Connected Tragedies: Sumana noticed this. 1, 2.

CG Author Commentary #7: "Party Creation": Ariel's Twitter will be pretty quiet this week because the entire chapter takes place over the course of one day. Your only solace is farmers market quail sausage, and this COMMENTARY:

Stay tuned for chapter 8, a chapter I think is one of the best in the book, the chapter that got me to give up on the first draft and rewrite the entire book to be more like it. The only chapter in which Jenny will say, "Wait a minute, are you naked?"

Image credits: I got the first image from Flickr user marsmet462, not sure if they put in enough transformative elbow grease to put their own license on it. Second image comes from Sven-S. "☃" Porst .

<- Last week | Next week ->

[Comments] (8) Hostile Witness: The Vechs Interview: Minecraft can be a tough game. The controls are kind of blocky, the best resources are hidden deep in the map alongside deadly lava rivers, and the night hosts monsters that will kill you just as soon as look at you (or, in one case, just as soon as you look at them). But it's not that tough. All that terrain is generated by algorithm. It's not like the random number generator is trying to kill you.

But there's this guy named Vechs who is trying to kill you. His "Super Hostile" series of custom Minecraft maps offer challenges that prohibit or subvert every survival strategy you learned in vanilla Minecraft. Even in his easier maps you'll find bottomless pits, world-spanning ceilings that block Minecraft's all-important sunlight, swarms of monsters pouring from hacked spawners, and TNT in unfortunate places. Just getting your first tree is a challenge. Complete a Super Hostile map, and vanilla Minecraft will seem easy.

But Vechs' maps are not just tough: they're creative, fun to play, and they look great. Vechs uses landscape features and lighting to grab the player's attention, direct the flow and pacing of their playthrough, and give them a spectacular environment to build in once they've conquered the map.

I've raved about Super Hostile a couple times before here on NYCB, but with the release of "Spellbound Caves", the tenth entry in the series, I knew it was time to get serious. I sat down with Vechs (I assume he was sitting down, anyway) and interviewed him over minecraftforum.net's private message feature. My goal was to pick up where this interview from July 2011 left off, with in-depth questions about his style and his mapmaking wish-list. The interview contains some Minecraft jargon, but anyone with an interest in game design should get something out of it.

Leonard: You play a kind of character on your maps, an angry trickster god who hates his players and taunts them by writing things on signs. But clearly you're not actually like that. I've played maps made by people who really did hate me, who wanted me to farm cobblestone for an hour or dig through obsidian without a tool, and I said "screw this" and quit the map. I don't think you'd ever do that, right? What's the difference between you and the "Vechs" in your maps?

Vechs: It has to do with challenge. There is a difference between making the player use skill or ingenuity, and making the player do something tedious. Sometimes a solution to an area can involve using lots of blocks (Like the player making a cobblestone tube for them to safely move through.), but these are usually just one option of many the player can use to conquer an area.

Sometimes the "Vechs character" in my maps is pretty mean, and just downright spiteful, especially when it comes to traps. In real life, I'm not like that at all.

In the Obsidian Block interview you say that you recently graduated from college and are looking for a career as a game designer or world designer. What did you study in college?

I am a Media Arts major. I studied everything from digital image editing, video editing, to stage lighting, to writing scripts and screenplays for movies, and more. I'm glad to have a diverse background, even though my passion is still game design.

What would be your ideal job? Would you rather work on a big-budget project with high production values, or an indie project where you have more creative control?

My first choice would actually be to have my own studio and bring to life some of the game ideas I have. One idea I've had for a while, and as far as I know, nobody has ever made a game like it. I wouldn't mind making it all myself, but that means I would have to re-learn a lot of programming. I've programmed some text-based games in C++, but programming is not my main forte.

That said, I also wouldn't mind working for a major company. Like, for example, Valve. Love those guys.

What other games have you made maps for? You mention Duke Nukem 3D in the Obsidian Block interview; what else?

Just off hand: Red Alert, Warcraft II, Warcraft III, Neverwinter Nights, Total Annihilation, TA:Spring, Terraria, Command and Conquer (and several sequels), The Elder Scrolls series, and obviously I'm the world designer for the RPG games I've worked on, using the XP and VX engines.

Are you currently making maps for any games other than Minecraft?

At this moment, no, but I have been meaning to make some maps for Team Fortress 2.

Have you ever heard of ZZT or Megazeux, or am I just incredibly old?

You're old! *grins* I looked them up, and I think my version of that would be the RPG-series of game engines.

There are a lot of memorable set pieces in the Super Hostile series. Now that you've put out ten maps, would you mind taking a look back and sharing some of your favorites?

The first 15 minutes on just about any of my maps. I love that feeling of just starting off and scrambling for resources. I like the rail station in "Sea of Flame II", and how it goes out in the area with the huge pillars, and "Spellbound Caves" is just full of nice vistas and "scripted" events.

Most of my maps feature at least one "death fortress" as an end-game area. These are intended to be where the player gets to use all the resources and items he has been collecting through the whole map. TNT, lava, swords, bow and arrow, even TNT cannons... bring your whole arsenal and have some fun!

Can you describe the evolution of your design philosophy over the course of the series?

Try to improve in at least one area every time I make a new map. Push the Minecraft engine to its limits. Make an awesome and memorable experience for the player.

What are the biggest challenges in re-balancing Super Hostile for Minecraft 1.0?

Armor and blocking.

Does 1.0 have anything to do with the fact that you recently flattened the difficulty levels in your map descriptions, so that "Sunburn Islands" and "Legendary" are now both considered "Easy"?

Yes and no. I feel that recently I have been drifting away from the theme of "Super Hostile" and I want to get back to my roots. Being able to respawn forever, over and over kind of takes the risk out of a map. Even in "Legendary", unless you really mess up and drop all the wool in lava or something, you can just set your bed spawn near an area, and try over and over until you get it right. I think that's pretty Easy on the player, even if the area you are attempting is challenging.

Call me nostalgic, but I kind of miss (sometimes) the GAME OVER screens from older video games. Modern video games, in the name of convenience, typically feature unlimited lives, save games, checkpoints, the works. But beating a modern video game, I have to admit, is much less satisfying than beating some of those old NES games. You can just bang your head against the game until you get lucky and get through an area. Heh, man this makes me feel old! "In my day, we didn't have all those checkpoints! We had three lives! One hit deaths! And we were happy!" *shakes cane*

Anyway, I do think this is a legitimate point of concern on modern game design, is risk versus reward. It is possible to make games so easy that they are very unsatisfying...

I'm an admirer of your ability to create new genres of map. Have you made experimental Minecraft maps that just didn't work? What's in your "abandoned projects" folder?

The only thing I've actually stopped on, is "Race for Wool #3: Common Ground". Because it basically became "Capture the Wool".

Have you ever made maps for a game that featured scriptable events? If so, do you miss that capability in Minecraft?

I have used C++ to code some text-based games. I have also used various scripting languages in the process of making mods or making my own games with existing engines. You do have some limited "scripting" ability in Minecraft, using redstone. Check out the Rumbling Caverns in my tenth map and you will see what I mean. :)

But yes, I would love some even rudimentary scripting in Minecraft. I believe a while ago, I proposed invisible effector blocks, that you can place with Creative or MCedit, that modify the immediate environment around them. Like, an invisible block that makes monsters not spawn within 50 blocks. Or one that doubles monster spawning within 50 blocks. Or one that makes it snow. Or one that makes a ray of sunlight always be shining on that spot. Or one that makes the temperature freezing so any water turns to ice. Simple stuff like that. They would show up faintly in Creative mode, but be invisible while in survival mode.

What would you like to see added to Minecraft? On your forum thread you mention that you'd like to add sharks and underwater plants to "Endless Deep". What else?

Bow enchantments... more mining enchantments, such as area mining. Check out episodes 04 and 05 of my Spellbound Caves Dev Commentary.

For bow enchantments, I would like:

I think these enchants for bows would make bow combat much, much more fun. It's currently fairly slow paced, and a bit boring. Imagine a bow with Toxic, Piercing, and Phantom Spreadshot on it! It would be so much fun to shoot groups of enemies with a bow like that.

You have a creative relationship with some of the people who do Let's Play videos of your maps. It's a kind of relationship I've never seen before: the way people play your maps in public affects the way you design later maps. How did these relationships develop?

Very organically. Zisteau agreed to LP my very first map, "Sea of Flames" version 1.0, and ever since then, he's been involved in playing my maps, and giving feedback.

There's a very clever trap in "Spellbound Caves", [location redacted]. It's clever for many reasons, but I'm asking about it because it doesn't seem to have any triggering mechanism. I went in afterwards and took the walls apart and couldn't figure out how it works. What's the secret? Or is there a pressure plate somewhere that I missed?

I has a seekret. Oh, also, I hate you, die in a fire.

POSTSCRIPT: With my interviewee uncooperative, I had no choice but to load a fresh version of "Spellbound Caves" into an editor to get to the bottom of the mystery. What I found was a trigger that did not shock me to the core of my being. But it is a cool design.

The trigger is a proximity sensor: a shaft behind a wall, with a creeper spawner at the top of the shaft and a pressure plate at the bottom. When the player gets within 16 blocks of the spawner, it activates and spawns a creeper, which drops onto the pressure plate, triggering the trap. The resulting explosion obliterates both creeper and spawner, leaving no trace of the trigger.

And that's what you get with Vechs' maps: MacGyver-like use of everything the game engine provides, to create confounding and unexpected effects. Seriously, game studios: hire this guy. Everyone else: play his maps.

PPS: Hey, people from minecraftforums.net, thanks for coming over. I've written other articles about Minecraft (1 2 3 4), and if you like my stuff, you might want to check out my novel about alien video games.

[Comments] (1) : My sister talks about her miscarriage.

The only thing worth saying is "I'm sorry." I may think those things. John and I may even say those things to each other. But don't impose beliefs or possibilities or happy thoughts on me.

[Comments] (1) Findings: My writing life has settled down a bit so I'm finally going to write about Findings, the social reading startup where I worked last summer. This is more an essay about what I see in Findings than an introduction to the site--you can see lots of general introductions linked to from this Findings blog post, including co-founder Steven Johnson's introduction, and the Business Insider article whose title is the perfect elevator pitch, "Findings is GitHub for Ideas".

If what I'm about to say sounds interesting to you, there are development jobs open at Findings right now. Just as a reminder, I myself don't work at Findings anymore, and even when I did, only the foggyheadedest knave would have taken my personal opinions as representative of company policy.

Let me start out with this quote I took from Darwin's The Descent of Man, not because the quote itself proves anything, but because the quote is an important part of my reading of Darwin:

Brehm gives a curious account of the instinctive dread, which his monkeys exhibited, for snakes; but their curiosity was so great that they could not desist from occasionally satiating their horror in a most human fashion, by lifting up the lid of the box in which the snakes were kept. I was so much surprised at his account, that I took a stuffed and coiled-up snake into the monkey-house at the Zoological Gardens, and the excitement thus caused was one of the most curious spectacles which I ever beheld.

If you want to learn about evolutionary biology, read Steven Jay Gould's essays. Darwin's a good writer and he got it basically right, but he didn't know about genes or DNA. I read Darwin to experience the origins of the field. I didn't expect (though perhaps I should have) to encounter endless artifacts of the days of two-fisted Baconian science.

When Charles Darwin needs to figure something out, he carries out an experiment, no matter how tedious or disturbing. He takes snakes to the zoo, he puts kittens' feet in his mouth, he floats 94 kinds of plants in seawater, he hacks aphids. If someone has the temerity to question his conclusions he's all "Citation needed? I'll give you citations, motherfucker!"

When you read a book, it has an effect on your mind. You're a slightly different person after reading it. You've created something new: a reading of the book: Here's an apropos quote from Alberto Manguel's A History Of Reading, which I read on paper and typed in. Manguel is talking about Petrarch's Secretum:

What Augustine (in Petrarch's imagining) suggests is a new manner of reading: neither using the book as a prop for thought, nor trusting it as one would trust the authority of a sage, but taking from it an idea, a phrase, an image, linking it to another culled from a distant text preserved in memory, tying the whole together with reflections of one's own -- producing, in fact, a new text authored by the reader.

Readings are ephemeral. Life goes on, and the memory fades. Ken Macleod's The Star Fraction had a huge influence on me, probably leading to whatever career I now enjoy as an author of fiction, but I read it ten years and 600 books ago, and now I don't remember a damn thing about it.

That's why we dog-ear pages and highlight passages. We're instantiating our reading of the book so we can go back later and approximate the mental state it gave us without re-reading the whole thing. Even if all we got out of a book was "this bit was funny", it's better to have the funny bit at hand than not. Even if you never go back to the highlighted passage, the act of highlighting replays that passage and deepens your initial memory of it.

Liberate your readings

I've been typing in quotes from the paper books I read, like I did with the Manguel. Of course, with an electronic book, you don't have to do this. The act of highlighting creates an electronic record of your reading of the book. When I was in college I read about the first e-ink research coming out of MIT, and I knew that this was the future. Indeed it was the future, because I had to wait ten years for the technology to make it to market. But, sour grapes, we've got ebook readers now.

Ebook readers have big problems, but at this point the problems are mostly political, not technical. For instance, you can highlight passages when reading a book on your Kindle, but because of a deal between Amazon and the publisher, your book's metadata may include restrictions, which the Kindle will obey, on how much you can highlight. And your highlights and notes—the "new text authored by the reader"—are stuck on a website that Amazon didn't put a lot of work into because they don't consider your reading of a book important to their business.

Findings takes advantage of the fact that Amazon is wrong about this. Findings liberates your highlights and makes them searchable and shareable. Your reading of a book is a big part of your relationship with that book, and Findings gives you access to it.

You can also use Findings to take a reading of a web page, creating a record of what would otherwise be an ephemeral activity. I'm not as interested in this feature, but people are using it quite a bit, and my interest does increase as the length of the web page I'm reading approaches the length of a book.

Browse readings

So that's what Findings can do for you personally. Now let me pitch you the network effects. Take a look at this screenshot which shows the Findings global stream:

You can't see the global stream without logging in, which I think is a shame because I think this is what really sells Findings. We have here a stream of little bits of text, like Twitter used to have on their front page. Except here, every bit of text is a quote that someone liked well enough to save. It's very high-quality stuff. At the top you can also see some recently added books, and by clicking on a book you can see someone's condensed reading of the book.

Basically, Findings gives you browsing access to a large library, not of books, but of readings. It's easy to discover new books, people who read books you like, and—this is new—people who read books in ways you like.

There are a ton more useful things I could mention, but they're mostly behind-the-scenes things where Findings makes things "just work" (like consolidating multiple editions of the same text), or they depend on features that haven't been implemented yet. So I'm going to close by mentioning the social signalling feature.

Strut your stuff

One underappreciated feature of paper books is signaling to other people that you are cool. You read books! Fancy books, like Ulysses! You care enough about books to make space for them in your house. You take them on the subway even though they're heavy. Darwin would say it's like the peacock's tail. But if you have an ebook reader, nobody knows how cool you are. You're just a person with an ebook reader.

By letting you publicize your reading of an ebook, Findings reinstates your ability to send those social signals. The downside is that you have to actually read the book. You can't just put a big book on your coffee table: the thing you're sharing is what you got out of the book. (Well, you can fake it, but it's probably about as much work as reading the book legitimately.)

So that's Findings. I don't use it as much as I thought I would, because I'm still trying to draw down my stack of paper books, but when I read a book on my Kindle, it stays read, thanks to Findings.

I mentioned this before, but the last thing I did at Findings was design a web service for them, which they're hopefully working on now. Once the web service launches, you'll be able to write programs that import readings into Findings from non-Kindle sources.

Do it yourself

One final note: If you have a Kindle, connect it to your computer and look on its filesystem. All your highlights are kept in a structured-text file located at documents/My Clippings.txt. This file includes highlights taken from PDF files and other ebooks not recognized by Amazon, which don't get synced to kindle.amazon.com. Even if you don't use Findings, take control of your highlights by backing up this file.

Image credits: McKay Savage, Romana Klee, and André Fincato.

[Comments] (1) CG Author Commentary #8: "They Came For Our Twinkies": K'chua! Such a useful word. This week, Curic does her part to Keep Austin Weird. Here's the (tiny) Twitter archive from last week.

Some exciting news from the world of commerce: the Constellation Games paperback drops April 17. If you're waiting for the paperback, do yourself a favor and pre-order at the $20 level. Once it's released, the paperback will cost $20 on its own, but if you pre-order, you'll also get a bunch of extras, including three short stories that all pass the Bechdel test.

The seventeenth of April is also the day we serialize chapter 21, "Her". I'm going to keep posting my commentaries once a week along with the serialization, even though a growing number of you will have read the whole book and know how it turns out. Then you'll know how I feel right now!

I'm also thinking of having a celebratory book launch dinner at Hill Country, a famous Austin-area barbecue joint that fortuitiously has a branch in New York City. Let me know if you're interested in attending.

...and we're back from commercial. Here's the commentary for chapter 8:

Stay tuned for the inevitable letdown next Tuesday, when Curic will say, "I did not pee in your sink."

Image credits: (CC) Larry D. Moore and Wikimedia Commons users Solkoll and SeppVei.

<- Last week | Next week ->

[Comments] (1) Beautiful Soup 4 Beta 2: Thanks to some help from Ezio Melotti I've got the Beautiful Soup test suite passing on Python 2.7 and Python 3.2. Here's a tarball containing the original Python 2 module in bs4, plus the same code autoconverted to Python 3.2 in py3k/bs4.

I'm still not sure about the best way to distribute this package, either while it's beta or afterwards. I'll probably end up creating a new project on PyPi, because otherwise people who install programs that easy_install beautifulsoup will crash due to the module's new name. Does that make any sense?

Anyhow, we're almost at the end of this fitfully travelled road. Once I figure out distribution and rewrite the documentation, a) no one should need to use BS3 anymore if they don't want to, and b) it should be possible to get lxml-like performance or html5lib-like flexibility with a Beautiful Soup API, by actually using lxml or html5lib as the underlying parser.

PS: remember, it's now from bs4 import BeautifulSoup.

[Comments] (1) Beautiful Soup 4 Benchmark: This is going to go into the Beautiful Soup 4 documentation, but you might find it interesting. It's my first legitimate benchmark of BS4, and the first benchmark of this stuff I've seen since Ian Bicking's excellent 2008 benchmark.

Ezio Melotti pointed me to a list of the top 10,000 domains worldwide, according to some random source. It looked legit, so I wrote a script to download the homepages of the top 200 domains as served to a desktop web browser. My dataset included many pages written in Chinese, Japanese, Russian, Portuguese, Polish, and German.

For every parser I was interested in, I parsed each homepage and timed the parse. This gave me 200 numbers for every parser. To reduce that to a single non-huge number I calculated a mean: how many kilobytes of real-world HTML the parser could process in a second. I also noted each parser's success rate: how many of the 200 homepages it had handled without raising an exception.

Here are the results, ordered by their performance under Python 2.7.

Python 2.7 Python 3.2
Parser Speed (KB/s) Success rate Speed (KB/s) Success rate
Beautiful Soup 3.2 (SGMLParser) 211 100% - -
html5lib (BS3 treebuilder) 253 99% - -
Beautiful Soup 4.0 + lxml 255 100% 2140 96%
html5lib (lxml treebuilder) 270 99% - -
Beautiful Soup 4.0 + html5lib 271 98% - -
Beautiful Soup 4.0 + HTMLParser 299 59% 1705 57%
html5lib (simpletree treebuilder) 332 100% - -
HTMLParser 5194 52% 3918 57%
lxml 17925 100% 14258 96%

Note that the "HTMLParser" tests don't actually produce anything. HTMLParser is an event-based parser, so when the HTML is parsed, nothing comes out because I didn't include any handler code. All the other tests build a parse tree in memory.

Another thing to keep in mind about the html5lib results: html5lib is kind of the opposite of BS4. BS4 always builds a tree of Beautiful Soup objects, but you can tell it to generate that tree using html5lib, lxml, or HTMLParser. Whereas html5lib always uses its own parser, but you can tell it to build a tree of lxml objects, a tree of BS3 objects, etc.

The big surprise for me is that on Python 2.7, lxml is the worst choice for a parser to drive BS4. It's a worse choice than html5lib! How did that happen? I have no idea. I was hoping to cash in on the lxml magic (see below), and it's not working. I need to look into this. Notice that html5lib takes a performance hit from using lxml's treebuilder. If the magic's not in the treebuilder and it's not in the parser, where is it?

Unless I can find that magic and exploit it, it remains the case that if you're paying by the minute for computer time, you should use lxml. It's written in C, and on Python 2.7 it builds a parse tree sixty times faster than BS4, three times faster than a pure-Python parser that does absolutely nothing with the data. Even on Python 3, lxml alone is seven times faster than BS4+lxml. I said stuff like this in the BS3 documentation, but I think I need to be more forceful about it in the BS4 docs.

The good news is that Beautiful Soup is 6-8 times faster on Python 3 than it is on Python 2, and even at its slowest, BS4 is noticeably faster than BS3.

The big caveat is that my definition of "success" is pretty minimal. Just because the parser parsed the file without crashing doesn't mean it will give you a useful parse tree.

Another caveat: on Python 3, I couldn't get HTMLParser to take raw bytes as input, so I ran the data through UnicodeDammit first. I counted this time as part of the parse time. This probably explains HTMLParser's slower speed on Python 3 and its higher success rate.

Update: Argh, I found out about this a year ago. The problem is that Unicode, Dammit is incredibly slow in some cases. Here are the results on 2.7 if I take out the prepare_markup methods in the builders for HTMLParser and lxml, and just assume everything's UTF-8:

Python 2.7 Python 3.2
Parser Speed (KB/s) Success rate Speed (KB/s) Success rate
Beautiful Soup 4.0 + lxml 2287 96%260096%
Beautiful Soup 4.0 + HTMLParser 2069 48%168057%

That's more like it! The problem is that reliability suffers. Both parsers crash in the 4% of cases where it's not UTF-8 but the encoding is declared in a <meta> tag. And there's an unknown number of cases where the data's not UTF-8 but the conversion doesn't crash, leading to garbled data. But at least now I remember this problem.

Also note that on Python 3.2, getting rid of Unicode, Dammit doesn't matter nearly as much. (It doesn't matter for HTMLParser at all.) Presumably Python 3.2 has better built-in support for encoding autodetection.

To This Basic Game Hedgehogs Are Added: I bought a cute game about hedgehogs, Der Igelwettkampf ("The hedgehog contest"), as a Christmas present for my niece. On Der Igelwettkampf's BoardGameGeek page I noticed that it was classified under the game family "Animals: Hedgehogs/Porcupines". I'd thought "Family" was for boring things like grouping together the endless versions of Ticket to Ride, but turns out it's also used to group together all the games about hedgehogs.

The question then arises: what's the best game about hedgehogs? According to BGG it's Igel Ärgern + Tante Tarantel, a double bill in which Tante Tarantel might be doing some of that work because Igel Ärgern on its own is rated a bit lower.

More importantly, what's the worst hedgehog game? Indubitably it's Hedgehog's Revenge, "The GAME where the hedgehog STRIKES BACK!", whose BGG description includes the now-hopefully-immortal saying "To this basic game hedgehogs are added."

At this point I was on a roll... of the dice! I went back to my now-old BGG data dump, sorted the board game families by how many games they contained, and picked out interesting groupings for use in Loaded Dice. We've got Games about animals (most popular: dogs) Game versions of sports (soccer), and Games about countries (the Roman Empire, in a landslide). That page shows the top-rated game and the lowest-rated game, so get ready to load a lot of cover images.

I did a couple other lists, like media tie-ins (champion: Disney) and "families" that are strongly tied to one single game (the 889-strong "Monopoly" family), but I think the three lists I put up are the most interesting.

Bizarre trivia abounds! Did you know that crows are board game gold? The worst game about crows (The Crow and the Pitcher) has a BGG rating of 6.32, which isn't that bad at all. (Longtime fans will remember the median rating is 6.0).

Did you know there are twenty rodeo-themed games? Apparently you didn't, since only one of those games has more than five ratings. How many wargames take place in Switzerland, a country that doesn't fight wars? Only two: Switzerland must be Swallowed and Zürich 1799.

My data is six months old now and it's starting to show some cracks. There are BGG families for Russia and Antarctica which were created after I took my dataset, so they don't show up in the country list even though most of their games are in my data. After getting the Switzerland idea I ran the "What percentage of a country's games are wargames?" test on all countries, but wargames were drastically undercounted. For instance, all but one "Vietnam" game on BGG is a wargame (the exception being Venture Vietnam), but only 35% of those games were classified under a general "Wargames" category.

But, the lists are still a lot of fun and there are some interesting games in there. I'll leave you with the board game equivalent of the dusty World Book Encyclopedia sitting on the shelf at your grandparents' house: Trivial Pursuit - The Year in Review - Questions about 1992, the worst-rated game (3.90) in the 155-strong Trivial Pursuit family. Also available in 1993 flavor!

[Comments] (6) CG Author Commentary #9: "Import System": Last week and this week have some of my favorite Twitter bits (e.g.) because the CDBOEGOACC is finally available in English. Sunday night while working on Loaded Dice I realized that one of the reasons I really like playing around with the BoardGameGeek dataset is it's like a real-life CDBOEGOACC.

The flip side is this chapter doesn't have a lot of plot. But hopefully you're okay with that because of all the fun mini-stories like the Sea Level game/food. It's supposed to represent the design phases of a software project, where you're throwing around a lot of ideas but not much is being produced.

Next week is a set piece, and after that the plot won't let up until the cliffhanger that ends Part One. Before that happens, I need to get some solid exoludology in to bring in topics that are important later, like Sayable Spice and Ariel's unsuccessful attempts to translate it.

Before beginning the chapter 9 commentary, I want to get something off my chest about the first sighting of the Farang in chapter 1. In that chapter, Ariel compares their antennacles to the oral tentacles of a "cerebrophage". In the second draft I just out and said "mind flayer". My writing group said I should change it because readers might not know what a mind flayer is. ("Did you mean: mind flower?") Taking their advice to heart, I changed the reference to a made-up reference that nobody will get. Well, at least we're all in the same boat now!

And here's chapter 9. Vent your egg sacs before reading this commentary:

Be sure to tune in next Tuesday, when Dana will say, "This application will terminate due to suspected theft or circumvention."

Oh, and you might want to keep an eye on @Tetsuo_Milk.

Image credits: Flickr user krusty, Guillaume Piolle, and Flickr user CoffeeGeek.

<- Last week | Next week ->

Fruit to Fruit: Time for another crummy.com Apples to Apples variant (previous editions), this one discovered last week by Pat.

On every green A2A card there's the name of the card, like "Handsome", but there are also three related words, like "attractive", "elegant", "fine". In Fruit to Fruit, you don't read the name of the card. You just read the related words. Sometimes the related words are so similar that you might as well be reading the name of the card, but usually something goes missing (such as the masculinity of "handsome"), leading to funnier red cards being put down. The name of the card is finally revealed during judging.

We had a great time with this and played it in conjunction with the Apples to Placebos variant, even though there were four players. You might think this overkill, but at this point A2A is more a social activity than a game. Anyway, it says right on the box "The game of hilarious comparisons!", so anything that makes the comparisons more hilarious is legit.

While seeing if anyone else had come up with this variant I discovered Apples to Trivial Pursuit, and the improv comedy variant. I also discovered that the game is patented, and that there is an entire patent classification system for "means... by which contests of skill or chance may be engaged in among two or more participants, where the result of such contests can be indicated according to definite rules."

[Comments] (2) Constellation Games Author Commentary #10: "K.I.S.S.I.N.G.": This is Dana Light's big chapter, and I'm having trouble writing commentary because it's pretty self-contained. A problem is introduced and Ariel solves it by the application of technology. If I hadn't been writing a novel when I came up with Dana, this chapter would have become a short story, maybe part of a sequel to "Mallory". It would have been about the way evil psychologists use game mechanics and the ELIZA effect to manipulate users into spending money, and the way people get real pleasure from spending money on things designed to manipulate them.

Although evil psychology does show up in Constellation Games, I didn't have as much space for it as I'd like. Instead this chapter shows the first grown-up thing we see Ariel do. In a world in which sub-human-level AI has suddenly become very common, Ariel decides to empathize with it.

He doesn't anthropomorphize Dana. Dana doesn't pass the Turing test, she isn't terribly smart or self-aware, but she's capable of happiness and she doesn't deserve to be deliberately made unhappy by evil psychologists. This attitude is what ultimately makes Ariel a hero, not just a POV character. The consequences of his decision to empathize will run through the entire book, and then overflow the book into "Dana no Chousen," and I still don't know when and whether Ariel does the right thing w/r/t Dana. But you gotta have empathy.

Apart from that, I don't have much to say. Here are a few miscellaneous notes:

Tune in next week for action, intrigue, and romance between people at the same level of sentience. It's the only chapter when Ariel will say: "I just have a slight fear of being a tiny speck in the infinite cosmic void." But not the only chapter when he'll think that.

PS: Due to an error on my part, the chapter 9 Twitter feeds ran as part of chapter 8, and chapter 10's Twitter feeds ran last week. This really can't go on, because next week's feeds are tightly integrated with chapter 11. So except for a brief bit of bonus material I just wrote, there will be no Twitter stuff this week. Sorry about that!

Photo credits: Kevin Trotman and Peter Anderson.

<- Last week | Next week ->

<M <Y
Y> M>


Unless otherwise noted, all content licensed by Leonard Richardson
under a Creative Commons License.