(3) Mon Jan 02 2012 19:32 Frances Daily: We're out of the gate with the first Crummy project of the new year. I spent Christmas in Utah, and while I was there my sister Susanna foisted on me my mother's old Franklin day planner. She's already typed up my mother's Franklin diary from the 80s and 90s, a diary very similar to the weblog she kept from 2001 until she died in 2006, so all that was left of the day planner was an enormous block of calendars.

It's a very detailed calendar that will be useful for family history purposes but quite boring otherwise. Except, at the beginning of each month there was an "index" page on which my mother had written a brief summary of every day of that month. There were index pages from January 1987 to November 1990, with a few sporadic months from later on.

The cumulative effect of these daily summaries was incredibly powerful. This was a time of major upheaval in my family, leading up to (but not ending with) my father's death in 1992. The monthly summaries show my mother trying to keep it together while studying for a graduate degree and raising three kids pretty much on her own. It's inspiring and a bit horrifying.

I've started a Twitter account, @FrancesDaily, which is using Sycorax to reprint the daily summaries 25 years after my mother wrote them. (Here's the RSS feed.) The summary for each day will go up at 4 PM Pacific time. It's a little spotty in January, but once it picks up she doesn't miss many days for the next three years.

I don't know what the effect of the summaries will be when experienced in real time. Probably when I read one I'm going to mentally compare my day against the day Frances had 25 years earlier, and you might want to do the same.

This is a much lower-bandwidth project than Roy's Postcards, but where my father's writing almost never showed any emotion, these summaries pack a lot into just a couple words. Susanna read the index and said "Mom was really hard on herself." So please take entries like "wasted day" as accurate depictions of my mother's mental state, but not necessarily of reality.

A note about the names. I did the thing you frequently see in old journals that have passed through the hands of the journal-writer's descendants, and replaced most of the names with initials. For instance, the January 18 summary, "Mario's Eagle", became "M's Eagle". The Mario mentioned is Mario Canton, one of my dad's Boy Scouts and later a family friend. I initialized most of the names because you don't have the context to know who all these people are, and I think giving each entry an explanation significantly longer than the entry itself would ruin the effect.

In a few cases, making peoples' names into initials was also necessary to protect their privacy. I left a few names intact: mostly my mother's close relatives like her aunt LeJeune ("Jeuney"), her sister Anne, my father, and of course me and my sisters.

Again, that Twitter account is @FrancesDaily. Here's the RSS feed.

(1) Tue Jan 03 2012 08:49 CG Author Commentary #5: "The Stars My Screensaver": Yeah, you know it's getting serious now. The microblog archive is up, I'm feeling good and it's time for some commentary:

• The chapter title is one of my favorites, and I'm proud of so many lines in this chapter. I especially like "The aliens are not a dessert," the Paperwork Reduction Act line I used as last week's teaser, and Curic's unintentional poem about the nebula. I didn't realize until after I'd sold the book that the odd line breaks I used for Curic (meant to convey the rhythm of her native language) turn everything she says on IM into free verse.
• Ariel's older brother Raphael has been mentioned several times before, but I forgot to talk about him. Because there's not much to say. He never shows up in the novel except in stories Ariel tells about when they were kids. But I needed to establish Raph's existence so it's not a big surprise when Ariel pulls out those stories.

  In my mind, Raph is now an economist for the IMF or something. His parents are very proud, Ariel less so.

• The Bureau of Extraterrestrial Affairs first showed up in "Vanilla". It was presented as less sinister and more farcical, mainly because the protagonist of "Vanilla" is in a good position to manipulate the BEA, and Ariel is not.
• Want to know something hilarious? In the first version of this chapter (second draft), BEA agents Krakowski and Fowler didn't even have names. In his blog, Ariel called them Good Cop and Bad Cop, after their roles in this initial conversation. My writing group raised an enormous stink about this, and eventually Ariel made up ridiculous fake cop-show aliases: Krakowski and Tonsil. But as the second draft progressed and the BEA agents became essential to the plot, I found it harder and harder to have a character named "Tonsil" deliver important lines.

  Also, I'd never given them personalities. You couldn't tell who was talking. It was awful. Writing group conversation: "Why do you even have two of them?" "Well, they're Men in Black, there's always two of them..."

  After completing the second draft I considered a number of ways of distinguishing the characters, including making Krakowski a woman. That didn't happen, not least because it would turn Krakowski/Fowler into Scully/Mulder. But out of that thought process and the Good Cop/Bad Cop thing came the distinction I went with. (Highlight to reveal spoiler:) Fowler is everything Ariel hates: a reactionary thug getting by on his charm and connections. Krakowski is basically the same person as Ariel: an outsider, a xenophile, smart and driven. But Krakowski and Fowler have the exact same job.

• I'm pretty sure 9/11 didn't happen in this universe (and at this point you might be able to guess why not), but there's a Department of Homeland Security anyway. Go figure.
• One of my favorite feelings when writing is the realization that you can reuse a detail that earlier was merely incidental. "Limited Nuclear Exchange" is just a random game reference in this chapter, but much later in the book I needed to write a tense conversation between Ariel and Jenny, so I set the conversation during an LNE session and fleshed out the game: a board game, a cross between Twilight Struggle and Twilight Imperium that takes half an hour just to set up and generally ends abruptly with both players losing.

  That's a small example, but in a couple chapters there's another offhand reference that greatly shaped one of the most important scenes in the book. Watch for it! Specifically, watch for me pointing it out.

• At the end of this chapter you see exactly the thing Krakowski is afraid of at the beginning: a race to the bottom. Once one country opens up to the Constellation, the united front crumbles. How long until the USA acquiesces to the alien anarchists' demands in exchange for access? Not long at all, considering I need to keep the plot moving.

And there's the commentary. Stay glued to the proverbial set for chapter 7, when Ariel will say, "Well, her hardware's Chinese..."

Image credits: Gisela Giardino, The United States Department of State, and the East German postal service.

<- Last week Next week ->

Wed Jan 04 2012 11:38 GeekDad Reviews Constellation Games: Wired's Jonathan Liu got an advance copy and calls it "a perfect blend of aliens, video game geekiness, and modern social media." Other quotes relevant to my interests include "absolutely loved it — it might not be the best book I read in 2011, but it’s certainly in the top 10."

C'mon, folks, do I have to draw you a picture? Because I can't draw very well. I use words instead.

Sat Jan 07 2012 10:18 Dear Santa: Dinosaurs:

My niece's Christmas stocking project.

Mon Jan 09 2012 06:30 Connected Tragedies: Sumana noticed this. 1, 2.

Tue Jan 10 2012 08:43 CG Author Commentary #7: "Party Creation": Ariel's Twitter will be pretty quiet this week because the entire chapter takes place over the course of one day. Your only solace is farmers market quail sausage, and this COMMENTARY:

• In this chapter we meet Bai's girlfriend, Dana Light. Sort of. Dana is the bane of this minimal-spoiler commentary because it's a bit of a spoiler even to treat her as an important character instead of a bit of character development for Bai. But she is important--Bai was defined in terms of Dana, not the other way around, and I found Dana interesting enough to make her the star of the first bonus story, "Dana no Chousen."

  Chapter 10 will be a very Dana-heavy chapter. If I hadn't been writing a novel when I came up with the character, that chapter would have become a standalone short story. Of course, once I had the character, Dana quickly became important to the story (spoiler!!) and after selling the book I moved bits from Chapter 10 backwards into this chapter so as to introduce her earlier.

  More on Dana and Dana/Bai in a few weeks when I commentate chapter 10, but for now you might enjoy going back to chapters 3 and 4 and re-reading the earlier references to her.

• Bai is pretty ditzy when it comes to women, because that was the only way I could get him into this relationship without making him stupid or a Love Plus-style otaku. If I was a better writer I might have come up with a better solution, but I'm at the level I'm at. However, there are a couple moments later on when Bai is surprisingly perceptive about Ariel's love life.
• The other introduction in this chapter is Sayable Spice, probably the game most important to the plot. In the second draft, Ariel talked about Sayable Spice a whole lot, but it wasn't until chapter 14 that he mentioned that he was working on a remake. Now Jenny forces Ariel to commit and puts a stop to that nonsense right quick. Thanks, Jenny.
• When I got rid of Bruce, the character you'll never see, I gave most of his scenes to Bai. (Eg. it was originally Bruce playing Knockdown Dragout in chapter 4.) I'm pretty sure the scene in the backyard here is the only Bruce scene I gave to Jenny. Nobody cares!
• This is also the only chapter in which Bizarro Kate has a speaking part. In the second and third drafts, she hooked up with Bruce at this chapter's cookout, and later in the novel they were living together. Hers was a supporting role to begin with, and when I eliminated her major-character boyfriend, she got cut down to just a walk-on. But I really like the character, and I gave her a chance to shine (ie. be cranky) in "Found Objects", the Jenny story.
• I think Tennis For Two is the only real-life video game I mention in the novel. There was no other way to get the joke to work.
• The tattoo joke I had to cut from chapter 19 has landed on Twitter. Relive the saga: 1 2 3 4. I don't think I needed to actually present the translation. Oh well.
• Around this time I planned on changing Ariel's Twitter profile image to the duck from the Crispy Duck Games logo, but it didn't look good, so I think I'm/he's going to stick with the pissed-off pony.
• The only other thing I want to say about this chapter is you see a little bit more of how old ET video games just aren't as interesting to other people as they are to Ariel.

Stay tuned for chapter 8, a chapter I think is one of the best in the book, the chapter that got me to give up on the first draft and rewrite the entire book to be more like it. The only chapter in which Jenny will say, "Wait a minute, are you naked?"

Image credits: I got the first image from Flickr user marsmet462, not sure if they put in enough transformative elbow grease to put their own license on it. Second image comes from Sven-S. "☃" Porst .

<- Last week | Next week ->

(8) Tue Jan 10 2012 11:42 Hostile Witness: The Vechs Interview: Minecraft can be a tough game. The controls are kind of blocky, the best resources are hidden deep in the map alongside deadly lava rivers, and the night hosts monsters that will kill you just as soon as look at you (or, in one case, just as soon as you look at them). But it's not that tough. All that terrain is generated by algorithm. It's not like the random number generator is trying to kill you.

But there's this guy named Vechs who is trying to kill you. His "Super Hostile" series of custom Minecraft maps offer challenges that prohibit or subvert every survival strategy you learned in vanilla Minecraft. Even in his easier maps you'll find bottomless pits, world-spanning ceilings that block Minecraft's all-important sunlight, swarms of monsters pouring from hacked spawners, and TNT in unfortunate places. Just getting your first tree is a challenge. Complete a Super Hostile map, and vanilla Minecraft will seem easy.

But Vechs' maps are not just tough: they're creative, fun to play, and they look great. Vechs uses landscape features and lighting to grab the player's attention, direct the flow and pacing of their playthrough, and give them a spectacular environment to build in once they've conquered the map.

I've raved about Super Hostile a couple times before here on NYCB, but with the release of "Spellbound Caves", the tenth entry in the series, I knew it was time to get serious. I sat down with Vechs (I assume he was sitting down, anyway) and interviewed him over minecraftforum.net's private message feature. My goal was to pick up where this interview from July 2011 left off, with in-depth questions about his style and his mapmaking wish-list. The interview contains some Minecraft jargon, but anyone with an interest in game design should get something out of it.

---

Leonard: You play a kind of character on your maps, an angry trickster god who hates his players and taunts them by writing things on signs. But clearly you're not actually like that. I've played maps made by people who really did hate me, who wanted me to farm cobblestone for an hour or dig through obsidian without a tool, and I said "screw this" and quit the map. I don't think you'd ever do that, right? What's the difference between you and the "Vechs" in your maps?

Vechs: It has to do with challenge. There is a difference between making the player use skill or ingenuity, and making the player do something tedious. Sometimes a solution to an area can involve using lots of blocks (Like the player making a cobblestone tube for them to safely move through.), but these are usually just one option of many the player can use to conquer an area.

Sometimes the "Vechs character" in my maps is pretty mean, and just downright spiteful, especially when it comes to traps. In real life, I'm not like that at all.

In the Obsidian Block interview you say that you recently graduated from college and are looking for a career as a game designer or world designer. What did you study in college?

I am a Media Arts major. I studied everything from digital image editing, video editing, to stage lighting, to writing scripts and screenplays for movies, and more. I'm glad to have a diverse background, even though my passion is still game design.

What would be your ideal job? Would you rather work on a big-budget project with high production values, or an indie project where you have more creative control?

My first choice would actually be to have my own studio and bring to life some of the game ideas I have. One idea I've had for a while, and as far as I know, nobody has ever made a game like it. I wouldn't mind making it all myself, but that means I would have to re-learn a lot of programming. I've programmed some text-based games in C++, but programming is not my main forte.

That said, I also wouldn't mind working for a major company. Like, for example, Valve. Love those guys.

What other games have you made maps for? You mention Duke Nukem 3D in the Obsidian Block interview; what else?

Just off hand: Red Alert, Warcraft II, Warcraft III, Neverwinter Nights, Total Annihilation, TA:Spring, Terraria, Command and Conquer (and several sequels), The Elder Scrolls series, and obviously I'm the world designer for the RPG games I've worked on, using the XP and VX engines.

Are you currently making maps for any games other than Minecraft?

At this moment, no, but I have been meaning to make some maps for Team Fortress 2.

Have you ever heard of ZZT or Megazeux, or am I just incredibly old?

You're old! *grins* I looked them up, and I think my version of that would be the RPG-series of game engines.

There are a lot of memorable set pieces in the Super Hostile series. Now that you've put out ten maps, would you mind taking a look back and sharing some of your favorites?

The first 15 minutes on just about any of my maps. I love that feeling of just starting off and scrambling for resources. I like the rail station in "Sea of Flame II", and how it goes out in the area with the huge pillars, and "Spellbound Caves" is just full of nice vistas and "scripted" events.

Most of my maps feature at least one "death fortress" as an end-game area. These are intended to be where the player gets to use all the resources and items he has been collecting through the whole map. TNT, lava, swords, bow and arrow, even TNT cannons... bring your whole arsenal and have some fun!

Can you describe the evolution of your design philosophy over the course of the series?

Try to improve in at least one area every time I make a new map. Push the Minecraft engine to its limits. Make an awesome and memorable experience for the player.

What are the biggest challenges in re-balancing Super Hostile for Minecraft 1.0?

Armor and blocking.

Does 1.0 have anything to do with the fact that you recently flattened the difficulty levels in your map descriptions, so that "Sunburn Islands" and "Legendary" are now both considered "Easy"?

Yes and no. I feel that recently I have been drifting away from the theme of "Super Hostile" and I want to get back to my roots. Being able to respawn forever, over and over kind of takes the risk out of a map. Even in "Legendary", unless you really mess up and drop all the wool in lava or something, you can just set your bed spawn near an area, and try over and over until you get it right. I think that's pretty Easy on the player, even if the area you are attempting is challenging.

Call me nostalgic, but I kind of miss (sometimes) the GAME OVER screens from older video games. Modern video games, in the name of convenience, typically feature unlimited lives, save games, checkpoints, the works. But beating a modern video game, I have to admit, is much less satisfying than beating some of those old NES games. You can just bang your head against the game until you get lucky and get through an area. Heh, man this makes me feel old! "In my day, we didn't have all those checkpoints! We had three lives! One hit deaths! And we were happy!" *shakes cane*

Anyway, I do think this is a legitimate point of concern on modern game design, is risk versus reward. It is possible to make games so easy that they are very unsatisfying...

I'm an admirer of your ability to create new genres of map. Have you made experimental Minecraft maps that just didn't work? What's in your "abandoned projects" folder?

The only thing I've actually stopped on, is "Race for Wool #3: Common Ground". Because it basically became "Capture the Wool".

Have you ever made maps for a game that featured scriptable events? If so, do you miss that capability in Minecraft?

I have used C++ to code some text-based games. I have also used various scripting languages in the process of making mods or making my own games with existing engines. You do have some limited "scripting" ability in Minecraft, using redstone. Check out the Rumbling Caverns in my tenth map and you will see what I mean. :)

But yes, I would love some even rudimentary scripting in Minecraft. I believe a while ago, I proposed invisible effector blocks, that you can place with Creative or MCedit, that modify the immediate environment around them. Like, an invisible block that makes monsters not spawn within 50 blocks. Or one that doubles monster spawning within 50 blocks. Or one that makes it snow. Or one that makes a ray of sunlight always be shining on that spot. Or one that makes the temperature freezing so any water turns to ice. Simple stuff like that. They would show up faintly in Creative mode, but be invisible while in survival mode.

What would you like to see added to Minecraft? On your forum thread you mention that you'd like to add sharks and underwater plants to "Endless Deep". What else?

Bow enchantments... more mining enchantments, such as area mining. Check out episodes 04 and 05 of my Spellbound Caves Dev Commentary.

For bow enchantments, I would like:

• Piercing (Arrows go through monsters and cause damage in a straight line.)
• Toxic (Arrows cause poison.)
• Fire (Arrows ignite enemies.)
• Knockback (Arrows cause knockback.)
• Phantom Spreadshot (In addition to your one normal arrow, you fire out additional ghost arrows (higher ranks provide more) that act like normal arrows, only you cannot pick them up, and they quickly vanish after hitting the ground.)
• Explosive (Arrows cannot be reclaimed and cause a small explosion, could possibly be combined with the fire enchant, so arrows function like ghast projectiles, causing an explosion and leaving fire.)
• Charge Speed (You charge your bow faster.)
• Unbreaking (Your bow lasts longer.)

I think these enchants for bows would make bow combat much, much more fun. It's currently fairly slow paced, and a bit boring. Imagine a bow with Toxic, Piercing, and Phantom Spreadshot on it! It would be so much fun to shoot groups of enemies with a bow like that.

You have a creative relationship with some of the people who do Let's Play videos of your maps. It's a kind of relationship I've never seen before: the way people play your maps in public affects the way you design later maps. How did these relationships develop?

Very organically. Zisteau agreed to LP my very first map, "Sea of Flames" version 1.0, and ever since then, he's been involved in playing my maps, and giving feedback.

There's a very clever trap in "Spellbound Caves", [location redacted]. It's clever for many reasons, but I'm asking about it because it doesn't seem to have any triggering mechanism. I went in afterwards and took the walls apart and couldn't figure out how it works. What's the secret? Or is there a pressure plate somewhere that I missed?

I has a seekret. Oh, also, I hate you, die in a fire.

---

POSTSCRIPT: With my interviewee uncooperative, I had no choice but to load a fresh version of "Spellbound Caves" into an editor to get to the bottom of the mystery. What I found was a trigger that did not shock me to the core of my being. But it is a cool design.

The trigger is a proximity sensor: a shaft behind a wall, with a creeper spawner at the top of the shaft and a pressure plate at the bottom. When the player gets within 16 blocks of the spawner, it activates and spawns a creeper, which drops onto the pressure plate, triggering the trap. The resulting explosion obliterates both creeper and spawner, leaving no trace of the trigger.

And that's what you get with Vechs' maps: MacGyver-like use of everything the game engine provides, to create confounding and unexpected effects. Seriously, game studios: hire this guy. Everyone else: play his maps.

---

PPS: Hey, people from minecraftforums.net, thanks for coming over. I've written other articles about Minecraft (1 2 3 4), and if you like my stuff, you might want to check out my novel about alien video games.

(1) Thu Jan 12 2012 09:17: My sister talks about her miscarriage.

The only thing worth saying is "I'm sorry." I may think those things. John and I may even say those things to each other. But don't impose beliefs or possibilities or happy thoughts on me.

(1) Mon Jan 16 2012 10:50 Findings: My writing life has settled down a bit so I'm finally going to write about Findings, the social reading startup where I worked last summer. This is more an essay about what I see in Findings than an introduction to the site--you can see lots of general introductions linked to from this Findings blog post, including co-founder Steven Johnson's introduction, and the Business Insider article whose title is the perfect elevator pitch, "Findings is GitHub for Ideas".

If what I'm about to say sounds interesting to you, there are development jobs open at Findings right now. Just as a reminder, I myself don't work at Findings anymore, and even when I did, only the foggyheadedest knave would have taken my personal opinions as representative of company policy.

Let me start out with this quote I took from Darwin's The Descent of Man, not because the quote itself proves anything, but because the quote is an important part of my reading of Darwin:

Brehm gives a curious account of the instinctive dread, which his monkeys exhibited, for snakes; but their curiosity was so great that they could not desist from occasionally satiating their horror in a most human fashion, by lifting up the lid of the box in which the snakes were kept. I was so much surprised at his account, that I took a stuffed and coiled-up snake into the monkey-house at the Zoological Gardens, and the excitement thus caused was one of the most curious spectacles which I ever beheld.

If you want to learn about evolutionary biology, read Steven Jay Gould's essays. Darwin's a good writer and he got it basically right, but he didn't know about genes or DNA. I read Darwin to experience the origins of the field. I didn't expect (though perhaps I should have) to encounter endless artifacts of the days of two-fisted Baconian science.

When Charles Darwin needs to figure something out, he carries out an experiment, no matter how tedious or disturbing. He takes snakes to the zoo, he puts kittens' feet in his mouth, he floats 94 kinds of plants in seawater, he hacks aphids. If someone has the temerity to question his conclusions he's all "Citation needed? I'll give you citations, motherfucker!"

When you read a book, it has an effect on your mind. You're a slightly different person after reading it. You've created something new: a reading of the book: Here's an apropos quote from Alberto Manguel's A History Of Reading, which I read on paper and typed in. Manguel is talking about Petrarch's Secretum:

What Augustine (in Petrarch's imagining) suggests is a new manner of reading: neither using the book as a prop for thought, nor trusting it as one would trust the authority of a sage, but taking from it an idea, a phrase, an image, linking it to another culled from a distant text preserved in memory, tying the whole together with reflections of one's own -- producing, in fact, a new text authored by the reader.

Readings are ephemeral. Life goes on, and the memory fades. Ken Macleod's The Star Fraction had a huge influence on me, probably leading to whatever career I now enjoy as an author of fiction, but I read it ten years and 600 books ago, and now I don't remember a damn thing about it.

That's why we dog-ear pages and highlight passages. We're instantiating our reading of the book so we can go back later and approximate the mental state it gave us without re-reading the whole thing. Even if all we got out of a book was "this bit was funny", it's better to have the funny bit at hand than not. Even if you never go back to the highlighted passage, the act of highlighting replays that passage and deepens your initial memory of it.

Liberate your readings

I've been typing in quotes from the paper books I read, like I did with the Manguel. Of course, with an electronic book, you don't have to do this. The act of highlighting creates an electronic record of your reading of the book. When I was in college I read about the first e-ink research coming out of MIT, and I knew that this was the future. Indeed it was the future, because I had to wait ten years for the technology to make it to market. But, sour grapes, we've got ebook readers now.

Ebook readers have big problems, but at this point the problems are mostly political, not technical. For instance, you can highlight passages when reading a book on your Kindle, but because of a deal between Amazon and the publisher, your book's metadata may include restrictions, which the Kindle will obey, on how much you can highlight. And your highlights and notes—the "new text authored by the reader"—are stuck on a website that Amazon didn't put a lot of work into because they don't consider your reading of a book important to their business.

Findings takes advantage of the fact that Amazon is wrong about this. Findings liberates your highlights and makes them searchable and shareable. Your reading of a book is a big part of your relationship with that book, and Findings gives you access to it.

You can also use Findings to take a reading of a web page, creating a record of what would otherwise be an ephemeral activity. I'm not as interested in this feature, but people are using it quite a bit, and my interest does increase as the length of the web page I'm reading approaches the length of a book.

Browse readings

So that's what Findings can do for you personally. Now let me pitch you the network effects. Take a look at this screenshot which shows the Findings global stream:

You can't see the global stream without logging in, which I think is a shame because I think this is what really sells Findings. We have here a stream of little bits of text, like Twitter used to have on their front page. Except here, every bit of text is a quote that someone liked well enough to save. It's very high-quality stuff. At the top you can also see some recently added books, and by clicking on a book you can see someone's condensed reading of the book.

Basically, Findings gives you browsing access to a large library, not of books, but of readings. It's easy to discover new books, people who read books you like, and—this is new—people who read books in ways you like.

There are a ton more useful things I could mention, but they're mostly behind-the-scenes things where Findings makes things "just work" (like consolidating multiple editions of the same text), or they depend on features that haven't been implemented yet. So I'm going to close by mentioning the social signalling feature.

Strut your stuff

One underappreciated feature of paper books is signaling to other people that you are cool. You read books! Fancy books, like Ulysses! You care enough about books to make space for them in your house. You take them on the subway even though they're heavy. Darwin would say it's like the peacock's tail. But if you have an ebook reader, nobody knows how cool you are. You're just a person with an ebook reader.

By letting you publicize your reading of an ebook, Findings reinstates your ability to send those social signals. The downside is that you have to actually read the book. You can't just put a big book on your coffee table: the thing you're sharing is what you got out of the book. (Well, you can fake it, but it's probably about as much work as reading the book legitimately.)

So that's Findings. I don't use it as much as I thought I would, because I'm still trying to draw down my stack of paper books, but when I read a book on my Kindle, it stays read, thanks to Findings.

I mentioned this before, but the last thing I did at Findings was design a web service for them, which they're hopefully working on now. Once the web service launches, you'll be able to write programs that import readings into Findings from non-Kindle sources.

Do it yourself

One final note: If you have a Kindle, connect it to your computer and look on its filesystem. All your highlights are kept in a structured-text file located at documents/My Clippings.txt. This file includes highlights taken from PDF files and other ebooks not recognized by Amazon, which don't get synced to kindle.amazon.com. Even if you don't use Findings, take control of your highlights by backing up this file.

Image credits: McKay Savage, Romana Klee, and André Fincato.

(1) Tue Jan 17 2012 10:10 CG Author Commentary #8: "They Came For Our Twinkies": K'chua! Such a useful word. This week, Curic does her part to Keep Austin Weird. Here's the (tiny) Twitter archive from last week.

Some exciting news from the world of commerce: the Constellation Games paperback drops April 17. If you're waiting for the paperback, do yourself a favor and pre-order at the $20 level. Once it's released, the paperback will cost $20 on its own, but if you pre-order, you'll also get a bunch of extras, including three short stories that all pass the Bechdel test.

The seventeenth of April is also the day we serialize chapter 21, "Her". I'm going to keep posting my commentaries once a week along with the serialization, even though a growing number of you will have read the whole book and know how it turns out. Then you'll know how I feel right now!

I'm also thinking of having a celebratory book launch dinner at Hill Country, a famous Austin-area barbecue joint that fortuitiously has a branch in New York City. Let me know if you're interested in attending.

...and we're back from commercial. Here's the commentary for chapter 8:

• Hopefully this chapter doesn't seem too special now, but when I first wrote it it was a revelation. It was a narrative with a plot! Curic was an active character and you could see her chemistry with Ariel as they physically interacted! My writing group indicated in no uncertain terms that this was the stuff they had been waiting for.

  I wrote a couple more chapters in the old, boring style but it wasn't long before I gave up and started the second draft, which tried to make the whole book more like this chapter. To this end I introduced innovations like the long IM conversations, and Jenny. Imagine reading up to this point, except Jenny has only been mentioned once and Curic has only had three lines of dialogue. That was the first draft.

• Because of its importance to the book's history, and also because it's such a great set piece, I kept coming back to this chapter. Near the end of the book, I recontextualize it by showing the visit to Earth from Curic's perspective. And Jenny's remark that the plastic fractal "looks like Skewer Sue's bracelet" fed, as I hinted a few weeks ago, into one of the book's most important scenes.

  Which almost didn't happen. At this point I'd decided that Jenny was a big comics nerd and wanted her to reference a kinda bland comic book character name. My first try was "Titania". Well, the joke was on me because Titania is an actual Marvel villain. Sumana says she showed up during Dan Slott's run on She-Hulk, so that's probably where I got it, but I don't remember her. Bland, but already taken. Titania was out. Then I came up with "Skewer Sue", which is not a top-tier name like "Wonder Woman", but is definitely not bland. Then I decided "why should I deliberately come up with bland names?" and went with Skewer Sue. Believe me when I say that if I'd been able to name that character Titania, the offhand reference would have stayed offhand forever.

  ...And I can't even tell you all the other things in this chapter that become important later. Because they're all huge spoilers. But in one case also because it's quite embarrassing to admit how long I took to realize that I could reuse something. I should have called this chapter "Guns on the Mantelpiece". (Actually, I should have called the second chapter "Guns on the Mantelpiece"; I never liked "Corner Pieces".)

• Jenny's nephew Eduardo does not really show up again, except in the Twitter feeds and one of the deleted scenes, but if you want to make your protagonists look good you should show them doing nice things for kids. Just a little tip I picked up in Sid Meier's screenwriting class.

  Eddie was originally the son of Jenny's brother, James, who never shows up at all. I don't remember why James became Jenny's sister, but when I changed it I de-named the character so as not to use up a "named character" slot in your head.

  Actually, rereading this, I notice I didn't give Eddie any lines. That was kind of sloppy. I'd retcon it by saying he's intensely shy around Curic, but Curic's account of the day contradicts this! Oh no.

• The star-draw was added in the third draft. There's another star-draw in chapter 20, but a lot of other stuff is happening in that scene, and Ariel wasn't actually there for it, so the exposition was very difficult. Putting a star-draw here takes some of the load off the chapter 20 scene, and--only later did I realize this was much more important--shows you a fluid overlay in action. Albeit a two-person fluid overlay because Jenny and Bai are slackers and Eddie's a little kid.
• The celebrity voices used for the different ET vocalizers are a tiny import from "Vanilla", in which (among other things) the protagonist was shaken down by an ET who sounded like Garrison Keillor. Now that I write that down, I realize that I've only got one joke about this, because there's a very similar incongruous-voice bit in chapter 21. Maybe I should have declined to import this particular thing from "Vanilla"... it feels like an artifact from a less mature stage of my style.
• The bit where Bai introduces Dana to Curic was added after I sold the book, at editorial request to get more Dana early in the story. You'll thank my editor later.
• And I don't know when all of this came together, but I love Curic's mix of anthropological curiosity, disinclination to tolerate bullshit, and bizarre opinions on what's bullshit. Her reactions to Dana, to "Eddie wants to be an astronaut", and to the revelation that "fuck" is a swear word... she's got all the best lines. And she does the game reviews! This is the chapter where Curic came alive for me. Metaphorically.

Stay tuned for the inevitable letdown next Tuesday, when Curic will say, "I did not pee in your sink."

Image credits: (CC) Larry D. Moore and Wikimedia Commons users Solkoll and SeppVei.

<- Last week | Next week ->

(1) Fri Jan 20 2012 17:40 Beautiful Soup 4 Beta 2: Thanks to some help from Ezio Melotti I've got the Beautiful Soup test suite passing on Python 2.7 and Python 3.2. Here's a tarball containing the original Python 2 module in bs4, plus the same code autoconverted to Python 3.2 in py3k/bs4.

I'm still not sure about the best way to distribute this package, either while it's beta or afterwards. I'll probably end up creating a new project on PyPi, because otherwise people who install programs that easy_install beautifulsoup will crash due to the module's new name. Does that make any sense?

Anyhow, we're almost at the end of this fitfully travelled road. Once I figure out distribution and rewrite the documentation, a) no one should need to use BS3 anymore if they don't want to, and b) it should be possible to get lxml-like performance or html5lib-like flexibility with a Beautiful Soup API, by actually using lxml or html5lib as the underlying parser.

PS: remember, it's now from bs4 import BeautifulSoup.

(1) Sun Jan 22 2012 09:43 Beautiful Soup 4 Benchmark: This is going to go into the Beautiful Soup 4 documentation, but you might find it interesting. It's my first legitimate benchmark of BS4, and the first benchmark of this stuff I've seen since Ian Bicking's excellent 2008 benchmark.

Ezio Melotti pointed me to a list of the top 10,000 domains worldwide, according to some random source. It looked legit, so I wrote a script to download the homepages of the top 200 domains as served to a desktop web browser. My dataset included many pages written in Chinese, Japanese, Russian, Portuguese, Polish, and German.

For every parser I was interested in, I parsed each homepage and timed the parse. This gave me 200 numbers for every parser. To reduce that to a single non-huge number I calculated a mean: how many kilobytes of real-world HTML the parser could process in a second. I also noted each parser's success rate: how many of the 200 homepages it had handled without raising an exception.

Here are the results, ordered by their performance under Python 2.7.

	Python 2.7		Python 3.2
Parser	Speed (KB/s)	Success rate	Speed (KB/s)	Success rate
Beautiful Soup 3.2 (SGMLParser)	211	100%	-	-
html5lib (BS3 treebuilder)	253	99%	-	-
Beautiful Soup 4.0 + lxml	255	100%	2140	96%
html5lib (lxml treebuilder)	270	99%	-	-
Beautiful Soup 4.0 + html5lib	271	98%	-	-
Beautiful Soup 4.0 + HTMLParser	299	59%	1705	57%
html5lib (simpletree treebuilder)	332	100%	-	-
HTMLParser	5194	52%	3918	57%
lxml	17925	100%	14258	96%

Note that the "HTMLParser" tests don't actually produce anything. HTMLParser is an event-based parser, so when the HTML is parsed, nothing comes out because I didn't include any handler code. All the other tests build a parse tree in memory.

Another thing to keep in mind about the html5lib results: html5lib is kind of the opposite of BS4. BS4 always builds a tree of Beautiful Soup objects, but you can tell it to generate that tree using html5lib, lxml, or HTMLParser. Whereas html5lib always uses its own parser, but you can tell it to build a tree of lxml objects, a tree of BS3 objects, etc.

The big surprise for me is that on Python 2.7, lxml is the worst choice for a parser to drive BS4. It's a worse choice than html5lib! How did that happen? I have no idea. I was hoping to cash in on the lxml magic (see below), and it's not working. I need to look into this. Notice that html5lib takes a performance hit from using lxml's treebuilder. If the magic's not in the treebuilder and it's not in the parser, where is it?

Unless I can find that magic and exploit it, it remains the case that if you're paying by the minute for computer time, you should use lxml. It's written in C, and on Python 2.7 it builds a parse tree sixty times faster than BS4, three times faster than a pure-Python parser that does absolutely nothing with the data. Even on Python 3, lxml alone is seven times faster than BS4+lxml. I said stuff like this in the BS3 documentation, but I think I need to be more forceful about it in the BS4 docs.

The good news is that Beautiful Soup is 6-8 times faster on Python 3 than it is on Python 2, and even at its slowest, BS4 is noticeably faster than BS3.

The big caveat is that my definition of "success" is pretty minimal. Just because the parser parsed the file without crashing doesn't mean it will give you a useful parse tree.

Another caveat: on Python 3, I couldn't get HTMLParser to take raw bytes as input, so I ran the data through UnicodeDammit first. I counted this time as part of the parse time. This probably explains HTMLParser's slower speed on Python 3 and its higher success rate.

Update: Argh, I found out about this a year ago. The problem is that Unicode, Dammit is incredibly slow in some cases. Here are the results on 2.7 if I take out the prepare_markup methods in the builders for HTMLParser and lxml, and just assume everything's UTF-8:

	Python 2.7		Python 3.2
Parser	Speed (KB/s)	Success rate	Speed (KB/s)	Success rate
Beautiful Soup 4.0 + lxml	2287	96%	2600	96%
Beautiful Soup 4.0 + HTMLParser	2069	48%	1680	57%

That's more like it! The problem is that reliability suffers. Both parsers crash in the 4% of cases where it's not UTF-8 but the encoding is declared in a <meta> tag. And there's an unknown number of cases where the data's not UTF-8 but the conversion doesn't crash, leading to garbled data. But at least now I remember this problem.

Also note that on Python 3.2, getting rid of Unicode, Dammit doesn't matter nearly as much. (It doesn't matter for HTMLParser at all.) Presumably Python 3.2 has better built-in support for encoding autodetection.

Mon Jan 23 2012 11:43 To This Basic Game Hedgehogs Are Added: I bought a cute game about hedgehogs, Der Igelwettkampf ("The hedgehog contest"), as a Christmas present for my niece. On Der Igelwettkampf's BoardGameGeek page I noticed that it was classified under the game family "Animals: Hedgehogs/Porcupines". I'd thought "Family" was for boring things like grouping together the endless versions of Ticket to Ride, but turns out it's also used to group together all the games about hedgehogs.

The question then arises: what's the best game about hedgehogs? According to BGG it's Igel Ärgern + Tante Tarantel, a double bill in which Tante Tarantel might be doing some of that work because Igel Ärgern on its own is rated a bit lower.

More importantly, what's the worst hedgehog game? Indubitably it's Hedgehog's Revenge, "The GAME where the hedgehog STRIKES BACK!", whose BGG description includes the now-hopefully-immortal saying "To this basic game hedgehogs are added."

At this point I was on a roll... of the dice! I went back to my now-old BGG data dump, sorted the board game families by how many games they contained, and picked out interesting groupings for use in Loaded Dice. We've got Games about animals (most popular: dogs) Game versions of sports (soccer), and Games about countries (the Roman Empire, in a landslide). That page shows the top-rated game and the lowest-rated game, so get ready to load a lot of cover images.

I did a couple other lists, like media tie-ins (champion: Disney) and "families" that are strongly tied to one single game (the 889-strong "Monopoly" family), but I think the three lists I put up are the most interesting.

Bizarre trivia abounds! Did you know that crows are board game gold? The worst game about crows (The Crow and the Pitcher) has a BGG rating of 6.32, which isn't that bad at all. (Longtime fans will remember the median rating is 6.0).

Did you know there are twenty rodeo-themed games? Apparently you didn't, since only one of those games has more than five ratings. How many wargames take place in Switzerland, a country that doesn't fight wars? Only two: Switzerland must be Swallowed and Zürich 1799.

My data is six months old now and it's starting to show some cracks. There are BGG families for Russia and Antarctica which were created after I took my dataset, so they don't show up in the country list even though most of their games are in my data. After getting the Switzerland idea I ran the "What percentage of a country's games are wargames?" test on all countries, but wargames were drastically undercounted. For instance, all but one "Vietnam" game on BGG is a wargame (the exception being Venture Vietnam), but only 35% of those games were classified under a general "Wargames" category.

But, the lists are still a lot of fun and there are some interesting games in there. I'll leave you with the board game equivalent of the dusty World Book Encyclopedia sitting on the shelf at your grandparents' house: Trivial Pursuit - The Year in Review - Questions about 1992, the worst-rated game (3.90) in the 155-strong Trivial Pursuit family. Also available in 1993 flavor!

(6) Tue Jan 24 2012 09:06 CG Author Commentary #9: "Import System": Last week and this week have some of my favorite Twitter bits (e.g.) because the CDBOEGOACC is finally available in English. Sunday night while working on Loaded Dice I realized that one of the reasons I really like playing around with the BoardGameGeek dataset is it's like a real-life CDBOEGOACC.

The flip side is this chapter doesn't have a lot of plot. But hopefully you're okay with that because of all the fun mini-stories like the Sea Level game/food. It's supposed to represent the design phases of a software project, where you're throwing around a lot of ideas but not much is being produced.

Next week is a set piece, and after that the plot won't let up until the cliffhanger that ends Part One. Before that happens, I need to get some solid exoludology in to bring in topics that are important later, like Sayable Spice and Ariel's unsuccessful attempts to translate it.

Before beginning the chapter 9 commentary, I want to get something off my chest about the first sighting of the Farang in chapter 1. In that chapter, Ariel compares their antennacles to the oral tentacles of a "cerebrophage". In the second draft I just out and said "mind flayer". My writing group said I should change it because readers might not know what a mind flayer is. ("Did you mean: mind flower?") Taking their advice to heart, I changed the reference to a made-up reference that nobody will get. Well, at least we're all in the same boat now!

And here's chapter 9. Vent your egg sacs before reading this commentary:

• This chapter represents the absolute end of the abandoned first draft. Beyond this point everything is from the second or third draft.
• In a questionable move on my part, Ariel gets an Alien computer before he meets any Alien characters, requiring that I introduce you to the species with an infodump ("eight-foot monkey-lizards"). Don't worry, in just a couple weeks, Alien characters will show up and run off with the whole damn book.
• Speaking of infodumps, I want to do a little infodump of my own, about the Ip Shkoy. The Ip Shkoy were an ancient civilization of Aliens, much like the Roman Empire was an ancient civilization of humans. "Ip Shkoy" is not the native-language name for the Alien species. I tried to make this abundantly clear, but I've noticed well-intentioned people calling the Alien species "the Ip Shkoy" or ascribing to modern Aliens the (frequently pretty awful) opinions of the Ip Shkoy. Which would be like Curic thinking that Ariel regularly offers sacrifices to Jupiter Optimus Maximus.

  Star Trek has conditioned us to see an ET species as having a single homogenous culture that never changes, and this sort of confusion is why they do that on Star Trek. That said, I don't think this is anyone's fault but my own. If I'd presented modern Alien society in as much detail as I present the Ip Shkoy, the other probably wouldn't crowd out the one. It doesn't help that certain features are shared by both cultures, such as transitive pair bonding (aka polyamory).

• Recapture That Remarkable Taste, the Ip Shkoy remake of Sayable Spice, is not to be confused with the new William Gibson anthology, Distrust That Particular Flavor.
• Charlene Siph is mentioned again, which gives me a good excuse to talk about Alien names. The Aliens on the contact mission have all taken human first names, but their surnames are monosyllables which I usually generated by truncating creepy English words ("siphon", "somnolent") to four letters. The impression I want is of someone who's trying to be accommodating but doesn't quite have it down.

  This is another detail imported from "Vanilla", one that I'm really happy with, one that even becomes important to the plot in one place. And if you like symbolism, check this out: "Ariel Blum" could be an Alien name.

• In the second draft I wrote a whole review of Proty's Big Escape, but turns out it's a dumb idea to insert one game review wholesale into another game review. Let me know if you want the Proty review, and I'll make it the first CG Deleted Scene, even though it's only five paragraphs long.

Be sure to tune in next Tuesday, when Dana will say, "This application will terminate due to suspected theft or circumvention."

Oh, and you might want to keep an eye on @Tetsuo_Milk.

Image credits: Flickr user krusty, Guillaume Piolle, and Flickr user CoffeeGeek.

<- Last week | Next week ->

Sat Jan 28 2012 09:58 Fruit to Fruit: Time for another crummy.com Apples to Apples variant (previous editions), this one discovered last week by Pat.

On every green A2A card there's the name of the card, like "Handsome", but there are also three related words, like "attractive", "elegant", "fine". In Fruit to Fruit, you don't read the name of the card. You just read the related words. Sometimes the related words are so similar that you might as well be reading the name of the card, but usually something goes missing (such as the masculinity of "handsome"), leading to funnier red cards being put down. The name of the card is finally revealed during judging.

We had a great time with this and played it in conjunction with the Apples to Placebos variant, even though there were four players. You might think this overkill, but at this point A2A is more a social activity than a game. Anyway, it says right on the box "The game of hilarious comparisons!", so anything that makes the comparisons more hilarious is legit.

While seeing if anyone else had come up with this variant I discovered Apples to Trivial Pursuit, and the improv comedy variant. I also discovered that the game is patented, and that there is an entire patent classification system for "means... by which contests of skill or chance may be engaged in among two or more participants, where the result of such contests can be indicated according to definite rules."

(2) Tue Jan 31 2012 09:18 Constellation Games Author Commentary #10: "K.I.S.S.I.N.G.": This is Dana Light's big chapter, and I'm having trouble writing commentary because it's pretty self-contained. A problem is introduced and Ariel solves it by the application of technology. If I hadn't been writing a novel when I came up with Dana, this chapter would have become a short story, maybe part of a sequel to "Mallory". It would have been about the way evil psychologists use game mechanics and the ELIZA effect to manipulate users into spending money, and the way people get real pleasure from spending money on things designed to manipulate them.

Although evil psychology does show up in Constellation Games, I didn't have as much space for it as I'd like. Instead this chapter shows the first grown-up thing we see Ariel do. In a world in which sub-human-level AI has suddenly become very common, Ariel decides to empathize with it.

He doesn't anthropomorphize Dana. Dana doesn't pass the Turing test, she isn't terribly smart or self-aware, but she's capable of happiness and she doesn't deserve to be deliberately made unhappy by evil psychologists. This attitude is what ultimately makes Ariel a hero, not just a POV character. The consequences of his decision to empathize will run through the entire book, and then overflow the book into "Dana no Chousen," and I still don't know when and whether Ariel does the right thing w/r/t Dana. But you gotta have empathy.

Apart from that, I don't have much to say. Here are a few miscellaneous notes:

• As you might expect, a lot of this stuff will come up again in "Dana no Chousen". But the callback you probably won't notice unless I point it out is that Dana loves popcorn.
• I enjoy many bits of this chapter but my favorite is Bai's big moment of lucidity, when he immediately detects and shoots down Ariel's Manic Pixie Dream Girl fantasy. (And you can bet that's gonna come up again.)
• I'm sure that G'go Investigation: When You Gotta Die makes sense in cultural context. Like, imagine if the first thing you learned about 21st-century Earth was Mario Kart: Double Dash.
• I really like the design of the chainable memory cylinders on the Simulates Hi-Def False Daylight. In the second draft, "[False Daylight] games were distributed as a set of ROM chips, snapped onto standardized circuit boards, and enclosed in a removable plastic case to be plugged into the computer's game slot." This led to chips popping out, hiding in the carpet and stabbing people in the foot. That's a design in keeping with the generally poor quality of Ip Shkoy consumer goods, but it doesn't fit with the fact that the False Daylight is a clone of the Brain Embryo, so I switched to the much cooler chainable cylinders.
• Originally I transliterated Bai's "bro" as "bra". Everyone hated this. I changed it to "brah". The hate did not abate. What is wrong with you people? "Brah" is an accurate transliteration! It's so accurate.

Tune in next week for action, intrigue, and romance between people at the same level of sentience. It's the only chapter when Ariel will say: "I just have a slight fear of being a tiny speck in the infinite cosmic void." But not the only chapter when he'll think that.

PS: Due to an error on my part, the chapter 9 Twitter feeds ran as part of chapter 8, and chapter 10's Twitter feeds ran last week. This really can't go on, because next week's feeds are tightly integrated with chapter 11. So except for a brief bit of bonus material I just wrote, there will be no Twitter stuff this week. Sorry about that!

Photo credits: Kevin Trotman and Peter Anderson.

<- Last week | Next week ->