A Project for 2011: Beautiful Soup 4: I'm breaking my normal rule of not announcing projects until they're done, because I think it might help some people make plans if they know about this. In 2011 I'll be coming out with a major new version of Beautiful Soup that will work with Python 3, but that won't have the problems of the failed 3.1 branch.

The story so far: the most recent release of Beautiful Soup (3.2) uses a custom parser based on Python's standard-library SGMLParser. This was a really good parser back in 2005. Here in 2011, html5lib is better at handling bad markup, and lxml and ElementTree are much faster if the markup isn't too bad. Beautiful Soup's parser is no longer a competitive advantage.

What's more, SGMLParser goes away in Python 3, and its replacement is awful at handling bad markup. I tried to switch over in early 2009 and it just didn't work for anyone. So, Beautiful Soup has had the specter of death looming over it for two years.

Beautiful Soup 4 will not be a parser at all. It will be a tree-builder. You will plug a parser into Beautiful Soup, and you'll get an object tree that reflects how that parser sees a document. I have this working reasonably well for lxml and html5lib, which is why I'm comfortable announcing the project now.

Problems this will solve:

Problems this will not solve

I'll be spending alternate Fridays working on Beautiful Soup 4. I'll probably have a beta release in a few months. I'll be pushing my progress to this branch.

(nb. My code-in-progress includes some code from html5lib, and I'm not sure how the licenses interact, but since html5lib is MIT licensed I think it's just a matter of adding some more boilerplate, so I'm not too worried about it.)

[Comments] (2) : Hello, weblog. I am writing in you. I just finished "Four Kinds of Cargo", my first post-novel short story, so now I feel like I can write some other stuff. The story still needs to pass the Sumana "does this plot make basic sense" test, but I'm optimistic.

Beautiful Soup work has mostly focused on trying to get easy_install to work, working around a Launchpad bug--I'm still not totally sure if it's working. Last Friday I was in Dallas for work, so tomorrow will be a BS development day.

I seem to really like writing fiction about small businesses--"Mallory", "Awesome Dinosaurs", Constellation Games, and this new story all revolve around business partnerships. I'm just doing my part to boost the economy--did you know that stories about small businesses are responsible for most of this country's stories about job growth?

I'm still reading that space-astronomy book and I'm saving up more interesting bits to tell you. I'm also reading the first volume of Mark Twain's autobiography and it's awesome—he's just dictating random stories as they occur to him, and, like Groucho Marx, he's always in comedy mode. I don't think the printing of this autobiography will create many new famous Mark Twain quotes, if only because Mark Twain never said the existing "famous Mark Twain quotes". But this cubical advertisement for e-books is to me as America: the Book was to Sumana: every few minutes I'm disturbing her with some new weird kind of laugh.

Beautiful Soup 4 Status Report: The port is going pretty well. I've got almost all of the old test suite modernized and ported to the new framework, meaning that BS4 now works about as well as BS3.

There are a couple things I haven't figured out yet, the main one being the API for starting up a BeautifulSoup object with a given tree-builder. I'd like it to be as simple as passing a string like 'lxml' into the constructor, but I haven't thought through the details yet.

: A story I helped critique in writing group has been published: "Showoff" by L.K. Herndon. It's a story about an alien wereflamingo. If that doesn't sell you, you can't be sold.

[Comments] (1) : "Riddle me this, Batman! Can God make a stone so heavy, even He can't lift it? What about the problem of evil, Batman?"

: I've reached the end of the first volume of Mark Twain's autobiography, although I started at page 200 (this was recommended in the introduction). I had a pencil handy while I read the book, with which to mark up all the funniest and most interesting passages, and I thought I'd share a few with you.

Earlier I called this book "an advertisement for e-books", and although I was referring to its physical dimension I was more truthful than I knew, because more or less the second half of this book is scholarly notes. Even the notes are pretty interesting, but it would have been nice to have some kind of hyper-text mechanism to keep them at hand during the reading.

A few sample interesting notes:

The [Second Iowa] Regiment had been disgraced by general order for having failed to prevent vandals from stealing taxidermic specimens from McDowell College in St. Louis, which was being used as a prison.

In 1910 in "The Turning Point of My Life", Clemens recalled that Herndon told "an astonishing tale" about the "miraculous powers" of coca, instilling in him "a longing to ascend the Amazon" and "open up a trade in coca with all the world."

I'd like to buy the world some coke?

Ok, on with the fun. Quotes from the autobiography itself:

My parents removed to Missouri in the early thirties; I do not remember just when, for I was not born then, and cared nothing for such things. It was a long journey in those days, and must have been a rough and tiresome one. The home was made in the wee village of Florida, in Monroe County, and I was born there in 1835. The village contained a hundred people and I increased the population by 1 per cent. It is more than the best man in history ever did for any other town. It may not be modest in me to refer to this, but it is true. There is no record of a person doing as much—not even Shakspeare. But I did it for Florida, and it shows that I could have done it for any place—even London, I suppose.

On unconscious plagiarism: "all our phrasings are spiritualized shadows cast multitudinously from our readings."

"William Swinton was a brilliant creature, highly educated, accomplished. He was such a contrast to me that I did not know which of us most to admire, because both ends of a contrast are equally delightful to me."

"I conceived the idea of a magazine to be called The Back Number, and to contain nothing but ancient news; narratives culled from mouldy old newspapers and mouldy old books; narratives set down by eye-witnesses at the time that the episodes treated of happened."

The account of his duel in "About Dueling" is great. "I woke up Mr. Laird with some courtesies of the kind that were fashionable among newspaper editors in that region."

I also loved this section about James G. Blaine, the Continental liar from the state of Maine.

On election day we went to the polls and consummated our hellish design. At that time the voting was public. Any spectator could see how a man was voting—and straightaway this crime was known to the whole community.

I feel that this ties somehow into the "Horrid Tragedy In Private Life" saga:

Susy [Twain's daughter] and her nearest neighbor, Margaret Warner, often devised tragedies and played them in the schoolroom, with little Jean’s help—with closed doors—no admission to anybody. The chief characters were always a couple of queens, with a quarrel in stock—historical when possible, but a quarrel anyway, even if it had to be a work of the imagination. Jean always had one function—only one. She sat at a little table about a foot high and drafted death-warrants for these queens to sign. In the course of time, they completely wore out Elizabeth and Mary Queen of Scots—also all of Mrs. Clemens’s gowns that they could get hold of—for nothing charmed these monarchs like having four or five feet of gown dragging on the floor behind.

I think that's enough for now.

Disturbing Search Requests: (rot13ed to avoid giving anyone else the same idea.)

evpuneqfba gurfvf erfg jro freivpr

No, but thanks for playing!

Let's Check In With Roy's Postcards: "I think Matthew Barney made a movie with this plot." (Update: here's the front of that postcard, in case you wanted to see what it looked like.)

The Last Workshop on Theoretical Physics in the Soviet Union: Can Beatriz Gato-Rivera's paper "The Last Workshop on Theoretical Physics in the Soviet Union" live up to its awesome title? On the whole, I think it doesn't—relatively little of the narrative takes place at said workshop— but it's worth reading for the many good bits, some of which I'll extract for you:

I was allowed to keep the original key of the back door of Einstein’s house, that I saved in its way to the garbage truck... In the garden I noticed that the old door had been replaced but was still there lying on the wall with the key inside, so.....Back in Boston, Cumrun Vafa (my extra-official supervisor) was not amused when I showed him the Einstein’s key: 'Look, what you have done is, precisely, what Einstein didn’t want people to do!'
The last day of the workshop I was supposed to give a talk. Then someone told me: 'we are sorry, you cannot give your talk because the mathematicians have finished their workshop ahead of schedule and they have brought the blackboard along, we borrowed it from them'.
We took a night train and we were hosted by a female friend of his mother: the renowed mathematician Olga Ladyzhenskaya who was the leader of the 'Leningrad School of Partial Differential Equations'. When we arrived to her flat Olga received us with very low voice saying: 'Look, my niece is here in the living room. She works for the government and she is not allowed to have any contact with foreign people. So, please, rush through the living room, enter the corridor, and take the two rooms at the end'. So, we followed the instructions and ran stealthily while the niece was looking through the window giving us the back.
Natalia looked at me and said: 'You know, you are the second foreign person to enter in this flat. The first was Niels Bohr'. The surprise was enormous for me: I was the second after Niels Bohr in something!

Beautiful Soup 4 Status Report: Yesterday I ported some more tests and added basic doctype handling to the parser plugins. The work is slowing down a little because I'm porting tests where html5lib and lxml handle the same markup differently, such as incorrectly nested tables. I'm not going to find and test every such difference, but I want to have all the old tests working, and it'll give you an idea of what the differences are in common situations.

In BS3 you could choose to convert incoming entities into Unicode characters, or to leave them alone. You could also choose to convert Microsoft smart quotes into Unicode characters, XML entities, or HTML entities. In BS4 this will depend on the parser. Both lxml and html5lib convert everything to Unicode. I think this makes more sense--convert absolutely everything to Unicode, use Unicode internally, and optionally convert back to entities when writing the document out. (I'll probably have to write the "convert back to entities" part.)

[Comments] (1) : I felt listless today, so I did some Beautiful Soup work so that I wouldn't have wasted the day. I fixed the handling of CDATA sections and doctype declarations.

Fun fact: it doesn't seem to be legal to stick a CDATA section into an HTML document. (By this I mean something like "<![CDATA[foo]]>", not the contents of a <pre> tag.) My knowledge of weird HTML constructs like CDATA comes mostly from studying Python's SGMLParser, which handles CDATA sections just fine since it's an SGML parser. So I had BS3 just create objects for CDATA sections, even when they occurred in HTML documents. But the two parsers I'm using as my testbeds for BS4 basically ignore CDATA sections in HTML documents:

By default, lxml's elementtree implementation replaces CDATA sections with the actual character data, and has an option to leave the CDATA sections alone, but this only works for XML. When parsing HTML, CDATA sections are ignored altogether. The HTMLParser constructor has a "strip_cdata" argument, inherited from XMLParser, but setting it to False does nothing.

BS4 can't be used to parse XML yet (unless you want to parse it by HTML rules), but once I add that, I'll have the lxml elementtree builder preserve CDATA sections.

html5lib treats CDATA sections as broken comments, so "<![CDATA[foo]]>" becomes "<!-- [CDATA[foo]] -->" The latest version of html5lib will replace a CDATA section with the character data if the CDATA section happens within a <svg> or <math> tag (see test), but this is not in any released version.

That took me the morning to figure out, so I hope it saves someone some time. But that person would have to bear a suspicious resemblance to me.

Beautiful Soup 4 Status Report: What an exciting weblog I run. Actually this update is pretty cool. I've ported all the non-XML tests for BS4, which means you should now be able to use the code for all HTML processing purposes. If you want to try it, note that the module is now called 'beautifulsoup', not 'BeautifulSoup'. So from beautifulsoup import BeautifulSoup. I may rename it to bs4 just because I'm tired of typing "from BeautifulSoup import BeautifulSoup" for the past six years.

I also decided this would be a good time to run a performance test. Here's a moderately sized document:

Document is 66409 bytes
BS4 lxml time: 0.06
BS4 html5lib time: 0.25
BS3 time: 0.15

("BS3" here is the latest released version, 3.2.0.)

Pretty good! And here's a huge complicated document:

Document is 1329825 bytes
BS4 lxml time: 12.60
BS4 html5lib time: 2.88
BS3 time: 14.11

Okay, that's kind of random. The problem is in Unicode, Dammit. It takes a long time to figure out the encoding for this particular page. This is ultimately because the document is in ISO-8859-2, but it includes a <meta> tag that claims the document is in UTF-8. I don't yet understand the problem on any deeper level than that. If it's gonna be like that, I may just stop believing anything I see in a <meta> tag.

If you specify the encoding up front, the lxml time drops to 0.90 seconds. The html5lib tree builder doesn't have the problem because it uses html5lib's native Unicode conversion functionality instead of Unicode, Dammit.

Incidentally, BS3 has the same problem. Specify the encoding up front, and BS3 takes about 2 seconds on this page, which makes sense--faster than html5lib but slower than lxml. I find it very annoying that I'm only discovering this problem now—I think this has wasted a lot of cumulative time over the past couple years.

Anyway, now is a good time to start trying out BS4, if you're a fan of new things. I renamed all the major methods to be PEP-8 compliant--details are in the CHANGELOG.

Update: Profiler shows the bottleneck is in the chardet library, sbchardetprober.py, which goes through the file character by character and crunches some numbers for each character. If it can't make a decision until late in a huge file, there's your twelve seconds. So... I have a couple ideas, but it's not a bug in my code that I can just fix. But html5lib uses chardet, so it must be doable.

Black Planning: I'm doing more work on Beautiful Soup, but I'll spare you the details and share with you one of my favorite Ken Macleod quotes. This is from The Stone Canal, and it came up in brunchtime conversation with Evan:

"Because your Yank dingbat libertarian pals are right—the Western democracies are socialist! Big public sectors, big companies that plan production while officially everything's on the market... sort of black planning, like the East had a black market."

: Got XML parsing working in Beautiful Soup 4, and then added a feature I've been wanting to add for a while. Instead of separate BeautifulSoup and BeautifulStoneSoup classes[0], in BS4 there's just BeautifulSoup. To get a tree-builder that's optimized for XML, you write BeautifulSoup(markup, "xml"). HTML is the default, but if you want to make it explicit, you write BeautifulSoup(markup, "html").

But this is just the tip of a general feature. "html" and "xml" are just strings, features for which a tree-builder might or might not advertise support. The tree builders also publish other features, like "fast", "permissive", "html5", and library names like "lxml". So you can make semi-fine distinctions:

BeautifulSoup(markup, ["html", "fast"])
BeautifulSoup(markup, ["html", "permissive"])
BeautifulSoup(markup, ["html", "lxml"])

The BS constructor will try to find the best tree-builder that matches all the features you specify, and will raise an exception if it can't match them all (because you don't have lxml installed or something).

This is overkill right now because there are only three tree-builders (["lxml", "xml"], ["lxml", "html"], and ["html5lib"]). But this gives me an easy way to add tree-builders to the code base, and for you to plug in additional builders, without making end-users learn where the classes are.

This is looking good enough that I can do an alpha release soon. I'm not sure why I've been putting so much work into BS, but I'm sure it has something to do with the fact that my other projects are stalled, blocked, or I want to procrastinate on them.

[0] Those little classes like ICantBelieveItsBeautifulSoup are also gone, because distinguishing between different techniques for parsing markup is now the parser's job. And those classes were kind of silly to begin with.

Looking For Work: That's right. I'm on the market. I may just move to a different position within Canonical, but I'm taking the opportunity to talk to people at other companies. If you're reading this and want to hire/work with the person who brought the world Beautiful Soup and RESTful Web Services (the book, not the concept), send me email at leonardr@segfault.org.

Here's my resume. Ideally I'd like to do a combination of coding and writing, but tell me what you've got.

[Comments] (2) The Board Game Remix Kit: I imported the book of The Board Game Remix Kit from the UK. I probably should have bought the PDF instead because it's a really small book that, pound for pound, was rather expensive to import. But, what I didn't realize until I bought the book is that it's by Holly and Kevan! So, buy that sucker. It's a lot of rules for new games you can play with the pieces from other games that you're tired of. The one into which the most care was put is a game that turns Clue[do], the game with the most irrelevant pieces ever, into a tactical wargame where you fight zombies.

There's also (for instance) a game played with Monopoly title cards that's suspiciously like our Man Bites Dog game remix. The whole thing makes me glad Sumana accepted that Trivial Pursuit game from Beth when Beth was moving. ("Dadaist Pursuit: ...every other player turns over their top card and selects the funniest answer from those printed on it...")

[Comments] (1) Beautiful Soup 4.0.0a: I feel good enough about Beautiful Soup 4 to do an alpha release. It does not yet work with Python 3, but at this point I feel like the BS4 series is better than the BS3 series. BS4 will use html5lib if you've got it, or lxml if you've got that, or if all else fails it will use the built-in HTMLParser, with all its problems.

Try it out and file bugs. Check out my weblog archives for a glimpse of what's new in BS4.

[Comments] (1) By Pokemon Betrayed: Pat, Beth and I planned to go to the Nintendo store at Rockefeller Center to check out their museum of old hardware. It's a good thing we spent most of the day walking around doing other stuff (weird promotional peanut butter exhibition put on the by the peanut butter sandwich restaurant and brunch with Camille, farmers' market, retro game store), because when we got to the Nintendo store it had been taken over by the launch event for the latest Pokemon game. Huge line/crowd of people, waiting to buy the game and/or get a stamp for their Pokemon passport. Oh well. We'll go some other time. Hopefully not the weekend they launch the 3DS.

We went to the LEGO store instead, which was no more crowded than the rest of Rockefeller Center on a weekend. They have a Lego model of Rockefeller Center in the LEGO store, including a Lego model of the LEGO store itself, and you know that somewhere inside that model is a lego model of the Lego model of the LEGO store. Can't see it, though.

[Comments] (3) : While I was busy doing other stuff, the Roy's Postcards backlog ran out. Argh! At some point I'll slap another hundred postcards up, but right now I have to get ready for a trip so I don't have time to pay attention to your needs. <—situational irony

[Comments] (6) "API": Next Friday is my last day at Canonical. I've taken a 90-day contract at a startup, on which more later, and after that I'm not sure. I may join the startup as an employee, I may take a few months to revise RESTful Web Services, or... something else.

Apropos revising RESTful Web Services, I have a question for you. When I started talking with people as part of my job search, I noticed that nobody uses the term "web services" anymore. Everyone talks about "APIs".

I don't want to overstate the magnitude of the change here; people have been referring to web services as APIs for years. I did this myself until around the time RWS came out, and even today the Launchpad web service is officially called the "API". The change I noticed is just that the terms used to be used interchangeably, and now "web service" has pretty much died out. When I say "web service" people know what I'm talking about, but I feel like I'm speaking out of a phrasebook, or doggedly saying "free software" in an "open source" world.

This is one of those little nagging things that may or may not be important. I have an inkling that it's fairly important: that there's a reason people might have stopped saying "web service" around 2007, but that using "API" as a generic term leads to products that are worse than they could be, and that it's something I'd need to address in any second edition of RWS. But I would like to get a broader spectrum of opinion on this.

Has the term "web service" really died out in the past few years (or, as I suspect, changed its meaning)? When do you think this happened, and why?

Update: This post spawned an InfoQ article, so I should probably stop playing coy and say what I think.

I agree with commentor Foone. I think that WS-* successfully claimed the term "web service" in the mid-2000s, and that when people soured on WS-* they also stopped using the term "web service". While looking for work, the only companies I saw talking about their "web services" were big companies that were using SOAP/WSDL. Everyone who wanted to go above level 0 on the maturity heuristic was talking about "APIs".

The problem is, this terminology is backwards! SOAP/WSDL services are at level 0 just because they're not "web services". They don't use any of the web technologies. Their goal is to abstract away the network boundary in order to appear like a native client library. An API, if you will. As you adopt more of the web technologies, your product becomes less like a native client library ("API") and more like the web ("web service").

Thinking of all web services as APIs buys into the assumption that a web service is like a native client library. You can do a lot with that assumption, you can even get to level 3 of the maturity heuristic, but I don't think you can use that assumption to build complex hypermedia-driven applications of the kind seen in REST in Practice. Saying "API" is saying that SOAP/WSDL had the right idea, but the technologies were too heavyweight.

[Comments] (3) Some Call It The "Weekend": I'm unemployed! Until Monday.

: Roy's Postcards is back! Though possibly only for three weeks.

First Week: For the first time since 2005 I've been leaving the house every day to go work in an office. The office is Betaworks' big open-plan incubator office in the meatpacking district, holding a zillion startups, including Findings, the company I work for.

There are lots of cool features I want to work on, but I spent this week taking responsibility for the back end, adding useful things like unit tests. That's almost done, so next week I'll get to the really interesting part... which I can't talk about yet. (Another thing that hasn't happened since 2005.) But I think pretty much everyone who reads this weblog will be interested in this, for one reason or another.

My main complaint is the commute, which takes about an hour in each direction. I know, you've probably got it way worse, with your cars and your traffic jams. I'm starting to see the appeal of my friends' "do something after work in Manhattan, then go home and crash" lifestyles.

: Yesterday I suggested to Sumana that there should be a LEGO model of the International Space Station. I think we mainly wanted a LEGO Canadarm. It turns out there is an official ISS model, but reaction to it is mixed, so check out Augie Krater's indie version.

: Another bit from New Cosmic Horizons: Space Astronomy from the V2 to the Hubble Space Telescope, this time about the Hubble Space Telescope itself (née the Large Space Telescope):

[F]urther budgetary problems forced NASA to reduce the mirror diameter of the LST from 3.0 m to 2.4 m, saving an estimated $61m in 1975 dollars. Later in 1975 the Large Space Telescope's name was changed to Space Telescope, so as to avoid giving Congress and the tax payers the impression that NASA were being greedy in asking for anything 'large' during a time of financial stringency in the USA. Some astronomers were concerned that this change in name signaled that NASA were eventually going to cut its diameter even more to 1.8 m, which George Low, NASA's deputy administrator, assured them was not the case.

It did launch with the 2.4-meter mirror.

[Comments] (5) Solaris: We were talking to Zed Lopez and he mentioned there was some author one book of whose he's never read, because he doesn't want to live in a world where he's read all of this person's work. I don't think he meant it as dramatically as all that, but it brought to mind my own similar pledge about Stanislaw Lem. I've read all the available English translations of his work, except Solaris. I have Solaris, I just didn't want to live in a world in which I'd read all of Lem.

But that conversation with Zed made me realize that living in a world with no more Lem to read was much better than dying without having read all of Lem. And those are the only two options, barring a tricky race-condition maneuver. So over the past week I read Solaris on the subway. And now there's no more English Lem for me to read.

I don't think I'll ever write anything as good as Solaris but I'm pretty sure I can do better than the English translation of Solaris. Not really recommended; for pure mind-blowing Lem action it's gotta be His Master's Voice or Fiasco.

: While doing research for Roy's Postcards I found one of the upcoming postcards in a "vintage postcards" Flickr stream. This was a little depressing, but "vintage" just means "time-indexed" so really, just about any postcard is "vintage". Anyway, check out some old-ass postcards.

Findings: Week Two: We've now got good test coverage for all the server-side code, and as a side effect of writing those tests I've now gone through everything, I understand the code, and I've refactored the hell out of it. Next week I start on new features. The other big triumph of last week was getting a written-down list of the features we absolutely must have to launch a public beta.

These status updates are important for me to write but probably useless to you. I get the feeling we're playing our cards pretty close to our chest right now, but next week I'll also try to get some guidelines from Corey about what he's OK with me talking about.

Nothing In Biology Makes Sense Except In The Light Of Evolution:

Maggie is always asking "Do horses lay eggs?" or saying "Bears lay eggs!" etc. We finally made a chart to help her remember the different types of animals and which ones lay eggs.

Best activity ever!

: Adam is on fire with his poem-a-day April experiments. See his ZZT poems and The Average Unicode Character.

[Comments] (2) Findings: Week Three: This week we had a lot of meetings with people from other companies and I think I can talk about what Findings is doing in a way that's interesting without giving away all the secrets we're saving for the product launch.

When I read a good book, I encounter lots of interesting bits that really stick in my mind, for about five minutes. Once I finish the book I'm left with just a general impression of what it said, unless I engage with it while reading by taking notes and highlighting passages. But then I give the book away, so I never see the notes and highlights again. Or the book goes on my shelf, where I probably won't look at it again unless I need to look something up.

Since the 1990s, when I first read about electronic ink coming out of the MIT Media Lab, I've fantasized not just about having access to a lot of books in a small space, but of having random access to my distilled readings of all the books I've read. This would make me (functionally) much smarter and a better writer.

Fifteen years later, electronic-paper hardware is now good enough that I want to use it to read books.[0] But I'm mainly interested in the software. I don't like being done with a book after I read it. I want to pull that extratextual layer out of the book as data. At Findings we're writing software to manage this extratextual layer. Once you extract your reading of a book as data, a lot of interesting social interactions become possible around that data, so we're also writing software to catalyze those interactions.

Yeah, it's vague, but hopefully you get the idea.

[0] That's a pretty low bar, though--I never liked the original "book" hardware all that much. If ZaReason started putting out an electronic-paper book reader for which I could write software, I would buy ten.

[Comments] (2) : Yesterday was Sumana's and my fifth wedding anniversary. We've been married about half the time we've known each other! Pretty amazing. To celebrate (not to celebrate), Sumana did standup at a local bookstore yesterday and was great. If you're in New York you can see her on Saturday in Brooklyn.

I told Sumana: "The fifth anniversary is the wooden anniversary, so I got you Minecraft." She got this joke, but I don't think she appreciated it, so I'm throwing it out there for you.

[Comments] (3) Dada Dwarf Dozens: Yesterday I was a guest critic for Adam's "Reading and Writing Electronic Text" class. His students are preparing their final projects (to be performed on May 6!), and the sight was inspiring. It made me want to bring back the randomness that has kind of been missing from my life recently. In a similar vein, Adam mentioned that his poem-a-day project came from seeing his students being really creative while he wasn't really doing much.

And so, Adam and I have decided to bring our generative creativity to bear even after the end of National Poetry Month, with a feature I like to call... whatever we decide to call it when we think of a name. The idea is that we'll trade off posting poems and other textual projects, each trying to outdo the other. It's kind of like Layer Tennis, except we won't be directly riffing on each others' work, just trying to put out the best texts. But hopefully my work will provide inspiration for Adam and vice versa.

Anyway, here's my first entry, "Dada Dwarf Dozens". I took the poignant internal monologues from Dwarf Fortress and turned them into incongruous "your mama" jokes.

"Price Fixing"
Your mama's so conceited, she has suffered the pain of having to give somebody water lately.
She's so lazy, she has complained of the lack of dining tables lately.
She's so fat, she has altered the prices of goods.

"Organization Is For Chumps"
Your mama's so dumb, she became a parent.
She's so lazy, she was overjoyed to be able to help somebody to bed lately.
She's so conceited, she tries to live a well-organized life.

"Backhanded Compliments"
Your mama's so stupid, she is interested only in facts and the real world.
She's so fat, she is easily moved to pity.
She's so dumb, she conducted a meeting in a good setting recently.

"No Shoes"
Your mama's so lazy, she was embarrassed to have no shoes lately.
She's so stupid, she loves new and fresh ideas.
She's so fat, she ate a legendary meal lately.

"Production Shortfall"
Your mama's so stupid, she was upset to have disappointed a noble lately.
She's so lazy, she was unable to request weapon production lately.
She's so conceited, she is open-minded to new ideas.

"Something Unpleasant"
Your mama's so ugly, she has been tired lately.
She's so stupid, she has mandated the construction of certain goods.
She's so ugly, she saw something unpleasant in a pond recently.

"It May Actually Be Stress"
Your mama's so stupid, she was forced to eat a beloved creature to survive lately.
She's so conceited, she is impervious to the effects of stress.
She's so lazy, she is quick to wink at others.

I got the values from the Dwarf Fortress binary using strings, but I should have just scraped that wiki page. Live and learn!

[Comments] (1) Onward To Cuteness: Recently a video made the rounds of a penguin being tickled and making a cute noise, and it reminded me of the time in 2004 when Sumana and I visited the San Francisco zoo and witnessed a penguin making a cute noise when tickled. An even cuter noise, in fact, than the penguin in that video. More of a chuckling noise, I'd say. I bring this up this not to brag, but to tell the world that even cuter penguin videos are possible. However I don't suggest that we harass penguins like paparazzi and demand that they make cute noises; just let it happen, man.

Too Soon?: There are asteroids named after dinosaurs. Even worse: there's one called Brontosaurus.

Phrases You Don't Hear Outside Of Writing Group #2: "We don't know which squid god is responsible."

(First in the series.)

Dinosaur Graffiti: Seen when walking around Astoria.

I can only hope that some future Werner Herzog puts this in his movie.

[Comments] (2) Findings, Week k-(k+n): This feature went by the wayside, along with NYCB posting in general, because I get on the subway, and go to work, and get on the subway to go back home, and then I'm tired. Just like a real job!

Suffice to say that I'm gradually switching from implementing new features to improving the code that's already written, and we'll launch a semi-public beta in the next couple weeks.

A couple miscellaneous things learned from the week I spent writing web service clients:

[Comments] (2) : Lots of stuff I can't write about right now, but there's been an abundance of good news lately, which will surface in time. Also good: Dave Grohl's heavy metal tribute album "Probot" (and the new Foo Fighters album, and apparently any album Dave Grohl has ever been involved in in any capacity). The Battlestar Galactica board game (though it's way more complicated than it needs to be). Ice cream.

: Findings is hiring! Including, most likely, hiring a replacement for me. I'm staying at the company at least into July, beyond the time of my initial contract, but some other projects of mine are taking off and I need to start spending serious time on them. We're hiring a senior backend Python developer (ie. a replacement for me), a junior backend Python developer, and a senior-ish frontend web developer.

The site we're building is really amazing, and you know that I don't go around saying such things randomly. Findings is currently in private beta, but you can click around the site enough to get the general idea of what we've built, and you can ask me questions if you're interested. The main thing holding us back is that our team is tiny. We don't have enough person-hours to get the features and usability we want in a reasonable timeframe.

Here's the catch: despite the fact that we live in a networked world, and you may be reading this post from your reclaimed-scrap-metal treehouse in Greenland, to take this job you do need to be within commuting distance of Manhattan. The commute is the only part of this job I hate, but maybe you don't mind it, or maybe you're already commuting so it's a wash. If you're interested, send me mail at work: leonard@findings.com. The rest of you, sign up for the beta.

: The hazards of untyped data.

Update: A similar problem.

Update 2: And sometimes people cheat.

Mashteroids: Adam never responded to my Dada Dwarf Dozens, so I'm unilaterally posting another generative-text project to shame him. This project came out of my discovery of asteroids named after dinosaurs, and the quick follow-up discovery that a) if you name an asteroid you also get to write a little description of the person or place the asteroid is named after, and b) although you can write pretty much anything you want in that description, all the descriptions use the same rhetorical style.

There are many ways to have an asteroid named after you. You can be a colleague and/or friend of the discoverer (10965 van Leverink, 9945 Karinaxavier), a leader in some non-science field (8749 Beatles), or just a really big ravine (8933 Kurobe). But in all cases the description of your asteroid will be a few short sentences explaining why you are cool.

These sentences flow into each other so well, I decided to mash up some asteroids, creating multi-talented people whose accomplishments span the entire space of "things that might cause someone to name an asteroid after you". Here are a few of the new asteroids:

33017 Anomalocaris-Benkhoff-Malpighi

Anomalocaris, a large carnivorous arthropod, was one of the many uniquely shaped multicellular creatures that appeared during the Cambrian explosion. He was instrumental in organizing the ACM 2002 conference. A pioneer in the use of the microscope in anatomy, he made fundamental studies of the lungs.

29873 Fröbel-Carpino-Rebentrost

The Thuringian pedagogue Friedrich Fröbel (1782-1852) was committed to the education of young children. From his initial studies of satellite geodesy, he acquired a taste for extreme accuracy in orbit determination, applying these skills to the study of the dynamics of solar-system bodies in projects such as LONGSTOP and SPACEGUARD. As a reward for curing elector Johann Georg II, he was invited to pick some plants from the elector's garden.

22752 Johnreid-Novalis

Geologist John Barlow Reid (b. 1940), a teacher for 30 years at Hampshire College, Massachusetts, has studied the earth and moon using isotopic methods. In one of his novels he uses the image of a blue flower for the magical task poets have.

54263 Lyubimets-Vanessa-Mae-Inoutadataka-Racquetball-Stolte

Lyubimets, the Russian word for `darling', seems to be an appropriate name for Grigorij (b. 2000), grandson of the Crimean astronomer Lyudmila Karachkina. Her debut album in 1995 sold two million copies. He began surveying in Hokkaido at his own expense. Today the sport is played on a four-wall court by two to four players with a short racquet and a small rubber ball. A professor of media research, Stolte initiated international TV channels (ARTE, 3SAT) in a European cooperation.

30198 Chanwainam-Nicolewen-Tugendhat

Chan Wainam (b. 1919), who lives in Hong Kong, has devoted his energies to education and charity in China. Wen (b. 1989) is a finalist in the 2002 Discovery Channel Young Scientist Challenge (DCYSC), a middle school science competition, for her botany and zoology project. It was built by Mies van der Rohe in Brno in 1930 as a family villa of great elegance, following the idea of freely floating space.

36994 Vischer-Protea-D´Haveloose-Engström

Named for Peter Vischer the Elder (c.1460-1529), the greatest German brass-caster master in the Middle Ages. They are native to the Southern Hemisphere, where they grow wild. He was magnificient in his efforts to relieve suffering, but his own neck injuries finally made him unable to operate any more. Probably Sweden's best painter of caricatures, he is best known for his black and white drawings illustrating very short stories.

Mashteroids Addendum: Check out the (actual, not a mashup) description for asteroid 10355 Kojiroharada:

Kojiro Harada (b. 1926), mechanical engineer, is a member of Kawasaki Astronomical Association. A long-time observer of double stars, he has also written many fairy tales on stars, dramatizing and performing in them himself.

This guy sounds awesome! But I can't find any other English-language information about him. Who can help?

: The first Blamimation.

[Comments] (2) July: Month of Kickstarter: Back in the late 90s I had a running joke on this website that July was "Leonard Richardson Month", a month of festivals and educational lectures about me and my accomplishments, which (this is still part of the joke, not my 2011-era commentary) nobody attended because the whole idea was egotistical and stupid. Even with something as corny as that, though, I discovered that thinking of the month of my birthday as being special actually did make it a better month.

This year I'm bringing back the "special month" idea, but instead of an obsessive focus on me, the focus will be on other people and the cool things they're doing. Every day this month I will pledge money towards a Kickstarter project and post about it here.

I'll try to pick projects you'll find interesting because I really like the Kickstarter model and my goal is to get you into the idea of funding things that way as well. If nothing really grabs me on a given day, I'll make an investment more or less at random. Because it's July, mamajama, and weird things happen on this site in July.

Today's project is a local one for me: I pledged $25 to cover the production rental costs for "An Economic Cycle Through An Artist’s Perspective", a dance performance to be held in the Socrates Sculpture Park (where I went just today to check out the new farmer's market).

This may seem an odd choice for me. You don't have to know me very well to guess that I am not into dance. I don't know enough about the medium to appreciate it, although I do find the description of this performance ("each segment will represent a milestone that one would experience throughout the development of an idea") a lot more interesting than the typical dance performance description. But, I am really into living in a neighborhood and a city where people who are into dance go around giving and attending dance performances, because that means other people have fun and it enriches the city as a whole. So, this sucker is backed.

Tune in tomorrow for more action! This will also get me back to posting every day, which is something I've really been missing.

Month of Kickstarter: The Backfilling: My calendar informs me that I should have started this project yesterday. No problem! I've brought Month of Kickstarter up to date by also backing the Kikori Open Source CNC Gantry Router, which pushes my buttons with sentences like "It’s essentially a robot capable of milling complex three-dimensional shapes out of wood, soft metals, and plastics." It's also self-replication-capable!

Month of Kickstarter: Thousand Island: Today's project is "Uncovering the Mysterious Origin of 1000 Island Dressing", a documentary that tries to resolve the dispute about whose idea it was to mix ketchup and mayonnaise and call it salad dressing. I backed it because it reminds me of the "Compression" episode of the BBS Documentary. Ancient disputes may seem trivial now, but picking at them blows the lid off the septic tank we call the past.

I don't have a good feel for which Kickstarter projects are likely to succeed, but I gotta say that because of the amount of money they're trying to raise ($23,500), this one seems like a long shot. I don't see how it works unless a few backers decide to theme a vacation around the high-value pledge rewards.

Now, one of the nice things about Kickstarter is that you can back a long shot without worrying about losing your money. But I would like to spend a decent amount of money during the Month of Kickstarter, which means backing successful projects. And I'd like to do it without funding a bunch of projects that are clearly going to be funded anyway. I may end up compromising by diversifying my portfolio.

[Comments] (1) Month of Kickstarter: Patriotism Edition: When I was a kid I heard someone say "America has brought the world two great art forms: jazz and comics." I've long thought it's time to add "video games" to that list, and on today's TRIPLE EDITION of Month of Kickstarter I salute all three of these great art forms.

Jazz: "Recording original songs in the spirit of a 1940's Big Band". Check out that period dress! This is a man I trust to do a historical recreation.

Comics: Narbonic: The Perfect Collection. This project already reached its goal, so it feels kind of cheap, but I'm a huge Narbonic fan.

Video games: "The Videogame History Museum", "dedicated to preserving, archiving, and documenting the history of the videogame industry." These people have a lot of stuff and they want a place to put it. What could be more American?

Damn, that's patriotic. Even Sam the American Eagle can't say no.

[Comments] (1) Month of Kickstarter: A Young Lady's Guide to Treachery and Military Operations: I've got a lot to do today (all the Findings work that explains why I haven't posted much, plus bonus other work), so this one is quick: Lyssan, a board game that "combines tightly designed euro-style rules with the interactivity and flavor of an epic wargame." It looks fun and well thought out, so BACKED. The $50 price point to get a copy of the game is higher than the $25 I'm typically dropping on Month of Kickstarter projects, but what the heck. Some of those projects won't pan out, and it looks like this one will.

Month of Kickstarter: Decisions, Decisions: I'm seriously on the fence about "Strange Attractors: Investigations in Non-Humanoid Extraterrestrial Sexualities". When I saw the description I thought: BACKED. But, I was picturing a monograph with an underlying thesis and a consistent medical-diagram-ish art style, and this is an anthology with often cartoony art and big ol' artist statements (and humanoid extraterrestrials, judging from the sample pages!). Plus it's another project where I'd need to shell out $50 to get the book. So I dunno. It's definitely on my short list though.

This sculpture also looks cool, but Month of Kickstarter has a strict "no Burning Man" policy.

And so on. A couple more projects today went on my list for a future slow day or multi-project bonanza day, but today's backed project is a project to reissue John Isaacson's Do-It-Yourself Screenprinting. "Whether you want to turn your bedroom into a t-shirt factory or just make a couple stickers or posters for your band's next tour, Isaacson has got your back."

Month of Kickstarter: Think Positive: Today's project: "Bursts of Light: an Anthology of Positive Speculative Fiction". Because I have a vested interest in making Kickstarter a good place to raise money for an anthology you're putting out.

[Comments] (2) Month of Kickstarter: Tess Tess Tess Tessellations: "Cookie Connections__Clever Cookie Cutters Cut Clever Cookies". You, sir, are a mouthful! As are the cookies, one hopes. For you see, this project is tessellation cookie cutters! It took me a few minutes of staring at the 3D models to see all the shapes, but yes, there's four stereotypical Christmas cookie shapes coming out of a single cutter, with no waste. (Halloween-themed shapes also being produced.) For reasons I hope are obvious (TESSELLATIONS) I am super excited about this one.

Month of Kickstarter: Birthday Backlog Blowout: It's time for another TRIPLE EDITION of Month of Kickstarter. On my birthday I'm backing three projects specifically recommended by friends.

  1. My cousin Camilla recommended "WWJD", a film of a play, being produced by a friend of hers.
    WWJD is a feature film following a college student named Tom and his three roommates over the course of one memorable day when they receive a visit from an unexpected houseguest--Jesus. He washes their dishes and goes skateboarding and miniature golfing with them--but for some reason, Tom, the only believer in the house, can't see him.

    In other nepotism news, Camilla has an Etsy shop where she sells jewelry and reproduction medieval clothing. Check it out!

  2. Pat said I should back "The Power of the Crystals", "NYC's premiere motivational seminar/rock opera". Sumana, who I just showed it to, agrees! So... I backed the project.
  3. Also from Pat, it's "Raiding Parties", "a card game that takes place in the Golden Age." The Golden Age of what, you ask? The answer is PIRATES.

Non-bonus bonus!: Here are a couple projects I backed prior to beginning Month of Kickstarter, which I'm worried aren't going to make it, so I'm publicising them here.

  1. "Chocolate! Handmade from bean to bar", which only has about a day to go.
    We are a small bean to bar chocolate making company that believes chocolate making is not only a craft, but also an art. We plan to draw from a pool of artists on a rotating basis to design custom packaging featuring their artwork. In addition, we will collaborate with artists who will create unique molds that will allow us to make one of a kind edible sculptures.

    We're talking chocolate Space Invaders here.

  2. "Project Gado: An Open-Source Photo Digitization Robot", which scans old photos quickly without damaging them.
    The Gado 1 is in operation at the archives of the Baltimore Afro American Newspaper, where it has already scanned nearly 1000 historical images. The paper has a collection of over 1.5 million images spanning 115 years, though, so there's still a long way to go!

Month of Kickstarter: Recreating Fake Things: You know I love it. "50's Monster Movie Serial!" "[W]hat's it about? Rubber monster suit, women in 1950's bikini's, and the mother of all cliff-hangers."

Ultra bonus: Evan has started his own Month of Kickstarter!

Month of Kickstarter: Abandoned: Today it's "Documenting the Midwest Through Analog Photography". Specifically the parts of the Midwest that are abandoned buildings.

Each building is an individual battle within a war that most people fail to see. When I see a derelict building, I see a war that hardly anyone else can see in the same way. I want to preserve these images of this war between Man and Nature that, though slow, remains in constant flux.

I see a lot of projects where it doesn't look like you can fund the entire project through Kickstarter. Unless you have a preexisting fan base or are very good at marketing through non-Kickstarter media, it's tough to raise the ten or twenty thousand dollars you need to record an album or make a short film. But you can probably get ten percent of that to edit the film once it's done, or press the CDs once the album's recorded. This is kind of disappointing and getting more people investing in Kickstarter projects won't solve the problem, since that will also cause more people to post projects on Kickstarter.

Anyway, the point of this digression is that this photography project only needs $800, which I think is very doable.

Month of Kickstarter: Cloud Computing: Time is running out for "OpenPhoto, a photo service for your S3 or Dropbox account", so I gave it the ol' BACKED treatment. It's "a way to share your photos with others while retaining complete ownership of them."

Month of Kickstarter: The Astoria Project: First off, Sumana sent me a link to Kitt Hodsden's "Why I'll pledge to your Kickstarter project", which I basically agree with. I cannot stress enough the importance of having a realistic funding goal! You may only be able to raise enough money for part of your project! It's called Kickstarter, not Completethewholeprojecter.

Now it's time to indulge in another high-stakes metagame of backing a board-game project for $50 because my usual $25 MoK donation won't get me a copy of the game. Today it's "The Manhattan Project - Board Game", "a low-luck, mostly open information efficiency game in which players compete to build and operate the most effective atomic bomb program." It could be a double feature with Twilight Struggle, or Power Grid, or I'll never get enough people in one place long enough to play this game plus any other game so why bother.

Does it take me into "You may end your turn early!" territory to find this really interesting?:

The game features worker placement with a twist; There are no rounds and no end-of-round administration. Players retrieve their workers when they choose to or are forced to (by running out).

If so, I've come full circle.

Wednesday Bonus! "Bring a great computer history zine back to a new audience!" It's a project to re-release a historical computing zine that is, by this time, itself historical. You know you want to BACK it.

Month of Kickstarter: Wind: "Open Source Vertical Wind Turbine"

The intended outcome of this project is to develop a small wind turbine that can be built for cheap, and that can be used with more wind turbines to marginally increase electricity output. Some of the parts in this project are still conceptual and have not been built before.

Is that good or bad? Either way, it's a 3D-printed wind turbine, which is hard for me to pass up.

Month of Kickstarter: Foundry Objects: I took most of the week off work because Rachel was visiting, but now she's back in England and here I am helping "Double the size of the Columbus Idea Foundry!" before heading off to work.

On August 15th, we will have the opportunity to expand into our neighboring industrial unit (with which we already share several interior doors... an architectural match made in heaven!) By doing so, we will be able to split our workshop into a "dirty" space (our current location) with welding, woodworking, metal casting, metal fabrication, CNC machining, and other dust- and chip-making processes; and a "clean" space (the new wing) with, among other planned resources, a photographic dark room, a videography stage, a textile/upholstery station, a proper lithography room for etching/engraving circuits, metal, and screenprinting masks, and about a dozen more studio rental spaces for clean arts.

Sounds fun!

Bonus fact: according to this same Kickstarter entry, Columbus, Ohio is "the Indie Arts Capital of the World, as declared by the Columbus City Council..." [ellipsis in original]

Month of Kickstarter: Sun Text: Month of Kickstarter is halfway over! I've backed 19 projects (21 by the time this entry is done) and it's time to check back in on their progress.

So I'm looking at about a 1/3 success rate, which is pretty good given that I've avoided big, obviously-gonna-succeed projects like M.C. Frontalot's music video.

OK, with that navel-gazing out of the way, let's check out today's Projects of Month of Kickstarter! First we have "Open Source Programmer's Text Editor using Canvas and WebGL".

This will be a fully featured text editor implemented using the canvas element in HTML5. It will support WebGL for graphics acceleration for visually pleasing but uncluttered scrolling, anti-aliasing and other effects. This is designed not for the wow factor but to minimize visual irritation when working with text for long periods of time.

Second, it's SCI-ARC/Caltech's entry in the Solar Decathlon, "a U.S. Department of Energy Sponsored competition which challenges 20 architecture and engineering schools from around the world to design and build a solar-powered, 'net-zero' house." Bonus: the house looks like your robot pet from the future.

Month of Kickstarter: Milk Asteroids: Rock Hunter. Studly, closeted film star? No, it's a first-person Asteroids-type game to be made available under the AGPL.

Several weapon types are planned, and rocks will be dynamically simulated so that they can be cut apart, fragmented, and otherwise destroyed in unique ways every time.

In the second half of Month of Kickstarter I'm experimenting with backing two projects a day, since the success rate seems to be less than 50%. Today's second project is Milk Not Jails.

Rural New York is home to 90% of the state’s prisons, which provide jobs in a depressed rural economy. Meanwhile the majority of people in prison come from New York City’s communities of color and their families are forced to make long trips to visit them. The guards union and their elected officials oppose major reforms to the prison system because they fear it will destroy jobs in their community. As a result, New York’s prison system is racist, ineffective, and too expensive. This is not going to change unless we can develop a new economic relationship between urban and rural areas. MILK NOT JAILS looks to the state’s dairy industry – which comprises 80% of New York’s agricultural sales – for a delicious solution to this conundrum.

Month of Kickstarter: In Praise of Turtles: Dipping into my big list of bookmarked projects, since none of the new projects I saw this morning were as cool as stuff I'd earlier saved for later, later being now.

First off, it's "In Praise of Small Things: A Letterpress Adventure", a modest project to commemorate "the small, everyday 'unfancy' things and moments that we so often forget to pay attention to, but when we do, they glimmer."

And on the expensive side of the spectrum, there's Turtle Derby Documentary, raising funds for post-production.

Every July 4th for the past 50 years our small hometown in Pennsylvania has held an annual turtle race – a fun event that sounds like something from a Mark Twain story. But this year the state outlawed collecting turtles from the wild because they’re becoming endangered. It’s a fun and funny opportunity to see what happens when tradition and environmental issues collide.

I think I know which Mark Twain story you're talking about!

: Sorry to interrupt Month of Kickstarter with non-Kickstarter stuff, but I just discovered that Ian McMillan, who earlier recorded a reading of "Daisy" from Thoughtcrime Experiments, also recorded a reading of the first story in the anthology, "Welcome to the Federation". Way to go, Ian!

Month of Kickstarter: Heartbreak & Entrepreneurship: Really good crop today, and that means my day is off to a good start. First, "Heartbreak & Heroines RPG":

Heartbreak & Heroines is a fantasy roleplaying game about adventurous women who go and have awesome adventures -- saving the world, falling in love, building community, defeating evil. It's a game about relationships and romance, about fairy tales and feminism.

You play a fantasy heroine (or hero, if you prefer) whose heart has been broken. She's experienced some loss so great that she's taken up her sword, her tome, her staff, or her wand and walked away from her place in society -- by becoming one of its defenders, fighting back the darkness that endangers everyone.

When a woman picks up her tome, you know it's serious. The game system is based on one of the author's previous RPGs, Wandering Monsters High School (free download).

Today's second project is Entrepreneurship Exposed:

What does it take to start a company? Volumes of books exist that explore the workings of entrepreneurship, but no one has created a documentary that follows the entire creative process of conceiving an idea and developing it into a product.

Meta bonus: the product is a product about entrepreneurship.

Month of Kickstarter: Kombucha & Dragons: Here's one for Sumana: Kombucha Party. At least I intended it to be for Sumana. Only after I backed this project did I realize that "kombucha" isn't the name of the tea Sumana likes; that's hojicha. Kombucha is the tea-based drink they sell at Whole Foods. Fortunately, I signed up to get loose-leaf tea, not liters of kombucha delivered to my door, so it should work out.

Fortunately, Dragon Valley won't turn out to be something totally different from what I think it is: a line of organic dairy products made from dragon's milk. It's a board game, you say? Damn! Well, I like board games too. Dragon's Valley has some interesting features, like bringing in mechanics from cooperative board games (pulling useful things out of a pile and sharing them) without being a cooperative game.

Has someone made a bootleg RSS feed of new Kickstarter projects? I find it hard to believe that there's no official one, but I find it even harder to locate an official one. Going to the "recently launched" page every day is fine for Month of Kickstarter, but I don't want to keep doing that, and I don't see any other way to look at all the projects.

Big non-MoK announcement today!

Month of Kickstarter: Hot Love: The big non-MoK announcement has been delayed, so in the meantime enjoy some more Kickstarter projects. First, it's "Sizzle Sauce - The Savory Hot Sauce!". Longtime readers will know that I'm a sucker for a sauce that "doesn't sacrifice flavor for heat." It is a tomato-based hot sauce, which I'm not generally a huge fan of, but I'm keeping an open mind.

Second, it's "We Promote Knowledge & Love - Parade Day in Harlem":

We Promote Knowledge & Love is a social practice, community art performance project that borrows the aggressive street advertising tactics of pawnbrokers in urban communities as a vehicle to promote knowledge, self-empowerment, and love rather than commerce or monetary wealth.

'Nuff said.

Month of Kickstarter: Oboe Logo: Jennifer Ownby's "Oboe Chamber Music Recital" knows how to get me where I live: by appealing to the downtrodden, living-on-the-fringes-of-society lot of oboe players. I played the oboe in middle school band, because I was determined to announce to the world that I was a misfit, and I still have a soft spot for the instrument, especially when it's played by someone better at it than me. Anyway, enough about me, how about we let a real oboist speak:

My program consists of some lesser-known and underperformed, but still very good, solo and chamber works for oboe and various instruments. I'm starting off with Gilles Silvestrini's Six Études pour Hautbois, which is a lovely piece based on six different Impressionist paintings. I'm also doing a fun piece for English horn and Actress by Christopher Berg, called Why Else Do You Have an English Horn? The program will also include the Prokofiev Quintet, Op. 39, for violin, viola, bass, oboe, and clarinet. I've seriously wanted to perform this for about 10 years but haven't known the personnel willing to do it. Then I'll be finishing up with Omaggio a Bellini by Antonino Pasculli, for English horn and Harp.

Today's second project is "The Let's Make a Bunch of (Company/Product) Logos Project".

As a designer, I love making logos. So, I thought it would be fun, and challenging, to do a project with the straightforward concept of creating a whole bunch of different logos for a whole bunch of different companies and products. That's it.

That's it.

Month of Kickstarter: Cheese Plant: It's too hot to write interesting descriptions, but Month of Kickstarter chugs along. First it's "Cheese Vat Community Support Agriculture".

We are working to acquire a new cheese vat to replace the old "gas guzzler" with one that is energy efficient, and that will help us to make even better cheese, to ensure the continued success of a small local, sustainable agricultural, sixth generation, grass based farm.

Unfortunately, my idea that they should allow high-value Kickstarter donors to take surreptitious dips in the cheese vat was rejected.

Next it's the Field Guide to Phytoremediation:

In 2010, youarethecity created the Field Guide to Phytoremediation, a DIY handbook to cleaning up toxic soils in your own backyard, neighborhood vacant lot, or other urban space. Working with soil scientists, urban farming activists, community groups, and others interested on (and in) the ground, we have expanded this research.

Month of Kickstarter: Music!: Not in a mood to do clever writeups, so I did three music projects today, because those generally speak for themselves. First, Jazz in Africa "The Root of Jazz Expression".

In recent years, I have traveled through various parts of West Africa and had the opportunity to interact and share music with people from different local communities. It was an incredible experience: I had the good fortune to collaborate and perform with many accomplished, local musicians. One finding that permeated all these different impressions was the realization that the ‘Spirit of Blues’ expression is the ‘language’ link between Jazz and African music.

And there's also blue period project:

Classical music is in a bad way. Audiences are shrinking and aging. Orchestras are declaring bankruptcy... We staunchly believe that music is not the problem – rather, that audiences have lost interest in the experience: asocial seating, distant stages, disconnected performers and patrons, a stiff atmosphere. Dissatisfied with the fashion and spaces in which most classical music is presented, we want to experiment with new ideas.

Finally, I backed Jeff Brooks Quartet, "Just In Time":

I am completing my second original jazz album, “Just In Time” for the Jeff Brooks Quartet. The compositions are eclectic, ranging from depression era, blues, standard sounding jazz, to Iranian influenced beats.

Month of Kickstarter: Broken Puppets: It's time once again for Backscratching Monday, where I back the Kickstarter projects of my friends with some scratch. See? Back... scratch... moving on!

I got the idea for Backscratching Monday when I discovered that my friend Gus had put up a Kickstarter project for her Internet-literacy puppet show "The Media Show". Here it is: "The Media Show Explains Search, SEO, and Sock Puppets". It's funny because they're literally sock puppets! Man, I'm batting zero here.

Due to a shortage of Mondays, Month of Kickstarter will feature no more Backscratching Mondays. But feel free to start a Kickstarter project anyway and I'll see what I can do.

Keep it moving. In non-backscratching news, I backed a print run of small-press science fiction/superhero novel, "Broken".

In a post-war future world where First Contact has been made, humans are colonizing the stars, and the nations of Earth have been united under a central government, Extrahumans are required by law to belong to the Union. When a young man with visions of the future sets out on a mission to define the course of human history, he encounters a devastated former hero, a fascist dictatorship bent on world domination, and the realities of living in a society where affiliation is everything.

[Comments] (3) Month of Kickstarter: Picture Pants: I love pictures of things from the 1980s, and "London in the 1980s" definitely fits the bill. I think there's something about the film stock from the decade you grew up in that acts as a delivery system for nostalgia.

I also like wearing pants that fit, or at least I imagine I would, if that ever happened, so I dropped a Month of Kickstarter-record $60 on "Custom Fit Jeans". It feels less like backing a Kickstarter project and more like buying a pricey pair of jeans, but at this point Month of Kickstarter is on drunken, careening autopilot as I try to finish up my Findings work and gear up for the work surrounding the big announcement (still forthcoming).

Month of Kickstarter: Humble Dinosaurs: In a non-Kickstarter shocker, I dropped $25 on the Humble Indie Bundle #3. I haven't played it a lot due to busyness (can't even get a couple of the games to run on Linux), but I will say that VVVVVV is the game that Jet Set Willy should have been.

You've seen how on Month of Kickstarter I've made the tough decisions. You trust me to only back the projects that are right for America. The projects you hope you would back if you had the guts to take on crazy challenges like this one. That's why I've made the call to back the paper RPG "Dinosaurs...in SPAAACE!". Now with game mechanics!

Dinosaurs...in SPAAACE! runs on the Token Effort engine, a tried and tested rules-set that privileges pratfalls over point-stacking and mania over min-maxing. It quantifies humor and rewards you whether people are laughing with you, or at you.

[Comments] (2) Month of Kickstarter: Two Scoops: In today's "gift that keeps on giving" edition, I've backed "Jane's Easy Serve Ergonomic Ice Cream Scoop" at the $50 level. This will net me two commercial-quality ergonomic ice cream scoops. I can give one each to Susanna and Rachel, and hopefully stop the family feud over who gets ownership of the heirloom ice cream scoop (last known photo).

Month of Kickstarter: Fealty: Well, that's not a very fun Month of Kickstarter entry title. But not much can be done, because I've backed only one project today and it's got a one-word name: Fealty, a "territory control game that plays in a short amount of time but packs a solid strategic punch and game to game variety."

[Comments] (1) Month of Kickstarter: Chocolate Alphabet: Welcome to the PENULTIMATE, TRIPLE EDITION of Month of Kickstarter. Let's start the day off right with "The Art of Chocistry", a "virtual gourmet chocolate studio". Don't worry, only the studio is virtual. There's real chocolate.

Next, it's "COOP-made-in-USA book":

This book is an introduction to the worker-owned cooperatives in the United States, a reality that is not very well known, but prospered for more than 30 years.

The book presents several examples of worker-owned co-ops, with different governance methods and active in different fields, from retail to high-tech, with even nude dancing...

"Nude dancing" I could understand, but I'm not sure about "nude dancing...".

And finally, "The Endangered Alphabets Project" brings together the previously disparate worlds of wood carving and linguistic diversity:

The world has between 6,000 and 7,000 languages, but as many as half of them will be extinct by the end of this century. Another and even more dramatic way in which this cultural diversity is shrinking concerns the alphabets in which those languages are written. Writing has become so dominated by a small number of global cultures that those 6,000-7,000 languages are written in fewer than 100 alphabets.

Month of Kickstarter: Grand Finale: It's the final day of Month of Kickstarter, and time to go out with an astounding QUADRUPLE EDITION. If you know one thing about me now that you didn't know before Month of Kickstarter, it's that I'm a sucker for games with really insane names. Long story short, I've chosen to back "Oh My God! There's An Axe In My Head. The Game of International Diplomacy".

I also like games with non-insane names, and to reflect that side of my personality I've also backed "StarLife - A 4x Turn-Based Space Strategy Indie Game", even though it's not totally certain that there'll be a Linux version.

Finally, in an attempt to seem more cerebral, I'm backing a math project and a classical music project that have been in my list for a while. First, "Relatively Prime: Stories from the Mathematical Domain":

Relatively Prime will be an 8 episode audio podcast featuring stories from the world of mathematics. Tackling questions like: is it true that you are only 7 seven handshakes from the President, what exactly is a micromort, and how did 39 people commenting on a blog manage to prove a deep theorem.

And the music project, "Chopin Revolutionary Etudes": Even though Chopin set out to create technical works, each etude is a beautiful piece of music and the technique is used ONLY in creating a beautiful piece of music. Some of the greatest melodies of all times are contained in these etudes.

This project will bring out all the unique elements of Chopin's Etudes. But it will also teach a lot about playing and listening to music.

And that's it! In about two months, once all the projects I backed have reached their deadlines, I'll report back with statistics, and once the backer rewards start rolling in I'll mention any especially interesting ones.

Although I won't be backing Kickstarter projects at the frenetic rate that obtained during MoK (what with no longer having a well-paying job), I'll keep looking at projects, backing one occasionally, and posting occasional dumps here on NYCB.

The big announcement is happening tomorrow, for real this time.

[Comments] (7) Constellation Games: I've sold my first novel, Constellation Games, to Candlemark & Gleam. (Here's the official announcement.) Starting in November, the novel will be serialized online, one chapter a week, with a collected book to follow later.

I cannot stress this enough: you should subscribe to this book. I pulled out all the stops to make Constellation Games the most fun-to-read thing I could write. It's a near-future space opera set in Austin, Texas. It's got everything I've ever wanted/loved to see in a sci-fi epic: zero-gravity sex, hive minds, terraforming, paleontology, fine art, warps in space-time, existential horror, and shipping containers.

But most of all, it's got video games. My goal was to write the most compelling fictional treatment of not only gaming culture (which already gets a lot of attention in spec-fic) but the creative act of making a game, how our society treats software and games as artifacts, and how other cultures might do things differently. Think of "Mallory", except in the context of what the Candlemark editor flatteringly calls "one of the most stunningly plausible first-contact novels out there."

I'm very happy to have Candlemark & Gleam as my publisher. I showed Constellation Games to some traditional-print-publishing people, and they really liked the story but didn't like the blog-heavy format. C&G immediately got on board with the format, and thanks to the serialization model, we can present the story in a way that conveys the feeling of living through a huge societal disruption. In case you don't get enough of that already.

Constellation Games Progress Report: Probably going to move this onto Twitter because a) I really need experience writing for Twitter, for reasons which will become clear, and b) that may make it feel less like I'm working alone. But here's the intro.

As July was Month of Kickstarter around here, so is August the Month of Constellation Games Rewrites. I've long been interested in, and somewhat disturbed by, the relationship between different drafts of a work. I think it'll be interesting to create a record of the rewrite process, and a fun challenge to do it without giving away huge spoilers or saying stuff that only makes sense once you've read the book. With that in mind, I have four main tasks for this month:

  1. Write two new scenes: one near the beginning, one near the end.
  2. Merge two of the minor characters into a single entity. They're kind of similar, so this will make the novel tighter. I'd been considering this as a nice-to-have for a while, but turns out one of those characters needs a lot more screen time near the beginning. The quickest solution is to just give him someone else's scenes.
  3. Use the additional screen time from #2 to foreshadow the appearance of a third character, who doesn't really exist until about halfway through the book.
  4. There are a lot of blog posts in the story that read more like normal prose. Kate (my editor) and I agreed to tweak the framing device a little bit and just turn those chunks into normal prose. The big problem here is that weblogs are written in a very recent past tense. I need to convert some of the blog posts to a more distant past tense, as though you were hearing a story recounted later, without causing big tense dislocations when you encounter an authentic blog post from the period.

Today I finished one of the scenes for #1. #2 causes me serious cognitive dissonance. I need to tackle that next so I can stop thinking about both sets of characters.

[Comments] (1) Late Adopter: I started posting about my writing progress on identi.ca, mirrored to Twitter. This is two years after Sumana started using identi.ca, so you know I'm behind the curve.

Anyway, today at work I merged those two characters, and although there are still some rough spots to fix, it was easier than I'd feared. I don't even miss Bruce, which is confirmation for me that he needed to go. Bruce was kind of the novel's straight man, and that's not a position that needs filling. Any given scene needs a straight man, but it doesn't need to be the same person every time.

[Comments] (1) The MST3K-IMDB Effect, Quantified: Sometimes when I rewatch an MST3K episode I go to the movie's IMDB page to learn more about it. Inevitably I'm annoyed by the comments of people who give these movies one-star reviews solely on the basis of having watched an edited version on MST3K. But even greater than my annoyance is my desire to quantify the phenomenon. Today, I have quantified it.

What does being on MST3K do to a movie's IMDB rating? My best guess is that it knocks 2.9 stars off what what the rating would have been if the movie hadn't been on MST3K. But read on to see how I came up with that number, and why it depends on the director.


I am not a statistician. I'm not even a data scientist. I know how to get data out of the Internet. I know the difference between mean, median, mode, and standard deviation. And that's about it.


Here's an example in case you're not familiar with the MST3K-IMDB effect, which there's no reason you should since that's a name I just made up for it. Consider "Speech: The Function of Gestures", a short film directed by Arthur H. Wolf. It's got 5 votes and an IMDB rating of 5.2. Now here's another short film in the same series, "Speech: Platform Posture and Appearance". Same director, same writer, same lead actor, but this film had the misfortune to be double-billed with Red Zone Cuba on MST3K. As a result, it's got 98 votes and an IMDB rating of 1.6.

Call me skeptical, but I've watched both films and I'm not convinced there's really a three-and-a-half star difference between them. Another film in the series, "Speech: Using Your Voice", was also featured on MST3K, but in a less memorable episode ("Earth vs. the Spider"), and it struggles along with an IMDB rating of 2.4.)


Since the "Speech" films are part of a series, it makes sense to suppose that the difference between them is mostly due to the MST3K-IMDB effect. Of course, most films aren't part of a series. So I went by director instead. I picked up the filmography of every director who directed a film that was on MST3K. I split their films into two lists, "Normal" (not featured on MST3K) and "MST" (featured on MST3K). The "Normal" set only includes films that had enough IMDB votes to be given a rating. I included shorts and episodes of TV shows. This isn't perfect, because IMDB's plain-text data dump sometimes (but not always) gives a director's credit where their website gives a writer's credit. But it's close enough.

I took the average rating of the "Normal" list and the "MST" list. The difference between the two averages is how much it hurt that director to have one of their films featured on MST3K. As we'll see see, some directors were hurt a lot, and some of them shrugged it off, both for interesting reasons.

For the sake of comparison, the mean rating for a movie on IMDB at large is 6.4 stars, the median is 6.6 stars, and the standard deviation is 1.6 stars. However, "one star" is not a consistent unit of measurement. I'm considering redoing this table with normalized percentiles, but I'm not convinced there's a big demand for that, so for now you get stars.


Here's a big table with the data for every director who had at at least five films in the "Normal" set and at least one in the "MST" set. Normalm and Normalstd are the mean and standard deviation for the IMDB ratings of that director's non-MST films, and Normaln is the sample size. MSTm, MSTstd, and MSTn are the same thing for the director's MST film(s).

Effect1 is what we're looking for: for a given director, how many stars does a film lose just from being on MST3K? But wait! What if the director made some good stuff and some bad stuff, and only the bad stuff ended up on MST3K? The MST3K set would have lower ratings, but it wouldn't be because of MST3K. That's where Effect2 comes in, and that's why the table is sorted by Effect2. I'll explain Effect2 after you get a look at the data.

Click here to skip the table.

Director Normalm Normalstd Normaln MSTm MSTstd MSTn Effect1 Effect2 MSTed Films
Bava, Mario 6.2 0.9 22 6.3 - 1 -0.1 -0.1 Diabolik (1968)
Francisci, Pietro 5.0 0.9 9 4.8 0.7 2 0.2 0.3 Ercole e la regina di Lidia (1959), Le fatiche di Ercole (1958)
Tucker, Phil (I) 3.1 0.9 6 2.8 - 1 0.3 0.3 Robot Monster (1953)
Steckler, Ray Dennis 3.1 1.3 22 2.1 - 1 1.0 0.8 The Incredibly Strange Creatures Who Stopped Living and Became Mixed-Up Zombies!!? (1964)
Rebane, Bill 2.7 0.7 8 2.1 0.6 2 0.6 0.8 Monster a-Go Go (1965), The Giant Spider Invasion (1975)
Burke, Martyn 5.8 1.5 7 4.4 - 1 1.4 0.9 The Last Chase (1981)
Sachs, William 4.3 1.1 9 3.2 - 1 1.1 0.9 The Incredible Melting Man (1977)
Warren, Jerry 2.4 0.6 8 1.8 - 1 0.6 0.9 The Wild World of Batwoman (1966)
Maetzig, Kurt 5.4 1.3 13 4.0 - 1 1.4 1.1 Der schweigende Stern (1960)
Buchanan, Larry 3.3 1.2 25 2.0 - 1 1.3 1.1 The Eye Creatures (1965) (TV)
Yuasa, Noriaki 5.0 1.6 12 3.1 - 1 1.9 1.1 Gamera tai daiakuju Giron (1969)
Gordon, Bert I. 4.2 0.8 13 3.2 0.6 8 1.0 1.2 Beginning of the End (1957), Earth vs. the Spider (1958), King Dinosaur (1955), The Amazing Colossal Man (1957), The Magic Sword (1962), Tormented (1960), Village of the Giants (1965), War of the Colossal Beast (1958)
Brannon, Fred C. 5.8 1.3 43 4.2 - 1 1.6 1.2 Radar Men from the Moon (1952)
Mikels, Ted V. 3.3 1.3 19 1.8 - 1 1.5 1.2 Girl in Gold Boots (1968)
Wood Jr., Edward D. 4.0 1.0 15 2.8 0.8 2 1.2 1.2 Bride of the Monster (1955), The Sinister Urge (1960)
Zarindast, Tony 4.2 2.0 10 1.7 - 1 2.5 1.2 Werewolf (1996) (V)
Bradley, David (I) 4.9 1.9 6 2.4 - 1 2.5 1.3 12 to the Moon (1960)
Ludwig, Edward 6.1 1.0 33 4.7 - 1 1.4 1.3 The Black Scorpion (1957)
Clark, Greydon (I) 3.5 1.2 19 1.9 0.0 2 1.6 1.3 Angels' Brigade (1979), Final Justice (1985)
Franco, Jesus 4.1 1.2 166 2.5 - 1 1.6 1.3 The Castle of Fu Manchu (1969)
Eason, B. Reeves 5.8 0.9 46 4.5 - 1 1.3 1.4 Undersea Kingdom (1936)
Pyun, Albert 4.5 1.5 43 2.5 - 1 2.0 1.4 Alien from L.A. (1988)
Sturges, John 6.5 0.7 42 5.5 - 1 1.0 1.4 Marooned (1969)
Neumann, Kurt (I) 6.2 0.9 51 4.8 - 1 1.4 1.4 Rocketship X-M (1950)
Rou, Aleksandr 6.8 1.5 14 4.5 - 1 2.3 1.5 Morozko (1965)
Zens, Will 4.4 1.8 7 1.6 - 1 2.8 1.6 The Starfighters (1964)
Corman, Roger 5.4 1.4 44 3.2 0.8 6 2.1 1.6 Gunslinger (1956), It Conquered the World (1956), Swamp Women (1956), Teenage Cave Man (1958), The Saga of the Viking Women and Their Voyage to the Waters of the Great Sea Serpent (1957), The Undead (1957)
Fukuda, Jun (I) 5.8 1.3 10 3.6 - 1 2.2 1.7 Gojira tai Megaro (1973)
Piquer Simón, Juan 3.5 1.0 11 1.8 - 1 1.7 1.7 Los nuevos extraterrestres (1983)
Crichton, Charles 6.5 1.7 58 3.6 - 1 2.9 1.7 Cosmic Princess (1982) (TV)
Portillo, Rafael (I) 4.7 1.6 10 1.9 - 1 2.8 1.7 La momia azteca contra el robot humano (1958)
Beaudine, William 6.1 1.2 128 4.1 - 1 2.0 1.7 Design for Dreaming (1956)
Peshak, Ted 3.7 0.7 14 2.5 0.1 2 1.2 1.7 Appreciating Your Parents (1950), What to Do on a Date (1951)
Conway, James L. (I) 7.2 1.2 74 5.0 - 1 2.2 1.8 Hangar 18 (1980)
Worth, David (II) 4.3 1.3 20 2.0 - 1 2.3 1.8 Warrior of the Lost World (1983)
Grefe, William 4.3 1.5 11 1.7 - 1 2.6 1.8 Wild Rebels (1967)
Shonteff, Lindsay 4.5 0.9 19 3.0 0.6 2 1.6 1.8 Devil Doll (1964), The Million Eyes of Sumuru (1967)
Kane, Joseph (I) 6.2 0.9 127 4.5 - 1 1.7 1.8 Undersea Kingdom (1936)
Yarbrough, Jean 6.5 1.7 87 3.4 - 1 3.1 1.8 The Brute Man (1946)
Hessler, Gordon 5.7 1.2 48 3.4 - 1 2.3 1.9 "The Master" (1984)
Kessler, Bruce 6.5 1.6 67 3.4 - 1 3.1 1.9 "The Master" (1984)
Lawrence, Quentin 7.1 1.4 12 4.4 - 1 2.7 1.9 The Trollenberg Terror (1958)
Fox, Wallace 6.1 1.1 31 4.0 - 1 2.1 2.0 The Corpse Vanishes (1942)
Kincaid, Tim (I) 6.5 2.2 25 2.0 - 1 4.5 2.0 Robot Holocaust (1986) (V)
Malatesta, Guido 4.3 1.4 14 1.5 - 1 2.8 2.1 Maciste contro i cacciatori di teste (1963)
Dein, Edward 6.1 1.0 7 4.1 - 1 2.0 2.1 The Leech Woman (1960)
Juran, Nathan 6.4 1.0 53 4.2 - 1 2.2 2.1 The Deadly Mantis (1957)
Baldanello, Gianfranco 4.8 1.3 9 2.0 - 1 2.8 2.1 Il raggio infernale (1967)
Beebe, Ford 6.1 0.7 50 4.6 - 1 1.5 2.1 The Phantom Creeps (1939)
Winters, David (I) 5.5 1.7 17 1.8 - 1 3.7 2.1 Space Mutiny (1988)
Harvey, Herk 5.1 1.1 22 2.7 0.4 3 2.4 2.2 Cheating (1952), What About Juvenile Delinquency? (1955), Why Study Industrial Arts? (1956)
Medak, Peter 6.7 1.4 63 3.6 - 1 3.1 2.2 Cosmic Princess (1982) (TV)
Corona Blake, Alfonso 5.6 1.3 13 2.7 - 1 2.9 2.2 Santo vs. las mujeres vampiro (1962)
D'Amato, Joe 4.7 1.3 141 1.9 - 1 2.8 2.2 Ator l'invincibile 2 (1984)
Pierce, Charles B. 4.9 1.3 10 1.9 - 1 3.0 2.2 The Barbaric Beast of Boggy Creek, Part II (1985)
Rich, David Lowell 6.6 1.2 99 3.8 - 1 2.8 2.3 SST: Death Flight (1977) (TV)
Strock, Herbert L. 6.1 1.5 23 2.6 - 1 3.5 2.3 The Crawling Hand (1963)
Katzin, Lee H. 6.6 1.3 71 3.6 - 1 3.0 2.3 The Stranger (1973) (TV)
Ulmer, Edgar G. 5.7 1.0 38 3.4 - 1 2.3 2.3 The Amazing Transparent Man (1960)
Sloane, Rick 3.4 0.7 14 1.7 - 1 1.7 2.3 Hobgoblins (1988)
Cardos, John 'Bud' 5.1 1.4 9 1.8 - 1 3.3 2.4 Outlaw of Gor (1989)
Castellari, Enzo G. 5.5 1.1 39 2.9 - 1 2.6 2.4 Fuga dal Bronx (1983)
Mahon, Barry 4.4 1.3 34 1.3 - 1 3.1 2.4 Rocket Attack U.S.A. (1961)
Vogel, Virgil W. 7.1 1.1 141 4.4 - 1 2.7 2.5 The Mole People (1956)
Giancola, David 3.9 0.7 7 2.0 - 1 1.9 2.5 Tangents (1994)
Francis, Freddie 5.7 1.1 30 3.0 - 1 2.7 2.6 The Deadly Bees (1967)
Baker, Roy Ward 7.0 1.2 89 3.7 - 1 3.3 2.7 Moon Zero Two (1969)
Nicol, Alex (I) 5.6 1.0 9 2.9 - 1 2.7 2.7 The Screaming Skull (1958)
Sears, Fred F. 6.1 1.0 45 3.3 - 1 2.8 2.7 Teen-Age Crime Wave (1955)
Rakoff, Alvin 6.5 1.2 25 3.3 - 1 3.2 2.8 City on Fire (1979)
Lieberman, Jeff (I) 5.7 0.7 9 3.9 - 1 1.8 2.8 Squirm (1976)
Cahn, Edward L. 5.5 1.0 106 2.8 - 1 2.7 2.8 The She-Creature (1956)
Arnold, Jack (I) 6.7 1.0 109 3.9 1.2 2 2.8 2.8 Revenge of the Creature (1955), The Space Children (1958)
Heyes, Douglas 7.6 1.1 42 4.4 - 1 3.2 2.8 Kitten with a Whip (1964)
Ferroni, Giorgio 5.3 1.1 16 2.0 - 1 3.3 3.0 New York chiama Superdrago (1966)
Fowler Jr., Gene 6.8 1.1 9 3.4 1.1 2 3.4 3.0 I Was a Teenage Werewolf (1957), The Rebel Set (1959)
Bava, Lamberto 5.4 1.1 30 2.1 - 1 3.3 3.0 Shark: Rosso nell'oceano (1984)
Wolf, Arthur H. 4.6 0.8 8 2.0 0.4 2 2.6 3.1 Speech: Platform Posture and Appearance (1949), Speech: Using Your Voice (1950)
Jameson, Jerry 6.1 1.2 109 2.5 0.6 2 3.6 3.1 Superdome (1978) (TV), The Bat People (1974)
Koch, Howard W. 6.3 1.3 20 2.4 - 1 3.9 3.1 Untamed Youth (1957)
Morse, Hollingsworth 6.6 1.3 105 2.5 0.6 2 4.0 3.1 Crash of Moons (1954) (TV), Manhunt in Space (1956) (TV)
Cottafavi, Vittorio 6.2 0.8 17 3.6 - 1 2.6 3.2 Ercole alla conquista di Atlantide (1961)
Moxey, John Llewellyn 6.7 1.2 110 2.9 - 1 3.8 3.2 "San Francisco International Airport" (1970) {San Francisco International (#1.0)}
McLaglen, Andrew V. 6.5 1.3 215 2.3 - 1 4.2 3.2 Mitchell (1975)
Trikonis, Gus 5.8 1.3 72 1.7 - 1 4.1 3.2 Five the Hard Way (1969)
De Martino, Alberto (I) 5.2 0.9 25 2.2 0.2 2 3.0 3.3 L'uomo puma (1980), OK Connery (1967)
Cardona, René (I) 5.6 1.0 45 2.0 - 1 3.6 3.4 Santa Claus (1959)
Miner, Allen H. 7.6 1.5 25 2.4 - 1 5.2 3.4 The Days of Our Years (1955)
Newfield, Sam (I) 5.5 0.9 145 2.4 0.5 4 3.2 3.5 I Accuse My Parents (1944), Lost Continent (1951), Radar Secret Service (1950), The Mad Monster (1942)
Sholem, Lee 6.7 1.4 44 1.9 - 1 4.8 3.5 Catalina Caper (1967)
Turner, Ken (I) 7.3 1.5 7 2.2 - 1 5.1 3.5 Revenge of the Mysterons from Mars (1981) (TV)
Haas, Charles F. 6.8 1.2 31 2.6 - 1 4.2 3.6 Girls Town (1959)
Kowalski, Bernard L. 6.9 1.1 80 2.9 0.2 2 4.0 3.7 Attack of the Giant Leeches (1959), Night of the Blood Beast (1958)
Fukasaku, Kinji 7.1 0.9 53 3.8 - 1 3.3 3.8 The Green Slime (1968)
Gentilomo, Giacomo 5.6 0.8 11 2.4 - 1 3.2 3.8 Maciste e la regina di Samar (1964)
McDougall, Don 7.2 1.4 158 1.7 - 1 5.5 3.8 Riding with Death (1976) (TV)
Webster, Nicholas 6.8 1.1 13 2.3 - 1 4.5 3.9 Santa Claus Conquers the Martians (1964)
Austin, Ray (I) 6.6 1.1 135 2.1 - 1 4.5 3.9 "The Master" (1984) {Hostages (#1.4)}
Oswald, Gerd 6.7 1.2 52 1.8 - 1 4.9 4.1 Agent for H.A.R.M. (1966)
Lynn, Robert (II) 5.5 0.8 12 2.2 - 1 3.3 4.4 Revenge of the Mysterons from Mars (1981) (TV)
Szwarc, Jeannot 7.3 1.0 167 2.5 - 1 4.8 4.6 Code Name: Diamond Head (1977) (TV)
Myerson, Alan 7.0 1.1 101 2.1 - 1 4.9 4.6 "The Master" (1984) {State of the Union (#1.3)}
Lipstadt, Aaron 7.2 1.1 73 2.1 - 1 5.1 4.7 City Limits (1984)
Rondeau, Charles R. 7.0 1.0 58 2.2 - 1 4.8 4.7 The Girl in Lovers Lane (1960)
Collins, Lewis D. 6.0 0.9 56 1.7 - 1 4.3 4.9 Jungle Goddess (1948)
Green, Alfred E. 6.4 0.7 73 2.4 - 1 4.0 5.4 Invasion USA (1952)
Levi, Alan J. 6.9 0.9 122 1.7 - 1 5.2 5.7 Riding with Death (1976) (TV)
Lane, David (I) 7.2 0.8 19 2.1 - 1 5.1 6.3 Invaders from the Deep (1981)
Greidanus, Tjardus 6.3 0.7 13 1.7 - 1 4.6 6.3 The Final Sacrifice (1990)
Saunders, Desmond (I) 6.7 0.7 14 2.1 - 1 4.6 6.5 Invaders from the Deep (1981)
Ptushko, Aleksandr 7.2 0.4 8 4.2 1.0 3 2.9 6.9 Ilya Muromets (1956), Sadko (1953), Sampo (1959)
Williams, Douglas (I) 8.5 0.7 6 2.1 - 1 6.4 9.6 Overdrawn at the Memory Bank (1983) (TV)
Morgan, William (I) 6.2 0.4 10 2.6 - 1 3.6 10.1 The Violent Years (1956)
Elliott, David (II) 6.7 0.3 9 2.1 - 1 4.6 14.1 Invaders from the Deep (1981)
Average 2.9


Now, for the explanation of Effect2. From Normalstd we know how likely this director is to make a film that's substantially better or worse than their average. If they made one bad film that was on MST3K, and there was no MST3K-IMDB effect for that director, the rating for that film would most likely be within two standard deviations of the director's average. But if there were a strong MST3K-IMDB effect for that director, the rating for the MSTed film would be much lower than the director's other bad films. So, Effect2 is: how many standard deviations below Normalm is MSTm?

Let's look at the extremes of the list. First, the directors with very low Effect2:

And this is the big thing I learned doing the project: you can calculate the MST3K-IMDB effect, but you must also look at the director's average movie rating to see what it means. A low Effect2 just means that being on MST3K doesn't hurt a director's ratings very much. It doesn't say anything about the movie's quality.

OTOH, a director with a high Effect2 is probably worth a second look in a non-MST3K context.

And so on. The MST3K-IMDB effect is real--ninety percent of the directors in this table have an Effect2 of more than one standard deviation, and for sixty percent of them, it's more than two standard deviations. But it doesn't affect all directors equally.

Let's close out by taking a look at some of MST3K's favorite directors.


I'm still annoyed by those one-star reviews, but I understand them a little better now. When you watch, say, "The Function of Gestures", you enjoy it for its camp value, you have fun with it, and you give it a relatively good rating. But when you watch "Platform Posture and Appearance" or "Using Your Voice" on MST3K, you're watching someone else making fun of it, you have fun at its expense, and you give it a bad rating as a sign of solidarity with the MST3K characters.

Finally, I'd like to thank IMDB for, in a relic of its geeky past, making plain-text dumps of its data available. It's a strange feeling to have a file open in an Emacs buffer that lists nearly every movie ever made. (There are about 2 million, if you're curious.) Now that I have the data and scripts to process it, I may run other cinematic experiments in the future. One thing I would like to see added is IMDB links for the people and movies. It's a pain to look all these things up, which is why there aren't as many links in this post as you'd think.

[Comments] (2) Loaded Dice: Last month I downloaded a bunch of data from BoardGameGeek's web service for use in an art project. I'll be announcing the art project soon, but today I'm announcing "Loaded Dice", a data-mining project using the same data.

I've been writing scripts that analyze the BGG data and produce interesting charts and tables. I'll keep adding stuff to these pages until I get bored with this data. I've put up thirteen experiments so far. Here are some highlights:

Beautiful Soup 4 Beta: Now With Python 3 Support: The main thing holding back Beautiful Soup 4 from release was that it didn't work with Python 3. Fortunately, Thomas Kluyver stepped in and wrote some code, and now I can present the first BS4 beta release.

There's still some work to do, and it'll be a while before I get to it, but the work that remains is pretty minor compared to the advantages you get from using BS4 instead of BS3. Try out the beta, and if it gets good reviews I may just make a 4.0 release and deal with the minor things afterwards.

Board Game Dadaist: I mentioned when I announced "Loaded Dice" that I got all this BoardGameGeek data for an art project. I'm now announcing the art project, Board Game Dadaist. This page uses an as-yet-unnamed algorithm to mash up game titles, descriptions, and BGG comments into new, intriguing games like "Plastic Walls Are High", "Shopping - Destroy", and "Armchair to Hell". ("First driver to complete 3 laps is the winner. Capture them, brainwash them, throw them into your dungeon or consume them for spells.")

The BGD page is updated every 5 minutes with new games, and there's a daily RSS feed. Special thanks to Beth for the logo. (My logo looked awful.)

Queneau Assembly: That's my name for the formerly-unnamed technique I used in "Board Game Dadaist". It all started in April, the night I was guest critic for Adam's ITP class. Afterwards I went out to dinner with Adam and Rob, and Adam was talking up Markov chains. Dude loves him some Markov chains. I said "Man, I'm tired of Markov chains. Markov chains are so 70s, they have little coke spoons dangling from them. I'm gonna come up with a better algorithm for creating generative text."

Big talk, but fortunately I didn't have to come up with a better algorithm, because I already had. Back in 2008 I released a project called "Spurious", which generates new Shakespearean sonnets by picking lines from the existing sonnets. It generates two sonnets at once using two different algorithms. Algorithm B (the one lower down on that page) is totally random: you could get a new sonnet made entirely of the first lines of other sonnets. But Algorithm A (the first one on that page) creates what I'm calling a Queneau assembly. The first line of a new sonnet is the first line of some existing sonnet. The second line is the second line of some other sonnet. The third line comes from the set of third lines, and so on to the end.

Oulipo founder Raymond Queneau did something very similar in his 1961 book "Hundred Thousand Billion Poems". This may be where I got the algorithm I used in "Spurious", though I don't think it was a conscious homage. In "Hundred Thousand Billion Poems" there are ten sonnets bound such that you can "turn the page" for a single line of the sonnet, changing that line while leaving the rest of the poem intact. Each generated poem feels like a sonnet because it starts with a "first line" and ends with a "last line" and every line in between is placed where it was in some manually generated sonnet.

I've named the technique in honor of Queneau because I can't find anyone who used it earlier. It's not universally better than a Markov chain, because it only works in certain cases:

That said, the Queneau assembly gives very entertaining results, and it's now my go-to dada technique, promoted over Markov chains and even unadulterated randomness.

The simple algorithm

I've come up with a number of algorithms for making Queneau assemblies. I'll talk about the simplest first, just so you'll see how this works. This is a refined version of the algorithm I used for "Mashteroids" (yes, those asteroid descriptions were me reinventing Queneau assemblies). It's not the algorithm I used for "Board Game Dadaist"; I'll talk about that later.

You've got a body of N texts, T0, T1, ..., TN-1. Each text can be split into some number of chunks, eg. T00, T01, ..., T0M-1.

Split each text into chunks and assign each chunk to one of three buckets. The first chunk from each text goes into the "first" bucket. The last chunk from each text goes into the "last" bucket". All the other chunks go into the "middle" bucket.

Also keep track of how text lengths are distributed: how likely it is that a text consists of one chunk, how likely that it consists of two chunks, and so on.

Now you're ready to assemble. Pick a length for your new text that reflects the length distribution of the existing texts. Then pick a chunk from the "first" bucket. If your target length is greater than 1, pick a chunk from the "last" bucket". If your target length is greater than 2, pick chunks from the "middle" bucket intil you've got enough. Concatenate the chunks first-middle-last, and you've got a Queneau assembly!

Paragraphs made from sentences

Now let's look at the scales on which you might create a Queneau assembly. Outside of poetry, the paragraph is the Queneau assembly's natural habitat. A pragraph has a flow to it, especially when you've got something like a description of a board game or an asteroid that's only one paragraph long.

You need to handle things like quotes and parentheses that open in one sentence and don't close by the end of the sentence, or that close without having opened. I wrote code for this in BGD but it doesn't catch all the cases.

Phrases made from words

In "Board Game Dadaist", the names of games are also Queneau assemblies. Here the chunks are words. I take the first word from the name of game A, the second word from the name of game B, and so on. So "Pirates! Denver" might come from "Pirates! Miniature Battles on the High Seas" and "Monopoly: Denver Broncos".

Quotes and parentheses are still problems, though it's not as bad. The big problem I ran into was repeated words, and words like "the" which are not allowed to end a game name. (The simple algorithm, with its "last" bucket, prevents "the" from showing up last unless it showed up as the last word of an existing game. In the algorithm I used for "Board Game Dadaist", I had to special-case this.)

In general, Queneau assemblies will not create coherent English sentences. Much as it pains me to admit, a Markov chain is better for that. It works for board game titles because we allow titles a lot of creative license, even up to the point of suspending the rules of grammar. "Pirates! Denver" makes no sense as a sentence, but it's a perfectly good game title.

Words made from letter-chunks

Many games have single-word titles, eg. "Carcassonne". I wanted to have single-word titles in BGD, but I didn't want to duplicate real names. So I applied the Queneau assembly algorithm on the word level.

Here, the chunk is a run of letters that's all vowels or all consonants. So "Carcassonne" would be split into the chunks ["C", "a", "rc", "a", "ss", "o", "nn", e"]. I keep two sets of buckets, one for vowels and one for consonants. If the first chunk was a vowel chunk, the second chunk is a consonant chunk, and I alternate til I reach the end.

This means that single-word BGD titles are almost never English words, but they do capture the feel of those one-word titles that aren't words (examples: "Zajekan", "Fraseda", "Kongin", "Q-blardo").

The BGD algorithm

Now that you see how it works, I'll explain the algorithm I actually use for "Board Game Dadaist". Instead of three buckets, I have a lot of numbered buckets. When I split a text into T00, T01, ..., T0M-1, I put T00 into bucket 0, T01 into bucket 1, and so on, with T0M-1 going into bucket M-1. I create an assembly by picking from bucket 0, then bucket 1, and so on until I've reached the target length.

This is the algorithm that "Hundred Thousand Billion Poems" uses, and when the texts have more structure than "beginning/middle/end", this algorithm works a lot better. I don't think it matters much for BGD descriptions, but I do think it matters for game names. I would like to combine this algorithm with the "last" bucket from the simple algorithm, because right now board game descriptions sometimes end abruptly with a sentence like "Contents:".

Children Formed by Plants or Objects : I just discovered that you can search the USPTO's trademark registry by "design code", to find trademarks that use a certain graphical element. Furthermore, the design code classifications are fairly insane. Some choice quotes:

02.01.04 Religious figures, men wearing robes, shepherds, monks and priests

Excluding: Asian-Pacific men (02.01.11) and wizards (04.01.25) are not coded in this section.

02.01.32 Other men, including frogmen, men wearing space suits and men wearing monocles

02.03.25 Other women including hobos, women holding fans and women with weaponry

02.05.27 Other grotesque children including children formed by plants or objects

Mythological beings are cross-coded in the Human Category only when they are depicted as ordinary humans having no indicia of their mythological powers.

03.23.15 Micro-organisms (including sperms)

Only ladders which fold open to form a triangular profile are in 14.09.01. Other straight or extension ladders are in 14.09.02.

Clock radios are double coded in 16.01.03 and 17.01.02.

Concentric circles (26.01.17 and 26.01.18) and circles within a circle (26.01.20) are not considered three circles for purposes of coding.

[Comments] (1) "No Module Named BeautifulSoup": Since Beautiful Soup 4 is not backwards compatible with Beautiful Soup 3, I put it in a different module: bs4 instead of BeautifulSoup. In a non-ironic twist, the module rename has itself turned out to be the biggest compatibility problem between BS3 and BS4. The new module name has caused problems on several occasions where users thought BS4 worked just like BS3, or didn't even know they were using BS4 (1 2 3).

Why would you be using BS4 without knowing it? It's an unreleased beta. Well, that's happened before. When I made the BS4 alpha release, I put the tarball in /software/BeautifulSoup/download/4.x, and PyPI picked it up because it knows /software/BeautifulSoup/download/ is where I keep my tarballs. PyPI believed the 4.0 alpha to be the latest release of BeautifulSoup and started recommending it it to all and sundry, which was not what I wanted. So I moved the 4.x tarballs into a different directory that PyPI doesn't know about: /software/BeautifulSoup/unreleased/4.x/, and that solved the problem.

But now it was happening again. Some installation process or other was finding my /unreleased/ directory, picking up the beta tarball, and installing it by default as the latest version of Beautiful Soup. Why?

Thanks to the bug Brian Shumate filed today I tracked the problem down. It turns out the pip package-installer program scrapes the Beautiful Soup homepage (using regexes, not Beautiful Soup, ha). It looks for tarballs and picks the one with the biggest version number. So just by linking to the beta tarball and giving it a "4.0" name I was declaring 4.0 ready for prime time.

So I got a couple problems:

You Can't Be Serious: It's time for another big HTML table! This time I'm interested in movie connections. IMDB's dataset relates movies to each other using many different predicates: "edited into", "remake of", "alternate language version of", and so on. I'm interested in two of the most common predicates, "referenced in" and "spoofed in". Specifically, I want to answer these questions:

I think my table speaks for itself, but I'll give a legend above it and a little commentary below it. The table has two columns:

  1. The most spoofed movies and TV shows (by number of "spoofed in" references)
  2. The movies and TV shows most referenced in earnest (by number of "referenced in" references)

The little numbers are the counts of "spoofed in" or "referenced in" references for that movie or TV show.

A title in bold shows up on only one list. This doesn't mean that, for instance, "The X-Files" has never been spoofed, only that it's not spoofed enough to make it onto the "most spoofed" list. A title in italics shows up on both lists (or would, if I extended the lists a little bit), but it's in a much higher position on the "spoof" list (left column) or the "non-spoof" list (right column). If a title is neither bolded nor italicized, then it's in approximately the same position on the "spoof" and "non-spoof" lists.

At this point I should probably let the table do the talking, so here it is. If you hate data, you can skip the table.
Most SpoofedMost Referenced Seriously
1Star Wars 279Star Wars 1793
2The Wizard of Oz 199The Wizard of Oz 1397
3"Star Trek" 180"Star Trek" 1270
4The Godfather 155The Godfather 737
5The Matrix 148Psycho 693
62001: A Space Odyssey 141Casablanca 622
7Psycho 139Star Wars: Episode V - The Empire Strikes Back 587
8Raiders of the Lost Ark 134Jaws 585
9Jaws 120"The Simpsons" 573
10Star Wars: Episode V - The Empire Strikes Back 119Gone with the Wind 534
11The Exorcist 96King Kong 527
12King Kong 96The Terminator 485
13"Batman" 95E.T.: The Extra-Terrestrial 449
14Pulp Fiction 932001: A Space Odyssey 448
15Titanic 93"Sesame Street" 448
16Superman 89Raiders of the Lost Ark 440
17E.T.: The Extra-Terrestrial 86Apocalypse Now 422
18Apocalypse Now 85Frankenstein 379
19The Shining 84The Exorcist 374
20"The Twilight Zone" 83"The Twilight Zone" 366
21The Terminator 81"Saturday Night Live" 363
22Casablanca 75Scarface 361
23Jurassic Park 74Citizen Kane 358
24Frankenstein 72Pulp Fiction 350
25Taxi Driver 71Titanic 350
26Rocky 68The Shining 348
27Alien 68"Doctor Who" 348
28The Silence of the Lambs 67Alien 346
29The Blair Witch Project 61"The Oprah Winfrey Show" 344
30Il buono, il brutto, il cattivo. 59Taxi Driver 343
31Terminator 2: Judgment Day 59Ghost Busters 343
32The Graduate 58"The Flintstones" 334
33Ghost Busters 57Rocky 333
34Gone with the Wind 56Star Wars: Episode VI - Return of the Jedi 328
35Star Wars: Episode VI - Return of the Jedi 56The Matrix 326
36It's a Wonderful Life 54Back to the Future 323
37Forrest Gump 52The Silence of the Lambs 323
38Goldfinger 52"Batman" 317
39Back to the Future 52A Clockwork Orange 306
40Dr. No 49Terminator 2: Judgment Day 302
41Star Wars: Episode I - The Phantom Menace 49"Happy Days" 301
42Gojira 48Snow White and the Seven Dwarfs 298
43Dr. Jekyll and Mr. Hyde 48The Sound of Music 293
44Scarface 47A Nightmare on Elm Street 289
45Planet of the Apes 47Superman 285
46"Cops" 47"Gilligan's Island" 284
47"Scooby Doo, Where Are You!" 47Dracula 282
48Snow White and the Seven Dwarfs 46"Star Trek: The Next Generation" 276
49Dracula 45"The X Files" 275
50The Sound of Music 44"The Brady Bunch" 271
51Reservoir Dogs 44Dr. No 270
52Batman 44"I Love Lucy" 269
53Citizen Kane 43First Blood 268
54Goodfellas 43Night of the Living Dead 262
55Night of the Living Dead 43"American Idol: The Search for a Superstar" 261
56Carrie 43Gojira 259
57"Jeopardy!" 43Jurassic Park 258
58Saturday Night Fever 42Dirty Harry 257
59The Texas Chain Saw Massacre 41The Texas Chain Saw Massacre 254
60"American Idol: The Search for a Superstar" 40Vertigo 253
61Mary Poppins 40It's a Wonderful Life 253
62Full Metal Jacket 40Aliens 243
63Dirty Harry 40Planet of the Apes 243
64The Lord of the Rings: The Fellowship of the Ring 39Batman 239
65The Karate Kid 38Il buono, il brutto, il cattivo. 236
66"The Brady Bunch" 38"Scooby Doo, Where Are You!" 231
67Friday the 13th 37The Graduate 229
68RoboCop 37Goldfinger 227
69Risky Business 37Deliverance 226
70"I Love Lucy" 36The Lord of the Rings: The Fellowship of the Ring 221
71Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb 35Blade Runner 220
72"Star Trek: The Next Generation" 35Die Hard 218
73"The Tonight Show Starring Johnny Carson" 34"Jeopardy!" 215
74Close Encounters of the Third Kind 34"Seinfeld" 215
75"Baywatch" 33Rosemary's Baby 214
76Scream 33Star Wars: Episode I - The Phantom Menace 213
77Flashdance 33The Lion King 211
78Lady and the Tramp 33Saturday Night Fever 210
79First Blood 33Mary Poppins 206
80"The Oprah Winfrey Show" 32Bambi 205
81Willy Wonka & the Chocolate Factory 32The Karate Kid 202
82The Lion King 32"Friends" 201
83"The Flintstones" 32"The Tonight Show Starring Johnny Carson" 200
84Indiana Jones and the Temple of Doom 32Halloween 199
85"24" 32Reservoir Dogs 199
86"Mission: Impossible" 32Top Gun 198
87The Seven Year Itch 32Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb 196
88Halloween 32"The Muppet Show" 194
89Spider-Man 31"Buffy the Vampire Slayer" 193
90Patton 31"The Andy Griffith Show" 192
91Rain Man 31Forrest Gump 188
92Thriller 31Dawn of the Dead 188
93Singin' in the Rain 30Friday the 13th 188
94Aliens 30Jerry Maguire 187
95A Clockwork Orange 30Close Encounters of the Third Kind 186
96Monty Python and the Holy Grail 30Singin' in the Rain 185
97Grease 30"Dancing with the Stars" 180
98Deliverance 30West Side Story 177
99Mission: Impossible 30"Baywatch" 176
100"The Sopranos" 30Grease 175

I'm not terribly happy with this data. I suspect many "referenced in" references are actually spoofs, or are throwaway jokes that don't even rise to the level of "spoof". Are there really 179 non-spoof references to "The Lion King"? You know everyone's just riffing on the baby-lifting shot.

However, the reverse problem ("incorrectly regarded as spoofs") is nonexistent, so it's easy to spot things like The Blair Witch Project and "Cops" which only exist in our culture as things to make fun of; as well as things that are occasionally referenced seriously but much more frequently spoofed (The Matrix).

You Can't Be Serious: Addendum: I Should Be In That Spoof: After messing around with the IMDB movie connections for the original "You Can't Be Serious", I've decided to measure a movie's spoofability with a ratio instead of just counting the number of times it's been spoofed. Counting spoofs only measures the impact a movie has on our culture. Star Wars is the most-spoofed movie by far, but also the most-referenced movie by far. Measuring the ratio of spoofs to earnest references will find movies whose impact on culture was primarily to give us something to spoof.

(I came into this hating the word "spoof", BTW, and the more I type it the more I hate it.)

I calculated the spoof/reference ratio for all IMDB entries with more than one spoof and more than 5 references. Surprisingly, the movies with spoof/reference ratios near or above unity aren't movies; they're almost all TV shows:
"Crocodile Hunter" 1.62138
"TMZ on TV" 1.56149
"The Undersea World of Jacques Cousteau" 1.43107
"The Twilight Zone" 1.43107
"Hardball with Chris Matthews" 1.402115
The Perils of Pauline 1.2997
"The McLaughlin Group" 1.1298
"Inside the Actors Studio" 1.071514
The Six Million Dollar Man 1.0088
Kids 1.0077
Der 90. Geburtstag oder Dinner for One 1.0077
Bigfoot 1.0066
"The Tomorrow Show" 1.001111
"Aquaman" 1.0066
Riverdance: The Show 0.911011
"Behind the Music" 0.911011
Uncle Tom's Cabin 0.90910
"Through the Keyhole" 0.8878
King's Quest: Quest for the Crown 0.8667
"The French Chef" 0.8356

I'll let you look up the ones you don't recognize, though I will say that "Der 90. Geburtstag oder Dinner for One" looks pretty great, and "Bigfoot" is exactly what you think it is: the one-minute 1967 film. (IMDB rating: 8.2!)

Calculating the average spoof/reference ratio is an iffy proposition, but for movies with a lot of references, it's around 0.15.

What movies have a very low ratio? Are there movies that are referenced, say, 100 times more often than they're spoofed? Once again, the question of what distinguishes a "reference" from a "spoof" rears its mediocre-looking head, but maybe it cancels out when we're calculating a ratio between the two. Let's find out.

Sex, Lies, and Videotape 0.012138
Deep Throat 0.02292
"Little House on the Prairie" 0.02287
"Hogan's Heroes" 0.02283
Guess Who's Coming to Dinner 0.03277
Brazil 0.03273
The Searchers 0.03273
THX 1138 0.03271
"Green Acres" 0.03269
"Will & Grace" 0.03264
Vertigo 0.038253
Mr. Smith Goes to Washington 0.03263
Sleeping Beauty 0.03262
"Two and a Half Men" 0.03261
A Wild Hare 0.03260
The Way We Were 0.03260
To Kill a Mockingbird 0.03260
Tootsie 0.03259
"Captain Kangaroo" 0.03258
Sophie's Choice 0.03258

I was skeptical about this list, but upon investigation it's pretty good! Certainly better results than I got on Sunday. Sex, Lies, and Videotape the movie wasn't that influential, but it has one of the most influential titles in cinema history. Similarly for Mr. Smith Goes to Washington: lots of title references and lots of conspicuously visible movie posters. A Wild Hare was the origin of the phrase "What's up, Doc?". Guess Who's Coming to Dinner scores whenever a character wryly cracks that phrase at the end of a scene. And so on.

: On Monday Sumana and I went to the Socrates Sculpture Park to see "Odysseus at Hell Gate", a production that mashed up the Odyssey and New York City history and puppets. Little did we know that it was not a dramatic production in which the puppets would interact, but a "puppetscape" in which the puppets would wander around the park for all to marvel at. I'm not complaining, as the puppets were cool. I took pictures but they didn't turn out, so enjoy pictures from someone else who was there.

My favorite puppet by far was Robert Moses as Polyphemus, with the Unisphere for a head and a traffic light for an eye, operated by puppeteers in hard hats and surrounded by a flock of sheep-cars honking their bicycle horns. But in terms of puppeteering excellence, the shades were the best.

[Comments] (2) Loaded Dice, Round 2: At Pat's suggestion I did some more number-crunching and put the results in Loaded Dice. First, check out Standard deviation of ratings over time. Although the average rating is higher for new games than for old games, the standard deviation is always about 0.8. That is: the average rating for any 1980 game is some number, plus or minus 0.8; and the average rating for any 2010 game is a slightly higher number, plus or minus 0.8. The range of opinion is surprisingly limited.

Also take a look at the huge new section on Ownership and the Trade Market, full of graphs and tables taken from BGG's information about how many people own a game, how many people want to own it, and how many want to get rid of it. Including but not limited to:

[Comments] (2) : Whoever sets up the old-photography exhibits for the Metropolitan Museum is on fire. Earlier this year they put up "Our Future Is In The Air", an exhibit of photos taken in the 1910s, when cameras became cheap and portable enough for ordinary people to own them and carry them around photographing random crap. One frequently-lovestruck teenage boy put together a scrapbook of "Girls I Have Known" ('Titled, signed, and inscribed in ink on cover board: "GIRLS // I // HAVE // KNOWN [underlined] // D. ROCHFORD // DANGER KEEP OUT [underlined] // PrIVATE"; extensive inscriptions and attachments on 83 pages and on inside front and back boards.') Anti-child-labor crusaders surreptitiously photographed child laborers at work. And so on. Great exhibit.

In related news, I hit the Met again today and caught the next amazing photograph exhibit: "Night Vision: Photography After Dark". Since these photos weren't taken 100 years ago, they're not public domain, and a lot of the good ones are NO IMAGE AVAILABLE on the Met's website, but here are a couple good ones: "Broadway at Night", this odd untitled bit of meta-voyeurism.

"Night Vision" goes away next week but it's likely to be replaced by something else awesome. I'm not really into photography as an art medium, but whoever puts together these exhibitions is doing a really good job of grabbing my attention. Good job, anonymous curator.

Papers!: Recently I read some academic papers that were comprehensible and interesting, and since I'm not a professional academic, that's really all I need out of such things. Check 'em out.

First, "It was twenty years ago today", Paul Ginsparg's history of arXiv.org, which includes some history of the Web from the perspective of the community for which it was invented:

In early 1994, I happened to serve on a committee advising the APS about putting Physical Review Letters online. I suggested that a Web interface along the lines of the xxx.lanl.gov prototype might be a good way for the APS to disseminate its documents. A response came back from another committee member: “Installing and learning to use a WorldWideWeb browser is a complicated and difficult task — we can’t possibly expect this of the average physicist.” So the APS went with a different (and short-lived) platform. Meanwhile, the CERN website had partitioned its linked list of ‘all the web servers in the world’ into geographic regions, as if keeping such lists could still be a sensible methodology for navigating information.

Second, Scott Aaronson's tour-de-force, "Why Philosophers Should Care About Computational Complexity". Sample punch to the jaw:

Suppose we want to claim, for example, that a computation that plays chess is “equivalent” to some other computation that simulates a waterfall. Then our claim is only non-vacuous if it’s possible to exhibit the equivalence (i.e., give the reductions) within a model of computation that isn’t itself powerful enough to solve the chess or waterfall problems.

In other highbrow news, right now I'm reading a draft of Mike Amundsen's Building Hypermedia APIs with HTML5 and Node, and it's pretty awesome.

Indefinite Period of Kickstarter: There are still four unfinalized projects from my Month of Kickstarter project, so I won't be posting an analysis for a few days. I will say that it doesn't look good for three of those four projects, including one ("London in the 1980s") for which I'm still the only backer.

But in the month and a half since MoK I've still been checking that damn "recent projects" page every day and putting money towards projects. I haven't been doing two projects every day, or posting about it, but I did recently pass 100 projects backed (not counting the five that have been cancelled), so I thought it was time for an update.

One thing I did learn from MoK is that my linking to a Kickstarter project doesn't do a whole lot for its success, but I'm always happy to do what I can. So here's a few of the projects I've backed recently that I think you'd be interested in:

In other news, recently I was contacted by Larry Hastings, who wanted to start a Python podcast called "Radio Free Python". He searched the proverbial net to see if anyone had used that phrase before, and it turned out I had, in the examples for a presentation I gave at PyCon in 2003.

Displaying a cravenness some might consider unseemly, Larry offered to pay me a nominal fee for free-and-clear use of the name. Since "Radio Free Python" was a random phrase I came up with eight years ago to populate a sample HTML page, I rejected his money as syntactically invalid, and said he should back a Kickstarter project instead, telling me which one he backed. Well, he backed two: "Carnival - A deck and dice game." and the not-as-expensive-as-putting-a-hyperspectral-sensor-on-the-ISS "InTanj - A new software development methodology".

And the story has a happy ending, as the real-life Radio Free Python is up and running.

That's it for this edition of Indefinite Period of Kickstarter. As always, you can check out the Kickstarter projects I've backed, and the projects I'm interested in but probably won't back unless there's a chance I can push them over the edge.

[Comments] (5) robotfindspainting: Beth with her robotfindskitten painting Two exciting pieces of news from the long-dormant world of kittens and the robots who find them. First, Star Simpson has ported rfk to jQuery, proving that robotfindskitten remains relevant in today's world of Javascript frameworks.

Second, as you can see on the right, Beth Lerman painted a robotfindskitten still life! Here's a better picture on Beth's art site.

The still life has been in progress for a while. Beth and I collaborated to find NKIs that would work well in a still life:

(Punch bowl and punch not pictured.)

Beth presented me with the finished painting Monday night in a gala unveiling at my apartment, and it's now being classy in the living room. Speaking of classy, be sure to visit Beth's Etsy store.

[Comments] (6) Month Of Kickstarter After-Action Report: Crispy Duck Games Here you see the logo for Crispy Duck Games, one of the fictional studios in my novel Constellation Games (coming soon!). It's the first reward I've gotten out of July's Month of Kickstarter project. (Thanks, Brandon Eck; I didn't tell you anything except the name of the company, but you came up with something funny and appropriate.) Now that all the Kickstarter projects I backed have passed their deadlines, I'm going to take a graph-filled look at how they did, and take a few tiny stabs at guidelines for future projects that use Kickstarter or the Street Performer Protocol more generally.

Success rate

This is not a random sample of Kickstarter projects. I picked projects I actually wanted to back.

In all, I backed 52 projects for MoK. 31 of them succeeded, 19 failed, and two were canceled. My success rate was 60%, versus 40-45% for Kickstarter projects as a whole.

I think this shows good judgment on my part: I usually backed projects the day they started, and I avoided projects like MC Frontalot's music video that were obviously going to make their goal (or I backed them and didn't count them towards MoK.)

I don't have any information about the canceled projects, but I gathered basic information about the other 50: their funding goal, how much money they actually raised, how many backers they had, and how many updates their founders posted. This information became the graphs you're about to see.

I pledged at least $25 to each project, and more if necessary to get a cool reward, so my total expenditure was at least $725, and probably closer to $900.

This graph shows how much money each project raised. The line is the success line: projects on or above the line met their goals, and projects below the line failed.

Success rate graph

Let's zoom in on the projects that failed:

Failure graph

Check that out. Most failed projects raised pretty much nothing. But many raised thousands of dollars but didn't get any of it. If the cheese vat project had asked for $3k instead of $20k, they could have made their goal, and still been able to produce all their prizes. They wouldn't have gotten a new cheese vat, but $3k is better than nothing.

Why do projects fail?

I looked at the failed projects and came up with four classes of failure. These are my subjective opinions about what went wrong with the projects.

  1. About 30% of failed projects didn't hustle. The project creator put up a project on Kickstarter in the belief that hundreds of people would come by and back their project. Instead, they got me, the guy who looks at every single project despite the fact that Kickstarter really doesn't make it easy to do this. (eg. "50's Monster Movie Serial!")
  2. Of course, hustling is no guarantee of success. For about 30% of the projects, the project creator hustled, and got money, but not as much as they asked for. They should have used a different rewards system, or assuming they could have delivered the existing rewards with less money, they should have asked for less. (eg. the "Bursts of Light" anthology.)
  3. About 30% of failed projects clearly had both problems: they didn't hustle and they asked for way too much money. (eg. the Thousand Island dressing documentary).
  4. About 10% of projects hustled towards a reasonable goal, but didn't make it because the project or the rewards were too niche. I think the best example is the oboe chamber music recital, which offered oboe reeds or oboe lessons at the $25 level.

Not shown: a much larger population of failed projects that I didn't back, which may have failed for other reasons.

What is the hustle?

How can I judge projects based on a vague quantity like "hustle"? I'm using the number of updates posted to the Kickstarter project as a very weak indicator of hustle. Here's a graph of updates versus backers, for all projects.

Attempting to measure "hustle"

Kickstarter updates do not cause backers, if only because nobody but existing backers cares about your updates. But in my dataset, no project with zero updates ever got more than 50 backers. Updates and backers are both signs of this invisible third thing, "hustle". Updates are a good indicator that the project founder is hustling. Many update messages are exhortations to existing backers, asking them to propagate the project through their social networks.

It's quite possible to hustle and fail anyway. But if you don't find yourself writing some updates, it's a sign of a problem with your strategy.

Mean contribution

Here's a big difference between successful projects and failed projects. The mean mean contribution to a successful project is $76. That is, dividing the number of contributors by the money raised gives a certain number, and the mean of those numbers is $76. The median mean is $66. The standard deviation of the means is a huge $39, but I don't know if that has any statistical meaning.

(CAUTION: Sumana finds these graphs confusing, because the x-axis doesn't mean anything. It's just all the projects lined up next to each other. But I couldn't think of a better way to present this information.)

Mean contribution (successful projects)

The mean mean contribution to a failed project is $43 (the median mean contribution is $38, the standard deviation of the mean a somewhat smaller $23).

Mean contribution (failed projects)

I was pretty shocked about this. Even the numbers for failed projects greatly overshadow the $25 I usually kicked in (and still usually kick in). However, for a lot of board game projects, I contributed $40 or $50, because that was how much you had to contribute to get a copy of the game, and all those projects succeeded.

Takeaway lessons

  1. Start the really good rewards at around $50. (You can go a little lower if you're doing a book.)
  2. Try to get people who put a lot of money into a few Kickstarter projects, rather than people like me who spread it around.
  3. If you're not sure how much you can raise, try something in the $1k-$2k range.
  4. Hustle, dammit.

What's next

I'm a little loath to do this because it means a bunch more data entry, but I want to take a closer look at at least one of these projects, to figure out what "hustle" looks like in more detail. There's one MoK project that for me just defines "hustle", a project that reached a really high goal with a mean backer contribution of only $23. I figure that's a good one to investigate more closely. So stay tuned, or not.

[Comments] (3) Month Of Kickstarter After-Action Report 2: Details: I decided to take a detailed look at the tesselated cookie cutter project. I chose this one for a couple reasons. First, the project succeeded, so I'm not rubbing it in when I suggest things that could have been done better. Second, a lot of interesting things happened over the course of this project. Third, the product is cool.

I scraped the list of project backers to make this graph of backers over time:

Backers over time

The red diamonds single out days when the Keith Kritselis, the project owner, did an update.

This project was very popular among people who don't use Kickstarter. This pie chart shows how many other Kickstarter projects this project's backers (including me) have backed:

Other projects backed

Half of the people who backed this project signed up for Kickstarter just for this project, and never used Kickstarter again. These are the numbers as of today, so someone who joined Kickstarter because of this project but then went on to back another project would show up in the "1" slice of the pie, not the "None" slice.

Despite the broad appeal, the main reason this project succeeded was that something really anomalous happened on the last day. According to this update, what happened was Stephanie Nelson, who has a huge following as "The Coupon Mom", posted about the project on Facebook. This brought in a huge number of new Kickstarter users.

Here's another pie chart that looks at the Kickstarter experience of people who backed the cookie cutter project on the last day. 77% of those people signed up for Kickstarter just to back this project and never backed another project.

Other projects backed (last day only)

If you go to the bit.ly statistics page for the Kickstarter page's shortened link, you'll see the effect of Stephanie Nelson's Facebook post there as well. The vast majority of clicks on that shortened link happened on August 16.

This is not the first time the project had gotten a lot of publicity. Update #3 shows that in early August, the project got a write-up in the local newsweekly, the Austin Chronicle. But that only brought in ten new backers, probably because you'd have to type in a long URL.

Keith Kritselis hustled quite a bit on this project, both online and off; you can see this by looking at the content of the updates, and clever Kickstarter hacks like this:

Spread the word.

It's also clear that Kritselis's hustling worked: the project was funded. But it would have come up about $1500 short if the hustle hadn't grabbed the attention of one person with a huge preexisting audience. It's not smart to risk the success of your project on being able to find that person (unless you are that person).

Obvious equation and nonobvious corollaries

Your project will succeed if B*Cm>=G. B is the number of backers, Cm is the mean contribution, and G is the amount set as the goal. (British newspapers, feel free to use this section for meaningless filler.) You have control over all three factors. You set G directly, you increase B by hustling, and Cm depends on how you've set up the rewards system.

How big can B possibly get? Here's the number of backers for each successful Month of Kickstarter project:

Number of contributors for each successful project.

Obviously this varies widely based on your hustle and the appeal of your project, but realistically you're not going to get more than 350 backers, and 150 is more likely. The project all the way to the right on that graph is The Endangered Alphabets Project, which got a huge amount of press coverage from mainstream sources including the New York Times. That project has 533 backers. If your plan assumes you'll get 1000 backers, your plan is probably wrong.

Because your number of backers is going to be pretty low, you need to make each backer count. That means raising the mean contribution or lowering the goal.

Recall from last time that the mean contribution for a successful MoK project was $76, plus or minus $39. The best solution is to have a product that's worth $50-$100, and to make a $50-$100 tier that is the first one with a really cool reward. This, I think, is why board games do so well on Kickstarter.

If you don't have something that's worth $50, get that tier up as high as you can. Don't undersell your product! It will doom your project! It very nearly doomed the cookie cutter project.

Let me show you what I mean. Here's the data on the pledge levels and how popular they were among the 317 backers of this project:

The $5 is the first pledge level where you get something real in return: one set of the cookie cutters. At $10 you get two sets, and at $20 you get four sets. (The retail price of one set of cutters is given as $12.) These three levels account for 85% of the backers.

These prices are way too low. (Again, I say this knowing that the project was funded--your project might not be so lucky.) To fund this project using only the $5-$10-$20 funding levels, you'd need 550 backers. It needs to be more like $10-$20-$40. This is the part that I don't have data on, but I'm willing to bet that getting someone to spend $10 instead of $5 on your $12 cookie cutters is easy, compared to selling them on your cookie cutters in the first place.

BTW, if you sum up all the backers at the listed pledge levels, you get a total of $5800 raised—two hundred dollars short of the goal. So even with all the hustling and the big last-day influx and everything, the project succeeded because people pledged more money than was strictly necessary to get the corresponding rewards.

In short, the tesselated cookie cutter story is a story of bad incentive design overcome by hard work, generosity, luck, and network effects. But do some work up front and you won't have to rely so much on that stuff. Look at other projects in the same space and see how they succeeded or failed. Look at the tiers they set up, see how many people pledged at each level, see how much money they actually raised and where it came from. A cool video can get people wanting to back your project, but the reward tiers and the goal you set will determine how much money you see.

RFK Update Redux: Two more exciting bits of rfk news bring life to this seemingly-abandoned weblog. First, you can now get prints of Beth Lerman's rfk painting! You can order a print online at Imagekind, where everything is 20% off until Friday, according to the big banner at the top of every page. Also of interest to NYCB readers: Beth's Magritte 95 painting and... most of her prints, actually.

If you live in the SF bay area, you can also buy the prints in person. Bay Homes and Linens in San Mateo stocks prints that have been color-corrected by the artist.

Second, I discovered another robotfindskittenlike game via clickolinko: Don't Find The Kitty. It's the anti-rfk, in which you lose when you find the kitten. I haven't played it, so I'm not sure if it's possible to not lose.

[Comments] (3) Deck-Building Wargame: I was talking to Pat about the hot new deck-building wargame A Few Acres Of Snow (great game title, BTW). I was at a conversational disadvantage since I've never played the game and didn't know anything about it other than "deck-building wargame". For discussion purposes I made up a game in my head that had nothing in common with the real game, and Pat said "that's totally wrong." BUT, I think the game I made up could be a really good game. All it requires is months of work and fine-tuning which I don't have time to do:

Picture a card-driven wargame like your Memoir '44. You have units on the board, and you draw cards that tell you which units you can move. Now picture Dominion, the original deck-building game, in which you buy cards that go into an endlessly recycled, ever-expanding deck. Now... imagine that your Dominion deck contains the orders for the units on your Memoir '44 board.

Now you have the war-torn chaos of Memoir '44, in which you can't fire your artillery because you don't have the card that lets you give the order, mitigated by the strategic buying of Dominion, with which you can choose to stock up on "fire the artillery" at the expense of other orders.

Units and orders are both for sale. Maybe orders are bought with "money" cards like Dominion, while units are put onto the board with special "deploy" cards. Maybe when you buy a unit you also get two or three different order cards for your deck. I don't know; months of work go here. But it would be a fun game, and hopefully someone's already done this, or will soon.

Bonus Mashup: spice up your Memoir '44 game by introducing the drafting mechanic from Seven Wonders. Pass two hands of seven cards back and forth, replenishing when you run out. Since this doesn't require months of work, Pat and I will actually be trying this, and I'll let you know how it goes.

Sycorax: Constellation Games begins serialization later this month (preorders open soon!), and I've working on the auxiliary material that will ensure a fun experience for subscribers. Things like bonus stories, and the Twitter feeds that will play out as the serialization progresses.

In general, a chapter of Constellation Games covers 5-7 days of story time. So... I can script those characters' Twitter lives ahead of time, and then enact them during the week, so that they talk to each other in a way that looks natural. If one day a character stays up late, they'll post to Twitter late at night and then go silent until early afternoon. What could be simpler?

Except, when you search for software to automatically post to Twitter on a schedule, you find software that is designed to make your personal Twitter account look like a robot is behind it, timing your tweets for optimum "penetration" into your "social network" in the hopes of "going viral", all at the cost of your "personal agency". Talk about the ELIZA effect!

So, I wrote my own client, Sycorax. You write a script with approximate timings in a text file, and Sycorax a) time-codes everything in a way that looks natural, and b) enacts the script on Twitter. I wrote more documentation and made the usage more friendly than was necessary for my personal use, in hopes that you'll find a use for it.

(My preferred name for this program was "Prospero", for reasons that will become clear, but Twitter said that Prospero was taken, so I went with the other magician from The Tempest.)

Sycorax is written in Python and published under a BSD-style license.

[Comments] (1) Game Roundup: Minecraft Edition: When I was a teenager the hot game in my BBS circle was ZZT, a top-down game about running around a grid and collecting things. ZZT came with an editor that let you fill up the grid with characters from the IBM extended ASCII character set. Some characters had special meaning to the game: π was a tiger which, like the tigers in In Watermelon Sugar, would hunt you down and eat you. (Also like real tigers, I guess.) ⇿ was a slider, which could be pushed left or right, but would never move up or down. And so on.

The hot game in my circle today is Minecraft, a 3D game about running around a grid and collecting things. I played the beta a while ago and exhausted the game's non-obsessive-compulsive possibilities, but I picked it back up when I discovered that people have been making and sharing custom Minecraft maps in a way that strongly reminds me of the ZZT days. In honor of the impending release of Minecraft 1.0, I present an overview of the custom map scene, with links to my favorites and commentary from a game-design perspective.

I got all these maps from the Minecraft forums. If you go there you'll see lots of people posting maps, advertising in the title the number of downloads each has received and the genre of the map. ZZT's game mechanics gave rise to certain genres: RPG, slider puzzle, the nebulous "adventure", and so on. Minecraft's mechanics have given rise to significantly different genres:

Survival: The genre that's the name of one of Minecraft's game modes. In this genre, you're put into a situation and you have to do... whatever you want. It's like playing Minecraft on someone else's map. Usually they've constructed some cool things for you to explore, or constrained your access to resources in some way, or at least created some arbitrary challenges like "build 64 bookcases". For me, all the memorable instances of this genre fall into the subgenre of:

Survival puzzle. (My term, it's lumped in with [SURV] on the forums.) In vanilla Minecraft, you have easy access to unlimited amounts of dirt, grass, water, wood, and stone, which can be crafted into unlimited amounts of food and basic tools. In a survival puzzle map, severe constraints are put on these inputs such that you have to figure out unusual ways of reaching certain desirable spots in the game world (cool-looking cave) or in the crafting graph (useful pickaxe).

The best pure survival puzzle map is Skyblock. ("500,000+ DOWNLOADS") Skyblock dominates this genre to such an extent that any new entry in the genre will be derided as a Skyblock ripoff. That's because it's very difficult to come up with a new survival puzzle. Survival puzzles depend very heavily on the implementation of Minecraft: facts about what inputs you can craft into what outputs, and facts about the environment. Facts that you can't change without modding. This is where the differences between Minecraft and ZZT begin to emerge.

Your best bet is to either come up with a totally new survival puzzle, or to combine the puzzles popularized in Skyblock with some fun pure-survival content. The Pit does the former, and the Complete The Monument genre (q.v.) does the latter.

Adventure: Unlike with ZZT (and games in general), the "Adventure" genre is very clearly defined. In an Adventure map, you have to go from one place to another, and you are not allowed to place or break blocks. In other words, the game stops being Minecraft, and becomes a 3D platformer with awkward controls and blocky graphics.

I've played a couple Adventure maps that showed off really impressive designs, notably Deep Space Turtle Chase (requires a mod that remakes Minecraft as a world made of spaceship components) and this Indiana Jones trapfest where you should probably just watch the video. But prohibiting the game mechanics that Minecraft does really well, and relying solely on its mediocre platforming, is a recipe for boredom. Specifically, my boredom.

There's a subgenre of Adventure called Parkour, which I haven't even tried because it requires very precise jumps, and I can't do that. Parkour is popular enough to rate a top-level genre on the forums, but it operates under the same constraints as Adventure, because if you could place blocks there'd be no need for very precise jumps.

Complete The Monument: My favorite genre, pioneered by the excellent Super Hostile series, and spread by the fact that the Super Hostile author put up a "toolkit" full of useful idioms to copy-and-paste into your own CTM map.

Basically, CTM turns Minecraft into Zelda. To win the game you need to collect a bunch of identical things (blocks of different-colored wool, generally). You explore an overworld until you find some dungeons, and at the end of each dungeon there's one of the identical things. If you're smart, while you're in the dungeon you can usually score an upgrade to your equipment (the Survival Puzzle aspect).

Turning Minecraft into Zelda works great! It leaves the building/crafting mechanics alone, it gives some goal-direction to the aimlessness of vanilla Minecraft, and it gives the creator a chance to show off some cool designs. Unlike Zelda, CTM games tend to be brutally difficult (it's called Super Hostile for a reason), but a little strategic cheating always works if the fun starts to fade. In addition to Super Hostile I've enjoyed the Twisted Logic series, the Forgotten series, the Corrupt Lands series, and Timetoslide's maps. See? Lots of series, just like Zelda.

If I were twenty years younger I'd be making Minecraft maps right now. Instead I'm the age I am, but I did have an idea for an arthouse Minecraft map: "Return To The Town Of ZZT". I'd build ZZT screens in Minecraft, making those stereotypical ANSI-yellow borders out of glowstone. I'd fill those screens with all the awful cliches of ZZT design: the broken slider puzzles, the "X of ZZT" naming convention, the talking trees. And it wouldn't matter because everything would be one block tall and you could just jump over the obstacles.

Except... I don't think the awful cliches of ZZT design can even be implemented in Minecraft. ZZT had a simple object-oriented programming language. In Minecraft if you want a programmable computer you have to lay out the circuits yourself. ZZT made it easy to display messages to the player. The Minecraft maps I linked to above are covered in little signs, 60 characters to a sign, worse than Twitter. A long message will be stuck on a wall with a grid of 9 or 16 signs!

The ZZT level editor was designed to make games to share with people, and Minecraft is designed to make an environment to walk around and enjoy. The genius of Minecraft is that the level editor is the game. But writing code isn't really a game, it's work. So instead of programming stupid games with ZZT, kids are building roller coasters and pixel art and Skyblock ripoffs with Minecraft. Clearly it's not Minecraft's job to be a programming environment, but this feels like a step backwards.

I'm giving some thought to minimal additions to Minecraft that would give players more scope for programmer-style creativity, without changing the nature of the game. I'll post my ideas in a separate entry.

Constellation Games: Subscribe!: Starting today, you can buy a subscription to the serialization of Constellation Games, my space opera novel about video games. Here's the blurb:

First contact isn't all fun and games.

Ariel Blum is pushing thirty and doesn't have much to show for it. His computer programming skills are producing nothing but pony-themed video games for little girls. His love life is a slow-motion train wreck, and whenever he tries to make something of his life, he finds himself back on the couch, replaying the games of his youth.

Then the aliens show up.

Out of the sky comes the Constellation: a swarm of anarchist anthropologists, exploring our seas, cataloguing our plants, editing our wikis and eating our Twinkies. No one knows how to respond--except for nerds like Ariel who've been reading, role-playing and wargaming first-contact scenarios their entire lives. Ariel sees the aliens' computers, and he knows that wherever there are computers, there are video games.

Ariel just wants to start a business translating alien games so they can be played on human computers. But a simple cultural exchange turns up ancient secrets, government conspiracies, and unconventional anthropology techniques that threaten humanity as we know it. If Ariel wants his species to have a future, he's going to have to take the step that nothing on Earth could make him take.

He'll have to grow up.

The ESRB would rate Constellation Games M for Mature. It contains strong language, sexual content, alcohol and drug references, comic mischief, and cartoon violence. It's a little bit "Mallory" and a little bit "Let Us Now Praise Awesome Dinosaurs".

Interested? $5 gets you the story: one chapter a week, delivered electronically, starting November 29th. You'll also be able to follow along with the Twitter feeds, as the characters live the events of the novel. (You can follow along with the Twitter feeds anyway, but they might not make much sense.)

Pay more than $5 and you start getting bonuses. Seven bucks gets you the serialization, plus a bonus short story afterwards. $20 gets you three bonus stories, and a hard copy of the trade paperback, and a phrasebook for the Pey Shkoy language Adam Parrish created just for Constellation Games. And so on. There is a ton of extra stuff available, with more on the way. Read the first two chapters now (requires Flash), pick a subscription level, and let's get this started.

(n.b. You might notice a few artifacts in the cover image. All but one of the artifacts will be gone in the final version.)

[Comments] (1) Programmable Minecraft 1: Circuit Layout: I had two distinct kinds of ideas about making Minecraft more programmable, so I'm going to do two posts. (The end of this post explains why I was thinking about this.) This first post is all about circuit design. I'm not trying to get to Python or even ZZT-OOP here, just trying to pull Minecraft up from the "do your own wiring" level without betraying its aesthetic. I don't really build mechanisms in Minecraft because the primitives are so primitive. I got plenty of that in college. I suspect other people are in the same boat, or at least other boats in the same flotilla.

If you're really into redstone circuits then any of this stuff may, for you, betray the Minecraft aesthetic. If so, take heart, for I am not a Minecraft developer and I doubt anything like this will ever be implemented except in a mod, for performance reasons if nothing else. (Here are some mods with blocks for logic gates and basic digital functions.)

My second post will be about ways to programatically interact with the environment, and I think even purists will be able to appreciate that. For now, here are ideas for making circuit layout easier.

Infraredstone repeater: When this block receives a redstone or infraredstone signal on one side, it sends out an infraredstone signal on all other sides. Infraredstone works like redstone, but it's a beam that operates across line-of-sight, rather than a current through a wire. This makes wiring easier and can also be used to make electric eyes, since an intervening mob/opaque block will block the signal.

Logic gates: Don't make players build their own logic gates out of redstone and torches. Just provide them ready-made. Logic gates don't have to be ugly boxes labelled "AND" and "XOR". I really like how a redstone torch acts as a NOT gate. You could add objects to the game that happen to act as logic gates if you hook them up correctly. For instance, the infraredstone repeater, as described above, acts as an OR gate.

Data: Now it's gonna get heavy. I want you to imagine that data itself is an object in Minecraft. You can carry it around and put it in chests and hold it in your hand and dig holes in the dirt with it. Data has no use, but unlike other objects, which are stackable up to 64, data is stackable up to 255. This lets you carry around an eight-bit value in one slot.

How do you get data objects? One way is to use a:

Display: like a tiny chest for data. It has space for one eight-bit value, and on all sides of the block it displays its current value in a big font (using IBM's CP437 character set, like ZZT). You can right-click a display and use keyboard input to set its value to any keyboard-enterable character. If you chain multiple displays together, you can type more than one character at a time, as you can when placing a sign.

A display has an "input" side and an "output" side. If you stick a redstone torch on the "input" side of a display, a single data object will appear inside it, it will start showing a ☺ (character 01 in CP437), and the "output" side will go live with a redstone signal.

But instead of the 1-bit signals of normal redstone, a display block sends and receives data through an 8-bit data bus. Basically I'm increasing the bandwidth of redstone from one bit to eight. Existing equipment such as redstone repeaters will work on an 8-bit signal just as they currently work on a 1-bit signal. Instead of "1", a redstone torch sends "00000001". If you type an "A" into a Display, a stack of 65 data objects will be placed in it, and its output side will read "01000001". By the same token, if you open up a Display and dump a stack of 65 data objects into it, it will start reading "A".

Now that we have redstone data buses, we can support some more interesting blocks:

Multiplexer: Takes two redstone bus signals: "input" and "select", and outputs a redstone bus signal. A single-block multiplexer is useless and never outputs anything, but if you chain two of them together you get a 1-bit multiplexer whose "select" chooses between two input signals based on its low bit. You can chain together up to 256 multiplexer blocks to use all eight bits of the "select" signal.

Similarly, you can chain together up to 256 Demultiplexer blocks to make a demux. I originally proposed a Register block, but the Display is almost a register already--it just needs a "set" line so it doesn't change whenever its "input" line changes.

Perhaps at this point, even non-purists are thinking, "Leonard, all these fancy blocks are spoiling my enjoyment of Minecraft! I like laying out complicated circuits in three dimensions so that I can make a frigging flip-flop! Well, I don't like that, exactly, but I do like having a relatively small number of core blocks, and I don't like where this Multiplexer/Demultiplexer/Register business is going!"

That's why I'd like to introduce you to The Item World. I learned of this insane concept when Dr. Aaditya Rangan showed me the Disgaea series of RPGs. In Disgaea, you can go through a dungeon and kill a demon and collect a sword, just like in any other RPG. But only in Disgaea do strong magics exist that let you go inside the sword, where you'll find another dungeon full of demons, which you can kill to level up the sword.

The Item World is a crazy time-sink in an RPG, but it's a really useful time-sink when you need to lay out circuits. The circuit layout program I used in college had an Item World: you could lay out a circuit with logic gates, then zoom out a level and treat that circuit as a tiny black box in a larger circuit. Minecraft could do this too. Let's introduce a block called the:

Computer. When you place this block and right-click it, you're sent into a translucent 15x15x15 room which you can decorate as you see fit. You can build sophisticated mechanisms inside the Computer block, but to the outside world it looks like a single-block black box.

The bottom four rows of one wall form the display. By putting Display blocks on this wall, you can achieve the same effect as putting a sign on a block. Only here, the message on the sign can be dynamic. This lets you do display output and keyboard input.

Opposite the display wall is the input wall. If you send an 8-bit redstone signal to the side of the Computer block opposite the display, then inside the Computer, every block on the input wall will go live with that signal. Every other wall of the block is an output wall. An 8-bit signal sent to any block of that wall will leave the Computer and be emitted by the corresponding side of the Computer block. If you send more than one signal to an output wall, they get ANDed.

With Computer blocks you can implement a logic gate, a multiplexer, a demultiplexer, or a register in a single block, without adding any code to Minecraft itself. Of course, you can put one Computer block inside another. Since this will not actually be implemented, let's suppose you can nest Computer blocks to arbitrary depth.

Building a sophisticated system will still be a huge pain if you have to craft every AND gate by hand. So I'll also introduce a new crafting block, the:

Replicator. This device takes raw materials and produces a copy of its input. Non-redstone example: I want to make a torch. I drop an existing torch into the Replicator. It says: "Gimme 1 stick and 1 coal." I drop a stick and a coal into the Replicator, and it spits out a torch. That's stupid, but you get the idea. Now say I spent five hours building and debugging a Computer block that acts as a shift register. I want another shift register. I drop my Computer into the Replicator. It says "Gimme 861 redstone, 201 sticks, 52 wooden planks, 8 stone, 29 cobblestone, etc. etc. etc." I dump all that stuff in and it gives me another shift register.

OK, you get the idea. The next post in this series will take a much different tactic. I'll accept that circuit layout isn't going to get easier, and suggest some blocks that can be used to make fun maps that aren't currently possible.

[Part 2]

From Hiroshima to the Moon: At a flea market Sumana got me a really good book: From Hiroshima to the Moon: Chronicles of Life in the Atomic Age by Daniel Lang. It was published in 1959, so the "Moon" in the title is purely aspirational--nothing in this book even gets close to the moon. But Philip Morrison does go to Hiroshima, a month after the bomb is dropped, and Lang gets him to tell the story of that initial fact-finding mission.

The commander started things off by gravely explaining that he had asked one of his aides, a young major, to tell the Americans the "facts of the disaster." "They always referred to it as 'the disaster,'" Morrison said. "It made me feel as though I were a member of an earthquake commission."

"A Fine Moral Point" is one of many amazing essays in the book, all written for magazines in the 1940s and 50s, when the world was just starting to measure the effects of fallout, learning how to treat the diseases caused by radiation, and coming to terms with the inevitable destruction of civilization. As Lloyd Smith says in one of the essays, "If war should come, we scientists might die the same as anyone else, but at least we would understand exactly what was causing our death." It's a time that's obscured in hindsight by our view of the Cold War and the Apollo program, but there's a lot of interesting and/or horrifying stuff going on here.

Other books I've read about the atomic bomb focus heavily on the scientists who designed it. Lang has a lot of scientists, but he also writes a lot about their families, the workers at plants like Oak Ridge, and the soldiers who work with the bombs—the people who are buying the magazines.

In "What's Up There?" Lang visits the White Sands Proving Ground where they're strapping scientific instruments onto leftover Nazi V-2 rockets. Capt. Edward Detchmendy shows him around the base and introduces him to Lt. James Kincannon, the Recovery Officer. Once the rockets go up, Kincannon very much cares where they come down, because he has to go out in a Jeep and recover the instruments. Lang asks to tag along:

"Well," [Kincannon] said, "meet me outside the blockhouse right after the shoot. Don't be late. Look for the Monstrosity—that's the name of my jeep. Can't help spotting it. It'll be kind of loaded. I carry ten gallons of water for the vehicle, five for passengers. Also four quarts of canned oil. Tools for the vehicle and rocket extricators will be in it. It has two-way radio-communication equipment, a bedroll, and binoculars. I carry a first-aid kit and a snake-bite kit with serum. I always have my forty-five for snakes and mountain lions, and also for coyotes. And don't bring a pillow, the way some people have. That's sissy stuff." Kincannon abruptly proceeded into the blockhouse, and Detchmendy said to me, "Take along a pillow."

Apropos Werner von Braun and the other German scientists, Lang gets off this line:

They were originally signed on for one year at a small wage and six dollars a day for expenses. Since the expiration of the contract, however, they have come into substantial raises. It is probably the first time that the kidnapper has also paid the ransom.

In "Bombs Away!" (1952) Lang visits Yucca Flat for an Army field test involving a live bomb drop. It's like getting to see Dr. Strangelove twelve years early:

From [a helicopter] emerged a lieutenant general, who strode past us halfway up the knoll, turned around, and delivered a talk. He had been up forward with the troops, he said, and the boys had made jokes immediately before and after the explosion. The weapon, he declared, had to be regarded as so much firepower. From a tactical point of view, he went on clinically, the day's bomb had been too big, because it had prevented the troops from advancing quickly enough. "We learned in the war that you have to follow close behind your firepower to capture your objective," he said.


Of the various officials I talked with during the tours and between lectures, none awaited the impending test more eagerly than the civil-defense people. "We're counting heavily on this bomb," one of them told me. "It's a tough job selling accident insurance."


Eight hours after the detonation, a hundred troops who had been in the foxholes and trenches that morning were marched into the City Hall auditorium. They were ranged against the walls in groups, by states, for the convenience of newspapermen interested in local stories. Almost instantly, the barnlike structure was alive with the din of feature stories. I wandered down one of the aisles, listening to snatches of the interviews, and found that the atomic G.I. sounded very much like his counterpart of a few years ago. An Arizona boy had prayed. A chipper California man said that he'd take the atomic bomb any day over those German 88s he'd known in Sicily. An Illinois corporal said that he'd drawn a stranger as his foxhole mate, but that after the hot earthquake they'd experienced together he was sure they'd be buddies for life. A very young blond New York City corporal wanted the reporter talking to him to do him a favor. "My name is Geiger, Vincent Geiger," he said, "and all the fellows in my company keep asking me if my father's the guy who invented that counter. I would appreciate it if you wrote that he isn't." The most hopeful, though unconsciously hopeful, words I heard were uttered by a New Mexican, an earnest, swarthy private first class named Evaristo Hernandez. "I passed up my furlough to be in on this test," he told his interviewer. "I figured I might never have another chance to see an atom bomb."

I could just go on and on. The book ends with a story on the impending launch of the first Vanguard rocket, a postmortem on the failure of said launch, and the super-speculative early phases (1958) of "what is known to researchers, in and out of the U.S. government, as 'the man-in-space program.'" Apparently the word "astronaut" hasn't yet found currency outside the pages of Astounding, because the still-hypothetical astronaut is always referred to by ominous terms like "the space man" or "the space traveller".

Thanks, GameSetWatch: As this GameSetWatch post implies, Constellation Games was triggered by, and originally written for serialization on, that site. Here's what happened: at the end of 2008 Simon Carless emailed me and said (paraphrase) "Hey, I liked 'Mallory', would you like to write some kind of game-related serial on GSW?" Later that day I sent him a pitch that still describes the novel pretty well. Turnaround was quick because I took the setting wholesale from my 2007 novella "Vanilla", which now needs to be rewritten because I completely cannibalized it for the novel.

Writing the book was not quick at all. The original plan was to build up a backlog and then start the serialization, the way webcomics work. I did about a quarter-draft that was awful, and then I had a conversation with some published novelists in which the novelists were unanimous that they would never start serializing something without having at least a complete draft. I started over and told Simon I'd like to finish a draft before we serialized.

Fortunately, the second draft was much better, I finished it by the end of 2010, and we decided it would be cool to try and find a real publisher for the novel. (Serializing the novel on GSW would pay me as a columnist, which--spoiler alert--is not very much.) I was apprehensive about chasing this particular carrot, anticipating years of bitter failure and publishing-industry drama, but I was able to get Candlemark & Gleam interested quickly, and the rest of the process has moved very smoothly.

So, thanks, Simon and GSW, for giving me the push I needed to write something novel-length. I'm also happy that on GSW I will forever be "Leonard Richardson (robotfindskitten)". It's like the way the Columbia alumni magazine refers to "Barack Obama ('83)".

Sycorax 1.9.1: The link to the initial release of Sycorax was broken, and although several people tried to download it, nobody was distressed enough by its unavailability to complain. However, this will change once the CG Twitter feeds start up and you see how useful Sycorax is for making fictional characters bitch at each other on Twitter. Then, you'll download my software!

Oh, uh, the point of this post is that a new version of Sycorax is out. This one has sanity checks like "is a tweet longer than 140 characters?" and "do a chapter's tweets overflow their alloted time?". This version also has a script that makes it easy to get OAuth access tokens for your characters. Plus, I checked that the link works.

Haul Of Kickstarter: Here's a photo of all the physical objects I've received because of Month of Kickstarter:

It's been almost four months, and I frankly expected a slightly bigger pile by now. It's possible that some of the projects asked for my address in an update rather than with a survey (please don't do this, folks--I'm getting 5-10 project updates a day, and I don't read them all), but the big items are board games, which just take a long time to produce, so I don't feel bad. I took this picture now instead of waiting for a bigger pile of stuff, because I have a feeling that pretty soon that bar of Firebird Chocolate is going to go "live on a farm" with the other three bars I was sent.

"The Day Alan Turing Came Out": One week from today serialization of Constellation Games begins. Today, as a pre-Thanksgiving promo, I'm putting online a story I sold to the Retro Spec anthology in 2010.

"The Day Alan Turing Came Out" is short and bittersweet. I've licensed it under CC-BY, the most permissive Creative Commons license. It doesn't have much in common with Constellation Games, but I've been wanting to put it online for a while, and this seems like a good time.

Retro Spec has lots of other F/SF set in the past (one of my favorite subgenres), so check it out.

Pey Shkoy Benefits Humans: If you're on the fence about subscribing to Constellation Games because you only want the trade paperback when it comes out next year, get off that fence! The trade paperback will cost $20 on its own, so spend $20 on the Plutonium package and you'll get the trade paperback, the bonus stories, and today's promo spotlight: the Leonard Richardson/Adam Parrish joint, Pey Shkoy For Humans.

In Constellation Games, Pey Shkoy is the language of an ancient alien civilization that produced some amazing (and some really awful) computer games. Adam took the fragments of Pey Shkoy found in the book, mostly names of people and companies, and reconstructed the language around those fragments, allowing us to present a basic guide for game importers and Creative Anachronism types.

The book's native-language title is Uiksel Pey Shkoy A Human, lit. Pey Shkoy Benefits Humans. Here it is in Pey Shkoy syllabic script:

The book will have a complete syllabary, a guide for transliterating English, and translations of handy phrases like "My secondary sexual characteristics explode with delight!" I've got to be a little vague on the details, partly because Adam and I just started working on this, and partly because one of the best bits of the book is who (within the fiction) wrote it. The latter being a tricky selling point when you, the potential buyer, haven't met any of the characters yet.

BTW, you can also see Pey Shkoy script on the Constellation Games front cover (it says "Perea", the name of the device manufacturer), and there'll be quite a bit more on the back cover.

Scientific Dadaism: This was an interesting history paper: 'Reactionaries and Einstein's Fame: "German Scientists for the Preservation of Pure Science," Relativity, and the Bad Nauheim Meeting'. The story of a fascist anti-relativity astroturf organization:

Indeed, the theory of relativity was being “thrown to the masses” in exactly the same way as “the dadaist gentlemen” promoted their wares, which had to do as little as relativity with the observation of nature. No one should be surprised, therefore, Weyland concluded, that a movement had arisen to counter this “scientific Dadaism”.

Update: In other anti-Semitism news, I also found interesting this paper on Moscow State University's "Jewish [Math] Problems".

[Comments] (2) @ArielBlum: Tomorrow the first chapter of Constellation Games is pushed out to subscribers! It's a weird situation because you can already read that chapter. In fact, you can read the first two chapters, in a nice PDF or semi-nice HTML format that you can pass around to your friends (a la Tales of MU), instead of the clunky Flash interface on BookBuzzr.

Although the launch is tomorrow, the Twitter feeds start running today. Here's Ariel, the novel's narrator, on Twitter. Right now it's May 30 for Ariel and he's fixing last-minute bugs in a video game about ponies. Tomorrow is May 31, the first day of Constellation Games. On May 31 Ariel's world will change irrevocably, and you get to watch the entire process.

Every week, the Twitter feeds will enact the events of the most recent chapter, using the Twitter-language of jokey status updates. If you subscribe to the serial, you can read each new chapter and then watch its drama play out on Twitter over the week, in something like real time.

You don't need to follow the Twitter feeds to understand Constellation Games, but they add another layer of fun. They contain tons of subplots and details that aren't in the novel because everything in a novel has to be useful to the main plotline—the polar opposite of the Twitter philosophy. I'll be archiving the feeds on my forthcoming "episode guide" page, so if you come in late, you can read the tweets for a given chapter en masse.

I said "feeds", plural, but right now there's just the one. Another feed will rise in a few weeks, but its owner hasn't entered the solar system yet and won't get a Twitter account for several chapters. I have created Twitter accounts for Jenny and Bai, the two other major human characters, but I won't be posting to those feeds. I grabbed the names because Ariel @talks to them a lot, Bob Newhart style, and I don't want someone else hijacking them/taking them innocently and getting very confused. The feeds are fun to write and I'd love to do four of them, but I'd rather spend my time working on subscriber bonuses. (If Brendan or someone who's read the story wants to role-play Jenny and/or Bai on Twitter, I'll hand over the keys.)

BTW, I got the idea for the Twitter feeds from J. Jacques of Questionable Content, who runs feeds for all of his characters. That's a lot of characters, and any given character doesn't post very often, and now I understand why.

Sycorax 0.9.2: As predicted (privately to myself, not with a sealed envelope on TV or anything), I needed to do a new release of Sycorax the first day I started using it for real. The new version, 0.9.2, will not generate new times for tweets that have already been posted: it will reuse the original times as recorded in the progress log. I also added a sanity check that keeps you from putting so many tweets into one chapter that it bleeds over into the next chapter.

[Comments] (5) Constellation Games Author Commentary #1: "Terrain Deformation": Check it out, bronies. The first chapter of Constellation Games, "Terrain Deformation", is in subscribers' inboxes. That means it's time for bonus author commentary here on NYCB. Every Tuesday I'll put up a post like this, containing whatever I have to say about this week's chapter.

Even if you haven't subscribed to CG, you can read the first chapter for free and then comprehend this post. But this magic is subtle, and will only work twice. So get on the gravy train before it leaves the... gravy station, I suppose.

These commentary posts won't usually be very long, but they'll give me a space to say whatever piece I might have. The Chapter 1 commentary would be super long, except I'm going to stretch my general comments out over the first few weeks. You can ask me questions in the comments or on Twitter. I don't intend to spoil later chapters, but eventually the callbacks will start coming due and I'll start referencing the early chapters a lot. And of course I will spoil whatever chapter I'm talking about that week.

So, on with the commentary! This is the shortest chapter in the book--I'm not doing anything but setting up the premise--and it's almost all that remains of my initial draft. I had to get rid of that draft because I tried to advance the plot entirely through blog posts. It got quite complicated, and boring. But the first contact with the Constellation is a very intense, breaking-news situation in which Ariel is not directly involved, so blog posts work well here. From start to finish I can't think of anything major I changed in this chapter.

The Twitter feeds are a lot busier this week than they will be in the future, because of all the crazy stuff that's going on in the story world. Also because I wrote a lot of tweets for the first couple chapters and then decided to scale it back so as not to overwhelm readers/myself. Pretty soon it'll settle down to 2-5 tweets per real-world day.

This chapter introduces the recurring game reviews on Ariel's blog. My original plan was to have one review per chapter (remember, this was originally going to be serialized on a gaming website). This did not work! At all! Most of the time there was no way to tie a game review into the action of the novel. I refocused on the action and now there's a review every few chapters, each one (hopefully) interesting on its own but also earning its keep, the way scenes in a novel are supposed to.

This chapter introduces the two main human characters, Ariel Blum (whose blog/framing device we're reading) and his BFF from college, Jenny Gallegos. I'm generally going to let the characters speak for themselves, but I want to talk briefly about Ariel's name.

I think super-symbolic names like "Adam Truman" are stupid, but character names are important for mechanical reasons. There are a lot of characters in a story, and if you don't give them distinctive names, readers get confused. It's worse for people like me who are bad with names in real life. And a character's name is often the first thing I come up with, with everything else flowing from that. It's not their destiny, but the name often says something about their upbringing and their life so far.

This is the sense in which Ariel's name is important. He was named after a character from Shakespeare, and shortly after he was born, his name become irrevocably associated with a woman from a Disney cartoon. Only certain kinds of characters can come out of that mold. But if you want to get super-symbolic with Ariel's name, connecting it to The Tempest and The Producers or Ulysses... well, I can't stop you, and you might not even be disappointed.

I think that's enough for now. Tune in next Tuesday for Chapter 2, when Ariel will say: "Nobody eats my coleslaw and disses NASA."

PS: I've put up an episode guide at constellation.crummy.com. There you'll find links to these commentaries, the archived Twitter feeds, etc. etc.

Next week ->

Constellation Games Tech Support + Gift Subscriptions: Some have had problems buying Constellation Games, problems like "not being able to place an order" or "not being able to have books shipped to the UK". These problems should all be fixed, but let me know if you still have them.

In related news, several people asked to be able to buy gift subscriptions for their friends (one assumes), and now you can go ahead and buy those gift subscriptions. It's a little hacky--you're buying a gift certificate which the recipient redeems--but it should work. Again, let me know if you have any problems. And big thanks to editor Kate Sullivan for doing the server-side work.

[Comments] (1) CG Author Commentary #2: "Corner Pieces": It's that time of the week again, the time when the air is filled with the navel-gazing sounds of Constellation Games author commentary. You can read chapter 2 for free, but this is the last week. Subscribe some time this week, or the commentary on stuff you haven't read will become increasingly mysterious.

The microblog archive for chapter 1 is up for those who don't use Twitter. I don't know how readable all that text is, but I also don't know how to design a website anymore. Let me know if you have suggestions. Note that, for your convenience, the tweets are filed under their in-story date, not the real-world date.

I'm trying out a list format for commentary this week. It might look weird, but coming up with segues between completely unrelated things looks weirder.

There we go. Tune in next week for chapter 3, when Ariel will say, "So are you female right now or what?"

<- Last week | Next week ->

[Comments] (4) Programmable Minecraft 2: Redstone Boogaloo: I think my first post in this series left me too optimistic about how much you could really do for Minecraft map designers without making programming easier. I did come up with three useful blocks, but I'm gonna save them for a brief follow-up, and in this post I'll go ahead and sketch out a ZZT-OOP-like language for Minecraft.

My reasons are fourfold: 1) This level of abstraction is what I really want, not the circuit stuff. 2) it's simpler to interpret a simple scripting language than to simulate an equally complex circuit. 3) Apparently the next big feature in Minecraft is official mod support, which means someone could actually implement this. 4) I think it's more important that people learn how to program than that they learn how to lay out electrical circuits.

So instead of last time's circuit-level Computer block, imagine a software Object block. An object can take the appearance of any other Minecraft block, or it can have a custom bitmap that's loaded from a map-specific resource directory. An object also has a script, which is probably loaded from a file because I really don't want to use Minecraft as an IDE. The script is where the magic happens.

Here's a ZZT-OOP manual. (I think it's a copy of the original manual, it seems very familiar.) ZZT-OOP is extremely primitive, but because it's so primitive it can be explained in a weblog post, and I'm still familiar with it despite not having used it for fifteen years. So let me adapt it to Minecraft and we'll see what happens.

You program an object in ZZT-OOP by defining its response to stimuli. In Minecraft, stimuli might include:

Maybe more. You get the idea. These are the events in the life of an Object.

You can program an object's response to a stimulus as a series of commands. A lot of these can be taken straight from ZZT-OOP:

Here's an object that tries to move north whenever it's hit with a snowball:

:hit_with snowball

In ZZT-OOP, {direction} means one of: north, south, east, west, randomly, away from player, towards player, in the direction of current movement, etc. In Minecraft you also have up and down, as well as "along the minecart track", "towards the sun", and perhaps others. In a multiplayer scenario, "player" is ambiguous, so I guess it's the nearest visible player.

Some Minecraft-specific responses to stimuli:

Here's a perimeter trap.

:can_see zombie
"Zombie alert!"

Some bits of ZZT-OOP don't make sense in Minecraft because they assume that the world is relatively small and divided into screens. ZZT-OOP has global flags, and the only conditional you can write is "is this flag set"? I propose replacing the global flags with player flags. This lets you track a player's progress and simulate an RPG's "items too important to keep in your normal inventory". It also works in multiplayer.

Conditional statements: I'm undecided. I can think of three ways to go on this, but explaining them all seems boring. (I'm eliding boring stuff throughout, actually.) So for the time being, I'll follow ZZT-OOP, and allow an if statement to check player flags but nothing else.

Now it's time to talk about named stimuli. A named stimulus is one you give a custom name and send out whenever you want. Instead of giving an object code that's triggered by :can_see creeper, you can give it code that runs when some other object emits :i_am_completely_surrounded_by_obsidian.

In ZZT-OOP, the range of a message is the current screen. But Minecraft doesn't have screens. How far should a message travel? For that matter, on a large map with many objects, how many of those objects should be simulated? This wiki talk page indicates that a redstone circuit stops working if you get more than 281 blocks away. It seems fine to say that that's the maximum range of a message, and that objects further away from that won't have their code run.

Time to take a step back. ZZT-OOP is a very primitive language, but the system I laid out above is good enough to script objects that make adventure maps a lot more interesting. It's also useful when you're just playing around on a vanilla Minecraft map. You can program objects to build bridges, lay rail track, scout ahead to light up areas, or create computational sculpture.

To build one of these Object blocks outside creative mode, you'd need something like the Replicator block from last time. Call it the Assembler. You'd load code from a file on disk, and drop in an appropriate object for each line of code: grey wool for the stimulus :adjacent_to grey_wool, raw porkchop + gunpowder for the command spawn creeper. If your object was capable of destroying blocks, placing blocks, or exploding you'd also need to provide the appropriate tools or materials.

Once you put in the necessary items, the Assembler would output your Object, which you could place and it would go to work. The correspondence between resources and code opens up a whole new type of Minecraft puzzle: that of programming an object to do what you want, given limited resources.

So, there you go. I think this is pretty good. I just ripped off ZZT-OOP, but ZZT and Minecraft have a lot in common, so it works. Another option is to rip off Robotic, Megazeux's scripting language. I don't know Robotic, but it certainly has a more active community than ZZT-OOP. The language doesn't matter a whole lot. Really all you really need from the language is stimulus/response. The fun is in tying the language into the game world so that objects can act on it.

[Part 3]

[Comments] (2) CG Author Commentary #3: "Rare Drop": It's Tuesday, and time for the first subscriber-exclusive chapter of Constellation Games. (Check your spam folder.) Been holding out for a better deal? Here it is: for the rest of December, when you subscribe you get a gift subscription to give to a friend, so you'll have a speculation partner.

Here's the microblog archive for chapter 2, let's get cracking on chapter 3! Sumana requested appropriate pictures to break up the huge wall of commentary text, so I'll be picking some from Wikimedia Commons/Flickr every week. This week's pictures are by Mike Baird and NASA Langley.

That's plenty long for one week's worth of commentary. Tune in next week for chapter 4, when Jenny will say, "We're going to use as little alcohol as possible."

<- Last week | Next week ->

Programmable Minecraft 3: The Simplest Thing That Could Possibly Work: In the previous installment of this series I went whole hog and created a simple event-based scripting language for Minecraft objects. Left in the dust were three ideas I had for blocks that let you do more with your existing redstone circuits. These blocks won't blow anyone away, but they would let creators add a lot more polish to their Adventure- and Complete The Monument-genre maps.

This is probably the last entry in this series, so take heart. I had an idea about using Minecraft blocks as placeable opcodes in the event-based scripting language, but... there you have it. That's my idea. And here are three more:

Hypothesis Confirmed: I know you're excited about the evidence for the Higgs boson, but I was going through NYCB's 2011 archives for the inevitable year-end "best of" entry, and I noticed something even less amazing.

Earlier this year as part of Loaded Dice I made a list of "Pre-1990 games still greatly in demand". I suggested that someone could make some money reprinting these games. When I reread the NYCB post that linked to that list, I remembered an ongoing but very polite dispute in which two different game companies plan to reprint "Merchant of Venus", having acquired the rights from two different sources. I checked on my "someone should reprint" list, and sure enough, "Merchant of Venus" was #4 on that list.

In fact, four of the top five games on that list were reprinted/remade in 2011, or will be in 2012. The other one is Crokinole, a public domain game that apparently just can't be manufactured cheaply because you have to use SFI-approved Canadian hardwood. Spot checks further down the list turned up other games like "Code 777" that were reprinted recently, and games like "Full Métal Planète" that were recently subject to unauthorized reimplementations. What I'm saying is, that list is marketing gold.

Month of Crowdfunding: Hey there. Readers know that I backed a lot of Kickstarter projects this year (182, to be exact), but I've also dabbled in other crowdfunding websites. I mostly use Kickstarter because their site is by far the easiest to use, but my loyalty is to interesting projects, not to the hosting site.

There are a buttload of these sites, so with one exception I'm only going to list the ones I've used. Let me know if you've had good experiences with any others, on the money-raising side or the funding side.

OK, I think that's enough crowdfunding stuff on this weblog for 2011. Notwithstanding that we already have enough, I'm going to post one more entry, about the Street Performer Protocol.

Street Performer Protocol: As far as I can tell, Kickstarter-style crowdfunding was introduced to the world in John Kelsey and Bruce Schneier's 1998 paper "The Street Performer Protocol". When I first heard about Kickstarter I thought "finally, a site that implements the Street Performer Protocol." Except Kickstarter doesn't implement the SPP.

The SPP funds the creation of public works. "[P]eople would place donations in escrow, to be released to an author in the event that the promised work be put in the public domain." Or whatever--Creative Commons didn't exist in 1998. A Kickstarter user could choose to implement the SPP, but almost nobody does. They use Kickstarter to fund (say) a book, and then only a few people get to see the book. Or then the book goes on sale. It's cool that the book was made/reprinted, but the world would be a better place if everyone could have a copy. (I hope it's clear that I'm talking about projects with zero marginal cost like texts, not things like ice cream trucks.)

Jason Scott is the big counterexample. He's using Kickstarter to fund his documentaries, which he releases under CC licenses. It's also pretty common to use Kickstarter to fund open source software. Why don't more people do this?

The biggest obstacle to implementing the Street Performer Protocol is that most people don't know about it. But even someone who knows might not want to use it, because the SPP is susceptible to the free rider problem. Why back a project when you can wait for someone else to back it and reap the rewards?

The Rational Street Performer Protocol deals with the free rider problem, but it's an iterative protocol and it's pretty complicated. In real life, people deal with free riders by introducing scarce rewards into the Kickstarter reward tiers. A copy of the work is a very effective reward, so a project runner may rationally decide to keep copies scarce with copyright.

I'm interested in alternatives. Physical copies of music/books are intrinsically scarce, and they make a good choice for that all-important $25-$50 tier, but a lot of people (including me) would generally be happier with an electronic copy because we don't need more physical objects in our lives.

Another possibility is time-limited scarcity: offer backers early electronic access to a work that will made public later. I was really excited about time-limited scarcity back in 2008 when it was just me thinking of crackpot ways to publish my fiction, but after getting some experience with crowdfunding, I'm not so sure. You gotta wait for this stuff anyway. It's been six months since Month of Kickstarter and I'm still getting things in the mail. (Just got a huge copy of Fealty.) If I have to wait for six months anyway, why shouldn't I wait nine months and get it for free? If I'm funding the creation of a work, rather than publication of something that already exists, the wait is even longer. Is a twelve-month wait really that much better than an eighteen-month wait?

I don't have any other ideas right now, but I wanted to write this down as a starting point. I'd really like to see crowdfunding used as a way to ransom works into the commons, but it's hard enough to get your project funded without taking away one of the reward tiers.

BTW, this is another reason why board games make great crowdfunding projects. They're intrinsically scarce. You can make an unauthorized copy of a board game, but it's a huge pain, and it'll probably remain a huge pain for the next five years.

[Comments] (3) CG Author Commentary #4: "Too Much Information": I'm experimenting with posting these commentaries on Wednesday to give you time to read the chapter. I don't think a lot of people read these commentaries right now, but feel free to complain if you want it sooner. It's easier for me to put the commentary up on Tuesday along with the Twitter archive, but I think this might be a little better for you.

So! This is where the engine of the novel starts running. We've introduced the human characters and their dynamic, they have a gateway to the Constellation in the form of Curic, and in this chapter we meet the main plot drivers for Part One: the Brain Embryo computer, and the Constellation Database of Electronic Games of a Certain Complexity (hereafter CDBOEGOACC).

Now's a good time to talk a bit about the three bonus stories, because one of them takes place right after this chapter. "Found Objects" is a story about Jenny Gallegos and what she does with the reentry foam that the Brain Embryo was packed in. "Dana no Chousen" features Dana Light, a character who's been mentioned (as Bai's girlfriend) but not introduced. I'm currently working on "The Time Somn Died", which features a character you haven't met yet.

If you subscribe at the Bronze level you'll get one story, probably "Dana". Subscribe at Gold or above and you'll get all three. There's a fourth story, "A Princess of Mars", which I won't have time to finish. But I might do it as a promo later, or write it to be sold separately.

How these happened: after finishing the second draft I realized that although I'd created a lot of interesting female characters, the POV is so tight around Ariel that a) the only one who'd gotten the time she deserved was Curic; b) my novel didn't pass the Bechdel test. I fixed b) with some cheap tricks which I'll point out when the time comes, but a) left me unsatisfied, so when it came time to plan bonus material, I decided to write stories about and from the POV of the women in the story.

Like I said, I'm not done with the stories, but I think this strategy works really well. Each bonus story presents a part of the world that wasn't shown in the novel, while recontextualizing what was shown.

And now, random notes on chapter 4:

Tune in next Tuesday for Chapter 5, when Curic will say "I am going to do research on human zombies to prove you wrong."

[Photo credits: BuenosAiresPhotographer.com and NASA.]

<- Last week | Next week ->

[Comments] (2) CG Author Commentary #5: "Let's Play": After some polite outcry we're back to commentary on Tuesday. I don't have a lot to say about this chapter, and/or I'm fighting off an artisan seasonal cold, so this one's relatively short.

I got excited while writing this commentary, because I was going through my earlier drafts and I remembered that Constellation Games has deleted scenes! But then I was a little less excited because the deleted scenes don't start until chapter 25. Let's not think about the deleted scenes for a while.

And there we go. Tune in next Tuesday for the heart-pounding chapter 6, when Ariel will say, "This form is a fake. There's no Paperwork Reduction Act notice."

[Drawing credits: Brandon Eck and NASA.]

<- Last week | Next week ->

[Comments] (4) The Crummy.com Review Of Things: 2011 Edition: Man, too busy even for proper navel-gazing this year. Fortunately I already did a lot of the work for this post, so it's just a matter of... doing a lot more work.

First, the main course: Best of Crummy for 2011! Yeah, the main course is coming first. Because we're all busy.

Twitter update: Every year since my curmudgeonly 2008 post I've dreaded having to write this, but this was the year I started posting to Twitter. Fortunately, I've been able to launder most of my pithy, trite observations through a fictional character.

Reading: My records are a little spotty but I don't think I read more than twenty books this year. Of these, the best was On The Origin Of Species, which is still very readable and holds up pretty well for a biology book written by someone who didn't know about genes.

The Crummy.com Non-Public Domain Book Of The Year is Carrying The Fire: An Astronaut's Journeys by Michael Collins, an incredibly informative biography and which I probably should have read earlier for novel research. At the end of Carrying The Fire Collins says he could write a whole book on the year he spent post-Apollo 11 as Assistant Secretary of State for Public Affairs during the Vietnam War. I would love to read that book.

Writing: On the other hand, I did quite a lot in 2011 to advance the cause of other peoples' reading. I completed, sold, revised and started serialization of Constellation Games. I completed three (almost four, but one needs a rewrite, so three) short stories. And, I worked four months for Findings.

Findings needs its own post, and I should have written that post in October, but in a nutshell, Findings takes the highlights and notes you created when you were reading a book on your Kindle, and copies them outside Amazon's ecosystem so you can share them, search them, and do work with them. For instance, that link I gave earlier to On The Origin of Species was not a useless link to a generic summary or storefront for one of the most famous books ever published; it's a link to my reading of the book, which is useful at least to myself.

With Findings you can also clip from web pages, and the last thing I did there was design a web service for them, so hopefully it can soon be made to work with software and readers other than Kindle. Findings solves a lot of the problems with the Kindle specifically, and with an ebook-based reading style in general. I'll definitely go into more detail later when I don't have a bunch of other aspects of 2011 to go into.

Audio: Best of newly subscribed podcasts: The Famicom Dojo podcast and The Dice Tower. The Crummy.com Album of the Year is "Princess Ghibli", an album of heavy metal covers of songs from Studio Ghibli movies. The new-to-me runners-up are also Japanese: chiptunes from YMCK and sample-based insanity from Plus-Tech Squeeze Box. Foo Fighters and Tally Hall also maintained their standard of quality with great new albums.

Video: I watch TV with Sumana, so I can't do anything more than echo her recommendations. Breaking Bad shows characters nerds can identify with doing horrible, horrible things. And The Dick Van Dyke Show is probably the most realistically-written sitcom I've ever seen. In web video, Noah Antwiler's "Counter Monkey" series of teenage RPG monologues, and Brad Jones' nostalgically obscene "'80s Dan". In between, the Something Awful translations of GameCenter CX episodes, which this year went from nowhere to having one of the translators get her fan art shown on the show.

Crummy.com Film Of The Year: The Muppets. All other films are disqualified for not being The Muppets. I run a tight ship.

Food: In 2011 a bunch of new restaurants opened up near my house! It was great. The best are Queens Comfort, Pachanga Patterson, and amazing surprise latecomer Butcher Bar, which is so new it doesn't have a proper website yet. Honorable mention to Sugar Freak for fried chicken.

In non-Astoria food: while working at Findings I ate a lot of good lunches from Chelsea Market (#1: quinoa salad from Amy's Bakery), and the burritos from the secret Chipotle were much better than any burrito I've had at any other Chipotle.

Games: Apart from "Minecraft" (Crummy.com Game Of The Year, honorable mention to Vechs' amazingly well-crafted "Super Hostile" series of Minecraft maps), a pretty lean year for video games. I didn't buy a 3DS, or any Wii games for myself. I did buy a lot of games in Humble Indie Bundles, but then didn't get a chance to play most of them.

I rediscovered a love of tactical games like "Frozen Synapse" and the "Tactics Ogre" remake for the PSP. But this was the year I decided I don't like JRPGs. "Radiant Historia" has a lot of really great ideas but the plot was generic and despite a tactical battle system, I only had to come up with new tactics twice. Both failures I place squarely at the feet of JRPG design. I bought two other well-reviewed DS JRPGs and was immediately bored with them. So, probably no more JRPGs for me, until they do more Paper Mario. Those games are hilarious.

Board games fared a lot better, as my gaming group tempered its tried-and-true conservatism with Pat's early-adopter enthusiasm. "7 Wonders", "Alien Frontiers", and "Pandemic" were the big newcomers, with strong supporting players like "Power Grid" and the surprisingly hilarious "Carcassonne: Hunters and Gatherers" (Crummy.com Board Game of the Year Because 7 Wonders Already Won The Spiel Des Jahres, It Can't Win Everything, Geez). "Dominion" and vanilla "Carcassonne" remain strong, "Quarriors" fizzled out early.

Next year: "A Few Acres of Snow", I hope? Also the Kickstarter reward games will really start getting to the table. If you live in New York and want to play board games occasionally, let me know.

And, looking into 2012 more generally: I can promise lots of cool Constellation Games stuff, since I'm already committed to or have already written it. Beautiful Soup 4 will leave beta. (It needs a little work and I need to write new documentation--but please use it now, it's already better than BS3 in my opinion.) There may be more REST stuff, and/or another novel. There will probably be more Month of Kickstarter-type experiments, if not a whole nother MoK. And more currently-secret projects to tantalize you, my friends and readers. Let's keep it going.

Want more? Here's Sumana's year-in-review.



Unless otherwise noted, all content licensed by Leonard Richardson
under a Creative Commons License.