News You Can Bruise for 2012 March

Hey, two in a row. The "release candidate" thing was a lie; the big change is that I ported and incorporated Simon Willison's soupselect project. So you can now combine the Beautiful Soup API with CSS selectors. Except I just realized that I ported an old version of the code, so I'll be doing another release. Anyway, here it is in the BS4 docs.

Recently I read a 2001 book by Jeffrey Kluger, Moon Hunters, about unmanned missions to non-planetary Solar System bodies. It was a little out of date but there was a lot of good early stuff, like how every time one of the Ranger missions failed, Khrushchev would use it as a laugh line in a speech. ("The Soviet pennant on the moon has been awaiting an American pennant for a long time. It is starting to become lonesome.")

And the book's its very out-of-dateness reminded me of something I'd forgotten about. The Cassini probe was launched when I was in college (I remember a flyer for an anti-Cassini protest at JPL, the point being that Cassini might explode on the launchpad like a Ranger and contaminate Cape Canaveral with radioactivity), and in Moon Hunters it's on its way to Saturn. But now it's there, like a jump cut!

And (this is the part I'd forgotten) Cassini included a probe, Huygens, whose job it was to land on Titan. That's why it was always called "Cassini-Huygens" on the news. It wasn't just NASA and ESA fighting over the name of the mission. And Huygens was instrumented with a microphone. Wow!

So I went to the Internet looking for the microphone data, and I was not disappointed. By that I mean: I found some sound files. The Planetary Society offers 'sounds from the Huygens "Microphone"', and those quote marks should be a clue as to how this is going to turn out. This semi-technical description of the Acoustic Sensor Unit explains all: the "microphone" is part of a set of instruments that examined Titan's atmosphere during the descent. It's designed to detect a thunderstorm. It takes a sample once every two seconds, and its share of the Huygens bandwidth is a measly 480 bits per second. It's basically taking Polaroid pictures of the ambient sound—not something the human sense of hearing can deal with.

But the Planetary Society gamely processed the data into sound files approximating what you would hear if the microphone was much better. And... it sounds like wind, because Huygens is falling through atmosphere. No thunderstorms. There are files reconstructed from the data recorded while Huygens was sitting on the surface. (Well, it's still sitting on the surface, but from back when the battery worked.) Unfortunately, according to Peter Falkner of ESA, "all the sound we can hear is likely internal to the microphone."

So in terms of the gee-whiz factor, the microphone is a bit of a bust. It doesn't help that Huygens's only visible-light image from Titan's surface looks like a daguerreotype of Mars (see comparison). No wonder I forgot all about Huygens. As an antidote, I recommend Cassini's amazing photos of Titan from orbit, including radar images of the hydrocarbon lakes.

This wasn't the Planetary Society's first venture into astroacoustics. In the 1990s, three Berkley scientists developed "The Mars Microphone", an actual human-ear-like microphone that would work on Mars. Unfortunately it went to Mars with the Mars Polar Lander, which was lost during landing. Another Mars Microphone was supposed to go on the ESA Netlander mission, but that mission was canceled for being too expensive.

The Phoenix lander had a Huygens-like low-resolution microphone as part of its Mars Descent Imager, but (I'm synthesizing contradictory reports here) MARDI was not turned on during descent because it could have screwed up the landing. The MARDI microphone was turned on after landing, but no data was received.

It's a legacy of heroic striving towards almost certain disappointment, but there's another MARDI on the Mars Science Laboratory, so let's check back in August.

Image credits: ESA/NASA/JPL/University of Arizona, Roel van der Hoorn/NASA

I just randomly discovered that a friend of mine, Will Thompson, cohosts a radio show about science fiction, and last week he put in a little plug for Constellation Games. Tune in at around 51:00 to hear me finally achieve my goal of having my work compared to Ken Macleod's. Admittedly by someone who hasn't read Constellation Games and doesn't seem very into Ken Macleod. But we don't get to choose how that kind of goal is achieved.

A few weeks ago I described the moment when I realized I'd written a novel that didn't pass the Bechdel test. I went back trying to "fix" the "problem". Should be easy, right? Five of the eight main characters are women. Well, I'm counting Curic as a woman because that's how Ariel thinks of her.

Actually, that's the problem: the whole novel is tight third-person limited from Ariel's POV. The women definitely have conversations that don't involve Ariel, but it's all off-camera. To dramatize such a conversation from Ariel's POV, he'd have to be spying on them or something.

Fortunately, there's a cheap fix: pull a Starbuck on male stock characters. I did this twice. In this chapter, I gender-swapped the Senator who gives Kinki Kwi the runaround. A similar thing will happen next week. In neither case is Ariel a direct party to the Bechdel-passing conversation. In this chapter, Curic recounts the conversation to him; in chapter 16 it's something he overhears on television.

So annoyed was I at the difficulty of a non-cheap fix, I decided to write all the bonus stories from the POV of the women. This made passing Bechdel trivial. Jenny talks to Bizarro Kate, Jenny talks to Curic. Done. You just have to be interested in what women might talk about.

(Attn. Bechdel nitpickers: if you're calling shenanigans because Curic never names the Senator, wait for chapter 16, geez.)

I hope you're hanging off a cliff. Here's last week's Twitter archive, and now the miscellaneous commentary:

• This is the one of the longest chapters in the book, because I'm out of time. Anything that needs to happen in Ring City needs to go into this chapter. But the chapters have been getting longer and longer over the course of Part One, so that they're now routinely over 4000 words. The first chapter of Part Two will reset to a shorter length, and then they'll start getting longer again. I didn't plan this, exactly, but I did shuffle scenes from one chapter to another so that they wouldn't get too long or too short.
• Before we leave Ring City, I want to mention that the awful motel atmosphere of Human Ring, especially the apartment layout, comes from "Vanilla". But in "Vanilla", what we saw of the other habitats wasn't much different. The ennui Cody Wicklund felt towards his living environment was just part of his general ennui. Ariel's anger is a lot more immediate because we've seen how much nicer Alien Ring and Farang Ring are for their inhabitants.

  However, I agree with BEA Agent Krakowski that it's kind of bitchy to spend a week in space and then complain about the accommodations.

• Curic's response to Ariel's general hopelessness, where she just walks into her hot tub and submerges for two minutes, always cracks me up. Sumana thought the bubbles were from Curic screaming where Ariel wouldn't hear. Could be, but I don't think screaming is Curic's style.
• The chapter title is one of those double-meaning titles they love to use on TV shows. "777" is a reference to Gliese 777Ad, but it's also what a slot machine shows when you hit the jackpot.
• Gliese 777A is a real star, part of a binary system for which two exoplanets are already known. It's 52 light years away, in the direction of Cygnus, which as we'll find out in chapter 24 is the general direction the Constellation is spreading from. The fictional Gliese 777Ad showed up in "Vanilla" (George was an expert on it), but this is the first real explanation as to what happened there.

  

  In "The Time Somn Died" it's revealed that the Constellation name for Gliese 777Ad is "Nobody's Home".

• I don't know what species Kinki Kwi is; she's probably Gweilo. In this commentary I originally called her "Kinki Bwi", because that was her name in the second draft. I changed it at some point and I have NO IDEA WHY and I FORGOT I'D EVEN MADE THE CHANGE. Why the heck did I do that? Not enough 'k' sounds in her name?
• In the second draft, Curic swore almost as much as Ariel does, at least when talking to him. I toned it down a lot in an effort to make Curic seem more cunning. In particular, I remember "Stick it in the overhead compartment" used to be "Stick it in the fucking overhead compartment." You see how the first one is more cunning? Of course, sometimes you want to go the other way. This writer stuff can be tricky.
• I'm kind of embarrassed by the cheapness of the tricks I used to make BEA Agent Fowler a super repulsive character. The guy practically kicks puppies. On the other hand, I do like this chapter's treatment of Krakowski. You can see what I said a few weeks ago behind spoiler blocks: he and Ariel are basically the same character in different situations.
• This chapter introduces the rivalry between the two big overlays, Plan C (end the contact mission) and Save The Humans (keep going). I feel like I didn't do a enough job explaining this rivalry. You may not think that now, because Curic spells it out pretty clearly, but in two months you'll have forgotten all about this chapter and Curic won't be any help.
• I'll talk a little about the Antarctica incident in next week's episode of Creative License. In this week's episode, I'd like to mention that many details of space program procedure in this book are made up or downright counterfactual.

  I could not find a solid number for a Space Shuttle astronaut's personal allowance. The best I could do was that it was more than the weight of 18 Montreal-style bagels. I don't know if there's someone currently doing a three-day stint in an Orion CM simulator so they can eliminate UI errors, but it fits with what I know about the way space programs test things. I could have done months more research, and it would still be good to know this stuff, but I wouldn't necessarily use it if I did find out. I do know how people on the ISS call home, but in Chapter 17 I'll be going with something I made up, because it's too complicated to explain all the failure modes in 20 words of dialogue. It's a situation that calls for... Creative License.

• Insofar as I had specific source texts for the space stuff, they were Mary Roach's Packing for Mars, my 2007 conversation with Janice Voss, my hazy memories of Tom Wolfe's The Right Stuff, and a 2010 HOPE talk called T 40: The Three Greatest Hacks of Apollo. I also highly recommend Michael Collins' Carrying The Fire, which has some cool Apollo tech details and backs up a lot of my thinking, but which can't count as a source text because I read it after selling the book.

  That's a reading list heavily weighted towards the Apollo era, but what Tammy says in this chapter is still true: you can be all the way at the end of that bell curve of competence and just lose your opportunity, or die, through no fault of your own.

And on that cheery note we end this week's commentary. Tune in next week for "False Daylight," the HEART-BEATING CONCLUSION to Part One, in which special guest star Charlene Siph will say, "Pardon my French."

Image credits: U.S. Congress (x2), Tropenmuseum of the Royal Tropical Institute, Wikimedia Commons user Silver_Spoon_Sockpop, NASA.

← Last week | Next week →

Last night I dreamed I was teaching a college-level class on the history of film. Despite my total lack of qualifications, the class went well, because I focused more on film metadata than on history or craft. One of the things I did in dream-class was compare different measures of film quality, as I've been doing recently on NYCB with TV shows. In particular, I compared the winner of each year's Best Picture Oscar to IMDB's highest-rated movie of that year.

Well, prepare for a dream come true, because when I woke up I created that comparison in real life, using my old standby, IMDB data. I also brought in Wikipedia data, because it looks like IMDB doesn't publish any machine-readable information about awards or nominations. Wikipedia doesn't either, but you may have heard of a little library called Beautiful Soup.

Without further ado, here's the table. Well, I need a little more ado to explain what the headings mean.

• BP_IMDB is the IMDB rating for a given year's Best Picture winner.
• If you sort all of a year's movies by their IMDB rating, then BP_rank is the Best Picture winner's rank in the resulting list. So according to IMDB, 1927's winner was actually the 9th best movie of my 1927 dataset. (This isn't all of 1927's movies—see below—but it should be all the big ones.)
• BIR_IMDB is the IMDB rating of a given year's highest-rated movie
• BIR_nom is ✓ if the highest-rated movie was also nominated for Best Picture, and ✓✓ if the highest-rated movie actually won Best Picture. If it's blank, the highest-rated movie didn't even get a nomination.
• "Alignment" is semi-complicated so I'll explain it after the table.

Year	Best Picture	BPIMDB	BPrank	Best IMDB Rating	BIRIMDB	BIRnom	Alignment
1927	Wings	7.80	#9	Metropolis	8.40		0%
1928	The Broadway Melody	6.50	#8	La passion de Jeanne d'Arc	8.30		0%
1929	All Quiet on the Western Front	8.10	#1	Chelovek s kino-apparatom	8.40		0%
1930	Cimarron	6.30	#20	All Quiet on the Western Front	8.10		0%
1931	Grand Hotel	7.70	#11	City Lights	8.60		12%
1932	Cavalcade	6.40	#24	Trouble in Paradise	8.20		10%
1934	It Happened One Night	8.30	#1	It Happened One Night	8.30	✓✓	25%
1935	Mutiny on the Bounty	7.90	#4	A Night at the Opera	8.10		41%
1936	The Great Ziegfeld	6.90	#57	Modern Times	8.50		30%
1937	The Life of Emile Zola	7.40	#31	La grande illusion	8.20		50%
1938	You Can't Take It With You	8.00	#4	The Lady Vanishes	8.10		30%
1939	Gone with the Wind	8.20	#3	Mr. Smith Goes to Washington	8.40	✓	40%
1940	Rebecca	8.40	#1	Rebecca	8.40	✓✓	40%
1941	How Green Was My Valley	7.90	#7	Citizen Kane	8.60	✓	60%
1942	Mrs. Miniver	7.70	#12	Casablanca	8.80		30%
1943	Casablanca	8.80	#1	The Ox-Bow Incident	8.20	✓	30%
1944	Going My Way	7.40	#31	Double Indemnity	8.60	✓	20%
1945	The Lost Weekend	8.10	#4	Les enfants du paradis	8.30		20%
1946	The Best Years of Our Lives	8.30	#2	It's a Wonderful Life	8.70	✓	40%
1947	Gentleman's Agreement	7.40	#25	Out of the Past	8.10		0%
1948	Hamlet	7.90	#12	Ladri di biciclette	8.50		40%
1949	All the King's Men	7.60	#18	The Third Man	8.50		20%
1950	All About Eve	8.50	#2	Sunset Blvd.	8.70		20%
1951	An American in Paris	7.30	#44	Strangers on a Train	8.30		0%
1952	The Greatest Show on Earth	6.70	#86	Singin' in the Rain	8.40		20%
1953	From Here to Eternity	7.90	#10	Le salaire de la peur	8.30		0%
1954	On the Waterfront	8.40	#3	Shichinin no samurai	8.80		20%
1955	Marty	7.70	#18	Les diaboliques	8.30		0%
1956	Around the World in 80 Days	6.8	?	The Killing	8.20		0%
1957	The Bridge on the River Kwai	8.40	#5	12 Angry Men	8.90	✓	60%
1958	Gigi	6.90	#77	Vertigo	8.50		20%
1959	Ben-Hur	8.20	#5	Ningen no jôken	8.80		20%
1960	The Apartment	8.40	#2	Psycho	8.70		20%
1961	West Side Story	7.70	#28	Ningen no jôken	8.80		20%
1962	Lawrence of Arabia	8.50	#1	Lawrence of Arabia	8.50	✓✓	40%
1963	Tom Jones	7.00	#74	Pour la suite du monde	8.50		0%
1964	My Fair Lady	7.90	#15	The T.A.M.I. Show	8.40		0%
1965	The Sound of Music	7.90	#20	Obyknovennyy fashizm	8.40		0%
1966	A Man for All Seasons	8.00	#8	Il buono, il brutto, il cattivo.	9.00		20%
1967	In the Heat of the Night	8.00	#8	Cool Hand Luke	8.30		20%
1968	Oliver!	7.50	#41	C'era una volta il West	8.80		20%
1969	Midnight Cowboy	8.00	#8	Le chagrin et la pitié	8.40		20%
1970	Patton	8.10	#4	Brigada Diverse intra în actiune	8.10		20%
1971	The French Connection	7.90	#12	A Clockwork Orange	8.50	✓	20%
1972	The Godfather	9.20	#1	The Godfather	9.20	✓✓	20%
1973	The Sting	8.40	#1	The Sting	8.40	✓✓	20%
1974	The Godfather Part II	9.00	#1	The Godfather: Part II	9.00	✓✓	40%
1975	One Flew Over the Cuckoo's Nest	8.90	#2	Hababam sinifi	9.00		40%
1976	Rocky	8.10	#8	Tosun Pasa	8.70		20%
1977	Annie Hall	8.20	#9	Star Wars	8.80	✓	20%
1978	The Deer Hunter	8.20	#5	Selvi boylum, al yazmalim	8.50		20%
1979	Kramer vs. Kramer	7.70	#27	Apocalypse Now	8.60	✓	20%
1980	Ordinary People	7.90	#8	The Shining	8.50		40%
1981	Chariots of Fire	7.20	#74	Raiders of the Lost Ark	8.70	✓	20%
1982	Gandhi	8.10	#8	Maratonci trce pocasni krug	8.40		0%
1983	Terms of Endearment	7.40	#42	El sur	8.20		0%
1984	Amadeus	8.40	#4	Balkanski spijun	8.50		20%
1985	Out of Africa	7.00	#83	Zügürt Aga	8.50		0%
1986	Platoon	8.20	#4	Aliens	8.50		20%
1987	The Last Emperor	7.80	#17	Muhsin Bey	8.40		0%
1988	Rain Man	8.00	#12	Nuovo Cinema Paradiso	8.50		0%
1989	Driving Miss Daisy	7.40	#45	Indiana Jones and the Last Crusade	8.30		0%
1990	Dances with Wolves	8.00	#3	Goodfellas	8.80	✓	40%
1991	The Silence of the Lambs	8.70	#1	The Silence of the Lambs	8.70	✓✓	60%
1992	Unforgiven	8.30	#3	Reservoir Dogs	8.40		20%
1993	Schindler's List	8.90	#1	Schindler's List	8.90	✓✓	40%
1994	Forrest Gump	8.70	#4	The Shawshank Redemption	9.20	✓	60%
1995	Braveheart	8.40	#3	Se7en	8.70		20%
1996	The English Patient	7.30	#73	Freebird... The Movie	8.30		20%
1997	Titanic	7.40	#51	La vita è bella	8.50		20%
1998	Shakespeare in Love	7.30	#83	American History X	8.60		20%
1999	American Beauty	8.60	#3	Fight Club	8.80		60%
2000	Gladiator	8.40	#3	Memento	8.70		20%
2001	A Beautiful Mind	8.00	#13	The Lord of the Rings: The Fellowship of the Ring	8.80	✓	20%
2002	Chicago	7.30	#155	The Last Just Man	9.40		40%
2003	The Lord of the Rings: The Return of the King	8.80	#1	The Lord of the Rings: The Return of the King	8.80	✓✓	20%
2004	Million Dollar Baby	8.20	#7	Eternal Sunshine of the Spotless Mind	8.50		0%
2005	Crash	8.50	#12	Babam Ve Oglum	8.70		0%
2006	The Departed	8.50	#3	Das Leben der Anderen	8.50		20%
2007	No Country for Old Men	8.20	#8	Jogo de Cena	8.70		0%
2008	Slumdog Millionaire	8.20	#7	The Dark Knight	8.90		0%
2009	The Hurt Locker	7.80	#30	Puskás Hungary	8.60		30%
2010	The King's Speech	8.30	#17	Inception	8.90	✓	20%
2011	The Artist	8.20	#15	Drive	9.00		0%

Ok, "Alignment". Take 1941 as an example. There were ten Best Picture nominees in 1941 (although it was called something different back then). So we take the top ten movies of 1941 by IMDB rating. Six of the Best Picture nominees are also in the top ten by IMDB rating, so the alignment for 1941 is 60%. At the other extreme, none of the five 1983 Best Picture nominees are in the IMDB top five for that year, so the alignment for 1983 is 0%.

For a few years I couldn't calculate BP_rank, generally because the IMDB year of the Oscar winner differs from the year it won an Oscar. Early on this happens a lot because until 1933 the Academy Awards covered parts of two years. That's why "All Quiet on the Western Front" got the nod in the 1929 Oscars, and then showed up as the best-rated IMDB film of 1930. The "1929" Oscars weren't just held in 1930, they actually covered some movies released in 1930. But sometimes the dates just don't match up. Casablanca is the top-rated film of 1942 and the winner of the 1943 Oscar. This still happens: The Hurt Locker won Best Picture in 2009 but IMDB says it was released in 2008. In most cases I was able to find the year the film was released, according to IMDB, and put down down its ranking within that year for BP_rank.

My dataset excludes TV shows, video games, direct-to-video releases, and shorts. (Excluding shorts required cross-referencing against IMDB's genre.list file.) I also excluded movies with fewer than 150 votes on IMDB. I did what I could to exclude movies that are mainly concert footage, although Freebird... The Movie still made it on there. I did not exclude documentaries or foreign films.

Finally, to fulfil the promise of this post's title. According to IMDB, the worst movie ever to win Best Picture is 1930/1931's winner, "Cimmaron" (IMDB:6.3). But if you look relative to what else came out the same year, the worst Best Picture is "Chicago" (IMDB:7.5), which IMDB data ranks at the 155th-best movie of 2002. However you look at it, the best movie ever to win Best Picture is 1974's The Godfather: Part II (IMDB:9.00).

PS: Why are the Oscar nominees linked and the IMDB champions not linked? Because IMDB DATASET DOESN'T INCLUDE ANY URLS ARGH.

PPS: I did something similar for board games as part of Loaded Dice. I called it the "People's Spiel des Jahres." I didn't put up the table because the results were uninterestingly full of wargames. But wargames generally don't get nominated for Spiel des Jahres, so maybe I should exclude them and try it again.

Here it is, the season finale! We've got the whole contact mission going to shit, plus a game review! Don't worry, everything will turn out fine. Maybe.

Last Friday I went to the Brooklyn Museum to take some pictures for my final Constellation Games commentary. (And if you can somehow turn that into a spoiler, I salute you.) It's a fun museum, like a much less formal version of the Met. While walking through the room of Indian sculpture I passed a curator cleaning one of the sculptures with a Shop Vac and a brush. When I showed a flinch of uncertainty about where the stairwell was, a security guard told me and talked my ear off about what I should see next, then opened up the cabinet containing the emergency fire hose and took out a "What's Happening" brochure, which she used for reference and then gave to me. Also, the neighboring Botanic Garden was free to get in because it's winter and everything's dead.

Friend of the show and beta reader Brendan Adkins has been writing erudite-ass essays about the novel's symbolism, and I'd make fun of him for being pretentious except he's right about most of it. My earlier coyness notwithstanding, I did reuse some of the character of Ariel from The Tempest, the guy with magic powers who gets bossed around all the time. Don't you think The Tempest would be more interesting if it were more about the PEOPLE WITH MAGIC POWERS and less about the Renaissance douchebags? We can only dream. For now, we sup the slender soup of the Twitter archive and this week's commentary:

• Something I forgot to mention last week: Conway's Life is the second real-life game to be mentioned in the novel, after Tennis For Two. There's just one more, I think. I really tried to only use real games when the name was necessary for a joke. E.g. in the second draft, Jenny called a game "alien Candyland", but I took that out because the joke wasn't good enough to justify the real-world reference.
• In this week's episode of Creative License we take on the ice sheet removal. After trial balloons like the kites went over like lead (trial) balloons, I wanted Save the Humans to turn to a plausible, but politically insane, freelance climate stabilization project. I read a number of popular sources on geoengineering, including Eli Kintisch's terrifying Hack the Planet, but they all focused on projects within the capabilities of human technology. So I got introductions to a couple friends-of-friends who are climate scientists, but they never responded to my emails, perhaps fearing that I was a crackpot. So I gave up.

  As such, I do not recommend using Constellation Games as a textbook in the earth sciences. In the end, I went ahead with my initial idea and slathered it with some... Creative License.

• Don't worry, the ice is being replaced with inert material of equivalent albedo. Or maybe that should worry you.
• This chapter has the second and final Bechdel-passing scene in the novel, the overheard conversation between Charlene Siph and Susan the news anchor. The anchor was originally named John, which is the name of my brother-in-law, so I changed it to the name of my sister.

  As you can see here, Charlene Siph's job is to act real folksy for the benefit of people who need to identify the Constellation with a single individual. Maybe a little more on this next week.

• One of the little jokes I put in for myself is that despite everything that happens in the rest of the novel, the ISPs never restore offworld Internet access.
• My original concept was to do the * review in a parody of the discursive style of Tim Rogers (now of Kotaku). You can still see some of that in the throat-clearing at the beginning, but then Ariel's natural despair takes over and produces something very different. Which is good, because this is a big character moment for Ariel, not random real-world people.

  This review is the clearest picture you'll get of the Ip Shkoy outside of Pey Shkoy Benefits Humans. Hopefully it shows why Tetsuo is interested in that society, and why Ariel is sort of clinging to their history as proof that humanity isn't doomed.

• According to Pey Shkoy Benefits Humans there is no "sch" sound in Pey Shkoy, only "ch" and "sh". Tetsuo must have changed the official transliteration sometime before writing PSBH. So "Schvei" is actually "Chvei". (It can't be "Shvei" because then "Ip Shkoy" would have originally been "Ip Schkoy".)

  Unlike "Curic"→"Huric", this was not intentional. I asked Adam about this and he said: "I was afraid you would ask this question, and I am glad that you have already come up with a retcon strategy."

• Af be Hui still doesn't exist as a character, but she's now got a little backstory and a creative arc. In A Tower of Sand she explored what it might feel like to be a Farang with a bicameral mind. In * she applied the same two-players-one-character mechanic to the conflict between the individual and society. This struggle between partners is one of the major themes of CG. Yeah, Brendan, write an essay about that. I know you already did! Uh, write another one.
• The ethics system in * is probably the most inventive game mechanic in the novel. I don't know how fun it would be to play, though. It would depend on your partner, as with board games.

  My vote for most fun game in the novel goes to another of Af be Hui's games, The Long Way Around, which Ariel and Tetsuo played last week and will talk about in the last scene they have together.

• I don't know how * ends.

With that, I'd like to thank you for following me through "Hardware", the first part of Constellation Games. After a short season break of seven days, we'll pick up with Part Two, "Software." It all starts next Tuesday, when Ariel will say, "Probably the most expensive penis in history."

Image credits: Wikimedia Commons user Anynobody, Paul Mutant, U.S. Air Force

← Last week | Next week →

It's been nearly two weeks since the release of the last BS4 beta, and no one has reported problems with the code. I'm sure there are still problems, but at this point the best way to find them is to do an official release. So, I present the first full release of Beautiful Soup 4, 4.0.1![0]

If you're just tuning in, Beautiful Soup 4 is nearly a complete rewrite that works on Python 2 and Python 3. Instead of a custom-built parser from 2006, Beautiful Soup 4 sits on top of lxml (for speed) or html5lib (for browser-like parsing) or the built-in HTMLParser (for convenience). Methods and attributes are renamed for PEP 8 compliance, and Beautiful Soup 4 incorporates the soupselect project to provide basic CSS selector support. I completely rewrote the documentation, Beautiful Soup's secret weapon since 3.0, for clarity and completeness.

That's the major stuff. Even though most of the code has changed, my goal was not to add a bunch more features, but to make sure Beautiful Soup will still be usable and useful years into the future.

Beautiful Soup 4 is mostly but not entirely backwards compatible with Beautiful Soup 3. Most users should be able to switch from 3 to 4 just by changing an import line. In the Python tradition of sticking a number on the end of your module name when you break backwards compatibility, I've released it as a separate package, beautifulsoup4.

This release also inaugurates the Beautiful Soup Hall of Fame, featuring the uses of Beautiful Soup that I personally find the coolest or highest-profile.

So, try out Beautiful Soup 4 the next time you need to do some screen-scraping. If you've used Beautiful Soup 3, I think you'll be pleasantly surprised. If not, I'll just say I hope you like it.

I've thanked them before, but special thanks are once again due to Thomas Kluyver and Ezio Melotti for helping me get everything working under Python 3.

[0] The first release is called 4.0.1 instead of 4.0.0 because I've been bitten by clever packagers before and I don't want them thinking "4.0.0" is an earlier version than "4.0.0b10".

Hey, folks, Leonard here, telling you that if you haven't bought your copy of Constellation Games, still the greatest novel about video games from outer space, now is probably the single best time to buy.

Sure, you were skeptical at first. Ever since standing in line for that midnight showing of The Phantom Menace, you've been wary of things that seem awesome. You thought, "can this guy bring to comedic science fiction the same epic scope we saw in RESTful Web Services?" But now Part One of the novel has been sent to subscribers, and random commentary readers are calling it "STONE COLD BRILLIANT" and "some of the most fun I've had in years". Even normally reputable publications like Wired's GeekDad have called it a "wild ride" that's "so much fun to read".

Now here's where your late-adopterhood pays off: with the completion of Part One, all subscribers have been given access to a compiled PDF of the novel so far. That's about 50,000 words in a single unencumbered file you can drop onto your ebook reader or your fancy smartphone.

This means you can subscribe to CG for $5, read the first sixteen chapters in one huge gulp, and then start reading the rest of the story as the chapters come out every Tuesday. Or you can subscribe at the $20 level, read the PDF at a more leisurely pace, and finish the whole story when the paperback comes out next month. For $20 you'll also get three bonus stories that take place before/during/after the novel, and an irreverent guide to a pathologically strange alien language.

With all this stuff on the table, you silently think, why not keep waiting? Won't we just offer more in the future? THE ANSWER IS NO. Once the paperback comes out, the bonus stories and language guide stop being pack-ins and become "sold separately"s. The paperback on its own will cost $20. (I don't know exactly how this is going to happen, but that's the gist of it.) So the best deal is to shell out $20 now for early access to Part One and a lot of preorder bonuses. If you hate paper, you can pay $5, catch up on the novel the way you would a web comic, and buy the bonus material later.

Friend, don't let the fact that I seem to think it's a great idea to call you "friend" in a sales pitch, dissuade you from shelling out your hard-earned PayPal balance for this quality entertainment. Here's the subscription page, and here are the first two chapters so you can see what you're getting. The whole thing could be yours for the cost of a really, really enormous gumball, a gumball that won't fit in your mouth so why even bother? This is a much better deal.

On Friday I decoded a BCDIC punch card that my dad used to sign up for classes at UCLA in 1968. It says, "C 6088312496U40" What drove me to this? Well:

• Friday's BCDIC adventure inspired by C.E. Mackenzie, "Coded Character Sets, History and Development" (1979), a book ½ boring and ½ AMAZING.
• If I'm reading "About The Author" right, Charles Mackenzie was IBM's man on the committee that created ASCII.
• In celebration of his great book, today I broadcast the best bits that can be summarized in 140 characters. Why limit it like this?
• "How many readers of this book have experienced the aggravation of trying to squeeze 81 characters into an 80-column card?" (p269)
• Because I laughed at that quote the first time I read it, then realized I had in fact experienced that aggravation.
• Supplementary information from my friend @enf's paper "The Evolution of Character Codes, 1874-1968"
• Let's begin with [E]BCDIC tidbits! ¤ is "the international 'Currency Symbol'", a placeholder for your country-specific symbol. (EBCDIC 0x5B)
• ¤ is not to be confused with ⌑, the "lozenge". An IBM control character not intended for customer use. Spoiler: customers used it. (p67)
• If you had a German IBM keypunch, the hole patterns for #@$ were represented by ÄöÜ. Similarly for other European countries. (p90)
• If you had a "scientific" (FORTRAN) installation, the "commercial" characters %⌑&#@ showed up as ()+='. You couldn't have both in BCDICv2
• This meant FORTRAN code X=(A+B)*(C-D) might print as X#%A&B⌑*%C-D⌑. "Such program listings, though bizarre, were unambiguous." (p92)
• But COBOL "created a requirement for both the 'commercial' and 'scientific' graphics within a single application"! (p381) Thanks, COBOL.
• IBM unified "commercial" and "scientific" in BCDIC v3, but seemingly forgot to add the plus sign! BCDIC v3 was quickly replaced. (p100)
• BCDIC control characters were given symbols designed to discourage use by customers: ␢ Δ γ ⵐ ᚒ ✓, and one that looks like ᚒ rotated 90°
• "Graphics for control characters should have no intrisic meaning... How well the graphics meet the criterion the reader can judge." (p102)
• (I guess people who can suddenly use both + and % might not feel the need for Δ for a while.)
• Death of the lozenge: "Attempts to persuade the standards committees to assign [⌑] in ASCII were unavailing." Thus, omitted from EBCDIC too.
• But wait! Customers demanded ⌑ on their System/360 printers! "It was provided... although it ostensibly did not exist in EBCDIC." (p152)
• "We shall learn in Chapter 18 that the assignment of Katakana in EBCDIC created complications." (p173)
• I don't understand the Katakana EBCDIC problem, but at one point character order broke collation order, a la "abcdefghijkluvwxyzmnopqrst"
• Exotic EBCDIC control characters: "Digit Select", "Start of Significance", "Reverse Line Feed", "Graphic Escape", "Eight Ones".
• And now for act 2: ASCII tidbits!
• AT&T demanded that ASCII start with Null and end with Delete. That's why Delete is on the other side of ASCII from other control chars (212)
• "A" is ASCII 0x41 instead of 0x40 for compatibility with other standards. Why do those standards use 0x41? "The author does not know." (p238)
• ASCII 0x5E and 0x5F (^ and _) were originally ↑ and ←, two characters for use in flowcharting. (p242, but Fischer says they were for ALGOL)
• The UK wanted to have "10" and "11" after "9" (for pennies in a shilling), so unimportant symbols : and ; were put in those spots. (p241)
• Similarly, @ goes right before A (0x40) so it can be replaced with À in France and Italy. (p243)
• The equal sign was placed between < and > for mnemonic reasons: less than, equal to, greater than (p243)
• ASCII 0x05 was originally WRU, "Who Are You?" "Causes the receiving equipment to transmit back information identifying itself" (Fischer)
• The 1963 draft of ASCII had unassigned space where lowercase letters would later go. (p246) Lots of resistance to lowercases. (Fischer)
• Now it gets bonkers. ASCII was designed without regard to how it would look on punch cards! It's a disaster! Takes up 80 pages in Mackenzie.
• Here's the big problem. Punch cards have ≥10 rows. Most data is numeric. Hollerith coding punches row 0 for "0", row 1 for "1", etc.
• But ASCII encoding of digits punches row 5 and 6 for *every* column, "lacing" it. The punch cards will fall apart!
• Problem #2: Hollerith coding encodes space as 0x00. That is, no punches on the card. But ASCII space is 0x32, which punches row 6! So ugly!
• One proposed solution: just replace the "Space" bar on keypunches with a "Null" bar. REJECTED. (p255-256)
• Problem #3: Hollerith lets you "overpunch" a card to represent -5 in a single column. ASCII makes you put "-" and "5" in separate columns.
• (Disclaimer: I'm simplifying that last one a little. But people were really attached to their overpunches.)
• Solution: A standardized Hollerith-type encoding that works for ASCII or EBCDIC. Space is an empty column, no overpunching for negative #s.
• And that's Mackenzie: the history, politics, and design of some of the most influential standards. Not riveting, but full of great stuff.
• Bonus: headings that read like pulp SF novel titles: "The Duals of BCDIC", "Bendix Prime", "The Plomondon Proposal". 0x04

Some addenda acquired from readers while I performed that blob of text on identica/Twitter:

• Evan Prodromou and Kragen Sitaker both recommended "An annotated history of some character codes".
• spacehobo asked what those uncooperative IBM customers used the lozenge for. According to Mackenzie they used it to indicate "final totals as contrasted to subtotals" (p67) and to fill up blank spaces on printed checks. (I don't have a page reference for that one.)
• spacehobo also provides some illumination on the Katakana problem: "kana are syllabic, so there's a received grid ordering for the characters that has lanthanide/actinide-style break-out areas"
• Zack Weinberg complained: "All these struggles over what punctuation to include, but nobody suggests cutting back on the number of control characters. Nearly all of which are now useless."

  Actually, lots of people wanted MORE control characters. Lowercase letters didn't get into the original ASCII because people wanted to reserve that space for control characters. I think they were imagining that as more types of technology were invented, control characters would have to be added to ASCII for each one. So by now ASCII would be full of modem commands and graphics primitives, but have no lowercase letters.

• Eric Fischer had a lot of helpful comments:
  • Why is "A" ASCII 0x41 instead of 0x40? "Ferranti had A as 1 instead of 0 so they could have NUL be 0 in a 5-bit code with mode shifts between alpha and digits."
  • Apropos @: "I really think @ is 0x40 because it is the most US-centric character so the easiest to do without in a 5-bit subset where it's NUL... But the major proposal that put @ there (Allen Whitman 4/2/1962) was arguing mostly for keyboard layout."
  • Why was ASCII not designed for use on punch cards? "They really expected computers to use a 6-bit subset subtracting 0x20, putting space at 0x00. 8-bit bytes came as a surprise."
  • Who says ↑ and ← were intended for ALGOL? "Oct 23 1961 draft refers to ↑ as 'Exponential' and ← as 'Is replaced by'." (The flowchart thing sounded fishy from the start--why 2 arrows instead of 4?)

    Kragen Sitaker says: "They're also used in Smalltalk: ↑ is "return" and ← is :=". Dan Hirsch adds: "This is the reason that old smalltalk sources use _ as an assignment operator." spacehobo says: "I think I read in Jennings that := was meant to be ascii-art for ← once _ won out"

This chapter has the best title in the whole book. Just gettin' that out of the way. This week sets up the plot for the next couple months while focusing the action on the emotional core of Part Two: Ariel's relationships with other humans people from Earth.

I really liked the comments from last week's commentary--two people I didn't know were reading said hello, Brendan responded to my evaluation of his reader commentary, and my friend Zack (whose name I stupidly misspelled) disputed my use of Creative License. If you're enjoying these commentaries, please do say hi in the comments.

Look on last week's Twitter archive, ye mighty, and despair. Tetsuo won't be posting for a while because of the Internet blackout. Here's this week's commentary:

• The original name for Part One was "Company", and Part Two was "Crunch Time". "Hardware" and "Software" aren't as snappy individually, but this way the names form a coherent mini-story. (The final part is called "Artwork".)
• In part one (I don't like the way that looks when capitalized, so let's try it not) we focused on two Constellation species, the Farang and the Aliens. Part two introduces the Gaijin, not through a single Curic-level character but through bit characters and the human use of their technology. I'll defer their commentary, since all you've seen at this point is Tammy mentioning that she talked to He Sees The Map And He Throws The Dart!.
• This chapter has the scene on the ISS I mentioned back in December 2009, where I fudged the system the astronauts use to call home. This has been... Creative License.
• Phillip (not Moe!) is another one of those fun one-shot characters like the hippie in chapter 2. Like many one-shots, I suspect Phillip is more interesting the less you know about him, so let's move on.
• Check out the "Mallory" reference! ("Mutant's Revenge cocktail arcade cabinet") In the same scene where Jenny mentions 3D printing! Does this mean Constellation Games takes place in the same universe as "Mallory"? No, sorry, they just share a fictional video game. By a supreme act of will I've not written a big block of nerd analysis about why they can't be the same universe. (The real reason is that if space aliens showed up in the "Mallory" universe, it would retroactively become ineligible for publication in Futurismic.)

  But, speaking of "Mallory", my friend Alexei recently told me about ANGELINA, an AI system that designs video games and then simulates a person playing the games to find the ones that are fun. This means basically everything from "Mallory" now exists in real life, a mostly horrifying prospect.

• Smart paper, the arrival of which Ariel is dreading, is another import from "Vanilla". But in "Vanilla" it was just a generic piece of near-future tech. Now it's Constellation tech that's effectively being given to humanity as a bribe. ("But not like flashy desperate jewelry.") We've seen with the False Daylight that a computer embodies the values of the culture that created it. Smart paper was kind of bland in "Vanilla", effectively a flexible iPad. Now it's a Gaijin computer repurposed by the Constellation for use by humans, which is a lot more fun.
• I hope you don't feel it's a terrible spoiler that I'm talking about smart paper like it's going to figure in the book at some point, totally eliminating the possibility that Ariel and Jenny have a huge conversation about it in this chapter but it never shows up.
• In "Vanilla" the smart paper was called vellum, but that was just so you'd know to differentiate it from static paper. "Vellum" was probably a brand name. Now it's just "smart paper", and near the end of the book, Ariel just calls it a "paper computer", because "smart paper" is the kind of dumb name you use for a technology when it's new.

  

• I think the diner banter between Ariel and Jenny came out really well. I wanna say the same of the Ariel/Tammy flirting, but an author praising their own sexy writing is just asking for trouble.
• As the story turns to Ariel's intraspecies relationships, he becomes a less reliable narrator. Here, he puts up a public blog post with part of a conversation, then "real life"s the remainder of the conversation, which is full of stuff he needs to keep secret.

  "So what's he editing out of the 'real life' sections?" you may ask. Well may you ask. Because of the tight POV I can't show you that within the novel, but you'll see some of it in "Found Objects", the Jenny story.

• Maybe you were wondering how Dana Light could possibly be a major character in the novel. Wonder no more!

  BTW, if you figured this out earlier, or if you guessed incorrectly, or even didn't think about it at all, I'd like to hear about it. It's always tricky putting together a plot twist without springing it prematurely, and I'm interested in hearing how readers react, because I really like when I'm reading a book and the author successfully pulls off something like this.

  Of course, since you've been reading the commentary you knew that something was going to happen involving Dana. My hope is that up to this point, most readers have thought of Dana as just a commentary on Bai's personality, when she's actually more important than Bai.

And there we go. Be sure to tune in next week, when Ariel's all like, that's right, motherfucker, you're not the only one who can use paper. Oh, and Tetsuo writes a game review!

Image credits: NASA, unknown, Flickr user puuikibeach.

← Last week | Next week →

Did you buy one of the really cheap Constellation Games packages, and are now regretting your decision? Sorry, no refunds. Oh, you want a package with more stuff? You're in luck! Use the Candlemark & Gleam contact form to ask for an upgrade, and Kate will upgrade your subscription and invoice you for the difference.

Be sure to say which package you want. "Gold" ($12) is the one with the (electronic) phrasebook and bonus stories.

Schmeckel, the Jewish transgender punk band most familiar to NYCB readers, has a Kickstarter project to get a sweet tour van. This lets me combine two great things in one post: crowdfunding and cool doods like Schmeckel frontman Lucian Kahn, who will probably get the Teenage Mutant Ninja Turtles airbrushed onto the tour van.

I'm still going through the Kickstarter firehose every day, but the number of projects I've backed has gone way down since the heady Month of Kickstarter. This one's an auto-back, though. Backing a crowdfunding project is like being pen pals with the Internet, so give it a shot.

Be warned! This week's commentary goes deep into the workings of a scene that was originally a disaster, and the rewrite process that made it hopefully only a minor disappointment. Fortify yourself with last week's Twitter archive before proceeding.

• We start with Tetsuo's review of Ariel's last game, Pôneis Brilhantes 5, a well-behaved scene which has never needed rewriting and is a common reader choice for favorite scene in the book. Tetsuo uses a kind of post-scarcity Marxist analysis to reach pretty good conclusions about the civilization that created the game, but is utterly baffled by the in-game economy, in which you do work for ponies who pay you in gold so you can buy them hats.
• Tetsuo's reference to "the mysterious Curic" is just him teasing her; he doesn't really find her mysterious. Oh, and the thing I said in chapter 4's commentary about Ariel's old company never being named isn't true--it's here in the metadata for the game review.
• The last paragraph of Tetsuo's review predates the ICP magnets meme, but sure, I'll take it.
• The chapter title is of course my tribute to one of my favorite B-movies, The Amazing Colossal Man, prefigured here. There was going to be another TACM reference in the book, but I botched it. Starting in chapter 20 there's a minor character, Colonel Mason, who was supposed to be named after Colonel Manning from TACM, except "Mason" isn't actually the same name as "Manning". I decided it didn't matter and stuck with "Mason".
• The "used-to-be-a-paint-store" vibe of the BEA field office was inspired by surprisingly classy old buildings in downtown Bakersfield, buildings which always house something super boring. Later on we actually get a street address for the field office, and I thought I'd show it to you on a map, but turns out it's a made-up address. Curse you, previous self! Here's the neighborhood, though.
• Nice moment of connection between Ariel and the right-wing gardening lady, who both hate the BEA but who frame it completely differently.
• I had a chance to put in a semi-Bechdel-passing scene by making the BEA clerk a woman, but I decided not to do it. Ariel doesn't hear what they're talking about, but it's probably not a man.
• The third draft features HUGE changes to the scene in the field office, an important scene that originally didn't work at all. Allow me to walk you through my shame.

  First, a little background. In the third draft I removed an Ariel/Curic scene from chapter 10, in which Ariel was afraid that if the contact mission ever turned into a "clusterfuck", he'd get shot as a collaborator for his relationship with Curic. Curic said that Ariel couldn't even imagine what a clusterfuck would look like in this context. She was trying to be encouraging.

  And here, in chapter 18, there was originally a very small subplot about Umi, the Farang you saw on TV in chapter 15. It didn't go anywhere so I moved its exposition (fluid overlays, how do they work) to the Ariel/Tammy scene in chapter 17. Incidentally: "umi" is Japanese for "sea", and I had in mind "uni", which is sea urchin. Lazy naming!

  So in his initial handwritten note to Curic, Ariel asks her about Umi, the person who "occasioned this clusterfuck" as he put it. Curic sends a heavily censored response dissing Umi and revealing that many people on the ice-transfer overlay don't really care what happens to Earth's sea level: they're trying to save the ice, which contains valuable paleoclimate data and ancient pollen that can be cloned. I took this out because this callous obsession with data collection is more typical of the Constellation Library than of people who might join a heavy-lifting overlay. (see "The Time Somn Died" for a similar situation)

  But now the disaster. Like I said, Curic's response was heavily censored, but it also contained a secret message from Curic to Ariel, steganographically encoded through the deliberate insertion of fake censorship markers. Decoding the secret message was supposed to be a fun little puzzle for the reader, but nobody in my writing group even saw that there was a puzzle to be solved. They said "where's the secret message Ariel talks about?"

  The simplest way to explain the formatting of the secret message is to present the Python code I just wrote to re-extract it from the second draft:

  import string
  for chunk in plaintext.split(" XXXXXX "):
      words = chunk.strip().split()
      print string.uppercase[len(words)-1],

  If you run that code on Curic's second-draft message it says "STILL NOT A CLUSTERFUCK". Except actually it says "RTILLNTACLSTRFUCKJ", which is worse than "BESURETODRINKYOUROVALTINE." Not only is the message impossible to decode and disappointing once you read it, it doesn't advance the plot and serves no purpose but to show Curic caring about Ariel's well-being, her committment to which will be periodically questioned in the chapters to come.

  So this scene was screwed up on every conceivable level including spelling, and the really important thing--Curic jerking Ariel around w/r/t the importance of the shipping container--was lost in the noise. It had to be rewritten. But I still wanted it to feature some clever evasion of the censorship system.

  The breakthrough idea was to make the cleverness Ariel's, not Curic's. Since Ariel is the narrator, I can show the uncensored message, use exposition to explain what's going on, and just assert that Curic (with Tetsuo's help) was able to decode it.

  I tackled the rewrite in April 2010, about two years ago, on an Amtrak train coming back from North Carolina. I put myself in Ariel's shoes and tried to communicate just one thing--we want Dana to learn Edink--past the censor. Without letting on to the eavesdropper what "Dana" is or what it means to teach her Edink. That's when I came up with the highly redundant medium of limericks. The poems won't win any Pulitzers, but now the subterfuge serves the plot.

• Apropos the censorship system itself: a real system of this type wouldn't rely so much on heavy-handed automation, given the low volume of traffic and the fact that there's already a human at one end of the air gap typing in the message. But 1) this is a satire, 2) real-world government firewalls work not just by denying access, but by degrading performance and inserting fake connection errors. For Pete's sake, allow me a little... Creative License.

Tune in next week for the shocking chapter 19, in which Ariel travels the well-worn road from "unreliable narrator" to "flat-out liar." The chapter in which BEA Agent Fowler will say the ridiculous line everyone tried to get me to cut, but I refused! For you, my readers.

Image credits: Flickr user windygig, Pepe Medina, Danny Cornelissen.

← Last week | Next week →