News You Can Bruise for 2006

Around the time when Cold War started, Bertrand Russell was giving a lecture on politics in England. Being a leftist in a conservative women's club, he was not received well at all: the ladies came up to him and started attacking him with whatever they could get their hands on. The guard, being an English gentleman, did not want to be rough to the ladies and yet needed to save Russell from them. He said, "But he is a great mathematician!" The ladies ignored him. The guard said again, "But he is a great philosopher!" The ladies ignore him again. In desperation, finally, he said, "But his brother is an earl!" Bert was saved.

Bertrand Russell's autobiography is presumably the original source for this anecdote. Here's the event as described in the autobiography (fake paragraphing inserted for clarity). According to the biography, it actually happened some time prior to 1918.

It was decided at Leeds to attempt to form organizations in the various districts of England and Scotland with a view to promoting workers' and soldiers' councils on the Russian model. In London a meeting for this purpose was held at the Brotherhood Church in Southgate Road. Patriotic newspapers distributed leavelts in all the neighbouring public houses (the district is a very poor one) saying that we were in communication with the Germans and signalled to their aeroplanes as to where to drop bombs. This made us somewhat unpopular in the neighbouthood, and a mob presently besieged the church. ...

The mob burst in led by a few officers; all except the officers were more or less drunk. The fiercest were viragos who used wooden boards full of rusty nails. An attempt was made by the officers to induce the women among us to retire first so that they might deal as they thought fit with the pacifist men, whom they supposed to be cowards. ...

Two of the drunken viragos began to attack me with their boards full of nails. While I was wondering how one defended oneself against this type of attack, one of the ladies among us went up to the police and suggested that they should defend me. The police, however, merely shrugged their shoulders. "But he is an eminent philosopher," said the lady, and the police still shrugged. "But he is famous all over the world as a man of learning," she continued. The police remained unmoved. "But he is the brother of an earl," she finally cried. At this the police rushed to my assistance. They were, however, too late to be of any service, and I owe my life to a young woman whom I did not know, who interposed herself between me and the viragos long enough for me to make my escape.

All the elements are in the original: the chivalry, the attacks with random objects, the comical deference to hierarchy. But the anecdote has undergone some well-placed mutations:

• The mob and the women are merged. This means fewer characters to remember and simpler interactions between them. It also changes the tone of the story from sinister (mob of toughs with nail-boards, police won't do anything) to somewhat comical (mob of elderly garden club ladies pummeling Bertrand Russell with their handbags, police try to politely stop them).
• The roles of the policemen and the women are switched. This also changes the tone: the rough chivalry of the mob ("Ladies, please leave so we can beat up the menfolk!") is replaced by the more gallant chivalry of the policemen. The only loss is the final bit of the anecdote, where a woman exploits the mob's chivalry to help Russell escape.

Altogether, I must say the later version is more memetically fit. Therefore, that's what actually happened. Another blow for the old primary sources, Jack. How long can they continue to be given their current respect when mutated versions are so much better?

Tomorrow: more saying goodbye to people. Goodbye!

I don't know much about the deep sea so I was thinking of writing a book that was just full of lies. Some people might call this "fiction" but these would be real actual lies. It would be okay because they would be funny lies, like the coffinfish interview. Actually this is starting to sound like an interesting book project, though somewhat different from the book ideas I brainstormed with Michael yesterday.

Yesterday I went with Dan, Beril, and Kevin to the Vietnamese restaurant near CollabNet. They (the restaurant) recently changed their name and rearranged their menu a little, so now it features clip art and pixellated pictures of food. It also features milkshakes that star the fruits of southeast Asia, including a durian milkshake. I ordered a durian milkshake.

The durian milkshake didn't smell bad (probably made from frozen durian) and it didn't taste good (turns out I don't like durian), so it was an all-around disappointment. But it's a cheap and harmless way to claim you've eaten durian.

Also, robot finds panties

• Hard Like Steel Pk5 (Not Classy Mix)
• Viagra Pro has become available since 1998 and

I wrote one more which I'll re-record once I get set up in New York. Goodbye, California.

• It came with lettuce and tomato, instead of that costing extra.
• It came with "raw onions" (ie. onions) instead of onion rings. Onion rings! Onion rings are a side dish, not a topping!

So clearly we have a thing like the British use of "salad" going on here. This diner's version of "California Burger" is more accurate, but less delicious than California's. Hamburgers in California don't have avocado on them by default, but they do have lettuce and tomato and onions by default. Not at this diner. This only confirms my suspicions that fresh vegetables are as rare spices here in the Northeast.

We walked through Central Park to a Japanese restaurant that Andy likes. While we ate, he brought us up to speed on his research into the creation of mutant mice. He creates a mutant mouse by taking a mouse embryo and replacing the DNA inside some of its stem cells. The mouse develops with two sets of DNA (it's a "chimeric" mouse), and you check the color of its coat.

The mouse embryo would naturally have, say, a white coat, and the new DNA is for a mouse with a black coat. If the chimeric mouse has a dark coat then it's a good bet that most of its cells came from the modified stem cells. Specifically, its sex cells probably came from the modified stem cells. This means you can breed it with another chimeric mouse and get a mutant mouse.

A mutant mouse comes from a modified sperm and a modified egg, so it's not a chimera. It's what you would have gotten if all the cells of the original mouse had come from modified stem cells.

Here is the part I'm not as clear on. Andy is trying to breed mutant mice which have a certain gene truncated. The truncated gene will not be able to produce a certain protein. The protein is in charge of attaching a chemical tag to cells that need to be garbage-collected (or maybe the protein is, in fact, that chemical tag; I'm not sure). So if Andy's mad plan succeeds, these mutant mice will not be able to engage in cell death.

At this point in the pop science article I would talk about the applications of this revolutionary new technique for "the fight against aging" and leave it at that. However I have two other things to tell you. The first is that these mice will not be immortal just because they can't kill their own cells. They will probably get some sort of anti-cancer and die (I speculated as such and Andy didn't correct me). The second thing is that this is not a really new ability. You can already create a mouse whose cells don't die by giving the mouse a new gene, one copied in from a virus. I don't remember what the virus gene does, but obviously it works differently from Andy's technique; I think it might prevent cells from accepting the garbage-collection tag. Andy's technique has the same end result, but it doesn't require genes from other organisms. Once he gets some mice whose cells behave in an interesting way (or he fails in an interesting way), he can write his dissertation and he's Dr. Andy.

I wrote this mainly as a writing excercise, but it's a pretty interesting set of hacks and I never really understood how genetic engineering works, so maybe you'll find this interesting too. Rewiring mice stem cells sounds like a pretty dull job to me, but Andy likes it. I have more interesting stuff to write from our conversation, such as the operations you can perform on a gene or genome, and when you can copy a gene from one species to another. I will write this other entry if people are interested, or if I am bored.

In the museum I told Andy lots of trivia about prehistoric animals and evolution which I'd think he would know, as a biology major, but he was appreciative of the trivia. Maybe he was just humoring me. For instance, they had an Irish elk skeleton. I told him how biologists used to think the Irish elk demonstrated that evolution could get stuck in a rut, with the male Irish elk getting larger and larger antlers long after it ceased being any kind of advantage. This continued until the males couldn't lift their heads anymore, leading to extinction.

This page yields some more Irish elk trivia. For instance, the antlers of the Irish elk are not particularly large for its size. They just look abnormally large because the animal is so big. And the Irish elk also figured in a more prominent scientific argument: whether or not animals could go extinct at all.

We are Internet squatters, taking a little bit of bandwidth from our neighbor NETGEAR. I am back to work on the Cookbook, which is now available! That is, the embarassing rough draft we're still working on is now available, thanks to O'Reilly's perfidious new "Rough Cuts" program. You can read my writing this very minute and then complain about it directly to me. There's one really awful recipe which I was hoping to replace with a better one before this happened, but I didn't get to it in time. Anyway, it'll improve the final quality of the book, though I'm a little distressed by the estimated publication date of September. I was hoping it would be out around July.

There was good ambience and lots of good-looking stuff in the restaurant, like macaroni and cheese and grilled sandwiches, but the hamburger was charred and, though it came with some vegetable fixings they were not very healthy-looking. There was no mustard or mayonaise to counteract the dryness of the patty. The french fries were okay but not great. Sumana's meal (green bean casserole and grilled sweet potato; yes, sweet potato cut in half and put on the grill) was far superior.

And what is the customer supposed to do? You can't complain that you got lowbrow food at a self-consciously lowbrow restaurant. You're trapped. Well played, Mr. Maples, well played. You'll be the death of me yet.

The CnC seems expensive to me (that burger cost $9) but it's probably average for Manhattan. The other stuff on the menu looked good, especially the wide variety of trashy desserts, so I'll probably go again eventually. It's near Union Square.

Today we came fr'over the short sea to see Tristram Shandy, which started out really well and then it got all Adaptiony which I could have done without. All extant material on this film (including the film itself) alleges that the source material is "unfilmable", but they were doing a really good job of filming it until they decided to stop. I'm going to be charitable and say they just didn't have enough money to film the whole thing as a period piece. It reminded me that I need to read the book, though. It's kind of sad how public domain books I download don't actually go into my book queue because of the physical stacks of non-public-domain books taking up space in my house and bugging me to read them.

I really like the idea of the Japanese pastry shop, but the thing I tried (a sherbet jelly made of grapes) just tastes like an Otter Pop. Here's some of the other stuff they had.

Then we went to another restaurant for dessert but the portions were really small. Camille vows that next time we will do things old-fashioned Brooklyn diner style so we can have huge pieces of pie.

Camille is somehow representing or works to book gigs for musicians. She was trying to expand her business and branch out, but then she decided to just go for a niche and own it. Her niche is "eclectic" musicians (eclectic as a group, or, I suppose, individually), so she'll find gigs for you if your act is just a guy with a zither and a girl with a triangle. Very vaudeville-like.

We also went over to see this exhibition of black-box analog synthesizers. Most of them did not respond well to my twiddlings, but I got a good sound out of one of them: a sort of funky metronome beat with Star Trek: TOS ship effects on top of it. People I thought about telling about this exhibit in a gloating manner: Jake, Seth, Brian Behlendorf. Then I decided that doing that would be equivalent to gloating that I lived in New York, rather than that I had gotten a chunk of culture into my otherwise philistine system, which isn't something I should gloat about.

Beautiful Soup needs an update, too, which will come probably this weekend.

• LEGO used to have 1x1 bricks with letters on them, so you could spell things out in bricks. I have a lot of the old bricks mentioned on that page (including a sign that says ESSO), but I never heard of those. A font of Lego! LEGO. This explains all those really short mills and really long amphitheatres you see in old sets.
• This guy mocks bad MAME cabinets. I think this one actually looks really cool, but not as a MAME cabinet.

• "Throwing-sticks in the National Museum"
• An Introduction to the Study of Browning

Neither as interesting as you'd think on first glance.

Also of interest: The New York Subway: Its Construction and Equipment. And from the classics section I recommend My Bondage and my Freedom, which among other things includes a novel reason for not revealing spoilers:

Keen is the scent of the slaveholder; like the fangs of the rattlesnake, his malice retains its poison long; and, although it is now nearly seventeen years since I made my escape, it is well to be careful, in dealing with the circumstances relating to it.

Shake and shake the ketchup bottle
None will come, then Quetzlcoatl

• Miller's Quest is a game you can just let run and it does the dull repetitive things for you. If you are a Windows person you can play its ancestor, Progress Quest. Written by a guy who I think used to write Segfault stories, though I can't prove this. Basically a non-bad Segfault story in game form.

  This game is written in Ruby, and it has a bizarre attitude towards generating colored text, but I'm probably the only person who will notice. I have been thinking about writing an MMORPG in which all the players are scripted robots, but the idea keeps running into problems. I've also been thinking about mentioning in the Ruby Cookbook the bizarre curses-less colored text hack used in this program, but I don't think I want to encourage this kind of behavior.

• Enemy Lines wins many points for extreme literalism (your enemies are lines) and its castlezzt.net-type presentation. It foolishly squanders them all by being kind of like Missile Command. Bah!

  The author has also put out three other games with the same name, only some of which are like Missile Command. The best-looking one is #4, which has a great cyberpunk Zaxxon look.

• You'd think a game whose file is called "todoom" would be "To-Doom", an action-packed slaughterfest organized around the principles of the Getting Things Done productivity philosophy. But it's actually a Roguelike game called Tower of Doom. Pretty good, not great. All your characters are various lowlife types (because the Tower of Doom is a punishment). Peculiarities: no enemy spawning and complete knowledge of all enemies on the level.

  Hmm, since I downloaded this game, the project seems to have undergone a freakish metamorphosis! It's now called Warp Rogue and is much different. That's what I get for writing reviews of ancient copies of games found on my hard drive while I have no Internet access. Developing...

• Zoid's Quest is potentially a really good side-scroller game like Jill of the Jungle. It has an excellent, original fighting mechanic, and good first-level music that reminds me of NES games like Faxanadu. But this game makes many of the tiny mistakes or misalignments that ruin so many NES-like open source games. I should make a big list of these and do a cookbook on how to avoid them, so here are a couple:
  • Jerky or disproportionate character movement. Zoid's Quest has pretty smooth character movement, but as in almost all of these platform games, the speed of the character is much slower than the character dimensions would indicate. A good rule of thumb is that I should not be continually jumping from place to place because it's faster than running.
  • Ladder behavior. When you jump and grab onto a ladder, your horizontal motion should stop or be absorbed by the ladder. The ladder is not a strip on the wall that lets you defy gravity; it's actively supporting you.

    The ladder is not a hole. It's bolted to its platform; it doesn't bisect it. You should be able to walk on top of a ladder square.

    If you stop moving on a ladder you should stay clinging to the ladder, rather than falling off of it.

  • Jumping and gravity behavior. First, you should be able to jump more than one square in height. This overlaps somewhat with disproportionate character movement. Maybe you think it's cliche that your character should be able to jump as high as Mario, but Mario jumps that high for a reason: jumping games aren't fun when you can't friggin' jump.

    Before dropping down below the edge of the screen, you should have some way of seeing whether or not a drop is infinite. You either need to provide some kind of "look up/down" facility, or design the level so that the player has a chance to experience the lower levels before the higher, or design the level so that it all fits on the screen. (This is actually in the Zoid's Quest TODO, as a magic spell.)

  • Enemies. You must have more than one kind of enemy, and each type of enemy must have different behavior, and in particular the first level must have more than one kind of enemy. I know your game is in development, but just make another kind of enemy for us.
  • Okay, that's enough for now. You get the idea.
• Holotz's Castle is the first game to ask, "What the hell are doing all these keys here?" But it never answers it. Instead, it disguises itself as two games with identical mechanics. In one, you must gather keys to escape the level. In the other, you must gather gems to... escape the level. After Zoid's Quest, the movement in this game felt perfect, and your characters gradually become faster and able to jump higher. Nice cut scenes use the game engine and have a somewhat juvenile Nintendo of America-approved sense of humor. The only problems: one touch of an enemy's hand kills you, and it's possible to get stuck in the corner of a block. And there's a big ol' cut scene cliffhanger. (though I should check a newer version; maybe it's done a Tower of Doom).

  "How do you know what I think?"

  "I KNOW EVERYTHING!"

• The other day I was laughing about the classic Rounded-up game Apricots. Specifically, I was recalling the limerick I'd written about the hilarious deaths which are impossible to avoid in that game. Sumana said I shouldn't talk that way online about death, because it might be read by someone who's thinking of committing suicide and it might give them the idea that suicide is somehow fun or that I approve of death in real life. Well, I'm here to tell you that suicide is not fun and that death is a bunch of crap. And there's no better game to tell you this than N.

  N is a game with a wonderful physics model and a million deaths. Your character is a greedy ninja with a lust for gold--and not being killed! Unfortunately you get killed a lot, usually in hilarious ways that top even the deaths in Apricots. In Apricots you might crash your plane into a tiny island in the middle of a lake. Explosion. In N you might do a ninja leap right into an active death ray. Your body plummets the length of the screen and splats on the ground. Then your wireframe corpse bounces around for a bit and hits a mine, which blows it into its little wireframe parts. The parts themselves will bounce around and get into all sorts of trouble: setting off more mines, being shot around the game board by trampolines, being zapped by zap-robots, and so on. It's the most morbid game with the simplest graphics I've ever seen, and I've seen at least five.

  But it's not fun to die, because that means you've got to start the whole frustrating Lode Runner-esque level over again instead of going to the next cool level. Which, under Nietzsche's theory of eternal recurrence, is basically the same thing that happens in real life. So I guess I'm saying it inculcates a healthier attitude towards your own, actual death than Apricots.

  It's also one of the best games I've ever seen on my Game Roundup scale of game-liking: "a flexible set of techniques to use towards your goals, and lots of random variation within well-defined parameters." Except it's not random here; varity here comes from a huge number of well-designed levels. There are also many user-created levels that simply use the player's (non-dead) body as a pinball in the machine of the board, bouncing you around narrowly avoiding rockets and zap-bots and all sorts of horrible fates.

  Oh, I should mention that it's composed of Flash and it's not open source. Good game though.

1. I know you're not supposed to ever ever cut paper with the sewing scissors. But can you cut cloth with scissors designed for cutting paper, or kitchen shears? I don't want to have to buy sewing scissors just to make the curtains.
2. What are the names of the parts of the hardware that I need to hang pots from the ceiling? O-hooks and S-hooks?

• The creation of different types of Lego minifig heads has angered the Elder Gods, and Cthulu Attack Lego is the result. I like the fake Lego catalog storyline the creator added.
• For a long time I've wondered how to see the worst-selling items on Amazon. I don't know why I never tried simply hacking the URL. I saw a cool-sounding book somewhere in the depths of low sales: "Broadcasting From Space".
• At last, the ultimate mola mola portal. Did you know that the Spanish name for the mola is "pez luna"? Moonfish! That always slays 'em down at the comparative linguistic zoology lab. Unfortunately, the field is so small that, having slain the lab inhabitants, there's no one left who thinks it's funny.
• Fractal Images of Formal Systems has a fractal representation of tic-tac-toe which you can play.

In said short, the Highway Patrol officer of "one of our western states" solemnly told us that, even though you may not feel anything, scienticians have proven that a mere four cocktails can affect your reflexes.

The mystery in the movie wasn't that interesting but we kept laughing and laughing at the character studies. We loved the dysfunctional Arrested Development-type family, including Lucille; a classic Buster+George Michael+Tobias; and Cesar Romero, probably my favorite movie-grade-spanning actor, as George Sr.+G.O.B. As Joel Hodgson once said, "Render unto Cesar what is Cesar's!" Also, there was a cop who gave a good impression of what Clark Kent would be like if he wasn't also Superman.

According to the chart, civilization stopped in the 1970s. Which it did, but it started back up again a bit later, and I'd like a chart that reflects that. Also, it's got some fossils of errors that died out in the 1970s. One of the first items of civilization is "Negroes move into Africa from Asia". Uh, no. It also looks like the 1970s, and in general I try to banish the 1970s from my house.

So I'm wondering what kind of cool wall charts you people know of. I've found various sites for updated charts, but I don't know which one's better, or what else is out there. Basically I want something for people to stare at if they're visiting us and they become incredibly bored, because sometimes that happens to me at other peoples' houses and things to stare at are much appreciated. The other main thing I want to get is the National Geographic map of the universe.

Reading John Henry Days, which is hilarious. "One Eye had been blinded in a tragic ironic quotes accident a few years before." I wish I could write like that. Like David Foster Wallace, but with more jokes instead of just wit.

I discovered that if you're making lasagna noodles you should roll them out as you build the lasagna. I rolled out all my noodles real thin, and folded them up for compact storage while I got the rest of the stuff ready. Then I discovered that my noodles had fused back together into thick skinny noodles, and I had to roll them out again. Great noodles though.

Malcolm

"I'll have you know I find this whole thing very distasteful," says Charmed. Slaved to the ship's fins, his wings carve microgravity.

"What part of 'thrall' don't you understand?" says Malcolm. "Ten minutes before last time. Do it n-- as soon as is safe." His voice commands; he can no longer seduce.

Charmed visualizes a familiar point in spacetime. He yawns wide and breathes neutrinos.

Just before last time, Malcolm wails as he dyes Malcolm's lips red. "Diffbuhphting," says Malcolm, but he cannot allow the plague to spread. Each time he does this, his new shape feels a little older.

Snow is really interesting; it's like a visualization of statistical distributions.

But according to this Smithsonian article, the John Frum cult is still active and is gradually turning into a real religion (Leonard's heuristic for cults turning into real religions: the eschatalogical event predicted by the cult keeps not happening, so it gets moved off into the indefinite future or the realm of metaphor). And I can't find the other article, so I guess the Smithsonian one wins. Probably there's just a few people doing cargo cult demonstrations on the side. It doesn't seem like they could support the whole tourist economy.

• Floating rock.
• Lego Churches #2: Church of the Flying Spaghetti Monster (see #1)
• Bacon Activity for Kevin.

Bonus quote:

"'Make money in your spare time?'"
"More like 'Make money by working in what would otherwise be your spare time.'"

That's not what I wanted to talk to you about. I wanted to talk to you about Elvis sandwiches. I'll do that in another entry.

Generally I don't order stuff at restaurants when I can make a superior version myself, and a peanut butter sandwich is well within my reach. So I decided to order "The Elvis", which I could make at home but never actually had. This is a peanut butter sandwich with bananas, honey, and bacon, which gets fried.

It was okay. The experience, and the recent visit of the Elvis sandwich to the Minutillo household, got me thinking: why couldn't I make my own Elvis sandwich? I decided to try it.

It was good; better than the restaurant one, certainly. While making it I figured out a couple of things about the Elvis sandwich. Longtime readers (that is, since December) may remember Eggnog Transitive Closure, where it was revealed that eggnog, ice cream, and French toast have the same ingredients. Well, an Elvis sandwich is basically French toast minus eggnog.

Peanut butter, bacon, bananas, and honey are all things that could reasonably go into the middle of a piece of french toast before you dip it in the eggnog and fry it. Load up a piece of french toast with all of those things, and don't dip it in the eggnog, and you've got an Elvis sandwich. The chain of foods continues apace! Chain chain chain, chain of foods.

Anyway, here's a Windows game that looks pretty cool.

I am not a huge fan of dim sum because honestly I find genuine Chinese food kind of weird. I feel that this is in some sense okay because I read an article in Gourmet about the article writer taking some visiting top Chinese chefs to the French Laundry and other fancy Bay Area restaurants, and them generally thinking everything was gross and weird (though one chef liked the oysters at a Fisherman's Wharf oyster bar). I figure if it's okay for actual chefs to be that un-cosmopolitan, I can be too.

Not that I went hungry. I like the sticky pork buns a lot, so I had two of those. There was also a vegetable of the brocolli/cauliflower/brussel sprout family which was also very good, and a dessert bun with red bean paste. But some of the other stuff I tried made me queasy. And I always felt like a friend's mother was coming around with a cart every few minutes urging me to eat more.

The occupants of cars #1 and #2 now roll down their windows and begin to argue. Who knows what about.

This argument continues long after the light in front of car #2 turns green, and the cars in front of it have moved on. Now car #2 could move too, but the driver's still arguing with the driver of car #1. Everyone loses! Honk honk. Thats's New York.

I think Crow is made out of a nickel alloy, but I can't find a source for that. All I could find was a non-canon Bill Corbett saying Crow is "some strange gold alloy". But the inclusion of Crow in the comic makes up for any inaccuracies. Anyway, clearly he should be made of chromium. Chroooomium.

Today's cool thing is Ruby's midilib, the... wait for it... Ruby interface to MIDI files. I did a recipe that shows how to visualize data as a MIDI file, and how to compose algorithmic music (sample composition).

While learning about midilib I came up with an observation about MIDI, which is that it's good for accurately representing what happens when you play a real instrument. It's kind of frustrating and unnormalized if you need to programatically compose music. I ended up taking a tip from the excellent MIDI Scripter project, and showing how to write a little wrapper on top of midilib that makes composition easier.

The experience gave me a huge number of ideas for musical programming projects. With my near-total lack of theoretical training, I will be unencumbered by traditional notions of "harmony" and "taste". For one, I plan to get into fractal music, though probably not the way the author of that web page defines it. Hard to tell, with that color scheme.

Actually no. I'm just making stuff up to make the comic more interesting while keeping it in its chosen genre of dumb comedy. There's a really interesting idea here about how team sports would work when the players are all part of a hive superorganism. But the actual cartoon takes up the entire back of a cereal box and the payoff is a dumb sight gag and a couple lame puns. Plus it violates the rules of baseball. I may ask Jack Masters for another metaconstrained comic that does this idea better.

[0] The cartoon refers to Buzz as "he", but "he" has a stinger => worker bee => female.

Update: I don't know anything.

Soon I'm going to make some chapati, because how hard could it be?

The constraints were:

1. The comic must be non-linear
2. The Xth panel of the comic must contain a character who's name starts with letter X of the alphabet. See below for a full list.
3. The comic must be a murder mystery, and for every character in the comic there must be an ending where they are revealed to be the murderer
4. You and I must be in the comic. I couldn't find any pictures of Jack, so I represented him by the kangaroo from the movie "Kangaroo Jack".
5. The comic must advertize WHAMMO, made by satan's minions

Sumana requested a suggested order for the panels, to deal with the non-linearity. The best I can do is 4-9-5-1-11-12, and then 7 as the PSA at the end. The rest of the panels are just possible endings.

List of characters: Adam (played by Adam Kaplan), Beelzebub, Cerberus, the Devil (in snake form), Eve (played by a character from the video game Parasite Eve; probably not actually named Eve), Furfur (look it up! I did!), Giant Anthropomorphic Pineapple, Hastur, Imp, Jack Masters, Kraken, and yours truly.

"I'll teach you how to make cheese," he said. It was a lie. "My hand slipped," he'd say. "It's thermodynamically impossible." Always something. She'd had enough of the excuses, the anthills left by unfinished projects on the back porch.

It had to stop, so she stopped it. With biscuits. The sour milk, the sawdust. She cleaned it all up, not the way a woman usually does, but for the first and last time.

"Look," she said then. "I've invented fiberboard."

The main problem is the arbitrary restriction of Amazon Web Services where you can't get more than 250 pages of query results. If it weren't for this restriction, you could pick a random number from a huge range and do a Browse Node search to get the corresponding page of results. This is still a pretty good solution if you want to search for products in a restrictive set like the set of science fiction books: find the corresponding Browse Node and do a browseBestSellers.

As it is, the simplest solution is to only search for a single word, instead of Jarno's two. Here's some code that does that and works pretty well.

import amazon
import random

def random_product(keyword, max_page=250, associate="crummthesite-20"):
    result = None
    while result == None:
        page = random.randint(1, max_page)
        try:
            results = amazon.searchByKeyword(keyword, page=page, type="lite",
                                             associate=associate)
            if results:
                result = random.choice(results)
        except amazon.AmazonError:
            results = None
        if not results:
            max_page /= 2
    return result

def format_product(p):
    print '<a href="%s">%s</a>' % (p.URL, p.ProductName)

keyword = random.choice(open("american.0").readlines()).strip()
format_product(random_product(keyword))

Sample book: Reutilization of Waste Materials.

The other solution is screen-scraping. As covered earlier, there is an Amazon URL such that you can randomly vary a parameter and get a more or less random set of books. Here's some code that fetches one of those URLs and grabs an ASIN at random. Then it does an AWS lookup of that ASIN because life is too short to scrape all the appropriate information off that big ol page. This is a little less reliable than the earlier implementation, but it gets a better random distribution of books (since the other one picks a word to search on, and word distribution is not random).

from BeautifulSoup import BeautifulSoup
import amazon
import random
import urllib2
import re
import socket

URL = "http://www.amazon.com/exec/obidos/tg/new-for-you/top-sellers/-/books/all/books/0/1/%s/1/"
def random_book(max_page=150000, associate="crummthesite-20"):

    product = None

    while product == None:
        url = URL % random.randint(1, max_page)

        req = urllib2.Request(url)
        try:
            soup = BeautifulSoup(urllib2.urlopen(req).read())
            asins = soup.fetch("input", {"type" : "hidden",
                                         "name" : re.compile("asin\.*")})
            if asins:
                asin = random.choice(asins)['name'][5:]
                product = amazon.searchByASIN(asin, "lite", associate=associate)[0]
        except socket.error:
            product = None
        if not product:
            max_page /= 2
    return product

def format_product(p):
    print '<a href="%s">%s</a>' % (p.URL, p.ProductName)

print format_product(random_book())

Sample book: Choice, Welfare and Measurement

Now I've discovered Rice to Riches, which only sells rice pudding with lots of different toppings. I think if I start a restaurant it would be easier to start one of the really specialized ones. Let's say, I would sell only choux puffs with various sweet and savory fillings. Meanwhile, here's a copycat recipe for their Rocky Road rice pudding.

• Just when you thought Lucene couldn't get any cooler, now it has an Atom-based web service API.
• What causes double rainbows? The surprising answer... after this!
• A cool-sounding movie about the sun. No details are available about this cool-sounding movie, but you can learn fun facts about the sun.
• This PBEM game about the Singularity sounds like a great idea, but after reading the rules I think the dressing is the main cool thing, and the game mechanics are kind of generic. Let's come up with some Singularity-specific game mechanics.
• Displaying large datasets atop Google Maps

Turns out double rainbows are caused by inflation. Now here's Bob with the weather!

  The Bard subsists on simpler food:
    A dinner, not severely plain,
  A pint or so of really good
        Champagne--

I don't understand anything about poetic meter or why one meter is funny and another one is serious, but that one's funny. It's a meter with a punchline.

Just part of my longstanding tradition of taking the logic of Transformers at face value when nobody else does.

• General MIDI
• Wikipedia recently deleted pages.
• Black Like Me
• Production Code
• Laser harp
• Suspension bridge
• Dog Latin
• Intellectual rights to magic methods

Unfortunately, this web meme has no way of propagating, so it is sure to die out. Just like the laser harp.

For a while before I A game was designed; Non- reliability of conversion and the inclusion of mendacity of translator. Boredom without finding that very, one condition in order to do the game went the tool which in order to distort message with use to converting that of the automatic translator on front and back is automated. Multibabel There is a kind of thing which I had with heart.

The latest entrant into this fray is lib.rario.us, which looks just like del.icio.us except it's for keeping track of things that have an ASIN or ISBN. I like the del.icio.us interface a lot (mainly because it's easy to hide it behind a NewsBruiser interface), but lib.rario.us doesn't offer even one tiny web service.

Amazingly, though, the traditional del.icio.us rip-off name proves itself useful for once: it's given me the obvious-in-retrospect idea of just using del.icio.us to keep track of my wish list. Use it as a platform. Why didn't I think of that before? It could work, once I put up something between me and del.icio.us that looks up ISBNs from titles.

"Troodon: Dinosaur Genius" is about the Troodon, which had an opposable finger and the largest brain/body mass ratio of any dinosaur. Pretty soon the documentary will reach the traditional "unwarranted extrapolation" phase, where people will speculate that, were it not for the K/T event, Troodon could have continued its trend towards greater intelligence and become a sentient lizardoid species. Yes, just like in that episode of Star Trek: Voyager.

The main problem with Paleoworld is that there's no warning when it goes to commercial, and the voiceover announcer for the show sounds just like the voiceover announcer in the commercials. So you hear things like this. "But the story of Troodon does not end here. It launches us from the dinosaur age to the space age. The latest Mars Orbiter is about to arrive!" And "Troodon was a dinosaur with incomparable brainpower. All of us have internal plumbing. But sometimes our pipes doesn't work as well as they should."

I once worked with a guy who was going to voiceover school nights, and he said that there is a standard set of voices that you learn in voiceover school. I guess this is so the school and employers can grade you on an objective scale, but it's really creating a voiceover monoculture.

I first noticed the popularity feedback loop back in the BBS days. A common feature of BBSes was a "Most Popular Files" list, and people just kept downloading the files on the list, which to my mind defeated the purpose of advertising these files. I set up a "Least Popular Files" list that listed all the files that had never been downloaded. It didn't really work.

• List of fictional plants
• List of fictional planets

Bonus Wikipedia link: Kappa

The White House maintains that Gamera is a friend to all children.

I redid the outline so it shows the reviewer coverage of the various recipes. I'm also working on a script that grabs the code snippets out of the recipes, runs them, and sees the results match what it says in the book. That one's a little tricker.

Incidentally, I recently became a Texas oil millionaire. Why you laughin', li'l buttercup? Ain't it the truth, why it says so right here on this printout of a web page about my bank account.

So, the variant: you start off with two ending cards. Whenever it's your turn, you must play one of your ending cards before you can play a story card. You can just play the ending card and start tying it into the story, or you can talk for a few sentences before playing the ending card, to connect the current story to the ending. No one can interrupt you before you play your ending card, but you can't play any of your story cards either.

Once you play the ending card, your turn continues as in vanilla: you play story cards and try to get to the point where you can play your second ending card. If someone interrupts you, you must draw a story card and an ending card (so you begin each of your turns with two ending cards).

If someone's story gets to the point where you could play an ending card, you can interrupt with the ending card and start your story from there. (This actually happened.)

Since OUaT stories jump around a lot anyway, it's not difficult to use an ending card to close one thread of the story or to spawn a new thread. The additional constraint is generally what I need to feel comfortable making up an OUaT story.

Update: See comments for feedback from OUaT co-author James Wallis.

Does anyone else remember this paper? Maybe I should have developed an intense emotional hangup when I first discovered it, so that I'd always be revisiting that moment and I could remember the URL.

I am reading The Dalkey Archive but so far it's a much less funny ripoff of The Third Policeman. It better get surreal and fast, or I'll... regret my purchase!

But come on. They look like ghosts. Specifically they look like Charlie Brown ghost costumes, baggy sheets over the head with holes cut for eyes. That's why people think they're ghosts, not because one port twenty-five years ago couldn't handle five moving objects on the screen at once. People don't see random things flickering and say "oh, that must be a ghost". There has to be some preexisting resemblance to a ghost. So don't blame people for thinking that the monsters are ghosts.

Instead of simply asserting that the monsters are not ghosts, it's pretty easy to prove it, a la Aristotle. A ghost is the disembodied essence of a dead person. When Pac-man, under the influence of a power pellet, eats a monster, the essence of the monster is separated from its body and goes into the penalty box to regenerate. What is that disembodied essence but the monster's ghost? How can a ghost have a ghost? It does not make sense.

Hopefully this proof can bring some civility to the enlightenment of those who think the Pac-Man monsters are ghosts. This is my Pac-Man heresy, brought on, I believe, by Pac-Man fever.

I originally tried to write this entry in the style of an Arrested Development episode, but though it was formally correct there were no jokes in it. This is how you repay me for how I repay you?!?!

I developed a samosa song which I sing when I'm awaiting samosas. It goes: "Samosa!" Except you say it on the fourth beat of the measure. I think it could be the next "Tequila".

I reread this essay today and I must have gone through postmodern puberty since my first reading, because this time I saw the subtext. Human culture is supposed to change and propagate quickly, by Lamarckian means. But the blind, semantics-ignorant copying seen in these textbooks shows that culture propagates by slower Darwinian processes too.

Textbooks evolve through sexual selection. Nothing selects against this bizarre and useless (and, Gould claims, wrong) comparison between Eohippus and a fox terrier, because nobody ever reads that sentence and tries to make sense of it. It is junk DNA. The whole text of the textbook only exists to make it larger, so it looks more attractive to school districts. All the resources are going into improving the textbook's plumage.

The reason I read the essay in the first place is that I've seen the fox terrier comparison three times in the past month or so. This is twenty years after big-shot paleontologist Gould publishes an essay totally slamming the comparison, with no one coming forward to explicitly defend it. The first time I saw it was next to an Eohippus in the horse evolution exhibit at the American Museum of Natural History. I guess it was about the right size, but I've seen a real Eohippus fossil and I've never seen a real fox terrier, so what do I know? Could we get a more helpful comparison in a real museum exhibit right next to the actual skeleton? (The Gould essay makes a tenuous connection between the AMNH and the original source of the comparison, so maybe they're just being stubborn.)

The second place I don't remember where I saw it. The third place was in a book I just read called The Eternal Frontier, an ecological history of North America. Apart from the fox terrier comparison it was a pretty good book, though it was written by an Australian which means it exhibits the cowboy obsession common to non-North Americans who write about North America. It was also pretty depressing since it portrays the entire history of life as organisms going in and really screwing things up for other organisms (though occasionally an non-organism like an asteroid or an ice cap really screws things up for everybody).

Nothing has really happened except slow spotty progress on editing the book. People are starting to ask me what I'll do next. Next I will decide what to do next.

I could buy the robonic (were they trying to get a trademark?) modifications if the show took place in a Harrison Bergeron society, where as a matter of social policy people at the Stooges' level of competence were given special abilities to bring them up to the level of the general population. But it ain't so. There aren't even any Three Laws of Robonics. The only thing this cartoon has going for it is that according to IMDB, the guy who dispatches the Six Million Dollar Stooges is code-named Agent 000. The show predates and has the same basic plot as Inspector Gadget (minus the PSAs, but if you started doing PSAs after Stooges shorts you'd never be able to stop). You think that'd be worth something, but it's not.

Obviously the Stooges are not some wonderful cultural icon that has been demeaned by this stupid cartoon, but that doesn't excuse anything. Though by any measure the Stooges come up short when compared to the Marx Brothers, they have the same basic dynamic, and it's one you don't see enough anymore. Characters with no marketable skills, who cling together more through inertia than attraction or synergy. Coveting a position in high society, they end up going for it because they've seen that nothing really separates them from the current inhabitants of high society (who have no marketable skills either). But their plans are jeopardized by their own self-destructive behavior. <-- This paragraph doubles as my "outreach" attempt to explain to women the appeal of the Stooges, such as it is.

Would you like to learn more about earwax? Too bad. Already the knowledge burrows through your tympanum and into your brain!

• It starts with a joke.
• It ends with a real conversation I've had. (Conversation was fictionalized to fit the Cat and Girl characters.)

• The strip should start with a bad pun, and deal with the repercussions of that pun.
• At least three panels must be in languages other than English; understanding them should be helpful but not necessary for appreciating the comic.
• Cat wins.

With further ado, the comic:

Tomorrow: Kevan's constraints, if I have time.

That can't be a good use of my time.

Also, I think this is the first review of my music not written by someone who knows me.

There are some twisted songs out on the net, and if you search for the good stuff, you're bound to come across a couple that, bad as they are, they're really catchy, and you laugh everytime you even think about the song.

Bad but catchy: that's me.

Incidentally, my gee-tars were delivered the other day, so once the book is done and I'm in the post-book era I can resume recording my new album.

There once was a man from Peru
This limerick ends on page 2.

It's in the book. I've got all kinds of silly stuff in this book. I had an earlier limerick and cut it, but this one's in there.

In the first two to three days of a stick Spam wave therefore the applied shares rise over up to two per cent. Afterwards the course consolidates, in order to sink in the consequence strongly. Apparently this is in all rule frequently the case if those coat the Spam clients their profit. This behavior is statistically significant in each case, avowed Rainer Boehme.

He avows it! It must be a real effect! But not drastic enough to see just by looking at graphs.

Actually I just gave the stock spam code to another German guy (a Ph.D student), but he's not one of the ones who did this report. And last July I gave the data to yet another researcher, but he was at U Penn. So this is a previously unknown use of my data.

Everyone has relativity and genetics and evolutionary biology and quantum physics and plate tectonics, but only the US has creationism (similarly for intelligent design, or as I call it, c******nism). Only the USSR under Stalin has Lysenko-style evolutionary biology. Everyone has astrology and alchemy, but everyone's astrology and alchemy is different. And only the UK has this special newspaper-based "formula for the perfect x" branch of science.

This is where you take something vague, dissect it into a bunch of vague contributing factors, assemble them back into a vague equation, and claim that it represents perfection. This gets you in all the papers and nowadays the weblogs as well. As far as I know only one article two years ago has ever dared to point out the ridiculousness, and it's still going on.

Oddly, NYCB has previously encountered Len Fisher, unwitting founder of this particular branch of pseudoscience. He's the one who started it with his measurements of liquid absorption in biscuits, and his use of a real, preexisting equation to describe it. I think he deserves some kind of meta-Ignobel for the monster he's created.

Entering the realm of speculation, there's also an indication that stock spam actually destroys wealth by reducing the capitalization of the spammed companies over the long-term (though maybe the wealth just leaves the stock market and goes into the spammers' pockets). Also includes a movie-plot scenario where spammers use the techniques of stock spam to start runs on banks. An entertaining paper. And it mentions me, which is awesome.

Inspired by Kevan. The stress of the replaced words is often wrong, and sometimes the result just doesn't scan, but it's sufficient for the secret also-Kevan-inspired project for which I wrote it. It works especially well on cadences that have been drummed into your head for years.

A: Leave your rabbit alone!

Not only does it find great pictures but it links to things like this summary of really old travel books (summary contains durian), a genre that has always fascinated me. As you travel further, your reports back home get less accurate.

• The first line of dialogue should be taken from a recent entry in a weblog you've read today. This was a very difficult constraint because it was so wide-ranging. In the end I just picked a recent sentence from Kevan's weblog.
• The strip should contain a complete recipe (simplified as much as you like, but enough that someone could reasonably prepare it from those instructions).
• Neither Cat nor Girl should appear visibly in the comic.

Sumana

We see love between others thousands of times but only participate in it a few times. Can we hope to find a new kind of love? How would we know it if we did? The best we can do is to explore the deeps of one great love, always finding new aspects of it.

Five years ago I stumbled upon a vast love which my lamp and map have still not charted. This is where I want to live, Sumana, among the crystals, the faraway noise of an underground city, the glimmering lights in the ceiling, with you. Marry me, Sumana.

But the success of 24 proves that you can get people to incorporate crazy political ideas into their worldview by presenting those ideas in story form. So I had an idea for a political thriller TV show. The elevator pitch is 24 meets the last season of The West Wing. It starts with a third party senator (like Jesse Ventura, but more politically savvy) who decides to run for president. He starts out wanting to act as a spoiler, but then he meets a programmer who works for the company that makes the voting machines, and he decides to win.

At the end of the pilot, you've got got two overlapping stories. The senator tries to make his spoiler campaign look like something that could really win the election, and the programmer works to form a conspiracy within the voting machine company. Not only does the conspiracy have to cover their tracks, they must form an ad hoc political organization, changing their voting machine hacks to deal with downticket races and otherwise make the voting results look realistic, without having any contact with the real campaign.

This can go for half a season or a full season. There are twists and turns but the two teams win/steal the election. Then the real fun begins. The conspiracy is discovered and publicised, but it's too late.

This will never be made into a TV show, mainly because nothing I ever say will be made into a TV show, but also because all American TV shows seem to have a premise that can extend, in theory, for five seasons. I don't see any way to make a presidential election and a constitutional crisis run more than two seasons.

We got married in the Shakespeare Garden in Central Park. Nandini, Andy, and Camille were there. The officiant was the Reverend Tony Johnson. We read our weblog proposals to each other, and also poems. I read Shakespeare's Sonnet 116, and Sumana read Yevgeny Yevtushenko's "Colors".

Then we were married! It was awesome. We devoured snacks that I brought: pears, pear-based muffins, and brownies. Camille left to go to work, but Nandini, Andy, Sumana and I went to Belvedere Castle and looked down at the turtles in the turtle pond.

So far we've gotten two gifts! (But don't send us more gifts; see below). Andy's family gave us a great new Shun knife with a caricature of Alton Brown's face on it--a knife that claims to be dishwasher safe! And Brendan and Maria sent us a bamboo steamer, in reference to when they were in New York and I went with them and a friend to Joe's Shanghai in Flushing and ate juicy dumplings.

So, instead of gifts, we would like you to send us advice about marriage. And nice letters. Sumana says that if you want to spend money, you should give to one of the charities Steve and Alice listed when they got married, starting here. Also, Sumana needs a new Brita pitcher and a bookcase that folds up--but don't tell her I said so!

Nandini and Camille took pictures, so soon you'll be able to see pictures.

The other major new feature is that you can now rip out a chunk from the parse tree with the extract method. You can use the chunk and abandon the rest of the tree, or vice versa. This is especially useful because the data structures you abandoned can now be garbage-collected: in current Beautiful Soup, the whole tree stays in memory forever because every Tag and NavigableText is connected to every other Tag and NavigableText through an intricate web of lies. And by "lies", I mean "instance variables".

There are some more new features, but I have to take a shower now to go and meet Pete Peterson II for dinner. Test it out; I'll be rewriting the documentation over the next month or so, and hopefully by then I'll have gotten enough feedback to release it.

I heard from Michael that the Ruby cookbook is in the neighborhood of 900 pages. That's as big as the second editions of the Perl and Python Cookbooks, which makes sense because it's got about as many recipes. We just didn't know before because the wiki doesn't automatically generate the Docbook output that gets printed on pages.

900 pages! I wrote a Neal Stephenson book! (Sumana: "Does it have an ending?" Me: "No, it just kind of ends." Sumana: "Yup.")

More miscellany coming soon; I typed up my mother's annotations to her copy of the James Beard Cookbook. I'll put that on a separate page instead of in NYCB.

After a long morning we thought we had it all done. Then I opened a cupboard and there were about ten boxes: "Rachel's Schoolwork", "Leonard's Postcards", "Susanna's Certificates", etc. etc. We let out a collective groan.

But what a find. "Roy's Postcards" fill up two boxes. There are more than 500 of them, and the vast majority were never sent to anyone. My dad was a compulsive postcard buyer. He bought them for himself, and wrote on each postcard a description of how he got it, or what he'd done that day.

In our excavations we have encountered many of my dad's journals. Usually he'd only written on the first five or ten pages. But these boxes contain a journal going from 1982 to 1991. My dad could keep a journal going; he just needed the right medium. He needed a LiveJournal.

The vacations, museum visits, and other memorable moments of my youth are especially well-represented in the postcard box, since that's where he usually got the postcards. So on balance, I'm glad we found all those extra boxes of stuff that needs to be scanned.

I haven't even mentioned "Leonard's Postcards", which include many examples of me-learning-to-write hilarity such as this one from 11/23/1984. In this piece, I start off following the rules of a mailed postcard, but as space runs out I get more and more desperate, finally piling all the letters atop each other in the corner like a typewriter that won't line-feed:

TO DAY WE WENT
TO DISNE-
YlAND
ANDWE WENTOBIG THUN-
DER MOUNTAI
NANDIDIDNOTL[big jumble of letters]

I think that Sisyphus, Brendan's Anacrusis from May 4, is apropos.

Continuing the tradition unwittingly started by my postcard panic entry, I give you an e.e. cummings-esque letter sent by my mother to her aunt LeJeune:

                        oct ober 12  1956
                           o                d
                                             ear jeuny

                thankyou for the new c clothes
An ne likes the shoes and hat


            very much
                      so we hunng them up high.


                            love frances

Sorry, nothing else going on.

"I want the book about 'Being A'... and 'Most Excellent'... and it has woodcuts."

''.join([e for e in soup.recursiveChildGenerator() if isinstance(e,unicode)])

Where soup is your soup object. I really like the if conditional in list comprehensions; I don't think Ruby has anything like it. I can think of ways to simulate it but they kill its simplicity, which is what I like about it.

Short entries these days, because I'm spending all my time writing a story.

"You're the apple-finder!"
"Wait, I'm the Apple Finder?"

This is a good movie but, pace Brendan, not the best ever made. The problem is that while it's stylistically excellent, it follows the noir mystery formula too faithfully. The best noir mysteries have a non-stylistic twist on the formula (The Big Lebowski, which for my money remains the king of stylistic twists as well). The best noir movies abandon the mystery element altogether and just go after the depravity of man (Double Indemnity). Or you can just be an extremely confusing mystery like The Big Sleep, which gives you some points in my book. In between you have movies like The Maltese Falcon and Brick, which are very stylish but formulaic.

I think the other major noir subgenre is the underworld film, but I haven't seen any of those yet, so I'm even less able than usual to pass ill-informed judgement. Those movies seem kind of sensationalistic and Manichean, but I could be wrong.

Phyla

"We are the whales," says Phyla. "the ocean's voice. Keepers of the deep eternal secrets." She strokes the fishbowl of warm seawater.

"We want krill," she says, surprising herself. "You have boats. Go north. Bring us krill." Even this crowd hoots and leaves. A few stay.

"Prepare to be boarded!" says Phyla, still translating. "Drop nets, me hearties!"

"Good job," say the whales. "Now..."

Then, Phyla gets another radio in her head: one eye's made of crystal. "Fishing craft, change course ten degrees."

"I know who you are," she says, "and why you're here."

To her crew: "Load the cannons."

"Parrots don't make that noise!" said Sumana.

"They do if they're around penguins."

We met Dorothy of Cat and Girl, who was as shy as I would be if I had written an amazing webcomic and had to stand around while people fawned over me all day. WHAT I AM SAYING IS THAT I SYMPATHIZE.

We wandered for a while (usually but not always a good strategy) and found a good vegan pan-Asian restaurant and a chocolate-obsessed store. Now we are admiring our huge stacks of comic books.

Here's a still Jimbo sent us, though not from footage that was actually used. I wonder if this qualifies us for IMDB entries. I'm only credited as "Leonard".

I recommend you watch it if only to use our gaudy dress as markers for the amazing continuity errors. We warp around the bar, appearing in almost every scene because we were the only extras who showed up on time for the shoot.

The episode itself is best summarized by a Trevor line: "That's the second-best song I've ever heard about big butts!" Later I'll tell you about a funny scene that they cut.

PS: our band is called "early internet".

Then we went to karaoke, which was fun. Jake and I did a duet on "A Boy Named Sue". Karaoke tip: if you're going to sing "Piano Man", save it for the karaoke contest.

Then I think I figured it out with a Flatland analogy. Suppose Flatland exists in a 3D universe, but one in which nothing is more than a Planck distance tall. The Flatlanders extend in three dimensions, but they only perceive two. They would say that the third dimension was "rolled up very small", even though that doesn't make sense.

Wikipedia seems to back me up on this one, ladies, so the first question is, am I right? Does the analogy hold? Second question is, what is with the "rolled up" nonsense? Are they trying to convey that the structure of space makes it impossible for something to be large in that dimension ("the universe measured along these additional dimensions is subatomic in size")?

"I got a map of the Pacific Ocean."

"I don't like maps of oceans. They're too big. I thought you knew that."

"Well, what if it's like a MAD fold-in? See? Now you can walk from Idaho to Japan."

"That's better."

Then we practiced and played the gig (which went well), and the next day my left hand hurt. The elasticity of the inner tube that made it so easy on the shoulders made it very hard on my left hand, which had to support the neck of the guitar as well as do the fingering. Take that, people who reflexively build anything they see described on a weblog! Fortunately, the pain is now mostly gone.

If we can figure out a way to do that kind of strap that doesn't stress the left hand, we'll have it made. It was really comfortable until I stopped using it.

Sumana likes the highbrow comic books they put out nowadays. I've read some of hers and I didn't like those a whole lot either. So what does picky me like? Well, I like ACKs. I like books of comic strips. Though I don't like the comic books themselves, I love the insane crowdedness of the Marvel and DC universes, caused by the respective companies having to crank out twenty books a month for decades.

Jake has a bunch of comic books, and I trust Jake's taste, so when I was in Worcester I decided to give the traditional comic book a try. Verdict: not great. The plots are pretty dull and the dialogue is either stilted or fake-casual. The drawing's good but I don't really notice that kind of thing.

Highlights:

• The community of comic book creators, as revealed in the books, is interesting. They dish dirt on each other, make up ridiculous retcons and silly nicknames. It seems pretty tight-knit and full of love-hate relationships.
• The Green Lantern comics are a lot like really old science fiction, especially the Lensman novels. I think those stories work a lot better as comic books. But I wasn't crazy about the Lensman novels and I don't want to read a lot of Green Lantern either.
• Silver Surfer has a ludicrous story and some of the most stilted dialogue I've ever seen, but it's really compelling. I think this is what comic books feel like to big fans.
• The Nethack phrase "undergo a freakish metamorphosis" shows up a lot in comic books. I don't know what the original source is.

But the big highlight was Jake's set of Ambush Bug comics, which were everything I want comic books to be. Unfortunately, Ambush Bug can only exist in the context of everything I find boring about comic books.

Ambush Bug is if Checkerboard Nightmare lived in the DC universe and thought he was Superman's pal. He had bit parts in Superman comics (including one where he kind of casually figures out Superman's secret identity), and then two mini-series of his own. Which actually ended the same way Checkerboard Nightmare ended. But which mainly focused on the weirdness of obscure comic book characters and the nature of continuity. It's really funny stuff and I recommend it. The miniseries also has parodies of those comic book ads that sell junk that doesn't work, which aren't that funny, but you can never have enough parodies of those ads.

Now I've read it and I'm happy to report that there's not much overlap. It's true that these are the only two pieces of fiction I know of that use MAME as a dramatic device, but the purpose is totally different. I would actually say this book is closer to Frog/Antifrog.

Okay, about the book itself. The middle is a Douglas Coupland-esque slog, but the beginning is fun and the end has a twist that I should have seen coming but didn't. Good job--the best kind of twist. Plus, there's a tiny piece of exposition hidden in the book that makes the central reveal semi-plausible sci-fi instead of squishy novel mysticism. I think many people who didn't like the ending missed that piece of exposition. But I say, thanks for including it, D. B. Weiss.

• Flatland goes to war in Dangen, the geometric shoot-em-up. Minimalist Galaga-like sharpshooter action. Fun and elegant.
• Less elegant but more three-dimensional-looking is Triplex Invaders, featuring hot triplex action. Everything is rendered in real-time as big Dactyl Nightmare-style polygonal shapes. Cool Ghostbusters-like beam weapons add to the fun. Lousy projectile weapons subtract from the fun. Can this game achieve a fun surplus?
• Lämmerlinge is a fun Javascript game where you guide sheep around a board, feeding them flowers and preventing them from wandering off or drowning. Documentation is both in German and totally superfluous. Levels are password-protected: also superfluous since the passwords are right there in a supplimentary Javascript file.
• Deadly Rooms of Death is a fun shareware puzzle game with a video game sense of humor. This is not a sophisticated sense of humor, not an interactive fiction or Ealing Studios humor. Maybe a ZZT game humor. If you saw it in a sitcom it would be undistinguished, but so few video games these days have any sense of humor that DROD's juvenile antics are charming. DROD has lots of puzzle elements and scriptability that make it more enjoyable than Rocks 'n' Diamonds. Fun voice acting separates this game from free software games with no voice acting.
• Trackballs (Debian: trackballs) captures the sleek futuristic look of Marble Madness but seems really obsessed with narrow passageways for your marble to navigate, instead of the sweeping ski-slope wireframe mountains of the original. On the plus side, your marble can jump, and the game levels are scriptable with Guile. So make some levels that don't rely on split-second marble manipulation cause I don't do that. Oh, this game also has enemies that you can squash with your marble. Unfortunately this does not make your marble bigger a la Katamari Damacy.
• How's that crackpot scheme of yours to mine Mars for mercy and sell it back on Earth? Not going too well? Maybe you should have played the cautionary game Mars: Land of No Mercy. You'll find out that anything that lands on the surface of Mars can only move once and then has to wait for the opponent's move, which never comes. Basically the game is still in a very early stage. Nice realistic Mars graphics though.

Confusing the point of a restaurant with the mission of a "Saturday Night Live" skit...

In the name of "new style sushi" Ninja employs rice cakes as beds - or sometimes graves - for a rectangle of truffle-flecked omelet (it tasted like soggy French toast), a sliver of sautéed foie gras (pleasant, but how could it not be?) and a finger of seaweed-crowned mackerel (fishy in the extreme).

Adam thought this was orientalism at its finest, but this restaurant is an offshoot of one in Tokyo, so there's clearly some self-orientalism going on as well.

The only cool part: the restaurant is camoflauged. It looks like an apartment building and you go down in the elevator and the door opens to reveal a ninja standing in front of you. They should cut the dining experience short right there and call it "Be Scared by a Ninja For $5".

• Overgod looks like one of the games that comes built into your Hitchhiker's Guide. It's a visually appealing 2D space shooter where you're free to move about the arena shooting at other ships. It's a lot of fun, but would be more fun if your ship weren't so slow to maneuver.
• Linux Shuttle Lander is a frustrating take on those frustrating "lunar lander" games (in fact, LSL author Thoma Raes also wrote one of those), where you have to land the fuel-less Space Shuttle. The viewfield is cleverly centered on the piece of the sky you just went through, so that you don't see the ground until a split second before you hit it. It doesn't end well. Let's hope the real Space Shuttle fares better. At least the game code is reentrant.
• Mega Mario, not to be confused with Molto Mario, is a clone of the Super Mario Bros. engine. Which one? Well, all of them, sort of. The graphics and sound are taken from the three NES games and the SNES Super Mario World, which gives a very weird effect. You haven't lived until you've seen SMW Mario stomp a SMB1 Goomba. Ironically, the Goomba has lived right up to that point. In what was surely the defining feature of this software project, Mega Mario features monochrome red blood spurting in precise arcs out of you and the creatures you stomp.

  The mechanics are kinda off: for instance, your Mario is just slightly too big which means that if you don't position yourself just right you'll punch two bricks instead of one. I don't generally make a big deal out of these little issues but c'mon, it's-a me, Mario! The Mario games are classics because of their polished, flexible gameplay. Anyway, the level design is the big problem here. It doesn't have the cleverness of any real Mario game. There are just mazes and coin bonanzas. I've seen the same thing in ZZT games and Rocks 'n' Diamonds boards.

  Speaking of games that rip off the Mario sprites, Super Mario PAC looks like fun but I can't play it because it's a Windows game. I think it's a game where Mario forms an organization to buy off Mushroom Kingdom politicians.

• On a very special Game Roundup, it's Pango: Mudbath, a pygame game by Python guy Moshe Zadka. You're a penguin and you wander around an unobstructed 2D field shooting at squares. Because squares are getting you down, man. Pretty good start on a game, but the name is too similar to the arcade classic Pengo.
• 304. This card game has not been modified.
• Paratim is a clone of an old DOS game called Paratrooper. Unfortunately, the original is totally superior in every respect except for the DOSsy circumstances of its birth. The original was a game that played the Toccata and Fugue in D Minor--not the Phantom of the Opera part but the part that sounds like a helicopter--and then tries to kill you with helicopters. This game can't live up to that.
• A while ago I rounded up a game called Tower of Doom, which it turned out had changed into a totally different game called Warp Rogue. Despite the promising name the game is not that good as a Roguelike. There's no exploring, only tactical combat and corpse-looting for the resale value. It does have a nice advancement system (you spend experience points on new abilities and improved attributes).
• And now what I hope to be a new feature of Game Roundup, Interactive Fiction Roundup. I haven't played any IF in years and I have lost track of the "scene" assuming it is the kind of thing that has a "scene" and not just a room with a description and geographically-defined exits. But the release of Inform 7 rekindled my interest, so I'm working to catch up on what people have done since, say, 2001. The results of my working will make it into future Game Roundups.

  And this one! Today we have Chris Klimas's Blue Chairs, a 2004 game that is kind of a modern version of IF chestnut A Fable. Fable was not even a real game, only part of a game, written by Stan Heller in 1985 to test out GAGS (game development precursor to AGT). Somehow it got included as one of the sample AGT games, which was a bad idea because from a game design perspective it's not good (even by AGT standards). I liked the writing though. You wandered around a landscape experiencing vignettes from your past.

  Anyway, the game I'm actually reviewing. Blue Chairs has all the good aspects of Fable, but it also has a real (surreal) plot, good NPC interaction, and really good descriptions. It mixes linear and nonlinear elements well, with many endings that are nonetheless easy to all see with a strategically placed saved game. It's also got this (spoiler), and the near perfect example of using characterization as a puzzle element.

A complex system that works is invariably found to have evolved from a simple system that worked.

(The first sentence of Gall's law)

I decided to buy this book and read it to make sure I wasn't quoting a book that said crackpot things about freemasonry or something. Of course, now that I've bought it I'm able to search on a key phrase and find the whole first edition online.

I had my hopes for this book. In my mind it was a romantic relic of the age of cybernetics. It would have feedback diagrams and equations vaguely remembered from CS112. It would have lots of other quotable bits backed up by distilled essence of case study. It would be the Mythical Man-Month or the Peopleware of systems design.

The truth is more prosaic: it's a pop-psychology book. The bibliography includes pop-psych classics I'm OK, You're OK and Games People Play. It mentions General Semantics on the third page. It talks about Murphy and Parkinson and Peter--pop psychologists all--as though they were Weiner and Taylor. There are no data and not even many anecdotes, just thought experiments.

Plus it's got dumb Watergate jokes. Hey, that's real funny to you guys in 1975 but I'm reading your book in the twenty-first century. You savvy my future-speak? To us, that happened decades ago, and we have higher standards for jokes about it. (Sample Watergate joke that's still funny. Non-broken image link for same.) Nothing wrong with trying to make a book funny, but the Poignant Guide to Ruby is pretty funny, and it's also got code you can run that works.

Okay, so it's a bunch of thought experiments. That's not too bad if you run the experiments and they jive with your experience, which is true here for the most part. The book has some more good quotes ("Systems work best when designed to run downhill") as well as good ideas I'd never thought of ("the confusion of Input and Output"). I'll probably pick up the revised edition of the book, The Systems Bible, which will surely have fewer Watergate jokes. But the pop psychology emphasis really decreases the authority of a quote from this book.

I could say "A complex system that works is invariably found to have evolved from a simple system that worked." myself, and it would have the same force as quoting John Gall in this book. I've probably got more hard evidence than the book does, because I live in the future. I quote him the way you quote Mark Twain: because you like the way he put it. So maybe I should just find a quote about systems design from Mark Twain.

As with anything, if you put it on the web I'll randomize it. Thus, my latest feature, Mystery Loaf. It names and creates a new dish every five minutes. Eating them is your problem.

Right now every REST web service is slightly different. You access them all with simple tools: your language's HTTP library, and probably its XML parser. Which is great because it keeps the services simple. What's not so great is that you have to learn each REST service separately.

What's more, it's useful to package and distribute your implementation of a particular service in a particular language. There's PyAmazon and Ruby/Amazon and a4j, PySearch and Yahoo-Ruby. That's always seemed wrong to me.

With a SOAP/WSDL service you just load up the WSDL file and call or generate the native language methods. Even though the underlying technology has more layers and is much more complex, when WSDL is one of those layers (and nothing goes wrong), everything just works. Sometimes there are language-specific wrappers on top of a SOAP/WSDL service (PyGoogle and Ruby/Google), but they don't do much: they make your code look nicer, or compensate for the shortcomings of the underlying API.

WADL is the best way I've found to get that ease of use for REST services, without creating documents that don't make sense to humans, or selling out the resource-oriented architecture of the web. WADL is very human-readable and it describes resources, dammit. I like it a lot and I think it'll make things easier for programmers if it gets widely adopted.

First a note about pages like that, which as a good IF writer but a middling static fiction writer, I find kind of frustrating. The page is oriented towards beginning writers. Like most web pages that allege to help you be a writer, it's heavy on "don't misspell words" and less heavy on "don't have a hackneyed plot" and "don't create puzzles that make no freaking sense" and things I don't know or can't articulate. Such pages turn bad writers into readable bad writers, but won't get anyone's work up to really good quality. As I've found out the hard way, good ideas and a good grasp of English don't automatically translate into readable stories. There are additional skills.

These pages chop the head off of a Sturgeon's Power Law that says the vast majority of bad writing is bad for obvious reasons (scroll down to the handy "context of rejection"). With static fiction I can routinely hit the midway point on this particular ring-the-bell carnival game, but I haven't found many good resources for getting higher. I have found one extremely useful page, but the higher-level craft seems to be something you have to learn one-on-one with someone who already knows, or something subjective you learn with practice. Or something that no one has written about because existing documents are enough to get the unintersting people out of your hair.

The problems in the first-timer list are divided into problems of fiction and problems of game design. I'm not going to discuss the problems of fiction because I don't find them very interesting. I can, however, tackle the problems of game design because much less work has been done exploring elementary problems in game design. Because this entry is already huge I'm going to cover one item at a time, over a period of several centuries.

Item 1: bad point systems. There are actually two possible problems here. The first is a poorly-scaled point system where you get 100 points for finding a key. The second is an overgenerous point system where you get 5 points for getting out of bed.

If a piece of IF has a scoring system, it imposes a limit on the score. I don't know of any counterexamples. In general, your score goes up when you do a one-time action that progresses you toward the conclusion of the plot. Your score at any given time is a measure of your progress (Zork III is a notable, and obnoxious, exception).

But if a video game has a scoring system, it imposes no limit on the score (except any imposed by the hardware). Again, I don't know of any counterexamples, though I can conceive of games where your score is represented as a percentage. The score of a video game has nothing to do with the game per se: your advancement towards the end of the game is measured by things like stage numbers. It's just a way of keeping... score. I think this problem is caused by treating a piece of IF like a video game.

It's fine to give the player 100 points for finding a key in a video game, but ridiculous to do the same thing in a piece of IF. This is because people don't handle big numbers well. If your maximum score in an IF game is a big number, it's difficult to tell how close you are to the maximum, and how much 100 points really contributes towards the maximum. A big inscrutable number is cool when you're claiming to be an awesome dude, but it's not as cool when everyone who completes the game gets approximately that same score.

By the same token, you shouldn't give out points for actions that don't advance the plot or don't involve any cleverness. If you do, the score will cease to have meaning as a way of measuring the state of the plot and your cleverness so far.

BeautifulSoup, c'est l'un des meilleurs parseurs HTML du monde mondial, c'est ce que j'utilise pour mon script de conversion blueflagz => RSS

"Monde mondial"? Does that mean "real-world"? If so, that's a great idiom.

I don't know when I'm going to be fixing Rubyful Soup to have the features of Beautiful Soup 3.0, but it's probably going to be after a bunch of WADL work.

When he entered, President Bush walked within two feet of me, and for some reason, he turned and looked me in the eye and said, "Let's eat!".

Ned Batchelder and his wife's White House adventure: parts 1 and 2.

In New York, right next to the place where the maple syrup comes from, they don't do any of this. I've been to fancy breakfast restaurants, as fancy as any I ever went to in California, and they all treat maple syrup like cheap jelly. It's made by Smuckers out of corn syrup and it comes in little plastic tubs with pull-off tops, like non-dairy creamer. They have to stock this stuff because all the restaurants here do delivery, and how are you going to deliver maple syrup in a little syrup ramekin. But they take it a step further and also give it to the people who came over to their restaurant in person! So when I go out to breakfast I never order anything with a dependency on maple syrup. I guess I could be a huge snob and bring my own little container of maple syrup, but then why don't I stay home and make my own waffles?

I've eaten at places in the south (IHOP being the exception), where the restaurant just buys a 20-pack of Log Cabin squeeze bottles and plunks one down on each table. New Yorkers probably think their system is better, but I don't see how. I never thought I'd have anything good to say about IHOP's syrup, but at least it doesn't look obviously fake.

Incidentally, Rocks'n'Diamonds is turning into ZZT.

Where will the trail of improvisation lead next? Ninjas? Robots? Some other nerdy role-play?

I'm not a huge fan of doing that kind of thing; I managed to stay about an hour and a half before it got to me and I had to leave. Did it help my writing? Maybe, but not $13 worth, which is what I spent on food. I'm going to try the cafe three blocks away tomorrow, but I think that mid-day work-trips to cafes will just be occasional treats to eat a lunch I didn't make.

Speaking of which, I went early this morning to the farmer's market in Union Square, which has really picked up since the dead of winter when I first went. Now it's full of cheap carrots and radishes and stone fruits, reasonably priced tomatoes, slightly expensive berries, and expensive garlic. Someone said "These tomatoes are severely deformed! Are you going to charge me extra for this?". I also saw a woman in chef garb pushing a metal cart around with some food on it--could this be one of the "celebrity chefs" who frequent the Union Square market, buying "celebrity food" for their "celebrity restaurants"? Who cares, really? Remind me to tell you about restaurant food sometime.

Usually when I go into Manhattan I try to wait until at least 10, because otherwise you're crammed into a subway car with a bunch of commuters and it feels like a demonstration of Avogadro's number. But today I went along with Sumana so I could buy stuff at the farmer's market and get home in time to make gazpacho and give it lots of time to soak. Sumana invited some people from Fog Creek over for dinner and the theme was to be "cold".

So I made gazpacho, and a tasty couscous/tofu/cucumber/tomato dish, and sliced deformed tomatoes with salt (not really cold, but you have to eat that at every opportunity while the farmer's market tomatoes are around). For dessert: vanilla ice cream with rasperries. Theme: embodied!

On an absolute timescale, dinner preparations kept me busy enough that I didn't write as much as I'd have liked. But I did discover some interesting facts which will go in the book.

• 1 box firm tofu
• 2 cups vegetable broth
• 2 tsp olive oil
• 2 cups couscous
• 1/2 red onion, chopped
• 1 cucumber, chopped
• 2 tomatoes, chopped
• 1 bunch parsley, chopped

Okay. Drain the tofu, slice it into rectangles, fry it in peanut oil, slice it into strips.

Boil the olive oil and broth. Pour it on the couscous and fluff with a fork. Dump in all the other ingredients and mix it up some more.

Tastes like a particulate Cobb salad or something.

Relatedly, my cousin Jill came to visit with her friend today. Said friend recounted a story of overhearing one of her sons saying to the other, "No, let it bite you, and you'll get superpowers like Spider-Man!" Friend put a stop to that right quick.

Can't think of a punchline, so I'll delegate to Penny Arcade. It's one of those ones where the punchline is in the first panel.

Among the topics I need to write about is my antipathy towards libraries that don't let you browse the stacks. But the really huge libraries with closed stacks also tend to be the ones that have things on display like a freaking Gutenberg Bible. Just right there in a case in the middle of an otherwise boring room. Do people know about this? I didn't.

To overcome my closed-stack anguish we went across the street to the mid-Manhattan library and checked out some books. This was great, especially the self-service machine you use to check the books out. But on the way out there was a different form of anguish, a lengthly fracas with the guy who makes sure you don't take books out of the library without checking them out. That's the totality of this poor guy's job, so when I triggered his "unchecked books being removed from library" alarm he went right to work on me.

"You need to check those out." "I did check them out." "Sir, those are library books. You need to check them out." "I did check them out. Do you want to see the receipts?" "Sir, you need to check those books out." "What do you do to check them out besides use the checkout machine?" "You can't take books without checking them out." Yes, I KNOW HOW A LIBRARY WORKS. (I didn't say that. I only say rude things like that afterwards, in weblog entries.)

Eventually he looked at the receipts and conceded that I had probably checked out my library books, deviously disguising myself as an honest citizen and cheating him out of a juicy apprehension. Paperwork beats alarm in this rock-paper-scissors. Sumana suggests that the self-service checkout machine demagnetizes or magnetizes some RFID-like object in the book, and I'd picked my books back up before the machine had had a chance to do that.

Then we decided to go to the transit museum for real, so we took the subway. Sumana has a greedy algorithm for taking the subway through a bottleneck, but one of its implicit assumptions is that the subway graph looks like San Francisco's, with few paths through a bottleneck. Now that we've moved, the algorithm needs generalization. The end result is that, due to stereotypically unintelligible service announcements, we ended up near the Brooklyn Bridge with no simple subway path to the museum. So we just walked the bridge. Which was fun, but once we got to the museum Sumana sat down on a bench and fell asleep, leaving me to explore the museum by myself.

This entry is long, so I'll publish it and write another one.

The cars were mostly decorated with period advertising placards, which were pretty interesting, but which destroyed the time-travel metaphor in a Vonnegutian way. Instead of having ads from a specific time, each car had a collection of ads from the entirety of its run, all unstuck in time and smushed together in one car. There were WWII-era ads next to Prohibition-era ads, including one that claimed "One day all women will vote... for [brand of soap]". I don't think that ever happened. Incidentally, the soap ad was one of those ads that features terrible and irrelevant racist cartoons.

Another soap ad had a Depression-era mother saying, "My children must purify hands before eating." Well, I hope your children enjoy starving, since nothing they do will ever be good enough for you.

I walked around for a while thinking these cynical thoughts. Not helped by an ad for subway ad space I'd seen upstairs in a historical exhibit, which claimed that subway riders "have learned to believe implicitly in the advertising displayed in the New York City Surface Cars because it merits and invites confidence."

This attitude was mocked by a contemporaneous newspaper cartoon where, having nowhere else to put ads, the conductor was looping signs over the necks of the passengers. It was from the golden age of political cartoons, when there were lots of strange details but not the loopy word balloons full of cursive dialogue that's poorly blocked and illegible, and not worth reading since it just explains in great detail "Oh! The Situation depicted in this cartoon, though considered by the reader a japish Metaphor, is something that truly does affect my corporeal Form!"

The truth is that most of the subway ads did, and do, not merit or inspire confidence. The golden age of the subway ads were the 1950s, a time when advertisers had forgotten that words had connotations as well as denotations, when you had ridiculous claims like "84 out of 100 women prefer men who wear hats" (an ad for hats, or for men?) and needed gorgeous MAD magazine-style paintings to make up for them.

Trivia: the original name of the Brooklyn Dodgers was the Brooklyn Trolley Dodgers, answering my decades-old question of what the Dodgers were dodging. Also, an old sign once posted near an elevator says "Meddlers Take Notice" in a preemptive anti-Scooby-Doo move.

"Actually there is a connection between Abner Doubleday and the Civil War," I said, ever the vigilant connection-spotter and nitpicker. I looked it up in Bully for Brontosaurus and, yes, Abner Doubleday did start the Civil War. He was a Union officer who fired the first shot from Fort Sumter.

Actually his Confederate counterpart is the one who really started the war, but it's more true to say that he started the Civil War than to say he invented baseball.

I wrote an article for the O'Reilly Network (which I think is technically not part of O'Reilly, somehow) about how we tested the code in the Cookbook. Which I should nudge the editors about. Lucas lives in Portland, so he doesn't have to pay for a plane ticket to go to OSCON and talk peoples' ears off about the book.

My ill-formed plan was to do a virtual book tour: guest-post on technical weblogs about the coolest recipes from the book. Other people have done this in the past, notably Greg Knauss, but I don't think it's been done with a technical book before. And I never actually emailed anyone about doing this, so it's probably too late now. I got an invitation to join the O'Reilly Ruby group weblog, so maybe I'll just post once a week on there about one of the cool recipes.

Does anyone else have ideas for book promotion? I want to focus on my current project but I should set aside some time for selling what I've already written.

"Lou, I think we ought to stick around a while longer." "You know, Acme Wallpaper is the only thing today's handyman needs to 'stick around' the house. It's inexpensive, durable, and when properly applied it forms a Penrose tiling! Ask your dealer to see a catalog today." Thus allowing the straight man to set up the exact same line again: "I say, Lou, that we ought to stick around a while longer and watch after Doris Day."

This is the model for this series of weblog entries, where I show you how to do cool things with Ruby but it turns out all to be a scheme to plug the Ruby Cookbook, bad boy of the O'Reilly cookbook family. In the first three entries of this series, I'll show you the system I rigged up to track the sales of my books. I don't really have a plan after that, but I can work from a list I made of the coolest recipes in the Cookbook.

The Book Sales Trilogy:

1. Getting book information with Ruby/Amazon
2. Generating graphs with Gruff.
3. Generating sparklines with the sparklines gem.

The problem

When you write books but don't publish them, it's difficult to know how many you've sold. The publisher gives you a total maybe quarterly. So a lot of authors (including me) use their Amazon sales rank as a proxy for books sold.

There are numerous problems with this, stemming from two facts: 1) Amazon sales rank only covers books sold through Amazon, and 2) it only conveys how well your book is selling _compared to all other books_. Booms and crashes in the book market as a whole won't make any difference to your sales rank. But you can't beat something with nothing, and sales rank is the only way I know of to get anything like real-time sales figures for a book. I talked with Mike Loukides, my O'Reilly editor, today, and he said that giving authors access to sales information is a low-priority project. And we all know about low-priority projects.

So, whenever a book of mine goes up on Amazon (first Learning Python and now the Ruby Cookbook), I register its ISBN with a script that runs once an hour and uses Amazon Web Services to collect the sales rank numbers. My goal is to encapsulate this logic into a class which maps an ISBN to a text file full of timestamped sales ranks. It will also work with Amazon-specific ASINs: for books, the ASIN is just the ISBN.

I use the Ruby/Amazon library, which is covered in Recipe 16.1, "Searching for Books on Amazon". The recipe focuses on keyword searches: finding books or other objects to match criteria. Here, we search by ASIN.

The script is based around a SalesReport class that encapsulates data about one product's sales rank over time. Here's the first draft of SalesReport.

#!/usr/bin/ruby
require 'amazon/search'
class SalesReport < Array
  attr_accessor :name, :asin
  def initialize(asin, name)
    @asin, @name = asin, name
    open(self.class.filename(@asin)).each do |line| 
      self << line.split.collect { |x| x.to_f }
    end
  end

  def self.filename(asin)
    asin + '.txt'
  end

When you create a SalesReport object for a product, it opens up a file named after that product's ASIN and reads in sales data.

Where does that file come from? The update! method is in charge of creating and maintaining it. This is also where we make the actual AWS request (in bold).

  # Updates sales rank for a number of products, and yields a
  # SalesReport object for each one.
  def self.update!(aws_key, *asins)
    now = Time.now.to_i
    req = Amazon::Search::Request.new(aws_key)
    req.asin_search(asins) do |product|
      asin, name = product.asin, product.product_name

      # Update the file with the new sales rank.
      open(self.filename(asin), 'a') do |f| 
        f << now << ' ' << product.sales_rank << "\n"
      end

      # Parse the file into an SalesReport object and yield it.
      yield self.new(asin, name)
    end
  end
end

Three things about this method's signature. First, like filename this is a class method: it's a factory for SalesReport objects rather than something you run on a particular SalesReport. For more on this, see recipe 8.18, "Creating Class and Singleton Methods". The self here is the SalesReport class itself.

Second, this method takes a variable number of ASIN arguments, for which see Recipe 8.11, "Accepting or Passing a Variable Number of Arguments".

Third, as I've mentioned in every book I've ever written, to use AWS you need to sign up for a unique string called a "developer token". This method uses that string to create an Amazon::Search::Request object, which will be our interface to Amazon Web Services.

You can reuse a Request object to do multiple searches, but we're only doing one. To save time and bandwidth, we pass all of our ASINs in a single search: "give me info on these products". The query returns a big XML document, which Ruby/Amazon parses into struct-like Ruby objects. It yields each object to a code block.

Inside the code block, I have access to a variable product which represents what Amazon knows about one particular book. I take its sales rank reading and stick it along with a timestamp into that book's file. Within the code block I also build a SalesReport object, which I yield to another code block somewhere outside this function. The code that calls SalesReport.update! gets yielded a series of SalesReport objects to do with what they will. In this case, my will is to generate cool-looking graphs.

If code blocks confuse you (they confused me when I was new to Ruby), the first few recipes of chapter 7 will help. If you're not familiar with file access, the basics (including appending to a file) are covered in Recipe 6.7, "Writing to A File".

Okay, that's all the SalesReport code for today. Here's how you'd call SalesReport.update! with a code block:

key = 'my AWS developer token whatever'
SalesReport.update!(key, "0596523696", "0764596543") do |product|
  puts "I updated #{product.name} (Rank: #{product[-1][1].to_i})"
end
# I updated Ruby Cookbook (Cookbooks (O'Reilly)) (Rank: 1412)
# I updated Beginning Python (Programmer to Programmer) (Rank: 219370)

So now I've got two files on Crummy's hard disk full of sales rank readings. What to do with that data? That code block doesn't really do anything., Well, I'd love to compare the sales ranks to hard sales numbers, but I've never gotten any for the Python book, and the one data point I got from Michael today doesn't mean much. So I've been comparing these numbers to their earlier selves by graphing them versus time. This leads into tomorrow's episode, Graphic Violence, featuring Chico Marx.

On the plus side, the code is much nicer now, and it turns out errata get fixed in the next printing of the book, not the next edition as I'd always thought for some reason.

Also, you know those allergen disclaimers on food that say "may contain traces of nuts"? Today I believe I succeeded in isolating the nuts! First, a recipe:

Natur-Swiit Oatmeal

• 1 cup soy milk
• 1/2 cup non-soy milk (I've been bumping up the soy milk percentage but 100% soy milk still tastes gross in this)
• 1 handful dried fruit
• pinch salt
• 3/4 cup oats

Instructions: Boil the fruit in the milks and the salt. Add oats and cook for 5 minutes on low heat.

Okay, it's just the instructions from the back of the oatmeal canister, except for the fruit. But the fruit is magic. When you boil the fruit in the milk, it swells up and sweetens the oatmeal, so you don't have to add brown sugar to the oatmeal. Adding fresh fruit afterwards never makes it sweet enough for me.

I eat this for breakfast about half the time. Usually I make it with cut up dried apricots, but today I went with all different types of fruit. I dumped them in the saucepan, turned on the heat, and then went away to work on the promotional tutorial and forgot about the boiling milk. Then I remembered and ran back into the kitchen, where the milk proteins had formed huge bubbles in the pan. And suspended in the bubbles were... tiny pieces of walnut skin!

I'm pretty sure that's what they were; they could have been pieces of fruit skin but they sure looked and felt like walnut. I'm pretty sure I've seen the fabled "traces of nuts" and my life is now complete.

It also features graphs! Recipe 12.4, "Graphing Data", introduces Geoffery Grosenbach's Gruff library, by amazing chance also the subject of today's promotional tutorial. Gruff makes it easy to turn data structures into graphs and write them to PNG files. So the old clock on the wall says it's time for part two of the Book Sales Trilogy:

1. Getting book information with Ruby/Amazon.
2. Generating graphs with Gruff.
3. Generating sparklines with the sparklines gem.

The hardest part of Gruff is installing the dang ImageMagick or RMagick libraries and their dependencies in the first place. It's easy on Debian and other systems with a good packaging system, but otherwise it can be a real pain. The second hardest part is working around Gruff when its simplifying assumptions don't apply to you. I glossed over these in the book but I'll tackle the second one a little bit in this tutorial.

Anyway, yesterday I showed you code to take periodic readings of books' Amazon sales rank. And then I showed it to you again today because the code I wrote yesterday was crap. So read that entry even if you read it earlier. The new code makes the rest of the trilogy much easier to present.

You'll recall (from like ten seconds ago when you read it) that we have a SalesReport class that encapsulates sales rank information from a book. Yesterday, though, I didn't show you anything interesting to do with this information. But the night is ours! Tonight, we graph!

Let's open up the SalesReport class again and make a sales report capable of expressing itself as a line graph:

require 'rubygems'
require 'gruff'

class SalesReport
  # Make a Gruff graph for the sales of this product.
  def make_graph(graph_path)
    g = SalesRankGraph.new(800)
    g.title = "Salesrank over time: #{name}"	
    g.theme_37signals
    g.colors = ["black"]
    g.title_font_size = 20
    g.hide_legend = true

This is mostly self-explanatory setup code. In the book I claim that most of the Gruff themes are ugly, but that theme_37signals is okay. Well, that's just, like, my opinion, man, but incontrovertible fact is -- and I should have mentioned this in the book -- that theme_37signals's idea of a good time is to graph the first dataset with a yellow line on what's basically a white background.

That's a really bad idea. I believe there's a UI maxim to the effect of: "Some other color than yellow on white, graph-reader's delight. White under yellow, dangerously confuse a fellow." So I go with theme_37signals but tell Gruff to draw the data line in black: the original high-contrast color for white backgrounds.

The other thing I do is hide the legend, because this graph is only for one product. I would like to graph sales for all of my books on a single graph, but I haven't figured out a good way to do it yet. One of Gruff's simplifying assumptions is that all your data points are spaced evenly along the X-axis starting at X=0. I'd have to insert a bunch of bogus data points for books that came out later; worse, most of my timestamps don't line up precisely, so I'd have to write code to group multiple times into a single data point. So right now I just do one book per graph.

Now we've got one line of code that's very important, because it's where I decide how the data will be represented.

    g.data(@name, collect { |date, rank| 1/rank })

The data method takes an array, and adds it to the graph as a data set. SalesReport is an array of dates and ranks, so I could just pass in the ranks, but that would yield a graph like this:

This is a lousy graph. Unimportant details (long stretches early on where no one bought the book) are the most obvious features, and you can't even see the release of the book. But the data isn't useless; it's just not presented well. We're accustomed to seeing charts go up when the numbers go up (see: any TV commercial featuring a chart), but a good sales rank is very small. Also, as all Web 2.0 types know, book sales follow a power law distribution. A book at 400K sells one copy and jumps to 200K, but you have to sell a mess of books to go from #100 to #90. Displaying the sales rank as though it were linear distorts the data.

I don't know the exact distribution for book sales, but simply taking the inverse of the sales rank gives the graph the right shape. In this graph, the release of the book is obvious, and the time leading up to it makes sense:

What about those labels on the X-axis? Where do they come from? They come from this code:

    label_hash = {}      
    [0, (self.size/2).round, self.size-1].each do |i|
      label_hash[i] = Time.at(self[i][0]).strftime('%m/%d/%Y') if self[i]
    end
    g.labels = label_hash

The graph's labels are a hash that maps positions on the X-axis to strings. Remember, the positions on the X-axis are the indices to the array(s) you passed into the data method. You don't get to choose these values. The X axis starts at zero, and ends at the maximum index of the largest array you passed into data. I choose three labels: one at the beginning, one at the end, and one halfway between. Here's a graph with a lot more history than the Ruby Cookbook one:

Finally, having created the graph, we write it to disk:

    g.write(File.join(graph_path, "#{asin}-salesrank.png"))    
  end
end

Well, not quite finally. I sneakily referenced a class called SalesRankGraph a while back, and never defined it. That class derives from Gruff::Line, but if you do this graph with a Gruff::Line it'll have weird numbers on the Y-axis:

Those labels are just what you'd think: they're the numbers being graphed. This mighty graph stretches from about zero to about 0.001. Of course, the graph is "really" measuring the inverses of those numbers, but there's no way to put that in the labels. It's another of Gruff's simplifying assumptions. You can choose your X-axis labels but not your X-axis points; you can choose your Y-axis points but not your Y-axis labels. I couldn't find an easy way to fix this, so I just hacked the draw_line_markers to not draw them. You can shut off both the X- and Y-axis labels by setting hide_line_markers, but I like the X-axis labels.

class SalesRankGraph < Gruff::Line
  def draw_line_markers
  end
end

So now I've got some pretty nice-looking graphs to track my sales rank. But I don't have time to look at graphs! I should have spent today working on my new project, but instead I wasted the morning fixing problems with the preivous entry in this series, and then spent the afternoon making pizza sauce and writing this entry! What to do? If only there were some post-literate infographic that would convey sales rank information at a glance! Something like the graphics I laboriously put on the crummy.com homepage this afternoon! Stay tuned for tomorrow's episode, The Spark of Line!

Sumana: I dreamed I was having dinner with the president, and I was cracking all these jokes, but nobody was laughing.
Leonard: You dreamed you were Stephen Colbert?

Jason pointed out that usually when you make a decision it's like the poem from high school, where you never find out what would have happened to your life if you'd chosen differently. But occasionally you do get a glimpse. He said that soon I would get a glimpse of what I'd be doing if I'd gone to work at Google with Greg and him.

And so I have. It seems I would be working on Google Code, a site for hosting open source projects a lot like Tigris and the other CollabNet software development sites.

Hum. Okay, Google is probably a more fun place to work than CollabNet, and Google Code has some great features (I like the radically simple approach to bug tracking). But I don't think Counterfactual Leonard would be happy quitting one job in frustration only to go do another one that's basically the same. I think I'm happier on the road less travelled.

Sumana: There should be a Lord of the Rings II.
Leonard: What would be in this Lord of the Rings II?
Sumana: The ring is back! But now we have machine guns.

On previous installations I showed you how to get sales rank information about a book from Amazon, and how to create a graph of the data over time. Now my vengeance will be complete: I will unleash sparklines upon the world!

Sparklines are really interesting: bits of anonymous data that add quantitative analysis to text, in some cases without breaking the flow of a sentence. The meaning of a sparkline depends on context or a tiny textual label, not on big sets of axial markings. I've been a fan of sparklines for a while, and what better way to propagandize them than by inclusion into a book of cool tricks? THERE IS NONE. So here we go with the final part of the trilogy:

1. Getting book information with Ruby/Amazon.
2. Generating graphs with Gruff.
3. Generating sparklines with the sparklines gem.

As you know, Bob, in previous episodes we defined a SalesReport class which encapsulates information about a product's sales rank over time. Then we extended the class to give it the ability to write out a big graph describing the history of its sales rank. Now we're going to extend it again, and give it the ability to write out a sparkline.

Like Gruff, the sparklines gem is a great piece of code by Geoffrey Grosenbach. It's simpler than Gruff because sparklines are simpler than graphs. It does have some issues you need to watch out for, though. I cover sparklines in Recipe 12.5, "Adding Graphical Context with Sparklines". In the book I give some silly examples focused on embedding sparklines into HTML pages with the data: URI scheme, and incorporating sparklines into Rails views with the sparkline_generator gem. Here, I'll show you to show how to write sparkline graphics to static PNG files.

Those images from the Crummy homepage (Cookbook and Beginning Python ) show sales rank values for the past 30 days. How do I make them? Let's see.

class SalesReport
  # Make a sparkline for the sales of this product.
  def make_sparkline(graph_path, time_units=30, samples_per_unit=24)
    path = File.join(graph_path, "#{@asin}-salesrank-sparkline.png")

    # Gather a sample of the data
    sample = []
    (size-(time_units*samples_per_unit)-1).step(size-1, samples_per_unit) do |i| 
	sample << 1/self[i][1]
    end

A sparkline needs to be small, and unlike Gruff, sparklines doesn't compress data points. Gruff takes an image size in pixels, and whether you give it 8 data points or 800, your graph is that size. But if you give sparklines 800 data points, you get a really long sparkline. I can't use all the data I've gathered since these books showed up on Amazon: I need to economize.

This is fine. I don't need the whole history for a sparkline like I do with a big graph. I'll settle for the sales rank history from the past 30 days. But I run my data collector every hour, and that's still 24*30=720 data points. Solution: I go back 720 hours, and skip ahead 24 samples at at time. This way I pick a sample sales rank for every day, ending with the most recent sample. You can change time_units and samples_per_unit to customize your sparkline.

Now I have an array and I just have to send it to be turned into a sparkline:

    Sparklines.plot_to_file(path, scale(sample), :type => 'smooth', 
                            :line_color => 'black')
  end
end

Well, not quite. Earlier I mentioned that sparklines line graphs don't compress data horizontally. There's a fixed number of pixels between each point. Well, sparklines doesn't compress data vertically either. A line graph can handle values between 0 and 100. You can make the sparkline bigger but I'm pretty sure you can't increase that range. If you give values outside that range they get clipped or ignored.

In this case, our numbers are already between 0 and 100, since we're taking the reciprocals of numbers greater than one. But they're also between 0 and 1, as we saw in the previous installment. When the range is 0-100, this makes for boring sparklines. If your book rocketed to the top of the Amazon charts and stayed there, its amazing success would look like this: (caution: embedded sparkline not visible in IE).

To make our data visible, we need to stretch it out. We might as well go for maximum stretch: treat the smallest sampled rank as zero, and the largest as 100. That's what the scale method mentioned above does. Here's the definition:

  # Scale data so that the smallest item becomes 0 and the largest becomes 
  # 100.
  def scale(data, bottom=0, top=100)
    min, max = data.min, data.max
    scale_ratio = (top-bottom)/(max-min)
    data.collect { |x| bottom + (x-min) * scale_ratio}
  end

I gave a similar method in the sparklines recipe in the book, but it's hard-coded to scale a range to 0-100. This implementation is more general and you can use it anywhere. I really should have made this a separate recipe because it kept coming up. In 12.14 "Generating MIDI Music" (which I hope to write about later) I reused this formula for a different range and got it wrong, and had to send in an erratum. In 2.12 "Using Complex Numbers" I used a similar formula to scale an ASCII drawing of the Mandelbrot set to any desired size. So it turns out this is a useful method to have.

Anyway, now we can scale any data set to take maximum advantage of the vertical space allotted us by a sparkline. In the book I lightly discuss the ramifications of scaling all these different data sets to the same 0-100 range. The Cookbook sells much better than Beginning Python, but you can't tell that from the sparklines. Which is fine here, because I want these sparklines to show trends.

The sparklines gem lets you do other kinds of sparklines: pie charts are my favorite, as you'll see in the book. I'm really excited about the possibility of sparklines: they're like the little crawls at the bottom of television news stations, except they're classy and related to the main text.

The full sales rank monitor program is available here. I hope other authors find it useful.

What's next for Best of Ye Book of Ruby-Receipts? I don't know. These promotional tutorials take surprisingly long to write: it's almost as much work as was writing the original recipes. So I may give it a rest for a while. But let me know if there's a recipe in the Cookbook you'd like to see me explore in more detail, in the context of a real-world program like the sales rank monitor.

Anyway, I'm just going to review one game, an IF game, and I'm cheating because it's a game I played and liked when I was younger. It's called Crusade Adventure and it came with AGT as a sample game.

I'm a little surprised that no one has written a program to convert AGT code to Inform. AGT and Inform are abstruse in opposite directions, though, so maybe I'm the only one nostalgic enough about AGT to be interested. And as I look at these old games I become less interested. AGT runs just fine in a DOS emulator, after all.

I remember Crusade Adventure as being full of action, with a caving section more sophisticated than Colossal Cave. Well, it's not full of action, it's full of the action-evasion. And its caving section is only sophisticated in that it's a small adjunct to a larger map. The game relies heavily on random events: you can avoid some puzzles (including a fairly clever one) by relying on chance. The map is disjoint, and you have to use a magic word to move around.

Unlike most AGT games, Crusade Adventure is pretty nonlinear and has good atmosphere that I associate with Sir Walter Scott. Unfortunately, it turns out I only associate the atmosphere with Sir Walter Scott because the game makes explicit reference to Ivanhoe, in what might be the most jarring anachronism I've ever seen in an IF game. As Mark Twain would say, "the genuine and wholesome civilization of the nineteenth century is curiously confused and commingled with the Walter Scott Middle-Age sham civilization". I've never read any Scott but his "dreams and phantoms" fit well into an AGT setting: misty woods, suits of armor, talking skulls.

I remember playing a very slightly racier version of Crusade Adventure. But the one on the IF archive is fine because it makes this game stands in stark contrast to the all-out sleaziness of E. L. Cheney's GAGS games, also often distributed with AGT. There was one really disturbing one set in an ultra-sleazy San Francisco that featured lines like: "Lucy's firm young breasts are the center of your focus, You'd love to play around with them, but do you have the time?" Oh, gee, look at the time! A web search reveals that E. L. Cheney now draws cartoons about golf.

As a palate cleanser for the sleaze let me offer more of Twain's Fennimore Cooper-class vitriol against Scott:

It was Sir Walter that made every gentleman in the South a Major or a Colonel, or a General or a Judge, before the war; and it was he, also, that made these gentlemen value these bogus decorations.

Bam!

I had an idea based on Google Sightseeing: a journal of negative results. You could post to this service saying that you'd scanned a certain region and found nothing especially interesting.

Actually, is there a journal of positive results that's machine-accessible? A fully general way to describe parts of the map. No reason the negative results couldn't go in there too.

The puppeteers are visible, they walk around the stage and they emote along with their Muppets, but in the fiction of the show they're supposed to not exist? Or they're supposed to be the insensate bodies of the Muppet characters? I dunno how it's supposed to work. How do Muppet operators see their Muppets? I always thought it was like the relationship between ventriloquist and dummy.

If I may self-analyze, I think my biggest fiction failing is a pathological fear of writing infodumps. I relentlessly avoid explaining anything except in sidelong glances. The result is a pointillistic story that you can't understand without reading it multiple times. To make you want to read the story multiple times I cram it full of ideas. Which must be explained. In sidelong glances. My stories look like flip books. You have to work to see the plot.

Does anyone else have this problem? (Especially published writers whose work I can look at?) I'll let you know what strategies I can figure out against it.

Proto-Kermit: Have some Willkins coffee?
Proto-Waldorf: No thanks, gimme Brand X!
[Proto-Kermit shoves an X-shaped branding iron into Proto-Waldorf's chest.]

I've never seen such violent Muppets.

All day I gnashed my teeth trying to figure out how to get that damn scene into its chronological place in the story. So many dependencies, so many facts it establishes that I need later. I finally just dropped the scene into the right place and started hacking at the dangling references. The guts are still showing, but amazingly the story now has a plot. One thing happens, and it makes another thing happen, and things cause other things until the story ends, and that's a plot. There are no random notes from the future coming in and obscuring the connection between things x and x+1.

Maybe it's not that strict a rule, but I'm a well-known anti-fan of flashbacks, so for me it's chronological order or totally disassociated sequence of images. It can't be half and half, it can't be now and then.

It's a spacewar type game with a separate spaceship for every key on the keyboard. When you press a key, that character shows up on the screen as a spaceship and joins the game. You control the spaceship with the corresponding key. Tapping the key rotates your ship clockwise, and holding it down accelerates.

The health of a spaceship is represented as a color from green to red. Accelerating drains your health, and so does colliding with another ship. You kill other spaceships by ramming them: this does some damage to you but more to the other ship. If your health goes to super-red your ship explodes.

In theory you can have as many players as there are keys on the keyboard, assuming they've all got long tentacles for fingers. In practice you will only have a few players, but each player can try to control multiple ships. Will you use ships as disposable missiles or try to set up pincer movements? Or just try to hit the other player's keys and mess up their navigation?

The builder came to court and shrugged its collective shoulders comically and said that it had never heard about these other builders or which one of them was responsible for the inspection, but the contract gave it no reason to believe that it was that unfortunate builder.

Ever since then I wondered what happened to those commas. Now thanks to Teresa Nielsen Hayden I see that one of them has shown up in Canada. Where's the other one?

I got this map laminated, but I always thought my craftsmanship on it was rather shoddy, so I never hung it up. Recently I foisted it on Jake Berendes, who pretended to appreciate it.

But once I got it out of my head, a similar idea took its place. Google Maps has pregenerated static image tiles for maps at different scales, and it wouldn't be hard to write a program that tiled them together like Google Maps does, except not helpfully.

So I present Dada Maps, which uses tiles from all over the world to create a new evocative map every 5 minutes. It's what I wanted my earlier project to look like, and it's not subject to scarcity. Enjoy!

I've discussed before how part of being a politician is going out and smiling the morning after a crushing defeat and pretending everything will turn out fine. I've seen my share of this up close, but Joe Lieberman was incredible. All through the 2004 campaign he acted like the nomination was his in all but name, right up until he dropped out. Defeats were recast into milestones that he'd had to meet, and that he'd just met. He had the Joementum.

Except no one could figure out what "Joementum" was. Clearly it was not being used according to its dictionary meaning ("an electroplated filligree added in modern times to a medieval baton or weapon"). Nor could it legitimately be considered any kind of pun on "momentum".

I think I've figured it out. Yesterday Joe Lieberman lost his Senate primary (note to non-US readers, who don't care anyway: this is quite a feat). He's still smilin'. He's going to run as an independent, against the Democratic nominee and the (fairly pathetic) Republican.

That's when it struck me. Joementum is not specific to Joe Lieberman, and it's not an attribute of the campaign, as I'd always thought. It's a personal quality of the candidate. It's just the name for that thing inside you that makes you act like the bad news isn't happening, that everything will work out fine and you'll win. It's the momentum that keeps you physically working even when people aren't voting for you. Joe Lieberman just has more of it than most people in politics.

It's not denial; it's the ability to act as though you were in denial. As I discussed in my earlier entry, this is a neccessary quality in a politician, because admitting that any bad news is real will instantly doom you. To gain any nontrivial victory, the candidate must posess enough Joementum to be stubborn and firm-jawed about bad news and impossible odds. But usually when the bad news is that you lost and conceded the election, you can relax.

I also discovered a creepy sub-floor called the Henry R. Luce Center, which is a sort of browsable storage space for the museum. It's full of furniture and paintings and bottles just lined up on shelves behind glass, with minimal signage.

One was for Bud Lite. I guess if you're spending a million dollars a day or whatever on advertising, hiring an airplane to fly over a beach is just a drop in the ocean. But it's not neccessary, because the Budweiser brand is one of those select few that's associated with the concept of obnoxious advertising in general. Ask someone: "Say you're at the beach and someone flies a plane in front of you. What is the plane advertising?" "Uh, Bud Lite?" You don't need to hire an advertising plane, because any advertising plane makes people think of your product.

One was for Snakes on a Plane. It had a picture of a snake, and it said "Snakes + Plane = In theaters August 18th". As much as I would like to believe that there were promotional snakes on the plane that pulled that banner, I know that that's not true. The banner merely called attention to one of the many planes above this planet that contain no snakes at all. Was this a good marketing move? I don't know, but this is a ridiculous B-movie-style movie being marketed as an A-movie.

The other airborne banner was the most ridiculous yet. It was tied to its plane by two tethers, which somehow had folded the banner up so that the advertising was not visible to anyone on the beach. It was as obnoxious as the other two ads, but there was no content. Made me think of Bud Lite for some reason.

So I'm sorry I gave you a lousy inscription, and I hope you don't feel insulted. If you're reading this, I'm happy to send you a replacement sticker-inscription with the cliche of your choice. Or you can just keep it around to embarass me with my own callousness.

To everyone else: enjoy your cliche-inscribed Cookbooks!

Update: Maybe the Dinosaur Comics plug.

We can increase the price of your stock and we can increase average day trading volume. We can increase price up to 200-250% in 2-3 weeks and also we can increase volume by 10 times each trading day.

You pay them according to how much they jack up the stock price in one day.

WE GUARANTY LONG TERM GROWTH

WE PROMOTE THE COMPANY AND IT'S BUSINESS

Who could doubt it?

The second product I don't understand, unless it's a degenerate case of the first product. Here's the relevant part of the email:

You want to invest some money and you want to make big profits? Our company can help you in that. We know exactly which stocks will increase in price and wich will be falling down. We will give you advices and according to that advices you will make decisions. We will prove it for 1 day for free. We will tell you the price of the stock in advance and you will see that our stock promotion group works hard in order to make your investment portfolio grow.

Payment: Our price for that is 25-30% from your income. Before the deal you are telling us on which ammount of money you are buying/selling stock and we are counting the future profit and take 30% from that profit. You don't have to pay anything in advance, you pay only then you receive your profit.

So they know how stocks will go, but they want you to buy the stock instead of doing it themselves? Clearly there's spamming going on here ("our stock promotion group works hard"), but maybe they also do something with the knowledge about how much you're going to buy or sell? I don't see it.

This demonstrates that stock spamming has become a service that can itself be packaged and sold via spam. It's a little different from generic spamming services (or spam for spamming tools that you yourself sell through spam), because there's an implication that pumping stocks is hard work best left to the spamming professionals.

We ate with Rachel Chalmers at Chez Panisse. We sat right next to the open kitchen and watched people make food. The atmosphere was great and conducive to conversation. We talked about science fiction and Rachel said that I should spend my time writing space opera for her delectation, which is good work if you can get it.

We all had the vegetarian prix fixe. Sumana and Rachel shared a half-bottle of fancy wine and I had two glasses of a really good raspberry Italian soda. We started with bread and butter. Then, the menu!

Appetizer was a pair of crostini, one with tomatoes and one with some green vegetable I don't know what--possibly squash--and cheese. They both were topped with an excellent vinaigrette. The tomato crostini was better, because tomatoes are more awesome than random vegetable.

Then a garlic broth soup with a poached egg floating in it. You cut open the egg with your spoon and the yolk flooded out to be cooked by the soup.

Then the entree, which was a soufflé. It was okay but what I think the non-vegetarian option (something porkish) would have been better. What was really great were the corn, green beans, and other vegetables served around the soufflé. Also there were zucchini blossoms at the top of the soufflé, which was awesome.

Dessert was a great fig tart, and there were some tiny after-dessert cookies, also great.

Chez Panisse was quite an experience, but I was pretty disappointed in the soufflé, a nontrivial part of the meal. I don't really like souflés except for the dessert ones with chocolate in them.

The next day I went to Greens with Adam, Kim, Kris, and Erica. Sumana got sick from the plane trip so she didn't come. I, on the other hand, waited until the vacation was over to get sick from a plane trip. Cough, sneeze.

At Greens I had lemonade and zucchini griddle cakes and pasta with tomatoes and chocolate cake. The lemonade was pretty lame (I officially give up on restaurant lemonade, except for the lemonade at the cafe nearest my house). Everything else was excellent.

I've been to Greens many a time and Chez Panisse only once, but I think Greens has better food, as well as being easier to get into and a little less expensive.

I have a more general entry on this topic percolating, but I think there's a point of diminishing returns at which making restaurant food more expensive doesn't make it proportinately better. I hypothesize that this point comes when you stop getting your food from Sysco or a Sysco wannabe and start getting it from diverse local sources. Greens and Chez Panisse are both at this point.

I do know the source of these remarks: they stem from the eternal conflict between two philosophes in my mind. These are the hippy-dippy Carl Sagan atheist who says "You don't need God to have a good time, man; look at all the wonder in the natural universe!", and the cranky Richard Dawkins atheist who says "The wonder is all in your brain, you pothead! Not distributed throughout the universe!"

The Carl Sagan philosophe was formed in a transcendental experience I had when I was six years old. You know those charts that go on the walls in elementary school classrooms? Instead of putting a few of them on the walls, my kindergarten teacher, Jim Murchison, bought an enormous number of them. He punched holes in them and turned them into a huge flip chart. Every day or two he would flip over a sheet and reveal the new topic of discussion. One day he flipped over a sheet and I saw the solar system.

I'd surely been to the Griffith Observatory before then, but somehow I hadn't seen any pictures of the planets before this reveal. Jupiter, in particular, blew me away. Seeing drawings of the planets triggered my first and biggest transcendental experience. Since I always ruin Sumana's transcendental experiences it's only fair that I should ruin my own. Jupiter isn't transcendental; it just looks really creepy. It's a planet, a gasball.

What's more, the universe does not contain a magical kind of thing called a "planet". Planetness is a social construct. The solar system is not the sun and nine planets, as depicted on the flip chart of my youth. It's a fusing gasball, four non-fusing gasballs, a few million rockballs, a few billion snowballs, and a big dust cloud.

Recently, the International Astronomical Union decided to make a proper scientific definition out of the social construct "planet". This used to seem easy and even superfluous, but in recent years it's become clear that to judge every object in the solar system by a uniform standard of planetness, you must make a hard decision. Either Pluto is not a planet, or there are lots of Kuiper Belt objects which are planets on the same criteria as Pluto. These planets are very distant, mostly undiscovered, and not very interesting once discovered. There might be hundreds of these tiny boring planets. So the IAU created a definition that excludes these objects, including Pluto.

It's not nice to fool Mother Nature, but it's even less nice to naturalize Mother Social Construct. People are used to the planets being the non-fusing gasballs, four of the biggest rockballs, and Pluto. A lot of people got mad at what they viewed as a bunch of eggheads usurping their social construct. And now the eggheads are arguing amongst themselves about the definition. It's a huge mess.

The real problem with this definition is that, to get the "right" result, the IAU restricted the definition to apply only to our solar system. It's the Bush v. Gore of scientific definitions! They had to do this because extrasolar systems perform instant reductio on any attempt to turn "planet" into an objective concept. There are too many weird things in the universe. It would have been simpler if they'd just enumerated by fiat the eight things in our solar system that are "real" planets.

Astronomers have found things like two non-fusing gasballs orbiting each other in interstellar space. Are they planets? Only if we decide to call them "planets". What they are is big non-fusing gasballs that orbit each other. A gasball can have natural starness, but it cannot have natural planetness, any more than (as Michael Shermer says) it can have natural meaningness. This is what social constructs are for.

I personally don't care if Pluto is called a planet or not. My money is still on "alien disco ball". But if we must have a definition of planetness, it should recognize the inevitable subjectivity. If you don't do this, you need an artificial ban on discussing what might make an extrasolar object a "planet". Because sooner or later the smart-aleck universe will toss you a planet that doesn't meets the technical criteria, or (less likely) a non-planet that does. Then you have to tweak the definition and it's like adding epicycles to the Ptolemaic model of the solar system.

Antigamesgeburtstagsdinosauriergewinnspiel (I can almost pronounce that now) is a Something Awful-style Photoshop contest presenting video game screenshots that have been modified to include a dinosaur. This is in accordance with Kevan's discovery that any game can be made more fun by the addition of dinosaurs. The winners are here and here. Highlights include Tetris, Grand Theft Auto, and Nintendogs.

I discovered Antigamesgeburtstagsdinosauriergewinnspiel via referer logs, when a post-contest entry presented Nethack with dinosaurs, and a commenter pointed out that I already put dinosaurs in Nethack.

Leonard: I used to wonder if Germans ever made up really really long words just as a joke.

Richard: Yes, we do. You can stop wondering. Really. Stop.

Leonard: "Antigamesgeburtstagsdinosauriergewinnspiel" sure looks funny but was it intended to be?

Richard: It does and it was.

Leonard: Presumably that's just how you say "birthday dinosaur contest".

Richard: Yes, that' s really how you say it.

Leonard: And maybe it's acceptable to incorporate the name of your website in a word.

Richard: Nope, usually not acceptable. Any OTHER questions?

Leonard: Die deutsche Richtung der Stimmung ist ein Geheimnis zu mir.

Richard: Lake, it' s funny because… uh… wave, because. Oh, what the brightly, it' s A mystery tons of ME, too. Happy now?

Actually I should have realized that the German sense of humor is not that much of a mystery; the Antigamesgeburtstagsdinosauriergewinnspiel proves that Germans speak the universal language of Photoshop contests. Only unresolved question: is there a German word for the habit of making up humorous long German words?

PS: Also, I just said "It is now empathy time."

• Planetographic coordinates. Defining the prime meridian of a gasball is trickly.
• Extreeeme extrasolar planets (PSR_B1620-26c is 12.7 billion years old!)
• Asteroid moons, asteroid families, and Kirkwood gaps

These were the first comic books I've ever bought, so this is a big step for me. I also got an issue of Hero Squared, written by Ambush Bug creator Keith Giffen. It's kind of funny in a wacky-sitcom way but lacks the Marx Brothers anarchy of Ambush Bug which is what made me actually go out and buy a comic book. I also got an issue of the aforementioned Buckaroo Banzai comic book, which is not really very funny at all but is great if you like Earl Mac Rauch's Buckaroo Banzai worldbuilding. Though the passage of time has made Buckaroo Banzai seem less eccentric, and more isolated from society and even vaguely sinister.

Totally gross! You know what we're talking about. You've skinned your elbow while crashing your bicycle getting extreme over a gnarly jump in your neighborhood.

I never meant to get extreme! It was all a big misunderstanding!

Gumbasia reminded me of old films like the ones I saw at the Exploratorium, so I went digging. I found Norman McLaren's Synchromy, which is still excellent though on my computer it was slightly... out of sync. Oskar Fischinger's Composition in Blue I could not find, but there is another film of his, seemingly a commercial that anthropomorphizes cigarettes. It's fun except it was made in 1934 and you're all the time worried it's going to turn into a cigarette Nuremberg rally. Which is totally unfair because Oskar Fischinger's art was degenerate and unwelcome, and he left Germany for Hollywood in 1936, so why am I thinking that?

There's also Norman McLaren's Neighbors, which some crackpot Wired article claimed was one of the inspirations for Star Wars. Why, because there's a swordfight? (Note: I don't feel like finding the reference because the Wired article might actually have a point, and then where would I be?)

I think this technique is pretty interesting and I'm really happy with the quality improvements and extra confidence I got out of it. I even considered going to RubyConf and giving a talk about it. But going to RubyConf to give a talk costs me money, money I don't have, whereas writing an article on the same topic would earn me a pitifully small amount of money. I decided to write an article. This is the sort of business acumen that has made me the financial giant I am today.

Now the article's been published: "Unit testing your documentation". Original title was the less-prescriptive "Unit testing a book", which got changed to the even-less-prescriptive "Unit testing the Ruby Cookbook" but now it seems I'm prescripting away! Whee!

[0] Tiresome but not neccessarily unrealistic. I could definitely imagine aliens in real life constantly flaunting their pop culture knowledge, but it would also be tiresome in real life.

A lesser known fact is that pre-Socratic philosophy is freaking hilarious. Both phases yield funny results (Everything is made of water! No, earth and water! No, air!). They sound ridiculous now but "everything is made of air" was actually a pretty sophisticated model; it supposed that you got other materials by compressing air to greater or lesser densities. Occasionally the book gives out kudos: "[Anaximenes's] view that the stars give no warmth because of their great distance contradicts Anaximander's theory that the stars are closer to earth than the moon and sun are. It is also true." I believe this author was the first to discover that Anaximenes and Anaximander were actually different people.

I just got to the chapter on Pythagoras, who is portrayed as an L. Ron Hubbard/Paul Bunyan/Chuck Norris figure of folklore and Internet meme. Here are some useful Pythagoras Facts (p79):

His followers were devoted to his sayings, which they collected, memorized, and passed down. He was a charismatic figure who became the subject of legends: he killed a poisionous snake by biting it; a river hailed him by name; he made predictions; he appeared simultaneously in two different places; he had a golden thigh. The people of Croton addressed him as Hyperborean Apollo. Pythagoreans identified three types of rational beings: gods, humans, and beings like Pythagoras.

My first reaction is to mock Heraclitus for saying that if you just figure out how the universe works you'll see how the universe works; but one thing this book has taught me is that these ideas had to be invented, and it looks like Heraclitus might have invented the idea that there is such a system. Plus he has this to say about phase two of philosophy: "Let us not make random conjectures about the greatest matters."

To refresh your memory, the basic plot is that a real estate company is trying to make money by building houses. In the pilot, the company has only a model home: an incomplete shell of a house designed to sell other houses. Over the course of the series they build more houses, but their fortunes never improve, because the company is run by incompetents who care only about polishing their egos and stabbing each other in the back. Now, I'm not a big reader of Variety (or, as I believe it's now called, More of the Same), so I'm not up on all the industry gossip, but that dysfunctional environment seems a lot the one in which you put together a television show. Especially a relatively highbrow show that lacks support from the network.

Maybe the reason no one has pointed this out before is that over time the metaphor was presented in more and more blatant ways, making it seem obvious. The most famous intrusion is at the beginning of the third season (which was cut from 22 episodes to 13), where it's revealed that the company had a contract to build 22 houses but it's been cut to 13. As the show's fortunes got worse and worse, more and more of its fans' anguish and desperate, doomed attempts to save the show bled into the fiction as attempts to save the real estate company. But I think the basic metaphor (Bluth company = Arrested Development, building houses = producing television) was there from the very beginning.

Anyway, while trying to bring to my story the scrupulous concern for scientific accuracy that is the hallmark of the Lovecraftian style, I became curious about where Eris is, exactly. Fourmilab's Solar System Live orrery only shows the nine textbook planets, but you can show an additional object on the map by pasting in its orbital elements.

Orbital elements are sets of astronomical measurements that resemble the tables used to enforce copyright protection in old computer games. The orbital elements are published, Kilgore Trout-style, in dirty magazines such as the IAU's Minor Planet Electronic Circular. It is rawther difficult to use the IAU site to find the MPEC issue for a particular minor planet, but it's easy if you use a Google site: search. Eris's orbital elements were published in MPEC 2005-041, and they are as follows:

2003 UB313
Epoch 2005 Aug. 18.0 TT = JDT 2453600.5                 MPC
M 197.53790              (2000.0)            P               Q
n   0.00176902     Peri.  151.31153     -0.91258509     -0.02028701
a  67.7091000      Node    35.87500     -0.34877687     -0.48266077
e   0.4416129      Incl.   44.17700     +0.21340843     -0.87557240
P 557              H   -1.1           G   0.15           U   5

Paste that into Solar System Live and you'll see that Eris is currently in the general direction of Neptune, as seen in the accompanying graphic.

Maybe there's nothing to explain. These guys weren't dumb, so their ideas are usually internally consistent. But they didn't know as much as we do, so it's easy for us to pick holes in their cosmologies. Except that it was also easy for people like Cicero to pick holes in their cosmologies, mainly using the rhetorical form of Something Awful-type mockery. Also, once I got past Parmenides in my book, I stopped seeing general crackpot ideas and started seeing ones that looked a lot like modern crackpot ideas I'd encountered before. I believe that crackpottery took a great leap with the founder of the Eleatic school.

Parmenides was so impressed with deductive logic, he personified it as a goddess. According to my book this was the Greek method of dealing with "anything that exists independently of human will or effort, which is everlasting and has effects beyond human control". I actually think Parmenides himself was not a crackpot, though he had a really weird way of expressing his ideas. He looks to me like a radical skeptic.

Parmenides's goddess of logic convinced him that the universe consists of only one thing, which cannot move, has no parts or distinguishing features, and which is the only possible object of thought. Parmenides's monist universe reminds me a lot of the universe you inhabit when reading Descartes, at least until the third Meditation where Descartes totally sells out. Your senses can't be trusted and the knowable universe consists of your mind, about which you know nothing except that it's a thing that thinks... about itself. The radical skeptic arguments are, IMO, impossible to refute and this issue has never been totally resolved (not even by Prof. Hsu).

But radical skepticism is not very interesting. Your sense impressions keep on having parties and you sit idly by, not trusting them enough to join in. The successors of Permenides can't ignore the radical skeptical arguments, but they also want to do some philosophy. So everyone from that point talks about the unreliability of the senses, but except for the Eleatics (like Zeno), they posit a universe containing multiple things. These two conditions, I believe, formed fertile ground for the Golden Age of Crackpotism.

I think the defining feature of a crackpot idea is that its author expects their idea to explain the universe, but they aren't interested in poking at the universe to affirm or disprove the idea. They would rather use pure reason to determine what qualities the universe must have. Except that their pure reason is not so pure: it incorporates premises that were originally acquired through poking.

The Eleatics didn't bother poking the universe because nothing reliable would come out of it, and anyway there's only one thing, it's alredy been explained, motion is impossible, and nothing exists that might do the poking. Because of the force of these arguments, non-Eleatics were also wary of the value of poking. But unlike Parmenides, these guys took as their starting point for their pure reason exercises the behavior of everyday materials, the weather, and atronomical phenomena. Their theories are brittle because they're based not on pure reason (which would give you radical skepticism) but on a small number of sense impressions.

This is not really the Presocratics' fault, because the topic of organized ex post facto universe-poking didn't come up for many centuries (even Aristotle didn't have a good handle on it). But now that we have it, when someone ignores it and does what the Presocratics did (sense impressions -> theory -X-> more sense impressions), we call it crackpot.

The guy who really made me start thinking about this was Anaxagoras, who believed that the fundamental forces of the universe are Love and Strife. Love makes like attract unlike; Strife makes like attract like. Really elegant. More elegant, in fact, than the similar modern crackpot theory (don't remember whose) that posited Suck and Blow as the two fundamental forces.

Also: Democritus, in addition to writing a dialogue between the mind and the senses, came up with his famous atomic theory. This theory started out pretty well, but ran into huge problems because of his insistence that everything is made of atoms. Even energy and ideas. When you see something or think about something, your atoms are colliding with atoms from the thing you're seeing or thinking about.

This reminds me a lot of a crackpot that I think Martin Gardner discussed in Fads and Fallacies. Said crackpot claimed that sound was a particle and not a wave. Sample argument: it's absurd that a cricket would be able to move such a huge volume of air that you could hear it from far away. Instead, the cricket is filling that same huge volume of air with particles, and you hear the particles.

Incidentally, check out the G.O.B.-like behavior of Empedocles, the non-Eleatic who firmly established the now-canonical four elements that show up in video games.

[One story of his death] is that he leapt into the crater of Mt. Elna "wishing to confirm the report about him that he had become a god." ... Empedocles' ego and flair for showmanship. In public he wore a purple robe, a gold crown, bronze shoes, and a laurel wreath. He wore his hair long, had a retinue of boys to attend him, and adopted a grave demeanor. He was known as a physician and magician (professons by no means distinct in antiquity). According to a widely known story he kept a woman alive for thirty days without breathing or pulse.

No, Socrates, it's my illusion!

One of the three stories I liked in Ted Chiang's Stories of Your Life and Others took place in the cosmology of ancient Babylon. In ancient Babylon, the story would have been a 50s pulp story with a lame twist at the end. But because we know the universe doesn't work that way, it was an exploration of large-scale counterfactuals, as big as the ones you get in grand space opera. So this pre-Socratics book is making me think about going back to messing with the cosmology in stories. Except not the way I did when I was twelve.

But also I was thinking, if science fiction had been around in antiquity, it might have explored the plethora of cosmologies offered by the Greeks. Aliens would not just look different from us or have a different psychology; they'd be from a different cosmology. For instance, Diogenes thought that air was the motive force behind life and intelligence. How would that work? What if a person from that cosmology spent a lot of time breathing the same air as someone from our cosmology? I guess we could still find out.

Okay, dude is so not a Puritan. He wears a big hat, and accasionally he remembers he's Christian. That's about it. Maybe a Puritan swordsman would be interested in... purifying the church? Imposing the priesthood of all believers? Kicking popery's ass? No, Solomon Kane cares not for these things. He does the same stuff as Howard's other heroes, except he does it in a dour brooding way. Current mood: stern.

Solomon Kane spends a lot of time in Africa. He discovers lost civilizations. He fights the undead. He has a magical staff his witch doctor buddy gave him. He's Conan, except smarter and not as fun to be around. The 1920s pulp great-chain-of-being racism stands out in stark relief here; it's not as noticeable when Conan is the viewpoint character.

Solomon Kane's schtick is protecting the weak. His modus operandi is to wander the earth, he knows not why, until he finds or hears about someone who's weak (inevitably a woman), and then he protects the hell out of that person to the exclusion of all other activities. His most interesting trait is this obsessiveness. In these stories he tracks people down for years to get revenge on someone who wronged a weak. More than any of the two-cents-a-word descriptions, this is the bit of characterization that makes Solomon Kane come alive.

The stories are never boring, there's some nice cosmos-mixing, and a great moment where Solomon Kane is the victim of great-chain-of-being racism. But I was really disappointed with the way the character was used. He's written as an AD&D paladin, full of do-gooding crusader spirit but with no specific religious beliefs. And the stories are mostly "lost continent" adventure tales divorced from the historical context. If I'd written these stories I'd have him fighting the Royalists. Okay, undead Royalists.

Brendan wrote his own version of "The Fungi From Eris", based on my prelimenary notes.

In related notes, I recently tried to catch the science fiction zeitgeist by writing a transhumance story, but it didn't work out well.

Unstrangely, the night after seeing this movie I had several very vivid dreams, including one that took the form of a MST3K skit. This is fitting because the movie's aesthetic was very MST3Kish. In my dream Joel went off somewhere to fight a space monster. The bots were worried about him, and when he came back unharmed they clamored to hear the story of the fight. Joel gathered them around in his parental way and started rambling on and on about strategy and fight psychology and everything but the fight itself. "And, you know, I'm not double-jointed or anything, us Hodgsons [sic] have always fought with the one set of joints." Etc. I'm not that big of a guy, but I'll crawl ya. The dream also involved Tom Servo's head being used as a disco ball.

Voyage to the Great Attractor does not answer these teleological questions. It tells of the discovery of the Great Attractor and of the subsequent attempts to demonstrate that the Great Attractor really exists and is not just observational error. Because the G.A. is hard to see: it turns out to be right in the part of the sky blocked from view by the Milky Way. Dastardly attractor! (I wonder whether or not this orientation is an accident. I can do a thought experiment where this happens naturally, but I can also poke big holes in the thought experiment. Since the book contains no speculation on this topic, I'm assuming it's an accident.)

Unlike the Hawking books, this book focuses less on results and more on the observing and computer-programming and arguing leading up to the results. Along the way you get a good overview of current (1990s) cosmology. The book also explains in detail concepts I was fuzzy on but that are neccessary for astronomers to do their work: what the red-shift is used for, why the cosmic microwave background is the frequency it is, etc.

This book contains no string theory, so if you don't believe in string theory you can learn all about the large-scale structure of the universe without having someone's unfalsifiable theory of its small-scale structure crammed down your throat. I'm going for the real niche markets here.

Most of the low-value stuff arrived within a week. The high-value stuff and two-thirds of the books disappeared off the face of the earth. I didn't buy tracking on the packages, so I can't track them from the USPS web site, but the post office has a way of tracking insured packages. They show the packages being accepted into the Bakersfield post office on May 17 and 19. That's it. My inheritance is gone.

As with any disaster I'm always revisiting what I could have done to avoid it. The most obvious thing is that I should have written my address directly on the plastic tubs. That way, no matter what happened I would eventually get the tubs along with whatever was in them at the time. The other obvious thing I should have done is not used USPS. I spent about $200 on postage, and it was only that low thanks to my liberal use of media mail. But the stuff I cared enough to insure -- especially the scrapbooks -- I should have cared enough to send UPS or FedEx.

After much badgering and form-filling and asking people here and in Bakersfield to check the back room, I've given up. After much more badgering and form-filling, today I got the post office to accept my insurance claims. In a few weeks they'll either find my packages or send me some money orders. I didn't buy a lot of insurance: just enough to (I thought) make the post office take my valuable packages seriously. But at least I'll get some closure.

When someone you love dies, the things they leave behind can keep your memories of them alive if you incorporate them into your own life. For a long time the only thing I had of my father's was one of his shirts, which is why I was so happy to find his postcards. Right now it looks like the tangible reminders of my mother's life are just more things that are gone forever.

Incidentally, WADL, and the Java implementation by its creator, now has its own webpage.

However the most prominent feature of Nextfest was the big-ass line that took an hour to get through. You really need to buy a ticket online. Okay, that ordeal over, there was some cool stuff in the various pavillions. For instance, Atari (??) had a pavillion full of entertainment stuff like a Claymation terrarium studio and a real-world fighting game, both products of Finnish ingenuity.

There were also many disappointments. Many exhibits had, instead of a thing, a flat-screen TV showing what the thing would look like if they were here. Example: the Lifestraw, one of the greatest inventions of this century, which is tiny and costs about $5 but they didn't have one on display. Ben was excited about seeing the "hybrid locomotive", until (according to him) he discovered that there was only a small model of said locomotive, and a picture of someone standing next to a larger model of the locomotive.

The "Exploration" (ie. "Space") section was kind of a disappointment -- as, indeed, has exploration itself been recently. There was a good exhibit on the Orion capsule, and according to the brochure an exhibit on detecting black holes which I missed.

I will forever consider the biggest rip-off "The Kitchen of the Future". You may have noticed a pattern here where you see something promised in the glossy color ad, with a description written by the same people who write quick, pat descriptions of new technology for Wired, but the thing itself is nowhere to be seen. There was no kitchen.

"Oh but it's not the future yet Giblets" you say, "You just need to wait til the video of the present becomes the kitchen of the future." Maybe it was the present this afternoon but now it's the future and still no kitchen!

What's more, when you call something "The Kitchen of the Future" and only provide a video, you're calling up memories of all those 1950s and 1960s videos on the same topic: Design For Dreaming et cetera. Here's a good historical overview that includes the new video. We expect some camp or at least some style. But this video was dull and creepy. You seemed to be inside some monolothic kitchen appliance, looking out at the people tapping on the touch-screen interface and following the recipe instructions displayed thereon. Both you and they seemed trapped.

And what are the benefits of this kitchen? RFID readers in the fridge, meal suggestions, shopping lists, automatic oven setting. The same things that these kitchens have been promising us forever; only the terminology has changed. I wrote a song eight years ago (called, yes, "The Kitchen of the Future") making fun of this exact thing. The song isn't very good, which is why no one but Jake has seen it, but I think I've made my point by now. If you can't get to the future in fifty years, then if it's world peace or artificial intelligence you should keep trying, but if it's a new kind of kitchen I think you should take a break and rethink the whole project.

Anyway, so as not to by totally cynical, an honestly cool thing about this kitchen of the future: the mega-appliance manages its energy by tricks like "using oven heat to warm dishwater". I've often thought that people could save a lot of energy with a lightweight, portable heat sink that plugs in elsewhere as a heat source. That's probably not possible though.

Looking through the catalog for other cool things I saw... Laser harp! Not really "future" but certainly futuristic and festive. Oh, we went to a demonstration on robots as entertainment, where we saw a great video from robotlab. They program industrial arm-robots to draw portraits, tune them so that you can play music on them, and so on. Lots of odd musical instruments in this show. The equivalent of the .org Pavillion here was "Robot Row", which was full of robots, including robots built by high school students to compete against other robots in a basketball-type game. That was a lot of fun, though I don't really get robots. You may have noticed that I'm not really a hardware guy.

Cool-sounding stuff I didn't see: text-free user interface, some kind of cameraphone-based barcode reader for doing your microloan business paperwork (?), a biofeedback game, and Pixelroller.

PS: here's a Wikitravel guide to the most recent World's Fair.

PPS: There were a bunch of cars. Boring. Except they were hybrid cars. Sensiboring. Some of them ran on ethanol. This was demonstrated by patches of corn surrounding the exhibit. The corn was fake, ie. plastic, ie. made from oil. Secret of ethanol -- revealed!

PPPS: Here's Design For Dreaming and I'm also gonna toss in a link to Explosives: Tools For Progress just on the basis of the name.

Fortunately, you can use a separate binder that's tasty and goes will with cookies: oatmeal. I got this idea from the fatless cookies at a weird restaurant near Sumana's work that caters to bodybuilders. Today I made these cookies that use an insane amount of oatmeal. And they're pretty good. Applesauce brownies make a pretty good substitute for butter brownies. These cookies are a different animal from a typical oatmeal cookie, but they're definitely tasty.

Since I have previously dabbled in recommendation engine design, some friends have asked me if I'm going to go for this prize. Well, I don't really need the money -- just now I got email telling me I'd won $2,500,000.00 -- but more importantly I don't think my Ultra Gleeper ideas are applicable to this domain. They mainly focus on improving the recommendations, whereas this contest is very tightly focused around predicting users' opinions given specific recommendations.

Not to say I don't have ideas. If I were going for the prize I would use IMDB and Amazon data for the movies to gather hypotheses about why people rate a movie high or low. I would then build profiles for the users, using the set of hypotheses as a basis. Then I would predict the users' future actions based on the profile. That sounds like handwaving but the basic idea (which others have seized on as well) is using objective data about the movies, instead of trying to figure out connections between them solely by how Netflix users rate them. And weighted vectors would probably be involved.

Of course, for all I know Netflix already uses external data to run their recommendation engine, so don't just run with this idea and expect to get anywhere.

• 1 can black beans
• 2 chipotle peppers
• 4 eggs
• 8 corn tortillas

Drain beans and mash with peppers. Scramble eggs. Serve on warm tortillas. Add anything that you would put on a burrito: salsa, yogurt, cheese, greens, guacamole, etc. It's filling and tasty.

Dear God,

If you let the dinasor not exstinct we would not have a country. You did the right thing.

Jonathan

That's tough but fair!

• Sales brochures from the Computer History Museum. Includes The Raytheon Computer Armchair Shopper, which sounds better than it is.
• Atari books
• An unplayable demo of Planetfall 2. Another "sounds better than it is", actually.
• Games on the C64, Amiga, BBC Micro. Remakes of same.
• The first DDR game machine.

[T]hese ancient people achieved much, yet were clearly prevented from achieving more by reason of a certain attitude of mind which seems to have been responsible for their failure to develop the scientific method.

If the pre-Socratic philosophers have taught me one thing, it's that motion is impossible. Also that the scientific method is not something lying around that any civilization would see were it not for attitudes of mind. It's a technology with a lot of prerequisites which must be developed. It's still being developed. We didn't get falsifiability until the 20th century.

If there is an attitude of mind that you have to get rid of, I'd say it's the attitude that induction is cheating or doesn't count towards knowledge. That if you thought really hard about things you could deduce the teleological reasons behind them. It takes a really long time for people to give up on this idea, but I'm skeptical that you'd get the scientific method much sooner by deciding to give it up.

Amateur philosophy of science now ends.

• TOFU GOOD OR BAD: TOFU GOOD
• What is the fancy term that killer whales use for navigation and finding food: They call it "mrrrgrrraighwhaaai!"
• real dinosaurs: Take it up with this kid.
• the myth about awful snowman Once there was a terrible snowman who cleaved the world from primal chaos with one swipe of his corncob pipe! From the huddled masses shivering in the howling winds, he demanded obedience! As a symbol of his power he created a great fire in the sky! Which melted him, causing a big flood. Some say yonder mountain range was once the buttons running down his chest!

I've been browsing psyop sites like this one which is frequently updated, not just when there's a war; and this link-filled one. You can spend many a disturbing hour reading through those articles and browsing the scanned propaganda leaflets.

Two of my favorite articles are these WWII case studies: Gray and Black Radio Propaganda against Nazi Germany and H.M.G.'s Secret Pornographer. "In 1939 it never occurred to me that one day my turn would come to wage war on Hitler by pornography." Click through if you like, but there's no topping that sentence.

I can't find my favorite leaflet, but it was put out by the Germans in 1945 and it basically said "Don't get killed in the last five minutes of the war! We've already lost, so give yourself up and stay safe in a POW camp until Germany officially surrenders!" Not really on message; I think oversight was getting a little lax by that point.

NES games: Top 100, bottom 20 with obviousness filters applied. All selections arbitrary.

Not sure how this happened; I'd blame Captain Planet, but I can't get the chronology to work out. Anyway, I'm tired of this so I was thinking of ways to improve on the concept. The obvious one is to use the modern elements, but that adds about a hundred new gameplay... elements, only a few of which are interesting. Another way is to go with the four fundamental forces. That would lead to some interesting gameplay. This is the Temple of the Strong Nuclear Force!

Hi all! Wife here. The folks competing for the Netflix prizes are having machine-learning fun with a big ol' dataset. There's Customer 305344, who rated more than 17,000 movies. There's constant wishing and jibing about getting a lower root mean squared error. And there's the question: why is Miss Congeniality the most-rated movie on Netflix?

The answer lies at the end of a lovely little essay by benjismith, which includes the SQL command ORDER BY hatred. As it turns out, Miss Congeniality shares something with Fahrenheit 9/11, The Passion of the Christ, and Con Air.

In case anyone is mistaken in thinking Grand Theft Auto was the first videogame to have players willingly commit a felony, Watergate Caper tempted gamers to “Break Into Watergate Yourself” 30 years prior.

Here's a Watergate Caper flyer.

Planespotting geeks have been instrumental in breaking the CIA secret prisons story. This doesn't endear them to the military:

A few months later, Ray learned through "back channels" that his post about the Base Camp planes had caused heads to roll somewhere in the shadowy world of military "black ops." Apparently, someone, somewhere, had lost a job because of Ray's photos. At a desert bar, one of his friends -- a man with an unspecified connection to Base Camp -- warned him over beers to "stop messing with those Base Camp guys or you'll wind up dead in the desert with two bullets in the back of your head."

"Wouldn't one bullet be enough?" asked Ray.

Leonard suggests that even the Lone Gunmen would not have said anything so nerdy. I find in Ray's response a kinship to the old joke about the lawyer, the doctor, and the engineer about to be guillotined. The blade doesn't drop on the first two, so they're freed. Then the engineer lies down and says, "Well, there's your problem...."

This happens within a sentence too. I'll start a sentence, then go somewhere else and tack something on to another sentence. Then, considering the start of the sentence to be a trivial corollary of the thing I just tacked on, I'll get right to work on the subordinate clause. Fortunately, the problem bottoms out here; I don't start writing a word and then go running off to write another word somewhere else.

I bring this up by way of introduction to this sentence I found just now in a book chapter, lying at the end of a paragraph like a gnawed bone:

If we , and , then when

Thanks a lot, prior self.

I went looking all over for a video of Super Mario Movie, but of course it's not a video, it's an NES cartridge. Get it from Cory's site. It's 15 minutes long and if I'd done it it would be about 7 minutes, but it's pretty fun and has great 8-bit music. You can also get the source, which reveals that there's a BASIC-like language for the 6502. Here's some info from a class on Game Design for the NES.

I, Mario is pretty funny but less interesting. Rather than making new things with the original Mario graphics, it retells the Mario story as accurately as possible within modern game conventions and using modern, anti-cartoon graphics. Rather like the disturbingly literal paintings of video games that if you know what I'm talking about you know what I'm talking about, and if you don't you probably shouldn't try to find out. It's a good concept but it suffers from a surfeit of world-building and belief that this will turn into a real game, and a shortage of people like the aforementioned painter who will just force their vision on the idea and let the chips fall where they may.

Anyway, I've now joined the ranks of the ascended. Now to work on that foodless zen pacifist caveman.

Also advertising on Starslip: This usually-funny raytraced comic, and a thing I'm not going to link to because it seems pretty sketchy, though it might just be arbitrage.

I used the interview as an excuse to update the Stock Spam Effectiveness Monitor main page to reflect some of the stock spam discoveries and developments in the past two years.

I said "To get more votes, he should change his name to B.A. Barackus Obama."

Sumana said "You have to post this." So I did.

Update: Then we started talking about economics and I said "I pity the greater fool!" And she said I had to post that too. I don't want to turn this into a Mr. T version of mola mola day, so hopefully it will stop here.

"Senator Bedfellow says he's for American jobs. But this one time, he ate a Toblerone! What's next, Senator--the killing of adorable kittens?"

Note to self: never claim that something would be "too flimsy to use in a campaign ad".

Sumana and I have started sharing a private notebook, which turns visibly into sharing a del.icio.us account. Today she found for me a statement of the principles of URI design in rant form. You know that I'm a connoisseur of rants about things that aren't really designed to be ranted about.

There are lots of books about Big Web Services: complex distributed-object systems that reproduce the mechanics of method calls over HTTP. The problem is 1) these systems are way too big for what they do, and 2) they're on the web but they aren't of the web. They don't use any of the web's features, or interact with anything else on the web. They just use HTTP as a transport protocol. They could just as easily run over TCP and get better performance.

These un-weblike systems were able to take the name "Web Services" away from the competition because the competition is... the web. Just writing programs that interact with the web. Sounds pretty sketchy! The web may be all right as a platform for serving movies, or selling books, or trading stocks, or democratizing publishing, or coordinating huge volunteer projects, or searching much of the information currently in existence, but there's no way it's up to the task of managing Accounts Payable! For that you need... Web Services!

As Big Web Services gathered steam, the pro-web forces rallied behind the banner of REST: a name for the design philosophy that made the human-visible web so successful. They preached simplicity, addressability, statelessness and the uniform interface of HTTP. They also got into lots of heated arguments about what REST really meant.

Some organizations created services that claimed to be RESTful, and others critiqued those services and said they weren't RESTful really, and is "RESTful" even a word? Meanwhile the world's programmers started finding options in their IDEs that generated code for Big Web Service clients and servers, so they wouldn't have to do so much programming.

I got the idea for a REST book when I started seriously trying to figure out what was and wasn't REST. I noticed that though certain best practices showed up repeatedly in REST folklore, they never really progressed beyond that point. I decided to write a book that would set down the folklore and hopefully create some new canon, some common ground. I got Sam Ruby to agree to do the hard parts. And when I started work on the book I discovered a basic rule of thumb, a framing device that really focuses on what's important about REST, and makes it easy to tell what's RESTful and what's not.

We call this framing device the Resource-Oriented Architecture (we're not the first, but the other uses are compatible with ours), and I'm going to be writing about it a lot more in NYCB. It's too good to keep hidden in a book along with a bunch of implementation details.

Some people might say, "Leonard, why are you guys even writing this book? There already are lots of books about the web, and REST is just the web, right? That's the whole point! We wrote those books back in the 90s."

To them I say: "Some people, think about Ajax for a bit. Ajax, as a technology, is only about two years old, but it's composed entirely of technologies that are much older, like Javascript and XMLHttpRequest and the browser DOM. So why are there all these books about Ajax? (Including Christian Gross's Ajax and REST Recipes, coming out later this month and seemingly the only other book to mention REST in the title.) Why does Ajax excite people when DHTML was a big yawn? Because Ajax is a way of framing these old technologies so that people can do awesome things.

"There's nothing on the market that explains what REST really means and how to make it work for you. Nothing about programming the web in a way that's more structured than screen-scraping without falling into the pit of Big Web Services. This is a way of framing preexisting technologies that lets people do awesome things."

Then I add, "Speaking of Ajax, did you know that an Ajax application is basically a REST web service client that runs in your web browser? That's why Christian Gross's book is called what it is, and that's just one of the amazing reality-twisting facts you'll learn from my new book, Jon." Because I'm talking to Jon Stewart! I'm on The Daily Show, plugging my new REST book! It's a dream come true! But wait--I'm not actually on The Daily Show! I'm just on a clip from The Daily Show that someone uploaded to YouTube! And now someone is sending a DELETE request to my URI! I can see the Authorization request header, its value is "DMCA Takedown Notice", and nooooooooooo

Then I wake up. Whew! It was just a regular dream, not a dream come true. I get up and go back to work on my REST book.

Ruby is awesome and the Ruby Cookbook was a lot of fun to write, but it wasn't my idea. It's part of a high-profile O'Reilly series. There was destined to be a Ruby Cookbook, written by somebody, from the moment Rails strapped an E motor to Ruby and hit the igniter. This book was my idea, and Sam and I are driving it. In the coming months I'll be writing a lot more on this topic. There's a lot of work we need to do in the real world, in addition to writing stuff down in a book.

PS: I say this on the homepage for the book, but in case you haven't clicked through and you're interested in REST, please email me or Sam. We'd love to have you help us with the folklore and best practices, and we need lots of reviewers.

Let me talk about light pollution. Recently I moved to a place where I have around 18 million neighbors. A lot of these people like to leave lights on at night. The effect is that every night the sky glows about as bright as the full moon (there's also a full moon tonight, which isn't helping). With curtains I can get a simalcrum of darkness, but I haven't seen (or, rather, been unable to see) real night darkness in months.

Now, I've seen the stars and they're not going anywhere, I don't need to look at the stars every night, but this lack of darkness is taking a toll on me. We're now at a point where there's about the same amount of ambient light from 5PM to 7AM, and the rest of the time there's normal daylight. I've never had any problems with SAD before I moved to New York, so I think it's not the darkness that's bothering me; it's the fact that it never gets really dark.

Also, almost everyone in the cast looked like someone else I'd already heard of. So you could pretend that the skits were being performed by an ensemble of Owen Wilson, Simon Pegg, Pauline Yates, Sumana's friend Steve Schultz, Aladdin Ullah from Uncle Morty's Dub Shack, and Jacquelyn, who I couldn't think of anyone she looks like.

The first problem is to sort out the terminology. There are many ways of classifying services, and we're introducing two more in the book: "Resource-Oriented Architecture", which I've mentioned before, and "The programmable web", which is a way of saying "web services" without implying Big Web Services, or even saying the word "services". We got the latter term from the excellent programmableweb.com. (We also introduce "Big Web Services" itself, which is a way of saying "web services" and implying only the SOAP/WSDL/WS-* stack, but I think I'm going to use that term a lot more in these entries than in the book.)

The main work thing I did today was draw diagrams: a taxonomy and Venn diagram. Don't worry, they'll be redrawn by real graphic designers before they're printed in the book. One or both might make it in; the Venn diagram is simpler but the taxonomy has examples and chapter references.

Both diagrams show two decisions I've made for the book: first, we draw a distinction between Resource-Oriented and Service-Oriented architectures. Second, we define HTTP+POX as a hybrid of the two architectures. The former because it's not useful to pit a technology against an architecture (as in "SOAP vs. REST").

The latter because the web services described as HTTP+POX, the ones that are kind-of-but-not-really RESTful, tend to deviate from REST in the direction of RPC and the Service-Oriented Architecture (rather than having no architecture at all, which was what I expected). Examples include the Flickr API, which includes explicit method names ("flickr.photos.search"), and the del.icio.us API, which has special URIs that you GET to invoke particular methods ("/posts/delete").

Some interesting edge cases: almost all web applications can be considered HTTP+POX web services. They're usually not all that RESTful, but they take standard HTTP requests with non-fancy entity bodies, and serve a non-fancy XML dialect (HTML or XHTML). This is how libraries like WWW-Mechanize look at the web.

In fact, I suspect the combination of URI- and RPC-based architectures to form HTTP+POX comes from our experience writing apps for the human web. You can't do nontrivial REST in a web browser without Ajax. HTML forms only support GET and POST, they don't let you set request headers, and they have hard-coded destination URIs (this is why URI templates are going to be great).

I used to think that, while the lack of PUT and DELETE was lame, people who obsessed about those methods as a determiner of RESTfullness were just being pedantic. But the ROA/SOA classification shows why it's important: if you have to overload POST, you end up creating extra URIs which the client POSTs to in different circumstances. This puts you well down the road to the RPC model. Your URIs no longer point to resources; they point to procedures you can call by POSTing. It's natural to keep following that model when you go from writing a web app for browsers, to writing a web service for computer programs.

Another edge case is that a website full of static pages is technically a RESTful web service. It's composed of resources (files), each with one URI and one representation (the data in the file). Each URI correctly supports a subset (GET and HEAD) of the uniform interface. This is how libraries like Beautiful Soup look at the web.

If you look at the Venn diagram you see that the entire human-visible web (static websites and web apps) is part of the programmable web. This is quite intentional. People were programming to the web long before anyone started talking about "web services". It's dirty work because the human web wasn't designed with software clients in mind, but sometimes it's the only way to get the data.

There's one thing in the diagram I'm not clear on. It might be a minor point or it might be an important aspect of our definition of HTTP+POX. The question is whether there are web applications that are so RPC-oriented they cross the limits of HTTP+POX and move in on the architectural turf of SOAP/WSDL services. I drew the Venn diagram with web apps that were neither REST nor HTTP+POX, thinking of really monolothic web apps that only expose a single URI, and do everything by POSTing to that URI.

The "POX" in HTTP+POX basically means "no SOAP", but what specifically don't HTTP+POX people like about SOAP? Is it the complexity of the SOAP message itself? Plain old XML can get pretty complex too. Or is it the fact that the SOAP message contains information (like the method name) that they think should go in the URI?

There are also, as one emailer pointed out, problems with calling anything a "web service" if we're drawing distinctions between a "service-oriented architecture" and something else. I'm not sure what to do about that. I'm tossing around the idea of switching that label to "RPC-oriented architecture", but I don't want to introduce yet another term of art.

I'd like to recommend open-uri for everything, including as a client for REST web services, but there are two big features it's missing. You can't use any HTTP method other than GET, and you can't send any body data (which makes sense if GET is the only supported method). Lately I've been building on my Cookbook work, coming closer and closer towards a general HTTP client library for Ruby, but really all I'm doing is reinventing open-uri with two extra features. It would be a lot better to just add those features to open-uri.

I submitted a patch for open-uri on this topic a while back but have yet to get any response, even from people saying it's stupid. I'm writing REST clients with open-uri under the assumption that that'll work by the time the book is published. For instance, here's some code from my Amazon S3 client:

  # Saves this bucket to S3.
  def save
    open(uri, :method => :put)
    return self
  end

  # Deletes this bucket and all its contents.
  def delete
    open(uri, :method => :delete)
  end

I guess you could argue that these features break the metaphor, that PUTting to a URI isn't really "opening" it, or that it's like opening a file for write access and you should have to write the body data inside a code block the way you do for a file.[0] But c'mon, let's have that argument. I don't want to have to keep telling everyone about fake reimplementations of open-uri just so they can write REST clients in Ruby. There's also precedent, like XMLHttpRequest.open in Javascript.

I'd also like to add caching support to open-uri, like the caching support in Joe Gregorio's httplib2 for Python. But I'll wait on that until the fate of this simpler patch is decided.

[0] I doubt this is really the intent of open-uri, since right now it doesn't support HEAD either. I think the intent of open-uri is to make it trivial to make an HTTP request from Ruby.

Good points, and highlighting an increasing source of uneasiness on my part. Which is good, despite the fact I generally don't like uneasiness, because it shows it's paying off to embarrass myself by formulating this book in public. People are interested enough in the topic to form uncoordinated revolts around ideas that are wrong or need work.

First a disclaimer about how much this matters: not a whole lot. This is a book about REST and the ROA. The name of the ROA might change yet for marketing reasons, but we've got its technical features pretty well defined. There is a chapter (#1) that covers the programmable web as a whole, a chapter (#3) whose first half covers services popularly considered RESTful that aren't really, and a chapter (#8) that compares REST/ROA to "the other guys".

I was calling "the other guys" "SOA" but that's starting to seem wrong. My current wild hare is to call them "Message-Oriented Architectures", though that has a bit of a collision with Message-Oriented Middleware. That way "Service-Oriented Architecture" is freed up to describe a higher-level architecture based on providing and consuming web services (ROA or MOA or both). Which seems closer to existing usage of "SOA", insofar as "SOA" means anything besides "your new issue of BusinessWeek is here."

Point is, our taxonomy could change completely and we'd only have to make real changes to those three chapters (and #8 isn't even started yet, partially because we're trying to sort this stuff out first). More important, much more detailed, and hopefully less contentious, will be our description of what goes on inside the "REST" part of the Venn diagram.

Anyway, our task here is the Confucian task of the rectification of names. We're trying to classify the set of techniques and technologies used on the programmable web. If it turns out to be best described with gradients instead of subsets, it might be a 2D or 3D map instead of a Venn diagram or taxonomy.

This doesn't have to be like the Linnean classification system where everyone signs onto it for all time, but it should be done with respect for the way "the other guys" see themselves. Sometimes in the text of the book we'll poke fun at what I call Big Web Services⁰. But we need to have a credible story about what Big Web Services really are, how they differ from RESTful web services, and what lives in between. Ideally someone who disagrees might say "Well, I wouldn't call it that, but I see how you draw the distinction."

So here are some unsolved mysteries of our taxonomy:

I don't think anyone will dispute that there's at least one rival web service architecture to REST/ROA: the RPC architecture that shows up in many web applications, in web services like Flickr's, and in XML-RPC. Can we consider this architecture as similar to the message processing architecture that rarely shows up in web applications, and that is commonly associated with SOAP? That is, granted there are differences, do they matter for a book about REST, or do they both differ from REST in a way we can capture in some snappy term like MOA?

Are all these architectures mutual opposites, or points on a gradient?

It it useful to distinguish between the typical robotic SOAP/WSDL interfaces autogenerated from Java or .NET code at the touch of a button, and the in potentia SOAP architecture that can be anything, even RESTful?

Consider the services popularly considered RESTful but seriously non-thesis-compliant. We currently term those things "HTTP+POX" in deference to common (kind of common) usage. Is that just another case of conflating architecture and technology? Is it worth coming up with another term? And given that there are multiple rival architectures for web services, do these not-quite-RESTful services consistently deviate from REST in the direction of some other architecture, or are they just freakish one-off mutants? Are they the missing links on a gradient between the rival architectures?

⁰ Said like in one of those political ads with lots of black-and-white closeups: "Senator Bedfellow doesn't want you to know about his millions in contributions from Big Web Services. When the FBI came knocking, he started shredding the XML documents. But he forgot to shred the envelopes. Senator Bedfellow: wrong on URIs. Wrong for the web."

He's Seth David Schoen!

However Sam and I did have a long IM conversation today, and cleared up a lot of taxonomy issues, as well as some of my misunderstandings about SOAP. I feel confident enough now to take another stab at my taxonomy of the programmable web. I got about halfway through before Ben came over to watch Battlestar Galactica, and that half is what I present to you tonight.

Again the disclaimer: though people will argue about these topics until the end of time, for purposes of this book we just need to get a coherent story about what it is we're cutting away as we zoom in on the programmable web to focus on the parts called "REST" and "ROA".

There are two major architectures on the programmable web. One is oriented around the passing of messages from place to place, and one is oriented around the manipulation of server-side objects called resources. We're (still tenatively) calling these the Message-Oriented and Resource-Oriented Architectures. Both architectures have real web applications and real web services to their credit.

These architectures can show up elsewhere than on the web, but we're not concerned with that. On the programmable web a service under either architecture uses the same underlying technologies: HTTP; a data format like XML, JSON, or form-encoded key-value pairs; and at least one URI.

Because they're built on HTTP, both architectures have a request cycle where the client asks the server to do something, and the server does it and sends a response. But we've found two questions which the MOA and the ROA answer differently:

1. How is the request scoped? How does the client tell the server what part of the application it wants to operate on? Let's say the purpose of the request is to fetch some data. Why should the server send back data X instead of data Y?
2. How does the client convey to the server what it wants the server to do? Why should the server send the requested data instead of modifying it somehow?

First a brief digression. When you send an HTTP request, you have two main options as to what to send in the entity-body. You can just send your data, or, if your data is an XML document, you can stick it in a box first. The name of the box is SOAP.

Just to avoid confusion: "SOAP" here is referring to the SOAP standard, not to the things people do with SOAP in the real world (that comes next). Sam says: "There isn't much to SOAP. It is three elements and a fault descriptor. Oh, that, and a world view." I'm just talking about the three elements and the fault descriptor. The worldview is the MOA.

Why would you stick your XML document in a box? Well, there are a variety of stickers that can be slapped onto the box without disturbing the contents. These stickers are the WS-* stack. That's the five-cent definition.

With that in mind, let's consider how the Resource-Oriented Architecture answers the two questions. A client for an ROA service scopes its request to a resource, by sending its request to a URI that identifies that resource. In an idealized ROA application there's one resource for every part of the application you might want to manipulate. Every resource has at least one URI, and you've got to send your request to some URI, so you might as well designate your resource by its URI. The URI goes in a special place in the HTTP request and can't be confused with anything else.

When conveying what it wants an ROA service to do to a resource, the client is restricted to a small and standardized list of methods. These methods are the famous HTTP GET, PUT, etc. Again, the HTTP method goes in a special place in the HTTP request and can't be confused with anything else.

If the client wants to convey an action that's not on that list, the problem is almost always that they've chosen the wrong resource. The service ought to expose some other resource for which the client's desired action translates naturally into one of the standard methods. The service might not actually expose this resource, but that just means the service can't do what the client wants. It's no different from the aggravation of a C library that's missing the function you need.

(There are edge cases I don't really understand, about overloading POST for atomic operations that span resources. I'm still trying to figure this part out. I think I may end up modelling this as a tiny MOA architecture inside the ROA.)

Now, depending on the service, when you need to send a document with your request, you might just send it as is, or you might stick it in a SOAP box first. The second possiblity, is, AFAIK, only theoretical. I don't know of any services I would describe as ROA where the client sticks its entity-bodies in a SOAP box before sending them.

This is partly for historical reasons. It's partly because it's simpler for the server to accept the entity-body as a document, instead of parsing it into a SOAP box covered in stickers, plus the "real" document. But it's mainly because HTTP already has stickers (HTTP headers). An HTTP request or response is already a box that can contain data and have stickers slapped on it.

HTTP stickers are more restricted in form than SOAP stickers, but they're good enough for most purposes. To the extent that ROA service programmers covet the features of WS-* stickers, they tend to devise ways of representing the same information in HTTP headers. Example: WS-Security's WSSE "ported" to the ROA as the X-WSSE header.

OK, enough about the ROA for a while. What about the MOA? And where do popular acronyms already in common usage fit in? Tune in tomorrow for the action-packed conclusion.

Always with deliberate irony, but my infernokrusher prose on the REST book page has two other goals. First, to market the book, to make people excited about it so I can ask them for help. Second, to act as a guideline and a goal. As the book nears completion I will find myself wrestling with the question of whether the manuscript does or does not put the "web" back into "web services". If it doesn't then we should probably do some more work on it. Does it drive all opposition before it, or is it just okay? Etc.

I'm exhausted, so no continuation of the taxonomy series tonight. But I would like to give an update on the name of the book. I got about 20 emails about it, almost uniformly against "with Ruby" in the title. On a semirelated note I can report that "with Ruby" is almost certainly gone from the title. The title is not finalized; editor Michael thinks "REST Web Services" is ungrammatical. But I think we agree on what kind of name will capture the goal of the book: an introduction to REST that has contains real examples but also has some philosophical heft to it.

The best argument for "with Ruby", I think, apart from "cheap trick to sell more books", is the same argument for using Ruby as our implementation language. We're not using Ruby because its XML support or HTTP client library is the best (it's not), but because Ruby is where the open source REST work is happening. This gives us lots to talk about without having to be polyglot throughout the whole book. Right now you don't see things like ActiveResource happening in other languages. (Note: this statement is precisely calculated to bring people out of the woodwork saying "you've neglected my awesome non-Ruby server-side REST tool!" so that I can take a look at said tools.)

The counterargument is that most peoples' interest in REST doesn't start on the server side. It starts when they write clients for big-company services like Yahoo!'s and Amazon's and Google's. They hear the term "REST" used (possibly incorrectly), and want to know more about the underlying architecture. There will always be more clients than servers, and the client programmers are using all kinds of languages.

Those whose interest does come from the server side are either trying to decide what to use, or just curious about RESt. Inside the book it's fine to say "Like REST? Your best bet right now is Ruby." But the book will fly past a chunk of its audience if the title says: "Like REST and Ruby? You'll like this book."

Strip includes cute fishlike aliens, and outdoes Battlestar Galactica in the low-tech science fiction aesthetic sweepstakes by having not just big clunky phones but big clunky payphones. Also you probably know this, but if you like science fiction comics with big space opera arcs and good character development you should check out Schlock Mercenary.

• How is the request scoped? How does the client tell the server what part of the application it wants to operate on? Let's say the purpose of the request is to fetch some data. Why should the server send back data X instead of data Y?
• How does the client convey to the server what it wants the server to do? Why should the server send the requested data instead of modifying it somehow?

Now I'm going to put out a description on the same principles of the Message-Oriented Architecture. Then I think I can make a case for how to classify weird cases like HTTP+POX.

To state the obvious (though it stops being obvious if you or we call the MOA something else), the Message-Oriented Architecture is about passing messages back and forth. Client sends server a message, server takes some action and sends client a message in response. A SOAP box with stickers on it is an example of a message. An HTTP request with headers is also an example of a message.

In an ideal MOA there is no information outside of the message. So if the message is in a SOAP box being transmitted over HTTP, then all the features of HTTP are just distractions and should be used as little as possible. There should be no leakage from the message into the transport protocol.

In the ROA, the fundamental server-side object is the resource. An ROA service usually has an infinite number of resources. In the MOA, the fundamental server-side object is the message processor, sometimes called an endpoint. A MOA service exposes a small number of message processors: usually one. A message processor is the thing that has a URI (or, possibly, the thing that can be the target of the WS-Addressing SOAP sticker).

Side note: A resource is just a message processor with a very limited vocabulary. So there's a pretty easy conceptual translation between the ROA and the MOA. This makes sense because HTTP is itself a message-oriented protocol. Of course, almost all ROA services expose too many "endpoints" to list. You either generate their URIs according to a rule, or you follow links in a document the server sends you.

An MOA might have ten or 200 or a million endpoints, but it's not ROA unless it has an endpoint or "resource" for every object the client might manipulate. Any service that exposes an infinite number of endpoints has at least some of the ROA in it.

This ties back into the famous triangle of "nouns" (resources), "verbs" (HTTP methods), and content types. The ROA constrains the "verbs" and lets the "nouns" run free. The MOA constrains the "nouns" and lets the "verbs" run free.

Side side note: an ROA service can contain a finite number of endpoints, but only if there's a finite number of objects the client might manipulate. A static website (like your cat's homepage, or the site that serves Google Maps image tiles) is an ROA service with a finite number of endpoints. A web service version of Parmenides's model of the universe would only have one endpoint, because there is only one proper object of discussion.

Now let's consider how the MOA answers the two questions:

How is the request scoped? Why should the server do x to data D1 instead of data D2? Well, in the URI it's scoped to the message processor. If your MOA service is actually ROA then that's all you need, because the "message processor" is the "resource" is the part of the application you're operating on. Otherwise, you need some additional scoping, and this is done by putting information in the message itself. This might be in the box or it might be in one of the stickers; the details don't matter.

How does the client convey to the server what it wants the server to do? Why should the server do x to data D1 instead of doing y to data D1? This too is scoped by information within the message itself. Again, this might be in the box or in one of the stickers; it doesn't matter to the MOA. Unlike with the ROA, the vocabulary of this sort of information is not constrained at all. If you constrained it you'd need to create more endpoints to handle the same actions with the more limited vocabulary, and you'd move towards the ROA.

Since in the MOA both of these pieces of information go into the message, there's a tendency to conflate them, and indeed that's what I used to do.

To recap from last time: the ROA puts scoping information in the URI, and puts the action information in the HTTP method. The big exception seems to be overloaded POST, where the action information can go in the entity-body. I'm now fairly certain that overloaded POST in the ROA is best described in terms of the MOA; I'll try to cover that next time.

Now I'd like to talk about a common design style for MOA services: the RPC style. In an RPC service, the scoping information corresponds to the information that would identify an object if this were an object-oriented program. The action information corresponds to a method that should be called on that object. As is proper with the MOA, both of these go into the message.

The message also contains some additional information, which corresponds to arguments passed into the procedure. The scoping information can be found among these arguments: things like the bank account number or the user ID. This is because an RPC service isn't really object-oriented. If it was, you'd be able to work directly with the objects (as, say, resources) instead of going through message processors. But with a little work you can get the same kind of "fake" OO you see in the GNOME project's C code, where the first argument to every function is a pointer to the "object". I put "fake" in scare quotes, possibly sending the most mixed message ever, because I think you can do real OO design in a non-OO system. But it looks strange, and you have to do work to get it.

In an RPC-style service, the HTTP request-response cycle is used to simulate a method or function call in a programming language. The request is the method invocation, and the response contains the return value of the method, or the exception it throws.

We all know that method calls are not neccessarily simple stuff where I ask you for information and you return it in a data structure. This "method call" might do anything: fetch information, store information, put a job in a processing queue, set up callbacks for later, etc. The response message might contain useful information, it might describe an exceptional condition, or it might just be a formality. If you register a callback the real data might be coming in later through the callback. So saying "RPC style" is not a negative value judgement about the capabilities of the style.

Let's take an easy case: XML-RPC. It's obviously an MOA architecture, since the only URI in an XML-RPC service is the URI to the XML-RPC service. It also rejects HTTP headers, methods, and status codes almost entirely, preferring to convey all information information in its XML message document.

It's also obviously an RPC architecture, because 1) it says so right in the name, and 2) its messages are full of incriminating tag names like methodCall and params.

Now, what about SOAP? Is SOAP an RPC style? No, that's a category error. SOAP came from XML-RPC (more accurately: they have a common ancestor) but they aren't the same kind of thing. XML-RPC is a way of describing method calls; SOAP is a way of sticking a message in a box and putting stickers on the box. This makes it useful for any MOA application, but it doesn't make it RPC.

So, riddle me this, Batman: how come basically the entire installed base of services that use SOAP also use the RPC style? Why does the average programmer think SOAP and then think RPC style?

SOAP is the message format of choice for automated tools that take Java or C# code you've already written, and with one click turn it into a web service. These tools are to web services as FrontPage is to web pages. The resulting "web service" exposes only one URI and accepts methods only in a very specific form that a person can't comprehend without a detailed map (a WSDL file) and a software tool on the other side (a WSDL client) that can read the map so you don't have to. When the software tool is done reading the map it's reconstructed something like the method signatures of the original Java or C# code.

This is why REST people dislike these services so much, and why that dislike often spills over into SOAP. This is why people tend to think that anything that doesn't fit this model is "REST". This is why I say these services are not really on the web: they're looking out at the web through a little peephole. To get any information out of them, you have to construct a very specific document, and then you have to put in a box and put the right stickers on it.

These services have the same design as XML-RPC services. They use XML Schema instead of XML-RPC's idiosyncratic data serialization format, and they use WSDL instead of the listMethods extension to XML-RPC. All the work done to make SOAP nothing but a good way of putting an XML document in a box and slapping stickers on the box, has passed these services by.

I don't like this because it's ugly and it pollutes the web. SOAP fans don't like this because it conflates SOAP with RPC when what they use SOAP for isn't RPC at all. I'm not really clear on what MOA styles they are using if not RPC, but I don't think it's going to come up a lot in a book about REST and the ROA, so I can just leave them to it and have Sam act as my lookout in case anything relevant to the book happens.

Who's the culprit here? I think it's those tools. They make it easy to get a web service that has the same interface as your code (that'd be a Procedure-Call interface that works Remotely), disregarding the fact that the architecture of the web doesn't look like that. Combine this disconnect with the complexity of the interface and the idiosyncracies of the tools, and you've made it easy for someone else to work with your service if they've got the exact same setup, and very difficult otherwise. This is pretty disgraceful since the real-world power of the web comes from its ability to connect everyone together.

The secondary culprit is WSDL. WSDL doesn't enforce RPC but it makes life easy for tools that want to force SOAP into an RPC mold. Sam is ambivalent about WADL. I suspect he's afraid that WADL will make it easy to force HTTP into an RPC mold, that tools for doing this will proliferate, and that we'll never see resources again. Sure, WADL makes you specify your resources, but a smart tool could easily subvert that and create services with a single "resource" that handles a whole bunch of method calls.

I think the scenario I've ascribed to Sam is not inevitable. A lot of the mess we see today comes from historical contigencies like SOAP's heritage in XML-RPC. But to prevent a future without resources we need a basic book that's willing to go out and fight for them. It means we need to straighten out the terminology—even if only informally and we only use the terminology locally—so we can create a frame for our argument.

Next time: HTTP+POX.

The Ajax chapter is also done, and I'd like someone who knows more than I do (ie. something) about Ajax to look at it.

There were two Picasso paintings (the really famous ones) that the MOMA owns and wouldn't lend to the Whitney. So the exhibit just put up the ripoff paintings and had tiny passive-aggressive reproductions of the originals along with explanations along the lines of "Girl Before a Mirror, in the collection of the Museum of Modern Arts since 1944, the cheap bastards!..."

This is great news for me as it demonstrates that I can write fiction people will buy. Look for it sometime next year, probably a little before the REST book is published.

The MOA field is dominated by RPC-style services, most of which use SOAP messages constrained by WSDL files. The client communicates its desire by sending a message (a box with stickers on it) to a message processor, which unpacks the box to find both the scoping information and the action information.

We've got two edge cases to explain: overloaded POST and so-called "HTTP+POX" services. It's been said (see link at end) that I shouldn't try to fit HTTP+POX services anywhere at all because they're just a mishmash, but I press on.

We've also got an unexplained coincidence. I've been describing SOAP as a way of putting a document in a box and slapping stickers on it. But you can look at HTTP the same way. The HTTP request (or response) is the box; the entity-body is the document; and the URI, headers, method name, and response code are all stickers.

HTTP+POX, as they say, is a mishmash, but I think I can identify the things it's a mishmash of. An HTTP+POX service is a message-oriented RPC-style service that uses HTTP as the box format, rather than using SOAP.

This is most obvious if you look at how you modify the dataset in HTTP+POX services (I'm using the del.icio.us API and Flickr's allegedly "REST" API as examples). You either send a GET or an overloaded POST to a URI that's not a resource: it's a description of the procedure you want to call. There's a message processor for every operation, rather than just one (as generally happens with SOAP+WSDL services), or one for every object on which you might want to operate (as with resource-oriented services). Examples: del.icio.us /v1/posts/add and /v1/posts/delete, Flickr's multitude of methodName arguments.

The confusing part is this: when the RPC method you're calling is "get some data", the HTTP method is usually GET, and the arguments to the RPC method usually go into the URI. This URI designates a resource, and you can GET a representation of that resource! This is why I thought that HTTP+POX services are hybrids of the resource-oriented and message-oriented architectures. In many cases, the message is one you'd send if you were using the uniform interface to get a representation of a resource. The service may not have been designed with resources in mind, but this part of it is functionally resource-oriented.

Examples. Consider an endpoint of the del.icio.us API: https://api.del.icio.us/v1/posts/get?tag=restbook That doesn't have the same ring as http://del.icio.us/leonardr/restbook, but it's the same kind of URI. Those are two URIs to two representations of the same resource: "my recent posts tagged with 'restbook'". There are infinitely many URIs of this form, each identifying a different resource.

Similarly for http://api.flickr.com/services/rest?method=flickr.photos.search&api_key=xxx&name=penguin. We're supposed to interpret that as a remote procedure call, but it's also the URI to a resource: "Pictures tagged 'penguin'". Another URI to a different representation of the same resource is http://flickr.com/photos/tags/penguin. Even a URI like /rest?method=flickr.people.findByEmail&find_email=leonardr@segfault.org is the URI to a resource, though a better URI for the same resource might be /people?email=leonardr@segfault.org.

We have our own recommendations about how to structure URIs in a resource-oriented service, but it's pretty small-minded to say that services aren't resource-oriented just because their URIs look funny and contain things that look like method names (like "get" and "method=flickr.photos.search"). From a functional standpoint, these services stop being resource-oriented when they stop using resources. And, in general, they stop using resources when it comes time to modify the data set.

The method names in those URIs are useless to the extent they agree with HTTP's uniform interface. The "get" and the "flickr.photos.search" basically mean HTTP GET, and the client's already using GET. When those strings become useful ("add" or "flickr.photos.comments.deleteComment") it's because the RPC interface supercedes the uniform interface. The URI can no longer be conceived as pointing to a resource.

/rest?method=flickr.photos.comments.deleteComment&comment_id=100 could be one of many URIs to the resource "comment #100". Maybe the "method" argument is just random junk. Maybe if you GET this URI you get the comment, and if you delete this URI you delete the comment. But of course if you GET this URI you delete the comment. This URI doesn't identify the comment: it identifies an operation on the comment. That's not a resource. It's just a procedure call.

You can see this by looking at web applications. A web application like a search engine is pretty resource-oriented. It exposes an infinite number of URIs that slice up the search engine data in various ways: by query, by page, by language searched. All of these URIs point to resources. You can't see any cracks between the message-oriented and resource-oriented models, because 1) the "messages" you're sending are standard HTTP requests, and 2) you can't modify the search engine data through the web interface, so you only use GET and you use it as intended.

Then again, consider NewsBruiser, the weblog program that publishes these words. I wrote NewsBruiser before the term "REST" was coined, but it's pretty typical of examples even today. I've got URIs like /nb.cgi/add, /nb.cgi/comment-add, /nb.cgi/configure, and so on. In general GETting one of these URIs gives you an HTML form, which you fill out and POST to the same URI to modify the data set.

Even putting aside the fact that you can't do PUT or DELETE with HTML forms, this is a very RPC-oriented application. It's got method names in the URIs. Yet when you're not changing the application you're using URIs like /nb.cgi/view/nycb/2006/11/03/1. That's got a method name ("view"), but it's redundant with the name of the HTTP method (GET), just like with the del.icio.us and Flickr examples. The rest of the URI describes a resource: the second News You Can Bruise entry of the third of November, 2006. Indeed, I've got a rewrite rule that cuts out all the redundant and default data so that you can use /2006/11/03/1 instead.

So that's HTTP+POX. It looks like there are two main paths to this architecture. You might come to it by designing web services on the exact same principles as web applications. This overlaps with the resource-oriented architecture when that architecture congrues with the capabilities of web browsers (GETting data from resources identified with URIs, sending data to hard-coded locations). It departs from the resource-oriented architecture when it comes to things you can't do with a web browser (using PUT and DELETE instead of overloading POST so much, sending data to URIs that the client generated dynamically).

Or you might come to it through an attempt to simplify the RPC style. You might be doing pushbutton SOAP+WSDL services and one day say "screw only having a single endpoint; I'm going to have an endpoint for every procedure and it's going to be identified in the URI and then I won't need WSDL". Or you might say "screw putting an XML document in an XML box; I'm already using HTTP, I'll just use it as my box and then I won't need SOAP".

Now a word about the name. I've decided "HTTP+POX" is inaccurate and I think I won't use it in the book except insofar as I need to translate real-world terms into my wondrous precise terminology. Here's the problem: like "AJAX" of old, "HTTP+POX" hard-codes a reference to XML where there might not be any XML.

To reiterate one of my earlier points in new language: what's "Plain Old" about Plain Old XML? What's "Plain Old" is that it's a document, not a box with a document inside it. You've already got a perfectly good box: the HTTP request. But that's a statement about the box. "POX" is a statement about the contents, and those don't have to be XML. An HTTP+POX web service might serve plain text, JSON, HTML microformats, or graphic files in an HTTP box. This holds even if you don't buy my argument that HTTP+POX is an RPC-style architecture.

I can think of two different names. One involves doing what they did to AJAX and lowercasing the "POX", making it a word instead of an acronym: HTTP+Pox (HTTP can stay an acronym because you are using HTTP and only HTTP). This has the advantage that it's an almost invisible change. But depending on how cocky I feel about my classification of these services as RPC services, I may try to push the term HTTP+RPC. Because "Pox" doesn't mean anything, and I don't like the trend towards stripping acronyms of their meaning when it becomes inconvenient (as also happened to SOAP).

The next installments will cover what to do with terms like "Service-Oriented Architecture" in the face of definitions that would a priori prevent REST from getting any of the pie, and also finally tackle the question of overloaded POST in resource-oriented architectures. I think it's going to be a while until the last one, at least, shows up. Before I try to figure out overloaded POST I want to incorporate into the book text my thinking in the series so far.

Pete quotes Anne Thomas Manes as saying:

Service oriented architecture (SOA) is a software design discipline in which application and infrastructure functionality are implemented as shared, reusable services.

As Pete points out this definition has a dependency on the definition of "service". Pete's definition is "a network available collection of related operations".

Okay, I'll buy that, and I'll also buy a $0.50 contract on Pete's "a resource is not a service". I'd say a resource is a service, albeit a very limited one that can only operate on one piece of data. An RSS feed is a good example of a one-resource service. But a collection of resources is definitely a service. In fact, "service" is what we call a collection of related resources.

A single resource can't have arbitrary operations, but you can expose arbitrary operations by exposing combinations of resources. It's not the same design as an RPC service, but only intertia associates RPC with "service-oriented" in the first place, and a service-oriented system could easily consume a mix of resource-oriented and RPC-style services.

PSEUDO-UPDATE: I finally finished the errata and decided to put up an update. Then I saw I had saved this entry as a draft and not published it. So this isn't a real update. Incidentally, the previous entry was one I wrote back in August and saved as draft instead of publsihing.

I've got much of the REST book converted to DocBook and am putting what I write online. Send me email if you want to do review.

> > I SHALL TAKE THE CHALLENGE!
> Join me or die! Can you do any less?
I can't think of any way I could do less in general, but I'm always on the lookout...

There are two different reactions to this: "We must stop them!" and/or "I remember doing that sort of thing." I am in the latter camp because the kids seem to know what they're doing; they play with the abandoned furniture instead of rooting around in the trash bags.

Sometimes it I think they know what they're doing a little too well. A while ago someone threw out an old wide-screen TV (a big box for a plasma-screen showed up around the same time). The other day some kids were sitting in front of the TV, coloring with markers and pretending to watch TV.

Some funny bits of conversation from our day:

"Wel-come to the social! Zune you'll be dead!" <-Cryptkeeper voice

"I don't know how all these little stores can survive."
"They survive because there are millions of people here, and they all demand breakfast sandwiches!"
"And punctuation errors."
"Yes. They 'demand' breakfast sandwich's."

I got a Met membership so that I can go whenever I want without having to pay the exorbitant "recommended donation", or having to deal with the passive-aggressive consequences of trying to get in for less. Last time I was at the big Met I found out that my favorite objects d'art in that museum, the ancient Chinese office supplies, are part of a temporary exhibit that goes away in January. Go see that exhibit! It's great! And it also has a bunch of cool Xu Bing art that I just discovered last time.

• Brandon Bird should paint an Adoration of the Magi in which the Magi are Kirk, Picard, and Sisko.
• Brandon Bird already painted a Last Supper, but perhaps some British equivalent to Brandon Bird could paint a Last Supper featuring all the incarnations of the Doctor from Doctor Who.
• Dun-da-dun-da-dun. Investor Gadget.
• It's well-known that a lot of the intro songs to James Bond movies are really lame and don't even have the same names as the movies. I now plan to write a song cycle of 21 songs, each with the name of a Bond movie.

• "Tequila"
• "Wipe-out"
• "Meow Mix Commercial" (theme)

I've been to a frozen custard place before: Carl's in Fredricksburg, Virginia. So I knew what to expect. And it was good, except here I ordered a concrete that included Valrhona chocolate disks or whatever they're called. I didn't really know what they are so I said "Disks? Sign me up!" Then I got the concrete and remembered what Valrhona chocolate disks are. Those guys are great... FOR BAKING. For putting in frozen drinks they're not so great. It's like putting chopped-up Snickers bars in a milkshake and then crunching on frozen Snickers bar all the time you're drinking it. When you get a concrete, stick to extra things that are soft.

Hamburger: eh. I probably shouldn't even have gotten this since Shake Shack had about 150 health code violations (not an exaggeration) last time it was inspected. It wasn't very good. Andy and I have been going out on excursions where we try out the city's hamburgers, and Shake Shack is pretty lousy. I'd rate it below Burger Joint (trashy take-out burger restaurant in fancy Parker Meridien hotel, primarily famous in class-obsessed New York due to its class-transvestitism; there are celebrity autographs on the wall inc. Stephen Colbert's), which doesn't even use real hamburger buns. In conclusion: Shake Shack is for frozen custard with soft things in it. That stuff is good and you're likely to run afoul of fewer health code violations.

If you're curious, best hamburger discovered so far: Zip Burger at 300 E 52nd. They have Belgian fries too, and non-perfunctory veggie options.

So instead she talked about Netscape's server business and the promise of the web and blah and blah. There were Powerpoint slides, which seems somewhat anachronistic for 1995.

Anyway, this morning we were doing a postmortem, and the conversation turned toward the mid-90s integration of the proprietary online networks (Compuserve, AOL, etc.) into the Internet. Specifically, the common shunning of AOL users. Any mention of shunning calls to Sumana's mind Gentlemen's Agreement, specifically the scene where Gregory Peck can't get a room at a New England inn.

"Yes, I'd like a room." <- Rugged Gregory Peck voice
"Certainly, sir, I think we can accommodate you. If you'll just put your email address down to get on our mailing list."
"gregorypeck22@aol.com..."
"Ah. Oh, I see, sir. I'll be quietly shunning you now, sir. It's a free service we provide... FOR THOSE WE DON'T GIVE ROOMS."

Almost all of REST Web Services so far is rewritten and in Docbook. I planned to get the whole thing done by the end of the month, but that's just another goal I didn't meet: still have half a chapter to redo. I'm very happy with the first 5 chapters so far, though. Total length in O'Reilly's stylesheet is 200 pages. Again, email me if you want to read the REST book as I write it.

"Hi. I'm a human..."
"...and I'm a Cylon."

Well, it didn't take me long to get up and search for those commercials online, where they now live. They're surprisingly similar to Douglas Adams's much more enjoyable Hyperland. In fact, I think Steven Levy was thinking about Hyperland when he described some aspects of Knowledge Navigator.

This year Backup Thanksgiving was roast chicken, orange-cranberry sauce, dressing, mashed potatoes, mashed sweet potatoes, carrots, green beans, and Parker House rolls. So delicious. But now Sumana's sick of the leftovers and I'm almost done reincorporating them. I made stock from the chicken carcasses and bread pudding from the cranberry sauce. Now I have to start thinking about what to make instead of just pulling stuff out of the fridge.

PS: The first six chapters of the REST book are now done!

The mail consisted entirely of commemorative postal covers addressed to President of the United States Dwight Eisenhower, other government officials, the Postmasters General of all members of the Universal Postal Union, and so on. They contained letters from United States Postmaster General Arthur E. Summerfield.

Reinforces my belief that the postal service makes most of its money acting as a kind of live-action MMORPG, arbitrarily creating things and declaring them to have value.

You do have to pick something in a field the other person knows fairly well, because the fun of the game is in being able to make a valid guess every time a new piece of information is revealed. I did very poorly when Sumana was going through "Hollywood Squares contestants" and "People from Cleveland". OTOH Sumana did well identifying the honorary Globetrotters (not an official Wikipedia category at the moment, though if you put the Futurama characters in "fictional honorary Globetrotters" you'd have one of the best Wikipedia category names ever). Sumana reminds me that I did well with "Fictional robots with emotion" and "Characters played by a member of the opposite sex". At the intersection of those two lies Gypsy.

Sumana says "You're really only indie and fresh if Wikipedia deletes you for not being notable."

• I got my Futurismic check today! (Plus T-shirt) I'm not yet a published fiction author but I am a paid fiction author.
• I thought of a publishing scheme that combines the street performer protocol with micropayments and the tradition of pulp magazines. I put up descriptions of stories with word counts. You send in payments at magazine rates (5-6 cents a word) and I reveal the appropriate number of words. If people think a story is boring they stop paying for it and it just cuts off in the middle.
• Recently I've noticed two comics that operate under similar constraints. If only there was some way to combine them...
• I got a copy of 1984's The Whole Earth Software Catalog, ostensibly for research for a 1980s steampunk story, but really because contemporaneous reviews of early personal computer software are SOLID GOLD. This is the tradition I like to deface in my Software and Game Roundups (been a while since the last one of those, actually).
• Diane Duane linked to my Star Trek slash pairing generator. This is vicariously exciting as Diane Duane wrote the best Trek-franchise novel among the far too many I read in grade school (that'd be Spock's World, which has the truest-to-life depictions I've seen of everyday life in the Federation).

He spoke to no one in particular. Most likely he was hawking fake or otherwise black-market Rolexen, getting plausible deniability from the crowd and his deadpan delivery. But I like to think of him as one of the new breed of mad monks, who take their mantras from the Heraclitan river of spam that flows through our lives. Strong buy strong buy one to watch about to explode strong buy...

what do the different parts of the christmas cracker mean?

What does a paperclip mean?

Here's one answer, though I can't determine the relative positions of tongue and cheek:

The Christmas cracker was invented, purely by chance, by an English baker called Tom Smith. He took the simple principal of the wrapped sweet or 'bon bon' and added first a love motto then, after much experimenting, a strip of paper impregnated with a compound which would 'crack' when opened. Over the course of time he dropped the sweet, lengthened the wrapper and introduced small novelty gifts.

I guess 2% inspiration and 98% perspiration moves an invention into "purely by chance" territory?

Much as I dislike the way SOAP is used these days, I'm not inclined to gloat, because the deprecation just means more work for me. I used the SOAP search service as an example in all three of my books. Now I've got to find another free, public, non-obscure SOAP service to use as an example (ideas?).

The official Google narrative is that the SOAP-based web service has been replaced by something called the "Google AJAX Search API". If you take this narrative at face value it means that Google has taken down their web service and put up an AJAX library in its place. What's AJAX? Who knows, it's AJAX. Here's a typical weblog entry on the topic.

Another victory for REST over WS-*? Nope -- Google doesn't have a REST API to replace it. Instead, something much more important is happening, and it could be that REST, WS-*, and the whole of open web data and mash-ups all end up on the losing side.

It's probably my recent proximity to Sam that's doing it, but I'm noticing a tendency in myself to draw fine distinctions. There's a sense in which this narrative is right and a sense in which it's not. I'm going to pick apart the narrative and show what exactly is disturbing about the Google AJAX Search API.

It's not true to say that "Google doesn't have a REST API to replace it." In fact, Google has two REST APIs, and one of them predates even the SOAP API. You've probably used this old API: its primary endpoint is http://www.google.com/.

Yes, the Google website is in fact a very RESTful web service. The downside of this web service is that it's a little bit difficult to use automatically, as opposed to through a web browser. It serves data in a human-oriented format (HTML), and you have to screen-scrape it into a data structure if you want to do anything for it.

There are libraries for doing this, but the other problem is Google doesn't want you to do it. It violates Google's Terms of Service ("No Automated Querying"). Lots of inconsiderate people write scripts that hammer Google's REST API day and night. Google tries to prevent this by sniffing out anything that might not be a web browser and preventing it from accessing the API. (To see this, set your browser's User-Agent to "libwww-perl" and try to use Google.)

But it can't be denied that people outside of Google have a powerful hankering for Google's dataset, so eventually someone (Nelson Minar, it seems) came up with a second web service API that was designed just for use by automated clients. The catch was that you had to sign up for a unique key to use it, and that key would only work for you 500 (later 1000) times a day.

In point of fact Nelson chose a SOAP/WSDL architecture for this web service. But there was no need to use any different architecture at all. Here's a possible different way of implementing the constraints above:

When you make an HTTP request to google.com, we try to figure out whether you're a web browser or an automated client. Ordinarily, if you're an automated client, we shut you out. But here's the deal. Now you can sign up for an "automated client key". When you make an HTTP request to google.com, stick your key into the Authorization header. Not only will we not shut you out, we'll try to make things easy for you. Instead of a human-oriented HTML document, we'll send you the appropriate data in an easy-to-parse XML format. But, we'll only do this for you 500 (later 1000) times a day. Then it's back to shutting you out.

This technique has a number of subtle benefits which I could bore you with for quite a while. But its obvious benefit is that it's got the exact same "API" as the Google website, which everyone knows how to use.

Anyway, instead of going down a route like this (which would, I think, have changed the history of web services quite a bit), Google went down the SOAP/WSDL route. Now they're deprecating the SOAP service in favor of some mysterious "AJAX API". This brings me to the second of Google's REST APIs.

There is no magical thing called an "Ajax request". An Ajax client makes normal HTTP requests, and processes the results automatically, just like a web service client. An Ajax client is a web service client.

What HTTP requests is the Google Ajax client making? I poked around a little bit and it looks like it mainly makes GET requests to URIs that look something like http://www.google.com/uds/GwebSearch?callback=GwebSearch.Raw&context=0&lstkp=0&v=1.0&key=xxxxxxxxxx&term=web+services. That's not exactly http://www.google.com/search?q=web+services, but it's not too far off either.

The Google AJAX API consists of a browser-side Javascript library and a server-side web service. The one acts as a client for the other. From what little I've seen of the web service I'd consider it quite RESTful. In fact, it's architecturally very similar to Yahoo!'s RESTful search API. They both use the same (IMO, fairly unsafe) trick to get a web browser to execute dynamically-generated Javascript code from another domain.

The main difference is that Yahoo's search API can also be made to send data (in JSON or ad-hoc XML format) instead of executable Javascript. That makes it possible for the service to be consumed by automated clients, not just by web browsers running client-side Ajax programs.

Let me just see if I can do something similar with the Google web service. The Javascript it serves is extremely close to also being a JSON document; I should be able to hack it a little and parse it as JSON.

Here's some Ruby code that gives you kind of a command-line Google search like people used to write for the old SOAP API. It requires the json gem.

You can skip the code.

#!/usr/bin/ruby
require 'rubygems'
require 'uri'
require 'open-uri'
require 'json'

KEYS = %w{GsearchResultClass unescapedUrl url visibleUrl cacheUrl
          title titleNoFormatting content results adResults
          content1 content2 impressionUrl}

def search(key, term)
  uri = "http://www.google.com/uds/GwebSearch" + 
    "?callback=GwebSearch.Raw&context=0&lstkp=0&v=1.0" + 
    "&key=#{key}&q=#{URI.escape(term)}"
  javascript = open(uri).read
  
  # Hack quotes around the hash keys to make the Javascript string
  # into JSON.
  KEYS.each do |key|
    find = Regexp.compile("\s*#{key}\s*:")
    json.gsub!(find, " \"#{key}\" : ")
  end 

  parsed = JSON.parse(json)
  return parsed["results"], parsed["adResults"]
end

# Command-line interface begins here

(puts "Usage: #{$0} [API key] [search term]"; exit) unless ARGV.size == 2
key, term = ARGV

results, ads = search(key, term)
puts "#{results.size} results for '#{term}':"
results.each do |result|
  puts result['titleNoFormatting']
  puts " #{result['url']}"
  puts " #{result['content'][0..70]}" unless result['content'].empty?
  puts 
end

unless ads.empty?
  puts "Look at some ads while you're at it:"
  puts '------------------------------------'
  ads.each do |ad|
      puts ad['title']
      puts ad['visibleUrl']
      puts " #{ad['content1']}"
      puts " #{ad['content2']}"
      puts
  end
end

Now, in old episodes of MacGyver, whenever MacGyver built a bomb out of baking soda and masking tape, the writers would change some crucial detail (like change the masking tape to Scotch tape) so that if kids copied MacGyver they wouldn't blow up the house. I've done something similar here. I've removed a crucial line of code from that program, so that people don't just go copying it and running it all over the place.

Why did I do that? Because when it works, that program violates the Google AJAX Search API Terms of Service. "The API is limited to allowing You to host and display Google Search Results on your site." I can use the old SOAP API to write a command-line search tool, but I can't use the new, RESTful API in that kind of application. My users can only access the RESTful API through a specific library (Google's Javascript library), running in a specific way (in their web browsers), for a specific purpose (displaying search results).

Wait a minute... running only in a web browser? Terms of Service? Bootleg scripts that hack the output into something a parser can understand? This REST web service is made available on exactly the same programming-unfriendly terms as the Google website "REST web service"!

Instead of screen-scraping a web page, I'm now screen-scraping a web service. I'm reverse-engineering undocumented URI formats, just like I do when I screen-scrape. So far, there's nothing on Google's end that sniffs my user-agent to make sure the web service only runs in a browser, but you can bet there will be as soon as that becomes a problem for Google.

The "blow to web services" narrative is incorrect. Google did in fact deprecate their SOAP API and expose a RESTful API. A win for REST!

Though incorrect, the "blow to web services" narrative is also correct. Google deprecated their SOAP API, exposed a RESTful API, and then erected a bunch of technological and legalese barriers around any attempt to actually use the RESTful API. You're only allowed to use it through one library in one language in one environment for one purpose. A loss for everyone!

On the level of technological choices, this move is a big improvement. They've gone from SOAP, which has a lot of overhead, to plain old HTTP, which has strictly less. Gone from an RPC style, which doesn't play well with the web, to a RESTful style, which does. This makes an enormous amount of technological sense. From its first day on the web, Google has exposed its dataset through a RESTful interface that gets orders of magnitude more traffic than any "web service" it might expose. In a sense, all they're doing now is unifying the architectures.

When it comes to getting information into the hands of people who can use it, Google has taken a big step backwards. The SOAP interface was serious overkill, but what you did with it was your business (though you could only do it 1000 times a day). The new RESTful interface is a technical improvement, but it's encumbered with restrictions that make it a museum piece. Unless you're writing an Ajax application using Google's library, its true value can only be obtained illicitly. And that's the other reason why I'm not inclined to gloat.

Leonard: I'll do it, but I'll be using scare quotes the whole time! "Scare quotes"!
Sumana: Yes, Leonard, scare quotes. That's what keeps you from being bourgeois.

I laughed and laughed.

Indeed, according to this random weblog entry the artificial beach is right next to a real beach. I guess that makes sense logistically but it takes us into the territory of a Vegas casino whose theme is "ripping off the other casinos on the Strip". Which undermines my point, because that would be the most awesome casino ever.

I finally got my sample web service written for the book, though it's still missing one feature. It's about 16K of code, which I don't know how many pages that translates into.

Clearly this is why my mother always signed registries as "Sigmund Freud".

• 2 chopped up zucchini
• 2 chopped up leeks
• 1 cup milk
• 4 eggs
• shredded hard cheese
• bread crumbs

Preheat oven to 375 degrees. Saute the leeks. Beat together milk, eggs, and cheese. Stack the zucchini in a 9-inch baking pan, mixed with the leeks. Pour the milk mixture on top and put bread crumbs on top of that. Bake for 50 minutes. It turned out great!

The web is full of stories of these arbitrary cutoffs, which I can now confirm in their relevant details. You get an email saying "we've cancelled your account based on classified information" and that's it. You get a pro forma chance to defend yourself, but it's not much good because you don't know what happened. It also doesn't make much difference, because the response always comes back: "we re-ran our classified algorithm on our classified dataset and it gave the same result as before, buh-bye".

Fortunately, I have a friend in the online ad business. Ryan's Project Wonderful mostly serves ads for webcomics right now, which is fine by me, as it means fewer "KEYWORD? Buy it now at creepysite.com" ads.

I bought the Waldrop book as a biography of Licklider (actually I bought it because of the BRIGHT YELLOW COVER that blocked out all other books in the bookstore), but it really strains to fill a whole book with his life and make it interesting. Fortunately the book's method of straining is to bring in all the other characters of American computer science from the 1940s through the 1970s, with a theme of Licklider as the networker between them, and that was very entertaining. Though I admit part of the entertainingness was the celebrity-spotting equivalent of those old Looney Tunes cartoons that feature charicatures of all the Warner Bros. contract stars of the 1940s. Oh look, it's {Peter Lorre,John McCarthy}.

The most interesting new information I found in the book is the history of ARPA, specifically of the Information Processing Techniques Office (Licklider was IPTO head twice). The major theme is the Vietnam-era mission anticreep that slowly pushed ARPA from advanced research projects, to the more circumscribed realm of advanced research projects that we can use on the battlefield within two years without fundamentally changing anything (somewhat parallel to the problems Xerox PARC had). Meanwhile Licklider's trying to scrape together a couple million to fund his crazy "Intergalactic Network" project, and his office is sticking boilerplate defense justifications onto incoming grant proposals so they can get ARPA money.

"All that language about military rationale wasn't in the Stanford version of the proposals," explains Ed Geigenbaum: it was slapped on at the very end by the ARPA funding officers back in Washington. "The only people who ever saw it were the students who would later dig it up under the Freedom of Information Act. Then they'd bring it on campus and say, 'See, McCarthy is working on such and such.' McCarthy would say, 'What do you mean? I never heard of that!'"

But the book also ties together things that I learned about in college with no that they were connected. For instance, ALOHA and Ethernet use the same collision resolution mechanism not because it's an obvious fact about the universe, but because Bob Metcalfe read a paper slamming ALOHA and decided to prove it could work. And multitasking came directly from time-sharing, as cheaper computers made time-sharing less of a selling point.

Fun fact: the book made it sound like ARPA was originally planned to encompass space research, and that NASA was created as a separate agency after a bureaucratic turf war. So the space program could have been part of ARPA.

Understanding the book requires no technical knowledge, but I don't think I would have enjoyed it as much if I didn't already have all these pieces in my mind ready to be connected. The book made me more interested in another item on my wishlist, John Markoff's What the Dormouse Said.

This entry got kind of long, so I'll talk about the Turing book in another entry.

One thing that jumped out of me was, reading the WWII sections, how much of the codebreaking relied not on abstract mathematical ingenuity but in ingenuity applied to exploiting the adversary's mistakes. For instance, the Germans often used a less secure version of Enigma for unclassified things like the weather report, not realizing that if Bletchley cracked the weather report they'd be 80% done cracking the secure cipher for that day. Not surprising that this got largely left out of Cryptonomicon, because fiction is usually more interesting when the adversaries are competent.

This creates a problem analagous to the problem of the anthropic principle in physics. It's not a perfect analogy because the development of the Internet took place within space-time, but I think you can see it. Why did all these unlikely contigencies happen?

I can think of three possible responses. The first is that the contigencies weren't unlikely at all. There is some hidden force of society that makes sure open solutions tend to win. This is the knee-jerk free software response. I like it okay, but I've grown more dissatisfied with it as I've read these books, because there are lots of other situations in the world where the open solutions lost big time. When does the hidden force work?

The second response is that we gravitate to whatever open solutions we can find to solve our problems. The general public can't use something unless it's open. If things had turned out differently we might be speculating on how unlikely it was that we would develop a tradition of collaborative biology or open letter-writing.

The third is to deny that there's anything special about the contigencies. We don't talk about things that don't happen, so if these contigencies hadn't happened, we wouldn't be talking about them. Saying that the Internet and the Web worked out the way they did is just a tautology. This response has its points when applied to the physics anthropic principle, but I find it unsatisfying in this case, because it looks like the same sorts of contigencies turned out the same way twice.

Anyway, just some high-concept rambling to see out the year.

I sustained an actual enthusiasm for clothes shopping for a record two stores, but by this time I was willing to settle for just trying on a topcoat and seeing how I looked in it. This resolve was strengthened by the $200 price tag on the topcoat.

The trying-on was a disaster. I'd thought I could just wear a topcoat on top of whatever and it would be warm and make me look classy, but I looked like a slob who'd stolen a topcoat from somebody. I was like the woman in Cryptonomicon who starts wearing stockings and then discovers that stockings dictate your entire wardrobe and lifestyle.

The topcoat works on Evan because Evan already has a compatible wardrobe, but it's totally out of my demographic. This was a fairly distressing realization. The coat would not transfer its class to me merely through a mercenary monetary transaction. This is the difference between class and mere wealth, and I don't even have the consolation that I became extremely wealthy and then discovered it.

I'm glad I got to spend a lot of time with my mother before she died, and I'm glad we got to see Susanna and John for Thanksgiving. My main 2006 regret is along the same lines: I wish we could have all been together for Christmas; I think we should start making plans for next year and get it set up as early as possible. Someone said that with Mom dead we would need to put in work to see each other, because we'd no longer have a natural gathering place, and I'm starting to see the truth in that.

My current plans for next year are writing-related: the REST book and lots more SF. Let me know if you want to read the noir space opera I'm trying to sell.