News You Can Bruise

Incidentally, if you want to make money writing, articles are where the money is at, not books. I made about 75% of my Python book advance writing those three articles, and it took me about 10% of the time. Plus my book advance still only exists in potentia, whereas I've already gotten and spent my article money.

Not feelin' good. Not gonna take it out on you.

I'm sure there's something you're supposed to do ahead of time to prevent Python from choking on those characters, and that I'll hear all about it soon, but this is still useful because you can use it to get rid of the Microsoft smart quotes. Everyone hates Microsoft smart quotes, right?

I'm actually a little frightened of the mysterious way Daily Python-URL chooses what to link to. For instance they linked to this back in January even though it was just a random observation about Python and not, in my opinion, worthy of being linked to. I feel like I should put disclaimers in my non-weighty Python entries ahead of time. Don't link to this entry, Daily Python-URL! Link to that PyZine tutorial instead. It's cooler than this rather obsessive post.

Incidentally, you can do base conversion in Python by passing in the base as the second argument to the int() function. Pretty neat, huh?

The name is Kris'. He came up with it in 1998 when I was complaining that the algorithms I was learning about in class didn't live up to their cool names, like the gift wrapping algorithm and Graham's Scan. Since any interesting name I could give this library would set the user up for an anticlimax, I figured I'd go all the way. Thanks, Kris.

In retrospect, though, Graham's Scan is pretty cool.

PS: If you know of any cool Wikis, post them in comments.

Sorry, no spectacular thing to show you today. I've got only one of the three things working that I wanted to have, and (sad to say) it's not the one that hits people in the gut and makes them think This Is Important, insofar as any of them do that.

This is an HTML document.

->

["This is an", [TAG name='a' attrs={href: 'http://www.w3.org/MarkUp/', title: "Brought to you by HTMLCorp"} child=[TAG name='b' child=["HTML"]], "document"], "."]

Update, much later: I wrote Beautiful Soup because I wasn't happy with any existing parser.

Update: I don't think this entry deserves to be on the Daily Python-URL, but I'll take what I can get.

PS: Hey, Past Leonard, you saw The Two Towers with your family and Return of the King with the rest of the technical team from the presidential campaign where you work.

Past Leonard: Yeah, right. Stop calling me here!

Anyway, I see that Saruman has not been idle since the War of the Ring: his love of perverting all that is good and natural is surely the driving force behind Perlthon. Look at the author's name! "Manura"? An obvious anagram.

Future Leonard: Hey, Present Leonard, you will watch the extended DVD version of LOTR to pass the time on your colony ship's voyage to Sirius.

Stop wasting my time!

OK, here you go. Resign Patterns, including "Simpleton" and "Flypaper". From Ned Batchelder's weblog.

At dinner last night I had an ice cream dessert, like a giant Mochi, called a "Tartouffe". I do not know anything about it or why it is named after a Moliere character, but it was tasty.

I talked with Moshe Zadka a bit after my talk. I talked to some other people as well but I bring up Moshe because there were several talks he was going to give yesterday but he didn't get here in time. These include his "text steganography" talk and his "Writing web clients in Python" talk. I've asked him for copies of the notes for each, and they'll be part of the inevitable EuroPython roundup, coming sometime next week to this space. I was able briefly to recharge my battery, but it's only at 50% and I want to write another rambling, incomprehensible entry tomorrow, so I'll sign off now.

Best talk so far: a talk about PyPy, a Python interpreter written in Python, which is currently up to 20,000 times slower than the one written in C. Armin Rigo, author of PyPy, also wrote his own presentation softwareusing PyGame, which presented his presentation as a brainstorm diagram that he navigated as he gave his talk. It's only a matter of time until Steve Jobs steals that idea.

I went to Amsterdam. I've been hanging out a lot with Jarno and his girlfriend PrustiTaina, because their travel plans have overlapped with mine to an unbelievable extent (eg. they planned to go to Amsterdam the same day I planned to, they chose the same hotels that I did, etc.) I also ate a waffle. My French is only good for reading and not for hearing what people are saying, nor for talking because when I try to talk to someone in French they just respond in English.

Also, what has the same sort of semiarbitrary, dependency-ridden semantics as a ZIP code, but Europeans know what it is?

Then there's the awesome ezPyCrypto, a GPLed (so beware)library which makes it incredibly easy to do crypto in Python. Encrypt everything, just because you can! Encrypt your own source code in some sort of obfuscation/trusted computing thing! Encrypt the very name of your program! Encrypt your cat!

Before you write code, you're supposed to write unit tests. It turns out I can bring myself to do this most of the time, but I do it grudgingly because I don't like writing the infrastructure for a new type of test. The way I generally do it is to write the tests but leave the expected value for the assertions blank (unless the expected value is really easy to figure out, or is actually defined and not just the result of applying some transform to some data I just made up). Then I write the code and all the assertions fail. I use the code I wrote to figure out what the asserted values should be, fixing bugs as I do, and paste in the final, correct values for later use.

Where unit tests are really great is when you find a bug in your code. If you've put in the work to write that infrastructure for that sort of bug, it's easy to record your finding of the bug in a way that will automatically notify you if it ever shows up again. This is lesson one: unit tests give you a place to codify the results of the debugging process. By putting in some incremental effort you can ensure that you're notified should the bug you're fixing ever recur. Usually it doesn't recur, but your real worry is not the bug recurring; it's the bug recurring without your knowledge. With unit tests, you know.

Sometimes when I refactor code I worry that some portion of the code I never exercise has broken due to refactoring. This is the worst thing about Python: unless you're using Leo (which I'm still not, despite my earlier touting of it), when you refactor the code the indentation changes, and when that happens you have to redo the indentation because the automatic indenter never works right, and all sorts of problems can crop up.[0] Unit tests find the problems, and if there are any you fix them and that's that. Lesson two: with unit tests, instead of worrying about code you broke while refactoring, your mind is free to worry about less rational things, like bloodthirsty packs of feral hamsters roaming the streets.

I have unit tests for all my Eaters, and some decent coverage for the HTML parser, but nothing yet for the user interface (even though I wrote the user interface to be drivable from Python code) or the thing that grabs URLs. This is because of lesson three: the closer a piece of code is to an actual user, a piece of hardware, or data from the outside world, the more aggravating it is to write unit tests for it. I am convinced this demonstrates a close affinity between unit tests and Lisp.

The big wrapper lesson is that unit tests are a way of creating a grammar with which to describe the behavior of the system so that you can offload to the computer the task of checking that behavior. Unit tests are easier to write when the elements of the grammar have simple APIs that you control, and harder to write when the elements are complex things with outside dependencies, like URLs and mouse movements. Even if you only write unit tests for the simple stuff, it means your agonizing over the more complicated stuff will be untroubled by the suspicion that the problem is something lower-level and simple (of course, then it turns out to actually be something lower-level and simple, and the code says "Gotcha!" like Lucky Ducky and struts off, but I can't help you there).

[0] Eventually I'm going to write my own autoindenting Emacs hook which operates on the very simple principle of moving the cursor line to where autoindent takes it, then moving every child line left or right the same number of spaces. I don't know why other indenters have these fancy algorithms that don't work, when that algorithm works fine. With my algorithm you have to go down arbitrarily many lines before moving the first line, to see how many lines are underneath the first one, then go back up to the top and go through all those lines again, but on the plus side it actually works.

(PS: format taken from They Have Blogs, a web toy of about a year ago).

[From time to time, "Software Roundup" will appear in a bonus edition, evaluating according to obscure criteria the members of an overly bifurcated software category. Only Crummy Premium subscribers ($24.95/year) will be spared this exclusive content.]

By my side-of-the-envelope calculation, there are more Python templating systems than there are Rocks 'n' Diamonds levels. I've skimmed some web pages and tried to get a few of them to work, so I am now an expert on Python templating systems, at least as regards whether or not I can use them in NewsBruiser.

Here are my showstoppers for any module I might want to incorporate into NewsBruiser. The first two are born of my design goal of making NewsBruiser work for anyone whose hosting service has Python available and who can unzip a tarball. The last one is due to the unfortunate reality of software licensing. I've looked at the template systems with these showstoppers in mind, which means that often I'll simply dismiss a possibly deserving module without really looking at it; this is not a reflection on the module.

• Not-obviously-fixable incompatibility with Python 1.5
• Requires separate installation step
• BSD-incompatible license

First, the ones I can't or won't use:

• Cheetah: Beautiful Velocity-like syntax. Seems written for Python 2.2 from beginning to end, unfortunately, and it's a lot of code. Otherwise I would definitely use this.

• Zebra: XML-based. Somewhat language-independent, which is a cool idea. GPLed, so I can't use it.

• PSP I couldn't get to work.

• empy is GPLed.

• htmltmpl is Python 2-specific.

• XIST looks like a good idea but is Python 2.

[STOP PRESS! Crummy Premium subscription now only $24.91/year! Such a deal!]

• Yet Another Python Templating Utility works, but I really don't like its template format. XYAPTU is much better-looking, but contains a 2.2ism which I can't get rid of. If I could make it 1.5-compatible, XYAPTU would be a contender; stay tuned.

• PyMeld has some of the control flow logic in Python, but not as much as template (see below). It's 2.2-only, though.

• Wasp is a whole web framework, and the syntax is acceptable, but again: Python 2.

Now, the actual contenders:

• template has all the control flow logic in Python rather than in the template, which I don't like, but I don't see any reason why it wouldn't do the job.

• AHTS has a slightly awkward API, and the examples don't work as is, but otherwise seems to work. It looks like it only accepts variable substitution by name, though.

• tinpy took just a little work to get working in 1.5. It has a ?SP-like syntax, which is okay, but you can't access members or make method calls from within templates; it has to be variable references, dictionary lookups, etc. Otherwise, I think this is the best one for me.

So, unless I find something else that's better (do you know of any?), or I can get XYAPTU to work, I'm probably going to hack up AHTS or tinpy to fit my needs.

• Whacked On The Head By Python
• Python In This World And The Next
• Bowling For Objects
• Lists Suck!

I forgot to mention my talk. It went well. I'll link to everything when I get back, and mention all the other interesting things seen at PyCon (including the presentation that compelled Guido van Rossum to say, "I don't think our insurance covers this.").

At the very least, Leo is the outline editor I've been looking for, and I can pretend to be one of those people, like Joseph Allen, who uses an editor named after them. At most, I will become one of those people who is a fanatic about Leo on wikis.

Note: I had to recompile Python to get Leo to work, because my preinstalled Python defined Py_UNICODE as wchar_t even though wchar_t on my system is 32 bits, in direct contradiction to what The Python/C API Reference Manual says. I mean, geez.