News You Can Bruise for 2008 March 13

Everybody on the net wrote about Gary Gygax's death but almost nobody wrote about Joseph Weizenbaum. The NYT has an obituary that covers the highlights: ELIZA and Computer Power and Human Reason. John McCarthy points out (Update: should probably be "pointed out", since he did it before I was born) that CP&HR contains a bunch of Leon Kass-esque "things man was not meant to know" stuff. This is true but the main thing I got out of the book is a warning about the ELIZA effect which I think is really insightful.

Throughout society we create rule-based systems and install them where there used to be people making decisions. If the rules are complex, maybe we put a person in charge of matching incoming events against the ruleset and carrying out responses. When dealing with the system we mentally fill in the blanks and it looks like there's volition there. But all the decisions were made up front and any people involved are just following a script.

This isn't a Chinese Room argument that no rule-based system can approach human intelligence. It's a pragmatic argument that says people will defer to and credit with intelligence systems that are actually really stupid. This is a serious problem because reactionary elements in society thrive on deference.

What's more, once we start crediting these stupid systems with intelligence we start modelling each other as stupid rule-based systems. After all, that's how the systems model us. We start thinking of each other (and ourselves) as less complex than we actually are. And thanks to the ELIZA effect, it works! We're happy! But our models are very wrong.

Here are some more contemporaneous reviews of CP&HR. Also see: Sherry Turkle's Edge Question Center thing.