(1) Sun Aug 07 2011 11:57 The MST3K-IMDB Effect, Quantified:
Sometimes when I rewatch an MST3K episode I go to the movie's IMDB page to learn more about it. Inevitably I'm annoyed by the comments of people who give these movies one-star reviews solely on the basis of having watched an edited version on MST3K. But even greater than my annoyance is my desire to quantify the phenomenon. Today, I have quantified it.
What does being on MST3K do to a movie's IMDB rating? My best guess is that it knocks 2.9 stars off what what the rating would have been if the movie hadn't been on MST3K. But read on to see how I came up with that number, and why it depends on the director.
Disclaimer
I am not a statistician. I'm not even a data scientist. I know how to get data out of the Internet. I know the difference between mean, median, mode, and standard deviation. And that's about it.
Example
Here's an example in case you're not familiar with the MST3K-IMDB effect, which there's no reason you should since that's a name I just made up for it. Consider "Speech: The Function of Gestures", a short film directed by Arthur H. Wolf. It's got 5 votes and an IMDB rating of 5.2. Now here's another short film in the same series, "Speech: Platform Posture and Appearance". Same director, same writer, same lead actor, but this film had the misfortune to be double-billed with Red Zone Cuba on MST3K. As a result, it's got 98 votes and an IMDB rating of 1.6.
Call me skeptical, but I've watched both films and I'm not convinced there's really a three-and-a-half star difference between them. Another film in the series, "Speech: Using Your Voice", was also featured on MST3K, but in a less memorable episode ("Earth vs. the Spider"), and it struggles along with an IMDB rating of 2.4.)
Methodology
Since the "Speech" films are part of a series, it makes sense to suppose that the difference between them is mostly due to the MST3K-IMDB effect. Of course, most films aren't part of a series. So I went by
director instead. I picked up the filmography of every director who
directed a film that was on MST3K. I split their films into two
lists, "Normal" (not featured on MST3K) and "MST" (featured on
MST3K). The "Normal" set only includes films that had enough IMDB
votes to be given a rating. I included shorts and episodes of TV
shows. This isn't perfect, because IMDB's plain-text data dump
sometimes (but not always) gives a director's credit where their
website gives a writer's credit. But it's close enough.
I took the average rating of the "Normal" list and the "MST"
list. The difference between the two averages is how much it
hurt that director to have one of their films featured on
MST3K. As we'll see see, some directors were hurt a lot, and some of
them shrugged it off, both for interesting reasons.
For the sake of comparison, the mean rating for a movie on IMDB at large is 6.4 stars, the median is 6.6 stars, and the standard deviation is 1.6 stars. However, "one star" is not a consistent unit of measurement. I'm considering redoing this table with normalized percentiles, but I'm not convinced there's a big demand for that, so for now you get stars.
Data
Here's a big table with the data for every director who had at at least five films in the "Normal" set and at least one in the "MST" set. Normalm and Normalstd are the mean and standard deviation for the IMDB ratings of that director's non-MST films, and Normaln is the sample size. MSTm, MSTstd, and MSTn are the same thing for the director's MST film(s).
Effect1 is what we're looking for: for a given director, how many stars does a film lose just from being on MST3K? But wait! What if the director made some good stuff and some bad stuff, and only the bad stuff ended up on MST3K? The MST3K set would have lower ratings, but it wouldn't be because of MST3K. That's where Effect2 comes in, and that's why the table is sorted by Effect2. I'll explain Effect2 after you get a look at the data.
Click here to skip the table.
Director |
Normalm |
Normalstd |
Normaln |
MSTm |
MSTstd |
MSTn |
Effect1 |
Effect2 |
MSTed Films |
Bava, Mario |
6.2 |
0.9 |
22 |
6.3 |
- |
1 |
-0.1 |
-0.1 |
Diabolik (1968) |
Francisci, Pietro |
5.0 |
0.9 |
9 |
4.8 |
0.7 |
2 |
0.2 |
0.3 |
Ercole e la regina di Lidia (1959), Le fatiche di Ercole (1958) |
Tucker, Phil (I) |
3.1 |
0.9 |
6 |
2.8 |
- |
1 |
0.3 |
0.3 |
Robot Monster (1953) |
Steckler, Ray Dennis |
3.1 |
1.3 |
22 |
2.1 |
- |
1 |
1.0 |
0.8 |
The Incredibly Strange Creatures Who Stopped Living and Became Mixed-Up Zombies!!? (1964) |
Rebane, Bill |
2.7 |
0.7 |
8 |
2.1 |
0.6 |
2 |
0.6 |
0.8 |
Monster a-Go Go (1965), The Giant Spider Invasion (1975) |
Burke, Martyn |
5.8 |
1.5 |
7 |
4.4 |
- |
1 |
1.4 |
0.9 |
The Last Chase (1981) |
Sachs, William |
4.3 |
1.1 |
9 |
3.2 |
- |
1 |
1.1 |
0.9 |
The Incredible Melting Man (1977) |
Warren, Jerry |
2.4 |
0.6 |
8 |
1.8 |
- |
1 |
0.6 |
0.9 |
The Wild World of Batwoman (1966) |
Maetzig, Kurt |
5.4 |
1.3 |
13 |
4.0 |
- |
1 |
1.4 |
1.1 |
Der schweigende Stern (1960) |
Buchanan, Larry |
3.3 |
1.2 |
25 |
2.0 |
- |
1 |
1.3 |
1.1 |
The Eye Creatures (1965) (TV) |
Yuasa, Noriaki |
5.0 |
1.6 |
12 |
3.1 |
- |
1 |
1.9 |
1.1 |
Gamera tai daiakuju Giron (1969) |
Gordon, Bert I. |
4.2 |
0.8 |
13 |
3.2 |
0.6 |
8 |
1.0 |
1.2 |
Beginning of the End (1957), Earth vs. the Spider (1958), King Dinosaur (1955), The Amazing Colossal Man (1957), The Magic Sword (1962), Tormented (1960), Village of the Giants (1965), War of the Colossal Beast (1958) |
Brannon, Fred C. |
5.8 |
1.3 |
43 |
4.2 |
- |
1 |
1.6 |
1.2 |
Radar Men from the Moon (1952) |
Mikels, Ted V. |
3.3 |
1.3 |
19 |
1.8 |
- |
1 |
1.5 |
1.2 |
Girl in Gold Boots (1968) |
Wood Jr., Edward D. |
4.0 |
1.0 |
15 |
2.8 |
0.8 |
2 |
1.2 |
1.2 |
Bride of the Monster (1955), The Sinister Urge (1960) |
Zarindast, Tony |
4.2 |
2.0 |
10 |
1.7 |
- |
1 |
2.5 |
1.2 |
Werewolf (1996) (V) |
Bradley, David (I) |
4.9 |
1.9 |
6 |
2.4 |
- |
1 |
2.5 |
1.3 |
12 to the Moon (1960) |
Ludwig, Edward |
6.1 |
1.0 |
33 |
4.7 |
- |
1 |
1.4 |
1.3 |
The Black Scorpion (1957) |
Clark, Greydon (I) |
3.5 |
1.2 |
19 |
1.9 |
0.0 |
2 |
1.6 |
1.3 |
Angels' Brigade (1979), Final Justice (1985) |
Franco, Jesus |
4.1 |
1.2 |
166 |
2.5 |
- |
1 |
1.6 |
1.3 |
The Castle of Fu Manchu (1969) |
Eason, B. Reeves |
5.8 |
0.9 |
46 |
4.5 |
- |
1 |
1.3 |
1.4 |
Undersea Kingdom (1936) |
Pyun, Albert |
4.5 |
1.5 |
43 |
2.5 |
- |
1 |
2.0 |
1.4 |
Alien from L.A. (1988) |
Sturges, John |
6.5 |
0.7 |
42 |
5.5 |
- |
1 |
1.0 |
1.4 |
Marooned (1969) |
Neumann, Kurt (I) |
6.2 |
0.9 |
51 |
4.8 |
- |
1 |
1.4 |
1.4 |
Rocketship X-M (1950) |
Rou, Aleksandr |
6.8 |
1.5 |
14 |
4.5 |
- |
1 |
2.3 |
1.5 |
Morozko (1965) |
Zens, Will |
4.4 |
1.8 |
7 |
1.6 |
- |
1 |
2.8 |
1.6 |
The Starfighters (1964) |
Corman, Roger |
5.4 |
1.4 |
44 |
3.2 |
0.8 |
6 |
2.1 |
1.6 |
Gunslinger (1956), It Conquered the World (1956), Swamp Women (1956), Teenage Cave Man (1958), The Saga of the Viking Women and Their Voyage to the Waters of the Great Sea Serpent (1957), The Undead (1957) |
Fukuda, Jun (I) |
5.8 |
1.3 |
10 |
3.6 |
- |
1 |
2.2 |
1.7 |
Gojira tai Megaro (1973) |
Piquer Simón, Juan |
3.5 |
1.0 |
11 |
1.8 |
- |
1 |
1.7 |
1.7 |
Los nuevos extraterrestres (1983) |
Crichton, Charles |
6.5 |
1.7 |
58 |
3.6 |
- |
1 |
2.9 |
1.7 |
Cosmic Princess (1982) (TV) |
Portillo, Rafael (I) |
4.7 |
1.6 |
10 |
1.9 |
- |
1 |
2.8 |
1.7 |
La momia azteca contra el robot humano (1958) |
Beaudine, William |
6.1 |
1.2 |
128 |
4.1 |
- |
1 |
2.0 |
1.7 |
Design for Dreaming (1956) |
Peshak, Ted |
3.7 |
0.7 |
14 |
2.5 |
0.1 |
2 |
1.2 |
1.7 |
Appreciating Your Parents (1950), What to Do on a Date (1951) |
Conway, James L. (I) |
7.2 |
1.2 |
74 |
5.0 |
- |
1 |
2.2 |
1.8 |
Hangar 18 (1980) |
Worth, David (II) |
4.3 |
1.3 |
20 |
2.0 |
- |
1 |
2.3 |
1.8 |
Warrior of the Lost World (1983) |
Grefe, William |
4.3 |
1.5 |
11 |
1.7 |
- |
1 |
2.6 |
1.8 |
Wild Rebels (1967) |
Shonteff, Lindsay |
4.5 |
0.9 |
19 |
3.0 |
0.6 |
2 |
1.6 |
1.8 |
Devil Doll (1964), The Million Eyes of Sumuru (1967) |
Kane, Joseph (I) |
6.2 |
0.9 |
127 |
4.5 |
- |
1 |
1.7 |
1.8 |
Undersea Kingdom (1936) |
Yarbrough, Jean |
6.5 |
1.7 |
87 |
3.4 |
- |
1 |
3.1 |
1.8 |
The Brute Man (1946) |
Hessler, Gordon |
5.7 |
1.2 |
48 |
3.4 |
- |
1 |
2.3 |
1.9 |
"The Master" (1984) |
Kessler, Bruce |
6.5 |
1.6 |
67 |
3.4 |
- |
1 |
3.1 |
1.9 |
"The Master" (1984) |
Lawrence, Quentin |
7.1 |
1.4 |
12 |
4.4 |
- |
1 |
2.7 |
1.9 |
The Trollenberg Terror (1958) |
Fox, Wallace |
6.1 |
1.1 |
31 |
4.0 |
- |
1 |
2.1 |
2.0 |
The Corpse Vanishes (1942) |
Kincaid, Tim (I) |
6.5 |
2.2 |
25 |
2.0 |
- |
1 |
4.5 |
2.0 |
Robot Holocaust (1986) (V) |
Malatesta, Guido |
4.3 |
1.4 |
14 |
1.5 |
- |
1 |
2.8 |
2.1 |
Maciste contro i cacciatori di teste (1963) |
Dein, Edward |
6.1 |
1.0 |
7 |
4.1 |
- |
1 |
2.0 |
2.1 |
The Leech Woman (1960) |
Juran, Nathan |
6.4 |
1.0 |
53 |
4.2 |
- |
1 |
2.2 |
2.1 |
The Deadly Mantis (1957) |
Baldanello, Gianfranco |
4.8 |
1.3 |
9 |
2.0 |
- |
1 |
2.8 |
2.1 |
Il raggio infernale (1967) |
Beebe, Ford |
6.1 |
0.7 |
50 |
4.6 |
- |
1 |
1.5 |
2.1 |
The Phantom Creeps (1939) |
Winters, David (I) |
5.5 |
1.7 |
17 |
1.8 |
- |
1 |
3.7 |
2.1 |
Space Mutiny (1988) |
Harvey, Herk |
5.1 |
1.1 |
22 |
2.7 |
0.4 |
3 |
2.4 |
2.2 |
Cheating (1952), What About Juvenile Delinquency? (1955), Why Study Industrial Arts? (1956) |
Medak, Peter |
6.7 |
1.4 |
63 |
3.6 |
- |
1 |
3.1 |
2.2 |
Cosmic Princess (1982) (TV) |
Corona Blake, Alfonso |
5.6 |
1.3 |
13 |
2.7 |
- |
1 |
2.9 |
2.2 |
Santo vs. las mujeres vampiro (1962) |
D'Amato, Joe |
4.7 |
1.3 |
141 |
1.9 |
- |
1 |
2.8 |
2.2 |
Ator l'invincibile 2 (1984) |
Pierce, Charles B. |
4.9 |
1.3 |
10 |
1.9 |
- |
1 |
3.0 |
2.2 |
The Barbaric Beast of Boggy Creek, Part II (1985) |
Rich, David Lowell |
6.6 |
1.2 |
99 |
3.8 |
- |
1 |
2.8 |
2.3 |
SST: Death Flight (1977) (TV) |
Strock, Herbert L. |
6.1 |
1.5 |
23 |
2.6 |
- |
1 |
3.5 |
2.3 |
The Crawling Hand (1963) |
Katzin, Lee H. |
6.6 |
1.3 |
71 |
3.6 |
- |
1 |
3.0 |
2.3 |
The Stranger (1973) (TV) |
Ulmer, Edgar G. |
5.7 |
1.0 |
38 |
3.4 |
- |
1 |
2.3 |
2.3 |
The Amazing Transparent Man (1960) |
Sloane, Rick |
3.4 |
0.7 |
14 |
1.7 |
- |
1 |
1.7 |
2.3 |
Hobgoblins (1988) |
Cardos, John 'Bud' |
5.1 |
1.4 |
9 |
1.8 |
- |
1 |
3.3 |
2.4 |
Outlaw of Gor (1989) |
Castellari, Enzo G. |
5.5 |
1.1 |
39 |
2.9 |
- |
1 |
2.6 |
2.4 |
Fuga dal Bronx (1983) |
Mahon, Barry |
4.4 |
1.3 |
34 |
1.3 |
- |
1 |
3.1 |
2.4 |
Rocket Attack U.S.A. (1961) |
Vogel, Virgil W. |
7.1 |
1.1 |
141 |
4.4 |
- |
1 |
2.7 |
2.5 |
The Mole People (1956) |
Giancola, David |
3.9 |
0.7 |
7 |
2.0 |
- |
1 |
1.9 |
2.5 |
Tangents (1994) |
Francis, Freddie |
5.7 |
1.1 |
30 |
3.0 |
- |
1 |
2.7 |
2.6 |
The Deadly Bees (1967) |
Baker, Roy Ward |
7.0 |
1.2 |
89 |
3.7 |
- |
1 |
3.3 |
2.7 |
Moon Zero Two (1969) |
Nicol, Alex (I) |
5.6 |
1.0 |
9 |
2.9 |
- |
1 |
2.7 |
2.7 |
The Screaming Skull (1958) |
Sears, Fred F. |
6.1 |
1.0 |
45 |
3.3 |
- |
1 |
2.8 |
2.7 |
Teen-Age Crime Wave (1955) |
Rakoff, Alvin |
6.5 |
1.2 |
25 |
3.3 |
- |
1 |
3.2 |
2.8 |
City on Fire (1979) |
Lieberman, Jeff (I) |
5.7 |
0.7 |
9 |
3.9 |
- |
1 |
1.8 |
2.8 |
Squirm (1976) |
Cahn, Edward L. |
5.5 |
1.0 |
106 |
2.8 |
- |
1 |
2.7 |
2.8 |
The She-Creature (1956) |
Arnold, Jack (I) |
6.7 |
1.0 |
109 |
3.9 |
1.2 |
2 |
2.8 |
2.8 |
Revenge of the Creature (1955), The Space Children (1958) |
Heyes, Douglas |
7.6 |
1.1 |
42 |
4.4 |
- |
1 |
3.2 |
2.8 |
Kitten with a Whip (1964) |
Ferroni, Giorgio |
5.3 |
1.1 |
16 |
2.0 |
- |
1 |
3.3 |
3.0 |
New York chiama Superdrago (1966) |
Fowler Jr., Gene |
6.8 |
1.1 |
9 |
3.4 |
1.1 |
2 |
3.4 |
3.0 |
I Was a Teenage Werewolf (1957), The Rebel Set (1959) |
Bava, Lamberto |
5.4 |
1.1 |
30 |
2.1 |
- |
1 |
3.3 |
3.0 |
Shark: Rosso nell'oceano (1984) |
Wolf, Arthur H. |
4.6 |
0.8 |
8 |
2.0 |
0.4 |
2 |
2.6 |
3.1 |
Speech: Platform Posture and Appearance (1949), Speech: Using Your Voice (1950) |
Jameson, Jerry |
6.1 |
1.2 |
109 |
2.5 |
0.6 |
2 |
3.6 |
3.1 |
Superdome (1978) (TV), The Bat People (1974) |
Koch, Howard W. |
6.3 |
1.3 |
20 |
2.4 |
- |
1 |
3.9 |
3.1 |
Untamed Youth (1957) |
Morse, Hollingsworth |
6.6 |
1.3 |
105 |
2.5 |
0.6 |
2 |
4.0 |
3.1 |
Crash of Moons (1954) (TV), Manhunt in Space (1956) (TV) |
Cottafavi, Vittorio |
6.2 |
0.8 |
17 |
3.6 |
- |
1 |
2.6 |
3.2 |
Ercole alla conquista di Atlantide (1961) |
Moxey, John Llewellyn |
6.7 |
1.2 |
110 |
2.9 |
- |
1 |
3.8 |
3.2 |
"San Francisco International Airport" (1970) {San Francisco International (#1.0)} |
McLaglen, Andrew V. |
6.5 |
1.3 |
215 |
2.3 |
- |
1 |
4.2 |
3.2 |
Mitchell (1975) |
Trikonis, Gus |
5.8 |
1.3 |
72 |
1.7 |
- |
1 |
4.1 |
3.2 |
Five the Hard Way (1969) |
De Martino, Alberto (I) |
5.2 |
0.9 |
25 |
2.2 |
0.2 |
2 |
3.0 |
3.3 |
L'uomo puma (1980), OK Connery (1967) |
Cardona, René (I) |
5.6 |
1.0 |
45 |
2.0 |
- |
1 |
3.6 |
3.4 |
Santa Claus (1959) |
Miner, Allen H. |
7.6 |
1.5 |
25 |
2.4 |
- |
1 |
5.2 |
3.4 |
The Days of Our Years (1955) |
Newfield, Sam (I) |
5.5 |
0.9 |
145 |
2.4 |
0.5 |
4 |
3.2 |
3.5 |
I Accuse My Parents (1944), Lost Continent (1951), Radar Secret Service (1950), The Mad Monster (1942) |
Sholem, Lee |
6.7 |
1.4 |
44 |
1.9 |
- |
1 |
4.8 |
3.5 |
Catalina Caper (1967) |
Turner, Ken (I) |
7.3 |
1.5 |
7 |
2.2 |
- |
1 |
5.1 |
3.5 |
Revenge of the Mysterons from Mars (1981) (TV) |
Haas, Charles F. |
6.8 |
1.2 |
31 |
2.6 |
- |
1 |
4.2 |
3.6 |
Girls Town (1959) |
Kowalski, Bernard L. |
6.9 |
1.1 |
80 |
2.9 |
0.2 |
2 |
4.0 |
3.7 |
Attack of the Giant Leeches (1959), Night of the Blood Beast (1958) |
Fukasaku, Kinji |
7.1 |
0.9 |
53 |
3.8 |
- |
1 |
3.3 |
3.8 |
The Green Slime (1968) |
Gentilomo, Giacomo |
5.6 |
0.8 |
11 |
2.4 |
- |
1 |
3.2 |
3.8 |
Maciste e la regina di Samar (1964) |
McDougall, Don |
7.2 |
1.4 |
158 |
1.7 |
- |
1 |
5.5 |
3.8 |
Riding with Death (1976) (TV) |
Webster, Nicholas |
6.8 |
1.1 |
13 |
2.3 |
- |
1 |
4.5 |
3.9 |
Santa Claus Conquers the Martians (1964) |
Austin, Ray (I) |
6.6 |
1.1 |
135 |
2.1 |
- |
1 |
4.5 |
3.9 |
"The Master" (1984) {Hostages (#1.4)} |
Oswald, Gerd |
6.7 |
1.2 |
52 |
1.8 |
- |
1 |
4.9 |
4.1 |
Agent for H.A.R.M. (1966) |
Lynn, Robert (II) |
5.5 |
0.8 |
12 |
2.2 |
- |
1 |
3.3 |
4.4 |
Revenge of the Mysterons from Mars (1981) (TV) |
Szwarc, Jeannot |
7.3 |
1.0 |
167 |
2.5 |
- |
1 |
4.8 |
4.6 |
Code Name: Diamond Head (1977) (TV) |
Myerson, Alan |
7.0 |
1.1 |
101 |
2.1 |
- |
1 |
4.9 |
4.6 |
"The Master" (1984) {State of the Union (#1.3)} |
Lipstadt, Aaron |
7.2 |
1.1 |
73 |
2.1 |
- |
1 |
5.1 |
4.7 |
City Limits (1984) |
Rondeau, Charles R. |
7.0 |
1.0 |
58 |
2.2 |
- |
1 |
4.8 |
4.7 |
The Girl in Lovers Lane (1960) |
Collins, Lewis D. |
6.0 |
0.9 |
56 |
1.7 |
- |
1 |
4.3 |
4.9 |
Jungle Goddess (1948) |
Green, Alfred E. |
6.4 |
0.7 |
73 |
2.4 |
- |
1 |
4.0 |
5.4 |
Invasion USA (1952) |
Levi, Alan J. |
6.9 |
0.9 |
122 |
1.7 |
- |
1 |
5.2 |
5.7 |
Riding with Death (1976) (TV) |
Lane, David (I) |
7.2 |
0.8 |
19 |
2.1 |
- |
1 |
5.1 |
6.3 |
Invaders from the Deep (1981) |
Greidanus, Tjardus |
6.3 |
0.7 |
13 |
1.7 |
- |
1 |
4.6 |
6.3 |
The Final Sacrifice (1990) |
Saunders, Desmond (I) |
6.7 |
0.7 |
14 |
2.1 |
- |
1 |
4.6 |
6.5 |
Invaders from the Deep (1981) |
Ptushko, Aleksandr |
7.2 |
0.4 |
8 |
4.2 |
1.0 |
3 |
2.9 |
6.9 |
Ilya Muromets (1956), Sadko (1953), Sampo (1959) |
Williams, Douglas (I) |
8.5 |
0.7 |
6 |
2.1 |
- |
1 |
6.4 |
9.6 |
Overdrawn at the Memory Bank (1983) (TV) |
Morgan, William (I) |
6.2 |
0.4 |
10 |
2.6 |
- |
1 |
3.6 |
10.1 |
The Violent Years (1956) |
Elliott, David (II) |
6.7 |
0.3 |
9 |
2.1 |
- |
1 |
4.6 |
14.1 |
Invaders from the Deep (1981) |
Average | |
|
|
|
|
|
2.9 |
|
Analysis
Now, for the explanation of Effect2. From Normalstd we know how likely this director is to make a film that's substantially better or worse than their average. If they made one bad film that was on MST3K, and there was no MST3K-IMDB effect for that director, the rating for that film would most likely be within two standard deviations of the director's average. But if there were a strong MST3K-IMDB effect for that director, the rating for the MSTed film would be much lower than the director's other bad films. So, Effect2 is: how many standard deviations below Normalm is MSTm?
Let's look at the extremes of the list. First, the directors with very low Effect2:
- Mario Bava seems to have actually benefited from having his movie Diabolik appear on MST3K! Looking at the IMDB reviews, it seems that the nostalgic 60s-spy-movie fans have taken the upper hand over the MST3K-IMDB effect and given this film a rating in keeping with the director's other films.
- At the top but in non-anomaly territory, we have Pietro Francisci,
who directed a bunch of sword-and-sandal movies
(Normalm=5.0), including two of the Hercules
movies shown on MST3K (MSTm=4.8). These movies are
nothing special but they're pretty fun, such that being on MST3K
barely hurts them at all.
- On the other hand, we have Ray Dennis Steckler, who directed
movies so bad (Normalm=3.1) that having one of
them appear on MST3K (MSTm=2.1) barely hurts the rating
at all.
And this is the big thing I learned doing the project: you can
calculate the MST3K-IMDB effect, but you must also look at the
director's average movie rating to see what it means. A low
Effect2 just means that being on MST3K doesn't hurt a
director's ratings very much. It doesn't say anything about the movie's
quality.
OTOH, a director with a high Effect2 is probably worth a
second look in a non-MST3K context.
- Look at the director at the
bottom, David Elliot, with Effect2 of
14.1. He worked with Gerry Anderson,
creator of "Thunderbirds", and Invaders From The Deep is made up of recut episodes of "Stingray", a "Thunderbirds"-type show. Cutting episodes of a TV show into a movie is never a good
idea, but "Stingray" has an IMDB rating of 7.9, which is quite a way from the MSTm of 2.1.
I'd couldn't even remember Invaders From The Deep being on
MST3K. Investigation reveals
it was on the very first non-pilot episode of MST3K. This episode is lost and no one's seen it for over
twenty years, so why is Effect2 so large? Could it
be that a bunch of MST3K fans gave this movie a one-star rating
without even seeing it on MST3K?
- William
Morgan made the questionable decision to direct the Ed Wood-penned The
Violent Years (MSTm=2.6), but he also directed
six-star movies like Mr. District Attorney and Disney's
cartoon compilation Fun and Fancy-Free
(Normalm=6.2). This is a case where it's hard to
isolate the MST3K-IMDB effect. It's probably there, but we're
looking at a guy who made one really bad film in an otherwise
average career, and that bad film also ended up on MST3K.
- Douglas Williams
directed Overdrawn at the Memory Bank, a movie that was
pretty good for made-for-TV but whose rating (MSTm=2.1)
is almost ten standard deviations away from his stellar
Normalm of 8.5, which he got from directing episodes
of Fraggle Rock.
- Robert
Carlisle directed Last Clear Chance
(MSTm=2.3), a short film that was a hoot on MST3K, but
he also directed a series of shorts called "Unusual Occupations"
that looks interesting. ("A daisy grower invents tinted daisies; a
woman makes 3-D seaweed art[.]") (Normalm=7.3)
- Alan J. Levi
directed TV shows from "Battlestar Galactica" in the 1970s to "Airwolf" in
the 1980s to "Columbo" and "Quantum Leap" in the 1990s to "NCIS"
today. (Normalm=7.2). But when the mediocre "Gemini
Man" (IMDB rating: 6.4) had some of its episodes re-cut into Riding
With Death, suddenly the premise becamse ridiculous and the director
earned an MSTm of 1.7.
- I always
enjoyed Aleksandr
Ptushko's lush Soviet epics, and IMDB agrees
(Normalm=7.2). But put one of his movies on MST3K and it
instantly loses three stars. (MSTm=4.2)
And so on. The MST3K-IMDB effect is real--ninety percent of the
directors in this table have an Effect2 of more than one
standard deviation, and for sixty percent of them, it's more than two standard deviations. But it doesn't affect all directors equally.
Let's close out by taking a look at some of MST3K's favorite
directors.
- Bert I. Gordon has a Normalm=4.2, which is
pretty bad. But his record 8 MST3K movies have an MSTm of
3.2, which isn't much worse. Put one of his movies on MST3K and it
loses only one star.
- Roger Corman, the B-movie king, has a pretty respectable (if you're Roger Corman) Normalm=5.4, but the same
MSTm as Bert I. Gordon: 3.2.
- Ed Wood isn't especially well-known for being on MST3K but he does
provide a pretty clear example of the MST3K-IMDB effect. His fifteen
non-MST movies have a Normalm=4.0. His two MST movies
have a MSTm of 2.8. Is The Sinister Urge (IMDB rating: 2.11)
really the worst movie Ed Wood ever directed?
- Unfortunately, there is no way to calculate the MST3K-IMDB effect
for the notorious Coleman Francis: since every film he directed
was on MST3K, we can't calculate Normalm.
Conclusion
I'm still annoyed by those one-star reviews, but I understand them a little better now. When you watch, say, "The Function of Gestures", you enjoy it for its camp value, you have fun with it, and you give it a relatively good rating. But when you watch "Platform Posture and Appearance" or "Using Your Voice" on MST3K, you're watching someone else making fun of it, you have fun at its expense, and you give it a bad rating as a sign of solidarity with the MST3K characters.
Finally, I'd like to thank IMDB for, in a relic of its geeky past,
making plain-text dumps
of its data available. It's a strange feeling to have a file open
in an Emacs buffer that lists nearly every movie ever made. (There are
about 2 million, if you're curious.) Now that I have the data and
scripts to process it, I may run other cinematic experiments in the
future. One thing I would like to see added is IMDB links for the people and movies. It's a pain to look all these things up, which is why there aren't as many links in this post as you'd think.
Thu Aug 18 2011 10:39 Queneau Assembly:
That's my name for the formerly-unnamed technique I used in "Board Game
Dadaist". It all started in April, the night I was guest critic
for Adam's ITP class. Afterwards I went out to dinner with Adam
and Rob, and Adam was talking up Markov chains. Dude loves him some
Markov chains. I said "Man, I'm tired of Markov chains. Markov chains are so 70s, they have little coke spoons dangling from them. I'm gonna come up with a better algorithm for creating generative text."
Big talk, but fortunately I didn't have to come up with a better
algorithm, because I already had. Back in 2008 I released a project
called "Spurious",
which generates new Shakespearean sonnets by picking lines from the
existing sonnets. It generates two sonnets at once using two different algorithms. Algorithm B (the one
lower down on that page) is totally random: you could get a new sonnet
made entirely of the first lines of other sonnets. But Algorithm A
(the first one on that page) creates what I'm calling a Queneau
assembly. The first line of a new sonnet is the first line of some
existing sonnet. The second line is the second line of some other
sonnet. The third line comes from the set of third lines, and so on to
the end.
Oulipo founder Raymond Queneau did something very similar in his 1961 book "Hundred
Thousand Billion Poems". This may be where I got the algorithm
I used in "Spurious", though I don't think it was a conscious
homage. In "Hundred Thousand Billion Poems" there are ten sonnets
bound such that you can "turn the page" for a single line of the
sonnet, changing that line while leaving the rest of the poem
intact. Each generated poem feels like a sonnet because it
starts with a "first line" and ends with a "last line" and every line
in between is placed where it was in some manually generated sonnet.
I've named the technique in honor of Queneau because I can't find
anyone who used it earlier. It's not universally better than a Markov
chain, because it only works in certain cases:
- It doesn't work on free text, only on texts with a small-scale internal structure (a poem, a paragraph, a short biography).
- You must have many source texts with that structure.
- You might not like how much of the original text it preserves.
That said, the Queneau assembly gives very entertaining results, and it's
now my go-to dada technique, promoted over Markov chains and even
unadulterated randomness.
The simple algorithm
I've come up with a number of algorithms for making Queneau
assemblies. I'll talk about the simplest first, just so you'll see how
this works. This is a refined version of the algorithm I used for "Mashteroids" (yes, those asteroid
descriptions were me reinventing Queneau assemblies). It's not the
algorithm I used for "Board Game Dadaist"; I'll talk about that later.
You've got a body of N texts, T0, T1, ...,
TN-1. Each text can be split into some number of chunks,
eg. T00, T01, ...,
T0M-1.
Split each text into chunks and assign each chunk to one of three
buckets. The first chunk from each text goes into the "first"
bucket. The last chunk from each text goes into the "last"
bucket". All the other chunks go into the "middle" bucket.
Also keep track of how text lengths are distributed: how likely it
is that a text consists of one chunk, how likely that it consists of
two chunks, and so on.
Now you're ready to assemble. Pick a length for your new text that
reflects the length distribution of the existing texts. Then pick a
chunk from the "first" bucket. If your target length is greater than
1, pick a chunk from the "last" bucket". If your target length is
greater than 2, pick chunks from the "middle" bucket intil you've got
enough. Concatenate the chunks first-middle-last, and you've got a
Queneau assembly!
Paragraphs made from sentences
Now let's look at the scales on which you might create a Queneau
assembly. Outside of poetry, the paragraph is the Queneau assembly's
natural habitat. A pragraph has a flow to it, especially when you've
got something like a description of a board game or an asteroid that's
only one paragraph long.
You need to handle things like quotes and parentheses that open in
one sentence and don't close by the end of the sentence, or that close
without having opened. I wrote code for this in BGD but it doesn't
catch all the cases.
Phrases made from words
In "Board Game Dadaist", the names of games are also Queneau
assemblies. Here the chunks are words. I take the first word from the
name of game A, the second word from the name of game B, and so on. So "Pirates! Denver" might come from "Pirates! Miniature Battles on the High Seas" and "Monopoly: Denver Broncos".
Quotes and parentheses are still problems, though it's not as
bad. The big problem I ran into was repeated words, and words like
"the" which are not allowed to end a game name. (The simple algorithm,
with its "last" bucket, prevents "the" from showing up last unless it
showed up as the last word of an existing game. In the algorithm I
used for "Board Game Dadaist", I had to special-case this.)
In general, Queneau assemblies will not create coherent English
sentences. Much as it pains me to admit, a Markov chain is better for
that. It works for board game titles because we allow titles a lot of
creative license, even up to the point of suspending the rules of
grammar. "Pirates! Denver" makes no sense as a sentence, but it's a perfectly good game title.
Words made from letter-chunks
Many games have single-word titles, eg. "Carcassonne". I wanted to
have single-word titles in BGD, but I didn't want to duplicate real
names. So I applied the Queneau assembly algorithm on the word level.
Here, the chunk is a run of letters that's all vowels or all
consonants. So "Carcassonne" would be split into the chunks ["C", "a",
"rc", "a", "ss", "o", "nn", e"]. I keep two sets of buckets, one for
vowels and one for consonants. If the first chunk was a vowel chunk,
the second chunk is a consonant chunk, and I alternate til I reach the
end.
This means that single-word BGD titles are almost never English words,
but they do capture the feel of those one-word titles that aren't
words (examples: "Zajekan", "Fraseda", "Kongin", "Q-blardo").
The BGD algorithm
Now that you see how it works, I'll explain the algorithm I
actually use for "Board Game Dadaist". Instead of three buckets, I
have a lot of numbered buckets. When I split a text into
T00, T01, ...,
T0M-1, I put T00 into
bucket 0, T01 into bucket 1, and so on, with
T0M-1 going into bucket M-1. I create an
assembly by picking from bucket 0, then bucket 1, and so on until I've
reached the target length.
This is the algorithm that "Hundred Thousand Billion Poems"
uses, and when the texts have more structure than "beginning/middle/end", this algorithm works a lot better. I don't think it matters much for BGD descriptions, but I do think it matters for game names. I would like to combine this algorithm with the "last" bucket from the simple algorithm, because right now board game descriptions sometimes end abruptly with a sentence like "Contents:".
Sun Aug 28 2011 22:15 You Can't Be Serious:
It's time for another big HTML table! This time I'm interested in movie connections. IMDB's dataset relates movies to each other using many different predicates: "edited into", "remake of", "alternate language version of", and so on. I'm interested in two of the most common predicates, "referenced in" and "spoofed in". Specifically, I want to answer these questions:
- What movies have been most influential on other movies?
- Are there movies that are commonly parodied, but almost never referenced in earnest? (And vice versa.)
I think my table speaks for itself, but I'll give a legend above it and a little commentary below it. The table has two columns:
- The most spoofed movies and TV shows (by number of "spoofed in" references)
- The movies and TV shows most referenced in earnest (by number of "referenced in" references)
The little numbers are the counts of "spoofed in" or "referenced in" references for that movie or TV show.
A title in bold shows up on only one list. This doesn't mean that, for instance, "The X-Files" has never been spoofed, only that it's not spoofed enough to make it onto the "most spoofed" list. A title in italics shows up on both lists (or would, if I extended the lists a little bit), but it's in a much higher position on the "spoof" list (left column) or the "non-spoof" list (right column). If a title is neither bolded nor italicized, then it's in approximately the same position on the "spoof" and "non-spoof" lists.
At this point I should probably let the table do the talking, so here it is. If you hate data, you can skip the table.
| Most Spoofed | Most Referenced Seriously |
---|
1 | Star Wars 279 | Star Wars 1793 |
2 | The Wizard of Oz 199 | The Wizard of Oz 1397 |
3 | "Star Trek" 180 | "Star Trek" 1270 |
4 | The Godfather 155 | The Godfather 737 |
5 | The Matrix 148 | Psycho 693 |
6 | 2001: A Space Odyssey 141 | Casablanca 622 |
7 | Psycho 139 | Star Wars: Episode V - The Empire Strikes Back 587 |
8 | Raiders of the Lost Ark 134 | Jaws 585 |
9 | Jaws 120 | "The Simpsons" 573 |
10 | Star Wars: Episode V - The Empire Strikes Back 119 | Gone with the Wind 534 |
11 | The Exorcist 96 | King Kong 527 |
12 | King Kong 96 | The Terminator 485 |
13 | "Batman" 95 | E.T.: The Extra-Terrestrial 449 |
14 | Pulp Fiction 93 | 2001: A Space Odyssey 448 |
15 | Titanic 93 | "Sesame Street" 448 |
16 | Superman 89 | Raiders of the Lost Ark 440 |
17 | E.T.: The Extra-Terrestrial 86 | Apocalypse Now 422 |
18 | Apocalypse Now 85 | Frankenstein 379 |
19 | The Shining 84 | The Exorcist 374 |
20 | "The Twilight Zone" 83 | "The Twilight Zone" 366 |
21 | The Terminator 81 | "Saturday Night Live" 363 |
22 | Casablanca 75 | Scarface 361 |
23 | Jurassic Park 74 | Citizen Kane 358 |
24 | Frankenstein 72 | Pulp Fiction 350 |
25 | Taxi Driver 71 | Titanic 350 |
26 | Rocky 68 | The Shining 348 |
27 | Alien 68 | "Doctor Who" 348 |
28 | The Silence of the Lambs 67 | Alien 346 |
29 | The Blair Witch Project 61 | "The Oprah Winfrey Show" 344 |
30 | Il buono, il brutto, il cattivo. 59 | Taxi Driver 343 |
31 | Terminator 2: Judgment Day 59 | Ghost Busters 343 |
32 | The Graduate 58 | "The Flintstones" 334 |
33 | Ghost Busters 57 | Rocky 333 |
34 | Gone with the Wind 56 | Star Wars: Episode VI - Return of the Jedi 328 |
35 | Star Wars: Episode VI - Return of the Jedi 56 | The Matrix 326 |
36 | It's a Wonderful Life 54 | Back to the Future 323 |
37 | Forrest Gump 52 | The Silence of the Lambs 323 |
38 | Goldfinger 52 | "Batman" 317 |
39 | Back to the Future 52 | A Clockwork Orange 306 |
40 | Dr. No 49 | Terminator 2: Judgment Day 302 |
41 | Star Wars: Episode I - The Phantom Menace 49 | "Happy Days" 301 |
42 | Gojira 48 | Snow White and the Seven Dwarfs 298 |
43 | Dr. Jekyll and Mr. Hyde 48 | The Sound of Music 293 |
44 | Scarface 47 | A Nightmare on Elm Street 289 |
45 | Planet of the Apes 47 | Superman 285 |
46 | "Cops" 47 | "Gilligan's Island" 284 |
47 | "Scooby Doo, Where Are You!" 47 | Dracula 282 |
48 | Snow White and the Seven Dwarfs 46 | "Star Trek: The Next Generation" 276 |
49 | Dracula 45 | "The X Files" 275 |
50 | The Sound of Music 44 | "The Brady Bunch" 271 |
51 | Reservoir Dogs 44 | Dr. No 270 |
52 | Batman 44 | "I Love Lucy" 269 |
53 | Citizen Kane 43 | First Blood 268 |
54 | Goodfellas 43 | Night of the Living Dead 262 |
55 | Night of the Living Dead 43 | "American Idol: The Search for a Superstar" 261 |
56 | Carrie 43 | Gojira 259 |
57 | "Jeopardy!" 43 | Jurassic Park 258 |
58 | Saturday Night Fever 42 | Dirty Harry 257 |
59 | The Texas Chain Saw Massacre 41 | The Texas Chain Saw Massacre 254 |
60 | "American Idol: The Search for a Superstar" 40 | Vertigo 253 |
61 | Mary Poppins 40 | It's a Wonderful Life 253 |
62 | Full Metal Jacket 40 | Aliens 243 |
63 | Dirty Harry 40 | Planet of the Apes 243 |
64 | The Lord of the Rings: The Fellowship of the Ring 39 | Batman 239 |
65 | The Karate Kid 38 | Il buono, il brutto, il cattivo. 236 |
66 | "The Brady Bunch" 38 | "Scooby Doo, Where Are You!" 231 |
67 | Friday the 13th 37 | The Graduate 229 |
68 | RoboCop 37 | Goldfinger 227 |
69 | Risky Business 37 | Deliverance 226 |
70 | "I Love Lucy" 36 | The Lord of the Rings: The Fellowship of the Ring 221 |
71 | Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb 35 | Blade Runner 220 |
72 | "Star Trek: The Next Generation" 35 | Die Hard 218 |
73 | "The Tonight Show Starring Johnny Carson" 34 | "Jeopardy!" 215 |
74 | Close Encounters of the Third Kind 34 | "Seinfeld" 215 |
75 | "Baywatch" 33 | Rosemary's Baby 214 |
76 | Scream 33 | Star Wars: Episode I - The Phantom Menace 213 |
77 | Flashdance 33 | The Lion King 211 |
78 | Lady and the Tramp 33 | Saturday Night Fever 210 |
79 | First Blood 33 | Mary Poppins 206 |
80 | "The Oprah Winfrey Show" 32 | Bambi 205 |
81 | Willy Wonka & the Chocolate Factory 32 | The Karate Kid 202 |
82 | The Lion King 32 | "Friends" 201 |
83 | "The Flintstones" 32 | "The Tonight Show Starring Johnny Carson" 200 |
84 | Indiana Jones and the Temple of Doom 32 | Halloween 199 |
85 | "24" 32 | Reservoir Dogs 199 |
86 | "Mission: Impossible" 32 | Top Gun 198 |
87 | The Seven Year Itch 32 | Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb 196 |
88 | Halloween 32 | "The Muppet Show" 194 |
89 | Spider-Man 31 | "Buffy the Vampire Slayer" 193 |
90 | Patton 31 | "The Andy Griffith Show" 192 |
91 | Rain Man 31 | Forrest Gump 188 |
92 | Thriller 31 | Dawn of the Dead 188 |
93 | Singin' in the Rain 30 | Friday the 13th 188 |
94 | Aliens 30 | Jerry Maguire 187 |
95 | A Clockwork Orange 30 | Close Encounters of the Third Kind 186 |
96 | Monty Python and the Holy Grail 30 | Singin' in the Rain 185 |
97 | Grease 30 | "Dancing with the Stars" 180 |
98 | Deliverance 30 | West Side Story 177 |
99 | Mission: Impossible 30 | "Baywatch" 176 |
100 | "The Sopranos" 30 | Grease 175 |
I'm not terribly happy with this data. I suspect many "referenced in" references are actually spoofs, or are throwaway jokes that don't even rise to the level of "spoof". Are there really 179 non-spoof references to "The Lion King"? You know everyone's just riffing on the baby-lifting shot.
However, the reverse problem ("incorrectly regarded as spoofs") is nonexistent, so it's easy to spot things like The Blair Witch Project and "Cops" which only exist in our culture as things to make fun of; as well as things that are occasionally referenced seriously but much more frequently spoofed (The Matrix).