<D <M <Y
Y> M> D>

[Comments] (1) The MST3K-IMDB Effect, Quantified: Sometimes when I rewatch an MST3K episode I go to the movie's IMDB page to learn more about it. Inevitably I'm annoyed by the comments of people who give these movies one-star reviews solely on the basis of having watched an edited version on MST3K. But even greater than my annoyance is my desire to quantify the phenomenon. Today, I have quantified it.

What does being on MST3K do to a movie's IMDB rating? My best guess is that it knocks 2.9 stars off what what the rating would have been if the movie hadn't been on MST3K. But read on to see how I came up with that number, and why it depends on the director.

Disclaimer

I am not a statistician. I'm not even a data scientist. I know how to get data out of the Internet. I know the difference between mean, median, mode, and standard deviation. And that's about it.

Example

Here's an example in case you're not familiar with the MST3K-IMDB effect, which there's no reason you should since that's a name I just made up for it. Consider "Speech: The Function of Gestures", a short film directed by Arthur H. Wolf. It's got 5 votes and an IMDB rating of 5.2. Now here's another short film in the same series, "Speech: Platform Posture and Appearance". Same director, same writer, same lead actor, but this film had the misfortune to be double-billed with Red Zone Cuba on MST3K. As a result, it's got 98 votes and an IMDB rating of 1.6.

Call me skeptical, but I've watched both films and I'm not convinced there's really a three-and-a-half star difference between them. Another film in the series, "Speech: Using Your Voice", was also featured on MST3K, but in a less memorable episode ("Earth vs. the Spider"), and it struggles along with an IMDB rating of 2.4.)

Methodology

Since the "Speech" films are part of a series, it makes sense to suppose that the difference between them is mostly due to the MST3K-IMDB effect. Of course, most films aren't part of a series. So I went by director instead. I picked up the filmography of every director who directed a film that was on MST3K. I split their films into two lists, "Normal" (not featured on MST3K) and "MST" (featured on MST3K). The "Normal" set only includes films that had enough IMDB votes to be given a rating. I included shorts and episodes of TV shows. This isn't perfect, because IMDB's plain-text data dump sometimes (but not always) gives a director's credit where their website gives a writer's credit. But it's close enough.

I took the average rating of the "Normal" list and the "MST" list. The difference between the two averages is how much it hurt that director to have one of their films featured on MST3K. As we'll see see, some directors were hurt a lot, and some of them shrugged it off, both for interesting reasons.

For the sake of comparison, the mean rating for a movie on IMDB at large is 6.4 stars, the median is 6.6 stars, and the standard deviation is 1.6 stars. However, "one star" is not a consistent unit of measurement. I'm considering redoing this table with normalized percentiles, but I'm not convinced there's a big demand for that, so for now you get stars.

Data

Here's a big table with the data for every director who had at at least five films in the "Normal" set and at least one in the "MST" set. Normalm and Normalstd are the mean and standard deviation for the IMDB ratings of that director's non-MST films, and Normaln is the sample size. MSTm, MSTstd, and MSTn are the same thing for the director's MST film(s).

Effect1 is what we're looking for: for a given director, how many stars does a film lose just from being on MST3K? But wait! What if the director made some good stuff and some bad stuff, and only the bad stuff ended up on MST3K? The MST3K set would have lower ratings, but it wouldn't be because of MST3K. That's where Effect2 comes in, and that's why the table is sorted by Effect2. I'll explain Effect2 after you get a look at the data.

Click here to skip the table.

Director Normalm Normalstd Normaln MSTm MSTstd MSTn Effect1 Effect2 MSTed Films
Bava, Mario 6.2 0.9 22 6.3 - 1 -0.1 -0.1 Diabolik (1968)
Francisci, Pietro 5.0 0.9 9 4.8 0.7 2 0.2 0.3 Ercole e la regina di Lidia (1959), Le fatiche di Ercole (1958)
Tucker, Phil (I) 3.1 0.9 6 2.8 - 1 0.3 0.3 Robot Monster (1953)
Steckler, Ray Dennis 3.1 1.3 22 2.1 - 1 1.0 0.8 The Incredibly Strange Creatures Who Stopped Living and Became Mixed-Up Zombies!!? (1964)
Rebane, Bill 2.7 0.7 8 2.1 0.6 2 0.6 0.8 Monster a-Go Go (1965), The Giant Spider Invasion (1975)
Burke, Martyn 5.8 1.5 7 4.4 - 1 1.4 0.9 The Last Chase (1981)
Sachs, William 4.3 1.1 9 3.2 - 1 1.1 0.9 The Incredible Melting Man (1977)
Warren, Jerry 2.4 0.6 8 1.8 - 1 0.6 0.9 The Wild World of Batwoman (1966)
Maetzig, Kurt 5.4 1.3 13 4.0 - 1 1.4 1.1 Der schweigende Stern (1960)
Buchanan, Larry 3.3 1.2 25 2.0 - 1 1.3 1.1 The Eye Creatures (1965) (TV)
Yuasa, Noriaki 5.0 1.6 12 3.1 - 1 1.9 1.1 Gamera tai daiakuju Giron (1969)
Gordon, Bert I. 4.2 0.8 13 3.2 0.6 8 1.0 1.2 Beginning of the End (1957), Earth vs. the Spider (1958), King Dinosaur (1955), The Amazing Colossal Man (1957), The Magic Sword (1962), Tormented (1960), Village of the Giants (1965), War of the Colossal Beast (1958)
Brannon, Fred C. 5.8 1.3 43 4.2 - 1 1.6 1.2 Radar Men from the Moon (1952)
Mikels, Ted V. 3.3 1.3 19 1.8 - 1 1.5 1.2 Girl in Gold Boots (1968)
Wood Jr., Edward D. 4.0 1.0 15 2.8 0.8 2 1.2 1.2 Bride of the Monster (1955), The Sinister Urge (1960)
Zarindast, Tony 4.2 2.0 10 1.7 - 1 2.5 1.2 Werewolf (1996) (V)
Bradley, David (I) 4.9 1.9 6 2.4 - 1 2.5 1.3 12 to the Moon (1960)
Ludwig, Edward 6.1 1.0 33 4.7 - 1 1.4 1.3 The Black Scorpion (1957)
Clark, Greydon (I) 3.5 1.2 19 1.9 0.0 2 1.6 1.3 Angels' Brigade (1979), Final Justice (1985)
Franco, Jesus 4.1 1.2 166 2.5 - 1 1.6 1.3 The Castle of Fu Manchu (1969)
Eason, B. Reeves 5.8 0.9 46 4.5 - 1 1.3 1.4 Undersea Kingdom (1936)
Pyun, Albert 4.5 1.5 43 2.5 - 1 2.0 1.4 Alien from L.A. (1988)
Sturges, John 6.5 0.7 42 5.5 - 1 1.0 1.4 Marooned (1969)
Neumann, Kurt (I) 6.2 0.9 51 4.8 - 1 1.4 1.4 Rocketship X-M (1950)
Rou, Aleksandr 6.8 1.5 14 4.5 - 1 2.3 1.5 Morozko (1965)
Zens, Will 4.4 1.8 7 1.6 - 1 2.8 1.6 The Starfighters (1964)
Corman, Roger 5.4 1.4 44 3.2 0.8 6 2.1 1.6 Gunslinger (1956), It Conquered the World (1956), Swamp Women (1956), Teenage Cave Man (1958), The Saga of the Viking Women and Their Voyage to the Waters of the Great Sea Serpent (1957), The Undead (1957)
Fukuda, Jun (I) 5.8 1.3 10 3.6 - 1 2.2 1.7 Gojira tai Megaro (1973)
Piquer Simón, Juan 3.5 1.0 11 1.8 - 1 1.7 1.7 Los nuevos extraterrestres (1983)
Crichton, Charles 6.5 1.7 58 3.6 - 1 2.9 1.7 Cosmic Princess (1982) (TV)
Portillo, Rafael (I) 4.7 1.6 10 1.9 - 1 2.8 1.7 La momia azteca contra el robot humano (1958)
Beaudine, William 6.1 1.2 128 4.1 - 1 2.0 1.7 Design for Dreaming (1956)
Peshak, Ted 3.7 0.7 14 2.5 0.1 2 1.2 1.7 Appreciating Your Parents (1950), What to Do on a Date (1951)
Conway, James L. (I) 7.2 1.2 74 5.0 - 1 2.2 1.8 Hangar 18 (1980)
Worth, David (II) 4.3 1.3 20 2.0 - 1 2.3 1.8 Warrior of the Lost World (1983)
Grefe, William 4.3 1.5 11 1.7 - 1 2.6 1.8 Wild Rebels (1967)
Shonteff, Lindsay 4.5 0.9 19 3.0 0.6 2 1.6 1.8 Devil Doll (1964), The Million Eyes of Sumuru (1967)
Kane, Joseph (I) 6.2 0.9 127 4.5 - 1 1.7 1.8 Undersea Kingdom (1936)
Yarbrough, Jean 6.5 1.7 87 3.4 - 1 3.1 1.8 The Brute Man (1946)
Hessler, Gordon 5.7 1.2 48 3.4 - 1 2.3 1.9 "The Master" (1984)
Kessler, Bruce 6.5 1.6 67 3.4 - 1 3.1 1.9 "The Master" (1984)
Lawrence, Quentin 7.1 1.4 12 4.4 - 1 2.7 1.9 The Trollenberg Terror (1958)
Fox, Wallace 6.1 1.1 31 4.0 - 1 2.1 2.0 The Corpse Vanishes (1942)
Kincaid, Tim (I) 6.5 2.2 25 2.0 - 1 4.5 2.0 Robot Holocaust (1986) (V)
Malatesta, Guido 4.3 1.4 14 1.5 - 1 2.8 2.1 Maciste contro i cacciatori di teste (1963)
Dein, Edward 6.1 1.0 7 4.1 - 1 2.0 2.1 The Leech Woman (1960)
Juran, Nathan 6.4 1.0 53 4.2 - 1 2.2 2.1 The Deadly Mantis (1957)
Baldanello, Gianfranco 4.8 1.3 9 2.0 - 1 2.8 2.1 Il raggio infernale (1967)
Beebe, Ford 6.1 0.7 50 4.6 - 1 1.5 2.1 The Phantom Creeps (1939)
Winters, David (I) 5.5 1.7 17 1.8 - 1 3.7 2.1 Space Mutiny (1988)
Harvey, Herk 5.1 1.1 22 2.7 0.4 3 2.4 2.2 Cheating (1952), What About Juvenile Delinquency? (1955), Why Study Industrial Arts? (1956)
Medak, Peter 6.7 1.4 63 3.6 - 1 3.1 2.2 Cosmic Princess (1982) (TV)
Corona Blake, Alfonso 5.6 1.3 13 2.7 - 1 2.9 2.2 Santo vs. las mujeres vampiro (1962)
D'Amato, Joe 4.7 1.3 141 1.9 - 1 2.8 2.2 Ator l'invincibile 2 (1984)
Pierce, Charles B. 4.9 1.3 10 1.9 - 1 3.0 2.2 The Barbaric Beast of Boggy Creek, Part II (1985)
Rich, David Lowell 6.6 1.2 99 3.8 - 1 2.8 2.3 SST: Death Flight (1977) (TV)
Strock, Herbert L. 6.1 1.5 23 2.6 - 1 3.5 2.3 The Crawling Hand (1963)
Katzin, Lee H. 6.6 1.3 71 3.6 - 1 3.0 2.3 The Stranger (1973) (TV)
Ulmer, Edgar G. 5.7 1.0 38 3.4 - 1 2.3 2.3 The Amazing Transparent Man (1960)
Sloane, Rick 3.4 0.7 14 1.7 - 1 1.7 2.3 Hobgoblins (1988)
Cardos, John 'Bud' 5.1 1.4 9 1.8 - 1 3.3 2.4 Outlaw of Gor (1989)
Castellari, Enzo G. 5.5 1.1 39 2.9 - 1 2.6 2.4 Fuga dal Bronx (1983)
Mahon, Barry 4.4 1.3 34 1.3 - 1 3.1 2.4 Rocket Attack U.S.A. (1961)
Vogel, Virgil W. 7.1 1.1 141 4.4 - 1 2.7 2.5 The Mole People (1956)
Giancola, David 3.9 0.7 7 2.0 - 1 1.9 2.5 Tangents (1994)
Francis, Freddie 5.7 1.1 30 3.0 - 1 2.7 2.6 The Deadly Bees (1967)
Baker, Roy Ward 7.0 1.2 89 3.7 - 1 3.3 2.7 Moon Zero Two (1969)
Nicol, Alex (I) 5.6 1.0 9 2.9 - 1 2.7 2.7 The Screaming Skull (1958)
Sears, Fred F. 6.1 1.0 45 3.3 - 1 2.8 2.7 Teen-Age Crime Wave (1955)
Rakoff, Alvin 6.5 1.2 25 3.3 - 1 3.2 2.8 City on Fire (1979)
Lieberman, Jeff (I) 5.7 0.7 9 3.9 - 1 1.8 2.8 Squirm (1976)
Cahn, Edward L. 5.5 1.0 106 2.8 - 1 2.7 2.8 The She-Creature (1956)
Arnold, Jack (I) 6.7 1.0 109 3.9 1.2 2 2.8 2.8 Revenge of the Creature (1955), The Space Children (1958)
Heyes, Douglas 7.6 1.1 42 4.4 - 1 3.2 2.8 Kitten with a Whip (1964)
Ferroni, Giorgio 5.3 1.1 16 2.0 - 1 3.3 3.0 New York chiama Superdrago (1966)
Fowler Jr., Gene 6.8 1.1 9 3.4 1.1 2 3.4 3.0 I Was a Teenage Werewolf (1957), The Rebel Set (1959)
Bava, Lamberto 5.4 1.1 30 2.1 - 1 3.3 3.0 Shark: Rosso nell'oceano (1984)
Wolf, Arthur H. 4.6 0.8 8 2.0 0.4 2 2.6 3.1 Speech: Platform Posture and Appearance (1949), Speech: Using Your Voice (1950)
Jameson, Jerry 6.1 1.2 109 2.5 0.6 2 3.6 3.1 Superdome (1978) (TV), The Bat People (1974)
Koch, Howard W. 6.3 1.3 20 2.4 - 1 3.9 3.1 Untamed Youth (1957)
Morse, Hollingsworth 6.6 1.3 105 2.5 0.6 2 4.0 3.1 Crash of Moons (1954) (TV), Manhunt in Space (1956) (TV)
Cottafavi, Vittorio 6.2 0.8 17 3.6 - 1 2.6 3.2 Ercole alla conquista di Atlantide (1961)
Moxey, John Llewellyn 6.7 1.2 110 2.9 - 1 3.8 3.2 "San Francisco International Airport" (1970) {San Francisco International (#1.0)}
McLaglen, Andrew V. 6.5 1.3 215 2.3 - 1 4.2 3.2 Mitchell (1975)
Trikonis, Gus 5.8 1.3 72 1.7 - 1 4.1 3.2 Five the Hard Way (1969)
De Martino, Alberto (I) 5.2 0.9 25 2.2 0.2 2 3.0 3.3 L'uomo puma (1980), OK Connery (1967)
Cardona, René (I) 5.6 1.0 45 2.0 - 1 3.6 3.4 Santa Claus (1959)
Miner, Allen H. 7.6 1.5 25 2.4 - 1 5.2 3.4 The Days of Our Years (1955)
Newfield, Sam (I) 5.5 0.9 145 2.4 0.5 4 3.2 3.5 I Accuse My Parents (1944), Lost Continent (1951), Radar Secret Service (1950), The Mad Monster (1942)
Sholem, Lee 6.7 1.4 44 1.9 - 1 4.8 3.5 Catalina Caper (1967)
Turner, Ken (I) 7.3 1.5 7 2.2 - 1 5.1 3.5 Revenge of the Mysterons from Mars (1981) (TV)
Haas, Charles F. 6.8 1.2 31 2.6 - 1 4.2 3.6 Girls Town (1959)
Kowalski, Bernard L. 6.9 1.1 80 2.9 0.2 2 4.0 3.7 Attack of the Giant Leeches (1959), Night of the Blood Beast (1958)
Fukasaku, Kinji 7.1 0.9 53 3.8 - 1 3.3 3.8 The Green Slime (1968)
Gentilomo, Giacomo 5.6 0.8 11 2.4 - 1 3.2 3.8 Maciste e la regina di Samar (1964)
McDougall, Don 7.2 1.4 158 1.7 - 1 5.5 3.8 Riding with Death (1976) (TV)
Webster, Nicholas 6.8 1.1 13 2.3 - 1 4.5 3.9 Santa Claus Conquers the Martians (1964)
Austin, Ray (I) 6.6 1.1 135 2.1 - 1 4.5 3.9 "The Master" (1984) {Hostages (#1.4)}
Oswald, Gerd 6.7 1.2 52 1.8 - 1 4.9 4.1 Agent for H.A.R.M. (1966)
Lynn, Robert (II) 5.5 0.8 12 2.2 - 1 3.3 4.4 Revenge of the Mysterons from Mars (1981) (TV)
Szwarc, Jeannot 7.3 1.0 167 2.5 - 1 4.8 4.6 Code Name: Diamond Head (1977) (TV)
Myerson, Alan 7.0 1.1 101 2.1 - 1 4.9 4.6 "The Master" (1984) {State of the Union (#1.3)}
Lipstadt, Aaron 7.2 1.1 73 2.1 - 1 5.1 4.7 City Limits (1984)
Rondeau, Charles R. 7.0 1.0 58 2.2 - 1 4.8 4.7 The Girl in Lovers Lane (1960)
Collins, Lewis D. 6.0 0.9 56 1.7 - 1 4.3 4.9 Jungle Goddess (1948)
Green, Alfred E. 6.4 0.7 73 2.4 - 1 4.0 5.4 Invasion USA (1952)
Levi, Alan J. 6.9 0.9 122 1.7 - 1 5.2 5.7 Riding with Death (1976) (TV)
Lane, David (I) 7.2 0.8 19 2.1 - 1 5.1 6.3 Invaders from the Deep (1981)
Greidanus, Tjardus 6.3 0.7 13 1.7 - 1 4.6 6.3 The Final Sacrifice (1990)
Saunders, Desmond (I) 6.7 0.7 14 2.1 - 1 4.6 6.5 Invaders from the Deep (1981)
Ptushko, Aleksandr 7.2 0.4 8 4.2 1.0 3 2.9 6.9 Ilya Muromets (1956), Sadko (1953), Sampo (1959)
Williams, Douglas (I) 8.5 0.7 6 2.1 - 1 6.4 9.6 Overdrawn at the Memory Bank (1983) (TV)
Morgan, William (I) 6.2 0.4 10 2.6 - 1 3.6 10.1 The Violent Years (1956)
Elliott, David (II) 6.7 0.3 9 2.1 - 1 4.6 14.1 Invaders from the Deep (1981)
Average 2.9

Analysis

Now, for the explanation of Effect2. From Normalstd we know how likely this director is to make a film that's substantially better or worse than their average. If they made one bad film that was on MST3K, and there was no MST3K-IMDB effect for that director, the rating for that film would most likely be within two standard deviations of the director's average. But if there were a strong MST3K-IMDB effect for that director, the rating for the MSTed film would be much lower than the director's other bad films. So, Effect2 is: how many standard deviations below Normalm is MSTm?

Let's look at the extremes of the list. First, the directors with very low Effect2:

And this is the big thing I learned doing the project: you can calculate the MST3K-IMDB effect, but you must also look at the director's average movie rating to see what it means. A low Effect2 just means that being on MST3K doesn't hurt a director's ratings very much. It doesn't say anything about the movie's quality.

OTOH, a director with a high Effect2 is probably worth a second look in a non-MST3K context.

And so on. The MST3K-IMDB effect is real--ninety percent of the directors in this table have an Effect2 of more than one standard deviation, and for sixty percent of them, it's more than two standard deviations. But it doesn't affect all directors equally.

Let's close out by taking a look at some of MST3K's favorite directors.

Conclusion

I'm still annoyed by those one-star reviews, but I understand them a little better now. When you watch, say, "The Function of Gestures", you enjoy it for its camp value, you have fun with it, and you give it a relatively good rating. But when you watch "Platform Posture and Appearance" or "Using Your Voice" on MST3K, you're watching someone else making fun of it, you have fun at its expense, and you give it a bad rating as a sign of solidarity with the MST3K characters.

Finally, I'd like to thank IMDB for, in a relic of its geeky past, making plain-text dumps of its data available. It's a strange feeling to have a file open in an Emacs buffer that lists nearly every movie ever made. (There are about 2 million, if you're curious.) Now that I have the data and scripts to process it, I may run other cinematic experiments in the future. One thing I would like to see added is IMDB links for the people and movies. It's a pain to look all these things up, which is why there aren't as many links in this post as you'd think.


[Main]

Unless otherwise noted, all content licensed by Leonard Richardson
under a Creative Commons License.