What does being on MST3K do to a movie's IMDB rating? My best guess is that it knocks 2.9 stars off what what the rating would have been if the movie hadn't been on MST3K. But read on to see how I came up with that number, and why it depends on the director.
Disclaimer
I am not a statistician. I'm not even a data scientist. I know how to get data out of the Internet. I know the difference between mean, median, mode, and standard deviation. And that's about it.
Example
Here's an example in case you're not familiar with the MST3K-IMDB effect, which there's no reason you should since that's a name I just made up for it. Consider "Speech: The Function of Gestures", a short film directed by Arthur H. Wolf. It's got 5 votes and an IMDB rating of 5.2. Now here's another short film in the same series, "Speech: Platform Posture and Appearance". Same director, same writer, same lead actor, but this film had the misfortune to be double-billed with Red Zone Cuba on MST3K. As a result, it's got 98 votes and an IMDB rating of 1.6.
Call me skeptical, but I've watched both films and I'm not convinced there's really a three-and-a-half star difference between them. Another film in the series, "Speech: Using Your Voice", was also featured on MST3K, but in a less memorable episode ("Earth vs. the Spider"), and it struggles along with an IMDB rating of 2.4.)
Methodology Since the "Speech" films are part of a series, it makes sense to suppose that the difference between them is mostly due to the MST3K-IMDB effect. Of course, most films aren't part of a series. So I went by
director instead. I picked up the filmography of every director who
directed a film that was on MST3K. I split their films into two
lists, "Normal" (not featured on MST3K) and "MST" (featured on
MST3K). The "Normal" set only includes films that had enough IMDB
votes to be given a rating. I included shorts and episodes of TV
shows. This isn't perfect, because IMDB's plain-text data dump
sometimes (but not always) gives a director's credit where their
website gives a writer's credit. But it's close enough.
I took the average rating of the "Normal" list and the "MST"
list. The difference between the two averages is how much it
hurt that director to have one of their films featured on
MST3K. As we'll see see, some directors were hurt a lot, and some of
them shrugged it off, both for interesting reasons.
For the sake of comparison, the mean rating for a movie on IMDB at large is 6.4 stars, the median is 6.6 stars, and the standard deviation is 1.6 stars. However, "one star" is not a consistent unit of measurement. I'm considering redoing this table with normalized percentiles, but I'm not convinced there's a big demand for that, so for now you get stars.
Data Here's a big table with the data for every director who had at at least five films in the "Normal" set and at least one in the "MST" set. Normalm and Normalstd are the mean and standard deviation for the IMDB ratings of that director's non-MST films, and Normaln is the sample size. MSTm, MSTstd, and MSTn are the same thing for the director's MST film(s).
Effect1 is what we're looking for: for a given director, how many stars does a film lose just from being on MST3K? But wait! What if the director made some good stuff and some bad stuff, and only the bad stuff ended up on MST3K? The MST3K set would have lower ratings, but it wouldn't be because of MST3K. That's where Effect2 comes in, and that's why the table is sorted by Effect2. I'll explain Effect2 after you get a look at the data.
Now, for the explanation of Effect2. From Normalstd we know how likely this director is to make a film that's substantially better or worse than their average. If they made one bad film that was on MST3K, and there was no MST3K-IMDB effect for that director, the rating for that film would most likely be within two standard deviations of the director's average. But if there were a strong MST3K-IMDB effect for that director, the rating for the MSTed film would be much lower than the director's other bad films. So, Effect2 is: how many standard deviations below Normalm is MSTm?
Let's look at the extremes of the list. First, the directors with very low Effect2:
And this is the big thing I learned doing the project: you can
calculate the MST3K-IMDB effect, but you must also look at the
director's average movie rating to see what it means. A low
Effect2 just means that being on MST3K doesn't hurt a
director's ratings very much. It doesn't say anything about the movie's
quality.
OTOH, a director with a high Effect2 is probably worth a
second look in a non-MST3K context.
I'd couldn't even remember Invaders From The Deep being on
MST3K. Investigation reveals
it was on the very first non-pilot episode of MST3K. This episode is lost and no one's seen it for over
twenty years, so why is Effect2 so large? Could it
be that a bunch of MST3K fans gave this movie a one-star rating
without even seeing it on MST3K?
![[Comments]](/nb//themes/Default/comments.png "1 comment")
(1) Sun Aug 07 2011 11:57 The MST3K-IMDB Effect, Quantified:
Sometimes when I rewatch an MST3K episode I go to the movie's IMDB page to learn more about it. Inevitably I'm annoyed by the comments of people who give these movies one-star reviews solely on the basis of having watched an edited version on MST3K. But even greater than my annoyance is my desire to quantify the phenomenon. Today, I have quantified it.
Director
Normalm
Normalstd
Normaln
MSTm
MSTstd
MSTn
Effect1
Effect2
MSTed Films
Bava, Mario
6.2
0.9
22
6.3
-
1
-0.1
-0.1
Diabolik (1968)
Francisci, Pietro
5.0
0.9
9
4.8
0.7
2
0.2
0.3
Ercole e la regina di Lidia (1959), Le fatiche di Ercole (1958)
Tucker, Phil (I)
3.1
0.9
6
2.8
-
1
0.3
0.3
Robot Monster (1953)
Steckler, Ray Dennis
3.1
1.3
22
2.1
-
1
1.0
0.8
The Incredibly Strange Creatures Who Stopped Living and Became Mixed-Up Zombies!!? (1964)
Rebane, Bill
2.7
0.7
8
2.1
0.6
2
0.6
0.8
Monster a-Go Go (1965), The Giant Spider Invasion (1975)
Burke, Martyn
5.8
1.5
7
4.4
-
1
1.4
0.9
The Last Chase (1981)
Sachs, William
4.3
1.1
9
3.2
-
1
1.1
0.9
The Incredible Melting Man (1977)
Warren, Jerry
2.4
0.6
8
1.8
-
1
0.6
0.9
The Wild World of Batwoman (1966)
Maetzig, Kurt
5.4
1.3
13
4.0
-
1
1.4
1.1
Der schweigende Stern (1960)
Buchanan, Larry
3.3
1.2
25
2.0
-
1
1.3
1.1
The Eye Creatures (1965) (TV)
Yuasa, Noriaki
5.0
1.6
12
3.1
-
1
1.9
1.1
Gamera tai daiakuju Giron (1969)
Gordon, Bert I.
4.2
0.8
13
3.2
0.6
8
1.0
1.2
Beginning of the End (1957), Earth vs. the Spider (1958), King Dinosaur (1955), The Amazing Colossal Man (1957), The Magic Sword (1962), Tormented (1960), Village of the Giants (1965), War of the Colossal Beast (1958)
Brannon, Fred C.
5.8
1.3
43
4.2
-
1
1.6
1.2
Radar Men from the Moon (1952)
Mikels, Ted V.
3.3
1.3
19
1.8
-
1
1.5
1.2
Girl in Gold Boots (1968)
Wood Jr., Edward D.
4.0
1.0
15
2.8
0.8
2
1.2
1.2
Bride of the Monster (1955), The Sinister Urge (1960)
Zarindast, Tony
4.2
2.0
10
1.7
-
1
2.5
1.2
Werewolf (1996) (V)
Bradley, David (I)
4.9
1.9
6
2.4
-
1
2.5
1.3
12 to the Moon (1960)
Ludwig, Edward
6.1
1.0
33
4.7
-
1
1.4
1.3
The Black Scorpion (1957)
Clark, Greydon (I)
3.5
1.2
19
1.9
0.0
2
1.6
1.3
Angels' Brigade (1979), Final Justice (1985)
Franco, Jesus
4.1
1.2
166
2.5
-
1
1.6
1.3
The Castle of Fu Manchu (1969)
Eason, B. Reeves
5.8
0.9
46
4.5
-
1
1.3
1.4
Undersea Kingdom (1936)
Pyun, Albert
4.5
1.5
43
2.5
-
1
2.0
1.4
Alien from L.A. (1988)
Sturges, John
6.5
0.7
42
5.5
-
1
1.0
1.4
Marooned (1969)
Neumann, Kurt (I)
6.2
0.9
51
4.8
-
1
1.4
1.4
Rocketship X-M (1950)
Rou, Aleksandr
6.8
1.5
14
4.5
-
1
2.3
1.5
Morozko (1965)
Zens, Will
4.4
1.8
7
1.6
-
1
2.8
1.6
The Starfighters (1964)
Corman, Roger
5.4
1.4
44
3.2
0.8
6
2.1
1.6
Gunslinger (1956), It Conquered the World (1956), Swamp Women (1956), Teenage Cave Man (1958), The Saga of the Viking Women and Their Voyage to the Waters of the Great Sea Serpent (1957), The Undead (1957)
Fukuda, Jun (I)
5.8
1.3
10
3.6
-
1
2.2
1.7
Gojira tai Megaro (1973)
Piquer Simón, Juan
3.5
1.0
11
1.8
-
1
1.7
1.7
Los nuevos extraterrestres (1983)
Crichton, Charles
6.5
1.7
58
3.6
-
1
2.9
1.7
Cosmic Princess (1982) (TV)
Portillo, Rafael (I)
4.7
1.6
10
1.9
-
1
2.8
1.7
La momia azteca contra el robot humano (1958)
Beaudine, William
6.1
1.2
128
4.1
-
1
2.0
1.7
Design for Dreaming (1956)
Peshak, Ted
3.7
0.7
14
2.5
0.1
2
1.2
1.7
Appreciating Your Parents (1950), What to Do on a Date (1951)
Conway, James L. (I)
7.2
1.2
74
5.0
-
1
2.2
1.8
Hangar 18 (1980)
Worth, David (II)
4.3
1.3
20
2.0
-
1
2.3
1.8
Warrior of the Lost World (1983)
Grefe, William
4.3
1.5
11
1.7
-
1
2.6
1.8
Wild Rebels (1967)
Shonteff, Lindsay
4.5
0.9
19
3.0
0.6
2
1.6
1.8
Devil Doll (1964), The Million Eyes of Sumuru (1967)
Kane, Joseph (I)
6.2
0.9
127
4.5
-
1
1.7
1.8
Undersea Kingdom (1936)
Yarbrough, Jean
6.5
1.7
87
3.4
-
1
3.1
1.8
The Brute Man (1946)
Hessler, Gordon
5.7
1.2
48
3.4
-
1
2.3
1.9
"The Master" (1984)
Kessler, Bruce
6.5
1.6
67
3.4
-
1
3.1
1.9
"The Master" (1984)
Lawrence, Quentin
7.1
1.4
12
4.4
-
1
2.7
1.9
The Trollenberg Terror (1958)
Fox, Wallace
6.1
1.1
31
4.0
-
1
2.1
2.0
The Corpse Vanishes (1942)
Kincaid, Tim (I)
6.5
2.2
25
2.0
-
1
4.5
2.0
Robot Holocaust (1986) (V)
Malatesta, Guido
4.3
1.4
14
1.5
-
1
2.8
2.1
Maciste contro i cacciatori di teste (1963)
Dein, Edward
6.1
1.0
7
4.1
-
1
2.0
2.1
The Leech Woman (1960)
Juran, Nathan
6.4
1.0
53
4.2
-
1
2.2
2.1
The Deadly Mantis (1957)
Baldanello, Gianfranco
4.8
1.3
9
2.0
-
1
2.8
2.1
Il raggio infernale (1967)
Beebe, Ford
6.1
0.7
50
4.6
-
1
1.5
2.1
The Phantom Creeps (1939)
Winters, David (I)
5.5
1.7
17
1.8
-
1
3.7
2.1
Space Mutiny (1988)
Harvey, Herk
5.1
1.1
22
2.7
0.4
3
2.4
2.2
Cheating (1952), What About Juvenile Delinquency? (1955), Why Study Industrial Arts? (1956)
Medak, Peter
6.7
1.4
63
3.6
-
1
3.1
2.2
Cosmic Princess (1982) (TV)
Corona Blake, Alfonso
5.6
1.3
13
2.7
-
1
2.9
2.2
Santo vs. las mujeres vampiro (1962)
D'Amato, Joe
4.7
1.3
141
1.9
-
1
2.8
2.2
Ator l'invincibile 2 (1984)
Pierce, Charles B.
4.9
1.3
10
1.9
-
1
3.0
2.2
The Barbaric Beast of Boggy Creek, Part II (1985)
Rich, David Lowell
6.6
1.2
99
3.8
-
1
2.8
2.3
SST: Death Flight (1977) (TV)
Strock, Herbert L.
6.1
1.5
23
2.6
-
1
3.5
2.3
The Crawling Hand (1963)
Katzin, Lee H.
6.6
1.3
71
3.6
-
1
3.0
2.3
The Stranger (1973) (TV)
Ulmer, Edgar G.
5.7
1.0
38
3.4
-
1
2.3
2.3
The Amazing Transparent Man (1960)
Sloane, Rick
3.4
0.7
14
1.7
-
1
1.7
2.3
Hobgoblins (1988)
Cardos, John 'Bud'
5.1
1.4
9
1.8
-
1
3.3
2.4
Outlaw of Gor (1989)
Castellari, Enzo G.
5.5
1.1
39
2.9
-
1
2.6
2.4
Fuga dal Bronx (1983)
Mahon, Barry
4.4
1.3
34
1.3
-
1
3.1
2.4
Rocket Attack U.S.A. (1961)
Vogel, Virgil W.
7.1
1.1
141
4.4
-
1
2.7
2.5
The Mole People (1956)
Giancola, David
3.9
0.7
7
2.0
-
1
1.9
2.5
Tangents (1994)
Francis, Freddie
5.7
1.1
30
3.0
-
1
2.7
2.6
The Deadly Bees (1967)
Baker, Roy Ward
7.0
1.2
89
3.7
-
1
3.3
2.7
Moon Zero Two (1969)
Nicol, Alex (I)
5.6
1.0
9
2.9
-
1
2.7
2.7
The Screaming Skull (1958)
Sears, Fred F.
6.1
1.0
45
3.3
-
1
2.8
2.7
Teen-Age Crime Wave (1955)
Rakoff, Alvin
6.5
1.2
25
3.3
-
1
3.2
2.8
City on Fire (1979)
Lieberman, Jeff (I)
5.7
0.7
9
3.9
-
1
1.8
2.8
Squirm (1976)
Cahn, Edward L.
5.5
1.0
106
2.8
-
1
2.7
2.8
The She-Creature (1956)
Arnold, Jack (I)
6.7
1.0
109
3.9
1.2
2
2.8
2.8
Revenge of the Creature (1955), The Space Children (1958)
Heyes, Douglas
7.6
1.1
42
4.4
-
1
3.2
2.8
Kitten with a Whip (1964)
Ferroni, Giorgio
5.3
1.1
16
2.0
-
1
3.3
3.0
New York chiama Superdrago (1966)
Fowler Jr., Gene
6.8
1.1
9
3.4
1.1
2
3.4
3.0
I Was a Teenage Werewolf (1957), The Rebel Set (1959)
Bava, Lamberto
5.4
1.1
30
2.1
-
1
3.3
3.0
Shark: Rosso nell'oceano (1984)
Wolf, Arthur H.
4.6
0.8
8
2.0
0.4
2
2.6
3.1
Speech: Platform Posture and Appearance (1949), Speech: Using Your Voice (1950)
Jameson, Jerry
6.1
1.2
109
2.5
0.6
2
3.6
3.1
Superdome (1978) (TV), The Bat People (1974)
Koch, Howard W.
6.3
1.3
20
2.4
-
1
3.9
3.1
Untamed Youth (1957)
Morse, Hollingsworth
6.6
1.3
105
2.5
0.6
2
4.0
3.1
Crash of Moons (1954) (TV), Manhunt in Space (1956) (TV)
Cottafavi, Vittorio
6.2
0.8
17
3.6
-
1
2.6
3.2
Ercole alla conquista di Atlantide (1961)
Moxey, John Llewellyn
6.7
1.2
110
2.9
-
1
3.8
3.2
"San Francisco International Airport" (1970) {San Francisco International (#1.0)}
McLaglen, Andrew V.
6.5
1.3
215
2.3
-
1
4.2
3.2
Mitchell (1975)
Trikonis, Gus
5.8
1.3
72
1.7
-
1
4.1
3.2
Five the Hard Way (1969)
De Martino, Alberto (I)
5.2
0.9
25
2.2
0.2
2
3.0
3.3
L'uomo puma (1980), OK Connery (1967)
Cardona, René (I)
5.6
1.0
45
2.0
-
1
3.6
3.4
Santa Claus (1959)
Miner, Allen H.
7.6
1.5
25
2.4
-
1
5.2
3.4
The Days of Our Years (1955)
Newfield, Sam (I)
5.5
0.9
145
2.4
0.5
4
3.2
3.5
I Accuse My Parents (1944), Lost Continent (1951), Radar Secret Service (1950), The Mad Monster (1942)
Sholem, Lee
6.7
1.4
44
1.9
-
1
4.8
3.5
Catalina Caper (1967)
Turner, Ken (I)
7.3
1.5
7
2.2
-
1
5.1
3.5
Revenge of the Mysterons from Mars (1981) (TV)
Haas, Charles F.
6.8
1.2
31
2.6
-
1
4.2
3.6
Girls Town (1959)
Kowalski, Bernard L.
6.9
1.1
80
2.9
0.2
2
4.0
3.7
Attack of the Giant Leeches (1959), Night of the Blood Beast (1958)
Fukasaku, Kinji
7.1
0.9
53
3.8
-
1
3.3
3.8
The Green Slime (1968)
Gentilomo, Giacomo
5.6
0.8
11
2.4
-
1
3.2
3.8
Maciste e la regina di Samar (1964)
McDougall, Don
7.2
1.4
158
1.7
-
1
5.5
3.8
Riding with Death (1976) (TV)
Webster, Nicholas
6.8
1.1
13
2.3
-
1
4.5
3.9
Santa Claus Conquers the Martians (1964)
Austin, Ray (I)
6.6
1.1
135
2.1
-
1
4.5
3.9
"The Master" (1984) {Hostages (#1.4)}
Oswald, Gerd
6.7
1.2
52
1.8
-
1
4.9
4.1
Agent for H.A.R.M. (1966)
Lynn, Robert (II)
5.5
0.8
12
2.2
-
1
3.3
4.4
Revenge of the Mysterons from Mars (1981) (TV)
Szwarc, Jeannot
7.3
1.0
167
2.5
-
1
4.8
4.6
Code Name: Diamond Head (1977) (TV)
Myerson, Alan
7.0
1.1
101
2.1
-
1
4.9
4.6
"The Master" (1984) {State of the Union (#1.3)}
Lipstadt, Aaron
7.2
1.1
73
2.1
-
1
5.1
4.7
City Limits (1984)
Rondeau, Charles R.
7.0
1.0
58
2.2
-
1
4.8
4.7
The Girl in Lovers Lane (1960)
Collins, Lewis D.
6.0
0.9
56
1.7
-
1
4.3
4.9
Jungle Goddess (1948)
Green, Alfred E.
6.4
0.7
73
2.4
-
1
4.0
5.4
Invasion USA (1952)
Levi, Alan J.
6.9
0.9
122
1.7
-
1
5.2
5.7
Riding with Death (1976) (TV)
Lane, David (I)
7.2
0.8
19
2.1
-
1
5.1
6.3
Invaders from the Deep (1981)
Greidanus, Tjardus
6.3
0.7
13
1.7
-
1
4.6
6.3
The Final Sacrifice (1990)
Saunders, Desmond (I)
6.7
0.7
14
2.1
-
1
4.6
6.5
Invaders from the Deep (1981)
Ptushko, Aleksandr
7.2
0.4
8
4.2
1.0
3
2.9
6.9
Ilya Muromets (1956), Sadko (1953), Sampo (1959)
Williams, Douglas (I)
8.5
0.7
6
2.1
-
1
6.4
9.6
Overdrawn at the Memory Bank (1983) (TV)
Morgan, William (I)
6.2
0.4
10
2.6
-
1
3.6
10.1
The Violent Years (1956)
Elliott, David (II)
6.7
0.3
9
2.1
-
1
4.6
14.1
Invaders from the Deep (1981)
Average
2.9
And so on. The MST3K-IMDB effect is real--ninety percent of the directors in this table have an Effect2 of more than one standard deviation, and for sixty percent of them, it's more than two standard deviations. But it doesn't affect all directors equally.
Let's close out by taking a look at some of MST3K's favorite directors.
- Bert I. Gordon has a Normalm=4.2, which is pretty bad. But his record 8 MST3K movies have an MSTm of 3.2, which isn't much worse. Put one of his movies on MST3K and it loses only one star.
- Roger Corman, the B-movie king, has a pretty respectable (if you're Roger Corman) Normalm=5.4, but the same MSTm as Bert I. Gordon: 3.2.
- Ed Wood isn't especially well-known for being on MST3K but he does provide a pretty clear example of the MST3K-IMDB effect. His fifteen non-MST movies have a Normalm=4.0. His two MST movies have a MSTm of 2.8. Is The Sinister Urge (IMDB rating: 2.11) really the worst movie Ed Wood ever directed?
- Unfortunately, there is no way to calculate the MST3K-IMDB effect for the notorious Coleman Francis: since every film he directed was on MST3K, we can't calculate Normalm.
Conclusion
I'm still annoyed by those one-star reviews, but I understand them a little better now. When you watch, say, "The Function of Gestures", you enjoy it for its camp value, you have fun with it, and you give it a relatively good rating. But when you watch "Platform Posture and Appearance" or "Using Your Voice" on MST3K, you're watching someone else making fun of it, you have fun at its expense, and you give it a bad rating as a sign of solidarity with the MST3K characters.
Finally, I'd like to thank IMDB for, in a relic of its geeky past, making plain-text dumps of its data available. It's a strange feeling to have a file open in an Emacs buffer that lists nearly every movie ever made. (There are about 2 million, if you're curious.) Now that I have the data and scripts to process it, I may run other cinematic experiments in the future. One thing I would like to see added is IMDB links for the people and movies. It's a pain to look all these things up, which is why there aren't as many links in this post as you'd think.
- Comments:
Posted by Sumana at Sun Aug 07 2011 12:54
I love that you did this. Mushy mushy mush. And yay plaintext dumps!
