Data Analysis Exonerates Rotten Tomatoes for Hollywood’s Failures

Last week, the New York Times published an article about Hollywood studio executives blaming the influence of Rotten Tomatoes for its failures at the box office. This seemed silly, and it was practically an admission that the movies these execs are making suck. Well, now we have data that shows the critical consensus on movies is not killing profits.

Yves Bergquist manages the Data & Analytics Project at USC’s Entertainment Technology Center and he’s helpfully looked over the data from Rotten Tomatoes over the last 17 years to give us some firm answers on the correlation between box office returns and fresh ratings.

First, Bergquist checked out how Rotten Tomatoes scores may have impacted theater attendance in 2017, the year these whiny executives are crying about. He pulled the critics and audience scores for the 150 movies released this year that made more than $1 million. Then he compared those scores to the sales receipts and used Product-Moment Correlation Coefficient (PMCC) analysis to figure out if there was a linear correlation to be found. There wasn’t.

Bergquist’s data showed that there was only a 12 percent PMCC correlation between good or bad ratings on Rotten Tomatoes and the amount of money Hollywood raked in. When he just looked at how a film performed on its ever-important opening weekend, that number dropped to 8 percent. Narrowing the field further to the summer season (May through Labor Day), the number fell to 7 percent.

More damning to Hollywood’s case that snobby professional critics are turning off the audience is the fact that critics have been fairly kind to the movies this year. From Bergquist’s post:

Overall, Rotten Tomatoes scores for all movies grossing more than $2 million worldwide have been pretty stable since 2000: the median score was 51 during the 2000s and 53 during the 2010s so far. It’s actually gone up quite significantly from 2015 (46.5) until 2017 (71).

So, not only are critical scores going up but the median box office is going up with it—$26.6 million in 2015 versus $66.1 million in 2017.

What’s most shocking about Bergquist’s data analysis is that audiences and professional critics are actually reporting nearly the same scores for movies. In 2017, there was an 87 percent correlation between the pros and general audience’s scores for films on Rotten Tomatoes. That’s up from 76 percent in 2013. There are three possibilities going on here: audiences’ level of reading a film is falling into line with critics; Rotten Tomatoes is including more critics that align with audience’s general taste; or both.

Based on this selection from the New York Times, the case for a wider professional critics pool seems pretty strong:

[Fandango president Paul] Yanover rejected [complaints that Rotten Tomatoes casts too wide of a critical net], pointing to the site’s posted requirements. (“Online critics must have published no less than 100 reviews across two calendar years at a single, Tomatometer-approved publication,” for instance.) He also noted that critics at traditional outlets tended to be white men and that Rotten Tomatoes wanted to include female and minority voices.

It’s possible that Rotten Tomatoes is bringing in less-snobby voices and that media-savvy audiences are raising their standards. Bergquist leans towards the latter in his interpretation of the data, he writes, “And I don’t care how old you are, you’re a film expert: how many hours of film and TV have you consumed since birth?”

So, if critics and audiences are actually responding pretty warmly to what Hollywood is producing, and median profits are up, what’s to blame for falling profits? The data points suggest that budgets are to blame. From the 1970s to the 2010s, there was a growing correlation between a film’s production budget and its box office returns. In 2014, that correlation hit 88 percent as opposed to 44 percent in the ‘70s. But that relationship has begun to decline, and in 2017 it dropped to 77 percent. This year, there were 30 more releases with budgets over $100 million than there were in 2016. Those 11 percentage points are a big deal when you’re dealing with billions of dollars.

The area in which audiences’ and professional critics’ scores are diverging the most actually comes in the area of production budgets. The correlation between a films budget and its audience score was 58 percent this year, as opposed to 77 percent from critics. The pro critics are being far kinder to the CGI-heavy blockbusters than audiences!

Bergquist suggests that the novelty of more realistic special effects is wearing off with audiences and promises more data soon about why old-fashioned storytelling could be the secret to future success. You can see the full report right here.

[Yves Bergquist via Polygon]