The sabermetrics of the fall’s new television series

It is somewhat hard to believe this, but we are already five weeks into the new fall season. Five weeks is forever in internet time, so this feels like a great week to rush to judgment on all the fall’s new series. Well, those series that are still alive. May you rest in peace, Playboy Club, How to be A Gentleman, Charlie’s Angels and Free Agents. And may you burn in hell, H8R.

This slate of programs isn’t overwhelmingly impressive, but it also isn’t completely dreadful. There are few bright spots, yet most of the flat-out duds have already been mercy killed anyway. So basically, we’re dealing with a lot of “meh” right now. Now that I’ve seen most of the season’s new fall series (we still have a few others debuting soonish), I’ve been thinking about how to organize them in an interesting way and I hope I found it.

You might not know this about me, but I’m a big sports fan. Like many (probably most?) sports fans, I’m starting to embrace advanced stats and metrics that tell us much more than we could have ever guessed about certain sports. I love Ken Pomroy’s college basketball stuff, I’m a big fan of the Football Outsiders and the DVOA rankings and I did my best to start to understand the 913 crucial metrics in baseball this season. As I was checking up on some random baseball player’s stats the other day (I think it was WAR [wins above replacement] and Ryan Braun, if you’re curious), I started thinking about applying some kind of metric to a television series, particularly a new series. What is the WAR of something like The Secret Circle? How can we determine if one series is more valuable than the other?

WELL, you’re in luck because I spent way too much time last night coming up with television’s own version of WAR that brings together various completely objective metrics.* I’m calling this VACSINY, or Value Above CSI: New York. I see CSI: New York as the most baseline, innocuous television series out there. It’s not terrible, it’s not great and it provides a solid service for CBS, the network that it’s on. No one would ever think that CSI: NY is a valuable or important program, but we can also recognize its purpose.* This is our starting point.

*This is completely untrue. Everything I’m doing here is subjective. You don’t care. You’re too impressed/depressed that I spent more than an hour figuring this out. I know. I KNOW.

Of course, now you’re asking an important question: Cory, how do you calculate VACSINY? Well, it’s pretty simple really. I’ve created five basic qualifiers that I use to evaluate television programming:

Entertainment Value: Simply put, is the series entertaining? Do I enjoy myself while watching it? This can swing both ways, however, as very terrible series tend to bring lots of entertainment to the table.

Clarity: Does the series’ make sense? Is it unnecessarily convoluted? Does the world feel at least somewhat formed?

Awareness: Good series know what they are, who their audience is and how those things impact scenes, episodes and seasons. Don’t take yourself too seriously or be too glib about matters.

Performances: All good series need good performances. Pretty easy to explain.

Production qualifiers (Writing, direction, set design, etc.): This is our miscellaneous production category that takes stock of the quality of writing, directing and anything else related to making a series.

What you do is take these five categories and then assign them numerical values 1-5, with 5 being the highest. The total amount of “points” a series in this regard is 25 (5 multiplied by 5 everyone!). This is step one. Step two asks us to divide that raw score by the series’ value or importance to a network’s schedule, brand and success. Included in this is whether or not a series is successful creatively and in its timeslot. We determine the the brand variable simply by assigning a numerical value between 1-10, this time with 1 being the highest and 10 being the lowest. Series that are super-valuable (your American Idols, your Modern Familys) obtain the very rare 1 score in this category.  The raw score divided by the brand variable figure gives us a series VACSINY. Just to make sure you’re not confused (and why would you be, this is so simple!), I’ll do a few examples for you:

Raw score = 12.5 (2.5+2.5+2.5+2.5+2.)
Brand variable = 5
Total VACSINY= 2.5*

*Just a note, added after the fact: The primary way in which this starts to get confusing in relationship to WAR is that the “replacement” and “average” designation. In WAR, replacement level players are said to be worth around 0, average players around 2. This process isn’t exactly the same. If this were a real, important statistical process, CSI: NY would be worth 0 or maybe 1, because the name is “above” CSI:NY and it therefore couldn’t have a value above itself. However, for the purposes of argument, I’m assuming that my baseline VACSINY is 2.5. I’m not a statistician and I understand the faults in that, but this is my fake system. Basically, the problem occurs because the name was too catchy to pass up but the stats don’t necessarily back that up. So if you’ve got a better name, let me know. 

Breaking Bad
Raw score = 25 (5+5+5+5+5)
Brand variable = 1.2
Total VACSINY =20.83

Modern Family
Raw score = 18 (3+3+4+4+4)
Brand variable = 1
Total VACSINY = 18

Easy, right? Let’s determine the VACSINY of all the new series that are still on the air. This list is presented in reverse chronological order, with the highest valued new series first and the lowest valued at the end.

New Girl
Raw score = 21.5 (4.5+5+3.5+4.5+4)
Brand variable = 1.4
Total VACSINY = 15.36

Raw score = 25 (5+5+5+5+5)
Brand variable = 2.4
Total VACSINY = 10.42

2 Broke Girls
Raw score = 19.5 (4+3.5+4+4.5+3.5)
Brand variable = 2.3
Total VACSINY = 8.48

Raw score = 22.5 (5+4.5+5+3.5+4.5)
Brand variable = 3.0
Total VACSINY = 7.5

Last Man Standing
Raw score = 10 (2+2+2+2+2)
Brand variable = 1.4
Total VACSINY = 7.14

Raw score = 21 (4.5+4+4+4.5+4)
Brand variable = 3.4
Total VACSINY = 6.17

Raw score = 11 (3.5+2+1+2.5+2)
Brand variable = 1.8
Total VACSINY = 6.11

Up All Night
Raw score = 19.5 (4+4+3.5+4+4)
Brand variable = 3.2
Total VACSINY = 6.09

Terra Nova
Raw score = 16 (4+3+2.5+3+3.5)
Brand variable = 2.7
Total VACSINY = 5.93

Hart of Dixie
Raw score = 20 (4+3.5+4.5+4+4)
Brand variable = 3.8
Total VACSINY = 5.26

Prime Suspect
Raw score = 19 (3+4+4+4+4)
Brand variable = 3.5
Total VACSINY = 5.43

Person of Interest
Raw score = 18 (4+3.5+3.5+3.5+3.5)
Brand variable = 3.6
Total VACSINY = 5

Raw score = 17 (3+3.5+4.5+3.5+2.5)
Brand variable = 3.5
Total VACSINY = 4.86

The Secret Circle
Raw score = 18 (3.5+3.5+4.5+3+3.5)
Brand variable = 3.7
Total VACSINY = 4.86

American Horror Story
Raw score = 9.5(5+0+.5+2+2)
Brand variable = 2.2
Total VACSINY = 4.31

Pan Am
Raw score = 20.5 (4+3.5+4+4+5)
Brand variable = 5.5
Total VACSINY = 3.7

Raw score = 8 (2+2+2+1+1)
Brand variable = 2.8
Total VACSINY = 3.2

Raw score = 12.5 (2.5+2.5+2.5+3+2)
Brand variable = 5.3
Total VACSINY = 2.36

A Gifted Man
Raw score = 15 (2.5+3+3+4+2.5)
Brand variable = 6.9
Total VACSINY = 2.17

Man Up!
Raw score = 12.5 (2.5+3+3+2+2)
Brand variable = 6.7
Total VACSINY = 1.87

Looking over the numbers – which again, I didn’t manipulate with subjectivity at all – it seems like the VACSINY cruncher tells us that New Girl, Homeland, 2 Broke Girls, Revenge and Last Man Standing are our most valuable new series. New Girl and Homeland are high-powered outliers and the other three fit more in track with the rest of the pack. One of the things this system seems to value is quality, which I think is fair. Series like American Horror Story or Last Man Standing might be somewhat high because of their brand variable, but their low raw scores knock them down dramatically. On the other hand, a rock-solid program like Pan Am takes a major hit because of its severely problematic ratings in what is a very valuable timeslot for ABC. Only three of the new series come in under the base level VACSINY of 2.5 (Enlightened, A Gifted Man, Man Up!). All the series that were already cancelled would have been right there, however.

Obviously, one of the big things to point out here is that these figures will change with more data. Series will lose or gain in the quality raw data department and the brand variable will change as well. This is a fluid metric that takes into account only what’s happened thus far. Also, you’ll notice that a few series that have lower Nielsen ratings end up higher using the VACSINY system. There is something like 10 million more people who watch Unforgettable than who watch Ringer. However, the latter has a much more successful brand variable figure because of its overall place within the CW brand/schedule and because ratings don’t matter as much over there. Future analysis will require the metrics to be further examined and complicated, perhaps with a “network modifier” included that takes into account each network and their specific priorities. I will head back to the TVS Lab and get to work on that. Look for more VACSINY figures in the future. Because I know you won’t be able to restrain yourself from asking for them.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: