This is an article that, ostensibly, is introducing an attempt to quantify an element of basketball that we’ve talked about for a long time: an offense’s variability, or, the degree to which an offense runs many different kinds of plays. In reality, however, this article is a call for help from an armchair blogger who feels like he stumbled on an idea that he doesn’t quite have the tools to tackle correctly.
It’s been long suggested that a more variable offense is a better offense. If a team runs more and more varied plays, that’s more work for the defense, and the offense will be better as a result, right?
It turns out that that’s not actually the case. Despite conventional wisdom, a team that runs post ups almost as often as it runs pick-and-rolls almost as often as it runs isolation plays is not going to be as effective as an offense that relies more strictly on plays that results with a spot-up shot, or pick-and-rolls.
This may seem, for a moment and to some, counter-intuitive, but let me explain. Some plays are just inherently more efficient than others. Pick-and-rolls and spot-up shots result in better results than post-ups and isolations on average, for example.
Similarly, some teams are just better at some types of plays than others, and every team is likely to have a weakness. So, teams that are running every different kind of play equally are likely spending an inordinate amount of time on play types that they are ill suited for.
Teams that know their strengths and play to them, or who bias more efficient play types, or both, are, on average, going to be better off than teams who are trying a little bit of everything.
To demonstrate, I attempted to quantify a team’s “offensive variability” by finding the variance in how often each team finished a possession with a certain play type, per mySynergySports.com.
I defined a team’s offensive variability as being the degree to which a team uses each play the same amount. So, a lower variance would result in a greater offensive variability, because it would mean more uniformity among the different usages of different play types.
I weighted (very lightly) each team’s percentage of possessions used on a given play by the percentile of each team’s success on those plays, so that a team may not get credit for being “variable” if they ran hand-off plays 4% of the time (around average) but were worst in the league in them, since those plays would not have impacted the team’s play very much, if at all.
After all of that, I subtracted the variance from 1, so that the variability value would increase relative to greater team flexibility. The higher the variability, the more the offense uses different play types.
There are a few caveats to be taken here: first and foremost, mySynergySports data is limited on the degree to which “play type” is broken down. Spot-Up Shots, for example, are often created as the result of a greater play; sometimes a post-up, sometimes a pick-and-roll. I’d be willing to bet that a greater variety of a team’s playbook might result in more spot-up shots (though, more on that later).
As well, the pick-and-roll category could be broken down into a billion different types of plays, but instead only “pick-and-roll ball handler” and “pick-and-roll big man” are available. Still, this data gives a good indication how much a team is relying on different types of players and plays to do different things. A team with greater variability will have the scoring load more distributed among all the players, for example, with all those players playing many roles.
The second caveat is that I only have access to one season’s worth of mySynergySports play-by-play data for all teams, which limits the findings to a sample size of only 30 data points, meaning there’s a potential error of about .019. That error could, theoretically, seriously damage the findings.
Nonetheless, here were the results of my variability calculations, in a chart that includes all the data that I correlated the variability with:
Here, too, are the results of the correlation between offensive variability and offensive efficiency. The correlation is solid — if certainly not a 1:1 relationship — and decidedly negative.
Given the clear correlation between offensive efficiency and variability, most of the other expected correlations exist. There’s a weaker negative correlation between variability and Win% (weaker because defense is part of winning, of course), and there’s a negative relationship between variability and three point shooting (since greater variability also generally means less spot-up shooting).
Perhaps most interesting of these findings, however, is that there was functionally no correlation between the variability of an offense and how well that offense performed against top 5 defenses. I never expected the correlation to be large, but an analysis I did for the AnalyticsGame blog indicated that having a flexible offense might have a greater impact against defensive teams who function by forcing an offense to do one thing or another.
Of course, it’s worth pointing out that an offense’s variability is not necessarily the same thing as its flexibility; i.e a team may have relative uniformity in plays they run, but also have many different players with different skillsets who can get the most out of the few plays the team runs. That team may be flexible, but not variable.
This, in and of itself, begs some further analysis. If we can define variability by the different plays a team can run, is there analysis that can be done on flexibility? The answer, I’m sure, is yes (especially with the awesome introduction of teamSPACE on this site).
All of that said, as with any such analysis, the primary question is causality. If variability and efficiency are inversely correlated, are teams less efficient because they are more variable? Or are teams more variable on average because they are worse, e.g, because they have players who can’t fill defined roles, or because there’s nothing that the team is especially good at?
That’s not an easy question to answer, and to be honest, I’m not going to attempt to.
This all brings me back to my call for help: there are quite a few potential lessons to be had, here, but I’m limited on my ability to learn them.
So, for example, to the people who know how to make a causal inference, is there a way to know whether variability hurts an offense or vice versa?
Similarly, if someone has access to the data, is it the case that plays broken down on a more micro level (e.g, a team with more plays like “Horns,” or “elevator doors,” and their variations) leads to a better offense, or no? I would imagine that more of those plays would be reflected in a greater percentage of spot-up attempts or pick-and-roll attempts per game, and there’s no correlation between either and a better offense1, but this is clearly a place that’s ripe for further analysis.
Seth Partnow wrote beautifully on this site during its opening that micro-analyses of play running and player tendencies was going to always be more useful and relevant to basketball analysis until we could find a way to make those macro-analyses relevant to those who have to act on the findings.
The subject of variability is not such a magical link between the two levels of analysis, but it is a step forward in that direction, and personally, I’d like to see it move even closer, with some help.