Is our desire to analyze leading us to oversimplify a complex game?
If a=b and b=c, then a=c.
That immutable truth your junior high math teacher taught you is unquestionable. It’s tidy. It’s neat. It’s linear. It’s easy to work with.
Basketball, on the other hand, is not easy to work with. Every moment of on-court action has so many variables in play at the same time, that a=b is hardly ever true in an absolute way, to say nothing of c or d or e or…
That doesn’t stop us from trying, though.
Why do we do jump to conclusions that way? Some of it is human nature. A lot of great writing[ref]Here are a couple of fantastic books that come to mind[/ref] has been done lately on the psychologically hard-wired need humans have to simplify information, even at the expense of validity. In other words, we’re searching so hard for answers that we’re looking right past them.
Analytics are good[ref]How’s that for linear?[/ref]… So is basketball knowledge
Basketball analysis is a field of thought that has taken off over the last decade, giving us dozens or even hundreds of ways to analyze trends. The overall impact of the movement is undoubtedly positive. We’re thinking about the game more carefully, we’ve got a new and more holistic way to think about player value, and we can quickly assess the outcomes associated with a variety of different decisions.
But at every step, we risking doing so in an overly simplistic way.
ESPN’s Chris Broussard spoke with two anonymous league executives about what could either be described as a growing emphasis on statistical analysis or a waning emphasis on basketball knowledge. The former isn’t a problem at all; the latter certainly is.
This conversation went all the predictable places[ref]”Analytics can’t measure heart!” Matt Harpring would say.[/ref], but it also articulated some really interesting truths about just how un-simple this game really is.
The more defensive/cynical of the two executives talked about people for whom a lifetime of studying the game from all angles represents an education – a Basketball Ph.D., if you will. He sounds borderline xenophobic about these shifty newcomers with their fancy spreadsheets who “haven’t been taught, trained or educated by any basketball professors or gone to any basketball classes.”
The other executive who chimes in on the piece is far more balanced, stating clearly that the new methods for extracting data are a positive development.
But be careful, he cautions. “Looking at any subject from one vantage point limits your ability to be flexible and achieve maximum results.” He hastens to point out that analysis of numbers have been a part of league and team culture forever, and that statistical analysis should “absolutely” be a part of the process.
The piece is an absolute must-read for those interested in the relationship between people the “basketball people” and their new peers.
Like most people who haven’t played meaningful basketball since their teenage years, I get a bit defensive when I hear the term “basketball people.” It connotes that unless you can legitimize yourself with a Fleer trading card, you can’t dissect the game.
I categorically disagree. Seen through that lens, would we consider Erik Spoelstra a “basketball person”? Spo never played professionally outside of a short stint in Germany, but learned the game inside and out as a video coordinator. I would argue he has a better understanding of motion, adjustments, countermoves and decision points than a lot of people who have played for several years. In fact, I think analyzing the game is a completely separate skill from playing the game, or coaching the game, or writing about the game, and so forth. Being good at one of those skill sets doesn’t mean you’re guaranteed to be good at another.
But in the same way, analyzing basketball and analyzing numbers are two skills that can either co-exist or not.
Let’s apply this to a corporate setting. Basketball teams are deemed successful or not based on winning & losing, corporations based on their financial results. So what would happen if a Board of Directors decide to remove all the executives from a company and turn the organization over to just the financial arm of the company?
You’d now have a bunch of accountants who understand one aspect of the company really well. They’d know what aspects of the company really help the bottom line and which parts don’t, and they could probably make the company very profitable in the short term. Over time, though, that might not be sustainable. They might not have the background on how different parts of the company interact. They might not know how to negotiate important deals, or take care of customers, or develop employees.[ref]Although we should also point out that they could learn these skills. A lot of Fortune 500 CEOs come from a finance background, just like a person who never played pro ball can learn how to break down a play, establish a system or coach a team.[/ref]
They would have all of the answers to some of the questions.
Asking the right questions
Last month, Former NBA coach Stan Van Gundy zinged a room full of statheads when he dismissed the value of tracking data that shows how far a player runs in a game. What can I do with that information as a coach?, he asked.
Thing is, Van Gundy was generally regarded as a very analytics-friendly coach, so let’s not push this aside as the cynic’s attempt to discredit the movement. His point is that something like miles covered is, at best, an interesting answer to a question nobody is asking.
Anayltics are only relevant if they answer questions that help players and teams get better. Otherwise, they’re just like the lifeguard who sees you drowning and stops to describe the water.
Jazz coach Ty Corbin similarly landed in stat geek prison when he famously said things like, “Not based on plus/minus!” in response to a question about playing time decisions, or more recently when he was asked about the value of threes and he retorted, “I like shots that go in.”
Van Gundy and Corbin might sound curmudgeony or, worse, behind the times. The fact is, they’re expressing what a lot of “basketball people” feel about some of the new data: that it’s not answering the right questions. What Corbin probably meant to say[ref]Although I have no idea why he gets so defensive when talking about metrics or PT decisions.[/ref] is that some of the things that make a shot qualitatively good or bad can’t be easily measured. As a directional tool for basketball decision-makers, analytics only help if they answer the right questions.
If I asked you who runs the pick & roll most effectively, you’d go consult Synergy and bring back an answer you felt relatively confident in. But some P&R possessions end with a player scoring off a cut, or an open shooter cashing in on the attention to the P&R, so the play gets logged that way. The question here (which team gets the most out of the P&R) doesn’t match the answer (which team gets the most specifically when the designated user of the possession was either the P&R screener or the handler). See how non-linear that is?[ref]Van Gundy’s main beef with play logging systems is that the play loggers often don’t know the team’s system so they focus on arbitrary definitions. For example, they said NYK uses isos 17% of the time, and SVG says that literally half of NYK’s plays start as an iso. We’re trusting a video intern to get it right, but not even a Spo-style video intern that knows the plays.[/ref]
Derrick Favors is another excellent example. For months we asked ourselves what was wrong with Favors’ defense because we looked at his D rating, his Synergy stats, his counterpart PER, etc. Then last week, he told us he’s still by far the #1 defender in the Jazz’s own proprietary system that rates defenders based on making the right decisions within the defensive system. Now we have to figure out the disconnect between doing the right things and the statistical outcomes represented in those other tools, but do you see how now we’re asking the right questions?
That’s why Van Gundy said, “There’s no substitute for watching film. Over and over and over… The analytics can be useful, but if you’re using that in place of watching the game, you’re making a big mistake.”[ref]Credit here to @basketballtalk and @ESPNNBA, who tweeted these quotes.[/ref]
Basketball is not linear
Statistical systems most valuable when they can tell us something directional. Analytics should be a powerful tool to describe trends, track progress, and reevaluate performance. But to capture every complex, layered, nuanced hypothetical within a basketball situation, you need a basketball brain, too.
If we spent half the time studying offensive & defensive systems on video that we spend resorting data and applying new filters, we’d understand much better what the numbers showed us.
(Of course, the inverse is also true: people who rely only on basketball pedigrees and don’t try to understand the relationship those actions have with outcomes are only seeing part of the picture.)
Saying “the Jazz are bad defensively” or “the Spurs’ offense is first-rate” doesn’t do much. Show me how the play developed, how the defense adjusted, how the Spurs recognized the moments of opportunity/choice. Show me the countermove, the reads, the whole flow.
Don’t tell me a player is bad at defending the screen. Show me how he’s bad, where he’s not blitzing hard enough, where he’s playing into the handler’s strengths. Then let’s watch the numbers and see how he improves over time.
A lot of people do that very well and are fluent both in X-and-O language and in terms of measurement. When you read a Zach Lowe piece (or even SCH’s own Ben Dowsett), you almost always see video or stills incorporated into the conversation. Yes, they’ll quantify the trends with numbers, but then they’ll show you what they mean.
They get what SVG gets: that basketball doesn’t always fit in a spreadsheet cell.
They get that a=b might be true, but there’s a lot more going on in every second of NBA basketball.