As you may have seen, Luke Bornn set Twitter on fire yesterday (to the tune of nearly 500 RTs) re-posting something that Sam Ventura mentioned previously on why radar charts are bad.
— Luke Bornn (@LukeBornn) May 17, 2017
Obviously, a lot of eyes turned toward me, since it is probably my fault they exist at all in soccer/football, and possibly my fault they have crept into other sports. Daryl Morey then managed to do a drive-by on my career so far, posting this tweet
— Daryl Morey (@dmorey) May 17, 2017
Which I don’t think was calling my entire analytics career into question, but could be interpreted as such. THANKS DARYL. I am pleased to note that at least I don't use pie charts or 2 Y axes. Anyway, none of this is personal to me and please don’t assume I took it as such. I do have incredible respect for Luke, Daryl, and Sam though, so I thought this topic was actually worth revisiting. In addition to hot takes, that thread under Luke’s tweet generated a lot of great discussion.
The fact that lots of people have reactions to this type of work is a good thing, not a bad one. Anyway… many smart, analytically savvy people hate radars mostly for the reasons explained in that thread.
They can be misleading. Ordering of variables matters. There are more precise, accurate ways to convey the data.
The thing is, I knew all of this before I started down this road.
My stuff used to just feature tables of numbers. Then I spent the better part of six months doing a deep dive into data vis before I ever spat out a silly radar. And yet, some might say despite my education, I still did it.
It's obviously the result of a choice, not of ignorance. Before I’m tried and hanged for data visualization crimes against humanity, I’d at least like a chance to mount my defence. Often when someone allegedly smart (that’s me) continues to do something somewhat controversial in the face of some serious criticism, there are things we can learn.
Learn to Communicate
I have been to a lot of analytics conferences at this point, and the biggest point of emphasis on the sports side is always communication is key. You need to understand your audience (usually coaches, sometimes executives), and take steps to deliver your analysis in a form and language that they can accept. Rephrase that a bit, and you end up with:
Audience. Dictates. Delivery.
In order to succeed, you need to take account of the audience you are pitching to and give them something they can understand. Even better, give them something they want to understand. (It helps if it's pretty.) In soccer/football circa 2014, the fanbase had no real statistical knowledge. The media was just glomming on to the idea that maybe stripping out penalties from goalscoring stats made sense, assists might be vaguely interesting, and the concept of rate stats wasn't completely insane.
I’m not being glib here, this was how it was.
“xG” (or Expected Goals) was seriously weird and controversial and people seemed to think, presumably via the result of someone else's misguided analysis, that possession had something to do with the probable final score. In situations like this, visuals go a long way toward opening the conversation. If you show a table of numbers to a coach who isn’t already on board, you’re dead.
Only mostly dead.
Interesting... Tell me more.
The same was true of the general public. Radars grabbed people in a way almost nothing else did. I think part of that is related to the fact that various soccer/football video games had used spider charts for a long time already, so they were somewhat familiar. Math = bad. Familiar = less scary = good.
Right, we have a vis style that grabs attention - can I fix the flaws?
Rewinding, when faced with a cool visualisation framework that would allow us to talk about player stats in an accessible way – something ALMOST NO ONE WAS DOING IN SOCCER at the time – I set about seeing if I could correct radars for some flaws. Major flaws with radars:
- Order of variables matters
- Area vs length issue means potential misinterpretation
- Axes represent different independent scales
So what did I do?
- Added the 95th/5th percentile cutoffs to normalize for population. Suddenly axes weren’t really on independent scales, even if it seemed like they were
- Broke the stats we care about for different positions into their own templates
- Clustered similar element stats together. Shooting over here. Passing over here. Defensive over here, etc.
One thing I was also clear about up front was that I wanted to include actual output numbers, not just percentiles. This was another choice about audience impact. Sports quants mostly care about percentiles. Normal fans barely cared at all about numbers, so percentiles would be even more abstract.
Plus no one had ever done percentile work for most of the stats in football.
What is a high number of dribbles per game? No one knows. Putting percentile info made even less sense then, because we were just starting to have conversations about basic stats. Going back to my youth collecting baseball cards, I wanted people to be able to talk and argue about Messi vs Ronaldo from a stats perspective, and the only way to make that happen was to have some actual numbers on the vis. I don’t even know if this was successful, but it was a design impetus that was constantly in my head.
Impact vs Accuracy
Most of the people ranting about radar charts on Twitter yesterday are pretty hardcore quants. To many of them, sacrificing precision for anything is strictly verboten. The problem with this perspective for me was: radars aren’t for you. Hell, radars aren’t even for me. I work in the database, and my conclusions are largely drawn from that perspective.
The minor inaccuracy issues of radars don’t affect my work.
BUT I wanted to talk to a resistant public about soccer stats, and this enabled discussion. I needed to talk to coaches about skill sets and recruitment, and this was a vital way of bringing statistics into that discussion while comparing potential recruits to their own players.
As I designed them, radars exist to help you open the door with statistical novices, and from that perspective they have been wildly successful. Even in 2017, football/soccer doesn't have the volume of knowledgeable fans that basketball and baseball have in the U.S. We also don't have coaches who are comfortable with almost any statistical discourse, although that is definitely changing in the last year.
Actual, practical feedback
So a funny thing happened on the way to the boardroom: In football, radars became accepted as a default visualization type. I’ve visited a number of clubs who just incorporated the work as part of a basic suite of soccer vis, only occasionally to my chagrin.
“My coaches love these. They want us to do physical stats in this form because they feel like they are easy to understand.” “This is cool. I like the way the shapes become recognizable as you use them more, and clearly indicate different types of player.”
At Brentford, we took two non-stats guys, taught them the basics of interpretation, and churned through over 1000 potential recruits in a year. Football isn’t like American sports. Players can come from a ridiculous number of vectors, and radars were the best, most easily understandable unit of analysis I could find.
Combine no money, huge squad needs, and limited recruitment personnel, and the only way we could hope to succeed was via efficiency and volume. They were not the end of the analysis. In fact, for recruits we liked, they only comprised a tiny portion of the evaluation cycle. From a volume perspective though, radars were the most used form of evaluation in the process.
On StatsBomb IQ, our analytics platform, even non-recruitment people seem to be taking a deep dive in a way they never have before. One person researched nearly 1000 players and teams over the course of the first two weeks, just because they liked learning about the game and the stats in this way.
In my own opinion, having researched many alternatives, I feel like radars are the fastest way to get a handle on what skill set a player may or may not have, and include some basic statistical context.
You can’t prevent misuse of statistics
This tweet from Luke yesterday made me laugh
@stat_sam In case people think this is well-known, last month I met a pro soccer team using radar plot area as a global metric to value players...
— Luke Bornn (@LukeBornn) May 17, 2017
Two words: competitive advantage. If you’re going to take the research for free and apply it while failing to understand how to actually use it, you deserve what you get. I’m 98% certain this club never talked to me, or else I would have forcefully steered them away from that type of analysis. And the problem is, you can’t prevent people from doing bad analysis on any type of stats.
Thinking the wrong stats are important?
Happens all the time, even from smart, highly educated individuals. Bad interpretations of basic visualizations? Check newspapers almost every day. Bad/useless visualization? So many, it’s a surprise we don’t all walk around with our eyes bleeding.
Look, everyone makes mistakes in their jobs.
We try to be objective, but everyone in stats and analytics also makes mistakes. Daryl Morey drafted Joey Dorsey, even though understanding age curves and competition cohorts is a pretty basic concept. Soccer stats once thought possession% was important. Pep Guardiola apparently thought he could win a Premier League with a bunch of fullbacks over 30. I said nice things about Luke Bornn.
It happens to the best of us.
The fact of the matter is, unless you are there talking to the users every day, you can’t prevent people from taking your work and potentially using it poorly. It doesn’t matter if that work is in tables or numbers, or bar charts, or radars, or fans, or code, or in a shot map, or whatever. The interpretation, application, and execution of analysis will remain more important than simply having the information from now until the end of time.
The bashing of radars is almost a yearly event at this point, and like I said at the start, I concede that there are flaws in the vis style that even my adjustments haven’t completely overcome. With that in mind – and despite the fact that customers actually seem very happy with our current style of vis – we will probably add an alternate form of player data vis to StatsBomb IQ by the end of the year.
I’m not sure exactly what it will be, but as football clubs move from novice to intermediate to advanced statistical analysis, precision will become more important and I want us to stay ahead of that curve. In the meantime, I hope my defense of radars above has at least explained why I made this horrible, unforgiveable visualisation choice in 2014 and continued to stick with it over the years. Communication and opening doors to talk about stats in football with coaches, analysts, and owners remains the most important hurdle we have to overcome.
Radars start a conversation.
They get a reaction.
And for whatever reason, football people are often more comfortable talking about and digesting them than almost any other vis type I have encountered.
Maybe that will change in the future. Until then radars remain a pretty damned good visualisation for displaying most of the different elements in player skill sets*, which is the most important conversation topic we have in player recruitment.
*In my opinion at least, and provided you correct for their flaws and educate your users.