Expected goals has found itself outside the confines of the analytics community this season, which has brought renewed questions regarding its flaws, foibles and failures. The poster-child for expected goals flaws has been Burnley and their over-performing defence, even finding themselves in the New York Times courtesy of Rory Smith. Smith's article is a fine piece blending comments and insights from the analytics community and Sean Dyche himself. Smith quotes Dyche describing Burnley's game-plan when defending:
The way it is designed is to put a player in a position that it is statistically, visually and from experience, harder to score from.
Several analyses have dug deeper into Burnley's defence last season, including an excellent piece by Mark Thompson for StatsBomb. In his article, Mark used data from Stratagem to describe how Burnley put more men between the shooter and the goal than their peers, which may go some way to explaining their over-performance compared with expected goals.
Further work on the EightyFivePoints blog quantified how the number of intervening defenders affected chance quality. The author found that when comparing an expected goal model with and without information on the number of intervening defenders and a rating for defensive pressure, Burnley saw the largest absolute difference between these models (approximately 4 goals over the season). If there is a quibble with Smith's article it is that it mainly focuses on this season, which was only 12 games old at the time of writing. Much can happen in small samples where variance often reigns, so perhaps expanding the analysis to more seasons would be prudent. The table below summarises Burnley's goals and expected goals conceded over the past three and a bit seasons.
Each season is marked by over-performance, with fewer goals conceded than expected. This ranges from 5 goals last season to a scarcely believable 15 goals during their promotion season in the Championship. The above table and cited articles paint a picture of a team that has developed a game-plan that somewhat flummoxes expected goals and gains Burnley an edge when defending. However, if we dig a little deeper, the story isn't quite as neat as would perhaps be expected. Below are cumulative totals for goals and expected goals as well as the cumulative difference over each season.
In their 2014/15 season, Burnley were actually conceding more goals than expected for the majority of the season until a run of clean sheets at the end of the season saw them out-perform expected goals. After the 12 game mark in their Championship season, they steadily out-performed expected goals, yielding a huge over-performance. This continued into their 2016/17 season in the Premier League over the first 10 games where they conceded only 12 goals compared to an expected 19. However, over the remainder of the season, they slightly under-performed as they conceded 39 goals compared with 36 expected goals.
The above illustrates that Burnley's over-performance in previous Premier League seasons is actually driven by just a handful of games, rather than a systematic edge when examining expected goals. This leaves us needing to put a lot of weight on their Championship season if we're going to believe that Burnley are magical when it comes to expected goals. While the descriptive and predictive qualities of expected goals in the Premier League is well-established, there is less supporting evidence for the Championship. Consequently it may be wise to take their Championship figures with a few grains of salt.
This season and last has seen Burnley get off to hot starts, with what looks like a stone-cold classic example of regression to the mean last season. If we ignore positive variance for a moment, perhaps their opponents got wise to their defensive tactics and adapted last season but then you have to assume that they've either forgotten the lessons learned this season or Dyche has instigated a new and improved game-plan. The cumulative timelines paint a different picture to the season aggregated numbers, which might lead us to conclude that Burnley's tactics don't give them quite the edge that we've been led to believe. In truth we'll never be able to pin down exactly how much is positive variance and how much is driven by their game-plan. However, we can state that given our knowledge of the greater predictive qualities of expected goals when compared to actual goals, we would expect Burnley's goals against column to be closer to their expected rate (1.3 goals per game) than their current rate (0.6 goals per game) over the rest of the season.
Time will tell.