Why aren't Manchester City better?

Manchester City are in second place in the Premier League. They just won the League Cup. After going to Madrid and beating Real 1-2 they’re likely on their way to the Champions League quarterfinals. But they’re only on pace for slightly over 80 points. That’s a major dip from 98 last season and 100 the year before that. Why aren’t Manchester City better? The basic underlying numbers suggest that there might be some slight deterioration over the last couple of seasons, but nothing major. City’s defense seems slightly worse than it has been, but again this seems like largely the same performance level just with massively different results. Given that you’d expect that perhaps the side either massively overperformed their expected goals last season or were massively underperforming it this year. But, that’s not particularly true either. In fact while City have had stretches where they’ve been over xG, sometimes by quite a lot, they haven’t had very many long periods where they’ve been classic underperformers. What we can see on the chart from this season and last is that while City did run pretty hot early in the 2018-19 season, what we’ve seen is the side’s goal difference converge with it’s xG difference (which is gratifying to stats nerds since that’s exactly what’s supposed to happen), while that xG average has fluctuated around a relatively consistent level. But, just because those topline numbers are only minorly different, doesn’t mean there isn’t more to investigate under the hood. Prying apart those xG numbers show a team which has some major differences in both performance and execution year over year. During City’s great 2018-19 season they ran way over expectation when the match was tied, putting up 39 goals from 27.15 xG. It’s nice work if you can get it. Last season, City was the best team while also getting fortuitous bounces that regularly put Pep Guardiola’s side ahead. This season is a different story. City are actually slightly below their xG when drawing, 24 goals from 25.80 expected. The natural result of these differing fortunes is that even when they do ultimately pull ahead it’s taking longer for them to do so. That means just by dint of being tied longer, they’re giving up more goal scoring opportunities and risking falling behind more than they did last season. Already this season they’ve conceded nine goals from just under eight xG. All of last season they only conceded five from just over 7 xG This is one of the ways that variance and actual performance levels are inextricably intertwined. It is absolutely fair to say that City have been a worse defensive team when they’ve been drawing this season than last. But the only reason for that is that they’ve had a harder time scoring goals, due to what appears to be a natural swing in variance. Luck, or lack thereof, in one part of the pitch leads to actual deterioration in another. There are of course other factors at play. There have been real personnel changes, with Fernandinho aging out of central midfield and injuries across the defensive line. Perhaps relatedly, the teams defensive solidity has suffered. Specifically, teams complete more passes and create somewhat more high quality shots when they break through City’s outstanding aggressive press. The luck factor certainly doesn’t absolve the defense. They’ve become somewhat more porous and no longer seem to have the magical ability to press aggressively while magically never getting exposed on the counterattack. But, given the rest of the data it’s possible that without the bad luck City would be doing enough in attack to entirely offset the defensive problems. The City example is one that illustrates how much art there is in the science of parsing a team’s statistics. The line between luck and skill is often a lot blurrier than a casual observation might led you to believe. Good luck in one area can impact skill in another and vice versa. In the long run the vast majority of these factors average themselves out and become indistinguishable from noise. But, over the short term, when trying to figure out discrete questions like why isn’t City better, the line between luck and skill, becomes much much harder to parse.