Month: October 2014

That is all.

Assessing the skills of goalkeepers is exceptionally difficult, and it’s why I have never attempted to do it. As well as the basic and fundamental skill of shot stopping, the best goalkeepers will be able to effectively assess situations and decide whether to advance or stay on their line. How they deal with a high ball is also important, as is their distribution and their communication and organisational skills. Combine those altogether and you have a range of skills that would be very difficult to measure using conventional statistics.

Although I don’t think we are in the position of being able to rate goalkeepers in terms of their entire skillset we are in the position of being able to assess their shot stopping attributes. I’ve been told that the best way to eat an elephant is “one bit at a time”, and so we’ll take the same approach to rating goalkeepers. Let’s start the process by having a look at goalkeepers’ shot stopping numbers. The dataset that I’ll use for this analysis is Opta data for the four complete seasons from 2010/11 to 2013/14 covering the Big 5 leagues (EPL, La Liga, Serie A, Bundesliga and Ligue 1). This gives me a dataset of more than 64,000 on Target shots which were faced by 393 goalkeepers.

Goalkeepers with Best Save Percentage

To get our bearings we’ll take an initial glance at the Top 12 goalkeepers from the last four seasons as ranked by Save Percentage (Saved Shots / Total OnTarget shots faced) and I’ve applied a cut-off of a minimum of 300 shots.

This list seems to make some sense; we have Buffon topping it and it includes other accomplished net minders such as Abbiati, Sirigu, Neuer, Cech, De Gea and Hart. With the possible exception of Victor Valdes, it’s fair to say that this list of Top 12 Shot stoppers (ranked by Save %) includes most of the names that would quickly spring to mind. The football world will be glad to hear that, even when looking solely at numbers, some goalkeepers appear to be better than others. OK, so that isn’t exactly ground breaking; but it’s a starting point. At this point it’s also worth considering whether, just because the best shot stoppers have the highest save percentage, it means that the keepers with the highest save percentage are necessarily the best shot stoppers.

Repeatability

In terms of assessing the shot stopping skills of goalkeepers, the question about whether we can tell the great stoppers apart from the average ones is not the most important question for me. What is important is the timescale, or quantity of On Target shots, that we need to observe before we are in a position to be able to soundly judge shot stopping skills.

Why is this important?

If it takes a very long period of time before we can be confident that a goalkeeper’s save numbers are repeatable then that has to have implications for football teams. How do teams scout for a goalkeeper?

How can they possibly tell whether the saves a potential new signing made in the handful of games he was watched is likely to be repeatable going forward?

How do they know when to drop a goalkeeper due to a few bad performances?

For me, it’s apparent that we need to be able to assess how repeatable shot stopping performances are for a goalkeeper from one period of time to another. To do otherwise means that any decisions or judgements that are based on the outcomes achieved during the first period of time may be built on shaky foundations. This viewpoint seems to be shared by Billy Beane, as in an interview last week with Sean Ingle he was quoted as saying:

“You don’t have a lot of time to be right in football. So ultimately, before you mark on anything quantitative, you have to make sure you have scrutinised the data and have certainty with what you are doing, because the risk is very high.”

 

How to Measure Shot Stopping ability

For this analysis I am going to use two forms of measurement for assessing the quality of shot stopping performances. The first measure is the simple Save %, and this was the metric that the first table in this article was ranked by.

The second measure is based on our (created with Constantinos Chappas) Expected Goals model, or specifically our ExpG2 component of the model. This ExpG2 value is the expected value of the shot AFTER it has been struck. This means it takes into account all of the factors that existed at the point the shot was struck (ie location, shot and movement type etc) but it also includes the shot placement, but doesn't include the location of the GK at the time of the shot.

As expected, the shot placement is a huge driver of the ExpG2 value. A shot arrowed for the top corner will have a much higher ExpG2 value than a shot which was taken from the same location, but which was placed centrally in the goal. ExpG2 can be used to measure the placement skill of the shooter, but it also has great use in measuring the shot stopping performance of goalkeepers.

It is only right that a ball that hits the net in the very top corner reflects less badly on the keeper than one which squirms through his body in the middle of the goal; the ExpG2 metric achieves this. This analysis will use ExpG2 Ratio, which is calculated as: ExpG2 / Actual Goals Conceded An example: 12.34 ExpG2, but the GK conceded 14 goals. The ExpG2 ratio in this case is 0.88. A value of 1.00 means that saves have been in line with expectation, a value greater than 1 suggests the keeper has performed better than the average keeper would have done for the shots he faced, and a value of less than 1 means he allowed more goals than the average keeper would have done.

Repeatability (yes, that word again)

The key point in this analysis is not to measure the shot stopping performance of any goalkeeper, but to instead look at how repeatable the shot stopping performances are from one period to another. After all if they aren’t repeatable, be that due to variance, luck or something else we aren’t currently measuring, decisions and actions taken by teams shouldn’t be the same as those that they would take if they were known to be repeatable.

Isn’t that right football industry?

In March this year, Sander Ijtsma published a piece where he suggested that you could practically ignore save percentages.  I wanted to expand on that concept a little.

Method of Analysis

I sorted all the On Target shots by date and sequentially numbered each shot faced by each goalkeeper. I created a variable, n. This allowed me to divide the shots faced by each keeper into sets of n size. I then calculated a single correlation value for each level of n by plotting all the individual save performances achieved in Set 1 (which will be of size n) on the x asis against the individual save performances achieved in Set 2 (which will be of size n) on the y axis.

It would probably help to use an example; let’s say n = 50. For each GK I measured the save performance for their shots numbered 1 – 50, 51 – 100, 101 – 150, 151 – 200 etc. I then plotted the relationshion between shots 1 – 50 and 51 - 100. I did this as I wanted to see how repeatable the save performances were for goalkeepers from one set to the next. I also plotted the relationships between shots 51 – 100 and 101 – 150, as well as between 101 – 150 and 151 – 200 etc. I continued on using this logic to plot the relationship values for each consecutive set for each goalkeeper until I couldn’t compare any more sets of n on target shots for them.

A single correlation value was then calculated for each level of n. I made n a variable so it could be changed to assess the level of correlation (repeatability) between consecutive sets of On Target shots faced by goalkeepers based on the number of shots in each sample. The table below sets out the extent of the correlations for varying sizes of n.

The values in the two columns on the right of the table show the correlation for the save performance of a goalkeeper over two consecutive sets of facing n On Target shots. The third column shows the correlation for the simple Save % metric, while the final column shows the correlation for the ExpG2 Ratio measurement.

A brief reminder that a value of 1 means the save performance of the two consecutive sets are perfectly positively correlated, while a value of 0 indicates that no correlation exists whatsoever. I don’t want to make this piece any more technical than it already is but I’ll simply state that the correlation coefficients themselves have confidence intervals around them, these are primarily driven by the number of pairs for each level of n.

Let’s walk through one of the lines in the above table, we’ll use n = 100. There were 305 instances (pairs) in my data set where goalkeepers faced 100 on target shots followed by another 100 on target shots. The correlation in their save performance when measured by the simple save ratio between the first and second sets of 100 shots was just 0.127. When we instead measure the keepers’ save performances by the ExpG2 metric the correlation increases to 0.232. A correlation of 0.232 is a weak correlation as it means that just 5% (0.232^2) of the variability in the second set of 100 on target shots is explained by the first set of 100 on target shots. That is startling.

Even when we use the advanced ExpG2 metric to assess how well a goalkeeper performed over a series of 100 on target shots we can still only expect it to explain just 5% of their performance over the next 100 on target shots he faces.

As an average goalkeeper faces approximately 4 on target shots per game this means we need to assess a keeper over about 25 games to only get a 5% steer towards how he will perform over the next 25 games. Pause and think of the implications of that. Even a sample size of 250 shots (n = 250) has a correlation, using the more detailed ExpG2 metric, of just 0.405, this level of correlation is generally described as a moderate correlation as it gives an r2 (variance) value of just 0.16.

At this point I cannot calculate correlations for n greater than 250 as I do not have enough data in my dataset. To give you an idea of the amount of dispersion of save numbers from one set to the next I have produced below the plot for the 31 consecutive sets of 250 shots (which means the goalkeeper has had to face at least 500 on target shots):

Does the shot order matter?

Daniel Altman read a draft of this piece and he suggested I should look at whether I am introducing some bias into the correlations due to the way that I populate the two groups of n shots. You will recall that I carried out the above analysis by splitting the shots into groups based on the sequential order the keeper faced those shots, and doing this inevitably builds a time and age factor into the makeup of the groups.

This is fine as I initially only wanted to measure sequential relibality (as this is what happens in the real world when it comes to assessing a goalkeeper's performances at a certain point in time), but this may cause difficulties if we attempt to measure innate talent. To address Altman's prescient point I randomised the order of the shots and conducted the same analysis as before.  The correlation values between consecutive sets of n shots (with the shot groupings decided on a random basis) are as follows:

As expected, because we chose a different basis on which to create our sets of n, there are some differences between the correlation values in this table and the one that appeared earlier in this article.  However, at n = 250 the correlation value of 0.467 still means that we would only expect 22% of variability in the second set of 250 on target shots to be explained by the first set of 250.  This is still quite a low value and suggests that even when we strip away any impact of age / time bias there doesn't seem to be a great level of repeatability in terms of goalkeeper save performances.

Conclusion

Of course there are goalkeepers that save shots better than others. But for every goalkeeper such as David de Gea that have consistently over performed (1.23 and 1.21 is the ExpG2 Ratio for his two sets of 250 shots) we have a Stephane Ruffier who notched up ExpG2 ratios of 1.15 and 0.98 in his two sets of 250 shots.

If those two players had been assessed after their first batch of 250 on target shots (which would have taken almost two full seasons to amass) they would both have been assumed to be well above average shot stoppers. However, only one of them went on to repeat it again after they faced another batch of 250 on target shots. Imagine the analyst that recommended signing Ruffier on the strength of his save performances using an advanced metric over a “large” dataset of 250 shots or 60 games. This is a good time to recall Billy Beane's assertion that we need to make very sure that we know what we are doing with our data as the stakes will be high for any club that truly embraces the use of data in their decision making process.

When this level of variance exists after 250 shots, it is easy to see how Simon Mignolet went from being one of the best shot stoppers in the EPL in the 2012/13 season (ExpG2 ratio of 1.25) to being one of the worst in 2013/14 (ExpG2 ratio of 0.88). Very simply, trying to judge how good a goalkeeper is at saving shots based on one season’s worth of data is little more than a craps shoot such is the divergence on performance over 150 shots. Just 11% of his performance next season can be explained by his performance in the season just passed.

It looks to me that the very fact a player is good enough to be a goalkeeper for a top tier club means that he has achieved a level of performance that is difficult for even advanced numbers to distinguish, at least not until he has faced a very large number of shots. I’m just not sure yet how large that number needs to be. Remind me never to advise a club when they are looking at buying a goalkeeper as the shot stopping facet of the goalkeeper’s skillset was supposed to be one of the easier ones to measure and interpret.

Thanks to Constantinos Chappas and Daniel Altman for allowing me to bounce off them some of the more stat heavy angles of this piece. Cover picture by Steve Bardens.

  A chart created by Christoffer Johansen made its way into my Twitter timeline last week. This chart was fairly stark in that showed a steady and perceptible decline in the output of Man United’s Robin van Persie over the last 4 or 5 seasons. Christoffer’s chart was as follows: That chart doesn’t need much commentary, so I’ll give it none. Johansen then went on to show that van Persie’s decline extended to more than just the rate that he shot over the last five seasons. He showed that the year on year provision of assists is another category that has seen a decline from the Dutch captain:   Wider Attacking Contribution On a team with as much attacking talent as this current Man United side possess, it is obvious that both the shots and the headlines will be shared around. Not everyone can take the final shot, or even play the assist for the shot; this is especially true when the attacking talent includes all of Falcao, Rooney, Di Maria, Mata and van Persie. This desire to award attacking players the recognition that their involvement deserves is what motivated me to create the Attacking Contribution metric . Previously, unless they played the final pass or had the shot, their part in attacking moves would have gone unnoticed by the statistics that are currently reported on. An introduction to this metric can be found in an article published last week on Statsbomb.  In summary, it records the number of times that a player was involved in the final four events of an attacking move that culminated in a shot. Due to the various ways that forwards play it can be difficult to compare their outputs. Some forwards excel in holding up play and linking with others, while some are simply there to score the goals. I decided to include the final four events in the calculation of the Attacking Contribution metric as this will, generally, capture all the players that were integral to the attacking move. If attacking players are regularly failing to be involved in the final four events of their teams’ attacking moves I think that questions should be asked of them. What exactly is their role in the team? What does the coaching staff want them to achieve? And, most importantly, are they taking the position of a player that has more to give to the team than they themselves are?   Robin van Persie’s Attacking Contribution From Johansson’s charts we can see that RVP’s shot numbers have declined and that he’s also providing a minimal level of assists. This in itself might not be a problem.  With all the attacking talent at Louis van Gaal’s disposal it is possible that van Persie is being involved earlier in the moves. Such an earlier involvement would not see him gaining recognition under the two categories of stats that Christoffer Johansen covered in his charts. If he was central to United’s attacking moves, moves which were being finished by the likes of Di Maria, Rooney or Falcao then supporters of RVP could rightly say that the 2014 version of the Dutch forward is about more than just scoring goals. But is this actually the case? The Attacking Contribution Metric can help us answer this question: I only have data for games played from the start of the 2010/11 season. Robin van Persie was remarkably consistent during the spell from 2010/11 to 2012/13, during these three seasons he was involved in almost 50% of the shots his teams took while he was on the pitch. RVP’s productivity numbers noticeably tail off last season (2013/14) as he is involved in only 39% of United’s shots. Although the table above doesn’t include his playing minutes, I can tell you that he played less than 1700 minutes last season compared to the 3500 and 3350 minutes that he clocked up respectively in each of the two preceding seasons. The Dutchman obviously struggled with his fitness last season; he missed plenty of game time, and when he did play he wasn’t as productive as in previous terms. If last season was disappointing for van Persie, then this current one has started off very badly. His involvement in just 28% of United’s shots is an extremely poor individual return for a front line attacker and represents a serious decline from the exceptionally high numbers we have grown used to seeing van Persie deliver, first at Arsenal and then in Ferguson’s final season at Old Trafford.   Man United’s Individual Attacking Contributors The table below shows the attacking involvement of United’s attacking players this season: I know that the season is young, but we can see that six other United players have had a greater attacking input that van Persie has so far. How can Anders Herrera have had a greater influence (in terms of the percentage of attacking moves he has been involved in) than RVP has had? United’s attacks are passing van Persie by, this is a trend that I picked up few times this season in the commentaries that I made on some of Man United’s Player Positional Trackers, an example of which is United’s defeat against Leicester. Given the attacking firepower that currently resides inside Old Trafford I don’t think, in his current form, that van Persie is deserving of a start in United’s line up. With Mata, Falcao, Rooney, Di Maria, and Herrera real possibilities for the five available attacking spots (and that’s even before we consider Januzaj or Valencia), Robin van Persie should no longer expect to be one of first names on the Man United team sheet. Maybe the injuries have finally taken their toll. The fact that he has recently turned 31 will not help him either, but there is no doubt that his ability to influence games is clearly waning, and United will need more from all of their attacking players if they are to successfully secure a Top 4 league position this season.   Tale of Two Dutch Strikers In a brief Twitter conversation with Simon Gleave on Saturday, Simon mentioned that there was quite a bit of chatter in the Dutch media around van Persie and Huntelaar. Comparisons were being made between the two, presumably around which of the players should receive the nod to start up front as the Oranje played Kazakhstan on Friday night. Van Persie led the line whilst the Schalke striker had to be content with a place on the bench, although Huntelaar did come on in the 56^th minute and he grabbed the Dutch equalising goal just six minutes later. This article is mainly concerned with van Persie, but given the circumstances I decided to widen it out to briefly include some of Klaas-Jan Huntelaar’s numbers.   Van Persie or Huntelaar When van Persie was in his prime there was no contest around which of the two were the more productive player. However, is this still the case now in late 2014? It’s impossible to answer this question with the use of just one metric but let’s take a look at Huntelaar’s Attacking Contribution metric over the last four and a bit seasons: A few seasons back (2010 – 2012) we can see that at with an attacking involvement of approximately 35% in Schalke’s shots he was considerably less involved than van Persie was at Arsenal and Man United. However, as Father Time has quickly caught up with RVP it looks as though Huntelaar’s attacking performances haven’t yet taken the very noticeable decline that van Persie’s has; this despite the fact that just six days separated their birth. I’d expect the 42% contribution rate that Huntelaar has posted so far this season to reduce a little, but at this stage he still looks to be a player that will contribute to about one third of Schalke’s attacks. I’m conscious that this Attacking Contribution metric isn’t all encompassing. It doesn’t assess the quality of chances, nor the rate at which they convert their chances and we also need to be aware that we are looking at players that play in two different leagues. But, even being mindful of all of those caveats, it could be argued that van Persie has regressed to the point that he and Huntelaar could be expected to have a similar attacking contribution for the Netherlands.

Is every goal unique? The instinct says yes, but one needs only to remember Steven Gerrard to realise that it is possible to make a fine career out of scoring the same three goals over and over. The truth sits somewhere in the boring middle, but it's undeniable that many goals share similarities, including how they are created. It is this similarity that I set out to explore here.

My analysis rest on the notion of possession chains. For every on-the-ball event -- such as a goal -- it is possible to find the unbroken, ordered sequence of previous events leading to it. This is the idea on which Colin Trainor based his recent article about players' attacking contributions. My definition of a chain is likely different from his, and too technical to give it in full here, but the broad outline is as follows:

• I only look at chains terminating in a goal,
• The events in the chain are strictly consecutive (ie. no intermediate events are excluded),
• Only actions by the scoring team belong in a chain,
• A set piece can only be the first event in the chain (ie. we never look past a set piece),
• Ditto possession regain events (tackle, interception, recovery).

The numerous minor choices I had to make on top of these may mean that the overall definition is so arbitrary that I am unsure how much of what follows is insightful (never mind useful), and how much is just having unwholesome fun with the data. Caveat emptor.

The data I looked at was kindly provided by Opta and comprises all games from 2010/11 to 2013/14 in the top divisions of England, Spain, Italy, France and Germany. For every goal I derived the possession chain and grouped identical chains. It turns out that the 50 most common goals look like this:

[Yep, that's a screenshot.]

As you can see, by far the most common goal is scored with the team's first touch in the chain. I think this is partly a testament to the randomness of the game itself, but also to the strictness of the definition of the chain: if a defender manages to get a touch just before an intricate move is about to be crowned with a goal, none of the move will count in the chain. A penalty and a header from a corner complete the top 3.

Another summary of the results is provided in the figure below. Apologies for the terse, but hopefully still unambiguous codes for individual events. Note that listed event can occur anywhere in the chain for the goal to be counted, so for example the "head" bar comprises not only the headed goals, but also any goals where there was a headed pass in the move.

It turns out that only 64.5% of goals have a completed pass in the buildup (again, under my restrictive definition of buildup). I was delighted to discover that this agrees nicely with the classic analysis of Reep and Benjamin ("Skill and Chance in Association Football" J. R. Stat. Soc. 134(4):623-9, 1968, cited here after The Numbers Game), whose number is 60.6%. A quarter of goals involve a cross (but not necessarily as an assist), and about 1 in 19 see a shot saved before the ball goes into the net. Own goals are 3% of the total.

Finally, for the theory minded, here is the distribution of frequencies of individual chains on a log-log scale. It's tempting to drop some names here (cough Zipf cough), but in truth so many things look linear-ish on a log-log plot that it's best not to. Perhaps if and when the definition of the chain is made more robust, the distribution plot will be more interesting.

There is a pass that David Silva and Mesut Ozil, Premier League's outstanding playmakers, are very fond of. Standing just inside the opposition penalty area close to a corner, with options inside and outside the box, they slip the ball instead to the overlapping fullback who crosses it in. Why do they do this if, as the common wisdom has it, crossing is low percentage play?

The obvious answer is that not all crosses are equal, and with good setup play a cross is a dangerous weapon. I think it is particularly true of short, low crosses, precisely the kind Ozil and Silva encourage. I set out to investigate this hypothesis, only to realise that I don't have a clean way of separating low and high crosses in my database. What follows are two simple analyses trying to work around this problem.

Completion and conversion

Two definitions: I consider a cross completed if the next on-the-ball action is performed by the player from the crossing team. A cross is converted if the crossing team scores within 5 seconds of the cross. This is an arbitrary window, but it should catch all the goals (including own goals) which are "due" to the cross in significant degree. This will include own goals, rebounds and goals from brief goal-line scrambles. Note that conversion and completion are independent in this formulation: a completed cross may be converted or not, and a converted cross needn't have been completed. I looked at the last four full seasons of the five big European leagues and only considered locations where I have more than 1000 attempted crosses. Unless indicated otherwise, only open-play crosses were considered.

As expected, the premium crossing area is on the edge of the penalty and inside it (shall we call it the Zabaleta Zone?), where around 5% of crosses are converted. If this sounds low, then consider that the average cross conversion rate is just 1.76%. What was a bit of surprise to me is that it seems to be easier to complete a cross from the wide areas, farther away from the box. I suspect this is due to the fact that with a short cross the area is on average more crowded and who takes the next touch becomes more random. Average completion across all areas is 23.58%.

Crossing and success

There is a weak positive relationship between success (measured in points per game) and the proportion of cross-assisted shots that aren't headers. (Here assist is taken in the strict sense and not in the sense of conversion defined above.) The correlation coefficient for the attached graphic is 0.43, which drops to 0.32 when Manchester City, the ultimate low-crossing team, are removed. Interestingly, this relationship doesn't exist in the Bundesliga and Ligue 1.

I don't want to speculate on the nature of this relationship beyond what Devin Pleuler said on Twitter on Monday:

https://twitter.com/devinpleuler/status/519196963129294850 https://twitter.com/devinpleuler/status/519197095887380481

That is, a preference for low crossing will come naturally to better teams. Success and reliance on crosses are inversely related: the higher proportion of a team's shots come from crosses, the lower points-per-game. The strength of this relationship is similar to the previous, and, once again the effect is not found in Germany or France.

Conclusions

There can be no firm conclusions until I find a way of separating low crosses from the rest. However, it does appear that not all crosses are equal, and that a team that relies heavily on crosses for chance creation should make sure they know what they're doing.   Data provided by Opta.