This post has been written by Dan Kennett.

In the last year, this site has led the way on Goalkeeping analytics.   In that time the focus has become more on “Expected Saves” rather than the humble old Save% (Sv%) i.e. [Total Number of Saves] / [Total Shots On Target Faced]

Following a chance discovery on the NBC Sports website (example here), it’s been possible to quickly collect Sv% data going back to 2007/08 for England & Germany and 2008/09 for Italy & Spain, resulting in almost 48,000 Shots On Target with an average of 360 saves for 95 Goalkeepers (Individual Goalkeeper sample size is currently the main constraint with Goalkeeping models).

With this data it’s now possible to re-revisit the humble old Sv% and ask if some Goalkeepers are betting at saving shots than others over the course of their career. All Goalkeepers with > 100 saves were put into a funnel plot and the results are below:

GK_Funnel_v2

The purpose of the funnel plot is to show how randomness decreases as the sample size increases. In this case, as a Goalkeeper faces more Shots On Target. This is represented by the curved lines getting closer to the horizontal line.

 

  • The horizontal black line is the overall Sv% for the 48,000 shots (72.13%)
  • The curved black lines represent 2 standard errors above and below the mean

 

At the top centre of the chart there are 5 dots above the curved line that tally with the received wisdom of “good goalkeepers”: Gigi Buffon, Marc-Andre Ter Stegen, Petr Cech, Christian Abbiati and Manuel Neuer. Maybe Neuer genuinely does deserve his unofficial title of “world’s best Goalkeeper”?

Some love should also be shown for Tim Howard who has been consistently excellent for Everton over 7 seasons.

At the bottom centre of the chart there’s a cluster of 3 keepers from the Premier League who should set alarm bells ringing for fans: Brad Guzan, Boaz Myhill and Tim Krul. This statto will also be keeping close tabs on Ruben Ivan Martinez of Rayo Vallecano from now on!

 

 

Geek Notes

Sv% is normally distributed (p = 0.256)

There is a weak relationship between Sv% and SoT Faced (r = 0.345) but the p-value is very low (0.001)

We can rule out “league effects” as once goalkeepers from different leagues are grouped together and compared, there is no significant difference amongst the means (p=0.471)

Data

The following is a copy of the collated data:

[table id=63 /]

  • n0r

    Reading the Sherwood article it was clear that Spurs conceded a lot of valuable opportunities to their opponents. Watching Lloris he is clearly a good keeper, statistically here he doesnt look so good (who would with spurd def this season ;)). I guess the next (long) step is to compare expectant goal scoring opportunity to each shot conceded by each team relative to the saves made by the keepers.

    • Andrew

      I was a little surprised at first that ExpG wasn’t part of this, since it was being used to evaluate goalscorers, but then I realized how much work it would be for goalkeepers who face all the shots taken by the opposing team.

  • http://americansocceranalysis.wordpress.com Matthias Kullowatz

    Was there any adjustment in that funnel graph for multiple testing? Even if not, there seem to be more than 5% of the keepers outside the funnel, indicating the potential for some sort of skill.

  • beige

    Excellent post. I was very interested in the significance of this stuff – I followed the method explained here:
    http://www.real-statistics.com/binomial-and-related-distributions/beta-distribution/

    alpha (a)=saves, beta (b)=’failed saves’

    So for each keeper we can learn more about their sv %
    mean sv% =1/(a+b)
    95% confidence interval around sv%= BETAINV(0.025, a,b) to BETAINV(0.975, a,b)
    p value of test that keeper sv% is > population sv% (0.7213)= BETADIST(0.7213, a, b)

    Found 14 keepers with pValue < 5% (in a sample of 96 keepers we would expect to find 4.8).

    Keeper, pValue
    Weidenfeller 4.9%
    Mannone 3.9%
    Bizzarri 2.5%
    Schwarzer 2.3%
    Reina 2.0%
    Hart 1.1%
    Drobny 0.9%
    Buffon 0.6%
    Howard 0.6%
    Navas 0.5%
    Ter Stegen 0.3%
    Cech 0.0%
    Abbiati 0.0%
    Neuer 0.0%

    This is pretty convincing evidence that SOME GOALKEEPERS ARE BETTER AT SAVING SHOTS THAN OTHERS!

  • http://americansocceranalysis.wordpress.com Matthias Kullowatz

    Good stuff, Beige on the binomial distribution. It’s one of my favorites.

    Jumping across the ocean, I just plugged some figures in really quickly from our MLS database from 2013. We rate each goalkeeper’s “saves ability” based mostly on origin and placement of the shot, comparing actual goals allowed to expected goals allowed.

    There were 29 keepers that recorded at least 50 shots on target (against) in 2013. The correlation in keeper rating between the first 25 and second 25 shots was -0.01.

    Upping the split to 100 and 100, we had 17 keepers qualify. The correlation was -0.03.

    My methodology could be a little flawed in using SOT as a proxy for timeline (rather than xGoals against, for instance), but I’m not sure that would have much effect on the correlations. Basically within a season in MLS, there is very little stabilization among keepers’ saves abilities.

    I’ll try to put something more rigorous together in the coming weeks, but thanks for the inspiring article 🙂

    • Dan

      It’s worth noting that the NBC site also has 5 seasons of MLS Saves/SoT data so that could easily be added to the dataset above. e.g. would a one-way ANOVA show the MLS mean Sv% is not significantly different to the big 4 European leagues?
      Example here: http://scores.nbcsports.msnbc.com/mls/players.asp?player=1989

    • beige

      Matthias, those are some very interesting numbers!

      This is a pretty tough problem
      1 – Shot quality is so hard to measure (there are obvious things like power as well as things like keeper ‘sightedness’, striker deception (ie giving the keeper ‘the eyes’) which probably matter).
      2 – No doubt differences in skill are very small at a given level

      IIRC there was some work done on how a keeper is the most value enhancing place to spend your transfer £££. I think it was in Soccernomics. So they must have some evidence for a difference in skill between cheap and expensive keepers.

  • Dan

    Beige/Matthias
    Thanks for the kind words and glad you enjoyed the post
    I think there’s a clear lesson here in that if the data for the post is published, we can learn as much, if not more, from the ‘below the line comments’ as we can in the post!

  • Tuiuan

    This is very interesting, and the comments here add even more knowledge. It will be more interesting to evaluate keepers when we upgrade our available data to be able to differentiate the types of shots and their degree of “difficulty” to defend. But this data is good enough for us to make some assumptions. I always thought of Buffon as the World’s best keeper, and he was for a pretty long time. But it seems that age had finally taken a little bit from him.

  • Luke

    If David Marshall remains in the Premier League next season, it would be extremely interesting to see his performance here. Marshall made the highest number of saves this season and has widely been praised highly in the media for his performances in a very poor Cardiff side.

    Great work here Colin (and Beige).

    • Colin Trainor

      Luke, it was Dan Kennett’s article I just posted it.

  • Daniele

    Excellent post, and thanks for sharing the data. An important point, though:
    The correlation between save pct and number of saves is perhaps not strong, but it is highly statistically significant (that’s what p<0.001 means). This suggests that maybe goalies that are observed to have higher save percentages are also picked to play more by their coaches, survive longer in the top flight, and therefore have the chance to rack up more saves. If you looked at the correlation between save percentage and shots on target per 90 minutes, you would probably observe a negative correlation: Buffon, playing for a top team in his league, doesn't face that many shots, and also faces shots of lower quality.

  • Pingback: ‘NOT JUST A SHOT-STOPPER’: CLAIMS% AS A MEASURE OF GOALKEEPER AERIAL PROWESS | outsiders | in()

  • Pingback: Mind the Gap: Are Championship goalkeeping performances transferable to the Premier League? | outsiders | in()