Predictive variables extravaganza (part 1)

Much verbiage lately on what you can count on in terms of how a team performs this year influencing how it performs next year.  The standard approach is to take your variable of interest, run a correlation from one year to the next, and look at the result.

The better the correlation, the more confident you can be that you are likely looking at a repeatable skill as opposed to just random variation.  My general rule of thumb: 1/3 (up to 33%) is uncorrelated, 33% to 66% is a real correlation but with a generous number of confounding factors, and 67% and above is highly indicative.  Of what, you have to figure out – but it’s definitely indicating something.

One last warning: these are correlations.  A classic sciencey mistake is to confuse correlation (two things seen together) and causation (one causes the other).  You gotta use some brains and logic to determine if you’re truly seeing a causative relationship.

So, for part 1 of this two (maybe three) part series, let’s take a look at a number of variables from 2013/14 to this season (we’ll make the hopefully safe assumption that with a single game left, the numbers for 2014 are not going to change in any meaningful way), and see which ones “stuck”.

All data you see here is sourced from the terrific folks over at war-on-ice, and the points from NHL.com.

Let’s start with the grandaddy – points.  If you’re good (or bad) one year, you should be good or bad next year, right?

Points

figure_1 grandaddy pts

As you’d expect, there’s a pretty good relationship one year to the next.  Is it unsurprising or surprising to you that it’s only about 55%?

Even Strength SCF%

figure_3

As expected, it’s pretty sticky.  Better than points, even.  This is what the stats geek are pointing to when they talk about repeatability and predictability.

EV CF%

figure_7

Unsurprisingly – the venerable EV Corsi is also representative of a repeatable skill.  Again, stickier than points.

EV PDO

figure_14

This shouldn’t – but may – come as a surprise that PDO (Sh% plus Sv%) does not carry over year to year.  It means there is an awful lot of randomness embedded in this number (and since the two of them automatically sum to 1.0, this is where the idea that they will regress comes from).

Corollary: you can’t coach PDO.

By the way, notice the three incredibly horrible PDO teams this year.  Those are ARI, CAR, and EDM.  Notice that all three were decent (99-100) in PDO last year.  Unlucky for us that these things didn’t carry over from last year to this year.  Lucky for us these things are unlikely to carry over to next year, yes?

It means we can expect the Oilers to be a bad team next year because of managerial incompetence, not PDO!

All Situations Sh%

figure_22

On first glance, that correlation is low but higher than expected – 30%?  But the chart tells a clearer story, that it’s mostly a random effect.  There’s not much of a correlation there.  It’s more a small cluster of teams sitting right around the middle.  Regressing to the meat, you might say. Team shooting is not nearly as repeatable a skill as people think it is.

All Situations Sv%

figure_29

Again, you’d think that there would be a higher measure of consistency in goaltending from year to year, but there really isn’t.  I suspect that’s because a. goalies are voodoo, with inherently high variability in performance year to year, and b. teams with horrific goaltending one year tend to replace the goalie the next year, and that guy often comes from a team with good goaltending.

All Situations FO%

figure_36

Ah hah!  Sanity restored.  Good faceoff teams have a reasonable tendency to stay good faceoff teams.

So there you have it: count on SCF% and CF%.

Definitely some repeatability to points and FO%.

Do not count on PDO, Sh%, or Sv% to stay where they are.

Next edition: a look at which variables correlate well with other, different variables, year to year.

P.S. I have posted the CSV file where I gathered this information in the Data section, in case you want to try any of this cr*p for yourself!

Advertisements

2 thoughts on “Predictive variables extravaganza (part 1)

  1. Hey G-Man,

    Couple notes from someone who spends significant time editing other’s data presentations:

    (1) It’s not clear what data you’re using. All players in league, or some specific set of the 25-odd # of points plotted?

    (2) Similarly, what’s the definition of pearsonr? p? Why present two types of correlation measures?

    (3) Why bars and smooth Gaussian-type fits of the vert./horiz. projections? As in point (2), it’s almost always clearer to present less options when it doesn’t take away from the data

    (4) What are the hourglass shapes wrapped around your trend lines? If they are supposed to add some determinative value, I can’t fugger them out.

    (4) I would totally dock marks for your axes titles. “13 AS FO%” is awfully hard to decipher at first glance. As a guideline, I wouldn’t repeat information that’s in the title. If you made the title “All Situations Face Off % (FO%)”, then make your axes “2014 FO%” and “2013 FO%”, it could be read much easier.

    (5) Q: Why am I writing all these trivial critiques?

    (6) A: I’m waiting for someone.

    (7) Nope, they’re not here yet, better go scan Lowetide comments again…

    Like

  2. Heh heh, all feedback is appreciated. I have what I believe are reasonable answers to all your queries!

    1. Data is a team by team comparison. So for example, AS Sv% compares on a team to team basis the correlation between their all situations sv% year to year.

    2. Pearsonr is the Pearson Rsquared correlation, the full formal nomenclature for correlation. p is the significance level for the fit. If you apply the standard 95% confidence, you would be looking for p < 0.05.

    3. The bell shapes are called Kernel Density Estimates, which help with certain types of data to more easily ascertain the comparative 'shape' of the distribution.

    4. The hourglass shapes are the 95% confidence intervals. Like most correlations, they get much wider as you go away from the middle of the data.

    4b. Heh, yeah they are pretty cryptic. The reason is that those are the titles straight out of the dataset. Highly descriptive variables are more than a bit of a pain when scripting. You want to know something cool? The script that generated all 36 charts (every 2013 variable vs every 2014 variable) is seven lines of code! It's why I can generate these datasets nearly on a whim, so there's the tradeoff.

    5. Because you're waiting for someone?

    6. I knew it!

    7. See you there.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s