How We Rebuilt Our Expected Goals Model

2026-10-20 · xG · methodology · validation

Measured on shots it had never seen, our old expected goals model was not as good as we thought. We found where the data and the model were wrong and rebuilt both. Here is what changed, why, and how much better the new model is.

Figures computed 2026-10-19, against the models as they stood that day. We change them — see the methodology page for where they are now.

Expected goals sit underneath almost everything on this site. Player ratings, goalie GSAx, team shot quality, power rankings and projections all start from the same question: how likely was this shot to go in?

So when we tested our own expected goals model properly, on shots it had never seen, the answer mattered, and it was not flattering. This post explains what we changed and why, and how the new model, which we are calling v3, compares with the old one.

TL;DR

• Our old model (v2) was not as good as we thought. An accuracy figure we had quoted internally (0.81 AUC) was measured on the shots the model learned from. On shots it had never seen, it was about 0.77.
• Most of the fix was in the data, not the model. Mislabelled situations, unflagged empty nets, missing power-play timing and phantom shifts were all distorting the inputs.
• On 283,227 shots neither model trained on, v3 ranks shots better than v2 (AUC .788 against .768), and across all situations its total expected goals land within half a percent of actual goals.
• Shooter talent is now a separate number. xG describes the shot. xG(sh) adds what we know about the shooter, and goalie GSAx uses it.
• Rebuilt over every season since 2010-11, team totals of our player ratings track the standings better than before (mean R² .762 to .781), and a goalie bug we found along the way is fixed.

Our objective

We set one rule before changing anything: every change has to win on data the model has never seen.

That rules out the easy way to look good. A model graded on the shots it learned from always looks better than it is, and ours did. So every model number in this post is held out, in one of two ways:

• Leave one season out: train on every season except one, then score that season.
• Forward only: train only on seasons before the one being scored, the way a model works in real life.

Testing is one thing; what the site shows is another. The model on the site is trained on every regular season through 2025-26, so its values for past seasons are fitted to those seasons, as is usual for a published xG model. This season’s values are true predictions: the model has never seen these shots.

We also used three player-level tests that are standard in public hockey analytics:

1. Same season: does a player’s xG per 60 track his actual goals per 60 in the same season? We use the out-of-sample version.
2. Next season: does this season’s xG per 60 predict next season’s goals per 60? This rewards real signal over noise.
3. Split half: does the first half of a season predict the second?

What we changed in the model

Who we learned from

We borrowed several ideas from Post Hockey’s published method, and we are glad to say so:

• a season scoring environment that keeps xG honest as league scoring rises and falls;
• a walk-forward shooter-talent estimate, described below;
• dropping misses the NHL marks short or a failed bank attempt, which can never be goals. CJ Turtoro identified that these explain much of the post-2023 inflation in public xG.

The changes

What we tested and did not ship

Each of these was scored by the same held-out tests. None earned a place:

Two numbers: xG and xG(sh)

xG answers “how likely was this shot to score?” It does not know who shot it. It is the headline number for shot quality, team xG and on-ice expected goals for and against.

xG(sh) adds a running estimate of the shooter’s finishing talent, updated after every shot using only his earlier shots. It is built to predict, and it is what GSAx uses: a goalie who stops an elite shooter deserves more credit than one who stops a fourth-liner. We label it wherever it appears.

Results

Shot by shot

Protocol: even-strength unblocked shots, regular season 2023-24 to 2025-26, the same 283,227 shots for both models. Leave one season out: each season is scored by a model trained on every other season. Forward only: trained only on earlier seasons, which is possible for 2023-24 and 2024-25.

AUC is how well a model ranks goals above non-goals, where 0.5 is a coin flip. Log loss is lower when the probabilities themselves are closer to what happened. Calibration is total expected goals divided by actual goals, where 1.00 is perfect. v3 wins every season under both tests. On empty nets, v2 as it ran on the site had a calibration of .48: it predicted fewer than half the goals.

Player level

Protocol: 5v5 regular season, 2023-24 to 2025-26, skaters with 500 or more minutes, out-of-sample models. The numbers are correlations (r) between xG per 60 and goals per 60.

Playoffs

We did not build a separate playoff model, because the regular-season model did not need one. Protocol: playoff shots from 2022-23 to 2025-26, which the model never trained on, scored with that season’s end-of-season environment. v3’s AUC is .789 against v2’s .765, and its calibration is 1.01 against .98.

Rebuilding every season

A new xG model changes everything built on it, so when we switched we rebuilt the whole site’s history: every season from 2010-11, every per-game number, the on-ice tables, RAPM, GAR and WAR, and the projections. We rehearsed the whole rebuild on a copy of the database first. The figures below compare the site just before the switch with the site just after. The rebuild also carries a cleanup of our shift data: the NHL’s HTML shift reports put a summary table under each player that we had been reading as about 20 extra shifts a game, and some games had been loaded twice. That inflated ice time and every on-ice rate, so these figures reflect both changes.

Protocol: team standings R² is the share of the spread in each team’s standings points explained by the summed GAR of its skaters and goalies, computed per season and averaged over the 15 regular seasons from 2010-11 to 2025-26, leaving out the 48-game 2012-13 season. Repeatability is the correlation of a skater’s RAPM-based GAR from one season to the next, skaters with 500 or more minutes in both, averaged over every pair of seasons.

RAPM-based GAR improved in 8 of the 15 seasons and dipped in the other seven, by .005 to .044 of R². Six of those are 2018-19 or later, the seasons where the phantom shifts were removed. To separate the two, we rebuilt the copy a second time with the old model and the cleaned shifts. That scored .770, and v3 beat it in 12 of the 15 seasons, so the recent dips come from the shift cleanup, not from the new model.

For individual players the change is a refinement, not a reshuffle. In 2025-26, skater WAR under v3 correlates .993 with v2, and 19 of the top 20 stay in the top 20. Goalie GSAx correlates .972.

A goalie bug we found on the way

Rebuilding every season exposed a bug in how we total goalie GSAx, and it had nothing to do with the new model. To credit missed shots to the right goalie, we assign each one to the goalie who faced the last shot on goal. Shots into an empty net have no goalie, so they were being charged to the goalie who had just been pulled for the extra attacker, along with any empty-net goals. One version also counted shootout attempts.

Teams pull their goalie when they are losing, so this quietly mixed team results into goalie ratings, and it flattered the old model. Measured as the standings R² of each team’s goalie GSAx (same protocol as above), v2 scored .289 with the bug and .260 without it. Under v3, which prices empty-net shots correctly, the same bug pushed the other way. With it fixed, v3 scores .264 against v2’s .260. Empty-net and shootout attempts no longer count toward any goalie’s GSAx.

What this changes on the site

With v3 live, every xG-based number from 2010-11 on has updated together:

• player and team expected goals, on-ice numbers included;
• goalie GSAx, which now uses xG(sh) and leaves out empty-net and shootout attempts;
• GAR and WAR, which are built on expected goals through RAPM;
• game scores, and the power rankings through roster WAR and 5v5 expected goals (team Elo is built from results alone, so it does not move);
• line and WOWY numbers on the lines pages;
• live expected goals during games in progress.

We keep every v2 value, so we can always show what changed. The moves you are most likely to notice, all from 2025-26:

• Connor McDavid’s WAR rises from 6.25 to 6.73, the biggest gain of any skater. He was first before and is first now.
• Jack Eichel, Evan Bouchard and Jack Hughes move into the top 11 by WAR, from 19th, 17th and 18th to 9th, 10th and 11th.
• Ilya Sorokin and Jeremy Swayman stay among the league’s best goalies, but their GSAx falls from +34.0 to +25.5 and from +31.0 to +24.8.
• John Gibson and Filip Gustavsson move the other way: +11.2 to +16.5, and −5.3 to +1.1.
• The Bruins and Canadiens allowed less shot quality than v2 said: their expected goals against fall by 20 and 16 over the season. The Devils’ expected goals for fall by 16.

Limitations

• Goals against stays the hardest test. On-ice goals against depends heavily on the goalie, which shot quality alone cannot see. It is the hardest test for any expected goals model, and ours is no exception.
• The power play is still the weakest part. Calibration improved a lot; telling which power-play chances are better than others is still the weakest part of the model.
• Before 2010-11, shots lack the event data v3 needs, so we no longer show expected goals, or GSAx, GAR and WAR built on them, for 2007-08 to 2009-10.
• One model, measured against itself. This post compares v3 with our old model, not with other public models. We will rerun these tests each season and publish the results, good or bad.

What is next

• A separate model for the puck-tracking era, revisited once there are more seasons of it to learn from.
• The same test suite, rerun every season and published here.

Data: NHL play-by-play, shift charts and situation codes. Methods credited to Post Hockey and CJ Turtoro where noted. Model tests are held out as described in each section. Model tests as of October 18, 2026; the rebuild figures compare the site before and after the switch on October 19, 2026.

More from the Notebook

Explaining Our Power Rankings

Readers asked why our power rankings still count last season eight games in. We replayed 13 seasons to find out what predicts the rest of a season: the record so far is the weakest option, dropping last season makes the rankings worse, and this season's 5v5 expected goals make them better. Here is the model, and what we changed.

Talent vs Production: Why the Box Score Misleads

Two wingers score 20 goals; only one of them will do it again. The gap between what a player produced and the process underneath it is the most useful idea in hockey analytics - and it is the split our expected-goals and finishing models are built to make.

How Good Is Our WAR Model? We Ran Four Tests and Lost Two

We tested our WAR model against HockeyStats and Evolving Hockey on the four questions an honest player-value model has to answer, matched to each benchmark's published protocol. Summed to a team it loses both tests - it forecasts next season worse than simply reusing last season's standings. Measured per player it wins both, repeating at 0.77 against 0.62 and 0.46.

Related: All Posts Methodology Glossary