How Our Expected Goals (xG) Model Works

2026-07-04 · methodology · expected goals · xG · model

A transparent look inside Hockey Alchemy's xG model: four situation-specific XGBoost models, 53 features, 16 seasons and 1.6M shots, with cross-validated log loss, AUC, and calibration.

An expected-goals (xG) model is only as good as the data underneath it — and in the NHL, that data keeps moving. The league's shot tracking has shifted over the years: how missed shots are logged, where shots get recorded from, which shot types exist. The game itself keeps changing too. And that's a problem, because an xG model frozen on old seasons quietly drifts out of calibration — it starts expecting goals modern hockey doesn't produce, and crediting goalies and shooters for the wrong things. Public models have run into exactly this: as the tracking drifted, their goal expectations drifted with it. Staying accurate isn't a one-time training job; it's a rebuild-and-retrain discipline.

That's the lens for this post. xG is the backbone of everything we publish — goalie work above expected (GSAx), on-ice shot quality, and the expected-goals component of GAR/WAR — so if the model drifts, all of it drifts with it. Here's how we keep ours honest: what it measures, the four separate models under the hood, the features it reads, how we validate it, and where it still falls short.

What xG measures

xG assigns every unblocked shot attempt a probability — from 0 to 1 — that it becomes a goal. A shot with xG = 0.15 is the kind of chance that historically scores about 15% of the time. We model P(goal | unblocked shot attempt), the Fenwick-based definition used across public hockey analytics. The probability comes entirely from the shot's context:

Where — distance from the net, angle, and raw coordinates.
How — shot type, whether it's a rebound or a rush chance, and shot speed when tracking data is available.
When — score state, period, strength situation, and seconds elapsed into a power play or penalty kill.
What happened just before — the prior event, which team it belonged to, and how far and fast the puck moved to get to the shot.

Why four models, not one

Scoring behaves very differently by game state. A power-play shot from the slot is a different animal than a 5-on-5 shot from the same spot — different passing lanes, different goalie movement, different rebound dynamics. Rather than force one model to average across all of it, we train four situation-specific models so each can learn its own patterns:

Model	Situation	Shots trained on
Even strength	Equal skaters — the bulk of the game	1,334,000
Power play	Shooting team has the advantage	143,081
Shorthanded	Shooting team is down a skater	129,276
Empty net	Opposing goalie pulled	dedicated formula

A unified model trained on all 1.6 million shots serves as a fallback when a situation-specific model isn't appropriate.

The features

The model reads 53 features per shot — a deliberately rich feature set. They fall into six groups:

Group	Examples
Core shot geometry	Distance, angle, coordinates, shot type, period, time
Situation	5v5 / PP / PK / 4v4 / 3v3, strength differential, empty net
Rebound & rush	Rebound (≤3s after a save), rush (shot within 4s of zone entry), shot speed
Prior event	Type of the previous event, which team, time / distance / speed to the shot
Power-play timing	Seconds elapsed since the power play or penalty kill began
Score state	Goal differential, trailing / leading (teams behind take riskier chances)

The prior-event, power-play-timing, and score-state groups are the biggest additions over our earlier model — they capture how the play developed, not just where the puck ended up.

How it's trained and validated

The models are XGBoost gradient-boosting classifiers trained on 16 NHL seasons (2010-11 through 2025-26) — 1,606,355 unblocked shots and 108,080 goals. That's more history than most public models train on, which matters for stable calibration across eras.

We evaluate with 5-fold, season-stratified cross-validation: every shot is scored by a model that never saw it during training, and folds are drawn so a season isn't split across the train/test boundary in a way that leaks information. Three metrics do the judging:

Log loss (our primary metric) — rewards well-calibrated confidence and heavily punishes confident wrong predictions. Lower is better; random guessing at the NHL base rate is ≈ 0.24.
AUC — can the model rank a goal above a non-goal? Measures ordering quality, independent of calibration.
Calibration — do predicted probabilities match reality? If we predict 100 shots at 0.10 each, we should see ~10 goals (ratio = 1.00).

Results

Cross-validated performance by model:

Model	Log loss	AUC	Calibration
Even strength	0.166	0.857	0.999
Power play	0.241	—	0.998
Shorthanded	0.240	—	0.998
Unified (all shots)	0.178	—	1.000

Calibration is the number we're proudest of: within a fraction of a percent of perfect at every strength, which means our xG totals can be trusted as goal-equivalents rather than a relative index that needs rescaling.

An honest caveat: the shorthanded model is our weakest. Small samples and the chaos of 4-on-5 sequences make it the hardest state to model, and it's the area we'd most like to improve.

Better than our previous model

The current model cut log loss by roughly 28% versus our previous version, and added about +0.12 of AUC — a large jump, and a consistent one across all 16 seasons. Almost all of it came from the new context features (prior event, power-play timing, score state), not from more data alone: the model now understands the sequence of play leading into a shot, which is where a lot of shot quality actually lives.

Season	Old log loss	New log loss	Old AUC	New AUC
2013-14	0.203	0.146	0.734	0.856
2018-19	0.217	0.159	0.717	0.837
2021-22	0.232	0.160	0.754	0.857
2024-25	0.223	0.152	0.770	0.860
16-season avg	0.213	0.155	0.734	0.849

Lower log loss and higher AUC in every single season — the gain isn't a one-year artifact.

How it stacks up against other public models

The other question is how it lines up against the field. Start with the methodology, where the differences are structural:

	Hockey Alchemy	Typical public model
Models	4 situation-specific (EV / PP / SH) + empty-net formula	usually one blended model
Training data	16 seasons · 1.6M shots	~7–8 seasons
Features	53 (incl. prior-event, PP timing, score state)	~15–20

That shows up in the numbers. On a common set of shots, our even-strength model posts a 0.166 log loss and 0.857 AUC. The widely-used public gradient-boosted models we benchmarked against — scored on the same shots — came in around 0.185 and 0.215 log loss and roughly 0.78–0.79 AUC. More seasons of training and the extra context features are most of the difference.

Two honest caveats. First, those competitor figures are our computations on a shared shot set — an apples-to-apples test on our data, not numbers published by those sites, which is why we've left them unnamed. Second, privately-tracked models with camera and pre-shot-movement data remain a tier public models can't fully reach; that gap is real, even as public data improves.

Limitations

Shorthanded is our weakest state, as above.
We work from the NHL's public play-by-play. Like all public models, we don't see off-puck movement or the full pre-shot passing sequence the way camera-tracked private models can — we approximate it with prior-event features.
NHL shot-tracking drifts over time. Recording conventions have shifted across seasons; we retrain on the full history to keep calibration honest, but data-quality maintenance is permanent, not one-and-done.
Empty-net shots use a formula, not a trained model, given how few of them there are.

What it powers

Everything downstream. Expected goals feed GSAx (goalies saving above expected), on-ice xGF% and shot-quality metrics, and the expected-goals component of our GAR and WAR models. It's the single model that, when it improves, lifts the most of the site at once — which is why we keep it transparent and keep retraining it.

For the shorthand definitions, see the expected goals glossary entry; for how xG fits into the broader system, see our methodology.

Figures are 5-fold, season-stratified cross-validated results on 16 NHL seasons (2010-11 through 2025-26), 1,606,355 unblocked shot attempts. Point-in-time as of publication; the model is retrained as new data lands.