The projection that was always too small
Our college football spread projections were running 25% too small. A game the model called at 40 points was landing closer to 50 in the results; one we called at 32 was landing near 40. Across the full season, that compression held at every level — big projected margins were too small by roughly the same proportion as small ones.
Predicting a game's margin means picking a number: the favorite wins by 10, or by 40, or by 56. Over a season, you can check whether those numbers track reality — whether the games where the model projects a large margin actually produce large margins on average, and whether the tight-game calls stay close. That check is calibration.
We ran it. The model was consistently off in one direction. It had been configured to be. The tool we were using to tune the key parameter was quietly pushing it toward smaller projections every time we tried to improve the model.
What a scale parameter does
A football model works in two stages. First, rate each team — how good are they, measured as expected scoring above an average opponent? A team that went 12-0 against strong competition gets a high rating. A team that went 4-8 does not.
Then translate those ratings into projected margins. That translation runs through a function with a single dial: turn it one way, and a 20-point rating difference becomes a 45-point projected spread. Turn it the other way, and the same difference becomes a 28. That dial is the scale parameter.
Setting it right matters. Too high and every game becomes a predicted rout. Too low and the model can't distinguish a 7-point favorite from a 35-point one. The right value is wherever the projections track what football actually produces.
The question is: what do you measure to find it?
Why optimizing for accuracy made things worse
The natural answer is to try different values and see which produces the smallest average error — how far off, in points, the projections are from actual game results across a full season. Lower is better.
Mean absolute error is minimized by hedging toward the mean. In a noisy sport, a model that shrinks every projection toward the average wins more often against the distribution of real outcomes than one that makes bolder calls — even when the bolder calls are correct in aggregate. It's not that the math is broken. The measure does what it says — minimizes the average miss. But a compressed projection misses differently than a well-calibrated one, and average error can't tell the two apart.
When we found the scale setting that minimized average error, we had found the one that did the most hedging. Every projected blowout came out looking less like a blowout. Every tight game came out a little tighter than that. The number we were trying to minimize went down. The projections stopped tracking the actual spread of college football margins.
There's no easy way to see that from the error score alone. A compressed projection that misses a big game by 12 looks similar to a well-calibrated projection that misses a different big game by 11. Both look like imprecision. One is systematic miscalibration; the other is football.
The value our optimization chose left the model 25% below where calibration said the evidence pointed. The score got better. The predictions got worse.
What calibration checks instead
Calibration asks the right question. When the model projects a 40-point margin, do the games at that level actually average close to 40? That's not the same as asking whether any individual game lands at 40 — it won't. It's asking whether the projections, as a system, are the right size.
The test is a regression. Take all the games from a full season. Plot projected margins on one axis and actual margins on the other. Measure the slope of the line that runs through them. A slope of 1.0 means each projected point corresponds to roughly one real point in the results — the model is tracking reality. A slope below 1.0 means projections are running consistently short. Above 1.0 means they're running hot.
Before the fix, our slope for college spread projections was below 1.0. The model was systematically under-predicting big games — not because of random game-to-game noise, which averages out, but because the scale parameter had been set to produce it. Driving that slope toward 1.0, rather than minimizing average error, is what pointed us to the right value.
The gap between 12.95 and 12.99
The calibrated version of the parameter produced a mean absolute error of 12.95 points on the test season. The error-minimized version — the one that left every projected spread 25% too small — had been at 12.99. Four hundredths of a point.
The metric we'd been using to choose between parameter settings couldn't see the difference between a model that was well-calibrated and one that was systematically underestimating every game. Both looked nearly identical. That's not a coincidence: when projections run short, some games where the result was large end up close to the compressed call anyway, and the errors on those games cancel out some of the errors elsewhere. Averaged across a full season, the two approaches produce almost identical total error.
On calibration, they're not close at all. One tells you a game is a 40-point mismatch and means it. The other says 40 but is describing something closer to 50.
For anyone reading the projected margins on the board, those four hundredths of a point hide a real difference. A game the calibrated model projects at 45 has been measured against 45-point results. The same game projected at 36 — what you get from an error-minimized scale — is telling you something too, but not the same thing.
What this doesn't say
It doesn't say mean absolute error is a bad measure. For most comparisons — does this method beat that one, is this board improving season over season, what's the typical gap between projection and result — it's exactly right. The specific place it fails is when you're choosing a scale parameter, because the error-minimizing choice always compresses that scale downward. Use it as a tiebreaker between two options that are equivalent on calibration. Don't let it override calibration.
It doesn't say calibration is sufficient. A model can pass the slope test and still be wrong about individual games for real reasons. Calibration tells you projections are the right size. It doesn't tell you they're built on the right evidence. Early in the college season, the model prices last year's results while the market prices current rosters — that gap closes as games are played, and the boards reflect it honestly.
And it doesn't say this trap is unique to football models. Any prediction that outputs a number on a continuous scale and any parameter that controls how spread out those outputs are is at the same risk if you tune it on average error. The check is always the same: run the regression, look at the slope, and confirm that what the model calls a 45-point game is actually producing 45-point situations.
Sources
Every number here comes from the graded ledger the boards publish. Research from a statistical model, not betting advice; no outcome is guaranteed. Method: how MatchWiz works.