← All findingssource · memory/project_adjusted_projection_audit.md
Trigger: Kennedy Burke o9.5 pts (2026-08-26 vs GSV) served as board pick #1; snapshot showed baseline_q50 5.15 → adjusted_projection 2.26 against a 9.5 line. Actual: 2 pts / 21.5 min. Question: is the projection H2H-driven, and has it ever been scored? Answer: yes, and no — until now.
REPLICATION VALIDATED (hold this bar for any future audit): DH commit c8dd572 ported to Python; 632/632 cells exact (<=0.01) vs 4 live prod snapshots, 79 players x 8 stats; full end-to-end rebuild from raw upstreams 618/624 (the 6 misses = one player, SDS season-averages lagging its own game log by one game, straddling roleAnchorMinDeltaMinutes=5.0). Burke reproduced from as-of data: q50 5.15351, adj 2.25589. GOTCHA: Go's math.Round is half-away-from-zero, NOT banker's — round2 on WindowStats.Avg feeds applyAdjustments.
THE HEADLINE — adjusted_projection is NOT fit for purpose as a betting input.
- vs CLOSING MARKET LINE (points, n=2,688): MAE 5.428 vs 4.764 = +13.9% WORSE, CI [+0.501,+0.828], t=+7.94. Also loses on rebounds (+8.0%), assists (+8.7%), threes (+2.0%). Restricting to >=3-book props STRENGTHENS the result; restricting to 45-55% de-vig rows (so the line IS the median) also holds — not an MAE-metric artifact.
- vs NAIVE BASELINES: loses to a plain L10 mean on all four stats, and to a minutes-aware naive on all four.
- THE ADJUSTMENT LAYER DEGRADES ITS OWN INPUT: adj vs baseline_q50 = +6.2% (pts) / +5.1% (reb) / +3.2% (ast) / +4.1% (threes) worse. Strictly value-destroying on every stat.
- DIRECTIONAL: projection's side hits 53.19% vs always-bet-the-under 53.18% on identical rows (n=8,336) — +0.01pp of information over a constant needing no data/model/service. Market's own de-vigged favourite hits 54.9% and beats both. Raw-looking rates (51-56%) are ENTIRELY the under base rate: the projection picks UNDER on 65-68% of rows because its bias is -1.08.
THE H2H TERM IS A UNITS-MISMATCH BUG (diagnostic.go:1362-1375). Shift computed as
MatchupH2H.Avg - L10.Avg then added to baseline, which is the engine q50 on 62% of rows — a
different quantity. Proof by split: engine-q50 rows cost +0.4449 [0.345,0.543]; L10-fallback rows (where
baseline==L10.Avg so it collapses to a coherent 0.5L10+0.5H2H blend) cost only +0.1010 [-0.014,0.232]
= NOT significant. 4x more harmful precisely where the differenced windows aren't what it's added to.
Fires on 24.5% of rows (51% of points rows); harmful on 8/8 stats, CIs exclude zero on 8/8; of the 2,220
rows where it moved the projection >=2.0, 62.7% moved it AWAY from the actual (coin flip = 50%).
BUT FIXING THE UNITS DOESN'T MAKE IT PAY. Weight sweep on a properly rebuilt term
(w*(H2H.avg - L10.avg); w=0.5 reproduces the shipped number to 1e-10):
w: 0.0→MAE 1.9604 | 0.05→1.9605 | 0.10→1.9633 | 0.25→1.9820 | 0.50→2.0479 | 1.00→2.2869
Monotone increasing from zero — MAE-minimising weight is 0. More history doesn't rescue it
(n=3 +0.577, n=4 +0.391, n=5 +0.334). Removing the |shift|>=0.5 gate changes nothing.
THE RECONCILIATION (this is the nuance — do not flatten it to "H2H is worthless"): H2H is worthless as
an AUTOMATIC ADJUSTMENT to a point estimate, at any weight, because 3-6 games cannot measure a matchup.
It is NOT worthless to SURFACE. The project_analytics_engine /ask design rule already gets this right —
"never gate, always label", and not_measurable is the COMMON case for vs-team splits BY DESIGN. Correct
treatment: render "6 meetings vs GSV, 6.8 ppg, at 22.6 min in her old bench role — n=6, not_measurable"
and let the human judge. Wrong treatment: silently rewrite 5.15 into 2.26 where nobody can see or challenge it.
OTHER ADJUSTMENTS: pace = NULL (+0.002, z=1.41, fires on 68% of rows — harmless, does nothing). Blowout = mildly harmful. (Separately measured and refuted as a GATE: see project_dvp_blowout_gate_refuted.)
SHIPPABLE FIX: delete the 12-line H2H block. Recovers 0.0874 MAE overall [0.0642,0.1116] z=+7.68, and 0.357 on the 24.5% of rows where it fires. Halves the gap to Pinnacle (12.7% → 6.6%). Highest-value change found in the whole audit. If any matchup signal is ever rebuilt it must be a MULTIPLICATIVE ratio applied to the same quantity it was derived from, not an additive delta across mismatched baselines.
BLAST RADIUS: changes NO bet. Money formulas read line_edges (built from the raw L10 window), not
adjusted_projection. What changes is what humans, LLM surfaces, debate personas and the verdict/case
builder SEE. That is the point — the system has been publishing a self-generated number worse than a
ten-game average, and it misled a real decision on 2026-08-26.
Scripts: session a1827200 scratchpad proj/ (dhrepl.py + panel/ablation/sweep). Companion:
project_two_month_retro_2026_08, project_dvp_blowout_gate_refuted.