DIME
Get the picksJoin

PROJECT · findings note

project_projections_engine_state

[…] is BUILT + DEPLOYED end-to-end (NBA+WNBA crons writing nightly slates) but UNVALIDATED and NOT the source of truth. Blockers: (1) bet settlement n_settled=0, (2) ~0 model lift over L5, (3) not wired into live picks. Gap-audit 2026-05-29.

← All findingssource · memory/project_projections_engine_state.md

UPDATE 2026-07-07 (audit, project_wnba_audit_2026_07): WNBA validation now EXISTS from prod paper-trading: 51 slates 6/14–7/6 → 50.8% hit / −1.1% ROI / +0.42pp CLV over 1,748 settled staked bets, with 33% of staked bets grading no_data (join failures suspected — odds player_id is now 100% populated, so the name-matching shim may be misfiring; instrument reconcile). WNBA models DO beat L5 MAE on all 9 stats (unlike NBA ~0 lift; minutes −10% remains the only big lift). Consumed in prod by data-hydrator (17-22 of ~37 snapshot players get quantiles — coverage gap unexplained) + live-service triggers; rest-of-game is deployed but NBA-only by explicit guard whose "WNBA live boxscores empty" premise predates live-service 8f6a8da. 2026-07-02 (3 games) produced ZERO slates silently; SNS alerts topic has no subscriber. Uncommitted since 5/29: historical_backtest.py v2f6 + backtest-mae-summary.json (the only artifact showing q50 beats the MARKET on threes/blocks/steals). No kill/cutover gate exists — define one pre-registered (e.g. CLV ≥ +1.5pp CI>0 over next 500 bets, else enrichment-only).

3 BLOCKING gaps to "goal achieved":

  1. [internal detail removed]
  2. No modeling signal. Per-stat models barely beat L5: points q50 R²=0.52–0.54 vs 0.70 target, ~0% MAE lift on most stats (only minutes lifts, −10%, MAE 4.66). SYSTEM_STATE.md:499 calls blowout_prob the largest available MAE win; per-position DvP + NegBin for low counts are the other two. Model is currently FROZEN. 485K-bet backtest CLV is NEGATIVE (−0.30/−0.49pp), 50.6% win, −1.78% ROI, edge buckets non-monotone. OBSERVATIONS_HARNESS.md:8: "the model has no structural edge over closing line."
  3. Not the source of truth. Live picks still come from L10-recency PropDiagnostic.LineEdges in data-hydrator (diagnostic.go:273/384-390); engine attached only as optional q10–q90 band (assembler.go:601). Phase 5 scout-service parity comparison NEVER run (SYSTEM_STATE.md:454) — cutover undecidable. Cutover criteria already defined: ≥ +0.50pp CLV vs M0, lower-95%-CI>0, monotone edge (OBSERVATIONS_HARNESS.md:143).

Absent/future: DFS optimizer (dfs/__init__.py 0 bytes); NCAAB+MLB (no code); NumPyro Bayesian cold-start (designed only — cold_start.py is a flag-gated WNBA-only archetype-clone, docstring disclaims it's fitted); q05/q95 tails (80% CI coverage only 76–89%); Pinnacle MISSING from historical backfill (us-region only) so historical CLV-vs-sharp impossible.

Critical path: close measurement loop (fix player_id joins via POST /api/v1/odds/correlate/pending; make grading-service settle CLV bets; tag engine as versioned formula_id; Shin de-vig + Pinnacle close) → run scout-service parity → earn signal (blowout_prob, per-position DvP, NegBin) → deploy Brier/ECE+drift Lambdas → fix WNBA plumbing → THEN cut over LineEdges to consume engine, gated on measured win.

Related: project_live_rest_of_game (orphaned live read path), feedback_model_edge_calibration ("edge %" not real edge — confirmed: 40–60pp edge bets win ~55% w/ ~0 CLV), project_multi_formula_clv_harness (M1–M4 merged/deployed, doc stale), project_wnba_projections_playbook, feedback_check_line_edges_before_quoting.

Open question: slates 5/13–5/28 carry ~17:44 UTC run_ts (looks like manual score_matchup.py, not the 09:30 cron) — unclear if unattended daily cron reliably writes forward slates or they're manually backfilled. Verify before trusting zero-touch.

TWO models live here — do not conflate (verified 2026-08-25)

1. Quantile-band slate model — GET /slates/{date} + /slates/{date}/{slug} Per-player q10/q25/q50/q75/q90 across 10 stats (points, rebounds, assists, 3PM, steals, blocks, turnovers, FGA, FTA, minutes), ~31 players + ~96 bets per slate, run_ts ~10:00 UTC. This is the one that is DARK (0 slates on 8/16-8/18 and 8/20-8/24). Consumed by data-hydrator (fetcher/projections.go → findProjectionsSlateSlug), which is why the snapshot reports projections_no_slate. Already measured to have NO betting edge: −1.1% ROI over 1,748.

2. Minutes model — GET /minutes/{league}/{slate_date}/{game_id} projected_minutes + low/high per player, plus market_total, spread_home, and a leans block (model_share vs market_share, model_q50 vs market_line). Still ALIVE and answering today — independent of slates. Measured: beats L5 on MINUTES (starters 16.7%) but does NOT transfer to props. ⚠️ Nothing in data-hydrator / grading / broadcaster / debate-v2 consumes it.

So the outage is low priority: the dark model is the one proven not to make money, and the live model is wired to nothing. Do not restate "projections are down" as if it were costing edge.

⚠️ leans cites a market_line for minutes, yet our books publish no minutes market (17,424 WNBA minutes bets, 0 closing lines). Unexplained — check the source before trusting that block.

FIXED + VERIFIED 2026-08-24 (projections-engine 53f15e0)

  • its data root every morning even in the offseason — the download in resolve_models_dir/resolve_data_root happens BEFORE any slate check — and WNBA then lands with nothing left. Whether a day produced slates tracked whether the container happened to be cold. It also retried 3x at 470-630s billed each.

Now: daily 2048 MB, weekly 4096, predictions 2048. Verified by reproducing the failure: fired NBA async to fill /tmp, then WNBA on the warm container → status: ok, errors: [], n_slates_written: 2. Snapshot force-refresh then showed errors: null and 18 of 29 players carrying projections; the watchdog's snapshot:section-errors went warn → ok five minutes later (it caches).

⚠️ The projections-engine checkout was 10 commits behind origin and the first cdk diff proposed DELETING MarketHistoryPublisher + 5 WNBA market-history rules. Always git fetch + cdk diff and read it for [-] before deploying. The deploy also shipped 61c852c, committed 08-10 and never deployed.

⚠️ download_models docstring still claims "the entire models tree is ~50 MB" — it is 406 MB non-archive for NBA (3.57 GB total, _archive/ skipped), 20 MB for WNBA. The NBA download also pulls models/wnba since that prefix nests underneath.