DIME
Get the picksJoin

PROJECT · findings note

project_wnba_model_audit

WNBA prop-model audit (2026-06-15) — confirmed bug cluster FIXED in working tree (not committed/deployed); 25 WNBA tests + 622 suite green.

← All findingssource · memory/project_wnba_model_audit.md

FIXES APPLIED 2026-06-15 in the WORKING TREE (NOT committed, NOT deployed) on main atop the user's in-progress WNBA work. Verified: 25 WNBA tests pass (8 new regression guards), full suite 622 passed, TS typecheck clean, py_compile clean.

  • Opponent reb/ast/turnover/pace signals de-inverted + made relative to pool size via is_top_third/is_bottom_third + num_ranked on WNBATeamContext. points-allowed def-rank restricted to SCORING_PROPS.
  • Push voided in accuracy_tracker resolution (excluded, not scored UNDER) + _hit_rate drops pushes from denominator.
  • Deploy gate rewritten: requires holdout_evaluated + abs floor 0.52 + must beat heuristic baseline (proj_delta>0.03 on holdout); killed current_holdout==0.0 auto-deploy/lock-open; deployedHoldoutAccuracy only written when measured.
  • Train crash fixed: effective_holdout guard (min_days<holdout_days) + try/except graceful skip + collect_training_data empty-slice guard.
  • Market edge: _prob_over (per-stat CV normal CDF) replaces clip(proj_delta*2+0.5).
  • Confidence/score reconciled via support=over_score*pred_sign (support<=0 → LOW + score 0).
  • ML override rebuilds non-contradictory reasoning when it flips the pick.
  • [internal detail removed]

--- original audit (line numbers as-of audit; heuristic_generator.py was 674 lines) --- Audit of sports-scout-service/src/batch/plugins/wnba/ on 2026-06-15 (8-dim multi-agent + adversarial verify).

Reprioritizing fact: sportlib WNBA odds_api_markets maps only points/rebounds/assists/threes/pra → bugs in turnovers/steals/blocks/stocks/pr/pa/ra branches are likely DEAD (no odds). Verify vs live odds feed.

CONFIRMED bug cluster (fix order):

  1. CRITICAL push-as-UNDER: accuracy_tracker.py:128 (actual > line else UNDER) — pushes on integer lines counted UNDER, corrupts BOTH accuracy metric AND training labels (poison path → ML override heuristic_generator.py:664-670, the NBA under-bias regression mechanism). Fix: void pushes.
  2. Opponent rebound signal inverted (live: rebounds+pra) heuristic_generator.py:551-559 — reb_rank from opponent's OWN rebounds_pg, boosts OVER vs strongest-rebounding D. _build_rationale:267 narrates correct intent → proves gate inverted.
  3. Opponent assist signal inverted (live: assists+pra) :561-569 — boosts assist OVER vs most-disruptive (high-steal) D. (turnover branch :571-580 also inverted vs its own comment but DEAD — no TO odds.) pra trips both reb+ast blocks.
  4. Deploy gate batch.py:352-353,385-386 — improvement >= -0.003 ships worse models; first model deploys untested (holdout 0.0) then locks deployedHoldoutAccuracy=0.0 → always deploys; no abs floor / no heuristic baseline. Echoes project_scout_model_under_bias.
  5. confidence label vs confidenceScore on independent axes :644-661 (can contradict). Market-edge model_over_prob=clip(proj_delta*2+0.5) :596 wildly overconfident + double-counts projection — see feedback_model_edge_calibration.
  6. train crashes first week: min_days=5 < holdout_days=7 empties train slice prop_model_trainer.py:93-99, no try/except in run_train_nightly.

Live medium: POR/TOR have nba_stats=None in sportlib → dropped from defensive priming (defensive_rank_calculator.py:58-68), zero opponent context (one plays 2026-06-15). Ranks computed over fetched subset w/ absolute >=11/<=5 thresholds; no-stats teams rank #1 D. backfill leak: target-game exclusion uses != game_date not < :311. byPropType always undefined (API reads byPropType, Python emits byStatType) api/plugins/wnba/index.ts:86-103.

Top enhancement: NO minutes feature (minutes fetched but unused in 13-vec) — biggest lift. Then cold-start shrinkage (project_wnba_projections_playbook), opponent stat-ALLOWED rates, calibration/CLV. scripts/wnba_backtest.py exists (walk-forward harness) — verify fixes against it.