DIME
Get the picksJoin

PROJECT · findings note

project_multi_formula_clv_harness

Multi-formula CLV harness — runs N pick-generation formulas in parallel, tags each pick with formula_id, measures CLV per formula. 5-7 day build. Design doc at grading-service/docs/multi-formula-clv-harness.md.

← All findingssource · memory/project_multi_formula_clv_harness.md

Why this exists

Today the stack has at least 3 independent pick signals (Pinnacle de-vig, system prop_diagnostic.line_edges, scout-service ML) and the operator has to eyeball-pick which one to trust per play. That produces repeating process failures — see feedback_check_line_edges_before_quoting.

The structural fix is to run all formulas in parallel, tag each pick with formula_id, measure CLV per formula over n>=100 picks, and let the data tell us which has real edge. Replaces operator judgment with empirical measurement.

Design doc

grading-service/docs/multi-formula-clv-harness.md — full file-level scope, schema changes, test plan, risks. Ready to hand to an implementation agent.

Status

Two-tier work merged to main + deployed 2026-06-05 (grading-service 96b54fd):

  • Diagnosed why both_agree=2 / disciplined=0: disciplined was never registered in DefaultAutoFormulas; both_agree intersects pinn_devig×system_lineedges but they (i) name 3PM differently — snapshot three_pointers_made vs odds threes → 0 matches (fixed via canonicalPropType), and (ii) are orthogonal signals (price-arb vs projection) so they rarely agree on a side = sparse BY DESIGN. player_id was FINE for NBA (44 by-id == 44 by-name — earlier null-id theory was WNBA-only).
  • Fixed (commit 9dc288a): canonicalPropType, both_agree name-fallback + v1.1.0 bump, registered disciplined (Core tier), added NEW disciplined_board formula = system_lineedges + odds-sanity gate WITHOUT pinn requirement (List tier, higher volume).
  • Shipped (commit 04f4e43): GET /api/v1/picks/nightly?date=&league= — two-tier card (core=disciplined, list=disciplined_board) grouped by game, read straight from bet_clv. Pure grouping fn unit-tested.
  • Also committed the pre-existing uncommitted disciplined display-card + CLV-UI feature (handlers/repository/disciplined.go, disciplined_llm.go, GetCalibration, clv_ui) — HEAD didn't build without it (orphaned-WIP pattern). Flagged "review before merge" in commit 8ab5056.
  • Pending: the next 18:00 UTC fanout populates the disciplined/disciplined_board buckets — until then /picks/nightly returns empty (verified live: valid shape, games:[]).

Earlier note (still true): SUBSTANTIALLY BUILT (verified 2026-05-30). grading-service has the full plugin framework live:

  • internal/formulas/formula.go — Formula interface (ID/Version/Generate→[]Pick); Pick maps 1:1 to bet_clv columns.
  • Reference formulas implemented: pinn_devig.go (de-vig Pinnacle via DevigShins, bet the soft non-Pinnacle book at its price), system_lineedges.go, both_agree.go (intersection of the two on player_id+prop_type+line+side), plus participant slots claude_combined.go, user_manual.go. scout_ml referenced.
  • [internal detail removed]
  • internal/clvcron/cron.go — slate-capture (10:30 UTC), close-capture (every 10 min), settle loops.
  • internal/clvcapture/settle.go — formula-agnostic settlement: settles by bet identity (gameID+playerID+propType+side+line), processes ALL settled_at IS NULL rows regardless of formula_id. So any well-formed bet_clv row gets close-captured + settled.
  • internal/repository/clv_by_formula.go, handlers/clv.go + clv_ui.go — per-formula CLV aggregation + UI.

So both_agree (deterministic Pinnacle-devig ∩ system line_edges) ALREADY EXISTS. The disciplined-card formula (project_disciplined_slate_card_prototype) differs from it only by (a) an LLM adversarial-verify layer (can't run in the Go cron) and (b) the odds-sanity gate (median-consensus, miskey/extreme-juice/book-disagreement kill). Open fork: add a deterministic disciplined Go formula (both_agree + odds-sanity) in-framework, vs inject the LLM-verified picks into bet_clv from the Python pipeline (formula-agnostic settle supports it).

Key facts (for future reference)

  • Foundation exists: bet_clv table in grading-service (migration v8) already has the CLV capture columns. v1 just needs to add formula_id + formula_version and extend UNIQUE constraint.
  • CLVTrustworthyThreshold: n=100 per (formula × league × bucket). Per the migration comment, n=200 was tried first but was too conservative for WNBA's bounded slate volume.
  • v1 formulas (4 to start): pinn_devig, system_lineedges, both_agree, scout_ml (last one gated on scout-service combo-stat fix landing healthy in production).
  • Repo home: grading-service (Go). Plugs into existing clvcapture/capture.go + clvcron/ pipeline.
  • Effort: 5-7 focused days. Single-PR per formula + schema migration
    • handlers + tests.

Why this is the right answer

Per feedback_model_edge_calibration: "CLV over ~100 bets per bucket is the only real validation." Without per-formula CLV breakdown, we have no way to know which signal is real edge vs noise. With it, the question "which formula should we trust?" becomes empirical instead of a daily judgment call.

Out of scope for v1

  • Live arbitrage execution
  • Auto-staking via Kelly
  • Real-time line monitoring
  • Multi-leg correlation modeling
  • Formula auto-retraining

All require the harness to be working + producing trusted CLV first. v2+.

Cross-references

CLV blind spots — audited 2026-08-24

⚠️ Only 5 markets are priced by our books at all. odds-service /api/v1/odds/props/bulk?league=wnba carries exactly: points, rebounds, assists, threes, pra, pr, pa, ra. It does NOT carry minutes, turnovers, steals, blocks, or any MLB prop. Those capture zero closing lines correctly — there is nothing to capture. The dashboard renders that as "0.0% positive CLV", which reads like a measured result rather than a blind spot. ~35k of 78k ledger bets (45%) have no CLV for this reason. Never read 0.0% CLV as a finding without checking n_with_close.

⚠️ Combos ARE priced and we barely bet them — pr/pra/pa/ra all appear in the odds feed (≈50-70 rows each), yet the ledger holds only 8 pra bets. Those 8 captured 8 closes at 87.5% positive CLV. Unexploited surface.

FIXED (grading 9879728): three_pointers_made never matched threes. clvcapture.lookupByIdentity keys on (player, prop_type) by EXACT string. 3,916 bets spelled three_pointers_made captured 0 closes; 1,662 spelled threes captured 88%. Same market, decided by spelling, no error anywhere. Now canonicalPropType() normalises both sides of the key. Only same-market spellings collapse — unpriced markets pass through, since aliasing them would price a bet against the wrong line.