feat(eval): VaR breach rate metric (#12) + HMM regime detector (#13) — rq-04 prep
CD / Lint / Test / Vet (push) Failing after 2s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped

#12 — VaR_breach_rate_99_oos_regime_cond metric:
- internal/eval/var.go: VaRBreachRate() + kupiecPOF() + LinearProbePredict() (stdlib math only)
- internal/eval/var_test.go: 8 golden tests (zero/all breach, perfect calibration, boundary)
- cmd/eval/main.go: -metric var flag (no-leakage probe → VaR → Kupiec P)
- scripts/var_breach.py: Python equivalent with METRIC_KEY constant (13 TDD tests)
- train.py LOCKED VaR EVAL BLOCK: writes VaR_breach_rate_99_oos_regime_cond + kupiec_p to metrics.json
- Fixed bug: train.py used bare 'os' before import; now uses module-level '_os' consistently

#13 — HMM regime detector + JEPA conditioning seam:
- scripts/prepare_regime.py: GaussianHMM (diag, 3-state) on realized_vol; states sorted by mean vol
  (0=calm, 1=stressed, 2=crisis); deterministic (random_state=42); outputs eurusd_regime.parquet
- tests/test_regime.py: 11 TDD tests (dtype, states, determinism, vol sort, daily fallback)
- train.py: JEPA_ENABLE_REGIME toggle + REGIME CONDITIONING SEAM (concat baseline, agent-editable)
- requirements.txt: hmmlearn>=0.3, scikit-learn>=1.4

78 Python + all Go tests green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-27 10:35:10 +02:00
co-authored by Claude Sonnet 4.6
parent 65a58fcca2
commit 68bf8f15c5
9 changed files with 743 additions and 1 deletions
+67
View File
@@ -0,0 +1,67 @@
"""Parametric 99% VaR breach rate + Kupiec POF p-value.
Used by train.py's LOCKED VaR EVAL BLOCK to write VaR_breach_rate_99_oos_regime_cond
to metrics.json so the autoresearch loop can optimise it.
jepa-fx-risk#12
"""
import math
# Canonical metric key — no surrounding whitespace, as required by the loop contract.
METRIC_KEY = "VaR_breach_rate_99_oos_regime_cond"
# Default normal 99th-percentile z-score.
Z99 = 2.326
def var_breach_rate(pred_vol, actual_vol, z99=Z99):
"""Compute VaR breach rate and Kupiec POF p-value.
Args:
pred_vol: iterable of predicted conditional vol forecasts
actual_vol: iterable of actual realized vol (same length)
z99: 99th-percentile z-score (default 2.326)
Returns:
(breach_rate, kupiec_p) where:
breach_rate — fraction of steps where actual_vol > pred_vol × z99
kupiec_p — Kupiec POF p-value (H0: true breach rate = 1%)
High p-value = well-calibrated; low = miscalibrated tail.
"""
pred_v = list(pred_vol)
act_v = list(actual_vol)
n = len(pred_v)
if n == 0 or n != len(act_v):
return 0.0, 1.0
n1 = sum(1 for p, a in zip(pred_v, act_v) if a > p * z99)
breach_rate = n1 / n
p = kupiec_pvalue(n, n1)
return breach_rate, p
def kupiec_pvalue(n, n1, p0=0.01):
"""Kupiec Proportion-of-Failures likelihood ratio test.
H0: true breach probability = p0.
Returns P(chi²(1) > LR) using the identity P(chi²(1)>x) = erfc(sqrt(x/2)).
Returns 1.0 for n=0 or LR<=0 (well-calibrated / over-conservative).
"""
if n == 0:
return 1.0
n0 = n - n1
phat = n1 / n
if n1 == 0:
# 0 × ln(0/p0) = 0 by convention; only n0 term contributes
lr = 2 * n0 * math.log((1 - phat) / (1 - p0))
elif n1 == n:
lr = 2 * n1 * math.log(phat / p0)
else:
lr = 2 * (n1 * math.log(phat / p0) + n0 * math.log((1 - phat) / (1 - p0)))
if lr <= 0:
return 1.0
# P(chi²(1) > LR) = erfc(sqrt(LR/2))
return math.erfc(math.sqrt(lr / 2))