generated from mathias/template-go-web
mathiasandClaude Sonnet 4.6
68bf8f15c5
#12 — VaR_breach_rate_99_oos_regime_cond metric: - internal/eval/var.go: VaRBreachRate() + kupiecPOF() + LinearProbePredict() (stdlib math only) - internal/eval/var_test.go: 8 golden tests (zero/all breach, perfect calibration, boundary) - cmd/eval/main.go: -metric var flag (no-leakage probe → VaR → Kupiec P) - scripts/var_breach.py: Python equivalent with METRIC_KEY constant (13 TDD tests) - train.py LOCKED VaR EVAL BLOCK: writes VaR_breach_rate_99_oos_regime_cond + kupiec_p to metrics.json - Fixed bug: train.py used bare 'os' before import; now uses module-level '_os' consistently #13 — HMM regime detector + JEPA conditioning seam: - scripts/prepare_regime.py: GaussianHMM (diag, 3-state) on realized_vol; states sorted by mean vol (0=calm, 1=stressed, 2=crisis); deterministic (random_state=42); outputs eurusd_regime.parquet - tests/test_regime.py: 11 TDD tests (dtype, states, determinism, vol sort, daily fallback) - train.py: JEPA_ENABLE_REGIME toggle + REGIME CONDITIONING SEAM (concat baseline, agent-editable) - requirements.txt: hmmlearn>=0.3, scikit-learn>=1.4 78 Python + all Go tests green. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
68 lines
2.0 KiB
Python
68 lines
2.0 KiB
Python
"""Parametric 99% VaR breach rate + Kupiec POF p-value.
|
||
|
||
Used by train.py's LOCKED VaR EVAL BLOCK to write VaR_breach_rate_99_oos_regime_cond
|
||
to metrics.json so the autoresearch loop can optimise it.
|
||
|
||
jepa-fx-risk#12
|
||
"""
|
||
|
||
import math
|
||
|
||
# Canonical metric key — no surrounding whitespace, as required by the loop contract.
|
||
METRIC_KEY = "VaR_breach_rate_99_oos_regime_cond"
|
||
|
||
# Default normal 99th-percentile z-score.
|
||
Z99 = 2.326
|
||
|
||
|
||
def var_breach_rate(pred_vol, actual_vol, z99=Z99):
|
||
"""Compute VaR breach rate and Kupiec POF p-value.
|
||
|
||
Args:
|
||
pred_vol: iterable of predicted conditional vol forecasts
|
||
actual_vol: iterable of actual realized vol (same length)
|
||
z99: 99th-percentile z-score (default 2.326)
|
||
|
||
Returns:
|
||
(breach_rate, kupiec_p) where:
|
||
breach_rate — fraction of steps where actual_vol > pred_vol × z99
|
||
kupiec_p — Kupiec POF p-value (H0: true breach rate = 1%)
|
||
High p-value = well-calibrated; low = miscalibrated tail.
|
||
"""
|
||
pred_v = list(pred_vol)
|
||
act_v = list(actual_vol)
|
||
n = len(pred_v)
|
||
if n == 0 or n != len(act_v):
|
||
return 0.0, 1.0
|
||
|
||
n1 = sum(1 for p, a in zip(pred_v, act_v) if a > p * z99)
|
||
breach_rate = n1 / n
|
||
p = kupiec_pvalue(n, n1)
|
||
return breach_rate, p
|
||
|
||
|
||
def kupiec_pvalue(n, n1, p0=0.01):
|
||
"""Kupiec Proportion-of-Failures likelihood ratio test.
|
||
|
||
H0: true breach probability = p0.
|
||
Returns P(chi²(1) > LR) using the identity P(chi²(1)>x) = erfc(sqrt(x/2)).
|
||
Returns 1.0 for n=0 or LR<=0 (well-calibrated / over-conservative).
|
||
"""
|
||
if n == 0:
|
||
return 1.0
|
||
n0 = n - n1
|
||
phat = n1 / n
|
||
|
||
if n1 == 0:
|
||
# 0 × ln(0/p0) = 0 by convention; only n0 term contributes
|
||
lr = 2 * n0 * math.log((1 - phat) / (1 - p0))
|
||
elif n1 == n:
|
||
lr = 2 * n1 * math.log(phat / p0)
|
||
else:
|
||
lr = 2 * (n1 * math.log(phat / p0) + n0 * math.log((1 - phat) / (1 - p0)))
|
||
|
||
if lr <= 0:
|
||
return 1.0
|
||
# P(chi²(1) > LR) = erfc(sqrt(LR/2))
|
||
return math.erfc(math.sqrt(lr / 2))
|