10 Commits
Author SHA1 Message Date
mathiasandClaude Sonnet 4.6 68bf8f15c5 feat(eval): VaR breach rate metric (#12) + HMM regime detector (#13) — rq-04 prep
CD / Lint / Test / Vet (push) Failing after 2s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
#12 — VaR_breach_rate_99_oos_regime_cond metric:
- internal/eval/var.go: VaRBreachRate() + kupiecPOF() + LinearProbePredict() (stdlib math only)
- internal/eval/var_test.go: 8 golden tests (zero/all breach, perfect calibration, boundary)
- cmd/eval/main.go: -metric var flag (no-leakage probe → VaR → Kupiec P)
- scripts/var_breach.py: Python equivalent with METRIC_KEY constant (13 TDD tests)
- train.py LOCKED VaR EVAL BLOCK: writes VaR_breach_rate_99_oos_regime_cond + kupiec_p to metrics.json
- Fixed bug: train.py used bare 'os' before import; now uses module-level '_os' consistently

#13 — HMM regime detector + JEPA conditioning seam:
- scripts/prepare_regime.py: GaussianHMM (diag, 3-state) on realized_vol; states sorted by mean vol
  (0=calm, 1=stressed, 2=crisis); deterministic (random_state=42); outputs eurusd_regime.parquet
- tests/test_regime.py: 11 TDD tests (dtype, states, determinism, vol sort, daily fallback)
- train.py: JEPA_ENABLE_REGIME toggle + REGIME CONDITIONING SEAM (concat baseline, agent-editable)
- requirements.txt: hmmlearn>=0.3, scikit-learn>=1.4

78 Python + all Go tests green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 10:35:10 +02:00
mathiasandClaude Sonnet 4.6 65a58fcca2 feat(loop): add --run-dir isolation, heartbeat, ntfy-on-crash; scaffold start command
CD / Lint / Test / Vet (push) Failing after 1s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
Closes jepa-fx-risk#11 Phase A.

- scripts/autoresearch_start.py: scaffold runs/<rq-id>/ from Council backlog leaf;
  fail-closed on non-autoresearch-ready; strips candidate_metric; writes program.md
  + run.json (provenance) + train.py copy. 19 TDD tests.
- loop.py: --run-dir flag redirects STATUS.md / metrics.json / HEARTBEAT / train.py
  into the run dir; METRICS_OUT env var passed to train subprocess so it writes
  metrics.json to the run dir; heartbeat file written each iter phase; ntfy-on-crash
  via NTFY_URL env var (best-effort).
- train.py: METRICS_OUT env var overrides metrics.json path (default unchanged).

Launch: LITELLM_KEY=xxx python loop.py --run-dir runs/rq-04

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 10:19:17 +02:00
mathiasandClaude Sonnet 4.6 bd8962f997 feat(multipair): G10 multi-pair pipeline + USE_MULTIPAIR knob (Option C)
- prepare_hourly.py: parameterize PAIR env var; OUT_DEFAULT per-pair; load_m1_from_zips(pair=)
- prepare_multipair.py: inner-join 5-pair hourly parquets on datetime → wide parquet
  cols: datetime, {pair}_ret, {pair}_rv × n_pairs; eurusd_rv = target
- fetch_multipair.py: download GBPUSD/USDJPY/USDCHF/AUDUSD M1 2008-2023 from histdata
- train.py: USE_MULTIPAIR knob (JEPA_USE_MULTIPAIR=1); build() reads multipair parquet
  with n_channels = n_pairs × 2; target = eurusd_rv
- Taskfile: data:fetch:multipair, data:prepare:pair, data:prepare:multipair, data:test updated
- 7 new tests in test_multipair.py; 34/35 pass (1 SKIP until multipair parquet built)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 14:00:57 +02:00
mathiasandClaude Sonnet 4.6 b2bc01ba9e feat(phase1): warm-start joint encoder fine-tuning (Option B)
CD / Lint / Test / Vet (push) Successful in 3s
CD / Build & Import (push) Failing after 7s
CD / Deploy via GitOps (push) Has been skipped
Two-phase phase-1:
  1a. Frozen warmup: head trains on pre-computed embeddings for PHASE1_EPOCHS=200
  1b. Joint fine-tune: encoder + head for PHASE1_JOINT_EPOCHS=30 at PHASE1_ENCODER_LR=3e-6

Key design decisions:
- Warm start prevents catastrophic forgetting (PHASE1_JOINT=1 cold-start → -32 R²)
- Normalize live encoder output with FROZEN stats (mu_e/sd_e) so head sees same
  embedding distribution it was warmed up on
- head LR reduced 10× in joint phase to prevent head from racing ahead

HPO sweep: 30ep@3e-6=0.3962, 30ep@1e-5=0.3930, 50ep@3e-6=0.3923
Baseline (frozen): 0.3908. New best: phase1_r2=0.3962 (+0.0054 OOS).

New knobs: JEPA_PHASE1_JOINT (default 1), JEPA_PHASE1_JOINT_EPOCHS (default 30),
JEPA_PHASE1_ENCODER_LR (default 3e-6). 4 new tests (tests 15-18). 28/28 pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 13:25:09 +02:00
mathiasandClaude Sonnet 4.6 de19bfeada fix(features): revert to 2-channel default; OHLCV features redundant
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 7s
CD / Deploy via GitOps (push) Has been skipped
HPO finding: hl_range≈realized_vol, ret_intrabar≈ret — correlation kills signal.
4ch D=128: 0.3503, 4ch D=256: 0.3807, 2ch D=128 baseline: 0.3908 (winner).
Parquet keeps hl_range+ret_intrabar; comment in build() documents the attempt.
test_build_uses_4_channels → test_build_uses_2_channels (tracks current default).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 13:14:01 +02:00
mathiasandClaude Sonnet 4.6 caccd1aa7b feat(features): add hl_range + ret_intrabar OHLCV features (4-channel input)
- prepare_hourly.py: keep O/H/L columns from M1 zips; compute per-hour
  hl_range=log(H/L) and ret_intrabar=log(close/open); backward-compat
  (falls back to 4-col output only when O/H/L present in input)
- train.py build(): auto-detect extra features from parquet columns
  (FEAT_COLS = [ret, realized_vol] + [hl_range, ret_intrabar] if present)
- 5 new tests (9 total in test_prepare_hourly); 24/24 pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 13:11:59 +02:00
mathiasandClaude Sonnet 4.6 e739f84afd feat(hpo): env-var knob overrides + sweep script (18 configs)
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 8s
CD / Deploy via GitOps (push) Has been skipped
- train.py knobs all readable from JEPA_* env vars (JEPA_WINDOW, JEPA_D_MODEL,
  JEPA_DEPTH, etc.) so hpo_sweep.py can override without touching source
- scripts/hpo_sweep.py: 3×2×3 grid over D_MODEL × DEPTH × WINDOW,
  logs to results/hpo/hpo_results.jsonl with leaderboard at end
- 3 new tests: env override correctness, configs() schema validation
- 19/19 tests pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 12:37:27 +02:00
mathiasandClaude Sonnet 4.6 d282571c96 feat(phase1): MLP supervised head on frozen HEPA embeddings
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 7s
CD / Deploy via GitOps (push) Has been skipped
SupervisedHead: Linear(D→D/2)→GELU→Linear(D/2→1), trained on standardised
targets with proper epoch iteration (not random 200 batches) + weight_decay=1e-4.
Root cause of earlier -803 R²: unstandardised targets + ~1.2 effective passes.

Results on 2008-2023 hourly OOS (n=11,641):
  val_vol_r2 (linear probe): 0.3585
  phase1_r2  (MLP head):     0.3737  (+0.015 over probe)

New knobs: PHASE1_EPOCHS=200, PHASE1_LR=1e-3. 16/16 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-26 12:33:28 +02:00
mathiasandClaude Sonnet 4.6 e31905dc43 feat(data): EUR/USD hourly pipeline + 2008-2023 M1 dataset (#2)
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 8s
CD / Deploy via GitOps (push) Has been skipped
- scripts/prepare_hourly.py: M1→hourly aggregation (realized_vol = sqrt(Σr²),
  MIN_BARS=30 threshold, no weekend rows, year-based split preserved)
- tests/test_prepare_hourly.py: 5 TDD tests, all green
- train.py: USE_HOURLY=True, WINDOW=240 (10-day), PATCH_LEN=24 (1-day patches);
  build() prefers eurusd_hourly.parquet, falls back to daily; EXPORT BLOCK updated
- Taskfile.yml: data:fetch:historical, data:prepare:hourly, data:prepare:all, data:test
- 98,591 hourly rows (2008-2023) covering GFC, Euro crisis, Brexit, COVID, Fed cycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 13:12:48 +02:00
mathiasandClaude Sonnet 4.6 bde651b0df feat(backbone): replace TS-JEPA+SIGReg with HEPA causal JEPA
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 8s
CD / Deploy via GitOps (push) Has been skipped
HEPA (Petersen et al., arXiv:2605.11130, ICML 2026 Spotlight):
- CausalEncoder: non-overlapping patches + per-patch LayerNorm +
  causal Transformer (generate_square_subsequent_mask) → all tokens (B, N, D)
- HorizonPredictor: MLP(cat(h_t, Δt)) → predicted future embedding;
  Δt sampled uniformly from [1, min(DELTA_T_MAX, N-1-c)] per epoch
- vicreg_loss: (1-α)·L1(norm(ĥ), norm(h*)) + α·(L_var + L_cov);
  joint training — no stop-gradient on target encoder
- Probe: last-token embedding [:, -1, :], fit on 2019-2021, eval on OOS

Results (true OOS 2022-2023):
  val_vol_r2: -0.45 (TS-JEPA+SIGReg) → +0.243/+0.276 (HEPA)
  effective_rank: 58.9/64 → 122.3/128 (near-full-rank, no collapse)
  Phase-0 gate on val_vol_r2: PASS ✓

Tests: 6/6 green (causal masking verified with non-uniform perturbation;
per-patch LayerNorm is mean-invariant so constant shifts are absorbed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-25 08:05:33 +02:00