docs: write DECISIONS.md — all post-grill architectural and methodology decisions

DECISIONS.md

@@ -0,0 +1,120 @@
+# DECISIONS.md
+
+Architecture and methodology decisions for `jepa-fx-risk`. Every non-obvious choice lives here with its rationale. Agents: read this before making any design decision.
+
+Last updated: 2026-05-27
+
+---
+
+## Language split: Go-first, Python-minimal
+
+**Decision:** Data pipeline, evaluation harness, CLI, and experiment runner in Go 1.24+. Python 3.12 + PyTorch only for the TS-JEPA model training loop, isolated in `model/`.
+
+**Rationale:** Python's dependency ecosystem is a reliability risk for a multi-year project (conflicting CUDA versions, transitive breakage, environment drift). Go produces a single static binary, has excellent CSV/Parquet support, is fast enough for all non-training workloads, and keeps the reproducible parts of the project dependency-free. The training loop genuinely requires PyTorch — that is the one place Python is unavoidable.
+
+**Boundary:** `model/` is the Python perimeter. Nothing outside it imports Python.
+
+---
+
+## Primary hypothesis
+
+**Decision:** Regime detection via embedding structural separability is the primary hypothesis. Distributional VaR forecasting is secondary and only pursued if Phase 1 and 2 succeed.
+
+**Rationale:** Regime detection is testable in Phase 1 without regulatory-quality outputs. If embeddings don't show regime structure, distributional VaR will also fail. Choosing a primary hypothesis prevents the project from retreating from one to the other on failure.
+
+**Success criterion (Phase 1):** Silhouette score > 0.35 on k-means clusters (k=3–5) vs. realised-volatility regime label (rolling 30-day HV percentile, high/low), computed on held-out test data including at least one structural break.
+
+---
+
+## Training data: 2008–2022, all G10 pairs
+
+**Decision:** Train on 15 years of all G10 FX pairs from DUKASCopy, 2008–2022. Do not start with a single 5-year EUR/USD window.
+
+**Rationale:** A single 5-year window (e.g. 2019–2024) is dominated by one or two regimes and gives the encoder insufficient regime diversity to learn regime-sensitive representations. Training on 2008–2022 ensures the encoder sees: GFC (2008), EUR sovereign debt (2011–2012), SNB cap removal (2015), COVID (2020), USD rate cycle (2022). Multi-pair training also allows cross-currency transfer (Phase 4).
+
+**Alternative considered:** Start simple with EUR/USD only, expand later. Rejected because regime diversity in training is a structural requirement, not a nice-to-have. Retrofitting it in Phase 2 would require retraining from scratch.
+
+---
+
+## Hard training cutoff: 2023-01-01
+
+**Decision:** All data from 2023-01-01 onward is sealed. No architecture, hyperparameter, or methodology decision may be informed by post-2023 data. Post-2023 test set opened only once, for final evaluation.
+
+**Rationale:** Out-of-sample integrity is essential for publishability and honest self-assessment. The held-out window (2023–2026) includes: 2023 US regional bank stress, 2024 JPY intervention episodes. These are the test of genuine generalisation.
+
+**Enforcement:** `data/raw/` is gitignored. The download script hard-stops at 2022-12-31 for training splits. Any deviation requires a DECISIONS.md entry explaining why.
+
+---
+
+## Phase 0: SSL feasibility gate before JEPA work
+
+**Decision:** Before any JEPA-specific implementation, run a Phase 0 experiment: masked autoencoder (MAE) baseline on EUR/USD hourly data. If MAE silhouette < 0.20, SSL-based regime detection is likely not feasible on this data — stop and investigate before proceeding to JEPA.
+
+**Rationale:** JEPA's complexity is only justified if the core SSL premise (that latent representations capture regime structure) holds for FX data. A failed MAE experiment tells us this in 2 weeks rather than 4 months. Added after Full Grill session (2026-05-27).
+
+**Go/no-go threshold:** MAE silhouette < 0.20 on held-out 2023 data → pause, investigate, do not proceed to Phase 1.
+
+---
+
+## Architecture: TS-JEPA as starting implementation
+
+**Decision:** Use TS-JEPA (Ennadir et al., 2025) as the starting JEPA implementation. MTS-JEPA (He et al., 2026) is the upgrade path if multi-resolution proves necessary.
+
+**Rationale:** TS-JEPA is simpler. Validate the concept before adding multi-resolution complexity. If Phase 1 succeeds with TS-JEPA, MTS-JEPA is an ablation, not a prerequisite.
+
+**Risk:** Both are preprints. Code reproducibility is unconfirmed. First task of Phase 0 is reproducing TS-JEPA on the paper's own benchmark — if this takes > 2 weeks, contact authors or fall back to implementing JEPA masking from scratch using V-JEPA codebase as reference.
+
+---
+
+## Input features: minimal for Phase 0/1
+
+**Decision:** Phase 0 and Phase 1 use three features only: log-return (hourly), rolling 20-period realised volatility (hourly), VIX (daily, interpolated to hourly).
+
+**Rationale:** Too many input features in early phases makes it impossible to distinguish "JEPA learned regime structure" from "JEPA learned to encode a feature that correlates with regime." Minimal features reduce confounding.
+
+**Expansion path:** Add DXY, G10 vol surface, yield spreads in Phase 2 if Phase 1 succeeds.
+
+---
+
+## Collapse diagnostic
+
+**Decision:** If PC1 of JEPA embeddings correlates > 0.85 with rolling 30-day HV, treat Phase 1 as a partial failure — the encoder learned volatility level, not regime structure. This is a useful finding but not the hypothesis.
+
+**Rationale:** EUR/USD hourly returns are strongly heteroskedastic. A JEPA encoder trained to predict future embeddings will strongly tend to encode current volatility as its primary latent dimension. This is predictable, not regime-sensitive. The linear probe and PC1 diagnostic together distinguish "learned volatility" from "learned regime."
+
+---
+
+## Linear probe as mandatory Phase 1 exit gate
+
+**Decision:** Train a linear model on frozen JEPA embeddings to predict realised volatility decile. R² < 0.4 → embeddings are not encoding useful risk structure → do not proceed to Phase 2.
+
+**Rationale:** If a simple linear model cannot extract volatility regime from the embeddings, the representations are not useful for risk management purposes regardless of their silhouette score. Interpretability-by-linear-probe is the minimum bar for any downstream use.
+
+---
+
+## Explicit out-of-scope (Phases 1–3)
+
+The following are explicitly out of scope for this research programme and go to a parking lot if they arise:
+
+- Options / derivatives pricing
+- Directional alpha generation
+- Live or paper trading
+- Exotic pairs beyond G10
+- Institutional deployment
+- Real-time inference systems
+
+---
+
+## Experiment spec required before any experiment
+
+**Decision:** Every experiment phase must have a written spec in `specs/` before any code runs. No specless experiments.
+
+**Rationale:** Consistent with spec-driven-dev way of working. Specs force falsifiable hypothesis statement, quantitative acceptance criteria, and explicit null-result protocol before results are known — preventing post-hoc rationalisation.
+
+---
+
+## Null results are results
+
+**Decision:** Null results (hypothesis rejected) are recorded in `results/summaries/` and treated as valid research outputs, not failures to be iterated away silently.
+
+**Rationale:** A confirmed null result (e.g. "SSL cannot find regime structure in FX hourly data") is publishable and scientifically valuable. Iterating hyperparameters until metrics pass without recording the failed attempts is p-hacking. Humble attitude; scientific approach.