experiment: Phase 0 — MAE baseline on EUR/USD (SSL feasibility gate) #5

Closed
opened 2026-05-27 22:01:44 +00:00 by mathias · 2 comments
Owner

What

Implement the Phase 0 MAE (Masked Autoencoder) baseline in Python, train it on EUR/USD hourly 2008–2022, and evaluate on held-out 2023 data using the Go evaluation harness.

This is the first actual experiment — spec is already written

See: specs/phase-0-ssl-feasibility.md

Steps

  1. Implement 1D temporal MAE in model/ (3-layer CNN encoder, 2-layer MLP decoder)
  2. Train on EUR/USD hourly 2008–2022, masking horizon ablation K ∈ {8h, 24h, 72h}
  3. Export embeddings for held-out 2023 data to experiments/RUNID/embeddings/
  4. Run Go eval harness: task eval:silhouette, task eval:probe, task eval:collapse
  5. Record result in results/summaries/phase-0-[pass|null].md
  6. Git tag: exp/YYYYMMDD-phase-0-mae-baseline

Acceptance criteria (from spec)

  • Silhouette > 0.20 on 2023 held-out (k=3, binary HV label)
  • MAE silhouette > PCA baseline
  • Rerun ×3 within ±10%
  • PC1/HV correlation < 0.95

Prerequisites

  • Issue #2 (data pipeline) complete
  • Issue #3 (TS-JEPA reproduction) complete or time-boxed
  • Issue #4 (eval harness) complete

Null result protocol

If criteria not met, follow specs/phase-0-ssl-feasibility.md null result protocol exactly. Do not iterate silently — record and escalate.

## What Implement the Phase 0 MAE (Masked Autoencoder) baseline in Python, train it on EUR/USD hourly 2008–2022, and evaluate on held-out 2023 data using the Go evaluation harness. ## This is the first actual experiment — spec is already written See: `specs/phase-0-ssl-feasibility.md` ## Steps 1. Implement 1D temporal MAE in `model/` (3-layer CNN encoder, 2-layer MLP decoder) 2. Train on EUR/USD hourly 2008–2022, masking horizon ablation K ∈ {8h, 24h, 72h} 3. Export embeddings for held-out 2023 data to `experiments/RUNID/embeddings/` 4. Run Go eval harness: `task eval:silhouette`, `task eval:probe`, `task eval:collapse` 5. Record result in `results/summaries/phase-0-[pass|null].md` 6. Git tag: `exp/YYYYMMDD-phase-0-mae-baseline` ## Acceptance criteria (from spec) - [ ] Silhouette > 0.20 on 2023 held-out (k=3, binary HV label) - [ ] MAE silhouette > PCA baseline - [ ] Rerun ×3 within ±10% - [ ] PC1/HV correlation < 0.95 ## Prerequisites - Issue #2 (data pipeline) complete - Issue #3 (TS-JEPA reproduction) complete or time-boxed - Issue #4 (eval harness) complete ## Null result protocol If criteria not met, follow `specs/phase-0-ssl-feasibility.md` null result protocol exactly. Do not iterate silently — record and escalate.
Author
Owner

Path B proxy gate result: NULL (2026-06-24)

Ran the gate on daily 2019-2023 EUR/USD (path B deviation — not hourly 2008-2022 + Go harness). Full results in results/summaries/phase-0-null.md.

Criterion Result Gate
TS-JEPA silhouette (3 seeds) 0.018 > 0.20 → FAIL
Beats PCA (0.136) 0.018 > PCA → FAIL
Seed stability 87% spread < 10% → FAIL
PC1/HV correlation 0.526 < 0.95 → pass

Sensitivity check: 2000 epochs + D_MODEL=64 worsened to 0.004 — not a training-time issue.

Root cause

Two daily features (ret, realized_vol) carry weak regime structure at daily resolution. More critically: SIGReg pushes embeddings toward isotropic Gaussian — this is correct for downstream regression (val_vol_r2 > 0) but actively resists the clustering structure that silhouette measures. The gate metric and the SIGReg objective are in tension.

What this means

The path B proxy gate is inconclusive, not a verdict on TS-JEPA for FX. The real gate needs:

  1. #4 (Go eval harness) — silhouette on proper eval pipeline
  2. #2 (hourly data, more features) — daily 2 features has no resolution
  3. Potentially: reconsider the gate metric (silhouette vs SIGReg-trained embeddings may always conflict — a linear probe gate might be more appropriate)

val_vol_r2 > 0 and improving (loop iterations) is the more honest feasibility signal for this backbone.

Next step

Leaving #5 open. Proceed to #4 (Go eval harness) as the prerequisite that unlocks a meaningful gate run.

## Path B proxy gate result: NULL (2026-06-24) Ran the gate on daily 2019-2023 EUR/USD (path B deviation — not hourly 2008-2022 + Go harness). Full results in `results/summaries/phase-0-null.md`. | Criterion | Result | Gate | |-----------|--------|------| | TS-JEPA silhouette (3 seeds) | 0.018 | > 0.20 → **FAIL** | | Beats PCA (0.136) | 0.018 | > PCA → **FAIL** | | Seed stability | 87% spread | < 10% → **FAIL** | | PC1/HV correlation | 0.526 | < 0.95 → pass | Sensitivity check: 2000 epochs + D_MODEL=64 worsened to 0.004 — **not a training-time issue**. ### Root cause Two daily features (ret, realized_vol) carry weak regime structure at daily resolution. More critically: SIGReg pushes embeddings toward isotropic Gaussian — this is correct for downstream regression (val_vol_r2 > 0) but actively resists the clustering structure that silhouette measures. The gate metric and the SIGReg objective are in tension. ### What this means The path B proxy gate is **inconclusive**, not a verdict on TS-JEPA for FX. The real gate needs: 1. **#4** (Go eval harness) — silhouette on proper eval pipeline 2. **#2** (hourly data, more features) — daily 2 features has no resolution 3. Potentially: reconsider the gate metric (silhouette vs SIGReg-trained embeddings may always conflict — a linear probe gate might be more appropriate) val_vol_r2 > 0 and improving (loop iterations) is the more honest feasibility signal for this backbone. ### Next step Leaving #5 open. Proceed to #4 (Go eval harness) as the prerequisite that unlocks a meaningful gate run.
Author
Owner

Phase-0 gate — v0.4.0 — INCONCLUSIVE

Date: 2026-06-24
Data: EUR/USD daily 2019-2023 (histdata M1 → daily RV), year-based split (train ≤2021, OOS ≥2022)

What was built

  • Go eval harness (cmd/eval) wired to train.py via EXPORT_EMBEDDINGS=1
  • embeddings.json exports both train (877 windows) and OOS (623 windows, 2022-2023) frozen embeddings
  • LinearProbeTrainTest: fits ridge on train embeddings, evaluates on OOS — no leakage
  • standardiseCompute / applyStandardise: train-stats-based normalisation applied to both sets

Metric bug discovered and fixed

Previous loop metric (val_vol_r2=0.18) was from a mixed-period 70/30 split of all 2019-2023 data — train.py was evaluating on some 2022-2023 windows while calling it "OOS". Fixed: build() now uses year-based split; the probe trains on 2019-2021 embeddings, evaluates on 2022-2023.

Gate results

Metric Value Gate criterion Pass?
val_vol_r2 (true OOS, Go harness) -0.36 > 0
val_vol_r2 (train.py internal) -0.45 > 0
silhouette (OOS, binary HV label) 0.043 > 0.20
effective rank 58.9 / 64 not collapsed

Root cause

TS-JEPA+SIGReg produces non-collapsed embeddings (effective_rank=58.9/64 — SIGReg is working) but the representations do not generalize across the 2021→2022 regime boundary. The 2022 period (Ukraine invasion, Fed rate hike cycle) is a genuinely out-of-distribution regime relative to 2019-2021 training data.

SIGReg pushes embeddings toward isotropic Gaussian — which is good for preventing collapse but provides no mechanism for regime-invariant feature learning.

Verdict: INCONCLUSIVE (gate not passed)

The current backbone doesn't demonstrate SSL feasibility for cross-regime FX vol prediction. Gate criterion should remain val_vol_r2 > 0 on true year-based OOS.

Next options

  1. HEPA backbone (#14) — designed for critical-event regimes including volatility; directly addresses the regime-generalization gap
  2. DUKASCopy hourly data (#2) — 2008-2022 gives 3 full regime cycles for training; more signal for cross-regime generalization
  3. Modified objective — contrastive loss or regime-invariant pretraining instead of standard masked prediction

The autoresearch loop is now correctly configured (true OOS metric); further iterations can search for architectures that cross the gate.

## Phase-0 gate — v0.4.0 — INCONCLUSIVE **Date:** 2026-06-24 **Data:** EUR/USD daily 2019-2023 (histdata M1 → daily RV), year-based split (train ≤2021, OOS ≥2022) ### What was built - Go eval harness (`cmd/eval`) wired to `train.py` via `EXPORT_EMBEDDINGS=1` - `embeddings.json` exports both train (877 windows) and OOS (623 windows, 2022-2023) frozen embeddings - `LinearProbeTrainTest`: fits ridge on train embeddings, evaluates on OOS — no leakage - `standardiseCompute` / `applyStandardise`: train-stats-based normalisation applied to both sets ### Metric bug discovered and fixed Previous loop metric (val_vol_r2=0.18) was from a **mixed-period 70/30 split** of all 2019-2023 data — train.py was evaluating on some 2022-2023 windows while calling it "OOS". Fixed: `build()` now uses year-based split; the probe trains on 2019-2021 embeddings, evaluates on 2022-2023. ### Gate results | Metric | Value | Gate criterion | Pass? | |--------|-------|---------------|-------| | val_vol_r2 (true OOS, Go harness) | -0.36 | > 0 | ❌ | | val_vol_r2 (train.py internal) | -0.45 | > 0 | ❌ | | silhouette (OOS, binary HV label) | 0.043 | > 0.20 | ❌ | | effective rank | 58.9 / 64 | not collapsed | ✅ | ### Root cause TS-JEPA+SIGReg produces non-collapsed embeddings (effective_rank=58.9/64 — SIGReg is working) but the representations **do not generalize across the 2021→2022 regime boundary**. The 2022 period (Ukraine invasion, Fed rate hike cycle) is a genuinely out-of-distribution regime relative to 2019-2021 training data. SIGReg pushes embeddings toward isotropic Gaussian — which is good for preventing collapse but provides no mechanism for regime-invariant feature learning. ### Verdict: INCONCLUSIVE (gate not passed) The current backbone doesn't demonstrate SSL feasibility for cross-regime FX vol prediction. Gate criterion should remain val_vol_r2 > 0 on true year-based OOS. ### Next options 1. **HEPA backbone (#14)** — designed for critical-event regimes including volatility; directly addresses the regime-generalization gap 2. **DUKASCopy hourly data (#2)** — 2008-2022 gives 3 full regime cycles for training; more signal for cross-regime generalization 3. **Modified objective** — contrastive loss or regime-invariant pretraining instead of standard masked prediction The autoresearch loop is now correctly configured (true OOS metric); further iterations can search for architectures that cross the gate.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/jepa-fx-risk#5