generated from mathias/template-go-web
feat: project setup — research structure, specs, decisions, conventions #1
@@ -0,0 +1,84 @@
|
||||
# Experiment Spec: Phase 0 — SSL Feasibility Gate
|
||||
|
||||
## Hypothesis
|
||||
|
||||
> We believe that a masked autoencoder (MAE) trained on FX hourly time-series will
|
||||
> produce latent embeddings that show structural separability by volatility regime
|
||||
> without explicit regime labels, measurable by silhouette score > 0.20 on k-means
|
||||
> clusters evaluated against a held-out realised-volatility regime label on 2023 data.
|
||||
|
||||
This hypothesis is FALSE if silhouette score ≤ 0.20 on the held-out evaluation.
|
||||
|
||||
## Background
|
||||
|
||||
Before investing in JEPA-specific machinery, we need to confirm that self-supervised
|
||||
representation learning can find regime structure in FX time-series at all. If the
|
||||
simplest SSL method (MAE) cannot find structure, JEPA will not either — and the root
|
||||
cause needs to be understood before proceeding.
|
||||
|
||||
Also validates that TS-JEPA code is reproducible: first task is running TS-JEPA on
|
||||
the paper's own benchmark, not on FX data. If reproduction takes > 2 weeks, contact
|
||||
authors or fall back to implementing JEPA masking from V-JEPA codebase.
|
||||
|
||||
Added post Full Grill (2026-05-27). See DECISIONS.md: "Phase 0: SSL feasibility gate."
|
||||
|
||||
## Design
|
||||
|
||||
### Data
|
||||
- **Source:** DUKASCopy, EUR/USD hourly OHLCV
|
||||
- **Train:** 2008-01-01 – 2022-12-31
|
||||
- **Held-out test:** 2023-01-01 – 2023-12-31 (sealed until final evaluation)
|
||||
- **Features:** log-return (hourly), rolling 20-period realised HV, VIX (daily → hourly interpolation)
|
||||
- **Regime label (evaluation only):** rolling 30-day HV percentile; binary: top 50% = high-vol, bottom 50% = low-vol
|
||||
|
||||
### Model
|
||||
- **Architecture:** 1D temporal Masked Autoencoder
|
||||
- Encoder: 3-layer 1D CNN + positional encoding
|
||||
- Decoder: 2-layer MLP reconstructing masked segment
|
||||
- **Masking:** contiguous temporal block (target); context window = 120h (5 days)
|
||||
- **Loss:** MSE reconstruction on masked segment
|
||||
|
||||
### TS-JEPA reproduction task (runs in parallel / before MAE)
|
||||
- Reproduce TS-JEPA paper results on the authors' benchmark dataset
|
||||
- Go/no-go: if reproduction fails within 2 weeks → contact authors or pivot to V-JEPA adaptation
|
||||
|
||||
### Baseline
|
||||
- **PCA** on raw feature vectors (same 120h context window, flattened)
|
||||
- Tests whether any dimensionality reduction finds regime structure; confirms SSL is adding something
|
||||
|
||||
### Ablations
|
||||
- Masking horizon K ∈ {8h, 24h, 72h} — does context length affect embedding quality?
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] MAE silhouette score > 0.20 on held-out 2023 data (k-means k=3, vs. binary HV regime label)
|
||||
- [ ] MAE silhouette exceeds PCA baseline silhouette
|
||||
- [ ] Rerun ×3 within ±10% of reported silhouette (reproducibility)
|
||||
- [ ] PC1 / rolling-HV correlation < 0.95 (encoder is not purely encoding volatility level)
|
||||
- [ ] TS-JEPA reproduced on paper's benchmark within 2 weeks of start
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- JEPA implementation (Phase 1)
|
||||
- Multi-pair training (Phase 1+)
|
||||
- VaR, ES, or any risk metric computation
|
||||
- Any data after 2022-12-31 (training); 2023 test set opened only for final evaluation
|
||||
- Hyperparameter search beyond the three masking horizons defined above
|
||||
|
||||
## Null Result Protocol
|
||||
|
||||
If MAE silhouette ≤ 0.20 on held-out data:
|
||||
1. Conclude: SSL-based regime detection is not straightforwardly feasible on EUR/USD hourly data with these three features
|
||||
2. Investigate in order: (a) try daily resolution instead of hourly, (b) expand feature set to 6 features (add DXY, yield spread), (c) try 4-class regime label (HV quartiles) instead of binary
|
||||
3. If all three investigations fail: conclude SSL regime detection is not viable on FX data; document and consider pivoting the primary hypothesis to distributional forecasting directly
|
||||
4. Record result in `results/summaries/phase-0-null.md`
|
||||
5. Write failure mode to brain: `brain_write wing=jepa-fx hall=failures`
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Canary | Mitigation |
|
||||
|---|---|---|
|
||||
| TS-JEPA code unreproducible | Benchmark result doesn't match paper within 2 weeks | Contact authors; fall back to V-JEPA adaptation |
|
||||
| MAE encoder collapses | Reconstruction loss plateau in first 10 epochs; all embeddings near-identical | Add batch normalisation; reduce LR; check masking ratio |
|
||||
| Regime label too coarse | Silhouette low even with visually structured embeddings | Also evaluate with 4-class HV quartile label |
|
||||
| PC1 is just volatility | PC1/HV correlation > 0.95 | Useful finding; record it; do not declare success |
|
||||
Reference in New Issue
Block a user