docs: add data/README.md — download instructions and split definitions

This commit is contained in:
mathias
2026-05-27 21:59:53 +00:00
parent a27200cc5e
commit 4608c8ee29
+36
View File
@@ -0,0 +1,36 @@
# Data
Raw and processed FX data is **gitignored** — never committed.
## Download: DUKASCopy G10 tick data
Use the provided fetch task:
```bash
task data:fetch # downloads raw tick data → data/raw/
task data:process # resamples to hourly OHLCV + features → data/processed/
task data:validate # gap detection, outlier report, regime coverage check
```
## Expected structure (local only)
```
data/
raw/ G10 pairs, tick OHLCV, 2003present (gitignored)
processed/ Hourly log-returns, rolling HV, VIX-merged (gitignored)
cache/ Intermediate artefacts (gitignored)
config.yaml Download configuration (committed)
```
## Pairs
EUR/USD, GBP/USD, USD/JPY, USD/CHF, AUD/USD, USD/CAD, NZD/USD, EUR/GBP, EUR/JPY, EUR/CHF
## Splits
| Split | Date range | Purpose |
|---|---|---|
| Train | 2008-01-01 2022-12-31 | Model training only |
| Held-out test | 2023-01-01 2023-12-31 | Final evaluation — do not open until evaluation |
**Training cutoff is hard: 2023-01-01. No post-2022 data informs any design decision.**