diff --git a/data/README.md b/data/README.md new file mode 100644 index 0000000..5485b07 --- /dev/null +++ b/data/README.md @@ -0,0 +1,36 @@ +# Data + +Raw and processed FX data is **gitignored** — never committed. + +## Download: DUKASCopy G10 tick data + +Use the provided fetch task: + +```bash +task data:fetch # downloads raw tick data → data/raw/ +task data:process # resamples to hourly OHLCV + features → data/processed/ +task data:validate # gap detection, outlier report, regime coverage check +``` + +## Expected structure (local only) + +``` +data/ + raw/ G10 pairs, tick OHLCV, 2003–present (gitignored) + processed/ Hourly log-returns, rolling HV, VIX-merged (gitignored) + cache/ Intermediate artefacts (gitignored) + config.yaml Download configuration (committed) +``` + +## Pairs + +EUR/USD, GBP/USD, USD/JPY, USD/CHF, AUD/USD, USD/CAD, NZD/USD, EUR/GBP, EUR/JPY, EUR/CHF + +## Splits + +| Split | Date range | Purpose | +|---|---|---| +| Train | 2008-01-01 – 2022-12-31 | Model training only | +| Held-out test | 2023-01-01 – 2023-12-31 | Final evaluation — do not open until evaluation | + +**Training cutoff is hard: 2023-01-01. No post-2022 data informs any design decision.**