feat(data): EUR/USD hourly pipeline + 2008-2023 M1 dataset (#2)
CD / Lint / Test / Vet (push) Successful in 4s
CD / Build & Import (push) Failing after 8s
CD / Deploy via GitOps (push) Has been skipped

- scripts/prepare_hourly.py: M1→hourly aggregation (realized_vol = sqrt(Σr²),
  MIN_BARS=30 threshold, no weekend rows, year-based split preserved)
- tests/test_prepare_hourly.py: 5 TDD tests, all green
- train.py: USE_HOURLY=True, WINDOW=240 (10-day), PATCH_LEN=24 (1-day patches);
  build() prefers eurusd_hourly.parquet, falls back to daily; EXPORT BLOCK updated
- Taskfile.yml: data:fetch:historical, data:prepare:hourly, data:prepare:all, data:test
- 98,591 hourly rows (2008-2023) covering GFC, Euro crisis, Brexit, COVID, Fed cycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-25 13:12:48 +02:00
co-authored by Claude Sonnet 4.6
parent bde651b0df
commit e31905dc43
5 changed files with 292 additions and 10 deletions
+13 -3
View File
@@ -92,11 +92,21 @@ def test_jepa_step_end_to_end(train_mod):
assert loss.item() < 100, "loss exploded"
# 6. build() still returns year-based OOS split (2022-2023)
# 6. build() returns year-based OOS split (2022-2023); hourly gives many more windows
def test_build_year_split(train_mod):
(Xtr, ytr), (Xte, yte) = train_mod.build()
assert Xtr.shape[1] == train_mod.WINDOW
assert Xte.shape[1] == train_mod.WINDOW
assert len(Xtr) > 0 and len(Xte) > 0
# OOS set should be ~600 windows (2 years of daily data)
assert 400 < len(Xte) < 900, f"OOS size unexpected: {len(Xte)}"
# OOS: daily ≈ 600; hourly ≈ 17,000 (2 years × ~8,500 trading hours/year)
assert len(Xte) > 400, f"OOS too small: {len(Xte)}"
# 7. hourly build gives > 10× more training windows than daily
def test_build_hourly_more_windows(train_mod):
import os
if not os.path.exists("data/processed/eurusd_hourly.parquet"):
pytest.skip("eurusd_hourly.parquet not present — run data:prepare:hourly first")
(Xtr, _), _ = train_mod.build()
# Daily had ~877 train windows; hourly with 2008-2021 should have > 50,000
assert len(Xtr) > 10_000, f"expected >10k hourly train windows, got {len(Xtr)}"