generated from mathias/template-go-web
feat(data): EUR/USD hourly pipeline + 2008-2023 M1 dataset (#2)
- scripts/prepare_hourly.py: M1→hourly aggregation (realized_vol = sqrt(Σr²), MIN_BARS=30 threshold, no weekend rows, year-based split preserved) - tests/test_prepare_hourly.py: 5 TDD tests, all green - train.py: USE_HOURLY=True, WINDOW=240 (10-day), PATCH_LEN=24 (1-day patches); build() prefers eurusd_hourly.parquet, falls back to daily; EXPORT BLOCK updated - Taskfile.yml: data:fetch:historical, data:prepare:hourly, data:prepare:all, data:test - 98,591 hourly rows (2008-2023) covering GFC, Euro crisis, Brexit, COVID, Fed cycle Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -18,6 +18,26 @@ tasks:
|
||||
deps: [generate]
|
||||
cmds: [go test ./... -race]
|
||||
|
||||
data:fetch:
|
||||
desc: "Download EUR/USD M1 from histdata (set YEARS env var)"
|
||||
cmds: [.venv/bin/python scripts/fetch_data.py]
|
||||
data:fetch:historical:
|
||||
desc: "Download EUR/USD M1 2008-2018 from histdata"
|
||||
cmds:
|
||||
- YEARS=2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018 .venv/bin/python scripts/fetch_data.py
|
||||
data:prepare:daily:
|
||||
desc: "Rebuild eurusd_daily.parquet from all M1 zips"
|
||||
cmds: [.venv/bin/python scripts/prepare_data.py]
|
||||
data:prepare:hourly:
|
||||
desc: "Build eurusd_hourly.parquet from all M1 zips"
|
||||
cmds: [.venv/bin/python scripts/prepare_hourly.py]
|
||||
data:prepare:all:
|
||||
desc: "Build both daily and hourly parquets"
|
||||
deps: [data:prepare:daily, data:prepare:hourly]
|
||||
data:test:
|
||||
desc: "Run Python data pipeline tests"
|
||||
cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py -v]
|
||||
|
||||
eval:probe:
|
||||
desc: "Run linear-probe (val_vol_r2) on embeddings from metrics.json"
|
||||
cmds: [./bin/eval -metric probe]
|
||||
|
||||
Reference in New Issue
Block a user