generated from mathias/template-go-web
feat(multipair): G10 multi-pair pipeline + USE_MULTIPAIR knob (Option C)
- prepare_hourly.py: parameterize PAIR env var; OUT_DEFAULT per-pair; load_m1_from_zips(pair=)
- prepare_multipair.py: inner-join 5-pair hourly parquets on datetime → wide parquet
cols: datetime, {pair}_ret, {pair}_rv × n_pairs; eurusd_rv = target
- fetch_multipair.py: download GBPUSD/USDJPY/USDCHF/AUDUSD M1 2008-2023 from histdata
- train.py: USE_MULTIPAIR knob (JEPA_USE_MULTIPAIR=1); build() reads multipair parquet
with n_channels = n_pairs × 2; target = eurusd_rv
- Taskfile: data:fetch:multipair, data:prepare:pair, data:prepare:multipair, data:test updated
- 7 new tests in test_multipair.py; 34/35 pass (1 SKIP until multipair parquet built)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+12
-1
@@ -34,9 +34,20 @@ tasks:
|
||||
data:prepare:all:
|
||||
desc: "Build both daily and hourly parquets"
|
||||
deps: [data:prepare:daily, data:prepare:hourly]
|
||||
data:fetch:multipair:
|
||||
desc: "Download G10 M1 data (GBPUSD/USDJPY/USDCHF/AUDUSD) 2008-2023 from histdata"
|
||||
cmds: [.venv/bin/python scripts/fetch_multipair.py]
|
||||
data:prepare:pair:
|
||||
desc: "Build {PAIR}_hourly.parquet from data/raw/{PAIR}/ (e.g. PAIR=gbpusd)"
|
||||
cmds: [PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}]
|
||||
vars:
|
||||
PAIR: '{{default "eurusd" .PAIR}}'
|
||||
data:prepare:multipair:
|
||||
desc: "Merge 5-pair hourly parquets into eurusd_multipair.parquet"
|
||||
cmds: [.venv/bin/python scripts/prepare_multipair.py]
|
||||
data:test:
|
||||
desc: "Run Python data pipeline tests"
|
||||
cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py -v]
|
||||
cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py tests/test_multipair.py -v]
|
||||
|
||||
eval:probe:
|
||||
desc: "Run linear-probe (val_vol_r2) on embeddings from metrics.json"
|
||||
|
||||
Reference in New Issue
Block a user