generated from mathias/template-go-web
feat(data): EUR/USD M1 fetch + daily realized-vol prep (toy slice, #2/#11)
fetch_data.py politely pulls EUR/USD M1 from histdata.com (maintained package handles the anti-hotlink token; per-year, spaced). prepare_data.py (LOCKED per Phase-1 contract) parses M1 -> daily series with realized_vol = the val_vol_r2 target (sqrt sum of squared intraday returns). Verified on 2019-2021: 937 days, March-2020 COVID RV spike 5.1x over 2019 median — real signal, target works. Data gitignored (DVC/MinIO = #10). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,31 @@
|
||||
"""Fetch EUR/USD M1 bars from histdata.com (free, research use).
|
||||
|
||||
Polite: one request per year, spaced; past years query month=None. Uses the
|
||||
maintained `histdata` package which handles histdata's anti-hotlink tk token.
|
||||
Output: data/raw/DAT_ASCII_EURUSD_M1_<year>.zip
|
||||
|
||||
YEARS=2019,2020,2021 python scripts/fetch_data.py
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
|
||||
from histdata import download_hist_data
|
||||
from histdata.api import Platform as P, TimeFrame as T
|
||||
|
||||
YEARS = [y.strip() for y in os.environ.get("YEARS", "2019,2020,2021").split(",")]
|
||||
|
||||
|
||||
def main():
|
||||
os.makedirs("data/raw", exist_ok=True)
|
||||
for yr in YEARS:
|
||||
f = download_hist_data(
|
||||
year=yr, month=None, pair="eurusd",
|
||||
platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
|
||||
output_directory="data/raw",
|
||||
)
|
||||
print("fetched", yr, "->", f)
|
||||
time.sleep(2) # be a good citizen
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user