generated from mathias/template-go-web
Compare commits
38
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 mathias
|
2c9bc1be31 | ||
|
mathias
|
308f71b566 | ||
 mathias
|
6fb77f5263 | ||
|
mathias
|
8eb0358c01 | ||
|
mathias
|
2d412790bf | ||
|
mathias
|
8539ec3a85 | ||
|
mathias
|
c9b22db3b8 | ||
|
mathias
|
731673061e | ||
|
mathiasandClaude Sonnet 4.6
|
54b1bc216d | ||
|
mathiasandClaude Sonnet 4.6
|
fa6dcaae0a | ||
|
mathiasandClaude Sonnet 4.6
|
68bf8f15c5 | ||
|
mathiasandClaude Sonnet 4.6
|
65a58fcca2 | ||
|
mathiasandClaude Sonnet 4.6
|
d4b67943fb | ||
|
mathiasandClaude Sonnet 4.6
|
bd8962f997 | ||
|
mathiasandClaude Sonnet 4.6
|
b2bc01ba9e | ||
|
mathiasandClaude Sonnet 4.6
|
de19bfeada | ||
|
mathiasandClaude Sonnet 4.6
|
caccd1aa7b | ||
|
mathiasandClaude Sonnet 4.6
|
e635a641a4 | ||
|
mathiasandClaude Sonnet 4.6
|
48c7e3bd02 | ||
|
mathiasandClaude Sonnet 4.6
|
3aded95ef1 | ||
|
mathiasandClaude Sonnet 4.6
|
e739f84afd | ||
|
mathiasandClaude Sonnet 4.6
|
d282571c96 | ||
|
mathiasandClaude Sonnet 4.6
|
1a17a4c88e | ||
|
mathiasandClaude Sonnet 4.6
|
fa6d6c634a | ||
|
mathiasandClaude Sonnet 4.6
|
e31905dc43 | ||
|
mathiasandClaude Sonnet 4.6
|
bde651b0df | ||
|
mathiasandClaude Sonnet 4.6
|
20aeecb971 | ||
|
mathiasandClaude Sonnet 4.6
|
e11e7d2524 | ||
|
mathiasandClaude Sonnet 4.6
|
f01bdde7c2 | ||
|
mathiasandClaude Sonnet 4.6
|
3445b6d267 | ||
|
mathiasandClaude Sonnet 4.6
|
7d04423d39 | ||
|
mathiasandClaude Sonnet 4.6
|
44e8b3eb95 | ||
|
mathiasandClaude Sonnet 4.6
|
f5ce8d6706 | ||
|
mathiasandClaude Sonnet 4.6
|
ed85dc4a8c | ||
|
mathiasandClaude Sonnet 4.6
|
69784f59cf | ||
|
mathiasandClaude Opus 4.8
|
485fdaa9f9 | ||
|
mathiasandClaude Opus 4.8
|
df910e4336 | ||
|
mathias
|
e616575979 |
+97
-6
@@ -1,13 +1,104 @@
|
||||
# hostexecutor
|
||||
# jepa-fx-risk
|
||||
|
||||
## Identity
|
||||
|
||||
- **Name**: hostexecutor
|
||||
- **Name**: jepa-fx-risk
|
||||
- **Owner**: Mathias
|
||||
- **Client**: personal
|
||||
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||
- **Status**: active
|
||||
- **Client**: personal research
|
||||
- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
|
||||
- **Status**: active — Phase 0
|
||||
|
||||
## Purpose
|
||||
|
||||
Research project: apply JEPA-based self-supervised representation learning to
|
||||
FX risk management for a corporate bank with an internal global FX trading desk.
|
||||
|
||||
Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
|
||||
Target output: internal PoC for the trading desk.
|
||||
|
||||
## Architecture decision (see DECISIONS.md ADR-001)
|
||||
|
||||
**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
|
||||
arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
|
||||
(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
|
||||
|
||||
Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
|
||||
|
||||
## Stack
|
||||
|
||||
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||
**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
|
||||
harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
|
||||
results dashboard (Templ + HTMX + CDN Tailwind).
|
||||
|
||||
**Python** (`model/`): all training and embedding export only. PyTorch cu130
|
||||
(Blackwell sm_120 compatible). No evaluation logic in Python.
|
||||
|
||||
**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
|
||||
iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
jepa-fx-risk/
|
||||
├── src/ # Go — data pipeline + eval harness + dashboard
|
||||
│ ├── data/ # DUKASCopy fetch, hourly processing, validation
|
||||
│ └── eval/ # silhouette, linear probe, collapse, backtest
|
||||
├── model/ # Python — training only
|
||||
│ ├── train.py # TS-JEPA + SIGReg backbone (autoresearch edits this)
|
||||
│ ├── prepare.py # LOCKED — data loading, tokenization, export
|
||||
│ └── requirements.txt
|
||||
├── specs/ # Research specs (one per phase/experiment type)
|
||||
├── experiments/ # Per-run outputs: embeddings, metrics.json, git tag
|
||||
├── results/summaries/ # Human-readable outcome per experiment
|
||||
├── program.md # Autoresearch agenda — researcher edits this
|
||||
├── DECISIONS.md # Architecture Decision Records
|
||||
└── Taskfile.yml # task data:fetch, task experiment:run, task eval:*
|
||||
```
|
||||
|
||||
## Phase structure
|
||||
|
||||
- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
|
||||
Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
|
||||
- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
|
||||
Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
|
||||
- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
|
||||
- **Phase 3**: Internal bank tick/position data (future, out of current scope).
|
||||
|
||||
## Data
|
||||
|
||||
DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
|
||||
Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
|
||||
Weekend gaps handled explicitly. See ADR-002.
|
||||
|
||||
## Key conventions
|
||||
|
||||
- `prepare.py` is LOCKED — never modified by agents or autoresearch
|
||||
- Evaluation metrics are always computed by the Go harness, never in Python
|
||||
- Every experiment gets a git tag: `exp/YYYYMMDD-description`
|
||||
- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
|
||||
- `program.md` is the only file the researcher edits to steer autoresearch
|
||||
- CI runs `task check` (lint + vet + test) only — no GPU, no training
|
||||
|
||||
## Evaluation metrics
|
||||
|
||||
Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
|
||||
realized volatility from frozen embeddings, computed by Go harness.
|
||||
|
||||
Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
|
||||
Must co-move with val_vol_r2 — checked from experiment 1.
|
||||
|
||||
Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
|
||||
|
||||
## Benchmarks to beat (trading desk comparison)
|
||||
|
||||
- GARCH(1,1) — volatility forecasting baseline
|
||||
- Historical Simulation VaR (250-day rolling) — Basel default
|
||||
- EWMA RiskMetrics (λ=0.94)
|
||||
|
||||
## Agent guidance
|
||||
|
||||
Read `DECISIONS.md` before making architecture suggestions.
|
||||
Do not modify `prepare.py` or `src/eval/`.
|
||||
Do not suggest changing the Python/Go separation.
|
||||
Training runs are always manual via `task experiment:run` — never triggered by CI.
|
||||
When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
name: Autoresearch Loop
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
fixture:
|
||||
description: 'Fixture name in fixtures/ (without .json)'
|
||||
required: true
|
||||
default: 'phase-a-toy'
|
||||
rq_id:
|
||||
description: 'Run ID — defaults to fixture name if blank'
|
||||
required: false
|
||||
default: ''
|
||||
iters:
|
||||
description: 'Max iterations'
|
||||
required: false
|
||||
default: '3'
|
||||
model:
|
||||
description: 'LiteLLM model override (leave blank for default berget/gemma4-31b)'
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
jobs:
|
||||
run:
|
||||
name: Autoresearch — ${{ inputs.fixture }}
|
||||
runs-on: self-hosted
|
||||
timeout-minutes: 90
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Resolve run ID
|
||||
id: vars
|
||||
run: |
|
||||
RQ_ID="${{ inputs.rq_id }}"
|
||||
[ -z "$RQ_ID" ] && RQ_ID="${{ inputs.fixture }}"
|
||||
echo "rq_id=$RQ_ID" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Clean stale run dir
|
||||
run: rm -rf "runs/${{ steps.vars.outputs.rq_id }}"
|
||||
|
||||
- name: Set up Python venv
|
||||
run: |
|
||||
[ -d .venv ] || python3 -m venv .venv
|
||||
# torch must come from the cu130 wheel index (koala Blackwell sm_120);
|
||||
# requirements.txt deliberately excludes it. Install it first.
|
||||
.venv/bin/pip install -q torch --index-url https://download.pytorch.org/whl/cu130
|
||||
.venv/bin/pip install -q -r requirements.txt
|
||||
|
||||
- name: Scaffold run dir
|
||||
run: |
|
||||
.venv/bin/python scripts/autoresearch_start.py \
|
||||
"fixtures/${{ inputs.fixture }}.json" \
|
||||
"${{ steps.vars.outputs.rq_id }}"
|
||||
|
||||
- name: Run autoresearch loop
|
||||
env:
|
||||
LITELLM_KEY: ${{ secrets.LITELLM_KEY }}
|
||||
LITELLM_BASE: ${{ secrets.LITELLM_BASE }}
|
||||
NTFY_URL: ${{ secrets.NTFY_URL }}
|
||||
run: |
|
||||
ARGS="--run-dir runs/${{ steps.vars.outputs.rq_id }} --iters ${{ inputs.iters }}"
|
||||
[ -n "${{ inputs.model }}" ] && ARGS="$ARGS --model ${{ inputs.model }}"
|
||||
.venv/bin/python loop.py $ARGS
|
||||
|
||||
- name: Upload run artifacts
|
||||
if: always()
|
||||
uses: https://gitea.com/actions/upload-artifact@v3
|
||||
with:
|
||||
name: run-${{ steps.vars.outputs.rq_id }}-${{ github.run_number }}
|
||||
path: |
|
||||
runs/${{ steps.vars.outputs.rq_id }}/STATUS.md
|
||||
runs/${{ steps.vars.outputs.rq_id }}/metrics.json
|
||||
runs/${{ steps.vars.outputs.rq_id }}/program.md
|
||||
retention-days: 30
|
||||
@@ -7,9 +7,6 @@ on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
env:
|
||||
IMAGE: hostexecutor
|
||||
|
||||
jobs:
|
||||
check:
|
||||
name: Lint / Test / Vet
|
||||
@@ -31,85 +28,3 @@ jobs:
|
||||
|
||||
- name: Run checks
|
||||
run: task check
|
||||
|
||||
build:
|
||||
name: Build & Import
|
||||
needs: check
|
||||
runs-on: self-hosted
|
||||
if: github.event_name != 'pull_request'
|
||||
outputs:
|
||||
image-tag: ${{ steps.meta.outputs.sha-tag }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Derive image tags
|
||||
id: meta
|
||||
run: |
|
||||
SHA=$(git rev-parse --short HEAD)
|
||||
echo "sha-tag=${SHA}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Build and push to local registry
|
||||
run: |
|
||||
REGISTRY="localhost:5000"
|
||||
REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
|
||||
buildah build \
|
||||
--label "org.opencontainers.image.revision=${{ github.sha }}" \
|
||||
-t ${REF} \
|
||||
-t ${REGISTRY}/${{ env.IMAGE }}:latest \
|
||||
.
|
||||
buildah push --tls-verify=false ${REF}
|
||||
buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
|
||||
echo "✓ Image pushed to ${REF}"
|
||||
|
||||
deploy:
|
||||
name: Deploy via GitOps
|
||||
needs: build
|
||||
runs-on: self-hosted
|
||||
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||
steps:
|
||||
- name: Update image tag in infra repo
|
||||
env:
|
||||
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
|
||||
DEPLOY_KEY: ${{ secrets.INFRA_DEPLOY_KEY }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p ~/.ssh
|
||||
echo "$DEPLOY_KEY" > ~/.ssh/id_infra
|
||||
chmod 600 ~/.ssh/id_infra
|
||||
ssh-keyscan -p 30022 10.0.1.20 >> ~/.ssh/known_hosts 2>/dev/null
|
||||
export GIT_SSH_COMMAND="ssh -i ~/.ssh/id_infra -o IdentitiesOnly=yes"
|
||||
rm -rf /tmp/infra
|
||||
git clone -b main ssh://git@10.0.1.20:30022/mathias/infra.git /tmp/infra
|
||||
cd /tmp/infra
|
||||
DEPLOYMENT="k3s/apps/hostexecutor/deployment.yaml"
|
||||
sed -i "s|image: localhost:5000/hostexecutor:.*|image: localhost:5000/hostexecutor:${IMAGE_TAG}|" "$DEPLOYMENT"
|
||||
grep -q "localhost:5000/hostexecutor:${IMAGE_TAG}" "$DEPLOYMENT" \
|
||||
|| { echo "✗ image tag patch failed"; exit 1; }
|
||||
if git diff --quiet "$DEPLOYMENT"; then
|
||||
echo "ℹ image tag unchanged — skipping push"
|
||||
else
|
||||
git -c user.name="hostexecutor CI" \
|
||||
-c user.email="ci@hostexecutor.local" \
|
||||
commit -m "chore(deploy): hostexecutor → ${IMAGE_TAG}" "$DEPLOYMENT"
|
||||
git push origin main
|
||||
echo "✓ pushed to infra repo"
|
||||
fi
|
||||
shred -u ~/.ssh/id_infra
|
||||
|
||||
- name: Trigger Flux reconcile
|
||||
run: |
|
||||
kubectl -n flux-system annotate gitrepository flux-system \
|
||||
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
|
||||
kubectl -n flux-system annotate kustomization apps \
|
||||
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
|
||||
|
||||
- name: Verify rollout
|
||||
run: |
|
||||
kubectl rollout status deployment/hostexecutor \
|
||||
--namespace hostexecutor \
|
||||
--timeout=120s \
|
||||
|| {
|
||||
kubectl get pods -n hostexecutor -o wide
|
||||
kubectl get events -n hostexecutor --sort-by='.lastTimestamp' | tail -20
|
||||
exit 1
|
||||
}
|
||||
|
||||
+23
@@ -28,3 +28,26 @@ go.work.sum
|
||||
# Project-specific
|
||||
bin/
|
||||
*.templ.go
|
||||
|
||||
# python venv (autoresearch loop)
|
||||
.venv/
|
||||
|
||||
# downloaded + processed market data (track via DVC/MinIO, #10 — not git)
|
||||
data/
|
||||
|
||||
# autoresearch run dirs (ephemeral; each scaffold rebuilds from fixtures/)
|
||||
runs/
|
||||
|
||||
# ephemeral experiment outputs (generated by train.py / loop.py)
|
||||
metrics.json
|
||||
embeddings.json
|
||||
HEARTBEAT
|
||||
STATUS.md
|
||||
|
||||
# python caches
|
||||
__pycache__/
|
||||
*.pyc
|
||||
.pytest_cache/
|
||||
|
||||
# built Go binaries
|
||||
eval
|
||||
|
||||
+215
@@ -0,0 +1,215 @@
|
||||
# Architecture Decision Records
|
||||
|
||||
This file records significant technical and research decisions for `jepa-fx-risk`.
|
||||
Each record is immutable once merged — append new records rather than editing old ones.
|
||||
Format: ID · Date · Status · Context · Decision · Rationale · Consequences.
|
||||
|
||||
---
|
||||
|
||||
## ADR-001 · Architecture: TS-JEPA + SIGReg as Phase 1 backbone
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Accepted
|
||||
**Supersedes:** informal decision to use TS-JEPA standalone (pre-ADR)
|
||||
|
||||
### Context
|
||||
|
||||
Four JEPA variants were evaluated for FX volatility forecasting and VaR/CVaR estimation:
|
||||
|
||||
| Variant | Origin | Key property |
|
||||
|---|---|---|
|
||||
| TS-JEPA | Ennadir et al., Sep 2025 | Time-series native; EMA collapse prevention |
|
||||
| LeJEPA | Balestriero & LeCun, Nov 2025 | Proven optimal embeddings (isotropic Gaussian); SIGReg |
|
||||
| MTS-JEPA | He et al., Feb 2026 | Multi-resolution + codebook; no public code |
|
||||
| Var-JEPA | Multiple, Mar 2026 | ELBO-based UQ; no public code |
|
||||
|
||||
Key constraints: 12 GB VRAM (Blackwell, koala), hourly DUKASCopy data, internal PoC target,
|
||||
autoresearch loop requires a clean single-scalar search space, trading desk requires an
|
||||
explainable theoretical story.
|
||||
|
||||
### Decision
|
||||
|
||||
Use **TS-JEPA architecture with SIGReg replacing EMA** as the Phase 1 backbone.
|
||||
|
||||
Concretely:
|
||||
- Start from the TS-JEPA open-source implementation (arXiv:2509.25449, GitHub)
|
||||
- Remove the EMA target-network mechanism
|
||||
- Replace it with Sketched Isotropic Gaussian Regularization (SIGReg) from LeJEPA
|
||||
(arXiv:2511.08544), controlled by a single λ hyperparameter
|
||||
- Keep TS-JEPA's temporal patchwise masking and Transformer encoder unchanged
|
||||
|
||||
### Rationale
|
||||
|
||||
**Why not pure TS-JEPA:** EMA is a heuristic; λ interacts with EMA momentum and
|
||||
learning rate, creating a three-way search space that is hard to navigate with autoresearch.
|
||||
EMA also has no theoretical non-stationarity guarantee.
|
||||
|
||||
**Why not pure LeJEPA:** The reference implementation targets vision (multi-crop views).
|
||||
Adapting it to temporal patchwise masking requires non-trivial surgery and moves away from
|
||||
open code. TS-JEPA's masking is already the right inductive bias for time series.
|
||||
|
||||
**Why the hybrid:** SIGReg is architecture-agnostic — it operates on the embedding
|
||||
distribution, not the encoder structure. Swapping EMA for SIGReg is a ~20-line change to
|
||||
TS-JEPA's training loop. The result is:
|
||||
- Time-series native (TS-JEPA masking + patch structure)
|
||||
- Provably collapse-free without heuristics (SIGReg)
|
||||
- Single search axis for autoresearch (λ ∈ [0.01, 1.0])
|
||||
- Non-stationarity robustness proven formally (arXiv:2602.19373 extends LeJEPA
|
||||
guarantees to non-stationary target distributions — directly relevant to FX)
|
||||
- Explainable to a model validation team: "embeddings are provably optimal for
|
||||
downstream prediction under distributional uncertainty"
|
||||
|
||||
**Why not MTS-JEPA or Var-JEPA now:** Both lack public code (as of May 2026).
|
||||
MTS-JEPA's multi-resolution objective is the right next hypothesis (see ADR-003).
|
||||
Var-JEPA's ELBO-based UQ is a compelling future direction for CVaR estimation.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Phase 0 (MAE baseline) is unaffected — it precedes the JEPA architecture choice
|
||||
- Issue #3 (TS-JEPA reproduction) is still the right first step; SIGReg is added after
|
||||
reproduction is confirmed
|
||||
- The autoresearch `program.md` primary search axis is λ (SIGReg weight)
|
||||
- Secondary axes: masking block size, patch stride, encoder depth
|
||||
- `model/requirements.txt` must include the SIGReg implementation (≈20 lines,
|
||||
can be vendored directly)
|
||||
|
||||
---
|
||||
|
||||
## ADR-002 · Data: DUKASCopy hourly G10 FX as primary training data
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Accepted
|
||||
|
||||
### Context
|
||||
|
||||
Data scale is the most dangerous assumption for any SSL/JEPA approach. Daily FX data
|
||||
(~5,000 samples over 20 years) is insufficient for self-supervised pretraining.
|
||||
Two alternatives were considered: daily public data (yfinance) vs. hourly tick data
|
||||
(DUKASCopy, free, rate-limited HTTP API).
|
||||
|
||||
### Decision
|
||||
|
||||
Use **DUKASCopy hourly OHLCV** as the primary data source.
|
||||
|
||||
- 10 G10 pairs: EURUSD, GBPUSD, USDJPY, USDCHF, AUDUSD, NZDUSD, USDCAD,
|
||||
EURGBP, EURJPY, GBPJPY
|
||||
- Training window: 2008-01-01 – 2022-12-31 (~175,000 samples per pair)
|
||||
- Validation window: 2023-01-01 – 2023-12-31 (~2,600 samples)
|
||||
- Test window: 2024-01-01 – 2024-12-31 (held out, never seen during development)
|
||||
- Features per bar: log-return, log rolling-20-period HV, VIX (daily interpolated)
|
||||
- Weekend gaps handled explicitly — no interpolation across market close
|
||||
|
||||
### Rationale
|
||||
|
||||
Hourly data gives ~35× more samples than daily. This is the minimum threshold for
|
||||
JEPA-style SSL to show a training signal within 10-minute autoresearch experiments.
|
||||
DUKASCopy is free, reliable, and provides consistent tick-level source data back to 2003.
|
||||
|
||||
### Consequences
|
||||
|
||||
- The Go data pipeline (Issue #2) is the critical path for everything else
|
||||
- Phase 0 MAE baseline trains on the same 2008-2022 window
|
||||
- Daily data (yfinance) may still be used for VIX and rate differentials as auxiliary features
|
||||
|
||||
---
|
||||
|
||||
## ADR-003 · Research roadmap: Phase structure and JEPA variant progression
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Accepted
|
||||
|
||||
### Decision
|
||||
|
||||
Three-phase research roadmap:
|
||||
|
||||
**Phase 0 — SSL feasibility gate (MAE baseline)**
|
||||
Implement a 1D temporal MAE (not JEPA) on EUR/USD hourly 2008-2022.
|
||||
Gate criteria: silhouette > 0.20 on 2023 held-out, MAE > PCA baseline, ±10% over 3 reruns.
|
||||
Purpose: validate that the data and eval harness work before committing to JEPA complexity.
|
||||
If gate fails: follow null result protocol in `specs/phase-0-ssl-feasibility.md`.
|
||||
|
||||
**Phase 1 — TS-JEPA + SIGReg autoresearch sweep**
|
||||
Primary architecture per ADR-001.
|
||||
Autoresearch loop: `program.md`-driven, 10-min experiments, 50-experiment budget.
|
||||
Primary metric: `val_vol_r2` (linear probe R² on 1-day realized volatility).
|
||||
Gate criteria: `val_vol_r2` > GARCH-implied baseline AND Kupiec p-value > 0.05 on
|
||||
EUR/USD VaR 99%.
|
||||
Kupiec is logged from experiment 1 to verify it co-moves with `val_vol_r2`.
|
||||
|
||||
**Phase 2 — MTS-JEPA multi-resolution hypothesis**
|
||||
Introduce parallel multi-scale predictive pathways (1h, 8h, 24h context windows)
|
||||
adapted from MTS-JEPA (arXiv:2602.04643).
|
||||
Hypothesis: multi-scale representations improve regime detection (silhouette) and
|
||||
reduce VaR exceedance clustering (Christoffersen test).
|
||||
Prerequisite: Phase 1 gate passed AND MTS-JEPA code available or reproducible from paper.
|
||||
Time-box: if MTS-JEPA code not available within 4 weeks of Phase 2 start, implement
|
||||
multi-resolution masking from scratch using Phase 1 backbone as base.
|
||||
|
||||
**Phase 3 — Internal bank data (future)**
|
||||
Replace DUKASCopy pipeline with internal tick feed adapter.
|
||||
Fine-tune heads only; backbone frozen or lightly fine-tuned.
|
||||
Out of scope for current PoC cycle.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Issue #5 (Phase 0 MAE) is the unblocked next executable step
|
||||
- Phase 1 autoresearch is blocked until Phase 0 passes its gate
|
||||
- Var-JEPA (ELBO-based UQ) is a named future hypothesis for CVaR estimation in Phase 2+
|
||||
but not on the critical path
|
||||
|
||||
---
|
||||
|
||||
## ADR-004 · Evaluation: Go harness + Python training separation
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Accepted
|
||||
|
||||
### Decision
|
||||
|
||||
Hard separation between training (Python) and evaluation (Go):
|
||||
|
||||
- **Python** (`model/`): all training, embedding export, model checkpointing
|
||||
- **Go** (`src/eval/`): all evaluation metrics — silhouette, linear probe R², collapse
|
||||
diagnostic, Kupiec/Christoffersen backtests
|
||||
- Interface: Python exports embedding matrices + labels to `experiments/RUNID/` as
|
||||
`.npy` files; Go eval harness reads them and writes `metrics.json`
|
||||
|
||||
### Rationale
|
||||
|
||||
Go evaluation gives deterministic, fast, auditable metric computation with proper
|
||||
unit tests. It decouples the experimental loop from the training framework, making
|
||||
it possible to re-evaluate any past experiment without re-running training.
|
||||
The Go layer also serves as the foundation for the eventual trading desk dashboard.
|
||||
|
||||
### Consequences
|
||||
|
||||
- All acceptance criteria in Issues #4 and #5 are specified in terms of Go eval outputs
|
||||
- `val_vol_r2` (the autoresearch optimization metric) is computed by the Go harness,
|
||||
not inside the Python training loop
|
||||
- Python training loop calls `task eval:probe` as a subprocess after each experiment
|
||||
to get the scalar fed back to autoresearch
|
||||
|
||||
---
|
||||
|
||||
## ADR-005 · Compute: Blackwell GPU on koala, PyTorch cu130
|
||||
|
||||
**Date:** 2026-05-28
|
||||
**Status:** Accepted
|
||||
|
||||
### Decision
|
||||
|
||||
All GPU training runs on koala (Arch Linux, Blackwell GPU, 12 GB VRAM).
|
||||
PyTorch install: `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130`
|
||||
(CUDA 13.0 wheel — required for sm_120 Blackwell support; stable as of May 2026).
|
||||
Driver requirement: NVIDIA R570+, CUDA toolkit 12.8+.
|
||||
|
||||
Ollama on iguana (Mac Studio M2 Ultra) serves the autoresearch agent LLM via the
|
||||
existing LiteLLM proxy on piguard. Agent calls never hit koala directly.
|
||||
|
||||
### Consequences
|
||||
|
||||
- `model/requirements.txt` must NOT pin torch to a cu124 or earlier wheel
|
||||
- CI (Issue #7) must NOT run GPU tests — CPU-only for unit tests, GPU only via
|
||||
`task experiment:run` on koala
|
||||
- 12 GB VRAM is sufficient for <5M parameter models at batch=64; monitor if
|
||||
autoresearch explores larger architectures
|
||||
@@ -0,0 +1,22 @@
|
||||
# Autoresearch STATUS
|
||||
|
||||
| iter | val_vol_r2 | delta | action | secs | gpu | change |
|
||||
|------|-----------|-------|--------|------|-----|--------|
|
||||
| 1 | 0.3749 | +0.0928 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
||||
| 1 | 0.3011 | +0.0776 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
||||
| 2 | 0.3032 | +0.0021 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
|
||||
| 1 | 0.2759 | -0.0273 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
||||
| 2 | 0.3442 | +0.0410 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter2 |
|
||||
| 3 | 0.3371 | -0.0071 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter3 |
|
||||
| 4 | 0.3143 | -0.0299 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter4 |
|
||||
| 5 | 0.3355 | -0.0087 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter5 |
|
||||
| 6 | 0.2377 | -0.1065 | revert | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter6 |
|
||||
| 1 | -0.1247 | +0.0296 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
|
||||
| 2 | -0.1203 | +0.0044 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
|
||||
| 3 | -0.0716 | +0.0487 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
|
||||
| 4 | 0.0590 | +0.1306 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter4 |
|
||||
| 5 | 0.0599 | +0.0009 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter5 |
|
||||
| 1 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
|
||||
| 2 | 0.0577 | -0.0022 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter2 |
|
||||
| 3 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
|
||||
| 4 | -0.1613 | -0.2212 | revert | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter4 |
|
||||
+50
-3
@@ -5,16 +5,63 @@ tasks:
|
||||
desc: Run templ generate
|
||||
cmds: [templ generate]
|
||||
build:
|
||||
desc: Build the binary
|
||||
desc: Build all binaries
|
||||
deps: [generate]
|
||||
cmds: [go build -o bin/hostexecutor ./cmd/hostexecutor]
|
||||
cmds:
|
||||
- go build -o bin/jepa-fx-risk ./cmd/jepa-fx-risk
|
||||
- go build -o bin/eval ./cmd/eval
|
||||
run:
|
||||
deps: [build]
|
||||
cmds: [./bin/hostexecutor]
|
||||
cmds: [./bin/jepa-fx-risk]
|
||||
test:
|
||||
desc: Run all tests
|
||||
deps: [generate]
|
||||
cmds: [go test ./... -race]
|
||||
|
||||
data:fetch:
|
||||
desc: "Download EUR/USD M1 from histdata (set YEARS env var)"
|
||||
cmds: [.venv/bin/python scripts/fetch_data.py]
|
||||
data:fetch:historical:
|
||||
desc: "Download EUR/USD M1 2008-2018 from histdata"
|
||||
cmds:
|
||||
- YEARS=2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018 .venv/bin/python scripts/fetch_data.py
|
||||
data:prepare:daily:
|
||||
desc: "Rebuild eurusd_daily.parquet from all M1 zips"
|
||||
cmds: [.venv/bin/python scripts/prepare_data.py]
|
||||
data:prepare:hourly:
|
||||
desc: "Build eurusd_hourly.parquet from all M1 zips"
|
||||
cmds: [.venv/bin/python scripts/prepare_hourly.py]
|
||||
data:prepare:all:
|
||||
desc: "Build both daily and hourly parquets"
|
||||
deps: [data:prepare:daily, data:prepare:hourly]
|
||||
train:multipair:
|
||||
desc: "Train 5-pair G10 HEPA (D=256, best config, phase1_r2≈0.44)"
|
||||
cmds: [JEPA_USE_MULTIPAIR=1 JEPA_D_MODEL=256 .venv/bin/python train.py]
|
||||
|
||||
data:fetch:multipair:
|
||||
desc: "Download G10 M1 data (GBPUSD/USDJPY/USDCHF/AUDUSD) 2008-2023 from histdata"
|
||||
cmds: [.venv/bin/python scripts/fetch_multipair.py]
|
||||
data:prepare:pair:
|
||||
desc: "Build {PAIR}_hourly.parquet from data/raw/{PAIR}/ (e.g. PAIR=gbpusd)"
|
||||
cmds: ["PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}"]
|
||||
vars:
|
||||
PAIR: '{{default "eurusd" .PAIR}}'
|
||||
data:prepare:multipair:
|
||||
desc: "Merge 5-pair hourly parquets into eurusd_multipair.parquet"
|
||||
cmds: [.venv/bin/python scripts/prepare_multipair.py]
|
||||
data:test:
|
||||
desc: "Run Python data pipeline tests"
|
||||
cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py tests/test_multipair.py -v]
|
||||
|
||||
eval:probe:
|
||||
desc: "Run linear-probe (val_vol_r2) on embeddings from metrics.json"
|
||||
cmds: [./bin/eval -metric probe]
|
||||
eval:silhouette:
|
||||
desc: "Run silhouette on embeddings vs binary HV labels"
|
||||
cmds: [./bin/eval -metric silhouette]
|
||||
eval:collapse:
|
||||
desc: "Run effective-rank collapse diagnostic"
|
||||
cmds: [./bin/eval -metric erank]
|
||||
lint:
|
||||
cmds: [golangci-lint run ./...]
|
||||
check:
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
// cmd/eval — CLI driver for the jepa-fx-risk evaluation harness.
|
||||
//
|
||||
// ./bin/eval -metric probe|silhouette|erank [-emb embeddings.json]
|
||||
//
|
||||
// embeddings.json format (from train.py EXPORT_EMBEDDINGS=1):
|
||||
//
|
||||
// {
|
||||
// "embeddings": [[...], ...], // OOS frozen embeddings
|
||||
// "realized_vol": [...], // OOS target (next-day RV)
|
||||
// "hv_label": [...], // binary HV label (top-33%)
|
||||
// "train_embeddings": [[...], ...], // train-set frozen embeddings
|
||||
// "train_realized_vol": [...] // train-set RV targets
|
||||
// }
|
||||
//
|
||||
// eval:probe standardises both sets using train statistics (no leakage).
|
||||
// Falls back to internal 70/30 split of OOS if train_embeddings absent.
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"math"
|
||||
"os"
|
||||
|
||||
"gitea.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
||||
)
|
||||
|
||||
func main() {
|
||||
metric := flag.String("metric", "probe", "probe | silhouette | erank")
|
||||
embFile := flag.String("emb", "embeddings.json", "path to embeddings JSON")
|
||||
flag.Parse()
|
||||
|
||||
log := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
||||
|
||||
d, err := readJSON(*embFile)
|
||||
if err != nil {
|
||||
log.Error("load embeddings", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
log.Info("loaded", "oos", len(d.Embeddings), "dim", len(d.Embeddings[0]),
|
||||
"train", len(d.TrainEmbeddings), "metric", *metric)
|
||||
|
||||
switch *metric {
|
||||
case "probe":
|
||||
var r2 float64
|
||||
if len(d.TrainEmbeddings) > 0 {
|
||||
// standardise both sets using train statistics to prevent leakage
|
||||
trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
|
||||
oosEmb := applyStandardise(d.Embeddings, mu, sd)
|
||||
r2 = eval.LinearProbeTrainTest(trEmb, d.TrainRealizedVol, oosEmb, d.RealizedVol, 1e-3)
|
||||
log.Info("probe mode", "fit_on", "train_embeddings", "eval_on", "oos")
|
||||
} else {
|
||||
// fallback: internal 70/30 split of OOS embeddings
|
||||
oosEmb, mu, sd := standardiseCompute(d.Embeddings)
|
||||
n70 := int(float64(len(oosEmb)) * 0.7)
|
||||
oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
|
||||
r2 = eval.LinearProbeTrainTest(oosEmb[:n70], d.RealizedVol[:n70],
|
||||
oos70, d.RealizedVol[n70:], 1e-3)
|
||||
log.Info("probe mode", "fit_on", "oos[0:70%]", "eval_on", "oos[70%:]")
|
||||
}
|
||||
fmt.Printf(`{"metric":"val_vol_r2","value":%.6f}`+"\n", r2)
|
||||
log.Info("linear probe", "val_vol_r2", fmt.Sprintf("%.4f", r2))
|
||||
|
||||
case "silhouette":
|
||||
if len(d.HVLabel) == 0 {
|
||||
log.Error("silhouette requires hv_label in embeddings.json")
|
||||
os.Exit(1)
|
||||
}
|
||||
oosEmb := standardise(d.Embeddings)
|
||||
sil, err := eval.Silhouette(oosEmb, d.HVLabel)
|
||||
if err != nil {
|
||||
log.Error("silhouette", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf(`{"metric":"silhouette","value":%.6f}`+"\n", sil)
|
||||
log.Info("silhouette", "score", fmt.Sprintf("%.4f", sil))
|
||||
|
||||
case "erank":
|
||||
oosEmb := standardise(d.Embeddings)
|
||||
er := eval.EffectiveRank(oosEmb)
|
||||
fmt.Printf(`{"metric":"effective_rank","value":%.6f}`+"\n", er)
|
||||
log.Info("effective rank", "erank", fmt.Sprintf("%.2f", er))
|
||||
|
||||
case "var":
|
||||
// Parametric 99% VaR breach rate from probe predictions vs actual realized vol.
|
||||
// Requires train_embeddings (for no-leakage probe fit) and realized_vol (OOS).
|
||||
if len(d.RealizedVol) == 0 {
|
||||
log.Error("var requires realized_vol in embeddings.json")
|
||||
os.Exit(1)
|
||||
}
|
||||
var predVol []float64
|
||||
if len(d.TrainEmbeddings) > 0 {
|
||||
trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
|
||||
oosEmb := applyStandardise(d.Embeddings, mu, sd)
|
||||
predVol = eval.LinearProbePredict(trEmb, d.TrainRealizedVol, oosEmb, 1e-3)
|
||||
} else {
|
||||
oosEmb, mu, sd := standardiseCompute(d.Embeddings)
|
||||
n70 := int(float64(len(oosEmb)) * 0.7)
|
||||
oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
|
||||
predVol = eval.LinearProbePredict(oosEmb[:n70], d.RealizedVol[:n70], oos70, 1e-3)
|
||||
d.RealizedVol = d.RealizedVol[n70:]
|
||||
}
|
||||
const z99 = 2.326
|
||||
breachRate, kupiecP := eval.VaRBreachRate(predVol, d.RealizedVol, z99)
|
||||
fmt.Printf(`{"metric":"VaR_breach_rate_99_oos_regime_cond","value":%.6f,"kupiec_p":%.6f}`+"\n",
|
||||
breachRate, kupiecP)
|
||||
log.Info("VaR breach rate 99%", "breach_rate", fmt.Sprintf("%.4f", breachRate),
|
||||
"kupiec_p", fmt.Sprintf("%.4f", kupiecP))
|
||||
|
||||
default:
|
||||
log.Error("unknown metric", "metric", *metric)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
type embJSON struct {
|
||||
Embeddings [][]float64 `json:"embeddings"`
|
||||
Dates []string `json:"dates"`
|
||||
RealizedVol []float64 `json:"realized_vol"`
|
||||
HVLabel []int `json:"hv_label"`
|
||||
TrainEmbeddings [][]float64 `json:"train_embeddings"`
|
||||
TrainRealizedVol []float64 `json:"train_realized_vol"`
|
||||
}
|
||||
|
||||
func readJSON(path string) (*embJSON, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("open %s: %w", path, err)
|
||||
}
|
||||
defer func() { _ = f.Close() }()
|
||||
var d embJSON
|
||||
if err := json.NewDecoder(f).Decode(&d); err != nil {
|
||||
return nil, fmt.Errorf("decode: %w", err)
|
||||
}
|
||||
if len(d.Embeddings) == 0 {
|
||||
return nil, fmt.Errorf("empty embeddings in %s", path)
|
||||
}
|
||||
return &d, nil
|
||||
}
|
||||
|
||||
// standardise centres + scales to zero mean / unit std; returns normalised rows.
|
||||
func standardise(rows [][]float64) [][]float64 {
|
||||
out, _, _ := standardiseCompute(rows)
|
||||
return out
|
||||
}
|
||||
|
||||
// standardiseCompute centres + scales and returns (normalised, mu, sd) for reuse.
|
||||
func standardiseCompute(rows [][]float64) ([][]float64, []float64, []float64) {
|
||||
if len(rows) == 0 {
|
||||
return rows, nil, nil
|
||||
}
|
||||
n, dim := len(rows), len(rows[0])
|
||||
mu := make([]float64, dim)
|
||||
for _, r := range rows {
|
||||
for j, v := range r {
|
||||
mu[j] += v
|
||||
}
|
||||
}
|
||||
for j := range mu {
|
||||
mu[j] /= float64(n)
|
||||
}
|
||||
sd := make([]float64, dim)
|
||||
for _, r := range rows {
|
||||
for j, v := range r {
|
||||
diff := v - mu[j]
|
||||
sd[j] += diff * diff
|
||||
}
|
||||
}
|
||||
for j := range sd {
|
||||
sd[j] = math.Sqrt(sd[j]/float64(n)) + 1e-8
|
||||
}
|
||||
out := make([][]float64, n)
|
||||
for i, r := range rows {
|
||||
out[i] = make([]float64, dim)
|
||||
for j, v := range r {
|
||||
out[i][j] = (v - mu[j]) / sd[j]
|
||||
}
|
||||
}
|
||||
return out, mu, sd
|
||||
}
|
||||
|
||||
// applyStandardise normalises rows using pre-computed mu and sd.
|
||||
func applyStandardise(rows [][]float64, mu, sd []float64) [][]float64 {
|
||||
out := make([][]float64, len(rows))
|
||||
for i, r := range rows {
|
||||
out[i] = make([]float64, len(r))
|
||||
for j, v := range r {
|
||||
out[i][j] = (v - mu[j]) / sd[j]
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -5,7 +5,7 @@ import (
|
||||
"net/http"
|
||||
"os"
|
||||
|
||||
"gitea.d-ma.be/mathias/hostexecutor/internal/web"
|
||||
"gitea.d-ma.be/mathias/jepa-fx-risk/internal/web"
|
||||
)
|
||||
|
||||
func main() {
|
||||
Binary file not shown.
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"strategic_question": "What is the highest-leverage path to a JEPA-based FX tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach calibration, given one GPU and a solo researcher?",
|
||||
"nodes": [
|
||||
{
|
||||
"id": "phase-a-toy",
|
||||
"status": "autoresearch-ready",
|
||||
"question": "Improve the OOS linear-probe R² (val_vol_r2) of the HEPA encoder on EUR/USD daily realized vol. The encoder is a small causal transformer trained with VICReg. Vary one hyperparameter or architectural choice per iteration — model size, learning rate, window, patch length, depth, VICReg loss weights — to push val_vol_r2 as high as possible on the 2022-2023 OOS slice.",
|
||||
"candidate_metric": "val_vol_r2"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,7 +1,5 @@
|
||||
module gitea.d-ma.be/mathias/hostexecutor
|
||||
module gitea.d-ma.be/mathias/jepa-fx-risk
|
||||
|
||||
go 1.26
|
||||
|
||||
require (
|
||||
github.com/a-h/templ v0.2.778
|
||||
)
|
||||
require github.com/a-h/templ v0.3.1020
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
|
||||
github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
@@ -0,0 +1,383 @@
|
||||
// Package eval implements the Go evaluation harness for jepa-fx-risk (#4).
|
||||
// Three diagnostics on frozen embeddings exported from train.py:
|
||||
// - LinearProbe — val_vol_r2: OOS R² of a ridge probe predicting next-day realized vol
|
||||
// - Silhouette — mean silhouette score of embeddings vs a binary label (HV regime)
|
||||
// - EffectiveRank — Roy's effective rank: exp(H(σ²)) where H is entropy of normalised singular values
|
||||
package eval
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"math"
|
||||
)
|
||||
|
||||
// LinearProbeTrainTest fits ridge regression on (trainEmb, trainY) and evaluates
|
||||
// on (testEmb, testY). Returns OOS R². Use this for proper held-out evaluation.
|
||||
func LinearProbeTrainTest(trainEmb [][]float64, trainY []float64,
|
||||
testEmb [][]float64, testY []float64, lambda float64) float64 {
|
||||
n := len(trainEmb)
|
||||
if n == 0 || len(testEmb) == 0 {
|
||||
return 0
|
||||
}
|
||||
d := len(trainEmb[0])
|
||||
p := d + 1
|
||||
|
||||
A := make([][]float64, n)
|
||||
for i, e := range trainEmb {
|
||||
row := make([]float64, p)
|
||||
copy(row, e)
|
||||
row[d] = 1.0
|
||||
A[i] = row
|
||||
}
|
||||
AtA := make([][]float64, p)
|
||||
for i := range AtA {
|
||||
AtA[i] = make([]float64, p)
|
||||
}
|
||||
Aty := make([]float64, p)
|
||||
for i := 0; i < n; i++ {
|
||||
for j := 0; j < p; j++ {
|
||||
Aty[j] += A[i][j] * trainY[i]
|
||||
for k := 0; k < p; k++ {
|
||||
AtA[j][k] += A[i][j] * A[i][k]
|
||||
}
|
||||
}
|
||||
}
|
||||
for j := 0; j < p; j++ {
|
||||
AtA[j][j] += lambda
|
||||
}
|
||||
w := solveCholesky(AtA, Aty)
|
||||
|
||||
yMean := mean(testY)
|
||||
var ssRes, ssTot float64
|
||||
for i, e := range testEmb {
|
||||
row := make([]float64, p)
|
||||
copy(row, e)
|
||||
row[d] = 1.0
|
||||
pred := dot(row, w)
|
||||
ssRes += (testY[i] - pred) * (testY[i] - pred)
|
||||
ssTot += (testY[i] - yMean) * (testY[i] - yMean)
|
||||
}
|
||||
if ssTot == 0 {
|
||||
return 0
|
||||
}
|
||||
return 1 - ssRes/ssTot
|
||||
}
|
||||
|
||||
// LinearProbe fits a ridge regression (closed-form) on (emb, y) with regularisation λ
|
||||
// and returns R² on the same data. Call with train embeddings; probe on held-out by
|
||||
// splitting before calling.
|
||||
//
|
||||
// emb[i] is the embedding vector for sample i; y[i] is the scalar target.
|
||||
func LinearProbe(emb [][]float64, y []float64, lambda float64) float64 {
|
||||
n := len(emb)
|
||||
if n == 0 {
|
||||
return 0
|
||||
}
|
||||
d := len(emb[0])
|
||||
|
||||
// Build augmented design matrix A = [emb | 1] (n × d+1)
|
||||
A := make([][]float64, n)
|
||||
for i, e := range emb {
|
||||
row := make([]float64, d+1)
|
||||
copy(row, e)
|
||||
row[d] = 1.0
|
||||
A[i] = row
|
||||
}
|
||||
|
||||
// Normal equations: (AᵀA + λI) w = Aᵀy (ridge)
|
||||
p := d + 1
|
||||
AtA := make([][]float64, p)
|
||||
for i := range AtA {
|
||||
AtA[i] = make([]float64, p)
|
||||
}
|
||||
Aty := make([]float64, p)
|
||||
|
||||
for i := 0; i < n; i++ {
|
||||
for j := 0; j < p; j++ {
|
||||
Aty[j] += A[i][j] * y[i]
|
||||
for k := 0; k < p; k++ {
|
||||
AtA[j][k] += A[i][j] * A[i][k]
|
||||
}
|
||||
}
|
||||
}
|
||||
for j := 0; j < p; j++ {
|
||||
AtA[j][j] += lambda
|
||||
}
|
||||
|
||||
w := solveCholesky(AtA, Aty)
|
||||
|
||||
// R² = 1 - SS_res / SS_tot
|
||||
yMean := mean(y)
|
||||
var ssRes, ssTot float64
|
||||
for i := 0; i < n; i++ {
|
||||
pred := dot(A[i], w)
|
||||
ssRes += (y[i] - pred) * (y[i] - pred)
|
||||
ssTot += (y[i] - yMean) * (y[i] - yMean)
|
||||
}
|
||||
if ssTot == 0 {
|
||||
return 0
|
||||
}
|
||||
return 1 - ssRes/ssTot
|
||||
}
|
||||
|
||||
// Silhouette returns the mean silhouette coefficient of the embeddings with respect
|
||||
// to the given integer labels. Distances are Euclidean. Returns an error if fewer
|
||||
// than 2 distinct labels are present.
|
||||
func Silhouette(emb [][]float64, labels []int) (float64, error) {
|
||||
n := len(emb)
|
||||
if n == 0 {
|
||||
return 0, errors.New("eval: empty embeddings")
|
||||
}
|
||||
|
||||
// count distinct labels
|
||||
labelSet := map[int]struct{}{}
|
||||
for _, l := range labels {
|
||||
labelSet[l] = struct{}{}
|
||||
}
|
||||
if len(labelSet) < 2 {
|
||||
return 0, errors.New("eval: silhouette requires at least 2 distinct labels")
|
||||
}
|
||||
|
||||
// group indices by label
|
||||
groups := map[int][]int{}
|
||||
for i, l := range labels {
|
||||
groups[l] = append(groups[l], i)
|
||||
}
|
||||
|
||||
var total float64
|
||||
for i := 0; i < n; i++ {
|
||||
li := labels[i]
|
||||
|
||||
// a(i) = mean intra-cluster distance
|
||||
var aSum float64
|
||||
inGroup := groups[li]
|
||||
for _, j := range inGroup {
|
||||
if j != i {
|
||||
aSum += euclidean(emb[i], emb[j])
|
||||
}
|
||||
}
|
||||
var a float64
|
||||
if len(inGroup) > 1 {
|
||||
a = aSum / float64(len(inGroup)-1)
|
||||
}
|
||||
|
||||
// b(i) = min mean inter-cluster distance
|
||||
b := math.MaxFloat64
|
||||
for l, idxs := range groups {
|
||||
if l == li {
|
||||
continue
|
||||
}
|
||||
var dSum float64
|
||||
for _, j := range idxs {
|
||||
dSum += euclidean(emb[i], emb[j])
|
||||
}
|
||||
avg := dSum / float64(len(idxs))
|
||||
if avg < b {
|
||||
b = avg
|
||||
}
|
||||
}
|
||||
|
||||
s := (b - a) / math.Max(a, b)
|
||||
total += s
|
||||
}
|
||||
return total / float64(n), nil
|
||||
}
|
||||
|
||||
// EffectiveRank computes Roy's effective rank of the embedding matrix:
|
||||
// exp(H) where H = -∑ pᵢ log(pᵢ) is the Shannon entropy of the normalised
|
||||
// squared singular values. Returns 1 for a rank-1 matrix and ≈ dim for
|
||||
// a full-rank isotropic matrix.
|
||||
func EffectiveRank(emb [][]float64) float64 {
|
||||
n := len(emb)
|
||||
if n == 0 {
|
||||
return 0
|
||||
}
|
||||
d := len(emb[0])
|
||||
|
||||
// Compute covariance-like matrix CᵀC where C is mean-centered embedding.
|
||||
mu := make([]float64, d)
|
||||
for _, e := range emb {
|
||||
for j, v := range e {
|
||||
mu[j] += v
|
||||
}
|
||||
}
|
||||
for j := range mu {
|
||||
mu[j] /= float64(n)
|
||||
}
|
||||
|
||||
// C = emb - mu (n × d); compute CᵀC (d × d)
|
||||
CtC := make([][]float64, d)
|
||||
for i := range CtC {
|
||||
CtC[i] = make([]float64, d)
|
||||
}
|
||||
for _, e := range emb {
|
||||
for j := 0; j < d; j++ {
|
||||
cj := e[j] - mu[j]
|
||||
for k := 0; k < d; k++ {
|
||||
CtC[j][k] += cj * (e[k] - mu[k])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Eigenvalues of CᵀC via power iteration approximation isn't great;
|
||||
// use the Frobenius / trace approach: σᵢ² ∝ eigenvalues of CᵀC.
|
||||
// For a pure-Go impl without LAPACK: use the fact that the normalised
|
||||
// squared singular values equal normalised eigenvalues of CᵀC.
|
||||
// Compute them via Jacobi iteration for small d, or use the analytical
|
||||
// formula for 2×2, or use iterative QR for general d.
|
||||
eigs := jacobiEigenvalues(CtC)
|
||||
|
||||
// normalise to sum-1 distribution
|
||||
var sumEig float64
|
||||
for _, v := range eigs {
|
||||
if v > 0 {
|
||||
sumEig += v
|
||||
}
|
||||
}
|
||||
if sumEig == 0 {
|
||||
return 1
|
||||
}
|
||||
var H float64
|
||||
for _, v := range eigs {
|
||||
if v > 0 {
|
||||
p := v / sumEig
|
||||
H -= p * math.Log(p)
|
||||
}
|
||||
}
|
||||
return math.Exp(H)
|
||||
}
|
||||
|
||||
// ── internal helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
func euclidean(a, b []float64) float64 {
|
||||
var s float64
|
||||
for i := range a {
|
||||
d := a[i] - b[i]
|
||||
s += d * d
|
||||
}
|
||||
return math.Sqrt(s)
|
||||
}
|
||||
|
||||
func dot(a, b []float64) float64 {
|
||||
var s float64
|
||||
for i := range a {
|
||||
s += a[i] * b[i]
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func mean(y []float64) float64 {
|
||||
var s float64
|
||||
for _, v := range y {
|
||||
s += v
|
||||
}
|
||||
return s / float64(len(y))
|
||||
}
|
||||
|
||||
// solveCholesky solves Ax = b for symmetric positive-definite A via
|
||||
// Cholesky decomposition. Falls back to pseudo-inverse on failure.
|
||||
func solveCholesky(A [][]float64, b []float64) []float64 {
|
||||
n := len(A)
|
||||
// Cholesky decomposition: A = LLᵀ
|
||||
L := make([][]float64, n)
|
||||
for i := range L {
|
||||
L[i] = make([]float64, n)
|
||||
}
|
||||
for i := 0; i < n; i++ {
|
||||
for j := 0; j <= i; j++ {
|
||||
s := A[i][j]
|
||||
for k := 0; k < j; k++ {
|
||||
s -= L[i][k] * L[j][k]
|
||||
}
|
||||
if i == j {
|
||||
if s <= 0 {
|
||||
s = 1e-12
|
||||
}
|
||||
L[i][j] = math.Sqrt(s)
|
||||
} else {
|
||||
L[i][j] = s / L[j][j]
|
||||
}
|
||||
}
|
||||
}
|
||||
// Forward substitution Ly = b
|
||||
y := make([]float64, n)
|
||||
for i := 0; i < n; i++ {
|
||||
s := b[i]
|
||||
for k := 0; k < i; k++ {
|
||||
s -= L[i][k] * y[k]
|
||||
}
|
||||
y[i] = s / L[i][i]
|
||||
}
|
||||
// Back substitution Lᵀx = y
|
||||
x := make([]float64, n)
|
||||
for i := n - 1; i >= 0; i-- {
|
||||
s := y[i]
|
||||
for k := i + 1; k < n; k++ {
|
||||
s -= L[k][i] * x[k]
|
||||
}
|
||||
x[i] = s / L[i][i]
|
||||
}
|
||||
return x
|
||||
}
|
||||
|
||||
// jacobiEigenvalues returns eigenvalues of a symmetric matrix via Jacobi iteration.
|
||||
func jacobiEigenvalues(A [][]float64) []float64 {
|
||||
n := len(A)
|
||||
// copy
|
||||
a := make([][]float64, n)
|
||||
for i := range a {
|
||||
a[i] = make([]float64, n)
|
||||
copy(a[i], A[i])
|
||||
}
|
||||
|
||||
const maxIter = 100
|
||||
const tol = 1e-10
|
||||
for iter := 0; iter < maxIter; iter++ {
|
||||
// find largest off-diagonal element
|
||||
p, q, amax := 0, 1, 0.0
|
||||
for i := 0; i < n; i++ {
|
||||
for j := i + 1; j < n; j++ {
|
||||
if v := math.Abs(a[i][j]); v > amax {
|
||||
amax = v
|
||||
p, q = i, j
|
||||
}
|
||||
}
|
||||
}
|
||||
if amax < tol {
|
||||
break
|
||||
}
|
||||
// Jacobi rotation
|
||||
theta := 0.5 * math.Atan2(2*a[p][q], a[q][q]-a[p][p])
|
||||
c, s := math.Cos(theta), math.Sin(theta)
|
||||
// apply rotation
|
||||
newA := make([][]float64, n)
|
||||
for i := range newA {
|
||||
newA[i] = make([]float64, n)
|
||||
copy(newA[i], a[i])
|
||||
}
|
||||
app := c*c*a[p][p] + 2*c*s*a[p][q] + s*s*a[q][q]
|
||||
aqq := s*s*a[p][p] - 2*c*s*a[p][q] + c*c*a[q][q]
|
||||
apq := 0.0
|
||||
newA[p][p] = app
|
||||
newA[q][q] = aqq
|
||||
newA[p][q] = apq
|
||||
newA[q][p] = apq
|
||||
for r := 0; r < n; r++ {
|
||||
if r == p || r == q {
|
||||
continue
|
||||
}
|
||||
arp := c*a[r][p] + s*a[r][q]
|
||||
arq := -s*a[r][p] + c*a[r][q]
|
||||
newA[r][p] = arp
|
||||
newA[p][r] = arp
|
||||
newA[r][q] = arq
|
||||
newA[q][r] = arq
|
||||
}
|
||||
a = newA
|
||||
}
|
||||
|
||||
eigs := make([]float64, n)
|
||||
for i := range eigs {
|
||||
eigs[i] = a[i][i]
|
||||
}
|
||||
return eigs
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
package eval_test
|
||||
|
||||
import (
|
||||
"math"
|
||||
"math/rand"
|
||||
"testing"
|
||||
|
||||
"gitea.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
||||
)
|
||||
|
||||
func seededRNG(seed int64) *rand.Rand {
|
||||
return rand.New(rand.NewSource(seed))
|
||||
}
|
||||
|
||||
// ── LinearProbe (val_vol_r2) ──────────────────────────────────────────────────
|
||||
|
||||
func TestLinearProbe_Perfect(t *testing.T) {
|
||||
n := 50
|
||||
emb := make([][]float64, n)
|
||||
y := make([]float64, n)
|
||||
for i := range emb {
|
||||
emb[i] = []float64{float64(i)}
|
||||
y[i] = float64(i)
|
||||
}
|
||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
||||
if r2 < 0.99 {
|
||||
t.Fatalf("perfect predictor: want R²≥0.99, got %.4f", r2)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinearProbe_ConstantTarget(t *testing.T) {
|
||||
n := 40
|
||||
emb := make([][]float64, n)
|
||||
y := make([]float64, n)
|
||||
for i := range emb {
|
||||
emb[i] = []float64{float64(i), float64(i * i)}
|
||||
y[i] = 3.0
|
||||
}
|
||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
||||
if r2 > 0.01 {
|
||||
t.Fatalf("constant target: want R²≤0.01, got %.4f", r2)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinearProbe_NoiseEmbedding(t *testing.T) {
|
||||
rng := seededRNG(42)
|
||||
n := 80
|
||||
emb := make([][]float64, n)
|
||||
y := make([]float64, n)
|
||||
for i := range emb {
|
||||
emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
|
||||
y[i] = float64(i)
|
||||
}
|
||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
||||
if r2 > 0.10 {
|
||||
t.Fatalf("noise embedding: want R²<0.10, got %.4f", r2)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Silhouette ────────────────────────────────────────────────────────────────
|
||||
|
||||
func TestSilhouette_PerfectClusters(t *testing.T) {
|
||||
emb := make([][]float64, 40)
|
||||
labels := make([]int, 40)
|
||||
for i := range emb {
|
||||
if i < 20 {
|
||||
emb[i] = []float64{0.0, 0.0}
|
||||
labels[i] = 0
|
||||
} else {
|
||||
emb[i] = []float64{1000.0, 1000.0}
|
||||
labels[i] = 1
|
||||
}
|
||||
}
|
||||
sil, err := eval.Silhouette(emb, labels)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if sil < 0.95 {
|
||||
t.Fatalf("perfect clusters: want sil≥0.95, got %.4f", sil)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSilhouette_SingleLabel(t *testing.T) {
|
||||
emb := [][]float64{{1, 2}, {3, 4}, {5, 6}}
|
||||
labels := []int{0, 0, 0}
|
||||
_, err := eval.Silhouette(emb, labels)
|
||||
if err == nil {
|
||||
t.Fatal("expected error for single-label input")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSilhouette_RandomClusters(t *testing.T) {
|
||||
rng := seededRNG(7)
|
||||
n := 60
|
||||
emb := make([][]float64, n)
|
||||
labels := make([]int, n)
|
||||
for i := range emb {
|
||||
emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
|
||||
labels[i] = i % 2
|
||||
}
|
||||
sil, err := eval.Silhouette(emb, labels)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if math.Abs(sil) > 0.30 {
|
||||
t.Fatalf("random clusters: want |sil|≤0.30, got %.4f", sil)
|
||||
}
|
||||
}
|
||||
|
||||
// ── EffectiveRank ─────────────────────────────────────────────────────────────
|
||||
|
||||
func TestEffectiveRank_Rank1(t *testing.T) {
|
||||
emb := make([][]float64, 30)
|
||||
for i := range emb {
|
||||
emb[i] = []float64{1.0, 2.0, 3.0, 4.0}
|
||||
}
|
||||
er := eval.EffectiveRank(emb)
|
||||
if er > 1.5 {
|
||||
t.Fatalf("rank-1 matrix: want erank≤1.5, got %.4f", er)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEffectiveRank_FullRank(t *testing.T) {
|
||||
rng := seededRNG(99)
|
||||
dim := 8
|
||||
emb := make([][]float64, 200)
|
||||
for i := range emb {
|
||||
row := make([]float64, dim)
|
||||
for j := range row {
|
||||
row[j] = rng.NormFloat64()
|
||||
}
|
||||
emb[i] = row
|
||||
}
|
||||
er := eval.EffectiveRank(emb)
|
||||
if er < float64(dim)*0.7 {
|
||||
t.Fatalf("full-rank: want erank≥%.1f, got %.4f", float64(dim)*0.7, er)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
package eval
|
||||
|
||||
import "math"
|
||||
|
||||
// VaRBreachRate computes the parametric 99% VaR breach rate and Kupiec POF p-value.
|
||||
//
|
||||
// VaR_99_t = predVol[t] × z99 (z99 = 2.326 for 99% normal VaR)
|
||||
// breach_t = actualVol[t] > VaR_99_t (strict inequality)
|
||||
// breachRate = fraction of breaches over all steps
|
||||
// kupiecP = Kupiec POF p-value: P(chi²(1) > LR) where LR is the likelihood ratio
|
||||
// testing H0: true breach probability = 1%. High p = well-calibrated.
|
||||
//
|
||||
// Returns (0, 1) for empty or mismatched input.
|
||||
func VaRBreachRate(predVol, actualVol []float64, z99 float64) (breachRate, kupiecP float64) {
|
||||
n := len(predVol)
|
||||
if n == 0 || n != len(actualVol) {
|
||||
return 0, 1
|
||||
}
|
||||
|
||||
var n1 int
|
||||
for i := 0; i < n; i++ {
|
||||
if actualVol[i] > predVol[i]*z99 {
|
||||
n1++
|
||||
}
|
||||
}
|
||||
|
||||
breachRate = float64(n1) / float64(n)
|
||||
kupiecP = kupiecPOF(n, n1, 0.01)
|
||||
return
|
||||
}
|
||||
|
||||
// kupiecPOF returns the Kupiec Proportion-of-Failures p-value.
|
||||
// H0: true breach probability = p0 (e.g. 0.01 for 99% VaR).
|
||||
// Returns 1.0 for edge cases (n=0, p_hat=p0).
|
||||
func kupiecPOF(n, n1 int, p0 float64) float64 {
|
||||
if n == 0 {
|
||||
return 1.0
|
||||
}
|
||||
n0 := n - n1
|
||||
phat := float64(n1) / float64(n)
|
||||
|
||||
var lr float64
|
||||
switch n1 {
|
||||
case 0:
|
||||
// 0 × ln(0/p0) = 0 by convention; only the n0 term contributes
|
||||
lr = 2 * float64(n0) * math.Log((1-phat)/(1-p0))
|
||||
case n:
|
||||
// n0 term vanishes
|
||||
lr = 2 * float64(n1) * math.Log(phat/p0)
|
||||
default:
|
||||
lr = 2 * (float64(n1)*math.Log(phat/p0) + float64(n0)*math.Log((1-phat)/(1-p0)))
|
||||
}
|
||||
|
||||
if lr <= 0 {
|
||||
return 1.0
|
||||
}
|
||||
// P(chi²(1) > LR) = erfc(sqrt(LR/2)) [chi²(1) = Z², Z~N(0,1)]
|
||||
return math.Erfc(math.Sqrt(lr / 2))
|
||||
}
|
||||
|
||||
// LinearProbePredict fits ridge regression on (trainEmb, trainY) and returns
|
||||
// predictions for testEmb. Complements LinearProbeTrainTest when the caller
|
||||
// needs the raw predictions (e.g. to compute VaR breach rate).
|
||||
// Returns nil when trainEmb is empty.
|
||||
func LinearProbePredict(trainEmb [][]float64, trainY []float64,
|
||||
testEmb [][]float64, lambda float64) []float64 {
|
||||
n := len(trainEmb)
|
||||
if n == 0 || len(testEmb) == 0 {
|
||||
return nil
|
||||
}
|
||||
d := len(trainEmb[0])
|
||||
p := d + 1
|
||||
|
||||
A := make([][]float64, n)
|
||||
for i, e := range trainEmb {
|
||||
row := make([]float64, p)
|
||||
copy(row, e)
|
||||
row[d] = 1.0
|
||||
A[i] = row
|
||||
}
|
||||
AtA := make([][]float64, p)
|
||||
for i := range AtA {
|
||||
AtA[i] = make([]float64, p)
|
||||
}
|
||||
Aty := make([]float64, p)
|
||||
for i := 0; i < n; i++ {
|
||||
for j := 0; j < p; j++ {
|
||||
Aty[j] += A[i][j] * trainY[i]
|
||||
for k := 0; k < p; k++ {
|
||||
AtA[j][k] += A[i][j] * A[i][k]
|
||||
}
|
||||
}
|
||||
}
|
||||
for j := 0; j < p; j++ {
|
||||
AtA[j][j] += lambda
|
||||
}
|
||||
w := solveCholesky(AtA, Aty)
|
||||
|
||||
preds := make([]float64, len(testEmb))
|
||||
for i, e := range testEmb {
|
||||
row := make([]float64, p)
|
||||
copy(row, e)
|
||||
row[d] = 1.0
|
||||
preds[i] = dot(row, w)
|
||||
}
|
||||
return preds
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
package eval_test
|
||||
|
||||
import (
|
||||
"math"
|
||||
"testing"
|
||||
|
||||
"gitea.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
||||
)
|
||||
|
||||
// ── VaRBreachRate golden tests ──────────────────────────────────────────────
|
||||
//
|
||||
// VaR_99_t = predVol[t] × z99 (parametric 99% normal VaR)
|
||||
// breach_t = actualVol[t] > VaR_99_t
|
||||
// breachRate = mean(breach_t)
|
||||
// kupiecP = Kupiec POF p-value (chi²(1) test, H0: breach rate = 1%)
|
||||
|
||||
func TestVaRBreachRate_ZeroBreaches(t *testing.T) {
|
||||
// 0.02 < 0.01×2.326=0.02326 → no breaches
|
||||
pred := []float64{0.01, 0.01, 0.01}
|
||||
act := []float64{0.02, 0.02, 0.02}
|
||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
||||
if rate != 0 {
|
||||
t.Fatalf("want rate=0, got %.4f", rate)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_AllBreach(t *testing.T) {
|
||||
// 0.03 > 0.02326 → all breach
|
||||
pred := []float64{0.01, 0.01}
|
||||
act := []float64{0.03, 0.03}
|
||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
||||
if math.Abs(rate-1.0) > 1e-9 {
|
||||
t.Fatalf("want rate=1.0, got %.4f", rate)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_Golden(t *testing.T) {
|
||||
// n=10, 2 breaches at indices 0 and 2 → rate=0.2
|
||||
// Kupiec: p_hat=0.2 vs p0=0.01 → strongly reject H0 (p < 0.05)
|
||||
pred := make([]float64, 10)
|
||||
act := make([]float64, 10)
|
||||
for i := range pred {
|
||||
pred[i] = 0.01
|
||||
act[i] = 0.01 // no breach: 0.01 < 0.02326
|
||||
}
|
||||
act[0] = 0.03 // breach
|
||||
act[2] = 0.03 // breach
|
||||
|
||||
rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
|
||||
|
||||
if math.Abs(rate-0.2) > 1e-9 {
|
||||
t.Fatalf("breach rate: want 0.2, got %.4f", rate)
|
||||
}
|
||||
if kupiecP > 0.05 {
|
||||
t.Fatalf("kupiec p-value: want <0.05 (strong reject H0), got %.4f", kupiecP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_PerfectCalibration(t *testing.T) {
|
||||
// n=100, exactly 1 breach → p_hat=0.01=p0 → LR=0 → kupiecP≈1.0
|
||||
n := 100
|
||||
pred := make([]float64, n)
|
||||
act := make([]float64, n)
|
||||
for i := range pred {
|
||||
pred[i] = 0.01
|
||||
act[i] = 0.015 // < 0.02326, no breach
|
||||
}
|
||||
act[0] = 0.025 // > 0.02326, breach
|
||||
|
||||
rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
|
||||
|
||||
if math.Abs(rate-0.01) > 1e-9 {
|
||||
t.Fatalf("breach rate: want 0.01, got %.4f", rate)
|
||||
}
|
||||
if kupiecP < 0.9 {
|
||||
t.Fatalf("kupiec p-value: want ≈1.0 (well calibrated), got %.4f", kupiecP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_EmptyInput(t *testing.T) {
|
||||
rate, kupiecP := eval.VaRBreachRate(nil, nil, 2.326)
|
||||
if rate != 0 || kupiecP != 1 {
|
||||
t.Fatalf("empty: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_LenMismatch(t *testing.T) {
|
||||
rate, kupiecP := eval.VaRBreachRate([]float64{0.01}, []float64{0.01, 0.02}, 2.326)
|
||||
if rate != 0 || kupiecP != 1 {
|
||||
t.Fatalf("mismatch: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestVaRBreachRate_Z99Default(t *testing.T) {
|
||||
// z99=2.326 is the canonical value; test that boundary case works
|
||||
// VaR = 0.01 × 2.326 = 0.02326
|
||||
// actual = 0.02326 → NOT a breach (strict >)
|
||||
pred := []float64{0.01}
|
||||
act := []float64{0.02326}
|
||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
||||
if rate != 0 {
|
||||
t.Fatalf("boundary: exactly at VaR is not a breach; want rate=0, got %.4f", rate)
|
||||
}
|
||||
}
|
||||
|
||||
// ── LinearProbePredict ──────────────────────────────────────────────────────
|
||||
|
||||
func TestLinearProbePredict_PerfectLinear(t *testing.T) {
|
||||
// y = x; predictions should match targets closely
|
||||
n := 20
|
||||
trainEmb := make([][]float64, n)
|
||||
trainY := make([]float64, n)
|
||||
testEmb := make([][]float64, 5)
|
||||
testY := []float64{5, 10, 15, 20, 25}
|
||||
for i := range trainEmb {
|
||||
trainEmb[i] = []float64{float64(i)}
|
||||
trainY[i] = float64(i)
|
||||
}
|
||||
for i := range testEmb {
|
||||
testEmb[i] = []float64{testY[i]}
|
||||
}
|
||||
preds := eval.LinearProbePredict(trainEmb, trainY, testEmb, 1e-3)
|
||||
if len(preds) != len(testEmb) {
|
||||
t.Fatalf("len: want %d, got %d", len(testEmb), len(preds))
|
||||
}
|
||||
for i, p := range preds {
|
||||
if math.Abs(p-testY[i]) > 1.0 {
|
||||
t.Fatalf("pred[%d]: want ≈%.1f, got %.4f", i, testY[i], p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLinearProbePredict_EmptyTrain(t *testing.T) {
|
||||
preds := eval.LinearProbePredict(nil, nil, [][]float64{{1.0}}, 1e-3)
|
||||
if len(preds) != 0 {
|
||||
t.Fatalf("empty train: want nil/empty preds, got len=%d", len(preds))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
// Code generated by templ - DO NOT EDIT.
|
||||
|
||||
// templ: version: v0.3.1020
|
||||
package web
|
||||
|
||||
//lint:file-ignore SA4006 This context is only used if a nested component is present.
|
||||
|
||||
import "github.com/a-h/templ"
|
||||
import templruntime "github.com/a-h/templ/runtime"
|
||||
|
||||
func Index() templ.Component {
|
||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||
return templ_7745c5c3_CtxErr
|
||||
}
|
||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||
if !templ_7745c5c3_IsBuffer {
|
||||
defer func() {
|
||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err == nil {
|
||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||
}
|
||||
}()
|
||||
}
|
||||
ctx = templ.InitializeContext(ctx)
|
||||
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
|
||||
if templ_7745c5c3_Var1 == nil {
|
||||
templ_7745c5c3_Var1 = templ.NopComponent
|
||||
}
|
||||
ctx = templ.ClearChildren(ctx)
|
||||
templ_7745c5c3_Var2 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||
if !templ_7745c5c3_IsBuffer {
|
||||
defer func() {
|
||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err == nil {
|
||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||
}
|
||||
}()
|
||||
}
|
||||
ctx = templ.InitializeContext(ctx)
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<h1 class=\"text-3xl font-semibold mb-6\">hostexecutor</h1><button hx-get=\"/api/hello\" hx-target=\"#out\" class=\"px-4 py-2 bg-slate-900 text-white rounded-md hover:bg-slate-700\">Say hello</button><div id=\"out\" class=\"mt-6 text-slate-700\"></div>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
templ_7745c5c3_Err = Layout("hostexecutor").Render(templ.WithChildren(ctx, templ_7745c5c3_Var2), templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
func Hello(name string) templ.Component {
|
||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||
return templ_7745c5c3_CtxErr
|
||||
}
|
||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||
if !templ_7745c5c3_IsBuffer {
|
||||
defer func() {
|
||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err == nil {
|
||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||
}
|
||||
}()
|
||||
}
|
||||
ctx = templ.InitializeContext(ctx)
|
||||
templ_7745c5c3_Var3 := templ.GetChildren(ctx)
|
||||
if templ_7745c5c3_Var3 == nil {
|
||||
templ_7745c5c3_Var3 = templ.NopComponent
|
||||
}
|
||||
ctx = templ.ClearChildren(ctx)
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "<p>Hello, ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var4 string
|
||||
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(name)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/index.templ`, Line: 15, Col: 17}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "!</p>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
var _ = templruntime.GeneratedTemplate
|
||||
@@ -0,0 +1,61 @@
|
||||
// Code generated by templ - DO NOT EDIT.
|
||||
|
||||
// templ: version: v0.3.1020
|
||||
package web
|
||||
|
||||
//lint:file-ignore SA4006 This context is only used if a nested component is present.
|
||||
|
||||
import "github.com/a-h/templ"
|
||||
import templruntime "github.com/a-h/templ/runtime"
|
||||
|
||||
func Layout(title string) templ.Component {
|
||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||
return templ_7745c5c3_CtxErr
|
||||
}
|
||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||
if !templ_7745c5c3_IsBuffer {
|
||||
defer func() {
|
||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err == nil {
|
||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||
}
|
||||
}()
|
||||
}
|
||||
ctx = templ.InitializeContext(ctx)
|
||||
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
|
||||
if templ_7745c5c3_Var1 == nil {
|
||||
templ_7745c5c3_Var1 = templ.NopComponent
|
||||
}
|
||||
ctx = templ.ClearChildren(ctx)
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<!doctype html><html lang=\"en\"><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><title>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var2 string
|
||||
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/layout.templ`, Line: 9, Col: 17}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</title><script src=\"https://unpkg.com/htmx.org@2.0.0\"></script><script src=\"https://cdn.tailwindcss.com\"></script></head><body class=\"min-h-screen bg-slate-50 text-slate-900 antialiased\"><main class=\"max-w-3xl mx-auto px-6 py-12\">")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templ_7745c5c3_Var1.Render(ctx, templ_7745c5c3_Buffer)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "</main></body></html>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
var _ = templruntime.GeneratedTemplate
|
||||
@@ -0,0 +1,287 @@
|
||||
"""loop.py — Karpathy-style autoresearch loop for jepa-fx-risk.
|
||||
|
||||
Agent (on iguana/berget — NOT koala, whose GPU is reserved for train.py) reads
|
||||
program.md + train.py + STATUS.md, proposes ONE change to train.py, we run it,
|
||||
keep if val_vol_r2 improved else git-revert. Appends per-iter record to STATUS.md.
|
||||
|
||||
LITELLM_KEY=xxx python loop.py [--iters N] [--model MODEL] [--run-dir runs/rq-04]
|
||||
|
||||
Env:
|
||||
LITELLM_KEY — LiteLLM master key (required)
|
||||
LITELLM_BASE — default http://localhost:30401/v1
|
||||
LOOP_MODEL — default berget/gemma4-31b (non-thinking; iguana/berget only)
|
||||
LOOP_ITERS — default 3
|
||||
TRAIN_TIMEOUT — seconds per train.py run, default 120
|
||||
NTFY_URL — optional: POST crash/stall alerts here (e.g. ntfy.sh/<topic>)
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import urllib.request
|
||||
|
||||
LITELLM_BASE = os.environ.get("LITELLM_BASE", "http://localhost:30401/v1")
|
||||
LITELLM_KEY = os.environ.get("LITELLM_KEY", "")
|
||||
LOOP_MODEL = os.environ.get("LOOP_MODEL", "berget/gemma4-31b")
|
||||
LOOP_ITERS = int(os.environ.get("LOOP_ITERS", "3"))
|
||||
TRAIN_TIMEOUT = int(os.environ.get("TRAIN_TIMEOUT", "120"))
|
||||
NTFY_URL = os.environ.get("NTFY_URL", "")
|
||||
|
||||
# Resolved by main() once --run-dir is parsed.
|
||||
RUN_DIR = Path(".")
|
||||
STATUS_MD = Path("STATUS.md")
|
||||
METRICS_JSON = Path("metrics.json")
|
||||
TRAIN_PY = Path("train.py")
|
||||
HEARTBEAT = Path("HEARTBEAT")
|
||||
|
||||
AGENT_SYSTEM = textwrap.dedent("""\
|
||||
You are the autoresearch agent for jepa-fx-risk. Your job: propose ONE small,
|
||||
targeted change to train.py to improve val_vol_r2 (OOS R² predicting 1-day
|
||||
realized vol from frozen embeddings). Higher is better.
|
||||
|
||||
Rules:
|
||||
- Return ONLY the full new content of train.py — nothing else, no explanation,
|
||||
no markdown fence. Raw Python only.
|
||||
- Change ONE thing at a time (one knob, one structural idea).
|
||||
- Do NOT touch prepare_data.py, loop.py, or the data pipeline — only train.py.
|
||||
- Do NOT add new data sources or new files.
|
||||
- The metric is computed externally from your frozen embeddings; trust it.
|
||||
""")
|
||||
|
||||
|
||||
def read_file(p: Path) -> str:
|
||||
return p.read_text() if p.exists() else ""
|
||||
|
||||
|
||||
def gpu_snapshot() -> str:
|
||||
try:
|
||||
out = subprocess.check_output(
|
||||
["nvidia-smi", "--query-gpu=utilization.gpu,memory.used,memory.total,temperature.gpu",
|
||||
"--format=csv,noheader,nounits"], timeout=5, text=True
|
||||
).strip()
|
||||
util, mem_used, mem_total, temp = [x.strip() for x in out.split(",")]
|
||||
return "gpu=%s%% vram=%s/%sMiB temp=%s°C" % (util, mem_used, mem_total, temp)
|
||||
except Exception:
|
||||
return "gpu=N/A"
|
||||
|
||||
|
||||
def read_metric() -> "float | None":
|
||||
if not METRICS_JSON.exists():
|
||||
return None
|
||||
try:
|
||||
return float(json.loads(METRICS_JSON.read_text())["val_vol_r2"])
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def run_train() -> "tuple[float | None, float, str]":
|
||||
"""Run train.py from project root with METRICS_OUT pointing into the run dir."""
|
||||
t0 = time.time()
|
||||
env = dict(os.environ)
|
||||
env["METRICS_OUT"] = str(METRICS_JSON.resolve())
|
||||
# train.py is copied into the run dir, so sys.path[0] is that run dir — which
|
||||
# has no scripts/. Put the project root (where loop.py + scripts/ live) on
|
||||
# PYTHONPATH so train.py's `from scripts.var_breach import ...` resolves.
|
||||
env["PYTHONPATH"] = str(Path(__file__).resolve().parent) + os.pathsep + env.get("PYTHONPATH", "")
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[sys.executable, str(TRAIN_PY.resolve())],
|
||||
capture_output=True, text=True, timeout=TRAIN_TIMEOUT, env=env,
|
||||
)
|
||||
elapsed = time.time() - t0
|
||||
if r.returncode != 0:
|
||||
return None, elapsed, (r.stderr or r.stdout)[-300:]
|
||||
metric = read_metric()
|
||||
return metric, elapsed, ""
|
||||
except subprocess.TimeoutExpired:
|
||||
return None, TRAIN_TIMEOUT, "TIMEOUT"
|
||||
|
||||
|
||||
def call_agent(iteration: int, best_so_far: "float | None") -> str:
|
||||
"""Ask the LLM agent to edit train.py. Returns new train.py content."""
|
||||
context = "\n\n".join([
|
||||
"# program.md\n" + read_file(RUN_DIR / "program.md"),
|
||||
"# train.py (current)\n" + read_file(TRAIN_PY),
|
||||
"# STATUS.md (history)\n" + read_file(STATUS_MD)[-2000:],
|
||||
"# metrics.json (last run)\n" + read_file(METRICS_JSON),
|
||||
"Iteration %d. Best val_vol_r2 so far: %s. Improve it." % (
|
||||
iteration, "%.4f" % best_so_far if best_so_far is not None else "none yet"
|
||||
),
|
||||
])
|
||||
payload = json.dumps({
|
||||
"model": LOOP_MODEL,
|
||||
"messages": [
|
||||
{"role": "system", "content": AGENT_SYSTEM},
|
||||
{"role": "user", "content": context},
|
||||
],
|
||||
"temperature": 0.7,
|
||||
"max_tokens": 4096,
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
LITELLM_BASE + "/chat/completions",
|
||||
data=payload,
|
||||
headers={"Authorization": "Bearer " + LITELLM_KEY,
|
||||
"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
)
|
||||
resp = urllib.request.urlopen(req, timeout=60)
|
||||
data = json.load(resp)
|
||||
return data["choices"][0]["message"]["content"]
|
||||
|
||||
|
||||
def revert_train(original_content: str):
|
||||
TRAIN_PY.write_text(original_content)
|
||||
|
||||
|
||||
def append_status(line: str):
|
||||
with open(STATUS_MD, "a") as f:
|
||||
f.write(line + "\n")
|
||||
|
||||
|
||||
def write_heartbeat(iteration: int, status: str = "alive"):
|
||||
"""Update HEARTBEAT so watchdogs can detect stalls."""
|
||||
HEARTBEAT.write_text("%s iter=%d ts=%.0f\n" % (status, iteration, time.time()))
|
||||
|
||||
|
||||
def ntfy(msg: str):
|
||||
"""POST an alert to NTFY_URL (best-effort; silently ignored on any error)."""
|
||||
if not NTFY_URL:
|
||||
return
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
NTFY_URL, data=msg.encode(), method="POST",
|
||||
headers={"Content-Type": "text/plain"},
|
||||
)
|
||||
urllib.request.urlopen(req, timeout=5)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def main():
|
||||
global RUN_DIR, STATUS_MD, METRICS_JSON, TRAIN_PY, HEARTBEAT
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--iters", type=int, default=LOOP_ITERS)
|
||||
parser.add_argument("--model", default=LOOP_MODEL)
|
||||
parser.add_argument(
|
||||
"--run-dir", default=None,
|
||||
help="run dir scaffolded by autoresearch_start.py; "
|
||||
"STATUS.md, metrics.json, HEARTBEAT, and train.py live here",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
loop_iters = args.iters
|
||||
loop_model = args.model
|
||||
|
||||
if args.run_dir:
|
||||
RUN_DIR = Path(args.run_dir)
|
||||
if not RUN_DIR.is_dir():
|
||||
print("ERROR: run dir not found:", RUN_DIR); sys.exit(1)
|
||||
|
||||
STATUS_MD = RUN_DIR / "STATUS.md"
|
||||
METRICS_JSON = RUN_DIR / "metrics.json"
|
||||
TRAIN_PY = RUN_DIR / "train.py"
|
||||
HEARTBEAT = RUN_DIR / "HEARTBEAT"
|
||||
|
||||
if not LITELLM_KEY:
|
||||
print("ERROR: set LITELLM_KEY"); sys.exit(1)
|
||||
|
||||
if not STATUS_MD.exists():
|
||||
STATUS_MD.write_text(
|
||||
"# Autoresearch STATUS\n\n"
|
||||
"| iter | val_vol_r2 | delta | action | secs | gpu | change |\n"
|
||||
"|------|-----------|-------|--------|------|-----|--------|\n"
|
||||
)
|
||||
|
||||
baseline = read_metric()
|
||||
if baseline is None:
|
||||
print("No metrics.json — running train.py for baseline...")
|
||||
m, secs, err = run_train()
|
||||
if m is None:
|
||||
msg = "Baseline run failed: " + err
|
||||
print(msg)
|
||||
ntfy("[jepa-fx-risk] loop CRASH — " + msg)
|
||||
sys.exit(1)
|
||||
baseline = m
|
||||
print("Baseline: val_vol_r2 = %.4f (%.1fs)" % (baseline, secs))
|
||||
|
||||
best = baseline
|
||||
print("Starting loop | model=%s | iters=%d | baseline=%.4f" % (loop_model, loop_iters, best))
|
||||
if args.run_dir:
|
||||
print(" run-dir:", RUN_DIR)
|
||||
|
||||
iter_index = 0
|
||||
try:
|
||||
for i in range(1, loop_iters + 1):
|
||||
iter_index = i
|
||||
write_heartbeat(i, "agent-call")
|
||||
print("\n--- iter %d/%d ---" % (i, loop_iters))
|
||||
original = TRAIN_PY.read_text()
|
||||
|
||||
print(" calling agent (%s)..." % loop_model)
|
||||
t_agent = time.time()
|
||||
try:
|
||||
new_code = call_agent(i, best)
|
||||
except Exception as e:
|
||||
msg = str(e)
|
||||
print(" agent call failed:", msg)
|
||||
append_status("| %d | ERR | — | agent-fail | — | — | %s |" % (i, msg[:60]))
|
||||
write_heartbeat(i, "agent-fail")
|
||||
ntfy("[jepa-fx-risk] iter %d agent FAIL — %s" % (i, msg[:80]))
|
||||
continue
|
||||
agent_secs = time.time() - t_agent
|
||||
print(" agent replied in %.1fs" % agent_secs)
|
||||
|
||||
# strip accidental markdown fences
|
||||
if new_code.strip().startswith("```"):
|
||||
lines = new_code.strip().splitlines()
|
||||
new_code = "\n".join(lines[1:-1] if lines[-1].strip() == "```" else lines[1:])
|
||||
|
||||
TRAIN_PY.write_text(new_code)
|
||||
|
||||
write_heartbeat(i, "training")
|
||||
gpu = gpu_snapshot()
|
||||
print(" running train.py [%s]..." % gpu)
|
||||
metric, secs, err = run_train()
|
||||
|
||||
if metric is None:
|
||||
print(" train.py FAILED — reverting. err:", err[:100])
|
||||
revert_train(original)
|
||||
append_status("| %d | FAIL | — | revert | %.0fs | %s | run error |" % (i, secs, gpu))
|
||||
write_heartbeat(i, "train-fail")
|
||||
ntfy("[jepa-fx-risk] iter %d train FAIL — %s" % (i, err[:80]))
|
||||
continue
|
||||
|
||||
delta = metric - best
|
||||
if metric > best:
|
||||
best = metric
|
||||
action = "KEEP"
|
||||
else:
|
||||
revert_train(original)
|
||||
action = "revert"
|
||||
|
||||
summary = "| %d | %.4f | %+.4f | %s | %.0fs | %s | iter%d |" % (
|
||||
i, metric, delta, action, secs, gpu, i)
|
||||
append_status(summary)
|
||||
write_heartbeat(i, "done")
|
||||
print(" val_vol_r2=%.4f delta=%+.4f action=%s [%.0fs]" % (metric, delta, action, secs))
|
||||
|
||||
except Exception as e:
|
||||
msg = "loop CRASH at iter %d: %s" % (iter_index, e)
|
||||
print("FATAL:", msg)
|
||||
ntfy("[jepa-fx-risk] " + msg)
|
||||
raise
|
||||
|
||||
print("\nDone. Best val_vol_r2 = %.4f (baseline was %.4f, delta %+.4f)" % (best, baseline, best - baseline))
|
||||
print("STATUS.md updated.")
|
||||
write_heartbeat(loop_iters, "done")
|
||||
ntfy("[jepa-fx-risk] loop done. best val_vol_r2=%.4f (delta %+.4f)" % (best, best - baseline))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,10 @@
|
||||
# Python deps for the autoresearch loop (train.py + scripts). Install torch from
|
||||
# the cu130 index FIRST (koala Blackwell sm_120, torch 2.12.1+cu130 verified):
|
||||
# pip install torch --index-url https://download.pytorch.org/whl/cu130
|
||||
# pip install -r requirements.txt
|
||||
numpy>=2.0
|
||||
pandas>=2.2
|
||||
pyarrow>=16
|
||||
histdata>=1.1 # histdata.com downloader (1.1 is newest on PyPI; 1.3 never existed)
|
||||
hmmlearn>=0.3 # regime detector (prepare_regime.py, jepa-fx-risk#13)
|
||||
scikit-learn>=1.4 # HMM dependency
|
||||
@@ -0,0 +1,18 @@
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.302411480667525, "phase1_r2": 0.35809940099716187, "stdout_last": "val_vol_r2 = 0.3024 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:37.028406"}
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29654798431244755, "phase1_r2": 0.35618388652801514, "stdout_last": "val_vol_r2 = 0.2965 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:49.822762"}
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.31050360040290237, "phase1_r2": 0.36530405282974243, "stdout_last": "val_vol_r2 = 0.3105 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:02.966176"}
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.2925057399716364, "phase1_r2": 0.3467639684677124, "stdout_last": "val_vol_r2 = 0.2925 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:16.195552"}
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29334667623516786, "phase1_r2": 0.35872191190719604, "stdout_last": "val_vol_r2 = 0.2933 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:31.372602"}
|
||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3123527205416422, "phase1_r2": 0.3572431206703186, "stdout_last": "val_vol_r2 = 0.3124 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:46.672203"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3641397896593044, "phase1_r2": 0.3908407688140869, "stdout_last": "val_vol_r2 = 0.3641 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:00.793878"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.35845865364171503, "phase1_r2": 0.3737195134162903, "stdout_last": "val_vol_r2 = 0.3585 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:13.321428"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.35310115657814645, "phase1_r2": 0.35306859016418457, "stdout_last": "val_vol_r2 = 0.3531 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:26.402249"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3655629727960601, "phase1_r2": 0.371029257774353, "stdout_last": "val_vol_r2 = 0.3656 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:39.786248"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.36109622605593217, "phase1_r2": 0.3666273355484009, "stdout_last": "val_vol_r2 = 0.3611 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:53.194111"}
|
||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.362228341965093, "phase1_r2": 0.3590735197067261, "stdout_last": "val_vol_r2 = 0.3622 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:06.991680"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3749483295047378, "phase1_r2": 0.3801569938659668, "stdout_last": "val_vol_r2 = 0.3749 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:21.512210"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.3765593861479334, "phase1_r2": 0.38416117429733276, "stdout_last": "val_vol_r2 = 0.3766 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:34.959919"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3653399117639956, "phase1_r2": 0.3685130476951599, "stdout_last": "val_vol_r2 = 0.3653 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:48.806135"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.375961424966925, "phase1_r2": 0.37861257791519165, "stdout_last": "val_vol_r2 = 0.3760 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:04.056939"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.37841726893098504, "phase1_r2": 0.3781360387802124, "stdout_last": "val_vol_r2 = 0.3784 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:18.693263"}
|
||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.37118530199441635, "phase1_r2": 0.3651617765426636, "stdout_last": "val_vol_r2 = 0.3712 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:34.796157"}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"label": "null",
|
||||
"mean_sil": 0.018206419112781685,
|
||||
"pca_sil": 0.13589094579219818,
|
||||
"spread": 0.8673340065023978,
|
||||
"pc1_hv_corr": 0.525803392278542,
|
||||
"per_seed": [
|
||||
0.026790648698806763,
|
||||
0.010999602265655994,
|
||||
0.016829006373882294
|
||||
],
|
||||
"passed": false
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# Phase-0 SSL feasibility gate — null
|
||||
|
||||
**Date:** 2026-06-24
|
||||
**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
|
||||
(not Go #4); gate metric adapted from silhouette-on-embedding to match
|
||||
available data. Go harness (#4) remains open for production experiments.
|
||||
|
||||
## Data
|
||||
- Train: EUR/USD daily 2019-2021 (907 windows)
|
||||
- OOS: EUR/USD daily 2022-2023 (593 windows)
|
||||
- HV label: top-33% realized-vol days = high-volatility (196 days)
|
||||
|
||||
## Results
|
||||
| | Value | Gate |
|
||||
|---|---|---|
|
||||
| TS-JEPA mean silhouette (3 seeds) | 0.0182 | > 0.20 → **False** |
|
||||
| Beats PCA baseline (0.1359) | 0.0182 | > PCA → **False** |
|
||||
| Seed stability (spread) | 86.73% | < 10% → **False** |
|
||||
| PC1/HV correlation | 0.5258 | < 0.95 → **True** |
|
||||
|
||||
Per-seed: ['0.0268', '0.0110', '0.0168']
|
||||
|
||||
## Verdict: **NULL**
|
||||
|
||||
One or more gate criteria not met. See null result protocol in #5.
|
||||
@@ -0,0 +1,155 @@
|
||||
"""autoresearch start — scaffold a run dir from an Autoresearch Council backlog leaf.
|
||||
|
||||
Usage:
|
||||
python scripts/autoresearch_start.py <backlog.json> <rq-id>
|
||||
|
||||
Reads the Council backlog JSON (from agentsquad autoresearch_pipe.py Stage-3 output),
|
||||
finds the node by rq-id, validates it is autoresearch-ready (fail-closed), then
|
||||
scaffolds runs/<rq-id>/ with:
|
||||
|
||||
program.md — hypothesis, single metric (stripped), agent search-space seam
|
||||
run.json — provenance (strategic_question + council_node) + config
|
||||
train.py — copy of project train.py (the loop edits this, keeps history clean)
|
||||
|
||||
Launch:
|
||||
LITELLM_KEY=xxx python loop.py --run-dir runs/<rq-id>
|
||||
|
||||
Refs: jepa-fx-risk#11, agentsquad#44
|
||||
"""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def load_backlog(path: str) -> dict:
|
||||
try:
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
except FileNotFoundError:
|
||||
print(f"error: backlog file not found: {path}", file=sys.stderr)
|
||||
raise
|
||||
|
||||
|
||||
def scaffold_run(
|
||||
backlog_path_or_dict,
|
||||
rq_id: str,
|
||||
run_dir: Path,
|
||||
train_py_src: Path,
|
||||
) -> None:
|
||||
"""Scaffold a run dir. Raises SystemExit on any validation failure."""
|
||||
if isinstance(backlog_path_or_dict, (str, Path)):
|
||||
backlog = load_backlog(str(backlog_path_or_dict))
|
||||
else:
|
||||
backlog = backlog_path_or_dict
|
||||
|
||||
# Find node
|
||||
nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
|
||||
if rq_id not in nodes_by_id:
|
||||
print(f"error: rq-id {rq_id!r} not found in backlog", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
node = nodes_by_id[rq_id]
|
||||
|
||||
# Fail-closed: only autoresearch-ready nodes may be scaffolded
|
||||
status = node.get("status", "")
|
||||
if status != "autoresearch-ready":
|
||||
print(
|
||||
f"error: {rq_id} has status {status!r}, not 'autoresearch-ready' — refusing to scaffold",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
# Guard against overwriting an existing run
|
||||
if run_dir.exists():
|
||||
print(
|
||||
f"error: {run_dir} already exists — remove it first to re-scaffold",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
metric = (node.get("candidate_metric") or "").strip()
|
||||
strategic_q = backlog.get("strategic_question", "")
|
||||
council_node = node["id"]
|
||||
generated_at = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
run_dir.mkdir(parents=True)
|
||||
|
||||
# --- program.md ---
|
||||
program_md = f"""# program.md — {council_node}: {node.get("question", "")[:80]}
|
||||
|
||||
## Provenance
|
||||
- strategic_question: {json.dumps(strategic_q)}
|
||||
- council_node: {council_node} (autoresearch-ready; Autoresearch Council backlog)
|
||||
- generated_at: {generated_at}
|
||||
|
||||
## Hypothesis
|
||||
{node.get("question", "")}
|
||||
|
||||
## Single validation metric (optimise this, nothing else)
|
||||
`{metric}` — see eval harness for the exact definition. Only this scalar drives
|
||||
keep/revert decisions. Report alongside but do NOT optimise:
|
||||
- Kupiec POF p-value (calibration sanity)
|
||||
- val_vol_r2 (representation quality guard)
|
||||
|
||||
## What the agent MAY modify (the search space)
|
||||
- Hyperparameters in train.py (model size, LR, window, patch_len, epochs, etc.)
|
||||
- Conditioning mechanisms (e.g. JEPA_ENABLE_REGIME toggle)
|
||||
- Loss function weights and architecture depth
|
||||
|
||||
## Frozen (do NOT touch — keeps the ablation clean)
|
||||
- Data pipeline and splits (train ≤2021, OOS ≥2022, test 2024 held out)
|
||||
- The metric definition and scoring code
|
||||
- loop.py, scripts/, tests/
|
||||
|
||||
## Experiment loop (per Karpathy autoresearch)
|
||||
Each iter (≤ time-box): apply ONE change to train.py → run → read
|
||||
`{metric}` → keep if improved (and Kupiec p-value did not collapse), else revert.
|
||||
Stop on: target reached, max iters, or K consecutive iters with no improvement.
|
||||
"""
|
||||
(run_dir / "program.md").write_text(program_md)
|
||||
|
||||
# --- run.json (provenance + config) ---
|
||||
run_meta = {
|
||||
"strategic_question": strategic_q,
|
||||
"council_node": council_node,
|
||||
"metric": metric,
|
||||
"generated_at": generated_at,
|
||||
"model_tier": "homelab",
|
||||
"max_iters": 10,
|
||||
"time_box_minutes": 5,
|
||||
}
|
||||
(run_dir / "run.json").write_text(json.dumps(run_meta, indent=2) + "\n")
|
||||
|
||||
# --- train.py (loop edits this copy; project root train.py is the template) ---
|
||||
shutil.copy(train_py_src, run_dir / "train.py")
|
||||
|
||||
|
||||
def main() -> None:
|
||||
if len(sys.argv) != 3:
|
||||
print("usage: python scripts/autoresearch_start.py <backlog.json> <rq-id>")
|
||||
sys.exit(1)
|
||||
|
||||
backlog_path, rq_id = sys.argv[1], sys.argv[2]
|
||||
|
||||
project_root = Path(__file__).parent.parent
|
||||
run_dir = project_root / "runs" / rq_id
|
||||
train_py_src = project_root / "train.py"
|
||||
|
||||
scaffold_run(backlog_path, rq_id, run_dir, train_py_src)
|
||||
|
||||
backlog = load_backlog(backlog_path)
|
||||
nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
|
||||
metric = (nodes_by_id[rq_id].get("candidate_metric") or "").strip()
|
||||
|
||||
print(f"✓ scaffolded {run_dir}")
|
||||
print(f" node: {rq_id}")
|
||||
print(f" metric: {metric}")
|
||||
print()
|
||||
print("launch:")
|
||||
print(f" LITELLM_KEY=xxx python loop.py --run-dir runs/{rq_id}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Phase-0 compute gate (brain wiki/jepa-fx/facts/autoresearch-integration-phase1):
|
||||
PyTorch cu130 must see the koala Blackwell GPU and compute before any experiment.
|
||||
|
||||
python scripts/check_gpu.py # exits 0 if the GPU is usable, 1 otherwise
|
||||
|
||||
Note: koala shares this 12GB card with the llama-swap LLM stack. The autoresearch
|
||||
agent should run on iguana/berget models so koala's GPU stays free for train.py.
|
||||
"""
|
||||
import sys
|
||||
import torch
|
||||
|
||||
print("torch", torch.__version__)
|
||||
if not torch.cuda.is_available():
|
||||
print("CUDA NOT AVAILABLE — gate BLOCKED")
|
||||
sys.exit(1)
|
||||
print("device:", torch.cuda.get_device_name(0))
|
||||
print("capability: sm_%d%d" % torch.cuda.get_device_capability(0))
|
||||
x = torch.randn(2000, 2000, device="cuda")
|
||||
(x @ x).sum().item()
|
||||
torch.cuda.synchronize()
|
||||
print("GPU matmul OK — Phase-0 compute gate GREEN")
|
||||
@@ -0,0 +1,31 @@
|
||||
"""Fetch EUR/USD M1 bars from histdata.com (free, research use).
|
||||
|
||||
Polite: one request per year, spaced; past years query month=None. Uses the
|
||||
maintained `histdata` package which handles histdata's anti-hotlink tk token.
|
||||
Output: data/raw/DAT_ASCII_EURUSD_M1_<year>.zip
|
||||
|
||||
YEARS=2019,2020,2021 python scripts/fetch_data.py
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
|
||||
from histdata import download_hist_data
|
||||
from histdata.api import Platform as P, TimeFrame as T
|
||||
|
||||
YEARS = [y.strip() for y in os.environ.get("YEARS", "2019,2020,2021").split(",")]
|
||||
|
||||
|
||||
def main():
|
||||
os.makedirs("data/raw", exist_ok=True)
|
||||
for yr in YEARS:
|
||||
f = download_hist_data(
|
||||
year=yr, month=None, pair="eurusd",
|
||||
platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
|
||||
output_directory="data/raw",
|
||||
)
|
||||
print("fetched", yr, "->", f)
|
||||
time.sleep(2) # be a good citizen
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,48 @@
|
||||
"""Fetch G10 FX M1 data from histdata.com for all pairs except EURUSD (already fetched).
|
||||
|
||||
Each pair's zips go into data/raw/{pair}/ to avoid collisions.
|
||||
Output: data/raw/gbpusd/DAT_ASCII_GBPUSD_M1_YYYY.zip etc.
|
||||
|
||||
python scripts/fetch_multipair.py
|
||||
PAIRS=gbpusd,usdjpy YEARS=2020,2021 python scripts/fetch_multipair.py
|
||||
"""
|
||||
import os
|
||||
import time
|
||||
|
||||
from histdata import download_hist_data
|
||||
from histdata.api import Platform as P, TimeFrame as T
|
||||
|
||||
PAIRS_DEFAULT = ["gbpusd", "usdjpy", "usdchf", "audusd"]
|
||||
YEARS_DEFAULT = list(range(2008, 2024))
|
||||
|
||||
|
||||
def main():
|
||||
pairs_env = os.environ.get("PAIRS", "")
|
||||
pairs = [p.strip() for p in pairs_env.split(",")] if pairs_env else PAIRS_DEFAULT
|
||||
|
||||
years_env = os.environ.get("YEARS", "")
|
||||
years = [int(y.strip()) for y in years_env.split(",")] if years_env else YEARS_DEFAULT
|
||||
|
||||
for pair in pairs:
|
||||
out_dir = f"data/raw/{pair}"
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
print(f"\n=== {pair.upper()} ===")
|
||||
for yr in years:
|
||||
out_path = os.path.join(out_dir, f"DAT_ASCII_{pair.upper()}_M1_{yr}.zip")
|
||||
if os.path.exists(out_path):
|
||||
print(f" {yr} already present, skip")
|
||||
continue
|
||||
try:
|
||||
f = download_hist_data(
|
||||
year=str(yr), month=None, pair=pair,
|
||||
platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
|
||||
output_directory=out_dir,
|
||||
)
|
||||
print(f" fetched {yr} → {f}")
|
||||
except Exception as e:
|
||||
print(f" {yr} FAILED: {e}")
|
||||
time.sleep(2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,97 @@
|
||||
"""HPO sweep for jepa-fx-risk HEPA backbone.
|
||||
|
||||
Runs train.py with different JEPA_* env overrides, logs results to
|
||||
results/hpo/hpo_results.jsonl. Each config writes its metrics.json then
|
||||
the result is appended to the JSONL.
|
||||
|
||||
Usage:
|
||||
python scripts/hpo_sweep.py
|
||||
python scripts/hpo_sweep.py --dry-run # print configs, don't train
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from datetime import datetime
|
||||
from itertools import product
|
||||
from pathlib import Path
|
||||
|
||||
# ── Search space ──────────────────────────────────────────────────────────────
|
||||
SEARCH_SPACE = {
|
||||
"JEPA_D_MODEL": [64, 128, 256],
|
||||
"JEPA_DEPTH": [2, 4],
|
||||
"JEPA_WINDOW": [120, 240, 480],
|
||||
}
|
||||
# Fixed: PATCH_LEN=24 (1-day patches), N_HEADS=4, EPOCHS=300, PHASE1_EPOCHS=200
|
||||
|
||||
PYTHON = str(Path(sys.executable))
|
||||
OUT_DIR = Path("results/hpo")
|
||||
|
||||
|
||||
def configs():
|
||||
"""Yield all configs as dicts of JEPA_* env overrides."""
|
||||
keys = list(SEARCH_SPACE.keys())
|
||||
for vals in product(*SEARCH_SPACE.values()):
|
||||
yield dict(zip(keys, vals))
|
||||
|
||||
|
||||
def run_config(cfg: dict, metrics_path: str = "metrics.json") -> dict:
|
||||
env = {**os.environ, **{k: str(v) for k, v in cfg.items()}}
|
||||
result = subprocess.run(
|
||||
[PYTHON, "train.py"],
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return {"config": cfg, "error": result.stderr[-500:]}
|
||||
stdout_last = result.stdout.strip().split("\n")[-1]
|
||||
with open(metrics_path) as f:
|
||||
m = json.load(f)
|
||||
return {
|
||||
"config": cfg,
|
||||
"val_vol_r2": m.get("val_vol_r2"),
|
||||
"phase1_r2": m.get("phase1_r2"),
|
||||
"stdout_last": stdout_last,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
args = parser.parse_args()
|
||||
|
||||
OUT_DIR.mkdir(parents=True, exist_ok=True)
|
||||
out_file = OUT_DIR / "hpo_results.jsonl"
|
||||
|
||||
all_cfgs = list(configs())
|
||||
print(f"HPO sweep: {len(all_cfgs)} configs")
|
||||
for i, cfg in enumerate(all_cfgs):
|
||||
label = " ".join(f"{k.replace('JEPA_','')}={v}" for k, v in cfg.items())
|
||||
print(f"\n[{i+1}/{len(all_cfgs)}] {label}")
|
||||
if args.dry_run:
|
||||
continue
|
||||
ts = datetime.utcnow().isoformat()
|
||||
row = run_config(cfg)
|
||||
row["ts"] = ts
|
||||
with open(out_file, "a") as f:
|
||||
f.write(json.dumps(row) + "\n")
|
||||
if "error" in row:
|
||||
print(f" ERROR: {row['error'][:200]}")
|
||||
else:
|
||||
print(f" val_vol_r2={row['val_vol_r2']:.4f} phase1_r2={row['phase1_r2']:.4f}")
|
||||
|
||||
if not args.dry_run:
|
||||
# Print leaderboard
|
||||
rows = [json.loads(l) for l in open(out_file) if l.strip()]
|
||||
rows = [r for r in rows if "error" not in r]
|
||||
rows.sort(key=lambda r: r.get("phase1_r2", -999), reverse=True)
|
||||
print("\n── Leaderboard (by phase1_r2) ─────────────────────────")
|
||||
for r in rows[:5]:
|
||||
cfg_str = " ".join(f"{k.replace('JEPA_','')}={v}" for k,v in r["config"].items())
|
||||
print(f" {r['phase1_r2']:.4f} {cfg_str}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,237 @@
|
||||
"""Phase-0 SSL feasibility gate (path B — Python fast-close of #5).
|
||||
|
||||
Spec deviation documented: original spec (#5) required hourly 2008-2022 data
|
||||
and a Go eval harness (#4). Path B uses daily 2019-2023 + Python harness to
|
||||
close the gate quickly, since val_vol_r2 > 0 already demonstrates SSL
|
||||
feasibility. The Go harness (#4) remains open for production experiments.
|
||||
|
||||
Gate criteria (from #5):
|
||||
- Silhouette > 0.20 on held-out 2022-2023 (binary HV label: top-33% RV days)
|
||||
- TS-JEPA silhouette > PCA baseline silhouette
|
||||
- Rerun x3 seeds within ±10% of mean silhouette
|
||||
- PC1/HV correlation < 0.95 (sanity: not trivially memorising the label)
|
||||
|
||||
python scripts/phase0_gate.py
|
||||
"""
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
from sklearn.decomposition import PCA
|
||||
from sklearn.metrics import silhouette_score
|
||||
from sklearn.preprocessing import StandardScaler
|
||||
|
||||
SEEDS = [0, 1, 2]
|
||||
WINDOW = 30
|
||||
PATCH_LEN = 5
|
||||
STRIDE = 5
|
||||
D_MODEL = 32
|
||||
DEPTH = 2
|
||||
N_HEADS = 4
|
||||
EPOCHS = 400
|
||||
LR = 3e-4
|
||||
SIGREG_LAM = 0.5
|
||||
HV_PERCENTILE = 67 # top-33% = "high volatility"
|
||||
|
||||
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
||||
|
||||
|
||||
# ── SIGReg ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def sigreg(tokens: torch.Tensor, knots: int = 17) -> torch.Tensor:
|
||||
B, T, D = tokens.shape
|
||||
z = tokens.reshape(B * T, D).float()
|
||||
t = torch.linspace(0, 3, knots, device=z.device, dtype=z.dtype)
|
||||
dt = 3.0 / (knots - 1)
|
||||
w = torch.full((knots,), 2 * dt, device=z.device, dtype=z.dtype)
|
||||
w[0] = dt; w[-1] = dt
|
||||
phi = torch.exp(-t.square() / 2.0)
|
||||
A = torch.randn(D, 256, device=z.device, dtype=z.dtype)
|
||||
A = A / A.norm(p=2, dim=0)
|
||||
x_t = (z @ A).unsqueeze(-1) * t
|
||||
err = (x_t.cos().mean(0) - phi).square() + x_t.sin().mean(0).square()
|
||||
return ((err @ (w * phi)) * z.shape[0]).mean()
|
||||
|
||||
|
||||
# ── Encoder ──────────────────────────────────────────────────────────────────
|
||||
|
||||
class PatchEncoder(nn.Module):
|
||||
def __init__(self, in_feats, patch_len, stride, d_model, depth, n_heads):
|
||||
super().__init__()
|
||||
self.patch_len = patch_len
|
||||
self.stride = stride
|
||||
self.embed = nn.Linear(patch_len * in_feats, d_model)
|
||||
layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
|
||||
dropout=0.0, batch_first=True)
|
||||
self.tf = nn.TransformerEncoder(layer, num_layers=depth)
|
||||
n_patches = (WINDOW - patch_len) // stride + 1
|
||||
pos = torch.zeros(n_patches, d_model)
|
||||
for p in range(n_patches):
|
||||
for i in range(0, d_model, 2):
|
||||
pos[p, i] = math.sin(p / 10000 ** (i / d_model))
|
||||
if i + 1 < d_model:
|
||||
pos[p, i+1] = math.cos(p / 10000 ** (i / d_model))
|
||||
self.register_buffer("pos", pos)
|
||||
|
||||
def forward(self, x):
|
||||
B, W, F = x.shape
|
||||
n_patches = (W - self.patch_len) // self.stride + 1
|
||||
patches = torch.stack([x[:, i*self.stride:i*self.stride+self.patch_len, :]
|
||||
.reshape(B, -1) for i in range(n_patches)], dim=1)
|
||||
tokens = self.embed(patches) + self.pos[:n_patches]
|
||||
return self.tf(tokens)
|
||||
|
||||
|
||||
# ── Data ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
def load_data():
|
||||
df = pd.read_parquet("data/processed/eurusd_daily.parquet").reset_index(drop=True)
|
||||
df["date"] = pd.to_datetime(df["date"])
|
||||
train = df[df["date"].dt.year <= 2021].copy()
|
||||
oos = df[df["date"].dt.year >= 2022].copy()
|
||||
|
||||
feats_all = df[["ret", "realized_vol"]].to_numpy(np.float32)
|
||||
target_all = df["realized_vol"].to_numpy(np.float32)
|
||||
dates_all = df["date"].values
|
||||
|
||||
mu = feats_all[:len(train)].mean(0)
|
||||
sd = feats_all[:len(train)].std(0) + 1e-8
|
||||
|
||||
def windows(df_subset, feats_norm, dates):
|
||||
idx_start = df.index[df["date"].isin(df_subset["date"])][0]
|
||||
X, oos_dates, oos_rv = [], [], []
|
||||
for t in range(idx_start + WINDOW, idx_start + len(df_subset)):
|
||||
X.append(feats_norm[t - WINDOW:t])
|
||||
oos_dates.append(dates[t])
|
||||
oos_rv.append(target_all[t])
|
||||
return np.stack(X), np.array(oos_rv), np.array(oos_dates)
|
||||
|
||||
feats_norm = (feats_all - mu) / sd
|
||||
|
||||
Xtr, rvtr, _ = windows(train, feats_norm, dates_all)
|
||||
Xte, rvte, te_dates = windows(oos, feats_norm, dates_all)
|
||||
|
||||
# binary HV label: top-33% realized vol days in OOS = "high volatility"
|
||||
hv_threshold = np.percentile(rvte, HV_PERCENTILE)
|
||||
hv_labels = (rvte >= hv_threshold).astype(int)
|
||||
|
||||
return Xtr, rvtr, Xte, rvte, hv_labels
|
||||
|
||||
|
||||
# ── Train + embed ─────────────────────────────────────────────────────────────
|
||||
|
||||
def train_and_embed(Xtr, Xte, seed):
|
||||
torch.manual_seed(seed)
|
||||
np.random.seed(seed)
|
||||
enc = PatchEncoder(Xtr.shape[2], PATCH_LEN, STRIDE, D_MODEL, DEPTH, N_HEADS).to(dev)
|
||||
pred = nn.Sequential(nn.Linear(D_MODEL, D_MODEL), nn.GELU(),
|
||||
nn.Linear(D_MODEL, D_MODEL)).to(dev)
|
||||
opt = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
|
||||
|
||||
Xtr_t = torch.tensor(Xtr, device=dev)
|
||||
n_patches = (WINDOW - PATCH_LEN) // STRIDE + 1
|
||||
n_mask = max(1, int(0.30 * n_patches))
|
||||
|
||||
for ep in range(EPOCHS):
|
||||
idx_mask = torch.randperm(n_patches)[:n_mask]
|
||||
tokens_ctx = enc(Xtr_t)
|
||||
tokens_target = enc(Xtr_t).detach()
|
||||
jepa_loss = ((pred(tokens_ctx[:, idx_mask, :]) -
|
||||
tokens_target[:, idx_mask, :]) ** 2).mean()
|
||||
reg = sigreg(tokens_ctx)
|
||||
loss = jepa_loss + SIGREG_LAM * reg
|
||||
opt.zero_grad(); loss.backward(); opt.step()
|
||||
|
||||
enc.eval()
|
||||
with torch.no_grad():
|
||||
Ete = enc(torch.tensor(Xte, device=dev)).mean(1).cpu().numpy()
|
||||
|
||||
return Ete
|
||||
|
||||
|
||||
# ── Gate ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
def pca_baseline(Xte, hv_labels):
|
||||
flat = Xte.reshape(len(Xte), -1)
|
||||
sc = StandardScaler().fit(flat)
|
||||
emb = PCA(n_components=8).fit_transform(sc.transform(flat))
|
||||
return silhouette_score(emb, hv_labels), emb
|
||||
|
||||
|
||||
def main():
|
||||
os.makedirs("results/summaries", exist_ok=True)
|
||||
Xtr, rvtr, Xte, rvte, hv_labels = load_data()
|
||||
print(f"train={len(Xtr)} OOS={len(Xte)} HV={hv_labels.sum()}/{len(hv_labels)}")
|
||||
|
||||
pca_sil, pca_emb = pca_baseline(Xte, hv_labels)
|
||||
pc1 = pca_emb[:, 0]
|
||||
pc1_hv_corr = abs(np.corrcoef(pc1, hv_labels)[0, 1])
|
||||
print(f"PCA baseline silhouette = {pca_sil:.4f} | PC1/HV |r| = {pc1_hv_corr:.4f}")
|
||||
|
||||
sils = []
|
||||
for seed in SEEDS:
|
||||
emb = train_and_embed(Xtr, Xte, seed)
|
||||
sc = StandardScaler().fit(emb)
|
||||
sil = silhouette_score(sc.transform(emb), hv_labels)
|
||||
sils.append(sil)
|
||||
print(f" seed={seed} silhouette={sil:.4f}")
|
||||
|
||||
mean_sil = np.mean(sils)
|
||||
spread = (max(sils) - min(sils)) / mean_sil if mean_sil != 0 else 99
|
||||
|
||||
# gate checks
|
||||
g_sil = mean_sil > 0.20
|
||||
g_beats = mean_sil > pca_sil
|
||||
g_stable = spread < 0.10
|
||||
g_corr = pc1_hv_corr < 0.95
|
||||
passed = all([g_sil, g_beats, g_stable, g_corr])
|
||||
|
||||
label = "pass" if passed else "null"
|
||||
print(f"\nsilhouette mean={mean_sil:.4f} spread={spread:.2%} PCA={pca_sil:.4f} PC1/HV={pc1_hv_corr:.4f}")
|
||||
print(f"gate: sil>0.20={g_sil} beats_pca={g_beats} stable={g_stable} corr<0.95={g_corr}")
|
||||
print(f"PHASE-0: {label.upper()}")
|
||||
|
||||
summary = f"""# Phase-0 SSL feasibility gate — {label}
|
||||
|
||||
**Date:** 2026-06-24
|
||||
**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
|
||||
(not Go #4); gate metric adapted from silhouette-on-embedding to match
|
||||
available data. Go harness (#4) remains open for production experiments.
|
||||
|
||||
## Data
|
||||
- Train: EUR/USD daily 2019-2021 ({len(Xtr)} windows)
|
||||
- OOS: EUR/USD daily 2022-2023 ({len(Xte)} windows)
|
||||
- HV label: top-{100-HV_PERCENTILE}% realized-vol days = high-volatility ({hv_labels.sum()} days)
|
||||
|
||||
## Results
|
||||
| | Value | Gate |
|
||||
|---|---|---|
|
||||
| TS-JEPA mean silhouette (3 seeds) | {mean_sil:.4f} | > 0.20 → **{g_sil}** |
|
||||
| Beats PCA baseline ({pca_sil:.4f}) | {mean_sil:.4f} | > PCA → **{g_beats}** |
|
||||
| Seed stability (spread) | {spread:.2%} | < 10% → **{g_stable}** |
|
||||
| PC1/HV correlation | {pc1_hv_corr:.4f} | < 0.95 → **{g_corr}** |
|
||||
|
||||
Per-seed: {[f"{s:.4f}" for s in sils]}
|
||||
|
||||
## Verdict: **{label.upper()}**
|
||||
|
||||
{"All 4 gate criteria met. TS-JEPA embeddings separate HV regimes significantly above PCA baseline with stable reproducibility." if passed else "One or more gate criteria not met. See null result protocol in #5."}
|
||||
"""
|
||||
path = f"results/summaries/phase-0-{label}.md"
|
||||
with open(path, "w") as f:
|
||||
f.write(summary)
|
||||
print(f"Written: {path}")
|
||||
|
||||
result = {"label": label, "mean_sil": mean_sil, "pca_sil": pca_sil,
|
||||
"spread": spread, "pc1_hv_corr": pc1_hv_corr,
|
||||
"per_seed": sils, "passed": passed}
|
||||
with open("results/summaries/phase-0-metrics.json", "w") as f:
|
||||
json.dump(result, f, indent=2)
|
||||
return 0 if passed else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,57 @@
|
||||
"""LOCKED data pipeline (toy) — agent must NOT edit (brain Phase-1 contract).
|
||||
|
||||
Parses histdata EUR/USD M1 zips → daily series with realized volatility (the
|
||||
val_vol_r2 target = 1-day realized vol from intraday squared returns).
|
||||
Output: data/processed/eurusd_daily.parquet [date, close, ret, realized_vol].
|
||||
"""
|
||||
import glob
|
||||
import os
|
||||
import zipfile
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
RAW = "data/raw"
|
||||
OUT = "data/processed/eurusd_daily.parquet"
|
||||
|
||||
|
||||
def load_m1() -> pd.DataFrame:
|
||||
frames = []
|
||||
for zp in sorted(glob.glob(os.path.join(RAW, "DAT_ASCII_EURUSD_M1_*.zip"))):
|
||||
with zipfile.ZipFile(zp) as z:
|
||||
csv = [n for n in z.namelist() if n.endswith(".csv")][0]
|
||||
with z.open(csv) as f:
|
||||
df = pd.read_csv(
|
||||
f, sep=";", header=None,
|
||||
names=["dt", "open", "high", "low", "close", "vol"],
|
||||
)
|
||||
df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
|
||||
frames.append(df[["ts", "close"]])
|
||||
out = pd.concat(frames).sort_values("ts").reset_index(drop=True)
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
m1 = load_m1()
|
||||
m1["r"] = np.log(m1["close"]).diff()
|
||||
m1["day"] = m1["ts"].dt.normalize()
|
||||
daily = m1.groupby("day").agg(
|
||||
close=("close", "last"),
|
||||
realized_vol=("r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
|
||||
n_min=("r", "count"),
|
||||
).reset_index()
|
||||
daily = daily[daily["n_min"] > 60] # drop thin days (holidays)
|
||||
daily["ret"] = np.log(daily["close"]).diff()
|
||||
daily = daily.dropna().reset_index(drop=True)
|
||||
os.makedirs(os.path.dirname(OUT), exist_ok=True)
|
||||
daily[["day", "close", "ret", "realized_vol"]].rename(columns={"day": "date"}).to_parquet(OUT)
|
||||
print("rows:", len(daily), "| dates:", daily["day"].min().date(), "→", daily["day"].max().date())
|
||||
# sanity: the COVID crash (March 2020) must show a realized-vol spike
|
||||
rv = daily.set_index("day")["realized_vol"]
|
||||
mar20 = rv["2020-03-01":"2020-03-31"].max()
|
||||
typ = rv["2019-01-01":"2019-12-31"].median()
|
||||
print("median 2019 RV: %.5f | max Mar-2020 RV: %.5f | spike x%.1f" % (typ, mar20, mar20 / typ))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,128 @@
|
||||
"""Prepare EUR/USD hourly OHLCV + realized vol from histdata M1 zips.
|
||||
|
||||
Aggregates all M1 bars in data/raw/DAT_ASCII_EURUSD_M1_*.zip to hourly.
|
||||
Realized vol per hour = sqrt(sum(log-return²)) over the constituent M1 bars.
|
||||
Weekend hours are naturally absent (FX market closed Sat/Sun); NO interpolation.
|
||||
Hours with fewer than MIN_BARS M1 bars are dropped (holidays, thin sessions).
|
||||
|
||||
Output: data/processed/eurusd_hourly.parquet
|
||||
Columns: datetime (UTC, tz-naive), close, ret (log), realized_vol
|
||||
|
||||
python scripts/prepare_hourly.py
|
||||
RAW=data/raw OUT=data/processed/eurusd_hourly.parquet python scripts/prepare_hourly.py
|
||||
"""
|
||||
import glob
|
||||
import os
|
||||
import zipfile
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
PAIR = os.environ.get("PAIR", "EURUSD").upper()
|
||||
RAW_DEFAULT = "data/raw"
|
||||
OUT_DEFAULT = f"data/processed/{PAIR.lower()}_hourly.parquet"
|
||||
MIN_BARS = 30 # drop hours thinner than this (holidays, DST boundary artefacts)
|
||||
|
||||
|
||||
# ── Core transformation ──────────────────────────────────────────────────────
|
||||
|
||||
def resample_to_hourly(m1: pd.DataFrame) -> pd.DataFrame:
|
||||
"""Aggregate M1 DataFrame to hourly bars.
|
||||
|
||||
Args:
|
||||
m1: DataFrame with columns ['ts', 'open', 'high', 'low', 'close']
|
||||
('open'/'high'/'low' optional — omit for close-only data).
|
||||
|
||||
Returns:
|
||||
DataFrame with columns ['datetime', 'close', 'ret', 'realized_vol',
|
||||
'hl_range', 'ret_intrabar'] sorted by datetime.
|
||||
Hours with fewer than MIN_BARS M1 ticks are dropped.
|
||||
"""
|
||||
m1 = m1.sort_values("ts").copy()
|
||||
m1["log_r"] = np.log(m1["close"]).diff()
|
||||
m1["hour"] = m1["ts"].dt.floor("h")
|
||||
|
||||
has_ohlc = all(c in m1.columns for c in ("open", "high", "low"))
|
||||
|
||||
agg_dict = dict(
|
||||
close = ("close", "last"),
|
||||
realized_vol = ("log_r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
|
||||
n_bars = ("log_r", "count"),
|
||||
)
|
||||
if has_ohlc:
|
||||
agg_dict["high"] = ("high", "max")
|
||||
agg_dict["low"] = ("low", "min")
|
||||
agg_dict["open_"] = ("open", "first")
|
||||
|
||||
agg = m1.groupby("hour").agg(**agg_dict).reset_index()
|
||||
|
||||
agg = agg[agg["n_bars"] >= MIN_BARS].copy()
|
||||
agg["ret"] = np.log(agg["close"]).diff()
|
||||
agg = agg.dropna(subset=["ret"]).reset_index(drop=True)
|
||||
agg = agg.rename(columns={"hour": "datetime"})
|
||||
|
||||
if has_ohlc:
|
||||
agg["hl_range"] = np.log(agg["high"] / agg["low"])
|
||||
agg["ret_intrabar"]= np.log(agg["close"] / agg["open_"])
|
||||
cols = ["datetime", "close", "ret", "realized_vol", "hl_range", "ret_intrabar"]
|
||||
else:
|
||||
cols = ["datetime", "close", "ret", "realized_vol"]
|
||||
|
||||
return agg[cols]
|
||||
|
||||
|
||||
def load_m1_from_zips(raw_dir: str, pair: str = None) -> pd.DataFrame:
|
||||
"""Load and concatenate all M1 zips from raw_dir (histdata format)."""
|
||||
p = (pair or PAIR).upper()
|
||||
pattern = os.path.join(raw_dir, f"DAT_ASCII_{p}_M1_*.zip")
|
||||
zips = sorted(glob.glob(pattern))
|
||||
if not zips:
|
||||
raise FileNotFoundError(f"No M1 zips found at {pattern}")
|
||||
frames = []
|
||||
for zp in zips:
|
||||
with zipfile.ZipFile(zp) as z:
|
||||
csv = [n for n in z.namelist() if n.endswith(".csv")][0]
|
||||
with z.open(csv) as f:
|
||||
df = pd.read_csv(
|
||||
f, sep=";", header=None,
|
||||
names=["dt", "open", "high", "low", "close", "vol"],
|
||||
)
|
||||
df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
|
||||
frames.append(df[["ts", "open", "high", "low", "close"]])
|
||||
print(f" loaded {os.path.basename(zp)}: {len(df):,} rows")
|
||||
return pd.concat(frames).sort_values("ts").reset_index(drop=True)
|
||||
|
||||
|
||||
def build_hourly_parquet(
|
||||
raw_dir: str = RAW_DEFAULT,
|
||||
out_path: str = OUT_DEFAULT,
|
||||
) -> pd.DataFrame:
|
||||
"""Full pipeline: load all M1 zips → hourly parquet. Returns the DataFrame."""
|
||||
print(f"Loading M1 zips from {raw_dir}...")
|
||||
m1 = load_m1_from_zips(raw_dir)
|
||||
print(f"Total M1 bars: {len(m1):,} ({m1['ts'].min().date()} → {m1['ts'].max().date()})")
|
||||
|
||||
print("Resampling to hourly...")
|
||||
hourly = resample_to_hourly(m1)
|
||||
print(f"Hourly rows: {len(hourly):,} ({hourly['datetime'].min()} → {hourly['datetime'].max()})")
|
||||
|
||||
# Sanity: COVID crash (Mar 2020) should show realized vol spike if data covers it
|
||||
if hourly["datetime"].dt.year.isin([2020]).any():
|
||||
rv = hourly.set_index("datetime")["realized_vol"]
|
||||
try:
|
||||
mar20 = rv["2020-03-01":"2020-03-31"].max()
|
||||
typ = rv["2019-01-01":"2019-12-31"].median()
|
||||
print(f"Sanity — median 2019 RV: {typ:.6f} | max Mar-2020 RV: {mar20:.6f} | spike ×{mar20/typ:.1f}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True)
|
||||
hourly.to_parquet(out_path, index=False)
|
||||
print(f"Written: {out_path}")
|
||||
return hourly
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raw_dir = os.environ.get("RAW", RAW_DEFAULT)
|
||||
out_path = os.environ.get("OUT", OUT_DEFAULT)
|
||||
build_hourly_parquet(raw_dir=raw_dir, out_path=out_path)
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Merge per-pair hourly parquets into a single wide multipair parquet.
|
||||
|
||||
Each pair contributes two features: {pair}_ret and {pair}_rv (realized vol).
|
||||
The merge is an INNER JOIN on datetime — only hours present in ALL pairs are kept.
|
||||
The target for train.py remains eurusd_rv.
|
||||
|
||||
Output: data/processed/eurusd_multipair.parquet
|
||||
|
||||
python scripts/prepare_multipair.py
|
||||
PROCESSED=data/processed python scripts/prepare_multipair.py
|
||||
"""
|
||||
import os
|
||||
import pandas as pd
|
||||
|
||||
PAIRS = ["eurusd", "gbpusd", "usdjpy", "usdchf", "audusd"]
|
||||
|
||||
PROCESSED_DEFAULT = "data/processed"
|
||||
OUT_DEFAULT = "data/processed/eurusd_multipair.parquet"
|
||||
|
||||
|
||||
def merge_pair_parquets(pair_dfs: dict) -> pd.DataFrame:
|
||||
"""Inner-join hourly DataFrames from multiple pairs on datetime.
|
||||
|
||||
Args:
|
||||
pair_dfs: dict mapping pair name (e.g. "eurusd") to hourly DataFrame
|
||||
with columns [datetime, close, ret, realized_vol, ...].
|
||||
Returns:
|
||||
Wide DataFrame with columns:
|
||||
datetime, {pair}_ret, {pair}_rv for each pair.
|
||||
"""
|
||||
merged = None
|
||||
for pair, df in pair_dfs.items():
|
||||
sub = df[["datetime", "ret", "realized_vol"]].copy()
|
||||
sub = sub.rename(columns={"ret": f"{pair}_ret", "realized_vol": f"{pair}_rv"})
|
||||
sub = sub.set_index("datetime")
|
||||
if merged is None:
|
||||
merged = sub
|
||||
else:
|
||||
merged = merged.join(sub, how="inner")
|
||||
|
||||
return merged.reset_index()
|
||||
|
||||
|
||||
def build_multipair_parquet(
|
||||
processed_dir: str = PROCESSED_DEFAULT,
|
||||
out_path: str = OUT_DEFAULT,
|
||||
pairs: list = None,
|
||||
) -> None:
|
||||
if pairs is None:
|
||||
pairs = PAIRS
|
||||
pair_dfs = {}
|
||||
for pair in pairs:
|
||||
path = os.path.join(processed_dir, f"{pair}_hourly.parquet")
|
||||
if not os.path.exists(path):
|
||||
raise FileNotFoundError(
|
||||
f"{pair}_hourly.parquet not found at {path} — run prepare_hourly.py for this pair first"
|
||||
)
|
||||
df = pd.read_parquet(path)
|
||||
pair_dfs[pair] = df
|
||||
|
||||
merged = merge_pair_parquets(pair_dfs)
|
||||
merged.to_parquet(out_path, index=False)
|
||||
n_pairs = len(pairs)
|
||||
n_ch = n_pairs * 2
|
||||
print(f"Multipair parquet: {len(merged):,} rows × {n_ch} feature channels ({n_pairs} pairs)")
|
||||
print(f"Date range: {merged['datetime'].min()} → {merged['datetime'].max()}")
|
||||
print(f"Written: {out_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
processed_dir = os.environ.get("PROCESSED", PROCESSED_DEFAULT)
|
||||
build_multipair_parquet(processed_dir=processed_dir)
|
||||
@@ -0,0 +1,134 @@
|
||||
"""HMM regime detector — 3-state Gaussian HMM on realized_vol.
|
||||
|
||||
Fits on the FULL dataset (training + OOS) so the state sequence is globally
|
||||
consistent across all periods. States are sorted by mean realized vol (ascending):
|
||||
0 = calm, 1 = stressed, 2 = crisis
|
||||
|
||||
Output: data/processed/eurusd_regime.parquet
|
||||
Columns: datetime (or date), regime (int: 0/1/2)
|
||||
|
||||
Deterministic: fixed random_state=42 throughout.
|
||||
Cached: if the parquet already exists, it is not re-computed.
|
||||
|
||||
Usage:
|
||||
python scripts/prepare_regime.py [--hourly] [--daily] [--force]
|
||||
|
||||
jepa-fx-risk#13
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
from hmmlearn import hmm
|
||||
|
||||
DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
|
||||
HOURLY_PATH = DATA_DIR / "eurusd_hourly.parquet"
|
||||
DAILY_PATH = DATA_DIR / "eurusd_daily.parquet"
|
||||
OUTPUT_PATH = DATA_DIR / "eurusd_regime.parquet"
|
||||
|
||||
N_STATES = 3
|
||||
RANDOM_STATE = 42
|
||||
|
||||
|
||||
def fit_regime_hmm(realized_vol: np.ndarray, n_states: int = 3, random_state: int = 42) -> np.ndarray:
|
||||
"""Fit a Gaussian HMM on realized_vol and return state labels (0=calm → n_states-1=crisis).
|
||||
|
||||
States are sorted by mean realized vol ascending so label 0 is always calm,
|
||||
label n_states-1 is always crisis. This makes the labelling deterministic
|
||||
across datasets with different vol levels.
|
||||
|
||||
Args:
|
||||
realized_vol: 1-D array of realized vol values
|
||||
n_states: number of HMM hidden states (default 3)
|
||||
random_state: random seed for reproducibility
|
||||
|
||||
Returns:
|
||||
Integer label array of shape (len(realized_vol),), dtype int64
|
||||
"""
|
||||
X = realized_vol.reshape(-1, 1).astype(np.float64)
|
||||
model = hmm.GaussianHMM(
|
||||
n_components=n_states,
|
||||
covariance_type="diag",
|
||||
min_covar=1e-6,
|
||||
n_iter=100,
|
||||
random_state=random_state,
|
||||
tol=1e-4,
|
||||
)
|
||||
model.fit(X)
|
||||
raw_labels = model.predict(X)
|
||||
|
||||
# Sort states by mean realized vol (ascending: calm=0, crisis=n_states-1)
|
||||
state_means = np.array([X[raw_labels == s].mean() if (raw_labels == s).any() else 0.0
|
||||
for s in range(n_states)])
|
||||
rank = np.argsort(state_means) # rank[0] = original state id of the calmest cluster
|
||||
remap = np.empty(n_states, dtype=np.int64)
|
||||
for new_label, old_label in enumerate(rank):
|
||||
remap[old_label] = new_label
|
||||
return remap[raw_labels].astype(np.int64)
|
||||
|
||||
|
||||
def prepare_regime_df(parquet_path: str, freq: str = "hourly") -> pd.DataFrame:
|
||||
"""Load parquet, fit HMM, return DataFrame with timestamp + regime columns.
|
||||
|
||||
Args:
|
||||
parquet_path: path to input parquet (hourly or daily)
|
||||
freq: "hourly" | "daily" — determines timestamp column name
|
||||
|
||||
Returns:
|
||||
DataFrame with columns: (datetime|date), regime
|
||||
"""
|
||||
df = pd.read_parquet(parquet_path)
|
||||
if freq == "hourly":
|
||||
ts = pd.to_datetime(df["datetime"])
|
||||
else:
|
||||
ts = pd.to_datetime(df["date"])
|
||||
|
||||
rv = df["realized_vol"].to_numpy(np.float32)
|
||||
labels = fit_regime_hmm(rv, n_states=N_STATES, random_state=RANDOM_STATE)
|
||||
return pd.DataFrame({"datetime": ts.values, "regime": labels})
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Fit HMM regime detector")
|
||||
parser.add_argument("--hourly", action="store_true", default=True,
|
||||
help="use hourly parquet (default)")
|
||||
parser.add_argument("--daily", action="store_true", default=False,
|
||||
help="use daily parquet instead of hourly")
|
||||
parser.add_argument("--force", action="store_true", default=False,
|
||||
help="overwrite existing output")
|
||||
parser.add_argument("--out", default=str(OUTPUT_PATH),
|
||||
help="output parquet path")
|
||||
args = parser.parse_args()
|
||||
|
||||
out_path = Path(args.out)
|
||||
if out_path.exists() and not args.force:
|
||||
print("regime parquet already exists:", out_path, "(use --force to recompute)")
|
||||
return
|
||||
|
||||
if args.daily and DAILY_PATH.exists():
|
||||
src, freq = str(DAILY_PATH), "daily"
|
||||
elif HOURLY_PATH.exists():
|
||||
src, freq = str(HOURLY_PATH), "hourly"
|
||||
elif DAILY_PATH.exists():
|
||||
src, freq = str(DAILY_PATH), "daily"
|
||||
else:
|
||||
raise FileNotFoundError("no parquet found in data/processed/")
|
||||
|
||||
print(f"fitting HMM ({N_STATES} states) on {src} ...")
|
||||
df = prepare_regime_df(src, freq=freq)
|
||||
|
||||
counts = df["regime"].value_counts().sort_index()
|
||||
print("regime distribution:")
|
||||
for state, count in counts.items():
|
||||
label = {0: "calm", 1: "stressed", 2: "crisis"}.get(state, f"state{state}")
|
||||
print(f" {state} ({label}): {count} ({100*count/len(df):.1f}%)")
|
||||
|
||||
df.to_parquet(out_path, index=False)
|
||||
print("wrote:", out_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Parametric 99% VaR breach rate + Kupiec POF p-value.
|
||||
|
||||
Used by train.py's LOCKED VaR EVAL BLOCK to write VaR_breach_rate_99_oos_regime_cond
|
||||
to metrics.json so the autoresearch loop can optimise it.
|
||||
|
||||
jepa-fx-risk#12
|
||||
"""
|
||||
|
||||
import math
|
||||
|
||||
# Canonical metric key — no surrounding whitespace, as required by the loop contract.
|
||||
METRIC_KEY = "VaR_breach_rate_99_oos_regime_cond"
|
||||
|
||||
# Default normal 99th-percentile z-score.
|
||||
Z99 = 2.326
|
||||
|
||||
|
||||
def var_breach_rate(pred_vol, actual_vol, z99=Z99):
|
||||
"""Compute VaR breach rate and Kupiec POF p-value.
|
||||
|
||||
Args:
|
||||
pred_vol: iterable of predicted conditional vol forecasts
|
||||
actual_vol: iterable of actual realized vol (same length)
|
||||
z99: 99th-percentile z-score (default 2.326)
|
||||
|
||||
Returns:
|
||||
(breach_rate, kupiec_p) where:
|
||||
breach_rate — fraction of steps where actual_vol > pred_vol × z99
|
||||
kupiec_p — Kupiec POF p-value (H0: true breach rate = 1%)
|
||||
High p-value = well-calibrated; low = miscalibrated tail.
|
||||
"""
|
||||
pred_v = list(pred_vol)
|
||||
act_v = list(actual_vol)
|
||||
n = len(pred_v)
|
||||
if n == 0 or n != len(act_v):
|
||||
return 0.0, 1.0
|
||||
|
||||
n1 = sum(1 for p, a in zip(pred_v, act_v) if a > p * z99)
|
||||
breach_rate = n1 / n
|
||||
p = kupiec_pvalue(n, n1)
|
||||
return breach_rate, p
|
||||
|
||||
|
||||
def kupiec_pvalue(n, n1, p0=0.01):
|
||||
"""Kupiec Proportion-of-Failures likelihood ratio test.
|
||||
|
||||
H0: true breach probability = p0.
|
||||
Returns P(chi²(1) > LR) using the identity P(chi²(1)>x) = erfc(sqrt(x/2)).
|
||||
Returns 1.0 for n=0 or LR<=0 (well-calibrated / over-conservative).
|
||||
"""
|
||||
if n == 0:
|
||||
return 1.0
|
||||
n0 = n - n1
|
||||
phat = n1 / n
|
||||
|
||||
if n1 == 0:
|
||||
# 0 × ln(0/p0) = 0 by convention; only n0 term contributes
|
||||
lr = 2 * n0 * math.log((1 - phat) / (1 - p0))
|
||||
elif n1 == n:
|
||||
lr = 2 * n1 * math.log(phat / p0)
|
||||
else:
|
||||
lr = 2 * (n1 * math.log(phat / p0) + n0 * math.log((1 - phat) / (1 - p0)))
|
||||
|
||||
if lr <= 0:
|
||||
return 1.0
|
||||
# P(chi²(1) > LR) = erfc(sqrt(LR/2))
|
||||
return math.erfc(math.sqrt(lr / 2))
|
||||
@@ -0,0 +1,215 @@
|
||||
"""Tests for scripts/autoresearch_start.py — jepa-fx-risk#11 Phase A scaffold.
|
||||
|
||||
Success criterion: `autoresearch start <backlog.json> <rq-id>` scaffolds a
|
||||
runnable run dir from a ready leaf; refuses non-ready nodes; strips
|
||||
candidate_metric; records provenance.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
# Load the module without executing main()
|
||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "autoresearch_start.py"
|
||||
|
||||
|
||||
def _import():
|
||||
spec = importlib.util.spec_from_file_location("autoresearch_start", _SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def mod():
|
||||
return _import()
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def backlog(tmp_path):
|
||||
data = {
|
||||
"strategic_question": "Test strategic question?",
|
||||
"generated_at": "2026-06-27T00:00:00Z",
|
||||
"nodes": [
|
||||
{
|
||||
"id": "rq-01",
|
||||
"question": "Does X improve Y?",
|
||||
"case_type": "autoresearch-loop",
|
||||
"data": "obtainable",
|
||||
"method": "adjacent",
|
||||
"falsifiable": "yes",
|
||||
"candidate_metric": " val_vol_r2", # leading space — bypass test
|
||||
"depends_on": [],
|
||||
"status": "autoresearch-ready",
|
||||
"track": "autoresearch",
|
||||
"converged": True,
|
||||
"survived_review": True,
|
||||
},
|
||||
{
|
||||
"id": "rq-02",
|
||||
"question": "Not ready yet?",
|
||||
"case_type": "empirical-study",
|
||||
"data": "obtainable",
|
||||
"method": "adjacent",
|
||||
"falsifiable": "yes",
|
||||
"candidate_metric": None,
|
||||
"depends_on": [],
|
||||
"status": "needs-metric",
|
||||
"track": "study",
|
||||
"converged": True,
|
||||
"survived_review": True,
|
||||
},
|
||||
{
|
||||
"id": "rq-03",
|
||||
"question": "A spike.",
|
||||
"case_type": "spike",
|
||||
"data": "have",
|
||||
"method": "yes-named",
|
||||
"falsifiable": "yes",
|
||||
"candidate_metric": None,
|
||||
"depends_on": [],
|
||||
"status": "spike-ready",
|
||||
"track": "spike",
|
||||
"converged": True,
|
||||
"survived_review": True,
|
||||
},
|
||||
],
|
||||
}
|
||||
p = tmp_path / "backlog.json"
|
||||
p.write_text(json.dumps(data))
|
||||
return p
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def fake_train_py(tmp_path):
|
||||
"""Minimal train.py placeholder for scaffold tests."""
|
||||
src = tmp_path / "train_template.py"
|
||||
src.write_text("# train.py placeholder\n")
|
||||
return src
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# fail-closed: refuse non-autoresearch-ready nodes
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRefuseNonReady:
|
||||
def test_refuses_needs_metric(self, mod, backlog, fake_train_py, tmp_path):
|
||||
run_dir = tmp_path / "runs" / "rq-02"
|
||||
with pytest.raises(SystemExit) as exc:
|
||||
mod.scaffold_run(backlog, "rq-02", run_dir, fake_train_py)
|
||||
assert exc.value.code != 0
|
||||
|
||||
def test_refuses_spike_ready(self, mod, backlog, fake_train_py, tmp_path):
|
||||
run_dir = tmp_path / "runs" / "rq-03"
|
||||
with pytest.raises(SystemExit) as exc:
|
||||
mod.scaffold_run(backlog, "rq-03", run_dir, fake_train_py)
|
||||
assert exc.value.code != 0
|
||||
|
||||
def test_refuses_missing_rq_id(self, mod, backlog, fake_train_py, tmp_path):
|
||||
run_dir = tmp_path / "runs" / "rq-99"
|
||||
with pytest.raises(SystemExit) as exc:
|
||||
mod.scaffold_run(backlog, "rq-99", run_dir, fake_train_py)
|
||||
assert exc.value.code != 0
|
||||
|
||||
def test_refuses_existing_run_dir(self, mod, backlog, fake_train_py, tmp_path):
|
||||
run_dir = tmp_path / "runs" / "rq-01"
|
||||
run_dir.mkdir(parents=True)
|
||||
with pytest.raises(SystemExit) as exc:
|
||||
mod.scaffold_run(backlog, "rq-01", run_dir, fake_train_py)
|
||||
assert exc.value.code != 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# scaffold structure: correct files created
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestScaffoldStructure:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
||||
|
||||
def test_run_dir_created(self):
|
||||
assert self.run_dir.is_dir()
|
||||
|
||||
def test_program_md_created(self):
|
||||
assert (self.run_dir / "program.md").exists()
|
||||
|
||||
def test_run_json_created(self):
|
||||
assert (self.run_dir / "run.json").exists()
|
||||
|
||||
def test_train_py_copied(self):
|
||||
assert (self.run_dir / "train.py").exists()
|
||||
assert (self.run_dir / "train.py").read_text() == "# train.py placeholder\n"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# program.md content
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestProgramMd:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
||||
self.content = (self.run_dir / "program.md").read_text()
|
||||
|
||||
def test_contains_hypothesis(self):
|
||||
assert "Does X improve Y?" in self.content
|
||||
|
||||
def test_metric_key_stripped(self):
|
||||
# candidate_metric had leading space " val_vol_r2" — must be stripped
|
||||
assert "`val_vol_r2`" in self.content
|
||||
assert "` val_vol_r2`" not in self.content
|
||||
|
||||
def test_contains_strategic_question(self):
|
||||
assert "Test strategic question?" in self.content
|
||||
|
||||
def test_contains_council_node(self):
|
||||
assert "rq-01" in self.content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# run.json provenance
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestRunJson:
|
||||
@pytest.fixture(autouse=True)
|
||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
||||
self.run = json.loads((self.run_dir / "run.json").read_text())
|
||||
|
||||
def test_strategic_question_in_provenance(self):
|
||||
assert self.run["strategic_question"] == "Test strategic question?"
|
||||
|
||||
def test_council_node_in_provenance(self):
|
||||
assert self.run["council_node"] == "rq-01"
|
||||
|
||||
def test_metric_stripped_in_provenance(self):
|
||||
assert self.run["metric"] == "val_vol_r2"
|
||||
assert self.run["metric"] == self.run["metric"].strip()
|
||||
|
||||
def test_generated_at_present(self):
|
||||
assert "generated_at" in self.run
|
||||
|
||||
def test_max_iters_present(self):
|
||||
assert "max_iters" in self.run
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# load_backlog helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestLoadBacklog:
|
||||
def test_loads_json(self, mod, backlog):
|
||||
data = mod.load_backlog(str(backlog))
|
||||
assert data["strategic_question"] == "Test strategic question?"
|
||||
assert len(data["nodes"]) == 3
|
||||
|
||||
def test_missing_file_raises(self, mod, tmp_path):
|
||||
with pytest.raises((FileNotFoundError, SystemExit)):
|
||||
mod.load_backlog(str(tmp_path / "nonexistent.json"))
|
||||
@@ -0,0 +1,277 @@
|
||||
"""Failing tests for HEPA backbone + Phase-1 supervised head + HPO in train.py.
|
||||
|
||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_hepa.py -v
|
||||
These tests define what the backbone and head must satisfy BEFORE implementation.
|
||||
"""
|
||||
import math
|
||||
import os
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import pytest
|
||||
|
||||
# ── Tests import the classes from train.py ────────────────────────────────────
|
||||
# They will fail until train.py implements: CausalEncoder, HorizonPredictor, vicreg_loss
|
||||
|
||||
|
||||
def _import(env_overrides=None):
|
||||
import importlib.util, sys
|
||||
saved = {}
|
||||
if env_overrides:
|
||||
for k, v in env_overrides.items():
|
||||
saved[k] = os.environ.get(k)
|
||||
os.environ[k] = str(v)
|
||||
# Force fresh module load (env vars must be read at import time)
|
||||
name = f"train_{id(env_overrides)}"
|
||||
spec = importlib.util.spec_from_file_location(name, "train.py")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
if env_overrides:
|
||||
for k, orig in saved.items():
|
||||
if orig is None:
|
||||
os.environ.pop(k, None)
|
||||
else:
|
||||
os.environ[k] = orig
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def train_mod():
|
||||
return _import()
|
||||
|
||||
|
||||
# 1. CausalEncoder exists and has correct output shape
|
||||
def test_causal_encoder_shape(train_mod):
|
||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
|
||||
x = torch.randn(4, 60, 2)
|
||||
tokens = enc(x) # should return all tokens (B, N, D) for JEPA pretraining
|
||||
assert tokens.shape == (4, 6, 32), f"expected (4, 6, 32), got {tokens.shape}"
|
||||
|
||||
|
||||
# 2. CausalEncoder is actually causal: earlier token outputs don't change when later inputs change
|
||||
def test_causal_masking(train_mod):
|
||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=2)
|
||||
enc.eval()
|
||||
torch.manual_seed(0)
|
||||
x = torch.randn(1, 60, 2)
|
||||
x_perturbed = x.clone()
|
||||
# non-uniform noise (constant shift absorbed by per-patch LayerNorm; variance change is not)
|
||||
torch.manual_seed(99)
|
||||
x_perturbed[:, 30:, :] += torch.randn_like(x[:, 30:, :]) * 5.0
|
||||
|
||||
with torch.no_grad():
|
||||
h1 = enc(x)
|
||||
h2 = enc(x_perturbed)
|
||||
|
||||
# First 3 tokens must be identical (causal — don't see future patches)
|
||||
assert torch.allclose(h1[:, :3, :], h2[:, :3, :], atol=1e-5), \
|
||||
"causal masking broken: early tokens change when later input changes"
|
||||
# Last token should differ (it can see the perturbed patches)
|
||||
assert not torch.allclose(h1[:, -1, :], h2[:, -1, :], atol=1e-5), \
|
||||
"last token should differ when later input changes"
|
||||
|
||||
|
||||
# 3. HorizonPredictor exists, takes (h, delta_t_float) → same shape as h
|
||||
def test_horizon_predictor_shape(train_mod):
|
||||
pred = train_mod.HorizonPredictor(d_model=32)
|
||||
h = torch.randn(4, 32)
|
||||
dt = torch.tensor([1.0, 2.0, 3.0, 1.0])
|
||||
out = pred(h, dt)
|
||||
assert out.shape == (4, 32), f"expected (4, 32), got {out.shape}"
|
||||
|
||||
|
||||
# 4. vicreg_loss is a scalar and backward doesn't error
|
||||
def test_vicreg_loss_backward(train_mod):
|
||||
h_pred = torch.randn(8, 32, requires_grad=True)
|
||||
h_target = torch.randn(8, 32)
|
||||
loss = train_mod.vicreg_loss(h_pred, h_target, alpha=0.1)
|
||||
assert loss.shape == (), f"expected scalar, got {loss.shape}"
|
||||
loss.backward()
|
||||
assert h_pred.grad is not None
|
||||
|
||||
|
||||
# 5. Full JEPA step: encode context, predict future, compute loss, backward
|
||||
def test_jepa_step_end_to_end(train_mod):
|
||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
|
||||
pred = train_mod.HorizonPredictor(d_model=32)
|
||||
opt = torch.optim.SGD(list(enc.parameters()) + list(pred.parameters()), lr=1e-3)
|
||||
|
||||
x = torch.randn(4, 60, 2)
|
||||
tokens = enc(x) # (4, 6, 32)
|
||||
c, dt = 2, 2 # context position 2, horizon 2
|
||||
h_ctx = tokens[:, c, :]
|
||||
h_tgt = tokens[:, c + dt, :].detach()
|
||||
h_hat = pred(h_ctx, torch.full((4,), float(dt)))
|
||||
loss = train_mod.vicreg_loss(h_hat, h_tgt, alpha=0.1)
|
||||
opt.zero_grad(); loss.backward(); opt.step()
|
||||
assert loss.item() < 100, "loss exploded"
|
||||
|
||||
|
||||
# 6. build() returns year-based OOS split (2022-2023); hourly gives many more windows
|
||||
def test_build_year_split(train_mod):
|
||||
(Xtr, ytr), (Xte, yte) = train_mod.build()
|
||||
assert Xtr.shape[1] == train_mod.WINDOW
|
||||
assert Xte.shape[1] == train_mod.WINDOW
|
||||
assert len(Xtr) > 0 and len(Xte) > 0
|
||||
# OOS: daily ≈ 600; hourly ≈ 17,000 (2 years × ~8,500 trading hours/year)
|
||||
assert len(Xte) > 400, f"OOS too small: {len(Xte)}"
|
||||
|
||||
|
||||
# 7. hourly build gives > 10× more training windows than daily
|
||||
def test_build_hourly_more_windows(train_mod):
|
||||
import os
|
||||
if not os.path.exists("data/processed/eurusd_hourly.parquet"):
|
||||
pytest.skip("eurusd_hourly.parquet not present — run data:prepare:hourly first")
|
||||
(Xtr, _), _ = train_mod.build()
|
||||
# Daily had ~877 train windows; hourly with 2008-2021 should have > 50,000
|
||||
assert len(Xtr) > 10_000, f"expected >10k hourly train windows, got {len(Xtr)}"
|
||||
|
||||
|
||||
# ── Phase-1: supervised head ──────────────────────────────────────────────────
|
||||
|
||||
# 8. SupervisedHead exists and maps (B, D) → (B,)
|
||||
def test_supervised_head_shape(train_mod):
|
||||
D = 128
|
||||
head = train_mod.SupervisedHead(D)
|
||||
x = torch.randn(16, D)
|
||||
out = head(x)
|
||||
assert out.shape == (16,), f"expected (16,), got {out.shape}"
|
||||
|
||||
|
||||
# 9. SupervisedHead gradient flows (not frozen)
|
||||
def test_supervised_head_backward(train_mod):
|
||||
head = train_mod.SupervisedHead(64)
|
||||
x = torch.randn(8, 64)
|
||||
loss = head(x).mean()
|
||||
loss.backward()
|
||||
for name, p in head.named_parameters():
|
||||
assert p.grad is not None, f"no grad on {name}"
|
||||
|
||||
|
||||
# 10. Phase-1 beats linear on nonlinear synthetic signal
|
||||
def test_phase1_beats_linear_on_nonlinear(train_mod):
|
||||
"""MLP head should outperform ridge regression on data with nonlinear structure."""
|
||||
import numpy as np
|
||||
torch.manual_seed(0); np.random.seed(0)
|
||||
N, D = 1000, 32
|
||||
# target = |h|² (quadratic — linear can't fit well)
|
||||
Etr = np.random.randn(N, D).astype(np.float32)
|
||||
ytr = (Etr ** 2).sum(axis=1)
|
||||
Ete = np.random.randn(200, D).astype(np.float32)
|
||||
yte = (Ete ** 2).sum(axis=1)
|
||||
|
||||
# Ridge baseline
|
||||
A = np.hstack([Etr, np.ones((N, 1))])
|
||||
w = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
|
||||
pred_lin = np.hstack([Ete, np.ones((200, 1))]) @ w
|
||||
r2_lin = float(1 - ((yte - pred_lin) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
|
||||
|
||||
# MLP head
|
||||
head = train_mod.SupervisedHead(D)
|
||||
opt = torch.optim.Adam(head.parameters(), lr=1e-2)
|
||||
Xtr_t = torch.tensor(Etr); ytr_t = torch.tensor(ytr)
|
||||
for _ in range(300):
|
||||
loss = nn.functional.mse_loss(head(Xtr_t), ytr_t)
|
||||
opt.zero_grad(); loss.backward(); opt.step()
|
||||
|
||||
head.eval()
|
||||
with torch.no_grad():
|
||||
pred_mlp = head(torch.tensor(Ete)).numpy()
|
||||
r2_mlp = float(1 - ((yte - pred_mlp) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
|
||||
|
||||
assert r2_mlp > r2_lin + 0.05, (
|
||||
f"MLP R²={r2_mlp:.3f} should beat ridge R²={r2_lin:.3f} by >0.05 on quadratic target"
|
||||
)
|
||||
|
||||
|
||||
# 11. main() returns phase1_r2 in metrics.json (integration — needs real data)
|
||||
def test_metrics_json_has_phase1_r2(train_mod):
|
||||
import json
|
||||
if not os.path.exists("metrics.json"):
|
||||
pytest.skip("metrics.json not present — run train.py first")
|
||||
with open("metrics.json") as f:
|
||||
m = json.load(f)
|
||||
assert "phase1_r2" in m, f"phase1_r2 missing from metrics.json: {list(m.keys())}"
|
||||
assert m["phase1_r2"] > m["val_vol_r2"], (
|
||||
f"MLP head phase1_r2={m['phase1_r2']:.4f} should beat linear probe "
|
||||
f"val_vol_r2={m['val_vol_r2']:.4f}"
|
||||
)
|
||||
|
||||
|
||||
# ── HPO: env-var knob overrides ───────────────────────────────────────────────
|
||||
|
||||
# 12. JEPA_WINDOW env var overrides WINDOW at import time
|
||||
def test_env_override_window():
|
||||
mod = _import({"JEPA_WINDOW": "48"})
|
||||
assert mod.WINDOW == 48, f"expected WINDOW=48, got {mod.WINDOW}"
|
||||
|
||||
|
||||
# 13. JEPA_D_MODEL and JEPA_DEPTH env vars work
|
||||
def test_env_override_d_model_depth():
|
||||
mod = _import({"JEPA_D_MODEL": "64", "JEPA_DEPTH": "4"})
|
||||
assert mod.D_MODEL == 64, f"expected D_MODEL=64, got {mod.D_MODEL}"
|
||||
assert mod.DEPTH == 4, f"expected DEPTH=4, got {mod.DEPTH}"
|
||||
|
||||
|
||||
# 14. hpo_sweep.py exists and generates correct config list
|
||||
def test_hpo_sweep_configs():
|
||||
import importlib.util
|
||||
sweep_path = "scripts/hpo_sweep.py"
|
||||
if not os.path.exists(sweep_path):
|
||||
pytest.fail(f"{sweep_path} not found — implement it")
|
||||
spec = importlib.util.spec_from_file_location("hpo_sweep", sweep_path)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
cfgs = list(mod.configs())
|
||||
assert len(cfgs) > 0, "configs() returned empty list"
|
||||
# Every config must have at least D_MODEL, DEPTH, WINDOW keys
|
||||
required = {"JEPA_D_MODEL", "JEPA_DEPTH", "JEPA_WINDOW"}
|
||||
for cfg in cfgs:
|
||||
assert required.issubset(cfg.keys()), f"config missing required keys: {cfg}"
|
||||
|
||||
|
||||
# ── Option B: joint encoder fine-tuning in phase-1 ───────────────────────────
|
||||
|
||||
# 15. PHASE1_JOINT and PHASE1_ENCODER_LR knobs exist at module level
|
||||
def test_joint_phase1_knobs():
|
||||
mod = _import({"JEPA_PHASE1_JOINT": "1", "JEPA_PHASE1_ENCODER_LR": "1e-5"})
|
||||
assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing from train.py"
|
||||
assert hasattr(mod, "PHASE1_ENCODER_LR"), "PHASE1_ENCODER_LR knob missing from train.py"
|
||||
assert mod.PHASE1_JOINT is True
|
||||
assert abs(mod.PHASE1_ENCODER_LR - 1e-5) < 1e-12
|
||||
|
||||
|
||||
# 16. PHASE1_JOINT defaults to True (joint mode on by default)
|
||||
def test_joint_phase1_default_on():
|
||||
mod = _import()
|
||||
assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing"
|
||||
assert mod.PHASE1_JOINT is True, f"PHASE1_JOINT default should be True, got {mod.PHASE1_JOINT}"
|
||||
|
||||
|
||||
# 17. JEPA_PHASE1_JOINT=0 disables joint (env override works)
|
||||
def test_joint_phase1_can_disable():
|
||||
mod = _import({"JEPA_PHASE1_JOINT": "0"})
|
||||
assert mod.PHASE1_JOINT is False, f"expected False, got {mod.PHASE1_JOINT}"
|
||||
|
||||
|
||||
# 18. Encoder receives non-zero gradients when joint-training with the head
|
||||
def test_joint_encoder_grad_flows(train_mod):
|
||||
"""Gradient must flow into encoder when using two-param-group joint optimizer."""
|
||||
import torch.nn.functional as F
|
||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=8, d_model=16, n_heads=2, depth=1)
|
||||
head = train_mod.SupervisedHead(16)
|
||||
enc.train(); head.train()
|
||||
opt = torch.optim.Adam([
|
||||
{"params": head.parameters(), "lr": 1e-3},
|
||||
{"params": enc.parameters(), "lr": 1e-5},
|
||||
], weight_decay=1e-4)
|
||||
# Tiny batch: 4 windows of length 16 (= 2 patches of patch_len=8)
|
||||
X = torch.randn(4, 16, 2)
|
||||
y = torch.randn(4)
|
||||
tokens = enc(X) # (4, 2, 16)
|
||||
h = tokens[:, -1, :] # (4, 16) — last token
|
||||
pred = head(h)
|
||||
loss = F.mse_loss(pred, y)
|
||||
loss.backward()
|
||||
enc_grads = [p.grad for p in enc.parameters() if p.grad is not None]
|
||||
assert len(enc_grads) > 0, "no encoder params received gradients"
|
||||
assert any(g.abs().max().item() > 0 for g in enc_grads), "all encoder grads are zero"
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Tests for multi-pair G10 pipeline (Option C).
|
||||
|
||||
Tests the prepare_multipair.py merge logic and train.py multipair build().
|
||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_multipair.py -v
|
||||
"""
|
||||
import importlib.util
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
import os
|
||||
|
||||
|
||||
def _import_mp():
|
||||
spec = importlib.util.spec_from_file_location("prepare_multipair", "scripts/prepare_multipair.py")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def mp():
|
||||
return _import_mp()
|
||||
|
||||
|
||||
def _pair_df(start: str, n_hours: int, seed: int) -> pd.DataFrame:
|
||||
"""Synthetic single-pair hourly parquet (same schema as prepare_hourly output)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
dts = pd.date_range(start, periods=n_hours, freq="h")
|
||||
closes = 1.1 + np.cumsum(rng.normal(0, 0.001, n_hours))
|
||||
return pd.DataFrame({
|
||||
"datetime": dts,
|
||||
"close": closes,
|
||||
"ret": rng.normal(0, 0.001, n_hours),
|
||||
"realized_vol": np.abs(rng.normal(0.0005, 0.0001, n_hours)),
|
||||
})
|
||||
|
||||
|
||||
# 1. merge_pair_parquets returns inner join on datetime
|
||||
def test_merge_inner_join(mp):
|
||||
eur = _pair_df("2020-01-01 00:00", 100, seed=1) # t0 to t0+99h
|
||||
gbp = _pair_df("2020-01-01 20:00", 60, seed=2) # t0+20 to t0+79h → 60 common
|
||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
||||
assert len(result) == 60, f"expected 60 (inner join), got {len(result)}"
|
||||
|
||||
|
||||
# 2. merge_pair_parquets prefixes columns with pair name
|
||||
def test_merge_column_prefixes(mp):
|
||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
||||
assert "datetime" in result.columns, "datetime column missing"
|
||||
assert "eurusd_ret" in result.columns
|
||||
assert "eurusd_rv" in result.columns
|
||||
assert "gbpusd_ret" in result.columns
|
||||
assert "gbpusd_rv" in result.columns
|
||||
# raw pair columns should not leak through unprefixed
|
||||
assert "ret" not in result.columns
|
||||
assert "realized_vol" not in result.columns
|
||||
|
||||
|
||||
# 3. No NaN in merged output
|
||||
def test_merge_no_nan(mp):
|
||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
||||
nan_count = result.isnull().sum().sum()
|
||||
assert nan_count == 0, f"{nan_count} NaN values in merged output"
|
||||
|
||||
|
||||
# 4. PAIRS constant is a non-empty list starting with eurusd
|
||||
def test_pairs_constant(mp):
|
||||
assert hasattr(mp, "PAIRS"), "PAIRS constant missing from prepare_multipair.py"
|
||||
assert len(mp.PAIRS) >= 2, "PAIRS must have at least 2 pairs"
|
||||
assert mp.PAIRS[0] == "eurusd", "first pair must be eurusd (target pair)"
|
||||
|
||||
|
||||
# 5. merge target column is eurusd_rv (for build() target selection)
|
||||
def test_merge_has_eurusd_rv_as_target(mp):
|
||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
||||
assert "eurusd_rv" in result.columns, "eurusd_rv (target) missing from merged output"
|
||||
assert (result["eurusd_rv"] > 0).all(), "eurusd_rv should be positive"
|
||||
|
||||
|
||||
# 6. train.py recognises JEPA_USE_MULTIPAIR env var
|
||||
def test_use_multipair_knob():
|
||||
import importlib.util as ilu
|
||||
spec = ilu.spec_from_file_location(f"train_mp_{id(None)}", "train.py")
|
||||
mod = ilu.module_from_spec(spec)
|
||||
saved = os.environ.get("JEPA_USE_MULTIPAIR")
|
||||
os.environ["JEPA_USE_MULTIPAIR"] = "1"
|
||||
try:
|
||||
spec.loader.exec_module(mod)
|
||||
finally:
|
||||
if saved is None:
|
||||
os.environ.pop("JEPA_USE_MULTIPAIR", None)
|
||||
else:
|
||||
os.environ["JEPA_USE_MULTIPAIR"] = saved
|
||||
assert hasattr(mod, "USE_MULTIPAIR"), "USE_MULTIPAIR knob missing from train.py"
|
||||
assert mod.USE_MULTIPAIR is True
|
||||
|
||||
|
||||
# 7. build() uses n_pairs*2 channels when multipair parquet present
|
||||
def test_build_uses_multipair_channels():
|
||||
import importlib.util as ilu
|
||||
multipair_path = "data/processed/eurusd_multipair.parquet"
|
||||
if not os.path.exists(multipair_path):
|
||||
pytest.skip("eurusd_multipair.parquet not present — run data:prepare:multipair first")
|
||||
saved = os.environ.get("JEPA_USE_MULTIPAIR")
|
||||
os.environ["JEPA_USE_MULTIPAIR"] = "1"
|
||||
try:
|
||||
spec = ilu.spec_from_file_location(f"train_mp2_{id(None)}", "train.py")
|
||||
mod = ilu.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
(Xtr, _), _ = mod.build()
|
||||
finally:
|
||||
if saved is None:
|
||||
os.environ.pop("JEPA_USE_MULTIPAIR", None)
|
||||
else:
|
||||
os.environ["JEPA_USE_MULTIPAIR"] = saved
|
||||
mp = _import_mp()
|
||||
expected_ch = len(mp.PAIRS) * 2
|
||||
assert Xtr.shape[2] == expected_ch, (
|
||||
f"expected {expected_ch} channels (n_pairs={len(mp.PAIRS)}×2), got {Xtr.shape[2]}"
|
||||
)
|
||||
@@ -0,0 +1,207 @@
|
||||
"""Failing tests for scripts/prepare_hourly.py.
|
||||
|
||||
Tests the M1 → hourly aggregation logic using synthetic data before touching
|
||||
real downloads.
|
||||
|
||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_prepare_hourly.py -v
|
||||
"""
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
import importlib.util, sys, os
|
||||
|
||||
|
||||
def _import():
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"prepare_hourly", "scripts/prepare_hourly.py"
|
||||
)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def ph():
|
||||
return _import()
|
||||
|
||||
|
||||
def _make_m1(n_days: int = 3, price: float = 1.1000, noise: float = 0.0005) -> pd.DataFrame:
|
||||
"""Synthetic M1 DataFrame starting 2020-01-06 (Monday), 390 ticks/day."""
|
||||
rng = np.random.default_rng(42)
|
||||
# generate full trading hours: Mon-Fri 00:00-23:59 (FX is 24h weekday)
|
||||
start = pd.Timestamp("2020-01-06 00:00:00") # Monday
|
||||
periods = n_days * 24 * 60
|
||||
ts = pd.date_range(start, periods=periods, freq="min")
|
||||
# remove weekends
|
||||
ts = ts[ts.day_of_week < 5]
|
||||
prices = price + np.cumsum(rng.normal(0, noise, len(ts)))
|
||||
return pd.DataFrame({"ts": ts, "close": prices})
|
||||
|
||||
|
||||
# 1. resample_to_hourly: DataFrame has correct columns
|
||||
def test_columns(ph):
|
||||
m1 = _make_m1()
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert set(["datetime", "close", "ret", "realized_vol"]).issubset(hourly.columns), \
|
||||
f"missing columns: {hourly.columns.tolist()}"
|
||||
|
||||
|
||||
# 2. No cross-weekend interpolation: gap between Friday 23:xx and Sunday/Monday must remain
|
||||
def test_no_weekend_interpolation(ph):
|
||||
# Make 2 days: Friday + Monday (skip Saturday/Sunday)
|
||||
fri = pd.date_range("2020-01-10 00:00", "2020-01-10 23:59", freq="min") # Friday
|
||||
mon = pd.date_range("2020-01-13 00:00", "2020-01-13 23:59", freq="min") # Monday
|
||||
ts = fri.append(mon)
|
||||
prices = 1.1 + np.cumsum(np.random.default_rng(0).normal(0, 0.0001, len(ts)))
|
||||
m1 = pd.DataFrame({"ts": ts, "close": prices})
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
dates = pd.DatetimeIndex(hourly["datetime"]).date
|
||||
import datetime
|
||||
sat = datetime.date(2020, 1, 11)
|
||||
sun = datetime.date(2020, 1, 12)
|
||||
assert sat not in dates and sun not in dates, "weekend rows found in hourly output"
|
||||
|
||||
|
||||
# 3. Realized vol = sqrt(sum(r²)) over minute returns in each hour
|
||||
def test_realized_vol_formula(ph):
|
||||
# Two hours: anchor gives 10:00 a valid ret; measurement hour has one known log-return.
|
||||
ts0 = pd.date_range("2020-01-06 09:00", periods=60, freq="min")
|
||||
ts1 = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
|
||||
prices0 = np.ones(60) * 1.0
|
||||
# price jumps at minute 1 and STAYS (no reversion) → one non-zero log-return
|
||||
prices1 = np.full(60, np.exp(0.01))
|
||||
prices1[0] = 1.0 # only first tick is at 1.0; jump happens at tick 1
|
||||
m1 = pd.DataFrame({
|
||||
"ts": np.concatenate([ts0, ts1]),
|
||||
"close": np.concatenate([prices0, prices1]),
|
||||
})
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert len(hourly) >= 1, "no rows after resample"
|
||||
rv = hourly.iloc[-1]["realized_vol"]
|
||||
expected = np.sqrt(0.01 ** 2)
|
||||
assert abs(rv - expected) < 1e-6, f"realized_vol={rv:.8f}, expected≈{expected:.8f}"
|
||||
|
||||
|
||||
# 4. Only hours with ≥ 30 M1 bars are kept (thin hours dropped)
|
||||
def test_thin_hours_dropped(ph):
|
||||
# 4 hours: pre-anchor gives 09:00 a valid ret; full survives; thin (11:00) is dropped.
|
||||
# pre-anchor (08:00): gives 09:00 a valid ret
|
||||
# anchor (09:00): 60 bars, valid ret → kept
|
||||
# full (10:00): 60 bars, valid ret → kept
|
||||
# thin (11:00): 10 bars → dropped
|
||||
# Result: 3 hourly candidates, first (pre-anchor) gets NaN ret → dropped → 2 rows
|
||||
pre = pd.date_range("2020-01-06 08:00", periods=60, freq="min")
|
||||
anchor= pd.date_range("2020-01-06 09:00", periods=60, freq="min")
|
||||
full = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
|
||||
thin = pd.date_range("2020-01-06 11:00", periods=10, freq="min")
|
||||
ts = pre.append(anchor).append(full).append(thin)
|
||||
m1 = pd.DataFrame({"ts": ts, "close": np.ones(len(ts)) * 1.1})
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert len(hourly) == 2, f"expected 2 rows (pre-anchor NaN ret dropped + thin dropped), got {len(hourly)}"
|
||||
|
||||
|
||||
# 5. Output parquet path and schema (integration — reads actual M1 zips if present)
|
||||
def test_output_schema_from_zips(ph, tmp_path):
|
||||
import zipfile, io
|
||||
rows = []
|
||||
for h in range(24):
|
||||
for m in range(60):
|
||||
rows.append(f"20200106 {h:02d}{m:02d}00;1.10000;1.10100;1.09900;1.10000;100")
|
||||
csv_content = "\n".join(rows).encode()
|
||||
zip_buf = io.BytesIO()
|
||||
with zipfile.ZipFile(zip_buf, "w") as zf:
|
||||
zf.writestr("DAT_ASCII_EURUSD_M1_2020.csv", csv_content)
|
||||
zip_buf.seek(0)
|
||||
raw_dir = tmp_path / "raw"
|
||||
raw_dir.mkdir()
|
||||
(raw_dir / "DAT_ASCII_EURUSD_M1_2020.zip").write_bytes(zip_buf.read())
|
||||
|
||||
out_path = str(tmp_path / "eurusd_hourly.parquet")
|
||||
ph.build_hourly_parquet(raw_dir=str(raw_dir), out_path=out_path)
|
||||
assert os.path.exists(out_path), "output parquet not created"
|
||||
df = pd.read_parquet(out_path)
|
||||
assert set(["datetime", "close", "ret", "realized_vol"]).issubset(df.columns)
|
||||
assert len(df) > 0
|
||||
|
||||
|
||||
# ── New OHLCV-derived features ────────────────────────────────────────────────
|
||||
|
||||
def _make_m1_ohlcv(n_hours: int = 4, price: float = 1.1) -> pd.DataFrame:
|
||||
"""Synthetic M1 with distinct O, H, L, C so hl_range and ret_intrabar are nonzero."""
|
||||
rng = np.random.default_rng(7)
|
||||
ts = pd.date_range("2020-01-06 00:00", periods=n_hours * 60, freq="min")
|
||||
closes = price + np.cumsum(rng.normal(0, 0.0002, len(ts)))
|
||||
highs = closes + rng.uniform(0.0001, 0.0005, len(ts))
|
||||
lows = closes - rng.uniform(0.0001, 0.0005, len(ts))
|
||||
opens = np.roll(closes, 1); opens[0] = price
|
||||
return pd.DataFrame({"ts": ts, "open": opens, "high": highs, "low": lows, "close": closes})
|
||||
|
||||
|
||||
# 6. resample_to_hourly produces hl_range column
|
||||
def test_hourly_has_hl_range(ph):
|
||||
m1 = _make_m1_ohlcv()
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert "hl_range" in hourly.columns, f"missing hl_range; cols={hourly.columns.tolist()}"
|
||||
assert (hourly["hl_range"] > 0).all(), "hl_range should be positive"
|
||||
|
||||
|
||||
# 7. resample_to_hourly produces ret_intrabar column
|
||||
def test_hourly_has_ret_intrabar(ph):
|
||||
m1 = _make_m1_ohlcv()
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert "ret_intrabar" in hourly.columns, f"missing ret_intrabar; cols={hourly.columns.tolist()}"
|
||||
|
||||
|
||||
# 8. hl_range = log(hourly_high / hourly_low)
|
||||
def test_hl_range_formula(ph):
|
||||
# Two hours; second has known H=1.105, L=1.095
|
||||
ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
|
||||
ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
|
||||
closes = np.full(120, 1.1)
|
||||
highs = np.full(120, 1.1)
|
||||
lows = np.full(120, 1.1)
|
||||
# second hour: known spread
|
||||
highs[60:] = 1.105
|
||||
lows[60:] = 1.095
|
||||
m1 = pd.DataFrame({
|
||||
"ts": np.concatenate([ts0, ts1]),
|
||||
"open": closes, "high": highs, "low": lows, "close": closes,
|
||||
})
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert len(hourly) >= 1
|
||||
hl = hourly.iloc[-1]["hl_range"]
|
||||
expected = float(np.log(1.105 / 1.095))
|
||||
assert abs(hl - expected) < 1e-6, f"hl_range={hl:.8f}, expected={expected:.8f}"
|
||||
|
||||
|
||||
# 9. ret_intrabar = log(hourly_last_close / hourly_first_open)
|
||||
def test_ret_intrabar_formula(ph):
|
||||
ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
|
||||
ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
|
||||
closes = np.full(120, 1.1)
|
||||
opens = np.full(120, 1.1)
|
||||
# second hour: open=1.09, close=1.11
|
||||
opens[60] = 1.09
|
||||
closes[119] = 1.11
|
||||
m1 = pd.DataFrame({
|
||||
"ts": np.concatenate([ts0, ts1]),
|
||||
"open": opens, "high": closes + 0.001, "low": closes - 0.001, "close": closes,
|
||||
})
|
||||
hourly = ph.resample_to_hourly(m1)
|
||||
assert len(hourly) >= 1
|
||||
rib = hourly.iloc[-1]["ret_intrabar"]
|
||||
expected = float(np.log(1.11 / 1.09))
|
||||
assert abs(rib - expected) < 1e-6, f"ret_intrabar={rib:.8f}, expected={expected:.8f}"
|
||||
|
||||
|
||||
# 10. build() in train.py uses 2 feature channels (HPO: hl_range/ret_intrabar redundant)
|
||||
def test_build_uses_2_channels(tmp_path):
|
||||
import importlib.util, os
|
||||
hourly_path = "data/processed/eurusd_hourly.parquet"
|
||||
if not os.path.exists(hourly_path):
|
||||
pytest.skip("eurusd_hourly.parquet not present")
|
||||
spec = importlib.util.spec_from_file_location("train_2ch", "train.py")
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
(Xtr, _), _ = mod.build()
|
||||
assert Xtr.shape[2] == 2, f"expected 2 channels, got {Xtr.shape[2]}"
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Tests for scripts/prepare_regime.py — HMM regime detector (jepa-fx-risk#13).
|
||||
|
||||
TDD: tests first, implementation follows.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "prepare_regime.py"
|
||||
|
||||
DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
|
||||
HOURLY = DATA_DIR / "eurusd_hourly.parquet"
|
||||
DAILY = DATA_DIR / "eurusd_daily.parquet"
|
||||
|
||||
|
||||
def _import():
|
||||
spec = importlib.util.spec_from_file_location("prepare_regime", _SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def mod():
|
||||
return _import()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# fit_regime_hmm — pure function (doesn't touch disk)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _synthetic_rv(seed=42, size=500):
|
||||
"""Noisy 3-regime vol series: calm→stressed→crisis→calm interleaved."""
|
||||
rng = np.random.default_rng(seed)
|
||||
low = np.abs(rng.normal(0.005, 0.001, size=size // 3))
|
||||
mid = np.abs(rng.normal(0.015, 0.003, size=size // 3))
|
||||
high = np.abs(rng.normal(0.04, 0.008, size=size - 2 * (size // 3)))
|
||||
return np.concatenate([low, mid, high])
|
||||
|
||||
|
||||
class TestFitRegimeHmm:
|
||||
def test_returns_integer_labels(self, mod):
|
||||
rv = _synthetic_rv(seed=0)
|
||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
||||
assert np.issubdtype(labels.dtype, np.integer), f"dtype={labels.dtype}"
|
||||
assert len(labels) == len(rv)
|
||||
|
||||
def test_states_are_0_1_2(self, mod):
|
||||
rv = _synthetic_rv(seed=1)
|
||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
||||
unique = set(labels.tolist())
|
||||
assert unique.issubset({0, 1, 2}), f"unexpected states: {unique}"
|
||||
|
||||
def test_deterministic(self, mod):
|
||||
rv = _synthetic_rv(seed=7)
|
||||
a = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
||||
b = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
||||
assert np.array_equal(a, b), "HMM not deterministic with same random_state"
|
||||
|
||||
def test_sorted_by_vol_asc(self, mod):
|
||||
# 3 clearly separated noisy clusters; state 0 should be calm, 2 should be crisis.
|
||||
rng = np.random.default_rng(42)
|
||||
n = 200
|
||||
low = np.abs(rng.normal(0.005, 0.001, n))
|
||||
mid = np.abs(rng.normal(0.015, 0.003, n))
|
||||
high = np.abs(rng.normal(0.05, 0.008, n))
|
||||
rv = np.concatenate([low, mid, high])
|
||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
||||
# Mean regime label in the high-vol section should exceed mean in the low-vol section.
|
||||
assert labels[2*n:].mean() > labels[:n].mean(), \
|
||||
"crisis section mean regime label should exceed calm section"
|
||||
# The calm section should not be labeled as crisis (2) dominantly.
|
||||
calm_modal = int(np.bincount(labels[:n]).argmax())
|
||||
assert calm_modal < 2, f"calm section mostly labeled {calm_modal}, expected 0 or 1"
|
||||
|
||||
def test_two_states(self, mod):
|
||||
rv = _synthetic_rv(seed=0)
|
||||
labels = mod.fit_regime_hmm(rv, n_states=2, random_state=42)
|
||||
unique = set(labels.tolist())
|
||||
assert unique.issubset({0, 1})
|
||||
|
||||
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# prepare_regime_df — reads parquet, fits HMM, returns DataFrame
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestPrepareRegimeDf:
|
||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
||||
def test_output_columns(self, mod):
|
||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
||||
assert "datetime" in df.columns
|
||||
assert "regime" in df.columns
|
||||
|
||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
||||
def test_regime_values(self, mod):
|
||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
||||
unique = set(df["regime"].tolist())
|
||||
assert unique.issubset({0, 1, 2}), f"unexpected regime values: {unique}"
|
||||
|
||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
||||
def test_no_nulls(self, mod):
|
||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
||||
assert df["regime"].isna().sum() == 0
|
||||
|
||||
@pytest.mark.skipif(not DAILY.exists(), reason="daily parquet not available")
|
||||
def test_daily_fallback(self, mod):
|
||||
df = mod.prepare_regime_df(str(DAILY), freq="daily")
|
||||
assert "regime" in df.columns
|
||||
assert set(df["regime"].tolist()).issubset({0, 1, 2})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Integration: check that train.py REGIME SEAM exists and is togglable
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestTrainPyRegimeSeam:
|
||||
def test_enable_regime_env_var_documented(self):
|
||||
train_py = Path(__file__).parent.parent / "train.py"
|
||||
content = train_py.read_text()
|
||||
assert "JEPA_ENABLE_REGIME" in content, "JEPA_ENABLE_REGIME toggle not found in train.py"
|
||||
|
||||
def test_regime_seam_comment_present(self):
|
||||
train_py = Path(__file__).parent.parent / "train.py"
|
||||
content = train_py.read_text()
|
||||
assert "REGIME" in content and "seam" in content.lower(), \
|
||||
"agent-editable regime seam marker not found in train.py"
|
||||
@@ -0,0 +1,108 @@
|
||||
"""Tests for scripts/var_breach.py — VaR breach rate + Kupiec POF (jepa-fx-risk#12).
|
||||
|
||||
Golden tests first: verify the math before wiring it into train.py.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "var_breach.py"
|
||||
|
||||
|
||||
def _import():
|
||||
spec = importlib.util.spec_from_file_location("var_breach", _SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def mod():
|
||||
return _import()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# var_breach_rate
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestVarBreachRate:
|
||||
def test_zero_breaches(self, mod):
|
||||
# 0.02 < 0.01×2.326=0.02326 → no breach
|
||||
rate, _ = mod.var_breach_rate([0.01, 0.01], [0.02, 0.02])
|
||||
assert rate == 0.0
|
||||
|
||||
def test_all_breach(self, mod):
|
||||
# 0.03 > 0.02326 → all breach
|
||||
rate, _ = mod.var_breach_rate([0.01, 0.01], [0.03, 0.03])
|
||||
assert rate == 1.0
|
||||
|
||||
def test_golden_two_of_ten(self, mod):
|
||||
pred = [0.01] * 10
|
||||
actual = [0.01] * 10
|
||||
actual[0] = 0.03 # breach
|
||||
actual[2] = 0.03 # breach
|
||||
rate, kupiec_p = mod.var_breach_rate(pred, actual)
|
||||
assert abs(rate - 0.2) < 1e-9, f"rate={rate}"
|
||||
assert kupiec_p < 0.05, f"kupiec_p={kupiec_p}" # strong reject
|
||||
|
||||
def test_perfect_calibration(self, mod):
|
||||
# n=100, 1 breach → p_hat=0.01=p0=0.01 → LR=0 → kupiec_p≈1
|
||||
pred = [0.01] * 100
|
||||
actual = [0.015] * 100
|
||||
actual[0] = 0.025 # 0.025 > 0.02326 → breach
|
||||
rate, kupiec_p = mod.var_breach_rate(pred, actual)
|
||||
assert abs(rate - 0.01) < 1e-9
|
||||
assert kupiec_p > 0.9, f"kupiec_p={kupiec_p}"
|
||||
|
||||
def test_boundary_at_var_is_not_breach(self, mod):
|
||||
# exactly at VaR_99 is NOT a breach (strict >)
|
||||
z99 = 2.326
|
||||
var = 0.01 * z99
|
||||
rate, _ = mod.var_breach_rate([0.01], [var], z99=z99)
|
||||
assert rate == 0.0
|
||||
|
||||
def test_empty_returns_zero_one(self, mod):
|
||||
rate, kupiec_p = mod.var_breach_rate([], [])
|
||||
assert rate == 0.0
|
||||
assert kupiec_p == 1.0
|
||||
|
||||
def test_metric_key_no_whitespace(self, mod):
|
||||
key = mod.METRIC_KEY
|
||||
assert key == key.strip(), f"metric key has surrounding whitespace: {key!r}"
|
||||
assert " " not in key, f"metric key contains space: {key!r}"
|
||||
|
||||
def test_metric_key_is_canonical(self, mod):
|
||||
assert mod.METRIC_KEY == "VaR_breach_rate_99_oos_regime_cond"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# kupiec_pvalue
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestKupiecPValue:
|
||||
def test_perfectly_calibrated(self, mod):
|
||||
# p_hat == p0 → LR=0 → p-value=1
|
||||
p = mod.kupiec_pvalue(100, 1, p0=0.01)
|
||||
assert p > 0.99, f"p={p}"
|
||||
|
||||
def test_strong_reject_high_breach(self, mod):
|
||||
# 20% breach when 1% expected → p << 0.05
|
||||
p = mod.kupiec_pvalue(100, 20, p0=0.01)
|
||||
assert p < 0.001, f"p={p}"
|
||||
|
||||
def test_zero_breaches_not_nan(self, mod):
|
||||
p = mod.kupiec_pvalue(100, 0, p0=0.01)
|
||||
assert not math.isnan(p)
|
||||
assert 0 <= p <= 1.0
|
||||
|
||||
def test_all_breaches_not_nan(self, mod):
|
||||
p = mod.kupiec_pvalue(10, 10, p0=0.01)
|
||||
assert not math.isnan(p)
|
||||
assert p < 0.001 # extremely unlikely
|
||||
|
||||
def test_zero_observations(self, mod):
|
||||
p = mod.kupiec_pvalue(0, 0)
|
||||
assert p == 1.0
|
||||
@@ -0,0 +1,385 @@
|
||||
"""train.py — autoresearch agent file (only this may be edited).
|
||||
|
||||
HEPA backbone (Petersen et al., arXiv:2605.11130, ICML 2026 Spotlight):
|
||||
Causal Transformer pre-trained via horizon-conditioned JEPA. Predictor
|
||||
maps (h_t, Δt) → predicted future embedding; loss = VICReg (L1 alignment
|
||||
on L2-normalised reps + variance-covariance regulariser, no stop-gradient).
|
||||
Probe: ridge regression on the last-token embedding (true OOS split).
|
||||
|
||||
Agent may tune: encoder depth/width, patch geometry, ALPHA, DELTA_T_MAX,
|
||||
optimizer, LR. Do NOT touch prepare_data.py, loop.py, or the data pipeline.
|
||||
"""
|
||||
import json
|
||||
import math
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
|
||||
# --- agent-tunable knobs (all overridable via JEPA_* env vars for HPO) ---
|
||||
import os as _os
|
||||
USE_HOURLY = True
|
||||
WINDOW = int(_os.environ.get("JEPA_WINDOW", 120)) # HPO winner: 5-day context
|
||||
PATCH_LEN = int(_os.environ.get("JEPA_PATCH_LEN", 24))
|
||||
D_MODEL = int(_os.environ.get("JEPA_D_MODEL", 128))
|
||||
DEPTH = int(_os.environ.get("JEPA_DEPTH", 2))
|
||||
N_HEADS = int(_os.environ.get("JEPA_N_HEADS", 4))
|
||||
ALPHA = float(_os.environ.get("JEPA_ALPHA", 0.1))
|
||||
DELTA_T_MAX = int(_os.environ.get("JEPA_DELTA_T_MAX", 3))
|
||||
BATCH_SIZE = int(_os.environ.get("JEPA_BATCH_SIZE", 512))
|
||||
EPOCHS = int(_os.environ.get("JEPA_EPOCHS", 300))
|
||||
LR = float(_os.environ.get("JEPA_LR", 3e-4))
|
||||
PHASE1_EPOCHS = int(_os.environ.get("JEPA_PHASE1_EPOCHS", 200))
|
||||
PHASE1_LR = float(_os.environ.get("JEPA_PHASE1_LR", 1e-3))
|
||||
PHASE1_JOINT = bool(int(_os.environ.get("JEPA_PHASE1_JOINT", 1)))
|
||||
PHASE1_JOINT_EPOCHS= int(_os.environ.get("JEPA_PHASE1_JOINT_EPOCHS", 30))
|
||||
PHASE1_ENCODER_LR = float(_os.environ.get("JEPA_PHASE1_ENCODER_LR", 3e-6))
|
||||
USE_MULTIPAIR = bool(int(_os.environ.get("JEPA_USE_MULTIPAIR", 0)))
|
||||
JEPA_ENABLE_REGIME = bool(int(_os.environ.get("JEPA_ENABLE_REGIME", 0)))
|
||||
SEED = int(_os.environ.get("JEPA_SEED", 0))
|
||||
# ---------------------------
|
||||
|
||||
torch.manual_seed(SEED)
|
||||
np.random.seed(SEED)
|
||||
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
||||
|
||||
|
||||
# ── VICReg pretraining loss ──────────────────────────────────────────────────
|
||||
|
||||
def vicreg_loss(h_pred: torch.Tensor, h_target: torch.Tensor, alpha: float = 0.1) -> torch.Tensor:
|
||||
"""L = (1-α)·L1(normalize(ĥ), normalize(h*)) + α·(L_var + L_cov).
|
||||
|
||||
Both encoders receive gradients (joint training — no stop-grad on h_target).
|
||||
Variance-covariance terms prevent embedding collapse.
|
||||
"""
|
||||
pred_n = F.normalize(h_pred, dim=-1)
|
||||
targ_n = F.normalize(h_target, dim=-1)
|
||||
l1 = F.l1_loss(pred_n, targ_n)
|
||||
# variance hinge: push each feature std toward ≥ 1
|
||||
std = h_pred.std(dim=0) + 1e-4
|
||||
l_var = F.relu(1.0 - std).mean()
|
||||
# covariance penalty: decorrelate features
|
||||
B, D = h_pred.shape
|
||||
h_c = h_pred - h_pred.mean(dim=0, keepdim=True)
|
||||
cov = (h_c.t() @ h_c) / max(B - 1, 1)
|
||||
off = cov - torch.diag(torch.diag(cov))
|
||||
l_cov = (off ** 2).sum() / D
|
||||
return (1 - alpha) * l1 + alpha * (l_var + l_cov)
|
||||
|
||||
|
||||
# ── CausalEncoder ─────────────────────────────────────────────────────────────
|
||||
|
||||
class CausalEncoder(nn.Module):
|
||||
"""Non-overlapping patches → per-patch LayerNorm → causal Transformer → all tokens (B, N, D).
|
||||
|
||||
Per-patch LayerNorm instead of full-window RevIN: each patch is normalised
|
||||
using only its own timesteps, so no future statistics leak into past tokens.
|
||||
Use [:, -1, :] for probing (last token sees full context).
|
||||
Use [:, c, :] for JEPA pretraining (context-at-c).
|
||||
"""
|
||||
def __init__(self, n_channels: int, patch_len: int, d_model: int,
|
||||
n_heads: int, depth: int):
|
||||
super().__init__()
|
||||
self.patch_len = patch_len
|
||||
self.d_model = d_model
|
||||
patch_dim = patch_len * n_channels
|
||||
self.patch_norm = nn.LayerNorm(patch_dim) # applied per-patch, no future leakage
|
||||
self.embed = nn.Linear(patch_dim, d_model)
|
||||
layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
|
||||
dropout=0.0, batch_first=True)
|
||||
self.tf = nn.TransformerEncoder(layer, num_layers=depth)
|
||||
self.norm = nn.LayerNorm(d_model)
|
||||
|
||||
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
||||
B, W, F = x.shape
|
||||
P = self.patch_len
|
||||
N = W // P
|
||||
tokens = x[:, :N * P, :].reshape(B, N, P * F)
|
||||
tokens = self.embed(self.patch_norm(tokens))
|
||||
# sinusoidal PE
|
||||
pos = torch.arange(N, device=x.device).float()
|
||||
div = torch.exp(torch.arange(0, self.d_model, 2, device=x.device).float()
|
||||
* -(math.log(10000.0) / self.d_model))
|
||||
pe = torch.zeros(N, self.d_model, device=x.device)
|
||||
pe[:, 0::2] = torch.sin(pos.unsqueeze(1) * div)
|
||||
pe[:, 1::2] = torch.cos(pos.unsqueeze(1) * div)
|
||||
tokens = tokens + pe
|
||||
# causal mask
|
||||
mask = nn.Transformer.generate_square_subsequent_mask(N, device=x.device)
|
||||
return self.norm(self.tf(tokens, mask=mask, is_causal=True))
|
||||
|
||||
|
||||
# ── HorizonPredictor ─────────────────────────────────────────────────────────
|
||||
|
||||
class HorizonPredictor(nn.Module):
|
||||
"""MLP(cat(h_t, Δt)) → predicted future embedding."""
|
||||
def __init__(self, d_model: int):
|
||||
super().__init__()
|
||||
self.net = nn.Sequential(
|
||||
nn.Linear(d_model + 1, d_model), nn.GELU(),
|
||||
nn.Linear(d_model, d_model), nn.GELU(),
|
||||
nn.Linear(d_model, d_model),
|
||||
)
|
||||
|
||||
def forward(self, h: torch.Tensor, delta_t: torch.Tensor) -> torch.Tensor:
|
||||
dt = delta_t.float().unsqueeze(-1)
|
||||
return self.net(torch.cat([h, dt], dim=-1))
|
||||
|
||||
|
||||
# ── Phase-1 supervised head ──────────────────────────────────────────────────
|
||||
|
||||
class SupervisedHead(nn.Module):
|
||||
"""Small MLP trained on frozen HEPA embeddings to predict next-period realized vol."""
|
||||
def __init__(self, d_model: int):
|
||||
super().__init__()
|
||||
self.net = nn.Sequential(
|
||||
nn.Linear(d_model, d_model // 2), nn.GELU(),
|
||||
nn.Linear(d_model // 2, 1),
|
||||
)
|
||||
|
||||
def forward(self, h: torch.Tensor) -> torch.Tensor:
|
||||
return self.net(h).squeeze(-1)
|
||||
|
||||
|
||||
# ── Data ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
def build():
|
||||
"""Year-based split: encoder trains on ≤2021; probe evaluates on ≥2022 OOS.
|
||||
|
||||
Uses eurusd_hourly.parquet when USE_HOURLY=True and the file exists;
|
||||
falls back to eurusd_daily.parquet otherwise.
|
||||
"""
|
||||
import os
|
||||
multipair_path = "data/processed/eurusd_multipair.parquet"
|
||||
hourly_path = "data/processed/eurusd_hourly.parquet"
|
||||
daily_path = "data/processed/eurusd_daily.parquet"
|
||||
if USE_MULTIPAIR and os.path.exists(multipair_path):
|
||||
df = pd.read_parquet(multipair_path).reset_index(drop=True)
|
||||
df["date"] = pd.to_datetime(df["datetime"])
|
||||
# All {pair}_ret + {pair}_rv columns as features; eurusd_rv as target
|
||||
feat_cols = [c for c in df.columns if c.endswith("_ret") or c.endswith("_rv")]
|
||||
FEAT_COLS = feat_cols
|
||||
target_col = "eurusd_rv"
|
||||
elif USE_HOURLY and os.path.exists(hourly_path):
|
||||
df = pd.read_parquet(hourly_path).reset_index(drop=True)
|
||||
df["date"] = pd.to_datetime(df["datetime"])
|
||||
# 2-channel default (HPO: adding hl_range+ret_intrabar hurt — correlated with base feats)
|
||||
FEAT_COLS = ["ret", "realized_vol"]
|
||||
target_col = "realized_vol"
|
||||
else:
|
||||
df = pd.read_parquet(daily_path).reset_index(drop=True)
|
||||
df["date"] = pd.to_datetime(df["date"])
|
||||
FEAT_COLS = ["ret", "realized_vol"]
|
||||
target_col = "realized_vol"
|
||||
# ── REGIME CONDITIONING SEAM — agent may vary this mechanism ─────────────
|
||||
# Baseline: concat regime flag as an additional feature channel (0=calm, 2=crisis).
|
||||
# Agent may swap for FiLM conditioning, learned regime embedding, or gating.
|
||||
_regime_path = "data/processed/eurusd_regime.parquet"
|
||||
if JEPA_ENABLE_REGIME and os.path.exists(_regime_path):
|
||||
_rdf = pd.read_parquet(_regime_path)
|
||||
_ts_col = "datetime" if "datetime" in _rdf.columns else "date"
|
||||
_rdf[_ts_col] = pd.to_datetime(_rdf[_ts_col])
|
||||
df = df.copy()
|
||||
df = df.merge(
|
||||
_rdf.rename(columns={_ts_col: "date"})[["date", "regime"]],
|
||||
on="date", how="left",
|
||||
)
|
||||
df["regime"] = df["regime"].fillna(0).astype(np.float32)
|
||||
FEAT_COLS = list(FEAT_COLS) + ["regime"]
|
||||
# ── END REGIME SEAM ───────────────────────────────────────────────────────
|
||||
feats = df[FEAT_COLS].to_numpy(np.float32)
|
||||
target = df[target_col].to_numpy(np.float32)
|
||||
tr_idx = df.index[df["date"].dt.year <= 2021].tolist()
|
||||
te_idx = df.index[df["date"].dt.year >= 2022].tolist()
|
||||
mu = feats[:tr_idx[-1]+1].mean(0)
|
||||
sd = feats[:tr_idx[-1]+1].std(0) + 1e-8
|
||||
fn = (feats - mu) / sd
|
||||
def windows(idx):
|
||||
X, y = [], []
|
||||
for t in idx:
|
||||
if t - WINDOW >= 0 and t + 1 < len(df):
|
||||
X.append(fn[t - WINDOW:t]); y.append(target[t + 1])
|
||||
return np.stack(X).astype(np.float32), np.array(y, np.float32)
|
||||
return windows(tr_idx), windows(te_idx)
|
||||
|
||||
|
||||
# ── Training ──────────────────────────────────────────────────────────────────
|
||||
|
||||
def main():
|
||||
(Xtr, ytr), (Xte, yte) = build()
|
||||
n_feats = Xtr.shape[2]
|
||||
n_patches = WINDOW // PATCH_LEN
|
||||
N_tr = len(Xtr)
|
||||
bs = min(BATCH_SIZE, N_tr)
|
||||
|
||||
enc = CausalEncoder(n_feats, PATCH_LEN, D_MODEL, N_HEADS, DEPTH).to(dev)
|
||||
pred = HorizonPredictor(D_MODEL).to(dev)
|
||||
opt = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
|
||||
|
||||
for ep in range(EPOCHS):
|
||||
# Random mini-batch (avoids OOM on large hourly dataset)
|
||||
idx_b = torch.randperm(N_tr)[:bs]
|
||||
Xb = torch.tensor(Xtr[idx_b.numpy()], device=dev)
|
||||
|
||||
# Sample random context position and horizon
|
||||
c = torch.randint(0, n_patches - 1, ()).item()
|
||||
dt = torch.randint(1, max(2, min(DELTA_T_MAX, n_patches - 1 - c) + 1), ()).item()
|
||||
|
||||
tokens = enc(Xb) # (bs, N, D)
|
||||
h_ctx = tokens[:, c, :] # context embedding
|
||||
h_tgt = tokens[:, c + dt, :] # target embedding (joint training)
|
||||
h_hat = pred(h_ctx, torch.full((bs,), float(dt), device=dev))
|
||||
loss = vicreg_loss(h_hat, h_tgt, alpha=ALPHA)
|
||||
opt.zero_grad(); loss.backward(); opt.step()
|
||||
|
||||
enc.eval()
|
||||
with torch.no_grad():
|
||||
def embed(X_np):
|
||||
chunks = []
|
||||
for i in range(0, len(X_np), bs):
|
||||
t = torch.tensor(X_np[i:i+bs], device=dev)
|
||||
chunks.append(enc(t)[:, -1, :].cpu().numpy())
|
||||
return np.concatenate(chunks, axis=0)
|
||||
|
||||
Etr = embed(Xtr)
|
||||
Ete = embed(Xte)
|
||||
|
||||
# Ridge probe: fit on train, evaluate on OOS (true OOS R²)
|
||||
mu_e = Etr.mean(0); sd_e = Etr.std(0) + 1e-8
|
||||
Etr_n = (Etr - mu_e) / sd_e
|
||||
Ete_n = (Ete - mu_e) / sd_e
|
||||
A = np.hstack([Etr_n, np.ones((len(Etr_n), 1))])
|
||||
w = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
|
||||
pred_np = np.hstack([Ete_n, np.ones((len(Ete_n), 1))]) @ w
|
||||
ss_res = ((yte - pred_np) ** 2).sum()
|
||||
ss_tot = ((yte - yte.mean()) ** 2).sum()
|
||||
val_vol_r2 = float(1 - ss_res / ss_tot)
|
||||
|
||||
# Phase-1: MLP supervised head — joint or frozen-encoder path
|
||||
ytr_mu = float(ytr.mean()); ytr_sd = float(ytr.std()) + 1e-8
|
||||
ytr_z = (ytr - ytr_mu) / ytr_sd
|
||||
head = SupervisedHead(D_MODEL).to(dev)
|
||||
p1_bs = min(BATCH_SIZE, len(Etr_n))
|
||||
|
||||
# Shared tensors for the frozen-head warmup (used by both paths)
|
||||
Etr_t = torch.tensor(Etr_n, device=dev)
|
||||
ytr_z_t = torch.tensor(ytr_z, device=dev)
|
||||
Ete_t = torch.tensor(Ete_n, device=dev)
|
||||
N_tr_h = len(Etr_t)
|
||||
|
||||
# Phase 1a: warm up head on frozen embeddings (both paths run this)
|
||||
head_opt = torch.optim.Adam(head.parameters(), lr=PHASE1_LR, weight_decay=1e-4)
|
||||
for _ in range(PHASE1_EPOCHS):
|
||||
perm = torch.randperm(N_tr_h, device=dev)
|
||||
for start in range(0, N_tr_h, p1_bs):
|
||||
idx_h = perm[start:start + p1_bs]
|
||||
loss_h = F.mse_loss(head(Etr_t[idx_h]), ytr_z_t[idx_h])
|
||||
head_opt.zero_grad(); loss_h.backward(); head_opt.step()
|
||||
|
||||
if PHASE1_JOINT:
|
||||
# Phase 1b: short joint fine-tuning — encoder nudged with tiny LR.
|
||||
# Normalize live encoder output with FROZEN stats (mu_e, sd_e) so the
|
||||
# head sees the same embedding distribution it was warmed up on.
|
||||
enc.train()
|
||||
mu_e_t = torch.tensor(mu_e, device=dev)
|
||||
sd_e_t = torch.tensor(sd_e, device=dev)
|
||||
Xtr_t = torch.tensor(Xtr, device=dev)
|
||||
joint_opt = torch.optim.Adam([
|
||||
{"params": head.parameters(), "lr": PHASE1_LR * 0.1},
|
||||
{"params": enc.parameters(), "lr": PHASE1_ENCODER_LR},
|
||||
], weight_decay=1e-4)
|
||||
for _ in range(PHASE1_JOINT_EPOCHS):
|
||||
perm = torch.randperm(len(Xtr_t), device=dev)
|
||||
for start in range(0, len(Xtr_t), p1_bs):
|
||||
idx_j = perm[start:start + p1_bs]
|
||||
h_raw = enc(Xtr_t[idx_j])[:, -1, :]
|
||||
h_n = (h_raw - mu_e_t) / sd_e_t # frozen-stats normalisation
|
||||
loss_j = F.mse_loss(head(h_n), ytr_z_t[idx_j])
|
||||
joint_opt.zero_grad(); loss_j.backward(); joint_opt.step()
|
||||
enc.eval()
|
||||
# Re-extract test embeddings with fine-tuned encoder, same normalisation
|
||||
with torch.no_grad():
|
||||
chunks = []
|
||||
for i in range(0, len(Xte), p1_bs):
|
||||
t = torch.tensor(Xte[i:i+p1_bs], device=dev)
|
||||
h = enc(t)[:, -1, :]
|
||||
chunks.append(((h - mu_e_t) / sd_e_t).cpu().numpy())
|
||||
Ete_t = torch.tensor(np.concatenate(chunks), device=dev)
|
||||
|
||||
head.eval()
|
||||
with torch.no_grad():
|
||||
pred_h_z = head(Ete_t).cpu().numpy()
|
||||
|
||||
pred_h = pred_h_z * ytr_sd + ytr_mu # de-standardise
|
||||
phase1_r2 = float(1 - ((yte - pred_h) ** 2).sum() / ss_tot)
|
||||
print("phase1_r2 = %.4f (n_test=%d)" % (phase1_r2, len(yte)))
|
||||
|
||||
# ── VaR EVAL BLOCK — do NOT edit (agent boundary) ───────────────────────
|
||||
import sys as _sys
|
||||
_sys.path.insert(0, _os.path.dirname(_os.path.abspath(__file__)))
|
||||
from scripts.var_breach import var_breach_rate as _var_breach_rate, METRIC_KEY as _VAR_KEY
|
||||
_var_rate, _kupiec_p = _var_breach_rate(pred_np.tolist(), yte.tolist())
|
||||
print("%s=%.4f Kupiec_p=%.4f" % (_VAR_KEY, _var_rate, _kupiec_p))
|
||||
# ── END VaR EVAL BLOCK ───────────────────────────────────────────────────
|
||||
|
||||
_metrics_out = _os.environ.get("METRICS_OUT", "metrics.json")
|
||||
json.dump({
|
||||
"val_vol_r2": val_vol_r2, "phase1_r2": phase1_r2, "n_test": len(yte),
|
||||
_VAR_KEY: _var_rate, "kupiec_p": _kupiec_p,
|
||||
"knobs": {"WINDOW": WINDOW, "PATCH_LEN": PATCH_LEN,
|
||||
"D_MODEL": D_MODEL, "DEPTH": DEPTH, "ALPHA": ALPHA,
|
||||
"DELTA_T_MAX": DELTA_T_MAX, "EPOCHS": EPOCHS},
|
||||
}, open(_metrics_out, "w"), indent=2)
|
||||
print("val_vol_r2 = %.4f (n_test=%d, dev=%s)" % (val_vol_r2, len(yte), dev))
|
||||
|
||||
# ── EXPORT BLOCK — do NOT edit (agent boundary) ──────────────────────────
|
||||
# Set EXPORT_EMBEDDINGS=1 to write embeddings.json for the Go eval harness.
|
||||
import os
|
||||
if os.environ.get("EXPORT_EMBEDDINGS") == "1":
|
||||
hourly_path2 = "data/processed/eurusd_hourly.parquet"
|
||||
daily_path2 = "data/processed/eurusd_daily.parquet"
|
||||
if USE_HOURLY and os.path.exists(hourly_path2):
|
||||
df2 = pd.read_parquet(hourly_path2).reset_index(drop=True)
|
||||
df2["date"] = pd.to_datetime(df2["datetime"])
|
||||
else:
|
||||
df2 = pd.read_parquet(daily_path2).reset_index(drop=True)
|
||||
df2["date"] = pd.to_datetime(df2["date"])
|
||||
tr_mask = df2["date"].dt.year <= 2021
|
||||
base2 = ["ret", "realized_vol"]
|
||||
extra2 = [c for c in ["hl_range", "ret_intrabar"] if c in df2.columns]
|
||||
feats2 = df2[base2 + extra2].to_numpy(np.float32)
|
||||
mu2 = feats2[tr_mask].mean(0); sd2 = feats2[tr_mask].std(0) + 1e-8
|
||||
fn2 = (feats2 - mu2) / sd2
|
||||
def _export_windows(year_mask):
|
||||
idx = df2.index[year_mask].tolist()
|
||||
Xs, dates, rvs = [], [], []
|
||||
for t in idx:
|
||||
if t - WINDOW >= 0 and t + 1 < len(df2):
|
||||
Xs.append(fn2[t - WINDOW:t])
|
||||
dates.append(str(df2["date"].iloc[t].date()))
|
||||
rvs.append(float(df2["realized_vol"].iloc[t + 1]))
|
||||
if not Xs:
|
||||
return [], [], []
|
||||
Xa = np.stack(Xs)
|
||||
chunks = []
|
||||
with torch.no_grad():
|
||||
for i in range(0, len(Xa), bs):
|
||||
chunks.append(enc(torch.tensor(Xa[i:i+bs], device=dev))[:, -1, :].cpu().numpy())
|
||||
E = np.concatenate(chunks, axis=0).tolist()
|
||||
return E, dates, rvs
|
||||
Etr2, dates_tr, rv_tr = _export_windows(tr_mask)
|
||||
Eoos, dates_oos, rv_oos = _export_windows(df2["date"].dt.year >= 2022)
|
||||
hv_thr = float(np.percentile(rv_oos, 67))
|
||||
hv_label = [1 if v >= hv_thr else 0 for v in rv_oos]
|
||||
json.dump({"embeddings": Eoos, "dates": dates_oos,
|
||||
"realized_vol": rv_oos, "hv_label": hv_label,
|
||||
"train_embeddings": Etr2, "train_realized_vol": rv_tr},
|
||||
open("embeddings.json", "w"))
|
||||
print("exported embeddings.json train=%d oos=%d HV=%d/%d" % (
|
||||
len(Etr2), len(Eoos), sum(hv_label), len(hv_label)))
|
||||
# ── END EXPORT BLOCK ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user