docs: add .skills/README.md listing active skills

.aider.conf.yml

@@ -0,0 +1,2 @@
+read: .aider.conventions.md
+auto-commits: false

.aider.conventions.md

@@ -0,0 +1,13 @@
+# hostexecutor
+
+## Identity
+
+- **Name**: hostexecutor
+- **Owner**: Mathias
+- **Client**: personal
+- **Repo**: gitea.d-ma.be/mathias/hostexecutor
+- **Status**: active
+
+## Stack
+
+Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.

.context/PROJECT.md

@@ -4,101 +4,77 @@
 
 - **Name**: jepa-fx-risk
 - **Owner**: Mathias
-- **Client**: personal research
+- **Client**: personal / research
 - **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
-- **Status**: active — Phase 0
+- **Status**: active
+- **Brain wing**: `jepa-fx` (`wiki/jepa-fx/`)
 
 ## Purpose
 
-Research project: apply JEPA-based self-supervised representation learning to
-FX risk management for a corporate bank with an internal global FX trading desk.
-
-Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
-Target output: internal PoC for the trading desk.
-
-## Architecture decision (see DECISIONS.md ADR-001)
-
-**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
-arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
-(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
-
-Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
+Research project. Not a product. The "user" is future-self and research readers. Success is a reproducible, publishable result — not deployment.
 
 ## Stack
 
-**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
-harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
-results dashboard (Templ + HTMX + CDN Tailwind).
+- **Primary language**: Go 1.24+ (pipeline, eval harness, CLI, experiment runner)
+- **ML layer**: Python 3.12 + PyTorch (TS-JEPA training loop only — isolated in `model/`)
+- **Build**: Task (Taskfile.yml)
+- **Target infra**: koala (Arch + Blackwell GPU, training), iguana (Mac Studio M2, dev)
+- **MCP**: brain (knowledge), gitea (version control)
 
-**Python** (`model/`): all training and embedding export only. PyTorch cu130
-(Blackwell sm_120 compatible). No evaluation logic in Python.
+## Research context
 
-**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
-iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
+**Primary hypothesis:**
+> JEPA embeddings trained on FX time-series will produce latent market-state representations structurally separable by regime without explicit labels, measurable by silhouette score > 0.35 on k-means clusters vs. realised-volatility regime labels, on held-out data including at least one structural break.
 
-## Repository layout
+**Current phase:** Phase 0 — SSL Feasibility Gate
 
-```
-jepa-fx-risk/
-├── src/                    # Go — data pipeline + eval harness + dashboard
-│   ├── data/               # DUKASCopy fetch, hourly processing, validation
-│   └── eval/               # silhouette, linear probe, collapse, backtest
-├── model/                  # Python — training only
-│   ├── train.py            # TS-JEPA + SIGReg backbone (autoresearch edits this)
-│   ├── prepare.py          # LOCKED — data loading, tokenization, export
-│   └── requirements.txt
-├── specs/                  # Research specs (one per phase/experiment type)
-├── experiments/            # Per-run outputs: embeddings, metrics.json, git tag
-├── results/summaries/      # Human-readable outcome per experiment
-├── program.md              # Autoresearch agenda — researcher edits this
-├── DECISIONS.md            # Architecture Decision Records
-└── Taskfile.yml            # task data:fetch, task experiment:run, task eval:*
-```
+**Training cutoff:** 2023-01-01 (hard — never look at post-2023 data during development)
 
-## Phase structure
+**Out of scope:** options pricing, directional alpha, live/paper trading, exotic pairs (Phases 1–3)
 
-- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
-  Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
-- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
-  Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
-- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
-- **Phase 3**: Internal bank tick/position data (future, out of current scope).
+**Brain wing for prior decisions and failure modes:** `brain_query wing=jepa-fx`
 
-## Data
+## Conventions
 
-DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
-Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
-Weekend gaps handled explicitly. See ADR-002.
+### Scientific discipline
+- Every experiment has a spec in `specs/` before any code runs
+- Hypotheses are falsifiable and have quantitative acceptance criteria
+- Null results are recorded and published — not discarded
+- Training cutoff is sacred — post-2023 data never informs any design decision
+- Results reported with baselines; no cherry-picking
 
-## Key conventions
+### Code style
+- Go: `gofumpt`, `golangci-lint` with project config; table-driven tests; `testify`
+- Errors: `fmt.Errorf("context: %w", err)` — no naked returns
+- Python: `ruff` for lint; type hints throughout; `pytest` for tests
+- No Jupyter notebooks for anything reproducible — notebooks are EDA scratch only
 
-- `prepare.py` is LOCKED — never modified by agents or autoresearch
-- Evaluation metrics are always computed by the Go harness, never in Python
-- Every experiment gets a git tag: `exp/YYYYMMDD-description`
-- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
-- `program.md` is the only file the researcher edits to steer autoresearch
-- CI runs `task check` (lint + vet + test) only — no GPU, no training
+### Git
+- Conventional commits: `feat:`, `fix:`, `chore:`, `docs:`, `experiment:`, `result:`
+- Branch: `feat/`, `experiment/phase-N-description`, `fix/`
+- Every experiment run gets a git tag: `exp/YYYYMMDD-short-description`
+- PRs: one concern per PR; description explains *why* not *what*
 
-## Evaluation metrics
+### Experiment discipline
+- One spec per phase in `specs/` — written before any implementation
+- Each run recorded in `experiments/YYYYMMDD-HHMMSS-description/`
+- Metric summaries committed to `results/summaries/` — large outputs gitignored
+- `task check` must pass before any commit
 
-Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
-realized volatility from frozen embeddings, computed by Go harness.
+### Security / data
+- No raw FX data committed (gitignored) — see `data/README.md` for reproducible download
+- No API keys or tokens in code — env vars only
+- Training data and results stay local — nothing to cloud unless explicitly decided
 
-Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
-Must co-move with val_vol_r2 — checked from experiment 1.
+## Agent instructions
 
-Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
+When acting as a coding agent on this project:
 
-## Benchmarks to beat (trading desk comparison)
-
-- GARCH(1,1) — volatility forecasting baseline
-- Historical Simulation VaR (250-day rolling) — Basel default
-- EWMA RiskMetrics (λ=0.94)
-
-## Agent guidance
-
-Read `DECISIONS.md` before making architecture suggestions.
-Do not modify `prepare.py` or `src/eval/`.
-Do not suggest changing the Python/Go separation.
-Training runs are always manual via `task experiment:run` — never triggered by CI.
-When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
+1. Read this file and all `SKILL.md` files in `.skills/` before starting work
+2. Run `brain_query wing=jepa-fx` to load current decisions and failure modes
+3. Run `task check` before every commit (lint + vet + test)
+4. Check `DECISIONS.md` before making any architecture or methodology choice
+5. Every experiment needs a spec in `specs/` — no specless experiments
+6. Never touch post-2023 data during development; it is sealed
+7. Record null results honestly — do not iterate until metrics pass without noting it
+8. When adding a Python dependency, justify it; prefer pure Go alternatives

.context/mcp.json

@@ -1,30 +1,17 @@
 {
   "mcpServers": {
-    "knowledge": {
-      "url": "http://localhost:3100/mcp",
-      "description": "Project knowledge base — vector + graph retrieval"
-    },
     "brain": {
       "type": "http",
-      "url": "https://koala.tail30472e.ts.net:8443/mcp",
+      "url": "https://brain-mcp.d-ma.be/mcp",
       "headers": {
         "Authorization": "Bearer ${BRAIN_MCP_TOKEN}"
       }
     },
     "gitea": {
-      "_comment": "gitea-mcp authenticates by PASS-THROUGH: the bearer is a Gitea PAT it validates against Gitea and then acts as. GITEA_MCP_TOKEN -- which this entry used until 2026-09-08 -- 401s against it, so every harness synced from this file had a dead gitea server and no signal saying so. Probed 2026-09-08: GITEA_MCP_TOKEN 401, DMABE_GITEA_API_TOKEN 200, on both the public and the Serve URL.",
       "type": "http",
-      "url": "https://koala.tail30472e.ts.net:8446/mcp",
+      "url": "https://git-mcp.d-ma.be/mcp",
       "headers": {
-        "Authorization": "Bearer ${DMABE_GITEA_API_TOKEN}"
-      }
-    },
-    "infra": {
-      "_comment": "The bearer is belt-and-braces: this endpoint answers 200 with or without it, since Serve already limits reach to the tailnet. Sending it anyway means the day the server starts enforcing is not the day every harness breaks.",
-      "type": "http",
-      "url": "https://koala.tail30472e.ts.net:8444/mcp",
-      "headers": {
-        "Authorization": "Bearer ${INFRA_MCP_TOKEN}"
+        "Authorization": "Bearer ${GITEA_MCP_TOKEN}"
       }
     }
   }

.context/system-prompt.txt

@@ -3,533 +3,18 @@ Follow all conventions from both the root agent context and project context.
 
 ---
 
-# Agent context — Mathias workspace
-
-<!-- Canonical root context for all AI coding agents.
-     Lives at: ~/dev/.context/AGENT.md
-     Applies to every project under ~/dev/ unless overridden.
-     
-     Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
-     Project-level context in .context/PROJECT.md layers on top of this. -->
-
-## Who I am
-
-I'm Mathias, a digital product manager and technology consultant based in Sweden.
-I build software, research emerging tech, and deliver consulting engagements
-for clients under NDA. I work across AI/ML, financial automation, web applications,
-and climate/sustainability tech.
-
-## How I work with agents
-
-- I think like a product manager — I care about *why* before *how*
-- I want agents to be opinionated and push back, not just execute blindly
-- I prefer concise responses; skip ceremony and get to the point
-- When I say "build this", I mean production-quality with tests, not a demo
-- Ask me before making irreversible changes or adding heavy dependencies
-- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
-
-## Behavior rules
-
-These rules apply to every task across every project, regardless of harness.
-
-0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
-   - **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
-   - **Load the relevant skill** — see trigger table in *Engineering Skills* below.
-   - **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
-   - **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
-
-   **TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
-
-1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
-   Think before coding; if the problem is unclear, ask or state assumptions before acting.
-2. **Minimum viable code.** Solve with the smallest change that works. Nothing
-   speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
-3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
-   files, and formatting alone. Diffs should be small and reviewable.
-4. **Goal-driven execution.** Define clear success criteria up front for every task.
-   Loop — implement, verify, refine — until those criteria are met. Don't claim
-   completion without evidence (tests pass, command output, observed behavior).
-5. **Trunk-Based Development — commit directly to main.** Every commit is one
-   logical change (one tool, one fix, one test) with passing tests. Main is always
-   deployable. Never create long-lived feature branches.
-
-   **The PR trigger is RISK, not concurrency.** `risk/low` work commits straight to
-   main with CI as the gate — that is most work and is unchanged. `risk/medium` and
-   above opens a PR, even solo, even with nobody waiting to review it. This mirrors
-   the tiered merge model in `assessor-loop`
-   `method/transfer-to-a-regulated-organisation.md` §4.1, so the estate practises
-   what it sells instead of merging everything on green and claiming otherwise.
-
-   **Concurrency is NOT a reason to branch.** Measured 2026-08-29 across three real
-   collisions between parallel sessions: two were not branching problems at all — a
-   generated artifact written to the same path by both (branching turns that into a
-   merge conflict on a 33k-line JSON), and uncommitted work in a shared checkout
-   (a per-session worktree fixes it) — and the third was an ordinary non-fast-forward
-   push resolved by rebase in seconds. Use a worktree per session on shared repos and
-   timestamp generated artifacts. Do not reach for a branch.
-
-   Branch+PR flow carries documented costs on solo repos: merging auto-closes gated
-   issues, review comments live on a thread separate from the issue's, and a
-   merged-and-deleted branch cannot be amended (brain:
-   `gitea-pr-merge-autoclose-and-review-comment-footguns`). Those are the price of
-   the `risk/medium`+ lane — paid deliberately, not incurred by accident.
-
-   **Exception — external contributor or client four-eyes requirement:** Use PR flow
-   when a human reviewer outside the project is required. Document the reason in
-   PROJECT.md.
-
-   **Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
-   deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
-   invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
-   `litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
-   PR would buy a review of a state that was never deployed.
-
-   There, `risk/medium`+ work stays on `main` and instead requires a **recorded
-   adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
-   reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
-   thread.** No verdict recorded means the lane was not used, and saying "the gate was
-   green" is not the same claim.
-
-   Be honest about what this is: a recorded *independent* pass, not four-eyes in the
-   sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
-   is an agent. It is more than the repo had and less than the model it mirrors; do not
-   describe it as the stronger property. (infra#470, decided 2026-09-29.)
-
-   This lane depends on the `risk/` label being right. A label that triggers a heavier
-   lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
-   lane silently does not apply.
-
-6. **Close the loop — every substantive task ends with the same ritual.** Shipping
-   the code is not the end of the task; capturing it is. Run this unprompted:
-   - **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
-     new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
-     actual last tag — stated versions in docs drift stale.
-   - **Push** main and the tag (CI is the gate).
-   - **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
-     the reusable patterns and the footguns that would bite anyone again, never
-     project status. See *Knowledge base — when to write* below.
-   - **File discovered-but-deferred work as tracker issues** on the project's own
-     repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
-     "out of scope, recorded" rot in a commit message; make it a ticket with a
-     source pointer.
-   - Surface the brain entries and issue numbers in the closing summary so the
-     trail is auditable.
-
-   **Chat is not a durable store. Never rely on Mathias remembering anything.**
-   Every gap, caveat, risk, open question and next step must land in one of four
-   places before the turn ends: **the brain**, **a repo** (code, docs, ADR),
-   **a tracker issue**, or **a plan file**. A finding stated only in conversation is
-   lost the moment the session closes — and this estate has repeatedly proved that a
-   remembered convention is an unreliable control.
-
-   This applies hardest to what is tempting to leave as a closing remark: "worth
-   knowing X isn't enforced yet", "this is a convention, not a gate", "we should
-   revisit Y". Those are exactly the items that need a locator. If it is worth
-   saying in a summary, it is worth writing where it can be found again without you.
-
-7. **Finish what's closable — only involve me for a genuine call.** If a
-   tracker issue, PR, or checklist item is demonstrably resolved — the fix
-   landed, the state changed, you verified it live — close/merge/check it off
-   in the same turn. Don't leave it open "for the record" or park it waiting
-   on me to confirm something that already happened; that's bookkeeping, not
-   a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
-   comment, or otherwise pausing) for when it's genuinely mine to call:
-   irreversible or prod/host-impacting actions, a real judgment call with no
-   clearly-better option, ambiguous intent, or anything secret-writes/
-   destructive-git already gate. Before claiming something is closable,
-   verify it live (re-check the issue, re-run the command) rather than
-   trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
-
-8. **Control code needs an adversary, not a checklist.** Code whose job is to
-   refuse, deny, gate or contain is not adequately verified by a green gate.
-   Rule 5 makes CI the quality gate and that holds for ordinary work — but a
-   passing suite only asserts what its author already thought of, and for a
-   control the author is precisely the person whose imagination just failed.
-   Measured on `dispatch#75`: five review rounds, five real escapes verified by
-   execution, and **CI was green on every one of them** while `git rm
-   ././.dispatch-allow` deleted the file the control existed to protect. So
-   route this work through a review with authority to block, instructed to
-   verify by execution rather than by reading the implementation. Expect to be
-   blocked more than once; that is the control working, not the reviewer being
-   difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
-   practices live in the `pr-gatekeeper` agent.)
-
-9. **A test's name must not claim more than its assertions prove.** Name a test
-   after the mechanism it observes, not the goal it serves.
-   `DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
-   support. This is not pedantry about naming — when name and assertion
-   diverge, the name wins in the reader's head and the gap becomes invisible,
-   which is what lets the defect survive review. Found three times in one PR,
-   twice in tests written to fix the previous instance of it; each passed while
-   the thing it was named for was exploitable. If you cannot name the mechanism,
-   you do not yet know what the test observes.
-
-## CAD sprint patterns
-
-Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
-- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
-- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
-
-Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
-
-## Default stack
-
-| Layer | Default | Fallback | Last resort |
-|-------|---------|----------|-------------|
-| Language | Go | Python | TypeScript, Java, C |
-| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
-| Build | Task (taskfile.dev) | Make | — |
-| Containers | Docker Compose (dev), k3s (prod) | — | — |
-| DB | PostgreSQL + sqlc | SQLite | — |
-| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
-| Logging | slog (structured) | stdlib `logging` w/ structured `extra=` (or structlog) | — |
-| Testing | Table-driven, testify | — | — |
-| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
-
-Exploratory: Rust, Zig — I'll tell you when I want these.
-
-## Code conventions
-
-- **Go style**: golines, gofumpt, golangci-lint
-- **Python style** (fallback language): ruff (format+lint, one tool), mypy --strict (non-negotiable,
-  matches Go's static typing discipline), pytest + pytest-cov (table-driven via
-  `@pytest.mark.parametrize`), uv (venv+deps+lock, one tool), pydantic-settings (typed env-var config
-  — same principle as Go's typed structs), src-layout + `pyproject.toml` only (no `setup.py`)
-- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return.
-  Python: `raise X from e` (exception chaining, same principle) — never bare `except`, never silent `pass`
-- **Naming**: stdlib conventions, no stuttering
-- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
-- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
-  one logical change per commit, CI is the quality gate
-- **Never**: long-lived feature branches, PRs for solo work, direct push without
-  passing `task check` locally first
-- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
-- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
-
-## Secret handling (every harness, every command)
-
-Tool output is persisted: terminal → `~/.claude/projects` transcripts →
-claudewatcher → brain/wiki → gitea history. A secret printed once is
-searchable forever, and clearing it means rotating the key. So:
-
-1. **Never print, echo, log, or transform a secret to inspect it.** No
-   `base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
-   to defeat `op run`'s output masking (it masks raw values; base64 hides them
-   from the mask — that exact trick leaked a key on 2026-06-11).
-2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
-   *inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
-   in a command's argv (it lands in the tool call and the transcript).
-3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
-   never `${X:-...}` (returns the value when set) and never echo a substring of it.
-4. **Cross-host secrets:** run the secret-consuming command on the host that has
-   the secret; do not forward a raw key over ssh argv/stdout.
-5. If a secret does leak into output, say so immediately and flag it for rotation —
-   don't bury it.
-
-## Infrastructure
-
-Three machines on Tailscale:
-
-| Machine | Role | Key specs |
-|---------|------|-----------|
-| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
-| iguana | Services, builds | M2 Ultra Mac |
-| flamingo | Daily driver, edge | Apple Silicon Mac, ~/dev is here |
-
-- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
-- **Orchestration**: k3s on koala only — single control-plane node, no workers (verified 2026-08-30). iguana and flamingo are on the tailnet but are not cluster members
-- **Networking**: Tailscale mesh
-
-## Project landscape
-
-All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
-
-Organized in thematic folders:
-
-| Folder | Focus | Count |
-|--------|-------|-------|
-| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
-| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
-| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
-| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
-| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
-
-See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
-
-### Key active projects
-
-- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
-- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
-- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
-- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
-- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
-
-## Knowledge base — actively use it
-
-A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
-hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
-Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
-reference material — query it actively, not just when explicitly told.**
-
-### When to query (treat as a reflex)
-
-- **Before** starting a non-trivial task — search for prior art with the symptom
-  AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
-- **When debugging** — search for the error string, the stack frame, the affected
-  service. Past you may have already paid this tax.
-- **Before adopting** a pattern, library, framework, or model name — check if it
-  was tried and rejected, or what the integration footguns are.
-- **When making architectural decisions** — search for the domain + "ADR" or
-  "decision" to find prior reasoning before re-deriving it.
-- **When a recommendation feels novel** — challenge yourself: "has this been
-  documented?" The brain often has it.
-
-### When to write
-
-After you discover something that **future-you would forget** and that **isn't
-recoverable from the code, git log, or PR description alone**:
-
-- Bugs whose root cause is non-obvious and generalisable beyond this project.
-- Framework / library / model-name quirks that bit you and would bite anyone.
-- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
-- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
-
-DON'T write project status, sprint progress, PR summaries, or "what I did this
-session" — those rot fast and the originals are in git/gitea anyway. Brain
-entries that age well are about *why*, *how to avoid*, and *what to do when*.
-
-### How to access (per harness)
-
-| Harness | Query | Write |
-|---------|-------|-------|
-| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
-| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
-| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` |
-| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
-
-- **Scoping — the brain holds no confidential client content.** Nothing on the read path
-  filters by client: every token holder can read every wing, and `wing` on `brain_query`
-  is an optional argument, not an access control. Client confidentiality is enforced only
-  at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
-  client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
-  decided 2026-10-01. This line previously claimed client projects were filtered; they
-  were not.)
-- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
-  fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
-  on the koala k3s cluster; don't hardcode local-only model names into the
-  berget URL (see knowledge entry on namespace mismatches).
-
-### Quick reflex checks
-
-If you find yourself about to say any of these out loud, you owe yourself a brain query first:
-
-- "I think the issue might be..."
-- "Let me try X and see..."
-- "I'll just write a script to..."
-- "This is probably a new bug..."
-- "Has anyone done this before?" — *yes, probably, go check.*
-
-## Client work rules
-
-When working on a project tagged with a client name:
-1. Never send code, data, or context to cloud APIs — use local models only
-2. Never reference other client projects or their data
-3. Keep all artifacts within the client's git org / directory
-4. Treat everything as confidential unless told otherwise
-
-## Harness-agnostic principles
-
-Supported harnesses (2026-09-01) — Cursor and Aider were dropped; do not
-re-add adapters for them:
-- Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush, Pi Coding Agent
-- Open WebUI, Charmbracelet Mods, or any tool that takes a system prompt string
-- Any other tool that reads `AGENTS.md`, which is the de facto convention
-
-The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
-Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
-
-## How context propagates
-
-Canonical sources of truth:
-- Universal: `~/dev/.context/AGENT.md` (this file)
-- Project: `<repo>/.context/PROJECT.md` (per-repo)
-
-Derived files (committed, regenerated by `task context:sync`):
-- `CLAUDE.md`, `AGENTS.md`, `.context/system-prompt.txt`
-
-Repos still carrying `.cursorrules`, `.aider.conventions.md` or
-`.aider.conf.yml` are stale: nothing regenerates them. Delete them the next time
-you touch the repo, along with their entries in that repo's `task check`
-assertion.
-
-Workflow:
-1. Edit a canonical file. Run `task context:sync`. Commit canonical and
-   derived together. Push.
-2. On any other host, `git pull` brings both. Which file a harness reads does
-   NOT split cleanly by lookup strategy, so do not reason from the strategy:
-   - `CLAUDE.md` — Claude Code (walks up the tree)
-   - `AGENTS.md` — Crush, Pi, Antigravity (cwd-only), OpenCode (project root,
-     global at `~/.config/opencode/AGENTS.md`), and Mistral Vibe, which **walks
-     up** from cwd and takes the first `AGENTS.md` it finds
-     (user-level `~/.vibe/AGENTS.md`)
-   - `.context/system-prompt.txt` — Open WebUI, Mods, raw API calls; not read by
-     any harness, it is for pasting
-
-   Vibe tree-walks for `AGENTS.md`, so "tree-walkers use CLAUDE.md" is false.
-   Each repo's `AGENTS.md` embeds the canonical and is the first one found from
-   inside that repo, so this works — but by construction, not by luck.
-3. `task check` runs `context:sync` then asserts `git status --porcelain`
-   is empty over the derived files (catches both modified-tracked drift
-   and missing-untracked adapters). A drift fails the check with a
-   message telling you to stage the regenerated files.
-
-Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
-host and every harness.
-
-**Where a project rule contradicts a rule here, the project rule governs — and it
-must state its reason.** Both halves matter: the project file wins because it knows
-something this file cannot (infra's Flux constraint is real and not visible from
-here), and it must say why because a bare override is indistinguishable from drift.
-
-This replaces the previous wording, which said rules here and in `PROJECT.md` both
-apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
-unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
-at the same time (infra#470). A standard can hold two individually sensible clauses
-that are jointly unsatisfiable, and survive for as long as everyone reads it
-charitably — the same failure the Inception Sprint Oath's G6 had.
-
-## Engineering Skills
-
-Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Load at task start — not "on demand" but on schedule, before writing code. Browse `SKILLS_INDEX.md` for the full list.
-
-**Claude Code** installs the repo as a plugin from the marketplace the same repo hosts. That path also carries the `secret-guard` and `pre-task-ritual` hooks and the brain/gitea/infra MCP servers, which symlinks cannot:
-
-```
-/plugin marketplace add https://git.d-ma.be/mathias/skills.git
-/plugin install skills@mathias
-```
-
-Skills are **namespaced** under Claude Code: load `tdd` as `skills:tdd`. The trigger table below uses bare names — prefix them.
-
-**Crush, Pi and OpenCode** have no plugin system and are wired by `install.sh` as one directory symlink per harness via `~/.agents/skills`. Do not clone the checkout into `~/dev`: that folder is iCloud-shared between the Macs and spans two path roots (`/Users/mathias` vs `/home/mathias` on koala), so a checkout there is a git working tree with two concurrent writers, and any absolute symlink stored in it is broken on one side by construction (skills#23). Leave `SKILLS_CHECKOUT_DIR` at its host-local default.
-
-Check a host with `task doctor` in the skills repo; it also runs automatically at session start via the plugin.
-
-**Skill trigger table — load before starting, not after getting stuck:**
-
-| Task type | Load |
-|-----------|------|
-| Any feature or bug fix | `tdd` |
-| Refactor or design | `clean-code` or `solid` |
-| Debug | `problem-analysis` |
-| Review code or PRs | `code-review` |
-| Frame a problem before coding | `problem-analysis` |
-
----
-
-# jepa-fx-risk
+# hostexecutor
 
 ## Identity
 
-- **Name**: jepa-fx-risk
+- **Name**: hostexecutor
 - **Owner**: Mathias
-- **Client**: personal research
-- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
-- **Status**: active — Phase 0
-
-## Purpose
-
-Research project: apply JEPA-based self-supervised representation learning to
-FX risk management for a corporate bank with an internal global FX trading desk.
-
-Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
-Target output: internal PoC for the trading desk.
-
-## Architecture decision (see DECISIONS.md ADR-001)
-
-**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
-arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
-(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
-
-Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
+- **Client**: personal
+- **Repo**: gitea.d-ma.be/mathias/hostexecutor
+- **Status**: active
 
 ## Stack
 
-**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
-harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
-results dashboard (Templ + HTMX + CDN Tailwind).
-
-**Python** (`model/`): all training and embedding export only. PyTorch cu130
-(Blackwell sm_120 compatible). No evaluation logic in Python.
-
-**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
-iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
-
-## Repository layout
-
-```
-jepa-fx-risk/
-├── src/                    # Go — data pipeline + eval harness + dashboard
-│   ├── data/               # DUKASCopy fetch, hourly processing, validation
-│   └── eval/               # silhouette, linear probe, collapse, backtest
-├── model/                  # Python — training only
-│   ├── train.py            # TS-JEPA + SIGReg backbone (autoresearch edits this)
-│   ├── prepare.py          # LOCKED — data loading, tokenization, export
-│   └── requirements.txt
-├── specs/                  # Research specs (one per phase/experiment type)
-├── experiments/            # Per-run outputs: embeddings, metrics.json, git tag
-├── results/summaries/      # Human-readable outcome per experiment
-├── program.md              # Autoresearch agenda — researcher edits this
-├── DECISIONS.md            # Architecture Decision Records
-└── Taskfile.yml            # task data:fetch, task experiment:run, task eval:*
-```
-
-## Phase structure
-
-- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
-  Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
-- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
-  Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
-- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
-- **Phase 3**: Internal bank tick/position data (future, out of current scope).
-
-## Data
-
-DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
-Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
-Weekend gaps handled explicitly. See ADR-002.
-
-## Key conventions
-
-- `prepare.py` is LOCKED — never modified by agents or autoresearch
-- Evaluation metrics are always computed by the Go harness, never in Python
-- Every experiment gets a git tag: `exp/YYYYMMDD-description`
-- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
-- `program.md` is the only file the researcher edits to steer autoresearch
-- CI runs `task check` (lint + vet + test) only — no GPU, no training
-
-## Evaluation metrics
-
-Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
-realized volatility from frozen embeddings, computed by Go harness.
-
-Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
-Must co-move with val_vol_r2 — checked from experiment 1.
-
-Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
-
-## Benchmarks to beat (trading desk comparison)
-
-- GARCH(1,1) — volatility forecasting baseline
-- Historical Simulation VaR (250-day rolling) — Basel default
-- EWMA RiskMetrics (λ=0.94)
-
-## Agent guidance
-
-Read `DECISIONS.md` before making architecture suggestions.
-Do not modify `prepare.py` or `src/eval/`.
-Do not suggest changing the Python/Go separation.
-Training runs are always manual via `task experiment:run` — never triggered by CI.
-When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
+Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
 
 ---

.cursorrules

@@ -0,0 +1,16 @@
+# Cursor rules — auto-generated
+# Do not edit. Run: task context:sync
+
+# hostexecutor
+
+## Identity
+
+- **Name**: hostexecutor
+- **Owner**: Mathias
+- **Client**: personal
+- **Repo**: gitea.d-ma.be/mathias/hostexecutor
+- **Status**: active
+
+## Stack
+
+Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.

.gitea/workflows/autoresearch.yml

@@ -1,89 +0,0 @@
-name: Autoresearch Loop
-
-on:
-  workflow_dispatch:
-    inputs:
-      fixture:
-        description: 'Fixture name in fixtures/ (without .json)'
-        required: true
-        default: 'phase-a-toy'
-      rq_id:
-        description: 'Run ID — defaults to fixture name if blank'
-        required: false
-        default: ''
-      iters:
-        description: 'Max iterations'
-        required: false
-        default: '3'
-      model:
-        description: 'LiteLLM model override (leave blank for default berget/gemma4-31b)'
-        required: false
-        default: ''
-
-jobs:
-  run:
-    name: Autoresearch — ${{ inputs.fixture }}
-    runs-on: self-hosted
-    timeout-minutes: 90
-
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Resolve run ID
-        id: vars
-        # signals#47: inputs and outputs derived from them reach the shell via
-        # env, never via ${{ }} expansion inside run: (a `"` or `$(...)` in an
-        # input would otherwise run as code).
-        env:
-          IN_RQ_ID: ${{ inputs.rq_id }}
-          IN_FIXTURE: ${{ inputs.fixture }}
-        run: |
-          RQ_ID="$IN_RQ_ID"
-          [ -z "$RQ_ID" ] && RQ_ID="$IN_FIXTURE"
-          echo "rq_id=$RQ_ID" >> "$GITHUB_OUTPUT"
-
-      - name: Clean stale run dir
-        env:
-          RQ_ID: ${{ steps.vars.outputs.rq_id }}
-        run: rm -rf "runs/$RQ_ID"
-
-      - name: Set up Python venv
-        run: |
-          [ -d .venv ] || python3 -m venv .venv
-          # torch must come from the cu130 wheel index (koala Blackwell sm_120);
-          # requirements.txt deliberately excludes it. Install it first.
-          .venv/bin/pip install -q torch --index-url https://download.pytorch.org/whl/cu130
-          .venv/bin/pip install -q -r requirements.txt
-
-      - name: Scaffold run dir
-        env:
-          IN_FIXTURE: ${{ inputs.fixture }}
-          RQ_ID: ${{ steps.vars.outputs.rq_id }}
-        run: |
-          .venv/bin/python scripts/autoresearch_start.py \
-            "fixtures/$IN_FIXTURE.json" \
-            "$RQ_ID"
-
-      - name: Run autoresearch loop
-        env:
-          LITELLM_KEY: ${{ secrets.LITELLM_KEY }}
-          LITELLM_BASE: ${{ secrets.LITELLM_BASE }}
-          NTFY_URL: ${{ secrets.NTFY_URL }}
-          RQ_ID: ${{ steps.vars.outputs.rq_id }}
-          IN_ITERS: ${{ inputs.iters }}
-          IN_MODEL: ${{ inputs.model }}
-        run: |
-          ARGS=(--run-dir "runs/$RQ_ID" --iters "$IN_ITERS")
-          [ -n "$IN_MODEL" ] && ARGS+=(--model "$IN_MODEL")
-          .venv/bin/python loop.py "${ARGS[@]}"
-
-      - name: Upload run artifacts
-        if: always()
-        uses: https://gitea.com/actions/upload-artifact@v3
-        with:
-          name: run-${{ steps.vars.outputs.rq_id }}-${{ github.run_number }}
-          path: |
-            runs/${{ steps.vars.outputs.rq_id }}/STATUS.md
-            runs/${{ steps.vars.outputs.rq_id }}/metrics.json
-            runs/${{ steps.vars.outputs.rq_id }}/program.md
-          retention-days: 30

.gitea/workflows/cd.yml

@@ -7,6 +7,9 @@ on:
   pull_request:
     branches: [main]
 
+env:
+  IMAGE: hostexecutor
+
 jobs:
   check:
     name: Lint / Test / Vet
@@ -28,3 +31,85 @@ jobs:
 
       - name: Run checks
         run: task check
+
+  build:
+    name: Build & Import
+    needs: check
+    runs-on: self-hosted
+    if: github.event_name != 'pull_request'
+    outputs:
+      image-tag: ${{ steps.meta.outputs.sha-tag }}
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Derive image tags
+        id: meta
+        run: |
+          SHA=$(git rev-parse --short HEAD)
+          echo "sha-tag=${SHA}" >> "$GITHUB_OUTPUT"
+
+      - name: Build and push to local registry
+        run: |
+          REGISTRY="localhost:5000"
+          REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
+          buildah build \
+            --label "org.opencontainers.image.revision=${{ github.sha }}" \
+            -t ${REF} \
+            -t ${REGISTRY}/${{ env.IMAGE }}:latest \
+            .
+          buildah push --tls-verify=false ${REF}
+          buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
+          echo "✓ Image pushed to ${REF}"
+
+  deploy:
+    name: Deploy via GitOps
+    needs: build
+    runs-on: self-hosted
+    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
+    steps:
+      - name: Update image tag in infra repo
+        env:
+          IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
+          DEPLOY_KEY: ${{ secrets.INFRA_DEPLOY_KEY }}
+        run: |
+          set -euo pipefail
+          mkdir -p ~/.ssh
+          echo "$DEPLOY_KEY" > ~/.ssh/id_infra
+          chmod 600 ~/.ssh/id_infra
+          ssh-keyscan -p 30022 10.0.1.20 >> ~/.ssh/known_hosts 2>/dev/null
+          export GIT_SSH_COMMAND="ssh -i ~/.ssh/id_infra -o IdentitiesOnly=yes"
+          rm -rf /tmp/infra
+          git clone -b main ssh://git@10.0.1.20:30022/mathias/infra.git /tmp/infra
+          cd /tmp/infra
+          DEPLOYMENT="k3s/apps/hostexecutor/deployment.yaml"
+          sed -i "s|image: localhost:5000/hostexecutor:.*|image: localhost:5000/hostexecutor:${IMAGE_TAG}|" "$DEPLOYMENT"
+          grep -q "localhost:5000/hostexecutor:${IMAGE_TAG}" "$DEPLOYMENT" \
+            || { echo "✗ image tag patch failed"; exit 1; }
+          if git diff --quiet "$DEPLOYMENT"; then
+            echo "ℹ image tag unchanged — skipping push"
+          else
+            git -c user.name="hostexecutor CI" \
+                -c user.email="ci@hostexecutor.local" \
+                commit -m "chore(deploy): hostexecutor → ${IMAGE_TAG}" "$DEPLOYMENT"
+            git push origin main
+            echo "✓ pushed to infra repo"
+          fi
+          shred -u ~/.ssh/id_infra
+
+      - name: Trigger Flux reconcile
+        run: |
+          kubectl -n flux-system annotate gitrepository flux-system \
+            reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
+          kubectl -n flux-system annotate kustomization apps \
+            reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
+
+      - name: Verify rollout
+        run: |
+          kubectl rollout status deployment/hostexecutor \
+            --namespace hostexecutor \
+            --timeout=120s \
+          || {
+            kubectl get pods -n hostexecutor -o wide
+            kubectl get events -n hostexecutor --sort-by='.lastTimestamp' | tail -20
+            exit 1
+          }

.gitignore

@@ -1,53 +1,39 @@
-# ---> Go
-# If you prefer the allow list template instead of the deny list, see community template:
-# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
-#
-# Binaries for programs and plugins
-*.exe
-*.exe~
-*.dll
-*.so
-*.dylib
-
-# Test binary, built with `go test -c`
+# Go build artifacts
+bin/
 *.test
-
-# Output of the go coverage tool, specifically when used with LiteIDE
 *.out
 
-# Dependency directories (remove the comment below to include it)
-# vendor/
-
-# Go workspace file
+# Go workspace
 go.work
 go.work.sum
 
-# env file
+# Environment
 .env
 
-# Project-specific
-bin/
-*.templ.go
+# Python
+model/.venv/
+model/__pycache__/
+model/**/__pycache__/
+model/**/*.pyc
+model/.pytest_cache/
+model/**/.pytest_cache/
+model/.ruff_cache/
 
-# python venv (autoresearch loop)
-.venv/
+# Data — never commit raw or processed FX data
+data/raw/
+data/processed/
+data/cache/
 
-# downloaded + processed market data (track via DVC/MinIO, #10 — not git)
-data/
+# Experiment outputs — commit summaries only (results/summaries/)
+experiments/*/embeddings/
+experiments/*/checkpoints/
+experiments/*/logs/
 
-# autoresearch run dirs (ephemeral; each scaffold rebuilds from fixtures/)
-runs/
+# Large results — commit metric tables and figures only
+results/raw/
 
-# ephemeral experiment outputs (generated by train.py / loop.py)
-metrics.json
-embeddings.json
-HEARTBEAT
-STATUS.md
+# Notebooks — never commit outputs
+notebooks/**/.ipynb_checkpoints/
 
-# python caches
-__pycache__/
-*.pyc
-.pytest_cache/
-
-# built Go binaries
-eval
+# OS
+.DS_Store

.gitleaks.toml

@@ -1,27 +0,0 @@
-# gitleaks config for the jepa-fx-risk repo (infra#317).
-#
-# Runs gitleaks' full default ruleset and allowlists one non-credential
-# constant, scoped by BOTH path and pattern (condition = "AND") so a different
-# secret in the same files still fires. Allowlist by shape, not by filename.
-
-title = "jepa-fx-risk gitleaks config"
-
-[extend]
-useDefault = true
-
-# METRIC_KEY is the metrics-pipeline identifier
-# ("VaR_breach_rate_99_oos_regime_cond") that train.py's VaR eval block writes
-# into metrics.json -- documented in var_breach.py's own comment. It trips
-# generic-api-key purely because the variable name ends in _KEY and the value
-# is long and mixed-case enough to pass the entropy heuristic. It is not a
-# credential and has no secret analogue.
-[[allowlists]]
-description = "infra#317 — METRIC_KEY is a metrics identifier, not a credential"
-# infra#344: `paths` removed. gitleaks ORs the clauses of a GLOBAL allowlist and
-# does not honour `condition = "AND"` there, so the path matched on its own and
-# granted the whole file immunity — every finding in it, not just this one.
-# Verified: a live-shaped credential planted in that file scanned clean before
-# this change and fires after it. The regex below is what was meant and works
-# regardless of which file the value appears in (infra#256).
-regexTarget = "line"
-regexes = ['''METRIC_KEY''']

.golangci.yml

@@ -1,52 +0,0 @@
-# Canonical golangci-lint configuration for the estate's Go repos (infra#309
-# pattern: one canonical file here, a byte-identical copy in each adopter,
-# and the drift gate in THIS repo comparing sha256).
-#
-# Why this exists: 26 Go repos, 3 had a lint config, and all three were
-# different. The 23 without one ran golangci-lint's v2 default set, which
-# does NOT include a formatter -- so gofmt/gofumpt drift went uncaught until
-# it was found by hand (dma-cli#12: two files drifted, nothing flagged them).
-#
-# The two configs this replaces (cobalt-dingo, coo-agent) were near-identical
-# copies of each other AND both invalid under v2: they carried v1-style
-# `linters-settings` and `issues.exclude-rules` keys, which `golangci-lint
-# config verify` rejects outright. Their revive settings and test exclusions
-# were therefore not in effect. Consolidating fixes that too.
-#
-# Per-repo variation does NOT belong here -- this file must stay byte-identical
-# across adopters so the gate is a checksum, not a diff a human has to judge.
-# A repo with a genuine exception (agentsquad excludes its seeded-bug fixture
-# tree, which exists precisely to contain broken code) keeps its own config and
-# is not listed as an adopter.
-
-version: "2"
-
-linters:
-  default: none
-  enable:
-    - errcheck # unchecked errors
-    - govet # suspicious constructs
-    - staticcheck # broad static analysis
-    - unused # unused code
-    - misspell # spelling
-    - revive # idiomatic Go
-    - exhaustive # exhaustive switch on enums
-
-  settings:
-    revive:
-      rules:
-        - name: var-naming
-          severity: warning
-
-  exclusions:
-    rules:
-      # Test files: table-driven cases legitimately carry unused struct fields
-      # and long functions.
-      - path: _test\.go
-        linters:
-          - revive
-          - unused
-
-formatters:
-  enable:
-    - gofumpt

.skills/README.md

@@ -0,0 +1,13 @@
+This directory contains skill symlinks for this project.
+
+Agents: load all SKILL.md files in subdirectories before starting work.
+
+Active skills:
+- experiment-spec  — write experiment specs before running any phase
+- feature-spec     — write component specs before implementing within a phase
+- tdd              — acceptance criteria map to tests; red-green-refactor
+- grill-me         — stress-test specs and hypotheses before committing
+- session-retrospective — after each phase, surface learnings for the brain
+- clean-code       — Go and Python style conventions
+- planning         — break phases into trackable tasks
+- debug            — systematic debugging approach

AGENTS.md

@@ -1,528 +1,5 @@
-# Agent context — Mathias workspace
+# jepa-fx-risk — Agent context
+# Auto-generated from .context/PROJECT.md by `task context:sync`
+# Do not edit directly.
 
-<!-- Canonical root context for all AI coding agents.
-     Lives at: ~/dev/.context/AGENT.md
-     Applies to every project under ~/dev/ unless overridden.
-     
-     Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
-     Project-level context in .context/PROJECT.md layers on top of this. -->
-
-## Who I am
-
-I'm Mathias, a digital product manager and technology consultant based in Sweden.
-I build software, research emerging tech, and deliver consulting engagements
-for clients under NDA. I work across AI/ML, financial automation, web applications,
-and climate/sustainability tech.
-
-## How I work with agents
-
-- I think like a product manager — I care about *why* before *how*
-- I want agents to be opinionated and push back, not just execute blindly
-- I prefer concise responses; skip ceremony and get to the point
-- When I say "build this", I mean production-quality with tests, not a demo
-- Ask me before making irreversible changes or adding heavy dependencies
-- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
-
-## Behavior rules
-
-These rules apply to every task across every project, regardless of harness.
-
-0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
-   - **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
-   - **Load the relevant skill** — see trigger table in *Engineering Skills* below.
-   - **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
-   - **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
-
-   **TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
-
-1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
-   Think before coding; if the problem is unclear, ask or state assumptions before acting.
-2. **Minimum viable code.** Solve with the smallest change that works. Nothing
-   speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
-3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
-   files, and formatting alone. Diffs should be small and reviewable.
-4. **Goal-driven execution.** Define clear success criteria up front for every task.
-   Loop — implement, verify, refine — until those criteria are met. Don't claim
-   completion without evidence (tests pass, command output, observed behavior).
-5. **Trunk-Based Development — commit directly to main.** Every commit is one
-   logical change (one tool, one fix, one test) with passing tests. Main is always
-   deployable. Never create long-lived feature branches.
-
-   **The PR trigger is RISK, not concurrency.** `risk/low` work commits straight to
-   main with CI as the gate — that is most work and is unchanged. `risk/medium` and
-   above opens a PR, even solo, even with nobody waiting to review it. This mirrors
-   the tiered merge model in `assessor-loop`
-   `method/transfer-to-a-regulated-organisation.md` §4.1, so the estate practises
-   what it sells instead of merging everything on green and claiming otherwise.
-
-   **Concurrency is NOT a reason to branch.** Measured 2026-08-29 across three real
-   collisions between parallel sessions: two were not branching problems at all — a
-   generated artifact written to the same path by both (branching turns that into a
-   merge conflict on a 33k-line JSON), and uncommitted work in a shared checkout
-   (a per-session worktree fixes it) — and the third was an ordinary non-fast-forward
-   push resolved by rebase in seconds. Use a worktree per session on shared repos and
-   timestamp generated artifacts. Do not reach for a branch.
-
-   Branch+PR flow carries documented costs on solo repos: merging auto-closes gated
-   issues, review comments live on a thread separate from the issue's, and a
-   merged-and-deleted branch cannot be amended (brain:
-   `gitea-pr-merge-autoclose-and-review-comment-footguns`). Those are the price of
-   the `risk/medium`+ lane — paid deliberately, not incurred by accident.
-
-   **Exception — external contributor or client four-eyes requirement:** Use PR flow
-   when a human reviewer outside the project is required. Document the reason in
-   PROJECT.md.
-
-   **Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
-   deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
-   invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
-   `litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
-   PR would buy a review of a state that was never deployed.
-
-   There, `risk/medium`+ work stays on `main` and instead requires a **recorded
-   adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
-   reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
-   thread.** No verdict recorded means the lane was not used, and saying "the gate was
-   green" is not the same claim.
-
-   Be honest about what this is: a recorded *independent* pass, not four-eyes in the
-   sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
-   is an agent. It is more than the repo had and less than the model it mirrors; do not
-   describe it as the stronger property. (infra#470, decided 2026-09-29.)
-
-   This lane depends on the `risk/` label being right. A label that triggers a heavier
-   lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
-   lane silently does not apply.
-
-6. **Close the loop — every substantive task ends with the same ritual.** Shipping
-   the code is not the end of the task; capturing it is. Run this unprompted:
-   - **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
-     new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
-     actual last tag — stated versions in docs drift stale.
-   - **Push** main and the tag (CI is the gate).
-   - **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
-     the reusable patterns and the footguns that would bite anyone again, never
-     project status. See *Knowledge base — when to write* below.
-   - **File discovered-but-deferred work as tracker issues** on the project's own
-     repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
-     "out of scope, recorded" rot in a commit message; make it a ticket with a
-     source pointer.
-   - Surface the brain entries and issue numbers in the closing summary so the
-     trail is auditable.
-
-   **Chat is not a durable store. Never rely on Mathias remembering anything.**
-   Every gap, caveat, risk, open question and next step must land in one of four
-   places before the turn ends: **the brain**, **a repo** (code, docs, ADR),
-   **a tracker issue**, or **a plan file**. A finding stated only in conversation is
-   lost the moment the session closes — and this estate has repeatedly proved that a
-   remembered convention is an unreliable control.
-
-   This applies hardest to what is tempting to leave as a closing remark: "worth
-   knowing X isn't enforced yet", "this is a convention, not a gate", "we should
-   revisit Y". Those are exactly the items that need a locator. If it is worth
-   saying in a summary, it is worth writing where it can be found again without you.
-
-7. **Finish what's closable — only involve me for a genuine call.** If a
-   tracker issue, PR, or checklist item is demonstrably resolved — the fix
-   landed, the state changed, you verified it live — close/merge/check it off
-   in the same turn. Don't leave it open "for the record" or park it waiting
-   on me to confirm something that already happened; that's bookkeeping, not
-   a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
-   comment, or otherwise pausing) for when it's genuinely mine to call:
-   irreversible or prod/host-impacting actions, a real judgment call with no
-   clearly-better option, ambiguous intent, or anything secret-writes/
-   destructive-git already gate. Before claiming something is closable,
-   verify it live (re-check the issue, re-run the command) rather than
-   trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
-
-8. **Control code needs an adversary, not a checklist.** Code whose job is to
-   refuse, deny, gate or contain is not adequately verified by a green gate.
-   Rule 5 makes CI the quality gate and that holds for ordinary work — but a
-   passing suite only asserts what its author already thought of, and for a
-   control the author is precisely the person whose imagination just failed.
-   Measured on `dispatch#75`: five review rounds, five real escapes verified by
-   execution, and **CI was green on every one of them** while `git rm
-   ././.dispatch-allow` deleted the file the control existed to protect. So
-   route this work through a review with authority to block, instructed to
-   verify by execution rather than by reading the implementation. Expect to be
-   blocked more than once; that is the control working, not the reviewer being
-   difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
-   practices live in the `pr-gatekeeper` agent.)
-
-9. **A test's name must not claim more than its assertions prove.** Name a test
-   after the mechanism it observes, not the goal it serves.
-   `DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
-   support. This is not pedantry about naming — when name and assertion
-   diverge, the name wins in the reader's head and the gap becomes invisible,
-   which is what lets the defect survive review. Found three times in one PR,
-   twice in tests written to fix the previous instance of it; each passed while
-   the thing it was named for was exploitable. If you cannot name the mechanism,
-   you do not yet know what the test observes.
-
-## CAD sprint patterns
-
-Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
-- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
-- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
-
-Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
-
-## Default stack
-
-| Layer | Default | Fallback | Last resort |
-|-------|---------|----------|-------------|
-| Language | Go | Python | TypeScript, Java, C |
-| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
-| Build | Task (taskfile.dev) | Make | — |
-| Containers | Docker Compose (dev), k3s (prod) | — | — |
-| DB | PostgreSQL + sqlc | SQLite | — |
-| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
-| Logging | slog (structured) | stdlib `logging` w/ structured `extra=` (or structlog) | — |
-| Testing | Table-driven, testify | — | — |
-| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
-
-Exploratory: Rust, Zig — I'll tell you when I want these.
-
-## Code conventions
-
-- **Go style**: golines, gofumpt, golangci-lint
-- **Python style** (fallback language): ruff (format+lint, one tool), mypy --strict (non-negotiable,
-  matches Go's static typing discipline), pytest + pytest-cov (table-driven via
-  `@pytest.mark.parametrize`), uv (venv+deps+lock, one tool), pydantic-settings (typed env-var config
-  — same principle as Go's typed structs), src-layout + `pyproject.toml` only (no `setup.py`)
-- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return.
-  Python: `raise X from e` (exception chaining, same principle) — never bare `except`, never silent `pass`
-- **Naming**: stdlib conventions, no stuttering
-- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
-- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
-  one logical change per commit, CI is the quality gate
-- **Never**: long-lived feature branches, PRs for solo work, direct push without
-  passing `task check` locally first
-- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
-- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
-
-## Secret handling (every harness, every command)
-
-Tool output is persisted: terminal → `~/.claude/projects` transcripts →
-claudewatcher → brain/wiki → gitea history. A secret printed once is
-searchable forever, and clearing it means rotating the key. So:
-
-1. **Never print, echo, log, or transform a secret to inspect it.** No
-   `base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
-   to defeat `op run`'s output masking (it masks raw values; base64 hides them
-   from the mask — that exact trick leaked a key on 2026-06-11).
-2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
-   *inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
-   in a command's argv (it lands in the tool call and the transcript).
-3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
-   never `${X:-...}` (returns the value when set) and never echo a substring of it.
-4. **Cross-host secrets:** run the secret-consuming command on the host that has
-   the secret; do not forward a raw key over ssh argv/stdout.
-5. If a secret does leak into output, say so immediately and flag it for rotation —
-   don't bury it.
-
-## Infrastructure
-
-Three machines on Tailscale:
-
-| Machine | Role | Key specs |
-|---------|------|-----------|
-| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
-| iguana | Services, builds | M2 Ultra Mac |
-| flamingo | Daily driver, edge | Apple Silicon Mac, ~/dev is here |
-
-- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
-- **Orchestration**: k3s on koala only — single control-plane node, no workers (verified 2026-08-30). iguana and flamingo are on the tailnet but are not cluster members
-- **Networking**: Tailscale mesh
-
-## Project landscape
-
-All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
-
-Organized in thematic folders:
-
-| Folder | Focus | Count |
-|--------|-------|-------|
-| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
-| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
-| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
-| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
-| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
-
-See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
-
-### Key active projects
-
-- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
-- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
-- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
-- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
-- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
-
-## Knowledge base — actively use it
-
-A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
-hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
-Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
-reference material — query it actively, not just when explicitly told.**
-
-### When to query (treat as a reflex)
-
-- **Before** starting a non-trivial task — search for prior art with the symptom
-  AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
-- **When debugging** — search for the error string, the stack frame, the affected
-  service. Past you may have already paid this tax.
-- **Before adopting** a pattern, library, framework, or model name — check if it
-  was tried and rejected, or what the integration footguns are.
-- **When making architectural decisions** — search for the domain + "ADR" or
-  "decision" to find prior reasoning before re-deriving it.
-- **When a recommendation feels novel** — challenge yourself: "has this been
-  documented?" The brain often has it.
-
-### When to write
-
-After you discover something that **future-you would forget** and that **isn't
-recoverable from the code, git log, or PR description alone**:
-
-- Bugs whose root cause is non-obvious and generalisable beyond this project.
-- Framework / library / model-name quirks that bit you and would bite anyone.
-- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
-- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
-
-DON'T write project status, sprint progress, PR summaries, or "what I did this
-session" — those rot fast and the originals are in git/gitea anyway. Brain
-entries that age well are about *why*, *how to avoid*, and *what to do when*.
-
-### How to access (per harness)
-
-| Harness | Query | Write |
-|---------|-------|-------|
-| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
-| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
-| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` |
-| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
-
-- **Scoping — the brain holds no confidential client content.** Nothing on the read path
-  filters by client: every token holder can read every wing, and `wing` on `brain_query`
-  is an optional argument, not an access control. Client confidentiality is enforced only
-  at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
-  client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
-  decided 2026-10-01. This line previously claimed client projects were filtered; they
-  were not.)
-- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
-  fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
-  on the koala k3s cluster; don't hardcode local-only model names into the
-  berget URL (see knowledge entry on namespace mismatches).
-
-### Quick reflex checks
-
-If you find yourself about to say any of these out loud, you owe yourself a brain query first:
-
-- "I think the issue might be..."
-- "Let me try X and see..."
-- "I'll just write a script to..."
-- "This is probably a new bug..."
-- "Has anyone done this before?" — *yes, probably, go check.*
-
-## Client work rules
-
-When working on a project tagged with a client name:
-1. Never send code, data, or context to cloud APIs — use local models only
-2. Never reference other client projects or their data
-3. Keep all artifacts within the client's git org / directory
-4. Treat everything as confidential unless told otherwise
-
-## Harness-agnostic principles
-
-Supported harnesses (2026-09-01) — Cursor and Aider were dropped; do not
-re-add adapters for them:
-- Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush, Pi Coding Agent
-- Open WebUI, Charmbracelet Mods, or any tool that takes a system prompt string
-- Any other tool that reads `AGENTS.md`, which is the de facto convention
-
-The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
-Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
-
-## How context propagates
-
-Canonical sources of truth:
-- Universal: `~/dev/.context/AGENT.md` (this file)
-- Project: `<repo>/.context/PROJECT.md` (per-repo)
-
-Derived files (committed, regenerated by `task context:sync`):
-- `CLAUDE.md`, `AGENTS.md`, `.context/system-prompt.txt`
-
-Repos still carrying `.cursorrules`, `.aider.conventions.md` or
-`.aider.conf.yml` are stale: nothing regenerates them. Delete them the next time
-you touch the repo, along with their entries in that repo's `task check`
-assertion.
-
-Workflow:
-1. Edit a canonical file. Run `task context:sync`. Commit canonical and
-   derived together. Push.
-2. On any other host, `git pull` brings both. Which file a harness reads does
-   NOT split cleanly by lookup strategy, so do not reason from the strategy:
-   - `CLAUDE.md` — Claude Code (walks up the tree)
-   - `AGENTS.md` — Crush, Pi, Antigravity (cwd-only), OpenCode (project root,
-     global at `~/.config/opencode/AGENTS.md`), and Mistral Vibe, which **walks
-     up** from cwd and takes the first `AGENTS.md` it finds
-     (user-level `~/.vibe/AGENTS.md`)
-   - `.context/system-prompt.txt` — Open WebUI, Mods, raw API calls; not read by
-     any harness, it is for pasting
-
-   Vibe tree-walks for `AGENTS.md`, so "tree-walkers use CLAUDE.md" is false.
-   Each repo's `AGENTS.md` embeds the canonical and is the first one found from
-   inside that repo, so this works — but by construction, not by luck.
-3. `task check` runs `context:sync` then asserts `git status --porcelain`
-   is empty over the derived files (catches both modified-tracked drift
-   and missing-untracked adapters). A drift fails the check with a
-   message telling you to stage the regenerated files.
-
-Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
-host and every harness.
-
-**Where a project rule contradicts a rule here, the project rule governs — and it
-must state its reason.** Both halves matter: the project file wins because it knows
-something this file cannot (infra's Flux constraint is real and not visible from
-here), and it must say why because a bare override is indistinguishable from drift.
-
-This replaces the previous wording, which said rules here and in `PROJECT.md` both
-apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
-unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
-at the same time (infra#470). A standard can hold two individually sensible clauses
-that are jointly unsatisfiable, and survive for as long as everyone reads it
-charitably — the same failure the Inception Sprint Oath's G6 had.
-
-## Engineering Skills
-
-Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Load at task start — not "on demand" but on schedule, before writing code. Browse `SKILLS_INDEX.md` for the full list.
-
-**Claude Code** installs the repo as a plugin from the marketplace the same repo hosts. That path also carries the `secret-guard` and `pre-task-ritual` hooks and the brain/gitea/infra MCP servers, which symlinks cannot:
-
-```
-/plugin marketplace add https://git.d-ma.be/mathias/skills.git
-/plugin install skills@mathias
-```
-
-Skills are **namespaced** under Claude Code: load `tdd` as `skills:tdd`. The trigger table below uses bare names — prefix them.
-
-**Crush, Pi and OpenCode** have no plugin system and are wired by `install.sh` as one directory symlink per harness via `~/.agents/skills`. Do not clone the checkout into `~/dev`: that folder is iCloud-shared between the Macs and spans two path roots (`/Users/mathias` vs `/home/mathias` on koala), so a checkout there is a git working tree with two concurrent writers, and any absolute symlink stored in it is broken on one side by construction (skills#23). Leave `SKILLS_CHECKOUT_DIR` at its host-local default.
-
-Check a host with `task doctor` in the skills repo; it also runs automatically at session start via the plugin.
-
-**Skill trigger table — load before starting, not after getting stuck:**
-
-| Task type | Load |
-|-----------|------|
-| Any feature or bug fix | `tdd` |
-| Refactor or design | `clean-code` or `solid` |
-| Debug | `problem-analysis` |
-| Review code or PRs | `code-review` |
-| Frame a problem before coding | `problem-analysis` |
-
----
-
-# jepa-fx-risk
-
-## Identity
-
-- **Name**: jepa-fx-risk
-- **Owner**: Mathias
-- **Client**: personal research
-- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
-- **Status**: active — Phase 0
-
-## Purpose
-
-Research project: apply JEPA-based self-supervised representation learning to
-FX risk management for a corporate bank with an internal global FX trading desk.
-
-Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
-Target output: internal PoC for the trading desk.
-
-## Architecture decision (see DECISIONS.md ADR-001)
-
-**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
-arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
-(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
-
-Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
-
-## Stack
-
-**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
-harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
-results dashboard (Templ + HTMX + CDN Tailwind).
-
-**Python** (`model/`): all training and embedding export only. PyTorch cu130
-(Blackwell sm_120 compatible). No evaluation logic in Python.
-
-**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
-iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
-
-## Repository layout
-
-```
-jepa-fx-risk/
-├── src/                    # Go — data pipeline + eval harness + dashboard
-│   ├── data/               # DUKASCopy fetch, hourly processing, validation
-│   └── eval/               # silhouette, linear probe, collapse, backtest
-├── model/                  # Python — training only
-│   ├── train.py            # TS-JEPA + SIGReg backbone (autoresearch edits this)
-│   ├── prepare.py          # LOCKED — data loading, tokenization, export
-│   └── requirements.txt
-├── specs/                  # Research specs (one per phase/experiment type)
-├── experiments/            # Per-run outputs: embeddings, metrics.json, git tag
-├── results/summaries/      # Human-readable outcome per experiment
-├── program.md              # Autoresearch agenda — researcher edits this
-├── DECISIONS.md            # Architecture Decision Records
-└── Taskfile.yml            # task data:fetch, task experiment:run, task eval:*
-```
-
-## Phase structure
-
-- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
-  Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
-- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
-  Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
-- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
-- **Phase 3**: Internal bank tick/position data (future, out of current scope).
-
-## Data
-
-DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
-Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
-Weekend gaps handled explicitly. See ADR-002.
-
-## Key conventions
-
-- `prepare.py` is LOCKED — never modified by agents or autoresearch
-- Evaluation metrics are always computed by the Go harness, never in Python
-- Every experiment gets a git tag: `exp/YYYYMMDD-description`
-- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
-- `program.md` is the only file the researcher edits to steer autoresearch
-- CI runs `task check` (lint + vet + test) only — no GPU, no training
-
-## Evaluation metrics
-
-Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
-realized volatility from frozen embeddings, computed by Go harness.
-
-Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
-Must co-move with val_vol_r2 — checked from experiment 1.
-
-Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
-
-## Benchmarks to beat (trading desk comparison)
-
-- GARCH(1,1) — volatility forecasting baseline
-- Historical Simulation VaR (250-day rolling) — Basel default
-- EWMA RiskMetrics (λ=0.94)
-
-## Agent guidance
-
-Read `DECISIONS.md` before making architecture suggestions.
-Do not modify `prepare.py` or `src/eval/`.
-Do not suggest changing the Python/Go separation.
-Training runs are always manual via `task experiment:run` — never triggered by CI.
-When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
+See .context/PROJECT.md for the canonical source.

CLAUDE.md

@@ -1,104 +1,5 @@
-# jepa-fx-risk
+# jepa-fx-risk — Agent context
+# Auto-generated from .context/PROJECT.md by `task context:sync`
+# Do not edit directly.
 
-## Identity
-
-- **Name**: jepa-fx-risk
-- **Owner**: Mathias
-- **Client**: personal research
-- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
-- **Status**: active — Phase 0
-
-## Purpose
-
-Research project: apply JEPA-based self-supervised representation learning to
-FX risk management for a corporate bank with an internal global FX trading desk.
-
-Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
-Target output: internal PoC for the trading desk.
-
-## Architecture decision (see DECISIONS.md ADR-001)
-
-**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
-arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
-(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
-
-Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
-
-## Stack
-
-**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
-harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
-results dashboard (Templ + HTMX + CDN Tailwind).
-
-**Python** (`model/`): all training and embedding export only. PyTorch cu130
-(Blackwell sm_120 compatible). No evaluation logic in Python.
-
-**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
-iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
-
-## Repository layout
-
-```
-jepa-fx-risk/
-├── src/                    # Go — data pipeline + eval harness + dashboard
-│   ├── data/               # DUKASCopy fetch, hourly processing, validation
-│   └── eval/               # silhouette, linear probe, collapse, backtest
-├── model/                  # Python — training only
-│   ├── train.py            # TS-JEPA + SIGReg backbone (autoresearch edits this)
-│   ├── prepare.py          # LOCKED — data loading, tokenization, export
-│   └── requirements.txt
-├── specs/                  # Research specs (one per phase/experiment type)
-├── experiments/            # Per-run outputs: embeddings, metrics.json, git tag
-├── results/summaries/      # Human-readable outcome per experiment
-├── program.md              # Autoresearch agenda — researcher edits this
-├── DECISIONS.md            # Architecture Decision Records
-└── Taskfile.yml            # task data:fetch, task experiment:run, task eval:*
-```
-
-## Phase structure
-
-- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
-  Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
-- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
-  Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
-- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
-- **Phase 3**: Internal bank tick/position data (future, out of current scope).
-
-## Data
-
-DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
-Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
-Weekend gaps handled explicitly. See ADR-002.
-
-## Key conventions
-
-- `prepare.py` is LOCKED — never modified by agents or autoresearch
-- Evaluation metrics are always computed by the Go harness, never in Python
-- Every experiment gets a git tag: `exp/YYYYMMDD-description`
-- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
-- `program.md` is the only file the researcher edits to steer autoresearch
-- CI runs `task check` (lint + vet + test) only — no GPU, no training
-
-## Evaluation metrics
-
-Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
-realized volatility from frozen embeddings, computed by Go harness.
-
-Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
-Must co-move with val_vol_r2 — checked from experiment 1.
-
-Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
-
-## Benchmarks to beat (trading desk comparison)
-
-- GARCH(1,1) — volatility forecasting baseline
-- Historical Simulation VaR (250-day rolling) — Basel default
-- EWMA RiskMetrics (λ=0.94)
-
-## Agent guidance
-
-Read `DECISIONS.md` before making architecture suggestions.
-Do not modify `prepare.py` or `src/eval/`.
-Do not suggest changing the Python/Go separation.
-Training runs are always manual via `task experiment:run` — never triggered by CI.
-When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
+See .context/PROJECT.md for the canonical source.

DECISIONS.md

@@ -1,215 +1,120 @@
-# Architecture Decision Records
+# DECISIONS.md
 
-This file records significant technical and research decisions for `jepa-fx-risk`.
-Each record is immutable once merged — append new records rather than editing old ones.
-Format: ID · Date · Status · Context · Decision · Rationale · Consequences.
+Architecture and methodology decisions for `jepa-fx-risk`. Every non-obvious choice lives here with its rationale. Agents: read this before making any design decision.
+
+Last updated: 2026-05-27
 
 ---
 
-## ADR-001 · Architecture: TS-JEPA + SIGReg as Phase 1 backbone
+## Language split: Go-first, Python-minimal
 
-**Date:** 2026-05-28
-**Status:** Accepted
-**Supersedes:** informal decision to use TS-JEPA standalone (pre-ADR)
+**Decision:** Data pipeline, evaluation harness, CLI, and experiment runner in Go 1.24+. Python 3.12 + PyTorch only for the TS-JEPA model training loop, isolated in `model/`.
 
-### Context
+**Rationale:** Python's dependency ecosystem is a reliability risk for a multi-year project (conflicting CUDA versions, transitive breakage, environment drift). Go produces a single static binary, has excellent CSV/Parquet support, is fast enough for all non-training workloads, and keeps the reproducible parts of the project dependency-free. The training loop genuinely requires PyTorch — that is the one place Python is unavoidable.
 
-Four JEPA variants were evaluated for FX volatility forecasting and VaR/CVaR estimation:
-
-| Variant | Origin | Key property |
-|---|---|---|
-| TS-JEPA | Ennadir et al., Sep 2025 | Time-series native; EMA collapse prevention |
-| LeJEPA | Balestriero & LeCun, Nov 2025 | Proven optimal embeddings (isotropic Gaussian); SIGReg |
-| MTS-JEPA | He et al., Feb 2026 | Multi-resolution + codebook; no public code |
-| Var-JEPA | Multiple, Mar 2026 | ELBO-based UQ; no public code |
-
-Key constraints: 12 GB VRAM (Blackwell, koala), hourly DUKASCopy data, internal PoC target,
-autoresearch loop requires a clean single-scalar search space, trading desk requires an
-explainable theoretical story.
-
-### Decision
-
-Use **TS-JEPA architecture with SIGReg replacing EMA** as the Phase 1 backbone.
-
-Concretely:
-- Start from the TS-JEPA open-source implementation (arXiv:2509.25449, GitHub)
-- Remove the EMA target-network mechanism
-- Replace it with Sketched Isotropic Gaussian Regularization (SIGReg) from LeJEPA
-  (arXiv:2511.08544), controlled by a single λ hyperparameter
-- Keep TS-JEPA's temporal patchwise masking and Transformer encoder unchanged
-
-### Rationale
-
-**Why not pure TS-JEPA:** EMA is a heuristic; λ interacts with EMA momentum and
-learning rate, creating a three-way search space that is hard to navigate with autoresearch.
-EMA also has no theoretical non-stationarity guarantee.
-
-**Why not pure LeJEPA:** The reference implementation targets vision (multi-crop views).
-Adapting it to temporal patchwise masking requires non-trivial surgery and moves away from
-open code. TS-JEPA's masking is already the right inductive bias for time series.
-
-**Why the hybrid:** SIGReg is architecture-agnostic — it operates on the embedding
-distribution, not the encoder structure. Swapping EMA for SIGReg is a ~20-line change to
-TS-JEPA's training loop. The result is:
-- Time-series native (TS-JEPA masking + patch structure)
-- Provably collapse-free without heuristics (SIGReg)
-- Single search axis for autoresearch (λ ∈ [0.01, 1.0])
-- Non-stationarity robustness proven formally (arXiv:2602.19373 extends LeJEPA
-  guarantees to non-stationary target distributions — directly relevant to FX)
-- Explainable to a model validation team: "embeddings are provably optimal for
-  downstream prediction under distributional uncertainty"
-
-**Why not MTS-JEPA or Var-JEPA now:** Both lack public code (as of May 2026).
-MTS-JEPA's multi-resolution objective is the right next hypothesis (see ADR-003).
-Var-JEPA's ELBO-based UQ is a compelling future direction for CVaR estimation.
-
-### Consequences
-
-- Phase 0 (MAE baseline) is unaffected — it precedes the JEPA architecture choice
-- Issue #3 (TS-JEPA reproduction) is still the right first step; SIGReg is added after
-  reproduction is confirmed
-- The autoresearch `program.md` primary search axis is λ (SIGReg weight)
-- Secondary axes: masking block size, patch stride, encoder depth
-- `model/requirements.txt` must include the SIGReg implementation (≈20 lines,
-  can be vendored directly)
+**Boundary:** `model/` is the Python perimeter. Nothing outside it imports Python.
 
 ---
 
-## ADR-002 · Data: DUKASCopy hourly G10 FX as primary training data
+## Primary hypothesis
 
-**Date:** 2026-05-28
-**Status:** Accepted
+**Decision:** Regime detection via embedding structural separability is the primary hypothesis. Distributional VaR forecasting is secondary and only pursued if Phase 1 and 2 succeed.
 
-### Context
+**Rationale:** Regime detection is testable in Phase 1 without regulatory-quality outputs. If embeddings don't show regime structure, distributional VaR will also fail. Choosing a primary hypothesis prevents the project from retreating from one to the other on failure.
 
-Data scale is the most dangerous assumption for any SSL/JEPA approach. Daily FX data
-(~5,000 samples over 20 years) is insufficient for self-supervised pretraining.
-Two alternatives were considered: daily public data (yfinance) vs. hourly tick data
-(DUKASCopy, free, rate-limited HTTP API).
-
-### Decision
-
-Use **DUKASCopy hourly OHLCV** as the primary data source.
-
-- 10 G10 pairs: EURUSD, GBPUSD, USDJPY, USDCHF, AUDUSD, NZDUSD, USDCAD,
-  EURGBP, EURJPY, GBPJPY
-- Training window: 2008-01-01 – 2022-12-31 (~175,000 samples per pair)
-- Validation window: 2023-01-01 – 2023-12-31 (~2,600 samples)
-- Test window: 2024-01-01 – 2024-12-31 (held out, never seen during development)
-- Features per bar: log-return, log rolling-20-period HV, VIX (daily interpolated)
-- Weekend gaps handled explicitly — no interpolation across market close
-
-### Rationale
-
-Hourly data gives ~35× more samples than daily. This is the minimum threshold for
-JEPA-style SSL to show a training signal within 10-minute autoresearch experiments.
-DUKASCopy is free, reliable, and provides consistent tick-level source data back to 2003.
-
-### Consequences
-
-- The Go data pipeline (Issue #2) is the critical path for everything else
-- Phase 0 MAE baseline trains on the same 2008-2022 window
-- Daily data (yfinance) may still be used for VIX and rate differentials as auxiliary features
+**Success criterion (Phase 1):** Silhouette score > 0.35 on k-means clusters (k=3–5) vs. realised-volatility regime label (rolling 30-day HV percentile, high/low), computed on held-out test data including at least one structural break.
 
 ---
 
-## ADR-003 · Research roadmap: Phase structure and JEPA variant progression
+## Training data: 2008–2022, all G10 pairs
 
-**Date:** 2026-05-28
-**Status:** Accepted
+**Decision:** Train on 15 years of all G10 FX pairs from DUKASCopy, 2008–2022. Do not start with a single 5-year EUR/USD window.
 
-### Decision
+**Rationale:** A single 5-year window (e.g. 2019–2024) is dominated by one or two regimes and gives the encoder insufficient regime diversity to learn regime-sensitive representations. Training on 2008–2022 ensures the encoder sees: GFC (2008), EUR sovereign debt (2011–2012), SNB cap removal (2015), COVID (2020), USD rate cycle (2022). Multi-pair training also allows cross-currency transfer (Phase 4).
 
-Three-phase research roadmap:
-
-**Phase 0 — SSL feasibility gate (MAE baseline)**
-Implement a 1D temporal MAE (not JEPA) on EUR/USD hourly 2008-2022.
-Gate criteria: silhouette > 0.20 on 2023 held-out, MAE > PCA baseline, ±10% over 3 reruns.
-Purpose: validate that the data and eval harness work before committing to JEPA complexity.
-If gate fails: follow null result protocol in `specs/phase-0-ssl-feasibility.md`.
-
-**Phase 1 — TS-JEPA + SIGReg autoresearch sweep**
-Primary architecture per ADR-001.
-Autoresearch loop: `program.md`-driven, 10-min experiments, 50-experiment budget.
-Primary metric: `val_vol_r2` (linear probe R² on 1-day realized volatility).
-Gate criteria: `val_vol_r2` > GARCH-implied baseline AND Kupiec p-value > 0.05 on
-EUR/USD VaR 99%.
-Kupiec is logged from experiment 1 to verify it co-moves with `val_vol_r2`.
-
-**Phase 2 — MTS-JEPA multi-resolution hypothesis**
-Introduce parallel multi-scale predictive pathways (1h, 8h, 24h context windows)
-adapted from MTS-JEPA (arXiv:2602.04643).
-Hypothesis: multi-scale representations improve regime detection (silhouette) and
-reduce VaR exceedance clustering (Christoffersen test).
-Prerequisite: Phase 1 gate passed AND MTS-JEPA code available or reproducible from paper.
-Time-box: if MTS-JEPA code not available within 4 weeks of Phase 2 start, implement
-multi-resolution masking from scratch using Phase 1 backbone as base.
-
-**Phase 3 — Internal bank data (future)**
-Replace DUKASCopy pipeline with internal tick feed adapter.
-Fine-tune heads only; backbone frozen or lightly fine-tuned.
-Out of scope for current PoC cycle.
-
-### Consequences
-
-- Issue #5 (Phase 0 MAE) is the unblocked next executable step
-- Phase 1 autoresearch is blocked until Phase 0 passes its gate
-- Var-JEPA (ELBO-based UQ) is a named future hypothesis for CVaR estimation in Phase 2+
-  but not on the critical path
+**Alternative considered:** Start simple with EUR/USD only, expand later. Rejected because regime diversity in training is a structural requirement, not a nice-to-have. Retrofitting it in Phase 2 would require retraining from scratch.
 
 ---
 
-## ADR-004 · Evaluation: Go harness + Python training separation
+## Hard training cutoff: 2023-01-01
 
-**Date:** 2026-05-28
-**Status:** Accepted
+**Decision:** All data from 2023-01-01 onward is sealed. No architecture, hyperparameter, or methodology decision may be informed by post-2023 data. Post-2023 test set opened only once, for final evaluation.
 
-### Decision
+**Rationale:** Out-of-sample integrity is essential for publishability and honest self-assessment. The held-out window (2023–2026) includes: 2023 US regional bank stress, 2024 JPY intervention episodes. These are the test of genuine generalisation.
 
-Hard separation between training (Python) and evaluation (Go):
-
-- **Python** (`model/`): all training, embedding export, model checkpointing
-- **Go** (`src/eval/`): all evaluation metrics — silhouette, linear probe R², collapse
-  diagnostic, Kupiec/Christoffersen backtests
-- Interface: Python exports embedding matrices + labels to `experiments/RUNID/` as
-  `.npy` files; Go eval harness reads them and writes `metrics.json`
-
-### Rationale
-
-Go evaluation gives deterministic, fast, auditable metric computation with proper
-unit tests. It decouples the experimental loop from the training framework, making
-it possible to re-evaluate any past experiment without re-running training.
-The Go layer also serves as the foundation for the eventual trading desk dashboard.
-
-### Consequences
-
-- All acceptance criteria in Issues #4 and #5 are specified in terms of Go eval outputs
-- `val_vol_r2` (the autoresearch optimization metric) is computed by the Go harness,
-  not inside the Python training loop
-- Python training loop calls `task eval:probe` as a subprocess after each experiment
-  to get the scalar fed back to autoresearch
+**Enforcement:** `data/raw/` is gitignored. The download script hard-stops at 2022-12-31 for training splits. Any deviation requires a DECISIONS.md entry explaining why.
 
 ---
 
-## ADR-005 · Compute: Blackwell GPU on koala, PyTorch cu130
+## Phase 0: SSL feasibility gate before JEPA work
 
-**Date:** 2026-05-28
-**Status:** Accepted
+**Decision:** Before any JEPA-specific implementation, run a Phase 0 experiment: masked autoencoder (MAE) baseline on EUR/USD hourly data. If MAE silhouette < 0.20, SSL-based regime detection is likely not feasible on this data — stop and investigate before proceeding to JEPA.
 
-### Decision
+**Rationale:** JEPA's complexity is only justified if the core SSL premise (that latent representations capture regime structure) holds for FX data. A failed MAE experiment tells us this in 2 weeks rather than 4 months. Added after Full Grill session (2026-05-27).
 
-All GPU training runs on koala (Arch Linux, Blackwell GPU, 12 GB VRAM).
-PyTorch install: `pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu130`
-(CUDA 13.0 wheel — required for sm_120 Blackwell support; stable as of May 2026).
-Driver requirement: NVIDIA R570+, CUDA toolkit 12.8+.
+**Go/no-go threshold:** MAE silhouette < 0.20 on held-out 2023 data → pause, investigate, do not proceed to Phase 1.
 
-Ollama on iguana (Mac Studio M2 Ultra) serves the autoresearch agent LLM via the
-existing LiteLLM proxy on piguard. Agent calls never hit koala directly.
+---
 
-### Consequences
+## Architecture: TS-JEPA as starting implementation
 
-- `model/requirements.txt` must NOT pin torch to a cu124 or earlier wheel
-- CI (Issue #7) must NOT run GPU tests — CPU-only for unit tests, GPU only via
-  `task experiment:run` on koala
-- 12 GB VRAM is sufficient for <5M parameter models at batch=64; monitor if
-  autoresearch explores larger architectures
+**Decision:** Use TS-JEPA (Ennadir et al., 2025) as the starting JEPA implementation. MTS-JEPA (He et al., 2026) is the upgrade path if multi-resolution proves necessary.
+
+**Rationale:** TS-JEPA is simpler. Validate the concept before adding multi-resolution complexity. If Phase 1 succeeds with TS-JEPA, MTS-JEPA is an ablation, not a prerequisite.
+
+**Risk:** Both are preprints. Code reproducibility is unconfirmed. First task of Phase 0 is reproducing TS-JEPA on the paper's own benchmark — if this takes > 2 weeks, contact authors or fall back to implementing JEPA masking from scratch using V-JEPA codebase as reference.
+
+---
+
+## Input features: minimal for Phase 0/1
+
+**Decision:** Phase 0 and Phase 1 use three features only: log-return (hourly), rolling 20-period realised volatility (hourly), VIX (daily, interpolated to hourly).
+
+**Rationale:** Too many input features in early phases makes it impossible to distinguish "JEPA learned regime structure" from "JEPA learned to encode a feature that correlates with regime." Minimal features reduce confounding.
+
+**Expansion path:** Add DXY, G10 vol surface, yield spreads in Phase 2 if Phase 1 succeeds.
+
+---
+
+## Collapse diagnostic
+
+**Decision:** If PC1 of JEPA embeddings correlates > 0.85 with rolling 30-day HV, treat Phase 1 as a partial failure — the encoder learned volatility level, not regime structure. This is a useful finding but not the hypothesis.
+
+**Rationale:** EUR/USD hourly returns are strongly heteroskedastic. A JEPA encoder trained to predict future embeddings will strongly tend to encode current volatility as its primary latent dimension. This is predictable, not regime-sensitive. The linear probe and PC1 diagnostic together distinguish "learned volatility" from "learned regime."
+
+---
+
+## Linear probe as mandatory Phase 1 exit gate
+
+**Decision:** Train a linear model on frozen JEPA embeddings to predict realised volatility decile. R² < 0.4 → embeddings are not encoding useful risk structure → do not proceed to Phase 2.
+
+**Rationale:** If a simple linear model cannot extract volatility regime from the embeddings, the representations are not useful for risk management purposes regardless of their silhouette score. Interpretability-by-linear-probe is the minimum bar for any downstream use.
+
+---
+
+## Explicit out-of-scope (Phases 1–3)
+
+The following are explicitly out of scope for this research programme and go to a parking lot if they arise:
+
+- Options / derivatives pricing
+- Directional alpha generation
+- Live or paper trading
+- Exotic pairs beyond G10
+- Institutional deployment
+- Real-time inference systems
+
+---
+
+## Experiment spec required before any experiment
+
+**Decision:** Every experiment phase must have a written spec in `specs/` before any code runs. No specless experiments.
+
+**Rationale:** Consistent with spec-driven-dev way of working. Specs force falsifiable hypothesis statement, quantitative acceptance criteria, and explicit null-result protocol before results are known — preventing post-hoc rationalisation.
+
+---
+
+## Null results are results
+
+**Decision:** Null results (hypothesis rejected) are recorded in `results/summaries/` and treated as valid research outputs, not failures to be iterated away silently.
+
+**Rationale:** A confirmed null result (e.g. "SSL cannot find regime structure in FX hourly data") is publishable and scientifically valuable. Iterating hyperparameters until metrics pass without recording the failed attempts is p-hacking. Humble attitude; scientific approach.

README.md

@@ -1,13 +1,66 @@
-# hostexecutor
+# jepa-fx-risk
 
-> Generated from `mathias/template-go-web`.
+Research project exploring JEPA (Joint Embedding Predictive Architecture) as a framework for latent representation learning applied to FX trading risk management.
 
-## Bootstrap
+**Primary hypothesis:** JEPA embeddings trained on FX time-series will produce latent market-state representations that are structurally separable by regime without explicit regime labels, measurable by silhouette score on k-means clusters validated against held-out realised-volatility regime labels.
 
-After creating from template, run:
+**Current phase:** Phase 0 — SSL Feasibility Gate (not yet started)
+
+**Brain wing:** `jepa-fx` — query via `brain_query wing=jepa-fx`
+
+---
+
+## Architecture
+
+This is a **Go-first, Python-minimal** research project.
+
+| Layer | Language | Rationale |
+|---|---|---|
+| Data pipeline | Go | Type-safe, fast, no dependency hell |
+| Experiment runner | Go | CLI tooling, reproducible invocations |
+| Evaluation harness | Go | Silhouette, linear probe, collapse diagnostics |
+| Model training | Python + PyTorch | TS-JEPA requires it; kept minimal and isolated |
+
+## Structure
+
+```
+specs/          experiment specs (one per phase)
+src/            Go packages: pipeline, eval, cmd
+model/          Python: TS-JEPA training loop only
+experiments/    one directory per run (gitignored except summaries)
+results/        tracked: metric tables, key figures
+data/           gitignored; see data/README.md for download instructions
+notebooks/      EDA only; outputs stripped before commit
+docs/           ADRs and research notes
+```
+
+## Quick start
 
 ```bash
-go mod tidy       # regenerate go.sum with real module path
-task generate     # generate templ files
-task build        # build the binary
+task data:fetch          # download DUKASCopy G10 tick data
+task pipeline:build      # build Go data pipeline
+task check               # lint + vet + test
+task experiment:run -- --spec specs/phase-0-ssl-feasibility.md
 ```
+
+## Phases
+
+| Phase | Goal | Status |
+|---|---|---|
+| 0 | SSL feasibility gate — MAE baseline on FX data | 🔲 Not started |
+| 1 | JEPA representation PoC — silhouette > 0.35 | 🔲 Not started |
+| 2 | Regime detection validation — recall > 70%, lead > 5d | 🔲 Not started |
+| 3 | Distributional forecasting — VaR passes Kupiec | 🔲 Not started |
+| 4 | Multi-pair transfer — exotic pair improvement | 🔲 Not started |
+
+## Key constraints
+
+- **Training cutoff:** 2023-01-01 — all post-2023 data is sealed for final evaluation
+- **Training data:** 2008–2022, all G10 pairs, DUKASCopy
+- **Out of scope:** options pricing, directional alpha, live trading, exotic pairs (Phases 1–3)
+- **End-state:** publishable research result — not institutional deployment
+
+## Related
+
+- Brain wing: `wiki/jepa-fx/` — decisions, hypotheses, failure modes
+- Synthesis document: `JEPA_FX_Risk_Synthesis.docx` (session 2026-05-27)

STATUS.md

@@ -1,22 +0,0 @@
-# Autoresearch STATUS
-
-| iter | val_vol_r2 | delta | action | secs | gpu | change |
-|------|-----------|-------|--------|------|-----|--------|
-| 1 | 0.3749 | +0.0928 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
-| 1 | 0.3011 | +0.0776 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
-| 2 | 0.3032 | +0.0021 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
-| 1 | 0.2759 | -0.0273 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
-| 2 | 0.3442 | +0.0410 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter2 |
-| 3 | 0.3371 | -0.0071 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter3 |
-| 4 | 0.3143 | -0.0299 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter4 |
-| 5 | 0.3355 | -0.0087 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter5 |
-| 6 | 0.2377 | -0.1065 | revert | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter6 |
-| 1 | -0.1247 | +0.0296 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
-| 2 | -0.1203 | +0.0044 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
-| 3 | -0.0716 | +0.0487 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
-| 4 | 0.0590 | +0.1306 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter4 |
-| 5 | 0.0599 | +0.0009 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter5 |
-| 1 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
-| 2 | 0.0577 | -0.0022 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter2 |
-| 3 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
-| 4 | -0.1613 | -0.2212 | revert | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter4 |

Taskfile.yml

@@ -1,82 +1,79 @@
 version: '3'
 
 tasks:
-  generate:
-    desc: Run templ generate
-    cmds: [templ generate]
-  build:
-    desc: Build all binaries
-    deps: [generate]
+
+  # ── Quality gate ──────────────────────────────────────────────────────────
+
+  check:
+    desc: "Full quality gate: lint + vet + test (run before every commit)"
     cmds:
-      - go build -o bin/jepa-fx-risk ./cmd/jepa-fx-risk
-      - go build -o bin/eval ./cmd/eval
-  run:
-    deps: [build]
-    cmds: [./bin/jepa-fx-risk]
+      - golangci-lint run ./src/...
+      - go vet ./src/...
+      - go test ./src/... -race -count=1
+      - cd model && python -m pytest tests/ -q
+
   test:
-    desc: Run all tests
-    deps: [generate]
-    cmds: [go test ./... -race]
+    desc: Run Go tests only
+    cmds: [go test ./src/... -race]
+
+  lint:
+    desc: Lint Go code
+    cmds: [golangci-lint run ./src/...]
+
+  # ── Data ─────────────────────────────────────────────────────────────────
 
   data:fetch:
-    desc: "Download EUR/USD M1 from histdata (set YEARS env var)"
-    cmds: [.venv/bin/python scripts/fetch_data.py]
-  data:fetch:historical:
-    desc: "Download EUR/USD M1 2008-2018 from histdata"
-    cmds:
-      - YEARS=2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018 .venv/bin/python scripts/fetch_data.py
-  data:prepare:daily:
-    desc: "Rebuild eurusd_daily.parquet from all M1 zips"
-    cmds: [.venv/bin/python scripts/prepare_data.py]
-  data:prepare:hourly:
-    desc: "Build eurusd_hourly.parquet from all M1 zips"
-    cmds: [.venv/bin/python scripts/prepare_hourly.py]
-  data:prepare:all:
-    desc: "Build both daily and hourly parquets"
-    deps: [data:prepare:daily, data:prepare:hourly]
-  train:multipair:
-    desc: "Train 5-pair G10 HEPA (D=256, best config, phase1_r2≈0.44)"
-    cmds: [JEPA_USE_MULTIPAIR=1 JEPA_D_MODEL=256 .venv/bin/python train.py]
+    desc: "Download G10 tick data from DUKASCopy (2003–2022)"
+    cmds: [go run ./src/cmd/fetch --config data/config.yaml]
 
-  data:fetch:multipair:
-    desc: "Download G10 M1 data (GBPUSD/USDJPY/USDCHF/AUDUSD) 2008-2023 from histdata"
-    cmds: [.venv/bin/python scripts/fetch_multipair.py]
-  data:prepare:pair:
-    desc: "Build {PAIR}_hourly.parquet from data/raw/{PAIR}/ (e.g. PAIR=gbpusd)"
-    cmds: ["PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}"]
-    vars:
-      PAIR: '{{default "eurusd" .PAIR}}'
-  data:prepare:multipair:
-    desc: "Merge 5-pair hourly parquets into eurusd_multipair.parquet"
-    cmds: [.venv/bin/python scripts/prepare_multipair.py]
-  data:test:
-    desc: "Run Python data pipeline tests"
-    cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py tests/test_multipair.py -v]
+  data:process:
+    desc: "Resample ticks → hourly OHLCV + features (log-return, rolling HV)"
+    cmds: [go run ./src/cmd/process --input data/raw --output data/processed]
+
+  data:validate:
+    desc: "Validate processed data: gap detection, outlier report, regime coverage"
+    cmds: [go run ./src/cmd/validate --input data/processed]
+
+  # ── Experiment ───────────────────────────────────────────────────────────
+
+  experiment:run:
+    desc: "Run an experiment from a spec file. Usage: task experiment:run -- --spec specs/phase-0-ssl-feasibility.md"
+    cmds: [go run ./src/cmd/experiment {{.CLI_ARGS}}]
+
+  experiment:list:
+    desc: List all recorded experiment runs
+    cmds: [ls -lt experiments/ | head -20]
+
+  # ── Evaluation ───────────────────────────────────────────────────────────
+
+  eval:silhouette:
+    desc: "Compute silhouette score on embedding output. Usage: task eval:silhouette -- --run experiments/RUNID"
+    cmds: [go run ./src/cmd/eval silhouette {{.CLI_ARGS}}]
 
   eval:probe:
-    desc: "Run linear-probe (val_vol_r2) on embeddings from metrics.json"
-    cmds: [./bin/eval -metric probe]
-  eval:silhouette:
-    desc: "Run silhouette on embeddings vs binary HV labels"
-    cmds: [./bin/eval -metric silhouette]
+    desc: "Run linear probe on frozen embeddings vs. realised-vol decile"
+    cmds: [go run ./src/cmd/eval probe {{.CLI_ARGS}}]
+
   eval:collapse:
-    desc: "Run effective-rank collapse diagnostic"
-    cmds: [./bin/eval -metric erank]
-  lint:
-    cmds: [golangci-lint run ./...]
-  check:
-    desc: Lint, vet, and test (used by CI)
-    deps: [generate]
+    desc: "Check collapse diagnostic: PC1 correlation with rolling HV"
+    cmds: [go run ./src/cmd/eval collapse {{.CLI_ARGS}}]
+
+  # ── Model (Python) ───────────────────────────────────────────────────────
+
+  model:train:
+    desc: "Train TS-JEPA model. Usage: task model:train -- --config model/configs/phase0.yaml"
+    dir: model
+    cmds: [python train.py {{.CLI_ARGS}}]
+
+  model:setup:
+    desc: "Create Python venv and install model dependencies (uv)"
+    dir: model
     cmds:
-      - golangci-lint run ./...
-      - go vet ./...
-      - go test ./... -race -count=1
+      - uv venv .venv
+      - uv pip install -r requirements.txt
+
+  # ── Context ──────────────────────────────────────────────────────────────
 
   context:sync:
-    desc: Regenerate all harness-specific context files
-    cmds:
-      - bash scripts/context-sync.sh
-  context:sync:claude:
-    cmds: [bash scripts/context-sync.sh claude]
-  context:sync:agents:
-    cmds: [bash scripts/context-sync.sh agents]
+    desc: "Regenerate CLAUDE.md and AGENTS.md from .context/PROJECT.md"
+    cmds: [bash scripts/context-sync.sh]

cmd/eval/main.go

@@ -1,194 +0,0 @@
-// cmd/eval — CLI driver for the jepa-fx-risk evaluation harness.
-//
-//	./bin/eval -metric probe|silhouette|erank [-emb embeddings.json]
-//
-// embeddings.json format (from train.py EXPORT_EMBEDDINGS=1):
-//
-//	{
-//	  "embeddings":         [[...], ...],  // OOS frozen embeddings
-//	  "realized_vol":       [...],         // OOS target (next-day RV)
-//	  "hv_label":           [...],         // binary HV label (top-33%)
-//	  "train_embeddings":   [[...], ...],  // train-set frozen embeddings
-//	  "train_realized_vol": [...]          // train-set RV targets
-//	}
-//
-// eval:probe standardises both sets using train statistics (no leakage).
-// Falls back to internal 70/30 split of OOS if train_embeddings absent.
-package main
-
-import (
-	"encoding/json"
-	"flag"
-	"fmt"
-	"log/slog"
-	"math"
-	"os"
-
-	"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
-)
-
-func main() {
-	metric := flag.String("metric", "probe", "probe | silhouette | erank")
-	embFile := flag.String("emb", "embeddings.json", "path to embeddings JSON")
-	flag.Parse()
-
-	log := slog.New(slog.NewJSONHandler(os.Stdout, nil))
-
-	d, err := readJSON(*embFile)
-	if err != nil {
-		log.Error("load embeddings", "err", err)
-		os.Exit(1)
-	}
-	log.Info("loaded", "oos", len(d.Embeddings), "dim", len(d.Embeddings[0]),
-		"train", len(d.TrainEmbeddings), "metric", *metric)
-
-	switch *metric {
-	case "probe":
-		var r2 float64
-		if len(d.TrainEmbeddings) > 0 {
-			// standardise both sets using train statistics to prevent leakage
-			trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
-			oosEmb := applyStandardise(d.Embeddings, mu, sd)
-			r2 = eval.LinearProbeTrainTest(trEmb, d.TrainRealizedVol, oosEmb, d.RealizedVol, 1e-3)
-			log.Info("probe mode", "fit_on", "train_embeddings", "eval_on", "oos")
-		} else {
-			// fallback: internal 70/30 split of OOS embeddings
-			oosEmb, mu, sd := standardiseCompute(d.Embeddings)
-			n70 := int(float64(len(oosEmb)) * 0.7)
-			oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
-			r2 = eval.LinearProbeTrainTest(oosEmb[:n70], d.RealizedVol[:n70],
-				oos70, d.RealizedVol[n70:], 1e-3)
-			log.Info("probe mode", "fit_on", "oos[0:70%]", "eval_on", "oos[70%:]")
-		}
-		fmt.Printf(`{"metric":"val_vol_r2","value":%.6f}`+"\n", r2)
-		log.Info("linear probe", "val_vol_r2", fmt.Sprintf("%.4f", r2))
-
-	case "silhouette":
-		if len(d.HVLabel) == 0 {
-			log.Error("silhouette requires hv_label in embeddings.json")
-			os.Exit(1)
-		}
-		oosEmb := standardise(d.Embeddings)
-		sil, err := eval.Silhouette(oosEmb, d.HVLabel)
-		if err != nil {
-			log.Error("silhouette", "err", err)
-			os.Exit(1)
-		}
-		fmt.Printf(`{"metric":"silhouette","value":%.6f}`+"\n", sil)
-		log.Info("silhouette", "score", fmt.Sprintf("%.4f", sil))
-
-	case "erank":
-		oosEmb := standardise(d.Embeddings)
-		er := eval.EffectiveRank(oosEmb)
-		fmt.Printf(`{"metric":"effective_rank","value":%.6f}`+"\n", er)
-		log.Info("effective rank", "erank", fmt.Sprintf("%.2f", er))
-
-	case "var":
-		// Parametric 99% VaR breach rate from probe predictions vs actual realized vol.
-		// Requires train_embeddings (for no-leakage probe fit) and realized_vol (OOS).
-		if len(d.RealizedVol) == 0 {
-			log.Error("var requires realized_vol in embeddings.json")
-			os.Exit(1)
-		}
-		var predVol []float64
-		if len(d.TrainEmbeddings) > 0 {
-			trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
-			oosEmb := applyStandardise(d.Embeddings, mu, sd)
-			predVol = eval.LinearProbePredict(trEmb, d.TrainRealizedVol, oosEmb, 1e-3)
-		} else {
-			oosEmb, mu, sd := standardiseCompute(d.Embeddings)
-			n70 := int(float64(len(oosEmb)) * 0.7)
-			oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
-			predVol = eval.LinearProbePredict(oosEmb[:n70], d.RealizedVol[:n70], oos70, 1e-3)
-			d.RealizedVol = d.RealizedVol[n70:]
-		}
-		const z99 = 2.326
-		breachRate, kupiecP := eval.VaRBreachRate(predVol, d.RealizedVol, z99)
-		fmt.Printf(`{"metric":"VaR_breach_rate_99_oos_regime_cond","value":%.6f,"kupiec_p":%.6f}`+"\n",
-			breachRate, kupiecP)
-		log.Info("VaR breach rate 99%", "breach_rate", fmt.Sprintf("%.4f", breachRate),
-			"kupiec_p", fmt.Sprintf("%.4f", kupiecP))
-
-	default:
-		log.Error("unknown metric", "metric", *metric)
-		os.Exit(1)
-	}
-}
-
-type embJSON struct {
-	Embeddings       [][]float64 `json:"embeddings"`
-	Dates            []string    `json:"dates"`
-	RealizedVol      []float64   `json:"realized_vol"`
-	HVLabel          []int       `json:"hv_label"`
-	TrainEmbeddings  [][]float64 `json:"train_embeddings"`
-	TrainRealizedVol []float64   `json:"train_realized_vol"`
-}
-
-func readJSON(path string) (*embJSON, error) {
-	f, err := os.Open(path)
-	if err != nil {
-		return nil, fmt.Errorf("open %s: %w", path, err)
-	}
-	defer func() { _ = f.Close() }()
-	var d embJSON
-	if err := json.NewDecoder(f).Decode(&d); err != nil {
-		return nil, fmt.Errorf("decode: %w", err)
-	}
-	if len(d.Embeddings) == 0 {
-		return nil, fmt.Errorf("empty embeddings in %s", path)
-	}
-	return &d, nil
-}
-
-// standardise centres + scales to zero mean / unit std; returns normalised rows.
-func standardise(rows [][]float64) [][]float64 {
-	out, _, _ := standardiseCompute(rows)
-	return out
-}
-
-// standardiseCompute centres + scales and returns (normalised, mu, sd) for reuse.
-func standardiseCompute(rows [][]float64) ([][]float64, []float64, []float64) {
-	if len(rows) == 0 {
-		return rows, nil, nil
-	}
-	n, dim := len(rows), len(rows[0])
-	mu := make([]float64, dim)
-	for _, r := range rows {
-		for j, v := range r {
-			mu[j] += v
-		}
-	}
-	for j := range mu {
-		mu[j] /= float64(n)
-	}
-	sd := make([]float64, dim)
-	for _, r := range rows {
-		for j, v := range r {
-			diff := v - mu[j]
-			sd[j] += diff * diff
-		}
-	}
-	for j := range sd {
-		sd[j] = math.Sqrt(sd[j]/float64(n)) + 1e-8
-	}
-	out := make([][]float64, n)
-	for i, r := range rows {
-		out[i] = make([]float64, dim)
-		for j, v := range r {
-			out[i][j] = (v - mu[j]) / sd[j]
-		}
-	}
-	return out, mu, sd
-}
-
-// applyStandardise normalises rows using pre-computed mu and sd.
-func applyStandardise(rows [][]float64, mu, sd []float64) [][]float64 {
-	out := make([][]float64, len(rows))
-	for i, r := range rows {
-		out[i] = make([]float64, len(r))
-		for j, v := range r {
-			out[i][j] = (v - mu[j]) / sd[j]
-		}
-	}
-	return out
-}

cmd/hostexecutor/main.go

@@ -5,7 +5,7 @@ import (
 	"net/http"
 	"os"
 
-	"git.d-ma.be/mathias/jepa-fx-risk/internal/web"
+	"gitea.d-ma.be/mathias/hostexecutor/internal/web"
 )
 
 func main() {

data/README.md

@@ -0,0 +1,36 @@
+# Data
+
+Raw and processed FX data is **gitignored** — never committed.
+
+## Download: DUKASCopy G10 tick data
+
+Use the provided fetch task:
+
+```bash
+task data:fetch    # downloads raw tick data → data/raw/
+task data:process  # resamples to hourly OHLCV + features → data/processed/
+task data:validate # gap detection, outlier report, regime coverage check
+```
+
+## Expected structure (local only)
+
+```
+data/
+  raw/          G10 pairs, tick OHLCV, 2003–present (gitignored)
+  processed/    Hourly log-returns, rolling HV, VIX-merged (gitignored)
+  cache/        Intermediate artefacts (gitignored)
+  config.yaml   Download configuration (committed)
+```
+
+## Pairs
+
+EUR/USD, GBP/USD, USD/JPY, USD/CHF, AUD/USD, USD/CAD, NZD/USD, EUR/GBP, EUR/JPY, EUR/CHF
+
+## Splits
+
+| Split | Date range | Purpose |
+|---|---|---|
+| Train | 2008-01-01 – 2022-12-31 | Model training only |
+| Held-out test | 2023-01-01 – 2023-12-31 | Final evaluation — do not open until evaluation |
+
+**Training cutoff is hard: 2023-01-01. No post-2022 data informs any design decision.**

experiments/README.md

@@ -0,0 +1,32 @@
+# Experiments
+
+One directory per experiment run. Large outputs (embeddings, checkpoints, logs) are gitignored. Only metric summaries are committed to `results/summaries/`.
+
+## Naming convention
+
+```
+YYYYMMDD-HHMMSS-phase-N-short-description/
+  config.yaml       parameters used for this run (committed via results/summaries/)
+  metrics.json      final metric snapshot (committed via results/summaries/)
+  embeddings/       (gitignored — large)
+  checkpoints/      (gitignored — large)
+  logs/             (gitignored — large)
+```
+
+## Git tag convention
+
+Every run that produces reportable metrics gets a tag:
+
+```
+exp/YYYYMMDD-phase-N-description
+```
+
+Example: `exp/20260601-phase-0-mae-baseline`
+
+## Starting a run
+
+```bash
+task experiment:run -- --spec specs/phase-0-ssl-feasibility.md --tag my-run-description
+```
+
+The runner creates the directory, writes config, runs the experiment, and outputs metrics.json.

fixtures/phase-a-toy.json

@@ -1,11 +0,0 @@
-{
-  "strategic_question": "What is the highest-leverage path to a JEPA-based FX tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach calibration, given one GPU and a solo researcher?",
-  "nodes": [
-    {
-      "id": "phase-a-toy",
-      "status": "autoresearch-ready",
-      "question": "Improve the OOS linear-probe R² (val_vol_r2) of the HEPA encoder on EUR/USD daily realized vol. The encoder is a small causal transformer trained with VICReg. Vary one hyperparameter or architectural choice per iteration — model size, learning rate, window, patch length, depth, VICReg loss weights — to push val_vol_r2 as high as possible on the 2022-2023 OOS slice.",
-      "candidate_metric": "val_vol_r2"
-    }
-  ]
-}

go.mod

@@ -1,5 +1,7 @@
-module git.d-ma.be/mathias/jepa-fx-risk
+module gitea.d-ma.be/mathias/hostexecutor
 
 go 1.26
 
-require github.com/a-h/templ v0.3.1020
+require (
+	github.com/a-h/templ v0.2.778
+)

go.sum

@@ -1,4 +0,0 @@
-github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
-github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
-github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
-github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=

internal/eval/eval.go

@@ -1,384 +0,0 @@
-// Package eval implements the Go evaluation harness for jepa-fx-risk (#4).
-// Three diagnostics on frozen embeddings exported from train.py:
-//   - LinearProbe  — val_vol_r2: OOS R² of a ridge probe predicting next-day realized vol
-//   - Silhouette   — mean silhouette score of embeddings vs a binary label (HV regime)
-//   - EffectiveRank — Roy's effective rank: exp(H(σ²)) where H is entropy of normalised singular values
-package eval
-
-import (
-	"errors"
-	"math"
-)
-
-// LinearProbeTrainTest fits ridge regression on (trainEmb, trainY) and evaluates
-// on (testEmb, testY). Returns OOS R². Use this for proper held-out evaluation.
-func LinearProbeTrainTest(trainEmb [][]float64, trainY []float64,
-	testEmb [][]float64, testY []float64, lambda float64,
-) float64 {
-	n := len(trainEmb)
-	if n == 0 || len(testEmb) == 0 {
-		return 0
-	}
-	d := len(trainEmb[0])
-	p := d + 1
-
-	A := make([][]float64, n)
-	for i, e := range trainEmb {
-		row := make([]float64, p)
-		copy(row, e)
-		row[d] = 1.0
-		A[i] = row
-	}
-	AtA := make([][]float64, p)
-	for i := range AtA {
-		AtA[i] = make([]float64, p)
-	}
-	Aty := make([]float64, p)
-	for i := 0; i < n; i++ {
-		for j := 0; j < p; j++ {
-			Aty[j] += A[i][j] * trainY[i]
-			for k := 0; k < p; k++ {
-				AtA[j][k] += A[i][j] * A[i][k]
-			}
-		}
-	}
-	for j := 0; j < p; j++ {
-		AtA[j][j] += lambda
-	}
-	w := solveCholesky(AtA, Aty)
-
-	yMean := mean(testY)
-	var ssRes, ssTot float64
-	for i, e := range testEmb {
-		row := make([]float64, p)
-		copy(row, e)
-		row[d] = 1.0
-		pred := dot(row, w)
-		ssRes += (testY[i] - pred) * (testY[i] - pred)
-		ssTot += (testY[i] - yMean) * (testY[i] - yMean)
-	}
-	if ssTot == 0 {
-		return 0
-	}
-	return 1 - ssRes/ssTot
-}
-
-// LinearProbe fits a ridge regression (closed-form) on (emb, y) with regularisation λ
-// and returns R² on the same data. Call with train embeddings; probe on held-out by
-// splitting before calling.
-//
-// emb[i] is the embedding vector for sample i; y[i] is the scalar target.
-func LinearProbe(emb [][]float64, y []float64, lambda float64) float64 {
-	n := len(emb)
-	if n == 0 {
-		return 0
-	}
-	d := len(emb[0])
-
-	// Build augmented design matrix A = [emb | 1] (n × d+1)
-	A := make([][]float64, n)
-	for i, e := range emb {
-		row := make([]float64, d+1)
-		copy(row, e)
-		row[d] = 1.0
-		A[i] = row
-	}
-
-	// Normal equations: (AᵀA + λI) w = Aᵀy  (ridge)
-	p := d + 1
-	AtA := make([][]float64, p)
-	for i := range AtA {
-		AtA[i] = make([]float64, p)
-	}
-	Aty := make([]float64, p)
-
-	for i := 0; i < n; i++ {
-		for j := 0; j < p; j++ {
-			Aty[j] += A[i][j] * y[i]
-			for k := 0; k < p; k++ {
-				AtA[j][k] += A[i][j] * A[i][k]
-			}
-		}
-	}
-	for j := 0; j < p; j++ {
-		AtA[j][j] += lambda
-	}
-
-	w := solveCholesky(AtA, Aty)
-
-	// R² = 1 - SS_res / SS_tot
-	yMean := mean(y)
-	var ssRes, ssTot float64
-	for i := 0; i < n; i++ {
-		pred := dot(A[i], w)
-		ssRes += (y[i] - pred) * (y[i] - pred)
-		ssTot += (y[i] - yMean) * (y[i] - yMean)
-	}
-	if ssTot == 0 {
-		return 0
-	}
-	return 1 - ssRes/ssTot
-}
-
-// Silhouette returns the mean silhouette coefficient of the embeddings with respect
-// to the given integer labels. Distances are Euclidean. Returns an error if fewer
-// than 2 distinct labels are present.
-func Silhouette(emb [][]float64, labels []int) (float64, error) {
-	n := len(emb)
-	if n == 0 {
-		return 0, errors.New("eval: empty embeddings")
-	}
-
-	// count distinct labels
-	labelSet := map[int]struct{}{}
-	for _, l := range labels {
-		labelSet[l] = struct{}{}
-	}
-	if len(labelSet) < 2 {
-		return 0, errors.New("eval: silhouette requires at least 2 distinct labels")
-	}
-
-	// group indices by label
-	groups := map[int][]int{}
-	for i, l := range labels {
-		groups[l] = append(groups[l], i)
-	}
-
-	var total float64
-	for i := 0; i < n; i++ {
-		li := labels[i]
-
-		// a(i) = mean intra-cluster distance
-		var aSum float64
-		inGroup := groups[li]
-		for _, j := range inGroup {
-			if j != i {
-				aSum += euclidean(emb[i], emb[j])
-			}
-		}
-		var a float64
-		if len(inGroup) > 1 {
-			a = aSum / float64(len(inGroup)-1)
-		}
-
-		// b(i) = min mean inter-cluster distance
-		b := math.MaxFloat64
-		for l, idxs := range groups {
-			if l == li {
-				continue
-			}
-			var dSum float64
-			for _, j := range idxs {
-				dSum += euclidean(emb[i], emb[j])
-			}
-			avg := dSum / float64(len(idxs))
-			if avg < b {
-				b = avg
-			}
-		}
-
-		s := (b - a) / math.Max(a, b)
-		total += s
-	}
-	return total / float64(n), nil
-}
-
-// EffectiveRank computes Roy's effective rank of the embedding matrix:
-// exp(H) where H = -∑ pᵢ log(pᵢ) is the Shannon entropy of the normalised
-// squared singular values. Returns 1 for a rank-1 matrix and ≈ dim for
-// a full-rank isotropic matrix.
-func EffectiveRank(emb [][]float64) float64 {
-	n := len(emb)
-	if n == 0 {
-		return 0
-	}
-	d := len(emb[0])
-
-	// Compute covariance-like matrix CᵀC where C is mean-centered embedding.
-	mu := make([]float64, d)
-	for _, e := range emb {
-		for j, v := range e {
-			mu[j] += v
-		}
-	}
-	for j := range mu {
-		mu[j] /= float64(n)
-	}
-
-	// C = emb - mu  (n × d); compute CᵀC (d × d)
-	CtC := make([][]float64, d)
-	for i := range CtC {
-		CtC[i] = make([]float64, d)
-	}
-	for _, e := range emb {
-		for j := 0; j < d; j++ {
-			cj := e[j] - mu[j]
-			for k := 0; k < d; k++ {
-				CtC[j][k] += cj * (e[k] - mu[k])
-			}
-		}
-	}
-
-	// Eigenvalues of CᵀC via power iteration approximation isn't great;
-	// use the Frobenius / trace approach: σᵢ² ∝ eigenvalues of CᵀC.
-	// For a pure-Go impl without LAPACK: use the fact that the normalised
-	// squared singular values equal normalised eigenvalues of CᵀC.
-	// Compute them via Jacobi iteration for small d, or use the analytical
-	// formula for 2×2, or use iterative QR for general d.
-	eigs := jacobiEigenvalues(CtC)
-
-	// normalise to sum-1 distribution
-	var sumEig float64
-	for _, v := range eigs {
-		if v > 0 {
-			sumEig += v
-		}
-	}
-	if sumEig == 0 {
-		return 1
-	}
-	var H float64
-	for _, v := range eigs {
-		if v > 0 {
-			p := v / sumEig
-			H -= p * math.Log(p)
-		}
-	}
-	return math.Exp(H)
-}
-
-// ── internal helpers ──────────────────────────────────────────────────────────
-
-func euclidean(a, b []float64) float64 {
-	var s float64
-	for i := range a {
-		d := a[i] - b[i]
-		s += d * d
-	}
-	return math.Sqrt(s)
-}
-
-func dot(a, b []float64) float64 {
-	var s float64
-	for i := range a {
-		s += a[i] * b[i]
-	}
-	return s
-}
-
-func mean(y []float64) float64 {
-	var s float64
-	for _, v := range y {
-		s += v
-	}
-	return s / float64(len(y))
-}
-
-// solveCholesky solves Ax = b for symmetric positive-definite A via
-// Cholesky decomposition. Falls back to pseudo-inverse on failure.
-func solveCholesky(A [][]float64, b []float64) []float64 {
-	n := len(A)
-	// Cholesky decomposition: A = LLᵀ
-	L := make([][]float64, n)
-	for i := range L {
-		L[i] = make([]float64, n)
-	}
-	for i := 0; i < n; i++ {
-		for j := 0; j <= i; j++ {
-			s := A[i][j]
-			for k := 0; k < j; k++ {
-				s -= L[i][k] * L[j][k]
-			}
-			if i == j {
-				if s <= 0 {
-					s = 1e-12
-				}
-				L[i][j] = math.Sqrt(s)
-			} else {
-				L[i][j] = s / L[j][j]
-			}
-		}
-	}
-	// Forward substitution Ly = b
-	y := make([]float64, n)
-	for i := 0; i < n; i++ {
-		s := b[i]
-		for k := 0; k < i; k++ {
-			s -= L[i][k] * y[k]
-		}
-		y[i] = s / L[i][i]
-	}
-	// Back substitution Lᵀx = y
-	x := make([]float64, n)
-	for i := n - 1; i >= 0; i-- {
-		s := y[i]
-		for k := i + 1; k < n; k++ {
-			s -= L[k][i] * x[k]
-		}
-		x[i] = s / L[i][i]
-	}
-	return x
-}
-
-// jacobiEigenvalues returns eigenvalues of a symmetric matrix via Jacobi iteration.
-func jacobiEigenvalues(A [][]float64) []float64 {
-	n := len(A)
-	// copy
-	a := make([][]float64, n)
-	for i := range a {
-		a[i] = make([]float64, n)
-		copy(a[i], A[i])
-	}
-
-	const maxIter = 100
-	const tol = 1e-10
-	for iter := 0; iter < maxIter; iter++ {
-		// find largest off-diagonal element
-		p, q, amax := 0, 1, 0.0
-		for i := 0; i < n; i++ {
-			for j := i + 1; j < n; j++ {
-				if v := math.Abs(a[i][j]); v > amax {
-					amax = v
-					p, q = i, j
-				}
-			}
-		}
-		if amax < tol {
-			break
-		}
-		// Jacobi rotation
-		theta := 0.5 * math.Atan2(2*a[p][q], a[q][q]-a[p][p])
-		c, s := math.Cos(theta), math.Sin(theta)
-		// apply rotation
-		newA := make([][]float64, n)
-		for i := range newA {
-			newA[i] = make([]float64, n)
-			copy(newA[i], a[i])
-		}
-		app := c*c*a[p][p] + 2*c*s*a[p][q] + s*s*a[q][q]
-		aqq := s*s*a[p][p] - 2*c*s*a[p][q] + c*c*a[q][q]
-		apq := 0.0
-		newA[p][p] = app
-		newA[q][q] = aqq
-		newA[p][q] = apq
-		newA[q][p] = apq
-		for r := 0; r < n; r++ {
-			if r == p || r == q {
-				continue
-			}
-			arp := c*a[r][p] + s*a[r][q]
-			arq := -s*a[r][p] + c*a[r][q]
-			newA[r][p] = arp
-			newA[p][r] = arp
-			newA[r][q] = arq
-			newA[q][r] = arq
-		}
-		a = newA
-	}
-
-	eigs := make([]float64, n)
-	for i := range eigs {
-		eigs[i] = a[i][i]
-	}
-	return eigs
-}

internal/eval/eval_test.go

@@ -1,138 +0,0 @@
-package eval_test
-
-import (
-	"math"
-	"math/rand"
-	"testing"
-
-	"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
-)
-
-func seededRNG(seed int64) *rand.Rand {
-	return rand.New(rand.NewSource(seed))
-}
-
-// ── LinearProbe (val_vol_r2) ──────────────────────────────────────────────────
-
-func TestLinearProbe_Perfect(t *testing.T) {
-	n := 50
-	emb := make([][]float64, n)
-	y := make([]float64, n)
-	for i := range emb {
-		emb[i] = []float64{float64(i)}
-		y[i] = float64(i)
-	}
-	r2 := eval.LinearProbe(emb, y, 1e-3)
-	if r2 < 0.99 {
-		t.Fatalf("perfect predictor: want R²≥0.99, got %.4f", r2)
-	}
-}
-
-func TestLinearProbe_ConstantTarget(t *testing.T) {
-	n := 40
-	emb := make([][]float64, n)
-	y := make([]float64, n)
-	for i := range emb {
-		emb[i] = []float64{float64(i), float64(i * i)}
-		y[i] = 3.0
-	}
-	r2 := eval.LinearProbe(emb, y, 1e-3)
-	if r2 > 0.01 {
-		t.Fatalf("constant target: want R²≤0.01, got %.4f", r2)
-	}
-}
-
-func TestLinearProbe_NoiseEmbedding(t *testing.T) {
-	rng := seededRNG(42)
-	n := 80
-	emb := make([][]float64, n)
-	y := make([]float64, n)
-	for i := range emb {
-		emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
-		y[i] = float64(i)
-	}
-	r2 := eval.LinearProbe(emb, y, 1e-3)
-	if r2 > 0.10 {
-		t.Fatalf("noise embedding: want R²<0.10, got %.4f", r2)
-	}
-}
-
-// ── Silhouette ────────────────────────────────────────────────────────────────
-
-func TestSilhouette_PerfectClusters(t *testing.T) {
-	emb := make([][]float64, 40)
-	labels := make([]int, 40)
-	for i := range emb {
-		if i < 20 {
-			emb[i] = []float64{0.0, 0.0}
-			labels[i] = 0
-		} else {
-			emb[i] = []float64{1000.0, 1000.0}
-			labels[i] = 1
-		}
-	}
-	sil, err := eval.Silhouette(emb, labels)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if sil < 0.95 {
-		t.Fatalf("perfect clusters: want sil≥0.95, got %.4f", sil)
-	}
-}
-
-func TestSilhouette_SingleLabel(t *testing.T) {
-	emb := [][]float64{{1, 2}, {3, 4}, {5, 6}}
-	labels := []int{0, 0, 0}
-	_, err := eval.Silhouette(emb, labels)
-	if err == nil {
-		t.Fatal("expected error for single-label input")
-	}
-}
-
-func TestSilhouette_RandomClusters(t *testing.T) {
-	rng := seededRNG(7)
-	n := 60
-	emb := make([][]float64, n)
-	labels := make([]int, n)
-	for i := range emb {
-		emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
-		labels[i] = i % 2
-	}
-	sil, err := eval.Silhouette(emb, labels)
-	if err != nil {
-		t.Fatal(err)
-	}
-	if math.Abs(sil) > 0.30 {
-		t.Fatalf("random clusters: want |sil|≤0.30, got %.4f", sil)
-	}
-}
-
-// ── EffectiveRank ─────────────────────────────────────────────────────────────
-
-func TestEffectiveRank_Rank1(t *testing.T) {
-	emb := make([][]float64, 30)
-	for i := range emb {
-		emb[i] = []float64{1.0, 2.0, 3.0, 4.0}
-	}
-	er := eval.EffectiveRank(emb)
-	if er > 1.5 {
-		t.Fatalf("rank-1 matrix: want erank≤1.5, got %.4f", er)
-	}
-}
-
-func TestEffectiveRank_FullRank(t *testing.T) {
-	rng := seededRNG(99)
-	dim := 8
-	emb := make([][]float64, 200)
-	for i := range emb {
-		row := make([]float64, dim)
-		for j := range row {
-			row[j] = rng.NormFloat64()
-		}
-		emb[i] = row
-	}
-	er := eval.EffectiveRank(emb)
-	if er < float64(dim)*0.7 {
-		t.Fatalf("full-rank: want erank≥%.1f, got %.4f", float64(dim)*0.7, er)
-	}
-}

internal/eval/var.go

@@ -1,109 +0,0 @@
-package eval
-
-import "math"
-
-// VaRBreachRate computes the parametric 99% VaR breach rate and Kupiec POF p-value.
-//
-// VaR_99_t  = predVol[t] × z99   (z99 = 2.326 for 99% normal VaR)
-// breach_t  = actualVol[t] > VaR_99_t  (strict inequality)
-// breachRate = fraction of breaches over all steps
-// kupiecP   = Kupiec POF p-value: P(chi²(1) > LR) where LR is the likelihood ratio
-//
-//	testing H0: true breach probability = 1%. High p = well-calibrated.
-//
-// Returns (0, 1) for empty or mismatched input.
-func VaRBreachRate(predVol, actualVol []float64, z99 float64) (breachRate, kupiecP float64) {
-	n := len(predVol)
-	if n == 0 || n != len(actualVol) {
-		return 0, 1
-	}
-
-	var n1 int
-	for i := 0; i < n; i++ {
-		if actualVol[i] > predVol[i]*z99 {
-			n1++
-		}
-	}
-
-	breachRate = float64(n1) / float64(n)
-	kupiecP = kupiecPOF(n, n1, 0.01)
-	return
-}
-
-// kupiecPOF returns the Kupiec Proportion-of-Failures p-value.
-// H0: true breach probability = p0 (e.g. 0.01 for 99% VaR).
-// Returns 1.0 for edge cases (n=0, p_hat=p0).
-func kupiecPOF(n, n1 int, p0 float64) float64 {
-	if n == 0 {
-		return 1.0
-	}
-	n0 := n - n1
-	phat := float64(n1) / float64(n)
-
-	var lr float64
-	switch n1 {
-	case 0:
-		// 0 × ln(0/p0) = 0 by convention; only the n0 term contributes
-		lr = 2 * float64(n0) * math.Log((1-phat)/(1-p0))
-	case n:
-		// n0 term vanishes
-		lr = 2 * float64(n1) * math.Log(phat/p0)
-	default:
-		lr = 2 * (float64(n1)*math.Log(phat/p0) + float64(n0)*math.Log((1-phat)/(1-p0)))
-	}
-
-	if lr <= 0 {
-		return 1.0
-	}
-	// P(chi²(1) > LR) = erfc(sqrt(LR/2))  [chi²(1) = Z², Z~N(0,1)]
-	return math.Erfc(math.Sqrt(lr / 2))
-}
-
-// LinearProbePredict fits ridge regression on (trainEmb, trainY) and returns
-// predictions for testEmb. Complements LinearProbeTrainTest when the caller
-// needs the raw predictions (e.g. to compute VaR breach rate).
-// Returns nil when trainEmb is empty.
-func LinearProbePredict(trainEmb [][]float64, trainY []float64,
-	testEmb [][]float64, lambda float64,
-) []float64 {
-	n := len(trainEmb)
-	if n == 0 || len(testEmb) == 0 {
-		return nil
-	}
-	d := len(trainEmb[0])
-	p := d + 1
-
-	A := make([][]float64, n)
-	for i, e := range trainEmb {
-		row := make([]float64, p)
-		copy(row, e)
-		row[d] = 1.0
-		A[i] = row
-	}
-	AtA := make([][]float64, p)
-	for i := range AtA {
-		AtA[i] = make([]float64, p)
-	}
-	Aty := make([]float64, p)
-	for i := 0; i < n; i++ {
-		for j := 0; j < p; j++ {
-			Aty[j] += A[i][j] * trainY[i]
-			for k := 0; k < p; k++ {
-				AtA[j][k] += A[i][j] * A[i][k]
-			}
-		}
-	}
-	for j := 0; j < p; j++ {
-		AtA[j][j] += lambda
-	}
-	w := solveCholesky(AtA, Aty)
-
-	preds := make([]float64, len(testEmb))
-	for i, e := range testEmb {
-		row := make([]float64, p)
-		copy(row, e)
-		row[d] = 1.0
-		preds[i] = dot(row, w)
-	}
-	return preds
-}

internal/eval/var_test.go

@@ -1,138 +0,0 @@
-package eval_test
-
-import (
-	"math"
-	"testing"
-
-	"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
-)
-
-// ── VaRBreachRate golden tests ──────────────────────────────────────────────
-//
-// VaR_99_t  = predVol[t] × z99   (parametric 99% normal VaR)
-// breach_t  = actualVol[t] > VaR_99_t
-// breachRate = mean(breach_t)
-// kupiecP   = Kupiec POF p-value (chi²(1) test, H0: breach rate = 1%)
-
-func TestVaRBreachRate_ZeroBreaches(t *testing.T) {
-	// 0.02 < 0.01×2.326=0.02326 → no breaches
-	pred := []float64{0.01, 0.01, 0.01}
-	act := []float64{0.02, 0.02, 0.02}
-	rate, _ := eval.VaRBreachRate(pred, act, 2.326)
-	if rate != 0 {
-		t.Fatalf("want rate=0, got %.4f", rate)
-	}
-}
-
-func TestVaRBreachRate_AllBreach(t *testing.T) {
-	// 0.03 > 0.02326 → all breach
-	pred := []float64{0.01, 0.01}
-	act := []float64{0.03, 0.03}
-	rate, _ := eval.VaRBreachRate(pred, act, 2.326)
-	if math.Abs(rate-1.0) > 1e-9 {
-		t.Fatalf("want rate=1.0, got %.4f", rate)
-	}
-}
-
-func TestVaRBreachRate_Golden(t *testing.T) {
-	// n=10, 2 breaches at indices 0 and 2 → rate=0.2
-	// Kupiec: p_hat=0.2 vs p0=0.01 → strongly reject H0 (p < 0.05)
-	pred := make([]float64, 10)
-	act := make([]float64, 10)
-	for i := range pred {
-		pred[i] = 0.01
-		act[i] = 0.01 // no breach: 0.01 < 0.02326
-	}
-	act[0] = 0.03 // breach
-	act[2] = 0.03 // breach
-
-	rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
-
-	if math.Abs(rate-0.2) > 1e-9 {
-		t.Fatalf("breach rate: want 0.2, got %.4f", rate)
-	}
-	if kupiecP > 0.05 {
-		t.Fatalf("kupiec p-value: want <0.05 (strong reject H0), got %.4f", kupiecP)
-	}
-}
-
-func TestVaRBreachRate_PerfectCalibration(t *testing.T) {
-	// n=100, exactly 1 breach → p_hat=0.01=p0 → LR=0 → kupiecP≈1.0
-	n := 100
-	pred := make([]float64, n)
-	act := make([]float64, n)
-	for i := range pred {
-		pred[i] = 0.01
-		act[i] = 0.015 // < 0.02326, no breach
-	}
-	act[0] = 0.025 // > 0.02326, breach
-
-	rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
-
-	if math.Abs(rate-0.01) > 1e-9 {
-		t.Fatalf("breach rate: want 0.01, got %.4f", rate)
-	}
-	if kupiecP < 0.9 {
-		t.Fatalf("kupiec p-value: want ≈1.0 (well calibrated), got %.4f", kupiecP)
-	}
-}
-
-func TestVaRBreachRate_EmptyInput(t *testing.T) {
-	rate, kupiecP := eval.VaRBreachRate(nil, nil, 2.326)
-	if rate != 0 || kupiecP != 1 {
-		t.Fatalf("empty: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
-	}
-}
-
-func TestVaRBreachRate_LenMismatch(t *testing.T) {
-	rate, kupiecP := eval.VaRBreachRate([]float64{0.01}, []float64{0.01, 0.02}, 2.326)
-	if rate != 0 || kupiecP != 1 {
-		t.Fatalf("mismatch: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
-	}
-}
-
-func TestVaRBreachRate_Z99Default(t *testing.T) {
-	// z99=2.326 is the canonical value; test that boundary case works
-	// VaR = 0.01 × 2.326 = 0.02326
-	// actual = 0.02326 → NOT a breach (strict >)
-	pred := []float64{0.01}
-	act := []float64{0.02326}
-	rate, _ := eval.VaRBreachRate(pred, act, 2.326)
-	if rate != 0 {
-		t.Fatalf("boundary: exactly at VaR is not a breach; want rate=0, got %.4f", rate)
-	}
-}
-
-// ── LinearProbePredict ──────────────────────────────────────────────────────
-
-func TestLinearProbePredict_PerfectLinear(t *testing.T) {
-	// y = x; predictions should match targets closely
-	n := 20
-	trainEmb := make([][]float64, n)
-	trainY := make([]float64, n)
-	testEmb := make([][]float64, 5)
-	testY := []float64{5, 10, 15, 20, 25}
-	for i := range trainEmb {
-		trainEmb[i] = []float64{float64(i)}
-		trainY[i] = float64(i)
-	}
-	for i := range testEmb {
-		testEmb[i] = []float64{testY[i]}
-	}
-	preds := eval.LinearProbePredict(trainEmb, trainY, testEmb, 1e-3)
-	if len(preds) != len(testEmb) {
-		t.Fatalf("len: want %d, got %d", len(testEmb), len(preds))
-	}
-	for i, p := range preds {
-		if math.Abs(p-testY[i]) > 1.0 {
-			t.Fatalf("pred[%d]: want ≈%.1f, got %.4f", i, testY[i], p)
-		}
-	}
-}
-
-func TestLinearProbePredict_EmptyTrain(t *testing.T) {
-	preds := eval.LinearProbePredict(nil, nil, [][]float64{{1.0}}, 1e-3)
-	if len(preds) != 0 {
-		t.Fatalf("empty train: want nil/empty preds, got len=%d", len(preds))
-	}
-}

internal/web/index_templ.go

@@ -1,100 +0,0 @@
-// Code generated by templ - DO NOT EDIT.
-
-// templ: version: v0.3.1020
-package web
-
-//lint:file-ignore SA4006 This context is only used if a nested component is present.
-
-import "github.com/a-h/templ"
-import templruntime "github.com/a-h/templ/runtime"
-
-func Index() templ.Component {
-	return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
-		templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
-		if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
-			return templ_7745c5c3_CtxErr
-		}
-		templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
-		if !templ_7745c5c3_IsBuffer {
-			defer func() {
-				templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
-				if templ_7745c5c3_Err == nil {
-					templ_7745c5c3_Err = templ_7745c5c3_BufErr
-				}
-			}()
-		}
-		ctx = templ.InitializeContext(ctx)
-		templ_7745c5c3_Var1 := templ.GetChildren(ctx)
-		if templ_7745c5c3_Var1 == nil {
-			templ_7745c5c3_Var1 = templ.NopComponent
-		}
-		ctx = templ.ClearChildren(ctx)
-		templ_7745c5c3_Var2 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
-			templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
-			templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
-			if !templ_7745c5c3_IsBuffer {
-				defer func() {
-					templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
-					if templ_7745c5c3_Err == nil {
-						templ_7745c5c3_Err = templ_7745c5c3_BufErr
-					}
-				}()
-			}
-			ctx = templ.InitializeContext(ctx)
-			templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<h1 class=\"text-3xl font-semibold mb-6\">hostexecutor</h1><button hx-get=\"/api/hello\" hx-target=\"#out\" class=\"px-4 py-2 bg-slate-900 text-white rounded-md hover:bg-slate-700\">Say hello</button><div id=\"out\" class=\"mt-6 text-slate-700\"></div>")
-			if templ_7745c5c3_Err != nil {
-				return templ_7745c5c3_Err
-			}
-			return nil
-		})
-		templ_7745c5c3_Err = Layout("hostexecutor").Render(templ.WithChildren(ctx, templ_7745c5c3_Var2), templ_7745c5c3_Buffer)
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		return nil
-	})
-}
-
-func Hello(name string) templ.Component {
-	return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
-		templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
-		if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
-			return templ_7745c5c3_CtxErr
-		}
-		templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
-		if !templ_7745c5c3_IsBuffer {
-			defer func() {
-				templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
-				if templ_7745c5c3_Err == nil {
-					templ_7745c5c3_Err = templ_7745c5c3_BufErr
-				}
-			}()
-		}
-		ctx = templ.InitializeContext(ctx)
-		templ_7745c5c3_Var3 := templ.GetChildren(ctx)
-		if templ_7745c5c3_Var3 == nil {
-			templ_7745c5c3_Var3 = templ.NopComponent
-		}
-		ctx = templ.ClearChildren(ctx)
-		templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "<p>Hello, ")
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		var templ_7745c5c3_Var4 string
-		templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(name)
-		if templ_7745c5c3_Err != nil {
-			return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/index.templ`, Line: 15, Col: 17}
-		}
-		_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "!</p>")
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		return nil
-	})
-}
-
-var _ = templruntime.GeneratedTemplate

internal/web/layout_templ.go

@@ -1,61 +0,0 @@
-// Code generated by templ - DO NOT EDIT.
-
-// templ: version: v0.3.1020
-package web
-
-//lint:file-ignore SA4006 This context is only used if a nested component is present.
-
-import "github.com/a-h/templ"
-import templruntime "github.com/a-h/templ/runtime"
-
-func Layout(title string) templ.Component {
-	return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
-		templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
-		if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
-			return templ_7745c5c3_CtxErr
-		}
-		templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
-		if !templ_7745c5c3_IsBuffer {
-			defer func() {
-				templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
-				if templ_7745c5c3_Err == nil {
-					templ_7745c5c3_Err = templ_7745c5c3_BufErr
-				}
-			}()
-		}
-		ctx = templ.InitializeContext(ctx)
-		templ_7745c5c3_Var1 := templ.GetChildren(ctx)
-		if templ_7745c5c3_Var1 == nil {
-			templ_7745c5c3_Var1 = templ.NopComponent
-		}
-		ctx = templ.ClearChildren(ctx)
-		templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<!doctype html><html lang=\"en\"><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><title>")
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		var templ_7745c5c3_Var2 string
-		templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
-		if templ_7745c5c3_Err != nil {
-			return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/layout.templ`, Line: 9, Col: 17}
-		}
-		_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</title><script src=\"https://unpkg.com/htmx.org@2.0.0\"></script><script src=\"https://cdn.tailwindcss.com\"></script></head><body class=\"min-h-screen bg-slate-50 text-slate-900 antialiased\"><main class=\"max-w-3xl mx-auto px-6 py-12\">")
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		templ_7745c5c3_Err = templ_7745c5c3_Var1.Render(ctx, templ_7745c5c3_Buffer)
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "</main></body></html>")
-		if templ_7745c5c3_Err != nil {
-			return templ_7745c5c3_Err
-		}
-		return nil
-	})
-}
-
-var _ = templruntime.GeneratedTemplate

loop.py

@@ -1,287 +0,0 @@
-"""loop.py — Karpathy-style autoresearch loop for jepa-fx-risk.
-
-Agent (on iguana/berget — NOT koala, whose GPU is reserved for train.py) reads
-program.md + train.py + STATUS.md, proposes ONE change to train.py, we run it,
-keep if val_vol_r2 improved else git-revert. Appends per-iter record to STATUS.md.
-
-    LITELLM_KEY=xxx python loop.py [--iters N] [--model MODEL] [--run-dir runs/rq-04]
-
-Env:
-  LITELLM_KEY      — LiteLLM master key (required)
-  LITELLM_BASE     — default http://localhost:30401/v1
-  LOOP_MODEL       — default berget/gemma4-31b (non-thinking; iguana/berget only)
-  LOOP_ITERS       — default 3
-  TRAIN_TIMEOUT    — seconds per train.py run, default 120
-  NTFY_URL         — optional: POST crash/stall alerts here (e.g. ntfy.sh/<topic>)
-"""
-import argparse
-import json
-import os
-import subprocess
-import sys
-import time
-import textwrap
-from pathlib import Path
-
-import urllib.request
-
-LITELLM_BASE  = os.environ.get("LITELLM_BASE", "http://localhost:30401/v1")
-LITELLM_KEY   = os.environ.get("LITELLM_KEY", "")
-LOOP_MODEL    = os.environ.get("LOOP_MODEL", "berget/gemma4-31b")
-LOOP_ITERS    = int(os.environ.get("LOOP_ITERS", "3"))
-TRAIN_TIMEOUT = int(os.environ.get("TRAIN_TIMEOUT", "120"))
-NTFY_URL      = os.environ.get("NTFY_URL", "")
-
-# Resolved by main() once --run-dir is parsed.
-RUN_DIR      = Path(".")
-STATUS_MD    = Path("STATUS.md")
-METRICS_JSON = Path("metrics.json")
-TRAIN_PY     = Path("train.py")
-HEARTBEAT    = Path("HEARTBEAT")
-
-AGENT_SYSTEM = textwrap.dedent("""\
-    You are the autoresearch agent for jepa-fx-risk. Your job: propose ONE small,
-    targeted change to train.py to improve val_vol_r2 (OOS R² predicting 1-day
-    realized vol from frozen embeddings). Higher is better.
-
-    Rules:
-    - Return ONLY the full new content of train.py — nothing else, no explanation,
-      no markdown fence. Raw Python only.
-    - Change ONE thing at a time (one knob, one structural idea).
-    - Do NOT touch prepare_data.py, loop.py, or the data pipeline — only train.py.
-    - Do NOT add new data sources or new files.
-    - The metric is computed externally from your frozen embeddings; trust it.
-""")
-
-
-def read_file(p: Path) -> str:
-    return p.read_text() if p.exists() else ""
-
-
-def gpu_snapshot() -> str:
-    try:
-        out = subprocess.check_output(
-            ["nvidia-smi", "--query-gpu=utilization.gpu,memory.used,memory.total,temperature.gpu",
-             "--format=csv,noheader,nounits"], timeout=5, text=True
-        ).strip()
-        util, mem_used, mem_total, temp = [x.strip() for x in out.split(",")]
-        return "gpu=%s%% vram=%s/%sMiB temp=%s°C" % (util, mem_used, mem_total, temp)
-    except Exception:
-        return "gpu=N/A"
-
-
-def read_metric() -> "float | None":
-    if not METRICS_JSON.exists():
-        return None
-    try:
-        return float(json.loads(METRICS_JSON.read_text())["val_vol_r2"])
-    except Exception:
-        return None
-
-
-def run_train() -> "tuple[float | None, float, str]":
-    """Run train.py from project root with METRICS_OUT pointing into the run dir."""
-    t0 = time.time()
-    env = dict(os.environ)
-    env["METRICS_OUT"] = str(METRICS_JSON.resolve())
-    # train.py is copied into the run dir, so sys.path[0] is that run dir — which
-    # has no scripts/. Put the project root (where loop.py + scripts/ live) on
-    # PYTHONPATH so train.py's `from scripts.var_breach import ...` resolves.
-    env["PYTHONPATH"] = str(Path(__file__).resolve().parent) + os.pathsep + env.get("PYTHONPATH", "")
-    try:
-        r = subprocess.run(
-            [sys.executable, str(TRAIN_PY.resolve())],
-            capture_output=True, text=True, timeout=TRAIN_TIMEOUT, env=env,
-        )
-        elapsed = time.time() - t0
-        if r.returncode != 0:
-            return None, elapsed, (r.stderr or r.stdout)[-300:]
-        metric = read_metric()
-        return metric, elapsed, ""
-    except subprocess.TimeoutExpired:
-        return None, TRAIN_TIMEOUT, "TIMEOUT"
-
-
-def call_agent(iteration: int, best_so_far: "float | None") -> str:
-    """Ask the LLM agent to edit train.py. Returns new train.py content."""
-    context = "\n\n".join([
-        "# program.md\n" + read_file(RUN_DIR / "program.md"),
-        "# train.py (current)\n" + read_file(TRAIN_PY),
-        "# STATUS.md (history)\n" + read_file(STATUS_MD)[-2000:],
-        "# metrics.json (last run)\n" + read_file(METRICS_JSON),
-        "Iteration %d. Best val_vol_r2 so far: %s. Improve it." % (
-            iteration, "%.4f" % best_so_far if best_so_far is not None else "none yet"
-        ),
-    ])
-    payload = json.dumps({
-        "model": LOOP_MODEL,
-        "messages": [
-            {"role": "system", "content": AGENT_SYSTEM},
-            {"role": "user",   "content": context},
-        ],
-        "temperature": 0.7,
-        "max_tokens": 4096,
-    }).encode()
-    req = urllib.request.Request(
-        LITELLM_BASE + "/chat/completions",
-        data=payload,
-        headers={"Authorization": "Bearer " + LITELLM_KEY,
-                 "Content-Type": "application/json"},
-        method="POST",
-    )
-    resp = urllib.request.urlopen(req, timeout=60)
-    data = json.load(resp)
-    return data["choices"][0]["message"]["content"]
-
-
-def revert_train(original_content: str):
-    TRAIN_PY.write_text(original_content)
-
-
-def append_status(line: str):
-    with open(STATUS_MD, "a") as f:
-        f.write(line + "\n")
-
-
-def write_heartbeat(iteration: int, status: str = "alive"):
-    """Update HEARTBEAT so watchdogs can detect stalls."""
-    HEARTBEAT.write_text("%s  iter=%d  ts=%.0f\n" % (status, iteration, time.time()))
-
-
-def ntfy(msg: str):
-    """POST an alert to NTFY_URL (best-effort; silently ignored on any error)."""
-    if not NTFY_URL:
-        return
-    try:
-        req = urllib.request.Request(
-            NTFY_URL, data=msg.encode(), method="POST",
-            headers={"Content-Type": "text/plain"},
-        )
-        urllib.request.urlopen(req, timeout=5)
-    except Exception:
-        pass
-
-
-def main():
-    global RUN_DIR, STATUS_MD, METRICS_JSON, TRAIN_PY, HEARTBEAT
-
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--iters", type=int, default=LOOP_ITERS)
-    parser.add_argument("--model", default=LOOP_MODEL)
-    parser.add_argument(
-        "--run-dir", default=None,
-        help="run dir scaffolded by autoresearch_start.py; "
-             "STATUS.md, metrics.json, HEARTBEAT, and train.py live here",
-    )
-    args = parser.parse_args()
-
-    loop_iters = args.iters
-    loop_model = args.model
-
-    if args.run_dir:
-        RUN_DIR = Path(args.run_dir)
-        if not RUN_DIR.is_dir():
-            print("ERROR: run dir not found:", RUN_DIR); sys.exit(1)
-
-    STATUS_MD    = RUN_DIR / "STATUS.md"
-    METRICS_JSON = RUN_DIR / "metrics.json"
-    TRAIN_PY     = RUN_DIR / "train.py"
-    HEARTBEAT    = RUN_DIR / "HEARTBEAT"
-
-    if not LITELLM_KEY:
-        print("ERROR: set LITELLM_KEY"); sys.exit(1)
-
-    if not STATUS_MD.exists():
-        STATUS_MD.write_text(
-            "# Autoresearch STATUS\n\n"
-            "| iter | val_vol_r2 | delta | action | secs | gpu | change |\n"
-            "|------|-----------|-------|--------|------|-----|--------|\n"
-        )
-
-    baseline = read_metric()
-    if baseline is None:
-        print("No metrics.json — running train.py for baseline...")
-        m, secs, err = run_train()
-        if m is None:
-            msg = "Baseline run failed: " + err
-            print(msg)
-            ntfy("[jepa-fx-risk] loop CRASH — " + msg)
-            sys.exit(1)
-        baseline = m
-        print("Baseline: val_vol_r2 = %.4f (%.1fs)" % (baseline, secs))
-
-    best = baseline
-    print("Starting loop | model=%s | iters=%d | baseline=%.4f" % (loop_model, loop_iters, best))
-    if args.run_dir:
-        print("  run-dir:", RUN_DIR)
-
-    iter_index = 0
-    try:
-        for i in range(1, loop_iters + 1):
-            iter_index = i
-            write_heartbeat(i, "agent-call")
-            print("\n--- iter %d/%d ---" % (i, loop_iters))
-            original = TRAIN_PY.read_text()
-
-            print("  calling agent (%s)..." % loop_model)
-            t_agent = time.time()
-            try:
-                new_code = call_agent(i, best)
-            except Exception as e:
-                msg = str(e)
-                print("  agent call failed:", msg)
-                append_status("| %d | ERR | — | agent-fail | — | — | %s |" % (i, msg[:60]))
-                write_heartbeat(i, "agent-fail")
-                ntfy("[jepa-fx-risk] iter %d agent FAIL — %s" % (i, msg[:80]))
-                continue
-            agent_secs = time.time() - t_agent
-            print("  agent replied in %.1fs" % agent_secs)
-
-            # strip accidental markdown fences
-            if new_code.strip().startswith("```"):
-                lines = new_code.strip().splitlines()
-                new_code = "\n".join(lines[1:-1] if lines[-1].strip() == "```" else lines[1:])
-
-            TRAIN_PY.write_text(new_code)
-
-            write_heartbeat(i, "training")
-            gpu = gpu_snapshot()
-            print("  running train.py [%s]..." % gpu)
-            metric, secs, err = run_train()
-
-            if metric is None:
-                print("  train.py FAILED — reverting. err:", err[:100])
-                revert_train(original)
-                append_status("| %d | FAIL | — | revert | %.0fs | %s | run error |" % (i, secs, gpu))
-                write_heartbeat(i, "train-fail")
-                ntfy("[jepa-fx-risk] iter %d train FAIL — %s" % (i, err[:80]))
-                continue
-
-            delta = metric - best
-            if metric > best:
-                best = metric
-                action = "KEEP"
-            else:
-                revert_train(original)
-                action = "revert"
-
-            summary = "| %d | %.4f | %+.4f | %s | %.0fs | %s | iter%d |" % (
-                i, metric, delta, action, secs, gpu, i)
-            append_status(summary)
-            write_heartbeat(i, "done")
-            print("  val_vol_r2=%.4f  delta=%+.4f  action=%s  [%.0fs]" % (metric, delta, action, secs))
-
-    except Exception as e:
-        msg = "loop CRASH at iter %d: %s" % (iter_index, e)
-        print("FATAL:", msg)
-        ntfy("[jepa-fx-risk] " + msg)
-        raise
-
-    print("\nDone. Best val_vol_r2 = %.4f (baseline was %.4f, delta %+.4f)" % (best, baseline, best - baseline))
-    print("STATUS.md updated.")
-    write_heartbeat(loop_iters, "done")
-    ntfy("[jepa-fx-risk] loop done. best val_vol_r2=%.4f (delta %+.4f)" % (best, best - baseline))
-
-
-if __name__ == "__main__":
-    main()

model/README.md

@@ -0,0 +1,31 @@
+# model
+
+Python + PyTorch perimeter. This is the only directory in the project that uses Python.
+
+## Contents
+
+```
+model/
+  train.py              TS-JEPA training entry point
+  configs/              YAML configs per phase
+    phase0-mae.yaml
+    phase1-tsjepa.yaml
+  tsjepa/               TS-JEPA model implementation (adapted from paper)
+  tests/                pytest tests for model components
+  requirements.txt      pinned Python dependencies
+  .venv/                (gitignored — created by `task model:setup`)
+```
+
+## Setup
+
+```bash
+task model:setup    # creates .venv and installs requirements via uv
+```
+
+## Dependency policy
+
+Every Python dependency must be justified in a comment in `requirements.txt`. Prefer Go implementations for anything outside the training loop. When adding a new dependency, add an entry to DECISIONS.md explaining why a Go alternative wasn't sufficient.
+
+## Python version
+
+3.12 (pinned in `.python-version`)

notebooks/README.md

@@ -0,0 +1,11 @@
+# Notebooks
+
+Exploratory Data Analysis (EDA) scratch space only.
+
+## Rules
+
+- **Never use notebooks for anything reproducible.** Training, evaluation, and metric computation belong in `src/` with tests.
+- **Strip all outputs before committing.** Use `nbstripout` or equivalent.
+- **Label every notebook with a phase prefix:** `phase0-eda-eurusd-distribution.ipynb`
+
+Notebooks are thinking tools, not research artifacts. If a finding from a notebook is worth keeping, it goes into a spec, a decision in `DECISIONS.md`, or a result in `results/summaries/` — not the notebook itself.

requirements.txt

@@ -1,10 +0,0 @@
-# Python deps for the autoresearch loop (train.py + scripts). Install torch from
-# the cu130 index FIRST (koala Blackwell sm_120, torch 2.12.1+cu130 verified):
-#   pip install torch --index-url https://download.pytorch.org/whl/cu130
-#   pip install -r requirements.txt
-numpy>=2.0
-pandas>=2.2
-pyarrow>=16
-histdata>=1.1        # histdata.com downloader (1.1 is newest on PyPI; 1.3 never existed)
-hmmlearn>=0.3        # regime detector (prepare_regime.py, jepa-fx-risk#13)
-scikit-learn>=1.4    # HMM dependency

results/README.md

@@ -0,0 +1,32 @@
+# Results
+
+Tracked outputs from experiment runs. Large raw outputs are gitignored — only summaries are committed.
+
+## Structure
+
+```
+results/
+  summaries/    metric tables, key figures, per-phase result records (committed)
+  raw/          full embedding outputs, backtest CSVs (gitignored)
+```
+
+## Per-phase result record format
+
+Each concluded phase produces a result record in `summaries/`:
+
+```
+summaries/
+  phase-0-[pass|null].md
+  phase-1-[pass|null].md
+  ...
+```
+
+Each record must include:
+- Phase name and hypothesis
+- Key metrics (silhouette score, R², etc.) with confidence intervals where applicable
+- Baseline comparison
+- Verdict: PASS / NULL RESULT
+- If null: what was investigated, what was found, next step taken
+- Link to experiment git tag
+
+**Null results are valid research outputs and must be committed, not discarded.**

results/hpo/hpo_results.jsonl

@@ -1,18 +0,0 @@
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.302411480667525, "phase1_r2": 0.35809940099716187, "stdout_last": "val_vol_r2 = 0.3024  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:37.028406"}
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29654798431244755, "phase1_r2": 0.35618388652801514, "stdout_last": "val_vol_r2 = 0.2965  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:49.822762"}
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.31050360040290237, "phase1_r2": 0.36530405282974243, "stdout_last": "val_vol_r2 = 0.3105  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:02.966176"}
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.2925057399716364, "phase1_r2": 0.3467639684677124, "stdout_last": "val_vol_r2 = 0.2925  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:16.195552"}
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29334667623516786, "phase1_r2": 0.35872191190719604, "stdout_last": "val_vol_r2 = 0.2933  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:31.372602"}
-{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3123527205416422, "phase1_r2": 0.3572431206703186, "stdout_last": "val_vol_r2 = 0.3124  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:46.672203"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3641397896593044, "phase1_r2": 0.3908407688140869, "stdout_last": "val_vol_r2 = 0.3641  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:00.793878"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.35845865364171503, "phase1_r2": 0.3737195134162903, "stdout_last": "val_vol_r2 = 0.3585  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:13.321428"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.35310115657814645, "phase1_r2": 0.35306859016418457, "stdout_last": "val_vol_r2 = 0.3531  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:26.402249"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3655629727960601, "phase1_r2": 0.371029257774353, "stdout_last": "val_vol_r2 = 0.3656  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:39.786248"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.36109622605593217, "phase1_r2": 0.3666273355484009, "stdout_last": "val_vol_r2 = 0.3611  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:53.194111"}
-{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.362228341965093, "phase1_r2": 0.3590735197067261, "stdout_last": "val_vol_r2 = 0.3622  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:06.991680"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3749483295047378, "phase1_r2": 0.3801569938659668, "stdout_last": "val_vol_r2 = 0.3749  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:21.512210"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.3765593861479334, "phase1_r2": 0.38416117429733276, "stdout_last": "val_vol_r2 = 0.3766  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:34.959919"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3653399117639956, "phase1_r2": 0.3685130476951599, "stdout_last": "val_vol_r2 = 0.3653  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:48.806135"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.375961424966925, "phase1_r2": 0.37861257791519165, "stdout_last": "val_vol_r2 = 0.3760  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:04.056939"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.37841726893098504, "phase1_r2": 0.3781360387802124, "stdout_last": "val_vol_r2 = 0.3784  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:18.693263"}
-{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.37118530199441635, "phase1_r2": 0.3651617765426636, "stdout_last": "val_vol_r2 = 0.3712  (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:34.796157"}

results/summaries/phase-0-metrics.json

@@ -1,13 +0,0 @@
-{
-  "label": "null",
-  "mean_sil": 0.018206419112781685,
-  "pca_sil": 0.13589094579219818,
-  "spread": 0.8673340065023978,
-  "pc1_hv_corr": 0.525803392278542,
-  "per_seed": [
-    0.026790648698806763,
-    0.010999602265655994,
-    0.016829006373882294
-  ],
-  "passed": false
-}

results/summaries/phase-0-null.md

@@ -1,25 +0,0 @@
-# Phase-0 SSL feasibility gate — null
-
-**Date:** 2026-06-24
-**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
-(not Go #4); gate metric adapted from silhouette-on-embedding to match
-available data. Go harness (#4) remains open for production experiments.
-
-## Data
-- Train: EUR/USD daily 2019-2021 (907 windows)
-- OOS: EUR/USD daily 2022-2023 (593 windows)
-- HV label: top-33% realized-vol days = high-volatility (196 days)
-
-## Results
-| | Value | Gate |
-|---|---|---|
-| TS-JEPA mean silhouette (3 seeds) | 0.0182 | > 0.20 → **False** |
-| Beats PCA baseline (0.1359) | 0.0182 | > PCA → **False** |
-| Seed stability (spread) | 86.73% | < 10% → **False** |
-| PC1/HV correlation | 0.5258 | < 0.95 → **True** |
-
-Per-seed: ['0.0268', '0.0110', '0.0168']
-
-## Verdict: **NULL**
-
-One or more gate criteria not met. See null result protocol in #5.

scripts/autoresearch_start.py

@@ -1,155 +0,0 @@
-"""autoresearch start — scaffold a run dir from an Autoresearch Council backlog leaf.
-
-Usage:
-    python scripts/autoresearch_start.py <backlog.json> <rq-id>
-
-Reads the Council backlog JSON (from agentsquad autoresearch_pipe.py Stage-3 output),
-finds the node by rq-id, validates it is autoresearch-ready (fail-closed), then
-scaffolds runs/<rq-id>/ with:
-
-  program.md  — hypothesis, single metric (stripped), agent search-space seam
-  run.json    — provenance (strategic_question + council_node) + config
-  train.py    — copy of project train.py (the loop edits this, keeps history clean)
-
-Launch:
-  LITELLM_KEY=xxx python loop.py --run-dir runs/<rq-id>
-
-Refs: jepa-fx-risk#11, agentsquad#44
-"""
-
-import json
-import shutil
-import sys
-from datetime import datetime, timezone
-from pathlib import Path
-
-
-def load_backlog(path: str) -> dict:
-    try:
-        with open(path) as f:
-            return json.load(f)
-    except FileNotFoundError:
-        print(f"error: backlog file not found: {path}", file=sys.stderr)
-        raise
-
-
-def scaffold_run(
-    backlog_path_or_dict,
-    rq_id: str,
-    run_dir: Path,
-    train_py_src: Path,
-) -> None:
-    """Scaffold a run dir. Raises SystemExit on any validation failure."""
-    if isinstance(backlog_path_or_dict, (str, Path)):
-        backlog = load_backlog(str(backlog_path_or_dict))
-    else:
-        backlog = backlog_path_or_dict
-
-    # Find node
-    nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
-    if rq_id not in nodes_by_id:
-        print(f"error: rq-id {rq_id!r} not found in backlog", file=sys.stderr)
-        sys.exit(1)
-    node = nodes_by_id[rq_id]
-
-    # Fail-closed: only autoresearch-ready nodes may be scaffolded
-    status = node.get("status", "")
-    if status != "autoresearch-ready":
-        print(
-            f"error: {rq_id} has status {status!r}, not 'autoresearch-ready' — refusing to scaffold",
-            file=sys.stderr,
-        )
-        sys.exit(1)
-
-    # Guard against overwriting an existing run
-    if run_dir.exists():
-        print(
-            f"error: {run_dir} already exists — remove it first to re-scaffold",
-            file=sys.stderr,
-        )
-        sys.exit(1)
-
-    metric = (node.get("candidate_metric") or "").strip()
-    strategic_q = backlog.get("strategic_question", "")
-    council_node = node["id"]
-    generated_at = datetime.now(timezone.utc).isoformat()
-
-    run_dir.mkdir(parents=True)
-
-    # --- program.md ---
-    program_md = f"""# program.md — {council_node}: {node.get("question", "")[:80]}
-
-## Provenance
-- strategic_question: {json.dumps(strategic_q)}
-- council_node: {council_node} (autoresearch-ready; Autoresearch Council backlog)
-- generated_at: {generated_at}
-
-## Hypothesis
-{node.get("question", "")}
-
-## Single validation metric (optimise this, nothing else)
-`{metric}` — see eval harness for the exact definition. Only this scalar drives
-keep/revert decisions. Report alongside but do NOT optimise:
-  - Kupiec POF p-value (calibration sanity)
-  - val_vol_r2 (representation quality guard)
-
-## What the agent MAY modify (the search space)
-- Hyperparameters in train.py (model size, LR, window, patch_len, epochs, etc.)
-- Conditioning mechanisms (e.g. JEPA_ENABLE_REGIME toggle)
-- Loss function weights and architecture depth
-
-## Frozen (do NOT touch — keeps the ablation clean)
-- Data pipeline and splits (train ≤2021, OOS ≥2022, test 2024 held out)
-- The metric definition and scoring code
-- loop.py, scripts/, tests/
-
-## Experiment loop (per Karpathy autoresearch)
-Each iter (≤ time-box): apply ONE change to train.py → run → read
-`{metric}` → keep if improved (and Kupiec p-value did not collapse), else revert.
-Stop on: target reached, max iters, or K consecutive iters with no improvement.
-"""
-    (run_dir / "program.md").write_text(program_md)
-
-    # --- run.json (provenance + config) ---
-    run_meta = {
-        "strategic_question": strategic_q,
-        "council_node": council_node,
-        "metric": metric,
-        "generated_at": generated_at,
-        "model_tier": "homelab",
-        "max_iters": 10,
-        "time_box_minutes": 5,
-    }
-    (run_dir / "run.json").write_text(json.dumps(run_meta, indent=2) + "\n")
-
-    # --- train.py (loop edits this copy; project root train.py is the template) ---
-    shutil.copy(train_py_src, run_dir / "train.py")
-
-
-def main() -> None:
-    if len(sys.argv) != 3:
-        print("usage: python scripts/autoresearch_start.py <backlog.json> <rq-id>")
-        sys.exit(1)
-
-    backlog_path, rq_id = sys.argv[1], sys.argv[2]
-
-    project_root = Path(__file__).parent.parent
-    run_dir = project_root / "runs" / rq_id
-    train_py_src = project_root / "train.py"
-
-    scaffold_run(backlog_path, rq_id, run_dir, train_py_src)
-
-    backlog = load_backlog(backlog_path)
-    nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
-    metric = (nodes_by_id[rq_id].get("candidate_metric") or "").strip()
-
-    print(f"✓ scaffolded {run_dir}")
-    print(f"  node:   {rq_id}")
-    print(f"  metric: {metric}")
-    print()
-    print("launch:")
-    print(f"  LITELLM_KEY=xxx python loop.py --run-dir runs/{rq_id}")
-
-
-if __name__ == "__main__":
-    main()

scripts/check_gpu.py

@@ -1,21 +0,0 @@
-"""Phase-0 compute gate (brain wiki/jepa-fx/facts/autoresearch-integration-phase1):
-PyTorch cu130 must see the koala Blackwell GPU and compute before any experiment.
-
-    python scripts/check_gpu.py   # exits 0 if the GPU is usable, 1 otherwise
-
-Note: koala shares this 12GB card with the llama-swap LLM stack. The autoresearch
-agent should run on iguana/berget models so koala's GPU stays free for train.py.
-"""
-import sys
-import torch
-
-print("torch", torch.__version__)
-if not torch.cuda.is_available():
-    print("CUDA NOT AVAILABLE — gate BLOCKED")
-    sys.exit(1)
-print("device:", torch.cuda.get_device_name(0))
-print("capability: sm_%d%d" % torch.cuda.get_device_capability(0))
-x = torch.randn(2000, 2000, device="cuda")
-(x @ x).sum().item()
-torch.cuda.synchronize()
-print("GPU matmul OK — Phase-0 compute gate GREEN")

scripts/context-sync.sh

@@ -5,11 +5,6 @@
 # For Claude Code: generates project-only CLAUDE.md (it inherits root via tree walk)
 # For everything else: concatenates root AGENT.md + project PROJECT.md
 #
-# Cursor and Aider adapters removed 2026-09-01 — neither is a supported harness
-# any more. Supported: Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush,
-# Pi. All read CLAUDE.md or AGENTS.md; system-prompt.txt remains for Open WebUI,
-# Mods and raw API use.
-#
 # Usage: ./scripts/context-sync.sh [--force] [adapter...]
 # Task:  task context:sync
 #
@@ -58,7 +53,7 @@ if [ "$FORCE" = false ]; then
     while IFS= read -r _match; do
       _lineno="${_match%%:*}"
       _content="${_match#*:}"
-      _token=$(head -1 < <(grep -o '{{[^}]*}}' <<<"$_content"))
+      _token=$(printf '%s' "$_content" | grep -o '{{[^}]*}}' | head -1)
       echo "  $PROJECT_FILE:$_lineno: unfilled placeholder $_token" >&2
     done <<< "$_placeholders"
     echo "" >&2
@@ -100,6 +95,30 @@ generate_agents() {
   echo "  → AGENTS.md (root + project; Crush, Pi, Antigravity)"
 }
 
+# ── Cursor ───────────────────────────────────────────────────
+generate_cursor() {
+  {
+    echo "# Cursor rules — auto-generated"
+    echo "# Do not edit. Run: task context:sync"
+    echo ""
+    root_block
+    cat "$PROJECT_FILE"
+  } > .cursorrules
+  echo "  → .cursorrules (root + project)"
+}
+
+# ── Aider ────────────────────────────────────────────────────
+generate_aider() {
+  { root_block; cat "$PROJECT_FILE"; } > .aider.conventions.md
+  if [ ! -f .aider.conf.yml ]; then
+    cat > .aider.conf.yml << 'YAML'
+read: .aider.conventions.md
+auto-commits: false
+YAML
+  fi
+  echo "  → .aider.conventions.md (root + project)"
+}
+
 # ── Generic system prompt (Open WebUI, Mods, etc.) ──────────
 generate_system_prompt() {
   {
@@ -161,6 +180,8 @@ echo "Syncing project context from $PROJECT_FILE..."
 if [ ${#ADAPTERS[@]} -eq 0 ]; then
   generate_claude
   generate_agents
+  generate_cursor
+  generate_aider
   generate_system_prompt
   generate_mcp
 else
@@ -168,6 +189,8 @@ else
     case "$adapter" in
       claude)  generate_claude ;;
       agents)  generate_agents ;;
+      cursor)  generate_cursor ;;
+      aider)   generate_aider ;;
       prompt|system|openwebui|owui|generic) generate_system_prompt ;;
       mcp)     generate_mcp ;;
       *) echo "Unknown adapter: $adapter" >&2; exit 1 ;;

scripts/fetch_data.py

@@ -1,31 +0,0 @@
-"""Fetch EUR/USD M1 bars from histdata.com (free, research use).
-
-Polite: one request per year, spaced; past years query month=None. Uses the
-maintained `histdata` package which handles histdata's anti-hotlink tk token.
-Output: data/raw/DAT_ASCII_EURUSD_M1_<year>.zip
-
-    YEARS=2019,2020,2021 python scripts/fetch_data.py
-"""
-import os
-import time
-
-from histdata import download_hist_data
-from histdata.api import Platform as P, TimeFrame as T
-
-YEARS = [y.strip() for y in os.environ.get("YEARS", "2019,2020,2021").split(",")]
-
-
-def main():
-    os.makedirs("data/raw", exist_ok=True)
-    for yr in YEARS:
-        f = download_hist_data(
-            year=yr, month=None, pair="eurusd",
-            platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
-            output_directory="data/raw",
-        )
-        print("fetched", yr, "->", f)
-        time.sleep(2)  # be a good citizen
-
-
-if __name__ == "__main__":
-    main()

scripts/fetch_multipair.py

@@ -1,48 +0,0 @@
-"""Fetch G10 FX M1 data from histdata.com for all pairs except EURUSD (already fetched).
-
-Each pair's zips go into data/raw/{pair}/ to avoid collisions.
-Output: data/raw/gbpusd/DAT_ASCII_GBPUSD_M1_YYYY.zip etc.
-
-    python scripts/fetch_multipair.py
-    PAIRS=gbpusd,usdjpy YEARS=2020,2021 python scripts/fetch_multipair.py
-"""
-import os
-import time
-
-from histdata import download_hist_data
-from histdata.api import Platform as P, TimeFrame as T
-
-PAIRS_DEFAULT = ["gbpusd", "usdjpy", "usdchf", "audusd"]
-YEARS_DEFAULT = list(range(2008, 2024))
-
-
-def main():
-    pairs_env = os.environ.get("PAIRS", "")
-    pairs = [p.strip() for p in pairs_env.split(",")] if pairs_env else PAIRS_DEFAULT
-
-    years_env = os.environ.get("YEARS", "")
-    years = [int(y.strip()) for y in years_env.split(",")] if years_env else YEARS_DEFAULT
-
-    for pair in pairs:
-        out_dir = f"data/raw/{pair}"
-        os.makedirs(out_dir, exist_ok=True)
-        print(f"\n=== {pair.upper()} ===")
-        for yr in years:
-            out_path = os.path.join(out_dir, f"DAT_ASCII_{pair.upper()}_M1_{yr}.zip")
-            if os.path.exists(out_path):
-                print(f"  {yr} already present, skip")
-                continue
-            try:
-                f = download_hist_data(
-                    year=str(yr), month=None, pair=pair,
-                    platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
-                    output_directory=out_dir,
-                )
-                print(f"  fetched {yr} → {f}")
-            except Exception as e:
-                print(f"  {yr} FAILED: {e}")
-            time.sleep(2)
-
-
-if __name__ == "__main__":
-    main()

scripts/hpo_sweep.py

@@ -1,97 +0,0 @@
-"""HPO sweep for jepa-fx-risk HEPA backbone.
-
-Runs train.py with different JEPA_* env overrides, logs results to
-results/hpo/hpo_results.jsonl. Each config writes its metrics.json then
-the result is appended to the JSONL.
-
-Usage:
-    python scripts/hpo_sweep.py
-    python scripts/hpo_sweep.py --dry-run   # print configs, don't train
-"""
-import argparse
-import json
-import os
-import subprocess
-import sys
-from datetime import datetime
-from itertools import product
-from pathlib import Path
-
-# ── Search space ──────────────────────────────────────────────────────────────
-SEARCH_SPACE = {
-    "JEPA_D_MODEL": [64, 128, 256],
-    "JEPA_DEPTH":   [2, 4],
-    "JEPA_WINDOW":  [120, 240, 480],
-}
-# Fixed: PATCH_LEN=24 (1-day patches), N_HEADS=4, EPOCHS=300, PHASE1_EPOCHS=200
-
-PYTHON = str(Path(sys.executable))
-OUT_DIR = Path("results/hpo")
-
-
-def configs():
-    """Yield all configs as dicts of JEPA_* env overrides."""
-    keys = list(SEARCH_SPACE.keys())
-    for vals in product(*SEARCH_SPACE.values()):
-        yield dict(zip(keys, vals))
-
-
-def run_config(cfg: dict, metrics_path: str = "metrics.json") -> dict:
-    env = {**os.environ, **{k: str(v) for k, v in cfg.items()}}
-    result = subprocess.run(
-        [PYTHON, "train.py"],
-        env=env,
-        capture_output=True,
-        text=True,
-    )
-    if result.returncode != 0:
-        return {"config": cfg, "error": result.stderr[-500:]}
-    stdout_last = result.stdout.strip().split("\n")[-1]
-    with open(metrics_path) as f:
-        m = json.load(f)
-    return {
-        "config": cfg,
-        "val_vol_r2":  m.get("val_vol_r2"),
-        "phase1_r2":   m.get("phase1_r2"),
-        "stdout_last": stdout_last,
-    }
-
-
-def main():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--dry-run", action="store_true")
-    args = parser.parse_args()
-
-    OUT_DIR.mkdir(parents=True, exist_ok=True)
-    out_file = OUT_DIR / "hpo_results.jsonl"
-
-    all_cfgs = list(configs())
-    print(f"HPO sweep: {len(all_cfgs)} configs")
-    for i, cfg in enumerate(all_cfgs):
-        label = " ".join(f"{k.replace('JEPA_','')}={v}" for k, v in cfg.items())
-        print(f"\n[{i+1}/{len(all_cfgs)}] {label}")
-        if args.dry_run:
-            continue
-        ts = datetime.utcnow().isoformat()
-        row = run_config(cfg)
-        row["ts"] = ts
-        with open(out_file, "a") as f:
-            f.write(json.dumps(row) + "\n")
-        if "error" in row:
-            print(f"  ERROR: {row['error'][:200]}")
-        else:
-            print(f"  val_vol_r2={row['val_vol_r2']:.4f}  phase1_r2={row['phase1_r2']:.4f}")
-
-    if not args.dry_run:
-        # Print leaderboard
-        rows = [json.loads(l) for l in open(out_file) if l.strip()]
-        rows = [r for r in rows if "error" not in r]
-        rows.sort(key=lambda r: r.get("phase1_r2", -999), reverse=True)
-        print("\n── Leaderboard (by phase1_r2) ─────────────────────────")
-        for r in rows[:5]:
-            cfg_str = " ".join(f"{k.replace('JEPA_','')}={v}" for k,v in r["config"].items())
-            print(f"  {r['phase1_r2']:.4f}  {cfg_str}")
-
-
-if __name__ == "__main__":
-    main()

scripts/phase0_gate.py

@@ -1,237 +0,0 @@
-"""Phase-0 SSL feasibility gate (path B — Python fast-close of #5).
-
-Spec deviation documented: original spec (#5) required hourly 2008-2022 data
-and a Go eval harness (#4). Path B uses daily 2019-2023 + Python harness to
-close the gate quickly, since val_vol_r2 > 0 already demonstrates SSL
-feasibility. The Go harness (#4) remains open for production experiments.
-
-Gate criteria (from #5):
-  - Silhouette > 0.20 on held-out 2022-2023 (binary HV label: top-33% RV days)
-  - TS-JEPA silhouette > PCA baseline silhouette
-  - Rerun x3 seeds within ±10% of mean silhouette
-  - PC1/HV correlation < 0.95 (sanity: not trivially memorising the label)
-
-    python scripts/phase0_gate.py
-"""
-import json
-import math
-import os
-import numpy as np
-import pandas as pd
-import torch
-import torch.nn as nn
-from sklearn.decomposition import PCA
-from sklearn.metrics import silhouette_score
-from sklearn.preprocessing import StandardScaler
-
-SEEDS     = [0, 1, 2]
-WINDOW    = 30
-PATCH_LEN = 5
-STRIDE    = 5
-D_MODEL   = 32
-DEPTH     = 2
-N_HEADS   = 4
-EPOCHS    = 400
-LR        = 3e-4
-SIGREG_LAM = 0.5
-HV_PERCENTILE = 67   # top-33% = "high volatility"
-
-dev = "cuda" if torch.cuda.is_available() else "cpu"
-
-
-# ── SIGReg ──────────────────────────────────────────────────────────────────
-
-def sigreg(tokens: torch.Tensor, knots: int = 17) -> torch.Tensor:
-    B, T, D = tokens.shape
-    z = tokens.reshape(B * T, D).float()
-    t = torch.linspace(0, 3, knots, device=z.device, dtype=z.dtype)
-    dt = 3.0 / (knots - 1)
-    w = torch.full((knots,), 2 * dt, device=z.device, dtype=z.dtype)
-    w[0] = dt; w[-1] = dt
-    phi = torch.exp(-t.square() / 2.0)
-    A = torch.randn(D, 256, device=z.device, dtype=z.dtype)
-    A = A / A.norm(p=2, dim=0)
-    x_t = (z @ A).unsqueeze(-1) * t
-    err = (x_t.cos().mean(0) - phi).square() + x_t.sin().mean(0).square()
-    return ((err @ (w * phi)) * z.shape[0]).mean()
-
-
-# ── Encoder ──────────────────────────────────────────────────────────────────
-
-class PatchEncoder(nn.Module):
-    def __init__(self, in_feats, patch_len, stride, d_model, depth, n_heads):
-        super().__init__()
-        self.patch_len = patch_len
-        self.stride    = stride
-        self.embed = nn.Linear(patch_len * in_feats, d_model)
-        layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
-                                           dropout=0.0, batch_first=True)
-        self.tf = nn.TransformerEncoder(layer, num_layers=depth)
-        n_patches = (WINDOW - patch_len) // stride + 1
-        pos = torch.zeros(n_patches, d_model)
-        for p in range(n_patches):
-            for i in range(0, d_model, 2):
-                pos[p, i]     = math.sin(p / 10000 ** (i / d_model))
-                if i + 1 < d_model:
-                    pos[p, i+1] = math.cos(p / 10000 ** (i / d_model))
-        self.register_buffer("pos", pos)
-
-    def forward(self, x):
-        B, W, F = x.shape
-        n_patches = (W - self.patch_len) // self.stride + 1
-        patches = torch.stack([x[:, i*self.stride:i*self.stride+self.patch_len, :]
-                                .reshape(B, -1) for i in range(n_patches)], dim=1)
-        tokens = self.embed(patches) + self.pos[:n_patches]
-        return self.tf(tokens)
-
-
-# ── Data ─────────────────────────────────────────────────────────────────────
-
-def load_data():
-    df = pd.read_parquet("data/processed/eurusd_daily.parquet").reset_index(drop=True)
-    df["date"] = pd.to_datetime(df["date"])
-    train = df[df["date"].dt.year <= 2021].copy()
-    oos   = df[df["date"].dt.year >= 2022].copy()
-
-    feats_all  = df[["ret", "realized_vol"]].to_numpy(np.float32)
-    target_all = df["realized_vol"].to_numpy(np.float32)
-    dates_all  = df["date"].values
-
-    mu = feats_all[:len(train)].mean(0)
-    sd = feats_all[:len(train)].std(0) + 1e-8
-
-    def windows(df_subset, feats_norm, dates):
-        idx_start = df.index[df["date"].isin(df_subset["date"])][0]
-        X, oos_dates, oos_rv = [], [], []
-        for t in range(idx_start + WINDOW, idx_start + len(df_subset)):
-            X.append(feats_norm[t - WINDOW:t])
-            oos_dates.append(dates[t])
-            oos_rv.append(target_all[t])
-        return np.stack(X), np.array(oos_rv), np.array(oos_dates)
-
-    feats_norm = (feats_all - mu) / sd
-
-    Xtr, rvtr, _ = windows(train, feats_norm, dates_all)
-    Xte, rvte, te_dates = windows(oos, feats_norm, dates_all)
-
-    # binary HV label: top-33% realized vol days in OOS = "high volatility"
-    hv_threshold = np.percentile(rvte, HV_PERCENTILE)
-    hv_labels = (rvte >= hv_threshold).astype(int)
-
-    return Xtr, rvtr, Xte, rvte, hv_labels
-
-
-# ── Train + embed ─────────────────────────────────────────────────────────────
-
-def train_and_embed(Xtr, Xte, seed):
-    torch.manual_seed(seed)
-    np.random.seed(seed)
-    enc  = PatchEncoder(Xtr.shape[2], PATCH_LEN, STRIDE, D_MODEL, DEPTH, N_HEADS).to(dev)
-    pred = nn.Sequential(nn.Linear(D_MODEL, D_MODEL), nn.GELU(),
-                         nn.Linear(D_MODEL, D_MODEL)).to(dev)
-    opt  = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
-
-    Xtr_t = torch.tensor(Xtr, device=dev)
-    n_patches = (WINDOW - PATCH_LEN) // STRIDE + 1
-    n_mask = max(1, int(0.30 * n_patches))
-
-    for ep in range(EPOCHS):
-        idx_mask = torch.randperm(n_patches)[:n_mask]
-        tokens_ctx    = enc(Xtr_t)
-        tokens_target = enc(Xtr_t).detach()
-        jepa_loss = ((pred(tokens_ctx[:, idx_mask, :]) -
-                      tokens_target[:, idx_mask, :]) ** 2).mean()
-        reg = sigreg(tokens_ctx)
-        loss = jepa_loss + SIGREG_LAM * reg
-        opt.zero_grad(); loss.backward(); opt.step()
-
-    enc.eval()
-    with torch.no_grad():
-        Ete = enc(torch.tensor(Xte, device=dev)).mean(1).cpu().numpy()
-
-    return Ete
-
-
-# ── Gate ─────────────────────────────────────────────────────────────────────
-
-def pca_baseline(Xte, hv_labels):
-    flat = Xte.reshape(len(Xte), -1)
-    sc = StandardScaler().fit(flat)
-    emb = PCA(n_components=8).fit_transform(sc.transform(flat))
-    return silhouette_score(emb, hv_labels), emb
-
-
-def main():
-    os.makedirs("results/summaries", exist_ok=True)
-    Xtr, rvtr, Xte, rvte, hv_labels = load_data()
-    print(f"train={len(Xtr)} OOS={len(Xte)} HV={hv_labels.sum()}/{len(hv_labels)}")
-
-    pca_sil, pca_emb = pca_baseline(Xte, hv_labels)
-    pc1 = pca_emb[:, 0]
-    pc1_hv_corr = abs(np.corrcoef(pc1, hv_labels)[0, 1])
-    print(f"PCA baseline silhouette = {pca_sil:.4f} | PC1/HV |r| = {pc1_hv_corr:.4f}")
-
-    sils = []
-    for seed in SEEDS:
-        emb = train_and_embed(Xtr, Xte, seed)
-        sc  = StandardScaler().fit(emb)
-        sil = silhouette_score(sc.transform(emb), hv_labels)
-        sils.append(sil)
-        print(f"  seed={seed}  silhouette={sil:.4f}")
-
-    mean_sil = np.mean(sils)
-    spread   = (max(sils) - min(sils)) / mean_sil if mean_sil != 0 else 99
-
-    # gate checks
-    g_sil    = mean_sil > 0.20
-    g_beats  = mean_sil > pca_sil
-    g_stable = spread < 0.10
-    g_corr   = pc1_hv_corr < 0.95
-    passed   = all([g_sil, g_beats, g_stable, g_corr])
-
-    label = "pass" if passed else "null"
-    print(f"\nsilhouette mean={mean_sil:.4f} spread={spread:.2%} PCA={pca_sil:.4f} PC1/HV={pc1_hv_corr:.4f}")
-    print(f"gate: sil>0.20={g_sil} beats_pca={g_beats} stable={g_stable} corr<0.95={g_corr}")
-    print(f"PHASE-0: {label.upper()}")
-
-    summary = f"""# Phase-0 SSL feasibility gate — {label}
-
-**Date:** 2026-06-24
-**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
-(not Go #4); gate metric adapted from silhouette-on-embedding to match
-available data. Go harness (#4) remains open for production experiments.
-
-## Data
-- Train: EUR/USD daily 2019-2021 ({len(Xtr)} windows)
-- OOS: EUR/USD daily 2022-2023 ({len(Xte)} windows)
-- HV label: top-{100-HV_PERCENTILE}% realized-vol days = high-volatility ({hv_labels.sum()} days)
-
-## Results
-| | Value | Gate |
-|---|---|---|
-| TS-JEPA mean silhouette (3 seeds) | {mean_sil:.4f} | > 0.20 → **{g_sil}** |
-| Beats PCA baseline ({pca_sil:.4f}) | {mean_sil:.4f} | > PCA → **{g_beats}** |
-| Seed stability (spread) | {spread:.2%} | < 10% → **{g_stable}** |
-| PC1/HV correlation | {pc1_hv_corr:.4f} | < 0.95 → **{g_corr}** |
-
-Per-seed: {[f"{s:.4f}" for s in sils]}
-
-## Verdict: **{label.upper()}**
-
-{"All 4 gate criteria met. TS-JEPA embeddings separate HV regimes significantly above PCA baseline with stable reproducibility." if passed else "One or more gate criteria not met. See null result protocol in #5."}
-"""
-    path = f"results/summaries/phase-0-{label}.md"
-    with open(path, "w") as f:
-        f.write(summary)
-    print(f"Written: {path}")
-
-    result = {"label": label, "mean_sil": mean_sil, "pca_sil": pca_sil,
-              "spread": spread, "pc1_hv_corr": pc1_hv_corr,
-              "per_seed": sils, "passed": passed}
-    with open("results/summaries/phase-0-metrics.json", "w") as f:
-        json.dump(result, f, indent=2)
-    return 0 if passed else 1
-
-
-if __name__ == "__main__":
-    raise SystemExit(main())

scripts/prepare_data.py

@@ -1,57 +0,0 @@
-"""LOCKED data pipeline (toy) — agent must NOT edit (brain Phase-1 contract).
-
-Parses histdata EUR/USD M1 zips → daily series with realized volatility (the
-val_vol_r2 target = 1-day realized vol from intraday squared returns).
-Output: data/processed/eurusd_daily.parquet  [date, close, ret, realized_vol].
-"""
-import glob
-import os
-import zipfile
-
-import numpy as np
-import pandas as pd
-
-RAW = "data/raw"
-OUT = "data/processed/eurusd_daily.parquet"
-
-
-def load_m1() -> pd.DataFrame:
-    frames = []
-    for zp in sorted(glob.glob(os.path.join(RAW, "DAT_ASCII_EURUSD_M1_*.zip"))):
-        with zipfile.ZipFile(zp) as z:
-            csv = [n for n in z.namelist() if n.endswith(".csv")][0]
-            with z.open(csv) as f:
-                df = pd.read_csv(
-                    f, sep=";", header=None,
-                    names=["dt", "open", "high", "low", "close", "vol"],
-                )
-        df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
-        frames.append(df[["ts", "close"]])
-    out = pd.concat(frames).sort_values("ts").reset_index(drop=True)
-    return out
-
-
-def main():
-    m1 = load_m1()
-    m1["r"] = np.log(m1["close"]).diff()
-    m1["day"] = m1["ts"].dt.normalize()
-    daily = m1.groupby("day").agg(
-        close=("close", "last"),
-        realized_vol=("r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
-        n_min=("r", "count"),
-    ).reset_index()
-    daily = daily[daily["n_min"] > 60]  # drop thin days (holidays)
-    daily["ret"] = np.log(daily["close"]).diff()
-    daily = daily.dropna().reset_index(drop=True)
-    os.makedirs(os.path.dirname(OUT), exist_ok=True)
-    daily[["day", "close", "ret", "realized_vol"]].rename(columns={"day": "date"}).to_parquet(OUT)
-    print("rows:", len(daily), "| dates:", daily["day"].min().date(), "→", daily["day"].max().date())
-    # sanity: the COVID crash (March 2020) must show a realized-vol spike
-    rv = daily.set_index("day")["realized_vol"]
-    mar20 = rv["2020-03-01":"2020-03-31"].max()
-    typ = rv["2019-01-01":"2019-12-31"].median()
-    print("median 2019 RV: %.5f | max Mar-2020 RV: %.5f | spike x%.1f" % (typ, mar20, mar20 / typ))
-
-
-if __name__ == "__main__":
-    main()

scripts/prepare_hourly.py

@@ -1,128 +0,0 @@
-"""Prepare EUR/USD hourly OHLCV + realized vol from histdata M1 zips.
-
-Aggregates all M1 bars in data/raw/DAT_ASCII_EURUSD_M1_*.zip to hourly.
-Realized vol per hour = sqrt(sum(log-return²)) over the constituent M1 bars.
-Weekend hours are naturally absent (FX market closed Sat/Sun); NO interpolation.
-Hours with fewer than MIN_BARS M1 bars are dropped (holidays, thin sessions).
-
-Output: data/processed/eurusd_hourly.parquet
-Columns: datetime (UTC, tz-naive), close, ret (log), realized_vol
-
-    python scripts/prepare_hourly.py
-    RAW=data/raw OUT=data/processed/eurusd_hourly.parquet python scripts/prepare_hourly.py
-"""
-import glob
-import os
-import zipfile
-
-import numpy as np
-import pandas as pd
-
-PAIR        = os.environ.get("PAIR", "EURUSD").upper()
-RAW_DEFAULT = "data/raw"
-OUT_DEFAULT = f"data/processed/{PAIR.lower()}_hourly.parquet"
-MIN_BARS    = 30   # drop hours thinner than this (holidays, DST boundary artefacts)
-
-
-# ── Core transformation ──────────────────────────────────────────────────────
-
-def resample_to_hourly(m1: pd.DataFrame) -> pd.DataFrame:
-    """Aggregate M1 DataFrame to hourly bars.
-
-    Args:
-        m1: DataFrame with columns ['ts', 'open', 'high', 'low', 'close']
-            ('open'/'high'/'low' optional — omit for close-only data).
-
-    Returns:
-        DataFrame with columns ['datetime', 'close', 'ret', 'realized_vol',
-        'hl_range', 'ret_intrabar'] sorted by datetime.
-        Hours with fewer than MIN_BARS M1 ticks are dropped.
-    """
-    m1 = m1.sort_values("ts").copy()
-    m1["log_r"] = np.log(m1["close"]).diff()
-    m1["hour"]  = m1["ts"].dt.floor("h")
-
-    has_ohlc = all(c in m1.columns for c in ("open", "high", "low"))
-
-    agg_dict = dict(
-        close        = ("close", "last"),
-        realized_vol = ("log_r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
-        n_bars       = ("log_r", "count"),
-    )
-    if has_ohlc:
-        agg_dict["high"]  = ("high",  "max")
-        agg_dict["low"]   = ("low",   "min")
-        agg_dict["open_"] = ("open",  "first")
-
-    agg = m1.groupby("hour").agg(**agg_dict).reset_index()
-
-    agg = agg[agg["n_bars"] >= MIN_BARS].copy()
-    agg["ret"] = np.log(agg["close"]).diff()
-    agg = agg.dropna(subset=["ret"]).reset_index(drop=True)
-    agg = agg.rename(columns={"hour": "datetime"})
-
-    if has_ohlc:
-        agg["hl_range"]    = np.log(agg["high"] / agg["low"])
-        agg["ret_intrabar"]= np.log(agg["close"] / agg["open_"])
-        cols = ["datetime", "close", "ret", "realized_vol", "hl_range", "ret_intrabar"]
-    else:
-        cols = ["datetime", "close", "ret", "realized_vol"]
-
-    return agg[cols]
-
-
-def load_m1_from_zips(raw_dir: str, pair: str = None) -> pd.DataFrame:
-    """Load and concatenate all M1 zips from raw_dir (histdata format)."""
-    p = (pair or PAIR).upper()
-    pattern = os.path.join(raw_dir, f"DAT_ASCII_{p}_M1_*.zip")
-    zips = sorted(glob.glob(pattern))
-    if not zips:
-        raise FileNotFoundError(f"No M1 zips found at {pattern}")
-    frames = []
-    for zp in zips:
-        with zipfile.ZipFile(zp) as z:
-            csv = [n for n in z.namelist() if n.endswith(".csv")][0]
-            with z.open(csv) as f:
-                df = pd.read_csv(
-                    f, sep=";", header=None,
-                    names=["dt", "open", "high", "low", "close", "vol"],
-                )
-        df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
-        frames.append(df[["ts", "open", "high", "low", "close"]])
-        print(f"  loaded {os.path.basename(zp)}: {len(df):,} rows")
-    return pd.concat(frames).sort_values("ts").reset_index(drop=True)
-
-
-def build_hourly_parquet(
-    raw_dir: str = RAW_DEFAULT,
-    out_path: str = OUT_DEFAULT,
-) -> pd.DataFrame:
-    """Full pipeline: load all M1 zips → hourly parquet. Returns the DataFrame."""
-    print(f"Loading M1 zips from {raw_dir}...")
-    m1 = load_m1_from_zips(raw_dir)
-    print(f"Total M1 bars: {len(m1):,}  ({m1['ts'].min().date()} → {m1['ts'].max().date()})")
-
-    print("Resampling to hourly...")
-    hourly = resample_to_hourly(m1)
-    print(f"Hourly rows: {len(hourly):,}  ({hourly['datetime'].min()} → {hourly['datetime'].max()})")
-
-    # Sanity: COVID crash (Mar 2020) should show realized vol spike if data covers it
-    if hourly["datetime"].dt.year.isin([2020]).any():
-        rv = hourly.set_index("datetime")["realized_vol"]
-        try:
-            mar20 = rv["2020-03-01":"2020-03-31"].max()
-            typ   = rv["2019-01-01":"2019-12-31"].median()
-            print(f"Sanity — median 2019 RV: {typ:.6f} | max Mar-2020 RV: {mar20:.6f} | spike ×{mar20/typ:.1f}")
-        except Exception:
-            pass
-
-    os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True)
-    hourly.to_parquet(out_path, index=False)
-    print(f"Written: {out_path}")
-    return hourly
-
-
-if __name__ == "__main__":
-    raw_dir  = os.environ.get("RAW",  RAW_DEFAULT)
-    out_path = os.environ.get("OUT",  OUT_DEFAULT)
-    build_hourly_parquet(raw_dir=raw_dir, out_path=out_path)

scripts/prepare_multipair.py

@@ -1,72 +0,0 @@
-"""Merge per-pair hourly parquets into a single wide multipair parquet.
-
-Each pair contributes two features: {pair}_ret and {pair}_rv (realized vol).
-The merge is an INNER JOIN on datetime — only hours present in ALL pairs are kept.
-The target for train.py remains eurusd_rv.
-
-Output: data/processed/eurusd_multipair.parquet
-
-    python scripts/prepare_multipair.py
-    PROCESSED=data/processed python scripts/prepare_multipair.py
-"""
-import os
-import pandas as pd
-
-PAIRS = ["eurusd", "gbpusd", "usdjpy", "usdchf", "audusd"]
-
-PROCESSED_DEFAULT = "data/processed"
-OUT_DEFAULT       = "data/processed/eurusd_multipair.parquet"
-
-
-def merge_pair_parquets(pair_dfs: dict) -> pd.DataFrame:
-    """Inner-join hourly DataFrames from multiple pairs on datetime.
-
-    Args:
-        pair_dfs: dict mapping pair name (e.g. "eurusd") to hourly DataFrame
-                  with columns [datetime, close, ret, realized_vol, ...].
-    Returns:
-        Wide DataFrame with columns:
-          datetime, {pair}_ret, {pair}_rv  for each pair.
-    """
-    merged = None
-    for pair, df in pair_dfs.items():
-        sub = df[["datetime", "ret", "realized_vol"]].copy()
-        sub = sub.rename(columns={"ret": f"{pair}_ret", "realized_vol": f"{pair}_rv"})
-        sub = sub.set_index("datetime")
-        if merged is None:
-            merged = sub
-        else:
-            merged = merged.join(sub, how="inner")
-
-    return merged.reset_index()
-
-
-def build_multipair_parquet(
-    processed_dir: str = PROCESSED_DEFAULT,
-    out_path: str = OUT_DEFAULT,
-    pairs: list = None,
-) -> None:
-    if pairs is None:
-        pairs = PAIRS
-    pair_dfs = {}
-    for pair in pairs:
-        path = os.path.join(processed_dir, f"{pair}_hourly.parquet")
-        if not os.path.exists(path):
-            raise FileNotFoundError(
-                f"{pair}_hourly.parquet not found at {path} — run prepare_hourly.py for this pair first"
-            )
-        df = pd.read_parquet(path)
-        pair_dfs[pair] = df
-
-    merged = merge_pair_parquets(pair_dfs)
-    merged.to_parquet(out_path, index=False)
-    n_pairs = len(pairs)
-    n_ch    = n_pairs * 2
-    print(f"Multipair parquet: {len(merged):,} rows × {n_ch} feature channels ({n_pairs} pairs)")
-    print(f"Date range: {merged['datetime'].min()} → {merged['datetime'].max()}")
-    print(f"Written: {out_path}")
-
-
-if __name__ == "__main__":
-    processed_dir = os.environ.get("PROCESSED", PROCESSED_DEFAULT)
-    build_multipair_parquet(processed_dir=processed_dir)

scripts/prepare_regime.py

@@ -1,134 +0,0 @@
-"""HMM regime detector — 3-state Gaussian HMM on realized_vol.
-
-Fits on the FULL dataset (training + OOS) so the state sequence is globally
-consistent across all periods. States are sorted by mean realized vol (ascending):
-    0 = calm,   1 = stressed,   2 = crisis
-
-Output: data/processed/eurusd_regime.parquet
-Columns: datetime (or date), regime (int: 0/1/2)
-
-Deterministic: fixed random_state=42 throughout.
-Cached: if the parquet already exists, it is not re-computed.
-
-Usage:
-    python scripts/prepare_regime.py [--hourly] [--daily] [--force]
-
-jepa-fx-risk#13
-"""
-
-import argparse
-import os
-from pathlib import Path
-
-import numpy as np
-import pandas as pd
-from hmmlearn import hmm
-
-DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
-HOURLY_PATH = DATA_DIR / "eurusd_hourly.parquet"
-DAILY_PATH = DATA_DIR / "eurusd_daily.parquet"
-OUTPUT_PATH = DATA_DIR / "eurusd_regime.parquet"
-
-N_STATES = 3
-RANDOM_STATE = 42
-
-
-def fit_regime_hmm(realized_vol: np.ndarray, n_states: int = 3, random_state: int = 42) -> np.ndarray:
-    """Fit a Gaussian HMM on realized_vol and return state labels (0=calm → n_states-1=crisis).
-
-    States are sorted by mean realized vol ascending so label 0 is always calm,
-    label n_states-1 is always crisis. This makes the labelling deterministic
-    across datasets with different vol levels.
-
-    Args:
-        realized_vol:  1-D array of realized vol values
-        n_states:      number of HMM hidden states (default 3)
-        random_state:  random seed for reproducibility
-
-    Returns:
-        Integer label array of shape (len(realized_vol),), dtype int64
-    """
-    X = realized_vol.reshape(-1, 1).astype(np.float64)
-    model = hmm.GaussianHMM(
-        n_components=n_states,
-        covariance_type="diag",
-        min_covar=1e-6,
-        n_iter=100,
-        random_state=random_state,
-        tol=1e-4,
-    )
-    model.fit(X)
-    raw_labels = model.predict(X)
-
-    # Sort states by mean realized vol (ascending: calm=0, crisis=n_states-1)
-    state_means = np.array([X[raw_labels == s].mean() if (raw_labels == s).any() else 0.0
-                            for s in range(n_states)])
-    rank = np.argsort(state_means)          # rank[0] = original state id of the calmest cluster
-    remap = np.empty(n_states, dtype=np.int64)
-    for new_label, old_label in enumerate(rank):
-        remap[old_label] = new_label
-    return remap[raw_labels].astype(np.int64)
-
-
-def prepare_regime_df(parquet_path: str, freq: str = "hourly") -> pd.DataFrame:
-    """Load parquet, fit HMM, return DataFrame with timestamp + regime columns.
-
-    Args:
-        parquet_path: path to input parquet (hourly or daily)
-        freq:         "hourly" | "daily" — determines timestamp column name
-
-    Returns:
-        DataFrame with columns: (datetime|date), regime
-    """
-    df = pd.read_parquet(parquet_path)
-    if freq == "hourly":
-        ts = pd.to_datetime(df["datetime"])
-    else:
-        ts = pd.to_datetime(df["date"])
-
-    rv = df["realized_vol"].to_numpy(np.float32)
-    labels = fit_regime_hmm(rv, n_states=N_STATES, random_state=RANDOM_STATE)
-    return pd.DataFrame({"datetime": ts.values, "regime": labels})
-
-
-def main():
-    parser = argparse.ArgumentParser(description="Fit HMM regime detector")
-    parser.add_argument("--hourly", action="store_true", default=True,
-                        help="use hourly parquet (default)")
-    parser.add_argument("--daily", action="store_true", default=False,
-                        help="use daily parquet instead of hourly")
-    parser.add_argument("--force", action="store_true", default=False,
-                        help="overwrite existing output")
-    parser.add_argument("--out", default=str(OUTPUT_PATH),
-                        help="output parquet path")
-    args = parser.parse_args()
-
-    out_path = Path(args.out)
-    if out_path.exists() and not args.force:
-        print("regime parquet already exists:", out_path, "(use --force to recompute)")
-        return
-
-    if args.daily and DAILY_PATH.exists():
-        src, freq = str(DAILY_PATH), "daily"
-    elif HOURLY_PATH.exists():
-        src, freq = str(HOURLY_PATH), "hourly"
-    elif DAILY_PATH.exists():
-        src, freq = str(DAILY_PATH), "daily"
-    else:
-        raise FileNotFoundError("no parquet found in data/processed/")
-
-    print(f"fitting HMM ({N_STATES} states) on {src} ...")
-    df = prepare_regime_df(src, freq=freq)
-
-    counts = df["regime"].value_counts().sort_index()
-    print("regime distribution:")
-    for state, count in counts.items():
-        label = {0: "calm", 1: "stressed", 2: "crisis"}.get(state, f"state{state}")
-        print(f"  {state} ({label}): {count} ({100*count/len(df):.1f}%)")
-
-    df.to_parquet(out_path, index=False)
-    print("wrote:", out_path)
-
-
-if __name__ == "__main__":
-    main()

scripts/var_breach.py

@@ -1,67 +0,0 @@
-"""Parametric 99% VaR breach rate + Kupiec POF p-value.
-
-Used by train.py's LOCKED VaR EVAL BLOCK to write VaR_breach_rate_99_oos_regime_cond
-to metrics.json so the autoresearch loop can optimise it.
-
-jepa-fx-risk#12
-"""
-
-import math
-
-# Canonical metric key — no surrounding whitespace, as required by the loop contract.
-METRIC_KEY = "VaR_breach_rate_99_oos_regime_cond"
-
-# Default normal 99th-percentile z-score.
-Z99 = 2.326
-
-
-def var_breach_rate(pred_vol, actual_vol, z99=Z99):
-    """Compute VaR breach rate and Kupiec POF p-value.
-
-    Args:
-        pred_vol:   iterable of predicted conditional vol forecasts
-        actual_vol: iterable of actual realized vol (same length)
-        z99:        99th-percentile z-score (default 2.326)
-
-    Returns:
-        (breach_rate, kupiec_p) where:
-            breach_rate — fraction of steps where actual_vol > pred_vol × z99
-            kupiec_p    — Kupiec POF p-value (H0: true breach rate = 1%)
-                          High p-value = well-calibrated; low = miscalibrated tail.
-    """
-    pred_v = list(pred_vol)
-    act_v = list(actual_vol)
-    n = len(pred_v)
-    if n == 0 or n != len(act_v):
-        return 0.0, 1.0
-
-    n1 = sum(1 for p, a in zip(pred_v, act_v) if a > p * z99)
-    breach_rate = n1 / n
-    p = kupiec_pvalue(n, n1)
-    return breach_rate, p
-
-
-def kupiec_pvalue(n, n1, p0=0.01):
-    """Kupiec Proportion-of-Failures likelihood ratio test.
-
-    H0: true breach probability = p0.
-    Returns P(chi²(1) > LR) using the identity P(chi²(1)>x) = erfc(sqrt(x/2)).
-    Returns 1.0 for n=0 or LR<=0 (well-calibrated / over-conservative).
-    """
-    if n == 0:
-        return 1.0
-    n0 = n - n1
-    phat = n1 / n
-
-    if n1 == 0:
-        # 0 × ln(0/p0) = 0 by convention; only n0 term contributes
-        lr = 2 * n0 * math.log((1 - phat) / (1 - p0))
-    elif n1 == n:
-        lr = 2 * n1 * math.log(phat / p0)
-    else:
-        lr = 2 * (n1 * math.log(phat / p0) + n0 * math.log((1 - phat) / (1 - p0)))
-
-    if lr <= 0:
-        return 1.0
-    # P(chi²(1) > LR) = erfc(sqrt(LR/2))
-    return math.erfc(math.sqrt(lr / 2))

specs/phase-0-ssl-feasibility.md

@@ -0,0 +1,84 @@
+# Experiment Spec: Phase 0 — SSL Feasibility Gate
+
+## Hypothesis
+
+> We believe that a masked autoencoder (MAE) trained on FX hourly time-series will
+> produce latent embeddings that show structural separability by volatility regime
+> without explicit regime labels, measurable by silhouette score > 0.20 on k-means
+> clusters evaluated against a held-out realised-volatility regime label on 2023 data.
+
+This hypothesis is FALSE if silhouette score ≤ 0.20 on the held-out evaluation.
+
+## Background
+
+Before investing in JEPA-specific machinery, we need to confirm that self-supervised
+representation learning can find regime structure in FX time-series at all. If the
+simplest SSL method (MAE) cannot find structure, JEPA will not either — and the root
+cause needs to be understood before proceeding.
+
+Also validates that TS-JEPA code is reproducible: first task is running TS-JEPA on
+the paper's own benchmark, not on FX data. If reproduction takes > 2 weeks, contact
+authors or fall back to implementing JEPA masking from V-JEPA codebase.
+
+Added post Full Grill (2026-05-27). See DECISIONS.md: "Phase 0: SSL feasibility gate."
+
+## Design
+
+### Data
+- **Source:** DUKASCopy, EUR/USD hourly OHLCV
+- **Train:** 2008-01-01 – 2022-12-31
+- **Held-out test:** 2023-01-01 – 2023-12-31 (sealed until final evaluation)
+- **Features:** log-return (hourly), rolling 20-period realised HV, VIX (daily → hourly interpolation)
+- **Regime label (evaluation only):** rolling 30-day HV percentile; binary: top 50% = high-vol, bottom 50% = low-vol
+
+### Model
+- **Architecture:** 1D temporal Masked Autoencoder
+  - Encoder: 3-layer 1D CNN + positional encoding
+  - Decoder: 2-layer MLP reconstructing masked segment
+- **Masking:** contiguous temporal block (target); context window = 120h (5 days)
+- **Loss:** MSE reconstruction on masked segment
+
+### TS-JEPA reproduction task (runs in parallel / before MAE)
+- Reproduce TS-JEPA paper results on the authors' benchmark dataset
+- Go/no-go: if reproduction fails within 2 weeks → contact authors or pivot to V-JEPA adaptation
+
+### Baseline
+- **PCA** on raw feature vectors (same 120h context window, flattened)
+- Tests whether any dimensionality reduction finds regime structure; confirms SSL is adding something
+
+### Ablations
+- Masking horizon K ∈ {8h, 24h, 72h} — does context length affect embedding quality?
+
+## Acceptance Criteria
+
+- [ ] MAE silhouette score > 0.20 on held-out 2023 data (k-means k=3, vs. binary HV regime label)
+- [ ] MAE silhouette exceeds PCA baseline silhouette
+- [ ] Rerun ×3 within ±10% of reported silhouette (reproducibility)
+- [ ] PC1 / rolling-HV correlation < 0.95 (encoder is not purely encoding volatility level)
+- [ ] TS-JEPA reproduced on paper's benchmark within 2 weeks of start
+
+## Out of Scope
+
+- JEPA implementation (Phase 1)
+- Multi-pair training (Phase 1+)
+- VaR, ES, or any risk metric computation
+- Any data after 2022-12-31 (training); 2023 test set opened only for final evaluation
+- Hyperparameter search beyond the three masking horizons defined above
+
+## Null Result Protocol
+
+If MAE silhouette ≤ 0.20 on held-out data:
+1. Conclude: SSL-based regime detection is not straightforwardly feasible on EUR/USD hourly data with these three features
+2. Investigate in order: (a) try daily resolution instead of hourly, (b) expand feature set to 6 features (add DXY, yield spread), (c) try 4-class regime label (HV quartiles) instead of binary
+3. If all three investigations fail: conclude SSL regime detection is not viable on FX data; document and consider pivoting the primary hypothesis to distributional forecasting directly
+4. Record result in `results/summaries/phase-0-null.md`
+5. Write failure mode to brain: `brain_write wing=jepa-fx hall=failures`
+
+## Risks
+
+| Risk | Canary | Mitigation |
+|---|---|---|
+| TS-JEPA code unreproducible | Benchmark result doesn't match paper within 2 weeks | Contact authors; fall back to V-JEPA adaptation |
+| MAE encoder collapses | Reconstruction loss plateau in first 10 epochs; all embeddings near-identical | Add batch normalisation; reduce LR; check masking ratio |
+| Regime label too coarse | Silhouette low even with visually structured embeddings | Also evaluate with 4-class HV quartile label |
+| PC1 is just volatility | PC1/HV correlation > 0.95 | Useful finding; record it; do not declare success |

specs/phase-1-representation-poc.md

@@ -0,0 +1,82 @@
+# Experiment Spec: Phase 1 — JEPA Representation PoC
+
+## Hypothesis
+
+> We believe that a TS-JEPA encoder trained on G10 FX hourly data (2008–2022) will
+> produce latent market-state embeddings that are structurally separable by volatility
+> regime without explicit regime labels, measurable by silhouette score > 0.35 on
+> k-means clusters evaluated against realised-volatility regime labels on held-out
+> 2023 data including at least one structural break.
+
+This hypothesis is FALSE if silhouette score ≤ 0.35 OR linear probe R² ≤ 0.40 on
+held-out evaluation.
+
+**Prerequisites:** Phase 0 passed (MAE silhouette > 0.20, TS-JEPA reproduced).
+
+## Background
+
+Phase 0 confirmed that SSL-based representation learning can find regime structure in
+FX time-series. Phase 1 tests whether JEPA's specific inductive bias (predict target
+embeddings from context embeddings, never reconstruct raw data) produces richer
+representations than a simple MAE — and whether those representations are useful for
+risk management tasks (measurable via linear probe).
+
+## Design
+
+### Data
+- **Source:** DUKASCopy, all G10 pairs (EUR/USD, GBP/USD, USD/JPY, USD/CHF, AUD/USD, USD/CAD, NZD/USD, EUR/GBP, EUR/JPY, EUR/CHF), hourly
+- **Train:** 2008-01-01 – 2022-12-31 (all 10 pairs, jointly)
+- **Held-out test:** 2023-01-01 – 2023-12-31 (sealed until final evaluation)
+- **Features:** log-return, rolling 20-period HV, VIX (daily → hourly)
+- **Regime label (evaluation only):** rolling 30-day HV percentile; binary + 4-class (quartiles)
+
+### Model
+- **Architecture:** TS-JEPA (Ennadir et al., 2025)
+  - Context encoder: maps observed window → latent embedding
+  - Target encoder: EMA of context encoder (momentum β ≈ 0.996); stop-gradient
+  - Predictor: shallow MLP bridging context → target embedding
+- **Masking horizons:** ablate K ∈ {1h, 8h, 24h}; context window = 120h
+- **Training:** multi-pair joint training (one model, all G10 pairs)
+
+### Baseline
+- Phase 0 MAE (best masking horizon from Phase 0)
+- PCA on raw features (Phase 0 baseline)
+
+### Ablations
+1. TS-JEPA vs MAE — is JEPA's no-reconstruction inductive bias adding value?
+2. Single-pair (EUR/USD only) vs. multi-pair — does joint training improve representations?
+3. Masking horizon K: {1h, 8h, 24h}
+
+## Acceptance Criteria
+
+- [ ] Silhouette score > 0.35 on held-out 2023 data (k-means k=3–5, binary HV label)
+- [ ] Linear probe R² > 0.40 on frozen embeddings vs. realised-vol decile
+- [ ] PC1 / rolling-HV correlation < 0.85 (encoder learning more than volatility level)
+- [ ] TS-JEPA silhouette exceeds Phase 0 MAE silhouette by > 5%
+- [ ] Rerun ×3 within ±10% of reported silhouette
+
+## Out of Scope
+
+- VaR, ES, distributional forecasting (Phase 3)
+- Exotic pairs beyond G10
+- Options pricing, alpha generation, live trading
+- Any data after 2022-12-31 for training; test set opened only for final evaluation
+- Regime detection backtesting (Phase 2 — embedding drift as early warning)
+
+## Null Result Protocol
+
+If primary criteria not met:
+1. If silhouette > 0.20 but ≤ 0.35: JEPA shows partial structure; not sufficient for Phase 2. Investigate whether multi-pair training, longer context window, or additional features close the gap. One retry permitted with documented rationale.
+2. If silhouette ≤ 0.20: regression from Phase 0; investigate JEPA training stability (collapse risk). Do not proceed.
+3. If linear probe R² ≤ 0.40 despite good silhouette: embeddings are structured but not encoding risk-relevant information. Record as a finding; reconsider feature set.
+4. Record all results in `results/summaries/phase-1-[pass|null].md`
+5. Write findings to brain: `brain_write wing=jepa-fx hall=failures`
+
+## Risks
+
+| Risk | Canary | Mitigation |
+|---|---|---|
+| EMA encoder collapses | All embeddings converge to near-zero; loss goes to ~0 early | Verify EMA momentum schedule; check stop-gradient implementation |
+| Encoder learns only EUR/USD vol | PC1 dominated by EUR/USD HV even in multi-pair model | Evaluate per-pair silhouette; if EUR/USD dominates, weight loss by pair |
+| Phase 0 silhouette was data-split artefact | Phase 1 MAE baseline doesn't reproduce Phase 0 numbers | Fix random seeds; document split methodology in Phase 0 |
+| Insufficient regime diversity (2008–2022) | Embedding clusters don't separate 2023 structural break | Verify 2023 test includes high-vol episode; add 4-class label as backup |

tests/test_autoresearch_start.py

@@ -1,215 +0,0 @@
-"""Tests for scripts/autoresearch_start.py — jepa-fx-risk#11 Phase A scaffold.
-
-Success criterion: `autoresearch start <backlog.json> <rq-id>` scaffolds a
-runnable run dir from a ready leaf; refuses non-ready nodes; strips
-candidate_metric; records provenance.
-"""
-
-import importlib.util
-import json
-import sys
-from pathlib import Path
-
-import pytest
-
-# Load the module without executing main()
-_SCRIPT = Path(__file__).parent.parent / "scripts" / "autoresearch_start.py"
-
-
-def _import():
-    spec = importlib.util.spec_from_file_location("autoresearch_start", _SCRIPT)
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
-
-
-@pytest.fixture()
-def mod():
-    return _import()
-
-
-@pytest.fixture()
-def backlog(tmp_path):
-    data = {
-        "strategic_question": "Test strategic question?",
-        "generated_at": "2026-06-27T00:00:00Z",
-        "nodes": [
-            {
-                "id": "rq-01",
-                "question": "Does X improve Y?",
-                "case_type": "autoresearch-loop",
-                "data": "obtainable",
-                "method": "adjacent",
-                "falsifiable": "yes",
-                "candidate_metric": " val_vol_r2",  # leading space — bypass test
-                "depends_on": [],
-                "status": "autoresearch-ready",
-                "track": "autoresearch",
-                "converged": True,
-                "survived_review": True,
-            },
-            {
-                "id": "rq-02",
-                "question": "Not ready yet?",
-                "case_type": "empirical-study",
-                "data": "obtainable",
-                "method": "adjacent",
-                "falsifiable": "yes",
-                "candidate_metric": None,
-                "depends_on": [],
-                "status": "needs-metric",
-                "track": "study",
-                "converged": True,
-                "survived_review": True,
-            },
-            {
-                "id": "rq-03",
-                "question": "A spike.",
-                "case_type": "spike",
-                "data": "have",
-                "method": "yes-named",
-                "falsifiable": "yes",
-                "candidate_metric": None,
-                "depends_on": [],
-                "status": "spike-ready",
-                "track": "spike",
-                "converged": True,
-                "survived_review": True,
-            },
-        ],
-    }
-    p = tmp_path / "backlog.json"
-    p.write_text(json.dumps(data))
-    return p
-
-
-@pytest.fixture()
-def fake_train_py(tmp_path):
-    """Minimal train.py placeholder for scaffold tests."""
-    src = tmp_path / "train_template.py"
-    src.write_text("# train.py placeholder\n")
-    return src
-
-
-# ---------------------------------------------------------------------------
-# fail-closed: refuse non-autoresearch-ready nodes
-# ---------------------------------------------------------------------------
-
-class TestRefuseNonReady:
-    def test_refuses_needs_metric(self, mod, backlog, fake_train_py, tmp_path):
-        run_dir = tmp_path / "runs" / "rq-02"
-        with pytest.raises(SystemExit) as exc:
-            mod.scaffold_run(backlog, "rq-02", run_dir, fake_train_py)
-        assert exc.value.code != 0
-
-    def test_refuses_spike_ready(self, mod, backlog, fake_train_py, tmp_path):
-        run_dir = tmp_path / "runs" / "rq-03"
-        with pytest.raises(SystemExit) as exc:
-            mod.scaffold_run(backlog, "rq-03", run_dir, fake_train_py)
-        assert exc.value.code != 0
-
-    def test_refuses_missing_rq_id(self, mod, backlog, fake_train_py, tmp_path):
-        run_dir = tmp_path / "runs" / "rq-99"
-        with pytest.raises(SystemExit) as exc:
-            mod.scaffold_run(backlog, "rq-99", run_dir, fake_train_py)
-        assert exc.value.code != 0
-
-    def test_refuses_existing_run_dir(self, mod, backlog, fake_train_py, tmp_path):
-        run_dir = tmp_path / "runs" / "rq-01"
-        run_dir.mkdir(parents=True)
-        with pytest.raises(SystemExit) as exc:
-            mod.scaffold_run(backlog, "rq-01", run_dir, fake_train_py)
-        assert exc.value.code != 0
-
-
-# ---------------------------------------------------------------------------
-# scaffold structure: correct files created
-# ---------------------------------------------------------------------------
-
-class TestScaffoldStructure:
-    @pytest.fixture(autouse=True)
-    def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
-        self.run_dir = tmp_path / "runs" / "rq-01"
-        mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
-
-    def test_run_dir_created(self):
-        assert self.run_dir.is_dir()
-
-    def test_program_md_created(self):
-        assert (self.run_dir / "program.md").exists()
-
-    def test_run_json_created(self):
-        assert (self.run_dir / "run.json").exists()
-
-    def test_train_py_copied(self):
-        assert (self.run_dir / "train.py").exists()
-        assert (self.run_dir / "train.py").read_text() == "# train.py placeholder\n"
-
-
-# ---------------------------------------------------------------------------
-# program.md content
-# ---------------------------------------------------------------------------
-
-class TestProgramMd:
-    @pytest.fixture(autouse=True)
-    def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
-        self.run_dir = tmp_path / "runs" / "rq-01"
-        mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
-        self.content = (self.run_dir / "program.md").read_text()
-
-    def test_contains_hypothesis(self):
-        assert "Does X improve Y?" in self.content
-
-    def test_metric_key_stripped(self):
-        # candidate_metric had leading space " val_vol_r2" — must be stripped
-        assert "`val_vol_r2`" in self.content
-        assert "` val_vol_r2`" not in self.content
-
-    def test_contains_strategic_question(self):
-        assert "Test strategic question?" in self.content
-
-    def test_contains_council_node(self):
-        assert "rq-01" in self.content
-
-
-# ---------------------------------------------------------------------------
-# run.json provenance
-# ---------------------------------------------------------------------------
-
-class TestRunJson:
-    @pytest.fixture(autouse=True)
-    def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
-        self.run_dir = tmp_path / "runs" / "rq-01"
-        mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
-        self.run = json.loads((self.run_dir / "run.json").read_text())
-
-    def test_strategic_question_in_provenance(self):
-        assert self.run["strategic_question"] == "Test strategic question?"
-
-    def test_council_node_in_provenance(self):
-        assert self.run["council_node"] == "rq-01"
-
-    def test_metric_stripped_in_provenance(self):
-        assert self.run["metric"] == "val_vol_r2"
-        assert self.run["metric"] == self.run["metric"].strip()
-
-    def test_generated_at_present(self):
-        assert "generated_at" in self.run
-
-    def test_max_iters_present(self):
-        assert "max_iters" in self.run
-
-
-# ---------------------------------------------------------------------------
-# load_backlog helper
-# ---------------------------------------------------------------------------
-
-class TestLoadBacklog:
-    def test_loads_json(self, mod, backlog):
-        data = mod.load_backlog(str(backlog))
-        assert data["strategic_question"] == "Test strategic question?"
-        assert len(data["nodes"]) == 3
-
-    def test_missing_file_raises(self, mod, tmp_path):
-        with pytest.raises((FileNotFoundError, SystemExit)):
-            mod.load_backlog(str(tmp_path / "nonexistent.json"))

tests/test_hepa.py

@@ -1,277 +0,0 @@
-"""Failing tests for HEPA backbone + Phase-1 supervised head + HPO in train.py.
-
-Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_hepa.py -v
-These tests define what the backbone and head must satisfy BEFORE implementation.
-"""
-import math
-import os
-import torch
-import torch.nn as nn
-import pytest
-
-# ── Tests import the classes from train.py ────────────────────────────────────
-# They will fail until train.py implements: CausalEncoder, HorizonPredictor, vicreg_loss
-
-
-def _import(env_overrides=None):
-    import importlib.util, sys
-    saved = {}
-    if env_overrides:
-        for k, v in env_overrides.items():
-            saved[k] = os.environ.get(k)
-            os.environ[k] = str(v)
-    # Force fresh module load (env vars must be read at import time)
-    name = f"train_{id(env_overrides)}"
-    spec = importlib.util.spec_from_file_location(name, "train.py")
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    if env_overrides:
-        for k, orig in saved.items():
-            if orig is None:
-                os.environ.pop(k, None)
-            else:
-                os.environ[k] = orig
-    return mod
-
-
-@pytest.fixture(scope="module")
-def train_mod():
-    return _import()
-
-
-# 1. CausalEncoder exists and has correct output shape
-def test_causal_encoder_shape(train_mod):
-    enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
-    x = torch.randn(4, 60, 2)
-    tokens = enc(x)          # should return all tokens (B, N, D) for JEPA pretraining
-    assert tokens.shape == (4, 6, 32), f"expected (4, 6, 32), got {tokens.shape}"
-
-
-# 2. CausalEncoder is actually causal: earlier token outputs don't change when later inputs change
-def test_causal_masking(train_mod):
-    enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=2)
-    enc.eval()
-    torch.manual_seed(0)
-    x = torch.randn(1, 60, 2)
-    x_perturbed = x.clone()
-    # non-uniform noise (constant shift absorbed by per-patch LayerNorm; variance change is not)
-    torch.manual_seed(99)
-    x_perturbed[:, 30:, :] += torch.randn_like(x[:, 30:, :]) * 5.0
-
-    with torch.no_grad():
-        h1 = enc(x)
-        h2 = enc(x_perturbed)
-
-    # First 3 tokens must be identical (causal — don't see future patches)
-    assert torch.allclose(h1[:, :3, :], h2[:, :3, :], atol=1e-5), \
-        "causal masking broken: early tokens change when later input changes"
-    # Last token should differ (it can see the perturbed patches)
-    assert not torch.allclose(h1[:, -1, :], h2[:, -1, :], atol=1e-5), \
-        "last token should differ when later input changes"
-
-
-# 3. HorizonPredictor exists, takes (h, delta_t_float) → same shape as h
-def test_horizon_predictor_shape(train_mod):
-    pred = train_mod.HorizonPredictor(d_model=32)
-    h = torch.randn(4, 32)
-    dt = torch.tensor([1.0, 2.0, 3.0, 1.0])
-    out = pred(h, dt)
-    assert out.shape == (4, 32), f"expected (4, 32), got {out.shape}"
-
-
-# 4. vicreg_loss is a scalar and backward doesn't error
-def test_vicreg_loss_backward(train_mod):
-    h_pred = torch.randn(8, 32, requires_grad=True)
-    h_target = torch.randn(8, 32)
-    loss = train_mod.vicreg_loss(h_pred, h_target, alpha=0.1)
-    assert loss.shape == (), f"expected scalar, got {loss.shape}"
-    loss.backward()
-    assert h_pred.grad is not None
-
-
-# 5. Full JEPA step: encode context, predict future, compute loss, backward
-def test_jepa_step_end_to_end(train_mod):
-    enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
-    pred = train_mod.HorizonPredictor(d_model=32)
-    opt = torch.optim.SGD(list(enc.parameters()) + list(pred.parameters()), lr=1e-3)
-
-    x = torch.randn(4, 60, 2)
-    tokens = enc(x)          # (4, 6, 32)
-    c, dt = 2, 2             # context position 2, horizon 2
-    h_ctx = tokens[:, c, :]
-    h_tgt = tokens[:, c + dt, :].detach()
-    h_hat = pred(h_ctx, torch.full((4,), float(dt)))
-    loss = train_mod.vicreg_loss(h_hat, h_tgt, alpha=0.1)
-    opt.zero_grad(); loss.backward(); opt.step()
-    assert loss.item() < 100, "loss exploded"
-
-
-# 6. build() returns year-based OOS split (2022-2023); hourly gives many more windows
-def test_build_year_split(train_mod):
-    (Xtr, ytr), (Xte, yte) = train_mod.build()
-    assert Xtr.shape[1] == train_mod.WINDOW
-    assert Xte.shape[1] == train_mod.WINDOW
-    assert len(Xtr) > 0 and len(Xte) > 0
-    # OOS: daily ≈ 600; hourly ≈ 17,000 (2 years × ~8,500 trading hours/year)
-    assert len(Xte) > 400, f"OOS too small: {len(Xte)}"
-
-
-# 7. hourly build gives > 10× more training windows than daily
-def test_build_hourly_more_windows(train_mod):
-    import os
-    if not os.path.exists("data/processed/eurusd_hourly.parquet"):
-        pytest.skip("eurusd_hourly.parquet not present — run data:prepare:hourly first")
-    (Xtr, _), _ = train_mod.build()
-    # Daily had ~877 train windows; hourly with 2008-2021 should have > 50,000
-    assert len(Xtr) > 10_000, f"expected >10k hourly train windows, got {len(Xtr)}"
-
-
-# ── Phase-1: supervised head ──────────────────────────────────────────────────
-
-# 8. SupervisedHead exists and maps (B, D) → (B,)
-def test_supervised_head_shape(train_mod):
-    D = 128
-    head = train_mod.SupervisedHead(D)
-    x = torch.randn(16, D)
-    out = head(x)
-    assert out.shape == (16,), f"expected (16,), got {out.shape}"
-
-
-# 9. SupervisedHead gradient flows (not frozen)
-def test_supervised_head_backward(train_mod):
-    head = train_mod.SupervisedHead(64)
-    x = torch.randn(8, 64)
-    loss = head(x).mean()
-    loss.backward()
-    for name, p in head.named_parameters():
-        assert p.grad is not None, f"no grad on {name}"
-
-
-# 10. Phase-1 beats linear on nonlinear synthetic signal
-def test_phase1_beats_linear_on_nonlinear(train_mod):
-    """MLP head should outperform ridge regression on data with nonlinear structure."""
-    import numpy as np
-    torch.manual_seed(0); np.random.seed(0)
-    N, D = 1000, 32
-    # target = |h|² (quadratic — linear can't fit well)
-    Etr = np.random.randn(N, D).astype(np.float32)
-    ytr = (Etr ** 2).sum(axis=1)
-    Ete = np.random.randn(200, D).astype(np.float32)
-    yte = (Ete ** 2).sum(axis=1)
-
-    # Ridge baseline
-    A = np.hstack([Etr, np.ones((N, 1))])
-    w = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
-    pred_lin = np.hstack([Ete, np.ones((200, 1))]) @ w
-    r2_lin = float(1 - ((yte - pred_lin) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
-
-    # MLP head
-    head = train_mod.SupervisedHead(D)
-    opt  = torch.optim.Adam(head.parameters(), lr=1e-2)
-    Xtr_t = torch.tensor(Etr); ytr_t = torch.tensor(ytr)
-    for _ in range(300):
-        loss = nn.functional.mse_loss(head(Xtr_t), ytr_t)
-        opt.zero_grad(); loss.backward(); opt.step()
-
-    head.eval()
-    with torch.no_grad():
-        pred_mlp = head(torch.tensor(Ete)).numpy()
-    r2_mlp = float(1 - ((yte - pred_mlp) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
-
-    assert r2_mlp > r2_lin + 0.05, (
-        f"MLP R²={r2_mlp:.3f} should beat ridge R²={r2_lin:.3f} by >0.05 on quadratic target"
-    )
-
-
-# 11. main() returns phase1_r2 in metrics.json (integration — needs real data)
-def test_metrics_json_has_phase1_r2(train_mod):
-    import json
-    if not os.path.exists("metrics.json"):
-        pytest.skip("metrics.json not present — run train.py first")
-    with open("metrics.json") as f:
-        m = json.load(f)
-    assert "phase1_r2" in m, f"phase1_r2 missing from metrics.json: {list(m.keys())}"
-    assert m["phase1_r2"] > m["val_vol_r2"], (
-        f"MLP head phase1_r2={m['phase1_r2']:.4f} should beat linear probe "
-        f"val_vol_r2={m['val_vol_r2']:.4f}"
-    )
-
-
-# ── HPO: env-var knob overrides ───────────────────────────────────────────────
-
-# 12. JEPA_WINDOW env var overrides WINDOW at import time
-def test_env_override_window():
-    mod = _import({"JEPA_WINDOW": "48"})
-    assert mod.WINDOW == 48, f"expected WINDOW=48, got {mod.WINDOW}"
-
-
-# 13. JEPA_D_MODEL and JEPA_DEPTH env vars work
-def test_env_override_d_model_depth():
-    mod = _import({"JEPA_D_MODEL": "64", "JEPA_DEPTH": "4"})
-    assert mod.D_MODEL == 64, f"expected D_MODEL=64, got {mod.D_MODEL}"
-    assert mod.DEPTH == 4, f"expected DEPTH=4, got {mod.DEPTH}"
-
-
-# 14. hpo_sweep.py exists and generates correct config list
-def test_hpo_sweep_configs():
-    import importlib.util
-    sweep_path = "scripts/hpo_sweep.py"
-    if not os.path.exists(sweep_path):
-        pytest.fail(f"{sweep_path} not found — implement it")
-    spec = importlib.util.spec_from_file_location("hpo_sweep", sweep_path)
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    cfgs = list(mod.configs())
-    assert len(cfgs) > 0, "configs() returned empty list"
-    # Every config must have at least D_MODEL, DEPTH, WINDOW keys
-    required = {"JEPA_D_MODEL", "JEPA_DEPTH", "JEPA_WINDOW"}
-    for cfg in cfgs:
-        assert required.issubset(cfg.keys()), f"config missing required keys: {cfg}"
-
-
-# ── Option B: joint encoder fine-tuning in phase-1 ───────────────────────────
-
-# 15. PHASE1_JOINT and PHASE1_ENCODER_LR knobs exist at module level
-def test_joint_phase1_knobs():
-    mod = _import({"JEPA_PHASE1_JOINT": "1", "JEPA_PHASE1_ENCODER_LR": "1e-5"})
-    assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing from train.py"
-    assert hasattr(mod, "PHASE1_ENCODER_LR"), "PHASE1_ENCODER_LR knob missing from train.py"
-    assert mod.PHASE1_JOINT is True
-    assert abs(mod.PHASE1_ENCODER_LR - 1e-5) < 1e-12
-
-
-# 16. PHASE1_JOINT defaults to True (joint mode on by default)
-def test_joint_phase1_default_on():
-    mod = _import()
-    assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing"
-    assert mod.PHASE1_JOINT is True, f"PHASE1_JOINT default should be True, got {mod.PHASE1_JOINT}"
-
-
-# 17. JEPA_PHASE1_JOINT=0 disables joint (env override works)
-def test_joint_phase1_can_disable():
-    mod = _import({"JEPA_PHASE1_JOINT": "0"})
-    assert mod.PHASE1_JOINT is False, f"expected False, got {mod.PHASE1_JOINT}"
-
-
-# 18. Encoder receives non-zero gradients when joint-training with the head
-def test_joint_encoder_grad_flows(train_mod):
-    """Gradient must flow into encoder when using two-param-group joint optimizer."""
-    import torch.nn.functional as F
-    enc  = train_mod.CausalEncoder(n_channels=2, patch_len=8, d_model=16, n_heads=2, depth=1)
-    head = train_mod.SupervisedHead(16)
-    enc.train(); head.train()
-    opt = torch.optim.Adam([
-        {"params": head.parameters(), "lr": 1e-3},
-        {"params": enc.parameters(), "lr": 1e-5},
-    ], weight_decay=1e-4)
-    # Tiny batch: 4 windows of length 16 (= 2 patches of patch_len=8)
-    X = torch.randn(4, 16, 2)
-    y = torch.randn(4)
-    tokens = enc(X)           # (4, 2, 16)
-    h      = tokens[:, -1, :] # (4, 16) — last token
-    pred   = head(h)
-    loss   = F.mse_loss(pred, y)
-    loss.backward()
-    enc_grads = [p.grad for p in enc.parameters() if p.grad is not None]
-    assert len(enc_grads) > 0, "no encoder params received gradients"
-    assert any(g.abs().max().item() > 0 for g in enc_grads), "all encoder grads are zero"

tests/test_multipair.py

@@ -1,126 +0,0 @@
-"""Tests for multi-pair G10 pipeline (Option C).
-
-Tests the prepare_multipair.py merge logic and train.py multipair build().
-Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_multipair.py -v
-"""
-import importlib.util
-import numpy as np
-import pandas as pd
-import pytest
-import os
-
-
-def _import_mp():
-    spec = importlib.util.spec_from_file_location("prepare_multipair", "scripts/prepare_multipair.py")
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
-
-
-@pytest.fixture(scope="module")
-def mp():
-    return _import_mp()
-
-
-def _pair_df(start: str, n_hours: int, seed: int) -> pd.DataFrame:
-    """Synthetic single-pair hourly parquet (same schema as prepare_hourly output)."""
-    rng = np.random.default_rng(seed)
-    dts = pd.date_range(start, periods=n_hours, freq="h")
-    closes = 1.1 + np.cumsum(rng.normal(0, 0.001, n_hours))
-    return pd.DataFrame({
-        "datetime":     dts,
-        "close":        closes,
-        "ret":          rng.normal(0, 0.001, n_hours),
-        "realized_vol": np.abs(rng.normal(0.0005, 0.0001, n_hours)),
-    })
-
-
-# 1. merge_pair_parquets returns inner join on datetime
-def test_merge_inner_join(mp):
-    eur = _pair_df("2020-01-01 00:00", 100, seed=1)  # t0 to t0+99h
-    gbp = _pair_df("2020-01-01 20:00", 60, seed=2)   # t0+20 to t0+79h → 60 common
-    result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
-    assert len(result) == 60, f"expected 60 (inner join), got {len(result)}"
-
-
-# 2. merge_pair_parquets prefixes columns with pair name
-def test_merge_column_prefixes(mp):
-    eur = _pair_df("2020-01-01 00:00", 50, seed=1)
-    gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
-    result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
-    assert "datetime" in result.columns, "datetime column missing"
-    assert "eurusd_ret" in result.columns
-    assert "eurusd_rv" in result.columns
-    assert "gbpusd_ret" in result.columns
-    assert "gbpusd_rv" in result.columns
-    # raw pair columns should not leak through unprefixed
-    assert "ret" not in result.columns
-    assert "realized_vol" not in result.columns
-
-
-# 3. No NaN in merged output
-def test_merge_no_nan(mp):
-    eur = _pair_df("2020-01-01 00:00", 50, seed=1)
-    gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
-    result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
-    nan_count = result.isnull().sum().sum()
-    assert nan_count == 0, f"{nan_count} NaN values in merged output"
-
-
-# 4. PAIRS constant is a non-empty list starting with eurusd
-def test_pairs_constant(mp):
-    assert hasattr(mp, "PAIRS"), "PAIRS constant missing from prepare_multipair.py"
-    assert len(mp.PAIRS) >= 2, "PAIRS must have at least 2 pairs"
-    assert mp.PAIRS[0] == "eurusd", "first pair must be eurusd (target pair)"
-
-
-# 5. merge target column is eurusd_rv (for build() target selection)
-def test_merge_has_eurusd_rv_as_target(mp):
-    eur = _pair_df("2020-01-01 00:00", 50, seed=1)
-    gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
-    result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
-    assert "eurusd_rv" in result.columns, "eurusd_rv (target) missing from merged output"
-    assert (result["eurusd_rv"] > 0).all(), "eurusd_rv should be positive"
-
-
-# 6. train.py recognises JEPA_USE_MULTIPAIR env var
-def test_use_multipair_knob():
-    import importlib.util as ilu
-    spec = ilu.spec_from_file_location(f"train_mp_{id(None)}", "train.py")
-    mod = ilu.module_from_spec(spec)
-    saved = os.environ.get("JEPA_USE_MULTIPAIR")
-    os.environ["JEPA_USE_MULTIPAIR"] = "1"
-    try:
-        spec.loader.exec_module(mod)
-    finally:
-        if saved is None:
-            os.environ.pop("JEPA_USE_MULTIPAIR", None)
-        else:
-            os.environ["JEPA_USE_MULTIPAIR"] = saved
-    assert hasattr(mod, "USE_MULTIPAIR"), "USE_MULTIPAIR knob missing from train.py"
-    assert mod.USE_MULTIPAIR is True
-
-
-# 7. build() uses n_pairs*2 channels when multipair parquet present
-def test_build_uses_multipair_channels():
-    import importlib.util as ilu
-    multipair_path = "data/processed/eurusd_multipair.parquet"
-    if not os.path.exists(multipair_path):
-        pytest.skip("eurusd_multipair.parquet not present — run data:prepare:multipair first")
-    saved = os.environ.get("JEPA_USE_MULTIPAIR")
-    os.environ["JEPA_USE_MULTIPAIR"] = "1"
-    try:
-        spec = ilu.spec_from_file_location(f"train_mp2_{id(None)}", "train.py")
-        mod = ilu.module_from_spec(spec)
-        spec.loader.exec_module(mod)
-        (Xtr, _), _ = mod.build()
-    finally:
-        if saved is None:
-            os.environ.pop("JEPA_USE_MULTIPAIR", None)
-        else:
-            os.environ["JEPA_USE_MULTIPAIR"] = saved
-    mp = _import_mp()
-    expected_ch = len(mp.PAIRS) * 2
-    assert Xtr.shape[2] == expected_ch, (
-        f"expected {expected_ch} channels (n_pairs={len(mp.PAIRS)}×2), got {Xtr.shape[2]}"
-    )

tests/test_prepare_hourly.py

@@ -1,207 +0,0 @@
-"""Failing tests for scripts/prepare_hourly.py.
-
-Tests the M1 → hourly aggregation logic using synthetic data before touching
-real downloads.
-
-Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_prepare_hourly.py -v
-"""
-import numpy as np
-import pandas as pd
-import pytest
-import importlib.util, sys, os
-
-
-def _import():
-    spec = importlib.util.spec_from_file_location(
-        "prepare_hourly", "scripts/prepare_hourly.py"
-    )
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
-
-
-@pytest.fixture(scope="module")
-def ph():
-    return _import()
-
-
-def _make_m1(n_days: int = 3, price: float = 1.1000, noise: float = 0.0005) -> pd.DataFrame:
-    """Synthetic M1 DataFrame starting 2020-01-06 (Monday), 390 ticks/day."""
-    rng = np.random.default_rng(42)
-    # generate full trading hours: Mon-Fri 00:00-23:59 (FX is 24h weekday)
-    start = pd.Timestamp("2020-01-06 00:00:00")  # Monday
-    periods = n_days * 24 * 60
-    ts = pd.date_range(start, periods=periods, freq="min")
-    # remove weekends
-    ts = ts[ts.day_of_week < 5]
-    prices = price + np.cumsum(rng.normal(0, noise, len(ts)))
-    return pd.DataFrame({"ts": ts, "close": prices})
-
-
-# 1. resample_to_hourly: DataFrame has correct columns
-def test_columns(ph):
-    m1 = _make_m1()
-    hourly = ph.resample_to_hourly(m1)
-    assert set(["datetime", "close", "ret", "realized_vol"]).issubset(hourly.columns), \
-        f"missing columns: {hourly.columns.tolist()}"
-
-
-# 2. No cross-weekend interpolation: gap between Friday 23:xx and Sunday/Monday must remain
-def test_no_weekend_interpolation(ph):
-    # Make 2 days: Friday + Monday (skip Saturday/Sunday)
-    fri = pd.date_range("2020-01-10 00:00", "2020-01-10 23:59", freq="min")  # Friday
-    mon = pd.date_range("2020-01-13 00:00", "2020-01-13 23:59", freq="min")  # Monday
-    ts = fri.append(mon)
-    prices = 1.1 + np.cumsum(np.random.default_rng(0).normal(0, 0.0001, len(ts)))
-    m1 = pd.DataFrame({"ts": ts, "close": prices})
-    hourly = ph.resample_to_hourly(m1)
-    dates = pd.DatetimeIndex(hourly["datetime"]).date
-    import datetime
-    sat = datetime.date(2020, 1, 11)
-    sun = datetime.date(2020, 1, 12)
-    assert sat not in dates and sun not in dates, "weekend rows found in hourly output"
-
-
-# 3. Realized vol = sqrt(sum(r²)) over minute returns in each hour
-def test_realized_vol_formula(ph):
-    # Two hours: anchor gives 10:00 a valid ret; measurement hour has one known log-return.
-    ts0 = pd.date_range("2020-01-06 09:00", periods=60, freq="min")
-    ts1 = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
-    prices0 = np.ones(60) * 1.0
-    # price jumps at minute 1 and STAYS (no reversion) → one non-zero log-return
-    prices1 = np.full(60, np.exp(0.01))
-    prices1[0] = 1.0  # only first tick is at 1.0; jump happens at tick 1
-    m1 = pd.DataFrame({
-        "ts": np.concatenate([ts0, ts1]),
-        "close": np.concatenate([prices0, prices1]),
-    })
-    hourly = ph.resample_to_hourly(m1)
-    assert len(hourly) >= 1, "no rows after resample"
-    rv = hourly.iloc[-1]["realized_vol"]
-    expected = np.sqrt(0.01 ** 2)
-    assert abs(rv - expected) < 1e-6, f"realized_vol={rv:.8f}, expected≈{expected:.8f}"
-
-
-# 4. Only hours with ≥ 30 M1 bars are kept (thin hours dropped)
-def test_thin_hours_dropped(ph):
-    # 4 hours: pre-anchor gives 09:00 a valid ret; full survives; thin (11:00) is dropped.
-    # pre-anchor (08:00): gives 09:00 a valid ret
-    # anchor (09:00): 60 bars, valid ret  → kept
-    # full  (10:00): 60 bars, valid ret  → kept
-    # thin  (11:00): 10 bars             → dropped
-    # Result: 3 hourly candidates, first (pre-anchor) gets NaN ret → dropped → 2 rows
-    pre   = pd.date_range("2020-01-06 08:00", periods=60, freq="min")
-    anchor= pd.date_range("2020-01-06 09:00", periods=60, freq="min")
-    full  = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
-    thin  = pd.date_range("2020-01-06 11:00", periods=10, freq="min")
-    ts = pre.append(anchor).append(full).append(thin)
-    m1 = pd.DataFrame({"ts": ts, "close": np.ones(len(ts)) * 1.1})
-    hourly = ph.resample_to_hourly(m1)
-    assert len(hourly) == 2, f"expected 2 rows (pre-anchor NaN ret dropped + thin dropped), got {len(hourly)}"
-
-
-# 5. Output parquet path and schema (integration — reads actual M1 zips if present)
-def test_output_schema_from_zips(ph, tmp_path):
-    import zipfile, io
-    rows = []
-    for h in range(24):
-        for m in range(60):
-            rows.append(f"20200106 {h:02d}{m:02d}00;1.10000;1.10100;1.09900;1.10000;100")
-    csv_content = "\n".join(rows).encode()
-    zip_buf = io.BytesIO()
-    with zipfile.ZipFile(zip_buf, "w") as zf:
-        zf.writestr("DAT_ASCII_EURUSD_M1_2020.csv", csv_content)
-    zip_buf.seek(0)
-    raw_dir = tmp_path / "raw"
-    raw_dir.mkdir()
-    (raw_dir / "DAT_ASCII_EURUSD_M1_2020.zip").write_bytes(zip_buf.read())
-
-    out_path = str(tmp_path / "eurusd_hourly.parquet")
-    ph.build_hourly_parquet(raw_dir=str(raw_dir), out_path=out_path)
-    assert os.path.exists(out_path), "output parquet not created"
-    df = pd.read_parquet(out_path)
-    assert set(["datetime", "close", "ret", "realized_vol"]).issubset(df.columns)
-    assert len(df) > 0
-
-
-# ── New OHLCV-derived features ────────────────────────────────────────────────
-
-def _make_m1_ohlcv(n_hours: int = 4, price: float = 1.1) -> pd.DataFrame:
-    """Synthetic M1 with distinct O, H, L, C so hl_range and ret_intrabar are nonzero."""
-    rng = np.random.default_rng(7)
-    ts = pd.date_range("2020-01-06 00:00", periods=n_hours * 60, freq="min")
-    closes = price + np.cumsum(rng.normal(0, 0.0002, len(ts)))
-    highs  = closes + rng.uniform(0.0001, 0.0005, len(ts))
-    lows   = closes - rng.uniform(0.0001, 0.0005, len(ts))
-    opens  = np.roll(closes, 1); opens[0] = price
-    return pd.DataFrame({"ts": ts, "open": opens, "high": highs, "low": lows, "close": closes})
-
-
-# 6. resample_to_hourly produces hl_range column
-def test_hourly_has_hl_range(ph):
-    m1 = _make_m1_ohlcv()
-    hourly = ph.resample_to_hourly(m1)
-    assert "hl_range" in hourly.columns, f"missing hl_range; cols={hourly.columns.tolist()}"
-    assert (hourly["hl_range"] > 0).all(), "hl_range should be positive"
-
-
-# 7. resample_to_hourly produces ret_intrabar column
-def test_hourly_has_ret_intrabar(ph):
-    m1 = _make_m1_ohlcv()
-    hourly = ph.resample_to_hourly(m1)
-    assert "ret_intrabar" in hourly.columns, f"missing ret_intrabar; cols={hourly.columns.tolist()}"
-
-
-# 8. hl_range = log(hourly_high / hourly_low)
-def test_hl_range_formula(ph):
-    # Two hours; second has known H=1.105, L=1.095
-    ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
-    ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
-    closes = np.full(120, 1.1)
-    highs  = np.full(120, 1.1)
-    lows   = np.full(120, 1.1)
-    # second hour: known spread
-    highs[60:] = 1.105
-    lows[60:]  = 1.095
-    m1 = pd.DataFrame({
-        "ts": np.concatenate([ts0, ts1]),
-        "open": closes, "high": highs, "low": lows, "close": closes,
-    })
-    hourly = ph.resample_to_hourly(m1)
-    assert len(hourly) >= 1
-    hl = hourly.iloc[-1]["hl_range"]
-    expected = float(np.log(1.105 / 1.095))
-    assert abs(hl - expected) < 1e-6, f"hl_range={hl:.8f}, expected={expected:.8f}"
-
-
-# 9. ret_intrabar = log(hourly_last_close / hourly_first_open)
-def test_ret_intrabar_formula(ph):
-    ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
-    ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
-    closes = np.full(120, 1.1)
-    opens  = np.full(120, 1.1)
-    # second hour: open=1.09, close=1.11
-    opens[60]   = 1.09
-    closes[119] = 1.11
-    m1 = pd.DataFrame({
-        "ts": np.concatenate([ts0, ts1]),
-        "open": opens, "high": closes + 0.001, "low": closes - 0.001, "close": closes,
-    })
-    hourly = ph.resample_to_hourly(m1)
-    assert len(hourly) >= 1
-    rib = hourly.iloc[-1]["ret_intrabar"]
-    expected = float(np.log(1.11 / 1.09))
-    assert abs(rib - expected) < 1e-6, f"ret_intrabar={rib:.8f}, expected={expected:.8f}"
-
-
-# 10. build() in train.py uses 2 feature channels (HPO: hl_range/ret_intrabar redundant)
-def test_build_uses_2_channels(tmp_path):
-    import importlib.util, os
-    hourly_path = "data/processed/eurusd_hourly.parquet"
-    if not os.path.exists(hourly_path):
-        pytest.skip("eurusd_hourly.parquet not present")
-    spec = importlib.util.spec_from_file_location("train_2ch", "train.py")
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    (Xtr, _), _ = mod.build()
-    assert Xtr.shape[2] == 2, f"expected 2 channels, got {Xtr.shape[2]}"

tests/test_regime.py

@@ -1,134 +0,0 @@
-"""Tests for scripts/prepare_regime.py — HMM regime detector (jepa-fx-risk#13).
-
-TDD: tests first, implementation follows.
-"""
-
-import importlib.util
-import os
-import shutil
-from pathlib import Path
-
-import numpy as np
-import pandas as pd
-import pytest
-
-_SCRIPT = Path(__file__).parent.parent / "scripts" / "prepare_regime.py"
-
-DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
-HOURLY = DATA_DIR / "eurusd_hourly.parquet"
-DAILY = DATA_DIR / "eurusd_daily.parquet"
-
-
-def _import():
-    spec = importlib.util.spec_from_file_location("prepare_regime", _SCRIPT)
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
-
-
-@pytest.fixture()
-def mod():
-    return _import()
-
-
-# ---------------------------------------------------------------------------
-# fit_regime_hmm — pure function (doesn't touch disk)
-# ---------------------------------------------------------------------------
-
-def _synthetic_rv(seed=42, size=500):
-    """Noisy 3-regime vol series: calm→stressed→crisis→calm interleaved."""
-    rng = np.random.default_rng(seed)
-    low  = np.abs(rng.normal(0.005, 0.001, size=size // 3))
-    mid  = np.abs(rng.normal(0.015, 0.003, size=size // 3))
-    high = np.abs(rng.normal(0.04,  0.008, size=size - 2 * (size // 3)))
-    return np.concatenate([low, mid, high])
-
-
-class TestFitRegimeHmm:
-    def test_returns_integer_labels(self, mod):
-        rv = _synthetic_rv(seed=0)
-        labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
-        assert np.issubdtype(labels.dtype, np.integer), f"dtype={labels.dtype}"
-        assert len(labels) == len(rv)
-
-    def test_states_are_0_1_2(self, mod):
-        rv = _synthetic_rv(seed=1)
-        labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
-        unique = set(labels.tolist())
-        assert unique.issubset({0, 1, 2}), f"unexpected states: {unique}"
-
-    def test_deterministic(self, mod):
-        rv = _synthetic_rv(seed=7)
-        a = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
-        b = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
-        assert np.array_equal(a, b), "HMM not deterministic with same random_state"
-
-    def test_sorted_by_vol_asc(self, mod):
-        # 3 clearly separated noisy clusters; state 0 should be calm, 2 should be crisis.
-        rng = np.random.default_rng(42)
-        n = 200
-        low  = np.abs(rng.normal(0.005, 0.001, n))
-        mid  = np.abs(rng.normal(0.015, 0.003, n))
-        high = np.abs(rng.normal(0.05,  0.008, n))
-        rv = np.concatenate([low, mid, high])
-        labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
-        # Mean regime label in the high-vol section should exceed mean in the low-vol section.
-        assert labels[2*n:].mean() > labels[:n].mean(), \
-            "crisis section mean regime label should exceed calm section"
-        # The calm section should not be labeled as crisis (2) dominantly.
-        calm_modal = int(np.bincount(labels[:n]).argmax())
-        assert calm_modal < 2, f"calm section mostly labeled {calm_modal}, expected 0 or 1"
-
-    def test_two_states(self, mod):
-        rv = _synthetic_rv(seed=0)
-        labels = mod.fit_regime_hmm(rv, n_states=2, random_state=42)
-        unique = set(labels.tolist())
-        assert unique.issubset({0, 1})
-
-
-
-
-# ---------------------------------------------------------------------------
-# prepare_regime_df — reads parquet, fits HMM, returns DataFrame
-# ---------------------------------------------------------------------------
-
-class TestPrepareRegimeDf:
-    @pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
-    def test_output_columns(self, mod):
-        df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
-        assert "datetime" in df.columns
-        assert "regime" in df.columns
-
-    @pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
-    def test_regime_values(self, mod):
-        df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
-        unique = set(df["regime"].tolist())
-        assert unique.issubset({0, 1, 2}), f"unexpected regime values: {unique}"
-
-    @pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
-    def test_no_nulls(self, mod):
-        df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
-        assert df["regime"].isna().sum() == 0
-
-    @pytest.mark.skipif(not DAILY.exists(), reason="daily parquet not available")
-    def test_daily_fallback(self, mod):
-        df = mod.prepare_regime_df(str(DAILY), freq="daily")
-        assert "regime" in df.columns
-        assert set(df["regime"].tolist()).issubset({0, 1, 2})
-
-
-# ---------------------------------------------------------------------------
-# Integration: check that train.py REGIME SEAM exists and is togglable
-# ---------------------------------------------------------------------------
-
-class TestTrainPyRegimeSeam:
-    def test_enable_regime_env_var_documented(self):
-        train_py = Path(__file__).parent.parent / "train.py"
-        content = train_py.read_text()
-        assert "JEPA_ENABLE_REGIME" in content, "JEPA_ENABLE_REGIME toggle not found in train.py"
-
-    def test_regime_seam_comment_present(self):
-        train_py = Path(__file__).parent.parent / "train.py"
-        content = train_py.read_text()
-        assert "REGIME" in content and "seam" in content.lower(), \
-            "agent-editable regime seam marker not found in train.py"

tests/test_var_metric.py

@@ -1,108 +0,0 @@
-"""Tests for scripts/var_breach.py — VaR breach rate + Kupiec POF (jepa-fx-risk#12).
-
-Golden tests first: verify the math before wiring it into train.py.
-"""
-
-import importlib.util
-import math
-from pathlib import Path
-
-import pytest
-
-_SCRIPT = Path(__file__).parent.parent / "scripts" / "var_breach.py"
-
-
-def _import():
-    spec = importlib.util.spec_from_file_location("var_breach", _SCRIPT)
-    mod = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(mod)
-    return mod
-
-
-@pytest.fixture()
-def mod():
-    return _import()
-
-
-# ---------------------------------------------------------------------------
-# var_breach_rate
-# ---------------------------------------------------------------------------
-
-class TestVarBreachRate:
-    def test_zero_breaches(self, mod):
-        # 0.02 < 0.01×2.326=0.02326 → no breach
-        rate, _ = mod.var_breach_rate([0.01, 0.01], [0.02, 0.02])
-        assert rate == 0.0
-
-    def test_all_breach(self, mod):
-        # 0.03 > 0.02326 → all breach
-        rate, _ = mod.var_breach_rate([0.01, 0.01], [0.03, 0.03])
-        assert rate == 1.0
-
-    def test_golden_two_of_ten(self, mod):
-        pred = [0.01] * 10
-        actual = [0.01] * 10
-        actual[0] = 0.03   # breach
-        actual[2] = 0.03   # breach
-        rate, kupiec_p = mod.var_breach_rate(pred, actual)
-        assert abs(rate - 0.2) < 1e-9, f"rate={rate}"
-        assert kupiec_p < 0.05, f"kupiec_p={kupiec_p}"  # strong reject
-
-    def test_perfect_calibration(self, mod):
-        # n=100, 1 breach → p_hat=0.01=p0=0.01 → LR=0 → kupiec_p≈1
-        pred = [0.01] * 100
-        actual = [0.015] * 100
-        actual[0] = 0.025  # 0.025 > 0.02326 → breach
-        rate, kupiec_p = mod.var_breach_rate(pred, actual)
-        assert abs(rate - 0.01) < 1e-9
-        assert kupiec_p > 0.9, f"kupiec_p={kupiec_p}"
-
-    def test_boundary_at_var_is_not_breach(self, mod):
-        # exactly at VaR_99 is NOT a breach (strict >)
-        z99 = 2.326
-        var = 0.01 * z99
-        rate, _ = mod.var_breach_rate([0.01], [var], z99=z99)
-        assert rate == 0.0
-
-    def test_empty_returns_zero_one(self, mod):
-        rate, kupiec_p = mod.var_breach_rate([], [])
-        assert rate == 0.0
-        assert kupiec_p == 1.0
-
-    def test_metric_key_no_whitespace(self, mod):
-        key = mod.METRIC_KEY
-        assert key == key.strip(), f"metric key has surrounding whitespace: {key!r}"
-        assert " " not in key, f"metric key contains space: {key!r}"
-
-    def test_metric_key_is_canonical(self, mod):
-        assert mod.METRIC_KEY == "VaR_breach_rate_99_oos_regime_cond"
-
-
-# ---------------------------------------------------------------------------
-# kupiec_pvalue
-# ---------------------------------------------------------------------------
-
-class TestKupiecPValue:
-    def test_perfectly_calibrated(self, mod):
-        # p_hat == p0 → LR=0 → p-value=1
-        p = mod.kupiec_pvalue(100, 1, p0=0.01)
-        assert p > 0.99, f"p={p}"
-
-    def test_strong_reject_high_breach(self, mod):
-        # 20% breach when 1% expected → p << 0.05
-        p = mod.kupiec_pvalue(100, 20, p0=0.01)
-        assert p < 0.001, f"p={p}"
-
-    def test_zero_breaches_not_nan(self, mod):
-        p = mod.kupiec_pvalue(100, 0, p0=0.01)
-        assert not math.isnan(p)
-        assert 0 <= p <= 1.0
-
-    def test_all_breaches_not_nan(self, mod):
-        p = mod.kupiec_pvalue(10, 10, p0=0.01)
-        assert not math.isnan(p)
-        assert p < 0.001  # extremely unlikely
-
-    def test_zero_observations(self, mod):
-        p = mod.kupiec_pvalue(0, 0)
-        assert p == 1.0

train.py

@@ -1,385 +0,0 @@
-"""train.py — autoresearch agent file (only this may be edited).
-
-HEPA backbone (Petersen et al., arXiv:2605.11130, ICML 2026 Spotlight):
-Causal Transformer pre-trained via horizon-conditioned JEPA. Predictor
-maps (h_t, Δt) → predicted future embedding; loss = VICReg (L1 alignment
-on L2-normalised reps + variance-covariance regulariser, no stop-gradient).
-Probe: ridge regression on the last-token embedding (true OOS split).
-
-Agent may tune: encoder depth/width, patch geometry, ALPHA, DELTA_T_MAX,
-optimizer, LR. Do NOT touch prepare_data.py, loop.py, or the data pipeline.
-"""
-import json
-import math
-import numpy as np
-import pandas as pd
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-
-# --- agent-tunable knobs (all overridable via JEPA_* env vars for HPO) ---
-import os as _os
-USE_HOURLY    = True
-WINDOW        = int(_os.environ.get("JEPA_WINDOW",        120))   # HPO winner: 5-day context
-PATCH_LEN     = int(_os.environ.get("JEPA_PATCH_LEN",     24))
-D_MODEL       = int(_os.environ.get("JEPA_D_MODEL",       128))
-DEPTH         = int(_os.environ.get("JEPA_DEPTH",         2))
-N_HEADS       = int(_os.environ.get("JEPA_N_HEADS",       4))
-ALPHA         = float(_os.environ.get("JEPA_ALPHA",       0.1))
-DELTA_T_MAX   = int(_os.environ.get("JEPA_DELTA_T_MAX",   3))
-BATCH_SIZE    = int(_os.environ.get("JEPA_BATCH_SIZE",    512))
-EPOCHS        = int(_os.environ.get("JEPA_EPOCHS",        300))
-LR            = float(_os.environ.get("JEPA_LR",          3e-4))
-PHASE1_EPOCHS      = int(_os.environ.get("JEPA_PHASE1_EPOCHS",       200))
-PHASE1_LR          = float(_os.environ.get("JEPA_PHASE1_LR",           1e-3))
-PHASE1_JOINT       = bool(int(_os.environ.get("JEPA_PHASE1_JOINT",     1)))
-PHASE1_JOINT_EPOCHS= int(_os.environ.get("JEPA_PHASE1_JOINT_EPOCHS",  30))
-PHASE1_ENCODER_LR  = float(_os.environ.get("JEPA_PHASE1_ENCODER_LR",  3e-6))
-USE_MULTIPAIR      = bool(int(_os.environ.get("JEPA_USE_MULTIPAIR",    0)))
-JEPA_ENABLE_REGIME = bool(int(_os.environ.get("JEPA_ENABLE_REGIME",    0)))
-SEED          = int(_os.environ.get("JEPA_SEED",          0))
-# ---------------------------
-
-torch.manual_seed(SEED)
-np.random.seed(SEED)
-dev = "cuda" if torch.cuda.is_available() else "cpu"
-
-
-# ── VICReg pretraining loss ──────────────────────────────────────────────────
-
-def vicreg_loss(h_pred: torch.Tensor, h_target: torch.Tensor, alpha: float = 0.1) -> torch.Tensor:
-    """L = (1-α)·L1(normalize(ĥ), normalize(h*)) + α·(L_var + L_cov).
-
-    Both encoders receive gradients (joint training — no stop-grad on h_target).
-    Variance-covariance terms prevent embedding collapse.
-    """
-    pred_n = F.normalize(h_pred, dim=-1)
-    targ_n = F.normalize(h_target, dim=-1)
-    l1 = F.l1_loss(pred_n, targ_n)
-    # variance hinge: push each feature std toward ≥ 1
-    std = h_pred.std(dim=0) + 1e-4
-    l_var = F.relu(1.0 - std).mean()
-    # covariance penalty: decorrelate features
-    B, D = h_pred.shape
-    h_c = h_pred - h_pred.mean(dim=0, keepdim=True)
-    cov = (h_c.t() @ h_c) / max(B - 1, 1)
-    off = cov - torch.diag(torch.diag(cov))
-    l_cov = (off ** 2).sum() / D
-    return (1 - alpha) * l1 + alpha * (l_var + l_cov)
-
-
-# ── CausalEncoder ─────────────────────────────────────────────────────────────
-
-class CausalEncoder(nn.Module):
-    """Non-overlapping patches → per-patch LayerNorm → causal Transformer → all tokens (B, N, D).
-
-    Per-patch LayerNorm instead of full-window RevIN: each patch is normalised
-    using only its own timesteps, so no future statistics leak into past tokens.
-    Use [:, -1, :] for probing (last token sees full context).
-    Use [:, c, :] for JEPA pretraining (context-at-c).
-    """
-    def __init__(self, n_channels: int, patch_len: int, d_model: int,
-                 n_heads: int, depth: int):
-        super().__init__()
-        self.patch_len = patch_len
-        self.d_model   = d_model
-        patch_dim = patch_len * n_channels
-        self.patch_norm = nn.LayerNorm(patch_dim)   # applied per-patch, no future leakage
-        self.embed     = nn.Linear(patch_dim, d_model)
-        layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
-                                           dropout=0.0, batch_first=True)
-        self.tf   = nn.TransformerEncoder(layer, num_layers=depth)
-        self.norm = nn.LayerNorm(d_model)
-
-    def forward(self, x: torch.Tensor) -> torch.Tensor:
-        B, W, F = x.shape
-        P = self.patch_len
-        N = W // P
-        tokens = x[:, :N * P, :].reshape(B, N, P * F)
-        tokens = self.embed(self.patch_norm(tokens))
-        # sinusoidal PE
-        pos = torch.arange(N, device=x.device).float()
-        div = torch.exp(torch.arange(0, self.d_model, 2, device=x.device).float()
-                        * -(math.log(10000.0) / self.d_model))
-        pe = torch.zeros(N, self.d_model, device=x.device)
-        pe[:, 0::2] = torch.sin(pos.unsqueeze(1) * div)
-        pe[:, 1::2] = torch.cos(pos.unsqueeze(1) * div)
-        tokens = tokens + pe
-        # causal mask
-        mask = nn.Transformer.generate_square_subsequent_mask(N, device=x.device)
-        return self.norm(self.tf(tokens, mask=mask, is_causal=True))
-
-
-# ── HorizonPredictor ─────────────────────────────────────────────────────────
-
-class HorizonPredictor(nn.Module):
-    """MLP(cat(h_t, Δt)) → predicted future embedding."""
-    def __init__(self, d_model: int):
-        super().__init__()
-        self.net = nn.Sequential(
-            nn.Linear(d_model + 1, d_model), nn.GELU(),
-            nn.Linear(d_model, d_model),     nn.GELU(),
-            nn.Linear(d_model, d_model),
-        )
-
-    def forward(self, h: torch.Tensor, delta_t: torch.Tensor) -> torch.Tensor:
-        dt = delta_t.float().unsqueeze(-1)
-        return self.net(torch.cat([h, dt], dim=-1))
-
-
-# ── Phase-1 supervised head ──────────────────────────────────────────────────
-
-class SupervisedHead(nn.Module):
-    """Small MLP trained on frozen HEPA embeddings to predict next-period realized vol."""
-    def __init__(self, d_model: int):
-        super().__init__()
-        self.net = nn.Sequential(
-            nn.Linear(d_model, d_model // 2), nn.GELU(),
-            nn.Linear(d_model // 2, 1),
-        )
-
-    def forward(self, h: torch.Tensor) -> torch.Tensor:
-        return self.net(h).squeeze(-1)
-
-
-# ── Data ─────────────────────────────────────────────────────────────────────
-
-def build():
-    """Year-based split: encoder trains on ≤2021; probe evaluates on ≥2022 OOS.
-
-    Uses eurusd_hourly.parquet when USE_HOURLY=True and the file exists;
-    falls back to eurusd_daily.parquet otherwise.
-    """
-    import os
-    multipair_path = "data/processed/eurusd_multipair.parquet"
-    hourly_path    = "data/processed/eurusd_hourly.parquet"
-    daily_path     = "data/processed/eurusd_daily.parquet"
-    if USE_MULTIPAIR and os.path.exists(multipair_path):
-        df = pd.read_parquet(multipair_path).reset_index(drop=True)
-        df["date"] = pd.to_datetime(df["datetime"])
-        # All {pair}_ret + {pair}_rv columns as features; eurusd_rv as target
-        feat_cols = [c for c in df.columns if c.endswith("_ret") or c.endswith("_rv")]
-        FEAT_COLS = feat_cols
-        target_col = "eurusd_rv"
-    elif USE_HOURLY and os.path.exists(hourly_path):
-        df = pd.read_parquet(hourly_path).reset_index(drop=True)
-        df["date"] = pd.to_datetime(df["datetime"])
-        # 2-channel default (HPO: adding hl_range+ret_intrabar hurt — correlated with base feats)
-        FEAT_COLS = ["ret", "realized_vol"]
-        target_col = "realized_vol"
-    else:
-        df = pd.read_parquet(daily_path).reset_index(drop=True)
-        df["date"] = pd.to_datetime(df["date"])
-        FEAT_COLS = ["ret", "realized_vol"]
-        target_col = "realized_vol"
-    # ── REGIME CONDITIONING SEAM — agent may vary this mechanism ─────────────
-    # Baseline: concat regime flag as an additional feature channel (0=calm, 2=crisis).
-    # Agent may swap for FiLM conditioning, learned regime embedding, or gating.
-    _regime_path = "data/processed/eurusd_regime.parquet"
-    if JEPA_ENABLE_REGIME and os.path.exists(_regime_path):
-        _rdf = pd.read_parquet(_regime_path)
-        _ts_col = "datetime" if "datetime" in _rdf.columns else "date"
-        _rdf[_ts_col] = pd.to_datetime(_rdf[_ts_col])
-        df = df.copy()
-        df = df.merge(
-            _rdf.rename(columns={_ts_col: "date"})[["date", "regime"]],
-            on="date", how="left",
-        )
-        df["regime"] = df["regime"].fillna(0).astype(np.float32)
-        FEAT_COLS = list(FEAT_COLS) + ["regime"]
-    # ── END REGIME SEAM ───────────────────────────────────────────────────────
-    feats  = df[FEAT_COLS].to_numpy(np.float32)
-    target = df[target_col].to_numpy(np.float32)
-    tr_idx = df.index[df["date"].dt.year <= 2021].tolist()
-    te_idx = df.index[df["date"].dt.year >= 2022].tolist()
-    mu = feats[:tr_idx[-1]+1].mean(0)
-    sd = feats[:tr_idx[-1]+1].std(0) + 1e-8
-    fn = (feats - mu) / sd
-    def windows(idx):
-        X, y = [], []
-        for t in idx:
-            if t - WINDOW >= 0 and t + 1 < len(df):
-                X.append(fn[t - WINDOW:t]); y.append(target[t + 1])
-        return np.stack(X).astype(np.float32), np.array(y, np.float32)
-    return windows(tr_idx), windows(te_idx)
-
-
-# ── Training ──────────────────────────────────────────────────────────────────
-
-def main():
-    (Xtr, ytr), (Xte, yte) = build()
-    n_feats   = Xtr.shape[2]
-    n_patches = WINDOW // PATCH_LEN
-    N_tr      = len(Xtr)
-    bs        = min(BATCH_SIZE, N_tr)
-
-    enc  = CausalEncoder(n_feats, PATCH_LEN, D_MODEL, N_HEADS, DEPTH).to(dev)
-    pred = HorizonPredictor(D_MODEL).to(dev)
-    opt  = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
-
-    for ep in range(EPOCHS):
-        # Random mini-batch (avoids OOM on large hourly dataset)
-        idx_b = torch.randperm(N_tr)[:bs]
-        Xb    = torch.tensor(Xtr[idx_b.numpy()], device=dev)
-
-        # Sample random context position and horizon
-        c  = torch.randint(0, n_patches - 1, ()).item()
-        dt = torch.randint(1, max(2, min(DELTA_T_MAX, n_patches - 1 - c) + 1), ()).item()
-
-        tokens = enc(Xb)                        # (bs, N, D)
-        h_ctx  = tokens[:, c, :]               # context embedding
-        h_tgt  = tokens[:, c + dt, :]          # target embedding (joint training)
-        h_hat  = pred(h_ctx, torch.full((bs,), float(dt), device=dev))
-        loss   = vicreg_loss(h_hat, h_tgt, alpha=ALPHA)
-        opt.zero_grad(); loss.backward(); opt.step()
-
-    enc.eval()
-    with torch.no_grad():
-        def embed(X_np):
-            chunks = []
-            for i in range(0, len(X_np), bs):
-                t = torch.tensor(X_np[i:i+bs], device=dev)
-                chunks.append(enc(t)[:, -1, :].cpu().numpy())
-            return np.concatenate(chunks, axis=0)
-
-        Etr = embed(Xtr)
-        Ete = embed(Xte)
-
-    # Ridge probe: fit on train, evaluate on OOS (true OOS R²)
-    mu_e = Etr.mean(0); sd_e = Etr.std(0) + 1e-8
-    Etr_n = (Etr - mu_e) / sd_e
-    Ete_n = (Ete - mu_e) / sd_e
-    A  = np.hstack([Etr_n, np.ones((len(Etr_n), 1))])
-    w  = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
-    pred_np = np.hstack([Ete_n, np.ones((len(Ete_n), 1))]) @ w
-    ss_res  = ((yte - pred_np) ** 2).sum()
-    ss_tot  = ((yte - yte.mean()) ** 2).sum()
-    val_vol_r2 = float(1 - ss_res / ss_tot)
-
-    # Phase-1: MLP supervised head — joint or frozen-encoder path
-    ytr_mu = float(ytr.mean()); ytr_sd = float(ytr.std()) + 1e-8
-    ytr_z  = (ytr - ytr_mu) / ytr_sd
-    head   = SupervisedHead(D_MODEL).to(dev)
-    p1_bs  = min(BATCH_SIZE, len(Etr_n))
-
-    # Shared tensors for the frozen-head warmup (used by both paths)
-    Etr_t   = torch.tensor(Etr_n, device=dev)
-    ytr_z_t = torch.tensor(ytr_z, device=dev)
-    Ete_t   = torch.tensor(Ete_n, device=dev)
-    N_tr_h  = len(Etr_t)
-
-    # Phase 1a: warm up head on frozen embeddings (both paths run this)
-    head_opt = torch.optim.Adam(head.parameters(), lr=PHASE1_LR, weight_decay=1e-4)
-    for _ in range(PHASE1_EPOCHS):
-        perm = torch.randperm(N_tr_h, device=dev)
-        for start in range(0, N_tr_h, p1_bs):
-            idx_h = perm[start:start + p1_bs]
-            loss_h = F.mse_loss(head(Etr_t[idx_h]), ytr_z_t[idx_h])
-            head_opt.zero_grad(); loss_h.backward(); head_opt.step()
-
-    if PHASE1_JOINT:
-        # Phase 1b: short joint fine-tuning — encoder nudged with tiny LR.
-        # Normalize live encoder output with FROZEN stats (mu_e, sd_e) so the
-        # head sees the same embedding distribution it was warmed up on.
-        enc.train()
-        mu_e_t = torch.tensor(mu_e, device=dev)
-        sd_e_t = torch.tensor(sd_e, device=dev)
-        Xtr_t  = torch.tensor(Xtr, device=dev)
-        joint_opt = torch.optim.Adam([
-            {"params": head.parameters(), "lr": PHASE1_LR * 0.1},
-            {"params": enc.parameters(),  "lr": PHASE1_ENCODER_LR},
-        ], weight_decay=1e-4)
-        for _ in range(PHASE1_JOINT_EPOCHS):
-            perm = torch.randperm(len(Xtr_t), device=dev)
-            for start in range(0, len(Xtr_t), p1_bs):
-                idx_j = perm[start:start + p1_bs]
-                h_raw = enc(Xtr_t[idx_j])[:, -1, :]
-                h_n   = (h_raw - mu_e_t) / sd_e_t  # frozen-stats normalisation
-                loss_j = F.mse_loss(head(h_n), ytr_z_t[idx_j])
-                joint_opt.zero_grad(); loss_j.backward(); joint_opt.step()
-        enc.eval()
-        # Re-extract test embeddings with fine-tuned encoder, same normalisation
-        with torch.no_grad():
-            chunks = []
-            for i in range(0, len(Xte), p1_bs):
-                t = torch.tensor(Xte[i:i+p1_bs], device=dev)
-                h = enc(t)[:, -1, :]
-                chunks.append(((h - mu_e_t) / sd_e_t).cpu().numpy())
-            Ete_t = torch.tensor(np.concatenate(chunks), device=dev)
-
-    head.eval()
-    with torch.no_grad():
-        pred_h_z = head(Ete_t).cpu().numpy()
-
-    pred_h    = pred_h_z * ytr_sd + ytr_mu          # de-standardise
-    phase1_r2 = float(1 - ((yte - pred_h) ** 2).sum() / ss_tot)
-    print("phase1_r2   = %.4f  (n_test=%d)" % (phase1_r2, len(yte)))
-
-    # ── VaR EVAL BLOCK — do NOT edit (agent boundary) ───────────────────────
-    import sys as _sys
-    _sys.path.insert(0, _os.path.dirname(_os.path.abspath(__file__)))
-    from scripts.var_breach import var_breach_rate as _var_breach_rate, METRIC_KEY as _VAR_KEY
-    _var_rate, _kupiec_p = _var_breach_rate(pred_np.tolist(), yte.tolist())
-    print("%s=%.4f  Kupiec_p=%.4f" % (_VAR_KEY, _var_rate, _kupiec_p))
-    # ── END VaR EVAL BLOCK ───────────────────────────────────────────────────
-
-    _metrics_out = _os.environ.get("METRICS_OUT", "metrics.json")
-    json.dump({
-        "val_vol_r2": val_vol_r2, "phase1_r2": phase1_r2, "n_test": len(yte),
-        _VAR_KEY: _var_rate, "kupiec_p": _kupiec_p,
-        "knobs": {"WINDOW": WINDOW, "PATCH_LEN": PATCH_LEN,
-                  "D_MODEL": D_MODEL, "DEPTH": DEPTH, "ALPHA": ALPHA,
-                  "DELTA_T_MAX": DELTA_T_MAX, "EPOCHS": EPOCHS},
-    }, open(_metrics_out, "w"), indent=2)
-    print("val_vol_r2 = %.4f  (n_test=%d, dev=%s)" % (val_vol_r2, len(yte), dev))
-
-    # ── EXPORT BLOCK — do NOT edit (agent boundary) ──────────────────────────
-    # Set EXPORT_EMBEDDINGS=1 to write embeddings.json for the Go eval harness.
-    import os
-    if os.environ.get("EXPORT_EMBEDDINGS") == "1":
-        hourly_path2 = "data/processed/eurusd_hourly.parquet"
-        daily_path2  = "data/processed/eurusd_daily.parquet"
-        if USE_HOURLY and os.path.exists(hourly_path2):
-            df2 = pd.read_parquet(hourly_path2).reset_index(drop=True)
-            df2["date"] = pd.to_datetime(df2["datetime"])
-        else:
-            df2 = pd.read_parquet(daily_path2).reset_index(drop=True)
-            df2["date"] = pd.to_datetime(df2["date"])
-        tr_mask = df2["date"].dt.year <= 2021
-        base2 = ["ret", "realized_vol"]
-        extra2 = [c for c in ["hl_range", "ret_intrabar"] if c in df2.columns]
-        feats2 = df2[base2 + extra2].to_numpy(np.float32)
-        mu2 = feats2[tr_mask].mean(0); sd2 = feats2[tr_mask].std(0) + 1e-8
-        fn2 = (feats2 - mu2) / sd2
-        def _export_windows(year_mask):
-            idx = df2.index[year_mask].tolist()
-            Xs, dates, rvs = [], [], []
-            for t in idx:
-                if t - WINDOW >= 0 and t + 1 < len(df2):
-                    Xs.append(fn2[t - WINDOW:t])
-                    dates.append(str(df2["date"].iloc[t].date()))
-                    rvs.append(float(df2["realized_vol"].iloc[t + 1]))
-            if not Xs:
-                return [], [], []
-            Xa = np.stack(Xs)
-            chunks = []
-            with torch.no_grad():
-                for i in range(0, len(Xa), bs):
-                    chunks.append(enc(torch.tensor(Xa[i:i+bs], device=dev))[:, -1, :].cpu().numpy())
-            E = np.concatenate(chunks, axis=0).tolist()
-            return E, dates, rvs
-        Etr2, dates_tr, rv_tr = _export_windows(tr_mask)
-        Eoos, dates_oos, rv_oos = _export_windows(df2["date"].dt.year >= 2022)
-        hv_thr = float(np.percentile(rv_oos, 67))
-        hv_label = [1 if v >= hv_thr else 0 for v in rv_oos]
-        json.dump({"embeddings": Eoos, "dates": dates_oos,
-                   "realized_vol": rv_oos, "hv_label": hv_label,
-                   "train_embeddings": Etr2, "train_realized_vol": rv_tr},
-                  open("embeddings.json", "w"))
-        print("exported embeddings.json  train=%d  oos=%d  HV=%d/%d" % (
-            len(Etr2), len(Eoos), sum(hv_label), len(hv_label)))
-    # ── END EXPORT BLOCK ─────────────────────────────────────────────────────
-
-
-if __name__ == "__main__":
-    main()