generated from mathias/template-go-web
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 mathias
|
50635cd80e | ||
|
mathias
|
a0d180e37d |
@@ -0,0 +1,2 @@
|
|||||||
|
read: .aider.conventions.md
|
||||||
|
auto-commits: false
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
# hostexecutor
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
|
- **Name**: hostexecutor
|
||||||
|
- **Owner**: Mathias
|
||||||
|
- **Client**: personal
|
||||||
|
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||||
|
- **Status**: active
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||||
@@ -3,29 +3,6 @@
|
|||||||
"knowledge": {
|
"knowledge": {
|
||||||
"url": "http://localhost:3100/mcp",
|
"url": "http://localhost:3100/mcp",
|
||||||
"description": "Project knowledge base — vector + graph retrieval"
|
"description": "Project knowledge base — vector + graph retrieval"
|
||||||
},
|
|
||||||
"brain": {
|
|
||||||
"type": "http",
|
|
||||||
"url": "https://koala.tail30472e.ts.net:8443/mcp",
|
|
||||||
"headers": {
|
|
||||||
"Authorization": "Bearer ${BRAIN_MCP_TOKEN}"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"gitea": {
|
|
||||||
"_comment": "gitea-mcp authenticates by PASS-THROUGH: the bearer is a Gitea PAT it validates against Gitea and then acts as. GITEA_MCP_TOKEN -- which this entry used until 2026-09-08 -- 401s against it, so every harness synced from this file had a dead gitea server and no signal saying so. Probed 2026-09-08: GITEA_MCP_TOKEN 401, DMABE_GITEA_API_TOKEN 200, on both the public and the Serve URL.",
|
|
||||||
"type": "http",
|
|
||||||
"url": "https://koala.tail30472e.ts.net:8446/mcp",
|
|
||||||
"headers": {
|
|
||||||
"Authorization": "Bearer ${DMABE_GITEA_API_TOKEN}"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"infra": {
|
|
||||||
"_comment": "The bearer is belt-and-braces: this endpoint answers 200 with or without it, since Serve already limits reach to the tailnet. Sending it anyway means the day the server starts enforcing is not the day every harness breaks.",
|
|
||||||
"type": "http",
|
|
||||||
"url": "https://koala.tail30472e.ts.net:8444/mcp",
|
|
||||||
"headers": {
|
|
||||||
"Authorization": "Bearer ${INFRA_MCP_TOKEN}"
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+6
-521
@@ -3,533 +3,18 @@ Follow all conventions from both the root agent context and project context.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Agent context — Mathias workspace
|
# hostexecutor
|
||||||
|
|
||||||
<!-- Canonical root context for all AI coding agents.
|
|
||||||
Lives at: ~/dev/.context/AGENT.md
|
|
||||||
Applies to every project under ~/dev/ unless overridden.
|
|
||||||
|
|
||||||
Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
|
|
||||||
Project-level context in .context/PROJECT.md layers on top of this. -->
|
|
||||||
|
|
||||||
## Who I am
|
|
||||||
|
|
||||||
I'm Mathias, a digital product manager and technology consultant based in Sweden.
|
|
||||||
I build software, research emerging tech, and deliver consulting engagements
|
|
||||||
for clients under NDA. I work across AI/ML, financial automation, web applications,
|
|
||||||
and climate/sustainability tech.
|
|
||||||
|
|
||||||
## How I work with agents
|
|
||||||
|
|
||||||
- I think like a product manager — I care about *why* before *how*
|
|
||||||
- I want agents to be opinionated and push back, not just execute blindly
|
|
||||||
- I prefer concise responses; skip ceremony and get to the point
|
|
||||||
- When I say "build this", I mean production-quality with tests, not a demo
|
|
||||||
- Ask me before making irreversible changes or adding heavy dependencies
|
|
||||||
- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
|
|
||||||
|
|
||||||
## Behavior rules
|
|
||||||
|
|
||||||
These rules apply to every task across every project, regardless of harness.
|
|
||||||
|
|
||||||
0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
|
|
||||||
- **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
|
|
||||||
- **Load the relevant skill** — see trigger table in *Engineering Skills* below.
|
|
||||||
- **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
|
|
||||||
- **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
|
|
||||||
|
|
||||||
**TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
|
|
||||||
|
|
||||||
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
|
|
||||||
Think before coding; if the problem is unclear, ask or state assumptions before acting.
|
|
||||||
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
|
|
||||||
speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
|
|
||||||
3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
|
|
||||||
files, and formatting alone. Diffs should be small and reviewable.
|
|
||||||
4. **Goal-driven execution.** Define clear success criteria up front for every task.
|
|
||||||
Loop — implement, verify, refine — until those criteria are met. Don't claim
|
|
||||||
completion without evidence (tests pass, command output, observed behavior).
|
|
||||||
5. **Trunk-Based Development — commit directly to main.** Every commit is one
|
|
||||||
logical change (one tool, one fix, one test) with passing tests. Main is always
|
|
||||||
deployable. Never create long-lived feature branches.
|
|
||||||
|
|
||||||
**The PR trigger is RISK, not concurrency.** `risk/low` work commits straight to
|
|
||||||
main with CI as the gate — that is most work and is unchanged. `risk/medium` and
|
|
||||||
above opens a PR, even solo, even with nobody waiting to review it. This mirrors
|
|
||||||
the tiered merge model in `assessor-loop`
|
|
||||||
`method/transfer-to-a-regulated-organisation.md` §4.1, so the estate practises
|
|
||||||
what it sells instead of merging everything on green and claiming otherwise.
|
|
||||||
|
|
||||||
**Concurrency is NOT a reason to branch.** Measured 2026-08-29 across three real
|
|
||||||
collisions between parallel sessions: two were not branching problems at all — a
|
|
||||||
generated artifact written to the same path by both (branching turns that into a
|
|
||||||
merge conflict on a 33k-line JSON), and uncommitted work in a shared checkout
|
|
||||||
(a per-session worktree fixes it) — and the third was an ordinary non-fast-forward
|
|
||||||
push resolved by rebase in seconds. Use a worktree per session on shared repos and
|
|
||||||
timestamp generated artifacts. Do not reach for a branch.
|
|
||||||
|
|
||||||
Branch+PR flow carries documented costs on solo repos: merging auto-closes gated
|
|
||||||
issues, review comments live on a thread separate from the issue's, and a
|
|
||||||
merged-and-deleted branch cannot be amended (brain:
|
|
||||||
`gitea-pr-merge-autoclose-and-review-comment-footguns`). Those are the price of
|
|
||||||
the `risk/medium`+ lane — paid deliberately, not incurred by accident.
|
|
||||||
|
|
||||||
**Exception — external contributor or client four-eyes requirement:** Use PR flow
|
|
||||||
when a human reviewer outside the project is required. Document the reason in
|
|
||||||
PROJECT.md.
|
|
||||||
|
|
||||||
**Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
|
|
||||||
deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
|
|
||||||
invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
|
|
||||||
`litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
|
|
||||||
PR would buy a review of a state that was never deployed.
|
|
||||||
|
|
||||||
There, `risk/medium`+ work stays on `main` and instead requires a **recorded
|
|
||||||
adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
|
|
||||||
reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
|
|
||||||
thread.** No verdict recorded means the lane was not used, and saying "the gate was
|
|
||||||
green" is not the same claim.
|
|
||||||
|
|
||||||
Be honest about what this is: a recorded *independent* pass, not four-eyes in the
|
|
||||||
sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
|
|
||||||
is an agent. It is more than the repo had and less than the model it mirrors; do not
|
|
||||||
describe it as the stronger property. (infra#470, decided 2026-09-29.)
|
|
||||||
|
|
||||||
This lane depends on the `risk/` label being right. A label that triggers a heavier
|
|
||||||
lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
|
|
||||||
lane silently does not apply.
|
|
||||||
|
|
||||||
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
|
||||||
the code is not the end of the task; capturing it is. Run this unprompted:
|
|
||||||
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
|
||||||
new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
|
|
||||||
actual last tag — stated versions in docs drift stale.
|
|
||||||
- **Push** main and the tag (CI is the gate).
|
|
||||||
- **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
|
|
||||||
the reusable patterns and the footguns that would bite anyone again, never
|
|
||||||
project status. See *Knowledge base — when to write* below.
|
|
||||||
- **File discovered-but-deferred work as tracker issues** on the project's own
|
|
||||||
repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
|
|
||||||
"out of scope, recorded" rot in a commit message; make it a ticket with a
|
|
||||||
source pointer.
|
|
||||||
- Surface the brain entries and issue numbers in the closing summary so the
|
|
||||||
trail is auditable.
|
|
||||||
|
|
||||||
**Chat is not a durable store. Never rely on Mathias remembering anything.**
|
|
||||||
Every gap, caveat, risk, open question and next step must land in one of four
|
|
||||||
places before the turn ends: **the brain**, **a repo** (code, docs, ADR),
|
|
||||||
**a tracker issue**, or **a plan file**. A finding stated only in conversation is
|
|
||||||
lost the moment the session closes — and this estate has repeatedly proved that a
|
|
||||||
remembered convention is an unreliable control.
|
|
||||||
|
|
||||||
This applies hardest to what is tempting to leave as a closing remark: "worth
|
|
||||||
knowing X isn't enforced yet", "this is a convention, not a gate", "we should
|
|
||||||
revisit Y". Those are exactly the items that need a locator. If it is worth
|
|
||||||
saying in a summary, it is worth writing where it can be found again without you.
|
|
||||||
|
|
||||||
7. **Finish what's closable — only involve me for a genuine call.** If a
|
|
||||||
tracker issue, PR, or checklist item is demonstrably resolved — the fix
|
|
||||||
landed, the state changed, you verified it live — close/merge/check it off
|
|
||||||
in the same turn. Don't leave it open "for the record" or park it waiting
|
|
||||||
on me to confirm something that already happened; that's bookkeeping, not
|
|
||||||
a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
|
|
||||||
comment, or otherwise pausing) for when it's genuinely mine to call:
|
|
||||||
irreversible or prod/host-impacting actions, a real judgment call with no
|
|
||||||
clearly-better option, ambiguous intent, or anything secret-writes/
|
|
||||||
destructive-git already gate. Before claiming something is closable,
|
|
||||||
verify it live (re-check the issue, re-run the command) rather than
|
|
||||||
trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
|
|
||||||
|
|
||||||
8. **Control code needs an adversary, not a checklist.** Code whose job is to
|
|
||||||
refuse, deny, gate or contain is not adequately verified by a green gate.
|
|
||||||
Rule 5 makes CI the quality gate and that holds for ordinary work — but a
|
|
||||||
passing suite only asserts what its author already thought of, and for a
|
|
||||||
control the author is precisely the person whose imagination just failed.
|
|
||||||
Measured on `dispatch#75`: five review rounds, five real escapes verified by
|
|
||||||
execution, and **CI was green on every one of them** while `git rm
|
|
||||||
././.dispatch-allow` deleted the file the control existed to protect. So
|
|
||||||
route this work through a review with authority to block, instructed to
|
|
||||||
verify by execution rather than by reading the implementation. Expect to be
|
|
||||||
blocked more than once; that is the control working, not the reviewer being
|
|
||||||
difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
|
|
||||||
practices live in the `pr-gatekeeper` agent.)
|
|
||||||
|
|
||||||
9. **A test's name must not claim more than its assertions prove.** Name a test
|
|
||||||
after the mechanism it observes, not the goal it serves.
|
|
||||||
`DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
|
|
||||||
support. This is not pedantry about naming — when name and assertion
|
|
||||||
diverge, the name wins in the reader's head and the gap becomes invisible,
|
|
||||||
which is what lets the defect survive review. Found three times in one PR,
|
|
||||||
twice in tests written to fix the previous instance of it; each passed while
|
|
||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
|
||||||
you do not yet know what the test observes.
|
|
||||||
|
|
||||||
## CAD sprint patterns
|
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
|
||||||
- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
|
|
||||||
- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
|
|
||||||
|
|
||||||
Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
|
|
||||||
|
|
||||||
## Default stack
|
|
||||||
|
|
||||||
| Layer | Default | Fallback | Last resort |
|
|
||||||
|-------|---------|----------|-------------|
|
|
||||||
| Language | Go | Python | TypeScript, Java, C |
|
|
||||||
| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
|
|
||||||
| Build | Task (taskfile.dev) | Make | — |
|
|
||||||
| Containers | Docker Compose (dev), k3s (prod) | — | — |
|
|
||||||
| DB | PostgreSQL + sqlc | SQLite | — |
|
|
||||||
| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
|
|
||||||
| Logging | slog (structured) | stdlib `logging` w/ structured `extra=` (or structlog) | — |
|
|
||||||
| Testing | Table-driven, testify | — | — |
|
|
||||||
| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
|
|
||||||
|
|
||||||
Exploratory: Rust, Zig — I'll tell you when I want these.
|
|
||||||
|
|
||||||
## Code conventions
|
|
||||||
|
|
||||||
- **Go style**: golines, gofumpt, golangci-lint
|
|
||||||
- **Python style** (fallback language): ruff (format+lint, one tool), mypy --strict (non-negotiable,
|
|
||||||
matches Go's static typing discipline), pytest + pytest-cov (table-driven via
|
|
||||||
`@pytest.mark.parametrize`), uv (venv+deps+lock, one tool), pydantic-settings (typed env-var config
|
|
||||||
— same principle as Go's typed structs), src-layout + `pyproject.toml` only (no `setup.py`)
|
|
||||||
- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return.
|
|
||||||
Python: `raise X from e` (exception chaining, same principle) — never bare `except`, never silent `pass`
|
|
||||||
- **Naming**: stdlib conventions, no stuttering
|
|
||||||
- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
|
|
||||||
- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
|
|
||||||
one logical change per commit, CI is the quality gate
|
|
||||||
- **Never**: long-lived feature branches, PRs for solo work, direct push without
|
|
||||||
passing `task check` locally first
|
|
||||||
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
|
|
||||||
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
|
|
||||||
|
|
||||||
## Secret handling (every harness, every command)
|
|
||||||
|
|
||||||
Tool output is persisted: terminal → `~/.claude/projects` transcripts →
|
|
||||||
claudewatcher → brain/wiki → gitea history. A secret printed once is
|
|
||||||
searchable forever, and clearing it means rotating the key. So:
|
|
||||||
|
|
||||||
1. **Never print, echo, log, or transform a secret to inspect it.** No
|
|
||||||
`base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
|
|
||||||
to defeat `op run`'s output masking (it masks raw values; base64 hides them
|
|
||||||
from the mask — that exact trick leaked a key on 2026-06-11).
|
|
||||||
2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
|
|
||||||
*inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
|
|
||||||
in a command's argv (it lands in the tool call and the transcript).
|
|
||||||
3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
|
|
||||||
never `${X:-...}` (returns the value when set) and never echo a substring of it.
|
|
||||||
4. **Cross-host secrets:** run the secret-consuming command on the host that has
|
|
||||||
the secret; do not forward a raw key over ssh argv/stdout.
|
|
||||||
5. If a secret does leak into output, say so immediately and flag it for rotation —
|
|
||||||
don't bury it.
|
|
||||||
|
|
||||||
## Infrastructure
|
|
||||||
|
|
||||||
Three machines on Tailscale:
|
|
||||||
|
|
||||||
| Machine | Role | Key specs |
|
|
||||||
|---------|------|-----------|
|
|
||||||
| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
|
|
||||||
| iguana | Services, builds | M2 Ultra Mac |
|
|
||||||
| flamingo | Daily driver, edge | Apple Silicon Mac, ~/dev is here |
|
|
||||||
|
|
||||||
- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
|
|
||||||
- **Orchestration**: k3s on koala only — single control-plane node, no workers (verified 2026-08-30). iguana and flamingo are on the tailnet but are not cluster members
|
|
||||||
- **Networking**: Tailscale mesh
|
|
||||||
|
|
||||||
## Project landscape
|
|
||||||
|
|
||||||
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
|
||||||
|
|
||||||
Organized in thematic folders:
|
|
||||||
|
|
||||||
| Folder | Focus | Count |
|
|
||||||
|--------|-------|-------|
|
|
||||||
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
|
|
||||||
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
|
|
||||||
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
|
|
||||||
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
|
|
||||||
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
|
|
||||||
|
|
||||||
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
|
|
||||||
|
|
||||||
### Key active projects
|
|
||||||
|
|
||||||
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
|
|
||||||
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
|
|
||||||
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
|
|
||||||
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
|
|
||||||
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
|
|
||||||
|
|
||||||
## Knowledge base — actively use it
|
|
||||||
|
|
||||||
A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
|
|
||||||
hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
|
|
||||||
Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
|
|
||||||
reference material — query it actively, not just when explicitly told.**
|
|
||||||
|
|
||||||
### When to query (treat as a reflex)
|
|
||||||
|
|
||||||
- **Before** starting a non-trivial task — search for prior art with the symptom
|
|
||||||
AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
|
|
||||||
- **When debugging** — search for the error string, the stack frame, the affected
|
|
||||||
service. Past you may have already paid this tax.
|
|
||||||
- **Before adopting** a pattern, library, framework, or model name — check if it
|
|
||||||
was tried and rejected, or what the integration footguns are.
|
|
||||||
- **When making architectural decisions** — search for the domain + "ADR" or
|
|
||||||
"decision" to find prior reasoning before re-deriving it.
|
|
||||||
- **When a recommendation feels novel** — challenge yourself: "has this been
|
|
||||||
documented?" The brain often has it.
|
|
||||||
|
|
||||||
### When to write
|
|
||||||
|
|
||||||
After you discover something that **future-you would forget** and that **isn't
|
|
||||||
recoverable from the code, git log, or PR description alone**:
|
|
||||||
|
|
||||||
- Bugs whose root cause is non-obvious and generalisable beyond this project.
|
|
||||||
- Framework / library / model-name quirks that bit you and would bite anyone.
|
|
||||||
- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
|
|
||||||
- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
|
|
||||||
|
|
||||||
DON'T write project status, sprint progress, PR summaries, or "what I did this
|
|
||||||
session" — those rot fast and the originals are in git/gitea anyway. Brain
|
|
||||||
entries that age well are about *why*, *how to avoid*, and *what to do when*.
|
|
||||||
|
|
||||||
### How to access (per harness)
|
|
||||||
|
|
||||||
| Harness | Query | Write |
|
|
||||||
|---------|-------|-------|
|
|
||||||
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
|
|
||||||
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
|
|
||||||
| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` |
|
|
||||||
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
|
||||||
|
|
||||||
- **Scoping — the brain holds no confidential client content.** Nothing on the read path
|
|
||||||
filters by client: every token holder can read every wing, and `wing` on `brain_query`
|
|
||||||
is an optional argument, not an access control. Client confidentiality is enforced only
|
|
||||||
at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
|
|
||||||
client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
|
|
||||||
decided 2026-10-01. This line previously claimed client projects were filtered; they
|
|
||||||
were not.)
|
|
||||||
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
|
||||||
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
|
||||||
on the koala k3s cluster; don't hardcode local-only model names into the
|
|
||||||
berget URL (see knowledge entry on namespace mismatches).
|
|
||||||
|
|
||||||
### Quick reflex checks
|
|
||||||
|
|
||||||
If you find yourself about to say any of these out loud, you owe yourself a brain query first:
|
|
||||||
|
|
||||||
- "I think the issue might be..."
|
|
||||||
- "Let me try X and see..."
|
|
||||||
- "I'll just write a script to..."
|
|
||||||
- "This is probably a new bug..."
|
|
||||||
- "Has anyone done this before?" — *yes, probably, go check.*
|
|
||||||
|
|
||||||
## Client work rules
|
|
||||||
|
|
||||||
When working on a project tagged with a client name:
|
|
||||||
1. Never send code, data, or context to cloud APIs — use local models only
|
|
||||||
2. Never reference other client projects or their data
|
|
||||||
3. Keep all artifacts within the client's git org / directory
|
|
||||||
4. Treat everything as confidential unless told otherwise
|
|
||||||
|
|
||||||
## Harness-agnostic principles
|
|
||||||
|
|
||||||
Supported harnesses (2026-09-01) — Cursor and Aider were dropped; do not
|
|
||||||
re-add adapters for them:
|
|
||||||
- Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush, Pi Coding Agent
|
|
||||||
- Open WebUI, Charmbracelet Mods, or any tool that takes a system prompt string
|
|
||||||
- Any other tool that reads `AGENTS.md`, which is the de facto convention
|
|
||||||
|
|
||||||
The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
|
|
||||||
Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
|
|
||||||
|
|
||||||
## How context propagates
|
|
||||||
|
|
||||||
Canonical sources of truth:
|
|
||||||
- Universal: `~/dev/.context/AGENT.md` (this file)
|
|
||||||
- Project: `<repo>/.context/PROJECT.md` (per-repo)
|
|
||||||
|
|
||||||
Derived files (committed, regenerated by `task context:sync`):
|
|
||||||
- `CLAUDE.md`, `AGENTS.md`, `.context/system-prompt.txt`
|
|
||||||
|
|
||||||
Repos still carrying `.cursorrules`, `.aider.conventions.md` or
|
|
||||||
`.aider.conf.yml` are stale: nothing regenerates them. Delete them the next time
|
|
||||||
you touch the repo, along with their entries in that repo's `task check`
|
|
||||||
assertion.
|
|
||||||
|
|
||||||
Workflow:
|
|
||||||
1. Edit a canonical file. Run `task context:sync`. Commit canonical and
|
|
||||||
derived together. Push.
|
|
||||||
2. On any other host, `git pull` brings both. Which file a harness reads does
|
|
||||||
NOT split cleanly by lookup strategy, so do not reason from the strategy:
|
|
||||||
- `CLAUDE.md` — Claude Code (walks up the tree)
|
|
||||||
- `AGENTS.md` — Crush, Pi, Antigravity (cwd-only), OpenCode (project root,
|
|
||||||
global at `~/.config/opencode/AGENTS.md`), and Mistral Vibe, which **walks
|
|
||||||
up** from cwd and takes the first `AGENTS.md` it finds
|
|
||||||
(user-level `~/.vibe/AGENTS.md`)
|
|
||||||
- `.context/system-prompt.txt` — Open WebUI, Mods, raw API calls; not read by
|
|
||||||
any harness, it is for pasting
|
|
||||||
|
|
||||||
Vibe tree-walks for `AGENTS.md`, so "tree-walkers use CLAUDE.md" is false.
|
|
||||||
Each repo's `AGENTS.md` embeds the canonical and is the first one found from
|
|
||||||
inside that repo, so this works — but by construction, not by luck.
|
|
||||||
3. `task check` runs `context:sync` then asserts `git status --porcelain`
|
|
||||||
is empty over the derived files (catches both modified-tracked drift
|
|
||||||
and missing-untracked adapters). A drift fails the check with a
|
|
||||||
message telling you to stage the regenerated files.
|
|
||||||
|
|
||||||
Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
|
|
||||||
host and every harness.
|
|
||||||
|
|
||||||
**Where a project rule contradicts a rule here, the project rule governs — and it
|
|
||||||
must state its reason.** Both halves matter: the project file wins because it knows
|
|
||||||
something this file cannot (infra's Flux constraint is real and not visible from
|
|
||||||
here), and it must say why because a bare override is indistinguishable from drift.
|
|
||||||
|
|
||||||
This replaces the previous wording, which said rules here and in `PROJECT.md` both
|
|
||||||
apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
|
|
||||||
unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
|
|
||||||
at the same time (infra#470). A standard can hold two individually sensible clauses
|
|
||||||
that are jointly unsatisfiable, and survive for as long as everyone reads it
|
|
||||||
charitably — the same failure the Inception Sprint Oath's G6 had.
|
|
||||||
|
|
||||||
## Engineering Skills
|
|
||||||
|
|
||||||
Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Load at task start — not "on demand" but on schedule, before writing code. Browse `SKILLS_INDEX.md` for the full list.
|
|
||||||
|
|
||||||
**Claude Code** installs the repo as a plugin from the marketplace the same repo hosts. That path also carries the `secret-guard` and `pre-task-ritual` hooks and the brain/gitea/infra MCP servers, which symlinks cannot:
|
|
||||||
|
|
||||||
```
|
|
||||||
/plugin marketplace add https://git.d-ma.be/mathias/skills.git
|
|
||||||
/plugin install skills@mathias
|
|
||||||
```
|
|
||||||
|
|
||||||
Skills are **namespaced** under Claude Code: load `tdd` as `skills:tdd`. The trigger table below uses bare names — prefix them.
|
|
||||||
|
|
||||||
**Crush, Pi and OpenCode** have no plugin system and are wired by `install.sh` as one directory symlink per harness via `~/.agents/skills`. Do not clone the checkout into `~/dev`: that folder is iCloud-shared between the Macs and spans two path roots (`/Users/mathias` vs `/home/mathias` on koala), so a checkout there is a git working tree with two concurrent writers, and any absolute symlink stored in it is broken on one side by construction (skills#23). Leave `SKILLS_CHECKOUT_DIR` at its host-local default.
|
|
||||||
|
|
||||||
Check a host with `task doctor` in the skills repo; it also runs automatically at session start via the plugin.
|
|
||||||
|
|
||||||
**Skill trigger table — load before starting, not after getting stuck:**
|
|
||||||
|
|
||||||
| Task type | Load |
|
|
||||||
|-----------|------|
|
|
||||||
| Any feature or bug fix | `tdd` |
|
|
||||||
| Refactor or design | `clean-code` or `solid` |
|
|
||||||
| Debug | `problem-analysis` |
|
|
||||||
| Review code or PRs | `code-review` |
|
|
||||||
| Frame a problem before coding | `problem-analysis` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# jepa-fx-risk
|
|
||||||
|
|
||||||
## Identity
|
## Identity
|
||||||
|
|
||||||
- **Name**: jepa-fx-risk
|
- **Name**: hostexecutor
|
||||||
- **Owner**: Mathias
|
- **Owner**: Mathias
|
||||||
- **Client**: personal research
|
- **Client**: personal
|
||||||
- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
|
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||||
- **Status**: active — Phase 0
|
- **Status**: active
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Research project: apply JEPA-based self-supervised representation learning to
|
|
||||||
FX risk management for a corporate bank with an internal global FX trading desk.
|
|
||||||
|
|
||||||
Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
|
|
||||||
Target output: internal PoC for the trading desk.
|
|
||||||
|
|
||||||
## Architecture decision (see DECISIONS.md ADR-001)
|
|
||||||
|
|
||||||
**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
|
|
||||||
arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
|
|
||||||
(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
|
|
||||||
|
|
||||||
Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
|
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
|
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||||
harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
|
|
||||||
results dashboard (Templ + HTMX + CDN Tailwind).
|
|
||||||
|
|
||||||
**Python** (`model/`): all training and embedding export only. PyTorch cu130
|
|
||||||
(Blackwell sm_120 compatible). No evaluation logic in Python.
|
|
||||||
|
|
||||||
**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
|
|
||||||
iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
|
|
||||||
|
|
||||||
## Repository layout
|
|
||||||
|
|
||||||
```
|
|
||||||
jepa-fx-risk/
|
|
||||||
├── src/ # Go — data pipeline + eval harness + dashboard
|
|
||||||
│ ├── data/ # DUKASCopy fetch, hourly processing, validation
|
|
||||||
│ └── eval/ # silhouette, linear probe, collapse, backtest
|
|
||||||
├── model/ # Python — training only
|
|
||||||
│ ├── train.py # TS-JEPA + SIGReg backbone (autoresearch edits this)
|
|
||||||
│ ├── prepare.py # LOCKED — data loading, tokenization, export
|
|
||||||
│ └── requirements.txt
|
|
||||||
├── specs/ # Research specs (one per phase/experiment type)
|
|
||||||
├── experiments/ # Per-run outputs: embeddings, metrics.json, git tag
|
|
||||||
├── results/summaries/ # Human-readable outcome per experiment
|
|
||||||
├── program.md # Autoresearch agenda — researcher edits this
|
|
||||||
├── DECISIONS.md # Architecture Decision Records
|
|
||||||
└── Taskfile.yml # task data:fetch, task experiment:run, task eval:*
|
|
||||||
```
|
|
||||||
|
|
||||||
## Phase structure
|
|
||||||
|
|
||||||
- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
|
|
||||||
Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
|
|
||||||
- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
|
|
||||||
Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
|
|
||||||
- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
|
|
||||||
- **Phase 3**: Internal bank tick/position data (future, out of current scope).
|
|
||||||
|
|
||||||
## Data
|
|
||||||
|
|
||||||
DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
|
|
||||||
Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
|
|
||||||
Weekend gaps handled explicitly. See ADR-002.
|
|
||||||
|
|
||||||
## Key conventions
|
|
||||||
|
|
||||||
- `prepare.py` is LOCKED — never modified by agents or autoresearch
|
|
||||||
- Evaluation metrics are always computed by the Go harness, never in Python
|
|
||||||
- Every experiment gets a git tag: `exp/YYYYMMDD-description`
|
|
||||||
- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
|
|
||||||
- `program.md` is the only file the researcher edits to steer autoresearch
|
|
||||||
- CI runs `task check` (lint + vet + test) only — no GPU, no training
|
|
||||||
|
|
||||||
## Evaluation metrics
|
|
||||||
|
|
||||||
Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
|
|
||||||
realized volatility from frozen embeddings, computed by Go harness.
|
|
||||||
|
|
||||||
Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
|
|
||||||
Must co-move with val_vol_r2 — checked from experiment 1.
|
|
||||||
|
|
||||||
Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
|
|
||||||
|
|
||||||
## Benchmarks to beat (trading desk comparison)
|
|
||||||
|
|
||||||
- GARCH(1,1) — volatility forecasting baseline
|
|
||||||
- Historical Simulation VaR (250-day rolling) — Basel default
|
|
||||||
- EWMA RiskMetrics (λ=0.94)
|
|
||||||
|
|
||||||
## Agent guidance
|
|
||||||
|
|
||||||
Read `DECISIONS.md` before making architecture suggestions.
|
|
||||||
Do not modify `prepare.py` or `src/eval/`.
|
|
||||||
Do not suggest changing the Python/Go separation.
|
|
||||||
Training runs are always manual via `task experiment:run` — never triggered by CI.
|
|
||||||
When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# Cursor rules — auto-generated
|
||||||
|
# Do not edit. Run: task context:sync
|
||||||
|
|
||||||
|
# hostexecutor
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
|
- **Name**: hostexecutor
|
||||||
|
- **Owner**: Mathias
|
||||||
|
- **Client**: personal
|
||||||
|
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||||
|
- **Status**: active
|
||||||
|
|
||||||
|
## Stack
|
||||||
|
|
||||||
|
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||||
@@ -1,89 +0,0 @@
|
|||||||
name: Autoresearch Loop
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_dispatch:
|
|
||||||
inputs:
|
|
||||||
fixture:
|
|
||||||
description: 'Fixture name in fixtures/ (without .json)'
|
|
||||||
required: true
|
|
||||||
default: 'phase-a-toy'
|
|
||||||
rq_id:
|
|
||||||
description: 'Run ID — defaults to fixture name if blank'
|
|
||||||
required: false
|
|
||||||
default: ''
|
|
||||||
iters:
|
|
||||||
description: 'Max iterations'
|
|
||||||
required: false
|
|
||||||
default: '3'
|
|
||||||
model:
|
|
||||||
description: 'LiteLLM model override (leave blank for default berget/gemma4-31b)'
|
|
||||||
required: false
|
|
||||||
default: ''
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
run:
|
|
||||||
name: Autoresearch — ${{ inputs.fixture }}
|
|
||||||
runs-on: self-hosted
|
|
||||||
timeout-minutes: 90
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Resolve run ID
|
|
||||||
id: vars
|
|
||||||
# signals#47: inputs and outputs derived from them reach the shell via
|
|
||||||
# env, never via ${{ }} expansion inside run: (a `"` or `$(...)` in an
|
|
||||||
# input would otherwise run as code).
|
|
||||||
env:
|
|
||||||
IN_RQ_ID: ${{ inputs.rq_id }}
|
|
||||||
IN_FIXTURE: ${{ inputs.fixture }}
|
|
||||||
run: |
|
|
||||||
RQ_ID="$IN_RQ_ID"
|
|
||||||
[ -z "$RQ_ID" ] && RQ_ID="$IN_FIXTURE"
|
|
||||||
echo "rq_id=$RQ_ID" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Clean stale run dir
|
|
||||||
env:
|
|
||||||
RQ_ID: ${{ steps.vars.outputs.rq_id }}
|
|
||||||
run: rm -rf "runs/$RQ_ID"
|
|
||||||
|
|
||||||
- name: Set up Python venv
|
|
||||||
run: |
|
|
||||||
[ -d .venv ] || python3 -m venv .venv
|
|
||||||
# torch must come from the cu130 wheel index (koala Blackwell sm_120);
|
|
||||||
# requirements.txt deliberately excludes it. Install it first.
|
|
||||||
.venv/bin/pip install -q torch --index-url https://download.pytorch.org/whl/cu130
|
|
||||||
.venv/bin/pip install -q -r requirements.txt
|
|
||||||
|
|
||||||
- name: Scaffold run dir
|
|
||||||
env:
|
|
||||||
IN_FIXTURE: ${{ inputs.fixture }}
|
|
||||||
RQ_ID: ${{ steps.vars.outputs.rq_id }}
|
|
||||||
run: |
|
|
||||||
.venv/bin/python scripts/autoresearch_start.py \
|
|
||||||
"fixtures/$IN_FIXTURE.json" \
|
|
||||||
"$RQ_ID"
|
|
||||||
|
|
||||||
- name: Run autoresearch loop
|
|
||||||
env:
|
|
||||||
LITELLM_KEY: ${{ secrets.LITELLM_KEY }}
|
|
||||||
LITELLM_BASE: ${{ secrets.LITELLM_BASE }}
|
|
||||||
NTFY_URL: ${{ secrets.NTFY_URL }}
|
|
||||||
RQ_ID: ${{ steps.vars.outputs.rq_id }}
|
|
||||||
IN_ITERS: ${{ inputs.iters }}
|
|
||||||
IN_MODEL: ${{ inputs.model }}
|
|
||||||
run: |
|
|
||||||
ARGS=(--run-dir "runs/$RQ_ID" --iters "$IN_ITERS")
|
|
||||||
[ -n "$IN_MODEL" ] && ARGS+=(--model "$IN_MODEL")
|
|
||||||
.venv/bin/python loop.py "${ARGS[@]}"
|
|
||||||
|
|
||||||
- name: Upload run artifacts
|
|
||||||
if: always()
|
|
||||||
uses: https://gitea.com/actions/upload-artifact@v3
|
|
||||||
with:
|
|
||||||
name: run-${{ steps.vars.outputs.rq_id }}-${{ github.run_number }}
|
|
||||||
path: |
|
|
||||||
runs/${{ steps.vars.outputs.rq_id }}/STATUS.md
|
|
||||||
runs/${{ steps.vars.outputs.rq_id }}/metrics.json
|
|
||||||
runs/${{ steps.vars.outputs.rq_id }}/program.md
|
|
||||||
retention-days: 30
|
|
||||||
@@ -7,6 +7,9 @@ on:
|
|||||||
pull_request:
|
pull_request:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
|
|
||||||
|
env:
|
||||||
|
IMAGE: hostexecutor
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
check:
|
check:
|
||||||
name: Lint / Test / Vet
|
name: Lint / Test / Vet
|
||||||
@@ -28,3 +31,85 @@ jobs:
|
|||||||
|
|
||||||
- name: Run checks
|
- name: Run checks
|
||||||
run: task check
|
run: task check
|
||||||
|
|
||||||
|
build:
|
||||||
|
name: Build & Import
|
||||||
|
needs: check
|
||||||
|
runs-on: self-hosted
|
||||||
|
if: github.event_name != 'pull_request'
|
||||||
|
outputs:
|
||||||
|
image-tag: ${{ steps.meta.outputs.sha-tag }}
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Derive image tags
|
||||||
|
id: meta
|
||||||
|
run: |
|
||||||
|
SHA=$(git rev-parse --short HEAD)
|
||||||
|
echo "sha-tag=${SHA}" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Build and push to local registry
|
||||||
|
run: |
|
||||||
|
REGISTRY="localhost:5000"
|
||||||
|
REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
|
||||||
|
buildah build \
|
||||||
|
--label "org.opencontainers.image.revision=${{ github.sha }}" \
|
||||||
|
-t ${REF} \
|
||||||
|
-t ${REGISTRY}/${{ env.IMAGE }}:latest \
|
||||||
|
.
|
||||||
|
buildah push --tls-verify=false ${REF}
|
||||||
|
buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
|
||||||
|
echo "✓ Image pushed to ${REF}"
|
||||||
|
|
||||||
|
deploy:
|
||||||
|
name: Deploy via GitOps
|
||||||
|
needs: build
|
||||||
|
runs-on: self-hosted
|
||||||
|
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
|
||||||
|
steps:
|
||||||
|
- name: Update image tag in infra repo
|
||||||
|
env:
|
||||||
|
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
|
||||||
|
DEPLOY_KEY: ${{ secrets.INFRA_DEPLOY_KEY }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
mkdir -p ~/.ssh
|
||||||
|
echo "$DEPLOY_KEY" > ~/.ssh/id_infra
|
||||||
|
chmod 600 ~/.ssh/id_infra
|
||||||
|
ssh-keyscan -p 30022 10.0.1.20 >> ~/.ssh/known_hosts 2>/dev/null
|
||||||
|
export GIT_SSH_COMMAND="ssh -i ~/.ssh/id_infra -o IdentitiesOnly=yes"
|
||||||
|
rm -rf /tmp/infra
|
||||||
|
git clone -b main ssh://git@10.0.1.20:30022/mathias/infra.git /tmp/infra
|
||||||
|
cd /tmp/infra
|
||||||
|
DEPLOYMENT="k3s/apps/hostexecutor/deployment.yaml"
|
||||||
|
sed -i "s|image: localhost:5000/hostexecutor:.*|image: localhost:5000/hostexecutor:${IMAGE_TAG}|" "$DEPLOYMENT"
|
||||||
|
grep -q "localhost:5000/hostexecutor:${IMAGE_TAG}" "$DEPLOYMENT" \
|
||||||
|
|| { echo "✗ image tag patch failed"; exit 1; }
|
||||||
|
if git diff --quiet "$DEPLOYMENT"; then
|
||||||
|
echo "ℹ image tag unchanged — skipping push"
|
||||||
|
else
|
||||||
|
git -c user.name="hostexecutor CI" \
|
||||||
|
-c user.email="ci@hostexecutor.local" \
|
||||||
|
commit -m "chore(deploy): hostexecutor → ${IMAGE_TAG}" "$DEPLOYMENT"
|
||||||
|
git push origin main
|
||||||
|
echo "✓ pushed to infra repo"
|
||||||
|
fi
|
||||||
|
shred -u ~/.ssh/id_infra
|
||||||
|
|
||||||
|
- name: Trigger Flux reconcile
|
||||||
|
run: |
|
||||||
|
kubectl -n flux-system annotate gitrepository flux-system \
|
||||||
|
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
|
||||||
|
kubectl -n flux-system annotate kustomization apps \
|
||||||
|
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
|
||||||
|
|
||||||
|
- name: Verify rollout
|
||||||
|
run: |
|
||||||
|
kubectl rollout status deployment/hostexecutor \
|
||||||
|
--namespace hostexecutor \
|
||||||
|
--timeout=120s \
|
||||||
|
|| {
|
||||||
|
kubectl get pods -n hostexecutor -o wide
|
||||||
|
kubectl get events -n hostexecutor --sort-by='.lastTimestamp' | tail -20
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|||||||
-23
@@ -28,26 +28,3 @@ go.work.sum
|
|||||||
# Project-specific
|
# Project-specific
|
||||||
bin/
|
bin/
|
||||||
*.templ.go
|
*.templ.go
|
||||||
|
|
||||||
# python venv (autoresearch loop)
|
|
||||||
.venv/
|
|
||||||
|
|
||||||
# downloaded + processed market data (track via DVC/MinIO, #10 — not git)
|
|
||||||
data/
|
|
||||||
|
|
||||||
# autoresearch run dirs (ephemeral; each scaffold rebuilds from fixtures/)
|
|
||||||
runs/
|
|
||||||
|
|
||||||
# ephemeral experiment outputs (generated by train.py / loop.py)
|
|
||||||
metrics.json
|
|
||||||
embeddings.json
|
|
||||||
HEARTBEAT
|
|
||||||
STATUS.md
|
|
||||||
|
|
||||||
# python caches
|
|
||||||
__pycache__/
|
|
||||||
*.pyc
|
|
||||||
.pytest_cache/
|
|
||||||
|
|
||||||
# built Go binaries
|
|
||||||
eval
|
|
||||||
|
|||||||
@@ -1,27 +0,0 @@
|
|||||||
# gitleaks config for the jepa-fx-risk repo (infra#317).
|
|
||||||
#
|
|
||||||
# Runs gitleaks' full default ruleset and allowlists one non-credential
|
|
||||||
# constant, scoped by BOTH path and pattern (condition = "AND") so a different
|
|
||||||
# secret in the same files still fires. Allowlist by shape, not by filename.
|
|
||||||
|
|
||||||
title = "jepa-fx-risk gitleaks config"
|
|
||||||
|
|
||||||
[extend]
|
|
||||||
useDefault = true
|
|
||||||
|
|
||||||
# METRIC_KEY is the metrics-pipeline identifier
|
|
||||||
# ("VaR_breach_rate_99_oos_regime_cond") that train.py's VaR eval block writes
|
|
||||||
# into metrics.json -- documented in var_breach.py's own comment. It trips
|
|
||||||
# generic-api-key purely because the variable name ends in _KEY and the value
|
|
||||||
# is long and mixed-case enough to pass the entropy heuristic. It is not a
|
|
||||||
# credential and has no secret analogue.
|
|
||||||
[[allowlists]]
|
|
||||||
description = "infra#317 — METRIC_KEY is a metrics identifier, not a credential"
|
|
||||||
# infra#344: `paths` removed. gitleaks ORs the clauses of a GLOBAL allowlist and
|
|
||||||
# does not honour `condition = "AND"` there, so the path matched on its own and
|
|
||||||
# granted the whole file immunity — every finding in it, not just this one.
|
|
||||||
# Verified: a live-shaped credential planted in that file scanned clean before
|
|
||||||
# this change and fires after it. The regex below is what was meant and works
|
|
||||||
# regardless of which file the value appears in (infra#256).
|
|
||||||
regexTarget = "line"
|
|
||||||
regexes = ['''METRIC_KEY''']
|
|
||||||
@@ -1,52 +0,0 @@
|
|||||||
# Canonical golangci-lint configuration for the estate's Go repos (infra#309
|
|
||||||
# pattern: one canonical file here, a byte-identical copy in each adopter,
|
|
||||||
# and the drift gate in THIS repo comparing sha256).
|
|
||||||
#
|
|
||||||
# Why this exists: 26 Go repos, 3 had a lint config, and all three were
|
|
||||||
# different. The 23 without one ran golangci-lint's v2 default set, which
|
|
||||||
# does NOT include a formatter -- so gofmt/gofumpt drift went uncaught until
|
|
||||||
# it was found by hand (dma-cli#12: two files drifted, nothing flagged them).
|
|
||||||
#
|
|
||||||
# The two configs this replaces (cobalt-dingo, coo-agent) were near-identical
|
|
||||||
# copies of each other AND both invalid under v2: they carried v1-style
|
|
||||||
# `linters-settings` and `issues.exclude-rules` keys, which `golangci-lint
|
|
||||||
# config verify` rejects outright. Their revive settings and test exclusions
|
|
||||||
# were therefore not in effect. Consolidating fixes that too.
|
|
||||||
#
|
|
||||||
# Per-repo variation does NOT belong here -- this file must stay byte-identical
|
|
||||||
# across adopters so the gate is a checksum, not a diff a human has to judge.
|
|
||||||
# A repo with a genuine exception (agentsquad excludes its seeded-bug fixture
|
|
||||||
# tree, which exists precisely to contain broken code) keeps its own config and
|
|
||||||
# is not listed as an adopter.
|
|
||||||
|
|
||||||
version: "2"
|
|
||||||
|
|
||||||
linters:
|
|
||||||
default: none
|
|
||||||
enable:
|
|
||||||
- errcheck # unchecked errors
|
|
||||||
- govet # suspicious constructs
|
|
||||||
- staticcheck # broad static analysis
|
|
||||||
- unused # unused code
|
|
||||||
- misspell # spelling
|
|
||||||
- revive # idiomatic Go
|
|
||||||
- exhaustive # exhaustive switch on enums
|
|
||||||
|
|
||||||
settings:
|
|
||||||
revive:
|
|
||||||
rules:
|
|
||||||
- name: var-naming
|
|
||||||
severity: warning
|
|
||||||
|
|
||||||
exclusions:
|
|
||||||
rules:
|
|
||||||
# Test files: table-driven cases legitimately carry unused struct fields
|
|
||||||
# and long functions.
|
|
||||||
- path: _test\.go
|
|
||||||
linters:
|
|
||||||
- revive
|
|
||||||
- unused
|
|
||||||
|
|
||||||
formatters:
|
|
||||||
enable:
|
|
||||||
- gofumpt
|
|
||||||
@@ -1,528 +1,13 @@
|
|||||||
# Agent context — Mathias workspace
|
# hostexecutor
|
||||||
|
|
||||||
<!-- Canonical root context for all AI coding agents.
|
|
||||||
Lives at: ~/dev/.context/AGENT.md
|
|
||||||
Applies to every project under ~/dev/ unless overridden.
|
|
||||||
|
|
||||||
Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
|
|
||||||
Project-level context in .context/PROJECT.md layers on top of this. -->
|
|
||||||
|
|
||||||
## Who I am
|
|
||||||
|
|
||||||
I'm Mathias, a digital product manager and technology consultant based in Sweden.
|
|
||||||
I build software, research emerging tech, and deliver consulting engagements
|
|
||||||
for clients under NDA. I work across AI/ML, financial automation, web applications,
|
|
||||||
and climate/sustainability tech.
|
|
||||||
|
|
||||||
## How I work with agents
|
|
||||||
|
|
||||||
- I think like a product manager — I care about *why* before *how*
|
|
||||||
- I want agents to be opinionated and push back, not just execute blindly
|
|
||||||
- I prefer concise responses; skip ceremony and get to the point
|
|
||||||
- When I say "build this", I mean production-quality with tests, not a demo
|
|
||||||
- Ask me before making irreversible changes or adding heavy dependencies
|
|
||||||
- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
|
|
||||||
|
|
||||||
## Behavior rules
|
|
||||||
|
|
||||||
These rules apply to every task across every project, regardless of harness.
|
|
||||||
|
|
||||||
0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
|
|
||||||
- **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
|
|
||||||
- **Load the relevant skill** — see trigger table in *Engineering Skills* below.
|
|
||||||
- **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
|
|
||||||
- **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
|
|
||||||
|
|
||||||
**TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
|
|
||||||
|
|
||||||
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
|
|
||||||
Think before coding; if the problem is unclear, ask or state assumptions before acting.
|
|
||||||
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
|
|
||||||
speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
|
|
||||||
3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
|
|
||||||
files, and formatting alone. Diffs should be small and reviewable.
|
|
||||||
4. **Goal-driven execution.** Define clear success criteria up front for every task.
|
|
||||||
Loop — implement, verify, refine — until those criteria are met. Don't claim
|
|
||||||
completion without evidence (tests pass, command output, observed behavior).
|
|
||||||
5. **Trunk-Based Development — commit directly to main.** Every commit is one
|
|
||||||
logical change (one tool, one fix, one test) with passing tests. Main is always
|
|
||||||
deployable. Never create long-lived feature branches.
|
|
||||||
|
|
||||||
**The PR trigger is RISK, not concurrency.** `risk/low` work commits straight to
|
|
||||||
main with CI as the gate — that is most work and is unchanged. `risk/medium` and
|
|
||||||
above opens a PR, even solo, even with nobody waiting to review it. This mirrors
|
|
||||||
the tiered merge model in `assessor-loop`
|
|
||||||
`method/transfer-to-a-regulated-organisation.md` §4.1, so the estate practises
|
|
||||||
what it sells instead of merging everything on green and claiming otherwise.
|
|
||||||
|
|
||||||
**Concurrency is NOT a reason to branch.** Measured 2026-08-29 across three real
|
|
||||||
collisions between parallel sessions: two were not branching problems at all — a
|
|
||||||
generated artifact written to the same path by both (branching turns that into a
|
|
||||||
merge conflict on a 33k-line JSON), and uncommitted work in a shared checkout
|
|
||||||
(a per-session worktree fixes it) — and the third was an ordinary non-fast-forward
|
|
||||||
push resolved by rebase in seconds. Use a worktree per session on shared repos and
|
|
||||||
timestamp generated artifacts. Do not reach for a branch.
|
|
||||||
|
|
||||||
Branch+PR flow carries documented costs on solo repos: merging auto-closes gated
|
|
||||||
issues, review comments live on a thread separate from the issue's, and a
|
|
||||||
merged-and-deleted branch cannot be amended (brain:
|
|
||||||
`gitea-pr-merge-autoclose-and-review-comment-footguns`). Those are the price of
|
|
||||||
the `risk/medium`+ lane — paid deliberately, not incurred by accident.
|
|
||||||
|
|
||||||
**Exception — external contributor or client four-eyes requirement:** Use PR flow
|
|
||||||
when a human reviewer outside the project is required. Document the reason in
|
|
||||||
PROJECT.md.
|
|
||||||
|
|
||||||
**Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
|
|
||||||
deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
|
|
||||||
invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
|
|
||||||
`litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
|
|
||||||
PR would buy a review of a state that was never deployed.
|
|
||||||
|
|
||||||
There, `risk/medium`+ work stays on `main` and instead requires a **recorded
|
|
||||||
adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
|
|
||||||
reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
|
|
||||||
thread.** No verdict recorded means the lane was not used, and saying "the gate was
|
|
||||||
green" is not the same claim.
|
|
||||||
|
|
||||||
Be honest about what this is: a recorded *independent* pass, not four-eyes in the
|
|
||||||
sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
|
|
||||||
is an agent. It is more than the repo had and less than the model it mirrors; do not
|
|
||||||
describe it as the stronger property. (infra#470, decided 2026-09-29.)
|
|
||||||
|
|
||||||
This lane depends on the `risk/` label being right. A label that triggers a heavier
|
|
||||||
lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
|
|
||||||
lane silently does not apply.
|
|
||||||
|
|
||||||
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
|
||||||
the code is not the end of the task; capturing it is. Run this unprompted:
|
|
||||||
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
|
||||||
new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
|
|
||||||
actual last tag — stated versions in docs drift stale.
|
|
||||||
- **Push** main and the tag (CI is the gate).
|
|
||||||
- **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
|
|
||||||
the reusable patterns and the footguns that would bite anyone again, never
|
|
||||||
project status. See *Knowledge base — when to write* below.
|
|
||||||
- **File discovered-but-deferred work as tracker issues** on the project's own
|
|
||||||
repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
|
|
||||||
"out of scope, recorded" rot in a commit message; make it a ticket with a
|
|
||||||
source pointer.
|
|
||||||
- Surface the brain entries and issue numbers in the closing summary so the
|
|
||||||
trail is auditable.
|
|
||||||
|
|
||||||
**Chat is not a durable store. Never rely on Mathias remembering anything.**
|
|
||||||
Every gap, caveat, risk, open question and next step must land in one of four
|
|
||||||
places before the turn ends: **the brain**, **a repo** (code, docs, ADR),
|
|
||||||
**a tracker issue**, or **a plan file**. A finding stated only in conversation is
|
|
||||||
lost the moment the session closes — and this estate has repeatedly proved that a
|
|
||||||
remembered convention is an unreliable control.
|
|
||||||
|
|
||||||
This applies hardest to what is tempting to leave as a closing remark: "worth
|
|
||||||
knowing X isn't enforced yet", "this is a convention, not a gate", "we should
|
|
||||||
revisit Y". Those are exactly the items that need a locator. If it is worth
|
|
||||||
saying in a summary, it is worth writing where it can be found again without you.
|
|
||||||
|
|
||||||
7. **Finish what's closable — only involve me for a genuine call.** If a
|
|
||||||
tracker issue, PR, or checklist item is demonstrably resolved — the fix
|
|
||||||
landed, the state changed, you verified it live — close/merge/check it off
|
|
||||||
in the same turn. Don't leave it open "for the record" or park it waiting
|
|
||||||
on me to confirm something that already happened; that's bookkeeping, not
|
|
||||||
a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
|
|
||||||
comment, or otherwise pausing) for when it's genuinely mine to call:
|
|
||||||
irreversible or prod/host-impacting actions, a real judgment call with no
|
|
||||||
clearly-better option, ambiguous intent, or anything secret-writes/
|
|
||||||
destructive-git already gate. Before claiming something is closable,
|
|
||||||
verify it live (re-check the issue, re-run the command) rather than
|
|
||||||
trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
|
|
||||||
|
|
||||||
8. **Control code needs an adversary, not a checklist.** Code whose job is to
|
|
||||||
refuse, deny, gate or contain is not adequately verified by a green gate.
|
|
||||||
Rule 5 makes CI the quality gate and that holds for ordinary work — but a
|
|
||||||
passing suite only asserts what its author already thought of, and for a
|
|
||||||
control the author is precisely the person whose imagination just failed.
|
|
||||||
Measured on `dispatch#75`: five review rounds, five real escapes verified by
|
|
||||||
execution, and **CI was green on every one of them** while `git rm
|
|
||||||
././.dispatch-allow` deleted the file the control existed to protect. So
|
|
||||||
route this work through a review with authority to block, instructed to
|
|
||||||
verify by execution rather than by reading the implementation. Expect to be
|
|
||||||
blocked more than once; that is the control working, not the reviewer being
|
|
||||||
difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
|
|
||||||
practices live in the `pr-gatekeeper` agent.)
|
|
||||||
|
|
||||||
9. **A test's name must not claim more than its assertions prove.** Name a test
|
|
||||||
after the mechanism it observes, not the goal it serves.
|
|
||||||
`DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
|
|
||||||
support. This is not pedantry about naming — when name and assertion
|
|
||||||
diverge, the name wins in the reader's head and the gap becomes invisible,
|
|
||||||
which is what lets the defect survive review. Found three times in one PR,
|
|
||||||
twice in tests written to fix the previous instance of it; each passed while
|
|
||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
|
||||||
you do not yet know what the test observes.
|
|
||||||
|
|
||||||
## CAD sprint patterns
|
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
|
||||||
- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
|
|
||||||
- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
|
|
||||||
|
|
||||||
Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
|
|
||||||
|
|
||||||
## Default stack
|
|
||||||
|
|
||||||
| Layer | Default | Fallback | Last resort |
|
|
||||||
|-------|---------|----------|-------------|
|
|
||||||
| Language | Go | Python | TypeScript, Java, C |
|
|
||||||
| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
|
|
||||||
| Build | Task (taskfile.dev) | Make | — |
|
|
||||||
| Containers | Docker Compose (dev), k3s (prod) | — | — |
|
|
||||||
| DB | PostgreSQL + sqlc | SQLite | — |
|
|
||||||
| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
|
|
||||||
| Logging | slog (structured) | stdlib `logging` w/ structured `extra=` (or structlog) | — |
|
|
||||||
| Testing | Table-driven, testify | — | — |
|
|
||||||
| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
|
|
||||||
|
|
||||||
Exploratory: Rust, Zig — I'll tell you when I want these.
|
|
||||||
|
|
||||||
## Code conventions
|
|
||||||
|
|
||||||
- **Go style**: golines, gofumpt, golangci-lint
|
|
||||||
- **Python style** (fallback language): ruff (format+lint, one tool), mypy --strict (non-negotiable,
|
|
||||||
matches Go's static typing discipline), pytest + pytest-cov (table-driven via
|
|
||||||
`@pytest.mark.parametrize`), uv (venv+deps+lock, one tool), pydantic-settings (typed env-var config
|
|
||||||
— same principle as Go's typed structs), src-layout + `pyproject.toml` only (no `setup.py`)
|
|
||||||
- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return.
|
|
||||||
Python: `raise X from e` (exception chaining, same principle) — never bare `except`, never silent `pass`
|
|
||||||
- **Naming**: stdlib conventions, no stuttering
|
|
||||||
- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
|
|
||||||
- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
|
|
||||||
one logical change per commit, CI is the quality gate
|
|
||||||
- **Never**: long-lived feature branches, PRs for solo work, direct push without
|
|
||||||
passing `task check` locally first
|
|
||||||
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
|
|
||||||
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
|
|
||||||
|
|
||||||
## Secret handling (every harness, every command)
|
|
||||||
|
|
||||||
Tool output is persisted: terminal → `~/.claude/projects` transcripts →
|
|
||||||
claudewatcher → brain/wiki → gitea history. A secret printed once is
|
|
||||||
searchable forever, and clearing it means rotating the key. So:
|
|
||||||
|
|
||||||
1. **Never print, echo, log, or transform a secret to inspect it.** No
|
|
||||||
`base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
|
|
||||||
to defeat `op run`'s output masking (it masks raw values; base64 hides them
|
|
||||||
from the mask — that exact trick leaked a key on 2026-06-11).
|
|
||||||
2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
|
|
||||||
*inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
|
|
||||||
in a command's argv (it lands in the tool call and the transcript).
|
|
||||||
3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
|
|
||||||
never `${X:-...}` (returns the value when set) and never echo a substring of it.
|
|
||||||
4. **Cross-host secrets:** run the secret-consuming command on the host that has
|
|
||||||
the secret; do not forward a raw key over ssh argv/stdout.
|
|
||||||
5. If a secret does leak into output, say so immediately and flag it for rotation —
|
|
||||||
don't bury it.
|
|
||||||
|
|
||||||
## Infrastructure
|
|
||||||
|
|
||||||
Three machines on Tailscale:
|
|
||||||
|
|
||||||
| Machine | Role | Key specs |
|
|
||||||
|---------|------|-----------|
|
|
||||||
| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
|
|
||||||
| iguana | Services, builds | M2 Ultra Mac |
|
|
||||||
| flamingo | Daily driver, edge | Apple Silicon Mac, ~/dev is here |
|
|
||||||
|
|
||||||
- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
|
|
||||||
- **Orchestration**: k3s on koala only — single control-plane node, no workers (verified 2026-08-30). iguana and flamingo are on the tailnet but are not cluster members
|
|
||||||
- **Networking**: Tailscale mesh
|
|
||||||
|
|
||||||
## Project landscape
|
|
||||||
|
|
||||||
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
|
||||||
|
|
||||||
Organized in thematic folders:
|
|
||||||
|
|
||||||
| Folder | Focus | Count |
|
|
||||||
|--------|-------|-------|
|
|
||||||
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
|
|
||||||
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
|
|
||||||
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
|
|
||||||
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
|
|
||||||
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
|
|
||||||
|
|
||||||
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
|
|
||||||
|
|
||||||
### Key active projects
|
|
||||||
|
|
||||||
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
|
|
||||||
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
|
|
||||||
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
|
|
||||||
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
|
|
||||||
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
|
|
||||||
|
|
||||||
## Knowledge base — actively use it
|
|
||||||
|
|
||||||
A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
|
|
||||||
hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
|
|
||||||
Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
|
|
||||||
reference material — query it actively, not just when explicitly told.**
|
|
||||||
|
|
||||||
### When to query (treat as a reflex)
|
|
||||||
|
|
||||||
- **Before** starting a non-trivial task — search for prior art with the symptom
|
|
||||||
AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
|
|
||||||
- **When debugging** — search for the error string, the stack frame, the affected
|
|
||||||
service. Past you may have already paid this tax.
|
|
||||||
- **Before adopting** a pattern, library, framework, or model name — check if it
|
|
||||||
was tried and rejected, or what the integration footguns are.
|
|
||||||
- **When making architectural decisions** — search for the domain + "ADR" or
|
|
||||||
"decision" to find prior reasoning before re-deriving it.
|
|
||||||
- **When a recommendation feels novel** — challenge yourself: "has this been
|
|
||||||
documented?" The brain often has it.
|
|
||||||
|
|
||||||
### When to write
|
|
||||||
|
|
||||||
After you discover something that **future-you would forget** and that **isn't
|
|
||||||
recoverable from the code, git log, or PR description alone**:
|
|
||||||
|
|
||||||
- Bugs whose root cause is non-obvious and generalisable beyond this project.
|
|
||||||
- Framework / library / model-name quirks that bit you and would bite anyone.
|
|
||||||
- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
|
|
||||||
- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
|
|
||||||
|
|
||||||
DON'T write project status, sprint progress, PR summaries, or "what I did this
|
|
||||||
session" — those rot fast and the originals are in git/gitea anyway. Brain
|
|
||||||
entries that age well are about *why*, *how to avoid*, and *what to do when*.
|
|
||||||
|
|
||||||
### How to access (per harness)
|
|
||||||
|
|
||||||
| Harness | Query | Write |
|
|
||||||
|---------|-------|-------|
|
|
||||||
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
|
|
||||||
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
|
|
||||||
| **Anything HTTP-only (curl, scripts)** | Tailnet only: `POST https://koala.tail30472e.ts.net:8443/query` with `{"query":"..."}`, sending `BRAIN_MCP_TOKEN` (not yet enforced on REST, hyperguild#149). The public `brain-mcp.d-ma.be` serves `/mcp` only (infra#517). | `POST .../write` with `{"content":"...","wing":"...","hall":"..."}` |
|
|
||||||
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
|
||||||
|
|
||||||
- **Scoping — the brain holds no confidential client content.** Nothing on the read path
|
|
||||||
filters by client: every token holder can read every wing, and `wing` on `brain_query`
|
|
||||||
is an optional argument, not an access control. Client confidentiality is enforced only
|
|
||||||
at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
|
|
||||||
client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
|
|
||||||
decided 2026-10-01. This line previously claimed client projects were filtered; they
|
|
||||||
were not.)
|
|
||||||
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
|
||||||
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
|
||||||
on the koala k3s cluster; don't hardcode local-only model names into the
|
|
||||||
berget URL (see knowledge entry on namespace mismatches).
|
|
||||||
|
|
||||||
### Quick reflex checks
|
|
||||||
|
|
||||||
If you find yourself about to say any of these out loud, you owe yourself a brain query first:
|
|
||||||
|
|
||||||
- "I think the issue might be..."
|
|
||||||
- "Let me try X and see..."
|
|
||||||
- "I'll just write a script to..."
|
|
||||||
- "This is probably a new bug..."
|
|
||||||
- "Has anyone done this before?" — *yes, probably, go check.*
|
|
||||||
|
|
||||||
## Client work rules
|
|
||||||
|
|
||||||
When working on a project tagged with a client name:
|
|
||||||
1. Never send code, data, or context to cloud APIs — use local models only
|
|
||||||
2. Never reference other client projects or their data
|
|
||||||
3. Keep all artifacts within the client's git org / directory
|
|
||||||
4. Treat everything as confidential unless told otherwise
|
|
||||||
|
|
||||||
## Harness-agnostic principles
|
|
||||||
|
|
||||||
Supported harnesses (2026-09-01) — Cursor and Aider were dropped; do not
|
|
||||||
re-add adapters for them:
|
|
||||||
- Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush, Pi Coding Agent
|
|
||||||
- Open WebUI, Charmbracelet Mods, or any tool that takes a system prompt string
|
|
||||||
- Any other tool that reads `AGENTS.md`, which is the de facto convention
|
|
||||||
|
|
||||||
The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
|
|
||||||
Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
|
|
||||||
|
|
||||||
## How context propagates
|
|
||||||
|
|
||||||
Canonical sources of truth:
|
|
||||||
- Universal: `~/dev/.context/AGENT.md` (this file)
|
|
||||||
- Project: `<repo>/.context/PROJECT.md` (per-repo)
|
|
||||||
|
|
||||||
Derived files (committed, regenerated by `task context:sync`):
|
|
||||||
- `CLAUDE.md`, `AGENTS.md`, `.context/system-prompt.txt`
|
|
||||||
|
|
||||||
Repos still carrying `.cursorrules`, `.aider.conventions.md` or
|
|
||||||
`.aider.conf.yml` are stale: nothing regenerates them. Delete them the next time
|
|
||||||
you touch the repo, along with their entries in that repo's `task check`
|
|
||||||
assertion.
|
|
||||||
|
|
||||||
Workflow:
|
|
||||||
1. Edit a canonical file. Run `task context:sync`. Commit canonical and
|
|
||||||
derived together. Push.
|
|
||||||
2. On any other host, `git pull` brings both. Which file a harness reads does
|
|
||||||
NOT split cleanly by lookup strategy, so do not reason from the strategy:
|
|
||||||
- `CLAUDE.md` — Claude Code (walks up the tree)
|
|
||||||
- `AGENTS.md` — Crush, Pi, Antigravity (cwd-only), OpenCode (project root,
|
|
||||||
global at `~/.config/opencode/AGENTS.md`), and Mistral Vibe, which **walks
|
|
||||||
up** from cwd and takes the first `AGENTS.md` it finds
|
|
||||||
(user-level `~/.vibe/AGENTS.md`)
|
|
||||||
- `.context/system-prompt.txt` — Open WebUI, Mods, raw API calls; not read by
|
|
||||||
any harness, it is for pasting
|
|
||||||
|
|
||||||
Vibe tree-walks for `AGENTS.md`, so "tree-walkers use CLAUDE.md" is false.
|
|
||||||
Each repo's `AGENTS.md` embeds the canonical and is the first one found from
|
|
||||||
inside that repo, so this works — but by construction, not by luck.
|
|
||||||
3. `task check` runs `context:sync` then asserts `git status --porcelain`
|
|
||||||
is empty over the derived files (catches both modified-tracked drift
|
|
||||||
and missing-untracked adapters). A drift fails the check with a
|
|
||||||
message telling you to stage the regenerated files.
|
|
||||||
|
|
||||||
Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
|
|
||||||
host and every harness.
|
|
||||||
|
|
||||||
**Where a project rule contradicts a rule here, the project rule governs — and it
|
|
||||||
must state its reason.** Both halves matter: the project file wins because it knows
|
|
||||||
something this file cannot (infra's Flux constraint is real and not visible from
|
|
||||||
here), and it must say why because a bare override is indistinguishable from drift.
|
|
||||||
|
|
||||||
This replaces the previous wording, which said rules here and in `PROJECT.md` both
|
|
||||||
apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
|
|
||||||
unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
|
|
||||||
at the same time (infra#470). A standard can hold two individually sensible clauses
|
|
||||||
that are jointly unsatisfiable, and survive for as long as everyone reads it
|
|
||||||
charitably — the same failure the Inception Sprint Oath's G6 had.
|
|
||||||
|
|
||||||
## Engineering Skills
|
|
||||||
|
|
||||||
Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Load at task start — not "on demand" but on schedule, before writing code. Browse `SKILLS_INDEX.md` for the full list.
|
|
||||||
|
|
||||||
**Claude Code** installs the repo as a plugin from the marketplace the same repo hosts. That path also carries the `secret-guard` and `pre-task-ritual` hooks and the brain/gitea/infra MCP servers, which symlinks cannot:
|
|
||||||
|
|
||||||
```
|
|
||||||
/plugin marketplace add https://git.d-ma.be/mathias/skills.git
|
|
||||||
/plugin install skills@mathias
|
|
||||||
```
|
|
||||||
|
|
||||||
Skills are **namespaced** under Claude Code: load `tdd` as `skills:tdd`. The trigger table below uses bare names — prefix them.
|
|
||||||
|
|
||||||
**Crush, Pi and OpenCode** have no plugin system and are wired by `install.sh` as one directory symlink per harness via `~/.agents/skills`. Do not clone the checkout into `~/dev`: that folder is iCloud-shared between the Macs and spans two path roots (`/Users/mathias` vs `/home/mathias` on koala), so a checkout there is a git working tree with two concurrent writers, and any absolute symlink stored in it is broken on one side by construction (skills#23). Leave `SKILLS_CHECKOUT_DIR` at its host-local default.
|
|
||||||
|
|
||||||
Check a host with `task doctor` in the skills repo; it also runs automatically at session start via the plugin.
|
|
||||||
|
|
||||||
**Skill trigger table — load before starting, not after getting stuck:**
|
|
||||||
|
|
||||||
| Task type | Load |
|
|
||||||
|-----------|------|
|
|
||||||
| Any feature or bug fix | `tdd` |
|
|
||||||
| Refactor or design | `clean-code` or `solid` |
|
|
||||||
| Debug | `problem-analysis` |
|
|
||||||
| Review code or PRs | `code-review` |
|
|
||||||
| Frame a problem before coding | `problem-analysis` |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# jepa-fx-risk
|
|
||||||
|
|
||||||
## Identity
|
## Identity
|
||||||
|
|
||||||
- **Name**: jepa-fx-risk
|
- **Name**: hostexecutor
|
||||||
- **Owner**: Mathias
|
- **Owner**: Mathias
|
||||||
- **Client**: personal research
|
- **Client**: personal
|
||||||
- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
|
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||||
- **Status**: active — Phase 0
|
- **Status**: active
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Research project: apply JEPA-based self-supervised representation learning to
|
|
||||||
FX risk management for a corporate bank with an internal global FX trading desk.
|
|
||||||
|
|
||||||
Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
|
|
||||||
Target output: internal PoC for the trading desk.
|
|
||||||
|
|
||||||
## Architecture decision (see DECISIONS.md ADR-001)
|
|
||||||
|
|
||||||
**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
|
|
||||||
arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
|
|
||||||
(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
|
|
||||||
|
|
||||||
Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
|
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
|
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||||
harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
|
|
||||||
results dashboard (Templ + HTMX + CDN Tailwind).
|
|
||||||
|
|
||||||
**Python** (`model/`): all training and embedding export only. PyTorch cu130
|
|
||||||
(Blackwell sm_120 compatible). No evaluation logic in Python.
|
|
||||||
|
|
||||||
**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
|
|
||||||
iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
|
|
||||||
|
|
||||||
## Repository layout
|
|
||||||
|
|
||||||
```
|
|
||||||
jepa-fx-risk/
|
|
||||||
├── src/ # Go — data pipeline + eval harness + dashboard
|
|
||||||
│ ├── data/ # DUKASCopy fetch, hourly processing, validation
|
|
||||||
│ └── eval/ # silhouette, linear probe, collapse, backtest
|
|
||||||
├── model/ # Python — training only
|
|
||||||
│ ├── train.py # TS-JEPA + SIGReg backbone (autoresearch edits this)
|
|
||||||
│ ├── prepare.py # LOCKED — data loading, tokenization, export
|
|
||||||
│ └── requirements.txt
|
|
||||||
├── specs/ # Research specs (one per phase/experiment type)
|
|
||||||
├── experiments/ # Per-run outputs: embeddings, metrics.json, git tag
|
|
||||||
├── results/summaries/ # Human-readable outcome per experiment
|
|
||||||
├── program.md # Autoresearch agenda — researcher edits this
|
|
||||||
├── DECISIONS.md # Architecture Decision Records
|
|
||||||
└── Taskfile.yml # task data:fetch, task experiment:run, task eval:*
|
|
||||||
```
|
|
||||||
|
|
||||||
## Phase structure
|
|
||||||
|
|
||||||
- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
|
|
||||||
Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
|
|
||||||
- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
|
|
||||||
Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
|
|
||||||
- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
|
|
||||||
- **Phase 3**: Internal bank tick/position data (future, out of current scope).
|
|
||||||
|
|
||||||
## Data
|
|
||||||
|
|
||||||
DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
|
|
||||||
Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
|
|
||||||
Weekend gaps handled explicitly. See ADR-002.
|
|
||||||
|
|
||||||
## Key conventions
|
|
||||||
|
|
||||||
- `prepare.py` is LOCKED — never modified by agents or autoresearch
|
|
||||||
- Evaluation metrics are always computed by the Go harness, never in Python
|
|
||||||
- Every experiment gets a git tag: `exp/YYYYMMDD-description`
|
|
||||||
- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
|
|
||||||
- `program.md` is the only file the researcher edits to steer autoresearch
|
|
||||||
- CI runs `task check` (lint + vet + test) only — no GPU, no training
|
|
||||||
|
|
||||||
## Evaluation metrics
|
|
||||||
|
|
||||||
Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
|
|
||||||
realized volatility from frozen embeddings, computed by Go harness.
|
|
||||||
|
|
||||||
Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
|
|
||||||
Must co-move with val_vol_r2 — checked from experiment 1.
|
|
||||||
|
|
||||||
Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
|
|
||||||
|
|
||||||
## Benchmarks to beat (trading desk comparison)
|
|
||||||
|
|
||||||
- GARCH(1,1) — volatility forecasting baseline
|
|
||||||
- Historical Simulation VaR (250-day rolling) — Basel default
|
|
||||||
- EWMA RiskMetrics (λ=0.94)
|
|
||||||
|
|
||||||
## Agent guidance
|
|
||||||
|
|
||||||
Read `DECISIONS.md` before making architecture suggestions.
|
|
||||||
Do not modify `prepare.py` or `src/eval/`.
|
|
||||||
Do not suggest changing the Python/Go separation.
|
|
||||||
Training runs are always manual via `task experiment:run` — never triggered by CI.
|
|
||||||
When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
|
|
||||||
|
|||||||
@@ -1,104 +1,13 @@
|
|||||||
# jepa-fx-risk
|
# hostexecutor
|
||||||
|
|
||||||
## Identity
|
## Identity
|
||||||
|
|
||||||
- **Name**: jepa-fx-risk
|
- **Name**: hostexecutor
|
||||||
- **Owner**: Mathias
|
- **Owner**: Mathias
|
||||||
- **Client**: personal research
|
- **Client**: personal
|
||||||
- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
|
- **Repo**: gitea.d-ma.be/mathias/hostexecutor
|
||||||
- **Status**: active — Phase 0
|
- **Status**: active
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Research project: apply JEPA-based self-supervised representation learning to
|
|
||||||
FX risk management for a corporate bank with an internal global FX trading desk.
|
|
||||||
|
|
||||||
Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
|
|
||||||
Target output: internal PoC for the trading desk.
|
|
||||||
|
|
||||||
## Architecture decision (see DECISIONS.md ADR-001)
|
|
||||||
|
|
||||||
**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
|
|
||||||
arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
|
|
||||||
(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].
|
|
||||||
|
|
||||||
Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).
|
|
||||||
|
|
||||||
## Stack
|
## Stack
|
||||||
|
|
||||||
**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
|
Go + Templ + HTMX + CDN Tailwind. See `~/dev/.context/AGENT.md` for cross-project conventions.
|
||||||
harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
|
|
||||||
results dashboard (Templ + HTMX + CDN Tailwind).
|
|
||||||
|
|
||||||
**Python** (`model/`): all training and embedding export only. PyTorch cu130
|
|
||||||
(Blackwell sm_120 compatible). No evaluation logic in Python.
|
|
||||||
|
|
||||||
**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
|
|
||||||
iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.
|
|
||||||
|
|
||||||
## Repository layout
|
|
||||||
|
|
||||||
```
|
|
||||||
jepa-fx-risk/
|
|
||||||
├── src/ # Go — data pipeline + eval harness + dashboard
|
|
||||||
│ ├── data/ # DUKASCopy fetch, hourly processing, validation
|
|
||||||
│ └── eval/ # silhouette, linear probe, collapse, backtest
|
|
||||||
├── model/ # Python — training only
|
|
||||||
│ ├── train.py # TS-JEPA + SIGReg backbone (autoresearch edits this)
|
|
||||||
│ ├── prepare.py # LOCKED — data loading, tokenization, export
|
|
||||||
│ └── requirements.txt
|
|
||||||
├── specs/ # Research specs (one per phase/experiment type)
|
|
||||||
├── experiments/ # Per-run outputs: embeddings, metrics.json, git tag
|
|
||||||
├── results/summaries/ # Human-readable outcome per experiment
|
|
||||||
├── program.md # Autoresearch agenda — researcher edits this
|
|
||||||
├── DECISIONS.md # Architecture Decision Records
|
|
||||||
└── Taskfile.yml # task data:fetch, task experiment:run, task eval:*
|
|
||||||
```
|
|
||||||
|
|
||||||
## Phase structure
|
|
||||||
|
|
||||||
- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
|
|
||||||
Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
|
|
||||||
- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
|
|
||||||
Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
|
|
||||||
- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
|
|
||||||
- **Phase 3**: Internal bank tick/position data (future, out of current scope).
|
|
||||||
|
|
||||||
## Data
|
|
||||||
|
|
||||||
DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
|
|
||||||
Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
|
|
||||||
Weekend gaps handled explicitly. See ADR-002.
|
|
||||||
|
|
||||||
## Key conventions
|
|
||||||
|
|
||||||
- `prepare.py` is LOCKED — never modified by agents or autoresearch
|
|
||||||
- Evaluation metrics are always computed by the Go harness, never in Python
|
|
||||||
- Every experiment gets a git tag: `exp/YYYYMMDD-description`
|
|
||||||
- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
|
|
||||||
- `program.md` is the only file the researcher edits to steer autoresearch
|
|
||||||
- CI runs `task check` (lint + vet + test) only — no GPU, no training
|
|
||||||
|
|
||||||
## Evaluation metrics
|
|
||||||
|
|
||||||
Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
|
|
||||||
realized volatility from frozen embeddings, computed by Go harness.
|
|
||||||
|
|
||||||
Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
|
|
||||||
Must co-move with val_vol_r2 — checked from experiment 1.
|
|
||||||
|
|
||||||
Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).
|
|
||||||
|
|
||||||
## Benchmarks to beat (trading desk comparison)
|
|
||||||
|
|
||||||
- GARCH(1,1) — volatility forecasting baseline
|
|
||||||
- Historical Simulation VaR (250-day rolling) — Basel default
|
|
||||||
- EWMA RiskMetrics (λ=0.94)
|
|
||||||
|
|
||||||
## Agent guidance
|
|
||||||
|
|
||||||
Read `DECISIONS.md` before making architecture suggestions.
|
|
||||||
Do not modify `prepare.py` or `src/eval/`.
|
|
||||||
Do not suggest changing the Python/Go separation.
|
|
||||||
Training runs are always manual via `task experiment:run` — never triggered by CI.
|
|
||||||
When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.
|
|
||||||
|
|||||||
@@ -1,22 +0,0 @@
|
|||||||
# Autoresearch STATUS
|
|
||||||
|
|
||||||
| iter | val_vol_r2 | delta | action | secs | gpu | change |
|
|
||||||
|------|-----------|-------|--------|------|-----|--------|
|
|
||||||
| 1 | 0.3749 | +0.0928 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
|
||||||
| 1 | 0.3011 | +0.0776 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
|
||||||
| 2 | 0.3032 | +0.0021 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
|
|
||||||
| 1 | 0.2759 | -0.0273 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter1 |
|
|
||||||
| 2 | 0.3442 | +0.0410 | KEEP | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter2 |
|
|
||||||
| 3 | 0.3371 | -0.0071 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter3 |
|
|
||||||
| 4 | 0.3143 | -0.0299 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter4 |
|
|
||||||
| 5 | 0.3355 | -0.0087 | revert | 2s | gpu=0% vram=10054/12227MiB temp=34°C | iter5 |
|
|
||||||
| 6 | 0.2377 | -0.1065 | revert | 2s | gpu=0% vram=10054/12227MiB temp=35°C | iter6 |
|
|
||||||
| 1 | -0.1247 | +0.0296 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
|
|
||||||
| 2 | -0.1203 | +0.0044 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=35°C | iter2 |
|
|
||||||
| 3 | -0.0716 | +0.0487 | KEEP | 4s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
|
|
||||||
| 4 | 0.0590 | +0.1306 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter4 |
|
|
||||||
| 5 | 0.0599 | +0.0009 | KEEP | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter5 |
|
|
||||||
| 1 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=35°C | iter1 |
|
|
||||||
| 2 | 0.0577 | -0.0022 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter2 |
|
|
||||||
| 3 | 0.0563 | -0.0036 | revert | 5s | gpu=0% vram=10054/12227MiB temp=36°C | iter3 |
|
|
||||||
| 4 | -0.1613 | -0.2212 | revert | 5s | gpu=0% vram=10054/12227MiB temp=37°C | iter4 |
|
|
||||||
+5
-50
@@ -5,63 +5,16 @@ tasks:
|
|||||||
desc: Run templ generate
|
desc: Run templ generate
|
||||||
cmds: [templ generate]
|
cmds: [templ generate]
|
||||||
build:
|
build:
|
||||||
desc: Build all binaries
|
desc: Build the binary
|
||||||
deps: [generate]
|
deps: [generate]
|
||||||
cmds:
|
cmds: [go build -o bin/hostexecutor ./cmd/hostexecutor]
|
||||||
- go build -o bin/jepa-fx-risk ./cmd/jepa-fx-risk
|
|
||||||
- go build -o bin/eval ./cmd/eval
|
|
||||||
run:
|
run:
|
||||||
deps: [build]
|
deps: [build]
|
||||||
cmds: [./bin/jepa-fx-risk]
|
cmds: [./bin/hostexecutor]
|
||||||
test:
|
test:
|
||||||
desc: Run all tests
|
desc: Run all tests
|
||||||
deps: [generate]
|
deps: [generate]
|
||||||
cmds: [go test ./... -race]
|
cmds: [go test ./... -race]
|
||||||
|
|
||||||
data:fetch:
|
|
||||||
desc: "Download EUR/USD M1 from histdata (set YEARS env var)"
|
|
||||||
cmds: [.venv/bin/python scripts/fetch_data.py]
|
|
||||||
data:fetch:historical:
|
|
||||||
desc: "Download EUR/USD M1 2008-2018 from histdata"
|
|
||||||
cmds:
|
|
||||||
- YEARS=2008,2009,2010,2011,2012,2013,2014,2015,2016,2017,2018 .venv/bin/python scripts/fetch_data.py
|
|
||||||
data:prepare:daily:
|
|
||||||
desc: "Rebuild eurusd_daily.parquet from all M1 zips"
|
|
||||||
cmds: [.venv/bin/python scripts/prepare_data.py]
|
|
||||||
data:prepare:hourly:
|
|
||||||
desc: "Build eurusd_hourly.parquet from all M1 zips"
|
|
||||||
cmds: [.venv/bin/python scripts/prepare_hourly.py]
|
|
||||||
data:prepare:all:
|
|
||||||
desc: "Build both daily and hourly parquets"
|
|
||||||
deps: [data:prepare:daily, data:prepare:hourly]
|
|
||||||
train:multipair:
|
|
||||||
desc: "Train 5-pair G10 HEPA (D=256, best config, phase1_r2≈0.44)"
|
|
||||||
cmds: [JEPA_USE_MULTIPAIR=1 JEPA_D_MODEL=256 .venv/bin/python train.py]
|
|
||||||
|
|
||||||
data:fetch:multipair:
|
|
||||||
desc: "Download G10 M1 data (GBPUSD/USDJPY/USDCHF/AUDUSD) 2008-2023 from histdata"
|
|
||||||
cmds: [.venv/bin/python scripts/fetch_multipair.py]
|
|
||||||
data:prepare:pair:
|
|
||||||
desc: "Build {PAIR}_hourly.parquet from data/raw/{PAIR}/ (e.g. PAIR=gbpusd)"
|
|
||||||
cmds: ["PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}"]
|
|
||||||
vars:
|
|
||||||
PAIR: '{{default "eurusd" .PAIR}}'
|
|
||||||
data:prepare:multipair:
|
|
||||||
desc: "Merge 5-pair hourly parquets into eurusd_multipair.parquet"
|
|
||||||
cmds: [.venv/bin/python scripts/prepare_multipair.py]
|
|
||||||
data:test:
|
|
||||||
desc: "Run Python data pipeline tests"
|
|
||||||
cmds: [.venv/bin/python -m pytest tests/test_prepare_hourly.py tests/test_hepa.py tests/test_multipair.py -v]
|
|
||||||
|
|
||||||
eval:probe:
|
|
||||||
desc: "Run linear-probe (val_vol_r2) on embeddings from metrics.json"
|
|
||||||
cmds: [./bin/eval -metric probe]
|
|
||||||
eval:silhouette:
|
|
||||||
desc: "Run silhouette on embeddings vs binary HV labels"
|
|
||||||
cmds: [./bin/eval -metric silhouette]
|
|
||||||
eval:collapse:
|
|
||||||
desc: "Run effective-rank collapse diagnostic"
|
|
||||||
cmds: [./bin/eval -metric erank]
|
|
||||||
lint:
|
lint:
|
||||||
cmds: [golangci-lint run ./...]
|
cmds: [golangci-lint run ./...]
|
||||||
check:
|
check:
|
||||||
@@ -80,3 +33,5 @@ tasks:
|
|||||||
cmds: [bash scripts/context-sync.sh claude]
|
cmds: [bash scripts/context-sync.sh claude]
|
||||||
context:sync:agents:
|
context:sync:agents:
|
||||||
cmds: [bash scripts/context-sync.sh agents]
|
cmds: [bash scripts/context-sync.sh agents]
|
||||||
|
context:sync:cursor:
|
||||||
|
cmds: [bash scripts/context-sync.sh cursor]
|
||||||
|
|||||||
@@ -1,194 +0,0 @@
|
|||||||
// cmd/eval — CLI driver for the jepa-fx-risk evaluation harness.
|
|
||||||
//
|
|
||||||
// ./bin/eval -metric probe|silhouette|erank [-emb embeddings.json]
|
|
||||||
//
|
|
||||||
// embeddings.json format (from train.py EXPORT_EMBEDDINGS=1):
|
|
||||||
//
|
|
||||||
// {
|
|
||||||
// "embeddings": [[...], ...], // OOS frozen embeddings
|
|
||||||
// "realized_vol": [...], // OOS target (next-day RV)
|
|
||||||
// "hv_label": [...], // binary HV label (top-33%)
|
|
||||||
// "train_embeddings": [[...], ...], // train-set frozen embeddings
|
|
||||||
// "train_realized_vol": [...] // train-set RV targets
|
|
||||||
// }
|
|
||||||
//
|
|
||||||
// eval:probe standardises both sets using train statistics (no leakage).
|
|
||||||
// Falls back to internal 70/30 split of OOS if train_embeddings absent.
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"log/slog"
|
|
||||||
"math"
|
|
||||||
"os"
|
|
||||||
|
|
||||||
"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
metric := flag.String("metric", "probe", "probe | silhouette | erank")
|
|
||||||
embFile := flag.String("emb", "embeddings.json", "path to embeddings JSON")
|
|
||||||
flag.Parse()
|
|
||||||
|
|
||||||
log := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
|
||||||
|
|
||||||
d, err := readJSON(*embFile)
|
|
||||||
if err != nil {
|
|
||||||
log.Error("load embeddings", "err", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
log.Info("loaded", "oos", len(d.Embeddings), "dim", len(d.Embeddings[0]),
|
|
||||||
"train", len(d.TrainEmbeddings), "metric", *metric)
|
|
||||||
|
|
||||||
switch *metric {
|
|
||||||
case "probe":
|
|
||||||
var r2 float64
|
|
||||||
if len(d.TrainEmbeddings) > 0 {
|
|
||||||
// standardise both sets using train statistics to prevent leakage
|
|
||||||
trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
|
|
||||||
oosEmb := applyStandardise(d.Embeddings, mu, sd)
|
|
||||||
r2 = eval.LinearProbeTrainTest(trEmb, d.TrainRealizedVol, oosEmb, d.RealizedVol, 1e-3)
|
|
||||||
log.Info("probe mode", "fit_on", "train_embeddings", "eval_on", "oos")
|
|
||||||
} else {
|
|
||||||
// fallback: internal 70/30 split of OOS embeddings
|
|
||||||
oosEmb, mu, sd := standardiseCompute(d.Embeddings)
|
|
||||||
n70 := int(float64(len(oosEmb)) * 0.7)
|
|
||||||
oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
|
|
||||||
r2 = eval.LinearProbeTrainTest(oosEmb[:n70], d.RealizedVol[:n70],
|
|
||||||
oos70, d.RealizedVol[n70:], 1e-3)
|
|
||||||
log.Info("probe mode", "fit_on", "oos[0:70%]", "eval_on", "oos[70%:]")
|
|
||||||
}
|
|
||||||
fmt.Printf(`{"metric":"val_vol_r2","value":%.6f}`+"\n", r2)
|
|
||||||
log.Info("linear probe", "val_vol_r2", fmt.Sprintf("%.4f", r2))
|
|
||||||
|
|
||||||
case "silhouette":
|
|
||||||
if len(d.HVLabel) == 0 {
|
|
||||||
log.Error("silhouette requires hv_label in embeddings.json")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
oosEmb := standardise(d.Embeddings)
|
|
||||||
sil, err := eval.Silhouette(oosEmb, d.HVLabel)
|
|
||||||
if err != nil {
|
|
||||||
log.Error("silhouette", "err", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Printf(`{"metric":"silhouette","value":%.6f}`+"\n", sil)
|
|
||||||
log.Info("silhouette", "score", fmt.Sprintf("%.4f", sil))
|
|
||||||
|
|
||||||
case "erank":
|
|
||||||
oosEmb := standardise(d.Embeddings)
|
|
||||||
er := eval.EffectiveRank(oosEmb)
|
|
||||||
fmt.Printf(`{"metric":"effective_rank","value":%.6f}`+"\n", er)
|
|
||||||
log.Info("effective rank", "erank", fmt.Sprintf("%.2f", er))
|
|
||||||
|
|
||||||
case "var":
|
|
||||||
// Parametric 99% VaR breach rate from probe predictions vs actual realized vol.
|
|
||||||
// Requires train_embeddings (for no-leakage probe fit) and realized_vol (OOS).
|
|
||||||
if len(d.RealizedVol) == 0 {
|
|
||||||
log.Error("var requires realized_vol in embeddings.json")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
var predVol []float64
|
|
||||||
if len(d.TrainEmbeddings) > 0 {
|
|
||||||
trEmb, mu, sd := standardiseCompute(d.TrainEmbeddings)
|
|
||||||
oosEmb := applyStandardise(d.Embeddings, mu, sd)
|
|
||||||
predVol = eval.LinearProbePredict(trEmb, d.TrainRealizedVol, oosEmb, 1e-3)
|
|
||||||
} else {
|
|
||||||
oosEmb, mu, sd := standardiseCompute(d.Embeddings)
|
|
||||||
n70 := int(float64(len(oosEmb)) * 0.7)
|
|
||||||
oos70 := applyStandardise(d.Embeddings[n70:], mu, sd)
|
|
||||||
predVol = eval.LinearProbePredict(oosEmb[:n70], d.RealizedVol[:n70], oos70, 1e-3)
|
|
||||||
d.RealizedVol = d.RealizedVol[n70:]
|
|
||||||
}
|
|
||||||
const z99 = 2.326
|
|
||||||
breachRate, kupiecP := eval.VaRBreachRate(predVol, d.RealizedVol, z99)
|
|
||||||
fmt.Printf(`{"metric":"VaR_breach_rate_99_oos_regime_cond","value":%.6f,"kupiec_p":%.6f}`+"\n",
|
|
||||||
breachRate, kupiecP)
|
|
||||||
log.Info("VaR breach rate 99%", "breach_rate", fmt.Sprintf("%.4f", breachRate),
|
|
||||||
"kupiec_p", fmt.Sprintf("%.4f", kupiecP))
|
|
||||||
|
|
||||||
default:
|
|
||||||
log.Error("unknown metric", "metric", *metric)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type embJSON struct {
|
|
||||||
Embeddings [][]float64 `json:"embeddings"`
|
|
||||||
Dates []string `json:"dates"`
|
|
||||||
RealizedVol []float64 `json:"realized_vol"`
|
|
||||||
HVLabel []int `json:"hv_label"`
|
|
||||||
TrainEmbeddings [][]float64 `json:"train_embeddings"`
|
|
||||||
TrainRealizedVol []float64 `json:"train_realized_vol"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func readJSON(path string) (*embJSON, error) {
|
|
||||||
f, err := os.Open(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("open %s: %w", path, err)
|
|
||||||
}
|
|
||||||
defer func() { _ = f.Close() }()
|
|
||||||
var d embJSON
|
|
||||||
if err := json.NewDecoder(f).Decode(&d); err != nil {
|
|
||||||
return nil, fmt.Errorf("decode: %w", err)
|
|
||||||
}
|
|
||||||
if len(d.Embeddings) == 0 {
|
|
||||||
return nil, fmt.Errorf("empty embeddings in %s", path)
|
|
||||||
}
|
|
||||||
return &d, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// standardise centres + scales to zero mean / unit std; returns normalised rows.
|
|
||||||
func standardise(rows [][]float64) [][]float64 {
|
|
||||||
out, _, _ := standardiseCompute(rows)
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
// standardiseCompute centres + scales and returns (normalised, mu, sd) for reuse.
|
|
||||||
func standardiseCompute(rows [][]float64) ([][]float64, []float64, []float64) {
|
|
||||||
if len(rows) == 0 {
|
|
||||||
return rows, nil, nil
|
|
||||||
}
|
|
||||||
n, dim := len(rows), len(rows[0])
|
|
||||||
mu := make([]float64, dim)
|
|
||||||
for _, r := range rows {
|
|
||||||
for j, v := range r {
|
|
||||||
mu[j] += v
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := range mu {
|
|
||||||
mu[j] /= float64(n)
|
|
||||||
}
|
|
||||||
sd := make([]float64, dim)
|
|
||||||
for _, r := range rows {
|
|
||||||
for j, v := range r {
|
|
||||||
diff := v - mu[j]
|
|
||||||
sd[j] += diff * diff
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := range sd {
|
|
||||||
sd[j] = math.Sqrt(sd[j]/float64(n)) + 1e-8
|
|
||||||
}
|
|
||||||
out := make([][]float64, n)
|
|
||||||
for i, r := range rows {
|
|
||||||
out[i] = make([]float64, dim)
|
|
||||||
for j, v := range r {
|
|
||||||
out[i][j] = (v - mu[j]) / sd[j]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return out, mu, sd
|
|
||||||
}
|
|
||||||
|
|
||||||
// applyStandardise normalises rows using pre-computed mu and sd.
|
|
||||||
func applyStandardise(rows [][]float64, mu, sd []float64) [][]float64 {
|
|
||||||
out := make([][]float64, len(rows))
|
|
||||||
for i, r := range rows {
|
|
||||||
out[i] = make([]float64, len(r))
|
|
||||||
for j, v := range r {
|
|
||||||
out[i][j] = (v - mu[j]) / sd[j]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
@@ -5,7 +5,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"os"
|
"os"
|
||||||
|
|
||||||
"git.d-ma.be/mathias/jepa-fx-risk/internal/web"
|
"gitea.d-ma.be/mathias/hostexecutor/internal/web"
|
||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
Binary file not shown.
@@ -1,11 +0,0 @@
|
|||||||
{
|
|
||||||
"strategic_question": "What is the highest-leverage path to a JEPA-based FX tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach calibration, given one GPU and a solo researcher?",
|
|
||||||
"nodes": [
|
|
||||||
{
|
|
||||||
"id": "phase-a-toy",
|
|
||||||
"status": "autoresearch-ready",
|
|
||||||
"question": "Improve the OOS linear-probe R² (val_vol_r2) of the HEPA encoder on EUR/USD daily realized vol. The encoder is a small causal transformer trained with VICReg. Vary one hyperparameter or architectural choice per iteration — model size, learning rate, window, patch length, depth, VICReg loss weights — to push val_vol_r2 as high as possible on the 2022-2023 OOS slice.",
|
|
||||||
"candidate_metric": "val_vol_r2"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
module git.d-ma.be/mathias/jepa-fx-risk
|
module gitea.d-ma.be/mathias/hostexecutor
|
||||||
|
|
||||||
go 1.26
|
go 1.26
|
||||||
|
|
||||||
require github.com/a-h/templ v0.3.1020
|
require (
|
||||||
|
github.com/a-h/templ v0.2.778
|
||||||
|
)
|
||||||
|
|||||||
@@ -1,4 +0,0 @@
|
|||||||
github.com/a-h/templ v0.3.1020 h1:ypAT/L5ySWEnZ6Zft/5yfoWXYYkhFNvEFOeeqecg4tw=
|
|
||||||
github.com/a-h/templ v0.3.1020/go.mod h1:A2DlK61v+K+NRoGnhmYbNYVmtYHcFO5/AisMvBdDxTM=
|
|
||||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
|
||||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
|
||||||
@@ -1,384 +0,0 @@
|
|||||||
// Package eval implements the Go evaluation harness for jepa-fx-risk (#4).
|
|
||||||
// Three diagnostics on frozen embeddings exported from train.py:
|
|
||||||
// - LinearProbe — val_vol_r2: OOS R² of a ridge probe predicting next-day realized vol
|
|
||||||
// - Silhouette — mean silhouette score of embeddings vs a binary label (HV regime)
|
|
||||||
// - EffectiveRank — Roy's effective rank: exp(H(σ²)) where H is entropy of normalised singular values
|
|
||||||
package eval
|
|
||||||
|
|
||||||
import (
|
|
||||||
"errors"
|
|
||||||
"math"
|
|
||||||
)
|
|
||||||
|
|
||||||
// LinearProbeTrainTest fits ridge regression on (trainEmb, trainY) and evaluates
|
|
||||||
// on (testEmb, testY). Returns OOS R². Use this for proper held-out evaluation.
|
|
||||||
func LinearProbeTrainTest(trainEmb [][]float64, trainY []float64,
|
|
||||||
testEmb [][]float64, testY []float64, lambda float64,
|
|
||||||
) float64 {
|
|
||||||
n := len(trainEmb)
|
|
||||||
if n == 0 || len(testEmb) == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
d := len(trainEmb[0])
|
|
||||||
p := d + 1
|
|
||||||
|
|
||||||
A := make([][]float64, n)
|
|
||||||
for i, e := range trainEmb {
|
|
||||||
row := make([]float64, p)
|
|
||||||
copy(row, e)
|
|
||||||
row[d] = 1.0
|
|
||||||
A[i] = row
|
|
||||||
}
|
|
||||||
AtA := make([][]float64, p)
|
|
||||||
for i := range AtA {
|
|
||||||
AtA[i] = make([]float64, p)
|
|
||||||
}
|
|
||||||
Aty := make([]float64, p)
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
Aty[j] += A[i][j] * trainY[i]
|
|
||||||
for k := 0; k < p; k++ {
|
|
||||||
AtA[j][k] += A[i][j] * A[i][k]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
AtA[j][j] += lambda
|
|
||||||
}
|
|
||||||
w := solveCholesky(AtA, Aty)
|
|
||||||
|
|
||||||
yMean := mean(testY)
|
|
||||||
var ssRes, ssTot float64
|
|
||||||
for i, e := range testEmb {
|
|
||||||
row := make([]float64, p)
|
|
||||||
copy(row, e)
|
|
||||||
row[d] = 1.0
|
|
||||||
pred := dot(row, w)
|
|
||||||
ssRes += (testY[i] - pred) * (testY[i] - pred)
|
|
||||||
ssTot += (testY[i] - yMean) * (testY[i] - yMean)
|
|
||||||
}
|
|
||||||
if ssTot == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
return 1 - ssRes/ssTot
|
|
||||||
}
|
|
||||||
|
|
||||||
// LinearProbe fits a ridge regression (closed-form) on (emb, y) with regularisation λ
|
|
||||||
// and returns R² on the same data. Call with train embeddings; probe on held-out by
|
|
||||||
// splitting before calling.
|
|
||||||
//
|
|
||||||
// emb[i] is the embedding vector for sample i; y[i] is the scalar target.
|
|
||||||
func LinearProbe(emb [][]float64, y []float64, lambda float64) float64 {
|
|
||||||
n := len(emb)
|
|
||||||
if n == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
d := len(emb[0])
|
|
||||||
|
|
||||||
// Build augmented design matrix A = [emb | 1] (n × d+1)
|
|
||||||
A := make([][]float64, n)
|
|
||||||
for i, e := range emb {
|
|
||||||
row := make([]float64, d+1)
|
|
||||||
copy(row, e)
|
|
||||||
row[d] = 1.0
|
|
||||||
A[i] = row
|
|
||||||
}
|
|
||||||
|
|
||||||
// Normal equations: (AᵀA + λI) w = Aᵀy (ridge)
|
|
||||||
p := d + 1
|
|
||||||
AtA := make([][]float64, p)
|
|
||||||
for i := range AtA {
|
|
||||||
AtA[i] = make([]float64, p)
|
|
||||||
}
|
|
||||||
Aty := make([]float64, p)
|
|
||||||
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
Aty[j] += A[i][j] * y[i]
|
|
||||||
for k := 0; k < p; k++ {
|
|
||||||
AtA[j][k] += A[i][j] * A[i][k]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
AtA[j][j] += lambda
|
|
||||||
}
|
|
||||||
|
|
||||||
w := solveCholesky(AtA, Aty)
|
|
||||||
|
|
||||||
// R² = 1 - SS_res / SS_tot
|
|
||||||
yMean := mean(y)
|
|
||||||
var ssRes, ssTot float64
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
pred := dot(A[i], w)
|
|
||||||
ssRes += (y[i] - pred) * (y[i] - pred)
|
|
||||||
ssTot += (y[i] - yMean) * (y[i] - yMean)
|
|
||||||
}
|
|
||||||
if ssTot == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
return 1 - ssRes/ssTot
|
|
||||||
}
|
|
||||||
|
|
||||||
// Silhouette returns the mean silhouette coefficient of the embeddings with respect
|
|
||||||
// to the given integer labels. Distances are Euclidean. Returns an error if fewer
|
|
||||||
// than 2 distinct labels are present.
|
|
||||||
func Silhouette(emb [][]float64, labels []int) (float64, error) {
|
|
||||||
n := len(emb)
|
|
||||||
if n == 0 {
|
|
||||||
return 0, errors.New("eval: empty embeddings")
|
|
||||||
}
|
|
||||||
|
|
||||||
// count distinct labels
|
|
||||||
labelSet := map[int]struct{}{}
|
|
||||||
for _, l := range labels {
|
|
||||||
labelSet[l] = struct{}{}
|
|
||||||
}
|
|
||||||
if len(labelSet) < 2 {
|
|
||||||
return 0, errors.New("eval: silhouette requires at least 2 distinct labels")
|
|
||||||
}
|
|
||||||
|
|
||||||
// group indices by label
|
|
||||||
groups := map[int][]int{}
|
|
||||||
for i, l := range labels {
|
|
||||||
groups[l] = append(groups[l], i)
|
|
||||||
}
|
|
||||||
|
|
||||||
var total float64
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
li := labels[i]
|
|
||||||
|
|
||||||
// a(i) = mean intra-cluster distance
|
|
||||||
var aSum float64
|
|
||||||
inGroup := groups[li]
|
|
||||||
for _, j := range inGroup {
|
|
||||||
if j != i {
|
|
||||||
aSum += euclidean(emb[i], emb[j])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
var a float64
|
|
||||||
if len(inGroup) > 1 {
|
|
||||||
a = aSum / float64(len(inGroup)-1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// b(i) = min mean inter-cluster distance
|
|
||||||
b := math.MaxFloat64
|
|
||||||
for l, idxs := range groups {
|
|
||||||
if l == li {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
var dSum float64
|
|
||||||
for _, j := range idxs {
|
|
||||||
dSum += euclidean(emb[i], emb[j])
|
|
||||||
}
|
|
||||||
avg := dSum / float64(len(idxs))
|
|
||||||
if avg < b {
|
|
||||||
b = avg
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
s := (b - a) / math.Max(a, b)
|
|
||||||
total += s
|
|
||||||
}
|
|
||||||
return total / float64(n), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// EffectiveRank computes Roy's effective rank of the embedding matrix:
|
|
||||||
// exp(H) where H = -∑ pᵢ log(pᵢ) is the Shannon entropy of the normalised
|
|
||||||
// squared singular values. Returns 1 for a rank-1 matrix and ≈ dim for
|
|
||||||
// a full-rank isotropic matrix.
|
|
||||||
func EffectiveRank(emb [][]float64) float64 {
|
|
||||||
n := len(emb)
|
|
||||||
if n == 0 {
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
d := len(emb[0])
|
|
||||||
|
|
||||||
// Compute covariance-like matrix CᵀC where C is mean-centered embedding.
|
|
||||||
mu := make([]float64, d)
|
|
||||||
for _, e := range emb {
|
|
||||||
for j, v := range e {
|
|
||||||
mu[j] += v
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := range mu {
|
|
||||||
mu[j] /= float64(n)
|
|
||||||
}
|
|
||||||
|
|
||||||
// C = emb - mu (n × d); compute CᵀC (d × d)
|
|
||||||
CtC := make([][]float64, d)
|
|
||||||
for i := range CtC {
|
|
||||||
CtC[i] = make([]float64, d)
|
|
||||||
}
|
|
||||||
for _, e := range emb {
|
|
||||||
for j := 0; j < d; j++ {
|
|
||||||
cj := e[j] - mu[j]
|
|
||||||
for k := 0; k < d; k++ {
|
|
||||||
CtC[j][k] += cj * (e[k] - mu[k])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Eigenvalues of CᵀC via power iteration approximation isn't great;
|
|
||||||
// use the Frobenius / trace approach: σᵢ² ∝ eigenvalues of CᵀC.
|
|
||||||
// For a pure-Go impl without LAPACK: use the fact that the normalised
|
|
||||||
// squared singular values equal normalised eigenvalues of CᵀC.
|
|
||||||
// Compute them via Jacobi iteration for small d, or use the analytical
|
|
||||||
// formula for 2×2, or use iterative QR for general d.
|
|
||||||
eigs := jacobiEigenvalues(CtC)
|
|
||||||
|
|
||||||
// normalise to sum-1 distribution
|
|
||||||
var sumEig float64
|
|
||||||
for _, v := range eigs {
|
|
||||||
if v > 0 {
|
|
||||||
sumEig += v
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if sumEig == 0 {
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
var H float64
|
|
||||||
for _, v := range eigs {
|
|
||||||
if v > 0 {
|
|
||||||
p := v / sumEig
|
|
||||||
H -= p * math.Log(p)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return math.Exp(H)
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── internal helpers ──────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
func euclidean(a, b []float64) float64 {
|
|
||||||
var s float64
|
|
||||||
for i := range a {
|
|
||||||
d := a[i] - b[i]
|
|
||||||
s += d * d
|
|
||||||
}
|
|
||||||
return math.Sqrt(s)
|
|
||||||
}
|
|
||||||
|
|
||||||
func dot(a, b []float64) float64 {
|
|
||||||
var s float64
|
|
||||||
for i := range a {
|
|
||||||
s += a[i] * b[i]
|
|
||||||
}
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
func mean(y []float64) float64 {
|
|
||||||
var s float64
|
|
||||||
for _, v := range y {
|
|
||||||
s += v
|
|
||||||
}
|
|
||||||
return s / float64(len(y))
|
|
||||||
}
|
|
||||||
|
|
||||||
// solveCholesky solves Ax = b for symmetric positive-definite A via
|
|
||||||
// Cholesky decomposition. Falls back to pseudo-inverse on failure.
|
|
||||||
func solveCholesky(A [][]float64, b []float64) []float64 {
|
|
||||||
n := len(A)
|
|
||||||
// Cholesky decomposition: A = LLᵀ
|
|
||||||
L := make([][]float64, n)
|
|
||||||
for i := range L {
|
|
||||||
L[i] = make([]float64, n)
|
|
||||||
}
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
for j := 0; j <= i; j++ {
|
|
||||||
s := A[i][j]
|
|
||||||
for k := 0; k < j; k++ {
|
|
||||||
s -= L[i][k] * L[j][k]
|
|
||||||
}
|
|
||||||
if i == j {
|
|
||||||
if s <= 0 {
|
|
||||||
s = 1e-12
|
|
||||||
}
|
|
||||||
L[i][j] = math.Sqrt(s)
|
|
||||||
} else {
|
|
||||||
L[i][j] = s / L[j][j]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Forward substitution Ly = b
|
|
||||||
y := make([]float64, n)
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
s := b[i]
|
|
||||||
for k := 0; k < i; k++ {
|
|
||||||
s -= L[i][k] * y[k]
|
|
||||||
}
|
|
||||||
y[i] = s / L[i][i]
|
|
||||||
}
|
|
||||||
// Back substitution Lᵀx = y
|
|
||||||
x := make([]float64, n)
|
|
||||||
for i := n - 1; i >= 0; i-- {
|
|
||||||
s := y[i]
|
|
||||||
for k := i + 1; k < n; k++ {
|
|
||||||
s -= L[k][i] * x[k]
|
|
||||||
}
|
|
||||||
x[i] = s / L[i][i]
|
|
||||||
}
|
|
||||||
return x
|
|
||||||
}
|
|
||||||
|
|
||||||
// jacobiEigenvalues returns eigenvalues of a symmetric matrix via Jacobi iteration.
|
|
||||||
func jacobiEigenvalues(A [][]float64) []float64 {
|
|
||||||
n := len(A)
|
|
||||||
// copy
|
|
||||||
a := make([][]float64, n)
|
|
||||||
for i := range a {
|
|
||||||
a[i] = make([]float64, n)
|
|
||||||
copy(a[i], A[i])
|
|
||||||
}
|
|
||||||
|
|
||||||
const maxIter = 100
|
|
||||||
const tol = 1e-10
|
|
||||||
for iter := 0; iter < maxIter; iter++ {
|
|
||||||
// find largest off-diagonal element
|
|
||||||
p, q, amax := 0, 1, 0.0
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
for j := i + 1; j < n; j++ {
|
|
||||||
if v := math.Abs(a[i][j]); v > amax {
|
|
||||||
amax = v
|
|
||||||
p, q = i, j
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if amax < tol {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
// Jacobi rotation
|
|
||||||
theta := 0.5 * math.Atan2(2*a[p][q], a[q][q]-a[p][p])
|
|
||||||
c, s := math.Cos(theta), math.Sin(theta)
|
|
||||||
// apply rotation
|
|
||||||
newA := make([][]float64, n)
|
|
||||||
for i := range newA {
|
|
||||||
newA[i] = make([]float64, n)
|
|
||||||
copy(newA[i], a[i])
|
|
||||||
}
|
|
||||||
app := c*c*a[p][p] + 2*c*s*a[p][q] + s*s*a[q][q]
|
|
||||||
aqq := s*s*a[p][p] - 2*c*s*a[p][q] + c*c*a[q][q]
|
|
||||||
apq := 0.0
|
|
||||||
newA[p][p] = app
|
|
||||||
newA[q][q] = aqq
|
|
||||||
newA[p][q] = apq
|
|
||||||
newA[q][p] = apq
|
|
||||||
for r := 0; r < n; r++ {
|
|
||||||
if r == p || r == q {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
arp := c*a[r][p] + s*a[r][q]
|
|
||||||
arq := -s*a[r][p] + c*a[r][q]
|
|
||||||
newA[r][p] = arp
|
|
||||||
newA[p][r] = arp
|
|
||||||
newA[r][q] = arq
|
|
||||||
newA[q][r] = arq
|
|
||||||
}
|
|
||||||
a = newA
|
|
||||||
}
|
|
||||||
|
|
||||||
eigs := make([]float64, n)
|
|
||||||
for i := range eigs {
|
|
||||||
eigs[i] = a[i][i]
|
|
||||||
}
|
|
||||||
return eigs
|
|
||||||
}
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
package eval_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"math"
|
|
||||||
"math/rand"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
|
||||||
)
|
|
||||||
|
|
||||||
func seededRNG(seed int64) *rand.Rand {
|
|
||||||
return rand.New(rand.NewSource(seed))
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── LinearProbe (val_vol_r2) ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
func TestLinearProbe_Perfect(t *testing.T) {
|
|
||||||
n := 50
|
|
||||||
emb := make([][]float64, n)
|
|
||||||
y := make([]float64, n)
|
|
||||||
for i := range emb {
|
|
||||||
emb[i] = []float64{float64(i)}
|
|
||||||
y[i] = float64(i)
|
|
||||||
}
|
|
||||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
|
||||||
if r2 < 0.99 {
|
|
||||||
t.Fatalf("perfect predictor: want R²≥0.99, got %.4f", r2)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLinearProbe_ConstantTarget(t *testing.T) {
|
|
||||||
n := 40
|
|
||||||
emb := make([][]float64, n)
|
|
||||||
y := make([]float64, n)
|
|
||||||
for i := range emb {
|
|
||||||
emb[i] = []float64{float64(i), float64(i * i)}
|
|
||||||
y[i] = 3.0
|
|
||||||
}
|
|
||||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
|
||||||
if r2 > 0.01 {
|
|
||||||
t.Fatalf("constant target: want R²≤0.01, got %.4f", r2)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLinearProbe_NoiseEmbedding(t *testing.T) {
|
|
||||||
rng := seededRNG(42)
|
|
||||||
n := 80
|
|
||||||
emb := make([][]float64, n)
|
|
||||||
y := make([]float64, n)
|
|
||||||
for i := range emb {
|
|
||||||
emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
|
|
||||||
y[i] = float64(i)
|
|
||||||
}
|
|
||||||
r2 := eval.LinearProbe(emb, y, 1e-3)
|
|
||||||
if r2 > 0.10 {
|
|
||||||
t.Fatalf("noise embedding: want R²<0.10, got %.4f", r2)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── Silhouette ────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
func TestSilhouette_PerfectClusters(t *testing.T) {
|
|
||||||
emb := make([][]float64, 40)
|
|
||||||
labels := make([]int, 40)
|
|
||||||
for i := range emb {
|
|
||||||
if i < 20 {
|
|
||||||
emb[i] = []float64{0.0, 0.0}
|
|
||||||
labels[i] = 0
|
|
||||||
} else {
|
|
||||||
emb[i] = []float64{1000.0, 1000.0}
|
|
||||||
labels[i] = 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
sil, err := eval.Silhouette(emb, labels)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if sil < 0.95 {
|
|
||||||
t.Fatalf("perfect clusters: want sil≥0.95, got %.4f", sil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSilhouette_SingleLabel(t *testing.T) {
|
|
||||||
emb := [][]float64{{1, 2}, {3, 4}, {5, 6}}
|
|
||||||
labels := []int{0, 0, 0}
|
|
||||||
_, err := eval.Silhouette(emb, labels)
|
|
||||||
if err == nil {
|
|
||||||
t.Fatal("expected error for single-label input")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSilhouette_RandomClusters(t *testing.T) {
|
|
||||||
rng := seededRNG(7)
|
|
||||||
n := 60
|
|
||||||
emb := make([][]float64, n)
|
|
||||||
labels := make([]int, n)
|
|
||||||
for i := range emb {
|
|
||||||
emb[i] = []float64{rng.NormFloat64(), rng.NormFloat64()}
|
|
||||||
labels[i] = i % 2
|
|
||||||
}
|
|
||||||
sil, err := eval.Silhouette(emb, labels)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
if math.Abs(sil) > 0.30 {
|
|
||||||
t.Fatalf("random clusters: want |sil|≤0.30, got %.4f", sil)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── EffectiveRank ─────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
func TestEffectiveRank_Rank1(t *testing.T) {
|
|
||||||
emb := make([][]float64, 30)
|
|
||||||
for i := range emb {
|
|
||||||
emb[i] = []float64{1.0, 2.0, 3.0, 4.0}
|
|
||||||
}
|
|
||||||
er := eval.EffectiveRank(emb)
|
|
||||||
if er > 1.5 {
|
|
||||||
t.Fatalf("rank-1 matrix: want erank≤1.5, got %.4f", er)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestEffectiveRank_FullRank(t *testing.T) {
|
|
||||||
rng := seededRNG(99)
|
|
||||||
dim := 8
|
|
||||||
emb := make([][]float64, 200)
|
|
||||||
for i := range emb {
|
|
||||||
row := make([]float64, dim)
|
|
||||||
for j := range row {
|
|
||||||
row[j] = rng.NormFloat64()
|
|
||||||
}
|
|
||||||
emb[i] = row
|
|
||||||
}
|
|
||||||
er := eval.EffectiveRank(emb)
|
|
||||||
if er < float64(dim)*0.7 {
|
|
||||||
t.Fatalf("full-rank: want erank≥%.1f, got %.4f", float64(dim)*0.7, er)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
package eval
|
|
||||||
|
|
||||||
import "math"
|
|
||||||
|
|
||||||
// VaRBreachRate computes the parametric 99% VaR breach rate and Kupiec POF p-value.
|
|
||||||
//
|
|
||||||
// VaR_99_t = predVol[t] × z99 (z99 = 2.326 for 99% normal VaR)
|
|
||||||
// breach_t = actualVol[t] > VaR_99_t (strict inequality)
|
|
||||||
// breachRate = fraction of breaches over all steps
|
|
||||||
// kupiecP = Kupiec POF p-value: P(chi²(1) > LR) where LR is the likelihood ratio
|
|
||||||
//
|
|
||||||
// testing H0: true breach probability = 1%. High p = well-calibrated.
|
|
||||||
//
|
|
||||||
// Returns (0, 1) for empty or mismatched input.
|
|
||||||
func VaRBreachRate(predVol, actualVol []float64, z99 float64) (breachRate, kupiecP float64) {
|
|
||||||
n := len(predVol)
|
|
||||||
if n == 0 || n != len(actualVol) {
|
|
||||||
return 0, 1
|
|
||||||
}
|
|
||||||
|
|
||||||
var n1 int
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
if actualVol[i] > predVol[i]*z99 {
|
|
||||||
n1++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
breachRate = float64(n1) / float64(n)
|
|
||||||
kupiecP = kupiecPOF(n, n1, 0.01)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// kupiecPOF returns the Kupiec Proportion-of-Failures p-value.
|
|
||||||
// H0: true breach probability = p0 (e.g. 0.01 for 99% VaR).
|
|
||||||
// Returns 1.0 for edge cases (n=0, p_hat=p0).
|
|
||||||
func kupiecPOF(n, n1 int, p0 float64) float64 {
|
|
||||||
if n == 0 {
|
|
||||||
return 1.0
|
|
||||||
}
|
|
||||||
n0 := n - n1
|
|
||||||
phat := float64(n1) / float64(n)
|
|
||||||
|
|
||||||
var lr float64
|
|
||||||
switch n1 {
|
|
||||||
case 0:
|
|
||||||
// 0 × ln(0/p0) = 0 by convention; only the n0 term contributes
|
|
||||||
lr = 2 * float64(n0) * math.Log((1-phat)/(1-p0))
|
|
||||||
case n:
|
|
||||||
// n0 term vanishes
|
|
||||||
lr = 2 * float64(n1) * math.Log(phat/p0)
|
|
||||||
default:
|
|
||||||
lr = 2 * (float64(n1)*math.Log(phat/p0) + float64(n0)*math.Log((1-phat)/(1-p0)))
|
|
||||||
}
|
|
||||||
|
|
||||||
if lr <= 0 {
|
|
||||||
return 1.0
|
|
||||||
}
|
|
||||||
// P(chi²(1) > LR) = erfc(sqrt(LR/2)) [chi²(1) = Z², Z~N(0,1)]
|
|
||||||
return math.Erfc(math.Sqrt(lr / 2))
|
|
||||||
}
|
|
||||||
|
|
||||||
// LinearProbePredict fits ridge regression on (trainEmb, trainY) and returns
|
|
||||||
// predictions for testEmb. Complements LinearProbeTrainTest when the caller
|
|
||||||
// needs the raw predictions (e.g. to compute VaR breach rate).
|
|
||||||
// Returns nil when trainEmb is empty.
|
|
||||||
func LinearProbePredict(trainEmb [][]float64, trainY []float64,
|
|
||||||
testEmb [][]float64, lambda float64,
|
|
||||||
) []float64 {
|
|
||||||
n := len(trainEmb)
|
|
||||||
if n == 0 || len(testEmb) == 0 {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
d := len(trainEmb[0])
|
|
||||||
p := d + 1
|
|
||||||
|
|
||||||
A := make([][]float64, n)
|
|
||||||
for i, e := range trainEmb {
|
|
||||||
row := make([]float64, p)
|
|
||||||
copy(row, e)
|
|
||||||
row[d] = 1.0
|
|
||||||
A[i] = row
|
|
||||||
}
|
|
||||||
AtA := make([][]float64, p)
|
|
||||||
for i := range AtA {
|
|
||||||
AtA[i] = make([]float64, p)
|
|
||||||
}
|
|
||||||
Aty := make([]float64, p)
|
|
||||||
for i := 0; i < n; i++ {
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
Aty[j] += A[i][j] * trainY[i]
|
|
||||||
for k := 0; k < p; k++ {
|
|
||||||
AtA[j][k] += A[i][j] * A[i][k]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for j := 0; j < p; j++ {
|
|
||||||
AtA[j][j] += lambda
|
|
||||||
}
|
|
||||||
w := solveCholesky(AtA, Aty)
|
|
||||||
|
|
||||||
preds := make([]float64, len(testEmb))
|
|
||||||
for i, e := range testEmb {
|
|
||||||
row := make([]float64, p)
|
|
||||||
copy(row, e)
|
|
||||||
row[d] = 1.0
|
|
||||||
preds[i] = dot(row, w)
|
|
||||||
}
|
|
||||||
return preds
|
|
||||||
}
|
|
||||||
@@ -1,138 +0,0 @@
|
|||||||
package eval_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"math"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"git.d-ma.be/mathias/jepa-fx-risk/internal/eval"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ── VaRBreachRate golden tests ──────────────────────────────────────────────
|
|
||||||
//
|
|
||||||
// VaR_99_t = predVol[t] × z99 (parametric 99% normal VaR)
|
|
||||||
// breach_t = actualVol[t] > VaR_99_t
|
|
||||||
// breachRate = mean(breach_t)
|
|
||||||
// kupiecP = Kupiec POF p-value (chi²(1) test, H0: breach rate = 1%)
|
|
||||||
|
|
||||||
func TestVaRBreachRate_ZeroBreaches(t *testing.T) {
|
|
||||||
// 0.02 < 0.01×2.326=0.02326 → no breaches
|
|
||||||
pred := []float64{0.01, 0.01, 0.01}
|
|
||||||
act := []float64{0.02, 0.02, 0.02}
|
|
||||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
|
||||||
if rate != 0 {
|
|
||||||
t.Fatalf("want rate=0, got %.4f", rate)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_AllBreach(t *testing.T) {
|
|
||||||
// 0.03 > 0.02326 → all breach
|
|
||||||
pred := []float64{0.01, 0.01}
|
|
||||||
act := []float64{0.03, 0.03}
|
|
||||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
|
||||||
if math.Abs(rate-1.0) > 1e-9 {
|
|
||||||
t.Fatalf("want rate=1.0, got %.4f", rate)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_Golden(t *testing.T) {
|
|
||||||
// n=10, 2 breaches at indices 0 and 2 → rate=0.2
|
|
||||||
// Kupiec: p_hat=0.2 vs p0=0.01 → strongly reject H0 (p < 0.05)
|
|
||||||
pred := make([]float64, 10)
|
|
||||||
act := make([]float64, 10)
|
|
||||||
for i := range pred {
|
|
||||||
pred[i] = 0.01
|
|
||||||
act[i] = 0.01 // no breach: 0.01 < 0.02326
|
|
||||||
}
|
|
||||||
act[0] = 0.03 // breach
|
|
||||||
act[2] = 0.03 // breach
|
|
||||||
|
|
||||||
rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
|
|
||||||
|
|
||||||
if math.Abs(rate-0.2) > 1e-9 {
|
|
||||||
t.Fatalf("breach rate: want 0.2, got %.4f", rate)
|
|
||||||
}
|
|
||||||
if kupiecP > 0.05 {
|
|
||||||
t.Fatalf("kupiec p-value: want <0.05 (strong reject H0), got %.4f", kupiecP)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_PerfectCalibration(t *testing.T) {
|
|
||||||
// n=100, exactly 1 breach → p_hat=0.01=p0 → LR=0 → kupiecP≈1.0
|
|
||||||
n := 100
|
|
||||||
pred := make([]float64, n)
|
|
||||||
act := make([]float64, n)
|
|
||||||
for i := range pred {
|
|
||||||
pred[i] = 0.01
|
|
||||||
act[i] = 0.015 // < 0.02326, no breach
|
|
||||||
}
|
|
||||||
act[0] = 0.025 // > 0.02326, breach
|
|
||||||
|
|
||||||
rate, kupiecP := eval.VaRBreachRate(pred, act, 2.326)
|
|
||||||
|
|
||||||
if math.Abs(rate-0.01) > 1e-9 {
|
|
||||||
t.Fatalf("breach rate: want 0.01, got %.4f", rate)
|
|
||||||
}
|
|
||||||
if kupiecP < 0.9 {
|
|
||||||
t.Fatalf("kupiec p-value: want ≈1.0 (well calibrated), got %.4f", kupiecP)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_EmptyInput(t *testing.T) {
|
|
||||||
rate, kupiecP := eval.VaRBreachRate(nil, nil, 2.326)
|
|
||||||
if rate != 0 || kupiecP != 1 {
|
|
||||||
t.Fatalf("empty: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_LenMismatch(t *testing.T) {
|
|
||||||
rate, kupiecP := eval.VaRBreachRate([]float64{0.01}, []float64{0.01, 0.02}, 2.326)
|
|
||||||
if rate != 0 || kupiecP != 1 {
|
|
||||||
t.Fatalf("mismatch: want (0,1), got (%.4f,%.4f)", rate, kupiecP)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVaRBreachRate_Z99Default(t *testing.T) {
|
|
||||||
// z99=2.326 is the canonical value; test that boundary case works
|
|
||||||
// VaR = 0.01 × 2.326 = 0.02326
|
|
||||||
// actual = 0.02326 → NOT a breach (strict >)
|
|
||||||
pred := []float64{0.01}
|
|
||||||
act := []float64{0.02326}
|
|
||||||
rate, _ := eval.VaRBreachRate(pred, act, 2.326)
|
|
||||||
if rate != 0 {
|
|
||||||
t.Fatalf("boundary: exactly at VaR is not a breach; want rate=0, got %.4f", rate)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ── LinearProbePredict ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
func TestLinearProbePredict_PerfectLinear(t *testing.T) {
|
|
||||||
// y = x; predictions should match targets closely
|
|
||||||
n := 20
|
|
||||||
trainEmb := make([][]float64, n)
|
|
||||||
trainY := make([]float64, n)
|
|
||||||
testEmb := make([][]float64, 5)
|
|
||||||
testY := []float64{5, 10, 15, 20, 25}
|
|
||||||
for i := range trainEmb {
|
|
||||||
trainEmb[i] = []float64{float64(i)}
|
|
||||||
trainY[i] = float64(i)
|
|
||||||
}
|
|
||||||
for i := range testEmb {
|
|
||||||
testEmb[i] = []float64{testY[i]}
|
|
||||||
}
|
|
||||||
preds := eval.LinearProbePredict(trainEmb, trainY, testEmb, 1e-3)
|
|
||||||
if len(preds) != len(testEmb) {
|
|
||||||
t.Fatalf("len: want %d, got %d", len(testEmb), len(preds))
|
|
||||||
}
|
|
||||||
for i, p := range preds {
|
|
||||||
if math.Abs(p-testY[i]) > 1.0 {
|
|
||||||
t.Fatalf("pred[%d]: want ≈%.1f, got %.4f", i, testY[i], p)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLinearProbePredict_EmptyTrain(t *testing.T) {
|
|
||||||
preds := eval.LinearProbePredict(nil, nil, [][]float64{{1.0}}, 1e-3)
|
|
||||||
if len(preds) != 0 {
|
|
||||||
t.Fatalf("empty train: want nil/empty preds, got len=%d", len(preds))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
// Code generated by templ - DO NOT EDIT.
|
|
||||||
|
|
||||||
// templ: version: v0.3.1020
|
|
||||||
package web
|
|
||||||
|
|
||||||
//lint:file-ignore SA4006 This context is only used if a nested component is present.
|
|
||||||
|
|
||||||
import "github.com/a-h/templ"
|
|
||||||
import templruntime "github.com/a-h/templ/runtime"
|
|
||||||
|
|
||||||
func Index() templ.Component {
|
|
||||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
|
||||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
|
||||||
return templ_7745c5c3_CtxErr
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
|
||||||
if !templ_7745c5c3_IsBuffer {
|
|
||||||
defer func() {
|
|
||||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err == nil {
|
|
||||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
}
|
|
||||||
ctx = templ.InitializeContext(ctx)
|
|
||||||
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
|
|
||||||
if templ_7745c5c3_Var1 == nil {
|
|
||||||
templ_7745c5c3_Var1 = templ.NopComponent
|
|
||||||
}
|
|
||||||
ctx = templ.ClearChildren(ctx)
|
|
||||||
templ_7745c5c3_Var2 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
|
||||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
|
||||||
if !templ_7745c5c3_IsBuffer {
|
|
||||||
defer func() {
|
|
||||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err == nil {
|
|
||||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
}
|
|
||||||
ctx = templ.InitializeContext(ctx)
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<h1 class=\"text-3xl font-semibold mb-6\">hostexecutor</h1><button hx-get=\"/api/hello\" hx-target=\"#out\" class=\"px-4 py-2 bg-slate-900 text-white rounded-md hover:bg-slate-700\">Say hello</button><div id=\"out\" class=\"mt-6 text-slate-700\"></div>")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
templ_7745c5c3_Err = Layout("hostexecutor").Render(templ.WithChildren(ctx, templ_7745c5c3_Var2), templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func Hello(name string) templ.Component {
|
|
||||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
|
||||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
|
||||||
return templ_7745c5c3_CtxErr
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
|
||||||
if !templ_7745c5c3_IsBuffer {
|
|
||||||
defer func() {
|
|
||||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err == nil {
|
|
||||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
}
|
|
||||||
ctx = templ.InitializeContext(ctx)
|
|
||||||
templ_7745c5c3_Var3 := templ.GetChildren(ctx)
|
|
||||||
if templ_7745c5c3_Var3 == nil {
|
|
||||||
templ_7745c5c3_Var3 = templ.NopComponent
|
|
||||||
}
|
|
||||||
ctx = templ.ClearChildren(ctx)
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "<p>Hello, ")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
var templ_7745c5c3_Var4 string
|
|
||||||
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(name)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/index.templ`, Line: 15, Col: 17}
|
|
||||||
}
|
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "!</p>")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
var _ = templruntime.GeneratedTemplate
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
// Code generated by templ - DO NOT EDIT.
|
|
||||||
|
|
||||||
// templ: version: v0.3.1020
|
|
||||||
package web
|
|
||||||
|
|
||||||
//lint:file-ignore SA4006 This context is only used if a nested component is present.
|
|
||||||
|
|
||||||
import "github.com/a-h/templ"
|
|
||||||
import templruntime "github.com/a-h/templ/runtime"
|
|
||||||
|
|
||||||
func Layout(title string) templ.Component {
|
|
||||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
|
||||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
|
||||||
return templ_7745c5c3_CtxErr
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
|
||||||
if !templ_7745c5c3_IsBuffer {
|
|
||||||
defer func() {
|
|
||||||
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err == nil {
|
|
||||||
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
}
|
|
||||||
ctx = templ.InitializeContext(ctx)
|
|
||||||
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
|
|
||||||
if templ_7745c5c3_Var1 == nil {
|
|
||||||
templ_7745c5c3_Var1 = templ.NopComponent
|
|
||||||
}
|
|
||||||
ctx = templ.ClearChildren(ctx)
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<!doctype html><html lang=\"en\"><head><meta charset=\"utf-8\"><meta name=\"viewport\" content=\"width=device-width,initial-scale=1\"><title>")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
var templ_7745c5c3_Var2 string
|
|
||||||
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(title)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/layout.templ`, Line: 9, Col: 17}
|
|
||||||
}
|
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "</title><script src=\"https://unpkg.com/htmx.org@2.0.0\"></script><script src=\"https://cdn.tailwindcss.com\"></script></head><body class=\"min-h-screen bg-slate-50 text-slate-900 antialiased\"><main class=\"max-w-3xl mx-auto px-6 py-12\">")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Err = templ_7745c5c3_Var1.Render(ctx, templ_7745c5c3_Buffer)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "</main></body></html>")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
var _ = templruntime.GeneratedTemplate
|
|
||||||
@@ -1,287 +0,0 @@
|
|||||||
"""loop.py — Karpathy-style autoresearch loop for jepa-fx-risk.
|
|
||||||
|
|
||||||
Agent (on iguana/berget — NOT koala, whose GPU is reserved for train.py) reads
|
|
||||||
program.md + train.py + STATUS.md, proposes ONE change to train.py, we run it,
|
|
||||||
keep if val_vol_r2 improved else git-revert. Appends per-iter record to STATUS.md.
|
|
||||||
|
|
||||||
LITELLM_KEY=xxx python loop.py [--iters N] [--model MODEL] [--run-dir runs/rq-04]
|
|
||||||
|
|
||||||
Env:
|
|
||||||
LITELLM_KEY — LiteLLM master key (required)
|
|
||||||
LITELLM_BASE — default http://localhost:30401/v1
|
|
||||||
LOOP_MODEL — default berget/gemma4-31b (non-thinking; iguana/berget only)
|
|
||||||
LOOP_ITERS — default 3
|
|
||||||
TRAIN_TIMEOUT — seconds per train.py run, default 120
|
|
||||||
NTFY_URL — optional: POST crash/stall alerts here (e.g. ntfy.sh/<topic>)
|
|
||||||
"""
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import time
|
|
||||||
import textwrap
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import urllib.request
|
|
||||||
|
|
||||||
LITELLM_BASE = os.environ.get("LITELLM_BASE", "http://localhost:30401/v1")
|
|
||||||
LITELLM_KEY = os.environ.get("LITELLM_KEY", "")
|
|
||||||
LOOP_MODEL = os.environ.get("LOOP_MODEL", "berget/gemma4-31b")
|
|
||||||
LOOP_ITERS = int(os.environ.get("LOOP_ITERS", "3"))
|
|
||||||
TRAIN_TIMEOUT = int(os.environ.get("TRAIN_TIMEOUT", "120"))
|
|
||||||
NTFY_URL = os.environ.get("NTFY_URL", "")
|
|
||||||
|
|
||||||
# Resolved by main() once --run-dir is parsed.
|
|
||||||
RUN_DIR = Path(".")
|
|
||||||
STATUS_MD = Path("STATUS.md")
|
|
||||||
METRICS_JSON = Path("metrics.json")
|
|
||||||
TRAIN_PY = Path("train.py")
|
|
||||||
HEARTBEAT = Path("HEARTBEAT")
|
|
||||||
|
|
||||||
AGENT_SYSTEM = textwrap.dedent("""\
|
|
||||||
You are the autoresearch agent for jepa-fx-risk. Your job: propose ONE small,
|
|
||||||
targeted change to train.py to improve val_vol_r2 (OOS R² predicting 1-day
|
|
||||||
realized vol from frozen embeddings). Higher is better.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
- Return ONLY the full new content of train.py — nothing else, no explanation,
|
|
||||||
no markdown fence. Raw Python only.
|
|
||||||
- Change ONE thing at a time (one knob, one structural idea).
|
|
||||||
- Do NOT touch prepare_data.py, loop.py, or the data pipeline — only train.py.
|
|
||||||
- Do NOT add new data sources or new files.
|
|
||||||
- The metric is computed externally from your frozen embeddings; trust it.
|
|
||||||
""")
|
|
||||||
|
|
||||||
|
|
||||||
def read_file(p: Path) -> str:
|
|
||||||
return p.read_text() if p.exists() else ""
|
|
||||||
|
|
||||||
|
|
||||||
def gpu_snapshot() -> str:
|
|
||||||
try:
|
|
||||||
out = subprocess.check_output(
|
|
||||||
["nvidia-smi", "--query-gpu=utilization.gpu,memory.used,memory.total,temperature.gpu",
|
|
||||||
"--format=csv,noheader,nounits"], timeout=5, text=True
|
|
||||||
).strip()
|
|
||||||
util, mem_used, mem_total, temp = [x.strip() for x in out.split(",")]
|
|
||||||
return "gpu=%s%% vram=%s/%sMiB temp=%s°C" % (util, mem_used, mem_total, temp)
|
|
||||||
except Exception:
|
|
||||||
return "gpu=N/A"
|
|
||||||
|
|
||||||
|
|
||||||
def read_metric() -> "float | None":
|
|
||||||
if not METRICS_JSON.exists():
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return float(json.loads(METRICS_JSON.read_text())["val_vol_r2"])
|
|
||||||
except Exception:
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def run_train() -> "tuple[float | None, float, str]":
|
|
||||||
"""Run train.py from project root with METRICS_OUT pointing into the run dir."""
|
|
||||||
t0 = time.time()
|
|
||||||
env = dict(os.environ)
|
|
||||||
env["METRICS_OUT"] = str(METRICS_JSON.resolve())
|
|
||||||
# train.py is copied into the run dir, so sys.path[0] is that run dir — which
|
|
||||||
# has no scripts/. Put the project root (where loop.py + scripts/ live) on
|
|
||||||
# PYTHONPATH so train.py's `from scripts.var_breach import ...` resolves.
|
|
||||||
env["PYTHONPATH"] = str(Path(__file__).resolve().parent) + os.pathsep + env.get("PYTHONPATH", "")
|
|
||||||
try:
|
|
||||||
r = subprocess.run(
|
|
||||||
[sys.executable, str(TRAIN_PY.resolve())],
|
|
||||||
capture_output=True, text=True, timeout=TRAIN_TIMEOUT, env=env,
|
|
||||||
)
|
|
||||||
elapsed = time.time() - t0
|
|
||||||
if r.returncode != 0:
|
|
||||||
return None, elapsed, (r.stderr or r.stdout)[-300:]
|
|
||||||
metric = read_metric()
|
|
||||||
return metric, elapsed, ""
|
|
||||||
except subprocess.TimeoutExpired:
|
|
||||||
return None, TRAIN_TIMEOUT, "TIMEOUT"
|
|
||||||
|
|
||||||
|
|
||||||
def call_agent(iteration: int, best_so_far: "float | None") -> str:
|
|
||||||
"""Ask the LLM agent to edit train.py. Returns new train.py content."""
|
|
||||||
context = "\n\n".join([
|
|
||||||
"# program.md\n" + read_file(RUN_DIR / "program.md"),
|
|
||||||
"# train.py (current)\n" + read_file(TRAIN_PY),
|
|
||||||
"# STATUS.md (history)\n" + read_file(STATUS_MD)[-2000:],
|
|
||||||
"# metrics.json (last run)\n" + read_file(METRICS_JSON),
|
|
||||||
"Iteration %d. Best val_vol_r2 so far: %s. Improve it." % (
|
|
||||||
iteration, "%.4f" % best_so_far if best_so_far is not None else "none yet"
|
|
||||||
),
|
|
||||||
])
|
|
||||||
payload = json.dumps({
|
|
||||||
"model": LOOP_MODEL,
|
|
||||||
"messages": [
|
|
||||||
{"role": "system", "content": AGENT_SYSTEM},
|
|
||||||
{"role": "user", "content": context},
|
|
||||||
],
|
|
||||||
"temperature": 0.7,
|
|
||||||
"max_tokens": 4096,
|
|
||||||
}).encode()
|
|
||||||
req = urllib.request.Request(
|
|
||||||
LITELLM_BASE + "/chat/completions",
|
|
||||||
data=payload,
|
|
||||||
headers={"Authorization": "Bearer " + LITELLM_KEY,
|
|
||||||
"Content-Type": "application/json"},
|
|
||||||
method="POST",
|
|
||||||
)
|
|
||||||
resp = urllib.request.urlopen(req, timeout=60)
|
|
||||||
data = json.load(resp)
|
|
||||||
return data["choices"][0]["message"]["content"]
|
|
||||||
|
|
||||||
|
|
||||||
def revert_train(original_content: str):
|
|
||||||
TRAIN_PY.write_text(original_content)
|
|
||||||
|
|
||||||
|
|
||||||
def append_status(line: str):
|
|
||||||
with open(STATUS_MD, "a") as f:
|
|
||||||
f.write(line + "\n")
|
|
||||||
|
|
||||||
|
|
||||||
def write_heartbeat(iteration: int, status: str = "alive"):
|
|
||||||
"""Update HEARTBEAT so watchdogs can detect stalls."""
|
|
||||||
HEARTBEAT.write_text("%s iter=%d ts=%.0f\n" % (status, iteration, time.time()))
|
|
||||||
|
|
||||||
|
|
||||||
def ntfy(msg: str):
|
|
||||||
"""POST an alert to NTFY_URL (best-effort; silently ignored on any error)."""
|
|
||||||
if not NTFY_URL:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
req = urllib.request.Request(
|
|
||||||
NTFY_URL, data=msg.encode(), method="POST",
|
|
||||||
headers={"Content-Type": "text/plain"},
|
|
||||||
)
|
|
||||||
urllib.request.urlopen(req, timeout=5)
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
global RUN_DIR, STATUS_MD, METRICS_JSON, TRAIN_PY, HEARTBEAT
|
|
||||||
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--iters", type=int, default=LOOP_ITERS)
|
|
||||||
parser.add_argument("--model", default=LOOP_MODEL)
|
|
||||||
parser.add_argument(
|
|
||||||
"--run-dir", default=None,
|
|
||||||
help="run dir scaffolded by autoresearch_start.py; "
|
|
||||||
"STATUS.md, metrics.json, HEARTBEAT, and train.py live here",
|
|
||||||
)
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
loop_iters = args.iters
|
|
||||||
loop_model = args.model
|
|
||||||
|
|
||||||
if args.run_dir:
|
|
||||||
RUN_DIR = Path(args.run_dir)
|
|
||||||
if not RUN_DIR.is_dir():
|
|
||||||
print("ERROR: run dir not found:", RUN_DIR); sys.exit(1)
|
|
||||||
|
|
||||||
STATUS_MD = RUN_DIR / "STATUS.md"
|
|
||||||
METRICS_JSON = RUN_DIR / "metrics.json"
|
|
||||||
TRAIN_PY = RUN_DIR / "train.py"
|
|
||||||
HEARTBEAT = RUN_DIR / "HEARTBEAT"
|
|
||||||
|
|
||||||
if not LITELLM_KEY:
|
|
||||||
print("ERROR: set LITELLM_KEY"); sys.exit(1)
|
|
||||||
|
|
||||||
if not STATUS_MD.exists():
|
|
||||||
STATUS_MD.write_text(
|
|
||||||
"# Autoresearch STATUS\n\n"
|
|
||||||
"| iter | val_vol_r2 | delta | action | secs | gpu | change |\n"
|
|
||||||
"|------|-----------|-------|--------|------|-----|--------|\n"
|
|
||||||
)
|
|
||||||
|
|
||||||
baseline = read_metric()
|
|
||||||
if baseline is None:
|
|
||||||
print("No metrics.json — running train.py for baseline...")
|
|
||||||
m, secs, err = run_train()
|
|
||||||
if m is None:
|
|
||||||
msg = "Baseline run failed: " + err
|
|
||||||
print(msg)
|
|
||||||
ntfy("[jepa-fx-risk] loop CRASH — " + msg)
|
|
||||||
sys.exit(1)
|
|
||||||
baseline = m
|
|
||||||
print("Baseline: val_vol_r2 = %.4f (%.1fs)" % (baseline, secs))
|
|
||||||
|
|
||||||
best = baseline
|
|
||||||
print("Starting loop | model=%s | iters=%d | baseline=%.4f" % (loop_model, loop_iters, best))
|
|
||||||
if args.run_dir:
|
|
||||||
print(" run-dir:", RUN_DIR)
|
|
||||||
|
|
||||||
iter_index = 0
|
|
||||||
try:
|
|
||||||
for i in range(1, loop_iters + 1):
|
|
||||||
iter_index = i
|
|
||||||
write_heartbeat(i, "agent-call")
|
|
||||||
print("\n--- iter %d/%d ---" % (i, loop_iters))
|
|
||||||
original = TRAIN_PY.read_text()
|
|
||||||
|
|
||||||
print(" calling agent (%s)..." % loop_model)
|
|
||||||
t_agent = time.time()
|
|
||||||
try:
|
|
||||||
new_code = call_agent(i, best)
|
|
||||||
except Exception as e:
|
|
||||||
msg = str(e)
|
|
||||||
print(" agent call failed:", msg)
|
|
||||||
append_status("| %d | ERR | — | agent-fail | — | — | %s |" % (i, msg[:60]))
|
|
||||||
write_heartbeat(i, "agent-fail")
|
|
||||||
ntfy("[jepa-fx-risk] iter %d agent FAIL — %s" % (i, msg[:80]))
|
|
||||||
continue
|
|
||||||
agent_secs = time.time() - t_agent
|
|
||||||
print(" agent replied in %.1fs" % agent_secs)
|
|
||||||
|
|
||||||
# strip accidental markdown fences
|
|
||||||
if new_code.strip().startswith("```"):
|
|
||||||
lines = new_code.strip().splitlines()
|
|
||||||
new_code = "\n".join(lines[1:-1] if lines[-1].strip() == "```" else lines[1:])
|
|
||||||
|
|
||||||
TRAIN_PY.write_text(new_code)
|
|
||||||
|
|
||||||
write_heartbeat(i, "training")
|
|
||||||
gpu = gpu_snapshot()
|
|
||||||
print(" running train.py [%s]..." % gpu)
|
|
||||||
metric, secs, err = run_train()
|
|
||||||
|
|
||||||
if metric is None:
|
|
||||||
print(" train.py FAILED — reverting. err:", err[:100])
|
|
||||||
revert_train(original)
|
|
||||||
append_status("| %d | FAIL | — | revert | %.0fs | %s | run error |" % (i, secs, gpu))
|
|
||||||
write_heartbeat(i, "train-fail")
|
|
||||||
ntfy("[jepa-fx-risk] iter %d train FAIL — %s" % (i, err[:80]))
|
|
||||||
continue
|
|
||||||
|
|
||||||
delta = metric - best
|
|
||||||
if metric > best:
|
|
||||||
best = metric
|
|
||||||
action = "KEEP"
|
|
||||||
else:
|
|
||||||
revert_train(original)
|
|
||||||
action = "revert"
|
|
||||||
|
|
||||||
summary = "| %d | %.4f | %+.4f | %s | %.0fs | %s | iter%d |" % (
|
|
||||||
i, metric, delta, action, secs, gpu, i)
|
|
||||||
append_status(summary)
|
|
||||||
write_heartbeat(i, "done")
|
|
||||||
print(" val_vol_r2=%.4f delta=%+.4f action=%s [%.0fs]" % (metric, delta, action, secs))
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
msg = "loop CRASH at iter %d: %s" % (iter_index, e)
|
|
||||||
print("FATAL:", msg)
|
|
||||||
ntfy("[jepa-fx-risk] " + msg)
|
|
||||||
raise
|
|
||||||
|
|
||||||
print("\nDone. Best val_vol_r2 = %.4f (baseline was %.4f, delta %+.4f)" % (best, baseline, best - baseline))
|
|
||||||
print("STATUS.md updated.")
|
|
||||||
write_heartbeat(loop_iters, "done")
|
|
||||||
ntfy("[jepa-fx-risk] loop done. best val_vol_r2=%.4f (delta %+.4f)" % (best, best - baseline))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
# Python deps for the autoresearch loop (train.py + scripts). Install torch from
|
|
||||||
# the cu130 index FIRST (koala Blackwell sm_120, torch 2.12.1+cu130 verified):
|
|
||||||
# pip install torch --index-url https://download.pytorch.org/whl/cu130
|
|
||||||
# pip install -r requirements.txt
|
|
||||||
numpy>=2.0
|
|
||||||
pandas>=2.2
|
|
||||||
pyarrow>=16
|
|
||||||
histdata>=1.1 # histdata.com downloader (1.1 is newest on PyPI; 1.3 never existed)
|
|
||||||
hmmlearn>=0.3 # regime detector (prepare_regime.py, jepa-fx-risk#13)
|
|
||||||
scikit-learn>=1.4 # HMM dependency
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.302411480667525, "phase1_r2": 0.35809940099716187, "stdout_last": "val_vol_r2 = 0.3024 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:37.028406"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29654798431244755, "phase1_r2": 0.35618388652801514, "stdout_last": "val_vol_r2 = 0.2965 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:37:49.822762"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.31050360040290237, "phase1_r2": 0.36530405282974243, "stdout_last": "val_vol_r2 = 0.3105 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:02.966176"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.2925057399716364, "phase1_r2": 0.3467639684677124, "stdout_last": "val_vol_r2 = 0.2925 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:16.195552"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.29334667623516786, "phase1_r2": 0.35872191190719604, "stdout_last": "val_vol_r2 = 0.2933 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:31.372602"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 64, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3123527205416422, "phase1_r2": 0.3572431206703186, "stdout_last": "val_vol_r2 = 0.3124 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:38:46.672203"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3641397896593044, "phase1_r2": 0.3908407688140869, "stdout_last": "val_vol_r2 = 0.3641 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:00.793878"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.35845865364171503, "phase1_r2": 0.3737195134162903, "stdout_last": "val_vol_r2 = 0.3585 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:13.321428"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.35310115657814645, "phase1_r2": 0.35306859016418457, "stdout_last": "val_vol_r2 = 0.3531 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:26.402249"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3655629727960601, "phase1_r2": 0.371029257774353, "stdout_last": "val_vol_r2 = 0.3656 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:39.786248"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.36109622605593217, "phase1_r2": 0.3666273355484009, "stdout_last": "val_vol_r2 = 0.3611 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:39:53.194111"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 128, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.362228341965093, "phase1_r2": 0.3590735197067261, "stdout_last": "val_vol_r2 = 0.3622 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:06.991680"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 120}, "val_vol_r2": 0.3749483295047378, "phase1_r2": 0.3801569938659668, "stdout_last": "val_vol_r2 = 0.3749 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:21.512210"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 240}, "val_vol_r2": 0.3765593861479334, "phase1_r2": 0.38416117429733276, "stdout_last": "val_vol_r2 = 0.3766 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:34.959919"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 2, "JEPA_WINDOW": 480}, "val_vol_r2": 0.3653399117639956, "phase1_r2": 0.3685130476951599, "stdout_last": "val_vol_r2 = 0.3653 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:40:48.806135"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 120}, "val_vol_r2": 0.375961424966925, "phase1_r2": 0.37861257791519165, "stdout_last": "val_vol_r2 = 0.3760 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:04.056939"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 240}, "val_vol_r2": 0.37841726893098504, "phase1_r2": 0.3781360387802124, "stdout_last": "val_vol_r2 = 0.3784 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:18.693263"}
|
|
||||||
{"config": {"JEPA_D_MODEL": 256, "JEPA_DEPTH": 4, "JEPA_WINDOW": 480}, "val_vol_r2": 0.37118530199441635, "phase1_r2": 0.3651617765426636, "stdout_last": "val_vol_r2 = 0.3712 (n_test=11641, dev=cuda)", "ts": "2026-06-26T10:41:34.796157"}
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
{
|
|
||||||
"label": "null",
|
|
||||||
"mean_sil": 0.018206419112781685,
|
|
||||||
"pca_sil": 0.13589094579219818,
|
|
||||||
"spread": 0.8673340065023978,
|
|
||||||
"pc1_hv_corr": 0.525803392278542,
|
|
||||||
"per_seed": [
|
|
||||||
0.026790648698806763,
|
|
||||||
0.010999602265655994,
|
|
||||||
0.016829006373882294
|
|
||||||
],
|
|
||||||
"passed": false
|
|
||||||
}
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
# Phase-0 SSL feasibility gate — null
|
|
||||||
|
|
||||||
**Date:** 2026-06-24
|
|
||||||
**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
|
|
||||||
(not Go #4); gate metric adapted from silhouette-on-embedding to match
|
|
||||||
available data. Go harness (#4) remains open for production experiments.
|
|
||||||
|
|
||||||
## Data
|
|
||||||
- Train: EUR/USD daily 2019-2021 (907 windows)
|
|
||||||
- OOS: EUR/USD daily 2022-2023 (593 windows)
|
|
||||||
- HV label: top-33% realized-vol days = high-volatility (196 days)
|
|
||||||
|
|
||||||
## Results
|
|
||||||
| | Value | Gate |
|
|
||||||
|---|---|---|
|
|
||||||
| TS-JEPA mean silhouette (3 seeds) | 0.0182 | > 0.20 → **False** |
|
|
||||||
| Beats PCA baseline (0.1359) | 0.0182 | > PCA → **False** |
|
|
||||||
| Seed stability (spread) | 86.73% | < 10% → **False** |
|
|
||||||
| PC1/HV correlation | 0.5258 | < 0.95 → **True** |
|
|
||||||
|
|
||||||
Per-seed: ['0.0268', '0.0110', '0.0168']
|
|
||||||
|
|
||||||
## Verdict: **NULL**
|
|
||||||
|
|
||||||
One or more gate criteria not met. See null result protocol in #5.
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
"""autoresearch start — scaffold a run dir from an Autoresearch Council backlog leaf.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python scripts/autoresearch_start.py <backlog.json> <rq-id>
|
|
||||||
|
|
||||||
Reads the Council backlog JSON (from agentsquad autoresearch_pipe.py Stage-3 output),
|
|
||||||
finds the node by rq-id, validates it is autoresearch-ready (fail-closed), then
|
|
||||||
scaffolds runs/<rq-id>/ with:
|
|
||||||
|
|
||||||
program.md — hypothesis, single metric (stripped), agent search-space seam
|
|
||||||
run.json — provenance (strategic_question + council_node) + config
|
|
||||||
train.py — copy of project train.py (the loop edits this, keeps history clean)
|
|
||||||
|
|
||||||
Launch:
|
|
||||||
LITELLM_KEY=xxx python loop.py --run-dir runs/<rq-id>
|
|
||||||
|
|
||||||
Refs: jepa-fx-risk#11, agentsquad#44
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import shutil
|
|
||||||
import sys
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def load_backlog(path: str) -> dict:
|
|
||||||
try:
|
|
||||||
with open(path) as f:
|
|
||||||
return json.load(f)
|
|
||||||
except FileNotFoundError:
|
|
||||||
print(f"error: backlog file not found: {path}", file=sys.stderr)
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def scaffold_run(
|
|
||||||
backlog_path_or_dict,
|
|
||||||
rq_id: str,
|
|
||||||
run_dir: Path,
|
|
||||||
train_py_src: Path,
|
|
||||||
) -> None:
|
|
||||||
"""Scaffold a run dir. Raises SystemExit on any validation failure."""
|
|
||||||
if isinstance(backlog_path_or_dict, (str, Path)):
|
|
||||||
backlog = load_backlog(str(backlog_path_or_dict))
|
|
||||||
else:
|
|
||||||
backlog = backlog_path_or_dict
|
|
||||||
|
|
||||||
# Find node
|
|
||||||
nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
|
|
||||||
if rq_id not in nodes_by_id:
|
|
||||||
print(f"error: rq-id {rq_id!r} not found in backlog", file=sys.stderr)
|
|
||||||
sys.exit(1)
|
|
||||||
node = nodes_by_id[rq_id]
|
|
||||||
|
|
||||||
# Fail-closed: only autoresearch-ready nodes may be scaffolded
|
|
||||||
status = node.get("status", "")
|
|
||||||
if status != "autoresearch-ready":
|
|
||||||
print(
|
|
||||||
f"error: {rq_id} has status {status!r}, not 'autoresearch-ready' — refusing to scaffold",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# Guard against overwriting an existing run
|
|
||||||
if run_dir.exists():
|
|
||||||
print(
|
|
||||||
f"error: {run_dir} already exists — remove it first to re-scaffold",
|
|
||||||
file=sys.stderr,
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
metric = (node.get("candidate_metric") or "").strip()
|
|
||||||
strategic_q = backlog.get("strategic_question", "")
|
|
||||||
council_node = node["id"]
|
|
||||||
generated_at = datetime.now(timezone.utc).isoformat()
|
|
||||||
|
|
||||||
run_dir.mkdir(parents=True)
|
|
||||||
|
|
||||||
# --- program.md ---
|
|
||||||
program_md = f"""# program.md — {council_node}: {node.get("question", "")[:80]}
|
|
||||||
|
|
||||||
## Provenance
|
|
||||||
- strategic_question: {json.dumps(strategic_q)}
|
|
||||||
- council_node: {council_node} (autoresearch-ready; Autoresearch Council backlog)
|
|
||||||
- generated_at: {generated_at}
|
|
||||||
|
|
||||||
## Hypothesis
|
|
||||||
{node.get("question", "")}
|
|
||||||
|
|
||||||
## Single validation metric (optimise this, nothing else)
|
|
||||||
`{metric}` — see eval harness for the exact definition. Only this scalar drives
|
|
||||||
keep/revert decisions. Report alongside but do NOT optimise:
|
|
||||||
- Kupiec POF p-value (calibration sanity)
|
|
||||||
- val_vol_r2 (representation quality guard)
|
|
||||||
|
|
||||||
## What the agent MAY modify (the search space)
|
|
||||||
- Hyperparameters in train.py (model size, LR, window, patch_len, epochs, etc.)
|
|
||||||
- Conditioning mechanisms (e.g. JEPA_ENABLE_REGIME toggle)
|
|
||||||
- Loss function weights and architecture depth
|
|
||||||
|
|
||||||
## Frozen (do NOT touch — keeps the ablation clean)
|
|
||||||
- Data pipeline and splits (train ≤2021, OOS ≥2022, test 2024 held out)
|
|
||||||
- The metric definition and scoring code
|
|
||||||
- loop.py, scripts/, tests/
|
|
||||||
|
|
||||||
## Experiment loop (per Karpathy autoresearch)
|
|
||||||
Each iter (≤ time-box): apply ONE change to train.py → run → read
|
|
||||||
`{metric}` → keep if improved (and Kupiec p-value did not collapse), else revert.
|
|
||||||
Stop on: target reached, max iters, or K consecutive iters with no improvement.
|
|
||||||
"""
|
|
||||||
(run_dir / "program.md").write_text(program_md)
|
|
||||||
|
|
||||||
# --- run.json (provenance + config) ---
|
|
||||||
run_meta = {
|
|
||||||
"strategic_question": strategic_q,
|
|
||||||
"council_node": council_node,
|
|
||||||
"metric": metric,
|
|
||||||
"generated_at": generated_at,
|
|
||||||
"model_tier": "homelab",
|
|
||||||
"max_iters": 10,
|
|
||||||
"time_box_minutes": 5,
|
|
||||||
}
|
|
||||||
(run_dir / "run.json").write_text(json.dumps(run_meta, indent=2) + "\n")
|
|
||||||
|
|
||||||
# --- train.py (loop edits this copy; project root train.py is the template) ---
|
|
||||||
shutil.copy(train_py_src, run_dir / "train.py")
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
|
||||||
if len(sys.argv) != 3:
|
|
||||||
print("usage: python scripts/autoresearch_start.py <backlog.json> <rq-id>")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
backlog_path, rq_id = sys.argv[1], sys.argv[2]
|
|
||||||
|
|
||||||
project_root = Path(__file__).parent.parent
|
|
||||||
run_dir = project_root / "runs" / rq_id
|
|
||||||
train_py_src = project_root / "train.py"
|
|
||||||
|
|
||||||
scaffold_run(backlog_path, rq_id, run_dir, train_py_src)
|
|
||||||
|
|
||||||
backlog = load_backlog(backlog_path)
|
|
||||||
nodes_by_id = {n["id"]: n for n in backlog.get("nodes", [])}
|
|
||||||
metric = (nodes_by_id[rq_id].get("candidate_metric") or "").strip()
|
|
||||||
|
|
||||||
print(f"✓ scaffolded {run_dir}")
|
|
||||||
print(f" node: {rq_id}")
|
|
||||||
print(f" metric: {metric}")
|
|
||||||
print()
|
|
||||||
print("launch:")
|
|
||||||
print(f" LITELLM_KEY=xxx python loop.py --run-dir runs/{rq_id}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
"""Phase-0 compute gate (brain wiki/jepa-fx/facts/autoresearch-integration-phase1):
|
|
||||||
PyTorch cu130 must see the koala Blackwell GPU and compute before any experiment.
|
|
||||||
|
|
||||||
python scripts/check_gpu.py # exits 0 if the GPU is usable, 1 otherwise
|
|
||||||
|
|
||||||
Note: koala shares this 12GB card with the llama-swap LLM stack. The autoresearch
|
|
||||||
agent should run on iguana/berget models so koala's GPU stays free for train.py.
|
|
||||||
"""
|
|
||||||
import sys
|
|
||||||
import torch
|
|
||||||
|
|
||||||
print("torch", torch.__version__)
|
|
||||||
if not torch.cuda.is_available():
|
|
||||||
print("CUDA NOT AVAILABLE — gate BLOCKED")
|
|
||||||
sys.exit(1)
|
|
||||||
print("device:", torch.cuda.get_device_name(0))
|
|
||||||
print("capability: sm_%d%d" % torch.cuda.get_device_capability(0))
|
|
||||||
x = torch.randn(2000, 2000, device="cuda")
|
|
||||||
(x @ x).sum().item()
|
|
||||||
torch.cuda.synchronize()
|
|
||||||
print("GPU matmul OK — Phase-0 compute gate GREEN")
|
|
||||||
+29
-6
@@ -5,11 +5,6 @@
|
|||||||
# For Claude Code: generates project-only CLAUDE.md (it inherits root via tree walk)
|
# For Claude Code: generates project-only CLAUDE.md (it inherits root via tree walk)
|
||||||
# For everything else: concatenates root AGENT.md + project PROJECT.md
|
# For everything else: concatenates root AGENT.md + project PROJECT.md
|
||||||
#
|
#
|
||||||
# Cursor and Aider adapters removed 2026-09-01 — neither is a supported harness
|
|
||||||
# any more. Supported: Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush,
|
|
||||||
# Pi. All read CLAUDE.md or AGENTS.md; system-prompt.txt remains for Open WebUI,
|
|
||||||
# Mods and raw API use.
|
|
||||||
#
|
|
||||||
# Usage: ./scripts/context-sync.sh [--force] [adapter...]
|
# Usage: ./scripts/context-sync.sh [--force] [adapter...]
|
||||||
# Task: task context:sync
|
# Task: task context:sync
|
||||||
#
|
#
|
||||||
@@ -58,7 +53,7 @@ if [ "$FORCE" = false ]; then
|
|||||||
while IFS= read -r _match; do
|
while IFS= read -r _match; do
|
||||||
_lineno="${_match%%:*}"
|
_lineno="${_match%%:*}"
|
||||||
_content="${_match#*:}"
|
_content="${_match#*:}"
|
||||||
_token=$(head -1 < <(grep -o '{{[^}]*}}' <<<"$_content"))
|
_token=$(printf '%s' "$_content" | grep -o '{{[^}]*}}' | head -1)
|
||||||
echo " $PROJECT_FILE:$_lineno: unfilled placeholder $_token" >&2
|
echo " $PROJECT_FILE:$_lineno: unfilled placeholder $_token" >&2
|
||||||
done <<< "$_placeholders"
|
done <<< "$_placeholders"
|
||||||
echo "" >&2
|
echo "" >&2
|
||||||
@@ -100,6 +95,30 @@ generate_agents() {
|
|||||||
echo " → AGENTS.md (root + project; Crush, Pi, Antigravity)"
|
echo " → AGENTS.md (root + project; Crush, Pi, Antigravity)"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# ── Cursor ───────────────────────────────────────────────────
|
||||||
|
generate_cursor() {
|
||||||
|
{
|
||||||
|
echo "# Cursor rules — auto-generated"
|
||||||
|
echo "# Do not edit. Run: task context:sync"
|
||||||
|
echo ""
|
||||||
|
root_block
|
||||||
|
cat "$PROJECT_FILE"
|
||||||
|
} > .cursorrules
|
||||||
|
echo " → .cursorrules (root + project)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# ── Aider ────────────────────────────────────────────────────
|
||||||
|
generate_aider() {
|
||||||
|
{ root_block; cat "$PROJECT_FILE"; } > .aider.conventions.md
|
||||||
|
if [ ! -f .aider.conf.yml ]; then
|
||||||
|
cat > .aider.conf.yml << 'YAML'
|
||||||
|
read: .aider.conventions.md
|
||||||
|
auto-commits: false
|
||||||
|
YAML
|
||||||
|
fi
|
||||||
|
echo " → .aider.conventions.md (root + project)"
|
||||||
|
}
|
||||||
|
|
||||||
# ── Generic system prompt (Open WebUI, Mods, etc.) ──────────
|
# ── Generic system prompt (Open WebUI, Mods, etc.) ──────────
|
||||||
generate_system_prompt() {
|
generate_system_prompt() {
|
||||||
{
|
{
|
||||||
@@ -161,6 +180,8 @@ echo "Syncing project context from $PROJECT_FILE..."
|
|||||||
if [ ${#ADAPTERS[@]} -eq 0 ]; then
|
if [ ${#ADAPTERS[@]} -eq 0 ]; then
|
||||||
generate_claude
|
generate_claude
|
||||||
generate_agents
|
generate_agents
|
||||||
|
generate_cursor
|
||||||
|
generate_aider
|
||||||
generate_system_prompt
|
generate_system_prompt
|
||||||
generate_mcp
|
generate_mcp
|
||||||
else
|
else
|
||||||
@@ -168,6 +189,8 @@ else
|
|||||||
case "$adapter" in
|
case "$adapter" in
|
||||||
claude) generate_claude ;;
|
claude) generate_claude ;;
|
||||||
agents) generate_agents ;;
|
agents) generate_agents ;;
|
||||||
|
cursor) generate_cursor ;;
|
||||||
|
aider) generate_aider ;;
|
||||||
prompt|system|openwebui|owui|generic) generate_system_prompt ;;
|
prompt|system|openwebui|owui|generic) generate_system_prompt ;;
|
||||||
mcp) generate_mcp ;;
|
mcp) generate_mcp ;;
|
||||||
*) echo "Unknown adapter: $adapter" >&2; exit 1 ;;
|
*) echo "Unknown adapter: $adapter" >&2; exit 1 ;;
|
||||||
|
|||||||
@@ -1,31 +0,0 @@
|
|||||||
"""Fetch EUR/USD M1 bars from histdata.com (free, research use).
|
|
||||||
|
|
||||||
Polite: one request per year, spaced; past years query month=None. Uses the
|
|
||||||
maintained `histdata` package which handles histdata's anti-hotlink tk token.
|
|
||||||
Output: data/raw/DAT_ASCII_EURUSD_M1_<year>.zip
|
|
||||||
|
|
||||||
YEARS=2019,2020,2021 python scripts/fetch_data.py
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import time
|
|
||||||
|
|
||||||
from histdata import download_hist_data
|
|
||||||
from histdata.api import Platform as P, TimeFrame as T
|
|
||||||
|
|
||||||
YEARS = [y.strip() for y in os.environ.get("YEARS", "2019,2020,2021").split(",")]
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
os.makedirs("data/raw", exist_ok=True)
|
|
||||||
for yr in YEARS:
|
|
||||||
f = download_hist_data(
|
|
||||||
year=yr, month=None, pair="eurusd",
|
|
||||||
platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
|
|
||||||
output_directory="data/raw",
|
|
||||||
)
|
|
||||||
print("fetched", yr, "->", f)
|
|
||||||
time.sleep(2) # be a good citizen
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
"""Fetch G10 FX M1 data from histdata.com for all pairs except EURUSD (already fetched).
|
|
||||||
|
|
||||||
Each pair's zips go into data/raw/{pair}/ to avoid collisions.
|
|
||||||
Output: data/raw/gbpusd/DAT_ASCII_GBPUSD_M1_YYYY.zip etc.
|
|
||||||
|
|
||||||
python scripts/fetch_multipair.py
|
|
||||||
PAIRS=gbpusd,usdjpy YEARS=2020,2021 python scripts/fetch_multipair.py
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import time
|
|
||||||
|
|
||||||
from histdata import download_hist_data
|
|
||||||
from histdata.api import Platform as P, TimeFrame as T
|
|
||||||
|
|
||||||
PAIRS_DEFAULT = ["gbpusd", "usdjpy", "usdchf", "audusd"]
|
|
||||||
YEARS_DEFAULT = list(range(2008, 2024))
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
pairs_env = os.environ.get("PAIRS", "")
|
|
||||||
pairs = [p.strip() for p in pairs_env.split(",")] if pairs_env else PAIRS_DEFAULT
|
|
||||||
|
|
||||||
years_env = os.environ.get("YEARS", "")
|
|
||||||
years = [int(y.strip()) for y in years_env.split(",")] if years_env else YEARS_DEFAULT
|
|
||||||
|
|
||||||
for pair in pairs:
|
|
||||||
out_dir = f"data/raw/{pair}"
|
|
||||||
os.makedirs(out_dir, exist_ok=True)
|
|
||||||
print(f"\n=== {pair.upper()} ===")
|
|
||||||
for yr in years:
|
|
||||||
out_path = os.path.join(out_dir, f"DAT_ASCII_{pair.upper()}_M1_{yr}.zip")
|
|
||||||
if os.path.exists(out_path):
|
|
||||||
print(f" {yr} already present, skip")
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
f = download_hist_data(
|
|
||||||
year=str(yr), month=None, pair=pair,
|
|
||||||
platform=P.GENERIC_ASCII, time_frame=T.ONE_MINUTE,
|
|
||||||
output_directory=out_dir,
|
|
||||||
)
|
|
||||||
print(f" fetched {yr} → {f}")
|
|
||||||
except Exception as e:
|
|
||||||
print(f" {yr} FAILED: {e}")
|
|
||||||
time.sleep(2)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
"""HPO sweep for jepa-fx-risk HEPA backbone.
|
|
||||||
|
|
||||||
Runs train.py with different JEPA_* env overrides, logs results to
|
|
||||||
results/hpo/hpo_results.jsonl. Each config writes its metrics.json then
|
|
||||||
the result is appended to the JSONL.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python scripts/hpo_sweep.py
|
|
||||||
python scripts/hpo_sweep.py --dry-run # print configs, don't train
|
|
||||||
"""
|
|
||||||
import argparse
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from datetime import datetime
|
|
||||||
from itertools import product
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# ── Search space ──────────────────────────────────────────────────────────────
|
|
||||||
SEARCH_SPACE = {
|
|
||||||
"JEPA_D_MODEL": [64, 128, 256],
|
|
||||||
"JEPA_DEPTH": [2, 4],
|
|
||||||
"JEPA_WINDOW": [120, 240, 480],
|
|
||||||
}
|
|
||||||
# Fixed: PATCH_LEN=24 (1-day patches), N_HEADS=4, EPOCHS=300, PHASE1_EPOCHS=200
|
|
||||||
|
|
||||||
PYTHON = str(Path(sys.executable))
|
|
||||||
OUT_DIR = Path("results/hpo")
|
|
||||||
|
|
||||||
|
|
||||||
def configs():
|
|
||||||
"""Yield all configs as dicts of JEPA_* env overrides."""
|
|
||||||
keys = list(SEARCH_SPACE.keys())
|
|
||||||
for vals in product(*SEARCH_SPACE.values()):
|
|
||||||
yield dict(zip(keys, vals))
|
|
||||||
|
|
||||||
|
|
||||||
def run_config(cfg: dict, metrics_path: str = "metrics.json") -> dict:
|
|
||||||
env = {**os.environ, **{k: str(v) for k, v in cfg.items()}}
|
|
||||||
result = subprocess.run(
|
|
||||||
[PYTHON, "train.py"],
|
|
||||||
env=env,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
)
|
|
||||||
if result.returncode != 0:
|
|
||||||
return {"config": cfg, "error": result.stderr[-500:]}
|
|
||||||
stdout_last = result.stdout.strip().split("\n")[-1]
|
|
||||||
with open(metrics_path) as f:
|
|
||||||
m = json.load(f)
|
|
||||||
return {
|
|
||||||
"config": cfg,
|
|
||||||
"val_vol_r2": m.get("val_vol_r2"),
|
|
||||||
"phase1_r2": m.get("phase1_r2"),
|
|
||||||
"stdout_last": stdout_last,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--dry-run", action="store_true")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
OUT_DIR.mkdir(parents=True, exist_ok=True)
|
|
||||||
out_file = OUT_DIR / "hpo_results.jsonl"
|
|
||||||
|
|
||||||
all_cfgs = list(configs())
|
|
||||||
print(f"HPO sweep: {len(all_cfgs)} configs")
|
|
||||||
for i, cfg in enumerate(all_cfgs):
|
|
||||||
label = " ".join(f"{k.replace('JEPA_','')}={v}" for k, v in cfg.items())
|
|
||||||
print(f"\n[{i+1}/{len(all_cfgs)}] {label}")
|
|
||||||
if args.dry_run:
|
|
||||||
continue
|
|
||||||
ts = datetime.utcnow().isoformat()
|
|
||||||
row = run_config(cfg)
|
|
||||||
row["ts"] = ts
|
|
||||||
with open(out_file, "a") as f:
|
|
||||||
f.write(json.dumps(row) + "\n")
|
|
||||||
if "error" in row:
|
|
||||||
print(f" ERROR: {row['error'][:200]}")
|
|
||||||
else:
|
|
||||||
print(f" val_vol_r2={row['val_vol_r2']:.4f} phase1_r2={row['phase1_r2']:.4f}")
|
|
||||||
|
|
||||||
if not args.dry_run:
|
|
||||||
# Print leaderboard
|
|
||||||
rows = [json.loads(l) for l in open(out_file) if l.strip()]
|
|
||||||
rows = [r for r in rows if "error" not in r]
|
|
||||||
rows.sort(key=lambda r: r.get("phase1_r2", -999), reverse=True)
|
|
||||||
print("\n── Leaderboard (by phase1_r2) ─────────────────────────")
|
|
||||||
for r in rows[:5]:
|
|
||||||
cfg_str = " ".join(f"{k.replace('JEPA_','')}={v}" for k,v in r["config"].items())
|
|
||||||
print(f" {r['phase1_r2']:.4f} {cfg_str}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,237 +0,0 @@
|
|||||||
"""Phase-0 SSL feasibility gate (path B — Python fast-close of #5).
|
|
||||||
|
|
||||||
Spec deviation documented: original spec (#5) required hourly 2008-2022 data
|
|
||||||
and a Go eval harness (#4). Path B uses daily 2019-2023 + Python harness to
|
|
||||||
close the gate quickly, since val_vol_r2 > 0 already demonstrates SSL
|
|
||||||
feasibility. The Go harness (#4) remains open for production experiments.
|
|
||||||
|
|
||||||
Gate criteria (from #5):
|
|
||||||
- Silhouette > 0.20 on held-out 2022-2023 (binary HV label: top-33% RV days)
|
|
||||||
- TS-JEPA silhouette > PCA baseline silhouette
|
|
||||||
- Rerun x3 seeds within ±10% of mean silhouette
|
|
||||||
- PC1/HV correlation < 0.95 (sanity: not trivially memorising the label)
|
|
||||||
|
|
||||||
python scripts/phase0_gate.py
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import math
|
|
||||||
import os
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
import torch
|
|
||||||
import torch.nn as nn
|
|
||||||
from sklearn.decomposition import PCA
|
|
||||||
from sklearn.metrics import silhouette_score
|
|
||||||
from sklearn.preprocessing import StandardScaler
|
|
||||||
|
|
||||||
SEEDS = [0, 1, 2]
|
|
||||||
WINDOW = 30
|
|
||||||
PATCH_LEN = 5
|
|
||||||
STRIDE = 5
|
|
||||||
D_MODEL = 32
|
|
||||||
DEPTH = 2
|
|
||||||
N_HEADS = 4
|
|
||||||
EPOCHS = 400
|
|
||||||
LR = 3e-4
|
|
||||||
SIGREG_LAM = 0.5
|
|
||||||
HV_PERCENTILE = 67 # top-33% = "high volatility"
|
|
||||||
|
|
||||||
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
|
||||||
|
|
||||||
|
|
||||||
# ── SIGReg ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def sigreg(tokens: torch.Tensor, knots: int = 17) -> torch.Tensor:
|
|
||||||
B, T, D = tokens.shape
|
|
||||||
z = tokens.reshape(B * T, D).float()
|
|
||||||
t = torch.linspace(0, 3, knots, device=z.device, dtype=z.dtype)
|
|
||||||
dt = 3.0 / (knots - 1)
|
|
||||||
w = torch.full((knots,), 2 * dt, device=z.device, dtype=z.dtype)
|
|
||||||
w[0] = dt; w[-1] = dt
|
|
||||||
phi = torch.exp(-t.square() / 2.0)
|
|
||||||
A = torch.randn(D, 256, device=z.device, dtype=z.dtype)
|
|
||||||
A = A / A.norm(p=2, dim=0)
|
|
||||||
x_t = (z @ A).unsqueeze(-1) * t
|
|
||||||
err = (x_t.cos().mean(0) - phi).square() + x_t.sin().mean(0).square()
|
|
||||||
return ((err @ (w * phi)) * z.shape[0]).mean()
|
|
||||||
|
|
||||||
|
|
||||||
# ── Encoder ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
class PatchEncoder(nn.Module):
|
|
||||||
def __init__(self, in_feats, patch_len, stride, d_model, depth, n_heads):
|
|
||||||
super().__init__()
|
|
||||||
self.patch_len = patch_len
|
|
||||||
self.stride = stride
|
|
||||||
self.embed = nn.Linear(patch_len * in_feats, d_model)
|
|
||||||
layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
|
|
||||||
dropout=0.0, batch_first=True)
|
|
||||||
self.tf = nn.TransformerEncoder(layer, num_layers=depth)
|
|
||||||
n_patches = (WINDOW - patch_len) // stride + 1
|
|
||||||
pos = torch.zeros(n_patches, d_model)
|
|
||||||
for p in range(n_patches):
|
|
||||||
for i in range(0, d_model, 2):
|
|
||||||
pos[p, i] = math.sin(p / 10000 ** (i / d_model))
|
|
||||||
if i + 1 < d_model:
|
|
||||||
pos[p, i+1] = math.cos(p / 10000 ** (i / d_model))
|
|
||||||
self.register_buffer("pos", pos)
|
|
||||||
|
|
||||||
def forward(self, x):
|
|
||||||
B, W, F = x.shape
|
|
||||||
n_patches = (W - self.patch_len) // self.stride + 1
|
|
||||||
patches = torch.stack([x[:, i*self.stride:i*self.stride+self.patch_len, :]
|
|
||||||
.reshape(B, -1) for i in range(n_patches)], dim=1)
|
|
||||||
tokens = self.embed(patches) + self.pos[:n_patches]
|
|
||||||
return self.tf(tokens)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Data ─────────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def load_data():
|
|
||||||
df = pd.read_parquet("data/processed/eurusd_daily.parquet").reset_index(drop=True)
|
|
||||||
df["date"] = pd.to_datetime(df["date"])
|
|
||||||
train = df[df["date"].dt.year <= 2021].copy()
|
|
||||||
oos = df[df["date"].dt.year >= 2022].copy()
|
|
||||||
|
|
||||||
feats_all = df[["ret", "realized_vol"]].to_numpy(np.float32)
|
|
||||||
target_all = df["realized_vol"].to_numpy(np.float32)
|
|
||||||
dates_all = df["date"].values
|
|
||||||
|
|
||||||
mu = feats_all[:len(train)].mean(0)
|
|
||||||
sd = feats_all[:len(train)].std(0) + 1e-8
|
|
||||||
|
|
||||||
def windows(df_subset, feats_norm, dates):
|
|
||||||
idx_start = df.index[df["date"].isin(df_subset["date"])][0]
|
|
||||||
X, oos_dates, oos_rv = [], [], []
|
|
||||||
for t in range(idx_start + WINDOW, idx_start + len(df_subset)):
|
|
||||||
X.append(feats_norm[t - WINDOW:t])
|
|
||||||
oos_dates.append(dates[t])
|
|
||||||
oos_rv.append(target_all[t])
|
|
||||||
return np.stack(X), np.array(oos_rv), np.array(oos_dates)
|
|
||||||
|
|
||||||
feats_norm = (feats_all - mu) / sd
|
|
||||||
|
|
||||||
Xtr, rvtr, _ = windows(train, feats_norm, dates_all)
|
|
||||||
Xte, rvte, te_dates = windows(oos, feats_norm, dates_all)
|
|
||||||
|
|
||||||
# binary HV label: top-33% realized vol days in OOS = "high volatility"
|
|
||||||
hv_threshold = np.percentile(rvte, HV_PERCENTILE)
|
|
||||||
hv_labels = (rvte >= hv_threshold).astype(int)
|
|
||||||
|
|
||||||
return Xtr, rvtr, Xte, rvte, hv_labels
|
|
||||||
|
|
||||||
|
|
||||||
# ── Train + embed ─────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def train_and_embed(Xtr, Xte, seed):
|
|
||||||
torch.manual_seed(seed)
|
|
||||||
np.random.seed(seed)
|
|
||||||
enc = PatchEncoder(Xtr.shape[2], PATCH_LEN, STRIDE, D_MODEL, DEPTH, N_HEADS).to(dev)
|
|
||||||
pred = nn.Sequential(nn.Linear(D_MODEL, D_MODEL), nn.GELU(),
|
|
||||||
nn.Linear(D_MODEL, D_MODEL)).to(dev)
|
|
||||||
opt = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
|
|
||||||
|
|
||||||
Xtr_t = torch.tensor(Xtr, device=dev)
|
|
||||||
n_patches = (WINDOW - PATCH_LEN) // STRIDE + 1
|
|
||||||
n_mask = max(1, int(0.30 * n_patches))
|
|
||||||
|
|
||||||
for ep in range(EPOCHS):
|
|
||||||
idx_mask = torch.randperm(n_patches)[:n_mask]
|
|
||||||
tokens_ctx = enc(Xtr_t)
|
|
||||||
tokens_target = enc(Xtr_t).detach()
|
|
||||||
jepa_loss = ((pred(tokens_ctx[:, idx_mask, :]) -
|
|
||||||
tokens_target[:, idx_mask, :]) ** 2).mean()
|
|
||||||
reg = sigreg(tokens_ctx)
|
|
||||||
loss = jepa_loss + SIGREG_LAM * reg
|
|
||||||
opt.zero_grad(); loss.backward(); opt.step()
|
|
||||||
|
|
||||||
enc.eval()
|
|
||||||
with torch.no_grad():
|
|
||||||
Ete = enc(torch.tensor(Xte, device=dev)).mean(1).cpu().numpy()
|
|
||||||
|
|
||||||
return Ete
|
|
||||||
|
|
||||||
|
|
||||||
# ── Gate ─────────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def pca_baseline(Xte, hv_labels):
|
|
||||||
flat = Xte.reshape(len(Xte), -1)
|
|
||||||
sc = StandardScaler().fit(flat)
|
|
||||||
emb = PCA(n_components=8).fit_transform(sc.transform(flat))
|
|
||||||
return silhouette_score(emb, hv_labels), emb
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
os.makedirs("results/summaries", exist_ok=True)
|
|
||||||
Xtr, rvtr, Xte, rvte, hv_labels = load_data()
|
|
||||||
print(f"train={len(Xtr)} OOS={len(Xte)} HV={hv_labels.sum()}/{len(hv_labels)}")
|
|
||||||
|
|
||||||
pca_sil, pca_emb = pca_baseline(Xte, hv_labels)
|
|
||||||
pc1 = pca_emb[:, 0]
|
|
||||||
pc1_hv_corr = abs(np.corrcoef(pc1, hv_labels)[0, 1])
|
|
||||||
print(f"PCA baseline silhouette = {pca_sil:.4f} | PC1/HV |r| = {pc1_hv_corr:.4f}")
|
|
||||||
|
|
||||||
sils = []
|
|
||||||
for seed in SEEDS:
|
|
||||||
emb = train_and_embed(Xtr, Xte, seed)
|
|
||||||
sc = StandardScaler().fit(emb)
|
|
||||||
sil = silhouette_score(sc.transform(emb), hv_labels)
|
|
||||||
sils.append(sil)
|
|
||||||
print(f" seed={seed} silhouette={sil:.4f}")
|
|
||||||
|
|
||||||
mean_sil = np.mean(sils)
|
|
||||||
spread = (max(sils) - min(sils)) / mean_sil if mean_sil != 0 else 99
|
|
||||||
|
|
||||||
# gate checks
|
|
||||||
g_sil = mean_sil > 0.20
|
|
||||||
g_beats = mean_sil > pca_sil
|
|
||||||
g_stable = spread < 0.10
|
|
||||||
g_corr = pc1_hv_corr < 0.95
|
|
||||||
passed = all([g_sil, g_beats, g_stable, g_corr])
|
|
||||||
|
|
||||||
label = "pass" if passed else "null"
|
|
||||||
print(f"\nsilhouette mean={mean_sil:.4f} spread={spread:.2%} PCA={pca_sil:.4f} PC1/HV={pc1_hv_corr:.4f}")
|
|
||||||
print(f"gate: sil>0.20={g_sil} beats_pca={g_beats} stable={g_stable} corr<0.95={g_corr}")
|
|
||||||
print(f"PHASE-0: {label.upper()}")
|
|
||||||
|
|
||||||
summary = f"""# Phase-0 SSL feasibility gate — {label}
|
|
||||||
|
|
||||||
**Date:** 2026-06-24
|
|
||||||
**Path B deviation:** Daily 2019-2023 (not hourly 2008-2022); Python harness
|
|
||||||
(not Go #4); gate metric adapted from silhouette-on-embedding to match
|
|
||||||
available data. Go harness (#4) remains open for production experiments.
|
|
||||||
|
|
||||||
## Data
|
|
||||||
- Train: EUR/USD daily 2019-2021 ({len(Xtr)} windows)
|
|
||||||
- OOS: EUR/USD daily 2022-2023 ({len(Xte)} windows)
|
|
||||||
- HV label: top-{100-HV_PERCENTILE}% realized-vol days = high-volatility ({hv_labels.sum()} days)
|
|
||||||
|
|
||||||
## Results
|
|
||||||
| | Value | Gate |
|
|
||||||
|---|---|---|
|
|
||||||
| TS-JEPA mean silhouette (3 seeds) | {mean_sil:.4f} | > 0.20 → **{g_sil}** |
|
|
||||||
| Beats PCA baseline ({pca_sil:.4f}) | {mean_sil:.4f} | > PCA → **{g_beats}** |
|
|
||||||
| Seed stability (spread) | {spread:.2%} | < 10% → **{g_stable}** |
|
|
||||||
| PC1/HV correlation | {pc1_hv_corr:.4f} | < 0.95 → **{g_corr}** |
|
|
||||||
|
|
||||||
Per-seed: {[f"{s:.4f}" for s in sils]}
|
|
||||||
|
|
||||||
## Verdict: **{label.upper()}**
|
|
||||||
|
|
||||||
{"All 4 gate criteria met. TS-JEPA embeddings separate HV regimes significantly above PCA baseline with stable reproducibility." if passed else "One or more gate criteria not met. See null result protocol in #5."}
|
|
||||||
"""
|
|
||||||
path = f"results/summaries/phase-0-{label}.md"
|
|
||||||
with open(path, "w") as f:
|
|
||||||
f.write(summary)
|
|
||||||
print(f"Written: {path}")
|
|
||||||
|
|
||||||
result = {"label": label, "mean_sil": mean_sil, "pca_sil": pca_sil,
|
|
||||||
"spread": spread, "pc1_hv_corr": pc1_hv_corr,
|
|
||||||
"per_seed": sils, "passed": passed}
|
|
||||||
with open("results/summaries/phase-0-metrics.json", "w") as f:
|
|
||||||
json.dump(result, f, indent=2)
|
|
||||||
return 0 if passed else 1
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
"""LOCKED data pipeline (toy) — agent must NOT edit (brain Phase-1 contract).
|
|
||||||
|
|
||||||
Parses histdata EUR/USD M1 zips → daily series with realized volatility (the
|
|
||||||
val_vol_r2 target = 1-day realized vol from intraday squared returns).
|
|
||||||
Output: data/processed/eurusd_daily.parquet [date, close, ret, realized_vol].
|
|
||||||
"""
|
|
||||||
import glob
|
|
||||||
import os
|
|
||||||
import zipfile
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
RAW = "data/raw"
|
|
||||||
OUT = "data/processed/eurusd_daily.parquet"
|
|
||||||
|
|
||||||
|
|
||||||
def load_m1() -> pd.DataFrame:
|
|
||||||
frames = []
|
|
||||||
for zp in sorted(glob.glob(os.path.join(RAW, "DAT_ASCII_EURUSD_M1_*.zip"))):
|
|
||||||
with zipfile.ZipFile(zp) as z:
|
|
||||||
csv = [n for n in z.namelist() if n.endswith(".csv")][0]
|
|
||||||
with z.open(csv) as f:
|
|
||||||
df = pd.read_csv(
|
|
||||||
f, sep=";", header=None,
|
|
||||||
names=["dt", "open", "high", "low", "close", "vol"],
|
|
||||||
)
|
|
||||||
df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
|
|
||||||
frames.append(df[["ts", "close"]])
|
|
||||||
out = pd.concat(frames).sort_values("ts").reset_index(drop=True)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
m1 = load_m1()
|
|
||||||
m1["r"] = np.log(m1["close"]).diff()
|
|
||||||
m1["day"] = m1["ts"].dt.normalize()
|
|
||||||
daily = m1.groupby("day").agg(
|
|
||||||
close=("close", "last"),
|
|
||||||
realized_vol=("r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
|
|
||||||
n_min=("r", "count"),
|
|
||||||
).reset_index()
|
|
||||||
daily = daily[daily["n_min"] > 60] # drop thin days (holidays)
|
|
||||||
daily["ret"] = np.log(daily["close"]).diff()
|
|
||||||
daily = daily.dropna().reset_index(drop=True)
|
|
||||||
os.makedirs(os.path.dirname(OUT), exist_ok=True)
|
|
||||||
daily[["day", "close", "ret", "realized_vol"]].rename(columns={"day": "date"}).to_parquet(OUT)
|
|
||||||
print("rows:", len(daily), "| dates:", daily["day"].min().date(), "→", daily["day"].max().date())
|
|
||||||
# sanity: the COVID crash (March 2020) must show a realized-vol spike
|
|
||||||
rv = daily.set_index("day")["realized_vol"]
|
|
||||||
mar20 = rv["2020-03-01":"2020-03-31"].max()
|
|
||||||
typ = rv["2019-01-01":"2019-12-31"].median()
|
|
||||||
print("median 2019 RV: %.5f | max Mar-2020 RV: %.5f | spike x%.1f" % (typ, mar20, mar20 / typ))
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,128 +0,0 @@
|
|||||||
"""Prepare EUR/USD hourly OHLCV + realized vol from histdata M1 zips.
|
|
||||||
|
|
||||||
Aggregates all M1 bars in data/raw/DAT_ASCII_EURUSD_M1_*.zip to hourly.
|
|
||||||
Realized vol per hour = sqrt(sum(log-return²)) over the constituent M1 bars.
|
|
||||||
Weekend hours are naturally absent (FX market closed Sat/Sun); NO interpolation.
|
|
||||||
Hours with fewer than MIN_BARS M1 bars are dropped (holidays, thin sessions).
|
|
||||||
|
|
||||||
Output: data/processed/eurusd_hourly.parquet
|
|
||||||
Columns: datetime (UTC, tz-naive), close, ret (log), realized_vol
|
|
||||||
|
|
||||||
python scripts/prepare_hourly.py
|
|
||||||
RAW=data/raw OUT=data/processed/eurusd_hourly.parquet python scripts/prepare_hourly.py
|
|
||||||
"""
|
|
||||||
import glob
|
|
||||||
import os
|
|
||||||
import zipfile
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
PAIR = os.environ.get("PAIR", "EURUSD").upper()
|
|
||||||
RAW_DEFAULT = "data/raw"
|
|
||||||
OUT_DEFAULT = f"data/processed/{PAIR.lower()}_hourly.parquet"
|
|
||||||
MIN_BARS = 30 # drop hours thinner than this (holidays, DST boundary artefacts)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Core transformation ──────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def resample_to_hourly(m1: pd.DataFrame) -> pd.DataFrame:
|
|
||||||
"""Aggregate M1 DataFrame to hourly bars.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
m1: DataFrame with columns ['ts', 'open', 'high', 'low', 'close']
|
|
||||||
('open'/'high'/'low' optional — omit for close-only data).
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame with columns ['datetime', 'close', 'ret', 'realized_vol',
|
|
||||||
'hl_range', 'ret_intrabar'] sorted by datetime.
|
|
||||||
Hours with fewer than MIN_BARS M1 ticks are dropped.
|
|
||||||
"""
|
|
||||||
m1 = m1.sort_values("ts").copy()
|
|
||||||
m1["log_r"] = np.log(m1["close"]).diff()
|
|
||||||
m1["hour"] = m1["ts"].dt.floor("h")
|
|
||||||
|
|
||||||
has_ohlc = all(c in m1.columns for c in ("open", "high", "low"))
|
|
||||||
|
|
||||||
agg_dict = dict(
|
|
||||||
close = ("close", "last"),
|
|
||||||
realized_vol = ("log_r", lambda x: np.sqrt(np.nansum(x.values ** 2))),
|
|
||||||
n_bars = ("log_r", "count"),
|
|
||||||
)
|
|
||||||
if has_ohlc:
|
|
||||||
agg_dict["high"] = ("high", "max")
|
|
||||||
agg_dict["low"] = ("low", "min")
|
|
||||||
agg_dict["open_"] = ("open", "first")
|
|
||||||
|
|
||||||
agg = m1.groupby("hour").agg(**agg_dict).reset_index()
|
|
||||||
|
|
||||||
agg = agg[agg["n_bars"] >= MIN_BARS].copy()
|
|
||||||
agg["ret"] = np.log(agg["close"]).diff()
|
|
||||||
agg = agg.dropna(subset=["ret"]).reset_index(drop=True)
|
|
||||||
agg = agg.rename(columns={"hour": "datetime"})
|
|
||||||
|
|
||||||
if has_ohlc:
|
|
||||||
agg["hl_range"] = np.log(agg["high"] / agg["low"])
|
|
||||||
agg["ret_intrabar"]= np.log(agg["close"] / agg["open_"])
|
|
||||||
cols = ["datetime", "close", "ret", "realized_vol", "hl_range", "ret_intrabar"]
|
|
||||||
else:
|
|
||||||
cols = ["datetime", "close", "ret", "realized_vol"]
|
|
||||||
|
|
||||||
return agg[cols]
|
|
||||||
|
|
||||||
|
|
||||||
def load_m1_from_zips(raw_dir: str, pair: str = None) -> pd.DataFrame:
|
|
||||||
"""Load and concatenate all M1 zips from raw_dir (histdata format)."""
|
|
||||||
p = (pair or PAIR).upper()
|
|
||||||
pattern = os.path.join(raw_dir, f"DAT_ASCII_{p}_M1_*.zip")
|
|
||||||
zips = sorted(glob.glob(pattern))
|
|
||||||
if not zips:
|
|
||||||
raise FileNotFoundError(f"No M1 zips found at {pattern}")
|
|
||||||
frames = []
|
|
||||||
for zp in zips:
|
|
||||||
with zipfile.ZipFile(zp) as z:
|
|
||||||
csv = [n for n in z.namelist() if n.endswith(".csv")][0]
|
|
||||||
with z.open(csv) as f:
|
|
||||||
df = pd.read_csv(
|
|
||||||
f, sep=";", header=None,
|
|
||||||
names=["dt", "open", "high", "low", "close", "vol"],
|
|
||||||
)
|
|
||||||
df["ts"] = pd.to_datetime(df["dt"], format="%Y%m%d %H%M%S")
|
|
||||||
frames.append(df[["ts", "open", "high", "low", "close"]])
|
|
||||||
print(f" loaded {os.path.basename(zp)}: {len(df):,} rows")
|
|
||||||
return pd.concat(frames).sort_values("ts").reset_index(drop=True)
|
|
||||||
|
|
||||||
|
|
||||||
def build_hourly_parquet(
|
|
||||||
raw_dir: str = RAW_DEFAULT,
|
|
||||||
out_path: str = OUT_DEFAULT,
|
|
||||||
) -> pd.DataFrame:
|
|
||||||
"""Full pipeline: load all M1 zips → hourly parquet. Returns the DataFrame."""
|
|
||||||
print(f"Loading M1 zips from {raw_dir}...")
|
|
||||||
m1 = load_m1_from_zips(raw_dir)
|
|
||||||
print(f"Total M1 bars: {len(m1):,} ({m1['ts'].min().date()} → {m1['ts'].max().date()})")
|
|
||||||
|
|
||||||
print("Resampling to hourly...")
|
|
||||||
hourly = resample_to_hourly(m1)
|
|
||||||
print(f"Hourly rows: {len(hourly):,} ({hourly['datetime'].min()} → {hourly['datetime'].max()})")
|
|
||||||
|
|
||||||
# Sanity: COVID crash (Mar 2020) should show realized vol spike if data covers it
|
|
||||||
if hourly["datetime"].dt.year.isin([2020]).any():
|
|
||||||
rv = hourly.set_index("datetime")["realized_vol"]
|
|
||||||
try:
|
|
||||||
mar20 = rv["2020-03-01":"2020-03-31"].max()
|
|
||||||
typ = rv["2019-01-01":"2019-12-31"].median()
|
|
||||||
print(f"Sanity — median 2019 RV: {typ:.6f} | max Mar-2020 RV: {mar20:.6f} | spike ×{mar20/typ:.1f}")
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
|
|
||||||
os.makedirs(os.path.dirname(os.path.abspath(out_path)), exist_ok=True)
|
|
||||||
hourly.to_parquet(out_path, index=False)
|
|
||||||
print(f"Written: {out_path}")
|
|
||||||
return hourly
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raw_dir = os.environ.get("RAW", RAW_DEFAULT)
|
|
||||||
out_path = os.environ.get("OUT", OUT_DEFAULT)
|
|
||||||
build_hourly_parquet(raw_dir=raw_dir, out_path=out_path)
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
"""Merge per-pair hourly parquets into a single wide multipair parquet.
|
|
||||||
|
|
||||||
Each pair contributes two features: {pair}_ret and {pair}_rv (realized vol).
|
|
||||||
The merge is an INNER JOIN on datetime — only hours present in ALL pairs are kept.
|
|
||||||
The target for train.py remains eurusd_rv.
|
|
||||||
|
|
||||||
Output: data/processed/eurusd_multipair.parquet
|
|
||||||
|
|
||||||
python scripts/prepare_multipair.py
|
|
||||||
PROCESSED=data/processed python scripts/prepare_multipair.py
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
PAIRS = ["eurusd", "gbpusd", "usdjpy", "usdchf", "audusd"]
|
|
||||||
|
|
||||||
PROCESSED_DEFAULT = "data/processed"
|
|
||||||
OUT_DEFAULT = "data/processed/eurusd_multipair.parquet"
|
|
||||||
|
|
||||||
|
|
||||||
def merge_pair_parquets(pair_dfs: dict) -> pd.DataFrame:
|
|
||||||
"""Inner-join hourly DataFrames from multiple pairs on datetime.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
pair_dfs: dict mapping pair name (e.g. "eurusd") to hourly DataFrame
|
|
||||||
with columns [datetime, close, ret, realized_vol, ...].
|
|
||||||
Returns:
|
|
||||||
Wide DataFrame with columns:
|
|
||||||
datetime, {pair}_ret, {pair}_rv for each pair.
|
|
||||||
"""
|
|
||||||
merged = None
|
|
||||||
for pair, df in pair_dfs.items():
|
|
||||||
sub = df[["datetime", "ret", "realized_vol"]].copy()
|
|
||||||
sub = sub.rename(columns={"ret": f"{pair}_ret", "realized_vol": f"{pair}_rv"})
|
|
||||||
sub = sub.set_index("datetime")
|
|
||||||
if merged is None:
|
|
||||||
merged = sub
|
|
||||||
else:
|
|
||||||
merged = merged.join(sub, how="inner")
|
|
||||||
|
|
||||||
return merged.reset_index()
|
|
||||||
|
|
||||||
|
|
||||||
def build_multipair_parquet(
|
|
||||||
processed_dir: str = PROCESSED_DEFAULT,
|
|
||||||
out_path: str = OUT_DEFAULT,
|
|
||||||
pairs: list = None,
|
|
||||||
) -> None:
|
|
||||||
if pairs is None:
|
|
||||||
pairs = PAIRS
|
|
||||||
pair_dfs = {}
|
|
||||||
for pair in pairs:
|
|
||||||
path = os.path.join(processed_dir, f"{pair}_hourly.parquet")
|
|
||||||
if not os.path.exists(path):
|
|
||||||
raise FileNotFoundError(
|
|
||||||
f"{pair}_hourly.parquet not found at {path} — run prepare_hourly.py for this pair first"
|
|
||||||
)
|
|
||||||
df = pd.read_parquet(path)
|
|
||||||
pair_dfs[pair] = df
|
|
||||||
|
|
||||||
merged = merge_pair_parquets(pair_dfs)
|
|
||||||
merged.to_parquet(out_path, index=False)
|
|
||||||
n_pairs = len(pairs)
|
|
||||||
n_ch = n_pairs * 2
|
|
||||||
print(f"Multipair parquet: {len(merged):,} rows × {n_ch} feature channels ({n_pairs} pairs)")
|
|
||||||
print(f"Date range: {merged['datetime'].min()} → {merged['datetime'].max()}")
|
|
||||||
print(f"Written: {out_path}")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
processed_dir = os.environ.get("PROCESSED", PROCESSED_DEFAULT)
|
|
||||||
build_multipair_parquet(processed_dir=processed_dir)
|
|
||||||
@@ -1,134 +0,0 @@
|
|||||||
"""HMM regime detector — 3-state Gaussian HMM on realized_vol.
|
|
||||||
|
|
||||||
Fits on the FULL dataset (training + OOS) so the state sequence is globally
|
|
||||||
consistent across all periods. States are sorted by mean realized vol (ascending):
|
|
||||||
0 = calm, 1 = stressed, 2 = crisis
|
|
||||||
|
|
||||||
Output: data/processed/eurusd_regime.parquet
|
|
||||||
Columns: datetime (or date), regime (int: 0/1/2)
|
|
||||||
|
|
||||||
Deterministic: fixed random_state=42 throughout.
|
|
||||||
Cached: if the parquet already exists, it is not re-computed.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python scripts/prepare_regime.py [--hourly] [--daily] [--force]
|
|
||||||
|
|
||||||
jepa-fx-risk#13
|
|
||||||
"""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
from hmmlearn import hmm
|
|
||||||
|
|
||||||
DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
|
|
||||||
HOURLY_PATH = DATA_DIR / "eurusd_hourly.parquet"
|
|
||||||
DAILY_PATH = DATA_DIR / "eurusd_daily.parquet"
|
|
||||||
OUTPUT_PATH = DATA_DIR / "eurusd_regime.parquet"
|
|
||||||
|
|
||||||
N_STATES = 3
|
|
||||||
RANDOM_STATE = 42
|
|
||||||
|
|
||||||
|
|
||||||
def fit_regime_hmm(realized_vol: np.ndarray, n_states: int = 3, random_state: int = 42) -> np.ndarray:
|
|
||||||
"""Fit a Gaussian HMM on realized_vol and return state labels (0=calm → n_states-1=crisis).
|
|
||||||
|
|
||||||
States are sorted by mean realized vol ascending so label 0 is always calm,
|
|
||||||
label n_states-1 is always crisis. This makes the labelling deterministic
|
|
||||||
across datasets with different vol levels.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
realized_vol: 1-D array of realized vol values
|
|
||||||
n_states: number of HMM hidden states (default 3)
|
|
||||||
random_state: random seed for reproducibility
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Integer label array of shape (len(realized_vol),), dtype int64
|
|
||||||
"""
|
|
||||||
X = realized_vol.reshape(-1, 1).astype(np.float64)
|
|
||||||
model = hmm.GaussianHMM(
|
|
||||||
n_components=n_states,
|
|
||||||
covariance_type="diag",
|
|
||||||
min_covar=1e-6,
|
|
||||||
n_iter=100,
|
|
||||||
random_state=random_state,
|
|
||||||
tol=1e-4,
|
|
||||||
)
|
|
||||||
model.fit(X)
|
|
||||||
raw_labels = model.predict(X)
|
|
||||||
|
|
||||||
# Sort states by mean realized vol (ascending: calm=0, crisis=n_states-1)
|
|
||||||
state_means = np.array([X[raw_labels == s].mean() if (raw_labels == s).any() else 0.0
|
|
||||||
for s in range(n_states)])
|
|
||||||
rank = np.argsort(state_means) # rank[0] = original state id of the calmest cluster
|
|
||||||
remap = np.empty(n_states, dtype=np.int64)
|
|
||||||
for new_label, old_label in enumerate(rank):
|
|
||||||
remap[old_label] = new_label
|
|
||||||
return remap[raw_labels].astype(np.int64)
|
|
||||||
|
|
||||||
|
|
||||||
def prepare_regime_df(parquet_path: str, freq: str = "hourly") -> pd.DataFrame:
|
|
||||||
"""Load parquet, fit HMM, return DataFrame with timestamp + regime columns.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
parquet_path: path to input parquet (hourly or daily)
|
|
||||||
freq: "hourly" | "daily" — determines timestamp column name
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame with columns: (datetime|date), regime
|
|
||||||
"""
|
|
||||||
df = pd.read_parquet(parquet_path)
|
|
||||||
if freq == "hourly":
|
|
||||||
ts = pd.to_datetime(df["datetime"])
|
|
||||||
else:
|
|
||||||
ts = pd.to_datetime(df["date"])
|
|
||||||
|
|
||||||
rv = df["realized_vol"].to_numpy(np.float32)
|
|
||||||
labels = fit_regime_hmm(rv, n_states=N_STATES, random_state=RANDOM_STATE)
|
|
||||||
return pd.DataFrame({"datetime": ts.values, "regime": labels})
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description="Fit HMM regime detector")
|
|
||||||
parser.add_argument("--hourly", action="store_true", default=True,
|
|
||||||
help="use hourly parquet (default)")
|
|
||||||
parser.add_argument("--daily", action="store_true", default=False,
|
|
||||||
help="use daily parquet instead of hourly")
|
|
||||||
parser.add_argument("--force", action="store_true", default=False,
|
|
||||||
help="overwrite existing output")
|
|
||||||
parser.add_argument("--out", default=str(OUTPUT_PATH),
|
|
||||||
help="output parquet path")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
out_path = Path(args.out)
|
|
||||||
if out_path.exists() and not args.force:
|
|
||||||
print("regime parquet already exists:", out_path, "(use --force to recompute)")
|
|
||||||
return
|
|
||||||
|
|
||||||
if args.daily and DAILY_PATH.exists():
|
|
||||||
src, freq = str(DAILY_PATH), "daily"
|
|
||||||
elif HOURLY_PATH.exists():
|
|
||||||
src, freq = str(HOURLY_PATH), "hourly"
|
|
||||||
elif DAILY_PATH.exists():
|
|
||||||
src, freq = str(DAILY_PATH), "daily"
|
|
||||||
else:
|
|
||||||
raise FileNotFoundError("no parquet found in data/processed/")
|
|
||||||
|
|
||||||
print(f"fitting HMM ({N_STATES} states) on {src} ...")
|
|
||||||
df = prepare_regime_df(src, freq=freq)
|
|
||||||
|
|
||||||
counts = df["regime"].value_counts().sort_index()
|
|
||||||
print("regime distribution:")
|
|
||||||
for state, count in counts.items():
|
|
||||||
label = {0: "calm", 1: "stressed", 2: "crisis"}.get(state, f"state{state}")
|
|
||||||
print(f" {state} ({label}): {count} ({100*count/len(df):.1f}%)")
|
|
||||||
|
|
||||||
df.to_parquet(out_path, index=False)
|
|
||||||
print("wrote:", out_path)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
"""Parametric 99% VaR breach rate + Kupiec POF p-value.
|
|
||||||
|
|
||||||
Used by train.py's LOCKED VaR EVAL BLOCK to write VaR_breach_rate_99_oos_regime_cond
|
|
||||||
to metrics.json so the autoresearch loop can optimise it.
|
|
||||||
|
|
||||||
jepa-fx-risk#12
|
|
||||||
"""
|
|
||||||
|
|
||||||
import math
|
|
||||||
|
|
||||||
# Canonical metric key — no surrounding whitespace, as required by the loop contract.
|
|
||||||
METRIC_KEY = "VaR_breach_rate_99_oos_regime_cond"
|
|
||||||
|
|
||||||
# Default normal 99th-percentile z-score.
|
|
||||||
Z99 = 2.326
|
|
||||||
|
|
||||||
|
|
||||||
def var_breach_rate(pred_vol, actual_vol, z99=Z99):
|
|
||||||
"""Compute VaR breach rate and Kupiec POF p-value.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
pred_vol: iterable of predicted conditional vol forecasts
|
|
||||||
actual_vol: iterable of actual realized vol (same length)
|
|
||||||
z99: 99th-percentile z-score (default 2.326)
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
(breach_rate, kupiec_p) where:
|
|
||||||
breach_rate — fraction of steps where actual_vol > pred_vol × z99
|
|
||||||
kupiec_p — Kupiec POF p-value (H0: true breach rate = 1%)
|
|
||||||
High p-value = well-calibrated; low = miscalibrated tail.
|
|
||||||
"""
|
|
||||||
pred_v = list(pred_vol)
|
|
||||||
act_v = list(actual_vol)
|
|
||||||
n = len(pred_v)
|
|
||||||
if n == 0 or n != len(act_v):
|
|
||||||
return 0.0, 1.0
|
|
||||||
|
|
||||||
n1 = sum(1 for p, a in zip(pred_v, act_v) if a > p * z99)
|
|
||||||
breach_rate = n1 / n
|
|
||||||
p = kupiec_pvalue(n, n1)
|
|
||||||
return breach_rate, p
|
|
||||||
|
|
||||||
|
|
||||||
def kupiec_pvalue(n, n1, p0=0.01):
|
|
||||||
"""Kupiec Proportion-of-Failures likelihood ratio test.
|
|
||||||
|
|
||||||
H0: true breach probability = p0.
|
|
||||||
Returns P(chi²(1) > LR) using the identity P(chi²(1)>x) = erfc(sqrt(x/2)).
|
|
||||||
Returns 1.0 for n=0 or LR<=0 (well-calibrated / over-conservative).
|
|
||||||
"""
|
|
||||||
if n == 0:
|
|
||||||
return 1.0
|
|
||||||
n0 = n - n1
|
|
||||||
phat = n1 / n
|
|
||||||
|
|
||||||
if n1 == 0:
|
|
||||||
# 0 × ln(0/p0) = 0 by convention; only n0 term contributes
|
|
||||||
lr = 2 * n0 * math.log((1 - phat) / (1 - p0))
|
|
||||||
elif n1 == n:
|
|
||||||
lr = 2 * n1 * math.log(phat / p0)
|
|
||||||
else:
|
|
||||||
lr = 2 * (n1 * math.log(phat / p0) + n0 * math.log((1 - phat) / (1 - p0)))
|
|
||||||
|
|
||||||
if lr <= 0:
|
|
||||||
return 1.0
|
|
||||||
# P(chi²(1) > LR) = erfc(sqrt(LR/2))
|
|
||||||
return math.erfc(math.sqrt(lr / 2))
|
|
||||||
@@ -1,215 +0,0 @@
|
|||||||
"""Tests for scripts/autoresearch_start.py — jepa-fx-risk#11 Phase A scaffold.
|
|
||||||
|
|
||||||
Success criterion: `autoresearch start <backlog.json> <rq-id>` scaffolds a
|
|
||||||
runnable run dir from a ready leaf; refuses non-ready nodes; strips
|
|
||||||
candidate_metric; records provenance.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import importlib.util
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
# Load the module without executing main()
|
|
||||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "autoresearch_start.py"
|
|
||||||
|
|
||||||
|
|
||||||
def _import():
|
|
||||||
spec = importlib.util.spec_from_file_location("autoresearch_start", _SCRIPT)
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def mod():
|
|
||||||
return _import()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def backlog(tmp_path):
|
|
||||||
data = {
|
|
||||||
"strategic_question": "Test strategic question?",
|
|
||||||
"generated_at": "2026-06-27T00:00:00Z",
|
|
||||||
"nodes": [
|
|
||||||
{
|
|
||||||
"id": "rq-01",
|
|
||||||
"question": "Does X improve Y?",
|
|
||||||
"case_type": "autoresearch-loop",
|
|
||||||
"data": "obtainable",
|
|
||||||
"method": "adjacent",
|
|
||||||
"falsifiable": "yes",
|
|
||||||
"candidate_metric": " val_vol_r2", # leading space — bypass test
|
|
||||||
"depends_on": [],
|
|
||||||
"status": "autoresearch-ready",
|
|
||||||
"track": "autoresearch",
|
|
||||||
"converged": True,
|
|
||||||
"survived_review": True,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "rq-02",
|
|
||||||
"question": "Not ready yet?",
|
|
||||||
"case_type": "empirical-study",
|
|
||||||
"data": "obtainable",
|
|
||||||
"method": "adjacent",
|
|
||||||
"falsifiable": "yes",
|
|
||||||
"candidate_metric": None,
|
|
||||||
"depends_on": [],
|
|
||||||
"status": "needs-metric",
|
|
||||||
"track": "study",
|
|
||||||
"converged": True,
|
|
||||||
"survived_review": True,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"id": "rq-03",
|
|
||||||
"question": "A spike.",
|
|
||||||
"case_type": "spike",
|
|
||||||
"data": "have",
|
|
||||||
"method": "yes-named",
|
|
||||||
"falsifiable": "yes",
|
|
||||||
"candidate_metric": None,
|
|
||||||
"depends_on": [],
|
|
||||||
"status": "spike-ready",
|
|
||||||
"track": "spike",
|
|
||||||
"converged": True,
|
|
||||||
"survived_review": True,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
}
|
|
||||||
p = tmp_path / "backlog.json"
|
|
||||||
p.write_text(json.dumps(data))
|
|
||||||
return p
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def fake_train_py(tmp_path):
|
|
||||||
"""Minimal train.py placeholder for scaffold tests."""
|
|
||||||
src = tmp_path / "train_template.py"
|
|
||||||
src.write_text("# train.py placeholder\n")
|
|
||||||
return src
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# fail-closed: refuse non-autoresearch-ready nodes
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestRefuseNonReady:
|
|
||||||
def test_refuses_needs_metric(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
run_dir = tmp_path / "runs" / "rq-02"
|
|
||||||
with pytest.raises(SystemExit) as exc:
|
|
||||||
mod.scaffold_run(backlog, "rq-02", run_dir, fake_train_py)
|
|
||||||
assert exc.value.code != 0
|
|
||||||
|
|
||||||
def test_refuses_spike_ready(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
run_dir = tmp_path / "runs" / "rq-03"
|
|
||||||
with pytest.raises(SystemExit) as exc:
|
|
||||||
mod.scaffold_run(backlog, "rq-03", run_dir, fake_train_py)
|
|
||||||
assert exc.value.code != 0
|
|
||||||
|
|
||||||
def test_refuses_missing_rq_id(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
run_dir = tmp_path / "runs" / "rq-99"
|
|
||||||
with pytest.raises(SystemExit) as exc:
|
|
||||||
mod.scaffold_run(backlog, "rq-99", run_dir, fake_train_py)
|
|
||||||
assert exc.value.code != 0
|
|
||||||
|
|
||||||
def test_refuses_existing_run_dir(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
run_dir = tmp_path / "runs" / "rq-01"
|
|
||||||
run_dir.mkdir(parents=True)
|
|
||||||
with pytest.raises(SystemExit) as exc:
|
|
||||||
mod.scaffold_run(backlog, "rq-01", run_dir, fake_train_py)
|
|
||||||
assert exc.value.code != 0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# scaffold structure: correct files created
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestScaffoldStructure:
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
|
||||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
|
||||||
|
|
||||||
def test_run_dir_created(self):
|
|
||||||
assert self.run_dir.is_dir()
|
|
||||||
|
|
||||||
def test_program_md_created(self):
|
|
||||||
assert (self.run_dir / "program.md").exists()
|
|
||||||
|
|
||||||
def test_run_json_created(self):
|
|
||||||
assert (self.run_dir / "run.json").exists()
|
|
||||||
|
|
||||||
def test_train_py_copied(self):
|
|
||||||
assert (self.run_dir / "train.py").exists()
|
|
||||||
assert (self.run_dir / "train.py").read_text() == "# train.py placeholder\n"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# program.md content
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestProgramMd:
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
|
||||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
|
||||||
self.content = (self.run_dir / "program.md").read_text()
|
|
||||||
|
|
||||||
def test_contains_hypothesis(self):
|
|
||||||
assert "Does X improve Y?" in self.content
|
|
||||||
|
|
||||||
def test_metric_key_stripped(self):
|
|
||||||
# candidate_metric had leading space " val_vol_r2" — must be stripped
|
|
||||||
assert "`val_vol_r2`" in self.content
|
|
||||||
assert "` val_vol_r2`" not in self.content
|
|
||||||
|
|
||||||
def test_contains_strategic_question(self):
|
|
||||||
assert "Test strategic question?" in self.content
|
|
||||||
|
|
||||||
def test_contains_council_node(self):
|
|
||||||
assert "rq-01" in self.content
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# run.json provenance
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestRunJson:
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _scaffold(self, mod, backlog, fake_train_py, tmp_path):
|
|
||||||
self.run_dir = tmp_path / "runs" / "rq-01"
|
|
||||||
mod.scaffold_run(backlog, "rq-01", self.run_dir, fake_train_py)
|
|
||||||
self.run = json.loads((self.run_dir / "run.json").read_text())
|
|
||||||
|
|
||||||
def test_strategic_question_in_provenance(self):
|
|
||||||
assert self.run["strategic_question"] == "Test strategic question?"
|
|
||||||
|
|
||||||
def test_council_node_in_provenance(self):
|
|
||||||
assert self.run["council_node"] == "rq-01"
|
|
||||||
|
|
||||||
def test_metric_stripped_in_provenance(self):
|
|
||||||
assert self.run["metric"] == "val_vol_r2"
|
|
||||||
assert self.run["metric"] == self.run["metric"].strip()
|
|
||||||
|
|
||||||
def test_generated_at_present(self):
|
|
||||||
assert "generated_at" in self.run
|
|
||||||
|
|
||||||
def test_max_iters_present(self):
|
|
||||||
assert "max_iters" in self.run
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# load_backlog helper
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestLoadBacklog:
|
|
||||||
def test_loads_json(self, mod, backlog):
|
|
||||||
data = mod.load_backlog(str(backlog))
|
|
||||||
assert data["strategic_question"] == "Test strategic question?"
|
|
||||||
assert len(data["nodes"]) == 3
|
|
||||||
|
|
||||||
def test_missing_file_raises(self, mod, tmp_path):
|
|
||||||
with pytest.raises((FileNotFoundError, SystemExit)):
|
|
||||||
mod.load_backlog(str(tmp_path / "nonexistent.json"))
|
|
||||||
@@ -1,277 +0,0 @@
|
|||||||
"""Failing tests for HEPA backbone + Phase-1 supervised head + HPO in train.py.
|
|
||||||
|
|
||||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_hepa.py -v
|
|
||||||
These tests define what the backbone and head must satisfy BEFORE implementation.
|
|
||||||
"""
|
|
||||||
import math
|
|
||||||
import os
|
|
||||||
import torch
|
|
||||||
import torch.nn as nn
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
# ── Tests import the classes from train.py ────────────────────────────────────
|
|
||||||
# They will fail until train.py implements: CausalEncoder, HorizonPredictor, vicreg_loss
|
|
||||||
|
|
||||||
|
|
||||||
def _import(env_overrides=None):
|
|
||||||
import importlib.util, sys
|
|
||||||
saved = {}
|
|
||||||
if env_overrides:
|
|
||||||
for k, v in env_overrides.items():
|
|
||||||
saved[k] = os.environ.get(k)
|
|
||||||
os.environ[k] = str(v)
|
|
||||||
# Force fresh module load (env vars must be read at import time)
|
|
||||||
name = f"train_{id(env_overrides)}"
|
|
||||||
spec = importlib.util.spec_from_file_location(name, "train.py")
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
if env_overrides:
|
|
||||||
for k, orig in saved.items():
|
|
||||||
if orig is None:
|
|
||||||
os.environ.pop(k, None)
|
|
||||||
else:
|
|
||||||
os.environ[k] = orig
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="module")
|
|
||||||
def train_mod():
|
|
||||||
return _import()
|
|
||||||
|
|
||||||
|
|
||||||
# 1. CausalEncoder exists and has correct output shape
|
|
||||||
def test_causal_encoder_shape(train_mod):
|
|
||||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
|
|
||||||
x = torch.randn(4, 60, 2)
|
|
||||||
tokens = enc(x) # should return all tokens (B, N, D) for JEPA pretraining
|
|
||||||
assert tokens.shape == (4, 6, 32), f"expected (4, 6, 32), got {tokens.shape}"
|
|
||||||
|
|
||||||
|
|
||||||
# 2. CausalEncoder is actually causal: earlier token outputs don't change when later inputs change
|
|
||||||
def test_causal_masking(train_mod):
|
|
||||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=2)
|
|
||||||
enc.eval()
|
|
||||||
torch.manual_seed(0)
|
|
||||||
x = torch.randn(1, 60, 2)
|
|
||||||
x_perturbed = x.clone()
|
|
||||||
# non-uniform noise (constant shift absorbed by per-patch LayerNorm; variance change is not)
|
|
||||||
torch.manual_seed(99)
|
|
||||||
x_perturbed[:, 30:, :] += torch.randn_like(x[:, 30:, :]) * 5.0
|
|
||||||
|
|
||||||
with torch.no_grad():
|
|
||||||
h1 = enc(x)
|
|
||||||
h2 = enc(x_perturbed)
|
|
||||||
|
|
||||||
# First 3 tokens must be identical (causal — don't see future patches)
|
|
||||||
assert torch.allclose(h1[:, :3, :], h2[:, :3, :], atol=1e-5), \
|
|
||||||
"causal masking broken: early tokens change when later input changes"
|
|
||||||
# Last token should differ (it can see the perturbed patches)
|
|
||||||
assert not torch.allclose(h1[:, -1, :], h2[:, -1, :], atol=1e-5), \
|
|
||||||
"last token should differ when later input changes"
|
|
||||||
|
|
||||||
|
|
||||||
# 3. HorizonPredictor exists, takes (h, delta_t_float) → same shape as h
|
|
||||||
def test_horizon_predictor_shape(train_mod):
|
|
||||||
pred = train_mod.HorizonPredictor(d_model=32)
|
|
||||||
h = torch.randn(4, 32)
|
|
||||||
dt = torch.tensor([1.0, 2.0, 3.0, 1.0])
|
|
||||||
out = pred(h, dt)
|
|
||||||
assert out.shape == (4, 32), f"expected (4, 32), got {out.shape}"
|
|
||||||
|
|
||||||
|
|
||||||
# 4. vicreg_loss is a scalar and backward doesn't error
|
|
||||||
def test_vicreg_loss_backward(train_mod):
|
|
||||||
h_pred = torch.randn(8, 32, requires_grad=True)
|
|
||||||
h_target = torch.randn(8, 32)
|
|
||||||
loss = train_mod.vicreg_loss(h_pred, h_target, alpha=0.1)
|
|
||||||
assert loss.shape == (), f"expected scalar, got {loss.shape}"
|
|
||||||
loss.backward()
|
|
||||||
assert h_pred.grad is not None
|
|
||||||
|
|
||||||
|
|
||||||
# 5. Full JEPA step: encode context, predict future, compute loss, backward
|
|
||||||
def test_jepa_step_end_to_end(train_mod):
|
|
||||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=10, d_model=32, n_heads=4, depth=1)
|
|
||||||
pred = train_mod.HorizonPredictor(d_model=32)
|
|
||||||
opt = torch.optim.SGD(list(enc.parameters()) + list(pred.parameters()), lr=1e-3)
|
|
||||||
|
|
||||||
x = torch.randn(4, 60, 2)
|
|
||||||
tokens = enc(x) # (4, 6, 32)
|
|
||||||
c, dt = 2, 2 # context position 2, horizon 2
|
|
||||||
h_ctx = tokens[:, c, :]
|
|
||||||
h_tgt = tokens[:, c + dt, :].detach()
|
|
||||||
h_hat = pred(h_ctx, torch.full((4,), float(dt)))
|
|
||||||
loss = train_mod.vicreg_loss(h_hat, h_tgt, alpha=0.1)
|
|
||||||
opt.zero_grad(); loss.backward(); opt.step()
|
|
||||||
assert loss.item() < 100, "loss exploded"
|
|
||||||
|
|
||||||
|
|
||||||
# 6. build() returns year-based OOS split (2022-2023); hourly gives many more windows
|
|
||||||
def test_build_year_split(train_mod):
|
|
||||||
(Xtr, ytr), (Xte, yte) = train_mod.build()
|
|
||||||
assert Xtr.shape[1] == train_mod.WINDOW
|
|
||||||
assert Xte.shape[1] == train_mod.WINDOW
|
|
||||||
assert len(Xtr) > 0 and len(Xte) > 0
|
|
||||||
# OOS: daily ≈ 600; hourly ≈ 17,000 (2 years × ~8,500 trading hours/year)
|
|
||||||
assert len(Xte) > 400, f"OOS too small: {len(Xte)}"
|
|
||||||
|
|
||||||
|
|
||||||
# 7. hourly build gives > 10× more training windows than daily
|
|
||||||
def test_build_hourly_more_windows(train_mod):
|
|
||||||
import os
|
|
||||||
if not os.path.exists("data/processed/eurusd_hourly.parquet"):
|
|
||||||
pytest.skip("eurusd_hourly.parquet not present — run data:prepare:hourly first")
|
|
||||||
(Xtr, _), _ = train_mod.build()
|
|
||||||
# Daily had ~877 train windows; hourly with 2008-2021 should have > 50,000
|
|
||||||
assert len(Xtr) > 10_000, f"expected >10k hourly train windows, got {len(Xtr)}"
|
|
||||||
|
|
||||||
|
|
||||||
# ── Phase-1: supervised head ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
# 8. SupervisedHead exists and maps (B, D) → (B,)
|
|
||||||
def test_supervised_head_shape(train_mod):
|
|
||||||
D = 128
|
|
||||||
head = train_mod.SupervisedHead(D)
|
|
||||||
x = torch.randn(16, D)
|
|
||||||
out = head(x)
|
|
||||||
assert out.shape == (16,), f"expected (16,), got {out.shape}"
|
|
||||||
|
|
||||||
|
|
||||||
# 9. SupervisedHead gradient flows (not frozen)
|
|
||||||
def test_supervised_head_backward(train_mod):
|
|
||||||
head = train_mod.SupervisedHead(64)
|
|
||||||
x = torch.randn(8, 64)
|
|
||||||
loss = head(x).mean()
|
|
||||||
loss.backward()
|
|
||||||
for name, p in head.named_parameters():
|
|
||||||
assert p.grad is not None, f"no grad on {name}"
|
|
||||||
|
|
||||||
|
|
||||||
# 10. Phase-1 beats linear on nonlinear synthetic signal
|
|
||||||
def test_phase1_beats_linear_on_nonlinear(train_mod):
|
|
||||||
"""MLP head should outperform ridge regression on data with nonlinear structure."""
|
|
||||||
import numpy as np
|
|
||||||
torch.manual_seed(0); np.random.seed(0)
|
|
||||||
N, D = 1000, 32
|
|
||||||
# target = |h|² (quadratic — linear can't fit well)
|
|
||||||
Etr = np.random.randn(N, D).astype(np.float32)
|
|
||||||
ytr = (Etr ** 2).sum(axis=1)
|
|
||||||
Ete = np.random.randn(200, D).astype(np.float32)
|
|
||||||
yte = (Ete ** 2).sum(axis=1)
|
|
||||||
|
|
||||||
# Ridge baseline
|
|
||||||
A = np.hstack([Etr, np.ones((N, 1))])
|
|
||||||
w = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
|
|
||||||
pred_lin = np.hstack([Ete, np.ones((200, 1))]) @ w
|
|
||||||
r2_lin = float(1 - ((yte - pred_lin) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
|
|
||||||
|
|
||||||
# MLP head
|
|
||||||
head = train_mod.SupervisedHead(D)
|
|
||||||
opt = torch.optim.Adam(head.parameters(), lr=1e-2)
|
|
||||||
Xtr_t = torch.tensor(Etr); ytr_t = torch.tensor(ytr)
|
|
||||||
for _ in range(300):
|
|
||||||
loss = nn.functional.mse_loss(head(Xtr_t), ytr_t)
|
|
||||||
opt.zero_grad(); loss.backward(); opt.step()
|
|
||||||
|
|
||||||
head.eval()
|
|
||||||
with torch.no_grad():
|
|
||||||
pred_mlp = head(torch.tensor(Ete)).numpy()
|
|
||||||
r2_mlp = float(1 - ((yte - pred_mlp) ** 2).sum() / ((yte - yte.mean()) ** 2).sum())
|
|
||||||
|
|
||||||
assert r2_mlp > r2_lin + 0.05, (
|
|
||||||
f"MLP R²={r2_mlp:.3f} should beat ridge R²={r2_lin:.3f} by >0.05 on quadratic target"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# 11. main() returns phase1_r2 in metrics.json (integration — needs real data)
|
|
||||||
def test_metrics_json_has_phase1_r2(train_mod):
|
|
||||||
import json
|
|
||||||
if not os.path.exists("metrics.json"):
|
|
||||||
pytest.skip("metrics.json not present — run train.py first")
|
|
||||||
with open("metrics.json") as f:
|
|
||||||
m = json.load(f)
|
|
||||||
assert "phase1_r2" in m, f"phase1_r2 missing from metrics.json: {list(m.keys())}"
|
|
||||||
assert m["phase1_r2"] > m["val_vol_r2"], (
|
|
||||||
f"MLP head phase1_r2={m['phase1_r2']:.4f} should beat linear probe "
|
|
||||||
f"val_vol_r2={m['val_vol_r2']:.4f}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ── HPO: env-var knob overrides ───────────────────────────────────────────────
|
|
||||||
|
|
||||||
# 12. JEPA_WINDOW env var overrides WINDOW at import time
|
|
||||||
def test_env_override_window():
|
|
||||||
mod = _import({"JEPA_WINDOW": "48"})
|
|
||||||
assert mod.WINDOW == 48, f"expected WINDOW=48, got {mod.WINDOW}"
|
|
||||||
|
|
||||||
|
|
||||||
# 13. JEPA_D_MODEL and JEPA_DEPTH env vars work
|
|
||||||
def test_env_override_d_model_depth():
|
|
||||||
mod = _import({"JEPA_D_MODEL": "64", "JEPA_DEPTH": "4"})
|
|
||||||
assert mod.D_MODEL == 64, f"expected D_MODEL=64, got {mod.D_MODEL}"
|
|
||||||
assert mod.DEPTH == 4, f"expected DEPTH=4, got {mod.DEPTH}"
|
|
||||||
|
|
||||||
|
|
||||||
# 14. hpo_sweep.py exists and generates correct config list
|
|
||||||
def test_hpo_sweep_configs():
|
|
||||||
import importlib.util
|
|
||||||
sweep_path = "scripts/hpo_sweep.py"
|
|
||||||
if not os.path.exists(sweep_path):
|
|
||||||
pytest.fail(f"{sweep_path} not found — implement it")
|
|
||||||
spec = importlib.util.spec_from_file_location("hpo_sweep", sweep_path)
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
cfgs = list(mod.configs())
|
|
||||||
assert len(cfgs) > 0, "configs() returned empty list"
|
|
||||||
# Every config must have at least D_MODEL, DEPTH, WINDOW keys
|
|
||||||
required = {"JEPA_D_MODEL", "JEPA_DEPTH", "JEPA_WINDOW"}
|
|
||||||
for cfg in cfgs:
|
|
||||||
assert required.issubset(cfg.keys()), f"config missing required keys: {cfg}"
|
|
||||||
|
|
||||||
|
|
||||||
# ── Option B: joint encoder fine-tuning in phase-1 ───────────────────────────
|
|
||||||
|
|
||||||
# 15. PHASE1_JOINT and PHASE1_ENCODER_LR knobs exist at module level
|
|
||||||
def test_joint_phase1_knobs():
|
|
||||||
mod = _import({"JEPA_PHASE1_JOINT": "1", "JEPA_PHASE1_ENCODER_LR": "1e-5"})
|
|
||||||
assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing from train.py"
|
|
||||||
assert hasattr(mod, "PHASE1_ENCODER_LR"), "PHASE1_ENCODER_LR knob missing from train.py"
|
|
||||||
assert mod.PHASE1_JOINT is True
|
|
||||||
assert abs(mod.PHASE1_ENCODER_LR - 1e-5) < 1e-12
|
|
||||||
|
|
||||||
|
|
||||||
# 16. PHASE1_JOINT defaults to True (joint mode on by default)
|
|
||||||
def test_joint_phase1_default_on():
|
|
||||||
mod = _import()
|
|
||||||
assert hasattr(mod, "PHASE1_JOINT"), "PHASE1_JOINT knob missing"
|
|
||||||
assert mod.PHASE1_JOINT is True, f"PHASE1_JOINT default should be True, got {mod.PHASE1_JOINT}"
|
|
||||||
|
|
||||||
|
|
||||||
# 17. JEPA_PHASE1_JOINT=0 disables joint (env override works)
|
|
||||||
def test_joint_phase1_can_disable():
|
|
||||||
mod = _import({"JEPA_PHASE1_JOINT": "0"})
|
|
||||||
assert mod.PHASE1_JOINT is False, f"expected False, got {mod.PHASE1_JOINT}"
|
|
||||||
|
|
||||||
|
|
||||||
# 18. Encoder receives non-zero gradients when joint-training with the head
|
|
||||||
def test_joint_encoder_grad_flows(train_mod):
|
|
||||||
"""Gradient must flow into encoder when using two-param-group joint optimizer."""
|
|
||||||
import torch.nn.functional as F
|
|
||||||
enc = train_mod.CausalEncoder(n_channels=2, patch_len=8, d_model=16, n_heads=2, depth=1)
|
|
||||||
head = train_mod.SupervisedHead(16)
|
|
||||||
enc.train(); head.train()
|
|
||||||
opt = torch.optim.Adam([
|
|
||||||
{"params": head.parameters(), "lr": 1e-3},
|
|
||||||
{"params": enc.parameters(), "lr": 1e-5},
|
|
||||||
], weight_decay=1e-4)
|
|
||||||
# Tiny batch: 4 windows of length 16 (= 2 patches of patch_len=8)
|
|
||||||
X = torch.randn(4, 16, 2)
|
|
||||||
y = torch.randn(4)
|
|
||||||
tokens = enc(X) # (4, 2, 16)
|
|
||||||
h = tokens[:, -1, :] # (4, 16) — last token
|
|
||||||
pred = head(h)
|
|
||||||
loss = F.mse_loss(pred, y)
|
|
||||||
loss.backward()
|
|
||||||
enc_grads = [p.grad for p in enc.parameters() if p.grad is not None]
|
|
||||||
assert len(enc_grads) > 0, "no encoder params received gradients"
|
|
||||||
assert any(g.abs().max().item() > 0 for g in enc_grads), "all encoder grads are zero"
|
|
||||||
@@ -1,126 +0,0 @@
|
|||||||
"""Tests for multi-pair G10 pipeline (Option C).
|
|
||||||
|
|
||||||
Tests the prepare_multipair.py merge logic and train.py multipair build().
|
|
||||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_multipair.py -v
|
|
||||||
"""
|
|
||||||
import importlib.util
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
import pytest
|
|
||||||
import os
|
|
||||||
|
|
||||||
|
|
||||||
def _import_mp():
|
|
||||||
spec = importlib.util.spec_from_file_location("prepare_multipair", "scripts/prepare_multipair.py")
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="module")
|
|
||||||
def mp():
|
|
||||||
return _import_mp()
|
|
||||||
|
|
||||||
|
|
||||||
def _pair_df(start: str, n_hours: int, seed: int) -> pd.DataFrame:
|
|
||||||
"""Synthetic single-pair hourly parquet (same schema as prepare_hourly output)."""
|
|
||||||
rng = np.random.default_rng(seed)
|
|
||||||
dts = pd.date_range(start, periods=n_hours, freq="h")
|
|
||||||
closes = 1.1 + np.cumsum(rng.normal(0, 0.001, n_hours))
|
|
||||||
return pd.DataFrame({
|
|
||||||
"datetime": dts,
|
|
||||||
"close": closes,
|
|
||||||
"ret": rng.normal(0, 0.001, n_hours),
|
|
||||||
"realized_vol": np.abs(rng.normal(0.0005, 0.0001, n_hours)),
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
# 1. merge_pair_parquets returns inner join on datetime
|
|
||||||
def test_merge_inner_join(mp):
|
|
||||||
eur = _pair_df("2020-01-01 00:00", 100, seed=1) # t0 to t0+99h
|
|
||||||
gbp = _pair_df("2020-01-01 20:00", 60, seed=2) # t0+20 to t0+79h → 60 common
|
|
||||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
|
||||||
assert len(result) == 60, f"expected 60 (inner join), got {len(result)}"
|
|
||||||
|
|
||||||
|
|
||||||
# 2. merge_pair_parquets prefixes columns with pair name
|
|
||||||
def test_merge_column_prefixes(mp):
|
|
||||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
|
||||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
|
||||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
|
||||||
assert "datetime" in result.columns, "datetime column missing"
|
|
||||||
assert "eurusd_ret" in result.columns
|
|
||||||
assert "eurusd_rv" in result.columns
|
|
||||||
assert "gbpusd_ret" in result.columns
|
|
||||||
assert "gbpusd_rv" in result.columns
|
|
||||||
# raw pair columns should not leak through unprefixed
|
|
||||||
assert "ret" not in result.columns
|
|
||||||
assert "realized_vol" not in result.columns
|
|
||||||
|
|
||||||
|
|
||||||
# 3. No NaN in merged output
|
|
||||||
def test_merge_no_nan(mp):
|
|
||||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
|
||||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
|
||||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
|
||||||
nan_count = result.isnull().sum().sum()
|
|
||||||
assert nan_count == 0, f"{nan_count} NaN values in merged output"
|
|
||||||
|
|
||||||
|
|
||||||
# 4. PAIRS constant is a non-empty list starting with eurusd
|
|
||||||
def test_pairs_constant(mp):
|
|
||||||
assert hasattr(mp, "PAIRS"), "PAIRS constant missing from prepare_multipair.py"
|
|
||||||
assert len(mp.PAIRS) >= 2, "PAIRS must have at least 2 pairs"
|
|
||||||
assert mp.PAIRS[0] == "eurusd", "first pair must be eurusd (target pair)"
|
|
||||||
|
|
||||||
|
|
||||||
# 5. merge target column is eurusd_rv (for build() target selection)
|
|
||||||
def test_merge_has_eurusd_rv_as_target(mp):
|
|
||||||
eur = _pair_df("2020-01-01 00:00", 50, seed=1)
|
|
||||||
gbp = _pair_df("2020-01-01 00:00", 50, seed=2)
|
|
||||||
result = mp.merge_pair_parquets({"eurusd": eur, "gbpusd": gbp})
|
|
||||||
assert "eurusd_rv" in result.columns, "eurusd_rv (target) missing from merged output"
|
|
||||||
assert (result["eurusd_rv"] > 0).all(), "eurusd_rv should be positive"
|
|
||||||
|
|
||||||
|
|
||||||
# 6. train.py recognises JEPA_USE_MULTIPAIR env var
|
|
||||||
def test_use_multipair_knob():
|
|
||||||
import importlib.util as ilu
|
|
||||||
spec = ilu.spec_from_file_location(f"train_mp_{id(None)}", "train.py")
|
|
||||||
mod = ilu.module_from_spec(spec)
|
|
||||||
saved = os.environ.get("JEPA_USE_MULTIPAIR")
|
|
||||||
os.environ["JEPA_USE_MULTIPAIR"] = "1"
|
|
||||||
try:
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
finally:
|
|
||||||
if saved is None:
|
|
||||||
os.environ.pop("JEPA_USE_MULTIPAIR", None)
|
|
||||||
else:
|
|
||||||
os.environ["JEPA_USE_MULTIPAIR"] = saved
|
|
||||||
assert hasattr(mod, "USE_MULTIPAIR"), "USE_MULTIPAIR knob missing from train.py"
|
|
||||||
assert mod.USE_MULTIPAIR is True
|
|
||||||
|
|
||||||
|
|
||||||
# 7. build() uses n_pairs*2 channels when multipair parquet present
|
|
||||||
def test_build_uses_multipair_channels():
|
|
||||||
import importlib.util as ilu
|
|
||||||
multipair_path = "data/processed/eurusd_multipair.parquet"
|
|
||||||
if not os.path.exists(multipair_path):
|
|
||||||
pytest.skip("eurusd_multipair.parquet not present — run data:prepare:multipair first")
|
|
||||||
saved = os.environ.get("JEPA_USE_MULTIPAIR")
|
|
||||||
os.environ["JEPA_USE_MULTIPAIR"] = "1"
|
|
||||||
try:
|
|
||||||
spec = ilu.spec_from_file_location(f"train_mp2_{id(None)}", "train.py")
|
|
||||||
mod = ilu.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
(Xtr, _), _ = mod.build()
|
|
||||||
finally:
|
|
||||||
if saved is None:
|
|
||||||
os.environ.pop("JEPA_USE_MULTIPAIR", None)
|
|
||||||
else:
|
|
||||||
os.environ["JEPA_USE_MULTIPAIR"] = saved
|
|
||||||
mp = _import_mp()
|
|
||||||
expected_ch = len(mp.PAIRS) * 2
|
|
||||||
assert Xtr.shape[2] == expected_ch, (
|
|
||||||
f"expected {expected_ch} channels (n_pairs={len(mp.PAIRS)}×2), got {Xtr.shape[2]}"
|
|
||||||
)
|
|
||||||
@@ -1,207 +0,0 @@
|
|||||||
"""Failing tests for scripts/prepare_hourly.py.
|
|
||||||
|
|
||||||
Tests the M1 → hourly aggregation logic using synthetic data before touching
|
|
||||||
real downloads.
|
|
||||||
|
|
||||||
Run: cd ~/dev/AI/jepa-fx-risk && .venv/bin/python -m pytest tests/test_prepare_hourly.py -v
|
|
||||||
"""
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
import pytest
|
|
||||||
import importlib.util, sys, os
|
|
||||||
|
|
||||||
|
|
||||||
def _import():
|
|
||||||
spec = importlib.util.spec_from_file_location(
|
|
||||||
"prepare_hourly", "scripts/prepare_hourly.py"
|
|
||||||
)
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(scope="module")
|
|
||||||
def ph():
|
|
||||||
return _import()
|
|
||||||
|
|
||||||
|
|
||||||
def _make_m1(n_days: int = 3, price: float = 1.1000, noise: float = 0.0005) -> pd.DataFrame:
|
|
||||||
"""Synthetic M1 DataFrame starting 2020-01-06 (Monday), 390 ticks/day."""
|
|
||||||
rng = np.random.default_rng(42)
|
|
||||||
# generate full trading hours: Mon-Fri 00:00-23:59 (FX is 24h weekday)
|
|
||||||
start = pd.Timestamp("2020-01-06 00:00:00") # Monday
|
|
||||||
periods = n_days * 24 * 60
|
|
||||||
ts = pd.date_range(start, periods=periods, freq="min")
|
|
||||||
# remove weekends
|
|
||||||
ts = ts[ts.day_of_week < 5]
|
|
||||||
prices = price + np.cumsum(rng.normal(0, noise, len(ts)))
|
|
||||||
return pd.DataFrame({"ts": ts, "close": prices})
|
|
||||||
|
|
||||||
|
|
||||||
# 1. resample_to_hourly: DataFrame has correct columns
|
|
||||||
def test_columns(ph):
|
|
||||||
m1 = _make_m1()
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert set(["datetime", "close", "ret", "realized_vol"]).issubset(hourly.columns), \
|
|
||||||
f"missing columns: {hourly.columns.tolist()}"
|
|
||||||
|
|
||||||
|
|
||||||
# 2. No cross-weekend interpolation: gap between Friday 23:xx and Sunday/Monday must remain
|
|
||||||
def test_no_weekend_interpolation(ph):
|
|
||||||
# Make 2 days: Friday + Monday (skip Saturday/Sunday)
|
|
||||||
fri = pd.date_range("2020-01-10 00:00", "2020-01-10 23:59", freq="min") # Friday
|
|
||||||
mon = pd.date_range("2020-01-13 00:00", "2020-01-13 23:59", freq="min") # Monday
|
|
||||||
ts = fri.append(mon)
|
|
||||||
prices = 1.1 + np.cumsum(np.random.default_rng(0).normal(0, 0.0001, len(ts)))
|
|
||||||
m1 = pd.DataFrame({"ts": ts, "close": prices})
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
dates = pd.DatetimeIndex(hourly["datetime"]).date
|
|
||||||
import datetime
|
|
||||||
sat = datetime.date(2020, 1, 11)
|
|
||||||
sun = datetime.date(2020, 1, 12)
|
|
||||||
assert sat not in dates and sun not in dates, "weekend rows found in hourly output"
|
|
||||||
|
|
||||||
|
|
||||||
# 3. Realized vol = sqrt(sum(r²)) over minute returns in each hour
|
|
||||||
def test_realized_vol_formula(ph):
|
|
||||||
# Two hours: anchor gives 10:00 a valid ret; measurement hour has one known log-return.
|
|
||||||
ts0 = pd.date_range("2020-01-06 09:00", periods=60, freq="min")
|
|
||||||
ts1 = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
|
|
||||||
prices0 = np.ones(60) * 1.0
|
|
||||||
# price jumps at minute 1 and STAYS (no reversion) → one non-zero log-return
|
|
||||||
prices1 = np.full(60, np.exp(0.01))
|
|
||||||
prices1[0] = 1.0 # only first tick is at 1.0; jump happens at tick 1
|
|
||||||
m1 = pd.DataFrame({
|
|
||||||
"ts": np.concatenate([ts0, ts1]),
|
|
||||||
"close": np.concatenate([prices0, prices1]),
|
|
||||||
})
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert len(hourly) >= 1, "no rows after resample"
|
|
||||||
rv = hourly.iloc[-1]["realized_vol"]
|
|
||||||
expected = np.sqrt(0.01 ** 2)
|
|
||||||
assert abs(rv - expected) < 1e-6, f"realized_vol={rv:.8f}, expected≈{expected:.8f}"
|
|
||||||
|
|
||||||
|
|
||||||
# 4. Only hours with ≥ 30 M1 bars are kept (thin hours dropped)
|
|
||||||
def test_thin_hours_dropped(ph):
|
|
||||||
# 4 hours: pre-anchor gives 09:00 a valid ret; full survives; thin (11:00) is dropped.
|
|
||||||
# pre-anchor (08:00): gives 09:00 a valid ret
|
|
||||||
# anchor (09:00): 60 bars, valid ret → kept
|
|
||||||
# full (10:00): 60 bars, valid ret → kept
|
|
||||||
# thin (11:00): 10 bars → dropped
|
|
||||||
# Result: 3 hourly candidates, first (pre-anchor) gets NaN ret → dropped → 2 rows
|
|
||||||
pre = pd.date_range("2020-01-06 08:00", periods=60, freq="min")
|
|
||||||
anchor= pd.date_range("2020-01-06 09:00", periods=60, freq="min")
|
|
||||||
full = pd.date_range("2020-01-06 10:00", periods=60, freq="min")
|
|
||||||
thin = pd.date_range("2020-01-06 11:00", periods=10, freq="min")
|
|
||||||
ts = pre.append(anchor).append(full).append(thin)
|
|
||||||
m1 = pd.DataFrame({"ts": ts, "close": np.ones(len(ts)) * 1.1})
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert len(hourly) == 2, f"expected 2 rows (pre-anchor NaN ret dropped + thin dropped), got {len(hourly)}"
|
|
||||||
|
|
||||||
|
|
||||||
# 5. Output parquet path and schema (integration — reads actual M1 zips if present)
|
|
||||||
def test_output_schema_from_zips(ph, tmp_path):
|
|
||||||
import zipfile, io
|
|
||||||
rows = []
|
|
||||||
for h in range(24):
|
|
||||||
for m in range(60):
|
|
||||||
rows.append(f"20200106 {h:02d}{m:02d}00;1.10000;1.10100;1.09900;1.10000;100")
|
|
||||||
csv_content = "\n".join(rows).encode()
|
|
||||||
zip_buf = io.BytesIO()
|
|
||||||
with zipfile.ZipFile(zip_buf, "w") as zf:
|
|
||||||
zf.writestr("DAT_ASCII_EURUSD_M1_2020.csv", csv_content)
|
|
||||||
zip_buf.seek(0)
|
|
||||||
raw_dir = tmp_path / "raw"
|
|
||||||
raw_dir.mkdir()
|
|
||||||
(raw_dir / "DAT_ASCII_EURUSD_M1_2020.zip").write_bytes(zip_buf.read())
|
|
||||||
|
|
||||||
out_path = str(tmp_path / "eurusd_hourly.parquet")
|
|
||||||
ph.build_hourly_parquet(raw_dir=str(raw_dir), out_path=out_path)
|
|
||||||
assert os.path.exists(out_path), "output parquet not created"
|
|
||||||
df = pd.read_parquet(out_path)
|
|
||||||
assert set(["datetime", "close", "ret", "realized_vol"]).issubset(df.columns)
|
|
||||||
assert len(df) > 0
|
|
||||||
|
|
||||||
|
|
||||||
# ── New OHLCV-derived features ────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def _make_m1_ohlcv(n_hours: int = 4, price: float = 1.1) -> pd.DataFrame:
|
|
||||||
"""Synthetic M1 with distinct O, H, L, C so hl_range and ret_intrabar are nonzero."""
|
|
||||||
rng = np.random.default_rng(7)
|
|
||||||
ts = pd.date_range("2020-01-06 00:00", periods=n_hours * 60, freq="min")
|
|
||||||
closes = price + np.cumsum(rng.normal(0, 0.0002, len(ts)))
|
|
||||||
highs = closes + rng.uniform(0.0001, 0.0005, len(ts))
|
|
||||||
lows = closes - rng.uniform(0.0001, 0.0005, len(ts))
|
|
||||||
opens = np.roll(closes, 1); opens[0] = price
|
|
||||||
return pd.DataFrame({"ts": ts, "open": opens, "high": highs, "low": lows, "close": closes})
|
|
||||||
|
|
||||||
|
|
||||||
# 6. resample_to_hourly produces hl_range column
|
|
||||||
def test_hourly_has_hl_range(ph):
|
|
||||||
m1 = _make_m1_ohlcv()
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert "hl_range" in hourly.columns, f"missing hl_range; cols={hourly.columns.tolist()}"
|
|
||||||
assert (hourly["hl_range"] > 0).all(), "hl_range should be positive"
|
|
||||||
|
|
||||||
|
|
||||||
# 7. resample_to_hourly produces ret_intrabar column
|
|
||||||
def test_hourly_has_ret_intrabar(ph):
|
|
||||||
m1 = _make_m1_ohlcv()
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert "ret_intrabar" in hourly.columns, f"missing ret_intrabar; cols={hourly.columns.tolist()}"
|
|
||||||
|
|
||||||
|
|
||||||
# 8. hl_range = log(hourly_high / hourly_low)
|
|
||||||
def test_hl_range_formula(ph):
|
|
||||||
# Two hours; second has known H=1.105, L=1.095
|
|
||||||
ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
|
|
||||||
ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
|
|
||||||
closes = np.full(120, 1.1)
|
|
||||||
highs = np.full(120, 1.1)
|
|
||||||
lows = np.full(120, 1.1)
|
|
||||||
# second hour: known spread
|
|
||||||
highs[60:] = 1.105
|
|
||||||
lows[60:] = 1.095
|
|
||||||
m1 = pd.DataFrame({
|
|
||||||
"ts": np.concatenate([ts0, ts1]),
|
|
||||||
"open": closes, "high": highs, "low": lows, "close": closes,
|
|
||||||
})
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert len(hourly) >= 1
|
|
||||||
hl = hourly.iloc[-1]["hl_range"]
|
|
||||||
expected = float(np.log(1.105 / 1.095))
|
|
||||||
assert abs(hl - expected) < 1e-6, f"hl_range={hl:.8f}, expected={expected:.8f}"
|
|
||||||
|
|
||||||
|
|
||||||
# 9. ret_intrabar = log(hourly_last_close / hourly_first_open)
|
|
||||||
def test_ret_intrabar_formula(ph):
|
|
||||||
ts0 = pd.date_range("2020-01-06 00:00", periods=60, freq="min")
|
|
||||||
ts1 = pd.date_range("2020-01-06 01:00", periods=60, freq="min")
|
|
||||||
closes = np.full(120, 1.1)
|
|
||||||
opens = np.full(120, 1.1)
|
|
||||||
# second hour: open=1.09, close=1.11
|
|
||||||
opens[60] = 1.09
|
|
||||||
closes[119] = 1.11
|
|
||||||
m1 = pd.DataFrame({
|
|
||||||
"ts": np.concatenate([ts0, ts1]),
|
|
||||||
"open": opens, "high": closes + 0.001, "low": closes - 0.001, "close": closes,
|
|
||||||
})
|
|
||||||
hourly = ph.resample_to_hourly(m1)
|
|
||||||
assert len(hourly) >= 1
|
|
||||||
rib = hourly.iloc[-1]["ret_intrabar"]
|
|
||||||
expected = float(np.log(1.11 / 1.09))
|
|
||||||
assert abs(rib - expected) < 1e-6, f"ret_intrabar={rib:.8f}, expected={expected:.8f}"
|
|
||||||
|
|
||||||
|
|
||||||
# 10. build() in train.py uses 2 feature channels (HPO: hl_range/ret_intrabar redundant)
|
|
||||||
def test_build_uses_2_channels(tmp_path):
|
|
||||||
import importlib.util, os
|
|
||||||
hourly_path = "data/processed/eurusd_hourly.parquet"
|
|
||||||
if not os.path.exists(hourly_path):
|
|
||||||
pytest.skip("eurusd_hourly.parquet not present")
|
|
||||||
spec = importlib.util.spec_from_file_location("train_2ch", "train.py")
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
(Xtr, _), _ = mod.build()
|
|
||||||
assert Xtr.shape[2] == 2, f"expected 2 channels, got {Xtr.shape[2]}"
|
|
||||||
@@ -1,134 +0,0 @@
|
|||||||
"""Tests for scripts/prepare_regime.py — HMM regime detector (jepa-fx-risk#13).
|
|
||||||
|
|
||||||
TDD: tests first, implementation follows.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import importlib.util
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "prepare_regime.py"
|
|
||||||
|
|
||||||
DATA_DIR = Path(__file__).parent.parent / "data" / "processed"
|
|
||||||
HOURLY = DATA_DIR / "eurusd_hourly.parquet"
|
|
||||||
DAILY = DATA_DIR / "eurusd_daily.parquet"
|
|
||||||
|
|
||||||
|
|
||||||
def _import():
|
|
||||||
spec = importlib.util.spec_from_file_location("prepare_regime", _SCRIPT)
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def mod():
|
|
||||||
return _import()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# fit_regime_hmm — pure function (doesn't touch disk)
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
def _synthetic_rv(seed=42, size=500):
|
|
||||||
"""Noisy 3-regime vol series: calm→stressed→crisis→calm interleaved."""
|
|
||||||
rng = np.random.default_rng(seed)
|
|
||||||
low = np.abs(rng.normal(0.005, 0.001, size=size // 3))
|
|
||||||
mid = np.abs(rng.normal(0.015, 0.003, size=size // 3))
|
|
||||||
high = np.abs(rng.normal(0.04, 0.008, size=size - 2 * (size // 3)))
|
|
||||||
return np.concatenate([low, mid, high])
|
|
||||||
|
|
||||||
|
|
||||||
class TestFitRegimeHmm:
|
|
||||||
def test_returns_integer_labels(self, mod):
|
|
||||||
rv = _synthetic_rv(seed=0)
|
|
||||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
|
||||||
assert np.issubdtype(labels.dtype, np.integer), f"dtype={labels.dtype}"
|
|
||||||
assert len(labels) == len(rv)
|
|
||||||
|
|
||||||
def test_states_are_0_1_2(self, mod):
|
|
||||||
rv = _synthetic_rv(seed=1)
|
|
||||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
|
||||||
unique = set(labels.tolist())
|
|
||||||
assert unique.issubset({0, 1, 2}), f"unexpected states: {unique}"
|
|
||||||
|
|
||||||
def test_deterministic(self, mod):
|
|
||||||
rv = _synthetic_rv(seed=7)
|
|
||||||
a = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
|
||||||
b = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
|
||||||
assert np.array_equal(a, b), "HMM not deterministic with same random_state"
|
|
||||||
|
|
||||||
def test_sorted_by_vol_asc(self, mod):
|
|
||||||
# 3 clearly separated noisy clusters; state 0 should be calm, 2 should be crisis.
|
|
||||||
rng = np.random.default_rng(42)
|
|
||||||
n = 200
|
|
||||||
low = np.abs(rng.normal(0.005, 0.001, n))
|
|
||||||
mid = np.abs(rng.normal(0.015, 0.003, n))
|
|
||||||
high = np.abs(rng.normal(0.05, 0.008, n))
|
|
||||||
rv = np.concatenate([low, mid, high])
|
|
||||||
labels = mod.fit_regime_hmm(rv, n_states=3, random_state=42)
|
|
||||||
# Mean regime label in the high-vol section should exceed mean in the low-vol section.
|
|
||||||
assert labels[2*n:].mean() > labels[:n].mean(), \
|
|
||||||
"crisis section mean regime label should exceed calm section"
|
|
||||||
# The calm section should not be labeled as crisis (2) dominantly.
|
|
||||||
calm_modal = int(np.bincount(labels[:n]).argmax())
|
|
||||||
assert calm_modal < 2, f"calm section mostly labeled {calm_modal}, expected 0 or 1"
|
|
||||||
|
|
||||||
def test_two_states(self, mod):
|
|
||||||
rv = _synthetic_rv(seed=0)
|
|
||||||
labels = mod.fit_regime_hmm(rv, n_states=2, random_state=42)
|
|
||||||
unique = set(labels.tolist())
|
|
||||||
assert unique.issubset({0, 1})
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# prepare_regime_df — reads parquet, fits HMM, returns DataFrame
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestPrepareRegimeDf:
|
|
||||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
|
||||||
def test_output_columns(self, mod):
|
|
||||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
|
||||||
assert "datetime" in df.columns
|
|
||||||
assert "regime" in df.columns
|
|
||||||
|
|
||||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
|
||||||
def test_regime_values(self, mod):
|
|
||||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
|
||||||
unique = set(df["regime"].tolist())
|
|
||||||
assert unique.issubset({0, 1, 2}), f"unexpected regime values: {unique}"
|
|
||||||
|
|
||||||
@pytest.mark.skipif(not HOURLY.exists(), reason="hourly parquet not available")
|
|
||||||
def test_no_nulls(self, mod):
|
|
||||||
df = mod.prepare_regime_df(str(HOURLY), freq="hourly")
|
|
||||||
assert df["regime"].isna().sum() == 0
|
|
||||||
|
|
||||||
@pytest.mark.skipif(not DAILY.exists(), reason="daily parquet not available")
|
|
||||||
def test_daily_fallback(self, mod):
|
|
||||||
df = mod.prepare_regime_df(str(DAILY), freq="daily")
|
|
||||||
assert "regime" in df.columns
|
|
||||||
assert set(df["regime"].tolist()).issubset({0, 1, 2})
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Integration: check that train.py REGIME SEAM exists and is togglable
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestTrainPyRegimeSeam:
|
|
||||||
def test_enable_regime_env_var_documented(self):
|
|
||||||
train_py = Path(__file__).parent.parent / "train.py"
|
|
||||||
content = train_py.read_text()
|
|
||||||
assert "JEPA_ENABLE_REGIME" in content, "JEPA_ENABLE_REGIME toggle not found in train.py"
|
|
||||||
|
|
||||||
def test_regime_seam_comment_present(self):
|
|
||||||
train_py = Path(__file__).parent.parent / "train.py"
|
|
||||||
content = train_py.read_text()
|
|
||||||
assert "REGIME" in content and "seam" in content.lower(), \
|
|
||||||
"agent-editable regime seam marker not found in train.py"
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
"""Tests for scripts/var_breach.py — VaR breach rate + Kupiec POF (jepa-fx-risk#12).
|
|
||||||
|
|
||||||
Golden tests first: verify the math before wiring it into train.py.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import importlib.util
|
|
||||||
import math
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
_SCRIPT = Path(__file__).parent.parent / "scripts" / "var_breach.py"
|
|
||||||
|
|
||||||
|
|
||||||
def _import():
|
|
||||||
spec = importlib.util.spec_from_file_location("var_breach", _SCRIPT)
|
|
||||||
mod = importlib.util.module_from_spec(spec)
|
|
||||||
spec.loader.exec_module(mod)
|
|
||||||
return mod
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
|
||||||
def mod():
|
|
||||||
return _import()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# var_breach_rate
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestVarBreachRate:
|
|
||||||
def test_zero_breaches(self, mod):
|
|
||||||
# 0.02 < 0.01×2.326=0.02326 → no breach
|
|
||||||
rate, _ = mod.var_breach_rate([0.01, 0.01], [0.02, 0.02])
|
|
||||||
assert rate == 0.0
|
|
||||||
|
|
||||||
def test_all_breach(self, mod):
|
|
||||||
# 0.03 > 0.02326 → all breach
|
|
||||||
rate, _ = mod.var_breach_rate([0.01, 0.01], [0.03, 0.03])
|
|
||||||
assert rate == 1.0
|
|
||||||
|
|
||||||
def test_golden_two_of_ten(self, mod):
|
|
||||||
pred = [0.01] * 10
|
|
||||||
actual = [0.01] * 10
|
|
||||||
actual[0] = 0.03 # breach
|
|
||||||
actual[2] = 0.03 # breach
|
|
||||||
rate, kupiec_p = mod.var_breach_rate(pred, actual)
|
|
||||||
assert abs(rate - 0.2) < 1e-9, f"rate={rate}"
|
|
||||||
assert kupiec_p < 0.05, f"kupiec_p={kupiec_p}" # strong reject
|
|
||||||
|
|
||||||
def test_perfect_calibration(self, mod):
|
|
||||||
# n=100, 1 breach → p_hat=0.01=p0=0.01 → LR=0 → kupiec_p≈1
|
|
||||||
pred = [0.01] * 100
|
|
||||||
actual = [0.015] * 100
|
|
||||||
actual[0] = 0.025 # 0.025 > 0.02326 → breach
|
|
||||||
rate, kupiec_p = mod.var_breach_rate(pred, actual)
|
|
||||||
assert abs(rate - 0.01) < 1e-9
|
|
||||||
assert kupiec_p > 0.9, f"kupiec_p={kupiec_p}"
|
|
||||||
|
|
||||||
def test_boundary_at_var_is_not_breach(self, mod):
|
|
||||||
# exactly at VaR_99 is NOT a breach (strict >)
|
|
||||||
z99 = 2.326
|
|
||||||
var = 0.01 * z99
|
|
||||||
rate, _ = mod.var_breach_rate([0.01], [var], z99=z99)
|
|
||||||
assert rate == 0.0
|
|
||||||
|
|
||||||
def test_empty_returns_zero_one(self, mod):
|
|
||||||
rate, kupiec_p = mod.var_breach_rate([], [])
|
|
||||||
assert rate == 0.0
|
|
||||||
assert kupiec_p == 1.0
|
|
||||||
|
|
||||||
def test_metric_key_no_whitespace(self, mod):
|
|
||||||
key = mod.METRIC_KEY
|
|
||||||
assert key == key.strip(), f"metric key has surrounding whitespace: {key!r}"
|
|
||||||
assert " " not in key, f"metric key contains space: {key!r}"
|
|
||||||
|
|
||||||
def test_metric_key_is_canonical(self, mod):
|
|
||||||
assert mod.METRIC_KEY == "VaR_breach_rate_99_oos_regime_cond"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# kupiec_pvalue
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
class TestKupiecPValue:
|
|
||||||
def test_perfectly_calibrated(self, mod):
|
|
||||||
# p_hat == p0 → LR=0 → p-value=1
|
|
||||||
p = mod.kupiec_pvalue(100, 1, p0=0.01)
|
|
||||||
assert p > 0.99, f"p={p}"
|
|
||||||
|
|
||||||
def test_strong_reject_high_breach(self, mod):
|
|
||||||
# 20% breach when 1% expected → p << 0.05
|
|
||||||
p = mod.kupiec_pvalue(100, 20, p0=0.01)
|
|
||||||
assert p < 0.001, f"p={p}"
|
|
||||||
|
|
||||||
def test_zero_breaches_not_nan(self, mod):
|
|
||||||
p = mod.kupiec_pvalue(100, 0, p0=0.01)
|
|
||||||
assert not math.isnan(p)
|
|
||||||
assert 0 <= p <= 1.0
|
|
||||||
|
|
||||||
def test_all_breaches_not_nan(self, mod):
|
|
||||||
p = mod.kupiec_pvalue(10, 10, p0=0.01)
|
|
||||||
assert not math.isnan(p)
|
|
||||||
assert p < 0.001 # extremely unlikely
|
|
||||||
|
|
||||||
def test_zero_observations(self, mod):
|
|
||||||
p = mod.kupiec_pvalue(0, 0)
|
|
||||||
assert p == 1.0
|
|
||||||
@@ -1,385 +0,0 @@
|
|||||||
"""train.py — autoresearch agent file (only this may be edited).
|
|
||||||
|
|
||||||
HEPA backbone (Petersen et al., arXiv:2605.11130, ICML 2026 Spotlight):
|
|
||||||
Causal Transformer pre-trained via horizon-conditioned JEPA. Predictor
|
|
||||||
maps (h_t, Δt) → predicted future embedding; loss = VICReg (L1 alignment
|
|
||||||
on L2-normalised reps + variance-covariance regulariser, no stop-gradient).
|
|
||||||
Probe: ridge regression on the last-token embedding (true OOS split).
|
|
||||||
|
|
||||||
Agent may tune: encoder depth/width, patch geometry, ALPHA, DELTA_T_MAX,
|
|
||||||
optimizer, LR. Do NOT touch prepare_data.py, loop.py, or the data pipeline.
|
|
||||||
"""
|
|
||||||
import json
|
|
||||||
import math
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
import torch
|
|
||||||
import torch.nn as nn
|
|
||||||
import torch.nn.functional as F
|
|
||||||
|
|
||||||
# --- agent-tunable knobs (all overridable via JEPA_* env vars for HPO) ---
|
|
||||||
import os as _os
|
|
||||||
USE_HOURLY = True
|
|
||||||
WINDOW = int(_os.environ.get("JEPA_WINDOW", 120)) # HPO winner: 5-day context
|
|
||||||
PATCH_LEN = int(_os.environ.get("JEPA_PATCH_LEN", 24))
|
|
||||||
D_MODEL = int(_os.environ.get("JEPA_D_MODEL", 128))
|
|
||||||
DEPTH = int(_os.environ.get("JEPA_DEPTH", 2))
|
|
||||||
N_HEADS = int(_os.environ.get("JEPA_N_HEADS", 4))
|
|
||||||
ALPHA = float(_os.environ.get("JEPA_ALPHA", 0.1))
|
|
||||||
DELTA_T_MAX = int(_os.environ.get("JEPA_DELTA_T_MAX", 3))
|
|
||||||
BATCH_SIZE = int(_os.environ.get("JEPA_BATCH_SIZE", 512))
|
|
||||||
EPOCHS = int(_os.environ.get("JEPA_EPOCHS", 300))
|
|
||||||
LR = float(_os.environ.get("JEPA_LR", 3e-4))
|
|
||||||
PHASE1_EPOCHS = int(_os.environ.get("JEPA_PHASE1_EPOCHS", 200))
|
|
||||||
PHASE1_LR = float(_os.environ.get("JEPA_PHASE1_LR", 1e-3))
|
|
||||||
PHASE1_JOINT = bool(int(_os.environ.get("JEPA_PHASE1_JOINT", 1)))
|
|
||||||
PHASE1_JOINT_EPOCHS= int(_os.environ.get("JEPA_PHASE1_JOINT_EPOCHS", 30))
|
|
||||||
PHASE1_ENCODER_LR = float(_os.environ.get("JEPA_PHASE1_ENCODER_LR", 3e-6))
|
|
||||||
USE_MULTIPAIR = bool(int(_os.environ.get("JEPA_USE_MULTIPAIR", 0)))
|
|
||||||
JEPA_ENABLE_REGIME = bool(int(_os.environ.get("JEPA_ENABLE_REGIME", 0)))
|
|
||||||
SEED = int(_os.environ.get("JEPA_SEED", 0))
|
|
||||||
# ---------------------------
|
|
||||||
|
|
||||||
torch.manual_seed(SEED)
|
|
||||||
np.random.seed(SEED)
|
|
||||||
dev = "cuda" if torch.cuda.is_available() else "cpu"
|
|
||||||
|
|
||||||
|
|
||||||
# ── VICReg pretraining loss ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def vicreg_loss(h_pred: torch.Tensor, h_target: torch.Tensor, alpha: float = 0.1) -> torch.Tensor:
|
|
||||||
"""L = (1-α)·L1(normalize(ĥ), normalize(h*)) + α·(L_var + L_cov).
|
|
||||||
|
|
||||||
Both encoders receive gradients (joint training — no stop-grad on h_target).
|
|
||||||
Variance-covariance terms prevent embedding collapse.
|
|
||||||
"""
|
|
||||||
pred_n = F.normalize(h_pred, dim=-1)
|
|
||||||
targ_n = F.normalize(h_target, dim=-1)
|
|
||||||
l1 = F.l1_loss(pred_n, targ_n)
|
|
||||||
# variance hinge: push each feature std toward ≥ 1
|
|
||||||
std = h_pred.std(dim=0) + 1e-4
|
|
||||||
l_var = F.relu(1.0 - std).mean()
|
|
||||||
# covariance penalty: decorrelate features
|
|
||||||
B, D = h_pred.shape
|
|
||||||
h_c = h_pred - h_pred.mean(dim=0, keepdim=True)
|
|
||||||
cov = (h_c.t() @ h_c) / max(B - 1, 1)
|
|
||||||
off = cov - torch.diag(torch.diag(cov))
|
|
||||||
l_cov = (off ** 2).sum() / D
|
|
||||||
return (1 - alpha) * l1 + alpha * (l_var + l_cov)
|
|
||||||
|
|
||||||
|
|
||||||
# ── CausalEncoder ─────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
class CausalEncoder(nn.Module):
|
|
||||||
"""Non-overlapping patches → per-patch LayerNorm → causal Transformer → all tokens (B, N, D).
|
|
||||||
|
|
||||||
Per-patch LayerNorm instead of full-window RevIN: each patch is normalised
|
|
||||||
using only its own timesteps, so no future statistics leak into past tokens.
|
|
||||||
Use [:, -1, :] for probing (last token sees full context).
|
|
||||||
Use [:, c, :] for JEPA pretraining (context-at-c).
|
|
||||||
"""
|
|
||||||
def __init__(self, n_channels: int, patch_len: int, d_model: int,
|
|
||||||
n_heads: int, depth: int):
|
|
||||||
super().__init__()
|
|
||||||
self.patch_len = patch_len
|
|
||||||
self.d_model = d_model
|
|
||||||
patch_dim = patch_len * n_channels
|
|
||||||
self.patch_norm = nn.LayerNorm(patch_dim) # applied per-patch, no future leakage
|
|
||||||
self.embed = nn.Linear(patch_dim, d_model)
|
|
||||||
layer = nn.TransformerEncoderLayer(d_model, n_heads, 2 * d_model,
|
|
||||||
dropout=0.0, batch_first=True)
|
|
||||||
self.tf = nn.TransformerEncoder(layer, num_layers=depth)
|
|
||||||
self.norm = nn.LayerNorm(d_model)
|
|
||||||
|
|
||||||
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
|
||||||
B, W, F = x.shape
|
|
||||||
P = self.patch_len
|
|
||||||
N = W // P
|
|
||||||
tokens = x[:, :N * P, :].reshape(B, N, P * F)
|
|
||||||
tokens = self.embed(self.patch_norm(tokens))
|
|
||||||
# sinusoidal PE
|
|
||||||
pos = torch.arange(N, device=x.device).float()
|
|
||||||
div = torch.exp(torch.arange(0, self.d_model, 2, device=x.device).float()
|
|
||||||
* -(math.log(10000.0) / self.d_model))
|
|
||||||
pe = torch.zeros(N, self.d_model, device=x.device)
|
|
||||||
pe[:, 0::2] = torch.sin(pos.unsqueeze(1) * div)
|
|
||||||
pe[:, 1::2] = torch.cos(pos.unsqueeze(1) * div)
|
|
||||||
tokens = tokens + pe
|
|
||||||
# causal mask
|
|
||||||
mask = nn.Transformer.generate_square_subsequent_mask(N, device=x.device)
|
|
||||||
return self.norm(self.tf(tokens, mask=mask, is_causal=True))
|
|
||||||
|
|
||||||
|
|
||||||
# ── HorizonPredictor ─────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
class HorizonPredictor(nn.Module):
|
|
||||||
"""MLP(cat(h_t, Δt)) → predicted future embedding."""
|
|
||||||
def __init__(self, d_model: int):
|
|
||||||
super().__init__()
|
|
||||||
self.net = nn.Sequential(
|
|
||||||
nn.Linear(d_model + 1, d_model), nn.GELU(),
|
|
||||||
nn.Linear(d_model, d_model), nn.GELU(),
|
|
||||||
nn.Linear(d_model, d_model),
|
|
||||||
)
|
|
||||||
|
|
||||||
def forward(self, h: torch.Tensor, delta_t: torch.Tensor) -> torch.Tensor:
|
|
||||||
dt = delta_t.float().unsqueeze(-1)
|
|
||||||
return self.net(torch.cat([h, dt], dim=-1))
|
|
||||||
|
|
||||||
|
|
||||||
# ── Phase-1 supervised head ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
class SupervisedHead(nn.Module):
|
|
||||||
"""Small MLP trained on frozen HEPA embeddings to predict next-period realized vol."""
|
|
||||||
def __init__(self, d_model: int):
|
|
||||||
super().__init__()
|
|
||||||
self.net = nn.Sequential(
|
|
||||||
nn.Linear(d_model, d_model // 2), nn.GELU(),
|
|
||||||
nn.Linear(d_model // 2, 1),
|
|
||||||
)
|
|
||||||
|
|
||||||
def forward(self, h: torch.Tensor) -> torch.Tensor:
|
|
||||||
return self.net(h).squeeze(-1)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Data ─────────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def build():
|
|
||||||
"""Year-based split: encoder trains on ≤2021; probe evaluates on ≥2022 OOS.
|
|
||||||
|
|
||||||
Uses eurusd_hourly.parquet when USE_HOURLY=True and the file exists;
|
|
||||||
falls back to eurusd_daily.parquet otherwise.
|
|
||||||
"""
|
|
||||||
import os
|
|
||||||
multipair_path = "data/processed/eurusd_multipair.parquet"
|
|
||||||
hourly_path = "data/processed/eurusd_hourly.parquet"
|
|
||||||
daily_path = "data/processed/eurusd_daily.parquet"
|
|
||||||
if USE_MULTIPAIR and os.path.exists(multipair_path):
|
|
||||||
df = pd.read_parquet(multipair_path).reset_index(drop=True)
|
|
||||||
df["date"] = pd.to_datetime(df["datetime"])
|
|
||||||
# All {pair}_ret + {pair}_rv columns as features; eurusd_rv as target
|
|
||||||
feat_cols = [c for c in df.columns if c.endswith("_ret") or c.endswith("_rv")]
|
|
||||||
FEAT_COLS = feat_cols
|
|
||||||
target_col = "eurusd_rv"
|
|
||||||
elif USE_HOURLY and os.path.exists(hourly_path):
|
|
||||||
df = pd.read_parquet(hourly_path).reset_index(drop=True)
|
|
||||||
df["date"] = pd.to_datetime(df["datetime"])
|
|
||||||
# 2-channel default (HPO: adding hl_range+ret_intrabar hurt — correlated with base feats)
|
|
||||||
FEAT_COLS = ["ret", "realized_vol"]
|
|
||||||
target_col = "realized_vol"
|
|
||||||
else:
|
|
||||||
df = pd.read_parquet(daily_path).reset_index(drop=True)
|
|
||||||
df["date"] = pd.to_datetime(df["date"])
|
|
||||||
FEAT_COLS = ["ret", "realized_vol"]
|
|
||||||
target_col = "realized_vol"
|
|
||||||
# ── REGIME CONDITIONING SEAM — agent may vary this mechanism ─────────────
|
|
||||||
# Baseline: concat regime flag as an additional feature channel (0=calm, 2=crisis).
|
|
||||||
# Agent may swap for FiLM conditioning, learned regime embedding, or gating.
|
|
||||||
_regime_path = "data/processed/eurusd_regime.parquet"
|
|
||||||
if JEPA_ENABLE_REGIME and os.path.exists(_regime_path):
|
|
||||||
_rdf = pd.read_parquet(_regime_path)
|
|
||||||
_ts_col = "datetime" if "datetime" in _rdf.columns else "date"
|
|
||||||
_rdf[_ts_col] = pd.to_datetime(_rdf[_ts_col])
|
|
||||||
df = df.copy()
|
|
||||||
df = df.merge(
|
|
||||||
_rdf.rename(columns={_ts_col: "date"})[["date", "regime"]],
|
|
||||||
on="date", how="left",
|
|
||||||
)
|
|
||||||
df["regime"] = df["regime"].fillna(0).astype(np.float32)
|
|
||||||
FEAT_COLS = list(FEAT_COLS) + ["regime"]
|
|
||||||
# ── END REGIME SEAM ───────────────────────────────────────────────────────
|
|
||||||
feats = df[FEAT_COLS].to_numpy(np.float32)
|
|
||||||
target = df[target_col].to_numpy(np.float32)
|
|
||||||
tr_idx = df.index[df["date"].dt.year <= 2021].tolist()
|
|
||||||
te_idx = df.index[df["date"].dt.year >= 2022].tolist()
|
|
||||||
mu = feats[:tr_idx[-1]+1].mean(0)
|
|
||||||
sd = feats[:tr_idx[-1]+1].std(0) + 1e-8
|
|
||||||
fn = (feats - mu) / sd
|
|
||||||
def windows(idx):
|
|
||||||
X, y = [], []
|
|
||||||
for t in idx:
|
|
||||||
if t - WINDOW >= 0 and t + 1 < len(df):
|
|
||||||
X.append(fn[t - WINDOW:t]); y.append(target[t + 1])
|
|
||||||
return np.stack(X).astype(np.float32), np.array(y, np.float32)
|
|
||||||
return windows(tr_idx), windows(te_idx)
|
|
||||||
|
|
||||||
|
|
||||||
# ── Training ──────────────────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
def main():
|
|
||||||
(Xtr, ytr), (Xte, yte) = build()
|
|
||||||
n_feats = Xtr.shape[2]
|
|
||||||
n_patches = WINDOW // PATCH_LEN
|
|
||||||
N_tr = len(Xtr)
|
|
||||||
bs = min(BATCH_SIZE, N_tr)
|
|
||||||
|
|
||||||
enc = CausalEncoder(n_feats, PATCH_LEN, D_MODEL, N_HEADS, DEPTH).to(dev)
|
|
||||||
pred = HorizonPredictor(D_MODEL).to(dev)
|
|
||||||
opt = torch.optim.AdamW(list(enc.parameters()) + list(pred.parameters()), lr=LR)
|
|
||||||
|
|
||||||
for ep in range(EPOCHS):
|
|
||||||
# Random mini-batch (avoids OOM on large hourly dataset)
|
|
||||||
idx_b = torch.randperm(N_tr)[:bs]
|
|
||||||
Xb = torch.tensor(Xtr[idx_b.numpy()], device=dev)
|
|
||||||
|
|
||||||
# Sample random context position and horizon
|
|
||||||
c = torch.randint(0, n_patches - 1, ()).item()
|
|
||||||
dt = torch.randint(1, max(2, min(DELTA_T_MAX, n_patches - 1 - c) + 1), ()).item()
|
|
||||||
|
|
||||||
tokens = enc(Xb) # (bs, N, D)
|
|
||||||
h_ctx = tokens[:, c, :] # context embedding
|
|
||||||
h_tgt = tokens[:, c + dt, :] # target embedding (joint training)
|
|
||||||
h_hat = pred(h_ctx, torch.full((bs,), float(dt), device=dev))
|
|
||||||
loss = vicreg_loss(h_hat, h_tgt, alpha=ALPHA)
|
|
||||||
opt.zero_grad(); loss.backward(); opt.step()
|
|
||||||
|
|
||||||
enc.eval()
|
|
||||||
with torch.no_grad():
|
|
||||||
def embed(X_np):
|
|
||||||
chunks = []
|
|
||||||
for i in range(0, len(X_np), bs):
|
|
||||||
t = torch.tensor(X_np[i:i+bs], device=dev)
|
|
||||||
chunks.append(enc(t)[:, -1, :].cpu().numpy())
|
|
||||||
return np.concatenate(chunks, axis=0)
|
|
||||||
|
|
||||||
Etr = embed(Xtr)
|
|
||||||
Ete = embed(Xte)
|
|
||||||
|
|
||||||
# Ridge probe: fit on train, evaluate on OOS (true OOS R²)
|
|
||||||
mu_e = Etr.mean(0); sd_e = Etr.std(0) + 1e-8
|
|
||||||
Etr_n = (Etr - mu_e) / sd_e
|
|
||||||
Ete_n = (Ete - mu_e) / sd_e
|
|
||||||
A = np.hstack([Etr_n, np.ones((len(Etr_n), 1))])
|
|
||||||
w = np.linalg.solve(A.T @ A + 1e-3 * np.eye(A.shape[1]), A.T @ ytr)
|
|
||||||
pred_np = np.hstack([Ete_n, np.ones((len(Ete_n), 1))]) @ w
|
|
||||||
ss_res = ((yte - pred_np) ** 2).sum()
|
|
||||||
ss_tot = ((yte - yte.mean()) ** 2).sum()
|
|
||||||
val_vol_r2 = float(1 - ss_res / ss_tot)
|
|
||||||
|
|
||||||
# Phase-1: MLP supervised head — joint or frozen-encoder path
|
|
||||||
ytr_mu = float(ytr.mean()); ytr_sd = float(ytr.std()) + 1e-8
|
|
||||||
ytr_z = (ytr - ytr_mu) / ytr_sd
|
|
||||||
head = SupervisedHead(D_MODEL).to(dev)
|
|
||||||
p1_bs = min(BATCH_SIZE, len(Etr_n))
|
|
||||||
|
|
||||||
# Shared tensors for the frozen-head warmup (used by both paths)
|
|
||||||
Etr_t = torch.tensor(Etr_n, device=dev)
|
|
||||||
ytr_z_t = torch.tensor(ytr_z, device=dev)
|
|
||||||
Ete_t = torch.tensor(Ete_n, device=dev)
|
|
||||||
N_tr_h = len(Etr_t)
|
|
||||||
|
|
||||||
# Phase 1a: warm up head on frozen embeddings (both paths run this)
|
|
||||||
head_opt = torch.optim.Adam(head.parameters(), lr=PHASE1_LR, weight_decay=1e-4)
|
|
||||||
for _ in range(PHASE1_EPOCHS):
|
|
||||||
perm = torch.randperm(N_tr_h, device=dev)
|
|
||||||
for start in range(0, N_tr_h, p1_bs):
|
|
||||||
idx_h = perm[start:start + p1_bs]
|
|
||||||
loss_h = F.mse_loss(head(Etr_t[idx_h]), ytr_z_t[idx_h])
|
|
||||||
head_opt.zero_grad(); loss_h.backward(); head_opt.step()
|
|
||||||
|
|
||||||
if PHASE1_JOINT:
|
|
||||||
# Phase 1b: short joint fine-tuning — encoder nudged with tiny LR.
|
|
||||||
# Normalize live encoder output with FROZEN stats (mu_e, sd_e) so the
|
|
||||||
# head sees the same embedding distribution it was warmed up on.
|
|
||||||
enc.train()
|
|
||||||
mu_e_t = torch.tensor(mu_e, device=dev)
|
|
||||||
sd_e_t = torch.tensor(sd_e, device=dev)
|
|
||||||
Xtr_t = torch.tensor(Xtr, device=dev)
|
|
||||||
joint_opt = torch.optim.Adam([
|
|
||||||
{"params": head.parameters(), "lr": PHASE1_LR * 0.1},
|
|
||||||
{"params": enc.parameters(), "lr": PHASE1_ENCODER_LR},
|
|
||||||
], weight_decay=1e-4)
|
|
||||||
for _ in range(PHASE1_JOINT_EPOCHS):
|
|
||||||
perm = torch.randperm(len(Xtr_t), device=dev)
|
|
||||||
for start in range(0, len(Xtr_t), p1_bs):
|
|
||||||
idx_j = perm[start:start + p1_bs]
|
|
||||||
h_raw = enc(Xtr_t[idx_j])[:, -1, :]
|
|
||||||
h_n = (h_raw - mu_e_t) / sd_e_t # frozen-stats normalisation
|
|
||||||
loss_j = F.mse_loss(head(h_n), ytr_z_t[idx_j])
|
|
||||||
joint_opt.zero_grad(); loss_j.backward(); joint_opt.step()
|
|
||||||
enc.eval()
|
|
||||||
# Re-extract test embeddings with fine-tuned encoder, same normalisation
|
|
||||||
with torch.no_grad():
|
|
||||||
chunks = []
|
|
||||||
for i in range(0, len(Xte), p1_bs):
|
|
||||||
t = torch.tensor(Xte[i:i+p1_bs], device=dev)
|
|
||||||
h = enc(t)[:, -1, :]
|
|
||||||
chunks.append(((h - mu_e_t) / sd_e_t).cpu().numpy())
|
|
||||||
Ete_t = torch.tensor(np.concatenate(chunks), device=dev)
|
|
||||||
|
|
||||||
head.eval()
|
|
||||||
with torch.no_grad():
|
|
||||||
pred_h_z = head(Ete_t).cpu().numpy()
|
|
||||||
|
|
||||||
pred_h = pred_h_z * ytr_sd + ytr_mu # de-standardise
|
|
||||||
phase1_r2 = float(1 - ((yte - pred_h) ** 2).sum() / ss_tot)
|
|
||||||
print("phase1_r2 = %.4f (n_test=%d)" % (phase1_r2, len(yte)))
|
|
||||||
|
|
||||||
# ── VaR EVAL BLOCK — do NOT edit (agent boundary) ───────────────────────
|
|
||||||
import sys as _sys
|
|
||||||
_sys.path.insert(0, _os.path.dirname(_os.path.abspath(__file__)))
|
|
||||||
from scripts.var_breach import var_breach_rate as _var_breach_rate, METRIC_KEY as _VAR_KEY
|
|
||||||
_var_rate, _kupiec_p = _var_breach_rate(pred_np.tolist(), yte.tolist())
|
|
||||||
print("%s=%.4f Kupiec_p=%.4f" % (_VAR_KEY, _var_rate, _kupiec_p))
|
|
||||||
# ── END VaR EVAL BLOCK ───────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_metrics_out = _os.environ.get("METRICS_OUT", "metrics.json")
|
|
||||||
json.dump({
|
|
||||||
"val_vol_r2": val_vol_r2, "phase1_r2": phase1_r2, "n_test": len(yte),
|
|
||||||
_VAR_KEY: _var_rate, "kupiec_p": _kupiec_p,
|
|
||||||
"knobs": {"WINDOW": WINDOW, "PATCH_LEN": PATCH_LEN,
|
|
||||||
"D_MODEL": D_MODEL, "DEPTH": DEPTH, "ALPHA": ALPHA,
|
|
||||||
"DELTA_T_MAX": DELTA_T_MAX, "EPOCHS": EPOCHS},
|
|
||||||
}, open(_metrics_out, "w"), indent=2)
|
|
||||||
print("val_vol_r2 = %.4f (n_test=%d, dev=%s)" % (val_vol_r2, len(yte), dev))
|
|
||||||
|
|
||||||
# ── EXPORT BLOCK — do NOT edit (agent boundary) ──────────────────────────
|
|
||||||
# Set EXPORT_EMBEDDINGS=1 to write embeddings.json for the Go eval harness.
|
|
||||||
import os
|
|
||||||
if os.environ.get("EXPORT_EMBEDDINGS") == "1":
|
|
||||||
hourly_path2 = "data/processed/eurusd_hourly.parquet"
|
|
||||||
daily_path2 = "data/processed/eurusd_daily.parquet"
|
|
||||||
if USE_HOURLY and os.path.exists(hourly_path2):
|
|
||||||
df2 = pd.read_parquet(hourly_path2).reset_index(drop=True)
|
|
||||||
df2["date"] = pd.to_datetime(df2["datetime"])
|
|
||||||
else:
|
|
||||||
df2 = pd.read_parquet(daily_path2).reset_index(drop=True)
|
|
||||||
df2["date"] = pd.to_datetime(df2["date"])
|
|
||||||
tr_mask = df2["date"].dt.year <= 2021
|
|
||||||
base2 = ["ret", "realized_vol"]
|
|
||||||
extra2 = [c for c in ["hl_range", "ret_intrabar"] if c in df2.columns]
|
|
||||||
feats2 = df2[base2 + extra2].to_numpy(np.float32)
|
|
||||||
mu2 = feats2[tr_mask].mean(0); sd2 = feats2[tr_mask].std(0) + 1e-8
|
|
||||||
fn2 = (feats2 - mu2) / sd2
|
|
||||||
def _export_windows(year_mask):
|
|
||||||
idx = df2.index[year_mask].tolist()
|
|
||||||
Xs, dates, rvs = [], [], []
|
|
||||||
for t in idx:
|
|
||||||
if t - WINDOW >= 0 and t + 1 < len(df2):
|
|
||||||
Xs.append(fn2[t - WINDOW:t])
|
|
||||||
dates.append(str(df2["date"].iloc[t].date()))
|
|
||||||
rvs.append(float(df2["realized_vol"].iloc[t + 1]))
|
|
||||||
if not Xs:
|
|
||||||
return [], [], []
|
|
||||||
Xa = np.stack(Xs)
|
|
||||||
chunks = []
|
|
||||||
with torch.no_grad():
|
|
||||||
for i in range(0, len(Xa), bs):
|
|
||||||
chunks.append(enc(torch.tensor(Xa[i:i+bs], device=dev))[:, -1, :].cpu().numpy())
|
|
||||||
E = np.concatenate(chunks, axis=0).tolist()
|
|
||||||
return E, dates, rvs
|
|
||||||
Etr2, dates_tr, rv_tr = _export_windows(tr_mask)
|
|
||||||
Eoos, dates_oos, rv_oos = _export_windows(df2["date"].dt.year >= 2022)
|
|
||||||
hv_thr = float(np.percentile(rv_oos, 67))
|
|
||||||
hv_label = [1 if v >= hv_thr else 0 for v in rv_oos]
|
|
||||||
json.dump({"embeddings": Eoos, "dates": dates_oos,
|
|
||||||
"realized_vol": rv_oos, "hv_label": hv_label,
|
|
||||||
"train_embeddings": Etr2, "train_realized_vol": rv_tr},
|
|
||||||
open("embeddings.json", "w"))
|
|
||||||
print("exported embeddings.json train=%d oos=%d HV=%d/%d" % (
|
|
||||||
len(Etr2), len(Eoos), sum(hv_label), len(hv_label)))
|
|
||||||
# ── END EXPORT BLOCK ─────────────────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
Reference in New Issue
Block a user