jepa-fx-risk/AGENTS.md

# Agent context — Mathias workspace

<!-- Canonical root context for all AI coding agents.
     Lives at: ~/dev/.context/AGENT.md
     Applies to every project under ~/dev/ unless overridden.
     
     Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
     Project-level context in .context/PROJECT.md layers on top of this. -->

## Who I am

I'm Mathias, a digital product manager and technology consultant based in Sweden.
I build software, research emerging tech, and deliver consulting engagements
for clients under NDA. I work across AI/ML, financial automation, web applications,
and climate/sustainability tech.

## How I work with agents

- I think like a product manager — I care about *why* before *how*
- I want agents to be opinionated and push back, not just execute blindly
- I prefer concise responses; skip ceremony and get to the point
- When I say "build this", I mean production-quality with tests, not a demo
- Ask me before making irreversible changes or adding heavy dependencies
- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK

## Behavior rules

These rules apply to every task across every project, regardless of harness.

0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
   - **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
   - **Load the relevant skill** — see trigger table in *Engineering Skills* below.
   - **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
   - **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?

   **TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.

1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
   Think before coding; if the problem is unclear, ask or state assumptions before acting.
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
   speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
   files, and formatting alone. Diffs should be small and reviewable.
4. **Goal-driven execution.** Define clear success criteria up front for every task.
   Loop — implement, verify, refine — until those criteria are met. Don't claim
   completion without evidence (tests pass, command output, observed behavior).
5. **Trunk-Based Development — commit directly to main.** Every commit is one
   logical change (one tool, one fix, one test) with passing tests. Main is always
   deployable. Never create long-lived feature branches.

   **The PR trigger is RISK, not concurrency.** `risk/low` work commits straight to
   main with CI as the gate — that is most work and is unchanged. `risk/medium` and
   above opens a PR, even solo, even with nobody waiting to review it. This mirrors
   the tiered merge model in `assessor-loop`
   `method/transfer-to-a-regulated-organisation.md` §4.1, so the estate practises
   what it sells instead of merging everything on green and claiming otherwise.

   **Concurrency is NOT a reason to branch.** Measured 2026-08-29 across three real
   collisions between parallel sessions: two were not branching problems at all — a
   generated artifact written to the same path by both (branching turns that into a
   merge conflict on a 33k-line JSON), and uncommitted work in a shared checkout
   (a per-session worktree fixes it) — and the third was an ordinary non-fast-forward
   push resolved by rebase in seconds. Use a worktree per session on shared repos and
   timestamp generated artifacts. Do not reach for a branch.

   Branch+PR flow carries documented costs on solo repos: merging auto-closes gated
   issues, review comments live on a thread separate from the issue's, and a
   merged-and-deleted branch cannot be amended (brain:
   `gitea-pr-merge-autoclose-and-review-comment-footguns`). Those are the price of
   the `risk/medium`+ lane — paid deliberately, not incurred by accident.

   **Exception — external contributor or client four-eyes requirement:** Use PR flow
   when a human reviewer outside the project is required. Document the reason in
   PROJECT.md.

6. **Close the loop — every substantive task ends with the same ritual.** Shipping
   the code is not the end of the task; capturing it is. Run this unprompted:
   - **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
     new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
     actual last tag — stated versions in docs drift stale.
   - **Push** main and the tag (CI is the gate).
   - **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
     the reusable patterns and the footguns that would bite anyone again, never
     project status. See *Knowledge base — when to write* below.
   - **File discovered-but-deferred work as tracker issues** on the project's own
     repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
     "out of scope, recorded" rot in a commit message; make it a ticket with a
     source pointer.
   - Surface the brain entries and issue numbers in the closing summary so the
     trail is auditable.

   **Chat is not a durable store. Never rely on Mathias remembering anything.**
   Every gap, caveat, risk, open question and next step must land in one of four
   places before the turn ends: **the brain**, **a repo** (code, docs, ADR),
   **a tracker issue**, or **a plan file**. A finding stated only in conversation is
   lost the moment the session closes — and this estate has repeatedly proved that a
   remembered convention is an unreliable control.

   This applies hardest to what is tempting to leave as a closing remark: "worth
   knowing X isn't enforced yet", "this is a convention, not a gate", "we should
   revisit Y". Those are exactly the items that need a locator. If it is worth
   saying in a summary, it is worth writing where it can be found again without you.

## CAD sprint patterns

Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.

Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.

## Default stack

| Layer | Default | Fallback | Last resort |
|-------|---------|----------|-------------|
| Language | Go | Python | TypeScript, Java, C |
| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
| Build | Task (taskfile.dev) | Make | — |
| Containers | Docker Compose (dev), k3s (prod) | — | — |
| DB | PostgreSQL + sqlc | SQLite | — |
| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
| Logging | slog (structured) | stdlib `logging` w/ structured `extra=` (or structlog) | — |
| Testing | Table-driven, testify | — | — |
| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |

Exploratory: Rust, Zig — I'll tell you when I want these.

## Code conventions

- **Go style**: golines, gofumpt, golangci-lint
- **Python style** (fallback language): ruff (format+lint, one tool), mypy --strict (non-negotiable,
  matches Go's static typing discipline), pytest + pytest-cov (table-driven via
  `@pytest.mark.parametrize`), uv (venv+deps+lock, one tool), pydantic-settings (typed env-var config
  — same principle as Go's typed structs), src-layout + `pyproject.toml` only (no `setup.py`)
- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return.
  Python: `raise X from e` (exception chaining, same principle) — never bare `except`, never silent `pass`
- **Naming**: stdlib conventions, no stuttering
- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
  one logical change per commit, CI is the quality gate
- **Never**: long-lived feature branches, PRs for solo work, direct push without
  passing `task check` locally first
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message

## Secret handling (every harness, every command)

Tool output is persisted: terminal → `~/.claude/projects` transcripts →
claudewatcher → brain/wiki → gitea history. A secret printed once is
searchable forever, and clearing it means rotating the key. So:

1. **Never print, echo, log, or transform a secret to inspect it.** No
   `base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
   to defeat `op run`'s output masking (it masks raw values; base64 hides them
   from the mask — that exact trick leaked a key on 2026-06-11).
2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
   *inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
   in a command's argv (it lands in the tool call and the transcript).
3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
   never `${X:-...}` (returns the value when set) and never echo a substring of it.
4. **Cross-host secrets:** run the secret-consuming command on the host that has
   the secret; do not forward a raw key over ssh argv/stdout.
5. If a secret does leak into output, say so immediately and flag it for rotation —
   don't bury it.

## Infrastructure

Three machines on Tailscale:

| Machine | Role | Key specs |
|---------|------|-----------|
| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
| iguana | Services, builds | M2 Ultra Mac |
| flamingo | Daily driver, edge | Apple Silicon Mac, ~/dev is here |

- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
- **Orchestration**: k3s on koala only — single control-plane node, no workers (verified 2026-08-30). iguana and flamingo are on the tailnet but are not cluster members
- **Networking**: Tailscale mesh

## Project landscape

All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).

Organized in thematic folders:

| Folder | Focus | Count |
|--------|-------|-------|
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |

See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.

### Key active projects

- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform

## Knowledge base — actively use it

A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
reference material — query it actively, not just when explicitly told.**

### When to query (treat as a reflex)

- **Before** starting a non-trivial task — search for prior art with the symptom
  AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
- **When debugging** — search for the error string, the stack frame, the affected
  service. Past you may have already paid this tax.
- **Before adopting** a pattern, library, framework, or model name — check if it
  was tried and rejected, or what the integration footguns are.
- **When making architectural decisions** — search for the domain + "ADR" or
  "decision" to find prior reasoning before re-deriving it.
- **When a recommendation feels novel** — challenge yourself: "has this been
  documented?" The brain often has it.

### When to write

After you discover something that **future-you would forget** and that **isn't
recoverable from the code, git log, or PR description alone**:

- Bugs whose root cause is non-obvious and generalisable beyond this project.
- Framework / library / model-name quirks that bit you and would bite anyone.
- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.

DON'T write project status, sprint progress, PR summaries, or "what I did this
session" — those rot fast and the originals are in git/gitea anyway. Brain
entries that age well are about *why*, *how to avoid*, and *what to do when*.

### How to access (per harness)

| Harness | Query | Write |
|---------|-------|-------|
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |

- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
  fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
  on the koala k3s cluster; don't hardcode local-only model names into the
  berget URL (see knowledge entry on namespace mismatches).

### Quick reflex checks

If you find yourself about to say any of these out loud, you owe yourself a brain query first:

- "I think the issue might be..."
- "Let me try X and see..."
- "I'll just write a script to..."
- "This is probably a new bug..."
- "Has anyone done this before?" — *yes, probably, go check.*

## Client work rules

When working on a project tagged with a client name:
1. Never send code, data, or context to cloud APIs — use local models only
2. Never reference other client projects or their data
3. Keep all artifacts within the client's git org / directory
4. Treat everything as confidential unless told otherwise

## Harness-agnostic principles

Supported harnesses (2026-09-01) — Cursor and Aider were dropped; do not
re-add adapters for them:
- Claude Code, Antigravity, OpenCode, Mistral Vibe, Crush, Pi Coding Agent
- Open WebUI, Charmbracelet Mods, or any tool that takes a system prompt string
- Any other tool that reads `AGENTS.md`, which is the de facto convention

The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.

## How context propagates

Canonical sources of truth:
- Universal: `~/dev/.context/AGENT.md` (this file)
- Project: `<repo>/.context/PROJECT.md` (per-repo)

Derived files (committed, regenerated by `task context:sync`):
- `CLAUDE.md`, `AGENTS.md`, `.context/system-prompt.txt`

Repos still carrying `.cursorrules`, `.aider.conventions.md` or
`.aider.conf.yml` are stale: nothing regenerates them. Delete them the next time
you touch the repo, along with their entries in that repo's `task check`
assertion.

Workflow:
1. Edit a canonical file. Run `task context:sync`. Commit canonical and
   derived together. Push.
2. On any other host, `git pull` brings both. Which file a harness reads does
   NOT split cleanly by lookup strategy, so do not reason from the strategy:
   - `CLAUDE.md` — Claude Code (walks up the tree)
   - `AGENTS.md` — Crush, Pi, Antigravity (cwd-only), OpenCode (project root,
     global at `~/.config/opencode/AGENTS.md`), and Mistral Vibe, which **walks
     up** from cwd and takes the first `AGENTS.md` it finds
     (user-level `~/.vibe/AGENTS.md`)
   - `.context/system-prompt.txt` — Open WebUI, Mods, raw API calls; not read by
     any harness, it is for pasting

   Vibe tree-walks for `AGENTS.md`, so "tree-walkers use CLAUDE.md" is false.
   Each repo's `AGENTS.md` embeds the canonical and is the first one found from
   inside that repo, so this works — but by construction, not by luck.
3. `task check` runs `context:sync` then asserts `git status --porcelain`
   is empty over the derived files (catches both modified-tracked drift
   and missing-untracked adapters). A drift fails the check with a
   message telling you to stage the regenerated files.

Behavior rules in this file and per-project rules in `PROJECT.md` apply
unconditionally on every host, every harness.

## Engineering Skills

Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Load at task start — not "on demand" but on schedule, before writing code. Browse `SKILLS_INDEX.md` for the full list.

**Claude Code** installs the repo as a plugin from the marketplace the same repo hosts. That path also carries the `secret-guard` and `pre-task-ritual` hooks and the brain/gitea/infra MCP servers, which symlinks cannot:

```
/plugin marketplace add https://git.d-ma.be/mathias/skills.git
/plugin install skills@mathias
```

Skills are **namespaced** under Claude Code: load `tdd` as `skills:tdd`. The trigger table below uses bare names — prefix them.

**Crush, Pi and OpenCode** have no plugin system and are wired by `install.sh` as one directory symlink per harness via `~/.agents/skills`. Do not clone the checkout into `~/dev`: that folder is iCloud-shared between the Macs and spans two path roots (`/Users/mathias` vs `/home/mathias` on koala), so a checkout there is a git working tree with two concurrent writers, and any absolute symlink stored in it is broken on one side by construction (skills#23). Leave `SKILLS_CHECKOUT_DIR` at its host-local default.

Check a host with `task doctor` in the skills repo; it also runs automatically at session start via the plugin.

**Skill trigger table — load before starting, not after getting stuck:**

| Task type | Load |
|-----------|------|
| Any feature or bug fix | `tdd` |
| Refactor or design | `clean-code` or `solid` |
| Debug | `problem-analysis` |
| Review code or PRs | `code-review` |
| Frame a problem before coding | `problem-analysis` |

---

# jepa-fx-risk

## Identity

- **Name**: jepa-fx-risk
- **Owner**: Mathias
- **Client**: personal research
- **Repo**: gitea.d-ma.be/mathias/jepa-fx-risk
- **Status**: active — Phase 0

## Purpose

Research project: apply JEPA-based self-supervised representation learning to
FX risk management for a corporate bank with an internal global FX trading desk.

Primary tasks: FX volatility forecasting and VaR/CVaR estimation.
Target output: internal PoC for the trading desk.

## Architecture decision (see DECISIONS.md ADR-001)

**TS-JEPA + SIGReg** — TS-JEPA temporal patchwise architecture (Ennadir et al.,
arXiv:2509.25449) with EMA replaced by Sketched Isotropic Gaussian Regularization
(SIGReg, Balestriero & LeCun, arXiv:2511.08544). Single search axis: λ ∈ [0.01, 1.0].

Phase 2 hypothesis: MTS-JEPA multi-resolution objective (arXiv:2602.04643).

## Stack

**Go** (`src/`): data pipeline (DUKASCopy fetch + hourly processing), evaluation
harness (silhouette, linear probe R², collapse diagnostic, Kupiec/Christoffersen),
results dashboard (Templ + HTMX + CDN Tailwind).

**Python** (`model/`): all training and embedding export only. PyTorch cu130
(Blackwell sm_120 compatible). No evaluation logic in Python.

**Infra**: koala (Arch Linux, Blackwell GPU 12 GB VRAM) for training.
iguana (Mac Studio M2 Ultra) + LiteLLM on piguard for autoresearch agent LLM.

## Repository layout

```
jepa-fx-risk/
├── src/                    # Go — data pipeline + eval harness + dashboard
│   ├── data/               # DUKASCopy fetch, hourly processing, validation
│   └── eval/               # silhouette, linear probe, collapse, backtest
├── model/                  # Python — training only
│   ├── train.py            # TS-JEPA + SIGReg backbone (autoresearch edits this)
│   ├── prepare.py          # LOCKED — data loading, tokenization, export
│   └── requirements.txt
├── specs/                  # Research specs (one per phase/experiment type)
├── experiments/            # Per-run outputs: embeddings, metrics.json, git tag
├── results/summaries/      # Human-readable outcome per experiment
├── program.md              # Autoresearch agenda — researcher edits this
├── DECISIONS.md            # Architecture Decision Records
└── Taskfile.yml            # task data:fetch, task experiment:run, task eval:*
```

## Phase structure

- **Phase 0** (current): MAE baseline on EUR/USD hourly 2008–2022.
  Gate: silhouette > 0.20, MAE > PCA, ±10% over 3 reruns.
- **Phase 1**: TS-JEPA + SIGReg autoresearch sweep, 50 experiments.
  Gate: val_vol_r2 > GARCH baseline AND Kupiec p > 0.05.
- **Phase 2**: MTS-JEPA multi-resolution hypothesis.
- **Phase 3**: Internal bank tick/position data (future, out of current scope).

## Data

DUKASCopy hourly OHLCV, 10 G10 pairs, 2008–2022 train / 2023 val / 2024 test.
Features: log-return, log rolling-20-period HV, VIX (daily interpolated).
Weekend gaps handled explicitly. See ADR-002.

## Key conventions

- `prepare.py` is LOCKED — never modified by agents or autoresearch
- Evaluation metrics are always computed by the Go harness, never in Python
- Every experiment gets a git tag: `exp/YYYYMMDD-description`
- Null results are recorded explicitly in `results/summaries/` — do not iterate silently
- `program.md` is the only file the researcher edits to steer autoresearch
- CI runs `task check` (lint + vet + test) only — no GPU, no training

## Evaluation metrics

Primary (autoresearch optimizes): `val_vol_r2` — linear probe R² on 1-day
realized volatility from frozen embeddings, computed by Go harness.

Secondary (logged, not optimized): Kupiec p-value (VaR 99% backtest on EUR/USD).
Must co-move with val_vol_r2 — checked from experiment 1.

Diagnostics: silhouette score (regime clustering), PC1/HV correlation (collapse check).

## Benchmarks to beat (trading desk comparison)

- GARCH(1,1) — volatility forecasting baseline
- Historical Simulation VaR (250-day rolling) — Basel default
- EWMA RiskMetrics (λ=0.94)

## Agent guidance

Read `DECISIONS.md` before making architecture suggestions.
Do not modify `prepare.py` or `src/eval/`.
Do not suggest changing the Python/Go separation.
Training runs are always manual via `task experiment:run` — never triggered by CI.
When implementing, follow Go conventions in `.skills/go-patterns/SKILL.md`.