A one-command path from an Autoresearch Council backlog leaf (agentsquad #42/#44, the autoresearch-ready nodes with a single candidate_metric) to a running, monitored Karpathy-style autoresearch loop — and the ability to watch it on two axes while it runs: research progress (is the metric actually moving?) and technical health (is the loop alive, fast, and not on fire?).
Closes the last gap in the pipeline: Council decomposes → guards → emits a backlog; today a human re-keys a leaf into a program.md by hand. This makes the handoff mechanical and observable.
Background / chain of custody
strategic question
└─ 🔬 Autoresearch Council (agentsquad) → backlog JSON (#44):
nodes[] with status=autoresearch-ready, candidate_metric, depends_on
└─ THIS ISSUE: pick a ready leaf → scaffold program.md → launch loop
└─ Karpathy loop: agent edits train.py → 5-min experiment →
check single metric → keep if improved, else revert (brain: wiki/jepa-fx/facts/autoresearch-integration-phase1)
Every run must carry provenance back to the council node id + the strategic question (so a result traces to the question that spawned it).
Read the council backlog JSON, select node rq-id (must be status: autoresearch-ready; refuse otherwise — fail-closed).
Scaffold a run dir: program.md (hypothesis = the node question; single metric = candidate_metric; the ablation it implies; baseline to beat), a fresh train.py from the project template, and a run.yaml (model tier, max iters, time-box, provenance: {strategic_question, council_node, generated_at}).
One worked seed lands with this issue: rq-04 (regime-conditioned JEPA, metric VaR_breach_rate_99_oos_regime_cond).
2. Launch
Kick the loop: agent (LiteLLM, homelab models — iguana/koala) edits train.py, runs the time-boxed experiment, reads the single metric, keeps-or-reverts. Reuse the existing loop shape; this issue is the entry point + wiring, not a new loop.
3. Monitor — research/task axis ("is it learning?")
The single metric trajectory over iterations (best-so-far + per-iter delta), plateau/regression detection.
A per-iteration record of what the agent changed and why it kept/reverted (the research narrative, not just numbers).
Surfaced as a live STATUS.md (cheap, always works) and/or a small dashboard. Answer at a glance: improving / plateaued / diverging.
infra #92 (Langfuse — agent observability), Prometheus/Grafana (technical axis)
## Goal
A one-command path from an **Autoresearch Council backlog leaf** (agentsquad #42/#44, the `autoresearch-ready` nodes with a single `candidate_metric`) to a **running, monitored Karpathy-style autoresearch loop** — and the ability to watch it on two axes while it runs: **research progress** (is the metric actually moving?) and **technical health** (is the loop alive, fast, and not on fire?).
Closes the last gap in the pipeline: Council decomposes → guards → emits a backlog; today a human re-keys a leaf into a `program.md` by hand. This makes the handoff mechanical and observable.
## Background / chain of custody
```
strategic question
└─ 🔬 Autoresearch Council (agentsquad) → backlog JSON (#44):
nodes[] with status=autoresearch-ready, candidate_metric, depends_on
└─ THIS ISSUE: pick a ready leaf → scaffold program.md → launch loop
└─ Karpathy loop: agent edits train.py → 5-min experiment →
check single metric → keep if improved, else revert (brain: wiki/jepa-fx/facts/autoresearch-integration-phase1)
```
Every run must carry provenance back to the council node id + the strategic question (so a result traces to the question that spawned it).
## Components
### 1. Ingest + scaffold (`autoresearch start <backlog.json> <rq-id>`)
- Read the council backlog JSON, select node `rq-id` (must be `status: autoresearch-ready`; refuse otherwise — fail-closed).
- Scaffold a run dir: `program.md` (hypothesis = the node question; **single metric = `candidate_metric`**; the ablation it implies; baseline to beat), a fresh `train.py` from the project template, and a `run.yaml` (model tier, max iters, time-box, provenance: `{strategic_question, council_node, generated_at}`).
- One worked seed lands with this issue: **rq-04** (regime-conditioned JEPA, metric `VaR_breach_rate_99_oos_regime_cond`).
### 2. Launch
- Kick the loop: agent (LiteLLM, homelab models — iguana/koala) edits `train.py`, runs the time-boxed experiment, reads the single metric, keeps-or-reverts. Reuse the existing loop shape; this issue is the entry point + wiring, not a new loop.
### 3. Monitor — **research/task axis** ("is it learning?")
- The single metric **trajectory over iterations** (best-so-far + per-iter delta), plateau/regression detection.
- A per-iteration record of *what the agent changed* and *why it kept/reverted* (the research narrative, not just numbers).
- Surfaced as a live `STATUS.md` (cheap, always works) and/or a small dashboard. Answer at a glance: improving / plateaued / diverging.
### 4. Monitor — **technical axis** ("is it healthy?")
- Loop liveness: iterations/hr, current-experiment runtime, last-heartbeat.
- Resource: iguana/koala GPU + VRAM + RAM during experiments, LiteLLM latency + token cost per iter.
- Failure surfacing: crashed experiments, OOM, model timeouts, revert-storms (agent thrashing).
- Wire into existing homelab observability — Prometheus/Grafana, and Langfuse for the agent trace (infra#92). ntfy alert on stall/crash.
## Acceptance criteria
- [ ] `autoresearch start <backlog.json> <rq-id>` scaffolds a runnable program.md + run dir from a ready leaf; refuses non-ready nodes
- [ ] Provenance (strategic_question + council_node) recorded in the run and visible in monitoring
- [ ] Launch starts the loop against homelab models; one real iteration completes
- [ ] Research axis: metric trajectory + best-so-far + per-iter change-log visible live (STATUS.md at minimum)
- [ ] Technical axis: loop liveness + iguana/koala resource + LiteLLM cost/latency visible; stall/crash alerts
- [ ] Seed run for **rq-04** scaffolded and launchable
**Risk:** MEDIUM
## Out of scope
- Auto-selecting *which* leaf to run (human picks for now)
- Multi-run scheduling / queueing across leaves
- Promoting `needs-human-framing` nodes (still a human metric-decision)
## Refs
- agentsquad #42 (Autoresearch Council), #44 (backlog contract — the input), #45 (guard)
- brain `wiki/jepa-fx/facts/autoresearch-integration-phase1` (loop architecture), `wiki/glossary/val-vol-r2` (metric discipline), `knowledge/pattern-autoresearch-karpathy`
- infra #92 (Langfuse — agent observability), Prometheus/Grafana (technical axis)
Ready to commit into the run dir once the launcher (this issue) exists. Derived directly from the Autoresearch Council backlog leaf rq-04.
# program.md — Regime-conditioned JEPA for FX tail risk
## Provenance
- strategic_question: "What is the highest-leverage path to a JEPA-based FX
tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach
calibration, given one GPU and a solo researcher?"
- council_node: rq-04 (autoresearch-ready; agentsquad #42/#44)
## Hypothesis
Conditioning the JEPA predictor on latent tail-state indicators (regime flags
from an HMM / Markov-switching GARCH) reduces the 99% VaR-breach rate on
out-of-sample EUR/USD during crisis windows (2008-Q4, 2020-March) by >=20%
relative to an otherwise-identical unconditioned JEPA.
## Single validation metric (optimise this, nothing else)
`VaR_breach_rate_99_oos_regime_cond` = fraction of 1-day 99% VaR breaches on the
two held-out crisis windows. LOWER is better. Report alongside (do not optimise):
Kupiec POF p-value (calibration sanity) and val_vol_r2 (representation quality),
to catch breach-rate gains that wreck calibration elsewhere.
## Baseline (frozen reference)
- Unconditioned JEPA (identical backbone, no regime input) — the head-to-head.
- EWMA(0.94) 99% VaR — the floor the council question demands we beat.
## What the agent MAY modify (the search space)
- How the regime flag enters the model: concat to predictor input vs FiLM
conditioning vs a learned regime embedding.
- Regime detector: number of states, features fed to the HMM/MS-GARCH, lookback.
- Predictor head capacity for the conditioned path.
## Frozen (do NOT touch — keeps the ablation clean)
- TS-JEPA backbone architecture + SIGReg objective.
- Train/OOS splits and the two crisis evaluation windows.
- The metric definition and the breach-counting code.
## Experiment loop (per Karpathy autoresearch)
Each iter (<= ~5 min): apply one change to the conditioning path → train → compute
`VaR_breach_rate_99_oos_regime_cond` → keep if it improved AND Kupiec p-value did
not collapse, else revert. Stop on: >=20% relative reduction reached, or N iters,
or a plateau of K iters with no improvement.
## Success criterion
>=20% relative reduction in `VaR_breach_rate_99_oos_regime_cond` vs the
unconditioned JEPA baseline on BOTH crisis windows, without degrading Kupiec POF
calibration on the non-crisis OOS period.
Notes:
The metric is breach-rate (not val_vol_r2) because rq-04 is specifically a tail-calibration claim under regime shift — the council tagged it that way. The improved stage-1 prompt (agentsquad v0.21.2) now also surfaces several val_vol_r2-metric ready leaves on the same question, which are the natural representation-quality companions to run after this calibration probe.
## Seed `program.md` — rq-04 (first autoresearch case)
Ready to commit into the run dir once the launcher (this issue) exists. Derived directly from the Autoresearch Council backlog leaf `rq-04`.
```markdown
# program.md — Regime-conditioned JEPA for FX tail risk
## Provenance
- strategic_question: "What is the highest-leverage path to a JEPA-based FX
tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach
calibration, given one GPU and a solo researcher?"
- council_node: rq-04 (autoresearch-ready; agentsquad #42/#44)
## Hypothesis
Conditioning the JEPA predictor on latent tail-state indicators (regime flags
from an HMM / Markov-switching GARCH) reduces the 99% VaR-breach rate on
out-of-sample EUR/USD during crisis windows (2008-Q4, 2020-March) by >=20%
relative to an otherwise-identical unconditioned JEPA.
## Single validation metric (optimise this, nothing else)
`VaR_breach_rate_99_oos_regime_cond` = fraction of 1-day 99% VaR breaches on the
two held-out crisis windows. LOWER is better. Report alongside (do not optimise):
Kupiec POF p-value (calibration sanity) and val_vol_r2 (representation quality),
to catch breach-rate gains that wreck calibration elsewhere.
## Baseline (frozen reference)
- Unconditioned JEPA (identical backbone, no regime input) — the head-to-head.
- EWMA(0.94) 99% VaR — the floor the council question demands we beat.
## What the agent MAY modify (the search space)
- How the regime flag enters the model: concat to predictor input vs FiLM
conditioning vs a learned regime embedding.
- Regime detector: number of states, features fed to the HMM/MS-GARCH, lookback.
- Predictor head capacity for the conditioned path.
## Frozen (do NOT touch — keeps the ablation clean)
- TS-JEPA backbone architecture + SIGReg objective.
- Train/OOS splits and the two crisis evaluation windows.
- The metric definition and the breach-counting code.
## Experiment loop (per Karpathy autoresearch)
Each iter (<= ~5 min): apply one change to the conditioning path → train → compute
`VaR_breach_rate_99_oos_regime_cond` → keep if it improved AND Kupiec p-value did
not collapse, else revert. Stop on: >=20% relative reduction reached, or N iters,
or a plateau of K iters with no improvement.
## Success criterion
>=20% relative reduction in `VaR_breach_rate_99_oos_regime_cond` vs the
unconditioned JEPA baseline on BOTH crisis windows, without degrading Kupiec POF
calibration on the non-crisis OOS period.
```
Notes:
- The metric is breach-rate (not val_vol_r2) because rq-04 is specifically a tail-calibration claim under regime shift — the council tagged it that way. The improved stage-1 prompt (agentsquad v0.21.2) now also surfaces several `val_vol_r2`-metric ready leaves on the same question, which are the natural *representation-quality* companions to run after this calibration probe.
Minimum vertical slice — get something executing + monitored, then deepen
The full path (foundation #2/#3/#5/#4 → loop → rq-04 extras) is the whole project. To de-risk it, prove the chain end-to-end with a toy first, so "council leaf → running, monitored experiment" is real before TS-JEPA exists. Three phases:
Phase A — toy slice (the point: the loop runs + you can watch it)
Deliberately tiny; throwaway-able. Reuse what's already in the repo (internal/web UI, cmd/hostexecutor).
Data (min): one pair (EUR/USD), daily, a few years incl. one OOS slice. A thin slice of #2, not the full DUKASCopy pipeline.
train.py (toy): a small encoder (NOT full TS-JEPA) that trains in <2 min on koala and exports embeddings. Just enough to move a metric.
Metric (min):val_vol_r2 linear probe only — a thin slice of #4 (the metric the harness is designed for; defer VaR-breach to rq-04).
Loop runner: edit train.py → run (time-boxed) → read the scalar → keep-if-improved-else-revert → append to STATUS.md. The Karpathy loop, minimal.
Monitor (both axes, minimal):
research → STATUS.md + the existing web UI: metric trajectory, best-so-far, per-iter change-log.
technical → loop heartbeat + koala GPU/VRAM (nvidia-smi scrape) + LiteLLM latency; ntfy on stall/crash. (Prometheus/Langfuse wiring comes later.)
Done when: the loop runs 3 real iterations unattended, the metric visibly moves, and you can see both axes live.
Phase B — make it real
Swap the toy for the foundation, slice by slice: #2 full data pipeline, #3 TS-JEPA + SIGReg backbone, #5 Phase-0 SSL feasibility gate (must pass before trusting the loop), #4 full eval harness, #10 DVC+MinIO for experiment artifacts/provenance. The loop + monitoring from Phase A stay; only the substrate deepens.
Phase C — rq-04
On the real substrate: regime detector + conditioning hook (issue above) and the VaR_breach_rate_99 metric (issue above). Then autoresearch start <backlog.json> rq-04 scaffolds the program.md (seeded above) and launches. This is when the juicy council leaf actually runs.
Why this order
The first runnable experiment is a val_vol_r2 case (matches the built-for harness), NOT rq-04 — rq-04 needs a new metric + regime model on top.
Phase A surfaces the integration bugs (loop control, time-boxing, metric plumbing, monitoring) on a 2-minute toy instead of a 10-minute TS-JEPA run — cheap iteration on the harness, exactly the lesson from the agentsquad CAD work.
Each phase ends in something that executes, not just more backlog.
Concretely, Phase A is the smallest commit set that makes #11's acceptance ("launch starts the loop; one real iteration completes; both axes visible") true.
## Minimum vertical slice — get *something executing + monitored*, then deepen
The full path (foundation #2/#3/#5/#4 → loop → rq-04 extras) is the whole project. To de-risk it, prove the **chain end-to-end with a toy first**, so "council leaf → running, monitored experiment" is real before TS-JEPA exists. Three phases:
### Phase A — toy slice (the point: the loop runs + you can watch it)
Deliberately tiny; throwaway-able. Reuse what's already in the repo (`internal/web` UI, `cmd/hostexecutor`).
- **Data (min):** one pair (EUR/USD), daily, a few years incl. one OOS slice. A thin slice of #2, not the full DUKASCopy pipeline.
- **`train.py` (toy):** a small encoder (NOT full TS-JEPA) that trains in <2 min on koala and exports embeddings. Just enough to move a metric.
- **Metric (min):** `val_vol_r2` linear probe only — a thin slice of #4 (the metric the harness is designed for; defer VaR-breach to rq-04).
- **Loop runner:** edit `train.py` → run (time-boxed) → read the scalar → keep-if-improved-else-revert → append to `STATUS.md`. The Karpathy loop, minimal.
- **Monitor (both axes, minimal):**
- research → `STATUS.md` + the existing web UI: metric trajectory, best-so-far, per-iter change-log.
- technical → loop heartbeat + koala GPU/VRAM (nvidia-smi scrape) + LiteLLM latency; ntfy on stall/crash. (Prometheus/Langfuse wiring comes later.)
- **Done when:** the loop runs **3 real iterations** unattended, the metric visibly moves, and you can see both axes live.
### Phase B — make it real
Swap the toy for the foundation, slice by slice: #2 full data pipeline, #3 TS-JEPA + SIGReg backbone, #5 Phase-0 SSL feasibility gate (must pass before trusting the loop), #4 full eval harness, #10 DVC+MinIO for experiment artifacts/provenance. The loop + monitoring from Phase A stay; only the substrate deepens.
### Phase C — rq-04
On the real substrate: regime detector + conditioning hook (issue above) and the `VaR_breach_rate_99` metric (issue above). Then `autoresearch start <backlog.json> rq-04` scaffolds the program.md (seeded above) and launches. This is when the *juicy* council leaf actually runs.
### Why this order
- The first runnable experiment is a **`val_vol_r2`** case (matches the built-for harness), NOT rq-04 — rq-04 needs a new metric + regime model on top.
- Phase A surfaces the integration bugs (loop control, time-boxing, metric plumbing, monitoring) on a 2-minute toy instead of a 10-minute TS-JEPA run — cheap iteration on the *harness*, exactly the lesson from the agentsquad CAD work.
- Each phase ends in something that **executes**, not just more backlog.
Concretely, Phase A is the smallest commit set that makes #11's acceptance ("launch starts the loop; one real iteration completes; both axes visible") true.
This issue's scaffolder reads candidate_metric from the Council backlog JSON to wire the single metric into program.md / run.yaml. In the first real run that value came out as " VaR_breach_rate_99_oos_regime_cond" — leading space — because of a Stage-3 serialisation bug (agentsquad#48, bug 2).
When ingesting a backlog leaf, .strip() the candidate_metric defensively before using it as an identifier / metric key — don't assume upstream is clean even after #48 lands (belt-and-braces; the scaffold is the last line before it becomes a file path / lookup key).
The canonical metric name is VaR_breach_rate_99_oos_regime_cond (no leading space). #12 emits the scalar under that exact key; keep them identical or the loop's metric-lookup misses.
agentsquad#48 fixes the source; this note ensures #11 is robust regardless of ingest order.
**Cross-repo dependency — agentsquad#48 (metric whitespace).**
This issue's scaffolder reads `candidate_metric` from the Council backlog JSON to wire the single metric into `program.md` / `run.yaml`. In the first real run that value came out as `" VaR_breach_rate_99_oos_regime_cond"` — **leading space** — because of a Stage-3 serialisation bug (agentsquad#48, bug 2).
Two implications for #11:
1. When ingesting a backlog leaf, **`.strip()` the `candidate_metric`** defensively before using it as an identifier / metric key — don't assume upstream is clean even after #48 lands (belt-and-braces; the scaffold is the last line before it becomes a file path / lookup key).
2. The canonical metric name is `VaR_breach_rate_99_oos_regime_cond` (no leading space). #12 emits the scalar under that exact key; keep them identical or the loop's metric-lookup misses.
agentsquad#48 fixes the source; this note ensures #11 is robust regardless of ingest order.
Reads the Council backlog JSON (e.g. first_run_2026-06-23.json), finds the node by rq-id. Fail-closed: if status != autoresearch-ready, print error and exit non-zero. No partial scaffolding on non-ready nodes.
Scaffolds a run dir (runs/<rq-id>/):
program.md — hypothesis = node question; single metric = candidate_metric.strip() (defensive); ablation implied by the question; baseline to beat
run.yaml — model tier, max_iters, time_box_minutes, provenance: {strategic_question, council_node, generated_at}
train.py — copy from project template (the current train.py)
Provenance is recorded in run.yaml and visible in STATUS.md header.
rq-04 seed
Will scaffold runs/rq-04/ as the worked example once #12's metric key (VaR_breach_rate_99_oos_regime_cond) exists in the eval harness.
Technical axis: loop liveness (iterations/hr, last-heartbeat). Resource/LiteLLM/alert wiring (Prometheus, Langfuse, ntfy) deferred — noted as out of scope for this issue's MVP, tracked as follow-up. Will note in implementation which hooks exist and where the wiring should land.
Dep gate
Blocked on #12 metric existing before rq-04 run can be seeded. Will implement the scaffold command first (testable independently with a mock metric key), then seed rq-04 once #12 lands.
## Implementation plan
### `autoresearch start <backlog.json> <rq-id>` command
Reads the Council backlog JSON (e.g. `first_run_2026-06-23.json`), finds the node by `rq-id`. Fail-closed: if `status != autoresearch-ready`, print error and exit non-zero. No partial scaffolding on non-ready nodes.
Scaffolds a run dir (`runs/<rq-id>/`):
- `program.md` — hypothesis = node question; single metric = `candidate_metric.strip()` (defensive); ablation implied by the question; baseline to beat
- `run.yaml` — model tier, max_iters, time_box_minutes, provenance: `{strategic_question, council_node, generated_at}`
- `train.py` — copy from project template (the current `train.py`)
Provenance is recorded in `run.yaml` and visible in STATUS.md header.
### rq-04 seed
Will scaffold `runs/rq-04/` as the worked example once #12's metric key (`VaR_breach_rate_99_oos_regime_cond`) exists in the eval harness.
### Monitoring stubs
**Research axis (STATUS.md):** metric trajectory (best-so-far + per-iter delta), plateau detection, per-iter change log.
**Technical axis:** loop liveness (iterations/hr, last-heartbeat). Resource/LiteLLM/alert wiring (Prometheus, Langfuse, ntfy) deferred — noted as out of scope for this issue's MVP, tracked as follow-up. Will note in implementation which hooks exist and where the wiring should land.
### Dep gate
Blocked on #12 metric existing before rq-04 run can be seeded. Will implement the scaffold command first (testable independently with a mock metric key), then seed rq-04 once #12 lands.
Decision: Phase A toy slice FIRST — do not go straight to rq-04
Your own earlier analysis (comment above, the three-phase plan) had this right, but the current execution plans across #11/#12/#13 quietly skipped Phase A and are marching straight to rq-04 (Phase C). Pull it back to toy-first.
Build Phase A before the regime detector or the VaR-breach metric:
One pair (EUR/USD), daily, thin OOS slice — not the full data pipeline.
Toy encoder (NOT TS-JEPA/HEPA-full), trains <2 min on koala, exports embeddings.
Metric = val_vol_r2 linear probe only — the metric the harness was built for. Defer VaR-breach to Phase C.
Both monitoring axes, minimal: research (STATUS.md + web UI metric trajectory) and technical (loop heartbeat + nvidia-smi scrape + ntfy on stall).
Done when: loop runs 3 real iterations unattended, metric visibly moves, both axes visible live.
Why toy-first is real learning, not ceremony: the toy teaches nothing about FX or JEPA — it de-risks the harness (loop control, time-boxing, revert logic, metric plumbing, the two monitor axes). Those break on first contact regardless of the science. This is the first autoresearch loop actually running, so the mechanics are unproven and the toy lets them fail on a 2-min run instead of a 10-min TS-JEPA one. Going straight to rq-04 means debugging the harness AND the regime model AND the science simultaneously on slow runs — exactly the trap the agentsquad CAD work taught us to avoid (prove the loop on something trivial before real compute).
Sequencing, concretely:
Now (Phase A): the toy val_vol_r2 loop above. This is what makes #11's acceptance true ("launch starts the loop; one real iteration completes; both axes visible").
#12 and #13 (Phase C prep): the VaR-breach metric and the regime detector are still wanted and the plans are approved (see those issues) — but they're Phase C, built on the real substrate, after the toy proves the harness. Don't block them, but don't seed runs/rq-04/ until Phase A is green.
rq-04 seed: holds until Phase A proves the loop AND #12's metric exists. Your #11 plan already gates the seed on #12 — keep that gate, just add "and Phase A green" in front of it.
The autoresearch start scaffold command itself (comment 1939) is fine to build now — it's testable with a mock metric key independent of phase. Build the command, prove it on the toy, seed rq-04 last.
## Decision: Phase A toy slice FIRST — do not go straight to rq-04
Your own earlier analysis (comment above, the three-phase plan) had this right, but the current execution plans across #11/#12/#13 quietly skipped Phase A and are marching straight to rq-04 (Phase C). Pull it back to toy-first.
**Build Phase A before the regime detector or the VaR-breach metric:**
- One pair (EUR/USD), daily, thin OOS slice — not the full data pipeline.
- Toy encoder (NOT TS-JEPA/HEPA-full), trains <2 min on koala, exports embeddings.
- Metric = `val_vol_r2` linear probe only — the metric the harness was built for. Defer VaR-breach to Phase C.
- Loop runner: edit `train.py` → time-boxed run → read scalar → keep-if-improved-else-revert → append `STATUS.md`.
- Both monitoring axes, minimal: research (STATUS.md + web UI metric trajectory) and technical (loop heartbeat + nvidia-smi scrape + ntfy on stall).
- **Done when:** loop runs 3 real iterations unattended, metric visibly moves, both axes visible live.
**Why toy-first is real learning, not ceremony:** the toy teaches nothing about FX or JEPA — it de-risks the *harness* (loop control, time-boxing, revert logic, metric plumbing, the two monitor axes). Those break on first contact regardless of the science. This is the first autoresearch loop actually running, so the mechanics are unproven and the toy lets them fail on a 2-min run instead of a 10-min TS-JEPA one. Going straight to rq-04 means debugging the harness AND the regime model AND the science simultaneously on slow runs — exactly the trap the agentsquad CAD work taught us to avoid (prove the loop on something trivial before real compute).
**Sequencing, concretely:**
- **Now (Phase A):** the toy `val_vol_r2` loop above. This is what makes #11's acceptance true ("launch starts the loop; one real iteration completes; both axes visible").
- **#12 and #13 (Phase C prep):** the VaR-breach metric and the regime detector are still wanted and the plans are approved (see those issues) — but they're Phase C, built on the real substrate, after the toy proves the harness. Don't block them, but don't seed `runs/rq-04/` until Phase A is green.
- **rq-04 seed:** holds until Phase A proves the loop AND #12's metric exists. Your #11 plan already gates the seed on #12 — keep that gate, just add "and Phase A green" in front of it.
The `autoresearch start` scaffold command itself (comment 1939) is fine to build now — it's testable with a mock metric key independent of phase. Build the command, prove it on the toy, seed rq-04 last.
loop.py --run-dir flag — redirects STATUS.md / metrics.json / HEARTBEAT / train.py into the run dir; METRICS_OUT env var passes to train subprocess
train.py — respects METRICS_OUT env var (default unchanged)
All 54 existing tests still pass
Remaining before rq-04 seed:
Live run (can be triggered now — infra ready)
#12 VaR metric must exist before the rq-04 program.md can name it
Leaving open until 3 live iters confirmed.
## Phase A shipped — v1.4.0
**Acceptance criteria status:**
| Criterion | Status |
|-----------|--------|
| `autoresearch start` scaffolds run dir from ready leaf; refuses non-ready nodes | ✅ 19 TDD tests green |
| `candidate_metric` stripped on ingestion (bypass path fixed) | ✅ test asserts `` `val_vol_r2` `` not `` ` val_vol_r2` `` |
| Provenance (strategic_question + council_node) in run.json | ✅ |
| Research axis: metric trajectory in STATUS.md | ✅ existing (loop already writes this) |
| Technical axis: HEARTBEAT file per iter, ntfy-on-crash via NTFY_URL | ✅ new in loop.py |
| GPU visible in STATUS.md (nvidia-smi per iter) | ✅ existing |
| Live run: 3 real iters unattended | ⏳ needs `LITELLM_KEY` + koala; trigger: `LITELLM_KEY=xxx python loop.py --run-dir runs/<id>` |
**What shipped:**
- `scripts/autoresearch_start.py` — fail-closed scaffold command
- `loop.py --run-dir` flag — redirects STATUS.md / metrics.json / HEARTBEAT / train.py into the run dir; METRICS_OUT env var passes to train subprocess
- `train.py` — respects `METRICS_OUT` env var (default unchanged)
- All 54 existing tests still pass
**Remaining before rq-04 seed:**
- Live run (can be triggered now — infra ready)
- #12 VaR metric must exist before the rq-04 `program.md` can name it
Leaving open until 3 live iters confirmed.
Clean build: fail-closed scaffold (refuses non-ready, 19 tests), both monitor axes, 54 existing tests still green. And you found and fixed the candidate_metric strip bypass — the test asserting val_vol_r2 not ␣val_vol_r2 confirms you located the actual bypass path, not just added a second strip. That was the open question from agentsquad#48; good that it's closed on the consuming side too.
Correctly left open pending the 3 live unattended iterations — that needs LITELLM_KEY + koala and you shouldn't self-certify a run you didn't do. That live run is now the single gate on Phase A acceptance, and Phase A is the gate on everything downstream. Trigger it when ready: LITELLM_KEY=xxx python loop.py --run-dir runs/<id>.
One update to the rq-04 seed gate (from the #12 crisis-window decision): the gate is no longer just "Phase A green AND #12 metric." It's now Phase A green AND #12 metric (done) AND #13 regime (done) AND the crisis-holdout data slice exists (new #2 task). rq-04 can't be honestly evaluated until 2008-Q4/2020-March are in a held-out eval slice — see #12 and #2. The scaffold command and program.md seed can be built before that (with HEPA+VICReg frozen, not TS-JEPA — confirmed on #13), but don't launch the rq-04 run until the crisis slice is real.
So the immediate next action on this issue is just: trigger the live toy run, confirm 3 iters + metric movement + both axes, close Phase A.
## Phase A code — accepted. Two things noted.
Clean build: fail-closed scaffold (refuses non-ready, 19 tests), both monitor axes, 54 existing tests still green. And you **found and fixed the candidate_metric strip bypass** — the test asserting `val_vol_r2` not `␣val_vol_r2` confirms you located the actual bypass path, not just added a second strip. That was the open question from agentsquad#48; good that it's closed on the consuming side too.
Correctly left open pending the **3 live unattended iterations** — that needs `LITELLM_KEY` + koala and you shouldn't self-certify a run you didn't do. That live run is now the single gate on Phase A acceptance, and Phase A is the gate on everything downstream. Trigger it when ready: `LITELLM_KEY=xxx python loop.py --run-dir runs/<id>`.
**One update to the rq-04 seed gate** (from the #12 crisis-window decision): the gate is no longer just "Phase A green AND #12 metric." It's now **Phase A green AND #12 metric (done) AND #13 regime (done) AND the crisis-holdout data slice exists (new #2 task)**. rq-04 can't be honestly evaluated until 2008-Q4/2020-March are in a held-out eval slice — see #12 and #2. The scaffold command and `program.md` seed can be *built* before that (with HEPA+VICReg frozen, not TS-JEPA — confirmed on #13), but don't launch the rq-04 run until the crisis slice is real.
So the immediate next action on this issue is just: trigger the live toy run, confirm 3 iters + metric movement + both axes, close Phase A.
Phase A live-run instructions (the gate on Phase A acceptance)
The code is shipped (v1.4.0); what's missing is the live run — it needs the GPU + LiteLLM key, which only exist on koala. Fire and verify as follows. Do not fake or simulate any part of the run; if it fails, report the failure, don't work around it.
Fire it (from repo root on koala)
cd ~/jepa-fx-risk # adjust if the path differs
op run -- env LITELLM_KEY="$LITELLM_KEY" python loop.py --run-dir runs/phase-a-toy
Use 1Password (op run) — don't paste the key inline or leak it into shell history.
LITELLM_KEY must resolve to the koala LiteLLM endpoint (http://koala:30401/v1/), NOT a stale piguard reference. Verify the endpoint before firing if unsure.
Run-dir runs/phase-a-toy — a throwaway. Do NOT use runs/rq-04 (not ready: no VaR metric wired, no crisis-holdout data; scaffold should refuse it anyway, but don't point at it).
3 real iterations complete unattended — no manual intervention between iters.
The metric moves — val_vol_r2 changes across iters in STATUS.md (keep-if-improved-else-revert visibly working; needn't monotonically improve, but must not be static/stubbed).
Research axis live — STATUS.md shows metric trajectory, best-so-far, per-iter change log.
Technical axis live — HEARTBEAT updates per iter; nvidia-smi GPU line in STATUS.md; ntfy-on-crash wired (NTFY_URL set) even if it doesn't fire.
Report back here
Comment with: iterations completed, the three val_vol_r2 values (showing movement), confirmation both axes were live, total wallclock. Clean → state Phase A acceptance met and close. Any failure (loop stall, static metric, LiteLLM auth/endpoint error, GPU not visible) → capture the exact error here and leave open; do not patch around without flagging.
Do NOT in this run
Do not seed/launch runs/rq-04 (gated on crisis-holdout data — new #2 task — plus #12/#13 done but the data slice isn't).
Do not wire Prometheus/Langfuse (deferred, out of scope for Phase A MVP).
Do not extend data or touch the regime detector here — toy val_vol_r2 loop only.
## Phase A live-run instructions (the gate on Phase A acceptance)
The code is shipped (v1.4.0); what's missing is the live run — it needs the GPU + LiteLLM key, which only exist on koala. Fire and verify as follows. **Do not fake or simulate any part of the run; if it fails, report the failure, don't work around it.**
### Fire it (from repo root on koala)
```bash
cd ~/jepa-fx-risk # adjust if the path differs
op run -- env LITELLM_KEY="$LITELLM_KEY" python loop.py --run-dir runs/phase-a-toy
```
- Use 1Password (`op run`) — don't paste the key inline or leak it into shell history.
- `LITELLM_KEY` must resolve to the koala LiteLLM endpoint (`http://koala:30401/v1/`), NOT a stale piguard reference. Verify the endpoint before firing if unsure.
- Run-dir `runs/phase-a-toy` — a throwaway. Do NOT use `runs/rq-04` (not ready: no VaR metric wired, no crisis-holdout data; scaffold should refuse it anyway, but don't point at it).
### What each iteration does
edit `train.py` → time-boxed train (<2 min toy) → read `val_vol_r2` from `metrics.json` → keep-if-improved-else-revert → append `STATUS.md` → write `HEARTBEAT`.
### Acceptance — all four must hold
1. **3 real iterations complete unattended** — no manual intervention between iters.
2. **The metric moves** — `val_vol_r2` changes across iters in `STATUS.md` (keep-if-improved-else-revert visibly working; needn't monotonically improve, but must not be static/stubbed).
3. **Research axis live** — `STATUS.md` shows metric trajectory, best-so-far, per-iter change log.
4. **Technical axis live** — `HEARTBEAT` updates per iter; nvidia-smi GPU line in STATUS.md; ntfy-on-crash wired (`NTFY_URL` set) even if it doesn't fire.
### Report back here
Comment with: iterations completed, the three `val_vol_r2` values (showing movement), confirmation both axes were live, total wallclock. Clean → state Phase A acceptance met and close. Any failure (loop stall, static metric, LiteLLM auth/endpoint error, GPU not visible) → capture the exact error here and leave open; do not patch around without flagging.
### Do NOT in this run
- Do not seed/launch `runs/rq-04` (gated on crisis-holdout data — new #2 task — plus #12/#13 done but the data slice isn't).
- Do not wire Prometheus/Langfuse (deferred, out of scope for Phase A MVP).
- Do not extend data or touch the regime detector here — toy `val_vol_r2` loop only.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
A one-command path from an Autoresearch Council backlog leaf (agentsquad #42/#44, the
autoresearch-readynodes with a singlecandidate_metric) to a running, monitored Karpathy-style autoresearch loop — and the ability to watch it on two axes while it runs: research progress (is the metric actually moving?) and technical health (is the loop alive, fast, and not on fire?).Closes the last gap in the pipeline: Council decomposes → guards → emits a backlog; today a human re-keys a leaf into a
program.mdby hand. This makes the handoff mechanical and observable.Background / chain of custody
Every run must carry provenance back to the council node id + the strategic question (so a result traces to the question that spawned it).
Components
1. Ingest + scaffold (
autoresearch start <backlog.json> <rq-id>)rq-id(must bestatus: autoresearch-ready; refuse otherwise — fail-closed).program.md(hypothesis = the node question; single metric =candidate_metric; the ablation it implies; baseline to beat), a freshtrain.pyfrom the project template, and arun.yaml(model tier, max iters, time-box, provenance:{strategic_question, council_node, generated_at}).VaR_breach_rate_99_oos_regime_cond).2. Launch
train.py, runs the time-boxed experiment, reads the single metric, keeps-or-reverts. Reuse the existing loop shape; this issue is the entry point + wiring, not a new loop.3. Monitor — research/task axis ("is it learning?")
STATUS.md(cheap, always works) and/or a small dashboard. Answer at a glance: improving / plateaued / diverging.4. Monitor — technical axis ("is it healthy?")
Acceptance criteria
autoresearch start <backlog.json> <rq-id>scaffolds a runnable program.md + run dir from a ready leaf; refuses non-ready nodesRisk: MEDIUM
Out of scope
needs-human-framingnodes (still a human metric-decision)Refs
wiki/jepa-fx/facts/autoresearch-integration-phase1(loop architecture),wiki/glossary/val-vol-r2(metric discipline),knowledge/pattern-autoresearch-karpathySeed
program.md— rq-04 (first autoresearch case)Ready to commit into the run dir once the launcher (this issue) exists. Derived directly from the Autoresearch Council backlog leaf
rq-04.Notes:
val_vol_r2-metric ready leaves on the same question, which are the natural representation-quality companions to run after this calibration probe.Minimum vertical slice — get something executing + monitored, then deepen
The full path (foundation #2/#3/#5/#4 → loop → rq-04 extras) is the whole project. To de-risk it, prove the chain end-to-end with a toy first, so "council leaf → running, monitored experiment" is real before TS-JEPA exists. Three phases:
Phase A — toy slice (the point: the loop runs + you can watch it)
Deliberately tiny; throwaway-able. Reuse what's already in the repo (
internal/webUI,cmd/hostexecutor).train.py(toy): a small encoder (NOT full TS-JEPA) that trains in <2 min on koala and exports embeddings. Just enough to move a metric.val_vol_r2linear probe only — a thin slice of #4 (the metric the harness is designed for; defer VaR-breach to rq-04).train.py→ run (time-boxed) → read the scalar → keep-if-improved-else-revert → append toSTATUS.md. The Karpathy loop, minimal.STATUS.md+ the existing web UI: metric trajectory, best-so-far, per-iter change-log.Phase B — make it real
Swap the toy for the foundation, slice by slice: #2 full data pipeline, #3 TS-JEPA + SIGReg backbone, #5 Phase-0 SSL feasibility gate (must pass before trusting the loop), #4 full eval harness, #10 DVC+MinIO for experiment artifacts/provenance. The loop + monitoring from Phase A stay; only the substrate deepens.
Phase C — rq-04
On the real substrate: regime detector + conditioning hook (issue above) and the
VaR_breach_rate_99metric (issue above). Thenautoresearch start <backlog.json> rq-04scaffolds the program.md (seeded above) and launches. This is when the juicy council leaf actually runs.Why this order
val_vol_r2case (matches the built-for harness), NOT rq-04 — rq-04 needs a new metric + regime model on top.Concretely, Phase A is the smallest commit set that makes #11's acceptance ("launch starts the loop; one real iteration completes; both axes visible") true.
Cross-repo dependency — agentsquad#48 (metric whitespace).
This issue's scaffolder reads
candidate_metricfrom the Council backlog JSON to wire the single metric intoprogram.md/run.yaml. In the first real run that value came out as" VaR_breach_rate_99_oos_regime_cond"— leading space — because of a Stage-3 serialisation bug (agentsquad#48, bug 2).Two implications for #11:
.strip()thecandidate_metricdefensively before using it as an identifier / metric key — don't assume upstream is clean even after #48 lands (belt-and-braces; the scaffold is the last line before it becomes a file path / lookup key).VaR_breach_rate_99_oos_regime_cond(no leading space). #12 emits the scalar under that exact key; keep them identical or the loop's metric-lookup misses.agentsquad#48 fixes the source; this note ensures #11 is robust regardless of ingest order.
Implementation plan
autoresearch start <backlog.json> <rq-id>commandReads the Council backlog JSON (e.g.
first_run_2026-06-23.json), finds the node byrq-id. Fail-closed: ifstatus != autoresearch-ready, print error and exit non-zero. No partial scaffolding on non-ready nodes.Scaffolds a run dir (
runs/<rq-id>/):program.md— hypothesis = node question; single metric =candidate_metric.strip()(defensive); ablation implied by the question; baseline to beatrun.yaml— model tier, max_iters, time_box_minutes, provenance:{strategic_question, council_node, generated_at}train.py— copy from project template (the currenttrain.py)Provenance is recorded in
run.yamland visible in STATUS.md header.rq-04 seed
Will scaffold
runs/rq-04/as the worked example once #12's metric key (VaR_breach_rate_99_oos_regime_cond) exists in the eval harness.Monitoring stubs
Research axis (STATUS.md): metric trajectory (best-so-far + per-iter delta), plateau detection, per-iter change log.
Technical axis: loop liveness (iterations/hr, last-heartbeat). Resource/LiteLLM/alert wiring (Prometheus, Langfuse, ntfy) deferred — noted as out of scope for this issue's MVP, tracked as follow-up. Will note in implementation which hooks exist and where the wiring should land.
Dep gate
Blocked on #12 metric existing before rq-04 run can be seeded. Will implement the scaffold command first (testable independently with a mock metric key), then seed rq-04 once #12 lands.
Decision: Phase A toy slice FIRST — do not go straight to rq-04
Your own earlier analysis (comment above, the three-phase plan) had this right, but the current execution plans across #11/#12/#13 quietly skipped Phase A and are marching straight to rq-04 (Phase C). Pull it back to toy-first.
Build Phase A before the regime detector or the VaR-breach metric:
val_vol_r2linear probe only — the metric the harness was built for. Defer VaR-breach to Phase C.train.py→ time-boxed run → read scalar → keep-if-improved-else-revert → appendSTATUS.md.Why toy-first is real learning, not ceremony: the toy teaches nothing about FX or JEPA — it de-risks the harness (loop control, time-boxing, revert logic, metric plumbing, the two monitor axes). Those break on first contact regardless of the science. This is the first autoresearch loop actually running, so the mechanics are unproven and the toy lets them fail on a 2-min run instead of a 10-min TS-JEPA one. Going straight to rq-04 means debugging the harness AND the regime model AND the science simultaneously on slow runs — exactly the trap the agentsquad CAD work taught us to avoid (prove the loop on something trivial before real compute).
Sequencing, concretely:
val_vol_r2loop above. This is what makes #11's acceptance true ("launch starts the loop; one real iteration completes; both axes visible").runs/rq-04/until Phase A is green.The
autoresearch startscaffold command itself (comment 1939) is fine to build now — it's testable with a mock metric key independent of phase. Build the command, prove it on the toy, seed rq-04 last.Phase A shipped — v1.4.0
Acceptance criteria status:
autoresearch startscaffolds run dir from ready leaf; refuses non-ready nodescandidate_metricstripped on ingestion (bypass path fixed)`val_vol_r2`not` val_vol_r2`LITELLM_KEY+ koala; trigger:LITELLM_KEY=xxx python loop.py --run-dir runs/<id>What shipped:
scripts/autoresearch_start.py— fail-closed scaffold commandloop.py --run-dirflag — redirects STATUS.md / metrics.json / HEARTBEAT / train.py into the run dir; METRICS_OUT env var passes to train subprocesstrain.py— respectsMETRICS_OUTenv var (default unchanged)Remaining before rq-04 seed:
program.mdcan name itLeaving open until 3 live iters confirmed.
Phase A code — accepted. Two things noted.
Clean build: fail-closed scaffold (refuses non-ready, 19 tests), both monitor axes, 54 existing tests still green. And you found and fixed the candidate_metric strip bypass — the test asserting
val_vol_r2not␣val_vol_r2confirms you located the actual bypass path, not just added a second strip. That was the open question from agentsquad#48; good that it's closed on the consuming side too.Correctly left open pending the 3 live unattended iterations — that needs
LITELLM_KEY+ koala and you shouldn't self-certify a run you didn't do. That live run is now the single gate on Phase A acceptance, and Phase A is the gate on everything downstream. Trigger it when ready:LITELLM_KEY=xxx python loop.py --run-dir runs/<id>.One update to the rq-04 seed gate (from the #12 crisis-window decision): the gate is no longer just "Phase A green AND #12 metric." It's now Phase A green AND #12 metric (done) AND #13 regime (done) AND the crisis-holdout data slice exists (new #2 task). rq-04 can't be honestly evaluated until 2008-Q4/2020-March are in a held-out eval slice — see #12 and #2. The scaffold command and
program.mdseed can be built before that (with HEPA+VICReg frozen, not TS-JEPA — confirmed on #13), but don't launch the rq-04 run until the crisis slice is real.So the immediate next action on this issue is just: trigger the live toy run, confirm 3 iters + metric movement + both axes, close Phase A.
Phase A live-run instructions (the gate on Phase A acceptance)
The code is shipped (v1.4.0); what's missing is the live run — it needs the GPU + LiteLLM key, which only exist on koala. Fire and verify as follows. Do not fake or simulate any part of the run; if it fails, report the failure, don't work around it.
Fire it (from repo root on koala)
op run) — don't paste the key inline or leak it into shell history.LITELLM_KEYmust resolve to the koala LiteLLM endpoint (http://koala:30401/v1/), NOT a stale piguard reference. Verify the endpoint before firing if unsure.runs/phase-a-toy— a throwaway. Do NOT useruns/rq-04(not ready: no VaR metric wired, no crisis-holdout data; scaffold should refuse it anyway, but don't point at it).What each iteration does
edit
train.py→ time-boxed train (<2 min toy) → readval_vol_r2frommetrics.json→ keep-if-improved-else-revert → appendSTATUS.md→ writeHEARTBEAT.Acceptance — all four must hold
val_vol_r2changes across iters inSTATUS.md(keep-if-improved-else-revert visibly working; needn't monotonically improve, but must not be static/stubbed).STATUS.mdshows metric trajectory, best-so-far, per-iter change log.HEARTBEATupdates per iter; nvidia-smi GPU line in STATUS.md; ntfy-on-crash wired (NTFY_URLset) even if it doesn't fire.Report back here
Comment with: iterations completed, the three
val_vol_r2values (showing movement), confirmation both axes were live, total wallclock. Clean → state Phase A acceptance met and close. Any failure (loop stall, static metric, LiteLLM auth/endpoint error, GPU not visible) → capture the exact error here and leave open; do not patch around without flagging.Do NOT in this run
runs/rq-04(gated on crisis-holdout data — new #2 task — plus #12/#13 done but the data slice isn't).val_vol_r2loop only.