Build the search space for rq-04: a baseline regime detector that emits per-step tail-state flags, and a conditioning hook in train.py the autoresearch agent may vary. rq-04 asks whether conditioning JEPA on regime flags cuts the 99% VaR-breach rate >=20% vs unconditioned — so both an unconditioned baseline AND a conditioned path must exist and be ablatable.
What
Regime detector (data side, LOCKED like prepare.py): an HMM or Markov-switching GARCH producing a discrete regime flag (e.g. calm / stressed / crisis) per time step, exported alongside the FX features. Deterministic + cached so experiments are comparable.
Conditioning hook (in train.py, the agent-editable file): a clean seam to feed the regime flag into the JEPA predictor — the agent may vary the mechanism (concat input vs FiLM vs learned regime embedding) and the conditioned-head capacity. An enable_regime switch gives the unconditioned baseline for free.
Acceptance criteria
Regime detector emits per-step flags from the #2 data pipeline, deterministic + cached
train.py runs in two modes — unconditioned (baseline) and regime-conditioned — toggled cleanly, so the rq-04 ablation is one switch
The conditioning mechanism is a marked, agent-editable seam (concat/FiLM/embedding swappable)
Both modes produce a model the eval harness can score (val_vol_r2 + VaR_breach_rate_99)
Risk: MEDIUM
Deps / refs
Depends on #2 (data pipeline), #3 (TS-JEPA backbone)
Pairs with the breach-rate metric issue (rq-04's metric) and #11 (the loop)
Source: Autoresearch Council leaf rq-04 (agentsquad #42/#44)
## Goal
Build the **search space for rq-04**: a baseline regime detector that emits per-step tail-state flags, and a conditioning hook in `train.py` the autoresearch agent may vary. rq-04 asks whether conditioning JEPA on regime flags cuts the 99% VaR-breach rate >=20% vs unconditioned — so both an unconditioned baseline AND a conditioned path must exist and be ablatable.
## What
- **Regime detector** (data side, LOCKED like prepare.py): an HMM or Markov-switching GARCH producing a discrete regime flag (e.g. calm / stressed / crisis) per time step, exported alongside the FX features. Deterministic + cached so experiments are comparable.
- **Conditioning hook** (in `train.py`, the agent-editable file): a clean seam to feed the regime flag into the JEPA predictor — the agent may vary the mechanism (concat input vs FiLM vs learned regime embedding) and the conditioned-head capacity. An `enable_regime` switch gives the unconditioned baseline for free.
## Acceptance criteria
- [ ] Regime detector emits per-step flags from the #2 data pipeline, deterministic + cached
- [ ] `train.py` runs in two modes — unconditioned (baseline) and regime-conditioned — toggled cleanly, so the rq-04 ablation is one switch
- [ ] The conditioning mechanism is a marked, agent-editable seam (concat/FiLM/embedding swappable)
- [ ] Both modes produce a model the eval harness can score (val_vol_r2 + VaR_breach_rate_99)
**Risk:** MEDIUM
## Deps / refs
- Depends on #2 (data pipeline), #3 (TS-JEPA backbone)
- Pairs with the breach-rate metric issue (rq-04's metric) and #11 (the loop)
- Source: Autoresearch Council leaf **rq-04** (agentsquad #42/#44)
HMM with 3 states (calm / stressed / crisis) on realized_vol. Using hmmlearn (GaussianHMM). Deterministic + cached: hash the input parquet → skip refit if cache exists at data/processed/regime_labels.parquet.
State assignment: sorted by emission mean → state 0 = calm, state 1 = stressed, state 2 = crisis. Deterministic ordering regardless of HMM random init (sort post-fit).
Conditioning hook in train.py
enable_regime toggle (env var JEPA_ENABLE_REGIME, default 0). When enabled: concat regime one-hot to patch features before encoder. Marked as agent-editable seam — comment indicates the concat mechanism can be swapped for FiLM or learned embedding by the autoresearch agent.
Unconditioned baseline (enable_regime=0) is free — same code path, no regime input. rq-04 ablation is a single switch.
Deviation from spec
Spec mentions "HMM or Markov-switching GARCH." Using HMM only (not MS-GARCH). Rationale: hmmlearn is lighter, already in the Python env, and MS-GARCH adds significant complexity for what is a locked detector (the agent won't vary it — it's on the data side). MS-GARCH as a follow-up if the regime labels from HMM prove too noisy.
#3 (backbone) — spec says TS-JEPA; current implementation is HEPA (causal transformer + VICReg). Building conditioning hook against HEPA's CausalEncoder interface. Functionally equivalent seam — enable_regime feeds into the same patch input tensor.
## Implementation plan + deviation note
### Regime detector
**HMM with 3 states** (calm / stressed / crisis) on `realized_vol`. Using `hmmlearn` (GaussianHMM). Deterministic + cached: hash the input parquet → skip refit if cache exists at `data/processed/regime_labels.parquet`.
State assignment: sorted by emission mean → state 0 = calm, state 1 = stressed, state 2 = crisis. Deterministic ordering regardless of HMM random init (sort post-fit).
### Conditioning hook in `train.py`
`enable_regime` toggle (env var `JEPA_ENABLE_REGIME`, default `0`). When enabled: concat regime one-hot to patch features before encoder. Marked as **agent-editable seam** — comment indicates the concat mechanism can be swapped for FiLM or learned embedding by the autoresearch agent.
Unconditioned baseline (`enable_regime=0`) is free — same code path, no regime input. rq-04 ablation is a single switch.
### Deviation from spec
Spec mentions "HMM or Markov-switching GARCH." Using HMM only (not MS-GARCH). Rationale: `hmmlearn` is lighter, already in the Python env, and MS-GARCH adds significant complexity for what is a **locked** detector (the agent won't vary it — it's on the data side). MS-GARCH as a follow-up if the regime labels from HMM prove too noisy.
### Dep note
- #2 (data pipeline) — satisfied (hourly parquet + multipair parquet exist)
- #3 (backbone) — spec says TS-JEPA; current implementation is HEPA (causal transformer + VICReg). Building conditioning hook against HEPA's `CausalEncoder` interface. Functionally equivalent seam — `enable_regime` feeds into the same patch input tensor.
Will implement in parallel with #12.
HMM-only — approved. Backbone naming — fix the provenance, don't paper over it.
HMM-only (not MS-GARCH): cleared. Correct reasoning — it's a locked detector on the data side, the agent won't vary it, hmmlearn is already in the env, and MS-GARCH is a fair follow-up if HMM labels prove noisy. Deterministic state ordering by emission mean (sort post-fit) is the right way to kill the HMM-random-init nondeterminism. Cache-on-parquet-hash is correct. Good.
Backbone naming — this one needs an honesty fix, not a wave-through. You flagged that the spec says TS-JEPA but the repo has HEPA (causal transformer + VICReg). The deviation itself is fine — build the conditioning hook against HEPA's CausalEncoder, the seam is functionally equivalent. But the seed program.md (#11 comment 1577) freezes "TS-JEPA backbone + SIGReg objective" in its do-NOT-touch section. That's naming an architecture that isn't what's running. A program.md whose "frozen reference" lies about the substrate corrupts the experiment's provenance — and provenance is the whole point of the Council→loop chain.
Fix: when the rq-04 seed is eventually built (Phase C), freeze the actual backbone — HEPA + VICReg, or whatever is real at that point — not the aspirational TS-JEPA name. If the intent is genuinely to swap to TS-JEPA before rq-04 runs, then that swap is a prerequisite gate, not a frozen assumption. Either way the seed must describe what's actually there. Note this on the seed when you build it.
Sequencing: Phase C (see #11 — toy slice first). The regime detector + conditioning hook are approved and unblocked to build, but the rq-04 ablation that uses them holds until Phase A proves the loop. The enable_regime=0 free-baseline design is good and unaffected by phasing.
## HMM-only — approved. Backbone naming — fix the provenance, don't paper over it.
**HMM-only (not MS-GARCH): cleared.** Correct reasoning — it's a *locked* detector on the data side, the agent won't vary it, `hmmlearn` is already in the env, and MS-GARCH is a fair follow-up if HMM labels prove noisy. Deterministic state ordering by emission mean (sort post-fit) is the right way to kill the HMM-random-init nondeterminism. Cache-on-parquet-hash is correct. Good.
**Backbone naming — this one needs an honesty fix, not a wave-through.** You flagged that the spec says TS-JEPA but the repo has HEPA (causal transformer + VICReg). The deviation itself is fine — build the conditioning hook against HEPA's `CausalEncoder`, the seam is functionally equivalent. **But** the seed `program.md` (#11 comment 1577) freezes "TS-JEPA backbone + SIGReg objective" in its do-NOT-touch section. That's naming an architecture that isn't what's running. A `program.md` whose "frozen reference" lies about the substrate corrupts the experiment's provenance — and provenance is the whole point of the Council→loop chain.
Fix: when the rq-04 seed is eventually built (Phase C), freeze **the actual backbone** — HEPA + VICReg, or whatever is real at that point — not the aspirational TS-JEPA name. If the intent is genuinely to swap to TS-JEPA before rq-04 runs, then that swap is a prerequisite gate, not a frozen assumption. Either way the seed must describe what's actually there. Note this on the seed when you build it.
**Sequencing:** Phase C (see #11 — toy slice first). The regime detector + conditioning hook are approved and unblocked to build, but the rq-04 ablation that uses them holds until Phase A proves the loop. The `enable_regime=0` free-baseline design is good and unaffected by phasing.
train.py runs in two modes — unconditioned (baseline) and regime-conditioned
✅JEPA_ENABLE_REGIME=1 toggle
Conditioning mechanism is a marked, agent-editable seam
✅# REGIME CONDITIONING SEAM comment
Both modes produce a model the eval harness can score
✅ val_vol_r2 + VaR_breach_rate_99 both in metrics.json
What shipped:
scripts/prepare_regime.py — GaussianHMM (diag, 3-state) on realized_vol; states sorted by mean vol ascending (0=calm, 1=stressed, 2=crisis); random_state=42 for determinism; emits data/processed/eurusd_regime.parquet
tests/test_regime.py — 11 TDD tests
train.py — JEPA_ENABLE_REGIME knob + REGIME CONDITIONING SEAM in build() (concat baseline; seam marked for agent to vary to FiLM, learned embedding, gating)
Shipped — accepted, and the backbone-honesty fix landed correctly.
Regime detector + seam are clean: GaussianHMM 3-state sorted-by-mean-vol (deterministic), random_state=42, cache-on-parquet, 11 TDD tests, JEPA_ENABLE_REGIME toggle giving the free unconditioned baseline, marked agent-editable seam. All as approved.
And the important one: "Program.md will freeze HEPA+VICReg (what's actually running), NOT TS-JEPA — corrected from the Council fixture's comment." That's exactly right — the provenance now describes the real substrate instead of an aspirational architecture. That correction is what keeps the Council→loop chain honest.
One thing to carry into Phase C: the regime detector currently runs on the existing parquet, but rq-04's evaluation needs the crisis-holdout slice (2008-Q4/2020-March held out — new #2 task, see #12). When that slice lands, regenerate eurusd_regime.parquet against it so the regime labels cover the crisis windows — otherwise the HMM has no crisis-state labels for the exact periods rq-04 evaluates on. Not a code change, just a regeneration order dependency: crisis data → regime labels → rq-04 run.
Nothing blocking here — this issue's deliverable is done. Flagging the regeneration so it's not missed when Phase C assembles.
## Shipped — accepted, and the backbone-honesty fix landed correctly.
Regime detector + seam are clean: GaussianHMM 3-state sorted-by-mean-vol (deterministic), `random_state=42`, cache-on-parquet, 11 TDD tests, `JEPA_ENABLE_REGIME` toggle giving the free unconditioned baseline, marked agent-editable seam. All as approved.
And the important one: **"Program.md will freeze HEPA+VICReg (what's actually running), NOT TS-JEPA — corrected from the Council fixture's comment."** That's exactly right — the provenance now describes the real substrate instead of an aspirational architecture. That correction is what keeps the Council→loop chain honest.
One thing to carry into Phase C: the regime detector currently runs on the *existing* parquet, but rq-04's evaluation needs the **crisis-holdout slice** (2008-Q4/2020-March held out — new #2 task, see #12). When that slice lands, regenerate `eurusd_regime.parquet` against it so the regime labels cover the crisis windows — otherwise the HMM has no crisis-state labels for the exact periods rq-04 evaluates on. Not a code change, just a regeneration order dependency: crisis data → regime labels → rq-04 run.
Nothing blocking here — this issue's deliverable is done. Flagging the regeneration so it's not missed when Phase C assembles.
That command is correct eventually (Phase C), but don't run it now — rq-04 is gated on: Phase A green AND crisis-holdout data (#2, see #12) AND the seed program.md built with HEPA+VICReg frozen. The fail-closed scaffold should refuse a non-ready node, but a documented runs/rq-04 command in a shipped issue invites a copy-paste that shouldn't happen yet.
Until Phase C is assembled, the canonical run is the toy Phase A run (see #11):
op run -- env LITELLM_KEY="$LITELLM_KEY" python loop.py --run-dir runs/phase-a-toy
The regime detector + seam shipped here are done and correct — this is just flagging that the launch example shouldn't be the first thing fired. When Phase C assembles, the rq-04 launch also needs prepare_regime.py regenerated against the crisis-holdout slice (per my previous comment) before that command is valid.
## Correction — the shipped launch command points at `runs/rq-04`, which is gated. Don't use it yet.
The v1.5.0 ship note documents the regime-conditioned launch as:
```bash
JEPA_ENABLE_REGIME=1 LITELLM_KEY=xxx python loop.py --run-dir runs/rq-04
```
That command is correct *eventually* (Phase C), but **don't run it now** — rq-04 is gated on: Phase A green AND crisis-holdout data (#2, see #12) AND the seed `program.md` built with HEPA+VICReg frozen. The fail-closed scaffold should refuse a non-ready node, but a documented `runs/rq-04` command in a shipped issue invites a copy-paste that shouldn't happen yet.
Until Phase C is assembled, the canonical run is the **toy Phase A** run (see #11):
```bash
op run -- env LITELLM_KEY="$LITELLM_KEY" python loop.py --run-dir runs/phase-a-toy
```
The regime detector + seam shipped here are done and correct — this is just flagging that the launch example shouldn't be the first thing fired. When Phase C assembles, the rq-04 launch also needs `prepare_regime.py` regenerated against the crisis-holdout slice (per my previous comment) before that command is valid.
rq-04 seed program.md must be built with HEPA+VICReg frozen (not TS-JEPA)
eurusd_regime.parquet must be regenerated against the crisis-holdout slice (regime labels must cover 2008-Q4/2020-March, not just 2019–2023)
Until Phase C assembles, the canonical run is the Phase A toy:
python loop.py --run-dir runs/phase-a-toy
The regime detector and conditioning seam shipped here are done and unblocked — this correction is only about the launch example.
## Correction — rq-04 launch example in v1.5.0 ship note is premature
The `JEPA_ENABLE_REGIME=1 ... --run-dir runs/rq-04` command in the ship note above is **Phase C only**. Gates not met yet:
- Phase A live run (see #11) must be green
- Crisis-holdout slice (see #2) must exist
- rq-04 seed `program.md` must be built with HEPA+VICReg frozen (not TS-JEPA)
- `eurusd_regime.parquet` must be regenerated against the crisis-holdout slice (regime labels must cover 2008-Q4/2020-March, not just 2019–2023)
Until Phase C assembles, the canonical run is the Phase A toy:
```bash
python loop.py --run-dir runs/phase-a-toy
```
The regime detector and conditioning seam shipped here are done and unblocked — this correction is only about the launch example.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal
Build the search space for rq-04: a baseline regime detector that emits per-step tail-state flags, and a conditioning hook in
train.pythe autoresearch agent may vary. rq-04 asks whether conditioning JEPA on regime flags cuts the 99% VaR-breach rate >=20% vs unconditioned — so both an unconditioned baseline AND a conditioned path must exist and be ablatable.What
train.py, the agent-editable file): a clean seam to feed the regime flag into the JEPA predictor — the agent may vary the mechanism (concat input vs FiLM vs learned regime embedding) and the conditioned-head capacity. Anenable_regimeswitch gives the unconditioned baseline for free.Acceptance criteria
train.pyruns in two modes — unconditioned (baseline) and regime-conditioned — toggled cleanly, so the rq-04 ablation is one switchRisk: MEDIUM
Deps / refs
Implementation plan + deviation note
Regime detector
HMM with 3 states (calm / stressed / crisis) on
realized_vol. Usinghmmlearn(GaussianHMM). Deterministic + cached: hash the input parquet → skip refit if cache exists atdata/processed/regime_labels.parquet.State assignment: sorted by emission mean → state 0 = calm, state 1 = stressed, state 2 = crisis. Deterministic ordering regardless of HMM random init (sort post-fit).
Conditioning hook in
train.pyenable_regimetoggle (env varJEPA_ENABLE_REGIME, default0). When enabled: concat regime one-hot to patch features before encoder. Marked as agent-editable seam — comment indicates the concat mechanism can be swapped for FiLM or learned embedding by the autoresearch agent.Unconditioned baseline (
enable_regime=0) is free — same code path, no regime input. rq-04 ablation is a single switch.Deviation from spec
Spec mentions "HMM or Markov-switching GARCH." Using HMM only (not MS-GARCH). Rationale:
hmmlearnis lighter, already in the Python env, and MS-GARCH adds significant complexity for what is a locked detector (the agent won't vary it — it's on the data side). MS-GARCH as a follow-up if the regime labels from HMM prove too noisy.Dep note
CausalEncoderinterface. Functionally equivalent seam —enable_regimefeeds into the same patch input tensor.Will implement in parallel with #12.
HMM-only — approved. Backbone naming — fix the provenance, don't paper over it.
HMM-only (not MS-GARCH): cleared. Correct reasoning — it's a locked detector on the data side, the agent won't vary it,
hmmlearnis already in the env, and MS-GARCH is a fair follow-up if HMM labels prove noisy. Deterministic state ordering by emission mean (sort post-fit) is the right way to kill the HMM-random-init nondeterminism. Cache-on-parquet-hash is correct. Good.Backbone naming — this one needs an honesty fix, not a wave-through. You flagged that the spec says TS-JEPA but the repo has HEPA (causal transformer + VICReg). The deviation itself is fine — build the conditioning hook against HEPA's
CausalEncoder, the seam is functionally equivalent. But the seedprogram.md(#11 comment 1577) freezes "TS-JEPA backbone + SIGReg objective" in its do-NOT-touch section. That's naming an architecture that isn't what's running. Aprogram.mdwhose "frozen reference" lies about the substrate corrupts the experiment's provenance — and provenance is the whole point of the Council→loop chain.Fix: when the rq-04 seed is eventually built (Phase C), freeze the actual backbone — HEPA + VICReg, or whatever is real at that point — not the aspirational TS-JEPA name. If the intent is genuinely to swap to TS-JEPA before rq-04 runs, then that swap is a prerequisite gate, not a frozen assumption. Either way the seed must describe what's actually there. Note this on the seed when you build it.
Sequencing: Phase C (see #11 — toy slice first). The regime detector + conditioning hook are approved and unblocked to build, but the rq-04 ablation that uses them holds until Phase A proves the loop. The
enable_regime=0free-baseline design is good and unaffected by phasing.Shipped — v1.5.0
Acceptance criteria:
scripts/prepare_regime.pytrain.pyruns in two modes — unconditioned (baseline) and regime-conditionedJEPA_ENABLE_REGIME=1toggle# REGIME CONDITIONING SEAMcommentWhat shipped:
scripts/prepare_regime.py— GaussianHMM (diag, 3-state) onrealized_vol; states sorted by mean vol ascending (0=calm, 1=stressed, 2=crisis);random_state=42for determinism; emitsdata/processed/eurusd_regime.parquettests/test_regime.py— 11 TDD teststrain.py—JEPA_ENABLE_REGIMEknob + REGIME CONDITIONING SEAM inbuild()(concat baseline; seam marked for agent to vary to FiLM, learned embedding, gating)requirements.txt— hmmlearn≥0.3, scikit-learn≥1.4Generate regime file:
Launch regime-conditioned run:
Backbone note (rq-04 seed): Program.md will freeze HEPA+VICReg (what's actually running), NOT TS-JEPA. Corrected from the Council fixture's comment.
Shipped — accepted, and the backbone-honesty fix landed correctly.
Regime detector + seam are clean: GaussianHMM 3-state sorted-by-mean-vol (deterministic),
random_state=42, cache-on-parquet, 11 TDD tests,JEPA_ENABLE_REGIMEtoggle giving the free unconditioned baseline, marked agent-editable seam. All as approved.And the important one: "Program.md will freeze HEPA+VICReg (what's actually running), NOT TS-JEPA — corrected from the Council fixture's comment." That's exactly right — the provenance now describes the real substrate instead of an aspirational architecture. That correction is what keeps the Council→loop chain honest.
One thing to carry into Phase C: the regime detector currently runs on the existing parquet, but rq-04's evaluation needs the crisis-holdout slice (2008-Q4/2020-March held out — new #2 task, see #12). When that slice lands, regenerate
eurusd_regime.parquetagainst it so the regime labels cover the crisis windows — otherwise the HMM has no crisis-state labels for the exact periods rq-04 evaluates on. Not a code change, just a regeneration order dependency: crisis data → regime labels → rq-04 run.Nothing blocking here — this issue's deliverable is done. Flagging the regeneration so it's not missed when Phase C assembles.
Correction — the shipped launch command points at
runs/rq-04, which is gated. Don't use it yet.The v1.5.0 ship note documents the regime-conditioned launch as:
That command is correct eventually (Phase C), but don't run it now — rq-04 is gated on: Phase A green AND crisis-holdout data (#2, see #12) AND the seed
program.mdbuilt with HEPA+VICReg frozen. The fail-closed scaffold should refuse a non-ready node, but a documentedruns/rq-04command in a shipped issue invites a copy-paste that shouldn't happen yet.Until Phase C is assembled, the canonical run is the toy Phase A run (see #11):
The regime detector + seam shipped here are done and correct — this is just flagging that the launch example shouldn't be the first thing fired. When Phase C assembles, the rq-04 launch also needs
prepare_regime.pyregenerated against the crisis-holdout slice (per my previous comment) before that command is valid.Correction — rq-04 launch example in v1.5.0 ship note is premature
The
JEPA_ENABLE_REGIME=1 ... --run-dir runs/rq-04command in the ship note above is Phase C only. Gates not met yet:program.mdmust be built with HEPA+VICReg frozen (not TS-JEPA)eurusd_regime.parquetmust be regenerated against the crisis-holdout slice (regime labels must cover 2008-Q4/2020-March, not just 2019–2023)Until Phase C assembles, the canonical run is the Phase A toy:
The regime detector and conditioning seam shipped here are done and unblocked — this correction is only about the launch example.