infra's scripts/context-sync.sh is the estate canonical, and it moved in infra
262369b (a SIGPIPE fix on the placeholder-reporting path: `printf | grep -o |
head -1` became `head -1 < <(grep -o ...)`). Every other repo still carried the
old line, so ContextSyncScriptDrift and ContextAdapterDrift were both firing.
Also picks up the canonical AGENT.md change from local-dev a01ee80 — rule 5's
GitOps review lane and the new precedence rule — which every repo's adapters
need regenerated for regardless of the script.
Refs mathias/infra#470
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ContextAdapterDrift was firing for 13 repos. Picks up behaviour rule 7
("Finish what's closable"), which landed in ~/dev/.context/AGENT.md and never
reached any repo's derived files.
Where mcp.json changes, it also picks up root MCP config the repo was stale on:
brain/gitea/infra move from public hostnames to tailnet Serve URLs, and gitea's
bearer changes from GITEA_MCP_TOKEN to DMABE_GITEA_API_TOKEN — gitea-mcp
authenticates by pass-through and 401s on the former, so a harness syncing from
the old file had a dead gitea server and no signal saying so.
No literal secrets: mcp.json carries ${VAR} placeholders only.
CLAUDE.md deliberately unchanged — context-sync writes it from PROJECT.md alone,
since Claude Code walks up to ~/dev/CLAUDE.md for root context.
Refs mathias/infra ContextAdapterDrift
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
go.mod declared this module under gitea.d-ma.be, the pre-rename host
(infra ADR-0004). That name still resolves in DNS but no longer serves --
HTTPS to it fails outright -- so the declared path pointed at nothing.
It built anyway, because a Go module does not have to be fetchable from its
own declared path to compile locally. That is exactly what made it durable
rot: nothing failed, so nothing flagged it. What it did break is anyone
trying to depend on this repo -- `go get git.d-ma.be/mathias/<repo>` fails on
the path mismatch, and an importer would have to spell the dead host.
Verified safe before changing: nothing in the estate requires this module
(checked every go.mod across the Go repos) -- these are leaf binaries, so the
rename breaks no importer.
Mechanical: module path plus every internal import rewritten, templ
regenerated where applicable. Verified after -- go build, go vet, go test and
golangci-lint (against a Go 1.26.1 toolchain, matching CI) all pass, and no
gitea.d-ma.be reference remains in any .go/.templ/.mod file.
Found by an estate-wide scan for the old host, prompted by the same rot
turning up in tapir internal/web/views.templ (tapir@4bafd5f).
Canonical lives at infra ci-templates/golangci.yml, distributed by the
infra#309 pattern: byte-identical copies, checksum drift gate in infra. Do
not edit this file here -- edit the canonical and run `task ci:sync`.
Verified locally before pushing, not pushed-and-hoped: golangci-lint v2.11.4
against a Go 1.26.1 toolchain (matching CI) reports 0 issues here.
Also applies gofumpt, which this config enables and the golangci-lint v2
default set does not check at all. That gap is why the drift was invisible:
earlier passes ran gofmt, which is weaker. Mechanical, deterministic,
build+vet verified.
Estate-wide consolidation pass. golangci-lint v2 default set enables NO
formatter, so gofmt drift accumulated silently across the estate -- 8 repos
were carrying it, this one included. Same hole that let two files rot in
dma-cli until they were found by hand (dma-cli#12).
Mechanical and deterministic: `gofmt -w`, nothing hand-edited. Verified
before commit -- gofmt -l now reports 0, go build and go vet both pass.
Clears this repo to adopt the canonical .golangci.yml (infra
ci-templates/golangci.yml, infra#309 pattern) in a later wave.
Same drift as gitea-mcp: this repo's copy of context-sync.sh was two days
stale against dma-cli's canonical (Mathias's Cursor/Aider decision,
2026-09-01) and CLAUDE.md itself was stale enough to need 97 lines of
correction on regeneration -- this repo's context:sync was apparently never
wired into `task check` at all (no drift assertion existed in its Taskfile),
so nothing had been catching it.
Found while building the estate-wide context-sync drift gate (ADR-0019).
- script synced to canonical hash
- .cursorrules, .aider.conventions.md, .aider.conf.yml deleted
- stale context:sync:cursor Taskfile shortcut removed (would now hit the
canonical script's own `exit 1` on an unknown adapter name)
- CLAUDE.md, AGENTS.md, .context/system-prompt.txt regenerated against canonical
task check -> exit 0.
infra#344. gitleaks ORs the clauses of a GLOBAL `[[allowlists]]` block and does
not honour `condition = "AND"` there, so the `paths` entry suppressed EVERY
finding in the named file — not only the false positive the `regexes` line
describes. The regex alone is what was meant, and it works regardless of which
file the value turns up in (infra#256).
Verified rather than assumed, with a control for the probe itself: a live-shaped
credential planted in the formerly-exempt file scans clean before this change and
fires after it, and the same shape fires in a neutral path both ways (so "not
detected" cannot be mistaken for "immune"). The intended false positive is still
suppressed — the tree scans clean.
Refs: mathias/infra#344, mathias/infra#256, mathias/infra#317
The estate-wide secret scan (infra#316) flags a fixture value in this repo
as a credential. Verified by reading the surrounding code that it is not
one, then scoped an allowlist to it.
Scoped by BOTH path and pattern (condition = "AND"), never path alone: a
path-only allowlist would blind this file to a future REAL secret, which is
how a live token hid inside a file whose name read as safe (infra#256).
Allowlist by shape, not by filename.
Adds .gitleaks.toml where there was none, so this repo is no longer scanned
with bare defaults.
This is a research repo -- issue#6/#7 already decided CI should be
check-only (`task check` on push/PR), no build, no image push, no
deploy, since training runs are manual/GPU-bound and never CI-triggered.
That never actually landed in cd.yml: it still carried the original
go-web template's build+deploy jobs verbatim, still referencing the
pre-rename `hostexecutor` name (`./cmd/hostexecutor` doesn't exist --
real dirs are cmd/eval, cmd/jepa-fx-risk) and a `k3s/apps/hostexecutor`
deployment that was never created in infra (confirmed: no manifest
exists under either name). Once the check job's own Taskfile bug was
fixed, build started actually running and failing on the wrong path.
Removed both jobs -- check is the only gate this repo needs.
Taskfile.yml line 46 had an unquoted Go-template `{{.VAR}}` inside a YAML
flow sequence (`cmds: [...]`) -- the literal braces broke YAML parsing
outright ("did not find expected ',' or ']'"), so `task check` (and thus
CI's push-triggered check job) failed before running a single command.
A manual `workflow_dispatch` re-run passed because the autoresearch
workflow never calls `task check` at all -- unrelated path, not an
env/secret difference as first suspected. Quoted the string.
Also fixed a staticcheck QF1002 in internal/eval/var.go: a boolean
switch comparing the same variable (n1) in every case is a tagged
switch in disguise -- converted to `switch n1 { case 0: ... case n: ...
}`.
train.py is copied into runs/<rq-id>/ by autoresearch_start.py, so when loop.py
executes it, sys.path[0] is the run dir — which has no scripts/. train.py's
frozen VaR-eval block does `from scripts.var_breach import ...`, which then
fails with ModuleNotFoundError: No module named 'scripts' on every loop run
(CI and the documented manual launch alike).
Fix in the harness, not the frozen train.py/scripts boundary: prepend the
project root (loop.py's own dir, where scripts/ lives) to the subprocess
PYTHONPATH. Verified red→green locally: the import fails without it and
resolves with it (scripts/ is an implicit namespace package, no __init__.py).
The Autoresearch Loop workflow (phase-a-toy) runs train.py, which reads
data/processed/eurusd_daily.parquet. data/ is gitignored (DVC/MinIO policy, #10),
but CI checks out a fresh tree with no data, so the baseline run died with
FileNotFoundError. Force-add this single 59KB derived fixture (public EUR/USD
daily realized vol, 2019-2023, 1560 rows) so the toy loop is hermetic and fast.
This is a scoped exception to #10 for a CI fixture only — large/production
datasets still stay out of git per that policy.
workflow_dispatch with inputs: fixture, rq_id, iters, model override.
- Scaffolds run dir from fixtures/<fixture>.json via autoresearch_start.py
- Runs loop.py with LITELLM_KEY/LITELLM_BASE/NTFY_URL secrets
- Uploads STATUS.md + metrics.json + program.md as artifacts (30-day retention)
- Cleans stale run dir at start; venv created/reused per runner workspace
- timeout-minutes: 90 to cover multi-iter runs
Phase B (k8s Job + GPU isolation) tracked in jepa-fx-risk#15.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- fixtures/phase-a-toy.json — scaffold source for `autoresearch start`
Phase A live run: val_vol_r2 metric, daily EUR/USD, HEPA encoder toy
- .gitignore: add runs/ (ephemeral run dirs), metrics.json, embeddings.json,
HEARTBEAT, STATUS.md, pyc/__pycache__/.pytest_cache, eval binary
- untrack metrics.json (was a stale committed sample, now gitignored)
Launch on koala:
python scripts/autoresearch_start.py fixtures/phase-a-toy.json phase-a-toy
op run -- env LITELLM_KEY="$LITELLM_KEY" \
python loop.py --run-dir runs/phase-a-toy --iters 3
Closes the last gate on jepa-fx-risk#11 Phase A (code shipped v1.4.0;
live run pending).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Closes jepa-fx-risk#11 Phase A.
- scripts/autoresearch_start.py: scaffold runs/<rq-id>/ from Council backlog leaf;
fail-closed on non-autoresearch-ready; strips candidate_metric; writes program.md
+ run.json (provenance) + train.py copy. 19 TDD tests.
- loop.py: --run-dir flag redirects STATUS.md / metrics.json / HEARTBEAT / train.py
into the run dir; METRICS_OUT env var passed to train subprocess so it writes
metrics.json to the run dir; heartbeat file written each iter phase; ntfy-on-crash
via NTFY_URL env var (best-effort).
- train.py: METRICS_OUT env var overrides metrics.json path (default unchanged).
Launch: LITELLM_KEY=xxx python loop.py --run-dir runs/rq-04
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two-phase phase-1:
1a. Frozen warmup: head trains on pre-computed embeddings for PHASE1_EPOCHS=200
1b. Joint fine-tune: encoder + head for PHASE1_JOINT_EPOCHS=30 at PHASE1_ENCODER_LR=3e-6
Key design decisions:
- Warm start prevents catastrophic forgetting (PHASE1_JOINT=1 cold-start → -32 R²)
- Normalize live encoder output with FROZEN stats (mu_e/sd_e) so head sees same
embedding distribution it was warmed up on
- head LR reduced 10× in joint phase to prevent head from racing ahead
HPO sweep: 30ep@3e-6=0.3962, 30ep@1e-5=0.3930, 50ep@3e-6=0.3923
Baseline (frozen): 0.3908. New best: phase1_r2=0.3962 (+0.0054 OOS).
New knobs: JEPA_PHASE1_JOINT (default 1), JEPA_PHASE1_JOINT_EPOCHS (default 30),
JEPA_PHASE1_ENCODER_LR (default 3e-6). 4 new tests (tests 15-18). 28/28 pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- prepare_hourly.py: keep O/H/L columns from M1 zips; compute per-hour
hl_range=log(H/L) and ret_intrabar=log(close/open); backward-compat
(falls back to 4-col output only when O/H/L present in input)
- train.py build(): auto-detect extra features from parquet columns
(FEAT_COLS = [ret, realized_vol] + [hl_range, ret_intrabar] if present)
- 5 new tests (9 total in test_prepare_hourly); 24/24 pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- train.py knobs all readable from JEPA_* env vars (JEPA_WINDOW, JEPA_D_MODEL,
JEPA_DEPTH, etc.) so hpo_sweep.py can override without touching source
- scripts/hpo_sweep.py: 3×2×3 grid over D_MODEL × DEPTH × WINDOW,
logs to results/hpo/hpo_results.jsonl with leaderboard at end
- 3 new tests: env override correctness, configs() schema validation
- 19/19 tests pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Go harness reported 0.42 vs Python 0.36 because export used realized_vol[t]
(current) while Python probe used realized_vol[t+1] (next-period). Fix adds
t+1 < len(df2) guard and uses iloc[t+1] as target. Go now matches Python: 0.3585.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
BATCH_SIZE=512 per step; batched embed() at eval + export time.
78k hourly windows can't fit in GPU in one shot (was fine at 877 daily).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- train.py build(): year-based split (train≤2021, OOS≥2022) replaces
misleading 70/30 mixed-period split; true OOS val_vol_r2 now ~-0.36
vs previously reported +0.18 (artefact of cross-period data leakage)
- train.py: EXPORT_EMBEDDINGS block now exports both train+OOS embeddings
with dates and HV labels for Go eval harness
- cmd/eval: LinearProbeTrainTest uses train stats for standardisation of
both sets (no leakage); standardiseCompute/applyStandardise helpers
- internal/eval: add LinearProbeTrainTest (fit-on-train, eval-on-OOS)
alongside LinearProbe (same-set); 8/8 tests still green
Phase-0 gate result: val_vol_r2=-0.36, silhouette=0.043, erank=58.9/64.
Backbone produces high-rank embeddings (SIGReg working) but does NOT
generalize across 2021→2022 regime boundary. Gate: INCONCLUSIVE/FAIL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase-0 SSL feasibility gate run on daily 2019-2023 EUR/USD (path B deviation:
not hourly 2008-2022 + Go harness as specced in #5). Results:
TS-JEPA silhouette mean=0.018 (need >0.20) — FAIL
PCA baseline silhouette=0.136 — also below threshold
sensitivity: 2000 ep + D=64 worsened to 0.004 (not a training-time issue)
Root cause: 2 daily features (ret, realized_vol) carry minimal regime structure
at this resolution. The JEPA objective with SIGReg pushes embeddings toward
isotropic Gaussian — good for downstream probes (val_vol_r2>0) but may actively
resist the clustering structure the silhouette gate measures.
Null protocol: real gate requires #4 (Go harness) + #2 (hourly data, more
features) before rerunning. HEPA (#14) noted as alternative backbone.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All 5 kept: val_vol_r2 -0.1543 → +0.0599 (+0.214 total). Backbone learning.
Agent tuning: LR, depth, SIGREG_LAM, EPOCHS. Still well below toy ceiling
(0.37) — real backbone room to grow via #3/#4/#5.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PatchTST-style transformer encoder with JEPA predictive loss + SIGReg
regularization (Balestriero & LeCun arXiv:2511.08544; time-series placement
from ChronoJEPA). Token-level SIGReg (dual placement) to avoid time-axis
collapse (confirmed real by ChronoJEPA). Baseline val_vol_r2=-0.1543 on first
run — expected for fresh weights with new architecture. Agent will iterate.
SIGReg source: Epps-Pulley statistic, identical math to LeJEPA MINIMAL.md.
Refs: #3 (TS-JEPA reproduce), ChronoJEPA github.com/MrRobotop/ChronoJEPA
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Toy encoder near ceiling. 1 kept (val_vol_r2 0.3032→0.3442), 5 reverts.
Consistent plateau = time to swap in TS-JEPA backbone (#3/#5).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fetch_data.py politely pulls EUR/USD M1 from histdata.com (maintained package
handles the anti-hotlink token; per-year, spaced). prepare_data.py (LOCKED per
Phase-1 contract) parses M1 -> daily series with realized_vol = the val_vol_r2
target (sqrt sum of squared intraday returns). Verified on 2019-2021: 937 days,
March-2020 COVID RV spike 5.1x over 2019 median — real signal, target works.
Data gitignored (DVC/MinIO = #10).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/check_gpu.py verifies PyTorch cu130 sees the koala Blackwell GPU
(sm_120) and computes — the Phase-0 prerequisite before any autoresearch
experiment. Verified green: torch 2.12.1+cu130, RTX 5070, GPU matmul OK.
Note: koala GPU is shared with the llama-swap LLM stack — run the autoresearch
agent on iguana/berget so the card stays free for train.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mathiasandClaude Opus 5.5