10 Commits
Author SHA1 Message Date
mathias 2c9bc1be31 fix(ci): drop Build & Import / Deploy via GitOps jobs (jepa-fx-risk#19)
CD / Lint / Test / Vet (push) Successful in 5s
This is a research repo -- issue#6/#7 already decided CI should be
check-only (`task check` on push/PR), no build, no image push, no
deploy, since training runs are manual/GPU-bound and never CI-triggered.
That never actually landed in cd.yml: it still carried the original
go-web template's build+deploy jobs verbatim, still referencing the
pre-rename `hostexecutor` name (`./cmd/hostexecutor` doesn't exist --
real dirs are cmd/eval, cmd/jepa-fx-risk) and a `k3s/apps/hostexecutor`
deployment that was never created in infra (confirmed: no manifest
exists under either name). Once the check job's own Taskfile bug was
fixed, build started actually running and failing on the wrong path.
Removed both jobs -- check is the only gate this repo needs.
2026-07-24 15:28:46 +02:00
mathias 308f71b566 fix(ci): Taskfile YAML syntax error + staticcheck tagged-switch (jepa-fx-risk#19)
CD / Build & Import (push) Failing after 10s
CD / Deploy via GitOps (push) Has been skipped
CD / Lint / Test / Vet (push) Successful in 7s
Taskfile.yml line 46 had an unquoted Go-template `{{.VAR}}` inside a YAML
flow sequence (`cmds: [...]`) -- the literal braces broke YAML parsing
outright ("did not find expected ',' or ']'"), so `task check` (and thus
CI's push-triggered check job) failed before running a single command.
A manual `workflow_dispatch` re-run passed because the autoresearch
workflow never calls `task check` at all -- unrelated path, not an
env/secret difference as first suspected. Quoted the string.

Also fixed a staticcheck QF1002 in internal/eval/var.go: a boolean
switch comparing the same variable (n1) in every case is a tagged
switch in disguise -- converted to `switch n1 { case 0: ... case n: ...
}`.
2026-07-24 15:26:01 +02:00
mathias 6fb77f5263 fix(ci): drop empty LOOP_MODEL env that blanked the model → LLM 400
CD / Lint / Test / Vet (push) Failing after 3s
CD / Deploy via GitOps (push) Has been skipped
CD / Build & Import (push) Has been skipped
`LOOP_MODEL: ${{ inputs.model }}` sets the env var to "" when no model input is
given. An env var set-but-empty is NOT unset, so it overrode loop.py's
berget/gemma4-31b default, and every agent call POSTed "model":"" → HTTP 400
Bad Request. The loop ran but optimized nothing (best == baseline, delta 0).

Real overrides already flow through the conditional `--model` arg, so this env
line was redundant as well as harmful. Remove it; loop.py's default applies.
2026-06-29 19:34:57 +00:00
mathias 8eb0358c01 fix(loop): put project root on PYTHONPATH for train.py subprocess
CD / Lint / Test / Vet (push) Failing after 2s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
train.py is copied into runs/<rq-id>/ by autoresearch_start.py, so when loop.py
executes it, sys.path[0] is the run dir — which has no scripts/. train.py's
frozen VaR-eval block does `from scripts.var_breach import ...`, which then
fails with ModuleNotFoundError: No module named 'scripts' on every loop run
(CI and the documented manual launch alike).

Fix in the harness, not the frozen train.py/scripts boundary: prepend the
project root (loop.py's own dir, where scripts/ lives) to the subprocess
PYTHONPATH. Verified red→green locally: the import fails without it and
resolves with it (scripts/ is an implicit namespace package, no __init__.py).
2026-06-29 21:32:25 +02:00
mathias 2d412790bf chore(ci): commit 59KB toy eurusd_daily.parquet as CI data fixture
CD / Lint / Test / Vet (push) Failing after 3s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
The Autoresearch Loop workflow (phase-a-toy) runs train.py, which reads
data/processed/eurusd_daily.parquet. data/ is gitignored (DVC/MinIO policy, #10),
but CI checks out a fresh tree with no data, so the baseline run died with
FileNotFoundError. Force-add this single 59KB derived fixture (public EUR/USD
daily realized vol, 2019-2023, 1560 rows) so the toy loop is hermetic and fast.

This is a scoped exception to #10 for a CI fixture only — large/production
datasets still stay out of git per that policy.
2026-06-29 21:20:41 +02:00
mathias 8539ec3a85 fix(ci): install torch from cu130 index in venv step
CD / Lint / Test / Vet (push) Failing after 2s
CD / Deploy via GitOps (push) Has been skipped
CD / Build & Import (push) Has been skipped
train.py imports torch but the venv step only installed requirements.txt,
which deliberately excludes torch (must come from the cu130 wheel index for
koala's Blackwell sm_120, per the requirements.txt header). Baseline run died
with ModuleNotFoundError: No module named 'torch'. Add the documented install.
2026-06-29 19:14:51 +00:00
mathias c9b22db3b8 fix(ci): pin upload-artifact to @v3 — v4 artifact protocol unsupported on Gitea
CD / Lint / Test / Vet (push) Failing after 2s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
Gitea's act_runner does not implement the @actions/artifact v2.0.0+ backend, so
upload-artifact@v4 fails the job with GHESNotSupportedError even when the loop
succeeds. v3 is the newest tag the Gitea runner supports.
2026-06-29 18:43:12 +00:00
mathias 731673061e fix(deps): histdata>=1.3 is unsatisfiable, pin to >=1.1 (newest on PyPI)
CD / Lint / Test / Vet (push) Failing after 3s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
The Autoresearch Loop workflow failed at "Set up Python venv" on every run:
  ERROR: Could not find a version that satisfies the requirement histdata>=1.3
          (from versions: 1.0, 1.1)
histdata never published 1.3 — 1.1 is the newest release. Lower the floor so
pip can resolve and the loop can actually start.
2026-06-29 18:41:07 +00:00
mathiasandClaude Sonnet 4.6 54b1bc216d feat(ci): Gitea Actions workflow for autoresearch loop (Phase A automation)
CD / Lint / Test / Vet (push) Failing after 3s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
workflow_dispatch with inputs: fixture, rq_id, iters, model override.
- Scaffolds run dir from fixtures/<fixture>.json via autoresearch_start.py
- Runs loop.py with LITELLM_KEY/LITELLM_BASE/NTFY_URL secrets
- Uploads STATUS.md + metrics.json + program.md as artifacts (30-day retention)
- Cleans stale run dir at start; venv created/reused per runner workspace
- timeout-minutes: 90 to cover multi-iter runs

Phase B (k8s Job + GPU isolation) tracked in jepa-fx-risk#15.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 17:20:44 +02:00
mathiasandClaude Sonnet 4.6 fa6dcaae0a chore: fixture backlog for Phase A toy run + gitignore cleanup
CD / Lint / Test / Vet (push) Failing after 3s
CD / Build & Import (push) Has been skipped
CD / Deploy via GitOps (push) Has been skipped
- fixtures/phase-a-toy.json — scaffold source for `autoresearch start`
  Phase A live run: val_vol_r2 metric, daily EUR/USD, HEPA encoder toy
- .gitignore: add runs/ (ephemeral run dirs), metrics.json, embeddings.json,
  HEARTBEAT, STATUS.md, pyc/__pycache__/.pytest_cache, eval binary
- untrack metrics.json (was a stale committed sample, now gitignored)

Launch on koala:
  python scripts/autoresearch_start.py fixtures/phase-a-toy.json phase-a-toy
  op run -- env LITELLM_KEY="$LITELLM_KEY" \
    python loop.py --run-dir runs/phase-a-toy --iters 3

Closes the last gate on jepa-fx-risk#11 Phase A (code shipped v1.4.0;
live run pending).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-27 17:03:55 +02:00
10 changed files with 112 additions and 104 deletions
+75
View File
@@ -0,0 +1,75 @@
name: Autoresearch Loop
on:
workflow_dispatch:
inputs:
fixture:
description: 'Fixture name in fixtures/ (without .json)'
required: true
default: 'phase-a-toy'
rq_id:
description: 'Run ID — defaults to fixture name if blank'
required: false
default: ''
iters:
description: 'Max iterations'
required: false
default: '3'
model:
description: 'LiteLLM model override (leave blank for default berget/gemma4-31b)'
required: false
default: ''
jobs:
run:
name: Autoresearch — ${{ inputs.fixture }}
runs-on: self-hosted
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Resolve run ID
id: vars
run: |
RQ_ID="${{ inputs.rq_id }}"
[ -z "$RQ_ID" ] && RQ_ID="${{ inputs.fixture }}"
echo "rq_id=$RQ_ID" >> "$GITHUB_OUTPUT"
- name: Clean stale run dir
run: rm -rf "runs/${{ steps.vars.outputs.rq_id }}"
- name: Set up Python venv
run: |
[ -d .venv ] || python3 -m venv .venv
# torch must come from the cu130 wheel index (koala Blackwell sm_120);
# requirements.txt deliberately excludes it. Install it first.
.venv/bin/pip install -q torch --index-url https://download.pytorch.org/whl/cu130
.venv/bin/pip install -q -r requirements.txt
- name: Scaffold run dir
run: |
.venv/bin/python scripts/autoresearch_start.py \
"fixtures/${{ inputs.fixture }}.json" \
"${{ steps.vars.outputs.rq_id }}"
- name: Run autoresearch loop
env:
LITELLM_KEY: ${{ secrets.LITELLM_KEY }}
LITELLM_BASE: ${{ secrets.LITELLM_BASE }}
NTFY_URL: ${{ secrets.NTFY_URL }}
run: |
ARGS="--run-dir runs/${{ steps.vars.outputs.rq_id }} --iters ${{ inputs.iters }}"
[ -n "${{ inputs.model }}" ] && ARGS="$ARGS --model ${{ inputs.model }}"
.venv/bin/python loop.py $ARGS
- name: Upload run artifacts
if: always()
uses: https://gitea.com/actions/upload-artifact@v3
with:
name: run-${{ steps.vars.outputs.rq_id }}-${{ github.run_number }}
path: |
runs/${{ steps.vars.outputs.rq_id }}/STATUS.md
runs/${{ steps.vars.outputs.rq_id }}/metrics.json
runs/${{ steps.vars.outputs.rq_id }}/program.md
retention-days: 30
-85
View File
@@ -7,9 +7,6 @@ on:
pull_request:
branches: [main]
env:
IMAGE: hostexecutor
jobs:
check:
name: Lint / Test / Vet
@@ -31,85 +28,3 @@ jobs:
- name: Run checks
run: task check
build:
name: Build & Import
needs: check
runs-on: self-hosted
if: github.event_name != 'pull_request'
outputs:
image-tag: ${{ steps.meta.outputs.sha-tag }}
steps:
- uses: actions/checkout@v4
- name: Derive image tags
id: meta
run: |
SHA=$(git rev-parse --short HEAD)
echo "sha-tag=${SHA}" >> "$GITHUB_OUTPUT"
- name: Build and push to local registry
run: |
REGISTRY="localhost:5000"
REF="${REGISTRY}/${{ env.IMAGE }}:${{ steps.meta.outputs.sha-tag }}"
buildah build \
--label "org.opencontainers.image.revision=${{ github.sha }}" \
-t ${REF} \
-t ${REGISTRY}/${{ env.IMAGE }}:latest \
.
buildah push --tls-verify=false ${REF}
buildah push --tls-verify=false ${REGISTRY}/${{ env.IMAGE }}:latest
echo "✓ Image pushed to ${REF}"
deploy:
name: Deploy via GitOps
needs: build
runs-on: self-hosted
if: github.ref == 'refs/heads/main' && github.event_name == 'push'
steps:
- name: Update image tag in infra repo
env:
IMAGE_TAG: ${{ needs.build.outputs.image-tag }}
DEPLOY_KEY: ${{ secrets.INFRA_DEPLOY_KEY }}
run: |
set -euo pipefail
mkdir -p ~/.ssh
echo "$DEPLOY_KEY" > ~/.ssh/id_infra
chmod 600 ~/.ssh/id_infra
ssh-keyscan -p 30022 10.0.1.20 >> ~/.ssh/known_hosts 2>/dev/null
export GIT_SSH_COMMAND="ssh -i ~/.ssh/id_infra -o IdentitiesOnly=yes"
rm -rf /tmp/infra
git clone -b main ssh://git@10.0.1.20:30022/mathias/infra.git /tmp/infra
cd /tmp/infra
DEPLOYMENT="k3s/apps/hostexecutor/deployment.yaml"
sed -i "s|image: localhost:5000/hostexecutor:.*|image: localhost:5000/hostexecutor:${IMAGE_TAG}|" "$DEPLOYMENT"
grep -q "localhost:5000/hostexecutor:${IMAGE_TAG}" "$DEPLOYMENT" \
|| { echo "✗ image tag patch failed"; exit 1; }
if git diff --quiet "$DEPLOYMENT"; then
echo " image tag unchanged — skipping push"
else
git -c user.name="hostexecutor CI" \
-c user.email="ci@hostexecutor.local" \
commit -m "chore(deploy): hostexecutor → ${IMAGE_TAG}" "$DEPLOYMENT"
git push origin main
echo "✓ pushed to infra repo"
fi
shred -u ~/.ssh/id_infra
- name: Trigger Flux reconcile
run: |
kubectl -n flux-system annotate gitrepository flux-system \
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
kubectl -n flux-system annotate kustomization apps \
reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite
- name: Verify rollout
run: |
kubectl rollout status deployment/hostexecutor \
--namespace hostexecutor \
--timeout=120s \
|| {
kubectl get pods -n hostexecutor -o wide
kubectl get events -n hostexecutor --sort-by='.lastTimestamp' | tail -20
exit 1
}
+17
View File
@@ -34,3 +34,20 @@ bin/
# downloaded + processed market data (track via DVC/MinIO, #10 — not git)
data/
# autoresearch run dirs (ephemeral; each scaffold rebuilds from fixtures/)
runs/
# ephemeral experiment outputs (generated by train.py / loop.py)
metrics.json
embeddings.json
HEARTBEAT
STATUS.md
# python caches
__pycache__/
*.pyc
.pytest_cache/
# built Go binaries
eval
+1 -1
View File
@@ -43,7 +43,7 @@ tasks:
cmds: [.venv/bin/python scripts/fetch_multipair.py]
data:prepare:pair:
desc: "Build {PAIR}_hourly.parquet from data/raw/{PAIR}/ (e.g. PAIR=gbpusd)"
cmds: [PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}]
cmds: ["PAIR={{.PAIR}} .venv/bin/python scripts/prepare_hourly.py {{.EXTRA_ARGS}}"]
vars:
PAIR: '{{default "eurusd" .PAIR}}'
data:prepare:multipair:
Binary file not shown.
+11
View File
@@ -0,0 +1,11 @@
{
"strategic_question": "What is the highest-leverage path to a JEPA-based FX tail-risk system that beats a GARCH/EWMA baseline on out-of-sample VaR-breach calibration, given one GPU and a solo researcher?",
"nodes": [
{
"id": "phase-a-toy",
"status": "autoresearch-ready",
"question": "Improve the OOS linear-probe R² (val_vol_r2) of the HEPA encoder on EUR/USD daily realized vol. The encoder is a small causal transformer trained with VICReg. Vary one hyperparameter or architectural choice per iteration — model size, learning rate, window, patch length, depth, VICReg loss weights — to push val_vol_r2 as high as possible on the 2022-2023 OOS slice.",
"candidate_metric": "val_vol_r2"
}
]
}
+3 -3
View File
@@ -40,11 +40,11 @@ func kupiecPOF(n, n1 int, p0 float64) float64 {
phat := float64(n1) / float64(n)
var lr float64
switch {
case n1 == 0:
switch n1 {
case 0:
// 0 × ln(0/p0) = 0 by convention; only the n0 term contributes
lr = 2 * float64(n0) * math.Log((1-phat)/(1-p0))
case n1 == n:
case n:
// n0 term vanishes
lr = 2 * float64(n1) * math.Log(phat/p0)
default:
+4
View File
@@ -84,6 +84,10 @@ def run_train() -> "tuple[float | None, float, str]":
t0 = time.time()
env = dict(os.environ)
env["METRICS_OUT"] = str(METRICS_JSON.resolve())
# train.py is copied into the run dir, so sys.path[0] is that run dir — which
# has no scripts/. Put the project root (where loop.py + scripts/ live) on
# PYTHONPATH so train.py's `from scripts.var_breach import ...` resolves.
env["PYTHONPATH"] = str(Path(__file__).resolve().parent) + os.pathsep + env.get("PYTHONPATH", "")
try:
r = subprocess.run(
[sys.executable, str(TRAIN_PY.resolve())],
-14
View File
@@ -1,14 +0,0 @@
{
"val_vol_r2": 0.3641397896593044,
"phase1_r2": 0.3908407688140869,
"n_test": 11641,
"knobs": {
"WINDOW": 120,
"PATCH_LEN": 24,
"D_MODEL": 128,
"DEPTH": 2,
"ALPHA": 0.1,
"DELTA_T_MAX": 3,
"EPOCHS": 300
}
}
+1 -1
View File
@@ -5,6 +5,6 @@
numpy>=2.0
pandas>=2.2
pyarrow>=16
histdata>=1.3 # histdata.com downloader (handles the tk token politely)
histdata>=1.1 # histdata.com downloader (1.1 is newest on PyPI; 1.3 never existed)
hmmlearn>=0.3 # regime detector (prepare_regime.py, jepa-fx-risk#13)
scikit-learn>=1.4 # HMM dependency