Author SHA1 Message Date
mathias 5ff79891b5 feat: add experiment-spec skill 2026-05-27 21:58:50 +00:00
mathias 5a2bef35c1 fix(ci): runs-on self-hosted to match koala runner labels
release / tag (push) Successful in 1s
The koala act_runner has labels [self-hosted, linux, amd64]. The
default ubuntu-latest label doesn't match, so the release workflow
silently produced zero runs on every push. Matches the convention
used by hyperguild + infra workflows.

Runner is offline at time of this commit — bring it up on koala via
the act_runner systemd unit before expecting tags to appear.

[skip-release]
2026-05-24 20:52:26 +02:00
mathias b1ff71a7fd feat: wire Crush + Antigravity harnesses
release / tag (push) Has been cancelled
Phase 2 of mathias/skills extraction. Adds two new per-tool wirers
alongside Claude Code:

- Crush — ~/.config/crush/skills/<name>/SKILL.md (charmbracelet/crush,
  successor to opencode-ai/opencode which is archived)
- Antigravity — ~/.gemini/antigravity/skills/<name>/SKILL.md (Google
  VS Code extension). Our SKILL.md frontmatter (name + description)
  is already in the format Antigravity expects — no manifest
  translation step needed.

Both wirers added to Taskfile.yml (install:crush + install:antigravity)
and install.sh (wire_crush + wire_antigravity). The aggregate `install`
target now calls all four targets (claude:global, claude:repo, crush,
antigravity). Idempotent symlinks, same shape as the Claude Code wirer.

Per-host env overrides documented in README: SKILLS_REF for tag pinning,
CRUSH_SKILLS_DIR + ANTIGRAVITY_SKILLS_DIR for non-default paths.

Skipped:
- opencode (opencode-ai/opencode): archived, succeeded by Crush; its
  Custom Commands surface is user prompt templates, not system skills
- gitea-resident agents: consume via per-repo .claude/skills or brain
  MCP, no special target needed

Bump-Type: minor
2026-05-24 20:51:06 +02:00
5 changed files with 296 additions and 11 deletions
+1 -1
View File
@@ -17,7 +17,7 @@ on:
jobs:
tag:
runs-on: ubuntu-latest
runs-on: self-hosted
steps:
- name: checkout
uses: actions/checkout@v4
+29 -7
View File
@@ -44,15 +44,37 @@ changes. CI tags every change as a `vX.Y.Z` release (see
## Wired harnesses
Phase 1 (current) wires **Claude Code** only:
| Harness | Target path | Notes |
|---|---|---|
| **Claude Code (global)** | `~/.claude/skills/<name>` | Visible in every Claude Code session on the host. |
| **Claude Code (per-repo)** | `<repo>/.claude/skills/<name>` | Created when `install` is invoked inside a git repo (not the skills repo itself). Gitignored globally via `**/.claude/skills/`. |
| **Crush** | `~/.config/crush/skills/<name>` | Charmbracelet Crush — successor to `opencode-ai/opencode` (archived). |
| **Antigravity** | `~/.gemini/antigravity/skills/<name>` | Google Antigravity (VS Code extension). Our `SKILL.md` frontmatter (`name` + `description`) is already in the format Antigravity expects, so symlinks suffice — no manifest translation. |
- Global: `~/.claude/skills/<name>``~/.local/share/skills/<name>`
- Per-repo: `<repo>/.claude/skills/<name>` (when invoked from a git
repo) → same
### Not wired (and why)
Phase 2 will add: Crush, opencode, antigravity, gitea-resident agents
(cobalt-dingo, agentsquad). See `mathias/infra` issue `infra#62`
addendum for the roadmap.
- **opencode** (`opencode-ai/opencode`): archive notice; succeeded by Crush.
Its only "skill"-like surface — Custom Commands at `~/.config/opencode/commands/`
is for user-facing prompt templates, not system-level instructions.
Misaligned with how we use skills. Skipped.
- **gitea-resident agents** (cobalt-dingo, agentsquad): they consume
skills via their containing project's `.claude/skills` directory
(populated by the per-repo wirer when run from the host), or via the
brain MCP. No special target needed.
### Per-host env-var overrides
Each target path can be overridden by setting the matching env var before
running `install.sh`:
| Env var | Default |
|---|---|
| `SKILLS_REPO_URL` | `https://gitea.d-ma.be/mathias/skills.git` |
| `SKILLS_REF` | `main` (set to e.g. `v0.1.0` to pin a release) |
| `SKILLS_CHECKOUT_DIR` | `$HOME/.local/share/skills` |
| `CLAUDE_SKILLS_DIR` | `$HOME/.claude/skills` |
| `CRUSH_SKILLS_DIR` | `$HOME/.config/crush/skills` |
| `ANTIGRAVITY_SKILLS_DIR` | `$HOME/.gemini/antigravity/skills` |
## Versioning
+48 -3
View File
@@ -12,6 +12,11 @@ vars:
REPO_URL: 'https://gitea.d-ma.be/mathias/skills.git'
CHECKOUT_DIR: '{{.HOME}}/.local/share/skills'
CLAUDE_GLOBAL_DIR: '{{.HOME}}/.claude/skills'
CRUSH_DIR: '{{.HOME}}/.config/crush/skills'
ANTIGRAVITY_DIR: '{{.HOME}}/.gemini/antigravity/skills'
# Anything at the repo root that is NOT a skill directory. Both
# Taskfile + install.sh share this exclusion regex.
NON_SKILL_ENTRIES: '^(Taskfile.yml|install.sh|README.md|SKILLS_INDEX.md|.gitea|.git$)$'
tasks:
@@ -26,17 +31,19 @@ tasks:
silent: true
install:
desc: 'Wire skills into every harness detected on this host (Claude Code today, more in phase 2).'
desc: 'Wire skills into every supported harness (Claude Code, Crush, Antigravity).'
cmds:
- task: install:claude:global
- task: install:claude:repo
- task: install:crush
- task: install:antigravity
install:claude:global:
desc: 'Per-skill symlinks under ~/.claude/skills/<name> — visible in every Claude Code project on this host.'
cmds:
- mkdir -p {{.CLAUDE_GLOBAL_DIR}}
- |
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "^(Taskfile.yml|install.sh|README.md|SKILLS_INDEX.md|.gitea|.git$)$"); do
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "{{.NON_SKILL_ENTRIES}}"); do
target="{{.TASKFILE_DIR}}/${skill}"
link="{{.CLAUDE_GLOBAL_DIR}}/${skill}"
if [ -L "$link" ] || [ -e "$link" ]; then
@@ -63,7 +70,7 @@ tasks:
fi
target_dir="$repo/.claude/skills"
mkdir -p "$target_dir"
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "^(Taskfile.yml|install.sh|README.md|SKILLS_INDEX.md|.gitea|.git$)$"); do
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "{{.NON_SKILL_ENTRIES}}"); do
target="{{.TASKFILE_DIR}}/${skill}"
link="${target_dir}/${skill}"
if [ -L "$link" ] || [ -e "$link" ]; then
@@ -77,6 +84,44 @@ tasks:
echo "linked $link → $target"
done
install:crush:
desc: 'Per-skill symlinks under ~/.config/crush/skills/<name> — visible in every Crush session on this host.'
cmds:
- mkdir -p {{.CRUSH_DIR}}
- |
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "{{.NON_SKILL_ENTRIES}}"); do
target="{{.TASKFILE_DIR}}/${skill}"
link="{{.CRUSH_DIR}}/${skill}"
if [ -L "$link" ] || [ -e "$link" ]; then
current=$(readlink "$link" 2>/dev/null || true)
if [ "$current" = "$target" ]; then
continue
fi
rm -rf "$link"
fi
ln -s "$target" "$link"
echo "linked $link → $target"
done
install:antigravity:
desc: 'Per-skill symlinks under ~/.gemini/antigravity/skills/<name>. SKILL.md frontmatter (name + description) is already antigravity-compatible.'
cmds:
- mkdir -p {{.ANTIGRAVITY_DIR}}
- |
for skill in $(ls -1 "{{.TASKFILE_DIR}}" | grep -Ev "{{.NON_SKILL_ENTRIES}}"); do
target="{{.TASKFILE_DIR}}/${skill}"
link="{{.ANTIGRAVITY_DIR}}/${skill}"
if [ -L "$link" ] || [ -e "$link" ]; then
current=$(readlink "$link" 2>/dev/null || true)
if [ "$current" = "$target" ]; then
continue
fi
rm -rf "$link"
fi
ln -s "$target" "$link"
echo "linked $link → $target"
done
update:
desc: 'git pull the canonical checkout then re-run install.'
cmds:
+192
View File
@@ -0,0 +1,192 @@
---
name: experiment-spec
description: Write a rigorous experiment spec for a research phase before any code or training runs. Use instead of feature-spec for scientific/ML research projects. Enforces falsifiable hypothesis, quantitative acceptance criteria, baseline comparison, and null-result protocol.
---
# Experiment Spec
## Overview
An experiment spec is the scientific contract for one research phase or experiment, written before any implementation or training begins. It is the research analogue of `feature-spec` — same discipline, different vocabulary.
**Core principle:** If you cannot write a falsifiable hypothesis with a quantitative acceptance criterion, you do not understand the experiment well enough to run it.
## When to Use
- Starting a new research phase (Phase 0, Phase 1, etc.)
- Running any experiment that will produce metrics used to make a go/no-go decision
- Any time the question is "does X work?" rather than "build X"
- Before touching training code, data, or hyperparameters for a new question
**When NOT to use:**
- Implementing a specific component whose behaviour is already defined by a phase spec (use `feature-spec` instead)
- Exploratory data analysis with no hypothesis (use a notebook; note it as EDA)
- Bug fixes or refactors
## Iron Laws
1. **The hypothesis must be falsifiable.** "JEPA is promising" is not a hypothesis. "JEPA embeddings will achieve silhouette > 0.35 on held-out data" is. If you cannot state conditions under which the hypothesis is false, it is not a hypothesis.
2. **Acceptance criteria must be quantitative and pre-registered.** Write the number before you run the experiment. Moving the goalposts after seeing results is p-hacking.
3. **A baseline is mandatory.** Every experiment must compare against at least one simpler baseline. "Better than nothing" is not a baseline.
4. **A null-result protocol is mandatory.** State what you will conclude and do if the hypothesis is rejected. "Try harder" is not a protocol.
5. **The training cutoff is sacred.** No post-cutoff data informs any decision in the spec or implementation.
## Spec Template
```markdown
# Experiment Spec: [Phase N — Short Name]
## Hypothesis
> "[Falsifiable claim]: We believe [X] will produce [Y], measurable by [Z]."
State conditions under which this hypothesis is FALSE.
## Background
Why this experiment? What does it build on? What prior result or decision motivates it?
(24 sentences. Reference DECISIONS.md or brain wing entries where relevant.)
## Design
### Data
- Source, date range, pairs/assets, features used
- Train / validation / test split (respect training cutoff)
### Model / method
- Architecture, configuration, key hyperparameters
- What is being varied vs. held fixed
### Baseline
- What simpler method is being compared against?
- Why is this the right baseline?
### Ablations (if any)
- What variants will be run to isolate the effect being studied?
## Acceptance Criteria
- [ ] [Primary criterion — quantitative threshold on primary metric]
- [ ] [Baseline comparison — e.g. "exceeds baseline by >X%"]
- [ ] [Reproducibility — reruns within ±Y% of reported metric]
- [ ] [Collapse/sanity check — e.g. "PC1/rolling-HV correlation < 0.85"]
## Out of Scope
What this experiment explicitly does NOT answer, even if related.
Anything plausibly in scope that is deferred goes here.
## Null Result Protocol
If the primary acceptance criterion is NOT met:
- What do we conclude?
- What is the next step? (Investigate X, pivot to Y, terminate programme)
- What gets written to the brain and results/summaries/?
## Risks
What could go wrong, and how would it be detected?
At least one risk must be listed.
```
## Worked Example
```markdown
# Experiment Spec: Phase 0 — SSL Feasibility Gate
## Hypothesis
> We believe that a masked autoencoder (MAE) trained on FX hourly data will produce
> latent embeddings that show structural separability by volatility regime without
> explicit regime labels, measurable by silhouette score > 0.20 on held-out 2023 data.
This hypothesis is FALSE if silhouette score ≤ 0.20 on the held-out evaluation.
## Background
Before investing in JEPA-specific machinery, we need to confirm that SSL-based
representation learning can find regime structure in FX time-series at all. MAE is
the simplest SSL baseline — if it cannot find structure, JEPA will not either.
Added post Full Grill (2026-05-27). See DECISIONS.md: "Phase 0: SSL feasibility gate".
## Design
### Data
- Source: DUKASCopy, EUR/USD hourly, 20082022 (train), 2023 (held-out test)
- Features: log-return, rolling 20-period HV, VIX (daily interpolated to hourly)
- Regime label (for evaluation only, not training): rolling 30-day HV percentile,
binary high/low threshold at 50th percentile
### Model
- Masked Autoencoder: 1D temporal masking (mask contiguous 24h window)
- Encoder: 3-layer 1D CNN + positional encoding
- Decoder: 2-layer MLP reconstructing masked segment
- Context window: 120 hours (5 days)
### Baseline
- PCA on raw feature vectors (same window) — tests whether any dimensionality
reduction shows regime structure, not just SSL
### Ablations
- Masking horizon: K ∈ {8h, 24h, 72h} — does horizon affect embedding quality?
## Acceptance Criteria
- [ ] Silhouette score > 0.20 on held-out 2023 data (k-means, k=3, vs. HV regime label)
- [ ] MAE silhouette exceeds PCA baseline silhouette
- [ ] Rerun within ±10% of reported silhouette
- [ ] PC1 / rolling-HV correlation < 0.95 (not purely encoding volatility level)
## Out of Scope
- JEPA implementation (Phase 1)
- Multi-pair training (Phase 1+)
- VaR or ES computation
- Any use of post-2023 data
## Null Result Protocol
If silhouette ≤ 0.20:
- Conclude: SSL cannot reliably find regime structure in EUR/USD hourly data with
these features at this resolution
- Next step: investigate whether (a) hourly resolution is too noisy (try daily),
(b) 3 features are insufficient, or (c) regime label definition is too coarse
- Record result in results/summaries/phase-0-null.md and brain wing jepa-fx/failures/
## Risks
- Encoder collapses to near-constant output: detect via reconstruction loss plateau
in first 10 epochs; mitigation: add batch norm, reduce learning rate
- Regime label too coarse (binary HV): silhouette may be low even with good structure;
mitigation: also evaluate with 4-class label (HV quartiles)
```
## Common Failure Modes
| Failure mode | What it looks like | Fix |
|---|---|---|
| Non-falsifiable hypothesis | "JEPA shows promise" | Rewrite with a number |
| Post-hoc criteria | Threshold chosen after seeing results | Write the number first, commit the spec |
| No baseline | Silhouette of 0.30 sounds good until PCA achieves 0.35 | Always include a dumber method |
| Missing null protocol | "We'll figure it out if it fails" | Write it now — it forces clarity about what you're actually betting on |
| Cutoff violation | Architecture choice informed by 2024 data patterns | Never open the test set during development |
## Brain MCP Integration
**At spec start:**
- `brain_query wing=jepa-fx hall=decisions` — load current architectural decisions
- `brain_query wing=jepa-fx hall=failures` — load known failure modes; address them in Risks section
**After spec is approved:**
- `brain_write` to `jepa-fx/hypotheses/` with the hypothesis and acceptance criteria
**After experiment concludes:**
- `brain_write` to `jepa-fx/failures/` with any new failure modes discovered
- `session_log` with outcome
## Cross-References
- Use `feature-spec` for implementing a specific component within an already-specced phase
- Use `grill-me` on the spec before running the experiment if the hypothesis feels shaky
- Use `tdd` once the spec is approved — each acceptance criterion maps to a test
- Use `session-retrospective` after the experiment concludes
+26
View File
@@ -17,6 +17,8 @@ REPO_URL="${SKILLS_REPO_URL:-https://gitea.d-ma.be/mathias/skills.git}"
REF="${SKILLS_REF:-main}"
CHECKOUT_DIR="${SKILLS_CHECKOUT_DIR:-$HOME/.local/share/skills}"
CLAUDE_GLOBAL_DIR="${CLAUDE_SKILLS_DIR:-$HOME/.claude/skills}"
CRUSH_DIR="${CRUSH_SKILLS_DIR:-$HOME/.config/crush/skills}"
ANTIGRAVITY_DIR="${ANTIGRAVITY_SKILLS_DIR:-$HOME/.gemini/antigravity/skills}"
log() { printf '[skills] %s\n' "$*"; }
@@ -69,6 +71,28 @@ wire_claude_global() {
done < <(list_skills)
}
wire_crush() {
# Crush reads skills from ~/.config/crush/skills/<name>/SKILL.md.
# Pre-creating the dir is cheap even when Crush isn't installed.
mkdir -p "$CRUSH_DIR"
while IFS= read -r skill; do
[ -n "$skill" ] || continue
link_skill "$CRUSH_DIR" "$skill"
done < <(list_skills)
}
wire_antigravity() {
# Antigravity (Google's VS Code extension) reads global skills from
# ~/.gemini/antigravity/skills/<name>/SKILL.md. Our SKILL.md files
# already carry the required `name` + `description` YAML frontmatter,
# so symlinks are sufficient — no manifest translation step.
mkdir -p "$ANTIGRAVITY_DIR"
while IFS= read -r skill; do
[ -n "$skill" ] || continue
link_skill "$ANTIGRAVITY_DIR" "$skill"
done < <(list_skills)
}
wire_claude_repo() {
# Only wire per-repo when invoked from inside a git repo and it isn't
# the skills repo itself.
@@ -92,6 +116,8 @@ main() {
ensure_checkout
wire_claude_global
wire_claude_repo
wire_crush
wire_antigravity
log "done — $(list_skills | wc -l | tr -d ' ') skill(s) wired at ref=$REF"
}