Compare commits

..
14 Commits
Author SHA1 Message Date
mathiasandClaude Opus 5 4edf42ce9f fix(test): run the secret guard via its own shebang, not through sh
CI / Check / Test (push) Successful in 3s
CI / Tag release (push) Successful in 0s
The suite piped hook input into `sh "$GUARD"`, which overrides the hook's
`#!/usr/bin/env bash` and tests an interpreter production never uses. Claude
Code executes the hook as a program.

On Debian /bin/sh is dash, which has no `set -o pipefail`, so the guard died
on line 23 before reading a byte of input:

    secret-guard.sh: 23: set: Illegal option -o pipefail
    secret_guard: 0 passed, 20 failed

while passing 20/20 on a host where /bin/sh happens to be bash. A security
gate that tests green and cannot start is the worst of both, and the test was
the only thing standing between those two states.

Found because baked-v16 added jq to the runner: this suite needs jq to build
its hook JSON, so it had been silently doing nothing in CI and started running
for the first time. Same shape as installing zip and immediately learning the
gitea backup check had never run -- a missing tool does not announce that it
is suppressing a whole suite.

Also asserts the hook is executable, since direct invocation depends on the
exec bit, and that assertion is what stops this regressing to `sh "$GUARD"`.

Checked the rest of the suite: every other `sh "$SUT"` targets a POSIX script,
which is correct. hooks/pre-task-ritual.sh is the other bash-shebang hook and
has no test at all -- filed separately rather than fixed here.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:23:06 +02:00
mathiasandClaude Opus 5 2f53346b8c docs(check): say plainly that CI can never verify the MCP env wiring
CI / Check / Test (push) Failing after 2s
CI / Tag release (push) Skipped
infra#368 added jq to the gitea-runner image on the stated grounds that
check-mcp-env would then verify in CI instead of skipping. Verified after
rolling baked-v16: jq IS present now, the `SKIP: jq not installed` branch is
gone -- and the check still reports unverified, because CI has no ~/.op-env.

The env-file branch was always the binding one. It holds live credentials, so
it is not on the runner and must not be, which means this check can only ever
verify on a host that has one. Putting ~/.op-env on the runner to turn the
NOTE green would be trading a real secret for a green tick.

So the message now says so, at the point someone reads it. The risk otherwise
is not the skip -- it is a future reader seeing a passing pipeline and
concluding the MCP wiring is covered. It is covered on developer hosts only.

jq was still worth adding: it is a genuine dependency of this checker, the
runner's tool-contract.txt now asserts it, and a future image built without it
is a red pod at roll time rather than a silently degraded check.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:21:10 +02:00
mathias dcc0677270 chore(release): plugin.json version 0.19.2
Skip-Release: true
2026-09-07 18:14:53 +00:00
mathiasandClaude Opus 5 9fd5a054b3 fix(check): tolerate an unverifiable check instead of dying on it
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
bin/check-mcp-env exits 2 for "could not run" -- no jq, or no env-file --
and the gate was written to report that as a NOTE rather than fail:

    "$DIR/check-mcp-env" "$DIR/.mcp.json" "$HOME/.op-env"
    case $? in 0) ;; 2) echo "NOTE: ..." ;; *) fail=1 ;; esac

That tolerance never executed. Task runs each cmds block under `sh -e`, so
a bare command exiting non-zero aborts the shell before `case $?` is
reached. main went red on the very commit that added the check (run #57),
with `task: Failed to run task "check": exit status 2`.

It passed locally because this host has both jq and ~/.op-env, so the
check exited 0 and the untested branch never ran. CI has neither. A
tolerated exit code that is only tolerated where it never fires is not
tolerated at all.

`|| rc=$?` puts the invocation in a compound command, which `set -e`
exempts, so `case` is reached. Both exit-2 paths verified: no env-file
(the new test's precondition) and no jq (PATH sandbox, same SKIP message
CI printed).

The test pins the tolerance itself rather than the checker: it points HOME
at an empty directory, which is exactly CI's condition, and asserts `task
check` exits 0 AND still says it could not verify -- silence would mean
skipped rather than tolerated.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 20:14:19 +02:00
mathiasandClaude Opus 5 da76a6b3d2 feat(mcp): unpark the gitea server — it needs no AgentSecrets token
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
gitea-mcp was parked alongside cad on the stated reason that nothing puts its
token into the session environment: GITEA_MCP_STATIC_TOKEN lives in the
AgentSecrets vault, which the session identity deliberately cannot read
(infra#224), so enabling it was blocked behind #32/#33.

That reason never applied to gitea. gitea-mcp authenticates by PASS-THROUGH:
the bearer is a Gitea PAT, which the server validates against Gitea and then
acts as. No dedicated MCP token exists to be missing. DMABE_GITEA_API_TOKEN is
op://HomeLab and already in the session environment.

Verified on koala 2026-09-07 against tailscale serve 8446 -> NodePort 30340:
initialize returns 200 with serverInfo gitea-mcp, tools/list returns 41 tools.
Two DIFFERENT PATs both authenticate while GITEA_MCP_TOKEN 401s -- that is what
proves pass-through validation rather than a static-token comparison, and it
also shows GITEA_MCP_TOKEN is stale (it 401s against git-mcp.d-ma.be too).

cad stays parked; its blocker is real and unchanged.

Also adds bin/check-mcp-env and tests/mcp_config_test.sh, wired into task check.
The fault it gates is silent: a header interpolating an undefined variable does
not error, it sends an empty Authorization, and Claude Code reports "requires
authentication" -- indistinguishable from a revoked credential. That cost a full
investigation on koala today before the credential turned out to be fine. The
check exits 2 rather than 0 when it cannot run (no jq, no env-file, as in CI),
so an unverified config is never reported as a clean one. Servers under _parked
are exempt: referencing an unprovided variable is what parked means.

Bump-Type: minor

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 07:24:23 +02:00
mathias fc6d5628a7 fix(adr): template emitted invalid YAML frontmatter
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
The frontmatter exists to be machine-read. Two shapes this template
suggested do not parse, so any ADR that copied them dropped silently out of
every "what is still open?" query while looking correct:

  supersedes:    -                 a bare dash is a YAML sequence entry
  deciders:      pending: Mathias  the unquoted second colon reads as a
                                   nested mapping

Both were found live during an estate-wide ADR survey on 2026-09-05:
openbanking-api ADR-0001 (the one genuinely open decision in the estate --
invisible to the survey until fixed) and infra ADR-0019, each copied
verbatim from here.

Template corrected to `null` and a quoted `"pending: <name>"`, plus a short
section after the fence explaining why and giving a one-line yaml.safe_load
check to run before committing. The explanation is deliberately OUTSIDE the
fenced block so it is not copied into the ADR being written.
2026-09-05 12:25:10 +02:00
mathias 1f7f53e6c0 chore(release): plugin.json version 0.18.1
Skip-Release: true
2026-08-30 09:49:22 +00:00
mathias bedb6d278d fix(test): stop the Antigravity cases failing on a host without jq
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
main has been red since e9ad692. The gate passed; the shell suite failed on
three Antigravity cases.

Not a bug in the wiring. install.sh treats jq as optional by design — every
other target is dependency-free, so wire_antigravity warns and skips when jq is
missing. It did exactly that on the runner:

  [skills] Antigravity: jq not found — skipping

The tests were the problem: they inherited whatever jq the host happened to
have. koala has it and reported 12/12; the runner does not and reported 3
failures on the same commit. Works-on-my-machine, caught by CI doing its job.

Two changes:

- The three merge-behaviour cases now run only where jq exists, and print a SKIP
  naming what is not covered. A case that fails because the host lacks an
  optional dependency is a false red, and false reds are how a suite stops being
  read.
- The skip path gets a real test of its own, because it is a contract rather than
  a fallback. install.sh gains SKILLS_JQ so the jq-absent branch is exercisable
  on a host that has jq — the same seam pattern as DOCTOR_LATEST_TAG_CMD. It
  asserts no skills.json is written, the skip is reported, and the other
  harnesses still wire.

Verified both ways on koala by building a jq-free PATH:
  with jq:    install: 13 passed, 0 failed, 0 skipped
  without jq: install: 10 passed, 0 failed, 3 skipped

Consequence worth naming: CI now SKIPS the merge assertions rather than running
them, so idempotency and entry-preservation are untested there until the runner
gets jq. Filed as #42.

Refs #41
Bump-Type: patch
2026-08-30 11:48:56 +02:00
mathiasandClaude Sonnet 5 e9ad692418 feat(install): re-add Antigravity as a wiring target
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
Antigravity discovers skills via a path-entries JSON
(~/.gemini/config/skills.json), not a symlink directory. One entry points
at $AGENTS_DIR — the same managed, surfaces-filtered, collision-resolved
directory Crush/OpenCode/Pi already read — so Antigravity sees exactly the
same skill set as every other harness, for free.

Requires jq; every other target in this script stays dependency-free on
purpose, so wire_antigravity skips with a warning (not a hard failure) if
jq is missing. Merges into skills.json rather than overwriting it: the
file is documented as user-editable (json_configs.md — e.g. a personal
skills dir), so a hand-added entry must survive a re-run.

Was previously removed as "unused" (mathias/skills#41 — Antigravity usage
picked back up). Mistral Vibe stays removed.

Closes #41

Bump-Type: minor

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-30 08:58:34 +02:00
mathias 86529d2ac5 chore(release): plugin.json version 0.18.0
Skip-Release: true
2026-08-29 13:34:27 +00:00
mathias d6e892ac4c feat(skill): traceability — a chain of checked links, and a tool that says what it cannot check
CI / Check / Test (push) Successful in 2s
CI / Tag release (push) Successful in 1s
Forward trace: requirement → decision → commit → gate → verification → CI on
that sha → release → closure. The distinction the skill exists for is that each
link is either checked or named as unchecked; a trace where every link is
asserted is a narrative, and narratives are what audits exist to defeat.

`trace.sh` does the mechanical part — commit, files, test files touched, release
containing it, referenced issues as URLs — and then always prints the four things
git cannot prove. The GREEN run's own accounting: "the tool did the mechanical
15%. Its real contribution is the 'Cannot verify from here' block, which is a
checklist, not an output." Kept, because it is true.

The RED run established the shape by tracing a commit by hand in sixteen steps.
Its decisive moves were reading issue COMMENTS — where the decisions actually
live — and running a probe that falsified a claim. Both are now steps.

The GREEN run then used step 5 against a gate I shipped this morning, and broke
it. It also found four things wrong with what I had written:

- The slug sed used `[^/]+?` expecting a lazy quantifier. POSIX ERE has none, so
  `.git` was swallowed and every emitted evidence URL carried the suffix. It
  happened to work against Gitea and would 404 on GitHub — a bad failure for the
  one tool whose pitch is that an unchecked link must not look checked.
- "Prints every issue it references" was overclaimed: it greps the commit body,
  so it structurally cannot find the issue that cites the COMMIT. That is the
  commoner break — the issue filed afterwards that falsifies the outcome — and
  it is now step 6 with the two searches that find it.
- Step 3 told you to check which job passed, and the estate's own
  `workflow_run_status` returns no jobs. A step whose tooling dead-ends is where
  a trace becomes "the run was green"; the REST fallback is now written out.
- The output format did not survive contact. It grew four slots, each forced by
  a real trace: two requirements differing in kind (a defect report and a
  standing bar), the gate as an artifact separate from the test, the RED/GREEN
  baseline against the parent, and the commit's factual claims checked. Without
  the last one, "commit → verified" means only "this sha exists".

Also: `git describe --contains` reads local tags, so the tool now says so on
every hit rather than only when it finds nothing.

Closes #10
Bump-Type: minor
2026-08-29 15:34:11 +02:00
mathias 4d6ebb626f fix(check): read the session_log payload, not the file
A traceability probe falsified this gate four hours after I shipped it. The
check was `grep -q 'session_id' "$dir/SKILL.md"` — file-scoped. So the probe
stripped session_id from all three documented payloads in tdd/SKILL.md, added
one prose line saying "the tool has a required session_id field", and the gate
returned 0. The exact regression it exists to prevent was back in the file, and
the gate said nothing.

The duration_ms rule was no better: it matched one spelling, the historical one.
Both of these passed — `"duration_ms": <wall-clock>` with a JSON-quoted key, and
`duration_ms: elapsed wall-clock ms` with no angle brackets. It was a regression
test against the nine strings that existed that morning, described in the commit
message as a rule.

Now each documented payload is checked on its own line and reported with a line
number, and duration_ms is rejected in any spelling unless it carries an actual
number — the objection was always to a placeholder no agent can fill, not to the
field.

Five new cases, each one a probe from that run: prose mentioning the word while
the payload omits it, one bad payload among several, both duration_ms spellings,
and a real measurement being allowed.

Also fixed while writing them: my first regex had `[[:space:]]*` before a
negated class that did not exclude whitespace, so the star matched empty and the
space itself tripped the check — a real measurement failed. The tests caught it
immediately, which is the argument for writing them from the probes rather than
from the intent.

Refs #36, #39
Bump-Type: patch
2026-08-29 15:34:11 +02:00
mathias 88a4fa407b chore(release): plugin.json version 0.17.1
Skip-Release: true
2026-08-29 13:27:06 +00:00
mathias 808af7b020 chore(release): plugin.json version 0.17.0
Skip-Release: true
2026-08-29 13:27:05 +00:00
17 changed files with 920 additions and 22 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "skills",
"description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.",
"version": "0.16.0",
"version": "0.19.2",
"author": {
"name": "Mathias",
"url": "https://git.d-ma.be/mathias"
+10 -9
View File
@@ -1,5 +1,5 @@
{
"_comment": "Tailscale Serve on koala, not the public d-ma.be hostnames. The public path exits to Cloudflare and NPM to reach a k3s service on the same machine most sessions run on, which imposes rate limits and message-size caps on tool results. Serve terminates TLS with a real Tailscale cert and proxies to the NodePort over loopback, so traffic reaches the service from koala itself and satisfies the default-deny-ingress NetworkPolicy with no cluster change. Clients need 'tailscale set --accept-dns=true' for the MagicDNS name to resolve; see mathias/skills#24.",
"_comment": "Tailscale Serve on koala, not the public d-ma.be hostnames. The public path exits to Cloudflare and NPM to reach a k3s service on the same machine most sessions run on, which imposes rate limits and message-size caps on tool results. Serve terminates TLS with a real Tailscale cert and proxies to the NodePort over loopback, so traffic reaches the service from koala itself and satisfies the default-deny-ingress NetworkPolicy with no cluster change. Clients need 'tailscale set --accept-dns=true' for the MagicDNS name to resolve; see mathias/skills#24. Every ${VAR} below must be one ~/.op-env defines -- bin/check-mcp-env enforces that, because an undefined one sends an empty Authorization header and reports as 'requires authentication', which is indistinguishable from a revoked credential.",
"mcpServers": {
"brain": {
"type": "http",
@@ -11,23 +11,24 @@
"infra": {
"type": "http",
"url": "https://koala.tail30472e.ts.net:8444/mcp"
},
"gitea": {
"_comment": "gitea-mcp authenticates by PASS-THROUGH: the bearer is a Gitea PAT, which the server validates against Gitea and then acts as. So no dedicated MCP token is needed, and the AgentSecrets blocker that parked this entry (skills#32, #33) never applied to it -- DMABE_GITEA_API_TOKEN is op://HomeLab and already in the session environment. Verified 2026-09-07: initialize 200 (serverInfo gitea-mcp) and tools/list returns 41 tools. Two DIFFERENT PATs both authenticate while GITEA_MCP_TOKEN 401s, which is what proves this is pass-through validation and not a static-token comparison.",
"type": "http",
"url": "https://koala.tail30472e.ts.net:8446/mcp",
"headers": {
"Authorization": "Bearer ${DMABE_GITEA_API_TOKEN}"
}
}
},
"_parked": {
"_reason": "cad and gitea-mcp are proxied, reachable and AUTHENTICATING: tailscale serve 8445 -> 127.0.0.1:31877 and 8446 -> 127.0.0.1:30340; an MCP initialize with the correct bearer returns 200 from both (serverInfo gitea-mcp 0.1.0, cad aed2aaf). They stay parked only because nothing puts those two tokens into the session environment. Both live in the 1Password AgentSecrets vault, and by deliberate design (infra#224, closed) no single identity on koala reads both vaults: koala-homelab is HomeLab-only and resolves ~/.op-env, koala-agentsecrets-write is AgentSecrets-only and lives in ~/.op-agent-write.env. Adding an AgentSecrets reference to ~/.op-env does NOT degrade gracefully -- `op run` fails wholesale and every other token, BRAIN_MCP_TOKEN included, goes unresolved with it. Enabling needs a read-scoped AgentSecrets identity in the session-env path; see mathias/skills#32, #33.",
"_reason": "cad is proxied, reachable and AUTHENTICATING: tailscale serve 8445 -> 127.0.0.1:31877, and an MCP initialize with the correct bearer returns 200 (serverInfo cad aed2aaf). It stays parked only because nothing puts CAD_MCP_TOKEN into the session environment. It lives in the 1Password AgentSecrets vault, and by deliberate design (infra#224, closed) no single identity on koala reads both vaults: koala-homelab is HomeLab-only and resolves ~/.op-env, koala-agentsecrets-write is AgentSecrets-only and lives in ~/.op-agent-write.env. Adding an AgentSecrets reference to ~/.op-env does NOT degrade gracefully -- `op run` fails wholesale and every other token, BRAIN_MCP_TOKEN included, goes unresolved with it. Enabling needs a read-scoped AgentSecrets identity in the session-env path; see mathias/skills#32, #33. gitea was parked here for the same stated reason and has been unparked: it needs no AgentSecrets token at all, because gitea-mcp is pass-through-authenticated.",
"cad": {
"type": "http",
"url": "https://koala.tail30472e.ts.net:8445/mcp",
"headers": {
"Authorization": "Bearer ${CAD_MCP_TOKEN}"
}
},
"gitea": {
"type": "http",
"url": "https://koala.tail30472e.ts.net:8446/mcp",
"headers": {
"Authorization": "Bearer ${GITEA_MCP_STATIC_TOKEN}"
}
}
}
}
+3 -1
View File
@@ -100,9 +100,11 @@ default to point into `~/dev`.
| `AGENTS_SKILLS_DIR` | `$HOME/.agents/skills` |
| `CRUSH_SKILLS_DIR` | `$HOME/.config/crush/skills` |
| `OPENCODE_SKILLS_DIR` | `$HOME/.config/opencode/skills` |
| `ANTIGRAVITY_SKILLS_JSON` | `$HOME/.gemini/config/skills.json` (registers `AGENTS_SKILLS_DIR` as a path entry; needs `jq`) |
| `SKILLS_EXTRA_SOURCES` | unset (colon-separated extra source directories) |
Antigravity and Mistral Vibe targets were removed — unused.
The Mistral Vibe target was removed — unused. Antigravity's was removed for
the same reason and re-added once it came back into use.
## Checking a host
+1
View File
@@ -49,6 +49,7 @@ the SKILL.md, not the table.
| `tdd` | Test-Driven Development enforcement | Starting any new feature or fixing a bug | claude-code, library |
| `telos-load` | Load TELOS intention substrate at session start | Starting any koala session; before architectural decisions; CAD pipeline entry | claude-code, library |
| `test-design` | Dave Farley test design scoring (Farley Index) | Reviewing or writing tests | claude-code, library |
| `traceability` | Forward trace — requirement to decision to commit to verification to release, and what cannot be checked | Proving why a change exists; auditing a claim; before an attestation or a compliance ask | claude-code, library |
| `trainer` | Two-phase brain curation (score candidates, write past quality gate) | Periodic brain pruning and curation | claude-code, library |
| `user-stories` | Elephant Carpaccio story slicing | Breaking down features | claude-code, library |
| `web-shot` | Screenshot a running web UI via a koala k3s Playwright job | Visual verification of a running UI when no local browser exists | claude-code, library |
+20
View File
@@ -138,6 +138,26 @@ tasks:
echo "ERROR: missing .claude-plugin/plugin.json" >&2; fail=1
fi
# Every ${VAR} a live .mcp.json server interpolates must be one the
# session env-file defines. An undefined one is silent: the header
# goes out empty and the server reports "requires authentication",
# which reads as a revoked credential. Exit 2 means the check could
# not run (no jq, or no env-file — CI has neither); that is reported
# and not fatal, but it is never treated as a pass.
if [ -x "{{.TASKFILE_DIR}}/bin/check-mcp-env" ]; then
# `|| rc=$?` is load-bearing, not style. Task runs this block under
# `sh -e`, so a bare invocation exiting non-zero kills the shell
# before `case` is reached -- the tolerance below never ran, and the
# gate died with `exit status 2` on the commit that added it.
rc=0
"{{.TASKFILE_DIR}}/bin/check-mcp-env" "{{.TASKFILE_DIR}}/.mcp.json" "$HOME/.op-env" || rc=$?
case $rc in
0) ;;
2) echo "NOTE: MCP env wiring unverified on this host" >&2 ;;
*) fail=1 ;;
esac
fi
if [ $fail -ne 0 ]; then
echo "check FAILED" >&2; exit 1
fi
+99
View File
@@ -0,0 +1,99 @@
#!/bin/sh
# check-mcp-env — assert every variable a live MCP server interpolates is one
# the session env-file actually defines.
#
# Why this is a gate and not a comment: the failure is silent. A header of
# "Bearer ${FOO}" where FOO is undefined does not error -- it goes out empty,
# the server answers 401, and Claude Code reports "requires authentication",
# which reads exactly like a revoked credential. On 2026-09-07 that cost a
# full investigation on koala before the credential turned out to be fine.
#
# Servers under `_parked` are exempt by design. Parked means "reachable, but
# nothing puts its token in the environment yet" -- flagging those would
# report the intended state as a fault.
#
# Usage: check-mcp-env <mcp.json> <env-file>
#
# Exit 0 every live server's variables are defined
# 1 at least one is not, named on stderr
# 2 could not verify (bad usage, unreadable input, jq absent).
# Deliberately distinct from 0: an unverified config is not a clean
# one, and a gate that cannot run should say so rather than pass.
set -u
CONFIG=${1:-}
ENVFILE=${2:-}
if [ -z "$CONFIG" ] || [ -z "$ENVFILE" ]; then
echo "usage: check-mcp-env <mcp.json> <env-file>" >&2
exit 2
fi
if ! command -v jq >/dev/null 2>&1; then
echo "SKIP: jq not installed -- MCP env wiring unverified" >&2
exit 2
fi
if [ ! -f "$CONFIG" ]; then
echo "ERROR: no such config: $CONFIG" >&2
exit 2
fi
if [ ! -f "$ENVFILE" ]; then
# EXPECTED IN CI, and permanently so. The env-file holds live credentials, so
# it is not on the runner and must not be -- which means this check can only
# ever verify on a host that has one. infra#368 added jq to the runner image
# believing that would let CI verify; it removed the jq branch below and
# changed nothing else, because this branch is the binding one.
#
# Said plainly so nobody concludes CI covers this. It does not, it cannot,
# and putting ~/.op-env on the runner to make it would be trading a real
# secret for a green tick.
echo "ERROR: no such env-file: $ENVFILE -- cannot verify MCP variables" >&2
echo " (expected in CI: this check only verifies on a host with ~/.op-env)" >&2
exit 2
fi
# Every "<server>\t<VAR>" pair a live server interpolates. `..` walks the whole
# value, so a variable is found wherever it is placed, not only under headers.
refs=$(jq -r '
.mcpServers // {}
| to_entries[]
| .key as $server
| (.value | .. | strings)
| select(test("\\$\\{"))
| [ scan("\\$\\{([A-Za-z_][A-Za-z0-9_]*)\\}")[] ]
| .[]
| "\($server)\t\(.)"
' "$CONFIG" 2>/dev/null) || {
echo "ERROR: $CONFIG is not valid JSON -- MCP env wiring unverified" >&2
exit 2
}
if [ -z "$refs" ]; then
echo "check-mcp-env: no interpolated variables in live servers"
exit 0
fi
fail=0
checked=0
for pair in $(printf '%s\n' "$refs" | tr '\t' ':'); do
server=${pair%%:*}
var=${pair#*:}
checked=$((checked + 1))
if grep -qE "^[[:space:]]*(export[[:space:]]+)?$var=" "$ENVFILE"; then
continue
fi
printf 'ERROR: MCP server %s interpolates %s, which %s does not define.\n' \
"$server" "$var" "$ENVFILE" >&2
printf ' It will send an empty Authorization header and report as "requires authentication".\n' >&2
fail=1
done
if [ "$fail" -ne 0 ]; then
exit 1
fi
printf 'check-mcp-env: %d interpolated variable(s) across live MCP servers, all defined in %s\n' \
"$checked" "$ENVFILE"
+29 -5
View File
@@ -25,15 +25,39 @@ for dir in "$ROOT"/skills/*/; do
grep -q 'session_log' "$dir/SKILL.md" || continue
if ! grep -q 'session_id' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents session_log without session_id — the tool's only required field" >&2
fail=1
# Read the PAYLOAD, not the file. The first version of this gate grepped the
# whole SKILL.md for "session_id", so a payload missing the field passed as
# long as the word appeared anywhere — including in prose explaining that the
# field is required. A traceability probe demonstrated it in one sed.
payloads=$(grep -n 'session_log' "$dir/SKILL.md" | grep '{' || true)
if [ -z "$payloads" ]; then
# Mentions the tool but documents no call. Nothing to check, and nothing to
# get wrong.
continue
fi
if grep -q 'duration_ms:[[:space:]]*<' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents duration_ms as a placeholder an agent cannot measure — omit it or pass a real measurement" >&2
printf '%s\n' "$payloads" | while IFS= read -r line; do
[ -n "$line" ] || continue
body=$(printf '%s' "$line" | sed 's/^[0-9]*://')
lineno=$(printf '%s' "$line" | cut -d: -f1)
printf '%s' "$body" | grep -q 'session_id' \
|| echo "ERROR: skills/$name:$lineno documents a session_log payload with no session_id — the tool's only required field" >&2
# Any spelling of the key, quoted or not, followed by anything that is not a
# bare number. A placeholder nobody can fill gets fabricated or dropped; a
# real measurement is fine.
if printf '%s' "$body" | grep -Eq '"?duration_ms"?[[:space:]]*:[[:space:]]*[^0-9,}[:space:]]'; then
echo "ERROR: skills/$name:$lineno documents duration_ms as something other than a real measurement — an agent does not observe a clock; omit it" >&2
fi
done > "$ROOT/.session_log_errors.$$" 2>&1
if [ -s "$ROOT/.session_log_errors.$$" ]; then
cat "$ROOT/.session_log_errors.$$" >&2
fail=1
fi
rm -f "$ROOT/.session_log_errors.$$"
done
exit $fail
+51 -2
View File
@@ -17,8 +17,8 @@
# /plugin install skills@mathias
#
# That path carries the hooks and MCP server definitions too, which symlinks
# cannot. This script covers the harnesses with no plugin system: Crush, Pi
# and OpenCode.
# cannot. This script covers the harnesses with no plugin system: Crush, Pi,
# OpenCode, and Antigravity.
#
# Wiring model — one managed directory, one link per harness:
#
@@ -27,6 +27,13 @@
# ~/.config/crush/skills -> ~/.agents/skills
# ~/.config/opencode/skills -> ~/.agents/skills
#
# Antigravity has no symlink-directory discovery — it reads a path-entries
# JSON instead (~/.gemini/config/skills.json:
# {"entries":[{"path":"..."}]}). One entry points at ~/.agents/skills, so it
# sees exactly the same surfaces-filtered, collision-resolved set every other
# harness does. Requires jq; skipped with a warning if jq is unavailable —
# every other target in this script stays dependency-free on purpose.
#
# ~/.agents/skills was itself a link to this checkout until #27. That encoded
# an assumption nobody had stated — that this checkout is the ONLY source of
# skills on the host — and on flamingo the only way to install this library
@@ -53,7 +60,12 @@ CHECKOUT_DIR="${SKILLS_CHECKOUT_DIR:-$HOME/.local/share/skills}"
AGENTS_DIR="${AGENTS_SKILLS_DIR:-$HOME/.agents/skills}"
CRUSH_DIR="${CRUSH_SKILLS_DIR:-$HOME/.config/crush/skills}"
OPENCODE_DIR="${OPENCODE_SKILLS_DIR:-$HOME/.config/opencode/skills}"
ANTIGRAVITY_SKILLS_JSON="${ANTIGRAVITY_SKILLS_JSON:-$HOME/.gemini/config/skills.json}"
EXTRA_SOURCES="${SKILLS_EXTRA_SOURCES:-}"
# Overridable so the jq-absent branch is testable on a host that has jq. That
# branch is a contract, not a fallback: jq is optional here while every other
# target in this script is dependency-free.
JQ="${SKILLS_JQ:-jq}"
# Recorded beside the managed directory rather than inside it: a stray file in
# there would look like a skill to every harness reading the dir.
SOURCES_MANIFEST="$(dirname "$AGENTS_DIR")/$(basename "$AGENTS_DIR")-sources"
@@ -213,6 +225,42 @@ link_dir() {
log "linked $link → $target"
}
# wire_antigravity — register $AGENTS_DIR in Antigravity's skills.json.
# Merges rather than overwrites: skills.json is a documented user-editable
# config (json_configs.md), so a hand-added entry (e.g. a personal skills
# dir) must survive a re-run untouched.
wire_antigravity() {
if ! command -v "$JQ" >/dev/null 2>&1; then
log "Antigravity: jq not found — skipping (needed to merge $ANTIGRAVITY_SKILLS_JSON safely)"
return
fi
local existing already updated
mkdir -p "$(dirname "$ANTIGRAVITY_SKILLS_JSON")"
if [ -s "$ANTIGRAVITY_SKILLS_JSON" ]; then
existing="$(cat "$ANTIGRAVITY_SKILLS_JSON")"
else
existing='{}'
fi
already="$(printf '%s' "$existing" | "$JQ" --arg p "$AGENTS_DIR" \
'[.entries // [] | .[] | select(.path == $p)] | length' 2>/dev/null)" || {
log "Antigravity: $ANTIGRAVITY_SKILLS_JSON is not valid JSON — leaving it untouched"
return 1
}
if [ "$already" -gt 0 ]; then
log "Antigravity: $AGENTS_DIR already registered in $ANTIGRAVITY_SKILLS_JSON"
return
fi
updated="$(printf '%s' "$existing" | "$JQ" --arg p "$AGENTS_DIR" \
'.entries = ((.entries // []) + [{path: $p}])')"
printf '%s\n' "$updated" > "$ANTIGRAVITY_SKILLS_JSON"
log "Antigravity: registered $AGENTS_DIR in $ANTIGRAVITY_SKILLS_JSON"
}
main() {
ensure_checkout
@@ -224,6 +272,7 @@ main() {
link_dir "$CRUSH_DIR" "$AGENTS_DIR"
link_dir "$OPENCODE_DIR" "$AGENTS_DIR"
wire_antigravity
log "done — $LINKED_COUNT skill(s) linked from $(sources | wc -l | tr -d ' ') source(s), ref=$REF"
log "Claude Code: install the plugin instead — /plugin marketplace add $REPO_URL"
+21 -2
View File
@@ -58,8 +58,8 @@ adr: 0004
title: <the decision, as a sentence — not "database choice">
status: proposed | accepted | superseded | amended
date: 2026-08-29
deciders: <who accepted it, by name — or "pending: <name>" while proposed>
supersedes: <adr number, or ->
deciders: <who accepted it, by name — or "pending: <name>" (QUOTED) while proposed>
supersedes: <adr number, or null>
---
# ADR-0004 — <same title>
@@ -97,6 +97,25 @@ the condition that fires a revisit. An ADR with no validation section is a prefe
with a template around it.
```
### The frontmatter must actually parse
It exists to be machine-read, so a block that does not load is worse than no block at
all: it drops that ADR silently out of every "what is still open?" query, while looking
correct. Two shapes this template used to suggest are invalid YAML, and both were found
live in the estate on 2026-09-05 — `openbanking-api` ADR-0001 and `infra` ADR-0019, each
copied verbatim from here:
| Wrong | Why | Right |
|---|---|---|
| `supersedes: -` | a bare dash is a YAML *sequence entry*, not a string | `supersedes: null` |
| `deciders: pending: Mathias` | the second, unquoted colon reads as a nested mapping | `deciders: "pending: Mathias"` |
Check before committing:
```sh
python3 -c "import yaml,sys;yaml.safe_load(open(sys.argv[1]).read().split('---')[1]);print('ok')" docs/adr/0004-*.md
```
## Preserving a productive tension
Some tradeoffs are not unresolved because nobody thought hard enough. They are
+170
View File
@@ -0,0 +1,170 @@
---
name: traceability
surfaces: [claude-code, library]
summary: "Forward trace — requirement to decision to commit to verification to release, and what cannot be checked"
use_when: "Proving why a change exists; auditing a claim; before an attestation or a compliance ask"
description: Establish the forward trace for a change — requirement, decision, commit, verification, CI, release, closure — as a chain of links that are each checked rather than asserted. Names the links that cannot be verified from the repository instead of letting them pass as verified. Use when auditing why a change exists, preparing an attestation, or when a claim needs to be provable rather than plausible.
---
# Traceability
## Overview
A trace is not a story about how a change came to be. It is a chain of links, each of
which is either **checked** or **named as unchecked**. The value is entirely in that
distinction: a trace where every link is asserted is a narrative, and narratives are what
audits exist to defeat.
The chain in this estate:
```
issue (requirement) → comment (decision) → commit → test/gate → CI run on that sha
→ release tag → closing comment (evidence)
```
Note where the decisions live: **in issue comments**, not issue bodies. A tool that reads
issue titles will trace the wrong thing.
## When to Use
- Auditing why a change exists, especially one nobody remembers making
- Preparing an attestation, a compliance answer, or a client-facing "how do you know"
- Before claiming a requirement is satisfied
- When a claim is repeated in several places, which is the condition under which nobody
checks it
**Do not use for:** a change you are about to make. That is `adr` (the decision) and
`spec-driven-dev` (the requirement). This skill reconstructs; it does not author.
## The tool
```sh
skills/traceability/trace.sh <sha> [repo-path]
```
Prints what git can prove — the commit, its files, the test files it touched, the release
that contains it, every issue it references as a URL — and then prints, every time, the
three things git **cannot** prove. That last section is the point of the tool.
## The sequence
**1. Run the tool.** It does the mechanical half in one command.
**2. Open the issues it printed, and read the comments.** The requirement is in the body;
the decision is almost always in a comment, and so is the closure record. A trace built
from commit messages alone will confidently cite an issue that says something else.
**3. Match CI on the head sha, not on the branch.** "The pipeline is green" is not
evidence about a commit. Find the run whose `head_sha` is this commit and read its
conclusion — and check which *job* and which *step* passed, because a run can be red for a
reason that has nothing to do with the gate you care about, and green overall while the
gate you care about was skipped.
The estate's `workflow_run_status` returns only the run-level conclusion — no jobs, no
steps. For that breakdown you have to drop to the API:
```sh
op run --env-file ~/.op-env -- sh -c 'curl -sS -H "Authorization: token $DMABE_GITEA_API_TOKEN" \
"https://git.d-ma.be/api/v1/repos/<owner>/<repo>/actions/runs/<id>/jobs"'
```
Said explicitly because a step whose tooling dead-ends is exactly where a trace becomes
"the run was green".
**4. Check that the tests actually cover the behaviour.** A commit that touched a test
file is not a commit that is verified. Read the assertions and ask what would have to
break for them to fail. If nothing would, the link is decorative.
**5. Probe the load-bearing claim — try to falsify it.** This is the step that separates a
trace from a summary, and it is the one that finds things.
Pick the claim the chain rests on, and attack it in a scratch copy:
```sh
git archive <sha> | (mkdir -p /tmp/probe && tar -x -C /tmp/probe)
# then break exactly the thing the claim protects, and run the gate
```
If the gate stays green, the claim is false — regardless of how many places assert it.
This is not hypothetical: the probe that shaped this skill falsified a "the list only
shrinks" claim that was stated in five places in the repository and enforced in none.
**6. Look for what cites the commit, not only what it cites.** A commit cannot reference
an issue filed after it, and that is precisely where an outcome gets falsified. Search the
tracker for the sha, across issues *and comments*, and `git log --all --grep=<sha>` for
later commits referring back. In the run that shaped this skill, the issue proving the
traced commit had *relocated* rather than fixed its defect was filed 90 minutes later and
was reachable only this way.
**7. Report unverifiable links as unverifiable.** Not as weak links, not as caveats at the
end. Each gets its own line saying what would be needed to close it.
## What a broken link looks like
| Symptom | What it means |
|---|---|
| The commit cites `#37`, and `#37` describes something else | The reference is decoration; someone renumbered or misremembered |
| A claim appears in five places and a check in none | Convention wearing a mechanism's clothes. Probe it |
| The gate greps for a heading | Presence is not compliance. A heading with nothing under it passes |
| `git describe` says unreleased | Usually stale local tags, not an unreleased commit. `git fetch --tags` first |
| The requirement is a bare `#5` with no URL | Recoverable only while the tracker lives. Not traceable from the repo alone |
| CI green, but on a different sha | The most common false link, and the easiest to accept by accident |
## Output
State the chain, then the gaps, in that order:
```
CHAIN
requirement #36 item 3 (defect report) → verified, read: body + comments
requirement #19 (standing execution bar) → verified, read
decision comment 8509 → verified; names the commit
commit b616c51 → verified
claim "nine skills, none named session_id" → verified against b616c51^: 9/9
claim "session_id is the only required field" → verified against the live tool schema
gate bin/check-session-log → RED on parent (17 errors), GREEN on it
wiring Taskfile check step 5 → verified; CI runs task check
verification tests/check_session_log_test.sh → verified, 4 cases, both directions
CI run 4880, job "Check / Test" → verified, head_sha matches, success
release v0.13.3 → verified against the REMOTE
GAPS
- The gate greps the file, not the payload. Probe: stripped session_id from
every payload, left the word in prose — gate returned 0. FALSIFIED.
- Defect relocated, not removed: #39, filed 90 minutes later by the same
author, says so. The commit's outcome claim does not hold.
```
Four slots that are not optional, each learned by leaving them out:
- **More than one requirement, and they differ in kind.** A defect report and a standing
policy bar are both requirements and collapsing them hides which one was not met.
- **The gate, separately from the test.** A commit that ships a check has three artifacts,
not two, and the chain usually rests on the one in the middle.
- **The RED/GREEN baseline.** Run the gate against the parent commit. Without it you have
shown a gate exists, not that it caught anything.
- **The commit's factual claims, checked.** "Nine skills, none named it" is falsifiable in
one command. Without a slot for it, "commit → verified" means only "this sha exists".
Gaps ranked by how much they matter, not by where you found them. A trace whose gap
section is empty should be suspicious of itself.
## Escalation
Read-only. Reports the chain; changes nothing, and closes no issue on the strength of
what it finds.
Stop and hand back when: a link's evidence is outside anything you can reach — a tracker
you cannot read, a CI system you have no access to, a decision made in conversation. Say
which link and what would close it. An inferred link presented as a traced one is worse
than a gap, because the gap is honest and the inference is citable.
Also stop before *acting* on a falsified claim. Finding that a gate does not enforce what
it says is a finding; deciding what to do about it is the owner's call.
## Cross-References
- `lineage` — the same discipline backwards: this asks "does the chain hold", that asks
"was it tried before and why did it stop".
- `adr` — the decision layer of the chain. A trace that reaches an ADR has found the
reasoning; a trace that reaches only a commit message has found a summary of it.
+107
View File
@@ -0,0 +1,107 @@
#!/bin/sh
# trace.sh — the forward trace for one commit, and an honest list of what it
# could not check.
#
# The chain this estate actually uses:
#
# issue (requirement) -> comment (decision) -> commit -> test/gate -> CI run
# -> release tag -> closing comment
#
# git can prove the middle of that and nothing else. The issue bodies, the
# comments where decisions actually live, and the CI run are all tracker state.
# So this prints what git knows and then names, with URLs, the links a human has
# to follow — because a link nobody checked must not read like a verified one.
#
# Usage: trace.sh <sha> [repo-path]
#
# Read-only.
set -u
[ $# -ge 1 ] || { echo "usage: trace.sh <sha> [repo-path]" >&2; exit 2; }
SHA=$1
REPO=${2:-.}
git -C "$REPO" rev-parse --git-dir >/dev/null 2>&1 || {
echo "trace: $REPO is not a git repository" >&2; exit 2; }
git -C "$REPO" rev-parse --verify --quiet "$SHA^{commit}" >/dev/null || {
echo "trace: $SHA is not a commit in $REPO" >&2; exit 2; }
TOP=$(git -C "$REPO" rev-parse --show-toplevel)
ORIGIN=$(git -C "$REPO" config --get remote.origin.url 2>/dev/null || true)
# Never print the URL as configured: url.insteadOf can expand it into a live
# token. Reduce it to owner/repo and use that.
SLUG=$(printf '%s' "$ORIGIN" | sed -E 's#\.git$##; s#^.*[/:]([^/]+/[^/]+)$#\1#' 2>/dev/null || true)
section() { printf '\n== %s ==\n' "$1"; }
section "Commit"
git -C "$REPO" log -1 --format='%h %ad %an%n%n%B' --date=short "$SHA"
section "Files"
git -C "$REPO" show --stat --format='' "$SHA"
section "Verification it touched"
tests=$(git -C "$REPO" show --name-only --format='' "$SHA" \
| grep -Ei '(^|/)(tests?|spec)/|_test\.|\.test\.|_spec\.' || true)
if [ -n "$tests" ]; then
printf '%s\n' "$tests"
else
printf 'none — this commit changed no test or spec file.\n'
printf 'That is not automatically wrong (docs, config), but if it changed behaviour\n'
printf 'it means the change has no verification of its own.\n'
fi
section "Released in"
tag=$(git -C "$REPO" describe --tags --contains "$SHA" 2>/dev/null | sed 's/[~^].*//' || true)
if [ -n "$tag" ]; then
printf '%s (LOCAL tags only — confirm with: git ls-remote --tags origin)\n' "$tag"
else
printf 'no release contains this commit in THIS checkout.\n'
printf 'Local tags go stale — run `git fetch --tags` and re-check before\n'
printf 'concluding it was never released.\n'
fi
section "Referenced issues"
refs=$(git -C "$REPO" log -1 --format='%B' "$SHA" | grep -oE '#[0-9]+' | sort -u || true)
if [ -n "$refs" ]; then
for r in $refs; do
n=${r#\#}
if [ -n "$SLUG" ]; then
printf '%s -> https://git.d-ma.be/%s/issues/%s\n' "$r" "$SLUG" "$n"
else
printf '%s\n' "$r"
fi
done
else
printf 'none — the commit cites no issue, so its requirement is unrecorded.\n'
fi
section "Issues that cite THIS commit"
printf 'Not searchable from the commit — an issue filed after it cannot appear in\n'
printf 'its message, and that is exactly where an outcome gets falsified. Check:\n\n'
printf ' git -C %s log --all --grep=%s # later commits referring back\n' "$REPO" "$SHA"
if [ -n "$SLUG" ]; then
printf ' tracker search for "%s" across issues AND comments\n' "$SHA"
fi
printf '\nIn the run that shaped this tool, the issue that falsified the traced\n'
printf 'commit was filed 90 minutes after it and was reachable only this way.\n'
section "Cannot verify from here"
cat <<'TXT'
git proves the commit, its files, and which tag contains it. It cannot prove:
1. That the issues above say what the commit claims they say. The DECISION is
usually in a comment, not the issue body — open them and read the comments.
2. That CI actually passed on this exact sha. A green pipeline elsewhere is not
evidence about this commit; match on head_sha.
3. That the tests listed above assert the behaviour this commit added. A test
file touched is not a behaviour covered.
4. That any gate this commit ships enforces what its message says it does.
Probe it: break exactly the thing the claim protects, in a scratch copy,
and see whether the gate goes red.
None of the three is a formality. Each has been the broken link in this estate:
a claim asserted in five places and enforced in none, a red run whose gate had
passed, and a gate that greps for a heading and calls it a contract.
TXT
+33
View File
@@ -63,5 +63,38 @@ expect 1 'documenting duration_ms as wall-clock fails' \
"$(fixture clock 'Call `session_log` with `{session_id: "<id>", skill: "alpha", duration_ms: <wall-clock>}`')" \
'duration_ms'
# --- the gate must read the PAYLOAD, not the file ---------------------------
# A traceability probe falsified the original: it grepped the whole file for the
# string "session_id", so stripping the field from every documented payload and
# leaving the word once in prose passed cleanly. The exact regression the gate
# exists to prevent was back in the file and the gate said nothing.
expect 1 'a payload without session_id fails even when prose mentions the word' \
"$(fixture prose 'The tool has a required session_id field.
Call `session_log` with `{skill: "alpha", phase: "red", final_status: "pass"}`')" \
'session_id'
expect 1 'one bad payload among several fails' \
"$(fixture mixed 'Call `session_log` with `{session_id: "<id>", skill: "alpha", phase: "red"}`
Then `session_log` with `{skill: "alpha", phase: "green", final_status: "pass"}`')" \
'session_id'
# duration_ms was banned by matching exactly one spelling of it — the historical
# one. Both of these passed the original gate.
expect 1 'duration_ms with a JSON-quoted key fails' \
"$(fixture jsonkey 'Call `session_log` with `{session_id: "<id>", "duration_ms": <wall-clock>}`')" \
'duration_ms'
expect 1 'duration_ms described in prose rather than angle brackets fails' \
"$(fixture prosems 'Call `session_log` with `{session_id: "<id>", duration_ms: elapsed wall-clock ms}`')" \
'duration_ms'
# A real measurement is allowed: the objection is to a placeholder no agent can
# fill, not to the field.
expect 0 'a duration_ms carrying a real value is allowed' \
"$(fixture realms 'Call `session_log` with `{session_id: "<id>", duration_ms: 1430}`')"
printf '\ncheck_session_log: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
@@ -0,0 +1,70 @@
#!/bin/sh
# `task check` must survive a check that reports "could not verify".
#
# The regression these exist for: bin/check-mcp-env exits 2 to mean "I could
# not run — no jq, or no env-file", which the gate deliberately treats as a
# reported NOTE rather than a failure. The Taskfile said exactly that:
#
# "$DIR/check-mcp-env" "$DIR/.mcp.json" "$HOME/.op-env"
# case $? in 0) ;; 2) echo "NOTE: ..." ;; *) fail=1 ;; esac
#
# and it did not work. Task runs each cmds block under `sh -e`, so a bare
# command exiting non-zero aborts the shell *before* `case $?` is ever
# reached. The whole gate died with `exit status 2`. Locally the check found
# jq and exited 0, so it went green; CI has neither jq nor ~/.op-env, so main
# went red on the very commit that added it (skills run #57, 2026-09-07).
#
# A tolerated exit code that is only tolerated on the host where it never
# fires is not tolerated at all. This pins the tolerance itself.
#
# No mocking: pointing HOME at an empty directory removes ~/.op-env, which is
# exactly the condition CI runs under, so check-mcp-env genuinely exits 2.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
pass=0
fail=0
ok() { pass=$((pass + 1)); printf ' ok %s\n' "$1"; }
no() { fail=$((fail + 1)); printf ' FAIL %s\n' "$1"; }
if ! command -v task >/dev/null 2>&1; then
printf ' SKIP: task not installed -- cannot exercise the gate\n'
exit 0
fi
TMP=$(mktemp -d)
trap 'rm -rf "$TMP"' EXIT
# Sanity: the precondition must actually hold, or the test proves nothing.
# An empty HOME means no ~/.op-env, so check-mcp-env cannot verify anything.
"$ROOT/bin/check-mcp-env" "$ROOT/.mcp.json" "$TMP/.op-env" >/dev/null 2>&1
rc=$?
if [ "$rc" -eq 2 ]; then
ok 'precondition: check-mcp-env exits 2 when the env-file is absent'
else
no "precondition: expected check-mcp-env to exit 2, got $rc"
fi
# The assertion. `task check` must still pass, and must say why it could not
# verify -- silence would mean the check was skipped rather than tolerated.
out=$(HOME="$TMP" task --dir "$ROOT" check 2>&1)
rc=$?
if [ "$rc" -eq 0 ]; then
ok 'task check passes when MCP env wiring cannot be verified'
else
no "task check exited $rc when check-mcp-env reported unverifiable"
printf '%s\n' "$out" | sed 's/^/ | /' | tail -8
fi
if printf '%s' "$out" | grep -q 'unverified'; then
ok 'the unverifiable check is reported, not silently skipped'
else
no 'no NOTE reported -- an unverifiable check must never look like a pass'
fi
printf ' %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+91 -1
View File
@@ -17,6 +17,7 @@ SUT="$ROOT/install.sh"
pass=0
fail=0
skipped=0
WORK=${TMPDIR:-/tmp}/install_test.$$
if [ ! -f "$SUT" ]; then
@@ -55,6 +56,7 @@ run_install() {
AGENTS_SKILLS_DIR="$_f/home/.agents/skills" \
CRUSH_SKILLS_DIR="$_f/home/.config/crush/skills" \
OPENCODE_SKILLS_DIR="$_f/home/.config/opencode/skills" \
ANTIGRAVITY_SKILLS_JSON="$_f/home/.gemini/config/skills.json" \
SKILLS_EXTRA_SOURCES="$_extra" \
bash "$SUT" 2>&1)
rc=$?
@@ -156,6 +158,40 @@ else
no 'configured sources are recorded for doctor' "$out"
fi
# --- Antigravity: jq is optional, so BOTH branches must be tested -----------
# These three cases inherited whatever jq the host happened to have. koala has
# it, the CI runner does not, so they passed locally and failed in CI — the
# classic works-on-my-machine gate. install.sh treats jq as optional by design,
# which means the skip is a contract and needs a test of its own.
# run_install_nojq <fixture> — same as run_install, with the jq lookup pointed
# at a binary that does not exist. Deterministic regardless of the host.
run_install_nojq() {
_f=$1
_out=$(HOME="$_f/home" \
SKILLS_SKIP_FETCH=1 \
SKILLS_CHECKOUT_DIR="$_f/ck" \
AGENTS_SKILLS_DIR="$_f/home/.agents/skills" \
CRUSH_SKILLS_DIR="$_f/home/.config/crush/skills" \
OPENCODE_SKILLS_DIR="$_f/home/.config/opencode/skills" \
SKILLS_JQ="definitely-not-jq-$$" \
bash "$SUT" 2>&1)
rc=$?
printf '%s' "$_out"
}
f=$(new_fixture antigravity_nojq)
out=$(run_install_nojq "$f")
if [ -e "$f/home/.gemini/config/skills.json" ]; then
no 'without jq, no skills.json is written' "$out"
elif ! printf '%s' "$out" | grep -qi 'jq not found'; then
no 'without jq, the skip is reported' "$out"
elif [ ! -L "$f/home/.agents/skills/alpha" ]; then
no 'without jq, the other harnesses are still wired' "$out"
else
ok 'without jq, Antigravity is skipped and the rest still wires'
fi
# --- 8. surfaces: the library link set is not "every skill" -----------------
# claude.ai has no shell, no cluster and no checkout, so a chat-only skill
# linked into Crush is a skill that can never fire, competing for selection
@@ -183,5 +219,59 @@ else
no 'a source with no surfaces convention is linked as-is' "$out"
fi
printf '\ninstall: %d passed, %d failed\n' "$pass" "$fail"
# --- 9-11. Antigravity's merge behaviour, only where jq exists ---------------
# These three assert what jq DOES. install.sh treats jq as optional, so on a
# host without it there is nothing to assert — and failing there is what turned
# main red: they passed on koala (jq present) and failed on the runner (absent).
# A skipped case is honest; a case that fails because the host lacks an optional
# dependency is a false red, and false reds are how a suite stops being read.
if ! command -v jq >/dev/null 2>&1; then
skipped=$((skipped + 3))
printf ' SKIP jq absent — Antigravity merge behaviour not covered on this host (3 cases)\n'
else
# skills.json entries must point at $AGENTS_DIR — that carries the
# surfaces-filtering, collision handling and multi-source merge. Pointing at
# the checkout directly would give Antigravity a different skill set than
# every other harness.
f=$(new_fixture antigravity)
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
if [ ! -f "$AG" ]; then
no 'Antigravity skills.json is created' "$out"
else
registered=$(jq -r --arg p "$f/home/.agents/skills" '[.entries[]? | select(.path == $p)] | length' "$AG")
if [ "$registered" = "1" ]; then
ok 'Antigravity registers the managed agents dir, not the raw checkout'
else
no 'Antigravity registers the managed agents dir, not the raw checkout' "$(cat "$AG")"
fi
fi
f=$(new_fixture antigravity_idempotent)
run_install "$f" >/dev/null
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
count=$(jq '.entries | length' "$AG" 2>/dev/null || echo "?")
if [ "$count" = "1" ]; then
ok 'Antigravity wiring is idempotent (no duplicate entries)'
else
no 'Antigravity wiring is idempotent (no duplicate entries)' "entries=$count"$'\n'"$out"
fi
# A user may hand-edit skills.json to point at their own personal skills dir
# (json_configs.md documents this). Our merge must not clobber it.
f=$(new_fixture antigravity_preserve)
mkdir -p "$f/home/.gemini/config"
printf '{"entries":[{"path":"/Users/mathias/personal-skills"}]}' > "$f/home/.gemini/config/skills.json"
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
personal=$(jq -r '[.entries[]? | select(.path == "/Users/mathias/personal-skills")] | length' "$AG" 2>/dev/null || echo 0)
if [ "$personal" = "1" ]; then
ok 'Antigravity wiring preserves pre-existing hand-authored entries'
else
no 'Antigravity wiring preserves pre-existing hand-authored entries' "$(cat "$AG")"$'\n'"$out"
fi
fi
printf '\ninstall: %d passed, %d failed, %d skipped\n' "$pass" "$fail" "$skipped"
[ "$fail" -eq 0 ]
+118
View File
@@ -0,0 +1,118 @@
#!/bin/sh
# Tests for bin/check-mcp-env.
#
# The fault this gate exists for is silent, which is why it needs a gate at
# all. An `.mcp.json` server whose Authorization header interpolates a
# variable nothing in the session environment defines does not error: the
# header goes out empty, the server answers 401, and Claude Code reports
# "requires authentication". That is indistinguishable from a revoked or
# expired credential, and on 2026-09-07 it cost a full investigation on koala
# to establish that the credential was fine and the variable was simply never
# set.
#
# So the property is: every ${VAR} a LIVE server interpolates must be a
# variable the session env-file actually defines.
#
# The `_parked` block is deliberately exempt. Parked servers reference
# variables nothing provides on purpose -- that is what parked means -- and a
# gate that flagged them would be reporting the intended state as a fault.
#
# Fixtures drive the checker through argv, so no test touches the real host.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
CHECK="$ROOT/bin/check-mcp-env"
pass=0
fail=0
WORK=${TMPDIR:-/tmp}/mcp_config_test.$$
if [ ! -f "$CHECK" ]; then
printf 'FAIL: %s does not exist\n' "$CHECK"
exit 1
fi
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
mkdir -p "$WORK"
if ! command -v jq >/dev/null 2>&1; then
printf 'SKIP: jq not installed -- bin/check-mcp-env cannot be exercised\n'
exit 0
fi
# ok <name> / bad <name> <detail>
ok() { pass=$((pass+1)); printf ' ok %s\n' "$1"; }
bad() { fail=$((fail+1)); printf ' FAIL %s %s\n' "$1" "${2:-}"; }
# run <mcp-json> <env-file> -> sets $rc and $out
run() {
out=$("$CHECK" "$1" "$2" 2>&1); rc=$?
}
ENVFILE="$WORK/op-env"
cat > "$ENVFILE" <<'ENVEOF'
BRAIN_MCP_TOKEN=op://HomeLab/brain/token
DMABE_GITEA_API_TOKEN=op://HomeLab/gitea/token
ENVEOF
# --- a live server whose variable IS defined ---------------------------------
cat > "$WORK/good.json" <<'JSONEOF'
{"mcpServers":{"brain":{"type":"http","url":"https://x/mcp",
"headers":{"Authorization":"Bearer ${BRAIN_MCP_TOKEN}"}}}}
JSONEOF
run "$WORK/good.json" "$ENVFILE"
[ "$rc" -eq 0 ] && ok "defined variable passes" || bad "defined variable passes" "rc=$rc $out"
# --- a live server whose variable is NOT defined ------------------------------
# The exact shape that broke: the server looks correct and fails at runtime.
cat > "$WORK/missing.json" <<'JSONEOF'
{"mcpServers":{"gitea":{"type":"http","url":"https://x/mcp",
"headers":{"Authorization":"Bearer ${GITEA_MCP_STATIC_TOKEN}"}}}}
JSONEOF
run "$WORK/missing.json" "$ENVFILE"
if [ "$rc" -ne 0 ]; then
case "$out" in
*gitea*GITEA_MCP_STATIC_TOKEN*) ok "undefined variable is flagged, naming server and variable" ;;
*) bad "undefined variable is flagged, naming server and variable" "report did not name both: $out" ;;
esac
else
bad "undefined variable is flagged, naming server and variable" "rc=0"
fi
# --- a server with no headers at all (the `infra` shape) ----------------------
# Unauthenticated by design. Nothing to resolve, so nothing to flag.
cat > "$WORK/noauth.json" <<'JSONEOF'
{"mcpServers":{"infra":{"type":"http","url":"https://x/mcp"}}}
JSONEOF
run "$WORK/noauth.json" "$ENVFILE"
[ "$rc" -eq 0 ] && ok "header-less server is not flagged" || bad "header-less server is not flagged" "rc=$rc $out"
# --- parked servers are exempt ------------------------------------------------
cat > "$WORK/parked.json" <<'JSONEOF'
{"mcpServers":{"brain":{"type":"http","url":"https://x/mcp",
"headers":{"Authorization":"Bearer ${BRAIN_MCP_TOKEN}"}}},
"_parked":{"_reason":"no identity provides these yet",
"cad":{"type":"http","url":"https://x/mcp",
"headers":{"Authorization":"Bearer ${CAD_MCP_TOKEN}"}}}}
JSONEOF
run "$WORK/parked.json" "$ENVFILE"
if [ "$rc" -eq 0 ]; then
ok "parked server is exempt"
else
bad "parked server is exempt" "rc=$rc $out"
fi
# --- malformed JSON must not pass silently ------------------------------------
# A config the checker cannot read is an unverified config, not a clean one.
printf '{"mcpServers":{' > "$WORK/broken.json"
run "$WORK/broken.json" "$ENVFILE"
[ "$rc" -ne 0 ] && ok "unparseable config does not report success" || bad "unparseable config does not report success" "rc=0"
# --- a missing env-file is unverifiable, not clean ----------------------------
run "$WORK/good.json" "$WORK/no-such-env"
[ "$rc" -ne 0 ] && ok "missing env-file does not report success" || bad "missing env-file does not report success" "rc=0"
printf '\n%s: %d passed, %d failed\n' "$(basename "$0")" "$pass" "$fail"
[ "$fail" -eq 0 ]
+15 -1
View File
@@ -24,7 +24,7 @@ decision_for() {
_tool=${2:-Bash}
jq -n --arg t "$_tool" --arg c "$_cmd" \
'{tool_name: $t, tool_input: {command: $c}}' \
| sh "$GUARD" \
| "$GUARD" \
| jq -r '.hookSpecificOutput.permissionDecision // "MISSING"'
}
@@ -42,6 +42,20 @@ expect() {
fi
}
# Invoked as a PROGRAM above, never as `sh "$GUARD"`. Claude Code executes the
# hook directly, so its `#!/usr/bin/env bash` shebang decides the interpreter;
# piping it into `sh` overrides that and tests something production never runs.
#
# It is not academic. On Debian /bin/sh is dash, which has no `set -o
# pipefail`, so the guard died on line 23 before reading a byte of input and
# every deny case returned empty -- 20 of 20 assertions failed in CI while
# passing on a host where /bin/sh happens to be bash. A security gate that
# tests green and cannot start is the worst of both.
if [ ! -x "$GUARD" ]; then
printf 'FAIL: %s is not executable -- it must run via its own shebang\n' "$GUARD"
exit 1
fi
if [ ! -f "$GUARD" ]; then
printf 'FAIL: %s does not exist\n' "$GUARD"
exit 1
+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# Tests for skills/traceability/trace.sh.
#
# The RED run for #10 established what a forward trace costs by hand: sixteen
# commands, of which the decisive ones were reading issue comments and running a
# probe against a claim. This tool does the mechanical half — what git can prove
# — and, more importantly, states what it CANNOT prove, so an unverifiable link
# never reads as a verified one.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
SUT="$ROOT/skills/traceability/trace.sh"
pass=0
fail=0
WORK=${TMPDIR:-/tmp}/traceability_test.$$
if [ ! -f "$SUT" ]; then
printf 'FAIL: %s does not exist\n' "$SUT"
exit 1
fi
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
mkdir -p "$WORK/repo/tests"
cd "$WORK/repo" || exit 1
git init -q .
git config user.email t@example.com
git config user.name test
# A remote is needed for the tool to turn "#37" into a checkable URL, which is
# the point of that section.
git remote add origin https://git.d-ma.be/mathias/demo.git
printf 'baseline\n' > README.md
git add -A && git commit -qm "chore: baseline"
printf 'gate\n' > gate.sh
printf 'case\n' > tests/gate_test.sh
git add -A && git commit -qm "feat(gate): add the thing
Implements the decision recorded in #5, filed as #37.
Closes #37"
SHA=$(git rev-parse --short HEAD)
git tag -a v1.2.0 -m "release v1.2.0"
printf 'more\n' >> gate.sh
git add -A && git commit -qm "chore: later work"
cd - >/dev/null || exit 1
out=$(sh "$SUT" "$SHA" "$WORK/repo" 2>&1)
check() {
if printf '%s' "$out" | grep -q "$1"; then
pass=$((pass + 1)); printf ' ok %s\n' "$2"
else
fail=$((fail + 1)); printf ' FAIL %s\n output had no %s\n%s\n' "$2" "$1" "$out"
fi
}
check '#5' 'lists every issue the commit references, not just the closing one'
check '#37' 'lists the closing reference'
check 'tests/gate_test.sh' 'names the test files the commit touched'
check 'v1.2.0' 'names the release that contains it'
check 'Cannot verify' 'states what it could not verify'
check 'issues/37' 'emits a checkable URL for the tracker link it cannot follow'
# A commit in no release must say so — "no tag" and "not checked" are different
# answers and only one of them is about the commit.
last=$(git -C "$WORK/repo" rev-parse --short HEAD)
out=$(sh "$SUT" "$last" "$WORK/repo" 2>&1)
if printf '%s' "$out" | grep -qi 'no release contains'; then
pass=$((pass + 1)); printf ' ok an unreleased commit is reported as unreleased\n'
else
fail=$((fail + 1)); printf ' FAIL an unreleased commit is reported as unreleased\n%s\n' "$out"
fi
printf '\ntraceability: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]