Compare commits

...
7 Commits
Author SHA1 Message Date
mathiasandClaude Opus 5 1328dcd8f3 feat(doctor): report golangci-lint built with a Go older than GOROOT
CI / Check / Test (push) Successful in 3s
CI / Tag release (push) Successful in 1s
Second recurrence of the same host fault (infra#375; first hit
openbanking-api 2026-09-03, then cobalt-dingo 2026-09-08). golangci-lint
embeds the go/types export-data reader from the Go it was BUILT with, so a
newer GOROOT makes it unable to decode the stdlib:

  panic: file requires newer Go version go1.27 (application built with go1.26)

An earlier form printed a typecheck error naming the user's own source file,
which sends people to debug code that is fine. It cost time in two repos before
anyone recognised it as a host fault rather than a repo fault.

CI never sees it — the runner image has matching versions — so it breaks
exactly the local `task check` that AGENT.md requires before every push, and
nothing else. The fix was always one reinstall; the missing thing was the
signal, which is what doctor is for.

Fails open by design: an unparseable version, or no Go at all, is a normal host
and not a fault doctor may assert. Version comparison is major.minor in POSIX
sh rather than sort -V, since this runs on macOS too and the export-data format
is minor-versioned.

Also makes the suite hermetic. The check reads the real host, so on a machine
whose golangci-lint is genuinely stale every unrelated fixture started failing.
The harness now prepends a matching fake pair for the whole file, and only the
four cases that opt in exercise the check.

Verified against koala, which is currently in exactly this state:

  FAIL toolchain: golangci-lint built with go1.26.1 but GOROOT is go1.27.0
       — lint panics on every Go repo; reinstall it under the current
       toolchain (infra#375)

doctor: 25 passed, 0 failed. Full suite green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 21:34:44 +02:00
mathias 6cc224f151 chore(release): plugin.json version 0.19.4
Skip-Release: true
2026-09-07 20:34:58 +00:00
mathiasandClaude Opus 5 8359d61e33 test(hooks): pin the pre-task-ritual hook to its registration
CI / Tag release (push) Successful in 1s
CI / Check / Test (push) Successful in 2s
Closes #44.

That issue overstated the risk and this corrects it. The hook is three lines
and echoes a string -- no bash-isms, so it cannot fail the way secret-guard
appeared to. The real exposure is narrower and duller: it can stop working
silently, and its entire job is to be visible.

THE ASSERTION THAT MATTERS IS REGISTRATION. A hook file that exists but is
absent from hooks.json does nothing at all and looks identical on disk to one
that works. So the test derives the command from hooks.json and runs it exactly
as registered, substituting CLAUDE_PLUGIN_ROOT the way the harness does, rather
than inventing its own invocation.

That choice is the lesson from #44's sibling. secret_guard_test.sh piped its
hook into `sh` while hooks.json runs it with `bash`; on Debian /bin/sh is dash,
which has no `set -o pipefail`, so the suite failed 20/20 in CI against a hook
that was FINE in production. The test was wrong, not the hook -- and the
earlier commit message for that fix said the shebang decides the interpreter,
which hooks.json shows is not true. Deriving the invocation removes the whole
class rather than fixing one instance of it.

No red phase to report: the hook was already correct, so this is regression
pinning, not TDD. Teeth proven by mutation against a throwaway copy instead:

  - hook runs but prints nothing      -> 6 failures
  - hook removed from hooks.json      -> caught, suite stops early
  - ONE ritual step dropped, rule 0
    still named                       -> caught (9 passed, 1 failed)

The third is the one worth having. Losing the whole line is obvious; quietly
losing "failing test" from the ritual while the banner still reads correctly is
the drift that would actually happen.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:33:52 +02:00
mathias 400d2b9b6b chore(release): plugin.json version 0.19.3
Skip-Release: true
2026-09-07 20:23:31 +00:00
mathiasandClaude Opus 5 4edf42ce9f fix(test): run the secret guard via its own shebang, not through sh
CI / Check / Test (push) Successful in 3s
CI / Tag release (push) Successful in 0s
The suite piped hook input into `sh "$GUARD"`, which overrides the hook's
`#!/usr/bin/env bash` and tests an interpreter production never uses. Claude
Code executes the hook as a program.

On Debian /bin/sh is dash, which has no `set -o pipefail`, so the guard died
on line 23 before reading a byte of input:

    secret-guard.sh: 23: set: Illegal option -o pipefail
    secret_guard: 0 passed, 20 failed

while passing 20/20 on a host where /bin/sh happens to be bash. A security
gate that tests green and cannot start is the worst of both, and the test was
the only thing standing between those two states.

Found because baked-v16 added jq to the runner: this suite needs jq to build
its hook JSON, so it had been silently doing nothing in CI and started running
for the first time. Same shape as installing zip and immediately learning the
gitea backup check had never run -- a missing tool does not announce that it
is suppressing a whole suite.

Also asserts the hook is executable, since direct invocation depends on the
exec bit, and that assertion is what stops this regressing to `sh "$GUARD"`.

Checked the rest of the suite: every other `sh "$SUT"` targets a POSIX script,
which is correct. hooks/pre-task-ritual.sh is the other bash-shebang hook and
has no test at all -- filed separately rather than fixed here.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:23:06 +02:00
mathiasandClaude Opus 5 2f53346b8c docs(check): say plainly that CI can never verify the MCP env wiring
CI / Check / Test (push) Failing after 2s
CI / Tag release (push) Skipped
infra#368 added jq to the gitea-runner image on the stated grounds that
check-mcp-env would then verify in CI instead of skipping. Verified after
rolling baked-v16: jq IS present now, the `SKIP: jq not installed` branch is
gone -- and the check still reports unverified, because CI has no ~/.op-env.

The env-file branch was always the binding one. It holds live credentials, so
it is not on the runner and must not be, which means this check can only ever
verify on a host that has one. Putting ~/.op-env on the runner to turn the
NOTE green would be trading a real secret for a green tick.

So the message now says so, at the point someone reads it. The risk otherwise
is not the skip -- it is a future reader seeing a passing pipeline and
concluding the MCP wiring is covered. It is covered on developer hosts only.

jq was still worth adding: it is a genuine dependency of this checker, the
runner's tool-contract.txt now asserts it, and a future image built without it
is a red pod at roll time rather than a silently degraded check.

Bump-Type: patch

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 22:21:10 +02:00
mathias dcc0677270 chore(release): plugin.json version 0.19.2
Skip-Release: true
2026-09-07 18:14:53 +00:00
6 changed files with 249 additions and 2 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "skills",
"description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.",
"version": "0.19.1",
"version": "0.19.4",
"author": {
"name": "Mathias",
"url": "https://git.d-ma.be/mathias"
+10
View File
@@ -41,7 +41,17 @@ if [ ! -f "$CONFIG" ]; then
fi
if [ ! -f "$ENVFILE" ]; then
# EXPECTED IN CI, and permanently so. The env-file holds live credentials, so
# it is not on the runner and must not be -- which means this check can only
# ever verify on a host that has one. infra#368 added jq to the runner image
# believing that would let CI verify; it removed the jq branch below and
# changed nothing else, because this branch is the binding one.
#
# Said plainly so nobody concludes CI covers this. It does not, it cannot,
# and putting ~/.op-env on the runner to make it would be trading a real
# secret for a green tick.
echo "ERROR: no such env-file: $ENVFILE -- cannot verify MCP variables" >&2
echo " (expected in CI: this check only verifies on a host with ~/.op-env)" >&2
exit 2
fi
+38
View File
@@ -327,6 +327,44 @@ if [ "$ONLINE" -eq 1 ]; then
fi
fi
# --- toolchain: golangci-lint must be built with Go >= GOROOT ---------------
# infra#375, second recurrence. golangci-lint embeds the go/types export-data
# reader from the Go it was BUILT with. When GOROOT is newer, that reader
# cannot decode the stdlib and the linter dies:
#
# panic: file requires newer Go version go1.27 (application built with go1.26)
#
# An earlier form printed a typecheck error naming the user's own file, which
# sends people to debug code that is fine. It cost time in two repos before
# anyone recognised it as a host fault rather than a repo fault.
#
# It is invisible to CI -- the runner image has matching versions -- so it
# breaks exactly the local `task check` that AGENT.md requires before a push,
# and nothing else. That is why it belongs in doctor: the signal was missing,
# not the fix.
#
# Fails open on anything unparseable. A host with no Go toolchain is a normal
# host, not a broken one, and doctor runs in a SessionStart hook everywhere.
ver_lt() {
# 0 when $1 < $2 on major.minor. Deliberately not sort -V: this is POSIX sh
# that also runs on macOS, and the export-data format is minor-versioned, so
# patch level does not enter into it.
_a=${1%%.*}; _ar=${1#*.}; _am=${_ar%%.*}
_b=${2%%.*}; _br=${2#*.}; _bm=${_br%%.*}
case "$_a$_am$_b$_bm" in *[!0-9]*) return 1 ;; esac
[ "$_a" -lt "$_b" ] && return 0
[ "$_a" -gt "$_b" ] && return 1
[ "$_am" -lt "$_bm" ]
}
if command -v golangci-lint >/dev/null 2>&1 && command -v go >/dev/null 2>&1; then
lint_go=$(golangci-lint --version 2>/dev/null | sed -n 's/.*built with go\([0-9][0-9.]*\).*/\1/p' | head -1)
root_go=$(go version 2>/dev/null | sed -n 's/.*go version go\([0-9][0-9.]*\).*/\1/p' | head -1)
if [ -n "$lint_go" ] && [ -n "$root_go" ] && ver_lt "$lint_go" "$root_go"; then
add FAIL toolchain "golangci-lint built with go$lint_go but GOROOT is go$root_go — lint panics on every Go repo; reinstall it under the current toolchain (infra#375)"
fi
fi
# --- output -----------------------------------------------------------------
if [ "$JSON" -eq 1 ]; then
printf '{"checkout":"%s","expected":%s,"plugin":{"installed":"%s","available":"%s"},"findings":[%s]}\n' \
+82
View File
@@ -24,6 +24,18 @@ if [ ! -f "$DOCTOR" ]; then
exit 1
fi
# The toolchain check (golangci-lint build-Go vs GOROOT) reads the real host
# unless we say otherwise, which made every unrelated fixture fail on a machine
# whose golangci-lint is genuinely stale. Give the whole suite a matching fake
# pair so only the cases that OPT IN exercise that check.
STUB_BIN="$WORK/_stub_bin"
mkdir -p "$STUB_BIN"
printf '#!/bin/sh\nprintf "golangci-lint has version 2.0.0 built with go1.27.0 from abc\\n"\n' > "$STUB_BIN/golangci-lint"
printf '#!/bin/sh\ncase "$1" in version) printf "go version go1.27.0 linux/amd64\\n";; *) printf "\\n";; esac\n' > "$STUB_BIN/go"
chmod +x "$STUB_BIN/golangci-lint" "$STUB_BIN/go"
PATH="$STUB_BIN:$PATH"
export PATH
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
@@ -291,5 +303,75 @@ else
fail=$((fail + 1)); printf ' FAIL --online reports an unreachable remote as its own fault\n rc=%s got: %s\n' "$_rc" "$_out"
fi
# --- toolchain: golangci-lint must be built with Go >= GOROOT ---------------
# Measured on koala 2026-09-08 (infra#375, second recurrence): golangci-lint
# 2.11.4 built with go1.26.1 against a go1.27.0 GOROOT PANICS on any Go repo --
#
# panic: file requires newer Go version go1.27 (application built with go1.26)
#
# and the earlier form of the same fault printed a typecheck error naming YOUR
# file, so the natural response is to debug code that is fine. It cost time in
# two repos. CI never sees it (the runner image matches), so it breaks only the
# local `task check` that AGENT.md requires before every push.
#
# The fixtures inject fake `go` and `golangci-lint` on PATH; nothing here reads
# the real host toolchain, so the suite still passes on a machine without Go.
mk_toolchain() { # $1=fixture $2=lint-build-go $3=goroot-go ("" = absent)
_b="$1/bin"; mkdir -p "$_b"
if [ -n "$2" ]; then
printf '#!/bin/sh\nprintf "golangci-lint has version 2.11.4 built with %s from abc on 2026-01-01\\n"\n' "$2" > "$_b/golangci-lint"
chmod +x "$_b/golangci-lint"
fi
if [ -n "$3" ]; then
printf '#!/bin/sh\ncase "$1" in version) printf "go version %s linux/amd64\\n";; *) printf "\\n";; esac\n' "$3" > "$_b/go"
chmod +x "$_b/go"
fi
printf '%s' "$_b"
}
f=$(new_fixture golangci_old); wire_clean "$f"
b=$(mk_toolchain "$f" go1.26.1 go1.27.0)
_out=$(PATH="$b:$PATH" HOME="$f/home" DOCTOR_CHECKOUT="$f/ck" "$DOCTOR" 2>&1); _rc=$?
if [ "$_rc" -ne 0 ] && printf '%s' "$_out" | grep -qi 'golangci'; then
pass=$((pass + 1)); printf ' ok golangci-lint older than GOROOT is reported\n'
else
fail=$((fail + 1)); printf ' FAIL golangci-lint older than GOROOT is reported\n rc=%s got: %s\n' "$_rc" "$_out"
fi
# Matching versions are the normal case and must stay silent.
f=$(new_fixture golangci_ok); wire_clean "$f"
b=$(mk_toolchain "$f" go1.27.0 go1.27.0)
_out=$(PATH="$b:$PATH" HOME="$f/home" DOCTOR_CHECKOUT="$f/ck" "$DOCTOR" 2>&1); _rc=$?
if [ "$_rc" -eq 0 ]; then
pass=$((pass + 1)); printf ' ok golangci-lint matching GOROOT is silent\n'
else
fail=$((fail + 1)); printf ' FAIL golangci-lint matching GOROOT is silent\n rc=%s got: %s\n' "$_rc" "$_out"
fi
# A NEWER lint than GOROOT is fine -- the constraint is one-directional.
f=$(new_fixture golangci_newer); wire_clean "$f"
b=$(mk_toolchain "$f" go1.28.0 go1.27.0)
_out=$(PATH="$b:$PATH" HOME="$f/home" DOCTOR_CHECKOUT="$f/ck" "$DOCTOR" 2>&1); _rc=$?
if [ "$_rc" -eq 0 ]; then
pass=$((pass + 1)); printf ' ok golangci-lint newer than GOROOT is not a fault\n'
else
fail=$((fail + 1)); printf ' FAIL golangci-lint newer than GOROOT is not a fault\n rc=%s got: %s\n' "$_rc" "$_out"
fi
# Fail OPEN on anything unparseable. doctor runs in a SessionStart hook on every
# machine; a toolchain it cannot read is not a fault it may assert. (A host with
# no Go at all takes the same path -- command -v simply misses.)
f=$(new_fixture golangci_garbled); wire_clean "$f"
_b="$f/bin"; mkdir -p "$_b"
printf '#!/bin/sh\nprintf "golangci-lint dev build\\n"\n' > "$_b/golangci-lint"
printf '#!/bin/sh\nprintf "go version devel\\n"\n' > "$_b/go"
chmod +x "$_b/golangci-lint" "$_b/go"
_out=$(PATH="$_b:$PATH" HOME="$f/home" DOCTOR_CHECKOUT="$f/ck" "$DOCTOR" 2>&1); _rc=$?
if [ "$_rc" -eq 0 ]; then
pass=$((pass + 1)); printf ' ok an unreadable toolchain version is not asserted as a fault\n'
else
fail=$((fail + 1)); printf ' FAIL an unreadable toolchain version is not asserted as a fault\n rc=%s got: %s\n' "$_rc" "$_out"
fi
printf '\ndoctor: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+103
View File
@@ -0,0 +1,103 @@
#!/bin/sh
# Tests for hooks/pre-task-ritual.sh — the UserPromptSubmit hook that surfaces
# rule 0 on every prompt (#44).
#
# WHY THIS EXISTS, AND WHAT IT DELIBERATELY DOES NOT CLAIM
# This hook is three lines and echoes a string. It has no bash-isms, so it
# cannot fail the way secret-guard appeared to; that risk was overstated when
# this test was requested. What it CAN do is stop working silently, and its
# whole job is to be visible -- if the text stops appearing, rule 0 stops being
# surfaced on every prompt and nothing announces that.
#
# THE ASSERTION THAT MATTERS IS REGISTRATION.
# A hook file that exists but is not in hooks.json does absolutely nothing, and
# looks identical on disk to one that works. So this derives the command from
# hooks.json rather than assuming one: the registration is the source of truth
# for how the hook is invoked, and a test that invokes it its own way is
# testing something the harness never runs.
#
# That distinction is not hypothetical. secret_guard_test.sh piped its hook into
# `sh` while hooks.json runs it with `bash`; on Debian /bin/sh is dash, which
# has no `set -o pipefail`, and the suite failed 20/20 in CI against a hook that
# was fine in production. The test was wrong, not the hook. Deriving the
# invocation removes the opportunity for that class of mistake entirely.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
HOOK="$ROOT/hooks/pre-task-ritual.sh"
REG="$ROOT/hooks/hooks.json"
pass=0
fail=0
ok() { pass=$((pass + 1)); printf ' ok %s\n' "$1"; }
no() { fail=$((fail + 1)); printf ' FAIL %s\n' "$1"; }
if ! command -v jq >/dev/null 2>&1; then
printf ' SKIP: jq not installed -- cannot read the hook registration\n'
exit 0
fi
[ -f "$HOOK" ] || { printf ' FAIL %s does not exist\n' "$HOOK"; exit 1; }
ok 'hook file exists'
[ -f "$REG" ] || { printf ' FAIL %s does not exist\n' "$REG"; exit 1; }
# 1. Registered, and on the right event. UserPromptSubmit is what makes it fire
# on every prompt; registered under any other event it would still "exist"
# and still do nothing useful.
CMD=$(jq -r '
.hooks.UserPromptSubmit[]?.hooks[]?
| select(.command | test("pre-task-ritual"))
| .command
' "$REG" 2>/dev/null | head -1)
if [ -n "$CMD" ]; then
ok 'registered on UserPromptSubmit in hooks.json'
else
no 'not registered on UserPromptSubmit -- the file would exist and never run'
printf ' %d passed, %d failed\n' "$pass" "$fail"
exit 1
fi
# 2. Run it EXACTLY as registered, with the harness variable substituted the way
# the harness substitutes it.
OUT=$(CLAUDE_PLUGIN_ROOT="$ROOT" sh -c "$CMD" 2>&1)
rc=$?
[ "$rc" -eq 0 ] && ok 'exits 0 as registered' \
|| no "exited $rc -- a non-zero UserPromptSubmit hook can block the prompt"
# 3. The content is the point. A hook that runs and says nothing is the same as
# no hook, and that is the failure this pins.
if [ -n "$OUT" ]; then
ok 'produces output'
else
no 'produced no output -- rule 0 would silently stop being surfaced'
fi
case "$OUT" in
*"Rule 0"*) ok 'names rule 0' ;;
*) no "does not name rule 0: '$OUT'" ;;
esac
# Each of the four steps, individually. Naming rule 0 while having quietly lost
# a step is the plausible drift, not losing the whole line.
for step in brain_query skill 'failing test' 'success criterion'; do
case "$OUT" in
*"$step"*) ok "states the '$step' step" ;;
*) no "lost the '$step' step from the ritual" ;;
esac
done
# 4. The interpreter named in the registration must actually exist, or the hook
# fails on that host with a message nobody reads.
INTERP=${CMD%% *}
if command -v "$INTERP" >/dev/null 2>&1; then
ok "registered interpreter '$INTERP' is available"
else
no "registered interpreter '$INTERP' not found on this host"
fi
printf ' %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+15 -1
View File
@@ -24,7 +24,7 @@ decision_for() {
_tool=${2:-Bash}
jq -n --arg t "$_tool" --arg c "$_cmd" \
'{tool_name: $t, tool_input: {command: $c}}' \
| sh "$GUARD" \
| "$GUARD" \
| jq -r '.hookSpecificOutput.permissionDecision // "MISSING"'
}
@@ -42,6 +42,20 @@ expect() {
fi
}
# Invoked as a PROGRAM above, never as `sh "$GUARD"`. Claude Code executes the
# hook directly, so its `#!/usr/bin/env bash` shebang decides the interpreter;
# piping it into `sh` overrides that and tests something production never runs.
#
# It is not academic. On Debian /bin/sh is dash, which has no `set -o
# pipefail`, so the guard died on line 23 before reading a byte of input and
# every deny case returned empty -- 20 of 20 assertions failed in CI while
# passing on a host where /bin/sh happens to be bash. A security gate that
# tests green and cannot start is the worst of both.
if [ ! -x "$GUARD" ]; then
printf 'FAIL: %s is not executable -- it must run via its own shebang\n' "$GUARD"
exit 1
fi
if [ ! -f "$GUARD" ]; then
printf 'FAIL: %s does not exist\n' "$GUARD"
exit 1