Compare commits

...
6 Commits
Author SHA1 Message Date
mathias b616c51a2c fix(skills): document a session_log call the tool would actually accept
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
Nine skills documented the payload. None named `session_id`, which is the
tool's only required field. So every documented call was one the tool rejects —
an agent following any of them either failed the call or invented an id, and
invented ids do not group, which means the pass-rate the /pass-rate endpoint
computes was being assembled from entries that cannot be joined.

Found by the #19 pressure test against `debug`, which invented
`debug-signed-artifact-missing-2026-08-29` and said so: "my three entries are
only groupable by luck of naming". The grep afterwards showed it was fleet-wide,
not a `debug` bug.

Also removed: `duration_ms: <wall-clock>` in eight of them. An agent does not
observe a clock. The same run omitted it "rather than fabricate", which is the
good outcome; the bad outcome is the run that fabricates, and a made-up number
in a metrics stream is worse than a missing one. It is an optional field and is
now simply absent.

bin/check-session-log gates both, with tests. This is exactly the case #19 has
in mind for a small tool over prose: the rule is a grep, so it should be a gate,
not a note asking people to remember.

Closes part of #36
Bump-Type: patch
2026-08-29 14:49:20 +02:00
mathias b8bacfa7b3 chore(release): plugin.json version 0.13.2
Skip-Release: true
2026-08-29 12:22:35 +00:00
mathias c377c23e21 fix(mcp): correct the parked reason — the tokens work, the delivery does not
CI / Check / Test (push) Successful in 2s
CI / Tag release (push) Successful in 1s
The previous note blamed infra#224 for cad and gitea-mcp being unreachable.
Both claims were wrong. infra#224 is closed, and closed deliberately: rather
than widening koala-homelab's grants, a second scoped token was provisioned at
~/.op-agent-write.env for AgentSecrets. Sourcing it makes `op vault list`
return AgentSecrets, and an MCP initialize against both endpoints returns 200 —
serverInfo gitea-mcp 0.1.0 and cad aed2aaf. Nothing about the services or the
proxying is blocked.

What is actually blocked is delivery into the session environment. By design no
single identity on koala reads both vaults, and the session env comes from one
`op run --env-file ~/.op-env` under the HomeLab-only token. A mixed-vault file
does not degrade gracefully: tested on a two-line copy, `op run` aborts on the
unresolvable reference and BRAIN_MCP_TOKEN goes unresolved with it — so
appending to ~/.op-env would take out the MCP servers that currently work.

Refs #32, #33
Bump-Type: patch
2026-08-29 14:22:17 +02:00
mathias 8f9d5dd8bd chore(release): plugin.json version 0.13.1
Skip-Release: true
2026-08-29 12:19:31 +00:00
mathias 7ba01abdbe chore(mcp): park cad and gitea-mcp entries, blocked on vault visibility
CI / Tag release (push) Successful in 1s
CI / Check / Test (push) Successful in 1s
Both services are now proxied from koala and reachable: tailscale serve 8445 ->
127.0.0.1:31877 (cad) and 8446 -> 127.0.0.1:30340 (gitea-mcp). Both answer 401
unauthenticated, so the path is right and auth is enforced.

They stay out of mcpServers because neither token resolves on this host. The
static tokens live in the 1Password AgentSecrets vault; koala's op identity sees
HomeLab only:

  op vault list                    -> HomeLab, and nothing else
  op://HomeLab/CAD_STATIC_TOKEN    -> could not find item
  op://HomeLab/gitea-mcp-shared-token -> could not find item

That is infra#224, not the OAuth-discovery bug #32 and #33 both flagged as the
likely blocker. A live entry that cannot authenticate would fail on every
session start, so the wiring is recorded rather than enabled.

The item names come from the ExternalSecrets, read by reference only — no
secret value was retrieved, printed, or transformed at any point.

Refs #32, #33
Bump-Type: patch
2026-08-29 14:18:38 +02:00
mathias 9a81a707af chore(release): plugin.json version 0.13.0
Skip-Release: true
2026-08-29 12:10:37 +00:00
14 changed files with 143 additions and 16 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "skills", "name": "skills",
"description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.", "description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.",
"version": "0.12.1", "version": "0.13.2",
"author": { "author": {
"name": "Mathias", "name": "Mathias",
"url": "https://git.d-ma.be/mathias" "url": "https://git.d-ma.be/mathias"
+17
View File
@@ -12,5 +12,22 @@
"type": "http", "type": "http",
"url": "https://koala.tail30472e.ts.net:8444/mcp" "url": "https://koala.tail30472e.ts.net:8444/mcp"
} }
},
"_parked": {
"_reason": "cad and gitea-mcp are proxied, reachable and AUTHENTICATING: tailscale serve 8445 -> 127.0.0.1:31877 and 8446 -> 127.0.0.1:30340; an MCP initialize with the correct bearer returns 200 from both (serverInfo gitea-mcp 0.1.0, cad aed2aaf). They stay parked only because nothing puts those two tokens into the session environment. Both live in the 1Password AgentSecrets vault, and by deliberate design (infra#224, closed) no single identity on koala reads both vaults: koala-homelab is HomeLab-only and resolves ~/.op-env, koala-agentsecrets-write is AgentSecrets-only and lives in ~/.op-agent-write.env. Adding an AgentSecrets reference to ~/.op-env does NOT degrade gracefully -- `op run` fails wholesale and every other token, BRAIN_MCP_TOKEN included, goes unresolved with it. Enabling needs a read-scoped AgentSecrets identity in the session-env path; see mathias/skills#32, #33.",
"cad": {
"type": "http",
"url": "https://koala.tail30472e.ts.net:8445/mcp",
"headers": {
"Authorization": "Bearer ${CAD_MCP_TOKEN}"
}
},
"gitea": {
"type": "http",
"url": "https://koala.tail30472e.ts.net:8446/mcp",
"headers": {
"Authorization": "Bearer ${GITEA_MCP_STATIC_TOKEN}"
}
}
} }
} }
+5 -1
View File
@@ -110,7 +110,11 @@ tasks:
# harness instead of erroring. # harness instead of erroring.
sh '{{.TASKFILE_DIR}}/bin/check-surfaces' '{{.TASKFILE_DIR}}' || fail=1 sh '{{.TASKFILE_DIR}}/bin/check-surfaces' '{{.TASKFILE_DIR}}' || fail=1
# 5. Plugin manifest version must match the latest tag, or installers # 5. A documented session_log call must be one the tool would accept:
# session_id is its only required field, and nine skills omitted it.
sh '{{.TASKFILE_DIR}}/bin/check-session-log' '{{.TASKFILE_DIR}}' || fail=1
# 6. Plugin manifest version must match the latest tag, or installers
# never see the update (Claude Code gates updates on this field). # never see the update (Claude Code gates updates on this field).
if [ -f '{{.PLUGIN_MANIFEST}}' ]; then if [ -f '{{.PLUGIN_MANIFEST}}' ]; then
manifest_v=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' '{{.PLUGIN_MANIFEST}}' | head -1) manifest_v=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' '{{.PLUGIN_MANIFEST}}' | head -1)
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# check-session-log — a skill that documents a session_log call must document a
# call the tool would actually accept.
#
# Nine skills documented the payload and none named `session_id`, which is the
# tool's only REQUIRED field. An agent following any of them either failed the
# call or invented an id — and invented ids do not group, so the pass-rate the
# endpoint computes was being assembled from entries that could not be joined
# (#36, found by a pressure test against `debug`).
#
# Also rejected: `duration_ms: <wall-clock>`. An agent does not observe a clock.
# A documented field nobody can honestly populate gets fabricated or silently
# dropped, and fabricated numbers in a metrics stream are worse than no numbers.
#
# Usage: check-session-log [checkout-root]
set -u
ROOT=${1:-$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)}
fail=0
for dir in "$ROOT"/skills/*/; do
[ -f "$dir/SKILL.md" ] || continue
name=$(basename "$dir")
grep -q 'session_log' "$dir/SKILL.md" || continue
if ! grep -q 'session_id' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents session_log without session_id — the tool's only required field" >&2
fail=1
fi
if grep -q 'duration_ms:[[:space:]]*<' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents duration_ms as a placeholder an agent cannot measure — omit it or pass a real measurement" >&2
fail=1
fi
done
exit $fail
+2 -2
View File
@@ -234,8 +234,8 @@ wrong" — it means "nothing on the known list is wrong."
Call `session_log` once at the end of every phase to record the outcome. Call `session_log` once at the end of every phase to record the outcome.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "cad-babysit", phase: "<phase-name>", final_status: "pass" | - `session_log` with `{session_id: "<this session's id>", skill: "cad-babysit", phase: "<phase-name>", final_status: "pass" |
"fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "fail" | "skip", message: "<one-line summary>", project_root:
"<absolute path>"}` "<absolute path>"}`
**Phases for this skill:** query, read-comments, read-actions-logs, classify-fault-vs-outcome, **Phases for this skill:** query, read-comments, read-actions-logs, classify-fault-vs-outcome,
+1 -1
View File
@@ -253,7 +253,7 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "code-review", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "code-review", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** phase-0-iron-laws, phase-1-context, phase-2-architectural-analysis, phase-3-go-checklist, phase-4-code-smell-detection, phase-5-test-quality **Phases for this skill:** phase-0-iron-laws, phase-1-context, phase-2-architectural-analysis, phase-3-go-checklist, phase-4-code-smell-detection, phase-5-test-quality
+1 -1
View File
@@ -154,7 +154,7 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "debug", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "debug", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** read-failure, generate-hypotheses, output **Phases for this skill:** read-failure, generate-hypotheses, output
+1 -1
View File
@@ -185,7 +185,7 @@ If silhouette ≤ 0.20:
**After experiment concludes:** **After experiment concludes:**
- `brain_write` to `jepa-fx/failures/` with any new failure modes discovered - `brain_write` to `jepa-fx/failures/` with any new failure modes discovered
- `session_log` with outcome - `session_log` with `{session_id: "<this session's id>", skill: "experiment-spec", phase: "conclude", final_status: "pass" | "fail" | "skip", message: "<one-line outcome>"}`
## Cross-References ## Cross-References
+1 -1
View File
@@ -245,7 +245,7 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "feature-spec", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "feature-spec", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** draft **Phases for this skill:** draft
+1 -1
View File
@@ -116,7 +116,7 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "session-retrospective", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "session-retrospective", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** surfaced **Phases for this skill:** surfaced
+1 -1
View File
@@ -332,7 +332,7 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of each phase:** **At end of each phase:**
- `session_log` with `{skill: "spec-driven-dev", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "spec-driven-dev", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** specify, plan, tasks, implement **Phases for this skill:** specify, plan, tasks, implement
+3 -3
View File
@@ -406,13 +406,13 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of `red` phase:** **At end of `red` phase:**
- `session_log` with `{skill: "tdd", phase: "red", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "red", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**At end of `green` phase:** **At end of `green` phase:**
- `session_log` with `{skill: "tdd", phase: "green", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "green", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**At end of `refactor` phase:** **At end of `refactor` phase:**
- `session_log` with `{skill: "tdd", phase: "refactor", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "refactor", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Status semantics:** **Status semantics:**
- `pass` — the phase's intended outcome was reached (red: test fails as expected; green: test passes; refactor: tests still pass after refactor). - `pass` — the phase's intended outcome was reached (red: test fails as expected; green: test passes; refactor: tests still pass after refactor).
+2 -2
View File
@@ -231,10 +231,10 @@ Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither. treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of `read` phase:** **At end of `read` phase:**
- `session_log` with `{skill: "trainer", phase: "read", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "trainer", phase: "read", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**At end of `write` phase:** **At end of `write` phase:**
- `session_log` with `{skill: "trainer", phase: "write", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", duration_ms: <wall-clock>, project_root: "<absolute path>"}` - `session_log` with `{session_id: "<this session's id>", skill: "trainer", phase: "write", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Phases for this skill:** read, write **Phases for this skill:** read, write
+67
View File
@@ -0,0 +1,67 @@
#!/bin/sh
# Tests for bin/check-session-log.
#
# The regression: nine skills documented a session_log payload and not one of
# them named `session_id` — the tool's ONLY required field. Every documented
# call was incomplete, so an agent following any of them either failed the call
# or invented an id, and invented ids do not group. Found by a pressure test
# against `debug`, then found to be fleet-wide (#36).
#
# Also gated: `duration_ms: <wall-clock>`, which an agent cannot measure. A
# documented field nobody can honestly populate gets fabricated or silently
# dropped; both are worse than omitting it.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
SUT="$ROOT/bin/check-session-log"
pass=0
fail=0
WORK=${TMPDIR:-/tmp}/check_session_log_test.$$
if [ ! -f "$SUT" ]; then
printf 'FAIL: %s does not exist\n' "$SUT"
exit 1
fi
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
# fixture <name> <body> — a checkout with one skill whose SKILL.md is <body>.
fixture() {
_f="$WORK/$1"
mkdir -p "$_f/skills/alpha"
printf -- '---\nname: alpha\ndescription: a\n---\n\n%s\n' "$2" > "$_f/skills/alpha/SKILL.md"
printf '%s' "$_f"
}
# expect <rc> <label> <fixture> [needle]
expect() {
_want=$1; _label=$2; _fx=$3; _needle=${4:-}
_out=$(sh "$SUT" "$_fx" 2>&1); _rc=$?
if [ "$_rc" -ne "$_want" ]; then
fail=$((fail + 1)); printf ' FAIL %s (want rc=%s got %s)\n%s\n' "$_label" "$_want" "$_rc" "$_out"; return
fi
if [ -n "$_needle" ] && ! printf '%s' "$_out" | grep -q "$_needle"; then
fail=$((fail + 1)); printf ' FAIL %s (rc ok, report never says %s)\n%s\n' "$_label" "$_needle" "$_out"; return
fi
pass=$((pass + 1)); printf ' ok %s\n' "$_label"
}
expect 0 'a skill that never mentions session_log is fine' \
"$(fixture quiet 'Just a skill. No logging here.')"
expect 0 'a documented call naming session_id passes' \
"$(fixture good 'Call `session_log` with `{session_id: "<id>", skill: "alpha", phase: "red", final_status: "pass"}`')"
expect 1 'a documented call omitting session_id fails' \
"$(fixture bad 'Call `session_log` with `{skill: "alpha", phase: "red", final_status: "pass"}`')" \
'session_id'
expect 1 'documenting duration_ms as wall-clock fails' \
"$(fixture clock 'Call `session_log` with `{session_id: "<id>", skill: "alpha", duration_ms: <wall-clock>}`')" \
'duration_ms'
printf '\ncheck_session_log: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]