fix(skills): document a session_log call the tool would actually accept
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s

Nine skills documented the payload. None named `session_id`, which is the
tool's only required field. So every documented call was one the tool rejects —
an agent following any of them either failed the call or invented an id, and
invented ids do not group, which means the pass-rate the /pass-rate endpoint
computes was being assembled from entries that cannot be joined.

Found by the #19 pressure test against `debug`, which invented
`debug-signed-artifact-missing-2026-08-29` and said so: "my three entries are
only groupable by luck of naming". The grep afterwards showed it was fleet-wide,
not a `debug` bug.

Also removed: `duration_ms: <wall-clock>` in eight of them. An agent does not
observe a clock. The same run omitted it "rather than fabricate", which is the
good outcome; the bad outcome is the run that fabricates, and a made-up number
in a metrics stream is worse than a missing one. It is an optional field and is
now simply absent.

bin/check-session-log gates both, with tests. This is exactly the case #19 has
in mind for a small tool over prose: the rule is a grep, so it should be a gate,
not a note asking people to remember.

Closes part of #36
Bump-Type: patch
This commit is contained in:
2026-08-29 14:49:20 +02:00
parent b8bacfa7b3
commit b616c51a2c
12 changed files with 124 additions and 14 deletions
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# check-session-log — a skill that documents a session_log call must document a
# call the tool would actually accept.
#
# Nine skills documented the payload and none named `session_id`, which is the
# tool's only REQUIRED field. An agent following any of them either failed the
# call or invented an id — and invented ids do not group, so the pass-rate the
# endpoint computes was being assembled from entries that could not be joined
# (#36, found by a pressure test against `debug`).
#
# Also rejected: `duration_ms: <wall-clock>`. An agent does not observe a clock.
# A documented field nobody can honestly populate gets fabricated or silently
# dropped, and fabricated numbers in a metrics stream are worse than no numbers.
#
# Usage: check-session-log [checkout-root]
set -u
ROOT=${1:-$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)}
fail=0
for dir in "$ROOT"/skills/*/; do
[ -f "$dir/SKILL.md" ] || continue
name=$(basename "$dir")
grep -q 'session_log' "$dir/SKILL.md" || continue
if ! grep -q 'session_id' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents session_log without session_id — the tool's only required field" >&2
fail=1
fi
if grep -q 'duration_ms:[[:space:]]*<' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents duration_ms as a placeholder an agent cannot measure — omit it or pass a real measurement" >&2
fail=1
fi
done
exit $fail