fix(skills): document a session_log call the tool would actually accept
Nine skills documented the payload. None named `session_id`, which is the tool's only required field. So every documented call was one the tool rejects — an agent following any of them either failed the call or invented an id, and invented ids do not group, which means the pass-rate the /pass-rate endpoint computes was being assembled from entries that cannot be joined. Found by the #19 pressure test against `debug`, which invented `debug-signed-artifact-missing-2026-08-29` and said so: "my three entries are only groupable by luck of naming". The grep afterwards showed it was fleet-wide, not a `debug` bug. Also removed: `duration_ms: <wall-clock>` in eight of them. An agent does not observe a clock. The same run omitted it "rather than fabricate", which is the good outcome; the bad outcome is the run that fabricates, and a made-up number in a metrics stream is worse than a missing one. It is an optional field and is now simply absent. bin/check-session-log gates both, with tests. This is exactly the case #19 has in mind for a small tool over prose: the rule is a grep, so it should be a gate, not a note asking people to remember. Closes part of #36 Bump-Type: patch
This commit is contained in:
Executable
+39
@@ -0,0 +1,39 @@
|
||||
#!/bin/sh
|
||||
# check-session-log — a skill that documents a session_log call must document a
|
||||
# call the tool would actually accept.
|
||||
#
|
||||
# Nine skills documented the payload and none named `session_id`, which is the
|
||||
# tool's only REQUIRED field. An agent following any of them either failed the
|
||||
# call or invented an id — and invented ids do not group, so the pass-rate the
|
||||
# endpoint computes was being assembled from entries that could not be joined
|
||||
# (#36, found by a pressure test against `debug`).
|
||||
#
|
||||
# Also rejected: `duration_ms: <wall-clock>`. An agent does not observe a clock.
|
||||
# A documented field nobody can honestly populate gets fabricated or silently
|
||||
# dropped, and fabricated numbers in a metrics stream are worse than no numbers.
|
||||
#
|
||||
# Usage: check-session-log [checkout-root]
|
||||
|
||||
set -u
|
||||
|
||||
ROOT=${1:-$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)}
|
||||
fail=0
|
||||
|
||||
for dir in "$ROOT"/skills/*/; do
|
||||
[ -f "$dir/SKILL.md" ] || continue
|
||||
name=$(basename "$dir")
|
||||
|
||||
grep -q 'session_log' "$dir/SKILL.md" || continue
|
||||
|
||||
if ! grep -q 'session_id' "$dir/SKILL.md"; then
|
||||
echo "ERROR: skills/$name documents session_log without session_id — the tool's only required field" >&2
|
||||
fail=1
|
||||
fi
|
||||
|
||||
if grep -q 'duration_ms:[[:space:]]*<' "$dir/SKILL.md"; then
|
||||
echo "ERROR: skills/$name documents duration_ms as a placeholder an agent cannot measure — omit it or pass a real measurement" >&2
|
||||
fail=1
|
||||
fi
|
||||
done
|
||||
|
||||
exit $fail
|
||||
Reference in New Issue
Block a user