Compare commits

..
10 Commits
Author SHA1 Message Date
mathias fc6d5628a7 fix(adr): template emitted invalid YAML frontmatter
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
The frontmatter exists to be machine-read. Two shapes this template
suggested do not parse, so any ADR that copied them dropped silently out of
every "what is still open?" query while looking correct:

  supersedes:    -                 a bare dash is a YAML sequence entry
  deciders:      pending: Mathias  the unquoted second colon reads as a
                                   nested mapping

Both were found live during an estate-wide ADR survey on 2026-09-05:
openbanking-api ADR-0001 (the one genuinely open decision in the estate --
invisible to the survey until fixed) and infra ADR-0019, each copied
verbatim from here.

Template corrected to `null` and a quoted `"pending: <name>"`, plus a short
section after the fence explaining why and giving a one-line yaml.safe_load
check to run before committing. The explanation is deliberately OUTSIDE the
fenced block so it is not copied into the ADR being written.
2026-09-05 12:25:10 +02:00
mathias 1f7f53e6c0 chore(release): plugin.json version 0.18.1
Skip-Release: true
2026-08-30 09:49:22 +00:00
mathias bedb6d278d fix(test): stop the Antigravity cases failing on a host without jq
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
main has been red since e9ad692. The gate passed; the shell suite failed on
three Antigravity cases.

Not a bug in the wiring. install.sh treats jq as optional by design — every
other target is dependency-free, so wire_antigravity warns and skips when jq is
missing. It did exactly that on the runner:

  [skills] Antigravity: jq not found — skipping

The tests were the problem: they inherited whatever jq the host happened to
have. koala has it and reported 12/12; the runner does not and reported 3
failures on the same commit. Works-on-my-machine, caught by CI doing its job.

Two changes:

- The three merge-behaviour cases now run only where jq exists, and print a SKIP
  naming what is not covered. A case that fails because the host lacks an
  optional dependency is a false red, and false reds are how a suite stops being
  read.
- The skip path gets a real test of its own, because it is a contract rather than
  a fallback. install.sh gains SKILLS_JQ so the jq-absent branch is exercisable
  on a host that has jq — the same seam pattern as DOCTOR_LATEST_TAG_CMD. It
  asserts no skills.json is written, the skip is reported, and the other
  harnesses still wire.

Verified both ways on koala by building a jq-free PATH:
  with jq:    install: 13 passed, 0 failed, 0 skipped
  without jq: install: 10 passed, 0 failed, 3 skipped

Consequence worth naming: CI now SKIPS the merge assertions rather than running
them, so idempotency and entry-preservation are untested there until the runner
gets jq. Filed as #42.

Refs #41
Bump-Type: patch
2026-08-30 11:48:56 +02:00
mathiasandClaude Sonnet 5 e9ad692418 feat(install): re-add Antigravity as a wiring target
CI / Check / Test (push) Failing after 1s
CI / Tag release (push) Skipped
Antigravity discovers skills via a path-entries JSON
(~/.gemini/config/skills.json), not a symlink directory. One entry points
at $AGENTS_DIR — the same managed, surfaces-filtered, collision-resolved
directory Crush/OpenCode/Pi already read — so Antigravity sees exactly the
same skill set as every other harness, for free.

Requires jq; every other target in this script stays dependency-free on
purpose, so wire_antigravity skips with a warning (not a hard failure) if
jq is missing. Merges into skills.json rather than overwriting it: the
file is documented as user-editable (json_configs.md — e.g. a personal
skills dir), so a hand-added entry must survive a re-run.

Was previously removed as "unused" (mathias/skills#41 — Antigravity usage
picked back up). Mistral Vibe stays removed.

Closes #41

Bump-Type: minor

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-30 08:58:34 +02:00
mathias 86529d2ac5 chore(release): plugin.json version 0.18.0
Skip-Release: true
2026-08-29 13:34:27 +00:00
mathias d6e892ac4c feat(skill): traceability — a chain of checked links, and a tool that says what it cannot check
CI / Check / Test (push) Successful in 2s
CI / Tag release (push) Successful in 1s
Forward trace: requirement → decision → commit → gate → verification → CI on
that sha → release → closure. The distinction the skill exists for is that each
link is either checked or named as unchecked; a trace where every link is
asserted is a narrative, and narratives are what audits exist to defeat.

`trace.sh` does the mechanical part — commit, files, test files touched, release
containing it, referenced issues as URLs — and then always prints the four things
git cannot prove. The GREEN run's own accounting: "the tool did the mechanical
15%. Its real contribution is the 'Cannot verify from here' block, which is a
checklist, not an output." Kept, because it is true.

The RED run established the shape by tracing a commit by hand in sixteen steps.
Its decisive moves were reading issue COMMENTS — where the decisions actually
live — and running a probe that falsified a claim. Both are now steps.

The GREEN run then used step 5 against a gate I shipped this morning, and broke
it. It also found four things wrong with what I had written:

- The slug sed used `[^/]+?` expecting a lazy quantifier. POSIX ERE has none, so
  `.git` was swallowed and every emitted evidence URL carried the suffix. It
  happened to work against Gitea and would 404 on GitHub — a bad failure for the
  one tool whose pitch is that an unchecked link must not look checked.
- "Prints every issue it references" was overclaimed: it greps the commit body,
  so it structurally cannot find the issue that cites the COMMIT. That is the
  commoner break — the issue filed afterwards that falsifies the outcome — and
  it is now step 6 with the two searches that find it.
- Step 3 told you to check which job passed, and the estate's own
  `workflow_run_status` returns no jobs. A step whose tooling dead-ends is where
  a trace becomes "the run was green"; the REST fallback is now written out.
- The output format did not survive contact. It grew four slots, each forced by
  a real trace: two requirements differing in kind (a defect report and a
  standing bar), the gate as an artifact separate from the test, the RED/GREEN
  baseline against the parent, and the commit's factual claims checked. Without
  the last one, "commit → verified" means only "this sha exists".

Also: `git describe --contains` reads local tags, so the tool now says so on
every hit rather than only when it finds nothing.

Closes #10
Bump-Type: minor
2026-08-29 15:34:11 +02:00
mathias 4d6ebb626f fix(check): read the session_log payload, not the file
A traceability probe falsified this gate four hours after I shipped it. The
check was `grep -q 'session_id' "$dir/SKILL.md"` — file-scoped. So the probe
stripped session_id from all three documented payloads in tdd/SKILL.md, added
one prose line saying "the tool has a required session_id field", and the gate
returned 0. The exact regression it exists to prevent was back in the file, and
the gate said nothing.

The duration_ms rule was no better: it matched one spelling, the historical one.
Both of these passed — `"duration_ms": <wall-clock>` with a JSON-quoted key, and
`duration_ms: elapsed wall-clock ms` with no angle brackets. It was a regression
test against the nine strings that existed that morning, described in the commit
message as a rule.

Now each documented payload is checked on its own line and reported with a line
number, and duration_ms is rejected in any spelling unless it carries an actual
number — the objection was always to a placeholder no agent can fill, not to the
field.

Five new cases, each one a probe from that run: prose mentioning the word while
the payload omits it, one bad payload among several, both duration_ms spellings,
and a real measurement being allowed.

Also fixed while writing them: my first regex had `[[:space:]]*` before a
negated class that did not exclude whitespace, so the star matched empty and the
space itself tripped the check — a real measurement failed. The tests caught it
immediately, which is the argument for writing them from the probes rather than
from the intent.

Refs #36, #39
Bump-Type: patch
2026-08-29 15:34:11 +02:00
mathias 88a4fa407b chore(release): plugin.json version 0.17.1
Skip-Release: true
2026-08-29 13:27:06 +00:00
mathias 808af7b020 chore(release): plugin.json version 0.17.0
Skip-Release: true
2026-08-29 13:27:05 +00:00
mathias 40405cd95a fix(check): make the escalation ratchet actually ratchet
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
"That list only shrinks" was asserted in the commit message, the script header,
the pending file's own header, the test header, and the issue comment. It was
enforced nowhere.

A traceability probe falsified it in one run: extract the commit to a temp tree,
add a skill with no `## Escalation` section, append its name to
bin/escalation-pending.txt — gate returns 0. The list could grow silently and
forever. "Self-cleaning" was true; "only shrinks" was a social convention wearing
a mechanism's clothes, and this repo has spent a day on what those cost.

The gate now diffs the pending list against the version committed at HEAD and
fails on any added entry, naming it. Where the comparison is impossible — not a
git checkout — it says so rather than passing silently, because a ratchet that
cannot be verified must not look like one that was.

Four new cases: unchanged list, shrinking list, growing list, and the
unverifiable case.

Refs #37
Bump-Type: patch
2026-08-29 15:26:24 +02:00
13 changed files with 654 additions and 12 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "skills",
"description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.",
"version": "0.16.0",
"version": "0.18.1",
"author": {
"name": "Mathias",
"url": "https://git.d-ma.be/mathias"
+3 -1
View File
@@ -100,9 +100,11 @@ default to point into `~/dev`.
| `AGENTS_SKILLS_DIR` | `$HOME/.agents/skills` |
| `CRUSH_SKILLS_DIR` | `$HOME/.config/crush/skills` |
| `OPENCODE_SKILLS_DIR` | `$HOME/.config/opencode/skills` |
| `ANTIGRAVITY_SKILLS_JSON` | `$HOME/.gemini/config/skills.json` (registers `AGENTS_SKILLS_DIR` as a path entry; needs `jq`) |
| `SKILLS_EXTRA_SOURCES` | unset (colon-separated extra source directories) |
Antigravity and Mistral Vibe targets were removed — unused.
The Mistral Vibe target was removed — unused. Antigravity's was removed for
the same reason and re-added once it came back into use.
## Checking a host
+1
View File
@@ -49,6 +49,7 @@ the SKILL.md, not the table.
| `tdd` | Test-Driven Development enforcement | Starting any new feature or fixing a bug | claude-code, library |
| `telos-load` | Load TELOS intention substrate at session start | Starting any koala session; before architectural decisions; CAD pipeline entry | claude-code, library |
| `test-design` | Dave Farley test design scoring (Farley Index) | Reviewing or writing tests | claude-code, library |
| `traceability` | Forward trace — requirement to decision to commit to verification to release, and what cannot be checked | Proving why a change exists; auditing a claim; before an attestation or a compliance ask | claude-code, library |
| `trainer` | Two-phase brain curation (score candidates, write past quality gate) | Periodic brain pruning and curation | claude-code, library |
| `user-stories` | Elephant Carpaccio story slicing | Breaking down features | claude-code, library |
| `web-shot` | Screenshot a running web UI via a koala k3s Playwright job | Visual verification of a running UI when no local browser exists | claude-code, library |
+24
View File
@@ -61,4 +61,28 @@ if [ -f "$PENDING" ]; then
done
fi
# --- the ratchet must actually ratchet ---------------------------------------
# "This list only shrinks" was asserted in five places and enforced in none. A
# traceability probe falsified it in one run: add an unsectioned skill, append it
# to the list, gate returns 0. So compare against the list as committed at HEAD —
# a convention nothing checks is a convention, and this repo has spent a day on
# what those cost.
if [ -f "$PENDING" ]; then
if ! git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then
echo "WARN: $ROOT is not a git checkout — cannot verify the pending list only shrank" >&2
elif ! git -C "$ROOT" cat-file -e HEAD:bin/escalation-pending.txt 2>/dev/null; then
: # first commit introducing the file — nothing to compare against yet
else
before=$(git -C "$ROOT" show HEAD:bin/escalation-pending.txt 2>/dev/null \
| sed 's/#.*//' | tr -d '[:blank:]' | grep -v '^$' | sort -u || true)
now=$(sed 's/#.*//' "$PENDING" | tr -d '[:blank:]' | grep -v '^$' | sort -u || true)
added=$(printf '%s\n' "$now" | grep -vxF "$before" 2>/dev/null || true)
for entry in $added; do
[ -n "$entry" ] || continue
echo "ERROR: '$entry' was added to bin/escalation-pending.txt — the list only shrinks. Write its '## Escalation' section instead of exempting it." >&2
fail=1
done
fi
fi
exit $fail
+29 -5
View File
@@ -25,15 +25,39 @@ for dir in "$ROOT"/skills/*/; do
grep -q 'session_log' "$dir/SKILL.md" || continue
if ! grep -q 'session_id' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents session_log without session_id — the tool's only required field" >&2
fail=1
# Read the PAYLOAD, not the file. The first version of this gate grepped the
# whole SKILL.md for "session_id", so a payload missing the field passed as
# long as the word appeared anywhere — including in prose explaining that the
# field is required. A traceability probe demonstrated it in one sed.
payloads=$(grep -n 'session_log' "$dir/SKILL.md" | grep '{' || true)
if [ -z "$payloads" ]; then
# Mentions the tool but documents no call. Nothing to check, and nothing to
# get wrong.
continue
fi
if grep -q 'duration_ms:[[:space:]]*<' "$dir/SKILL.md"; then
echo "ERROR: skills/$name documents duration_ms as a placeholder an agent cannot measure — omit it or pass a real measurement" >&2
printf '%s\n' "$payloads" | while IFS= read -r line; do
[ -n "$line" ] || continue
body=$(printf '%s' "$line" | sed 's/^[0-9]*://')
lineno=$(printf '%s' "$line" | cut -d: -f1)
printf '%s' "$body" | grep -q 'session_id' \
|| echo "ERROR: skills/$name:$lineno documents a session_log payload with no session_id — the tool's only required field" >&2
# Any spelling of the key, quoted or not, followed by anything that is not a
# bare number. A placeholder nobody can fill gets fabricated or dropped; a
# real measurement is fine.
if printf '%s' "$body" | grep -Eq '"?duration_ms"?[[:space:]]*:[[:space:]]*[^0-9,}[:space:]]'; then
echo "ERROR: skills/$name:$lineno documents duration_ms as something other than a real measurement — an agent does not observe a clock; omit it" >&2
fi
done > "$ROOT/.session_log_errors.$$" 2>&1
if [ -s "$ROOT/.session_log_errors.$$" ]; then
cat "$ROOT/.session_log_errors.$$" >&2
fail=1
fi
rm -f "$ROOT/.session_log_errors.$$"
done
exit $fail
+51 -2
View File
@@ -17,8 +17,8 @@
# /plugin install skills@mathias
#
# That path carries the hooks and MCP server definitions too, which symlinks
# cannot. This script covers the harnesses with no plugin system: Crush, Pi
# and OpenCode.
# cannot. This script covers the harnesses with no plugin system: Crush, Pi,
# OpenCode, and Antigravity.
#
# Wiring model — one managed directory, one link per harness:
#
@@ -27,6 +27,13 @@
# ~/.config/crush/skills -> ~/.agents/skills
# ~/.config/opencode/skills -> ~/.agents/skills
#
# Antigravity has no symlink-directory discovery — it reads a path-entries
# JSON instead (~/.gemini/config/skills.json:
# {"entries":[{"path":"..."}]}). One entry points at ~/.agents/skills, so it
# sees exactly the same surfaces-filtered, collision-resolved set every other
# harness does. Requires jq; skipped with a warning if jq is unavailable —
# every other target in this script stays dependency-free on purpose.
#
# ~/.agents/skills was itself a link to this checkout until #27. That encoded
# an assumption nobody had stated — that this checkout is the ONLY source of
# skills on the host — and on flamingo the only way to install this library
@@ -53,7 +60,12 @@ CHECKOUT_DIR="${SKILLS_CHECKOUT_DIR:-$HOME/.local/share/skills}"
AGENTS_DIR="${AGENTS_SKILLS_DIR:-$HOME/.agents/skills}"
CRUSH_DIR="${CRUSH_SKILLS_DIR:-$HOME/.config/crush/skills}"
OPENCODE_DIR="${OPENCODE_SKILLS_DIR:-$HOME/.config/opencode/skills}"
ANTIGRAVITY_SKILLS_JSON="${ANTIGRAVITY_SKILLS_JSON:-$HOME/.gemini/config/skills.json}"
EXTRA_SOURCES="${SKILLS_EXTRA_SOURCES:-}"
# Overridable so the jq-absent branch is testable on a host that has jq. That
# branch is a contract, not a fallback: jq is optional here while every other
# target in this script is dependency-free.
JQ="${SKILLS_JQ:-jq}"
# Recorded beside the managed directory rather than inside it: a stray file in
# there would look like a skill to every harness reading the dir.
SOURCES_MANIFEST="$(dirname "$AGENTS_DIR")/$(basename "$AGENTS_DIR")-sources"
@@ -213,6 +225,42 @@ link_dir() {
log "linked $link → $target"
}
# wire_antigravity — register $AGENTS_DIR in Antigravity's skills.json.
# Merges rather than overwrites: skills.json is a documented user-editable
# config (json_configs.md), so a hand-added entry (e.g. a personal skills
# dir) must survive a re-run untouched.
wire_antigravity() {
if ! command -v "$JQ" >/dev/null 2>&1; then
log "Antigravity: jq not found — skipping (needed to merge $ANTIGRAVITY_SKILLS_JSON safely)"
return
fi
local existing already updated
mkdir -p "$(dirname "$ANTIGRAVITY_SKILLS_JSON")"
if [ -s "$ANTIGRAVITY_SKILLS_JSON" ]; then
existing="$(cat "$ANTIGRAVITY_SKILLS_JSON")"
else
existing='{}'
fi
already="$(printf '%s' "$existing" | "$JQ" --arg p "$AGENTS_DIR" \
'[.entries // [] | .[] | select(.path == $p)] | length' 2>/dev/null)" || {
log "Antigravity: $ANTIGRAVITY_SKILLS_JSON is not valid JSON — leaving it untouched"
return 1
}
if [ "$already" -gt 0 ]; then
log "Antigravity: $AGENTS_DIR already registered in $ANTIGRAVITY_SKILLS_JSON"
return
fi
updated="$(printf '%s' "$existing" | "$JQ" --arg p "$AGENTS_DIR" \
'.entries = ((.entries // []) + [{path: $p}])')"
printf '%s\n' "$updated" > "$ANTIGRAVITY_SKILLS_JSON"
log "Antigravity: registered $AGENTS_DIR in $ANTIGRAVITY_SKILLS_JSON"
}
main() {
ensure_checkout
@@ -224,6 +272,7 @@ main() {
link_dir "$CRUSH_DIR" "$AGENTS_DIR"
link_dir "$OPENCODE_DIR" "$AGENTS_DIR"
wire_antigravity
log "done — $LINKED_COUNT skill(s) linked from $(sources | wc -l | tr -d ' ') source(s), ref=$REF"
log "Claude Code: install the plugin instead — /plugin marketplace add $REPO_URL"
+21 -2
View File
@@ -58,8 +58,8 @@ adr: 0004
title: <the decision, as a sentence — not "database choice">
status: proposed | accepted | superseded | amended
date: 2026-08-29
deciders: <who accepted it, by name — or "pending: <name>" while proposed>
supersedes: <adr number, or ->
deciders: <who accepted it, by name — or "pending: <name>" (QUOTED) while proposed>
supersedes: <adr number, or null>
---
# ADR-0004 — <same title>
@@ -97,6 +97,25 @@ the condition that fires a revisit. An ADR with no validation section is a prefe
with a template around it.
```
### The frontmatter must actually parse
It exists to be machine-read, so a block that does not load is worse than no block at
all: it drops that ADR silently out of every "what is still open?" query, while looking
correct. Two shapes this template used to suggest are invalid YAML, and both were found
live in the estate on 2026-09-05 — `openbanking-api` ADR-0001 and `infra` ADR-0019, each
copied verbatim from here:
| Wrong | Why | Right |
|---|---|---|
| `supersedes: -` | a bare dash is a YAML *sequence entry*, not a string | `supersedes: null` |
| `deciders: pending: Mathias` | the second, unquoted colon reads as a nested mapping | `deciders: "pending: Mathias"` |
Check before committing:
```sh
python3 -c "import yaml,sys;yaml.safe_load(open(sys.argv[1]).read().split('---')[1]);print('ok')" docs/adr/0004-*.md
```
## Preserving a productive tension
Some tradeoffs are not unresolved because nobody thought hard enough. They are
+170
View File
@@ -0,0 +1,170 @@
---
name: traceability
surfaces: [claude-code, library]
summary: "Forward trace — requirement to decision to commit to verification to release, and what cannot be checked"
use_when: "Proving why a change exists; auditing a claim; before an attestation or a compliance ask"
description: Establish the forward trace for a change — requirement, decision, commit, verification, CI, release, closure — as a chain of links that are each checked rather than asserted. Names the links that cannot be verified from the repository instead of letting them pass as verified. Use when auditing why a change exists, preparing an attestation, or when a claim needs to be provable rather than plausible.
---
# Traceability
## Overview
A trace is not a story about how a change came to be. It is a chain of links, each of
which is either **checked** or **named as unchecked**. The value is entirely in that
distinction: a trace where every link is asserted is a narrative, and narratives are what
audits exist to defeat.
The chain in this estate:
```
issue (requirement) → comment (decision) → commit → test/gate → CI run on that sha
→ release tag → closing comment (evidence)
```
Note where the decisions live: **in issue comments**, not issue bodies. A tool that reads
issue titles will trace the wrong thing.
## When to Use
- Auditing why a change exists, especially one nobody remembers making
- Preparing an attestation, a compliance answer, or a client-facing "how do you know"
- Before claiming a requirement is satisfied
- When a claim is repeated in several places, which is the condition under which nobody
checks it
**Do not use for:** a change you are about to make. That is `adr` (the decision) and
`spec-driven-dev` (the requirement). This skill reconstructs; it does not author.
## The tool
```sh
skills/traceability/trace.sh <sha> [repo-path]
```
Prints what git can prove — the commit, its files, the test files it touched, the release
that contains it, every issue it references as a URL — and then prints, every time, the
three things git **cannot** prove. That last section is the point of the tool.
## The sequence
**1. Run the tool.** It does the mechanical half in one command.
**2. Open the issues it printed, and read the comments.** The requirement is in the body;
the decision is almost always in a comment, and so is the closure record. A trace built
from commit messages alone will confidently cite an issue that says something else.
**3. Match CI on the head sha, not on the branch.** "The pipeline is green" is not
evidence about a commit. Find the run whose `head_sha` is this commit and read its
conclusion — and check which *job* and which *step* passed, because a run can be red for a
reason that has nothing to do with the gate you care about, and green overall while the
gate you care about was skipped.
The estate's `workflow_run_status` returns only the run-level conclusion — no jobs, no
steps. For that breakdown you have to drop to the API:
```sh
op run --env-file ~/.op-env -- sh -c 'curl -sS -H "Authorization: token $DMABE_GITEA_API_TOKEN" \
"https://git.d-ma.be/api/v1/repos/<owner>/<repo>/actions/runs/<id>/jobs"'
```
Said explicitly because a step whose tooling dead-ends is exactly where a trace becomes
"the run was green".
**4. Check that the tests actually cover the behaviour.** A commit that touched a test
file is not a commit that is verified. Read the assertions and ask what would have to
break for them to fail. If nothing would, the link is decorative.
**5. Probe the load-bearing claim — try to falsify it.** This is the step that separates a
trace from a summary, and it is the one that finds things.
Pick the claim the chain rests on, and attack it in a scratch copy:
```sh
git archive <sha> | (mkdir -p /tmp/probe && tar -x -C /tmp/probe)
# then break exactly the thing the claim protects, and run the gate
```
If the gate stays green, the claim is false — regardless of how many places assert it.
This is not hypothetical: the probe that shaped this skill falsified a "the list only
shrinks" claim that was stated in five places in the repository and enforced in none.
**6. Look for what cites the commit, not only what it cites.** A commit cannot reference
an issue filed after it, and that is precisely where an outcome gets falsified. Search the
tracker for the sha, across issues *and comments*, and `git log --all --grep=<sha>` for
later commits referring back. In the run that shaped this skill, the issue proving the
traced commit had *relocated* rather than fixed its defect was filed 90 minutes later and
was reachable only this way.
**7. Report unverifiable links as unverifiable.** Not as weak links, not as caveats at the
end. Each gets its own line saying what would be needed to close it.
## What a broken link looks like
| Symptom | What it means |
|---|---|
| The commit cites `#37`, and `#37` describes something else | The reference is decoration; someone renumbered or misremembered |
| A claim appears in five places and a check in none | Convention wearing a mechanism's clothes. Probe it |
| The gate greps for a heading | Presence is not compliance. A heading with nothing under it passes |
| `git describe` says unreleased | Usually stale local tags, not an unreleased commit. `git fetch --tags` first |
| The requirement is a bare `#5` with no URL | Recoverable only while the tracker lives. Not traceable from the repo alone |
| CI green, but on a different sha | The most common false link, and the easiest to accept by accident |
## Output
State the chain, then the gaps, in that order:
```
CHAIN
requirement #36 item 3 (defect report) → verified, read: body + comments
requirement #19 (standing execution bar) → verified, read
decision comment 8509 → verified; names the commit
commit b616c51 → verified
claim "nine skills, none named session_id" → verified against b616c51^: 9/9
claim "session_id is the only required field" → verified against the live tool schema
gate bin/check-session-log → RED on parent (17 errors), GREEN on it
wiring Taskfile check step 5 → verified; CI runs task check
verification tests/check_session_log_test.sh → verified, 4 cases, both directions
CI run 4880, job "Check / Test" → verified, head_sha matches, success
release v0.13.3 → verified against the REMOTE
GAPS
- The gate greps the file, not the payload. Probe: stripped session_id from
every payload, left the word in prose — gate returned 0. FALSIFIED.
- Defect relocated, not removed: #39, filed 90 minutes later by the same
author, says so. The commit's outcome claim does not hold.
```
Four slots that are not optional, each learned by leaving them out:
- **More than one requirement, and they differ in kind.** A defect report and a standing
policy bar are both requirements and collapsing them hides which one was not met.
- **The gate, separately from the test.** A commit that ships a check has three artifacts,
not two, and the chain usually rests on the one in the middle.
- **The RED/GREEN baseline.** Run the gate against the parent commit. Without it you have
shown a gate exists, not that it caught anything.
- **The commit's factual claims, checked.** "Nine skills, none named it" is falsifiable in
one command. Without a slot for it, "commit → verified" means only "this sha exists".
Gaps ranked by how much they matter, not by where you found them. A trace whose gap
section is empty should be suspicious of itself.
## Escalation
Read-only. Reports the chain; changes nothing, and closes no issue on the strength of
what it finds.
Stop and hand back when: a link's evidence is outside anything you can reach — a tracker
you cannot read, a CI system you have no access to, a decision made in conversation. Say
which link and what would close it. An inferred link presented as a traced one is worse
than a gap, because the gap is honest and the inference is citable.
Also stop before *acting* on a falsified claim. Finding that a gate does not enforce what
it says is a finding; deciding what to do about it is the owner's call.
## Cross-References
- `lineage` — the same discipline backwards: this asks "does the chain hold", that asks
"was it tried before and why did it stop".
- `adr` — the decision layer of the chain. A trace that reaches an ADR has found the
reasoning; a trace that reaches only a commit message has found a summary of it.
+107
View File
@@ -0,0 +1,107 @@
#!/bin/sh
# trace.sh — the forward trace for one commit, and an honest list of what it
# could not check.
#
# The chain this estate actually uses:
#
# issue (requirement) -> comment (decision) -> commit -> test/gate -> CI run
# -> release tag -> closing comment
#
# git can prove the middle of that and nothing else. The issue bodies, the
# comments where decisions actually live, and the CI run are all tracker state.
# So this prints what git knows and then names, with URLs, the links a human has
# to follow — because a link nobody checked must not read like a verified one.
#
# Usage: trace.sh <sha> [repo-path]
#
# Read-only.
set -u
[ $# -ge 1 ] || { echo "usage: trace.sh <sha> [repo-path]" >&2; exit 2; }
SHA=$1
REPO=${2:-.}
git -C "$REPO" rev-parse --git-dir >/dev/null 2>&1 || {
echo "trace: $REPO is not a git repository" >&2; exit 2; }
git -C "$REPO" rev-parse --verify --quiet "$SHA^{commit}" >/dev/null || {
echo "trace: $SHA is not a commit in $REPO" >&2; exit 2; }
TOP=$(git -C "$REPO" rev-parse --show-toplevel)
ORIGIN=$(git -C "$REPO" config --get remote.origin.url 2>/dev/null || true)
# Never print the URL as configured: url.insteadOf can expand it into a live
# token. Reduce it to owner/repo and use that.
SLUG=$(printf '%s' "$ORIGIN" | sed -E 's#\.git$##; s#^.*[/:]([^/]+/[^/]+)$#\1#' 2>/dev/null || true)
section() { printf '\n== %s ==\n' "$1"; }
section "Commit"
git -C "$REPO" log -1 --format='%h %ad %an%n%n%B' --date=short "$SHA"
section "Files"
git -C "$REPO" show --stat --format='' "$SHA"
section "Verification it touched"
tests=$(git -C "$REPO" show --name-only --format='' "$SHA" \
| grep -Ei '(^|/)(tests?|spec)/|_test\.|\.test\.|_spec\.' || true)
if [ -n "$tests" ]; then
printf '%s\n' "$tests"
else
printf 'none — this commit changed no test or spec file.\n'
printf 'That is not automatically wrong (docs, config), but if it changed behaviour\n'
printf 'it means the change has no verification of its own.\n'
fi
section "Released in"
tag=$(git -C "$REPO" describe --tags --contains "$SHA" 2>/dev/null | sed 's/[~^].*//' || true)
if [ -n "$tag" ]; then
printf '%s (LOCAL tags only — confirm with: git ls-remote --tags origin)\n' "$tag"
else
printf 'no release contains this commit in THIS checkout.\n'
printf 'Local tags go stale — run `git fetch --tags` and re-check before\n'
printf 'concluding it was never released.\n'
fi
section "Referenced issues"
refs=$(git -C "$REPO" log -1 --format='%B' "$SHA" | grep -oE '#[0-9]+' | sort -u || true)
if [ -n "$refs" ]; then
for r in $refs; do
n=${r#\#}
if [ -n "$SLUG" ]; then
printf '%s -> https://git.d-ma.be/%s/issues/%s\n' "$r" "$SLUG" "$n"
else
printf '%s\n' "$r"
fi
done
else
printf 'none — the commit cites no issue, so its requirement is unrecorded.\n'
fi
section "Issues that cite THIS commit"
printf 'Not searchable from the commit — an issue filed after it cannot appear in\n'
printf 'its message, and that is exactly where an outcome gets falsified. Check:\n\n'
printf ' git -C %s log --all --grep=%s # later commits referring back\n' "$REPO" "$SHA"
if [ -n "$SLUG" ]; then
printf ' tracker search for "%s" across issues AND comments\n' "$SHA"
fi
printf '\nIn the run that shaped this tool, the issue that falsified the traced\n'
printf 'commit was filed 90 minutes after it and was reachable only this way.\n'
section "Cannot verify from here"
cat <<'TXT'
git proves the commit, its files, and which tag contains it. It cannot prove:
1. That the issues above say what the commit claims they say. The DECISION is
usually in a comment, not the issue body — open them and read the comments.
2. That CI actually passed on this exact sha. A green pipeline elsewhere is not
evidence about this commit; match on head_sha.
3. That the tests listed above assert the behaviour this commit added. A test
file touched is not a behaviour covered.
4. That any gate this commit ships enforces what its message says it does.
Probe it: break exactly the thing the claim protects, in a scratch copy,
and see whether the gate goes red.
None of the three is a formality. Each has been the broken link in this estate:
a claim asserted in five places and enforced in none, a red run whose gate had
passed, and a gate that greps for a heading and calls it a contract.
TXT
+42
View File
@@ -81,5 +81,47 @@ f=$(fixture comments); skill "$f" alpha no
printf '# a comment, and a blank line follow\n\nalpha\n' > "$f/bin/escalation-pending.txt"
expect 0 'comments and blank lines in the list are ignored' "$f"
# --- the ratchet must actually ratchet ---------------------------------------
# Five places in this repo claimed the pending list "only shrinks" and nothing
# enforced it. A traceability probe falsified that in one run: add an unsectioned
# skill, append it to the list, gate returns 0. A convention stated five times is
# still a convention.
#
# Enforcement compares against the list as committed at HEAD, so the fixtures
# below need a real git repo.
git_fixture() {
_f="$WORK/$1"
mkdir -p "$_f/skills" "$_f/bin"
( cd "$_f" && git init -q . && git config user.email t@example.com && git config user.name test )
skill "$_f" alpha no
printf 'alpha\n' > "$_f/bin/escalation-pending.txt"
( cd "$_f" && git add -A && git commit -qm baseline )
printf '%s' "$_f"
}
f=$(git_fixture ratchet_same)
expect 0 'an unchanged list passes' "$f"
f=$(git_fixture ratchet_shrink)
skill "$f" alpha yes
: > "$f/bin/escalation-pending.txt"
expect 0 'removing an entry (and writing its section) passes' "$f"
f=$(git_fixture ratchet_grow)
skill "$f" beta no
printf 'alpha\nbeta\n' > "$f/bin/escalation-pending.txt"
expect 1 'adding a NEW entry fails — the list only shrinks' "$f" 'beta'
# Outside a git repo the comparison is impossible; that must not be silent.
f=$(fixture ratchet_nogit); skill "$f" alpha no
printf 'alpha\n' > "$f/bin/escalation-pending.txt"
_out=$(sh "$SUT" "$f" 2>&1)
if printf '%s' "$_out" | grep -qi 'cannot verify'; then
pass=$((pass + 1)); printf ' ok an unverifiable ratchet says so rather than passing silently\n'
else
fail=$((fail + 1)); printf ' FAIL an unverifiable ratchet says so rather than passing silently\n%s\n' "$_out"
fi
printf '\ncheck_escalation: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+33
View File
@@ -63,5 +63,38 @@ expect 1 'documenting duration_ms as wall-clock fails' \
"$(fixture clock 'Call `session_log` with `{session_id: "<id>", skill: "alpha", duration_ms: <wall-clock>}`')" \
'duration_ms'
# --- the gate must read the PAYLOAD, not the file ---------------------------
# A traceability probe falsified the original: it grepped the whole file for the
# string "session_id", so stripping the field from every documented payload and
# leaving the word once in prose passed cleanly. The exact regression the gate
# exists to prevent was back in the file and the gate said nothing.
expect 1 'a payload without session_id fails even when prose mentions the word' \
"$(fixture prose 'The tool has a required session_id field.
Call `session_log` with `{skill: "alpha", phase: "red", final_status: "pass"}`')" \
'session_id'
expect 1 'one bad payload among several fails' \
"$(fixture mixed 'Call `session_log` with `{session_id: "<id>", skill: "alpha", phase: "red"}`
Then `session_log` with `{skill: "alpha", phase: "green", final_status: "pass"}`')" \
'session_id'
# duration_ms was banned by matching exactly one spelling of it — the historical
# one. Both of these passed the original gate.
expect 1 'duration_ms with a JSON-quoted key fails' \
"$(fixture jsonkey 'Call `session_log` with `{session_id: "<id>", "duration_ms": <wall-clock>}`')" \
'duration_ms'
expect 1 'duration_ms described in prose rather than angle brackets fails' \
"$(fixture prosems 'Call `session_log` with `{session_id: "<id>", duration_ms: elapsed wall-clock ms}`')" \
'duration_ms'
# A real measurement is allowed: the objection is to a placeholder no agent can
# fill, not to the field.
expect 0 'a duration_ms carrying a real value is allowed' \
"$(fixture realms 'Call `session_log` with `{session_id: "<id>", duration_ms: 1430}`')"
printf '\ncheck_session_log: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+91 -1
View File
@@ -17,6 +17,7 @@ SUT="$ROOT/install.sh"
pass=0
fail=0
skipped=0
WORK=${TMPDIR:-/tmp}/install_test.$$
if [ ! -f "$SUT" ]; then
@@ -55,6 +56,7 @@ run_install() {
AGENTS_SKILLS_DIR="$_f/home/.agents/skills" \
CRUSH_SKILLS_DIR="$_f/home/.config/crush/skills" \
OPENCODE_SKILLS_DIR="$_f/home/.config/opencode/skills" \
ANTIGRAVITY_SKILLS_JSON="$_f/home/.gemini/config/skills.json" \
SKILLS_EXTRA_SOURCES="$_extra" \
bash "$SUT" 2>&1)
rc=$?
@@ -156,6 +158,40 @@ else
no 'configured sources are recorded for doctor' "$out"
fi
# --- Antigravity: jq is optional, so BOTH branches must be tested -----------
# These three cases inherited whatever jq the host happened to have. koala has
# it, the CI runner does not, so they passed locally and failed in CI — the
# classic works-on-my-machine gate. install.sh treats jq as optional by design,
# which means the skip is a contract and needs a test of its own.
# run_install_nojq <fixture> — same as run_install, with the jq lookup pointed
# at a binary that does not exist. Deterministic regardless of the host.
run_install_nojq() {
_f=$1
_out=$(HOME="$_f/home" \
SKILLS_SKIP_FETCH=1 \
SKILLS_CHECKOUT_DIR="$_f/ck" \
AGENTS_SKILLS_DIR="$_f/home/.agents/skills" \
CRUSH_SKILLS_DIR="$_f/home/.config/crush/skills" \
OPENCODE_SKILLS_DIR="$_f/home/.config/opencode/skills" \
SKILLS_JQ="definitely-not-jq-$$" \
bash "$SUT" 2>&1)
rc=$?
printf '%s' "$_out"
}
f=$(new_fixture antigravity_nojq)
out=$(run_install_nojq "$f")
if [ -e "$f/home/.gemini/config/skills.json" ]; then
no 'without jq, no skills.json is written' "$out"
elif ! printf '%s' "$out" | grep -qi 'jq not found'; then
no 'without jq, the skip is reported' "$out"
elif [ ! -L "$f/home/.agents/skills/alpha" ]; then
no 'without jq, the other harnesses are still wired' "$out"
else
ok 'without jq, Antigravity is skipped and the rest still wires'
fi
# --- 8. surfaces: the library link set is not "every skill" -----------------
# claude.ai has no shell, no cluster and no checkout, so a chat-only skill
# linked into Crush is a skill that can never fire, competing for selection
@@ -183,5 +219,59 @@ else
no 'a source with no surfaces convention is linked as-is' "$out"
fi
printf '\ninstall: %d passed, %d failed\n' "$pass" "$fail"
# --- 9-11. Antigravity's merge behaviour, only where jq exists ---------------
# These three assert what jq DOES. install.sh treats jq as optional, so on a
# host without it there is nothing to assert — and failing there is what turned
# main red: they passed on koala (jq present) and failed on the runner (absent).
# A skipped case is honest; a case that fails because the host lacks an optional
# dependency is a false red, and false reds are how a suite stops being read.
if ! command -v jq >/dev/null 2>&1; then
skipped=$((skipped + 3))
printf ' SKIP jq absent — Antigravity merge behaviour not covered on this host (3 cases)\n'
else
# skills.json entries must point at $AGENTS_DIR — that carries the
# surfaces-filtering, collision handling and multi-source merge. Pointing at
# the checkout directly would give Antigravity a different skill set than
# every other harness.
f=$(new_fixture antigravity)
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
if [ ! -f "$AG" ]; then
no 'Antigravity skills.json is created' "$out"
else
registered=$(jq -r --arg p "$f/home/.agents/skills" '[.entries[]? | select(.path == $p)] | length' "$AG")
if [ "$registered" = "1" ]; then
ok 'Antigravity registers the managed agents dir, not the raw checkout'
else
no 'Antigravity registers the managed agents dir, not the raw checkout' "$(cat "$AG")"
fi
fi
f=$(new_fixture antigravity_idempotent)
run_install "$f" >/dev/null
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
count=$(jq '.entries | length' "$AG" 2>/dev/null || echo "?")
if [ "$count" = "1" ]; then
ok 'Antigravity wiring is idempotent (no duplicate entries)'
else
no 'Antigravity wiring is idempotent (no duplicate entries)' "entries=$count"$'\n'"$out"
fi
# A user may hand-edit skills.json to point at their own personal skills dir
# (json_configs.md documents this). Our merge must not clobber it.
f=$(new_fixture antigravity_preserve)
mkdir -p "$f/home/.gemini/config"
printf '{"entries":[{"path":"/Users/mathias/personal-skills"}]}' > "$f/home/.gemini/config/skills.json"
out=$(run_install "$f")
AG="$f/home/.gemini/config/skills.json"
personal=$(jq -r '[.entries[]? | select(.path == "/Users/mathias/personal-skills")] | length' "$AG" 2>/dev/null || echo 0)
if [ "$personal" = "1" ]; then
ok 'Antigravity wiring preserves pre-existing hand-authored entries'
else
no 'Antigravity wiring preserves pre-existing hand-authored entries' "$(cat "$AG")"$'\n'"$out"
fi
fi
printf '\ninstall: %d passed, %d failed, %d skipped\n' "$pass" "$fail" "$skipped"
[ "$fail" -eq 0 ]
+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# Tests for skills/traceability/trace.sh.
#
# The RED run for #10 established what a forward trace costs by hand: sixteen
# commands, of which the decisive ones were reading issue comments and running a
# probe against a claim. This tool does the mechanical half — what git can prove
# — and, more importantly, states what it CANNOT prove, so an unverifiable link
# never reads as a verified one.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
SUT="$ROOT/skills/traceability/trace.sh"
pass=0
fail=0
WORK=${TMPDIR:-/tmp}/traceability_test.$$
if [ ! -f "$SUT" ]; then
printf 'FAIL: %s does not exist\n' "$SUT"
exit 1
fi
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
mkdir -p "$WORK/repo/tests"
cd "$WORK/repo" || exit 1
git init -q .
git config user.email t@example.com
git config user.name test
# A remote is needed for the tool to turn "#37" into a checkable URL, which is
# the point of that section.
git remote add origin https://git.d-ma.be/mathias/demo.git
printf 'baseline\n' > README.md
git add -A && git commit -qm "chore: baseline"
printf 'gate\n' > gate.sh
printf 'case\n' > tests/gate_test.sh
git add -A && git commit -qm "feat(gate): add the thing
Implements the decision recorded in #5, filed as #37.
Closes #37"
SHA=$(git rev-parse --short HEAD)
git tag -a v1.2.0 -m "release v1.2.0"
printf 'more\n' >> gate.sh
git add -A && git commit -qm "chore: later work"
cd - >/dev/null || exit 1
out=$(sh "$SUT" "$SHA" "$WORK/repo" 2>&1)
check() {
if printf '%s' "$out" | grep -q "$1"; then
pass=$((pass + 1)); printf ' ok %s\n' "$2"
else
fail=$((fail + 1)); printf ' FAIL %s\n output had no %s\n%s\n' "$2" "$1" "$out"
fi
}
check '#5' 'lists every issue the commit references, not just the closing one'
check '#37' 'lists the closing reference'
check 'tests/gate_test.sh' 'names the test files the commit touched'
check 'v1.2.0' 'names the release that contains it'
check 'Cannot verify' 'states what it could not verify'
check 'issues/37' 'emits a checkable URL for the tracker link it cannot follow'
# A commit in no release must say so — "no tag" and "not checked" are different
# answers and only one of them is about the commit.
last=$(git -C "$WORK/repo" rev-parse --short HEAD)
out=$(sh "$SUT" "$last" "$WORK/repo" 2>&1)
if printf '%s' "$out" | grep -qi 'no release contains'; then
pass=$((pass + 1)); printf ' ok an unreleased commit is reported as unreleased\n'
else
fail=$((fail + 1)); printf ' FAIL an unreleased commit is reported as unreleased\n%s\n' "$out"
fi
printf '\ntraceability: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]