pr-gatekeeper exists only in ~/.claude/agents on koala — the estate's most effective control is unversioned #51

Closed
opened 2026-09-28 21:23:47 +00:00 by mathias · 1 comment
Owner

The gap

pr-gatekeeper blocked dispatch#75 five times. Every block was a real, executable escape — twelve pathspec bypasses, arbitrary command execution through git -c alias, --upload-pack='rm …', a write primitive through --output=, ${IFS} word-splitting past the policy, and a path denylist that had been a no-op in the live flow since it was written.

task check was green on all five.

The agent definition exists in no repository. A code search across the estate finds only references to it — comments in dispatch source, a local-dev sprint doc. The file itself lives at:

~/.claude/agents/pr-gatekeeper.md      158 lines, koala only

Unversioned, unreviewed, not on any other host, and not covered by anything that would survive a disk failure. The single most effective quality control in this estate is the one artifact with no backup and no history.

Why it belongs here

This repo is already the canonical home for agent-facing instructions, installed as a plugin with install.sh wiring the harnesses that have no plugin system. Everything else an agent loads is versioned; this is not.

Versioning it also makes it reviewable. The procedure inside it is the reason the reviews worked, and right now nobody can diff a change to it or explain why a clause is there.

What to port, not just the file

The file is the artifact; the method is the value. brain a-green-gate-is-not-evidence-for-control-code records what actually found the defects, and the ported version should make each one explicit rather than implied:

  1. Verify by execution in a scratch repo, not by reading the matcher. Every finding came from running the command and observing the file change. The matcher said one thing and the filesystem said another, five times.
  2. Flip-test. Revert the fix, confirm the test goes red, restore. This found three behavioural changes shipped with no assertion at all.
  3. Read the survivors. After a differential, read what is still allowed, one by one. This caught git commit -F /etc/passwd — file exfiltration wearing a message flag — which neither the review nor the differential found, because both implementations agreed and both were wrong.
  4. Grade linters on exit code, never on grepping output. A crashed linter greps as clean.
  5. Refusal-by-default posture, and authority to block. The same model in the author's seat had convinced itself five times.

Done when

  • pr-gatekeeper lives in this repo and is installed by the existing wiring, with the local copy removed rather than left to drift
  • The five practices above are stated in the agent's procedure, not implied by it
  • task doctor reports whether the agent is present on the host, the way it already does for other wiring
  • A change to the agent is a reviewable diff

Why now

Two independent reasons this is live rather than tidy-up. ai-sessions ADR-0001 introduces 90-day pruning of raw transcripts — so the review transcripts that justify each clause in this file start expiring. And the agent is currently one rm away from being gone with no way to reconstruct it.

Refs

  • brain a-green-gate-is-not-evidence-for-control-code — the method and its evidence
  • dispatch#75 — the five review rounds
  • local-dev docs/sprints/2026-09-17b-templates-stop-minting-broken-children.md — an earlier block by the same agent, on a different repo

Created via git-mcp on behalf of @mathias

## The gap `pr-gatekeeper` blocked `dispatch#75` five times. Every block was a real, executable escape — twelve pathspec bypasses, arbitrary command execution through `git -c alias`, `--upload-pack='rm …'`, a write primitive through `--output=`, `${IFS}` word-splitting past the policy, and a path denylist that had been a **no-op in the live flow since it was written**. `task check` was green on all five. **The agent definition exists in no repository.** A code search across the estate finds only *references* to it — comments in `dispatch` source, a `local-dev` sprint doc. The file itself lives at: ``` ~/.claude/agents/pr-gatekeeper.md 158 lines, koala only ``` Unversioned, unreviewed, not on any other host, and not covered by anything that would survive a disk failure. The single most effective quality control in this estate is the one artifact with no backup and no history. ## Why it belongs here This repo is already the canonical home for agent-facing instructions, installed as a plugin with `install.sh` wiring the harnesses that have no plugin system. Everything else an agent loads is versioned; this is not. Versioning it also makes it **reviewable**. The procedure inside it is the reason the reviews worked, and right now nobody can diff a change to it or explain why a clause is there. ## What to port, not just the file The file is the artifact; the method is the value. brain `a-green-gate-is-not-evidence-for-control-code` records what actually found the defects, and the ported version should make each one explicit rather than implied: 1. **Verify by execution in a scratch repo, not by reading the matcher.** Every finding came from running the command and observing the file change. The matcher said one thing and the filesystem said another, five times. 2. **Flip-test.** Revert the fix, confirm the test goes red, restore. This found **three behavioural changes shipped with no assertion at all**. 3. **Read the survivors.** After a differential, read what is still *allowed*, one by one. This caught `git commit -F /etc/passwd` — file exfiltration wearing a message flag — which neither the review nor the differential found, because both implementations agreed and both were wrong. 4. **Grade linters on exit code, never on grepping output.** A crashed linter greps as clean. 5. **Refusal-by-default posture, and authority to block.** The same model in the author's seat had convinced itself five times. ## Done when - `pr-gatekeeper` lives in this repo and is installed by the existing wiring, with the local copy removed rather than left to drift - The five practices above are stated in the agent's procedure, not implied by it - `task doctor` reports whether the agent is present on the host, the way it already does for other wiring - A change to the agent is a reviewable diff ## Why now Two independent reasons this is live rather than tidy-up. ai-sessions ADR-0001 introduces **90-day pruning of raw transcripts** — so the review transcripts that justify each clause in this file start expiring. And the agent is currently one `rm` away from being gone with no way to reconstruct it. ## Refs - brain `a-green-gate-is-not-evidence-for-control-code` — the method and its evidence - `dispatch#75` — the five review rounds - `local-dev` `docs/sprints/2026-09-17b-templates-stop-minting-broken-children.md` — an earlier block by the same agent, on a different repo --- _Created via git-mcp on behalf of @mathias_
mathias added the risk/mediumsize/stype/ops labels 2026-09-28 21:24:26 +00:00
Author
Owner

Shipped in c25175e, released as v0.21.0, CI run 8208 green.

Verified

agent in the repo agents/pr-gatekeeper.md
shipped by the plugin ~/.claude/plugins/cache/mathias/skills/0.21.0/agents/pr-gatekeeper.md, 9767 bytes — the extended version, not the old one
doctor reports it new shadow check, tested both directions
host-local copy moved aside on koala
manifest/tag parity check OK — 45 skills, manifest matches tag

Takes effect on session restart — the harness loads agents at startup, so this session lost pr-gatekeeper the moment the host-local copy went and regains it on the next one. Stated rather than left as a surprise.

Ported, plus what the five rounds taught

The file already had flip-testing. Three habits that actually did the finding were implied at best and are now explicit (§4b):

  • Verify by execution, not by reading the implementation. Every finding came from running the command in a scratch repo and looking at the filesystem. The matcher said one thing and the filesystem another, five times.
  • Read the survivors. After an allow/deny sweep, read what is still permitted, one at a time. This caught git commit -F /etc/passwd — exfiltration wearing a message flag — which neither the review nor the differential found, because both implementations agreed and both were wrong. Agreement is not correctness.
  • Grade a tool on its exit code, never by grepping output. A crashed linter greps as clean.

Plus rule 9 in §4, and a note in §5 that CI green was worth nothing five times running on dispatch#75.

§4c — the incident this agent caused

New section, and the reason I did not just git mv the file. On 2026-09-19 a probe of this agent's own making printed /proc/$PPID/environ and ps e raw, and a git credential fill consulted the host's real store. Seven live credentials reached the transcript (infra#445). The finding was correct; the method was not. Versioning the file without that lesson would have shipped the incident along with the agent.

So: a probe that reads an environment prints a count, never the buffer; invented sentinels only; isolated HOME for anything touching git's credential subsystem; prove a primitive and stop.

Two things worth recording

The stray-entry guard caught me. Leaving agents/ out of task check's root-entry allowlist broke check_tolerates_unverifiable_test.sh. I checked whether that was pre-existing by running it against a pristine HEAD worktree — it passed there, so it was mine. That is #23's guard working exactly as designed: an undeclared root entry loads in no harness and errors in none either.

14 assertions, each named for what it observes, not what it guarantees. A grep proves a phrase is present, not that a procedure is followed; naming it otherwise would be the defect local-dev rule 9 exists for. Flip-tested the doctor check and the probe-safety phrase — each turns the suite red.

Unrelated finding, left alone

doctor reports a pre-existing drift on this host: "Claude Code plugin serves 45 skills, the other harnesses see 44 — the two copies have diverged." Present before this change and not caused by it. install.sh is the sanctioned fix; out of scope here rather than bundled in silently.


Created via git-mcp on behalf of @mathias

Shipped in `c25175e`, released as **v0.21.0**, CI run 8208 green. ## Verified | | | |---|---| | agent in the repo | `agents/pr-gatekeeper.md` | | shipped by the plugin | `~/.claude/plugins/cache/mathias/skills/0.21.0/agents/pr-gatekeeper.md`, 9767 bytes — the extended version, not the old one | | doctor reports it | new `shadow` check, tested both directions | | host-local copy | moved aside on koala | | manifest/tag parity | `check OK — 45 skills, manifest matches tag` | Takes effect on session restart — the harness loads agents at startup, so this session lost `pr-gatekeeper` the moment the host-local copy went and regains it on the next one. Stated rather than left as a surprise. ## Ported, plus what the five rounds taught The file already had flip-testing. Three habits that actually did the finding were implied at best and are now explicit (§4b): - **Verify by execution, not by reading the implementation.** Every finding came from running the command in a scratch repo and looking at the filesystem. The matcher said one thing and the filesystem another, five times. - **Read the survivors.** After an allow/deny sweep, read what is still permitted, one at a time. This caught `git commit -F /etc/passwd` — exfiltration wearing a message flag — which neither the review nor the differential found, because both implementations **agreed and both were wrong.** Agreement is not correctness. - **Grade a tool on its exit code, never by grepping output.** A crashed linter greps as clean. Plus rule 9 in §4, and a note in §5 that CI green was worth nothing five times running on `dispatch#75`. ## §4c — the incident this agent caused New section, and the reason I did not just `git mv` the file. On 2026-09-19 a probe of this agent's own making printed `/proc/$PPID/environ` and `ps e` raw, and a `git credential fill` consulted the host's real store. Seven live credentials reached the transcript (`infra#445`). **The finding was correct; the method was not.** Versioning the file without that lesson would have shipped the incident along with the agent. So: a probe that reads an environment prints a count, never the buffer; invented sentinels only; isolated `HOME` for anything touching git's credential subsystem; prove a primitive and stop. ## Two things worth recording **The stray-entry guard caught me.** Leaving `agents/` out of `task check`'s root-entry allowlist broke `check_tolerates_unverifiable_test.sh`. I checked whether that was pre-existing by running it against a pristine HEAD worktree — it passed there, so it was mine. That is `#23`'s guard working exactly as designed: an undeclared root entry loads in no harness and errors in none either. **14 assertions, each named for what it observes**, not what it guarantees. A grep proves a phrase is present, not that a procedure is followed; naming it otherwise would be the defect `local-dev` rule 9 exists for. Flip-tested the doctor check and the probe-safety phrase — each turns the suite red. ## Unrelated finding, left alone `doctor` reports a pre-existing drift on this host: *"Claude Code plugin serves 45 skills, the other harnesses see 44 — the two copies have diverged."* Present before this change and not caused by it. `install.sh` is the sanctioned fix; out of scope here rather than bundled in silently. --- _Created via git-mcp on behalf of @mathias_
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/skills#51