bedb6d278da6412c2d9a840598e4b81b484e36db
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 mathias
|
0c7fac3686 |
fix(debug): close the intake gate properly — and the contradiction it exposed
The GREEN run confirmed the gate works: given "builds are flaky again... what's the fix?" with no run id and no log, it produced zero hypotheses and an ordered list of artifacts, each justified by what it would discriminate. It also caught that the retry already shipped in that pipeline is destroying the evidence needed to diagnose it, which is a better finding than any hypothesis would have been. Then it found eight problems, and the first two are mine: **The template contradicted the rule.** I banned high/medium/low labels in Step 2 and left them in the Step 3 output template and all three worked-example hypotheses. "Prose loses to a format block every time" — correct, and the template is what gets imitated. Labels are gone from both; the slot is now WHY FIRST, which asks for the evidence that sets the order rather than a restatement of it. **The description never mentioned the gate.** That is the text a harness shows before an agent decides to load anything, and it read "Generate 3-5 ranked hypotheses" — so an agent routing on the description alone does exactly the thing this skill was rewritten to prevent. Both description and use_when now lead with the refusal. The rest: - Intake vs the failed-fixes section pointed opposite ways for a report that fires both, and nothing stated precedence. Intake wins, and says so: you cannot aim hypotheses at a premise you have no evidence about either. - "One run narrows the search to a single component" was overclaimed exactly where it gets used — CI flakiness. One run *that reproduces* narrows it; a passing run narrows nothing. - session_log had no phase for a correct refusal, so an intake stop logged as a failed read-failure. A metric that penalises the skill's best behaviour trains the next model to guess. Added an `intake` phase. - The Anti-Patterns table — the part people scan in a hurry — was missing the failure this whole rewrite is about: answering "what's the fix?" from a report with no artifact. - brain_query told you to search on an error string you may have just refused to proceed without. Now: query symptom + component, which may itself name the artifact to ask for. - condition-based-waiting was an unconditional load on any mention of a timeout, including during an intake stop, which is the wrong side of the gate. Closes #36 Bump-Type: minor |
||
|
mathias
|
cba0fe4337 |
docs(tdd): fix ten defects the GREEN run found in the language references
The pressure test that #18 and #20 were waiting on. All six original gaps came back ANSWERED with the lines quoted — and the run then found ten problems in what I had written. The worst two are mine, not inherited. **The Python exemplar was the evaluation task.** references/python.md demonstrated `split_vat` — the exact function the test scenario asks for, negative-amount case included. The run said so: "as a teaching example that's over-fitted; as an evaluation it's a leak... this task can't answer it — the reference handed over the finished artifact." So the run could not measure whether the guidance transfers, only whether copying works. The exemplar is now `parse_retry_after`, deliberately unrelated to anything in this estate, and it states its own contract rather than requiring the reader to divide 12500 by 1.25 to discover it. **mypy at RED pointed at the stub the same file forbids.** python.md taught that a pytest ImportError is a legitimate red, then presented `mypy --strict` as a matched pair without saying that mypy reports the same missing module — so following both literally, the way to get mypy green at RED is to create the module. Now stated: that error is the expected red; what must be fixed immediately is a --strict violation inside the test file you just wrote. The rest, each quoted in the run: - The checklist contradicted itself two lines apart — "strike it deliberately" above, "Can't check all boxes? You skipped TDD" below. A struck box is by definition unchecked, so the Python path routed straight into "start over". - `match=` was overclaimed as "free". It is `re.search`, so a message containing parens quietly matches almost anything; and it couples every error test to wording. The Go comparison was a strawman against `assert.Error` when `assert.ErrorIs` exists — the honest Python analogue is a custom exception type, with `match=` as the fallback. - `pytest.approx` was listed as the soft-assert analogue. It solves float tolerance and has nothing to do with collecting failures; that is pytest-subtests or pytest-check, neither of which ships with pytest. - The whole-shape assertion silently required value equality — on a plain class it compares identity and can never pass. - "When Stuck" is a language-neutral table that prescribed `t.Helper()`. - `__tracebackhide__` must be a local inside the helper body; at module level it does nothing, silently. Refs #18, #20 Bump-Type: patch |
||
|
mathias
|
1d140ec525 |
feat(check): every skill states when it stops and hands back
Implements #5's decision. The hard part was #37's, not #5's: which skills need an escalation rule? A hardcoded list rots, sniffing the prose for "does this write anything" is unfalsifiable, and a frontmatter flag is the option #5 rejected wearing a hat. So there is no acting set. Every skill answers the question, and a read-only one answers it in a line. That turns an absence into a claim someone can disagree with, which is the whole point — "this skill never hands back" is a design decision worth stating, and today it was indistinguishable from nobody having thought about it. Eleven written here, all advisory or read-only, where the honest answer is short and the value is in being specific rather than generic. `cognitive-load` stops when the metric disagrees with the people reading the code, because that is evidence about the metric. `test-design` stops when the tests are fine and the design under test is what makes them awkward. `code-review` stops when the diff is big enough that a review would be a skim, and says so rather than producing confident coverage of a fraction. The remaining thirty are the acting set — writes, dispatches, credentials, releases — and each needs a rule someone thought about rather than a rote paragraph, so they sit in bin/escalation-pending.txt. That list only shrinks, and it is self-cleaning: a listed skill that HAS a section fails the gate, so an exemption cannot outlive its reason. Mutation-verified both ways. Six test cases including the stale-entry case, because an exemption for a skill that no longer exists is how a gate quietly stops covering something. Refs #37, closes the mechanism half Bump-Type: minor |
||
|
mathias
|
1d368d4b5d |
docs(skills): drop the Mode 2 routing notes, fix the contributing steps
Seven skills carried a `## Mode 2 Routing Note` citing a routing pod ("Plan 6")
that does not exist. Several admitted in their own text that nothing about the
skill changes between modes — so it was roughly ten lines of context on every
load for zero behavioural effect. The content is not worthless, just not
per-skill: it now lives as one table in README.md, with the caveat that nothing
depends on it.
The related line inside the session_log sections stays, because that one does
motivate behaviour — but it no longer cites a component that has never shipped.
It now says what is true: pass-rate is what a future routing decision would be
based on, and a skill that never logs contributes no evidence either way.
While in README: the contributing steps were stale enough to fail CI for anyone
who followed them. They said to add a row to SKILLS_INDEX.md by hand — the table
is generated now, and `task check` fails on a hand-edited diff — and listed none
of the four frontmatter keys the gates require. Step 4 now names the
pressure-test bar from #19 rather than leaving it to be discovered.
Closes #36
Bump-Type: patch
|
||
|
mathias
|
b616c51a2c |
fix(skills): document a session_log call the tool would actually accept
Nine skills documented the payload. None named `session_id`, which is the tool's only required field. So every documented call was one the tool rejects — an agent following any of them either failed the call or invented an id, and invented ids do not group, which means the pass-rate the /pass-rate endpoint computes was being assembled from entries that cannot be joined. Found by the #19 pressure test against `debug`, which invented `debug-signed-artifact-missing-2026-08-29` and said so: "my three entries are only groupable by luck of naming". The grep afterwards showed it was fleet-wide, not a `debug` bug. Also removed: `duration_ms: <wall-clock>` in eight of them. An agent does not observe a clock. The same run omitted it "rather than fabricate", which is the good outcome; the bad outcome is the run that fabricates, and a made-up number in a metrics stream is worse than a missing one. It is an optional field and is now simply absent. bin/check-session-log gates both, with tests. This is exactly the case #19 has in mind for a small tool over prose: the rule is a grep, so it should be a gate, not a note asking people to remember. Closes part of #36 Bump-Type: patch |
||
|
mathias
|
08c935ba8a |
chore(debug): fold in two techniques, delete systematic-debugging
superpowers-systematic-debugging restated debug's core (hypothesis before fix) across 296 lines, with a fix-implementation phase that belongs to tdd and a rationalization table that mostly repeats debug's own Anti-Patterns. Two things in it were not restatement, and both are now in debug: instrument every component boundary before hypothesising when a bug spans components, and treat three failed fixes as evidence the mechanism was never identified rather than as a reason to try a fourth. Not deleted with it, contrary to the issue: find-polluter.sh and the three technique files (root-cause-tracing, defense-in-depth, condition-based-waiting) are real assets, one of them an actual tool. They move to debug/references/ with a stated load trigger each, because a reference nobody knows when to open is dead weight. The pressure-test transcripts and CREATION-LOG from the upstream fork go; they document obra's process, not ours. Pressure-tested per #19, and the RED phase refuted the issue's premise. Two subagents without the skill got the multi-component scenario and the four-failed- fixes scenario; both already instrumented boundaries unprompted, and the second refused to write a fifth fix on its own initiative. So these sections are not correcting a demonstrated gap in this model — they are insurance for weaker models and for prompts less well-posed than a test scenario. Recorded as such rather than claimed as a fix. The GREEN run applied both sections correctly and then found four defects in them, all fixed here: the boundary rule read as suspending Iron Law 2's 3-5 hypotheses; the trigger said "reduced" when fixes that change nothing are the stronger signal; "stop proposing a fourth" read as licence to return no hypotheses at all; and the references had no load triggers. Its remaining findings are pre-existing and filed as #36. 41 skills. debug 146 -> 188 lines. Closes #14 Bump-Type: minor |
||
|
mathias
|
c6787e04d7 |
feat(index): generate the skills table from frontmatter
Second half of #29. The index was hand-maintained and had already drifted once: dream and experiment-spec shipped with no row, their rows lost to a squash-merge after a directory reorg. bin/check-index caught that after the fact; generating removes the class. The curated summary and trigger move into each SKILL.md as `summary:` and `use_when:`, carried over verbatim from the existing table rather than derived from `description:` — description is a paragraph written for model selection, and truncating it would have produced a worse table than the one it replaced. Knowledge now lives once, next to the skill it describes, which is the argument that made a hand-maintained index the wrong shape to begin with. The table is written between markers, so the wiring docs, the relationships tree and the trigger table below it are untouched — those are prose, not data. Rows are alphabetical now; the conceptual grouping the old order carried lives in the relationships tree, which is the better place for it. A fourth column shows each skill's surfaces, which is what makes #29's field visible rather than merely enforced. bin/check-index and its test are deleted: a generated table cannot drift in either direction, so `generate-index --check` subsumes both of its assertions. Verified by mutation — deleting the tdd row turns `task check` red with "SKILLS_INDEX.md is out of date". Closes #29 Bump-Type: minor |
||
|
mathias
|
90a58f0b9e |
feat(skills): declare which surfaces each skill ships to
The repo distributes three ways — the Claude Code plugin, the symlinked library for Crush/Pi/OpenCode, and claude.ai, which has no fetch path at all — and treated every skill as belonging to all of them. claude.ai is not a fourth host to mirror: gitea-ci, playwright and web-shot have no shell and no cluster there, so a flat mirror installs skills that can never fire and degrades selection for the ones that can. Every SKILL.md now carries `surfaces:`, defaulting to [claude-code, library]; dream, own-the-task and grill-me add `chat`, matching what is actually installed on claude.ai today. Claude Code accepts the extra key — `claude plugin validate` passes and the skills still load. bin/check-surfaces validates values, not just presence, because the failure mode here is silent: a wrong surface removes a skill from a harness rather than erroring. install.sh links only skills whose surfaces include `library`, and a source that declares nothing at all is treated as foreign and linked as-is — Anthropic's official set has no reason to know this convention. doctor's missing-link check skips non-library skills for the same reason. Refs #29 Bump-Type: minor |
||
 mathiasandClaude Opus 5
|
56a947b2a0 |
refactor(plugin)!: ship as a Claude Code plugin, one dir link per harness
Skills move to skills/<name>/SKILL.md and the repo gains both manifests, so it is now a marketplace hosting exactly one plugin whose root is the repo root (source: "."). Claude Code installs it from git; no symlinks involved. Why: install.sh wired the library into every consuming repo as an ABSOLUTE symlink to $CHECKOUT_DIR. The checkout moved three times, most recently when local-dev@1966d5a dropped the embedded .skills/ copy, and nothing migrated the links — 243 dangling links across 12 repos, hidden from git by the **/.claude/skills/ ignore rule. Worse, ~/dev spans two path roots (/Users/mathias on the iCloud-shared Macs, /home/mathias on koala), so an absolute link stored there is broken on one side by construction. Re-running the installer cannot fix that; only removing the mechanism can. install.sh keeps the harnesses with no plugin system and drops from 42 links per harness to one directory link, via the convergent ~/.agents/skills convention that Pi and OpenCode both read: ~/.agents/skills -> $CHECKOUT_DIR/skills ~/.config/crush/skills -> ~/.agents/skills ~/.config/opencode/skills -> ~/.agents/skills wire_claude_repo is deleted outright rather than gated behind a flag — on a mixed-root fleet it cannot be made correct. Antigravity and Vibe targets go too; both unused. CHECKOUT_DIR keeps its host-local default deliberately: pointing it into ~/dev would put a git working tree inside iCloud with two concurrent writers. Taskfile drops five near-identical wiring loops (they had already drifted) and delegates to install.sh. `task check` gains frontmatter validation, a stray-root-entry gate, and manifest/tag version parity — the loose secret-hygiene.md sat unloadable for months because nothing checked layout. Verified: claude plugin validate . passes clean; a probe skill loaded as skills:zzprobe through --plugin-dir; task check green at 42 skills. Refs: #23 Bump-Type: minor Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BeAp5LTscnz2W7Ubt4eJ6m |