Files
skills/skills/debug/SKILL.md
T
mathias 0c7fac3686
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
fix(debug): close the intake gate properly — and the contradiction it exposed
The GREEN run confirmed the gate works: given "builds are flaky again... what's
the fix?" with no run id and no log, it produced zero hypotheses and an ordered
list of artifacts, each justified by what it would discriminate. It also caught
that the retry already shipped in that pipeline is destroying the evidence
needed to diagnose it, which is a better finding than any hypothesis would have
been.

Then it found eight problems, and the first two are mine:

**The template contradicted the rule.** I banned high/medium/low labels in
Step 2 and left them in the Step 3 output template and all three worked-example
hypotheses. "Prose loses to a format block every time" — correct, and the
template is what gets imitated. Labels are gone from both; the slot is now WHY
FIRST, which asks for the evidence that sets the order rather than a restatement
of it.

**The description never mentioned the gate.** That is the text a harness shows
before an agent decides to load anything, and it read "Generate 3-5 ranked
hypotheses" — so an agent routing on the description alone does exactly the
thing this skill was rewritten to prevent. Both description and use_when now
lead with the refusal.

The rest:

- Intake vs the failed-fixes section pointed opposite ways for a report that
  fires both, and nothing stated precedence. Intake wins, and says so: you
  cannot aim hypotheses at a premise you have no evidence about either.
- "One run narrows the search to a single component" was overclaimed exactly
  where it gets used — CI flakiness. One run *that reproduces* narrows it; a
  passing run narrows nothing.
- session_log had no phase for a correct refusal, so an intake stop logged as a
  failed read-failure. A metric that penalises the skill's best behaviour trains
  the next model to guess. Added an `intake` phase.
- The Anti-Patterns table — the part people scan in a hurry — was missing the
  failure this whole rewrite is about: answering "what's the fix?" from a report
  with no artifact.
- brain_query told you to search on an error string you may have just refused to
  proceed without. Now: query symptom + component, which may itself name the
  artifact to ask for.
- condition-based-waiting was an unconditional load on any mention of a timeout,
  including during an intake stop, which is the wrong side of the gate.

Closes #36
Bump-Type: minor
2026-08-29 15:17:16 +02:00

13 KiB

name, surfaces, summary, use_when, description
name surfaces summary use_when description
debug
claude-code
library
Hypothesis-first debugging discipline Any test failure or bug whose cause is not immediately obvious — or a bug report with no failure text, which this skill refuses rather than guesses at Systematic hypothesis-first debugging. Requires the failure itself — error text, stack, run id or log — and names what it needs and stops if the report has none, rather than hypothesising into a vacuum. Given evidence, generates 3-5 ranked, falsifiable hypotheses with verification steps before suggesting any fix. Use when encountering a bug, test failure, or unexpected behavior.

Debug

Overview

Debugging is hypothesis generation, not fix generation. The first instinct — "try this and see if it works" — wastes time and teaches nothing. A disciplined debug session produces a small set of ranked, falsifiable hypotheses, each with a concrete verification command. The fix comes after one hypothesis is confirmed, not before.

Core principle: A hypothesis you cannot verify with a single bounded, decisive observation is not a hypothesis — it is a guess. In a repo that observation is usually one command. Above one it may be an instrumented run, or partitioning the last twenty CI runs by trigger type; what makes it a hypothesis is that the observation discriminates, not that it fits on one line.

When to Use

  • Any test failure whose cause is not immediately obvious
  • Any production error or unexpected behavior
  • Any bug report from a user or stakeholder
  • Before reaching for a debugger or adding print statements: form hypotheses first

Do not use for:

  • Compile errors with clear messages (just fix the typo)
  • Already-diagnosed bugs where you know the cause (go straight to TDD with a regression test)

Is it actually diagnosed?

That last exclusion is the one people get wrong, because a confident wrong diagnosis feels exactly like a right one. It is diagnosed if you can state the mechanism and name the observation that would have proved it wrong. If the evidence is "we changed X and it got better", that is correlation with a deployment, and this skill applies.

The strongest tell is a run of failed fixes — see below. Three plausible fixes that did not eliminate the symptom is not bad luck, it is the diagnosis being wrong.

Intake: what you need before hypothesising

Before generating anything, confirm you have the failure itself — the error text, the stack, the failing run's id, the log. If the report is "it's flaky again" with no artifact, name the specific thing you need and stop. Iron Law 1 forbids "try X and see"; hypothesising into a vacuum is the same failure with better grammar, and it is worse because the output looks like analysis.

This is the rule cad-babysit already enforces from the other side: read the issue's own comments and the job log first.

Iron Laws

  1. Never suggest "try X and see what happens." Every hypothesis must have a specific expected outcome if correct.
  2. Generate 3-5 hypotheses, ordered by likelihood (most likely first). Fewer than 3 means you stopped thinking; more than 5 means you are not prioritizing.
  3. Diagnose only — do not fix in this skill. The fix happens in a separate TDD cycle (load tdd skill) once a hypothesis is confirmed.

When the bug spans components

A system with boundaries (CI → build → signing, API → service → database) hides the question "which component is wrong?" behind the question "why is the output wrong?". Specify, as a pre-step, what to log entering and leaving each boundary, plus whether config and environment actually propagated. One run that reproduces the failure narrows the search to a single component. On an intermittent fault a passing run narrows nothing, so leave the instrumentation in across several runs rather than concluding from one.

This does not suspend Iron Law 2. Still emit 3-5 hypotheses — the boundary spec comes first in the output, and each hypothesis says which boundary observation would kill it. What the pre-step forbids is hypotheses that quietly assume a component is the faulty one before any evidence says so.

When several fixes have already failed

Three or more plausible fixes that failed to eliminate a symptom is not a run of bad luck — it is evidence that nobody has identified the mechanism. (Fixes that changed nothing at all count double: they are stronger evidence of a misidentified mechanism than fixes that partially worked.) Each one has been narrowing a window rather than closing it, and a fourth guess buys another tenth of the rate plus a layer of interacting guards nobody can reason about.

Intake still gates this section. A report of three failed fixes with no artifacts fires both rules at once, and intake wins: you cannot aim hypotheses at a premise you have no evidence about either. Ask for the log, then aim.

Say that explicitly — the refusal is the finding, not a preamble to it. Then aim the hypotheses one level up: the premise those fixes share is itself the thing to falsify. This is not an exemption from the output format. It is the format, pointed at the assumption instead of at the symptom.

Process

Step 1: Read the failure

Before forming hypotheses:

  • Read the full error message and stack trace, not just the headline
  • Read the file where the failure originated, around the failing line
  • If the failure is from a test, read the test and the code under test
  • Identify the failure mode — what actually went wrong (e.g. "nil pointer dereference in goroutine spawned by handler") not just what the error says ("runtime error")

Step 2: Generate hypotheses

For each hypothesis, capture three things:

  • Mechanism: what specific code path or state would produce this exact failure
  • Verification: the single command or file inspection that confirms or denies it
  • Expected outcome if correct: the specific output you would see

Order by likelihood, most likely first. The ordering is the ranking — do not also attach high/medium/low labels. With five slots and three labels the labels stop discriminating (two hypotheses are genuinely both "high" and the label cannot say which comes first), and a reader then has two orderings that can disagree. Position already carries it.

Where you can say why one outranks another — a prior brain entry, a matching symptom, a recent change in that area — say that instead of a label. That is information; a label is a restatement of the order it sits next to.

Step 3: Output

Use this format:

HYPOTHESIS 1: <mechanism in one sentence>
WHY FIRST: <what makes it likeliest — a matching symptom, a recent change, a brain entry>
VERIFY: <exact command, or the bounded observation that discriminates>
EXPECTED IF CORRECT: <specific output, value, or condition>

HYPOTHESIS 2: <mechanism>
WHY FIRST: <or why it ranks here rather than first>
VERIFY: <exact command>
EXPECTED IF CORRECT: <specific output>

[... up to 5 ...]

RECOMMENDED NEXT STEP: Run VERIFY for hypothesis 1 first.

End with the recommendation, not a fix.

Worked Example

Failure:

--- FAIL: TestInvoiceParser_HandlesEmptyPDF (0.00s)
    parser_test.go:47: panic: runtime error: index out of range [0] with length 0

Output:

HYPOTHESIS 1: parser indexes into pages[0] before checking len(pages) > 0; empty PDFs produce a zero-page document
VERIFY: rg -n 'pages\[0\]' internal/parser/
EXPECTED IF CORRECT: at least one site reads pages[0] without a preceding length check

HYPOTHESIS 2: the test fixture is a zero-byte file rather than a valid empty PDF; pdf library returns nil pages slice instead of empty
VERIFY: ls -la testdata/empty.pdf && file testdata/empty.pdf
EXPECTED IF CORRECT: file size 0 bytes or "data" rather than "PDF document"

HYPOTHESIS 3: a recently changed dependency reordered the page-extraction API; pages[0] now refers to metadata, not content
VERIFY: git log --oneline -10 -- go.sum | grep -i pdf
EXPECTED IF CORRECT: a pdf-library bump in the last few commits

RECOMMENDED NEXT STEP: Run VERIFY for hypothesis 1 first.

Anti-Patterns

Anti-Pattern Why It Fails
"Maybe try restarting the service" Not a mechanism. Not falsifiable. Teaches nothing if it works.
"Could be a race condition" Mechanism without specifics. Which two operations race? On what state?
"Let me add some print statements and see" Skips hypothesis generation. Generates noise, not understanding.
Answering "what's the fix?" from a report with no artifact Hypothesising into a vacuum. The output looks like analysis, which is what makes it worse than saying "I need the log".
Single hypothesis presented as fact If you are sure, write the regression test, do not run a debug skill.
Mixing hypotheses with fix suggestions The skill is diagnose-only. The fix is a TDD task on its hypothesis.

Brain MCP Integration

The brain holds prior debug sessions across the project. Use it to skip rediscovering known failure modes.

At debug start:

  • Run brain_query with the error message snippet + the package name. On an intake stop you have neither — query the symptom and the component instead ("signed artifact missing CI"), which is worth doing precisely because it may name the artifact to ask for. Past sessions may have logged identical or similar failures with their resolved hypotheses.
  • A hit is evidence, so cite it inline in the hypothesis it informs — (brain: <entry-name>) — rather than mentioning it once at the top. If a prior entry is why a hypothesis ranks first, the reader needs that at the hypothesis, not in a preamble. If a prior entry contradicts your ranking, say so and rank it anyway if the evidence here is better; the brain records what was true then.

After a hypothesis is confirmed:

  • Run brain_write with the failure signature → confirmed mechanism. Future debug sessions on the same area get the answer immediately.

Never:

  • Run brain_write for an unconfirmed hypothesis. Speculation in the brain pollutes future queries.

Logging

Call session_log once at the end of every phase to record the outcome. Pass-rate is computed downstream by the /pass-rate HTTP endpoint, which treats pass as success, fail as failure, skip as neither.

At end of each phase:

  • session_log with {session_id: "<this session's id>", skill: "debug", phase: "<phase-name>", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}

Phases for this skill: intake, read-failure, generate-hypotheses, output

intake exists so that refusing correctly is recordable. A run that stops for want of a run id has done the right thing and logs intake / pass; without that phase the refusal logs as a failed read-failure, and a metric that penalises the skill's best behaviour teaches the next model to hypothesise into the vacuum instead.

Status semantics:

  • pass — the phase's intended outcome was reached.
  • fail — the phase's intended outcome was NOT reached.
  • skip — phase was skipped intentionally.

Why this matters: pass-rate is what a future routing decision would be based on — whether a given call can be served by a local model instead of a cloud one. A skill that never logs contributes no evidence either way.

Cross-References

  • After a hypothesis is confirmed, load tdd skill — write a failing regression test that proves the bug exists, then fix it.
  • For test failures specifically caused by mock-vs-real divergence, also load tdd/references/testing-anti-patterns.md.
  • Load code-review skill if the diagnosis surfaces a structural issue (god object, shotgun surgery) rather than a pointwise bug.

Techniques kept in debug/references/, inherited from the deleted superpowers-systematic-debugging (#14):

  • root-cause-tracing.md — load when the failure surfaces far from its trigger and the stack is the only thread back to it.
  • defense-in-depth.md — load after a root cause is confirmed, to decide which layers should have caught it.
  • condition-based-waiting.md — load whenever a timeout appears anywhere in the report, including as one of the fixes already tried. A bumped timeout is a symptom. Not during an intake stop: if you are not analysing yet, a symptom-analysis reference is on the wrong side of the gate.
  • find-polluter.sh — run when a test passes alone and fails in the suite, or when something in the tree appears that no test admits to creating.

Escalation

Diagnose only — Iron Law 3 is already an escalation rule: the fix is a separate TDD cycle, not this skill's output.

Stop and hand back when: the report contains no failure text and no locator, in which case name the specific artifact you need (run id, job log, stack trace) and stop rather than hypothesising into a vacuum; or the evidence needed to discriminate between your top two hypotheses requires access you do not have.