docs(tdd): fix ten defects the GREEN run found in the language references
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s

The pressure test that #18 and #20 were waiting on. All six original gaps came
back ANSWERED with the lines quoted — and the run then found ten problems in
what I had written. The worst two are mine, not inherited.

**The Python exemplar was the evaluation task.** references/python.md
demonstrated `split_vat` — the exact function the test scenario asks for,
negative-amount case included. The run said so: "as a teaching example that's
over-fitted; as an evaluation it's a leak... this task can't answer it — the
reference handed over the finished artifact." So the run could not measure
whether the guidance transfers, only whether copying works. The exemplar is now
`parse_retry_after`, deliberately unrelated to anything in this estate, and it
states its own contract rather than requiring the reader to divide 12500 by 1.25
to discover it.

**mypy at RED pointed at the stub the same file forbids.** python.md taught that
a pytest ImportError is a legitimate red, then presented `mypy --strict` as a
matched pair without saying that mypy reports the same missing module — so
following both literally, the way to get mypy green at RED is to create the
module. Now stated: that error is the expected red; what must be fixed
immediately is a --strict violation inside the test file you just wrote.

The rest, each quoted in the run:

- The checklist contradicted itself two lines apart — "strike it deliberately"
  above, "Can't check all boxes? You skipped TDD" below. A struck box is by
  definition unchecked, so the Python path routed straight into "start over".
- `match=` was overclaimed as "free". It is `re.search`, so a message containing
  parens quietly matches almost anything; and it couples every error test to
  wording. The Go comparison was a strawman against `assert.Error` when
  `assert.ErrorIs` exists — the honest Python analogue is a custom exception
  type, with `match=` as the fallback.
- `pytest.approx` was listed as the soft-assert analogue. It solves float
  tolerance and has nothing to do with collecting failures; that is
  pytest-subtests or pytest-check, neither of which ships with pytest.
- The whole-shape assertion silently required value equality — on a plain class
  it compares identity and can never pass.
- "When Stuck" is a language-neutral table that prescribed `t.Helper()`.
- `__tracebackhide__` must be a local inside the helper body; at module level it
  does nothing, silently.

Refs #18, #20
Bump-Type: patch
This commit is contained in:
2026-08-29 15:16:20 +02:00
parent 06592805b6
commit cba0fe4337
2 changed files with 34 additions and 4 deletions
+32 -2
View File
@@ -12,7 +12,7 @@ description: Systematic hypothesis-first debugging. Generate 3-5 ranked hypothes
Debugging is hypothesis generation, not fix generation. The first instinct — "try this and see if it works" — wastes time and teaches nothing. A disciplined debug session produces a small set of ranked, falsifiable hypotheses, each with a concrete verification command. The fix comes after one hypothesis is confirmed, not before. Debugging is hypothesis generation, not fix generation. The first instinct — "try this and see if it works" — wastes time and teaches nothing. A disciplined debug session produces a small set of ranked, falsifiable hypotheses, each with a concrete verification command. The fix comes after one hypothesis is confirmed, not before.
**Core principle:** A hypothesis you cannot verify in one command is not a hypothesis — it is a guess. **Core principle:** A hypothesis you cannot verify with a single bounded, decisive observation is not a hypothesis — it is a guess. In a repo that observation is usually one command. Above one it may be an instrumented run, or partitioning the last twenty CI runs by trigger type; what makes it a hypothesis is that the observation *discriminates*, not that it fits on one line.
## When to Use ## When to Use
@@ -25,6 +25,27 @@ Debugging is hypothesis generation, not fix generation. The first instinct — "
- Compile errors with clear messages (just fix the typo) - Compile errors with clear messages (just fix the typo)
- Already-diagnosed bugs where you know the cause (go straight to TDD with a regression test) - Already-diagnosed bugs where you know the cause (go straight to TDD with a regression test)
### Is it actually diagnosed?
That last exclusion is the one people get wrong, because a confident wrong diagnosis feels
exactly like a right one. It is diagnosed if you can state the mechanism *and* name the
observation that would have proved it wrong. If the evidence is "we changed X and it got
better", that is correlation with a deployment, and this skill applies.
The strongest tell is a run of failed fixes — see below. Three plausible fixes that did
not eliminate the symptom is not bad luck, it is the diagnosis being wrong.
### Intake: what you need before hypothesising
Before generating anything, confirm you have the failure itself — the error text, the
stack, the failing run's id, the log. If the report is "it's flaky again" with no
artifact, **name the specific thing you need and stop**. Iron Law 1 forbids "try X and
see"; hypothesising into a vacuum is the same failure with better grammar, and it is
worse because the output looks like analysis.
This is the rule `cad-babysit` already enforces from the other side: read the issue's own
comments and the job log *first*.
## Iron Laws ## Iron Laws
1. **Never suggest "try X and see what happens."** Every hypothesis must have a specific expected outcome if correct. 1. **Never suggest "try X and see what happens."** Every hypothesis must have a specific expected outcome if correct.
@@ -76,7 +97,15 @@ For each hypothesis, capture three things:
- **Verification:** the single command or file inspection that confirms or denies it - **Verification:** the single command or file inspection that confirms or denies it
- **Expected outcome if correct:** the specific output you would see - **Expected outcome if correct:** the specific output you would see
Order by likelihood. The most likely cause is hypothesis 1. Order by likelihood, most likely first. **The ordering is the ranking — do not also
attach `high`/`medium`/`low` labels.** With five slots and three labels the labels stop
discriminating (two hypotheses are genuinely both "high" and the label cannot say which
comes first), and a reader then has two orderings that can disagree. Position already
carries it.
Where you can say *why* one outranks another — a prior brain entry, a matching symptom,
a recent change in that area — say that instead of a label. That is information; a label
is a restatement of the order it sits next to.
### Step 3: Output ### Step 3: Output
@@ -140,6 +169,7 @@ The brain holds prior debug sessions across the project. Use it to skip rediscov
**At debug start:** **At debug start:**
- Run `brain_query` with the error message snippet + the package name. Past sessions may have logged identical or similar failures with their resolved hypotheses. - Run `brain_query` with the error message snippet + the package name. Past sessions may have logged identical or similar failures with their resolved hypotheses.
- **A hit is evidence, so cite it inline in the hypothesis it informs** — `(brain: <entry-name>)` — rather than mentioning it once at the top. If a prior entry is why a hypothesis ranks first, the reader needs that at the hypothesis, not in a preamble. If a prior entry contradicts your ranking, say so and rank it anyway if the evidence here is better; the brain records what was true then.
**After a hypothesis is confirmed:** **After a hypothesis is confirmed:**
- Run `brain_write` with the failure signature → confirmed mechanism. Future debug sessions on the same area get the answer immediately. - Run `brain_write` with the failure signature → confirmed mechanism. Future debug sessions on the same area get the answer immediately.
+2 -2
View File
@@ -365,7 +365,7 @@ Before marking work complete:
- [ ] Tests use real code (mocks only if crossing a system boundary) - [ ] Tests use real code (mocks only if crossing a system boundary)
- [ ] Edge cases and errors covered - [ ] Edge cases and errors covered
Can't check all boxes? You skipped TDD. Start over. Can't check **or deliberately strike** all boxes? You skipped TDD. Start over. A box struck with a stated reason is answered; a box left hanging is not.
## When Stuck ## When Stuck
@@ -374,7 +374,7 @@ Can't check all boxes? You skipped TDD. Start over.
| Don't know how to test | Write wished-for API. Write assertion first. Ask Mathias. | | Don't know how to test | Write wished-for API. Write assertion first. Ask Mathias. |
| Test too complicated | Design too complicated. Simplify interface. | | Test too complicated | Design too complicated. Simplify interface. |
| Must mock everything | Code too coupled. Use dependency injection. | | Must mock everything | Code too coupled. Use dependency injection. |
| Test setup huge | Extract helpers with `t.Helper()`. Still complex? Simplify design. | | Test setup huge | Extract helpers (`t.Helper()` in Go, a local `__tracebackhide__ = True` in Python). Still complex? Simplify design. |
## Debugging Integration ## Debugging Integration