docs(tdd): fix ten defects the GREEN run found in the language references
The pressure test that #18 and #20 were waiting on. All six original gaps came back ANSWERED with the lines quoted — and the run then found ten problems in what I had written. The worst two are mine, not inherited. **The Python exemplar was the evaluation task.** references/python.md demonstrated `split_vat` — the exact function the test scenario asks for, negative-amount case included. The run said so: "as a teaching example that's over-fitted; as an evaluation it's a leak... this task can't answer it — the reference handed over the finished artifact." So the run could not measure whether the guidance transfers, only whether copying works. The exemplar is now `parse_retry_after`, deliberately unrelated to anything in this estate, and it states its own contract rather than requiring the reader to divide 12500 by 1.25 to discover it. **mypy at RED pointed at the stub the same file forbids.** python.md taught that a pytest ImportError is a legitimate red, then presented `mypy --strict` as a matched pair without saying that mypy reports the same missing module — so following both literally, the way to get mypy green at RED is to create the module. Now stated: that error is the expected red; what must be fixed immediately is a --strict violation inside the test file you just wrote. The rest, each quoted in the run: - The checklist contradicted itself two lines apart — "strike it deliberately" above, "Can't check all boxes? You skipped TDD" below. A struck box is by definition unchecked, so the Python path routed straight into "start over". - `match=` was overclaimed as "free". It is `re.search`, so a message containing parens quietly matches almost anything; and it couples every error test to wording. The Go comparison was a strawman against `assert.Error` when `assert.ErrorIs` exists — the honest Python analogue is a custom exception type, with `match=` as the fallback. - `pytest.approx` was listed as the soft-assert analogue. It solves float tolerance and has nothing to do with collecting failures; that is pytest-subtests or pytest-check, neither of which ships with pytest. - The whole-shape assertion silently required value equality — on a plain class it compares identity and can never pass. - "When Stuck" is a language-neutral table that prescribed `t.Helper()`. - `__tracebackhide__` must be a local inside the helper body; at module level it does nothing, silently. Refs #18, #20 Bump-Type: patch
This commit is contained in:
+32
-2
@@ -12,7 +12,7 @@ description: Systematic hypothesis-first debugging. Generate 3-5 ranked hypothes
|
||||
|
||||
Debugging is hypothesis generation, not fix generation. The first instinct — "try this and see if it works" — wastes time and teaches nothing. A disciplined debug session produces a small set of ranked, falsifiable hypotheses, each with a concrete verification command. The fix comes after one hypothesis is confirmed, not before.
|
||||
|
||||
**Core principle:** A hypothesis you cannot verify in one command is not a hypothesis — it is a guess.
|
||||
**Core principle:** A hypothesis you cannot verify with a single bounded, decisive observation is not a hypothesis — it is a guess. In a repo that observation is usually one command. Above one it may be an instrumented run, or partitioning the last twenty CI runs by trigger type; what makes it a hypothesis is that the observation *discriminates*, not that it fits on one line.
|
||||
|
||||
## When to Use
|
||||
|
||||
@@ -25,6 +25,27 @@ Debugging is hypothesis generation, not fix generation. The first instinct — "
|
||||
- Compile errors with clear messages (just fix the typo)
|
||||
- Already-diagnosed bugs where you know the cause (go straight to TDD with a regression test)
|
||||
|
||||
### Is it actually diagnosed?
|
||||
|
||||
That last exclusion is the one people get wrong, because a confident wrong diagnosis feels
|
||||
exactly like a right one. It is diagnosed if you can state the mechanism *and* name the
|
||||
observation that would have proved it wrong. If the evidence is "we changed X and it got
|
||||
better", that is correlation with a deployment, and this skill applies.
|
||||
|
||||
The strongest tell is a run of failed fixes — see below. Three plausible fixes that did
|
||||
not eliminate the symptom is not bad luck, it is the diagnosis being wrong.
|
||||
|
||||
### Intake: what you need before hypothesising
|
||||
|
||||
Before generating anything, confirm you have the failure itself — the error text, the
|
||||
stack, the failing run's id, the log. If the report is "it's flaky again" with no
|
||||
artifact, **name the specific thing you need and stop**. Iron Law 1 forbids "try X and
|
||||
see"; hypothesising into a vacuum is the same failure with better grammar, and it is
|
||||
worse because the output looks like analysis.
|
||||
|
||||
This is the rule `cad-babysit` already enforces from the other side: read the issue's own
|
||||
comments and the job log *first*.
|
||||
|
||||
## Iron Laws
|
||||
|
||||
1. **Never suggest "try X and see what happens."** Every hypothesis must have a specific expected outcome if correct.
|
||||
@@ -76,7 +97,15 @@ For each hypothesis, capture three things:
|
||||
- **Verification:** the single command or file inspection that confirms or denies it
|
||||
- **Expected outcome if correct:** the specific output you would see
|
||||
|
||||
Order by likelihood. The most likely cause is hypothesis 1.
|
||||
Order by likelihood, most likely first. **The ordering is the ranking — do not also
|
||||
attach `high`/`medium`/`low` labels.** With five slots and three labels the labels stop
|
||||
discriminating (two hypotheses are genuinely both "high" and the label cannot say which
|
||||
comes first), and a reader then has two orderings that can disagree. Position already
|
||||
carries it.
|
||||
|
||||
Where you can say *why* one outranks another — a prior brain entry, a matching symptom,
|
||||
a recent change in that area — say that instead of a label. That is information; a label
|
||||
is a restatement of the order it sits next to.
|
||||
|
||||
### Step 3: Output
|
||||
|
||||
@@ -140,6 +169,7 @@ The brain holds prior debug sessions across the project. Use it to skip rediscov
|
||||
|
||||
**At debug start:**
|
||||
- Run `brain_query` with the error message snippet + the package name. Past sessions may have logged identical or similar failures with their resolved hypotheses.
|
||||
- **A hit is evidence, so cite it inline in the hypothesis it informs** — `(brain: <entry-name>)` — rather than mentioning it once at the top. If a prior entry is why a hypothesis ranks first, the reader needs that at the hypothesis, not in a preamble. If a prior entry contradicts your ranking, say so and rank it anyway if the evidence here is better; the brain records what was true then.
|
||||
|
||||
**After a hypothesis is confirmed:**
|
||||
- Run `brain_write` with the failure signature → confirmed mechanism. Future debug sessions on the same area get the answer immediately.
|
||||
|
||||
+2
-2
@@ -365,7 +365,7 @@ Before marking work complete:
|
||||
- [ ] Tests use real code (mocks only if crossing a system boundary)
|
||||
- [ ] Edge cases and errors covered
|
||||
|
||||
Can't check all boxes? You skipped TDD. Start over.
|
||||
Can't check **or deliberately strike** all boxes? You skipped TDD. Start over. A box struck with a stated reason is answered; a box left hanging is not.
|
||||
|
||||
## When Stuck
|
||||
|
||||
@@ -374,7 +374,7 @@ Can't check all boxes? You skipped TDD. Start over.
|
||||
| Don't know how to test | Write wished-for API. Write assertion first. Ask Mathias. |
|
||||
| Test too complicated | Design too complicated. Simplify interface. |
|
||||
| Must mock everything | Code too coupled. Use dependency injection. |
|
||||
| Test setup huge | Extract helpers with `t.Helper()`. Still complex? Simplify design. |
|
||||
| Test setup huge | Extract helpers (`t.Helper()` in Go, a local `__tracebackhide__ = True` in Python). Still complex? Simplify design. |
|
||||
|
||||
## Debugging Integration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user