Files
skills/skills/tdd/SKILL.md
T
mathias cba0fe4337
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
docs(tdd): fix ten defects the GREEN run found in the language references
The pressure test that #18 and #20 were waiting on. All six original gaps came
back ANSWERED with the lines quoted — and the run then found ten problems in
what I had written. The worst two are mine, not inherited.

**The Python exemplar was the evaluation task.** references/python.md
demonstrated `split_vat` — the exact function the test scenario asks for,
negative-amount case included. The run said so: "as a teaching example that's
over-fitted; as an evaluation it's a leak... this task can't answer it — the
reference handed over the finished artifact." So the run could not measure
whether the guidance transfers, only whether copying works. The exemplar is now
`parse_retry_after`, deliberately unrelated to anything in this estate, and it
states its own contract rather than requiring the reader to divide 12500 by 1.25
to discover it.

**mypy at RED pointed at the stub the same file forbids.** python.md taught that
a pytest ImportError is a legitimate red, then presented `mypy --strict` as a
matched pair without saying that mypy reports the same missing module — so
following both literally, the way to get mypy green at RED is to create the
module. Now stated: that error is the expected red; what must be fixed
immediately is a --strict violation inside the test file you just wrote.

The rest, each quoted in the run:

- The checklist contradicted itself two lines apart — "strike it deliberately"
  above, "Can't check all boxes? You skipped TDD" below. A struck box is by
  definition unchecked, so the Python path routed straight into "start over".
- `match=` was overclaimed as "free". It is `re.search`, so a message containing
  parens quietly matches almost anything; and it couples every error test to
  wording. The Go comparison was a strawman against `assert.Error` when
  `assert.ErrorIs` exists — the honest Python analogue is a custom exception
  type, with `match=` as the fallback.
- `pytest.approx` was listed as the soft-assert analogue. It solves float
  tolerance and has nothing to do with collecting failures; that is
  pytest-subtests or pytest-check, neither of which ships with pytest.
- The whole-shape assertion silently required value equality — on a plain class
  it compares identity and can never pass.
- "When Stuck" is a language-neutral table that prescribed `t.Helper()`.
- `__tracebackhide__` must be a local inside the helper body; at module level it
  does nothing, silently.

Refs #18, #20
Bump-Type: patch
2026-08-29 15:16:20 +02:00

432 lines
13 KiB
Markdown

---
name: tdd
surfaces: [claude-code, library]
summary: "Test-Driven Development enforcement"
use_when: "Starting any new feature or fixing a bug"
description: Write failing tests first, then minimal code to pass. Non-negotiable for all new features and bug fixes.
---
# Test-Driven Development (TDD)
## Overview
Write the test first. Watch it fail. Write minimal code to pass.
**Core principle:** If you didn't watch the test fail, you don't know if it tests the right thing.
**Violating the letter of the rules is violating the spirit of the rules.**
## When to Use
**Always:**
- New features
- Bug fixes
- Refactoring
- Behavior changes
**Exceptions (ask Mathias):**
- Throwaway prototypes
- Generated code (sqlc output, templ output)
- Configuration files
Thinking "skip TDD just this once"? Stop. That's rationalization.
## The Iron Law
```
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
```
Write code before the test? Delete it. Start over.
**No exceptions:**
- Don't keep it as "reference"
- Don't "adapt" it while writing tests
- Don't look at it
- Delete means delete
Implement fresh from tests. Period.
## Red-Green-Refactor
```
RED → verify fails correctly → GREEN → verify all pass → REFACTOR → stay green → RED (next)
```
### RED - Write Failing Test
Write one minimal test showing what should happen.
**Go example (good):**
```go
func TestRetryOperation_RetriesThreeTimes(t *testing.T) {
attempts := 0
op := func() error {
attempts++
if attempts < 3 {
return errors.New("fail")
}
return nil
}
err := RetryOperation(op, 3)
require.NoError(t, err)
assert.Equal(t, 3, attempts)
}
```
Clear name, tests real behavior, one thing.
**Go example (bad):**
```go
func TestRetry(t *testing.T) {
// Vague name, tests nothing meaningful
err := RetryOperation(nil, 0)
assert.NoError(t, err)
}
```
**Requirements:**
- One behavior per test
- Clear name describing the behavior
- Real code (no mocks unless crossing a system boundary)
### Verify RED - Watch It Fail
**MANDATORY. Never skip.**
```bash
go test -run TestRetryOperation_RetriesThreeTimes ./...
```
Confirm:
- Test fails, and fails because the feature is missing
- The failure message is the one you expected
- It is not failing on a typo in the test itself
**Test passes?** You're testing existing behavior. Fix test.
**Failing because the thing does not exist yet?** That is a legitimate red — a Go
compile error for an undefined function, a Python `ImportError` for a module you have
not written, a TypeScript "cannot find module". Read the message, confirm it names the
unit you are about to build, and proceed to GREEN. **Do not write a stub to make the
runner happier.** A stub is production code written before a failing test, which is the
Iron Law with extra steps.
**Failing on a typo or a broken import elsewhere?** Different thing. Fix it and re-run
until the failure is the one above.
### GREEN - Minimal Code
Write simplest code to pass the test.
**Good:**
```go
func RetryOperation(op func() error, maxRetries int) error {
for i := 0; i < maxRetries; i++ {
if err := op(); err == nil {
return nil
}
}
return op()
}
```
Just enough to pass.
**Bad:**
```go
func RetryOperation(op func() error, maxRetries int, opts ...RetryOption) error {
// YAGNI — don't add options, backoff, jitter before the test demands it
}
```
Over-engineered.
Don't add features, refactor other code, or "improve" beyond what the test demands.
### Verify GREEN - Watch It Pass
**MANDATORY.**
```bash
go test ./...
```
Confirm:
- Test passes
- All other tests still pass
- No race conditions: `go test -race ./...`
**Test fails?** Fix code, not test.
**Other tests fail?** Fix now.
### REFACTOR - Clean Up
After green only:
- Remove duplication
- Improve names
- Extract helpers
- Apply clean code principles (load `clean-code` skill)
Keep tests green. Don't add behavior.
### Repeat
Next failing test for next behavior.
## Other languages
The discipline above is language-neutral; the examples below it are Go, because Go is
this fleet's default stack and one good example beats five mediocre ones.
For anything else, the per-language conventions live in reference files — load the one
you need rather than translating Go idioms by eye. Translation is where the mistakes
happen: the Go table-driven exemplar below branches on a `wantErr` column inside the
loop, which is idiomatic Go and an anti-pattern in pytest, where it produces a
conditional assertion and folds two behaviours into one test.
- `references/python.md` — pytest, `@pytest.mark.parametrize`, `mypy --strict` in the
RED gate, `pytest.raises(..., match=)`, fixtures vs helpers
- `references/typescript.md` — vitest/jest, `it.each`, `expect(...).rejects`
- `references/rust.md` — `#[cfg(test)]` modules, table-driven via arrays, `#[should_panic]`
- `references/java.md` — JUnit 5, `@ParameterizedTest`, `assertThrows`
## Go-Specific TDD Notes
### Table-Driven Tests (Preferred Pattern)
```go
func TestValidateEmail(t *testing.T) {
tests := []struct {
name string
email string
wantErr bool
}{
{"valid email", "user@example.com", false},
{"empty email", "", true},
{"no at sign", "userexample.com", true},
{"no domain", "user@", true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := ValidateEmail(tt.email)
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
})
}
}
```
Table-driven tests are the Go idiom. Use them for behavior that varies across inputs.
### Subtests with t.Run
Use `t.Run` to name subtests clearly. Failure messages include the subtest name.
```go
t.Run("rejects empty input", func(t *testing.T) { ... })
t.Run("accepts valid UUID", func(t *testing.T) { ... })
```
### Test File Conventions
- File: `<package>_test.go` in the same directory
- Package: `package foo_test` for black-box testing (preferred), `package foo` for white-box
- Helpers: use `t.Helper()` so stack traces point to the caller, not the helper
```go
func assertNoError(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
```
### Running Tests
```bash
go test ./... # all packages
go test -run TestFoo ./pkg/... # specific test
go test -run TestFoo/subtest ./... # specific subtest
go test -race ./... # race detector (always run before commit)
go test -cover ./... # coverage
go test -v ./... # verbose
```
### testify
Pre-approved. Use `assert` (continues on failure) and `require` (stops on failure):
```go
require.NoError(t, err) // fatal if error
assert.Equal(t, expected, got) // non-fatal comparison
assert.ErrorIs(t, err, ErrFoo) // error chain check
```
## Good Tests
| Quality | Good | Bad |
|---------|------|-----|
| **Minimal** | One thing. "and" in name? Split it. | `TestValidatesEmailAndDomainAndWhitespace` |
| **Clear** | Name describes behavior | `TestFoo`, `Test1` |
| **Shows intent** | Demonstrates desired API | Obscures what code should do |
| **Table-driven** | Multiple cases, one test function | Copy-pasted test functions |
## Common Rationalizations
| Excuse | Reality |
|--------|---------|
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
| "I'll test after" | Tests passing immediately prove nothing. |
| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |
| "Already manually tested" | Ad-hoc ≠ systematic. No record, can't re-run. |
| "Deleting X hours is wasteful" | Sunk cost fallacy. Keeping unverified code is technical debt. |
| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |
| "Need to explore first" | Fine. Throw away exploration, start with TDD. |
| "Test hard = design unclear" | Listen to the test. Hard to test = hard to use. |
| "TDD will slow me down" | TDD faster than debugging. Pragmatic = test-first. |
| "Existing code has no tests" | You're improving it. Add tests for existing code. |
## Red Flags - STOP and Start Over
- Code before test
- Test after implementation
- Test passes immediately without seeing it fail
- Can't explain why test failed
- Tests added "later"
- Rationalizing "just this once"
- "Already manually tested it"
- "Tests after achieve the same purpose"
- "Keep as reference" or "adapt existing code"
- "Already spent X hours, deleting is wasteful"
- "TDD is dogmatic, I'm being pragmatic"
- "This is different because..."
**All of these mean: Delete code. Start over with TDD.**
## Example: Bug Fix
**Bug:** Empty email accepted
**RED**
```go
func TestSubmitForm_RejectsEmptyEmail(t *testing.T) {
result := submitForm(FormData{Email: ""})
assert.Equal(t, "email required", result.Error)
}
```
**Verify RED**
```bash
$ go test -run TestSubmitForm_RejectsEmptyEmail ./...
FAIL: expected "email required", got ""
```
**GREEN**
```go
func submitForm(data FormData) FormResult {
if strings.TrimSpace(data.Email) == "" {
return FormResult{Error: "email required"}
}
// ...
}
```
**Verify GREEN**
```bash
$ go test ./...
ok example.com/myapp 0.003s
```
**REFACTOR**
Extract validation for multiple fields if needed.
## Verification Checklist
Before marking work complete:
- [ ] Every new function/method has a test
- [ ] Watched each test fail before implementing
- [ ] Each test failed for expected reason (feature missing, not typo)
- [ ] Wrote minimal code to pass each test
- [ ] All tests pass
- [ ] The language's static gate is clean — `go vet`, `mypy --strict`, `tsc --noEmit`,
`cargo clippy`. A red type-check is a red build; it does not wait for GREEN.
- [ ] The language's concurrency gate is clean **where one exists and applies** —
`go test -race`, `cargo test` under Miri for unsafe code. There is no equivalent
for a pure Python function, and a checklist box that can never be ticked teaches
people to skip checklists. Tick it or strike it deliberately; do not leave it
hanging.
- [ ] Tests use real code (mocks only if crossing a system boundary)
- [ ] Edge cases and errors covered
Can't check **or deliberately strike** all boxes? You skipped TDD. Start over. A box struck with a stated reason is answered; a box left hanging is not.
## When Stuck
| Problem | Solution |
|---------|----------|
| Don't know how to test | Write wished-for API. Write assertion first. Ask Mathias. |
| Test too complicated | Design too complicated. Simplify interface. |
| Must mock everything | Code too coupled. Use dependency injection. |
| Test setup huge | Extract helpers (`t.Helper()` in Go, a local `__tracebackhide__ = True` in Python). Still complex? Simplify design. |
## Debugging Integration
Bug found? Write failing test reproducing it. Follow TDD cycle. Test proves fix and prevents regression.
Never fix bugs without a test.
## Testing Anti-Patterns
When adding test utilities or mocks, load the `references/testing-anti-patterns.md` to avoid:
- Testing mock behavior instead of real behavior
- Adding test-only methods to production types
- Mocking without understanding what the dependency does
## Brain MCP Integration
The brain MCP exposes session context across machines. Use it to make TDD cycles cumulative rather than one-shot.
**At session start:**
- Run `brain_query` with the feature name + "tdd" to surface prior cycles, anti-patterns, or testing decisions for this code area. Skip if the feature is brand new.
**Never:**
- Embed brain content in test code or assertions. The brain is context for you, not state for the system under test.
### Logging
Call `session_log` once at the end of every phase to record the outcome.
Pass-rate is computed downstream by the `/pass-rate` HTTP endpoint, which
treats `pass` as success, `fail` as failure, `skip` as neither.
**At end of `red` phase:**
- `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "red", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**At end of `green` phase:**
- `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "green", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**At end of `refactor` phase:**
- `session_log` with `{session_id: "<this session's id>", skill: "tdd", phase: "refactor", final_status: "pass" | "fail" | "skip", message: "<one-line summary>", project_root: "<absolute path>"}`
**Status semantics:**
- `pass` — the phase's intended outcome was reached (red: test fails as expected; green: test passes; refactor: tests still pass after refactor).
- `fail` — the phase's intended outcome was NOT reached (test compiled when it shouldn't, test still fails after green attempt, refactor broke tests).
- `skip` — phase was skipped intentionally (e.g. refactor not warranted).
**Why this matters:** pass-rate is what a future routing decision would be based on — whether a `tdd` call can be served by a local model instead of a cloud one. A skill that never logs contributes no evidence either way, and a decision made on no evidence is a coin flip wearing a metric.
## Final Rule
```
Production code → test exists and failed first
Otherwise → not TDD
```
No exceptions without Mathias's permission.