Files
skills/skills/debug/references/defense-in-depth.md
T
mathias 08c935ba8a
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
chore(debug): fold in two techniques, delete systematic-debugging
superpowers-systematic-debugging restated debug's core (hypothesis before fix)
across 296 lines, with a fix-implementation phase that belongs to tdd and a
rationalization table that mostly repeats debug's own Anti-Patterns. Two things
in it were not restatement, and both are now in debug: instrument every
component boundary before hypothesising when a bug spans components, and treat
three failed fixes as evidence the mechanism was never identified rather than
as a reason to try a fourth.

Not deleted with it, contrary to the issue: find-polluter.sh and the three
technique files (root-cause-tracing, defense-in-depth, condition-based-waiting)
are real assets, one of them an actual tool. They move to debug/references/ with
a stated load trigger each, because a reference nobody knows when to open is
dead weight. The pressure-test transcripts and CREATION-LOG from the upstream
fork go; they document obra's process, not ours.

Pressure-tested per #19, and the RED phase refuted the issue's premise. Two
subagents without the skill got the multi-component scenario and the four-failed-
fixes scenario; both already instrumented boundaries unprompted, and the second
refused to write a fifth fix on its own initiative. So these sections are not
correcting a demonstrated gap in this model — they are insurance for weaker
models and for prompts less well-posed than a test scenario. Recorded as such
rather than claimed as a fix.

The GREEN run applied both sections correctly and then found four defects in
them, all fixed here: the boundary rule read as suspending Iron Law 2's 3-5
hypotheses; the trigger said "reduced" when fixes that change nothing are the
stronger signal; "stop proposing a fourth" read as licence to return no
hypotheses at all; and the references had no load triggers. Its remaining
findings are pre-existing and filed as #36.

41 skills. debug 146 -> 188 lines.

Closes #14
Bump-Type: minor
2026-08-29 11:33:36 +02:00

3.6 KiB

Defense-in-Depth Validation

Overview

When you fix a bug caused by invalid data, adding validation at one place feels sufficient. But that single check can be bypassed by different code paths, refactoring, or mocks.

Core principle: Validate at EVERY layer data passes through. Make the bug structurally impossible.

Why Multiple Layers

Single validation: "We fixed the bug" Multiple layers: "We made the bug impossible"

Different layers catch different cases:

  • Entry validation catches most bugs
  • Business logic catches edge cases
  • Environment guards prevent context-specific dangers
  • Debug logging helps when other layers fail

The Four Layers

Layer 1: Entry Point Validation

Purpose: Reject obviously invalid input at API boundary

function createProject(name: string, workingDirectory: string) {
  if (!workingDirectory || workingDirectory.trim() === '') {
    throw new Error('workingDirectory cannot be empty');
  }
  if (!existsSync(workingDirectory)) {
    throw new Error(`workingDirectory does not exist: ${workingDirectory}`);
  }
  if (!statSync(workingDirectory).isDirectory()) {
    throw new Error(`workingDirectory is not a directory: ${workingDirectory}`);
  }
  // ... proceed
}

Layer 2: Business Logic Validation

Purpose: Ensure data makes sense for this operation

function initializeWorkspace(projectDir: string, sessionId: string) {
  if (!projectDir) {
    throw new Error('projectDir required for workspace initialization');
  }
  // ... proceed
}

Layer 3: Environment Guards

Purpose: Prevent dangerous operations in specific contexts

async function gitInit(directory: string) {
  // In tests, refuse git init outside temp directories
  if (process.env.NODE_ENV === 'test') {
    const normalized = normalize(resolve(directory));
    const tmpDir = normalize(resolve(tmpdir()));

    if (!normalized.startsWith(tmpDir)) {
      throw new Error(
        `Refusing git init outside temp dir during tests: ${directory}`
      );
    }
  }
  // ... proceed
}

Layer 4: Debug Instrumentation

Purpose: Capture context for forensics

async function gitInit(directory: string) {
  const stack = new Error().stack;
  logger.debug('About to git init', {
    directory,
    cwd: process.cwd(),
    stack,
  });
  // ... proceed
}

Applying the Pattern

When you find a bug:

  1. Trace the data flow - Where does bad value originate? Where used?
  2. Map all checkpoints - List every point data passes through
  3. Add validation at each layer - Entry, business, environment, debug
  4. Test each layer - Try to bypass layer 1, verify layer 2 catches it

Example from Session

Bug: Empty projectDir caused git init in source code

Data flow:

  1. Test setup → empty string
  2. Project.create(name, '')
  3. WorkspaceManager.createWorkspace('')
  4. git init runs in process.cwd()

Four layers added:

  • Layer 1: Project.create() validates not empty/exists/writable
  • Layer 2: WorkspaceManager validates projectDir not empty
  • Layer 3: WorktreeManager refuses git init outside tmpdir in tests
  • Layer 4: Stack trace logging before git init

Result: All 1847 tests passed, bug impossible to reproduce

Key Insight

All four layers were necessary. During testing, each layer caught bugs the others missed:

  • Different code paths bypassed entry validation
  • Mocks bypassed business logic checks
  • Edge cases on different platforms needed environment guards
  • Debug logging identified structural misuse

Don't stop at one validation point. Add checks at every layer.