Merge 2 techniques from superpowers-systematic-debugging into debug, then delete it #14

Closed
opened 2026-07-25 20:46:56 +00:00 by mathias · 1 comment
Owner

Overlap audit (fork, 2026-07-25): debug and superpowers-systematic-debugging cover the same core (hypothesis-before-fix) but debug is tighter and better-integrated: diagnose-only, ranked 3-5 hypotheses w/ VERIFY/EXPECTED format, delegates fix to tdd, brain/session_log-integrated (146 lines). systematic-debugging (296 lines) duplicates a fix-implementation phase that belongs to tdd, uses generic "your human partner" phrasing, and its rationalization table mostly restates debug's own Anti-Patterns table.

Action:

  1. Fold into debug: multi-component evidence-gathering technique (log at each boundary when a bug spans components) + the "3+ failed fixes → question architecture, not the fix" escalation rule. Not currently in debug.
  2. Drop everything else from systematic-debugging as pure restatement (phase-gate ceremony, duplicate fix-phase, restated rationalization table).
  3. Delete superpowers-systematic-debugging. Update SKILLS_INDEX.md + trigger table refs.

Target: debug grows ~146→~180 lines, stays the single debugging skill.

Overlap audit (fork, 2026-07-25): `debug` and `superpowers-systematic-debugging` cover the same core (hypothesis-before-fix) but `debug` is tighter and better-integrated: diagnose-only, ranked 3-5 hypotheses w/ VERIFY/EXPECTED format, delegates fix to `tdd`, brain/session_log-integrated (146 lines). `systematic-debugging` (296 lines) duplicates a fix-implementation phase that belongs to `tdd`, uses generic "your human partner" phrasing, and its rationalization table mostly restates debug's own Anti-Patterns table. Action: 1. Fold into `debug`: multi-component evidence-gathering technique (log at each boundary when a bug spans components) + the "3+ failed fixes → question architecture, not the fix" escalation rule. Not currently in `debug`. 2. Drop everything else from `systematic-debugging` as pure restatement (phase-gate ceremony, duplicate fix-phase, restated rationalization table). 3. Delete `superpowers-systematic-debugging`. Update `SKILLS_INDEX.md` + trigger table refs. Target: `debug` grows ~146→~180 lines, stays the single debugging skill.
mathias added the risk/lowsize/stype/chore labels 2026-08-27 05:25:45 +00:00
Author
Owner

Done in 08c935b — CI run #38 green. 41 skills; debug 146 → 188 lines (issue predicted ~180).

The pressure test refuted the premise

Two subagents, neither with the skill, got the exact scenarios this issue is about.

Multi-component evidence gathering — a 5% empty-200 bug across API → service → Postgres/Redis, two fixes already failed. Unprompted, it proposed per-boundary logging:

Log in A, per request: B's status code, Content-Length, first 200 bytes of B's body, the remote peer IP, and both build SHAs. Then map the failing peer IPs to pods. Evidence, not inference.

The 3-failed-fixes escalation — four shipped fixes for duplicate job processing, asked for a fifth. It opened with:

I'd refuse to ship a fifth fix yet. […] Four fixes that each reduced duplicates are four signs nobody has identified the mechanism. […] A fifth guess narrows it to 1% and we're back here in a fortnight, now with five interacting layers nobody can reason about.

So neither technique corrects a demonstrated gap in this model. Both are still in, and the commit says why: insurance for weaker models and for real prompts, which are far less well-posed than a test scenario. Recorded as insurance rather than claimed as a fix — #19's bar is about knowing which one you shipped.

Deviation: the sidecar files are not restatement

The issue says delete everything else. superpowers-systematic-debugging/ also held find-polluter.sh — a working bisection tool for finding which test pollutes shared state — plus root-cause-tracing.md, defense-in-depth.md and condition-based-waiting.md. Deleting those would have thrown away the one thing #19 explicitly prefers over prose. They moved to debug/references/, each with a load trigger, because a reference nobody knows when to open is dead weight. The upstream pressure-test transcripts and CREATION-LOG did go — they document obra's process, not ours.

The GREEN run found four defects in my own additions

All fixed before commit:

  • The boundary rule read as suspending Iron Law 2 — "the skill simultaneously forbade and mandated the only deliverable it defines". It now says the boundary spec comes first in the output and hypotheses still follow.
  • The trigger said fixes that reduced a symptom; fixes that changed nothing are the stronger signal and fell outside the literal wording. Now "failed to eliminate", with that noted.
  • "Stop proposing a fourth" read as licence to return no hypotheses. Now: aim them one level up, at the shared premise.
  • The references had no load triggers. Each has one — including condition-based-waiting.md, which is transparently relevant to any report whose first attempted fix was "we bumped the timeout".

Its remaining findings are pre-existing (no intake gate, a one-command verification rule that breaks on infra, a session_log payload missing session_id, a leaked Mode 2 roadmap note) and are filed as #36 rather than swept in here.

Done in `08c935b` — CI run [#38](https://git.d-ma.be/mathias/skills/actions/runs/4855) green. 41 skills; `debug` 146 → 188 lines (issue predicted ~180). ## The pressure test refuted the premise Two subagents, neither with the skill, got the exact scenarios this issue is about. **Multi-component evidence gathering** — a 5% empty-200 bug across API → service → Postgres/Redis, two fixes already failed. Unprompted, it proposed per-boundary logging: > Log in A, per request: B's status code, `Content-Length`, first 200 bytes of B's body, the remote peer IP, and both build SHAs. Then map the failing peer IPs to pods. Evidence, not inference. **The 3-failed-fixes escalation** — four shipped fixes for duplicate job processing, asked for a fifth. It opened with: > I'd refuse to ship a fifth fix yet. […] Four fixes that each *reduced* duplicates are four signs nobody has identified the mechanism. […] A fifth guess narrows it to 1% and we're back here in a fortnight, now with five interacting layers nobody can reason about. So neither technique corrects a demonstrated gap in this model. Both are still in, and the commit says why: insurance for weaker models and for real prompts, which are far less well-posed than a test scenario. Recorded as insurance rather than claimed as a fix — #19's bar is about knowing which one you shipped. ## Deviation: the sidecar files are not restatement The issue says delete everything else. `superpowers-systematic-debugging/` also held `find-polluter.sh` — a working bisection tool for finding which test pollutes shared state — plus `root-cause-tracing.md`, `defense-in-depth.md` and `condition-based-waiting.md`. Deleting those would have thrown away the one thing #19 explicitly prefers over prose. They moved to `debug/references/`, each with a load trigger, because a reference nobody knows when to open is dead weight. The upstream pressure-test transcripts and CREATION-LOG did go — they document obra's process, not ours. ## The GREEN run found four defects in my own additions All fixed before commit: - The boundary rule read as suspending Iron Law 2 — *"the skill simultaneously forbade and mandated the only deliverable it defines"*. It now says the boundary spec comes first in the output and hypotheses still follow. - The trigger said fixes that *reduced* a symptom; fixes that changed **nothing** are the stronger signal and fell outside the literal wording. Now "failed to eliminate", with that noted. - "Stop proposing a fourth" read as licence to return no hypotheses. Now: aim them one level up, at the shared premise. - The references had no load triggers. Each has one — including `condition-based-waiting.md`, which is transparently relevant to any report whose first attempted fix was "we bumped the timeout". Its remaining findings are pre-existing (no intake gate, a one-command verification rule that breaks on infra, a `session_log` payload missing `session_id`, a leaked Mode 2 roadmap note) and are filed as **#36** rather than swept in here.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/skills#14