Compare commits

...
11 Commits
Author SHA1 Message Date
mathias 21dbb35526 chore(skills): delete superpowers-test-driven-development
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 0s
Verified duplicate rather than assumed one. Section for section it is the same
skill as `tdd` — Overview, When to Use, Iron Law, Red-Green-Refactor with the
same dot graph, Good Tests, Common Rationalizations, Red Flags, Verification
Checklist, When Stuck, Final Rule — with TypeScript examples and obra's "your
human partner" phrasing instead of ours. `tdd` carries all of it plus the
Go-specific material (table-driven tests, t.Run, testify, go test -race), the
brain MCP integration and the Mode 2 routing note.

Its one non-SKILL.md file, testing-anti-patterns.md, is byte-identical to
skills/tdd/references/testing-anti-patterns.md, which `tdd` already points at.
Nothing was salvageable because nothing was different.

No references anywhere outside its own directory except the index row, removed
here. `atdd` is untouched — acceptance-level is a genuinely different altitude
and it already cross-refs `tdd` correctly.

Incidentally the first live catch by bin/check-index (#15): deleting the
directory turned `task check` red with "SKILLS_INDEX.md lists
'superpowers-test-driven-development', which has no ...SKILL.md" before I got
to the row.

42 skills now. Closes #13
Bump-Type: patch
2026-08-29 09:55:53 +02:00
mathias 8ad2079f86 chore(release): plugin.json version 0.8.4
Skip-Release: true
2026-08-29 07:54:53 +00:00
mathias b188965161 docs(index): place experiment-spec in the relationships diagram
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
Missed in 83b069a. #15 asked for three things and the table row was only two
of them: experiment-spec also belongs in the Skill Relationships tree, parallel
to feature-spec and downstream of spec-driven-dev, since it is the research/ML
substitute for feature-spec rather than a separate lifecycle.

dream needs no edge, same as trainer and session-retrospective — the issue said
so and that part was right.

Refs #15
Bump-Type: patch
2026-08-29 09:54:23 +02:00
mathias ae66840a03 chore(release): plugin.json version 0.8.3
Skip-Release: true
2026-08-29 07:53:48 +00:00
mathias e7e514390b docs(web-shot): resolve shot.sh from the install, not a checkout path
CI / Check / Test (push) Successful in 1s
CI / Tag release (push) Successful in 1s
The usage block pointed at ~/dev/.skills/web-shot/shot.sh four times. That
directory has not existed since the skills moved into this repo, and the
failure mode is worse than a broken link: web-shot exists for agents with no
local browser, so a 404 sends them to hand-roll the Job instead of looking
harder. One caller burned three attempts on playwright module resolution
before finding the script that documents that exact gotcha.

Replaced with a resolution line covering both delivery paths — the Claude Code
plugin and the ~/.agents/skills symlink — so no absolute checkout path is left
to rot. Verified on koala: resolves to the .agents link, and degrades to an
empty string plus a clear message where the skill is not installed.

The brain entry headless-browser-via-k3s-playwright-job carried the same stale
path in three places and referenced the retired wire-skills.sh wiring; updated
to match.

Closes #26
Bump-Type: patch
2026-08-29 09:53:33 +02:00
mathias 087685f2e3 chore(release): plugin.json version 0.8.2
Skip-Release: true
2026-08-29 07:52:16 +00:00
mathias 83b069afba docs(index): add dream + experiment-spec rows, and gate the index
CI / Check / Test (push) Successful in 0s
CI / Tag release (push) Successful in 1s
Both skills shipped without an index row. The rows existed — on the feature
branches deleted in #22 — and were dropped by the squash-merge that followed
a directory reorg: the SKILL.md landed under skills/, the row did not. Nothing
errored, the index just under-reported the library to every harness reading it.

Adding two rows by hand fixes today and not tomorrow, so the assertion goes in
`task check` via bin/check-index, following the same pattern as
bin/should-skip-release: a real script with a real test, not untested logic
buried in a Taskfile. It checks both directions — a skill with no row, and a
row naming a skill that no longer exists.

Verified by mutation: deleting the `dream` row turns `task check` red and the
report names it.

Closes #15
Bump-Type: patch
2026-08-29 09:52:03 +02:00
mathias a699ad8cbd chore(release): plugin.json version 0.8.1
Skip-Release: true
2026-08-29 07:50:52 +00:00
mathias a42ec18f40 fix(secret-guard): treat env as a dump only when no command follows
CI / Tag release (push) Successful in 1s
CI / Check / Test (push) Successful in 1s
`env -u VAR cmd` and `env FOO=bar cmd` exec a command with a modified
environment and print nothing themselves. The old rule matched any `env`
token followed anywhere later by a pipe into a filter, so it blocked the
documented 1Password write path and forced a `(unset VAR; cmd)` workaround.
False positives only — never a leak — but a guard people route around is a
guard on its way out.

The rule now denies only when everything after `env` is exhausted by options
and VAR=value assignments. That is strictly the dumping shape, and it also
closes a hole the two previous rules left open: `env FOO=bar` with no command
word prints the whole environment and was allowed.

Three new deny/allow pairs pin it, including the 1Password write shape that
motivated the report. 20 cases now, up from 14, all passing.

Closes #25
Bump-Type: patch
2026-08-29 09:50:08 +02:00
mathias 46ff1acdfe chore(release): plugin.json version 0.8.0
Skip-Release: true
2026-08-29 07:47:07 +00:00
mathias 3112b4ce92 chore(ci): gate the release tagger on check + test
CI / Check / Test (push) Successful in 0s
CI / Tag release (push) Successful in 0s
The repo had a `task check` structure gate and a 28-case shell test suite
that nothing ever ran. The only workflow auto-tagged every push to main, so
CI reported green having tested nothing — a gate that fails open, and the
reason `mathias/skills` shows up as repository-blocked in the estate backlog
classification.

release.yml becomes ci.yml with two jobs: `gate` runs check + test on every
push and PR; `tag` keeps the existing semver-from-trailer logic but declares
`needs: gate`, so a red check cannot ship a tag. The pull_request trigger
also gives `tbd_ship` a CI run to read on a branch head — without one it
fails closed and never merges (#22).

The tagger now pushes the manifest bump commit and the tag in a single
`git push`. Two pushes left a window where the gate ran `task check` on the
bump commit before the tag existed, and the manifest/tag parity assertion
would fail on a commit that was in fact correct.

tests/ci_gate_test.sh pins the wiring: PR trigger present, both tasks run,
tag needs gate, no second ungated workflow file.

Refs: #22
Bump-Type: minor
2026-08-29 09:46:10 +02:00
12 changed files with 275 additions and 708 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
{ {
"name": "skills", "name": "skills",
"description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.", "description": "Mathias's engineering skills library, plus the hooks and MCP servers that make it enforceable: TDD, clean code, debugging, specs, discovery, CI, and secret hygiene.",
"version": "0.7.0", "version": "0.8.4",
"author": { "author": {
"name": "Mathias", "name": "Mathias",
"url": "https://git.d-ma.be/mathias" "url": "https://git.d-ma.be/mathias"
@@ -1,23 +1,52 @@
name: release name: CI
# Auto-tag every push to main with a semver bump derived from the latest # Two jobs, in order:
#
# gate — `task check` (structure, frontmatter, manifest/tag parity) plus
# `task test` (hooks, doctor, release-skip). Runs on every push and
# every PR. Before this existed the suite ran nowhere: CI reported
# green having tested nothing (#22).
#
# tag — auto-tags a push to main with a semver bump derived from the latest
# commit footer: # commit footer:
# Bump-Type: major → vX.0.0 # Bump-Type: major → vX.0.0
# Bump-Type: minor → v0.Y.0 # Bump-Type: minor → v0.Y.0
# (default) → v0.0.Z (patch) # (default) → v0.0.Z (patch)
# # Skips when the commit carries the `Skip-Release: true` trailer (the
# Skips when the latest commit is itself the auto-tagger (commit message # tagger's own manifest bump) to prevent recursion. Gated on `gate`,
# contains "[skip-release]") to prevent recursion. Tag is pushed back # so a red check cannot ship a tag.
# to the repo using the GITEA_TOKEN provided by act_runner.
on: "on":
push: push:
branches: branches: ["**"]
- main pull_request:
branches: [main]
jobs: jobs:
tag: gate:
name: Check / Test
runs-on: self-hosted runs-on: self-hosted
steps:
- uses: actions/checkout@v4
with:
# `task check` compares plugin.json's version to `git describe`;
# a shallow checkout has no tags and would skip that assertion.
fetch-depth: 0
- name: Verify toolchain
run: task --version
- name: Structure gate
run: task check
- name: Shell test suite
run: task test
tag:
name: Tag release
needs: gate
runs-on: self-hosted
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
steps: steps:
- name: checkout - name: checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -72,16 +101,19 @@ jobs:
manifest=.claude-plugin/plugin.json manifest=.claude-plugin/plugin.json
version="${TAG#v}" version="${TAG#v}"
current=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$manifest" | head -1) current=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' "$manifest" | head -1)
git tag -a "$TAG" -m "release $TAG"
if [ "$current" != "$version" ]; then if [ "$current" != "$version" ]; then
sed -i "s/\"version\"[[:space:]]*:[[:space:]]*\"$current\"/\"version\": \"$version\"/" "$manifest" sed -i "s/\"version\"[[:space:]]*:[[:space:]]*\"$current\"/\"version\": \"$version\"/" "$manifest"
git add "$manifest" git add "$manifest"
# The trailer below stops this push from re-triggering the tagger. # The trailer below stops this push from re-triggering the tagger.
git commit -m "chore(release): plugin.json version $version" -m "Skip-Release: true" git commit -m "chore(release): plugin.json version $version" -m "Skip-Release: true"
git push origin HEAD:main # Commit and tag go up in ONE push: the gate job runs `task check`
# on the bump commit, and that asserts manifest == latest tag. Two
# separate pushes leave a window where it legitimately fails.
git push origin HEAD:main "$TAG"
echo "bumped $manifest to $version" echo "bumped $manifest to $version"
fi else
git tag -a "$TAG" -m "release $TAG"
# gitea-actions checkout uses an x-access-token URL; reuse it.
git push origin "$TAG" git push origin "$TAG"
fi
echo "pushed $TAG" echo "pushed $TAG"
+11 -8
View File
@@ -19,9 +19,11 @@ This index lists all available engineering skills. Load the full SKILL.md on dem
| `gitea-ci` | Gitea CI/CD pipeline with act_runner, buildah, k3s, GitHub mirror | Setting up or debugging CI on this infra | | `gitea-ci` | Gitea CI/CD pipeline with act_runner, buildah, k3s, GitHub mirror | Setting up or debugging CI on this infra |
| `debug` | Hypothesis-first debugging discipline | Any test failure or bug whose cause is not immediately obvious | | `debug` | Hypothesis-first debugging discipline | Any test failure or bug whose cause is not immediately obvious |
| `feature-spec` | Implementation-level feature spec (tighter than spec-driven-dev) | Scoping one feature inside an already-specced project | | `feature-spec` | Implementation-level feature spec (tighter than spec-driven-dev) | Scoping one feature inside an already-specced project |
| `experiment-spec` | Rigorous experiment spec — falsifiable hypothesis, quantitative acceptance, null-result protocol | Scoping a research/ML phase instead of a feature |
| `playwright` | Playwright E2E testing — setup, HTMX patterns, SSE streams, CI wiring | Adding browser tests to any project with a web UI | | `playwright` | Playwright E2E testing — setup, HTMX patterns, SSE streams, CI wiring | Adding browser tests to any project with a web UI |
| `session-retrospective` | Surface non-obvious learnings from a session log | After a coding session ends, before context is lost | | `session-retrospective` | Surface non-obvious learnings from a session log | After a coding session ends, before context is lost |
| `trainer` | Two-phase brain curation (score candidates, write past quality gate) | Periodic brain pruning and curation | | `trainer` | Two-phase brain curation (score candidates, write past quality gate) | Periodic brain pruning and curation |
| `dream` | Reflective consolidation pass over an agent's memory directory | Pruning/merging memory files, rebuilding a memory index |
| `grill-me` | Structured plan interrogation — Quick Poke, Full Grill, Pre-mortem | Stress-testing a plan before committing; end of Diamond 1; before promoting to pre-prod | | `grill-me` | Structured plan interrogation — Quick Poke, Full Grill, Pre-mortem | Stress-testing a plan before committing; end of Diamond 1; before promoting to pre-prod |
| `own-the-task` | Delegation ownership — Intake → Brief → Plan → Execute | A task is handed over open-ended; "take this off my plate"; before a plan exists | | `own-the-task` | Delegation ownership — Intake → Brief → Plan → Execute | A task is handed over open-ended; "take this off my plate"; before a plan exists |
| `telos-load` | Load TELOS intention substrate at session start | Starting any koala session; before architectural decisions; CAD pipeline entry | | `telos-load` | Load TELOS intention substrate at session start | Starting any koala session; before architectural decisions; CAD pipeline entry |
@@ -37,7 +39,6 @@ This index lists all available engineering skills. Load the full SKILL.md on dem
| `superpowers-dispatching-parallel-agents` | Fan work out across parallel subagents | Decomposing work for concurrent agents | | `superpowers-dispatching-parallel-agents` | Fan work out across parallel subagents | Decomposing work for concurrent agents |
| `superpowers-subagent-driven-development` | Subagent-driven development workflow | Building via orchestrated subagents | | `superpowers-subagent-driven-development` | Subagent-driven development workflow | Building via orchestrated subagents |
| `superpowers-systematic-debugging` | Hypothesis-first systematic debugging | Diagnosing a failure methodically | | `superpowers-systematic-debugging` | Hypothesis-first systematic debugging | Diagnosing a failure methodically |
| `superpowers-test-driven-development` | TDD discipline (superpowers variant) | Red → green → refactor on a change |
| `superpowers-verification-before-completion` | Verify behavior before claiming done | Before marking any task complete | | `superpowers-verification-before-completion` | Verify behavior before claiming done | Before marking any task complete |
| `superpowers-requesting-code-review` | Requesting a code review well | Asking another agent/human to review | | `superpowers-requesting-code-review` | Requesting a code review well | Asking another agent/human to review |
| `superpowers-receiving-code-review` | Acting on received review feedback | Responding to review comments | | `superpowers-receiving-code-review` | Acting on received review feedback | Responding to review comments |
@@ -107,14 +108,16 @@ problem-analysis
│ │
├── spec-driven-dev (PM-kickoff lifecycle) ├── spec-driven-dev (PM-kickoff lifecycle)
│ │ │ │
│ └── feature-spec (per-feature, mid-implementation) │ ├── feature-spec (per-feature, mid-implementation)
│ │ │
│ │ └── planning
│ │ │
│ │ └── atdd ──→ tdd (foundation)
│ │ ├──→ debug (when tests fail)
│ │ ├──→ clean-code (REFACTOR phase)
│ │ └──→ test-design (review)
│ │ │ │
│ └── planning │ └── experiment-spec (research/ML phase, instead of feature-spec)
│ │
│ └── atdd ──→ tdd (foundation)
│ ├──→ debug (when tests fail)
│ ├──→ clean-code (REFACTOR phase)
│ └──→ test-design (review)
│ │
└── planning └── planning
+6 -1
View File
@@ -94,7 +94,12 @@ tasks:
count=$((count + 1)) count=$((count + 1))
done done
# 3. Plugin manifest version must match the latest tag, or installers # 3. SKILLS_INDEX.md lists exactly the skills that exist. Delegated
# to bin/check-index so the rule is covered by
# tests/check_index_test.sh rather than living untested here.
sh '{{.TASKFILE_DIR}}/bin/check-index' '{{.TASKFILE_DIR}}' || fail=1
# 4. Plugin manifest version must match the latest tag, or installers
# never see the update (Claude Code gates updates on this field). # never see the update (Claude Code gates updates on this field).
if [ -f '{{.PLUGIN_MANIFEST}}' ]; then if [ -f '{{.PLUGIN_MANIFEST}}' ]; then
manifest_v=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' '{{.PLUGIN_MANIFEST}}' | head -1) manifest_v=$(sed -n 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' '{{.PLUGIN_MANIFEST}}' | head -1)
+42
View File
@@ -0,0 +1,42 @@
#!/bin/sh
# check-index — asserts SKILLS_INDEX.md lists exactly the skills that exist.
#
# The index is hand-maintained and drifts silently in both directions: a skill
# ships with no row (`dream` and `experiment-spec` did — their rows lived on
# branches squash-merged after a directory reorg, so the SKILL.md landed and
# the row did not, #15), or a row outlives the skill it names. Neither errors
# anywhere; the library just misreports itself to every harness that reads it.
#
# Usage: check-index [checkout-root] (defaults to this script's repo root)
set -u
ROOT=${1:-$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)}
INDEX="$ROOT/SKILLS_INDEX.md"
fail=0
if [ ! -f "$INDEX" ]; then
echo "ERROR: missing SKILLS_INDEX.md" >&2
exit 1
fi
# A row is a table line whose first cell is the skill name in backticks.
listed=$(sed -n 's/^|[[:space:]]*`\([^`]*\)`.*/\1/p' "$INDEX" | sort -u)
for dir in "$ROOT"/skills/*/; do
name=$(basename "$dir")
[ -f "$dir/SKILL.md" ] || continue
printf '%s\n' "$listed" | grep -qx "$name" || {
echo "ERROR: skills/$name has no row in SKILLS_INDEX.md" >&2
fail=1
}
done
for name in $listed; do
[ -f "$ROOT/skills/$name/SKILL.md" ] || {
echo "ERROR: SKILLS_INDEX.md lists '$name', which has no skills/$name/SKILL.md" >&2
fail=1
}
done
exit $fail
+12 -5
View File
@@ -50,11 +50,18 @@ if printf '%s' "$CMD" | grep -Eqi '(^|[;&|]|\s)op\s+document\s+get\b'; then
fi fi
# Raw env/secret-file dumps (infra#228/#229 root cause). # Raw env/secret-file dumps (infra#228/#229 root cause).
if printf '%s' "$CMD" | grep -Eqi '(^|[;&|]|\s)env(\s|$).*\|.*grep'; then #
deny "Blocked: 'env | grep' prints matching values in full. Use an existence-only check: [ -n \"\$VAR\" ] && echo set" # `env` is a dump only when nothing but options and VAR=value assignments
fi # follow it. `env -u VAR cmd` and `env FOO=bar cmd` EXEC a command with a
if printf '%s' "$CMD" | grep -Eq '(^|[;&|])[[:space:]]*env[[:space:]]*($|[;&|])'; then # modified environment and print nothing themselves — piping those into grep
deny "Blocked: bare 'env' prints every variable's value, including secrets. Use an existence-only check: [ -n \"\$VAR\" ] && echo set" # filters the command's own output, not the environment (#25). The earlier
# pattern matched any `env` token followed anywhere later by a pipe into
# grep, which blocked the documented 1Password write path
# (`env -u OP_SERVICE_ACCOUNT_TOKEN op item create`) and cost a workaround.
# Denying only the "no command word follows" shape also catches `env FOO=bar`
# with no command, which the two previous rules let through.
if printf '%s' "$CMD" | grep -Eqi '(^|[;&|]|[[:space:]])env([[:space:]]+-[iu][[:space:]]*[A-Za-z_][A-Za-z0-9_]*|[[:space:]]+-[iu]|[[:space:]]+[A-Za-z_][A-Za-z0-9_]*=[^[:space:]]*)*[[:space:]]*($|[|;&])'; then
deny "Blocked: this 'env' prints every variable's value, including secrets (no command word follows it, so it dumps rather than execs). Use an existence-only check: [ -n \"\$VAR\" ] && echo set. Note 'env -u VAR cmd' and 'env FOO=bar cmd' are allowed — they run a command."
fi fi
if printf '%s' "$CMD" | grep -Eqi '(^|[;&|]|\s)printenv\b'; then if printf '%s' "$CMD" | grep -Eqi '(^|[;&|]|\s)printenv\b'; then
deny "Blocked: printenv dumps all env values. Use: [ -n \"\$VAR\" ] && echo set" deny "Blocked: printenv dumps all env values. Use: [ -n \"\$VAR\" ] && echo set"
@@ -1,371 +0,0 @@
---
name: test-driven-development
description: Use when implementing any feature or bugfix, before writing implementation code
---
# Test-Driven Development (TDD)
## Overview
Write the test first. Watch it fail. Write minimal code to pass.
**Core principle:** If you didn't watch the test fail, you don't know if it tests the right thing.
**Violating the letter of the rules is violating the spirit of the rules.**
## When to Use
**Always:**
- New features
- Bug fixes
- Refactoring
- Behavior changes
**Exceptions (ask your human partner):**
- Throwaway prototypes
- Generated code
- Configuration files
Thinking "skip TDD just this once"? Stop. That's rationalization.
## The Iron Law
```
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
```
Write code before the test? Delete it. Start over.
**No exceptions:**
- Don't keep it as "reference"
- Don't "adapt" it while writing tests
- Don't look at it
- Delete means delete
Implement fresh from tests. Period.
## Red-Green-Refactor
```dot
digraph tdd_cycle {
rankdir=LR;
red [label="RED\nWrite failing test", shape=box, style=filled, fillcolor="#ffcccc"];
verify_red [label="Verify fails\ncorrectly", shape=diamond];
green [label="GREEN\nMinimal code", shape=box, style=filled, fillcolor="#ccffcc"];
verify_green [label="Verify passes\nAll green", shape=diamond];
refactor [label="REFACTOR\nClean up", shape=box, style=filled, fillcolor="#ccccff"];
next [label="Next", shape=ellipse];
red -> verify_red;
verify_red -> green [label="yes"];
verify_red -> red [label="wrong\nfailure"];
green -> verify_green;
verify_green -> refactor [label="yes"];
verify_green -> green [label="no"];
refactor -> verify_green [label="stay\ngreen"];
verify_green -> next;
next -> red;
}
```
### RED - Write Failing Test
Write one minimal test showing what should happen.
<Good>
```typescript
test('retries failed operations 3 times', async () => {
let attempts = 0;
const operation = () => {
attempts++;
if (attempts < 3) throw new Error('fail');
return 'success';
};
const result = await retryOperation(operation);
expect(result).toBe('success');
expect(attempts).toBe(3);
});
```
Clear name, tests real behavior, one thing
</Good>
<Bad>
```typescript
test('retry works', async () => {
const mock = jest.fn()
.mockRejectedValueOnce(new Error())
.mockRejectedValueOnce(new Error())
.mockResolvedValueOnce('success');
await retryOperation(mock);
expect(mock).toHaveBeenCalledTimes(3);
});
```
Vague name, tests mock not code
</Bad>
**Requirements:**
- One behavior
- Clear name
- Real code (no mocks unless unavoidable)
### Verify RED - Watch It Fail
**MANDATORY. Never skip.**
```bash
npm test path/to/test.test.ts
```
Confirm:
- Test fails (not errors)
- Failure message is expected
- Fails because feature missing (not typos)
**Test passes?** You're testing existing behavior. Fix test.
**Test errors?** Fix error, re-run until it fails correctly.
### GREEN - Minimal Code
Write simplest code to pass the test.
<Good>
```typescript
async function retryOperation<T>(fn: () => Promise<T>): Promise<T> {
for (let i = 0; i < 3; i++) {
try {
return await fn();
} catch (e) {
if (i === 2) throw e;
}
}
throw new Error('unreachable');
}
```
Just enough to pass
</Good>
<Bad>
```typescript
async function retryOperation<T>(
fn: () => Promise<T>,
options?: {
maxRetries?: number;
backoff?: 'linear' | 'exponential';
onRetry?: (attempt: number) => void;
}
): Promise<T> {
// YAGNI
}
```
Over-engineered
</Bad>
Don't add features, refactor other code, or "improve" beyond the test.
### Verify GREEN - Watch It Pass
**MANDATORY.**
```bash
npm test path/to/test.test.ts
```
Confirm:
- Test passes
- Other tests still pass
- Output pristine (no errors, warnings)
**Test fails?** Fix code, not test.
**Other tests fail?** Fix now.
### REFACTOR - Clean Up
After green only:
- Remove duplication
- Improve names
- Extract helpers
Keep tests green. Don't add behavior.
### Repeat
Next failing test for next feature.
## Good Tests
| Quality | Good | Bad |
|---------|------|-----|
| **Minimal** | One thing. "and" in name? Split it. | `test('validates email and domain and whitespace')` |
| **Clear** | Name describes behavior | `test('test1')` |
| **Shows intent** | Demonstrates desired API | Obscures what code should do |
## Why Order Matters
**"I'll write tests after to verify it works"**
Tests written after code pass immediately. Passing immediately proves nothing:
- Might test wrong thing
- Might test implementation, not behavior
- Might miss edge cases you forgot
- You never saw it catch the bug
Test-first forces you to see the test fail, proving it actually tests something.
**"I already manually tested all the edge cases"**
Manual testing is ad-hoc. You think you tested everything but:
- No record of what you tested
- Can't re-run when code changes
- Easy to forget cases under pressure
- "It worked when I tried it" ≠ comprehensive
Automated tests are systematic. They run the same way every time.
**"Deleting X hours of work is wasteful"**
Sunk cost fallacy. The time is already gone. Your choice now:
- Delete and rewrite with TDD (X more hours, high confidence)
- Keep it and add tests after (30 min, low confidence, likely bugs)
The "waste" is keeping code you can't trust. Working code without real tests is technical debt.
**"TDD is dogmatic, being pragmatic means adapting"**
TDD IS pragmatic:
- Finds bugs before commit (faster than debugging after)
- Prevents regressions (tests catch breaks immediately)
- Documents behavior (tests show how to use code)
- Enables refactoring (change freely, tests catch breaks)
"Pragmatic" shortcuts = debugging in production = slower.
**"Tests after achieve the same goals - it's spirit not ritual"**
No. Tests-after answer "What does this do?" Tests-first answer "What should this do?"
Tests-after are biased by your implementation. You test what you built, not what's required. You verify remembered edge cases, not discovered ones.
Tests-first force edge case discovery before implementing. Tests-after verify you remembered everything (you didn't).
30 minutes of tests after ≠ TDD. You get coverage, lose proof tests work.
## Common Rationalizations
| Excuse | Reality |
|--------|---------|
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
| "I'll test after" | Tests passing immediately prove nothing. |
| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |
| "Already manually tested" | Ad-hoc ≠ systematic. No record, can't re-run. |
| "Deleting X hours is wasteful" | Sunk cost fallacy. Keeping unverified code is technical debt. |
| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |
| "Need to explore first" | Fine. Throw away exploration, start with TDD. |
| "Test hard = design unclear" | Listen to test. Hard to test = hard to use. |
| "TDD will slow me down" | TDD faster than debugging. Pragmatic = test-first. |
| "Manual test faster" | Manual doesn't prove edge cases. You'll re-test every change. |
| "Existing code has no tests" | You're improving it. Add tests for existing code. |
## Red Flags - STOP and Start Over
- Code before test
- Test after implementation
- Test passes immediately
- Can't explain why test failed
- Tests added "later"
- Rationalizing "just this once"
- "I already manually tested it"
- "Tests after achieve the same purpose"
- "It's about spirit not ritual"
- "Keep as reference" or "adapt existing code"
- "Already spent X hours, deleting is wasteful"
- "TDD is dogmatic, I'm being pragmatic"
- "This is different because..."
**All of these mean: Delete code. Start over with TDD.**
## Example: Bug Fix
**Bug:** Empty email accepted
**RED**
```typescript
test('rejects empty email', async () => {
const result = await submitForm({ email: '' });
expect(result.error).toBe('Email required');
});
```
**Verify RED**
```bash
$ npm test
FAIL: expected 'Email required', got undefined
```
**GREEN**
```typescript
function submitForm(data: FormData) {
if (!data.email?.trim()) {
return { error: 'Email required' };
}
// ...
}
```
**Verify GREEN**
```bash
$ npm test
PASS
```
**REFACTOR**
Extract validation for multiple fields if needed.
## Verification Checklist
Before marking work complete:
- [ ] Every new function/method has a test
- [ ] Watched each test fail before implementing
- [ ] Each test failed for expected reason (feature missing, not typo)
- [ ] Wrote minimal code to pass each test
- [ ] All tests pass
- [ ] Output pristine (no errors, warnings)
- [ ] Tests use real code (mocks only if unavoidable)
- [ ] Edge cases and errors covered
Can't check all boxes? You skipped TDD. Start over.
## When Stuck
| Problem | Solution |
|---------|----------|
| Don't know how to test | Write wished-for API. Write assertion first. Ask your human partner. |
| Test too complicated | Design too complicated. Simplify interface. |
| Must mock everything | Code too coupled. Use dependency injection. |
| Test setup huge | Extract helpers. Still complex? Simplify design. |
## Debugging Integration
Bug found? Write failing test reproducing it. Follow TDD cycle. Test proves fix and prevents regression.
Never fix bugs without a test.
## Testing Anti-Patterns
When adding mocks or test utilities, read @testing-anti-patterns.md to avoid common pitfalls:
- Testing mock behavior instead of real behavior
- Adding test-only methods to production classes
- Mocking without understanding dependencies
## Final Rule
```
Production code → test exists and failed first
Otherwise → not TDD
```
No exceptions without your human partner's permission.
@@ -1,299 +0,0 @@
# Testing Anti-Patterns
**Load this reference when:** writing or changing tests, adding mocks, or tempted to add test-only methods to production code.
## Overview
Tests must verify real behavior, not mock behavior. Mocks are a means to isolate, not the thing being tested.
**Core principle:** Test what the code does, not what the mocks do.
**Following strict TDD prevents these anti-patterns.**
## The Iron Laws
```
1. NEVER test mock behavior
2. NEVER add test-only methods to production classes
3. NEVER mock without understanding dependencies
```
## Anti-Pattern 1: Testing Mock Behavior
**The violation:**
```typescript
// ❌ BAD: Testing that the mock exists
test('renders sidebar', () => {
render(<Page />);
expect(screen.getByTestId('sidebar-mock')).toBeInTheDocument();
});
```
**Why this is wrong:**
- You're verifying the mock works, not that the component works
- Test passes when mock is present, fails when it's not
- Tells you nothing about real behavior
**your human partner's correction:** "Are we testing the behavior of a mock?"
**The fix:**
```typescript
// ✅ GOOD: Test real component or don't mock it
test('renders sidebar', () => {
render(<Page />); // Don't mock sidebar
expect(screen.getByRole('navigation')).toBeInTheDocument();
});
// OR if sidebar must be mocked for isolation:
// Don't assert on the mock - test Page's behavior with sidebar present
```
### Gate Function
```
BEFORE asserting on any mock element:
Ask: "Am I testing real component behavior or just mock existence?"
IF testing mock existence:
STOP - Delete the assertion or unmock the component
Test real behavior instead
```
## Anti-Pattern 2: Test-Only Methods in Production
**The violation:**
```typescript
// ❌ BAD: destroy() only used in tests
class Session {
async destroy() { // Looks like production API!
await this._workspaceManager?.destroyWorkspace(this.id);
// ... cleanup
}
}
// In tests
afterEach(() => session.destroy());
```
**Why this is wrong:**
- Production class polluted with test-only code
- Dangerous if accidentally called in production
- Violates YAGNI and separation of concerns
- Confuses object lifecycle with entity lifecycle
**The fix:**
```typescript
// ✅ GOOD: Test utilities handle test cleanup
// Session has no destroy() - it's stateless in production
// In test-utils/
export async function cleanupSession(session: Session) {
const workspace = session.getWorkspaceInfo();
if (workspace) {
await workspaceManager.destroyWorkspace(workspace.id);
}
}
// In tests
afterEach(() => cleanupSession(session));
```
### Gate Function
```
BEFORE adding any method to production class:
Ask: "Is this only used by tests?"
IF yes:
STOP - Don't add it
Put it in test utilities instead
Ask: "Does this class own this resource's lifecycle?"
IF no:
STOP - Wrong class for this method
```
## Anti-Pattern 3: Mocking Without Understanding
**The violation:**
```typescript
// ❌ BAD: Mock breaks test logic
test('detects duplicate server', () => {
// Mock prevents config write that test depends on!
vi.mock('ToolCatalog', () => ({
discoverAndCacheTools: vi.fn().mockResolvedValue(undefined)
}));
await addServer(config);
await addServer(config); // Should throw - but won't!
});
```
**Why this is wrong:**
- Mocked method had side effect test depended on (writing config)
- Over-mocking to "be safe" breaks actual behavior
- Test passes for wrong reason or fails mysteriously
**The fix:**
```typescript
// ✅ GOOD: Mock at correct level
test('detects duplicate server', () => {
// Mock the slow part, preserve behavior test needs
vi.mock('MCPServerManager'); // Just mock slow server startup
await addServer(config); // Config written
await addServer(config); // Duplicate detected ✓
});
```
### Gate Function
```
BEFORE mocking any method:
STOP - Don't mock yet
1. Ask: "What side effects does the real method have?"
2. Ask: "Does this test depend on any of those side effects?"
3. Ask: "Do I fully understand what this test needs?"
IF depends on side effects:
Mock at lower level (the actual slow/external operation)
OR use test doubles that preserve necessary behavior
NOT the high-level method the test depends on
IF unsure what test depends on:
Run test with real implementation FIRST
Observe what actually needs to happen
THEN add minimal mocking at the right level
Red flags:
- "I'll mock this to be safe"
- "This might be slow, better mock it"
- Mocking without understanding the dependency chain
```
## Anti-Pattern 4: Incomplete Mocks
**The violation:**
```typescript
// ❌ BAD: Partial mock - only fields you think you need
const mockResponse = {
status: 'success',
data: { userId: '123', name: 'Alice' }
// Missing: metadata that downstream code uses
};
// Later: breaks when code accesses response.metadata.requestId
```
**Why this is wrong:**
- **Partial mocks hide structural assumptions** - You only mocked fields you know about
- **Downstream code may depend on fields you didn't include** - Silent failures
- **Tests pass but integration fails** - Mock incomplete, real API complete
- **False confidence** - Test proves nothing about real behavior
**The Iron Rule:** Mock the COMPLETE data structure as it exists in reality, not just fields your immediate test uses.
**The fix:**
```typescript
// ✅ GOOD: Mirror real API completeness
const mockResponse = {
status: 'success',
data: { userId: '123', name: 'Alice' },
metadata: { requestId: 'req-789', timestamp: 1234567890 }
// All fields real API returns
};
```
### Gate Function
```
BEFORE creating mock responses:
Check: "What fields does the real API response contain?"
Actions:
1. Examine actual API response from docs/examples
2. Include ALL fields system might consume downstream
3. Verify mock matches real response schema completely
Critical:
If you're creating a mock, you must understand the ENTIRE structure
Partial mocks fail silently when code depends on omitted fields
If uncertain: Include all documented fields
```
## Anti-Pattern 5: Integration Tests as Afterthought
**The violation:**
```
✅ Implementation complete
❌ No tests written
"Ready for testing"
```
**Why this is wrong:**
- Testing is part of implementation, not optional follow-up
- TDD would have caught this
- Can't claim complete without tests
**The fix:**
```
TDD cycle:
1. Write failing test
2. Implement to pass
3. Refactor
4. THEN claim complete
```
## When Mocks Become Too Complex
**Warning signs:**
- Mock setup longer than test logic
- Mocking everything to make test pass
- Mocks missing methods real components have
- Test breaks when mock changes
**your human partner's question:** "Do we need to be using a mock here?"
**Consider:** Integration tests with real components often simpler than complex mocks
## TDD Prevents These Anti-Patterns
**Why TDD helps:**
1. **Write test first** → Forces you to think about what you're actually testing
2. **Watch it fail** → Confirms test tests real behavior, not mocks
3. **Minimal implementation** → No test-only methods creep in
4. **Real dependencies** → You see what the test actually needs before mocking
**If you're testing mock behavior, you violated TDD** - you added mocks without watching test fail against real code first.
## Quick Reference
| Anti-Pattern | Fix |
|--------------|-----|
| Assert on mock elements | Test real component or unmock it |
| Test-only methods in production | Move to test utilities |
| Mock without understanding | Understand dependencies first, mock minimally |
| Incomplete mocks | Mirror real API completely |
| Tests as afterthought | TDD - tests first |
| Over-complex mocks | Consider integration tests |
## Red Flags
- Assertion checks for `*-mock` test IDs
- Methods only called in test files
- Mock setup is >50% of test
- Test fails when you remove mock
- Can't explain why mock is needed
- Mocking "just to be safe"
## The Bottom Line
**Mocks are tools to isolate, not things to test.**
If TDD reveals you're testing mock behavior, you've gone wrong.
Fix: Test real behavior or question why you're mocking at all.
+14 -4
View File
@@ -22,18 +22,28 @@ Screenshot a locally-running web UI without installing a browser on the agent's
## Usage ## Usage
`shot.sh` ships beside this SKILL.md, so resolve it from wherever the harness
installed the skill rather than from a checkout path — an absolute `~/dev/...`
went stale the first time the repo moved and cost a caller three hand-rolled
Job attempts before they found the working script (#26).
```bash ```bash
# Claude Code installs the plugin; Crush/Pi/OpenCode get ~/.agents/skills.
SHOT=$(ls ~/.claude/plugins/marketplaces/*/skills/web-shot/shot.sh \
~/.agents/skills/web-shot/shot.sh 2>/dev/null | head -1 || true)
[ -n "$SHOT" ] || { echo "web-shot not installed on this host: run install.sh or the plugin install" >&2; exit 1; }
# auto-detect Tailscale hostname, screenshot port 8099 root # auto-detect Tailscale hostname, screenshot port 8099 root
~/dev/.skills/web-shot/shot.sh --port 8099 "$SHOT" --port 8099
# explicit URL # explicit URL
~/dev/.skills/web-shot/shot.sh --url http://flamingo:5173 "$SHOT" --url http://flamingo:5173
# capture a search state too (types into input[name=q]) # capture a search state too (types into input[name=q])
~/dev/.skills/web-shot/shot.sh --port 8181 --search agentsquad "$SHOT" --port 8181 --search agentsquad
# different output dir # different output dir
~/dev/.skills/web-shot/shot.sh --port 3000 --out /tmp/shots "$SHOT" --port 3000 --out /tmp/shots
``` ```
## Flags ## Flags
+70
View File
@@ -0,0 +1,70 @@
#!/bin/sh
# Tests for bin/check-index.
#
# The regression these exist for: SKILLS_INDEX.md is maintained by hand, and
# two skills (`dream`, `experiment-spec`) shipped without a row — the index
# rows lived on feature branches that were squash-merged after a directory
# reorg, so the SKILL.md landed and the row did not (#15, #22). Nothing
# errored; the index just quietly under-reported the library.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
SUT="$ROOT/bin/check-index"
pass=0
fail=0
WORK=${TMPDIR:-/tmp}/check_index_test.$$
if [ ! -f "$SUT" ]; then
printf 'FAIL: %s does not exist\n' "$SUT"
exit 1
fi
cleanup() { rm -rf "$WORK"; }
trap cleanup EXIT INT TERM
# new_fixture <name> — a checkout with skills/alpha and skills/beta.
new_fixture() {
_f="$WORK/$1"
mkdir -p "$_f/skills/alpha" "$_f/skills/beta"
printf -- '---\nname: alpha\ndescription: a\n---\n' > "$_f/skills/alpha/SKILL.md"
printf -- '---\nname: beta\ndescription: b\n---\n' > "$_f/skills/beta/SKILL.md"
printf '%s' "$_f"
}
index() {
_dir=$1; shift
printf '# Engineering Skills Index\n\n| Skill | Description | Use when |\n|---|---|---|\n' > "$_dir/SKILLS_INDEX.md"
for _n in "$@"; do
printf '| `%s` | desc | when |\n' "$_n" >> "$_dir/SKILLS_INDEX.md"
done
}
# expect <0|1> <label> <fixture> [grep-for]
expect() {
_want=$1; _label=$2; _fx=$3; _needle=${4:-}
_out=$(sh "$SUT" "$_fx" 2>&1); _rc=$?
if [ "$_rc" -ne "$_want" ]; then
fail=$((fail + 1)); printf ' FAIL %s (want rc=%s got %s)\n%s\n' "$_label" "$_want" "$_rc" "$_out"; return
fi
if [ -n "$_needle" ] && ! printf '%s' "$_out" | grep -q "$_needle"; then
fail=$((fail + 1)); printf ' FAIL %s (rc ok, but report never names %s)\n%s\n' "$_label" "$_needle" "$_out"; return
fi
pass=$((pass + 1)); printf ' ok %s\n' "$_label"
}
fx=$(new_fixture complete); index "$fx" alpha beta
expect 0 'every skill has a row' "$fx"
fx=$(new_fixture missing); index "$fx" alpha
expect 1 'a skill with no row is reported' "$fx" beta
fx=$(new_fixture stale); index "$fx" alpha beta gamma
expect 1 'a row for a skill that does not exist is reported' "$fx" gamma
fx=$(new_fixture noindex)
expect 1 'a missing index is reported' "$fx" SKILLS_INDEX.md
printf '\ncheck_index: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+57
View File
@@ -0,0 +1,57 @@
#!/bin/sh
# Tests for the CI gate workflow.
#
# The regression these exist for: this repo had a release workflow that
# auto-tagged every push to main, and a `task check` / `task test` suite that
# nothing ever ran. CI reported green having tested nothing — a gate that
# fails *open*. These assertions pin the wiring so it cannot silently regress.
set -u
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
SUT="$ROOT/.gitea/workflows/ci.yml"
pass=0
fail=0
ok() { pass=$((pass + 1)); printf ' ok %s\n' "$1"; }
no() { fail=$((fail + 1)); printf ' FAIL %s\n' "$1"; }
if [ ! -f "$SUT" ]; then
printf 'FAIL: %s does not exist\n' "$SUT"
exit 1
fi
ok 'ci.yml exists'
# A pull_request trigger is what gives tbd_ship a CI run to read on the head
# commit of a branch; without one it fails closed and never merges (#22).
grep -qE '^ pull_request:' "$SUT" \
&& ok 'runs on pull_request' \
|| no 'runs on pull_request'
grep -qE '^ +push:' "$SUT" \
&& ok 'runs on push' \
|| no 'runs on push'
grep -qE '^ +- *(run: *)?task check$|task check' "$SUT" \
&& ok 'gate runs task check' \
|| no 'gate runs task check'
grep -qE 'task test' "$SUT" \
&& ok 'gate runs task test' \
|| no 'gate runs task test'
# The point of the gate: tagging must not happen on a red check.
tag_job=$(awk '/^ tag:/{f=1} f&&/^ [a-z][a-z_-]*:/&&!/^ tag:/{f=0} f' "$SUT")
printf '%s' "$tag_job" | grep -qE '^ +needs: *gate' \
&& ok 'tag job needs the gate' \
|| no 'tag job needs the gate'
# The old workflow file must be gone, or two workflows race on the same push
# and the ungated one still tags.
[ ! -f "$ROOT/.gitea/workflows/release.yml" ] \
&& ok 'ungated release.yml removed' \
|| no 'ungated release.yml removed'
printf '\nci_gate: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]
+11
View File
@@ -61,6 +61,17 @@ expect deny 'op read "op://vault/item/field"' 'op read'
expect deny 'env | grep -i gitea' 'env piped to grep' expect deny 'env | grep -i gitea' 'env piped to grep'
expect deny 'env' 'bare env' expect deny 'env' 'bare env'
expect deny 'printenv' 'printenv' expect deny 'printenv' 'printenv'
expect deny 'env -i | grep -i token' 'env -i with no command is still a dump'
expect deny 'env FOO=bar' 'env with only assignments and no command is a dump'
# --- allow: `env VAR=x cmd` / `env -u VAR cmd` RUN a command (#25) ---
# These print nothing themselves; a pipe into grep filters the command's own
# output, not the environment. Blocking them cost a documented workaround
# ((unset VAR; cmd)) on the 1Password write path, which is the shape below.
expect allow 'env -u ANTHROPIC_API_KEY claude --debug -p "ok" 2>&1 | grep -i plugin' 'env -u VAR runs a command'
expect allow 'env FOO=bar ./script.sh | grep x' 'env VAR=x runs a command'
expect allow 'env -u OP_SERVICE_ACCOUNT_TOKEN op item create - < item.json' 'env -u on the 1P write path'
expect deny 'env -u FOO ./run.sh; env | grep -i secret' 'a real dump later in the same line still denies'
# --- deny: kubernetes secret material --- # --- deny: kubernetes secret material ---
expect deny 'kubectl get secret brain-app -o yaml' 'kubectl get secret -o yaml' expect deny 'kubectl get secret brain-app -o yaml' 'kubectl get secret -o yaml'