mathias
39ac61c411
Ports the technique #11 asked for and ships the script #19 asks for: was this tried, what killed it, and is the killer still alive. The RED run did excellent archaeology unaided — fourteen steps, correct verdict, and it split the proposal into a goal worth keeping and a mechanism that had failed, which is the finding. What it never ran was `git log -S`, and what it leaned on hardest was the issue tracker. Both facts shaped the skill. The GREEN run then broke the tool's central claim, live. Asked about a feature this repo removed and argued about at length, `lineage.sh "per repo"` printed: no trace of "per repo" — that is a real finding: it means this is new here Flatly wrong; the term simply had a space in it. Worse, the skill's own anti-pattern table forbids that exact inference — "assuming absence means it was never considered" — so the tool was printing, in its own voice, the mistake the skill warns against. A no-match now says it is one string against one repo, and names the variants to try before anyone concludes novelty. The test asserts the refusal rather than the old message. Three more from the same run: - The tool searched `docs/adr/*` for decision records. This repo has no `docs/`, and this estate keeps decisions in the brain and in gitea comments — so that section silently searched nothing four times, and its null was indistinguishable from a real one. It now names both unsearchable sources on every run, and the skill makes `brain_query` a step: the single best artifact in the GREEN run came from the brain and nowhere else. - Step 1 steered readers toward path-scoped log and away from `-S`. Backwards. A function deleted from a file that still exists is the dominant removal shape, and `-S<identifier>` returns birth and death on one screen while the path search buries it. Rewritten to lead with the identifier. - "Which conditions still hold" had no method, and the run answered it by running commands against the live host in twenty seconds. Now a table: one command per claim, because "the record says X" and "X is true right now" are different answers and only the second is worth giving. Also fixed: the tool defaulted to the cwd and would return a confident null against the wrong repo in a worktree-heavy setup. It now prints which repo it searched, every run. Closes #11 Bump-Type: minor
91 lines
3.5 KiB
Bash
91 lines
3.5 KiB
Bash
#!/bin/sh
|
|
# Tests for skills/lineage/lineage.sh — the decision-archaeology helper.
|
|
#
|
|
# The techniques this automates are the ones people skip because they are
|
|
# fiddly to type: `git log -S` to find when a thing was introduced AND removed,
|
|
# --diff-filter=D to find files that once existed, and grepping decision records
|
|
# for the term. A skill that only *describes* those is a skill that gets read and
|
|
# not run (#11, #19's "small robust tools over prose").
|
|
|
|
set -u
|
|
|
|
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
|
SUT="$ROOT/skills/lineage/lineage.sh"
|
|
|
|
pass=0
|
|
fail=0
|
|
WORK=${TMPDIR:-/tmp}/lineage_test.$$
|
|
|
|
if [ ! -f "$SUT" ]; then
|
|
printf 'FAIL: %s does not exist\n' "$SUT"
|
|
exit 1
|
|
fi
|
|
|
|
cleanup() { rm -rf "$WORK"; }
|
|
trap cleanup EXIT INT TERM
|
|
|
|
# A repo where a thing was built, written about, and then removed.
|
|
mkdir -p "$WORK/repo/docs/adr"
|
|
cd "$WORK/repo" || exit 1
|
|
git init -q .
|
|
git config user.email t@example.com
|
|
git config user.name test
|
|
|
|
printf 'func WidgetCache() {}\n' > cache.go
|
|
git add -A && git commit -qm "feat: add WidgetCache for lookup speed"
|
|
|
|
printf -- '---\nadr: 0001\ntitle: use a WidgetCache\n---\nWe chose WidgetCache over a database index.\n' > docs/adr/0001-widgetcache.md
|
|
git add -A && git commit -qm "docs: record the WidgetCache decision"
|
|
|
|
git rm -q cache.go
|
|
git commit -qm "revert: drop WidgetCache, it went stale under concurrent writes"
|
|
|
|
cd - >/dev/null || exit 1
|
|
out=$(sh "$SUT" WidgetCache "$WORK/repo" 2>&1)
|
|
|
|
check() {
|
|
if printf '%s' "$out" | grep -q "$1"; then
|
|
pass=$((pass + 1)); printf ' ok %s\n' "$2"
|
|
else
|
|
fail=$((fail + 1)); printf ' FAIL %s\n output had no %s\n%s\n' "$2" "$1" "$out"
|
|
fi
|
|
}
|
|
|
|
check 'add WidgetCache' 'finds the commit that introduced it'
|
|
check 'drop WidgetCache' 'finds the commit that removed it — the revival question'
|
|
check '0001-widgetcache.md' 'finds the decision record naming it'
|
|
check 'cache.go' 'names the file that no longer exists'
|
|
|
|
# A no-match must NOT be reported as evidence the idea is new. The first version
|
|
# of this tool said "it means this is new here" and a pressure test caught it
|
|
# asserting exactly that about the most-discussed removed feature in this repo —
|
|
# the search term simply had a space in it. That is the inference the skill's own
|
|
# anti-pattern table forbids, printed in the tool's voice.
|
|
out=$(sh "$SUT" NeverExisted "$WORK/repo" 2>&1)
|
|
if printf '%s' "$out" | grep -q 'NOT evidence the idea is new' \
|
|
&& printf '%s' "$out" | grep -qi 'identifier'; then
|
|
pass=$((pass + 1)); printf ' ok a no-match refuses to imply novelty and suggests other terms\n'
|
|
else
|
|
fail=$((fail + 1)); printf ' FAIL a no-match refuses to imply novelty and suggests other terms\n%s\n' "$out"
|
|
fi
|
|
|
|
# The repo actually searched is always stated — a confident null against the
|
|
# wrong tree is this tool's worst failure mode, and the default is the cwd.
|
|
out=$(sh "$SUT" WidgetCache "$WORK/repo" 2>&1)
|
|
if printf '%s' "$out" | grep -q "searching: "; then
|
|
pass=$((pass + 1)); printf ' ok names the repo it searched\n'
|
|
else
|
|
fail=$((fail + 1)); printf ' FAIL names the repo it searched\n%s\n' "$out"
|
|
fi
|
|
|
|
# The sources it cannot reach are named every run, so their absence never reads
|
|
# as a null result.
|
|
if printf '%s' "$out" | grep -q 'brain_query' && printf '%s' "$out" | grep -qi 'closed issues'; then
|
|
pass=$((pass + 1)); printf ' ok names the sources it cannot search\n'
|
|
else
|
|
fail=$((fail + 1)); printf ' FAIL names the sources it cannot search\n%s\n' "$out"
|
|
fi
|
|
|
|
printf '\nlineage: %d passed, %d failed\n' "$pass" "$fail"
|
|
[ "$fail" -eq 0 ]
|