Template
fix(ci): make the template guard survive its own substitution
CD / Detect unsubstituted template (pull_request) Successful in 1s
CD / Lint / Test / Vet (pull_request) Skipped
CD / Build & Import (pull_request) Skipped
CD / Deploy via GitOps (pull_request) Skipped
CD / Detect unsubstituted template (push) Successful in 1s
CD / Lint / Test / Vet (push) Skipped
CD / Build & Import (push) Skipped
CD / Deploy via GitOps (push) Skipped
CD / Detect unsubstituted template (pull_request) Successful in 1s
CD / Lint / Test / Vet (pull_request) Skipped
CD / Build & Import (pull_request) Skipped
CD / Deploy via GitOps (pull_request) Skipped
CD / Detect unsubstituted template (push) Successful in 1s
CD / Lint / Test / Vet (push) Skipped
CD / Build & Import (push) Skipped
CD / Deploy via GitOps (push) Skipped
Closes #9. Closes #10. Every child repo born from this template has had fake-green CI: cd.yml's guard grepped a literal placeholder, and both scaffolders substitute every file in the tree -- including the guard. In a generated repo the check became `grep -q 'git.d-ma.be/mathias/<name>' go.mod` against a go.mod whose first line is `module git.d-ma.be/mathias/<name>`, so it matched forever, check/build/deploy all skipped, and the run reported success having built nothing. Confirmed in cad-atlas (#9) and openbanking-api (#10). The second occurrence landed directly underneath a comment warning against exactly this. That is the tell: an inline shell block in a workflow cannot be tested, so a fix for it can only be reviewed by reading, and reading is what missed it twice. So the detection moves to scripts/is-template.sh and gets a test. It greps the SHAPE of a placeholder rather than any literal token, which also catches leftovers the scaffolders know nothing about, and a missing target now exits non-zero instead of answering "false" -- failing open there would skip CI on a repo that is not a template at all, which is the same fake-green failure by another road. scripts/tests/is-template_test.sh runs a scaffolder-style substitution over a copy of the guard and asserts it still answers correctly. Both shipped bugs fail that test. The test assembles placeholder tokens at runtime rather than writing them whole, for the same reason the guard does: a fixture containing a literal token is substituted before the test runs, and the suite would then fail in every generated repo while passing here. The first draft of the guard was itself caught by this -- its explanatory comment named a token literally. Verified by simulating a generated child (whole tree substituted): child go.mod module git.d-ma.be/mathias/childproj guard child -> false (CI runs) guard template -> true (template still skips its own build) suite in child -> 8 passed, 0 failed suite in template -> 8 passed, 0 failed Wired into `task check`, so the guard that decides whether CI runs at all in every child is covered by the same gate as the Go tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit was merged in pull request #11.
This commit is contained in:
Executable
+105
@@ -0,0 +1,105 @@
|
||||
#!/usr/bin/env bash
|
||||
# Tests for scripts/is-template.sh
|
||||
#
|
||||
# This guard decides whether a checkout is still a raw template. Every job in
|
||||
# cd.yml keys off it, so when it answers wrongly the entire pipeline skips and
|
||||
# reports green — which is how `cad-atlas` and `openbanking-api` were born with
|
||||
# CI that had never built, tested or deployed them (#9, #10).
|
||||
#
|
||||
# The defect both times was the same: the guard grepped for a literal
|
||||
# placeholder, and the scaffolder substitutes placeholders in EVERY file,
|
||||
# including the guard itself. So in a generated repo the check became
|
||||
# `grep -q 'git.d-ma.be/mathias/<name>' go.mod` against a go.mod that reads
|
||||
# `module git.d-ma.be/mathias/<name>` — a match that can never fail.
|
||||
#
|
||||
# The third test below is the one that matters: it runs a scaffolder-style
|
||||
# substitution OVER THE GUARD and asserts it still answers correctly. Both
|
||||
# shipped bugs would have failed it.
|
||||
#
|
||||
# Usage: ./scripts/tests/is-template_test.sh
|
||||
|
||||
set -uo pipefail
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SCRIPT="$ROOT/scripts/is-template.sh"
|
||||
|
||||
pass=0; fail=0
|
||||
ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); }
|
||||
bad() { printf ' FAIL %s\n %s\n' "$1" "${2:-}"; fail=$((fail+1)); }
|
||||
|
||||
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
|
||||
echo "is-template tests"
|
||||
|
||||
# Placeholder tokens are ASSEMBLED, never written whole. The scaffolder rewrites
|
||||
# every file in the tree, this one included -- a fixture containing a literal
|
||||
# token would be substituted before the test ran, and the suite would fail in
|
||||
# every generated repo while passing here. Which is the same class of bug the
|
||||
# guard itself had.
|
||||
U="__"
|
||||
ph() { printf '%s%s%s' "$U" "$1" "$U"; }
|
||||
MODULE_PH="$(ph MODULE_PATH)"
|
||||
PROJECT_PH="$(ph PROJECT_NAME)"
|
||||
OTHER_PH="$(ph SOME_OTHER)"
|
||||
|
||||
if [ -f "$SCRIPT" ]; then ok "script exists"
|
||||
else bad "script exists" "$SCRIPT missing"; echo; echo "$pass passed, $fail failed"; exit 1; fi
|
||||
|
||||
# A raw template still carries placeholders.
|
||||
printf 'module git.d-ma.be/mathias/%s\n\ngo 1.26\n' "$MODULE_PH" > "$tmp/raw.go.mod"
|
||||
got="$(bash "$SCRIPT" "$tmp/raw.go.mod")"
|
||||
[ "$got" = "true" ] && ok "raw template is detected" \
|
||||
|| bad "raw template is detected" "got '$got', want 'true'"
|
||||
|
||||
# A generated repo does not. This is the assertion both shipped guards failed.
|
||||
printf 'module git.d-ma.be/mathias/openbanking-api\n\ngo 1.26\n' > "$tmp/real.go.mod"
|
||||
got="$(bash "$SCRIPT" "$tmp/real.go.mod")"
|
||||
[ "$got" = "false" ] && ok "a generated repo is NOT detected as a template" \
|
||||
|| bad "a generated repo is NOT detected as a template" "got '$got', want 'false'"
|
||||
|
||||
# THE ONE THAT MATTERS.
|
||||
#
|
||||
# Copy the guard and run a scaffolder-style substitution over it, exactly as
|
||||
# create_project_from_template and new-project.sh do to every file in the tree.
|
||||
# The substituted guard must still answer correctly. A guard that greps a whole
|
||||
# placeholder token gets rewritten here and starts matching forever.
|
||||
cp "$SCRIPT" "$tmp/substituted.sh"
|
||||
sed -i \
|
||||
-e "s|$MODULE_PH|git.d-ma.be/mathias/openbanking-api|g" \
|
||||
-e "s|$PROJECT_PH|openbanking-api|g" \
|
||||
"$tmp/substituted.sh"
|
||||
|
||||
got="$(bash "$tmp/substituted.sh" "$tmp/real.go.mod")"
|
||||
if [ "$got" = "false" ]; then
|
||||
ok "the guard survives being substituted itself"
|
||||
else
|
||||
bad "the guard survives being substituted itself" \
|
||||
"after substitution it answered '$got' for a generated repo — this is #9/#10"
|
||||
fi
|
||||
|
||||
got="$(bash "$tmp/substituted.sh" "$tmp/raw.go.mod")"
|
||||
[ "$got" = "true" ] && ok "the substituted guard still detects a raw template" \
|
||||
|| bad "the substituted guard still detects a raw template" "got '$got'"
|
||||
|
||||
# Belt and braces: the source must not contain a complete substitutable token.
|
||||
if grep -qE "$(printf '%s(MODULE_PATH|PROJECT_NAME)%s' "$U" "$U")" "$SCRIPT"; then
|
||||
bad "the guard contains no substitutable placeholder token" \
|
||||
"found one in $SCRIPT — the scaffolder will rewrite it"
|
||||
else
|
||||
ok "the guard contains no substitutable placeholder token"
|
||||
fi
|
||||
|
||||
# Any leftover placeholder counts, not just the two the scaffolder knows about —
|
||||
# a half-substituted tree is not a working repo either.
|
||||
printf 'module git.d-ma.be/mathias/thing\n\n// %s\ngo 1.26\n' "$OTHER_PH" > "$tmp/partial.go.mod"
|
||||
got="$(bash "$SCRIPT" "$tmp/partial.go.mod")"
|
||||
[ "$got" = "true" ] && ok "any leftover placeholder counts" \
|
||||
|| bad "any leftover placeholder counts" "got '$got', want 'true'"
|
||||
|
||||
# A missing file is not "this is a real repo" — failing open here would skip CI.
|
||||
bash "$SCRIPT" "$tmp/nope.go.mod" >/dev/null 2>&1
|
||||
rc=$?
|
||||
[ "$rc" -ne 0 ] && ok "a missing target fails loudly rather than answering false" \
|
||||
|| bad "a missing target fails loudly rather than answering false" "exit 0"
|
||||
|
||||
echo
|
||||
echo "$pass passed, $fail failed"
|
||||
[ "$fail" -eq 0 ]
|
||||
Reference in New Issue
Block a user