fix(ci): make the template guard survive its own substitution
CD / Detect unsubstituted template (pull_request) Successful in 1s
CD / Lint / Test / Vet (pull_request) Skipped
CD / Build & Import (pull_request) Skipped
CD / Deploy via GitOps (pull_request) Skipped
CD / Detect unsubstituted template (push) Successful in 1s
CD / Lint / Test / Vet (push) Skipped
CD / Build & Import (push) Skipped
CD / Deploy via GitOps (push) Skipped

Closes #9. Closes #10.

Every child repo born from this template has had fake-green CI: cd.yml's
guard grepped a literal placeholder, and both scaffolders substitute every
file in the tree -- including the guard. In a generated repo the check
became `grep -q 'git.d-ma.be/mathias/<name>' go.mod` against a go.mod whose
first line is `module git.d-ma.be/mathias/<name>`, so it matched forever,
check/build/deploy all skipped, and the run reported success having built
nothing. Confirmed in cad-atlas (#9) and openbanking-api (#10).

The second occurrence landed directly underneath a comment warning against
exactly this. That is the tell: an inline shell block in a workflow cannot
be tested, so a fix for it can only be reviewed by reading, and reading is
what missed it twice.

So the detection moves to scripts/is-template.sh and gets a test.

It greps the SHAPE of a placeholder rather than any literal token, which
also catches leftovers the scaffolders know nothing about, and a missing
target now exits non-zero instead of answering "false" -- failing open
there would skip CI on a repo that is not a template at all, which is the
same fake-green failure by another road.

scripts/tests/is-template_test.sh runs a scaffolder-style substitution over
a copy of the guard and asserts it still answers correctly. Both shipped
bugs fail that test.

The test assembles placeholder tokens at runtime rather than writing them
whole, for the same reason the guard does: a fixture containing a literal
token is substituted before the test runs, and the suite would then fail in
every generated repo while passing here. The first draft of the guard was
itself caught by this -- its explanatory comment named a token literally.

Verified by simulating a generated child (whole tree substituted):

    child  go.mod module git.d-ma.be/mathias/childproj
    guard  child    -> false   (CI runs)
    guard  template -> true    (template still skips its own build)
    suite  in child -> 8 passed, 0 failed
    suite  in template -> 8 passed, 0 failed

Wired into `task check`, so the guard that decides whether CI runs at all
in every child is covered by the same gate as the Go tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit was merged in pull request #11.
This commit is contained in:
2026-09-17 16:29:29 +02:00
co-authored by Claude Opus 5
parent 7c70489efe
commit b9bbbae3bb
4 changed files with 166 additions and 9 deletions
+105
View File
@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# Tests for scripts/is-template.sh
#
# This guard decides whether a checkout is still a raw template. Every job in
# cd.yml keys off it, so when it answers wrongly the entire pipeline skips and
# reports green — which is how `cad-atlas` and `openbanking-api` were born with
# CI that had never built, tested or deployed them (#9, #10).
#
# The defect both times was the same: the guard grepped for a literal
# placeholder, and the scaffolder substitutes placeholders in EVERY file,
# including the guard itself. So in a generated repo the check became
# `grep -q 'git.d-ma.be/mathias/<name>' go.mod` against a go.mod that reads
# `module git.d-ma.be/mathias/<name>` — a match that can never fail.
#
# The third test below is the one that matters: it runs a scaffolder-style
# substitution OVER THE GUARD and asserts it still answers correctly. Both
# shipped bugs would have failed it.
#
# Usage: ./scripts/tests/is-template_test.sh
set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
SCRIPT="$ROOT/scripts/is-template.sh"
pass=0; fail=0
ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); }
bad() { printf ' FAIL %s\n %s\n' "$1" "${2:-}"; fail=$((fail+1)); }
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
echo "is-template tests"
# Placeholder tokens are ASSEMBLED, never written whole. The scaffolder rewrites
# every file in the tree, this one included -- a fixture containing a literal
# token would be substituted before the test ran, and the suite would fail in
# every generated repo while passing here. Which is the same class of bug the
# guard itself had.
U="__"
ph() { printf '%s%s%s' "$U" "$1" "$U"; }
MODULE_PH="$(ph MODULE_PATH)"
PROJECT_PH="$(ph PROJECT_NAME)"
OTHER_PH="$(ph SOME_OTHER)"
if [ -f "$SCRIPT" ]; then ok "script exists"
else bad "script exists" "$SCRIPT missing"; echo; echo "$pass passed, $fail failed"; exit 1; fi
# A raw template still carries placeholders.
printf 'module git.d-ma.be/mathias/%s\n\ngo 1.26\n' "$MODULE_PH" > "$tmp/raw.go.mod"
got="$(bash "$SCRIPT" "$tmp/raw.go.mod")"
[ "$got" = "true" ] && ok "raw template is detected" \
|| bad "raw template is detected" "got '$got', want 'true'"
# A generated repo does not. This is the assertion both shipped guards failed.
printf 'module git.d-ma.be/mathias/openbanking-api\n\ngo 1.26\n' > "$tmp/real.go.mod"
got="$(bash "$SCRIPT" "$tmp/real.go.mod")"
[ "$got" = "false" ] && ok "a generated repo is NOT detected as a template" \
|| bad "a generated repo is NOT detected as a template" "got '$got', want 'false'"
# THE ONE THAT MATTERS.
#
# Copy the guard and run a scaffolder-style substitution over it, exactly as
# create_project_from_template and new-project.sh do to every file in the tree.
# The substituted guard must still answer correctly. A guard that greps a whole
# placeholder token gets rewritten here and starts matching forever.
cp "$SCRIPT" "$tmp/substituted.sh"
sed -i \
-e "s|$MODULE_PH|git.d-ma.be/mathias/openbanking-api|g" \
-e "s|$PROJECT_PH|openbanking-api|g" \
"$tmp/substituted.sh"
got="$(bash "$tmp/substituted.sh" "$tmp/real.go.mod")"
if [ "$got" = "false" ]; then
ok "the guard survives being substituted itself"
else
bad "the guard survives being substituted itself" \
"after substitution it answered '$got' for a generated repo — this is #9/#10"
fi
got="$(bash "$tmp/substituted.sh" "$tmp/raw.go.mod")"
[ "$got" = "true" ] && ok "the substituted guard still detects a raw template" \
|| bad "the substituted guard still detects a raw template" "got '$got'"
# Belt and braces: the source must not contain a complete substitutable token.
if grep -qE "$(printf '%s(MODULE_PATH|PROJECT_NAME)%s' "$U" "$U")" "$SCRIPT"; then
bad "the guard contains no substitutable placeholder token" \
"found one in $SCRIPT — the scaffolder will rewrite it"
else
ok "the guard contains no substitutable placeholder token"
fi
# Any leftover placeholder counts, not just the two the scaffolder knows about —
# a half-substituted tree is not a working repo either.
printf 'module git.d-ma.be/mathias/thing\n\n// %s\ngo 1.26\n' "$OTHER_PH" > "$tmp/partial.go.mod"
got="$(bash "$SCRIPT" "$tmp/partial.go.mod")"
[ "$got" = "true" ] && ok "any leftover placeholder counts" \
|| bad "any leftover placeholder counts" "got '$got', want 'true'"
# A missing file is not "this is a real repo" — failing open here would skip CI.
bash "$SCRIPT" "$tmp/nope.go.mod" >/dev/null 2>&1
rc=$?
[ "$rc" -ne 0 ] && ok "a missing target fails loudly rather than answering false" \
|| bad "a missing target fails loudly rather than answering false" "exit 0"
echo
echo "$pass passed, $fail failed"
[ "$fail" -eq 0 ]