From b9bbbae3bbfa69c8a253c3602734117b5cf6bfdc Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 17 Sep 2026 16:29:29 +0200 Subject: [PATCH] fix(ci): make the template guard survive its own substitution Closes #9. Closes #10. Every child repo born from this template has had fake-green CI: cd.yml's guard grepped a literal placeholder, and both scaffolders substitute every file in the tree -- including the guard. In a generated repo the check became `grep -q 'git.d-ma.be/mathias/' go.mod` against a go.mod whose first line is `module git.d-ma.be/mathias/`, so it matched forever, check/build/deploy all skipped, and the run reported success having built nothing. Confirmed in cad-atlas (#9) and openbanking-api (#10). The second occurrence landed directly underneath a comment warning against exactly this. That is the tell: an inline shell block in a workflow cannot be tested, so a fix for it can only be reviewed by reading, and reading is what missed it twice. So the detection moves to scripts/is-template.sh and gets a test. It greps the SHAPE of a placeholder rather than any literal token, which also catches leftovers the scaffolders know nothing about, and a missing target now exits non-zero instead of answering "false" -- failing open there would skip CI on a repo that is not a template at all, which is the same fake-green failure by another road. scripts/tests/is-template_test.sh runs a scaffolder-style substitution over a copy of the guard and asserts it still answers correctly. Both shipped bugs fail that test. The test assembles placeholder tokens at runtime rather than writing them whole, for the same reason the guard does: a fixture containing a literal token is substituted before the test runs, and the suite would then fail in every generated repo while passing here. The first draft of the guard was itself caught by this -- its explanatory comment named a token literally. Verified by simulating a generated child (whole tree substituted): child go.mod module git.d-ma.be/mathias/childproj guard child -> false (CI runs) guard template -> true (template still skips its own build) suite in child -> 8 passed, 0 failed suite in template -> 8 passed, 0 failed Wired into `task check`, so the guard that decides whether CI runs at all in every child is covered by the same gate as the Go tests. Co-Authored-By: Claude Opus 5 --- .gitea/workflows/cd.yml | 20 +++--- Taskfile.yml | 3 + scripts/is-template.sh | 47 +++++++++++++ scripts/tests/is-template_test.sh | 105 ++++++++++++++++++++++++++++++ 4 files changed, 166 insertions(+), 9 deletions(-) create mode 100755 scripts/is-template.sh create mode 100755 scripts/tests/is-template_test.sh diff --git a/.gitea/workflows/cd.yml b/.gitea/workflows/cd.yml index 783748d..36e35b4 100644 --- a/.gitea/workflows/cd.yml +++ b/.gitea/workflows/cd.yml @@ -19,16 +19,18 @@ jobs: steps: - uses: actions/checkout@v4 - id: detect + # The detection lives in scripts/is-template.sh so it can be TESTED. + # It was inline here twice and wrong both times (#9, #10): the grep + # named a placeholder literally, and the scaffolder substitutes every + # file including this one, so in a generated repo the guard matched + # forever and check/build/deploy all skipped while the run went green. + # + # An inline shell block in a workflow cannot be run by a test, which is + # why the second fix repeated the first bug under a comment warning + # about it. scripts/tests/is-template_test.sh now substitutes the guard + # and asserts it still answers correctly. run: | - # Grep the PLACEHOLDER, never the value substitution writes. A guard - # that greps its own substituted module path matches forever and - # skips every job below it -- see harg-bygglov-go#8, where that cost - # five weeks of empty green runs on a repo that never compiled. - if grep -q '__MODULE_PATH__' go.mod; then - echo "is_template=true" >> "$GITHUB_OUTPUT" - else - echo "is_template=false" >> "$GITHUB_OUTPUT" - fi + echo "is_template=$(bash scripts/is-template.sh go.mod)" >> "$GITHUB_OUTPUT" check: name: Lint / Test / Vet diff --git a/Taskfile.yml b/Taskfile.yml index f211ea3..585f059 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -24,6 +24,9 @@ tasks: - golangci-lint run ./... - go vet ./... - go test ./... -race -count=1 + # The template guard decides whether CI runs at all in every child repo, + # so its regression test belongs in the same gate as the Go tests. + - bash scripts/tests/is-template_test.sh context:sync: desc: Regenerate all harness-specific context files diff --git a/scripts/is-template.sh b/scripts/is-template.sh new file mode 100755 index 0000000..b3c1022 --- /dev/null +++ b/scripts/is-template.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# Is this checkout still a raw, unsubstituted template? +# +# Prints "true" or "false". cd.yml's guard job reads it and skips check/build/ +# deploy when true, so that a template repo does not try to build itself. +# +# WHY A PATTERN AND NOT A LITERAL +# +# The scaffolders (gitea-mcp create_project_from_template, local-dev +# scripts/new-project.sh) substitute placeholders in EVERY file of the tree — +# including this one. So a guard that greps a whole placeholder token is +# rewritten by the very substitution it exists to detect: in the template it +# greps the module-path placeholder, and in the generated repo that same line +# has become a grep for the real module path — which the generated go.mod +# declares on its first line. It matches forever, every job below skips, and the +# run reports green having built nothing. +# +# That shipped twice: #9 (cad-atlas) and #10 (openbanking-api), the second one +# sitting directly under a comment warning against exactly this. +# +# Note this comment names no placeholder literally, for the same reason. A +# comment that gets substituted stops explaining the thing it documents — and +# scripts/tests/is-template_test.sh enforces that, which is how the first draft +# of this file was caught. +# +# The regex below matches the SHAPE of a placeholder and contains no complete +# token, so there is nothing for a scaffolder to rewrite. It also catches any +# other leftover placeholder, not just the two the scaffolders know about. +# +# scripts/tests/is-template_test.sh substitutes a copy of this file and asserts +# it still answers correctly — that case is the regression test for the class. +set -euo pipefail + +target="${1:-go.mod}" + +if [ ! -f "$target" ]; then + # Never answer "false" here. A missing target would skip CI on a repo that is + # not a template at all, which is the same fake-green failure by another road. + echo "is-template: no such file: $target" >&2 + exit 2 +fi + +if grep -qE '__[A-Z][A-Z0-9_]*__' "$target"; then + echo "true" +else + echo "false" +fi diff --git a/scripts/tests/is-template_test.sh b/scripts/tests/is-template_test.sh new file mode 100755 index 0000000..85d1c04 --- /dev/null +++ b/scripts/tests/is-template_test.sh @@ -0,0 +1,105 @@ +#!/usr/bin/env bash +# Tests for scripts/is-template.sh +# +# This guard decides whether a checkout is still a raw template. Every job in +# cd.yml keys off it, so when it answers wrongly the entire pipeline skips and +# reports green — which is how `cad-atlas` and `openbanking-api` were born with +# CI that had never built, tested or deployed them (#9, #10). +# +# The defect both times was the same: the guard grepped for a literal +# placeholder, and the scaffolder substitutes placeholders in EVERY file, +# including the guard itself. So in a generated repo the check became +# `grep -q 'git.d-ma.be/mathias/' go.mod` against a go.mod that reads +# `module git.d-ma.be/mathias/` — a match that can never fail. +# +# The third test below is the one that matters: it runs a scaffolder-style +# substitution OVER THE GUARD and asserts it still answers correctly. Both +# shipped bugs would have failed it. +# +# Usage: ./scripts/tests/is-template_test.sh + +set -uo pipefail +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SCRIPT="$ROOT/scripts/is-template.sh" + +pass=0; fail=0 +ok() { printf ' ok %s\n' "$1"; pass=$((pass+1)); } +bad() { printf ' FAIL %s\n %s\n' "$1" "${2:-}"; fail=$((fail+1)); } + +tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT +echo "is-template tests" + +# Placeholder tokens are ASSEMBLED, never written whole. The scaffolder rewrites +# every file in the tree, this one included -- a fixture containing a literal +# token would be substituted before the test ran, and the suite would fail in +# every generated repo while passing here. Which is the same class of bug the +# guard itself had. +U="__" +ph() { printf '%s%s%s' "$U" "$1" "$U"; } +MODULE_PH="$(ph MODULE_PATH)" +PROJECT_PH="$(ph PROJECT_NAME)" +OTHER_PH="$(ph SOME_OTHER)" + +if [ -f "$SCRIPT" ]; then ok "script exists" +else bad "script exists" "$SCRIPT missing"; echo; echo "$pass passed, $fail failed"; exit 1; fi + +# A raw template still carries placeholders. +printf 'module git.d-ma.be/mathias/%s\n\ngo 1.26\n' "$MODULE_PH" > "$tmp/raw.go.mod" +got="$(bash "$SCRIPT" "$tmp/raw.go.mod")" +[ "$got" = "true" ] && ok "raw template is detected" \ + || bad "raw template is detected" "got '$got', want 'true'" + +# A generated repo does not. This is the assertion both shipped guards failed. +printf 'module git.d-ma.be/mathias/openbanking-api\n\ngo 1.26\n' > "$tmp/real.go.mod" +got="$(bash "$SCRIPT" "$tmp/real.go.mod")" +[ "$got" = "false" ] && ok "a generated repo is NOT detected as a template" \ + || bad "a generated repo is NOT detected as a template" "got '$got', want 'false'" + +# THE ONE THAT MATTERS. +# +# Copy the guard and run a scaffolder-style substitution over it, exactly as +# create_project_from_template and new-project.sh do to every file in the tree. +# The substituted guard must still answer correctly. A guard that greps a whole +# placeholder token gets rewritten here and starts matching forever. +cp "$SCRIPT" "$tmp/substituted.sh" +sed -i \ + -e "s|$MODULE_PH|git.d-ma.be/mathias/openbanking-api|g" \ + -e "s|$PROJECT_PH|openbanking-api|g" \ + "$tmp/substituted.sh" + +got="$(bash "$tmp/substituted.sh" "$tmp/real.go.mod")" +if [ "$got" = "false" ]; then + ok "the guard survives being substituted itself" +else + bad "the guard survives being substituted itself" \ + "after substitution it answered '$got' for a generated repo — this is #9/#10" +fi + +got="$(bash "$tmp/substituted.sh" "$tmp/raw.go.mod")" +[ "$got" = "true" ] && ok "the substituted guard still detects a raw template" \ + || bad "the substituted guard still detects a raw template" "got '$got'" + +# Belt and braces: the source must not contain a complete substitutable token. +if grep -qE "$(printf '%s(MODULE_PATH|PROJECT_NAME)%s' "$U" "$U")" "$SCRIPT"; then + bad "the guard contains no substitutable placeholder token" \ + "found one in $SCRIPT — the scaffolder will rewrite it" +else + ok "the guard contains no substitutable placeholder token" +fi + +# Any leftover placeholder counts, not just the two the scaffolder knows about — +# a half-substituted tree is not a working repo either. +printf 'module git.d-ma.be/mathias/thing\n\n// %s\ngo 1.26\n' "$OTHER_PH" > "$tmp/partial.go.mod" +got="$(bash "$SCRIPT" "$tmp/partial.go.mod")" +[ "$got" = "true" ] && ok "any leftover placeholder counts" \ + || bad "any leftover placeholder counts" "got '$got', want 'true'" + +# A missing file is not "this is a real repo" — failing open here would skip CI. +bash "$SCRIPT" "$tmp/nope.go.mod" >/dev/null 2>&1 +rc=$? +[ "$rc" -ne 0 ] && ok "a missing target fails loudly rather than answering false" \ + || bad "a missing target fails loudly rather than answering false" "exit 0" + +echo +echo "$pass passed, $fail failed" +[ "$fail" -eq 0 ] -- 2.54.0