CRITICAL: CD guard self-defeats — every child repo skips all CI (fake green) #9

Closed
opened 2026-07-19 12:39:58 +00:00 by mathias · 2 comments
Owner

Severity: critical — silent, affects every generated repo

.gitea/workflows/cd.yml guard job:

- id: detect
  run: |
    if grep -q 'git.d-ma.be/mathias/__PROJECT_NAME__' go.mod; then
      echo "is_template=true" >> "$GITHUB_OUTPUT"
    ...

The grep pattern contains __PROJECT_NAME__, which the template generator substitutes at create time along with every other placeholder. So in a generated repo the guard becomes grep -q 'git.d-ma.be/mathias/<name>' go.mod, and go.mod is module git.d-ma.be/mathias/<name> → always matches → is_template=true → check, build, and deploy all have if: needs.guard.outputs.is_template != 'true' → all skip.

Net: every child repo's CI is fake-green — runs report success because all real jobs are skipped, nothing is ever built, tested, or deployed. Found in cad-atlas (runs 16–18 "green", but namespace/image/pod never existed).

Fix

Detect leftover placeholders by a pattern that is NOT itself substituted:

if grep -qE '__[A-Z_]+__' go.mod; then
  echo "is_template=true" >> "$GITHUB_OUTPUT"

Verified in cad-atlas: substituted repo → is_template=false (CI runs); raw __PROJECT_NAME__ go.mod → is_template=true (skips). Applied there as a stopgap; the template is the real fix so future children aren't born with rubber-stamp CI.

Sibling bugs from the same dogfooding pass: #8 (templ version skew + gitignore pattern).

## Severity: critical — silent, affects every generated repo `.gitea/workflows/cd.yml` guard job: ```yaml - id: detect run: | if grep -q 'git.d-ma.be/mathias/__PROJECT_NAME__' go.mod; then echo "is_template=true" >> "$GITHUB_OUTPUT" ... ``` The grep pattern contains `__PROJECT_NAME__`, which the template generator **substitutes** at create time along with every other placeholder. So in a generated repo the guard becomes `grep -q 'git.d-ma.be/mathias/<name>' go.mod`, and `go.mod` is `module git.d-ma.be/mathias/<name>` → **always matches** → `is_template=true` → `check`, `build`, and `deploy` all have `if: needs.guard.outputs.is_template != 'true'` → **all skip**. Net: every child repo's CI is **fake-green** — runs report success because all real jobs are skipped, nothing is ever built, tested, or deployed. Found in `cad-atlas` (runs 16–18 "green", but namespace/image/pod never existed). ## Fix Detect leftover placeholders by a pattern that is NOT itself substituted: ```yaml if grep -qE '__[A-Z_]+__' go.mod; then echo "is_template=true" >> "$GITHUB_OUTPUT" ``` Verified in cad-atlas: substituted repo → `is_template=false` (CI runs); raw `__PROJECT_NAME__` go.mod → `is_template=true` (skips). Applied there as a stopgap; the template is the real fix so future children aren't born with rubber-stamp CI. Sibling bugs from the same dogfooding pass: #8 (templ version skew + gitignore pattern).
Author
Owner

Re-verified 2026-08-27 during estate backlog triage: still live, and the guard as it stands today is broken in the same way under a different name.

Current .gitea/workflows/cd.yml:

if grep -q '__MODULE_PATH__' go.mod; then
  echo "is_template=true" >> "$GITHUB_OUTPUT"

__MODULE_PATH__ is a literal placeholder that the generator substitutes. local-dev:scripts/new-project.sh:161-165:

MODULE_PATH="git.d-ma.be/mathias/$NAME"
while IFS= read -r -d '' f; do
  sed_inplace "s|__PROJECT_NAME__|$NAME|g" "$f"
  sed_inplace "s|__MODULE_PATH__|$MODULE_PATH|g" "$f"
done < <(find "$PROJECT_DIR" -type f -print0)

find ... -type f includes .gitea/workflows/cd.yml, so in a generated repo the guard becomes grep -q 'git.d-ma.be/mathias/foo' go.mod against module git.d-ma.be/mathias/foo — always matches, is_template=true, and check / build / deploy all skip. Identical failure to the one reported, with __MODULE_PATH__ swapped for __PROJECT_NAME__.

The fix this issue actually specified — grep -qE '__[A-Z_]+__' — works precisely because it is a regex character class, not a literal string, so sed cannot substitute it. That fix is live in cad-atlas (the stopgap noted in the description) but was never applied to the template.

Blast radius, checked rather than assumed. Every repo currently carrying a clause-6 test gate was inspected for this guard; only cad-atlas has one, and it has the correct pattern. So no existing repo is fake-green from this. The exposure is entirely forward: the next repo scaffolded from this template is born with rubber-stamp CI.

Deliberately not parked. The scaffolding-ownership question (local-dev#21) has parked most of the surrounding cluster behind blocked/decision. This issue is excluded from that: a template that ships fake-green CI is a defect in the artifact regardless of which tool ends up owning scaffolding, and it silently defeats the CI-verification clause that unattended agent dispatch depends on.

Suggested minimal fix, unchanged from the description:

if grep -qE '__[A-Z_]+__' go.mod; then

plus a regression case asserting the guard still fires on a raw template and does not fire after substitution — the property that broke twice now.

Re-verified 2026-08-27 during estate backlog triage: **still live, and the guard as it stands today is broken in the same way under a different name.** Current `.gitea/workflows/cd.yml`: ```yaml if grep -q '__MODULE_PATH__' go.mod; then echo "is_template=true" >> "$GITHUB_OUTPUT" ``` `__MODULE_PATH__` is a literal placeholder that the generator substitutes. `local-dev:scripts/new-project.sh:161-165`: ```bash MODULE_PATH="git.d-ma.be/mathias/$NAME" while IFS= read -r -d '' f; do sed_inplace "s|__PROJECT_NAME__|$NAME|g" "$f" sed_inplace "s|__MODULE_PATH__|$MODULE_PATH|g" "$f" done < <(find "$PROJECT_DIR" -type f -print0) ``` `find ... -type f` includes `.gitea/workflows/cd.yml`, so in a generated repo the guard becomes `grep -q 'git.d-ma.be/mathias/foo' go.mod` against `module git.d-ma.be/mathias/foo` — always matches, `is_template=true`, and `check` / `build` / `deploy` all skip. Identical failure to the one reported, with `__MODULE_PATH__` swapped for `__PROJECT_NAME__`. The fix this issue actually specified — `grep -qE '__[A-Z_]+__'` — works precisely because it is a regex character class, not a literal string, so `sed` cannot substitute it. That fix is live in `cad-atlas` (the stopgap noted in the description) but was never applied to the template. **Blast radius, checked rather than assumed.** Every repo currently carrying a clause-6 test gate was inspected for this guard; only `cad-atlas` has one, and it has the correct pattern. So no existing repo is fake-green from this. The exposure is entirely forward: the next repo scaffolded from this template is born with rubber-stamp CI. **Deliberately not parked.** The scaffolding-ownership question (`local-dev#21`) has parked most of the surrounding cluster behind `blocked/decision`. This issue is excluded from that: a template that ships fake-green CI is a defect in the artifact regardless of which tool ends up owning scaffolding, and it silently defeats the CI-verification clause that unattended agent dispatch depends on. Suggested minimal fix, unchanged from the description: ```yaml if grep -qE '__[A-Z_]+__' go.mod; then ``` plus a regression case asserting the guard still fires on a raw template and does not fire after substitution — the property that broke twice now.
mathias added the risk/highsize/xstype/fix labels 2026-08-27 05:12:29 +00:00
mathias added the risk/medium label 2026-08-27 05:25:47 +00:00
Author
Owner

Fixed in b9bbbae (PR #11, v0.2.0), verified 2026-09-17. Auto-closed by the merge, so recording the evidence here.

Your proposed fix — grep a pattern the scaffolder cannot rewrite — is what shipped, with two changes.

It moved out of the workflow. The detection now lives in scripts/is-template.sh, because an inline run: block cannot be executed by a test. That is not incidental: #10 was the same bug landing underneath a comment warning against it, which is what happens when the only available review is reading.

It has a test that both shipped bugs fail. scripts/tests/is-template_test.sh runs a scaffolder-style substitution over a copy of the guard and asserts it still answers correctly. Wired into task check.

Verified by simulating a generated child — whole tree substituted, the way create_project_from_template does it — rather than by reading the diff:

child go.mod       module git.d-ma.be/mathias/childproj
guard in child     -> false        (CI runs)
guard in template  -> true         (template still skips its own build)
suite in child     -> 8 passed, 0 failed
suite in template  -> 8 passed, 0 failed

Both directions were checked deliberately: a fix that only satisfies the child would have made the template start building itself.

Two things worth knowing that came out of it:

  • The guard's first draft failed its own test — an explanatory comment named a placeholder literally, which the scaffolder would have rewritten, leaving the documentation explaining nothing.
  • The test had to be made immune to substitution too (tokens assembled at runtime). A fixture containing a literal token is rewritten before the test runs, so the suite would pass in the template and fail in every child — shipping a template whose children fail task check from birth, which is the same defect one file over.

Still open and deliberately not fixed here: this repo cannot test itself, because the guard skips check along with build/deploy. That is #12, and it is why this merged on local evidence rather than a CI verdict. Your note about auditing repos generated since the guard was introduced also still stands — this stops new children being born broken, it does not find the existing ones.


Created via git-mcp on behalf of @mathias

**Fixed in `b9bbbae` (PR #11, v0.2.0), verified 2026-09-17.** Auto-closed by the merge, so recording the evidence here. Your proposed fix — grep a pattern the scaffolder cannot rewrite — is what shipped, with two changes. **It moved out of the workflow.** The detection now lives in `scripts/is-template.sh`, because an inline `run:` block cannot be executed by a test. That is not incidental: #10 was the same bug landing *underneath a comment warning against it*, which is what happens when the only available review is reading. **It has a test that both shipped bugs fail.** `scripts/tests/is-template_test.sh` runs a scaffolder-style substitution over a copy of the guard and asserts it still answers correctly. Wired into `task check`. Verified by simulating a generated child — whole tree substituted, the way `create_project_from_template` does it — rather than by reading the diff: ``` child go.mod module git.d-ma.be/mathias/childproj guard in child -> false (CI runs) guard in template -> true (template still skips its own build) suite in child -> 8 passed, 0 failed suite in template -> 8 passed, 0 failed ``` Both directions were checked deliberately: a fix that only satisfies the child would have made the template start building itself. Two things worth knowing that came out of it: - The guard's **first draft failed its own test** — an explanatory comment named a placeholder literally, which the scaffolder would have rewritten, leaving the documentation explaining nothing. - The test had to be made immune to substitution too (tokens assembled at runtime). A fixture containing a literal token is rewritten before the test runs, so the suite would pass in the template and fail in every child — shipping a template whose children fail `task check` from birth, which is the same defect one file over. Still open and deliberately not fixed here: **this repo cannot test itself**, because the guard skips `check` along with `build`/`deploy`. That is #12, and it is why this merged on local evidence rather than a CI verdict. Your note about auditing repos generated since the guard was introduced also still stands — this stops new children being born broken, it does not find the existing ones. --- _Created via git-mcp on behalf of @mathias_
Sign in to join this conversation.