On 2026-08-25 a deploy job ran with an empty IMAGE_TAG, wrote
`image: localhost:5000/gitea-mcp:` into the infra repo and pushed it.
That is invalid YAML, so Flux failed the ENTIRE apps Kustomization --
every app stopped reconciling for about an hour, including three
unrelated deploys queued behind it.
This job's existing check could not catch it. It greps for
`image: <repo>:${IMAGE_TAG}` after the sed, and with IMAGE_TAG empty that
pattern matches the corrupted line itself, so the verification passes
because of the bug rather than despite it.
Asserting the input up front is the fix: an empty tag now fails the job
before anything is written, let alone committed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011PL5XcG68SvQqz3vGVbWbf
Canonical gained two inputs so the repos still to migrate can adopt
without forking the script again: TARGET for multi-stage builds (cad,
agentsquad) and BUILD_SECRETS for buildkit secret mounts (agentsquad).
BUILD_SECRETS takes the SPEC only (`id=NAME,src=PATH`) -- the caller
writes the secret to PATH itself, so no secret value ever reaches this
script's argv or the CI log.
Unused here; this repo's copy just stays byte-identical, which is the
whole point. Written by `task ci:sync`, not by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011PL5XcG68SvQqz3vGVbWbf
A template is the highest-leverage copy of anything. This one carried its
own inline build step, so every generated repo was born with a private
fork of the build logic -- which is how rootless buildah reached a dozen
repos before infra#241, and why the fix then had to be rediscovered here
weeks after the app repos were done.
The step now calls .gitea/build-image.sh, a byte-identical copy of
mathias/infra's ci-templates/build-image.sh. Generated repos inherit the
script and the call; adding the new repo to infra's adopters.txt puts it
under the drift gate.
Invoked as `bash .gitea/build-image.sh` rather than relying on the exec
bit: files written through the Gitea API land 0644, and a template whose
CI depends on a file mode is a trap for every repo generated from it.
Byte-identical copy of mathias/infra's ci-templates/build-image.sh. A
template that carries its own copy of the build LOGIC manufactures a new
copy of every bug in it -- which is exactly how rootless buildah reached
a dozen repos (infra#241). Now it manufactures a copy of one file that a
drift gate keeps current instead.
Rootless buildah in gitea-runner's hardened securityContext (capabilities:
drop [ALL], allowPrivilegeEscalation: false) cannot remap the UID/GID range
newuidmap/newgidmap need on a fresh base-image pull, and the pod has no OCI
runtime at all. buildctl talks to the host's buildkitd over a socket, so the
privileged layer extraction happens on the host and the pod's securityContext
stops being in the picture. Same pattern as mathias/infra's segment-embedder.yml
and postgres18.yml.
A template is the highest-leverage place to fix this. Every repo generated from
here inherits its CI, so a buildah-based template does not just carry the bug —
it manufactures new instances of it. A fleet inventory for infra#241 found nine
repos still invoking buildah, and two of them are the templates.
Note the failure is latent, not immediate: warm base-image layers mask it, so a
generated repo builds fine until the first genuinely cold pull. That is exactly
the situation a NEW repo is in.
Verified by running the exact command against this repo's real build context:
buildctl --addr unix:///run/buildkit/buildkitd.sock build \
--frontend dockerfile.v0 --local context=. --local dockerfile=. \
--output type=oci,dest=$TAR
-> exporting manifest sha256:5a841529... DONE, 6.8MB OCI archive
That check matters here because this repo's own CI cannot validate the change:
the build job is deliberately skipped on the raw template (2ba2698), since
__PROJECT_NAME__ is not a legal image name. The next generated repo is the first
place it runs for real.
skopeo copy replaces `buildah push` — buildctl has no registry-push equivalent —
and runs twice to preserve the :sha and :latest tags the previous version pushed.
env.IMAGE: __PROJECT_NAME__ is only substituted with a real, lowercase
project name when a repo is *generated* from this template. On the
template repo itself it's still the literal placeholder, which
buildah/Docker reject outright ("repository name must be lowercase") --
build (and therefore deploy) can never succeed here no matter what else
changes. Gate both on github.repository != this template's own path so
the template's CI reads green on what it actually validates: the check
job (lint/vet/test), the thing every generated repo inherits.
mathiasandClaude Opus 5