mathias
6838baa071
Found from the other end: the koala cluster's CIHeadUncovered{repo=skills}
alert had been firing for 85 hours, and it is a true positive.
mathias/skills main HEAD f031c2b
workflow runs for that sha 0
Not a one-off. The previous release-bump commit has the same property:
6cc224f chore(release): plugin.json version 0.19.4 -> 0 runs
f031c2b chore(release): plugin.json version 0.19.5 -> 0 runs
The `tag` job pushes the manifest bump itself with secrets.GITEA_TOKEN, and a
push authenticated with the Actions token does not trigger workflows -- that is
deliberate platform behaviour to stop CI recursing. The `Skip-Release: true`
trailer stops the TAGGER recursing and works; nothing stopped the whole run,
`gate` included, from never firing on that commit.
So after every release, main's HEAD is a commit no CI has seen, until an
unrelated human push moves it. CIHeadUncovered therefore cannot clear on its
own -- and an alert that can only be cleared by doing something unrelated is
worse than a noisy one, because it teaches the reader that red is the normal
state of that rule.
The commit is not cosmetic either. plugin.json's version is what Claude Code
gates plugin updates on, `task check` asserts it is not behind the latest tag,
and the sed-and-retry logic that writes it already shipped a mismatched
manifest once -- v0.10.0 tagged with the manifest still reading 0.9.0, per the
comment above that very loop.
Fix: run `task check` on the bump commit before pushing it. The commit is
produced in that step, so it can be validated in that step -- one check instead
of a second full CI run, and no recursion risk to reintroduce.
Considered and rejected: re-triggering CI with a different credential after the
push (costs a full run per release and brings back what the token behaviour
exists to prevent), and teaching CIHeadUncovered to ignore manifest-only
commits (narrowing the measure to fit the system, and it would hide a real gap
too).
The assertion in tests/ci_gate_test.sh had to be anchored with ^[^#]*: the tag
job's own prose says "`task check` asserts the manifest is not behind the
latest tag", so a bare grep matched the comment explaining the problem and
reported the problem fixed.
Closes #46.
84 lines
3.2 KiB
Bash
84 lines
3.2 KiB
Bash
#!/bin/sh
|
|
# Tests for the CI gate workflow.
|
|
#
|
|
# The regression these exist for: this repo had a release workflow that
|
|
# auto-tagged every push to main, and a `task check` / `task test` suite that
|
|
# nothing ever ran. CI reported green having tested nothing — a gate that
|
|
# fails *open*. These assertions pin the wiring so it cannot silently regress.
|
|
|
|
set -u
|
|
|
|
ROOT=$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)
|
|
SUT="$ROOT/.gitea/workflows/ci.yml"
|
|
|
|
pass=0
|
|
fail=0
|
|
|
|
ok() { pass=$((pass + 1)); printf ' ok %s\n' "$1"; }
|
|
no() { fail=$((fail + 1)); printf ' FAIL %s\n' "$1"; }
|
|
|
|
if [ ! -f "$SUT" ]; then
|
|
printf 'FAIL: %s does not exist\n' "$SUT"
|
|
exit 1
|
|
fi
|
|
ok 'ci.yml exists'
|
|
|
|
# A pull_request trigger is what gives tbd_ship a CI run to read on the head
|
|
# commit of a branch; without one it fails closed and never merges (#22).
|
|
grep -qE '^ pull_request:' "$SUT" \
|
|
&& ok 'runs on pull_request' \
|
|
|| no 'runs on pull_request'
|
|
|
|
grep -qE '^ +push:' "$SUT" \
|
|
&& ok 'runs on push' \
|
|
|| no 'runs on push'
|
|
|
|
grep -qE '^ +- *(run: *)?task check$|task check' "$SUT" \
|
|
&& ok 'gate runs task check' \
|
|
|| no 'gate runs task check'
|
|
|
|
grep -qE 'task test' "$SUT" \
|
|
&& ok 'gate runs task test' \
|
|
|| no 'gate runs task test'
|
|
|
|
# The point of the gate: tagging must not happen on a red check.
|
|
tag_job=$(awk '/^ tag:/{f=1} f&&/^ [a-z][a-z_-]*:/&&!/^ tag:/{f=0} f' "$SUT")
|
|
printf '%s' "$tag_job" | grep -qE '^ +needs: *gate' \
|
|
&& ok 'tag job needs the gate' \
|
|
|| no 'tag job needs the gate'
|
|
|
|
# The old workflow file must be gone, or two workflows race on the same push
|
|
# and the ungated one still tags.
|
|
[ ! -f "$ROOT/.gitea/workflows/release.yml" ] \
|
|
&& ok 'ungated release.yml removed' \
|
|
|| no 'ungated release.yml removed'
|
|
|
|
# skills#46. The manifest-bump commit is created and pushed by the `tag` job
|
|
# using secrets.GITEA_TOKEN, and a push authenticated with the Actions token
|
|
# does not trigger workflows -- deliberate platform behaviour to stop CI
|
|
# recursing. So the ONE commit in this repo that no CI run has ever seen is the
|
|
# one the release job writes, every release:
|
|
#
|
|
# 6cc224f chore(release): plugin.json version 0.19.4 -> 0 runs
|
|
# f031c2b chore(release): plugin.json version 0.19.5 -> 0 runs
|
|
#
|
|
# The koala cluster's CIHeadUncovered{repo=skills} alert had been firing on it
|
|
# for 85 hours when this was found, and it cannot clear on its own -- only an
|
|
# unrelated human push moves main's HEAD to a tested commit.
|
|
#
|
|
# The commit is not nothing: plugin.json's version is what Claude Code gates
|
|
# plugin updates on, `task check` asserts it is not behind the latest tag, and
|
|
# the sed-and-retry logic that writes it already shipped a mismatched manifest
|
|
# once (v0.10.0 with the manifest reading 0.9.0).
|
|
#
|
|
# Anchored past comments: the tag job's prose already SAYS "`task check`
|
|
# asserts the manifest is not behind the latest tag", and a bare grep matches
|
|
# that and passes. A grep for a concept matches the rationale, and the more
|
|
# carefully a thing is documented the more likely that is.
|
|
printf '%s' "$tag_job" | grep -qE '^[^#]*task check' \
|
|
&& ok 'the release-bump commit is gated before it is pushed' \
|
|
|| no 'the release-bump commit is pushed with no gate -- nothing ever tests it (#46)'
|
|
|
|
printf '\nci_gate: %d passed, %d failed\n' "$pass" "$fail"
|
|
[ "$fail" -eq 0 ]
|