Every release leaves main's HEAD with zero CI runs, so CIHeadUncovered fires forever and can never clear #46

Open
opened 2026-09-11 08:37:40 +00:00 by mathias · 1 comment
Owner

Found 2026-09-11 from the other end: CIHeadUncovered{repo=skills} has been firing in the koala cluster for 85 hours, and it is a true positive.

The measurement

mathias/skills main HEAD   f031c2b77dd237083d121595e46189e97de2708f
workflow runs for that sha  0

Latest run of any kind is 1328dcd (2026-09-08 19:35, success). f031c2b is newer than that and has never been tested.

It is not a one-off. The previous release-bump commit has the same property:

6cc224f  chore(release): plugin.json version 0.19.4   →  0 runs
f031c2b  chore(release): plugin.json version 0.19.5   →  0 runs

Why

.gitea/workflows/ci.yml's tag job pushes the manifest bump itself:

- name: tag + push
  env:
    GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
  run: |
    git commit -m "chore(release): plugin.json version $version" -m "Skip-Release: true"
    git push origin HEAD:main

A push authenticated with the Actions token does not trigger workflows — that is deliberate platform behaviour to stop CI recursing. The Skip-Release: true trailer exists to stop the tagger recursing, and it works, but it is solving a different problem: the whole run, gate included, never fires on that commit.

So the sequence every release produces is:

  1. human pushes a real commit → gate runs, green → tag job tags it
  2. tag job pushes chore(release): ... to main → nothing runs
  3. main's HEAD is now a commit no CI has ever seen, until the next human push

Why it matters more than one untested one-line commit

The alert can never clear by itself. CIHeadUncovered asks "does main's HEAD have a CI run". After a release the answer is permanently no, and no amount of waiting changes it — only an unrelated human push does. An alert that cannot clear is worse than a noisy one: it teaches the reader that red is the normal state of that rule, which is exactly what the infra-side alert work (infra#355, infra#364) exists to stop. 85 hours unacted is that lesson being learned.

And the commit is not nothing. .claude-plugin/plugin.json's version is what Claude Code gates plugin updates on, and task check asserts the manifest is not behind the latest tag. The commit that moves it is the one commit in the repo that no gate ever sees. The for attempt in 1 2 3 retry loop with sed-based version rewriting is real logic running unverified — and the comment above it records that a previous version of this already shipped v0.10.0 with the manifest still reading 0.9.0.

Options

  1. Don't commit the bump at all. Derive the plugin version from the tag at read time, or have task check compare tag-to-manifest only when the manifest is authored by hand. Removes the untestable commit rather than testing it. Cleanest if the manifest can stop being a committed duplicate of the tag.
  2. Run the gate inside the tag job before pushing. The bump commit is produced there, so it can be validated there — task check after the sed, before git push. Cheap, and it covers the only thing that commit changes.
  3. workflow_dispatch after the push, re-triggering CI on the new HEAD with a different credential. Works, costs a second full run per release, and reintroduces the recursion risk the token behaviour exists to prevent.
  4. Teach CIHeadUncovered to ignore commits whose only change is the manifest. Rejected — that is narrowing the measure to fit the system, and it would also hide a genuine gap.

Recommend 2 as the immediate fix (smallest, testable, keeps the current release shape), with 1 as the better long-term answer if the manifest version can be derived rather than stored.

Done when

  • A release-bump commit either does not exist, or has a CI run attached.
  • CIHeadUncovered{repo=skills} clears without a human pushing something unrelated.
  • Whatever validates the bump covers the sed rewrite and the retry loop, since those are what previously shipped a mismatched manifest.

Refs infra#355, infra#364, infra#366 (the alert), skills#22 (CI that reported green having tested nothing — same family).


Created via git-mcp on behalf of @mathias

Found 2026-09-11 from the other end: `CIHeadUncovered{repo=skills}` has been firing in the koala cluster for **85 hours**, and it is a true positive. ## The measurement ``` mathias/skills main HEAD f031c2b77dd237083d121595e46189e97de2708f workflow runs for that sha 0 ``` Latest run of any kind is `1328dcd` (2026-09-08 19:35, success). `f031c2b` is *newer* than that and has never been tested. It is not a one-off. The previous release-bump commit has the same property: ``` 6cc224f chore(release): plugin.json version 0.19.4 → 0 runs f031c2b chore(release): plugin.json version 0.19.5 → 0 runs ``` ## Why `.gitea/workflows/ci.yml`'s `tag` job pushes the manifest bump itself: ```yaml - name: tag + push env: GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} run: | git commit -m "chore(release): plugin.json version $version" -m "Skip-Release: true" git push origin HEAD:main ``` A push authenticated with the Actions token **does not trigger workflows** — that is deliberate platform behaviour to stop CI recursing. The `Skip-Release: true` trailer exists to stop the *tagger* recursing, and it works, but it is solving a different problem: the whole run, `gate` included, never fires on that commit. So the sequence every release produces is: 1. human pushes a real commit → `gate` runs, green → `tag` job tags it 2. `tag` job pushes `chore(release): ...` to main → **nothing runs** 3. main's HEAD is now a commit no CI has ever seen, until the next human push ## Why it matters more than one untested one-line commit **The alert can never clear by itself.** `CIHeadUncovered` asks "does main's HEAD have a CI run". After a release the answer is permanently no, and no amount of waiting changes it — only an unrelated human push does. An alert that cannot clear is worse than a noisy one: it teaches the reader that red is the normal state of that rule, which is exactly what the infra-side alert work (infra#355, infra#364) exists to stop. 85 hours unacted is that lesson being learned. **And the commit is not nothing.** `.claude-plugin/plugin.json`'s version is what Claude Code gates plugin updates on, and `task check` asserts the manifest is not behind the latest tag. The commit that moves it is the one commit in the repo that no gate ever sees. The `for attempt in 1 2 3` retry loop with `sed`-based version rewriting is real logic running unverified — and the comment above it records that a previous version of this already shipped `v0.10.0` with the manifest still reading `0.9.0`. ## Options 1. **Don't commit the bump at all.** Derive the plugin version from the tag at read time, or have `task check` compare tag-to-manifest only when the manifest is authored by hand. Removes the untestable commit rather than testing it. Cleanest if the manifest can stop being a committed duplicate of the tag. 2. **Run the gate inside the `tag` job before pushing.** The bump commit is produced there, so it can be validated there — `task check` after the `sed`, before `git push`. Cheap, and it covers the only thing that commit changes. 3. **`workflow_dispatch` after the push**, re-triggering CI on the new HEAD with a different credential. Works, costs a second full run per release, and reintroduces the recursion risk the token behaviour exists to prevent. 4. **Teach `CIHeadUncovered` to ignore commits whose only change is the manifest.** Rejected — that is narrowing the measure to fit the system, and it would also hide a genuine gap. Recommend **2** as the immediate fix (smallest, testable, keeps the current release shape), with **1** as the better long-term answer if the manifest version can be derived rather than stored. ## Done when - A release-bump commit either does not exist, or has a CI run attached. - `CIHeadUncovered{repo=skills}` clears without a human pushing something unrelated. - Whatever validates the bump covers the `sed` rewrite and the retry loop, since those are what previously shipped a mismatched manifest. Refs infra#355, infra#364, infra#366 (the alert), skills#22 (CI that reported green having tested nothing — same family). --- _Created via git-mcp on behalf of @mathias_
mathias reopened this issue 2026-09-11 09:11:48 +00:00
Author
Owner

Reopened. 6838baa implements option 2 and my commit message said "Closes #46", which auto-closed it. That was wrong — option 2 satisfies one of the three "Done when" criteria, not all three.

What 6838baa actually achieves

task check now runs on the bump commit, in the tag job, after the sed and before git push origin HEAD:main. So the third criterion is met:

Whatever validates the bump covers the sed rewrite and the retry loop, since those are what previously shipped a mismatched manifest.

It does, and it fails the job rather than pushing an unvalidated commit.

What it does not achieve, and I should have caught before committing

CIHeadUncovered{repo=skills} clears without a human pushing something unrelated.

It will not. The alert asks whether main's HEAD has a workflow run — ci-red-monitor.sh queries the runs API by head_sha. Validating the commit's content inside a different run does not attach a run to that sha. The bump commit will still show zero runs, and the alert will still fire after every release.

So option 2 makes the content tested, which was the substantive risk, while leaving the signal wrong. That is a meaningful improvement and a worse-than-useless place to stop: an alert that is now firing about a commit that was in fact tested is a false positive, and false positives are how the channel gets ignored — the exact thing this issue is about.

Which means option 4 deserves re-examination

I rejected it in the body:

Teach CIHeadUncovered to ignore commits whose only change is the manifest. Rejected — that is narrowing the measure to fit the system, and it would also hide a genuine gap.

With option 2 in place that rejection is weaker than it was. The gap is no longer genuine: the commit is gated, just not by a run of its own. The honest objection is different now — the monitor would be trusting a claim it cannot verify, which is the same shape as a CI comment asserting it mirrors the local gate.

The option I now think is right

Option 1, properly: stop producing the commit at all. Have the human bump .claude-plugin/plugin.json in their own commit, and have task check assert it matches the version the tagger will apply. Then:

  • there is no untested commit, because there is no extra commit;
  • CIHeadUncovered is satisfied without being taught an exception;
  • the manifest is authored by whoever is making the release, which is where that decision belongs;
  • the sed-and-retry loop — the code that already shipped a mismatched manifest once — disappears rather than being tested.

The cost is real: releasing gains a manual step, and getting it wrong means a red task check instead of a silent self-correction. That is a trade worth making deliberately, and it is a change to how you release, so it is yours rather than mine.

Leaving this open for that. What is shipped is strictly an improvement on what was there; the alert will keep firing until the commit stops existing or the monitor learns about it.


Created via git-mcp on behalf of @mathias

Reopened. `6838baa` implements option 2 and my commit message said "Closes #46", which auto-closed it. **That was wrong — option 2 satisfies one of the three "Done when" criteria, not all three.** ## What `6838baa` actually achieves `task check` now runs on the bump commit, in the `tag` job, after the `sed` and before `git push origin HEAD:main`. So the third criterion is met: > Whatever validates the bump covers the `sed` rewrite and the retry loop, since those are what previously shipped a mismatched manifest. It does, and it fails the job rather than pushing an unvalidated commit. ## What it does not achieve, and I should have caught before committing > `CIHeadUncovered{repo=skills}` clears without a human pushing something unrelated. **It will not.** The alert asks whether main's HEAD has a workflow **run** — `ci-red-monitor.sh` queries the runs API by `head_sha`. Validating the commit's *content* inside a different run does not attach a run to that sha. The bump commit will still show zero runs, and the alert will still fire after every release. So option 2 makes the content tested, which was the substantive risk, while leaving the signal wrong. That is a meaningful improvement and a worse-than-useless place to stop: an alert that is now firing about a commit that *was* in fact tested is a false positive, and false positives are how the channel gets ignored — the exact thing this issue is about. ## Which means option 4 deserves re-examination I rejected it in the body: > **Teach `CIHeadUncovered` to ignore commits whose only change is the manifest.** Rejected — that is narrowing the measure to fit the system, and it would also hide a genuine gap. With option 2 in place that rejection is weaker than it was. The gap is no longer genuine: the commit *is* gated, just not by a run of its own. The honest objection is different now — the monitor would be trusting a claim it cannot verify, which is the same shape as a CI comment asserting it mirrors the local gate. ## The option I now think is right **Option 1, properly: stop producing the commit at all.** Have the human bump `.claude-plugin/plugin.json` in their own commit, and have `task check` assert it matches the version the tagger *will* apply. Then: - there is no untested commit, because there is no extra commit; - `CIHeadUncovered` is satisfied without being taught an exception; - the manifest is authored by whoever is making the release, which is where that decision belongs; - the `sed`-and-retry loop — the code that already shipped a mismatched manifest once — disappears rather than being tested. The cost is real: releasing gains a manual step, and getting it wrong means a red `task check` instead of a silent self-correction. That is a trade worth making deliberately, and it is a change to how you release, so it is yours rather than mine. Leaving this open for that. What is shipped is strictly an improvement on what was there; the alert will keep firing until the commit stops existing or the monitor learns about it. --- _Created via git-mcp on behalf of @mathias_
mathias added the risk/mediumsize/stype/fix labels 2026-09-16 22:21:53 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/skills#46