mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

cd.yml fails on a duplicate run: "nothing to commit" is treated as an error #101

New Issue
Closed
opened 2026-08-22 06:23:13 +00:00 by mathias · 1 comment
mathias commented 2026-08-22 06:23:13 +00:00
Owner
Copy Link
Copy Source

Seen merging #98 and #99 back to back on 2026-08-22.

What happened

Two merges landed within a couple of minutes. Each CI run's completion triggered cd.yml via workflow_run, and both CD runs resolved github.sha to the same branch head (b6ceccc). So:

  • Run 4098 built ingestion:b6ceccc, pushed it, bumped k3s/apps/supervisor/ingestion-deployment.yaml, committed, pushed. Deploy succeeded — pod is running that image, 0 restarts.
  • Run 4099 built and pushed the identical image, then found the manifest already at the right tag:
nothing to commit, working tree clean
❌  Failure - Main Update infra repo
🏁  Job failed

The deploy was correct and complete. CD still shows red on main.

Why it is worth fixing rather than shrugging at

A red CD run that means "the deploy was already done" is the most expensive kind of false alarm: it trains you to skim past CD failures, and the next red one will be real. It also cost me a log-dive during a routine merge to establish that production was fine.

The build work is duplicated too — a second full buildctl + skopeo copy of a byte-identical image.

Fix shape

In the "Update infra repo" step, treat an empty diff as success:

if git diff --quiet; then
  echo "manifest already at ${IMAGE_TAG} — nothing to do"
else
  git commit -am "chore(deploy): ingestion → ${IMAGE_TAG}"
  git push
fi

git commit --allow-empty would also exit 0, but it creates a pointless commit in the infra repo every time — worse than the check.

Optionally add concurrency control so the duplicate run does not do the build work at all:

concurrency:
  group: cd-${{ github.ref }}
  cancel-in-progress: true

Note cancel-in-progress: true cancels the first run mid-flight, which is wrong here — the first run is the one doing the real work. false (queue) is the safer choice, or leave concurrency alone and just make the commit step idempotent, which is the minimal fix.

Acceptance

  • caveman: robot not cry when work already done
  • Two merges in quick succession leave CD green
  • The deploy still happens exactly once, and the manifest still ends up at the newest SHA
  • Verified against a real back-to-back merge, not just reasoned about

Risk: LOW — CI-only change, no runtime effect.

Refs: runs 4098 (success) / 4099 (failure), .gitea/workflows/cd.yml.

Seen merging #98 and #99 back to back on 2026-08-22. ## What happened Two merges landed within a couple of minutes. Each CI run's completion triggered `cd.yml` via `workflow_run`, and both CD runs resolved `github.sha` to the same branch head (`b6ceccc`). So: - **Run 4098** built `ingestion:b6ceccc`, pushed it, bumped `k3s/apps/supervisor/ingestion-deployment.yaml`, committed, pushed. Deploy succeeded — pod is running that image, 0 restarts. - **Run 4099** built and pushed the identical image, then found the manifest already at the right tag: ``` nothing to commit, working tree clean ❌ Failure - Main Update infra repo 🏁 Job failed ``` The deploy was correct and complete. CD still shows red on `main`. ## Why it is worth fixing rather than shrugging at A red CD run that means "the deploy was already done" is the most expensive kind of false alarm: it trains you to skim past CD failures, and the next red one will be real. It also cost me a log-dive during a routine merge to establish that production was fine. The build work is duplicated too — a second full `buildctl` + `skopeo copy` of a byte-identical image. ## Fix shape In the "Update infra repo" step, treat an empty diff as success: ```sh if git diff --quiet; then echo "manifest already at ${IMAGE_TAG} — nothing to do" else git commit -am "chore(deploy): ingestion → ${IMAGE_TAG}" git push fi ``` `git commit --allow-empty` would also exit 0, but it creates a pointless commit in the infra repo every time — worse than the check. Optionally add concurrency control so the duplicate run does not do the build work at all: ```yaml concurrency: group: cd-${{ github.ref }} cancel-in-progress: true ``` Note `cancel-in-progress: true` cancels the *first* run mid-flight, which is wrong here — the first run is the one doing the real work. `false` (queue) is the safer choice, or leave concurrency alone and just make the commit step idempotent, which is the minimal fix. ## Acceptance - [ ] `caveman: robot not cry when work already done` - [ ] Two merges in quick succession leave CD green - [ ] The deploy still happens exactly once, and the manifest still ends up at the newest SHA - [ ] Verified against a real back-to-back merge, not just reasoned about **Risk:** LOW — CI-only change, no runtime effect. Refs: runs 4098 (success) / 4099 (failure), `.gitea/workflows/cd.yml`.
mathias added the risk/mediumsize/xstype/ops labels 2026-08-27 05:25:42 +00:00
mathias commented 2026-09-23 12:44:45 +00:00
Author
Owner
Copy Link
Copy Source

Duplicate of #118 (same defect, reported independently ~3 weeks apart), which was fixed and closed 2026-09-17 — this issue just never got linked/closed alongside it.

Verified current .gitea/workflows/cd.yml's "Update infra repo" step: git diff --cached --quiet idempotency check is in place exactly as this issue's fix-shape suggested, with a comment explicitly citing hyperguild#118 and explaining the two known no-op sources (manual re-run, and a since-removed tag-push CI run that used to shadow the branch-push one for the same sha).

Closing as resolved via #118. git commit --allow-empty was correctly avoided per this issue's own note.

🤖 Generated with Claude Code

Created via git-mcp on behalf of @mathias

Duplicate of #118 (same defect, reported independently ~3 weeks apart), which was fixed and closed 2026-09-17 — this issue just never got linked/closed alongside it. Verified current `.gitea/workflows/cd.yml`'s "Update infra repo" step: `git diff --cached --quiet` idempotency check is in place exactly as this issue's fix-shape suggested, with a comment explicitly citing `hyperguild#118` and explaining the two known no-op sources (manual re-run, and a since-removed tag-push CI run that used to shadow the branch-push one for the same sha). Closing as resolved via #118. `git commit --allow-empty` was correctly avoided per this issue's own note. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- _Created via git-mcp on behalf of @mathias_
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels risk/medium size/xs type/ops
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#101