create_project_from_template: substitution not durable against infra#179 (silent 0-file partial_failure) #50

Closed
opened 2026-07-04 10:42:51 +00:00 by mathias · 1 comment
Owner

Problem

create_project_from_template does not reliably substitute placeholders when gitea's async template-generate is slow (infra#179). On a real project creation it returned partial_failure with zero files substituted — the repo was created but left every __PROJECT_NAME__ / __MODULE_PATH__ placeholder intact, producing a repo that does not compile (go.mod read module __MODULE_PATH__).

Reproduction

Creating mathias/parallax (repo_id 54) from template-go-agent via the tool:

partial_failure: "repo created, but its branch (main) was not writable within 5s —
gitea's template-generate is slow-async on this instance (infra#179), so substitution
is incomplete (0 file(s) done). ... Underlying: write .env.example: not found:
{"message":"branch does not exist [repo_id: 54 name: main]"}"

files_substituted: null. Every placeholder remained. A human (koala Claude) then had to clone, substitute file contents, rename the cmd/__PROJECT_NAME__/ directory (path substitution), and push by hand — see parallax#1.

Root cause

Underlying is infra#179: gitea's /generate returns before the new branch is writable via the API (observed >40s in some cases), so the tool's per-file API writes 404 with "branch does not exist." The current 5s window is far too short, and the fill-in-the-blanks-via-API approach is structurally vulnerable to this race.

Why it matters

Project creation is supposed to be one call. Today, whenever infra#179 bites, it silently produces a non-functional repo that needs a full manual fix-up — including a directory rename the API-based path can't do cleanly. At a few projects/month this is recurring friction with a fiddly manual recovery.

Candidate techniques (decision belongs in this repo)

Not prescribing — options to evaluate:

  1. Retry-until-writable — poll the branch for writability (with backoff, generous timeout) before starting substitution. Simplest; still fights the race.
  2. Server-side one-shot on repo creation — a Gitea Actions workflow or webhook that runs substitution + directory rename + first-CI on the server, where the branch is writable and git handles path renames natively. Cleaner; more infra to build/own. (Note: path/dir substitution like cmd/__PROJECT_NAME__/ is hard via the file API but trivial with a git checkout.)
  3. Local clone-substitute-push — the tool (or a helper) clones, runs sed + git mv, pushes. Deterministic; sidesteps the API race entirely. This is effectively what the advertised hyperguild new-project fallback was meant to do (note: that subcommand does not currently exist in the shipped binary).

Acceptance (whatever technique wins)

  • Creating a project from a template yields a repo with all placeholders substituted (contents and paths, incl. cmd/<name>/), or a clear hard failure — never a silent half-created repo.
  • Works when the branch takes >40s to become writable.
  • Report accurately reflects what landed.

Related

  • infra#179 (underlying async-generate slowness)
  • parallax#1 (manual recovery, evidence)
  • Companion issue: dispatch_allow inherits this same fragility (filed separately).
## Problem `create_project_from_template` does not reliably substitute placeholders when gitea's async template-generate is slow (infra#179). On a real project creation it returned `partial_failure` with **zero** files substituted — the repo was created but left every `__PROJECT_NAME__` / `__MODULE_PATH__` placeholder intact, producing a repo that does not compile (`go.mod` read `module __MODULE_PATH__`). ## Reproduction Creating `mathias/parallax` (repo_id 54) from `template-go-agent` via the tool: ``` partial_failure: "repo created, but its branch (main) was not writable within 5s — gitea's template-generate is slow-async on this instance (infra#179), so substitution is incomplete (0 file(s) done). ... Underlying: write .env.example: not found: {"message":"branch does not exist [repo_id: 54 name: main]"}" ``` `files_substituted: null`. Every placeholder remained. A human (koala Claude) then had to clone, substitute file contents, rename the `cmd/__PROJECT_NAME__/` directory (path substitution), and push by hand — see parallax#1. ## Root cause Underlying is infra#179: gitea's `/generate` returns before the new branch is writable via the API (observed >40s in some cases), so the tool's per-file API writes 404 with "branch does not exist." The current 5s window is far too short, and the fill-in-the-blanks-via-API approach is structurally vulnerable to this race. ## Why it matters Project creation is supposed to be one call. Today, whenever infra#179 bites, it silently produces a non-functional repo that needs a full manual fix-up — including a directory rename the API-based path can't do cleanly. At a few projects/month this is recurring friction with a fiddly manual recovery. ## Candidate techniques (decision belongs in this repo) Not prescribing — options to evaluate: 1. **Retry-until-writable** — poll the branch for writability (with backoff, generous timeout) before starting substitution. Simplest; still fights the race. 2. **Server-side one-shot on repo creation** — a Gitea Actions workflow or webhook that runs substitution + directory rename + first-CI on the server, where the branch is writable and `git` handles path renames natively. Cleaner; more infra to build/own. (Note: path/dir substitution like `cmd/__PROJECT_NAME__/` is hard via the file API but trivial with a git checkout.) 3. **Local clone-substitute-push** — the tool (or a helper) clones, runs `sed` + `git mv`, pushes. Deterministic; sidesteps the API race entirely. This is effectively what the advertised `hyperguild new-project` fallback was meant to do (note: that subcommand does not currently exist in the shipped binary). ## Acceptance (whatever technique wins) - Creating a project from a template yields a repo with **all** placeholders substituted (contents *and* paths, incl. `cmd/<name>/`), or a clear hard failure — never a silent half-created repo. - Works when the branch takes >40s to become writable. - Report accurately reflects what landed. ## Related - infra#179 (underlying async-generate slowness) - parallax#1 (manual recovery, evidence) - Companion issue: dispatch_allow inherits this same fragility (filed separately).
Author
Owner

Fixed in v0.9.0 (b288462).

Added resume: bool. When true, skips template lookup and repo generation entirely (the destination must already exist) and jumps straight to the tree-walk substitution. This is safe to re-invoke repeatedly because substituteEntry already compares against the branch's current state on every call (GetTree is re-fetched fresh each time, no caching) — a file already fixed in a prior partial pass shows up already-correct (or already-renamed) and is a no-op. The actual blocker to resumability was purely the "destination must NOT exist" guard on the create path; flipped it for resume, no change needed to substitution itself.

infra179FinalizeMessage now points at the concrete recovery — "call this tool again with resume=true" — replacing the old manual clone/sed/push guidance.

Technique chosen: #1 from the investigation (idempotent re-invoke), not the retry-budget bump or the server-side-workflow spike — matches the tbd_ship (#48) precedent and avoids both the retry-budget's new hang-risk and the server-side workflow's unverified push-trigger-on-generate assumption.

Tests: resume with no destination (errors, names "nothing to resume"), resume continuing a partial substitution (asserts /generate is never re-called, only the still-wrong file gets rewritten — verified at the HTTP-request level, not just JSON output), resume when already fully done (success, not the fresh-create loud-fail).

Independent adversarial review: PASS, no blocking findings. One narrow non-blocking gap surfaced — filed as a follow-up (see next comment) rather than blocking this release, since it fails loud (not silently) and is a narrow corner case.

task check green (exit 0, 0 FAIL — verified, not grep-filtered). Closing #50.

Fixed in v0.9.0 (`b288462`). Added `resume: bool`. When `true`, skips template lookup and repo generation entirely (the destination must already exist) and jumps straight to the tree-walk substitution. This is safe to re-invoke repeatedly because `substituteEntry` already compares against the branch's **current** state on every call (`GetTree` is re-fetched fresh each time, no caching) — a file already fixed in a prior partial pass shows up already-correct (or already-renamed) and is a no-op. The actual blocker to resumability was purely the "destination must NOT exist" guard on the create path; flipped it for resume, no change needed to substitution itself. `infra179FinalizeMessage` now points at the concrete recovery — "call this tool again with resume=true" — replacing the old manual clone/sed/push guidance. **Technique chosen:** #1 from the investigation (idempotent re-invoke), not the retry-budget bump or the server-side-workflow spike — matches the `tbd_ship` (#48) precedent and avoids both the retry-budget's new hang-risk and the server-side workflow's unverified push-trigger-on-generate assumption. **Tests:** resume with no destination (errors, names "nothing to resume"), resume continuing a partial substitution (asserts `/generate` is never re-called, only the still-wrong file gets rewritten — verified at the HTTP-request level, not just JSON output), resume when already fully done (success, not the fresh-create loud-fail). **Independent adversarial review:** PASS, no blocking findings. One narrow non-blocking gap surfaced — filed as a follow-up (see next comment) rather than blocking this release, since it fails loud (not silently) and is a narrow corner case. `task check` green (exit 0, 0 FAIL — verified, not grep-filtered). Closing #50.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/gitea-mcp#50