create_project_from_template substitutes nothing (files_substituted: null) → produces a non-building repo #42

Closed
opened 2026-07-01 20:28:51 +00:00 by mathias · 1 comment
Owner

Symptom

create_project_from_template advertises "applying placeholder substitutions to known files" but performs no substitution at all — it returns files_substituted: null and copies the template verbatim, leaving every placeholder literal. The result is a repo that does not build, while the tool reports success. Silent contract violation.

Evidence (2026-07-01, template-go-agent → throwaway test-tpl-subst)

$ create_project_from_template owner=mathias name=test-tpl-subst template_name=template-go-agent
→ { "files_substituted": null, ... }

Reading the scaffolded repo:

  • go.modmodule __MODULE_PATH__ ← invalid module path
  • cmd/__PROJECT_NAME__/main.go → literal directory name + import "__MODULE_PATH__/pkg/litellm" ← unresolvable
  • DockerfileRUN ... go build -o /out/app ./cmd/__PROJECT_NAME__
  • CLAUDE.md, cd.yml, README.md, .env.example, AGENTS.md → all still contain __PROJECT_NAME__

go mod tidy / go build fail immediately (invalid module path + unresolvable import). There are two placeholder types, both un-substituted: __PROJECT_NAME__ and __MODULE_PATH__.

Severity: HIGH

The tool's primary purpose — create a working project from a template — produces a broken repo and reports success. Anyone using it as a one-shot gets a non-building scaffold with no error signal (same silent-green-but-broken class we've been squashing elsewhere this cycle).

Recommended fix (design A — make it a real one-shot)

The name + description already promise substitution; make it true:

  1. Substitute both __PROJECT_NAME__<name> and __MODULE_PATH__ → the module path (git.d-ma.be/<owner>/<name>).
  2. Handle the cmd/__PROJECT_NAME__/ directory rename (path substitution), not just file contents — this is the fiddly bit (git tree/path rewrite, not a simple content sed).
  3. Substitute across all text files (or a complete, tested known-files list covering go.mod, cmd/**, Dockerfile, .gitea/workflows/*, README.md, CLAUDE.md, AGENTS.md, AGENT_BOUNDARIES.md, .env.example).
  4. Return the real files_substituted list, and fail loudly if placeholders remain after the pass (don't return success with null).

Alternative (design B — explicit deferral)

If substitution is deliberately owned by a separate local step (task new-project / hyperguild new-project), then this tool should stop advertising substitution, stop being presented as producing "a working scaffold," and return an explicit "substitution deferred — finalize with hyperguild new-project" instead of a bare null. Caveat: the old local-finalize path ran scripts/context-sync.sh, which was removed from template-go-agent in infra#178 — so that path is now stale/incomplete for this template, which tilts the decision toward design A (make this tool the one-shot).

Refs

  • infra#178 (template-go-agent refresh — surfaced this; its acceptance test hit the null behavior)
  • template-go-agent placeholders: __PROJECT_NAME__, __MODULE_PATH__
## Symptom `create_project_from_template` advertises *"applying placeholder substitutions to known files"* but performs **no substitution at all** — it returns `files_substituted: null` and copies the template verbatim, leaving every placeholder literal. The result is a repo that **does not build**, while the tool reports success. Silent contract violation. ## Evidence (2026-07-01, template-go-agent → throwaway `test-tpl-subst`) ``` $ create_project_from_template owner=mathias name=test-tpl-subst template_name=template-go-agent → { "files_substituted": null, ... } ``` Reading the scaffolded repo: - `go.mod` → `module __MODULE_PATH__` ← invalid module path - `cmd/__PROJECT_NAME__/main.go` → literal **directory name** + `import "__MODULE_PATH__/pkg/litellm"` ← unresolvable - `Dockerfile` → `RUN ... go build -o /out/app ./cmd/__PROJECT_NAME__` - `CLAUDE.md`, `cd.yml`, `README.md`, `.env.example`, `AGENTS.md` → all still contain `__PROJECT_NAME__` `go mod tidy` / `go build` fail immediately (invalid module path + unresolvable import). There are **two** placeholder types, both un-substituted: `__PROJECT_NAME__` and `__MODULE_PATH__`. ## Severity: HIGH The tool's primary purpose — create a working project from a template — produces a broken repo and reports success. Anyone using it as a one-shot gets a non-building scaffold with no error signal (same silent-green-but-broken class we've been squashing elsewhere this cycle). ## Recommended fix (design A — make it a real one-shot) The name + description already promise substitution; make it true: 1. Substitute **both** `__PROJECT_NAME__` → `<name>` and `__MODULE_PATH__` → the module path (`git.d-ma.be/<owner>/<name>`). 2. Handle the **`cmd/__PROJECT_NAME__/` directory rename** (path substitution), not just file contents — this is the fiddly bit (git tree/path rewrite, not a simple content sed). 3. Substitute across **all text files** (or a complete, tested known-files list covering `go.mod`, `cmd/**`, `Dockerfile`, `.gitea/workflows/*`, `README.md`, `CLAUDE.md`, `AGENTS.md`, `AGENT_BOUNDARIES.md`, `.env.example`). 4. Return the real `files_substituted` list, and **fail loudly** if placeholders remain after the pass (don't return success with `null`). ## Alternative (design B — explicit deferral) If substitution is deliberately owned by a separate local step (`task new-project` / `hyperguild new-project`), then this tool should stop advertising substitution, stop being presented as producing "a working scaffold," and return an explicit "substitution deferred — finalize with `hyperguild new-project`" instead of a bare `null`. **Caveat:** the old local-finalize path ran `scripts/context-sync.sh`, which was removed from `template-go-agent` in infra#178 — so that path is now stale/incomplete for this template, which tilts the decision toward design A (make this tool the one-shot). ## Refs - infra#178 (template-go-agent refresh — surfaced this; its acceptance test hit the `null` behavior) - template-go-agent placeholders: `__PROJECT_NAME__`, `__MODULE_PATH__`
Author
Owner

Resolution (deployed 0395988)

The three root bugs are fixed and the silent-null contract violation is gone:

  • Empty generate branch → resolve explicitly (re-fetch repo; fall back to main). The old unit test hid this by mocking default_branch:"main".
  • Incomplete + rename-incapable → replaced the fixed 6-file list with a recursive tree walk: content-substitute every blob, and rename any placeholder-bearing path (cmd/__PROJECT_NAME__/cmd/<name>/, via POST-create new + delete old).
  • Stale module host__MODULE_PATH__ now git.d-ma.be/… (was gitea.d-ma.be, which breaks go mod download downstream — infra#74).
  • Fail loud → no more files_substituted:null masquerading as success; partial_failure is always populated when substitution can't complete.
  • Plus: migrated gitea-mcp's own mcp-chassis import to git.d-ma.be (its build was failing go mod download on the stale path — the infra#74 latent breakage, triggered by this rebuild).

Unit tests rewritten to drive the tree-walk and assert rename, module-host, empty-branch fallback, and the loud path.

⚠️ Full tool-side substitution is gated on infra#179

Live e2e exposed that gitea's template-generate on this instance doesn't make the repo writable for >40s (measured, idle koala — infra#179). No synchronous MCP tool can wait that long. So:

  • The write-readiness retry is bounded to 5s (a healthy gitea commits in ~1s and this catches it — substitution then completes automatically).
  • When the branch isn't writable in time (current reality), the tool returns fast with: "repo created, but its branch was not writable within 5s (infra#179) — finalize locally with hyperguild new-project." No hang, no silent null.

Leaving this open pending infra#179: once generate is fast, tool-side substitution completes end-to-end with no code change. Until then, the tool is honest and the local scaffold step is the finalize path (matches adr-local-dev-vs-hyperguild-new-project). Refs infra#179, infra#74.

## Resolution (deployed `0395988`) The three root bugs are **fixed** and the silent-null contract violation is gone: - **Empty generate branch** → resolve explicitly (re-fetch repo; fall back to `main`). The old unit test hid this by mocking `default_branch:"main"`. - **Incomplete + rename-incapable** → replaced the fixed 6-file list with a recursive **tree walk**: content-substitute every blob, and **rename** any placeholder-bearing path (`cmd/__PROJECT_NAME__/` → `cmd/<name>/`, via POST-create new + delete old). - **Stale module host** → `__MODULE_PATH__` now `git.d-ma.be/…` (was `gitea.d-ma.be`, which breaks `go mod download` downstream — infra#74). - **Fail loud** → no more `files_substituted:null` masquerading as success; `partial_failure` is always populated when substitution can't complete. - Plus: migrated gitea-mcp's own `mcp-chassis` import to `git.d-ma.be` (its build was failing `go mod download` on the stale path — the infra#74 latent breakage, triggered by this rebuild). Unit tests rewritten to drive the tree-walk and assert rename, module-host, empty-branch fallback, and the loud path. ## ⚠️ Full tool-side substitution is gated on infra#179 Live e2e exposed that **gitea's template-generate on this instance doesn't make the repo writable for >40s** (measured, idle koala — infra#179). No synchronous MCP tool can wait that long. So: - The write-readiness retry is bounded to **5s** (a healthy gitea commits in ~1s and this catches it — substitution then completes automatically). - When the branch isn't writable in time (current reality), the tool returns **fast** with: *"repo created, but its branch was not writable within 5s (infra#179) — finalize locally with `hyperguild new-project`."* No hang, no silent null. **Leaving this open** pending infra#179: once generate is fast, tool-side substitution completes end-to-end with no code change. Until then, the tool is honest and the local scaffold step is the finalize path (matches `adr-local-dev-vs-hyperguild-new-project`). Refs infra#179, infra#74.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/gitea-mcp#42