Compare commits

..
3 Commits
Author SHA1 Message Date
mathias 18d01b64d7 chore(context): restore the root rules in AGENTS.md and system-prompt.txt (#200) (#201)
CI / Lint / Test / Vet (push) Successful in 1m0s
CI / Mirror to GitHub (push) Successful in 4s
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-10-09 06:02:43 +00:00
mathiasandClaude Sonnet 5.5 5dfd2c0f7e docs: main is protected, ship by PR (local-dev#42)
CI / Lint / Test / Vet (push) Successful in 1m28s
CI / Mirror to GitHub (push) Successful in 3s
Gitea will refuse direct pushes to main in this repo. Rule 5's reviewed lane was enforced on the merge tools
(tbd_ship / pr_merge, gitea-mcp v0.17.0/v0.18.0) but not on a direct commit; this overrides rule 5's "risk/low straight
to main" for this repo and says why. Derived adapter files regenerated where the repo has them.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SZkhi49hfeSqBp5aZJNXsM
2026-10-08 08:34:50 +02:00
mathiasandClaude Opus 5.5 d6ddac4a03 fix(metrics): label an unmatched request "unmatched", never its raw path (#185)
CI / Mirror to GitHub (push) Successful in 5s
CI / Lint / Test / Vet (push) Successful in 1m4s
An authenticated REST request to an unregistered path reached the inner
mux, which left Pattern empty, and metrics.Middleware fell back to the
raw URL path: any token holder could create one Prometheus series per
unique path. The fallback is now the fixed label "unmatched"; real routes
keep their pattern. Tests: buildMux with an authenticated random path,
and the middleware unit test; the timing test now routes through a
ServeMux as production does.

Fixes #185.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 23:42:45 +02:00
14 changed files with 242 additions and 405 deletions
+12
View File
@@ -84,3 +84,15 @@ When acting as a coding agent on this project:
4. Never modify files outside the project root without explicit permission
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
for this repo; control code is where the lane matters.
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
+81 -17
View File
@@ -164,6 +164,63 @@ These rules apply to every task across every project, regardless of harness.
the thing it was named for was exploitable. If you cannot name the mechanism,
you do not yet know what the test observes.
10. **Short sessions, one rc per repo, edit only in a locked worktree of the repo
you are changing** (ADR-0002, accepted 2026-10-06).
- One session per issue or task, ending with rule 6's close ritual. Related
issues may share a session only when they share a worktree and an owner.
State that must outlive the session goes in an issue, not the chat.
- Start `claude rc` in the repo you will edit (`infra`, `trucker-ecommerce`,
`hyperguild`, ...), not in `~/dev`: the rc worktree is a worktree of the
launch directory's repo, so launched from `~/dev` it protects nothing you
edit (measured: 73% of edits landed in shared checkouts). Start it only
while working there.
- Keep at most one thin `~/dev` coordinator session. It files issues and hands
out tasks; it reads and edits no sub-repo code.
- Split cross-repo work per repo. The contract between the halves lives in an
issue on each repo.
- Edit in a worktree of that repo, locked at creation:
`scripts/wt.sh add <repo> <task>` does it (fetch, `wt/<task>` from origin/main, lock);
`scripts/wt.sh done <repo>-<task>` removes it after the push and refuses dirty or
unpushed work. By hand: `git worktree add ~/worktrees/<name> -b <branch> origin/main`
then `git worktree lock --reason "manual: <purpose>" ~/worktrees/<name>`.
- Before working a repo another session may touch, announce your issue list to
that session (`SendMessage`) and ask for theirs.
- Auto-memory is keyed by cwd, so a repo rc starts without it. Cross-repo
working rules live in brain (`wiki/homelab/decisions/2026-10-06-062416-auto.md`, wing homelab, hall
decisions) and the section below, not in per-directory memory.
## Standing footguns
Each cost real time once. Detail and evidence: brain `wiki/homelab/decisions/2026-10-06-062416-auto.md` (title standing-working-rules).
- **Gate then commit with `&&`, never `;`.** "Green" means exit code 0 and
`grep -c FAIL` is 0, not a filtered grep.
- **Stage by path, never `git add -A` or `git add .`** (they stage from the repo
root and swept in other sessions' work twice). Check `git status --porcelain` first.
- **Before committing in a shared checkout:** `git branch --show-current`.
- **Before tagging:** `git fetch origin --tags`, then `git describe --tags --abbrev=0 <sha>`
and `git tag --sort=-v:refname | head`; go above both. Never take the last tag
from a report.
- **Run only what the project's gate runs.** No repo-wide formatter unless the gate
runs it.
- **Reproduce a CI red at the cited commit before applying an issue's prescribed fix.**
- **Read an issue's comments (`issue_list_comments`), not just its body, before working it.**
- **A comment naming a control ("behind branch protection") is not evidence; query
the control.** A working detector can hide a missing preventer.
- **On koala the MCP servers are `mcp__plugin_skills_{brain,gitea,infra}__*`.** The
per-turn auth reminder is not evidence; call the tool or run `claude mcp list`.
Brain writes always pass `wing` + `hall`.
- **`cat /etc/hostname` before assuming which host you are on.**
- **Never run bare `git remote -v` or `git config --list`** (url.insteadOf expands
into a live token); use `git config --get remote.origin.url`.
- **On koala `sudo` is denied outright; k8s exec, env dumps and flux reconcile need
per-action approval; decrypting a SOPS file and printing in one Bash call is
hook-blocked.** Do the repo-side work and hand the host-side step over in an issue.
- **`set -o pipefail` plus an optional `grep | head | awk` dies on no match:** append `|| true`.
- **Gitea `file_write_branch` takes plain text, not base64.**
- **`OP_SERVICE_ACCOUNT_TOKEN` in the shell is read-only for `op`.** Operational
secrets go in the AgentSecrets vault via `set -a; . ~/.op-agent-write.env; set +a`.
## CAD sprint patterns
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
@@ -244,25 +301,20 @@ Three machines on Tailscale:
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
Organized in thematic folders:
Organized by **role**, because the role decides which rules apply (ADR-0003):
| Folder | Focus | Count |
|--------|-------|-------|
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
| Folder | Role | Rules that key off it |
|--------|------|-----------------------|
| `platform/` | infra, brain, hyperguild, gitea-mcp, dispatch, shared tooling | control code (rule 8), GitOps review lane |
| `products/` | tapir, cobalt-dingo, openbanking-api, ... | ordinary trunk-based work |
| `clients/` | client work | local models only, never cloud APIs (Client work rules) |
| `spikes/` | throwaway | no ceremony |
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
### Key active projects
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
New projects are born into a role folder (`task new-project -- <name> <client|personal>
<role-folder>`); a client tag requires `clients/` and `clients/` requires a client tag.
Existing repos still sit in the old `AGENTS/` and `AI/` folders until they are moved one by
one (ADR-0003 step 3). `~/dev/PROJECT_SUMMARY.md` (generated) lists every repo and where it
is. How a session runs on koala (worktrees, rc per repo, the reaper, memory): `docs/working-on-koala.md`.
## Knowledge base — actively use it
@@ -514,4 +566,16 @@ When acting as a coding agent on this project:
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
for this repo; control code is where the lane matters.
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
---
+81 -17
View File
@@ -159,6 +159,63 @@ These rules apply to every task across every project, regardless of harness.
the thing it was named for was exploitable. If you cannot name the mechanism,
you do not yet know what the test observes.
10. **Short sessions, one rc per repo, edit only in a locked worktree of the repo
you are changing** (ADR-0002, accepted 2026-10-06).
- One session per issue or task, ending with rule 6's close ritual. Related
issues may share a session only when they share a worktree and an owner.
State that must outlive the session goes in an issue, not the chat.
- Start `claude rc` in the repo you will edit (`infra`, `trucker-ecommerce`,
`hyperguild`, ...), not in `~/dev`: the rc worktree is a worktree of the
launch directory's repo, so launched from `~/dev` it protects nothing you
edit (measured: 73% of edits landed in shared checkouts). Start it only
while working there.
- Keep at most one thin `~/dev` coordinator session. It files issues and hands
out tasks; it reads and edits no sub-repo code.
- Split cross-repo work per repo. The contract between the halves lives in an
issue on each repo.
- Edit in a worktree of that repo, locked at creation:
`scripts/wt.sh add <repo> <task>` does it (fetch, `wt/<task>` from origin/main, lock);
`scripts/wt.sh done <repo>-<task>` removes it after the push and refuses dirty or
unpushed work. By hand: `git worktree add ~/worktrees/<name> -b <branch> origin/main`
then `git worktree lock --reason "manual: <purpose>" ~/worktrees/<name>`.
- Before working a repo another session may touch, announce your issue list to
that session (`SendMessage`) and ask for theirs.
- Auto-memory is keyed by cwd, so a repo rc starts without it. Cross-repo
working rules live in brain (`wiki/homelab/decisions/2026-10-06-062416-auto.md`, wing homelab, hall
decisions) and the section below, not in per-directory memory.
## Standing footguns
Each cost real time once. Detail and evidence: brain `wiki/homelab/decisions/2026-10-06-062416-auto.md` (title standing-working-rules).
- **Gate then commit with `&&`, never `;`.** "Green" means exit code 0 and
`grep -c FAIL` is 0, not a filtered grep.
- **Stage by path, never `git add -A` or `git add .`** (they stage from the repo
root and swept in other sessions' work twice). Check `git status --porcelain` first.
- **Before committing in a shared checkout:** `git branch --show-current`.
- **Before tagging:** `git fetch origin --tags`, then `git describe --tags --abbrev=0 <sha>`
and `git tag --sort=-v:refname | head`; go above both. Never take the last tag
from a report.
- **Run only what the project's gate runs.** No repo-wide formatter unless the gate
runs it.
- **Reproduce a CI red at the cited commit before applying an issue's prescribed fix.**
- **Read an issue's comments (`issue_list_comments`), not just its body, before working it.**
- **A comment naming a control ("behind branch protection") is not evidence; query
the control.** A working detector can hide a missing preventer.
- **On koala the MCP servers are `mcp__plugin_skills_{brain,gitea,infra}__*`.** The
per-turn auth reminder is not evidence; call the tool or run `claude mcp list`.
Brain writes always pass `wing` + `hall`.
- **`cat /etc/hostname` before assuming which host you are on.**
- **Never run bare `git remote -v` or `git config --list`** (url.insteadOf expands
into a live token); use `git config --get remote.origin.url`.
- **On koala `sudo` is denied outright; k8s exec, env dumps and flux reconcile need
per-action approval; decrypting a SOPS file and printing in one Bash call is
hook-blocked.** Do the repo-side work and hand the host-side step over in an issue.
- **`set -o pipefail` plus an optional `grep | head | awk` dies on no match:** append `|| true`.
- **Gitea `file_write_branch` takes plain text, not base64.**
- **`OP_SERVICE_ACCOUNT_TOKEN` in the shell is read-only for `op`.** Operational
secrets go in the AgentSecrets vault via `set -a; . ~/.op-agent-write.env; set +a`.
## CAD sprint patterns
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
@@ -239,25 +296,20 @@ Three machines on Tailscale:
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
Organized in thematic folders:
Organized by **role**, because the role decides which rules apply (ADR-0003):
| Folder | Focus | Count |
|--------|-------|-------|
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
| Folder | Role | Rules that key off it |
|--------|------|-----------------------|
| `platform/` | infra, brain, hyperguild, gitea-mcp, dispatch, shared tooling | control code (rule 8), GitOps review lane |
| `products/` | tapir, cobalt-dingo, openbanking-api, ... | ordinary trunk-based work |
| `clients/` | client work | local models only, never cloud APIs (Client work rules) |
| `spikes/` | throwaway | no ceremony |
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
### Key active projects
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
New projects are born into a role folder (`task new-project -- <name> <client|personal>
<role-folder>`); a client tag requires `clients/` and `clients/` requires a client tag.
Existing repos still sit in the old `AGENTS/` and `AI/` folders until they are moved one by
one (ADR-0003 step 3). `~/dev/PROJECT_SUMMARY.md` (generated) lists every repo and where it
is. How a session runs on koala (worktrees, rc per repo, the reaper, memory): `docs/working-on-koala.md`.
## Knowledge base — actively use it
@@ -508,3 +560,15 @@ When acting as a coding agent on this project:
4. Never modify files outside the project root without explicit permission
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
for this repo; control code is where the lane matters.
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
+12
View File
@@ -84,3 +84,15 @@ When acting as a coding agent on this project:
4. Never modify files outside the project root without explicit permission
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
for this repo; control code is where the lane matters.
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
+13
View File
@@ -159,3 +159,16 @@ func TestBuildMuxObservesRequestsAndServesMetrics(t *testing.T) {
require.Equal(t, http.StatusOK, code)
assert.Contains(t, body, `path="GET /pending"`)
}
// hyperguild#185: an authenticated request to an unregistered path reached
// the inner REST mux, which left Pattern empty, and the metrics label fell
// back to the raw URL path: one Prometheus series per unique path any token
// holder sends. It is now labelled with a fixed catch-all.
func TestBuildMuxLabelsAnUnregisteredPathWithAFixedLabel(t *testing.T) {
r := testMux(t)
code, _ := do(t, r, "GET", "/zqrandom-185-segment", "Bearer "+routesTok)
require.Equal(t, http.StatusNotFound, code)
_, body := do(t, r, "GET", "/metrics", "")
assert.NotContains(t, body, "zqrandom-185-segment", "the raw path became a metrics label")
assert.Contains(t, body, `path="unmatched"`)
}
-52
View File
@@ -22,10 +22,6 @@ import (
// ErrNotRegular is returned for an entry that is not a regular file.
var ErrNotRegular = errors.New("not a regular file")
// ErrLockChanged is returned by OpenLock when the lock file was replaced
// while it was being opened.
var ErrLockChanged = errors.New("lock file replaced while opening")
// ErrTooLarge is returned by ReadRegularMax for a file over the limit.
var ErrTooLarge = errors.New("file too large")
@@ -274,54 +270,6 @@ func (b *Brain) mkdirReal(rel string) error {
return nil
}
// OpenLock opens (creating if absent) a regular lock file at a clean
// brain-relative path for flock, through the root (hyperguild#192): every
// directory above it must be real, an existing lock must be a regular file,
// and the opened handle must be the file at the path (os.Root follows an
// in-root final symlink even with O_NOFOLLOW, so the Lstat pre-check is
// what refuses a dangling in-brain link). A link swapped in between the
// pre-check and the open can still have an empty file created at an in-brain
// target before the refusal: the #167 residual. ErrLockChanged reports a
// lock replaced while opening (a peer removed and recreated it); callers
// retry.
func (b *Brain) OpenLock(rel string) (*os.File, error) {
if err := b.RealParent(rel); err != nil {
return nil, err
}
if fi, err := b.root.Lstat(rel); err == nil && !fi.Mode().IsRegular() {
return nil, fmt.Errorf("%s: %w", rel, ErrNotRegular)
} else if err != nil && !errors.Is(err, fs.ErrNotExist) {
return nil, err
}
f, err := b.root.OpenFile(rel, os.O_RDWR|os.O_CREATE|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0o644)
if err != nil {
return nil, err
}
hfi, herr := f.Stat()
lfi, lerr := b.root.Lstat(rel)
if herr != nil || !hfi.Mode().IsRegular() {
_ = f.Close()
return nil, fmt.Errorf("%s: %w", rel, ErrNotRegular)
}
if lerr != nil || !os.SameFile(lfi, hfi) {
_ = f.Close()
if lerr == nil && !lfi.Mode().IsRegular() {
return nil, fmt.Errorf("%s: %w", rel, ErrNotRegular)
}
return nil, fmt.Errorf("%s: %w", rel, ErrLockChanged)
}
return f, nil
}
// LockIsCurrent reports whether f is still the file at rel: after taking a
// flock, a lock file removed and recreated by a peer means the flock is on
// an orphaned inode and does not exclude anyone (hyperguild#192).
func (b *Brain) LockIsCurrent(rel string, f *os.File) bool {
hfi, herr := f.Stat()
lfi, lerr := b.root.Lstat(rel)
return herr == nil && lerr == nil && os.SameFile(lfi, hfi)
}
// ReadRegular reads a brain-relative path, refusing anything that is not a
// regular file, both before the open and on the opened handle.
func (b *Brain) ReadRegular(rel string) ([]byte, error) {
@@ -529,49 +529,3 @@ func TestCreateRegularCreatesOnlyNewFilesThroughRealDirectories(t *testing.T) {
got, _ = os.ReadFile(filepath.Join(dir, "wiki", "new", "facts", "n.md"))
assert.Equal(t, "one", string(got))
}
// hyperguild#192: OpenLock creates or reuses a regular lock file through the
// root and refuses every other shape without creating anything at a link's
// target or hanging.
func TestOpenLockOpensOnlyARegularLockThroughRealDirectories(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.MkdirAll(filepath.Join(dir, "raw"), 0o755))
require.NoError(t, os.MkdirAll(filepath.Join(dir, "wiki", "w"), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, "raw", "v.md"), []byte("untouched"), 0o644))
require.NoError(t, os.Symlink(filepath.Join("..", "..", "raw", "created"), filepath.Join(dir, "wiki", "w", "dangling.lock")))
require.NoError(t, os.Symlink(filepath.Join("..", "..", "raw", "v.md"), filepath.Join(dir, "wiki", "w", "live.lock")))
require.NoError(t, syscall.Mkfifo(filepath.Join(dir, "wiki", "w", "fifo.lock"), 0o644))
require.NoError(t, os.MkdirAll(filepath.Join(dir, "wiki", "w", "dir.lock"), 0o755))
require.NoError(t, os.Symlink(filepath.Join("..", "raw"), filepath.Join(dir, "wiki", "alias")))
b, err := Open(dir)
require.NoError(t, err)
defer func() { _ = b.Close() }()
for _, rel := range []string{"wiki/w/dangling.lock", "wiki/w/live.lock", "wiki/w/fifo.lock", "wiki/w/dir.lock", "wiki/alias/x.lock", "wiki/w/../w/x.lock"} {
done := make(chan error, 1)
go func() {
f, err := b.OpenLock(rel)
if f != nil {
_ = f.Close()
}
done <- err
}()
select {
case err := <-done:
assert.ErrorIs(t, err, ErrNotRegular, rel)
case <-time.After(3 * time.Second):
t.Fatalf("OpenLock hung on %s", rel)
}
}
_, err = os.Lstat(filepath.Join(dir, "raw", "created"))
assert.True(t, os.IsNotExist(err), "nothing created at the dangling link's target")
got, _ := os.ReadFile(filepath.Join(dir, "raw", "v.md"))
assert.Equal(t, "untouched", string(got))
f, err := b.OpenLock("wiki/w/n.md.lock")
require.NoError(t, err, "positive control: creates a lock")
assert.True(t, b.LockIsCurrent("wiki/w/n.md.lock", f))
require.NoError(t, b.Remove("wiki/w/n.md.lock"))
assert.False(t, b.LockIsCurrent("wiki/w/n.md.lock", f), "a removed lock is no longer current")
_ = f.Close()
}
-118
View File
@@ -1,118 +0,0 @@
package mcp_test
import (
"os"
"path/filepath"
"syscall"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
)
// hyperguild#192: brain_edit snapshotted the note with os.ReadFile by path,
// which blocks on a FIFO held open by a writer, and hung the call.
func TestBrainEditReturnsPromptlyOnAFIFONoteWithAWriter(t *testing.T) {
dir := newBrainRepo(t)
fifo := filepath.Join(dir, "wiki", "homelab", "facts", "f.md")
require.NoError(t, syscall.Mkfifo(fifo, 0o644))
w, err := os.OpenFile(fifo, os.O_RDWR, 0)
require.NoError(t, err)
t.Cleanup(func() { _ = w.Close() })
srv := mcp.NewServer(dir, nil, nil, nil)
done := make(chan map[string]any, 1)
go func() {
done <- call(t, srv, "brain_edit", map[string]any{"path": "wiki/homelab/facts/f.md", "old_str": "a", "new_str": "b"})
}()
select {
case resp := <-done:
assert.Contains(t, resp, "error", "a FIFO note is refused")
case <-time.After(3 * time.Second):
t.Fatal("brain_edit hung on a FIFO note")
}
resp := call(t, srv, "brain_edit", map[string]any{"path": "wiki/homelab/facts/n.md", "old_str": "beta", "new_str": "BETA"})
require.NotContains(t, resp, "error", "positive control: a regular note is edited: %v", resp["error"])
}
// hyperguild#192: the edit lock "<note>.lock" was opened by path with
// O_CREATE, so a planted dangling .lock symlink made a successful edit create
// a file at the link's target, outside the brain or elsewhere in it.
func TestBrainEditDoesNotCreateAFileThroughADanglingLockSymlink(t *testing.T) {
for name, target := range map[string]func(t *testing.T, dir string) string{
"outside": func(t *testing.T, _ string) string { return filepath.Join(t.TempDir(), "created") },
// Relative, as Syncthing carries it: os.Root refuses an absolute
// link on its own, so only OpenLock's pre-check stops this one.
"in-brain": func(t *testing.T, dir string) string { return filepath.Join(dir, "raw", "created") },
} {
t.Run(name, func(t *testing.T) {
dir := newBrainRepo(t)
require.NoError(t, os.MkdirAll(filepath.Join(dir, "raw"), 0o755))
tgt := target(t, dir)
lock := filepath.Join(dir, "wiki", "homelab", "facts", "n.md.lock")
linkTo := tgt
if name == "in-brain" {
rel, err := filepath.Rel(filepath.Dir(lock), tgt)
require.NoError(t, err)
linkTo = rel
}
require.NoError(t, os.Symlink(linkTo, lock))
srv := mcp.NewServer(dir, nil, nil, nil)
resp := call(t, srv, "brain_edit", map[string]any{"path": "wiki/homelab/facts/n.md", "old_str": "beta", "new_str": "BETA"})
require.NotNil(t, resp["error"], "the edit is refused")
assert.Contains(t, resp["error"].(map[string]any)["message"], "lock", "refused because of the lock file")
_, err := os.Lstat(tgt)
assert.True(t, os.IsNotExist(err), "a file was created at the lock symlink's target")
})
}
}
// The rollback (when the stamp cannot be verified) writes the prior bytes
// through the brain root: a symlink swapped in for the note before the
// rollback is refused, and its target, in the brain or outside, is
// untouched.
func TestBrainEditRollbackDoesNotWriteThroughASymlinkSwappedInForTheNote(t *testing.T) {
for name, mk := range map[string]func(t *testing.T, dir string) (link, victim string){
"outside": func(t *testing.T, _ string) (string, string) {
v := filepath.Join(t.TempDir(), "victim.md")
return v, v
},
"in-brain": func(t *testing.T, dir string) (string, string) {
return filepath.Join("..", "..", "..", "raw", "victim.md"), filepath.Join(dir, "raw", "victim.md")
},
} {
t.Run(name, func(t *testing.T) {
dir := newBrainRepo(t)
require.NoError(t, os.MkdirAll(filepath.Join(dir, "raw"), 0o755))
cfg, err := classification.Load(dir)
require.NoError(t, err)
svc := capture.NewService(brainstore.New(dir), capFakeTracker{}, cfg, audit.NewSlogSink(nil))
srv := mcp.NewServer(dir, nil, nil, nil)
srv.WithCapture(svc, capFakeValidator{subject: "bad\xff"}, capStaticTok, "local-cli", capturehttp.NewOriginResolver(nil))
out := resultText(t, authCall(t, srv, "Bearer "+capStaticTok, "brain_write", map[string]any{
"content": "# R\n\nstatic body", "wing": "testing", "hall": "facts", "filename": "rb",
}))
path := out["path"].(string)
note := filepath.Join(dir, path)
link, victim := mk(t, dir)
require.NoError(t, os.WriteFile(victim, []byte("untouched"), 0o644))
defer mcp.SetBeforeStampHook(func() {
require.NoError(t, os.Remove(note))
require.NoError(t, os.Symlink(link, note))
})()
resp := authCall(t, srv, jwtAuthz, "brain_edit", map[string]any{"path": path, "old_str": "static body", "new_str": "injected"})
require.NotNil(t, resp["error"])
got, _ := os.ReadFile(victim)
assert.Equal(t, "untouched", string(got), "the rollback wrote through the swapped-in symlink")
})
}
}
-9
View File
@@ -1,9 +0,0 @@
package mcp
// SetBeforeStampHook installs beforeStampHook for a test and returns a
// restore func.
func SetBeforeStampHook(f func()) func() {
old := beforeStampHook
beforeStampHook = f
return func() { beforeStampHook = old }
}
+3 -18
View File
@@ -6,8 +6,8 @@ import (
"errors"
"fmt"
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainfs"
"log/slog"
"os"
"strings"
"time"
@@ -118,14 +118,7 @@ func (s *Server) brainEdit(ctx context.Context, args json.RawMessage) (json.RawM
return nil, err
}
// No snapshot, no edit: without one a failed stamp could not be undone.
// Read through the brain root, regular files only (hyperguild#192): a
// FIFO held open by a writer hung os.ReadFile.
bfs, err := brainfs.Open(s.brainDir)
if err != nil {
return nil, fmt.Errorf("snapshot before edit: %w", err)
}
defer func() { _ = bfs.Close() }()
prior, err := bfs.ReadRegular(target.Rel)
prior, err := os.ReadFile(target.Abs)
if err != nil {
return nil, fmt.Errorf("snapshot before edit: %w", err)
}
@@ -137,11 +130,8 @@ func (s *Server) brainEdit(ctx context.Context, args json.RawMessage) (json.RawM
// whoever edits a note is the author of what a reader now sees: re-stamp
// to the editor after every edit (PR #147 review, escape E1).
origin, writtenBy := provenanceFrom(ctx)
if beforeStampHook != nil {
beforeStampHook()
}
if serr := api.StampProvenanceFile(s.brainDir, res.Path, origin, writtenBy); serr != nil {
if werr := bfs.WriteRegular(target.Rel, prior); werr != nil {
if werr := os.WriteFile(target.Abs, prior, 0o644); werr != nil {
// Say exactly what state the note is in: edited, unverified.
return nil, fmt.Errorf("stamp provenance failed (%v) AND the edit could NOT be rolled back: %w; "+
"%s carries the edit under its previous attribution", serr, werr, res.Path)
@@ -151,11 +141,6 @@ func (s *Server) brainEdit(ctx context.Context, args json.RawMessage) (json.RawM
return out, nil
}
// beforeStampHook is a test seam (nil in production): it runs between the
// edit and the provenance stamp, so a test can swap the note before the
// rollback path runs.
var beforeStampHook func()
// brainEditRoot edits a governance file (CLAUDE.md). De-surfaced AND gated on
// the authenticated principal: only the static local-CLI token may call it, so
// a claude.ai-origin session — the one that may have just read an untrusted
+4 -3
View File
@@ -83,8 +83,9 @@ func (r *Registry) Observe(path, status string, d time.Duration) {
// Middleware wraps next, observing every request's duration + status.
// The metric label `path` uses the request's Pattern (Go 1.22+ ServeMux),
// falling back to the URL path if no Pattern is set. Pattern keeps
// cardinality bounded (one series per route, not one per unique URL).
// or the fixed label "unmatched" if no Pattern is set: falling back to the
// raw URL path gave any caller one series per unique path (hyperguild#185).
// Pattern keeps cardinality bounded (one series per route).
func (r *Registry) Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
rec := &statusRecorder{ResponseWriter: w, code: http.StatusOK}
@@ -92,7 +93,7 @@ func (r *Registry) Middleware(next http.Handler) http.Handler {
next.ServeHTTP(rec, req)
path := req.Pattern
if path == "" {
path = req.URL.Path
path = "unmatched"
}
r.Observe(path, statusClass(rec.code), time.Since(start))
})
+23 -2
View File
@@ -71,11 +71,16 @@ func TestMiddleware_RecordsTiming(t *testing.T) {
t.Parallel()
r := New()
handler := r.Middleware(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
// Routed through a ServeMux, as in production: the mux sets the
// request's Pattern, which is the label (an unrouted request is
// "unmatched", hyperguild#185).
mux := http.NewServeMux()
mux.HandleFunc("/query", func(w http.ResponseWriter, _ *http.Request) {
time.Sleep(2 * time.Millisecond)
w.WriteHeader(http.StatusOK)
_, _ = io.WriteString(w, "ok")
}))
})
handler := r.Middleware(mux)
srv := httptest.NewServer(handler)
defer srv.Close()
@@ -117,3 +122,19 @@ func TestStatusRecorder_DefaultsTo200(t *testing.T) {
t.Errorf("code %d, want 200", rec.Code)
}
}
// A request with no matched Pattern is labelled "unmatched", never with its
// raw URL path (unbounded cardinality, hyperguild#185).
func TestMiddleware_LabelsAnUnmatchedRequestWithAFixedLabel(t *testing.T) {
r := New()
h := r.Middleware(http.NotFoundHandler())
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/zqrandom-185-unit", nil))
rec := httptest.NewRecorder()
r.Handler().ServeHTTP(rec, httptest.NewRequest("GET", "/metrics", nil))
if strings.Contains(rec.Body.String(), "zqrandom-185-unit") {
t.Errorf("the raw path became a metrics label:\n%s", rec.Body.String())
}
if !strings.Contains(rec.Body.String(), `path="unmatched"`) {
t.Errorf("no unmatched label:\n%s", rec.Body.String())
}
}
+13 -41
View File
@@ -127,7 +127,7 @@ func Edit(brainDir, path, oldStr, newStr string, scope Scope) (Result, error) {
mu.Lock()
defer mu.Unlock()
unlock, err := flockFile(t)
unlock, err := flockFile(t.Abs)
if err != nil {
return Result{}, err
}
@@ -220,49 +220,21 @@ func lockFor(abs string) *sync.Mutex {
}
// flockFile takes an advisory lock covering other processes sharing the
// hostPath — during a RollingUpdate two ingestion pods do. The lock file is
// opened and removed through the brain root, never through a symlink
// (hyperguild#192): a planted dangling "<note>.lock" link made a successful
// edit create a file at its target.
//
// Unlock removes the lock file while still holding the flock, so a waiter
// can wake holding a lock on an orphaned inode while the next caller locks a
// fresh file: two editors at once (measured on main). After taking the
// flock, the file must still be the one at the path; otherwise retry.
func flockFile(t Target) (func(), error) {
lockRel := t.Rel + ".lock"
bfs, err := brainfs.Open(t.Brain)
// hostPath — during a RollingUpdate two ingestion pods do.
func flockFile(abs string) (func(), error) {
f, err := os.OpenFile(abs+".lock", os.O_CREATE|os.O_RDWR, 0o644)
if err != nil {
return nil, fmt.Errorf("notes: lock %s: %w", lockRel, err)
return nil, fmt.Errorf("notes: lock %s: %w", filepath.Base(abs), err)
}
for attempt := 0; attempt < 1000; attempt++ {
f, err := bfs.OpenLock(lockRel)
if errors.Is(err, brainfs.ErrLockChanged) {
continue
}
if err != nil {
_ = bfs.Close()
return nil, fmt.Errorf("notes: lock %s: %w", lockRel, err)
}
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
_ = f.Close()
_ = bfs.Close()
return nil, fmt.Errorf("notes: flock %s: %w", lockRel, err)
}
if !bfs.LockIsCurrent(lockRel, f) {
_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
_ = f.Close()
continue
}
return func() {
_ = bfs.Remove(lockRel)
_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
_ = f.Close()
_ = bfs.Close()
}, nil
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
_ = f.Close()
return nil, fmt.Errorf("notes: flock %s: %w", filepath.Base(abs), err)
}
_ = bfs.Close()
return nil, fmt.Errorf("notes: lock %s: kept changing while locking", lockRel)
return func() {
_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
_ = f.Close()
_ = os.Remove(abs + ".lock")
}, nil
}
// writeAtomic replaces the file via a same-directory temp + rename, then
@@ -1,82 +0,0 @@
package notes
import (
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
// lockChild runs inside a re-executed test binary: it takes the note's edit
// lock N times and, while holding it, creates an O_EXCL marker. Finding the
// marker already there means another process holds the "same" lock: mutual
// exclusion failed. It prints the overlap count.
func lockChild(brainDir string, rounds int) {
t, err := Resolve(brainDir, "wiki/w/facts/n.md", ScopeNotes)
if err != nil {
fmt.Println("resolve:", err)
os.Exit(2)
}
marker := filepath.Join(brainDir, "marker")
overlaps := 0
for i := 0; i < rounds; i++ {
unlock, err := flockFile(t)
if err != nil {
fmt.Println("lock:", err)
os.Exit(2)
}
f, err := os.OpenFile(marker, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)
if errors.Is(err, os.ErrExist) {
overlaps++
} else if err == nil {
_ = f.Close()
_ = os.Remove(marker)
}
unlock()
}
fmt.Println(overlaps)
os.Exit(0)
}
func TestMain(m *testing.M) {
if dir := os.Getenv("NOTES_LOCK_CHILD"); dir != "" {
n, _ := strconv.Atoi(os.Getenv("NOTES_LOCK_ROUNDS"))
lockChild(dir, n)
}
os.Exit(m.Run())
}
// hyperguild#192: the edit lock must exclude other processes (two ingestion
// pods share the hostPath brain during a RollingUpdate). Unlock removes the
// lock file, so without re-checking that the locked file is still the one at
// the path, a waiter and the next caller each held a lock (measured on main:
// hundreds of overlaps in 2000 rounds).
func TestEditLockExcludesOtherProcesses(t *testing.T) {
brain := t.TempDir()
require.NoError(t, os.MkdirAll(filepath.Join(brain, "wiki", "w", "facts"), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(brain, "wiki", "w", "facts", "n.md"), []byte("x"), 0o644))
const procs, rounds = 3, 400
cmds := make([]*exec.Cmd, procs)
outs := make([]*strings.Builder, procs)
for i := range cmds {
outs[i] = &strings.Builder{}
cmds[i] = exec.Command(os.Args[0], "-test.run=^$")
cmds[i].Env = append(os.Environ(), "NOTES_LOCK_CHILD="+brain, "NOTES_LOCK_ROUNDS="+strconv.Itoa(rounds))
cmds[i].Stdout, cmds[i].Stderr = outs[i], outs[i]
require.NoError(t, cmds[i].Start())
}
total := 0
for i, c := range cmds {
require.NoError(t, c.Wait(), "child %d: %s", i, outs[i].String())
n, err := strconv.Atoi(strings.TrimSpace(outs[i].String()))
require.NoError(t, err, "child %d output: %q", i, outs[i].String())
total += n
}
require.Zero(t, total, "processes held the edit lock at the same time")
}