Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
18d01b64d7 | ||
|
|
5dfd2c0f7e | ||
|
|
d6ddac4a03 | ||
|
|
4422dd1f69 | ||
|
|
f65f0b7826 | ||
|
|
f4636c7ef0 | ||
|
|
bd989f887f | ||
|
|
4770f97ffd |
@@ -84,3 +84,15 @@ When acting as a coding agent on this project:
|
|||||||
4. Never modify files outside the project root without explicit permission
|
4. Never modify files outside the project root without explicit permission
|
||||||
5. When adding a dependency, explain why in the commit message
|
5. When adding a dependency, explain why in the commit message
|
||||||
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
||||||
|
|
||||||
|
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
|
||||||
|
|
||||||
|
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
|
||||||
|
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
|
||||||
|
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
|
||||||
|
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
|
||||||
|
for this repo; control code is where the lane matters.
|
||||||
|
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
|
||||||
|
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
|
||||||
|
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
|
||||||
|
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
|
||||||
|
|||||||
+81
-17
@@ -164,6 +164,63 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
the thing it was named for was exploitable. If you cannot name the mechanism,
|
||||||
you do not yet know what the test observes.
|
you do not yet know what the test observes.
|
||||||
|
|
||||||
|
10. **Short sessions, one rc per repo, edit only in a locked worktree of the repo
|
||||||
|
you are changing** (ADR-0002, accepted 2026-10-06).
|
||||||
|
- One session per issue or task, ending with rule 6's close ritual. Related
|
||||||
|
issues may share a session only when they share a worktree and an owner.
|
||||||
|
State that must outlive the session goes in an issue, not the chat.
|
||||||
|
- Start `claude rc` in the repo you will edit (`infra`, `trucker-ecommerce`,
|
||||||
|
`hyperguild`, ...), not in `~/dev`: the rc worktree is a worktree of the
|
||||||
|
launch directory's repo, so launched from `~/dev` it protects nothing you
|
||||||
|
edit (measured: 73% of edits landed in shared checkouts). Start it only
|
||||||
|
while working there.
|
||||||
|
- Keep at most one thin `~/dev` coordinator session. It files issues and hands
|
||||||
|
out tasks; it reads and edits no sub-repo code.
|
||||||
|
- Split cross-repo work per repo. The contract between the halves lives in an
|
||||||
|
issue on each repo.
|
||||||
|
- Edit in a worktree of that repo, locked at creation:
|
||||||
|
`scripts/wt.sh add <repo> <task>` does it (fetch, `wt/<task>` from origin/main, lock);
|
||||||
|
`scripts/wt.sh done <repo>-<task>` removes it after the push and refuses dirty or
|
||||||
|
unpushed work. By hand: `git worktree add ~/worktrees/<name> -b <branch> origin/main`
|
||||||
|
then `git worktree lock --reason "manual: <purpose>" ~/worktrees/<name>`.
|
||||||
|
- Before working a repo another session may touch, announce your issue list to
|
||||||
|
that session (`SendMessage`) and ask for theirs.
|
||||||
|
- Auto-memory is keyed by cwd, so a repo rc starts without it. Cross-repo
|
||||||
|
working rules live in brain (`wiki/homelab/decisions/2026-10-06-062416-auto.md`, wing homelab, hall
|
||||||
|
decisions) and the section below, not in per-directory memory.
|
||||||
|
|
||||||
|
## Standing footguns
|
||||||
|
|
||||||
|
Each cost real time once. Detail and evidence: brain `wiki/homelab/decisions/2026-10-06-062416-auto.md` (title standing-working-rules).
|
||||||
|
|
||||||
|
- **Gate then commit with `&&`, never `;`.** "Green" means exit code 0 and
|
||||||
|
`grep -c FAIL` is 0, not a filtered grep.
|
||||||
|
- **Stage by path, never `git add -A` or `git add .`** (they stage from the repo
|
||||||
|
root and swept in other sessions' work twice). Check `git status --porcelain` first.
|
||||||
|
- **Before committing in a shared checkout:** `git branch --show-current`.
|
||||||
|
- **Before tagging:** `git fetch origin --tags`, then `git describe --tags --abbrev=0 <sha>`
|
||||||
|
and `git tag --sort=-v:refname | head`; go above both. Never take the last tag
|
||||||
|
from a report.
|
||||||
|
- **Run only what the project's gate runs.** No repo-wide formatter unless the gate
|
||||||
|
runs it.
|
||||||
|
- **Reproduce a CI red at the cited commit before applying an issue's prescribed fix.**
|
||||||
|
- **Read an issue's comments (`issue_list_comments`), not just its body, before working it.**
|
||||||
|
- **A comment naming a control ("behind branch protection") is not evidence; query
|
||||||
|
the control.** A working detector can hide a missing preventer.
|
||||||
|
- **On koala the MCP servers are `mcp__plugin_skills_{brain,gitea,infra}__*`.** The
|
||||||
|
per-turn auth reminder is not evidence; call the tool or run `claude mcp list`.
|
||||||
|
Brain writes always pass `wing` + `hall`.
|
||||||
|
- **`cat /etc/hostname` before assuming which host you are on.**
|
||||||
|
- **Never run bare `git remote -v` or `git config --list`** (url.insteadOf expands
|
||||||
|
into a live token); use `git config --get remote.origin.url`.
|
||||||
|
- **On koala `sudo` is denied outright; k8s exec, env dumps and flux reconcile need
|
||||||
|
per-action approval; decrypting a SOPS file and printing in one Bash call is
|
||||||
|
hook-blocked.** Do the repo-side work and hand the host-side step over in an issue.
|
||||||
|
- **`set -o pipefail` plus an optional `grep | head | awk` dies on no match:** append `|| true`.
|
||||||
|
- **Gitea `file_write_branch` takes plain text, not base64.**
|
||||||
|
- **`OP_SERVICE_ACCOUNT_TOKEN` in the shell is read-only for `op`.** Operational
|
||||||
|
secrets go in the AgentSecrets vault via `set -a; . ~/.op-agent-write.env; set +a`.
|
||||||
|
|
||||||
## CAD sprint patterns
|
## CAD sprint patterns
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||||
@@ -244,25 +301,20 @@ Three machines on Tailscale:
|
|||||||
|
|
||||||
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
||||||
|
|
||||||
Organized in thematic folders:
|
Organized by **role**, because the role decides which rules apply (ADR-0003):
|
||||||
|
|
||||||
| Folder | Focus | Count |
|
| Folder | Role | Rules that key off it |
|
||||||
|--------|-------|-------|
|
|--------|------|-----------------------|
|
||||||
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
|
| `platform/` | infra, brain, hyperguild, gitea-mcp, dispatch, shared tooling | control code (rule 8), GitOps review lane |
|
||||||
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
|
| `products/` | tapir, cobalt-dingo, openbanking-api, ... | ordinary trunk-based work |
|
||||||
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
|
| `clients/` | client work | local models only, never cloud APIs (Client work rules) |
|
||||||
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
|
| `spikes/` | throwaway | no ceremony |
|
||||||
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
|
|
||||||
|
|
||||||
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
|
New projects are born into a role folder (`task new-project -- <name> <client|personal>
|
||||||
|
<role-folder>`); a client tag requires `clients/` and `clients/` requires a client tag.
|
||||||
### Key active projects
|
Existing repos still sit in the old `AGENTS/` and `AI/` folders until they are moved one by
|
||||||
|
one (ADR-0003 step 3). `~/dev/PROJECT_SUMMARY.md` (generated) lists every repo and where it
|
||||||
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
|
is. How a session runs on koala (worktrees, rc per repo, the reaper, memory): `docs/working-on-koala.md`.
|
||||||
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
|
|
||||||
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
|
|
||||||
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
|
|
||||||
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
|
|
||||||
|
|
||||||
## Knowledge base — actively use it
|
## Knowledge base — actively use it
|
||||||
|
|
||||||
@@ -514,4 +566,16 @@ When acting as a coding agent on this project:
|
|||||||
5. When adding a dependency, explain why in the commit message
|
5. When adding a dependency, explain why in the commit message
|
||||||
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
||||||
|
|
||||||
|
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
|
||||||
|
|
||||||
|
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
|
||||||
|
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
|
||||||
|
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
|
||||||
|
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
|
||||||
|
for this repo; control code is where the lane matters.
|
||||||
|
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
|
||||||
|
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
|
||||||
|
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
|
||||||
|
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -159,6 +159,63 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
the thing it was named for was exploitable. If you cannot name the mechanism,
|
||||||
you do not yet know what the test observes.
|
you do not yet know what the test observes.
|
||||||
|
|
||||||
|
10. **Short sessions, one rc per repo, edit only in a locked worktree of the repo
|
||||||
|
you are changing** (ADR-0002, accepted 2026-10-06).
|
||||||
|
- One session per issue or task, ending with rule 6's close ritual. Related
|
||||||
|
issues may share a session only when they share a worktree and an owner.
|
||||||
|
State that must outlive the session goes in an issue, not the chat.
|
||||||
|
- Start `claude rc` in the repo you will edit (`infra`, `trucker-ecommerce`,
|
||||||
|
`hyperguild`, ...), not in `~/dev`: the rc worktree is a worktree of the
|
||||||
|
launch directory's repo, so launched from `~/dev` it protects nothing you
|
||||||
|
edit (measured: 73% of edits landed in shared checkouts). Start it only
|
||||||
|
while working there.
|
||||||
|
- Keep at most one thin `~/dev` coordinator session. It files issues and hands
|
||||||
|
out tasks; it reads and edits no sub-repo code.
|
||||||
|
- Split cross-repo work per repo. The contract between the halves lives in an
|
||||||
|
issue on each repo.
|
||||||
|
- Edit in a worktree of that repo, locked at creation:
|
||||||
|
`scripts/wt.sh add <repo> <task>` does it (fetch, `wt/<task>` from origin/main, lock);
|
||||||
|
`scripts/wt.sh done <repo>-<task>` removes it after the push and refuses dirty or
|
||||||
|
unpushed work. By hand: `git worktree add ~/worktrees/<name> -b <branch> origin/main`
|
||||||
|
then `git worktree lock --reason "manual: <purpose>" ~/worktrees/<name>`.
|
||||||
|
- Before working a repo another session may touch, announce your issue list to
|
||||||
|
that session (`SendMessage`) and ask for theirs.
|
||||||
|
- Auto-memory is keyed by cwd, so a repo rc starts without it. Cross-repo
|
||||||
|
working rules live in brain (`wiki/homelab/decisions/2026-10-06-062416-auto.md`, wing homelab, hall
|
||||||
|
decisions) and the section below, not in per-directory memory.
|
||||||
|
|
||||||
|
## Standing footguns
|
||||||
|
|
||||||
|
Each cost real time once. Detail and evidence: brain `wiki/homelab/decisions/2026-10-06-062416-auto.md` (title standing-working-rules).
|
||||||
|
|
||||||
|
- **Gate then commit with `&&`, never `;`.** "Green" means exit code 0 and
|
||||||
|
`grep -c FAIL` is 0, not a filtered grep.
|
||||||
|
- **Stage by path, never `git add -A` or `git add .`** (they stage from the repo
|
||||||
|
root and swept in other sessions' work twice). Check `git status --porcelain` first.
|
||||||
|
- **Before committing in a shared checkout:** `git branch --show-current`.
|
||||||
|
- **Before tagging:** `git fetch origin --tags`, then `git describe --tags --abbrev=0 <sha>`
|
||||||
|
and `git tag --sort=-v:refname | head`; go above both. Never take the last tag
|
||||||
|
from a report.
|
||||||
|
- **Run only what the project's gate runs.** No repo-wide formatter unless the gate
|
||||||
|
runs it.
|
||||||
|
- **Reproduce a CI red at the cited commit before applying an issue's prescribed fix.**
|
||||||
|
- **Read an issue's comments (`issue_list_comments`), not just its body, before working it.**
|
||||||
|
- **A comment naming a control ("behind branch protection") is not evidence; query
|
||||||
|
the control.** A working detector can hide a missing preventer.
|
||||||
|
- **On koala the MCP servers are `mcp__plugin_skills_{brain,gitea,infra}__*`.** The
|
||||||
|
per-turn auth reminder is not evidence; call the tool or run `claude mcp list`.
|
||||||
|
Brain writes always pass `wing` + `hall`.
|
||||||
|
- **`cat /etc/hostname` before assuming which host you are on.**
|
||||||
|
- **Never run bare `git remote -v` or `git config --list`** (url.insteadOf expands
|
||||||
|
into a live token); use `git config --get remote.origin.url`.
|
||||||
|
- **On koala `sudo` is denied outright; k8s exec, env dumps and flux reconcile need
|
||||||
|
per-action approval; decrypting a SOPS file and printing in one Bash call is
|
||||||
|
hook-blocked.** Do the repo-side work and hand the host-side step over in an issue.
|
||||||
|
- **`set -o pipefail` plus an optional `grep | head | awk` dies on no match:** append `|| true`.
|
||||||
|
- **Gitea `file_write_branch` takes plain text, not base64.**
|
||||||
|
- **`OP_SERVICE_ACCOUNT_TOKEN` in the shell is read-only for `op`.** Operational
|
||||||
|
secrets go in the AgentSecrets vault via `set -a; . ~/.op-agent-write.env; set +a`.
|
||||||
|
|
||||||
## CAD sprint patterns
|
## CAD sprint patterns
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||||
@@ -239,25 +296,20 @@ Three machines on Tailscale:
|
|||||||
|
|
||||||
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
|
||||||
|
|
||||||
Organized in thematic folders:
|
Organized by **role**, because the role decides which rules apply (ADR-0003):
|
||||||
|
|
||||||
| Folder | Focus | Count |
|
| Folder | Role | Rules that key off it |
|
||||||
|--------|-------|-------|
|
|--------|------|-----------------------|
|
||||||
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
|
| `platform/` | infra, brain, hyperguild, gitea-mcp, dispatch, shared tooling | control code (rule 8), GitOps review lane |
|
||||||
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
|
| `products/` | tapir, cobalt-dingo, openbanking-api, ... | ordinary trunk-based work |
|
||||||
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
|
| `clients/` | client work | local models only, never cloud APIs (Client work rules) |
|
||||||
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
|
| `spikes/` | throwaway | no ceremony |
|
||||||
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
|
|
||||||
|
|
||||||
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
|
New projects are born into a role folder (`task new-project -- <name> <client|personal>
|
||||||
|
<role-folder>`); a client tag requires `clients/` and `clients/` requires a client tag.
|
||||||
### Key active projects
|
Existing repos still sit in the old `AGENTS/` and `AI/` folders until they are moved one by
|
||||||
|
one (ADR-0003 step 3). `~/dev/PROJECT_SUMMARY.md` (generated) lists every repo and where it
|
||||||
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
|
is. How a session runs on koala (worktrees, rc per repo, the reaper, memory): `docs/working-on-koala.md`.
|
||||||
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
|
|
||||||
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
|
|
||||||
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
|
|
||||||
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
|
|
||||||
|
|
||||||
## Knowledge base — actively use it
|
## Knowledge base — actively use it
|
||||||
|
|
||||||
@@ -508,3 +560,15 @@ When acting as a coding agent on this project:
|
|||||||
4. Never modify files outside the project root without explicit permission
|
4. Never modify files outside the project root without explicit permission
|
||||||
5. When adding a dependency, explain why in the commit message
|
5. When adding a dependency, explain why in the commit message
|
||||||
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
||||||
|
|
||||||
|
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
|
||||||
|
|
||||||
|
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
|
||||||
|
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
|
||||||
|
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
|
||||||
|
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
|
||||||
|
for this repo; control code is where the lane matters.
|
||||||
|
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
|
||||||
|
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
|
||||||
|
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
|
||||||
|
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
|
||||||
|
|||||||
@@ -84,3 +84,15 @@ When acting as a coding agent on this project:
|
|||||||
4. Never modify files outside the project root without explicit permission
|
4. Never modify files outside the project root without explicit permission
|
||||||
5. When adding a dependency, explain why in the commit message
|
5. When adding a dependency, explain why in the commit message
|
||||||
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
|
||||||
|
|
||||||
|
## `main` is protected: ship by PR (local-dev#42, decided 2026-10-08)
|
||||||
|
|
||||||
|
Gitea refuses direct pushes to `main` in this repo, and a PR merges only after its CI status (`CI / Lint / Test / Vet (pull_request)`) passes.
|
||||||
|
**Why:** local-dev rule 5's reviewed lane (risk/medium+ is reviewed by a human) was enforced on the merge tools
|
||||||
|
(`tbd_ship`, `pr_merge`: gitea-mcp v0.17.0/v0.18.0, merge only for an issue labelled exactly `risk/low` with green CI)
|
||||||
|
but not on a direct commit to `main`, which bypassed it. This overrides rule 5's "risk/low commits straight to main"
|
||||||
|
for this repo; control code is where the lane matters.
|
||||||
|
**How:** `tbd_ship` for a one-file change, or branch -> PR -> `pr_merge` (name the `risk/low` issue); above `risk/low`
|
||||||
|
open the PR and a human merges it in the Gitea UI after review. Tags are not protected. CD jobs push image tags to
|
||||||
|
the **infra** repo with deploy keys, never to this repo's `main`. Undo (Mathias only): Settings -> Branches, or
|
||||||
|
`PATCH /repos/mathias/hyperguild/branch_protections/main {"enable_push": true}`.
|
||||||
|
|||||||
@@ -179,9 +179,12 @@ func alertingWarnings(getenv func(string) string) []string {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func buildNotifier() audit.Notifier {
|
// buildNotifier returns the ntfy notifier for one alert source, titled so
|
||||||
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
// the source is recognisable (hyperguild#163), or nil when ntfy is not
|
||||||
return n
|
// configured.
|
||||||
|
func buildNotifier(getenv func(string) string, title string) audit.Notifier {
|
||||||
|
if n := audit.NewNtfyNotifier(getenv("BRAIN_NTFY_URL"), getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
||||||
|
return n.WithTitle(title)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -203,7 +206,7 @@ func buildAuditSink(ctx context.Context, brainDir string, logger *slog.Logger) c
|
|||||||
logger.Error("capture audit buffer init", "err", err)
|
logger.Error("capture audit buffer init", "err", err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
notifier := buildNotifier()
|
notifier := buildNotifier(os.Getenv, "brain-capture audit degraded")
|
||||||
reconcileInterval := time.Duration(envInt("BRAIN_AUDIT_RECONCILE_INTERVAL", 60)) * time.Second
|
reconcileInterval := time.Duration(envInt("BRAIN_AUDIT_RECONCILE_INTERVAL", 60)) * time.Second
|
||||||
audit.StartReconcile(ctx, central, buffer, notifier, reconcileInterval)
|
audit.StartReconcile(ctx, central, buffer, notifier, reconcileInterval)
|
||||||
logger.Info("capture audit: loki+buffer sink", "loki", lokiURL, "reconcile_s", int(reconcileInterval.Seconds()))
|
logger.Info("capture audit: loki+buffer sink", "loki", lokiURL, "reconcile_s", int(reconcileInterval.Seconds()))
|
||||||
@@ -366,7 +369,7 @@ func main() {
|
|||||||
|
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
// hyperguild#157: report a symlink that appears in the brain while running.
|
// hyperguild#157: report a symlink that appears in the brain while running.
|
||||||
go symlinkguard.Watch(ctx, brainDir, symlinkScanInterval(os.Getenv), buildNotifier())
|
go symlinkguard.Watch(ctx, brainDir, symlinkScanInterval(os.Getenv), buildNotifier(os.Getenv, "brain symlink tripwire"))
|
||||||
|
|
||||||
// Never fatal on an unreachable issuer (#83): on a cold cluster boot this
|
// Never fatal on an unreachable issuer (#83): on a cold cluster boot this
|
||||||
// pod can beat dex to serving and get a 502 from ingress, and exiting meant
|
// pod can beat dex to serving and get a 502 from ingress, and exiting meant
|
||||||
@@ -546,7 +549,7 @@ func main() {
|
|||||||
// capture already gets, plus an out-of-band alert. Wired here because
|
// capture already gets, plus an out-of-band alert. Wired here because
|
||||||
// brain_edit_root needs the capture plumbing anyway — that is what
|
// brain_edit_root needs the capture plumbing anyway — that is what
|
||||||
// re-derives the static principal its gate checks.
|
// re-derives the static principal its gate checks.
|
||||||
mcpSrv.WithGovernanceObserver(auditSink, buildNotifier())
|
mcpSrv.WithGovernanceObserver(auditSink, buildNotifier(os.Getenv, "brain governance edit"))
|
||||||
logger.Info("capture enabled (REST + MCP tool)", "sovereign_principals", len(sovereign),
|
logger.Info("capture enabled (REST + MCP tool)", "sovereign_principals", len(sovereign),
|
||||||
"governance_observer", true)
|
"governance_observer", true)
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -159,3 +159,16 @@ func TestBuildMuxObservesRequestsAndServesMetrics(t *testing.T) {
|
|||||||
require.Equal(t, http.StatusOK, code)
|
require.Equal(t, http.StatusOK, code)
|
||||||
assert.Contains(t, body, `path="GET /pending"`)
|
assert.Contains(t, body, `path="GET /pending"`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// hyperguild#185: an authenticated request to an unregistered path reached
|
||||||
|
// the inner REST mux, which left Pattern empty, and the metrics label fell
|
||||||
|
// back to the raw URL path: one Prometheus series per unique path any token
|
||||||
|
// holder sends. It is now labelled with a fixed catch-all.
|
||||||
|
func TestBuildMuxLabelsAnUnregisteredPathWithAFixedLabel(t *testing.T) {
|
||||||
|
r := testMux(t)
|
||||||
|
code, _ := do(t, r, "GET", "/zqrandom-185-segment", "Bearer "+routesTok)
|
||||||
|
require.Equal(t, http.StatusNotFound, code)
|
||||||
|
_, body := do(t, r, "GET", "/metrics", "")
|
||||||
|
assert.NotContains(t, body, "zqrandom-185-segment", "the raw path became a metrics label")
|
||||||
|
assert.Contains(t, body, `path="unmatched"`)
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,9 +1,13 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
"syscall"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
@@ -63,3 +67,26 @@ func TestStartupSymlinkCheckRefusesAnUnlistableBrainRoot(t *testing.T) {
|
|||||||
t.Cleanup(func() { _ = os.Chmod(brain, 0o755) })
|
t.Cleanup(func() { _ = os.Chmod(brain, 0o755) })
|
||||||
assert.Error(t, startupSymlinkCheck(brain))
|
assert.Error(t, startupSymlinkCheck(brain))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// hyperguild#179: a FIFO in the brain is refused at startup with a clear
|
||||||
|
// message, instead of hanging or exiting later in a reader.
|
||||||
|
func TestStartupSymlinkCheckRefusesABrainWithAFIFO(t *testing.T) {
|
||||||
|
brain := t.TempDir()
|
||||||
|
require.NoError(t, syscall.Mkfifo(filepath.Join(brain, "classification.yaml"), 0o644))
|
||||||
|
err := startupSymlinkCheck(brain)
|
||||||
|
require.Error(t, err)
|
||||||
|
assert.Contains(t, err.Error(), "classification.yaml")
|
||||||
|
}
|
||||||
|
|
||||||
|
// hyperguild#163: each alert source gets its own ntfy title.
|
||||||
|
func TestBuildNotifierUsesTheGivenTitle(t *testing.T) {
|
||||||
|
var got string
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { got = r.Header.Get("Title") }))
|
||||||
|
defer srv.Close()
|
||||||
|
env := map[string]string{"BRAIN_NTFY_URL": srv.URL}
|
||||||
|
n := buildNotifier(func(k string) string { return env[k] }, "brain symlink tripwire")
|
||||||
|
require.NotNil(t, n)
|
||||||
|
require.NoError(t, n.Notify(context.Background(), "m"))
|
||||||
|
assert.Equal(t, "brain symlink tripwire", got)
|
||||||
|
assert.Nil(t, buildNotifier(func(string) string { return "" }, "x"), "no ntfy URL, no notifier")
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
package api_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hyperguild#178: an unreadable directory under wiki/ made the link check
|
||||||
|
// fail and brain_write refuse the note, losing the learning. The note is now
|
||||||
|
// written with its links as given.
|
||||||
|
func TestWriteNoteKeepsTheLearningWhenTheLinkCheckCannotIndex(t *testing.T) {
|
||||||
|
if os.Geteuid() == 0 {
|
||||||
|
t.Skip("root ignores directory permissions")
|
||||||
|
}
|
||||||
|
brain := t.TempDir()
|
||||||
|
require.NoError(t, os.MkdirAll(filepath.Join(brain, "wiki", "w", "facts"), 0o755))
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(brain, "wiki", "w", "facts", "real.md"), []byte("x"), 0o644))
|
||||||
|
locked := filepath.Join(brain, "wiki", "other", "locked")
|
||||||
|
require.NoError(t, os.MkdirAll(locked, 0o755))
|
||||||
|
require.NoError(t, os.Chmod(locked, 0o000))
|
||||||
|
t.Cleanup(func() { _ = os.Chmod(locked, 0o755) })
|
||||||
|
|
||||||
|
rel, err := api.WriteNote(brain, api.WriteNoteOptions{Wing: "w", Hall: "facts", Filename: "n", Content: "learning [[real]] and [[gone]]"})
|
||||||
|
require.NoError(t, err)
|
||||||
|
got, err := os.ReadFile(filepath.Join(brain, filepath.FromSlash(rel)))
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Contains(t, string(got), "[[gone]]", "links kept as given when the index is partial")
|
||||||
|
}
|
||||||
@@ -13,9 +13,14 @@ import (
|
|||||||
type NtfyNotifier struct {
|
type NtfyNotifier struct {
|
||||||
topicURL string
|
topicURL string
|
||||||
token string
|
token string
|
||||||
|
title string
|
||||||
http *http.Client
|
http *http.Client
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// defaultNtfyTitle is the title of a notifier built without WithTitle: the
|
||||||
|
// capture audit's degraded-state alert, its first user.
|
||||||
|
const defaultNtfyTitle = "brain-capture audit degraded"
|
||||||
|
|
||||||
// NewNtfyNotifier constructs a notifier for the given ntfy topic URL
|
// NewNtfyNotifier constructs a notifier for the given ntfy topic URL
|
||||||
// (e.g. https://ntfy.sh/my-topic). token is an optional bearer for
|
// (e.g. https://ntfy.sh/my-topic). token is an optional bearer for
|
||||||
// protected ntfy instances; it is held here and only sent in the
|
// protected ntfy instances; it is held here and only sent in the
|
||||||
@@ -27,17 +32,31 @@ func NewNtfyNotifier(topicURL, token string) *NtfyNotifier {
|
|||||||
return &NtfyNotifier{
|
return &NtfyNotifier{
|
||||||
topicURL: strings.TrimRight(topicURL, "/"),
|
topicURL: strings.TrimRight(topicURL, "/"),
|
||||||
token: token,
|
token: token,
|
||||||
|
title: defaultNtfyTitle,
|
||||||
http: &http.Client{Timeout: 10 * time.Second},
|
http: &http.Client{Timeout: 10 * time.Second},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithTitle returns a copy that sends title as the ntfy Title header, so each
|
||||||
|
// alert source (the capture audit, the symlink tripwire, governance edits)
|
||||||
|
// is recognisable on the phone (hyperguild#163). A nil notifier (ntfy not
|
||||||
|
// configured) stays nil.
|
||||||
|
func (n *NtfyNotifier) WithTitle(title string) *NtfyNotifier {
|
||||||
|
if n == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
c := *n
|
||||||
|
c.title = title
|
||||||
|
return &c
|
||||||
|
}
|
||||||
|
|
||||||
// Notify posts a message to the ntfy topic.
|
// Notify posts a message to the ntfy topic.
|
||||||
func (n *NtfyNotifier) Notify(ctx context.Context, msg string) error {
|
func (n *NtfyNotifier) Notify(ctx context.Context, msg string) error {
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, n.topicURL, strings.NewReader(msg))
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, n.topicURL, strings.NewReader(msg))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
req.Header.Set("Title", "brain-capture audit degraded")
|
req.Header.Set("Title", n.title)
|
||||||
req.Header.Set("Priority", "high")
|
req.Header.Set("Priority", "high")
|
||||||
req.Header.Set("Tags", "warning,brain")
|
req.Header.Set("Tags", "warning,brain")
|
||||||
if n.token != "" {
|
if n.token != "" {
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package audit
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hyperguild#163: every alert went out titled "brain-capture audit
|
||||||
|
// degraded", including the symlink tripwire's and governance alerts.
|
||||||
|
func TestNtfyNotifierSendsItsOwnTitle(t *testing.T) {
|
||||||
|
var got []string
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
got = append(got, r.Header.Get("Title"))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
base := NewNtfyNotifier(srv.URL, "")
|
||||||
|
require.NoError(t, base.Notify(context.Background(), "m"))
|
||||||
|
require.NoError(t, base.WithTitle("brain symlink tripwire").Notify(context.Background(), "m"))
|
||||||
|
require.NoError(t, base.Notify(context.Background(), "m"))
|
||||||
|
assert.Equal(t, []string{"brain-capture audit degraded", "brain symlink tripwire", "brain-capture audit degraded"}, got,
|
||||||
|
"WithTitle returns a copy; the base keeps its default title")
|
||||||
|
var nilN *NtfyNotifier
|
||||||
|
assert.Nil(t, nilN.WithTitle("x"), "nil (ntfy not configured) stays nil")
|
||||||
|
}
|
||||||
@@ -47,7 +47,12 @@ var (
|
|||||||
func StripUnresolvedLinks(brainDir, content string) (string, []string, error) {
|
func StripUnresolvedLinks(brainDir, content string) (string, []string, error) {
|
||||||
index, err := entryIndex(brainDir)
|
index, err := entryIndex(brainDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", nil, err
|
// A partial index cannot tell an invented link from one into the
|
||||||
|
// part it could not read, so unwrap nothing, and do not refuse the
|
||||||
|
// write either: this runs before every brain_write, and a refused
|
||||||
|
// write is a lost learning (hyperguild#178).
|
||||||
|
slog.Warn("link check skipped: cannot index the brain", "err", err)
|
||||||
|
return content, nil, nil
|
||||||
}
|
}
|
||||||
if len(index) == 0 {
|
if len(index) == 0 {
|
||||||
return content, nil, nil
|
return content, nil, nil
|
||||||
|
|||||||
@@ -0,0 +1,41 @@
|
|||||||
|
package brain_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hyperguild#178: StripUnresolvedLinks runs before every brain_write and
|
||||||
|
// brain_update. An unreadable directory under knowledge/ or wiki/ made it
|
||||||
|
// error, the write was refused and the learning lost. When the link index
|
||||||
|
// cannot be built, the content is now returned unchanged (no link is
|
||||||
|
// unwrapped, none is invented) and no error stops the write.
|
||||||
|
func TestStripUnresolvedLinksKeepsContentWhenTheIndexCannotBeBuilt(t *testing.T) {
|
||||||
|
if os.Geteuid() == 0 {
|
||||||
|
t.Skip("root ignores directory permissions")
|
||||||
|
}
|
||||||
|
b := t.TempDir()
|
||||||
|
require.NoError(t, os.MkdirAll(filepath.Join(b, "wiki", "w", "facts"), 0o755))
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(b, "wiki", "w", "facts", "real.md"), []byte("x"), 0o644))
|
||||||
|
locked := filepath.Join(b, "wiki", "other", "locked")
|
||||||
|
require.NoError(t, os.MkdirAll(locked, 0o755))
|
||||||
|
require.NoError(t, os.Chmod(locked, 0o000))
|
||||||
|
t.Cleanup(func() { _ = os.Chmod(locked, 0o755) })
|
||||||
|
|
||||||
|
in := "learning [[real]] and [[gone]]"
|
||||||
|
out, unlinked, err := brain.StripUnresolvedLinks(b, in)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, in, out, "content unchanged: no link unwrapped on a partial index")
|
||||||
|
assert.Empty(t, unlinked)
|
||||||
|
|
||||||
|
require.NoError(t, os.Chmod(locked, 0o755))
|
||||||
|
out, _, err = brain.StripUnresolvedLinks(b, in)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.NotContains(t, out, "[[gone]]", "positive control: with a readable tree the strip runs")
|
||||||
|
}
|
||||||
@@ -83,8 +83,9 @@ func (r *Registry) Observe(path, status string, d time.Duration) {
|
|||||||
|
|
||||||
// Middleware wraps next, observing every request's duration + status.
|
// Middleware wraps next, observing every request's duration + status.
|
||||||
// The metric label `path` uses the request's Pattern (Go 1.22+ ServeMux),
|
// The metric label `path` uses the request's Pattern (Go 1.22+ ServeMux),
|
||||||
// falling back to the URL path if no Pattern is set. Pattern keeps
|
// or the fixed label "unmatched" if no Pattern is set: falling back to the
|
||||||
// cardinality bounded (one series per route, not one per unique URL).
|
// raw URL path gave any caller one series per unique path (hyperguild#185).
|
||||||
|
// Pattern keeps cardinality bounded (one series per route).
|
||||||
func (r *Registry) Middleware(next http.Handler) http.Handler {
|
func (r *Registry) Middleware(next http.Handler) http.Handler {
|
||||||
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
|
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
|
||||||
rec := &statusRecorder{ResponseWriter: w, code: http.StatusOK}
|
rec := &statusRecorder{ResponseWriter: w, code: http.StatusOK}
|
||||||
@@ -92,7 +93,7 @@ func (r *Registry) Middleware(next http.Handler) http.Handler {
|
|||||||
next.ServeHTTP(rec, req)
|
next.ServeHTTP(rec, req)
|
||||||
path := req.Pattern
|
path := req.Pattern
|
||||||
if path == "" {
|
if path == "" {
|
||||||
path = req.URL.Path
|
path = "unmatched"
|
||||||
}
|
}
|
||||||
r.Observe(path, statusClass(rec.code), time.Since(start))
|
r.Observe(path, statusClass(rec.code), time.Since(start))
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -71,11 +71,16 @@ func TestMiddleware_RecordsTiming(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
r := New()
|
r := New()
|
||||||
handler := r.Middleware(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
// Routed through a ServeMux, as in production: the mux sets the
|
||||||
|
// request's Pattern, which is the label (an unrouted request is
|
||||||
|
// "unmatched", hyperguild#185).
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("/query", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
time.Sleep(2 * time.Millisecond)
|
time.Sleep(2 * time.Millisecond)
|
||||||
w.WriteHeader(http.StatusOK)
|
w.WriteHeader(http.StatusOK)
|
||||||
_, _ = io.WriteString(w, "ok")
|
_, _ = io.WriteString(w, "ok")
|
||||||
}))
|
})
|
||||||
|
handler := r.Middleware(mux)
|
||||||
|
|
||||||
srv := httptest.NewServer(handler)
|
srv := httptest.NewServer(handler)
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
@@ -117,3 +122,19 @@ func TestStatusRecorder_DefaultsTo200(t *testing.T) {
|
|||||||
t.Errorf("code %d, want 200", rec.Code)
|
t.Errorf("code %d, want 200", rec.Code)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A request with no matched Pattern is labelled "unmatched", never with its
|
||||||
|
// raw URL path (unbounded cardinality, hyperguild#185).
|
||||||
|
func TestMiddleware_LabelsAnUnmatchedRequestWithAFixedLabel(t *testing.T) {
|
||||||
|
r := New()
|
||||||
|
h := r.Middleware(http.NotFoundHandler())
|
||||||
|
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/zqrandom-185-unit", nil))
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
r.Handler().ServeHTTP(rec, httptest.NewRequest("GET", "/metrics", nil))
|
||||||
|
if strings.Contains(rec.Body.String(), "zqrandom-185-unit") {
|
||||||
|
t.Errorf("the raw path became a metrics label:\n%s", rec.Body.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(rec.Body.String(), `path="unmatched"`) {
|
||||||
|
t.Errorf("no unmatched label:\n%s", rec.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,104 @@
|
|||||||
|
package symlinkguard
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
// hyperguild#179: the tripwire looked only for symlinks, so a FIFO (or
|
||||||
|
// socket) in the brain passed it. A FIFO at classification.yaml then made
|
||||||
|
// startup exit; at a note path it could hang readers that skip the handle
|
||||||
|
// check.
|
||||||
|
func TestScanReportsFIFOsAndSocketsAsSpecialFiles(t *testing.T) {
|
||||||
|
brain := t.TempDir()
|
||||||
|
require.NoError(t, os.MkdirAll(filepath.Join(brain, "wiki", "w"), 0o755))
|
||||||
|
require.NoError(t, syscall.Mkfifo(filepath.Join(brain, "classification.yaml"), 0o644))
|
||||||
|
require.NoError(t, syscall.Mkfifo(filepath.Join(brain, "wiki", "w", "f.md"), 0o644))
|
||||||
|
l, err := net.Listen("unix", filepath.Join(brain, "s.sock"))
|
||||||
|
require.NoError(t, err)
|
||||||
|
defer func() { _ = l.Close() }()
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(brain, "wiki", "w", "n.md"), []byte("n"), 0o644))
|
||||||
|
|
||||||
|
r, err := Scan(brain)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, []string{"classification.yaml", "s.sock", "wiki/w/f.md"}, r.Special)
|
||||||
|
assert.False(t, r.Clean())
|
||||||
|
assert.Contains(t, r.String(), "special file")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Watch removes a FIFO as it removes symlinks, and leaves a regular file
|
||||||
|
// (which Scan never reports) alone.
|
||||||
|
func TestWatchRemovesAFIFOAndLeavesAnUnreportedRegularFile(t *testing.T) {
|
||||||
|
brain := t.TempDir()
|
||||||
|
fifo := filepath.Join(brain, "f.md")
|
||||||
|
require.NoError(t, syscall.Mkfifo(fifo, 0o644))
|
||||||
|
regular := filepath.Join(brain, "n.md")
|
||||||
|
require.NoError(t, os.WriteFile(regular, []byte("n"), 0o644))
|
||||||
|
n := &recNotifier{}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
go Watch(ctx, brain, 10*time.Millisecond, n)
|
||||||
|
require.Eventually(t, func() bool {
|
||||||
|
_, err := os.Lstat(fifo)
|
||||||
|
return os.IsNotExist(err)
|
||||||
|
}, 2*time.Second, 10*time.Millisecond, "the FIFO is removed")
|
||||||
|
require.Eventually(t, func() bool { return n.count() >= 1 }, 2*time.Second, 10*time.Millisecond)
|
||||||
|
b, err := os.ReadFile(regular)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Equal(t, "n", string(b))
|
||||||
|
n.mu.Lock()
|
||||||
|
defer n.mu.Unlock()
|
||||||
|
assert.Contains(t, n.msgs[0], "f.md")
|
||||||
|
}
|
||||||
|
|
||||||
|
// removeSpecial re-checks the type: a path that is a regular file or a
|
||||||
|
// directory at the re-check is not removed. (A swap between the re-check and
|
||||||
|
// the unlink can still remove one; Linux has no conditional unlink. The same
|
||||||
|
// window exists in removeSymlinks.)
|
||||||
|
func TestRemoveSpecialRefusesRegularFilesAndDirectories(t *testing.T) {
|
||||||
|
brain := t.TempDir()
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(brain, "n.md"), []byte("n"), 0o644))
|
||||||
|
require.NoError(t, os.MkdirAll(filepath.Join(brain, "d"), 0o755))
|
||||||
|
require.NoError(t, syscall.Mkfifo(filepath.Join(brain, "f.md"), 0o644))
|
||||||
|
removed, failed := removeSpecial(brain, []string{"n.md", "d", "f.md"})
|
||||||
|
assert.Equal(t, []string{"f.md"}, removed)
|
||||||
|
assert.Equal(t, []string{"n.md", "d"}, failed)
|
||||||
|
_, err := os.Stat(filepath.Join(brain, "n.md"))
|
||||||
|
assert.NoError(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A special file Watch cannot remove (its directory is read-only) stays
|
||||||
|
// reported: the persistent-finding alert names it.
|
||||||
|
func TestWatchReportsASpecialFileItCannotRemove(t *testing.T) {
|
||||||
|
if os.Geteuid() == 0 {
|
||||||
|
t.Skip("root ignores directory permissions")
|
||||||
|
}
|
||||||
|
brain := t.TempDir()
|
||||||
|
ro := filepath.Join(brain, "ro")
|
||||||
|
require.NoError(t, os.MkdirAll(ro, 0o755))
|
||||||
|
require.NoError(t, syscall.Mkfifo(filepath.Join(ro, "f.md"), 0o644))
|
||||||
|
require.NoError(t, os.Chmod(ro, 0o555))
|
||||||
|
t.Cleanup(func() { _ = os.Chmod(ro, 0o755) })
|
||||||
|
n := &recNotifier{}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
go Watch(ctx, brain, 10*time.Millisecond, n)
|
||||||
|
require.Eventually(t, func() bool { return n.count() >= 1 }, 2*time.Second, 10*time.Millisecond)
|
||||||
|
_, err := os.Lstat(filepath.Join(ro, "f.md"))
|
||||||
|
require.NoError(t, err, "the FIFO could not be removed")
|
||||||
|
n.mu.Lock()
|
||||||
|
defer n.mu.Unlock()
|
||||||
|
assert.Contains(t, n.msgs[0], "ro/f.md")
|
||||||
|
assert.Contains(t, n.msgs[0], "special file")
|
||||||
|
for _, m := range n.msgs {
|
||||||
|
assert.NotContains(t, m, "removed", "a file it could not remove was reported as removed")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -46,10 +46,16 @@ type Notifier interface {
|
|||||||
type Result struct {
|
type Result struct {
|
||||||
Symlinks []string
|
Symlinks []string
|
||||||
Unreadable []string
|
Unreadable []string
|
||||||
|
// Special holds FIFOs, sockets and devices (hyperguild#179): a reader
|
||||||
|
// that opens one by path can hang, and a FIFO at classification.yaml
|
||||||
|
// stops startup.
|
||||||
|
Special []string
|
||||||
}
|
}
|
||||||
|
|
||||||
// Clean reports whether the scan found nothing to act on.
|
// Clean reports whether the scan found nothing to act on.
|
||||||
func (r Result) Clean() bool { return len(r.Symlinks) == 0 && len(r.Unreadable) == 0 }
|
func (r Result) Clean() bool {
|
||||||
|
return len(r.Symlinks) == 0 && len(r.Unreadable) == 0 && len(r.Special) == 0
|
||||||
|
}
|
||||||
|
|
||||||
// String describes the findings for a log line or alert. Paths are quoted, so
|
// String describes the findings for a log line or alert. Paths are quoted, so
|
||||||
// a name containing a newline cannot blur two findings together.
|
// a name containing a newline cannot blur two findings together.
|
||||||
@@ -59,6 +65,9 @@ func (r Result) String() string {
|
|||||||
if len(r.Symlinks) > 0 {
|
if len(r.Symlinks) > 0 {
|
||||||
parts = append(parts, fmt.Sprintf("%d symlink(s): %s", len(r.Symlinks), q(r.Symlinks)))
|
parts = append(parts, fmt.Sprintf("%d symlink(s): %s", len(r.Symlinks), q(r.Symlinks)))
|
||||||
}
|
}
|
||||||
|
if len(r.Special) > 0 {
|
||||||
|
parts = append(parts, fmt.Sprintf("%d special file(s) (FIFO, socket or device; delete them): %s", len(r.Special), q(r.Special)))
|
||||||
|
}
|
||||||
if len(r.Unreadable) > 0 {
|
if len(r.Unreadable) > 0 {
|
||||||
parts = append(parts, fmt.Sprintf("%d unreadable dir(s): %s", len(r.Unreadable), q(r.Unreadable)))
|
parts = append(parts, fmt.Sprintf("%d unreadable dir(s): %s", len(r.Unreadable), q(r.Unreadable)))
|
||||||
}
|
}
|
||||||
@@ -113,12 +122,16 @@ func Scan(brainDir string) (Result, error) {
|
|||||||
if d.IsDir() && p == rootGit {
|
if d.IsDir() && p == rootGit {
|
||||||
return filepath.SkipDir
|
return filepath.SkipDir
|
||||||
}
|
}
|
||||||
if d.Type()&os.ModeSymlink != 0 {
|
switch {
|
||||||
|
case d.Type()&os.ModeSymlink != 0:
|
||||||
r.Symlinks = append(r.Symlinks, rel)
|
r.Symlinks = append(r.Symlinks, rel)
|
||||||
|
case !d.IsDir() && !d.Type().IsRegular():
|
||||||
|
r.Special = append(r.Special, rel)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
sort.Strings(r.Symlinks)
|
sort.Strings(r.Symlinks)
|
||||||
|
sort.Strings(r.Special)
|
||||||
sort.Strings(r.Unreadable)
|
sort.Strings(r.Unreadable)
|
||||||
if walkErr != nil {
|
if walkErr != nil {
|
||||||
// Only the root's own error reaches here (deeper errors are recorded
|
// Only the root's own error reaches here (deeper errors are recorded
|
||||||
@@ -182,14 +195,20 @@ func Watch(ctx context.Context, brainDir string, interval time.Duration, n Notif
|
|||||||
q.push(fmt.Sprintf("brain symlink tripwire (hyperguild#157): removed %d symlink(s) from the brain dir: %s",
|
q.push(fmt.Sprintf("brain symlink tripwire (hyperguild#157): removed %d symlink(s) from the brain dir: %s",
|
||||||
len(removed), Result{}.quoted(removed)))
|
len(removed), Result{}.quoted(removed)))
|
||||||
}
|
}
|
||||||
key = Result{Symlinks: failed, Unreadable: r.Unreadable}.String()
|
sRemoved, sFailed := removeSpecial(brainDir, r.Special)
|
||||||
|
if len(sRemoved) > 0 {
|
||||||
|
slog.Error("brain symlink tripwire removed special files", "count", len(sRemoved))
|
||||||
|
q.push(fmt.Sprintf("brain symlink tripwire (hyperguild#179): removed %d special file(s) (FIFO, socket or device) from the brain dir: %s",
|
||||||
|
len(sRemoved), Result{}.quoted(sRemoved)))
|
||||||
|
}
|
||||||
|
key = Result{Symlinks: failed, Unreadable: r.Unreadable, Special: sFailed}.String()
|
||||||
}
|
}
|
||||||
if key != reported {
|
if key != reported {
|
||||||
reported = key
|
reported = key
|
||||||
if key != "" {
|
if key != "" {
|
||||||
slog.Error("brain symlink tripwire", "finding", key)
|
slog.Error("brain symlink tripwire", "finding", key)
|
||||||
q.push("brain symlink tripwire (hyperguild#157): " + key +
|
q.push("brain symlink tripwire (hyperguild#157, #179): " + key +
|
||||||
". Readers may follow symlinks out of the brain; fix them.")
|
". Symlinks can lead readers out of the brain and special files can hang them; remove them.")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -322,6 +341,40 @@ func permanent(err error) bool {
|
|||||||
return errors.As(err, &p) && p.Permanent()
|
return errors.As(err, &p) && p.Permanent()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// removeSpecial unlinks each brain-relative FIFO, socket or device through
|
||||||
|
// an os.Root on the brain dir, after re-checking that it is still one: a
|
||||||
|
// path that is a regular file, a directory or a symlink at the re-check is
|
||||||
|
// left alone and reported as failed. The re-check narrows, but cannot close,
|
||||||
|
// the window before the unlink (Linux has no conditional unlink); the same
|
||||||
|
// window exists in removeSymlinks, and only a local writer can use it.
|
||||||
|
func removeSpecial(brainDir string, rels []string) (removed, failed []string) {
|
||||||
|
if len(rels) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
resolved, err := filepath.EvalSymlinks(brainDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, rels
|
||||||
|
}
|
||||||
|
root, err := os.OpenRoot(resolved)
|
||||||
|
if err != nil {
|
||||||
|
return nil, rels
|
||||||
|
}
|
||||||
|
defer func() { _ = root.Close() }()
|
||||||
|
for _, rel := range rels {
|
||||||
|
fi, err := root.Lstat(rel)
|
||||||
|
if err != nil || fi.IsDir() || fi.Mode().IsRegular() || fi.Mode()&os.ModeSymlink != 0 {
|
||||||
|
failed = append(failed, rel)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := root.Remove(rel); err != nil {
|
||||||
|
failed = append(failed, rel)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
removed = append(removed, rel)
|
||||||
|
}
|
||||||
|
return removed, failed
|
||||||
|
}
|
||||||
|
|
||||||
// removeSymlinks unlinks each brain-relative symlink through an os.Root on
|
// removeSymlinks unlinks each brain-relative symlink through an os.Root on
|
||||||
// the brain dir, after re-checking it is still a symlink. Removing a symlink
|
// the brain dir, after re-checking it is still a symlink. Removing a symlink
|
||||||
// never touches its target. It returns what was removed and what was not.
|
// never touches its target. It returns what was removed and what was not.
|
||||||
|
|||||||
@@ -381,18 +381,19 @@ func TestWatchQueueIsBoundedDuringAnOutage(t *testing.T) {
|
|||||||
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 2*time.Millisecond)
|
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 2*time.Millisecond)
|
||||||
}
|
}
|
||||||
n.down.Store(false)
|
n.down.Store(false)
|
||||||
require.Eventually(t, func() bool { return n.count() >= 1 }, 2*time.Second, 5*time.Millisecond)
|
// Synchronise on the summary being delivered (hyperguild#193: a fixed
|
||||||
time.Sleep(100 * time.Millisecond)
|
// sleep flaked under load); once it is, the queue has been flushed.
|
||||||
assert.LessOrEqual(t, n.count(), maxPending+1, "queued alerts plus at most one suppression summary")
|
require.Eventually(t, func() bool {
|
||||||
n.mu.Lock()
|
n.mu.Lock()
|
||||||
defer n.mu.Unlock()
|
defer n.mu.Unlock()
|
||||||
found := false
|
for _, m := range n.msgs {
|
||||||
for _, m := range n.msgs {
|
if strings.Contains(m, "suppressed") {
|
||||||
if strings.Contains(m, "suppressed") {
|
return true
|
||||||
found = true
|
}
|
||||||
}
|
}
|
||||||
}
|
return false
|
||||||
assert.True(t, found, "suppressed alerts are counted in a summary")
|
}, 10*time.Second, 5*time.Millisecond, "suppressed alerts are counted in a summary")
|
||||||
|
assert.LessOrEqual(t, n.count(), maxPending+1, "queued alerts plus at most one suppression summary")
|
||||||
}
|
}
|
||||||
|
|
||||||
// Within the bound, every alert queued during an outage is delivered after
|
// Within the bound, every alert queued during an outage is delivered after
|
||||||
@@ -411,11 +412,21 @@ func TestWatchDeliversEveryQueuedAlertAfterAShortOutage(t *testing.T) {
|
|||||||
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 2*time.Millisecond)
|
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 2*time.Millisecond)
|
||||||
}
|
}
|
||||||
n.down.Store(false)
|
n.down.Store(false)
|
||||||
require.Eventually(t, func() bool { return n.count() == 3 }, 2*time.Second, 5*time.Millisecond)
|
require.Eventually(t, func() bool { return n.count() >= 3 }, 10*time.Second, 5*time.Millisecond)
|
||||||
n.mu.Lock()
|
n.mu.Lock()
|
||||||
defer n.mu.Unlock()
|
defer n.mu.Unlock()
|
||||||
for i, name := range []string{"qa", "qb", "qc"} {
|
// Each removal delivered exactly once, in any order: the outage ends
|
||||||
assert.Contains(t, n.msgs[i], strconv.Quote(name))
|
// asynchronously, possibly mid-flush, so a later item can send before an
|
||||||
|
// earlier one (hyperguild#193). Order is not what this test is about.
|
||||||
|
assert.Len(t, n.msgs, 3)
|
||||||
|
for _, name := range []string{"qa", "qb", "qc"} {
|
||||||
|
hits := 0
|
||||||
|
for _, m := range n.msgs {
|
||||||
|
if strings.Contains(m, strconv.Quote(name)) {
|
||||||
|
hits++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assert.Equal(t, 1, hits, "%s delivered exactly once", name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -457,29 +468,30 @@ func TestWatchAccountsForEveryRemovalDuringAnOutage(t *testing.T) {
|
|||||||
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 1*time.Millisecond)
|
require.Eventually(t, func() bool { _, err := os.Lstat(link); return os.IsNotExist(err) }, 2*time.Second, 1*time.Millisecond)
|
||||||
}
|
}
|
||||||
n.down.Store(false)
|
n.down.Store(false)
|
||||||
require.Eventually(t, func() bool {
|
// Synchronise on the record being complete, not on a fixed sleep after
|
||||||
|
// the first summary (hyperguild#193): sum every "removed N" alert and
|
||||||
|
// every suppression count until they account for all plants.
|
||||||
|
account := func() (delivered, counted int) {
|
||||||
n.mu.Lock()
|
n.mu.Lock()
|
||||||
defer n.mu.Unlock()
|
defer n.mu.Unlock()
|
||||||
for _, m := range n.msgs {
|
for _, m := range n.msgs {
|
||||||
if strings.Contains(m, "suppressed") {
|
var k int
|
||||||
return true
|
if i := strings.Index(m, "removed "); i >= 0 && strings.Contains(m, "symlink(s)") {
|
||||||
|
if _, err := fmt.Sscanf(m[i+len("removed "):], "%d", &k); err == nil {
|
||||||
|
delivered += k
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if i := strings.Index(m, "): "); i >= 0 && strings.Contains(m, "suppressed") {
|
||||||
|
if _, err := fmt.Sscanf(m[i+3:], "%d", &k); err == nil {
|
||||||
|
counted += k
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return false
|
return delivered, counted
|
||||||
}, 2*time.Second, 5*time.Millisecond)
|
|
||||||
time.Sleep(50 * time.Millisecond)
|
|
||||||
n.mu.Lock()
|
|
||||||
defer n.mu.Unlock()
|
|
||||||
delivered, counted := 0, 0
|
|
||||||
for _, m := range n.msgs {
|
|
||||||
if strings.Contains(m, "removed 1 symlink(s)") {
|
|
||||||
delivered++
|
|
||||||
}
|
|
||||||
if i := strings.Index(m, "): "); i >= 0 && strings.Contains(m, "suppressed") {
|
|
||||||
_, _ = fmt.Sscanf(m[i+3:], "%d", &counted)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
assert.Equal(t, plants, delivered+counted, "delivered %d + counted %d must equal %d removals", delivered, counted, plants)
|
require.Eventually(t, func() bool { d, c := account(); return d+c >= plants }, 10*time.Second, 5*time.Millisecond)
|
||||||
|
d, c := account()
|
||||||
|
assert.Equal(t, plants, d+c, "delivered %d + counted %d must equal %d removals", d, c, plants)
|
||||||
}
|
}
|
||||||
|
|
||||||
// A poison message is still dropped (others send while it fails), and the
|
// A poison message is still dropped (others send while it fails), and the
|
||||||
|
|||||||
Reference in New Issue
Block a user