Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
442557ca31 | ||
|
|
42578c21b9 | ||
|
|
1d67e16c0d | ||
|
|
4f75279ceb | ||
|
|
110e8b8f75 | ||
|
|
0f932d8f77 | ||
|
|
062574dad8 | ||
|
|
f7fe00cecf |
@@ -78,27 +78,6 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
when a human reviewer outside the project is required. Document the reason in
|
when a human reviewer outside the project is required. Document the reason in
|
||||||
PROJECT.md.
|
PROJECT.md.
|
||||||
|
|
||||||
**Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
|
|
||||||
deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
|
|
||||||
invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
|
|
||||||
`litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
|
|
||||||
PR would buy a review of a state that was never deployed.
|
|
||||||
|
|
||||||
There, `risk/medium`+ work stays on `main` and instead requires a **recorded
|
|
||||||
adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
|
|
||||||
reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
|
|
||||||
thread.** No verdict recorded means the lane was not used, and saying "the gate was
|
|
||||||
green" is not the same claim.
|
|
||||||
|
|
||||||
Be honest about what this is: a recorded *independent* pass, not four-eyes in the
|
|
||||||
sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
|
|
||||||
is an agent. It is more than the repo had and less than the model it mirrors; do not
|
|
||||||
describe it as the stronger property. (infra#470, decided 2026-09-29.)
|
|
||||||
|
|
||||||
This lane depends on the `risk/` label being right. A label that triggers a heavier
|
|
||||||
lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
|
|
||||||
lane silently does not apply.
|
|
||||||
|
|
||||||
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
||||||
the code is not the end of the task; capturing it is. Run this unprompted:
|
the code is not the end of the task; capturing it is. Run this unprompted:
|
||||||
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
||||||
@@ -127,43 +106,6 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
revisit Y". Those are exactly the items that need a locator. If it is worth
|
revisit Y". Those are exactly the items that need a locator. If it is worth
|
||||||
saying in a summary, it is worth writing where it can be found again without you.
|
saying in a summary, it is worth writing where it can be found again without you.
|
||||||
|
|
||||||
7. **Finish what's closable — only involve me for a genuine call.** If a
|
|
||||||
tracker issue, PR, or checklist item is demonstrably resolved — the fix
|
|
||||||
landed, the state changed, you verified it live — close/merge/check it off
|
|
||||||
in the same turn. Don't leave it open "for the record" or park it waiting
|
|
||||||
on me to confirm something that already happened; that's bookkeeping, not
|
|
||||||
a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
|
|
||||||
comment, or otherwise pausing) for when it's genuinely mine to call:
|
|
||||||
irreversible or prod/host-impacting actions, a real judgment call with no
|
|
||||||
clearly-better option, ambiguous intent, or anything secret-writes/
|
|
||||||
destructive-git already gate. Before claiming something is closable,
|
|
||||||
verify it live (re-check the issue, re-run the command) rather than
|
|
||||||
trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
|
|
||||||
|
|
||||||
8. **Control code needs an adversary, not a checklist.** Code whose job is to
|
|
||||||
refuse, deny, gate or contain is not adequately verified by a green gate.
|
|
||||||
Rule 5 makes CI the quality gate and that holds for ordinary work — but a
|
|
||||||
passing suite only asserts what its author already thought of, and for a
|
|
||||||
control the author is precisely the person whose imagination just failed.
|
|
||||||
Measured on `dispatch#75`: five review rounds, five real escapes verified by
|
|
||||||
execution, and **CI was green on every one of them** while `git rm
|
|
||||||
././.dispatch-allow` deleted the file the control existed to protect. So
|
|
||||||
route this work through a review with authority to block, instructed to
|
|
||||||
verify by execution rather than by reading the implementation. Expect to be
|
|
||||||
blocked more than once; that is the control working, not the reviewer being
|
|
||||||
difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
|
|
||||||
practices live in the `pr-gatekeeper` agent.)
|
|
||||||
|
|
||||||
9. **A test's name must not claim more than its assertions prove.** Name a test
|
|
||||||
after the mechanism it observes, not the goal it serves.
|
|
||||||
`DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
|
|
||||||
support. This is not pedantry about naming — when name and assertion
|
|
||||||
diverge, the name wins in the reader's head and the gap becomes invisible,
|
|
||||||
which is what lets the defect survive review. Found three times in one PR,
|
|
||||||
twice in tests written to fix the previous instance of it; each passed while
|
|
||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
|
||||||
you do not yet know what the test observes.
|
|
||||||
|
|
||||||
## CAD sprint patterns
|
## CAD sprint patterns
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||||
@@ -307,13 +249,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
|
|||||||
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
|
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
|
||||||
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
||||||
|
|
||||||
- **Scoping — the brain holds no confidential client content.** Nothing on the read path
|
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
|
||||||
filters by client: every token holder can read every wing, and `wing` on `brain_query`
|
|
||||||
is an optional argument, not an access control. Client confidentiality is enforced only
|
|
||||||
at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
|
|
||||||
client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
|
|
||||||
decided 2026-10-01. This line previously claimed client projects were filtered; they
|
|
||||||
were not.)
|
|
||||||
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
||||||
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
||||||
on the koala k3s cluster; don't hardcode local-only model names into the
|
on the koala k3s cluster; don't hardcode local-only model names into the
|
||||||
@@ -383,20 +319,8 @@ Workflow:
|
|||||||
and missing-untracked adapters). A drift fails the check with a
|
and missing-untracked adapters). A drift fails the check with a
|
||||||
message telling you to stage the regenerated files.
|
message telling you to stage the regenerated files.
|
||||||
|
|
||||||
Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
|
Behavior rules in this file and per-project rules in `PROJECT.md` apply
|
||||||
host and every harness.
|
unconditionally on every host, every harness.
|
||||||
|
|
||||||
**Where a project rule contradicts a rule here, the project rule governs — and it
|
|
||||||
must state its reason.** Both halves matter: the project file wins because it knows
|
|
||||||
something this file cannot (infra's Flux constraint is real and not visible from
|
|
||||||
here), and it must say why because a bare override is indistinguishable from drift.
|
|
||||||
|
|
||||||
This replaces the previous wording, which said rules here and in `PROJECT.md` both
|
|
||||||
apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
|
|
||||||
unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
|
|
||||||
at the same time (infra#470). A standard can hold two individually sensible clauses
|
|
||||||
that are jointly unsatisfiable, and survive for as long as everyone reads it
|
|
||||||
charitably — the same failure the Inception Sprint Oath's G6 had.
|
|
||||||
|
|
||||||
## Engineering Skills
|
## Engineering Skills
|
||||||
|
|
||||||
|
|||||||
@@ -73,27 +73,6 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
when a human reviewer outside the project is required. Document the reason in
|
when a human reviewer outside the project is required. Document the reason in
|
||||||
PROJECT.md.
|
PROJECT.md.
|
||||||
|
|
||||||
**Exception — a GitOps repo where a branch cannot be reconciled.** In a repo whose
|
|
||||||
deployment watches one branch (`mathias/infra`: Flux watches `main`), a branch is
|
|
||||||
invisible to the thing that applies it. A `risk/medium` change to `alerts.yaml` or
|
|
||||||
`litellm-config.yaml` can be linted on a branch but never *exercised* there, so a
|
|
||||||
PR would buy a review of a state that was never deployed.
|
|
||||||
|
|
||||||
There, `risk/medium`+ work stays on `main` and instead requires a **recorded
|
|
||||||
adversarial review pass before the push** — `skills:pr-gatekeeper` or equivalent,
|
|
||||||
reviewing the staged diff. **The reviewer's verdict goes in the issue, not a PR
|
|
||||||
thread.** No verdict recorded means the lane was not used, and saying "the gate was
|
|
||||||
green" is not the same claim.
|
|
||||||
|
|
||||||
Be honest about what this is: a recorded *independent* pass, not four-eyes in the
|
|
||||||
sense `assessor-loop` §4.1 means for a regulated organisation, because the reviewer
|
|
||||||
is an agent. It is more than the repo had and less than the model it mirrors; do not
|
|
||||||
describe it as the stronger property. (infra#470, decided 2026-09-29.)
|
|
||||||
|
|
||||||
This lane depends on the `risk/` label being right. A label that triggers a heavier
|
|
||||||
lane is load-bearing, not cosmetic — if work that belongs in it is unlabelled, the
|
|
||||||
lane silently does not apply.
|
|
||||||
|
|
||||||
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
|
||||||
the code is not the end of the task; capturing it is. Run this unprompted:
|
the code is not the end of the task; capturing it is. Run this unprompted:
|
||||||
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
|
||||||
@@ -122,43 +101,6 @@ These rules apply to every task across every project, regardless of harness.
|
|||||||
revisit Y". Those are exactly the items that need a locator. If it is worth
|
revisit Y". Those are exactly the items that need a locator. If it is worth
|
||||||
saying in a summary, it is worth writing where it can be found again without you.
|
saying in a summary, it is worth writing where it can be found again without you.
|
||||||
|
|
||||||
7. **Finish what's closable — only involve me for a genuine call.** If a
|
|
||||||
tracker issue, PR, or checklist item is demonstrably resolved — the fix
|
|
||||||
landed, the state changed, you verified it live — close/merge/check it off
|
|
||||||
in the same turn. Don't leave it open "for the record" or park it waiting
|
|
||||||
on me to confirm something that already happened; that's bookkeeping, not
|
|
||||||
a decision. Reserve asking me (`AskUserQuestion`, a `DECISION NEEDED`
|
|
||||||
comment, or otherwise pausing) for when it's genuinely mine to call:
|
|
||||||
irreversible or prod/host-impacting actions, a real judgment call with no
|
|
||||||
clearly-better option, ambiguous intent, or anything secret-writes/
|
|
||||||
destructive-git already gate. Before claiming something is closable,
|
|
||||||
verify it live (re-check the issue, re-run the command) rather than
|
|
||||||
trusting your own earlier summary (brain: `filed-issue-not-same-as-fixed`).
|
|
||||||
|
|
||||||
8. **Control code needs an adversary, not a checklist.** Code whose job is to
|
|
||||||
refuse, deny, gate or contain is not adequately verified by a green gate.
|
|
||||||
Rule 5 makes CI the quality gate and that holds for ordinary work — but a
|
|
||||||
passing suite only asserts what its author already thought of, and for a
|
|
||||||
control the author is precisely the person whose imagination just failed.
|
|
||||||
Measured on `dispatch#75`: five review rounds, five real escapes verified by
|
|
||||||
execution, and **CI was green on every one of them** while `git rm
|
|
||||||
././.dispatch-allow` deleted the file the control existed to protect. So
|
|
||||||
route this work through a review with authority to block, instructed to
|
|
||||||
verify by execution rather than by reading the implementation. Expect to be
|
|
||||||
blocked more than once; that is the control working, not the reviewer being
|
|
||||||
difficult. (brain: `a-green-gate-is-not-evidence-for-control-code`; the
|
|
||||||
practices live in the `pr-gatekeeper` agent.)
|
|
||||||
|
|
||||||
9. **A test's name must not claim more than its assertions prove.** Name a test
|
|
||||||
after the mechanism it observes, not the goal it serves.
|
|
||||||
`DoesNotInheritEnv` is honest; `IsContained` is a claim the test cannot
|
|
||||||
support. This is not pedantry about naming — when name and assertion
|
|
||||||
diverge, the name wins in the reader's head and the gap becomes invisible,
|
|
||||||
which is what lets the defect survive review. Found three times in one PR,
|
|
||||||
twice in tests written to fix the previous instance of it; each passed while
|
|
||||||
the thing it was named for was exploitable. If you cannot name the mechanism,
|
|
||||||
you do not yet know what the test observes.
|
|
||||||
|
|
||||||
## CAD sprint patterns
|
## CAD sprint patterns
|
||||||
|
|
||||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||||
@@ -302,13 +244,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
|
|||||||
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
|
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
|
||||||
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
|
||||||
|
|
||||||
- **Scoping — the brain holds no confidential client content.** Nothing on the read path
|
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
|
||||||
filters by client: every token holder can read every wing, and `wing` on `brain_query`
|
|
||||||
is an optional argument, not an access control. Client confidentiality is enforced only
|
|
||||||
at write time (`capture`'s I1 gate refuses internal/confidential). So never `brain_write`
|
|
||||||
client material; generalise it first or keep it in the client's own repo. (hyperguild#144,
|
|
||||||
decided 2026-10-01. This line previously claimed client projects were filtered; they
|
|
||||||
were not.)
|
|
||||||
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
|
||||||
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
|
||||||
on the koala k3s cluster; don't hardcode local-only model names into the
|
on the koala k3s cluster; don't hardcode local-only model names into the
|
||||||
@@ -378,20 +314,8 @@ Workflow:
|
|||||||
and missing-untracked adapters). A drift fails the check with a
|
and missing-untracked adapters). A drift fails the check with a
|
||||||
message telling you to stage the regenerated files.
|
message telling you to stage the regenerated files.
|
||||||
|
|
||||||
Behavior rules in this file and per-project rules in `PROJECT.md` apply on every
|
Behavior rules in this file and per-project rules in `PROJECT.md` apply
|
||||||
host and every harness.
|
unconditionally on every host, every harness.
|
||||||
|
|
||||||
**Where a project rule contradicts a rule here, the project rule governs — and it
|
|
||||||
must state its reason.** Both halves matter: the project file wins because it knows
|
|
||||||
something this file cannot (infra's Flux constraint is real and not visible from
|
|
||||||
here), and it must say why because a bare override is indistinguishable from drift.
|
|
||||||
|
|
||||||
This replaces the previous wording, which said rules here and in `PROJECT.md` both
|
|
||||||
apply *"unconditionally"*. That was not a tiebreak but a contradiction, and it went
|
|
||||||
unnoticed until rule 5 and infra's "never create feature branches" had to be obeyed
|
|
||||||
at the same time (infra#470). A standard can hold two individually sensible clauses
|
|
||||||
that are jointly unsatisfiable, and survive for as long as everyone reads it
|
|
||||||
charitably — the same failure the Inception Sprint Oath's G6 had.
|
|
||||||
|
|
||||||
## Engineering Skills
|
## Engineering Skills
|
||||||
|
|
||||||
|
|||||||
@@ -147,37 +147,6 @@ func envInt(key string, fallback int) int {
|
|||||||
// buildNotifier returns the ntfy notifier, or a nil interface when
|
// buildNotifier returns the ntfy notifier, or a nil interface when
|
||||||
// BRAIN_NTFY_URL is unset. A nil *interface* (not a typed-nil) is required so
|
// BRAIN_NTFY_URL is unset. A nil *interface* (not a typed-nil) is required so
|
||||||
// DegradingSink/Reconcile and the governance observer skip it cleanly.
|
// DegradingSink/Reconcile and the governance observer skip it cleanly.
|
||||||
// alertingWarnings names every alert that will not fire under the current
|
|
||||||
// configuration (#107).
|
|
||||||
//
|
|
||||||
// buildNotifier returns a nil interface when BRAIN_NTFY_URL is unset, and every
|
|
||||||
// consumer skips a nil notifier silently — right for an unconfigured homelab,
|
|
||||||
// wrong for an alert whose entire value is that someone finds out. The cost of
|
|
||||||
// the silence is that "configured and working" and "never configured" are
|
|
||||||
// byte-identical in behaviour: no error, no warning, only a message that never
|
|
||||||
// arrives, which nobody notices because nobody is waiting for an alert they
|
|
||||||
// have never seen. Two alerts were dark in production for exactly that reason
|
|
||||||
// (#100), found by firing a real event rather than by reading anything.
|
|
||||||
//
|
|
||||||
// WARN rather than fatal, deliberately. The brain sibling rule
|
|
||||||
// (`empty-credential-must-refuse-to-start-not-switch-modes`) says an empty
|
|
||||||
// credential must be a startup error, never a silent mode switch — but that is
|
|
||||||
// about a control being removed. Here observability is degraded, not a gate
|
|
||||||
// bypassed, and refusing to boot would break the unconfigured-homelab case the
|
|
||||||
// nil notifier exists to support. What the rule does demand, and what this
|
|
||||||
// provides, is that the mode switch stops being silent.
|
|
||||||
//
|
|
||||||
// Pure and env-injected so the message is testable without a running server:
|
|
||||||
// the reason this gap survived is that nothing about it was observable.
|
|
||||||
func alertingWarnings(getenv func(string) string) []string {
|
|
||||||
var out []string
|
|
||||||
if getenv("BRAIN_NTFY_URL") == "" {
|
|
||||||
out = append(out, "alerting DISABLED: BRAIN_NTFY_URL unset — "+
|
|
||||||
"brain_edit_root governance rewrites and capture audit degradation will not notify")
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func buildNotifier() audit.Notifier {
|
func buildNotifier() audit.Notifier {
|
||||||
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
||||||
return n
|
return n
|
||||||
@@ -234,10 +203,6 @@ func systemHostname() string {
|
|||||||
func main() {
|
func main() {
|
||||||
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
|
||||||
|
|
||||||
for _, w := range alertingWarnings(os.Getenv) {
|
|
||||||
logger.Warn(w)
|
|
||||||
}
|
|
||||||
|
|
||||||
brainDir := envOr("INGEST_BRAIN_DIR", "../brain")
|
brainDir := envOr("INGEST_BRAIN_DIR", "../brain")
|
||||||
port := envOr("INGEST_PORT", "3300")
|
port := envOr("INGEST_PORT", "3300")
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ import (
|
|||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
@@ -37,42 +36,3 @@ func TestClaudeSink_IngestWritesToNonIndexedArchiveNotWiki(t *testing.T) {
|
|||||||
_, err = os.Stat(filepath.Join(dir, "wiki", "claude-sessions"))
|
_, err = os.Stat(filepath.Join(dir, "wiki", "claude-sessions"))
|
||||||
assert.True(t, os.IsNotExist(err), "raw transcripts must never land under wiki/ (ai-sessions#10 — BM25 pollution)")
|
assert.True(t, os.IsNotExist(err), "raw transcripts must never land under wiki/ (ai-sessions#10 — BM25 pollution)")
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- #107: an unconfigured alert must announce itself ----------------------
|
|
||||||
|
|
||||||
func TestAlertingWarnings(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
env map[string]string
|
|
||||||
want []string // substrings that must appear across the warnings
|
|
||||||
none bool // true = expect no warnings at all
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
// The failure this exists to prevent: two alerts dark in
|
|
||||||
// production, discovered only by firing a real event and watching
|
|
||||||
// nothing arrive (#100).
|
|
||||||
name: "notifier unconfigured names what will not fire",
|
|
||||||
env: map[string]string{},
|
|
||||||
want: []string{"BRAIN_NTFY_URL", "brain_edit_root", "audit"},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "configured notifier is silent",
|
|
||||||
env: map[string]string{"BRAIN_NTFY_URL": "http://ntfy/brain-alerts"},
|
|
||||||
none: true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
got := alertingWarnings(func(k string) string { return tt.env[k] })
|
|
||||||
if tt.none {
|
|
||||||
assert.Empty(t, got)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
joined := strings.Join(got, " ")
|
|
||||||
for _, w := range tt.want {
|
|
||||||
assert.Contains(t, joined, w)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,12 +1,6 @@
|
|||||||
package api
|
package api
|
||||||
|
|
||||||
import (
|
import "strings"
|
||||||
"strings"
|
|
||||||
|
|
||||||
// Aliased: this package's own frontmatter type (below) would collide
|
|
||||||
// with the imported package's name otherwise.
|
|
||||||
sharedfm "github.com/mathiasbq/hyperguild/ingestion/internal/frontmatter"
|
|
||||||
)
|
|
||||||
|
|
||||||
// frontmatter is an ordered, line-preserving view of a note's YAML
|
// frontmatter is an ordered, line-preserving view of a note's YAML
|
||||||
// frontmatter block. It deliberately avoids a full YAML round-trip: the
|
// frontmatter block. It deliberately avoids a full YAML round-trip: the
|
||||||
@@ -31,26 +25,19 @@ type fmLine struct {
|
|||||||
// frontmatter block is recognised only when the file opens with a `---`
|
// frontmatter block is recognised only when the file opens with a `---`
|
||||||
// fence and a closing `---` fence follows. Otherwise the whole input is
|
// fence and a closing `---` fence follows. Otherwise the whole input is
|
||||||
// the body and the returned frontmatter is empty.
|
// the body and the returned frontmatter is empty.
|
||||||
//
|
|
||||||
// Fence detection itself is frontmatter.Split (hyperguild#138) — this
|
|
||||||
// package's own contribution is purely the ordered, line-preserving fmLine
|
|
||||||
// shape below, which a shared parser has no business owning since it's
|
|
||||||
// specific to how brain_update does a surgical, byte-round-trip edit.
|
|
||||||
func parseFrontmatter(src string) (frontmatter, string) {
|
func parseFrontmatter(src string) (frontmatter, string) {
|
||||||
var fm frontmatter
|
var fm frontmatter
|
||||||
blockBytes, restBytes, ok := sharedfm.Split([]byte(src))
|
if !strings.HasPrefix(src, "---\n") {
|
||||||
if !ok {
|
|
||||||
return fm, src
|
return fm, src
|
||||||
}
|
}
|
||||||
// sharedfm.Split's block/rest boundary sits one byte earlier than this
|
rest := src[len("---\n"):]
|
||||||
// package's always has: block keeps the "\n" immediately before the
|
end := strings.Index(rest, "\n---\n")
|
||||||
// closing fence, and rest keeps the "\n" immediately after it (both by
|
if end < 0 {
|
||||||
// design, for Fix's own reconstruction). This package's block/body
|
// Opening fence with no closing fence — treat as bodyless content.
|
||||||
// never included either — TrimSuffix/TrimPrefix restore that exact
|
return fm, src
|
||||||
// boundary. Split guarantees ok=true only when "\n---\n" matched, so
|
}
|
||||||
// both trims are always exactly one byte, never ambiguous.
|
block := rest[:end]
|
||||||
block := strings.TrimSuffix(string(blockBytes), "\n")
|
body := rest[end+len("\n---\n"):]
|
||||||
body := strings.TrimPrefix(string(restBytes), "\n")
|
|
||||||
|
|
||||||
for _, line := range strings.Split(block, "\n") {
|
for _, line := range strings.Split(block, "\n") {
|
||||||
// An indented line is a continuation (a YAML block-list item, a
|
// An indented line is a continuation (a YAML block-list item, a
|
||||||
@@ -95,18 +82,6 @@ func (f *frontmatter) set(key, value string) {
|
|||||||
f.lines = append(f.lines, fmLine{key: key, value: value})
|
f.lines = append(f.lines, fmLine{key: key, value: value})
|
||||||
}
|
}
|
||||||
|
|
||||||
// del removes every top-level line for key. Continuation lines of a
|
|
||||||
// multi-line value are not tracked per key, so del is only for scalars.
|
|
||||||
func (f *frontmatter) del(key string) {
|
|
||||||
kept := f.lines[:0]
|
|
||||||
for _, l := range f.lines {
|
|
||||||
if l.key != key {
|
|
||||||
kept = append(kept, l)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
f.lines = kept
|
|
||||||
}
|
|
||||||
|
|
||||||
// render serialises the frontmatter back into a `---`-fenced block. An
|
// render serialises the frontmatter back into a `---`-fenced block. An
|
||||||
// empty frontmatter renders to the empty string so bodies without a
|
// empty frontmatter renders to the empty string so bodies without a
|
||||||
// header stay header-less.
|
// header stay header-less.
|
||||||
|
|||||||
@@ -118,20 +118,6 @@ type WriteNoteOptions struct {
|
|||||||
Wing string
|
Wing string
|
||||||
Hall string
|
Hall string
|
||||||
SourceType string // "internal" marks a first-party observation (e.g. claudewatcher) that needs no external citation
|
SourceType string // "internal" marks a first-party observation (e.g. claudewatcher) that needs no external citation
|
||||||
// Origin/WrittenBy are server-owned provenance (hyperguild#145): any
|
|
||||||
// origin/written_by in Content's own frontmatter is discarded, so a
|
|
||||||
// caller cannot claim a more trusted origin than it authenticated as.
|
|
||||||
Origin string
|
|
||||||
WrittenBy string
|
|
||||||
}
|
|
||||||
|
|
||||||
// provenanceOrigin normalises an origin for the frontmatter: absence must
|
|
||||||
// never read as trusted, so empty becomes "unknown".
|
|
||||||
func provenanceOrigin(o string) string {
|
|
||||||
if o == "" {
|
|
||||||
return "unknown"
|
|
||||||
}
|
|
||||||
return o
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// WriteNote writes a markdown note into the brain. Returns the path
|
// WriteNote writes a markdown note into the brain. Returns the path
|
||||||
@@ -194,11 +180,6 @@ func writeHallNote(brainDir string, opts WriteNoteOptions) (string, error) {
|
|||||||
fmt.Fprintf(&fm, "hall: %s\n", opts.Hall)
|
fmt.Fprintf(&fm, "hall: %s\n", opts.Hall)
|
||||||
fmt.Fprintf(&fm, "created_at: %s\n", time.Now().UTC().Format(time.RFC3339))
|
fmt.Fprintf(&fm, "created_at: %s\n", time.Now().UTC().Format(time.RFC3339))
|
||||||
emitted["wing"], emitted["hall"], emitted["created_at"] = true, true, true
|
emitted["wing"], emitted["hall"], emitted["created_at"] = true, true, true
|
||||||
fmt.Fprintf(&fm, "origin: %s\n", provenanceOrigin(opts.Origin))
|
|
||||||
emitted["origin"], emitted["written_by"] = true, true
|
|
||||||
if opts.WrittenBy != "" {
|
|
||||||
fmt.Fprintf(&fm, "written_by: %s\n", opts.WrittenBy)
|
|
||||||
}
|
|
||||||
|
|
||||||
// writeField merges one key: opts.Content's own value (if the note already
|
// writeField merges one key: opts.Content's own value (if the note already
|
||||||
// carries this field in its own frontmatter) always wins over the fallback,
|
// carries this field in its own frontmatter) always wins over the fallback,
|
||||||
@@ -314,16 +295,7 @@ func (h *Handler) Write(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeError(w, http.StatusBadRequest, "invalid JSON")
|
writeError(w, http.StatusBadRequest, "invalid JSON")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
// Field by field, not a struct conversion: provenance is server-owned and
|
relPath, err := WriteNote(h.brainDir, WriteNoteOptions(req))
|
||||||
// this door carries no principal, so it stamps origin: unknown
|
|
||||||
// (hyperguild#145). A JSON body must never be able to set it.
|
|
||||||
relPath, err := WriteNote(h.brainDir, WriteNoteOptions{
|
|
||||||
Content: req.Content,
|
|
||||||
Filename: req.Filename,
|
|
||||||
Wing: req.Wing,
|
|
||||||
Hall: req.Hall,
|
|
||||||
SourceType: req.SourceType,
|
|
||||||
})
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.logger.Error("write failed", "err", err)
|
h.logger.Error("write failed", "err", err)
|
||||||
writeError(w, http.StatusBadRequest, err.Error())
|
writeError(w, http.StatusBadRequest, err.Error())
|
||||||
|
|||||||
@@ -46,11 +46,6 @@ type UpdateNoteOptions struct {
|
|||||||
Slug string
|
Slug string
|
||||||
Content string // new full body (whole-note replace)
|
Content string // new full body (whole-note replace)
|
||||||
Reason string // optional; stamped as supersede_reason
|
Reason string // optional; stamped as supersede_reason
|
||||||
// Origin/WrittenBy re-stamp provenance to the updating principal: the
|
|
||||||
// body is replaced wholesale, so the last writer is the author of what
|
|
||||||
// a reader sees (hyperguild#145). Never taken from Content's fragment.
|
|
||||||
Origin string
|
|
||||||
WrittenBy string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// UpdateNote supersedes an existing note in place. It replaces the body
|
// UpdateNote supersedes an existing note in place. It replaces the body
|
||||||
@@ -115,31 +110,11 @@ func UpdateNote(brainDir string, opts UpdateNoteOptions) (relPath, contentHash,
|
|||||||
|
|
||||||
fm, _ := parseFrontmatter(string(prior))
|
fm, _ := parseFrontmatter(string(prior))
|
||||||
for _, l := range contentFM.lines {
|
for _, l := range contentFM.lines {
|
||||||
// updated_at/supersedes/supersede_reason are this function's own
|
if l.key != "" {
|
||||||
// output, not something a caller's fragment sets — updated_at and
|
|
||||||
// supersedes are already unconditionally re-set below regardless of
|
|
||||||
// what the loop does with them, but supersede_reason is only
|
|
||||||
// re-set when opts.Reason is non-empty, so without this skip a
|
|
||||||
// caller could set it via its own fragment and have it survive
|
|
||||||
// even when Reason is empty, bypassing the one intended entry
|
|
||||||
// point for that field (hyperguild#125, found in #111's review).
|
|
||||||
// Skipping all three here rather than relying on the unconditional
|
|
||||||
// overwrites below to save the other two keeps the invariant
|
|
||||||
// explicit at the one place it can be silently broken by a future
|
|
||||||
// reordering, not implicit in call order.
|
|
||||||
if l.key == "" || l.key == "updated_at" || l.key == "supersedes" || l.key == "supersede_reason" ||
|
|
||||||
l.key == "origin" || l.key == "written_by" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
fm.set(l.key, l.value)
|
fm.set(l.key, l.value)
|
||||||
}
|
}
|
||||||
fm.set("updated_at", time.Now().UTC().Format(time.RFC3339))
|
|
||||||
fm.set("origin", provenanceOrigin(opts.Origin))
|
|
||||||
if opts.WrittenBy != "" {
|
|
||||||
fm.set("written_by", opts.WrittenBy)
|
|
||||||
} else {
|
|
||||||
fm.del("written_by")
|
|
||||||
}
|
}
|
||||||
|
fm.set("updated_at", time.Now().UTC().Format(time.RFC3339))
|
||||||
fm.set("supersedes", priorHash)
|
fm.set("supersedes", priorHash)
|
||||||
if opts.Reason != "" {
|
if opts.Reason != "" {
|
||||||
fm.set("supersede_reason", opts.Reason)
|
fm.set("supersede_reason", opts.Reason)
|
||||||
|
|||||||
@@ -231,45 +231,6 @@ func TestUpdateThenReadRoundTripsHash(t *testing.T) {
|
|||||||
assert.Equal(t, hash, readHash, "update content_hash must round-trip through ReadNote")
|
assert.Equal(t, hash, readHash, "update content_hash must round-trip through ReadNote")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUpdateNoteSupersedeReasonNotSettableViaCallerFragmentWhenReasonOptionEmpty(t *testing.T) {
|
|
||||||
// hyperguild#125, found in #111's review: the merge loop applied every
|
|
||||||
// field from the caller's leading frontmatter fragment, including
|
|
||||||
// supersede_reason -- which is only re-set unconditionally when
|
|
||||||
// opts.Reason is non-empty, so a caller could set it via its own
|
|
||||||
// fragment and have it survive even with Reason left empty, bypassing
|
|
||||||
// the one intended entry point for that field.
|
|
||||||
brainDir := t.TempDir()
|
|
||||||
rel := seedNote(t, brainDir, "wiki/a/facts/n.md", "---\nwing: a\nhall: facts\n---\nold\n")
|
|
||||||
|
|
||||||
_, _, _, err := UpdateNote(brainDir, UpdateNoteOptions{
|
|
||||||
Path: rel,
|
|
||||||
Content: "---\nsupersede_reason: snuck in via the fragment\n---\nnew\n",
|
|
||||||
// Reason deliberately left empty.
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
got, err := os.ReadFile(filepath.Join(brainDir, filepath.FromSlash(rel)))
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.NotContains(t, string(got), "supersede_reason",
|
|
||||||
"supersede_reason must not appear at all when opts.Reason is empty, regardless of what the fragment tried to set")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestUpdateNoteSupersedeReasonFromOptionStillWorks(t *testing.T) {
|
|
||||||
// Regression pin: the legitimate entry point (opts.Reason) must be
|
|
||||||
// completely unaffected by #125's fix.
|
|
||||||
brainDir := t.TempDir()
|
|
||||||
rel := seedNote(t, brainDir, "wiki/a/facts/n.md", "---\nwing: a\nhall: facts\n---\nold\n")
|
|
||||||
|
|
||||||
_, _, _, err := UpdateNote(brainDir, UpdateNoteOptions{
|
|
||||||
Path: rel, Content: "new\n", Reason: "facts changed",
|
|
||||||
})
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
got, err := os.ReadFile(filepath.Join(brainDir, filepath.FromSlash(rel)))
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Contains(t, string(got), "supersede_reason: facts changed")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestContentHashStable(t *testing.T) {
|
func TestContentHashStable(t *testing.T) {
|
||||||
assert.Equal(t, ContentHash([]byte("abc")), ContentHash([]byte("abc")))
|
assert.Equal(t, ContentHash([]byte("abc")), ContentHash([]byte("abc")))
|
||||||
assert.NotEqual(t, ContentHash([]byte("abc")), ContentHash([]byte("abd")))
|
assert.NotEqual(t, ContentHash([]byte("abc")), ContentHash([]byte("abd")))
|
||||||
|
|||||||
@@ -53,8 +53,6 @@ func (s *Store) Write(ctx context.Context, n capture.Note) (capture.Ref, error)
|
|||||||
Filename: n.Filename,
|
Filename: n.Filename,
|
||||||
Wing: n.Wing,
|
Wing: n.Wing,
|
||||||
Hall: n.Hall,
|
Hall: n.Hall,
|
||||||
Origin: n.Origin,
|
|
||||||
WrittenBy: n.WrittenBy,
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return capture.Ref{}, err
|
return capture.Ref{}, err
|
||||||
@@ -76,7 +74,7 @@ func (s *Store) Update(ctx context.Context, slug string, n capture.Note) (captur
|
|||||||
}
|
}
|
||||||
n.Content = content
|
n.Content = content
|
||||||
|
|
||||||
opts := api.UpdateNoteOptions{Content: n.Content, Reason: n.Reason, Origin: n.Origin, WrittenBy: n.WrittenBy}
|
opts := api.UpdateNoteOptions{Content: n.Content, Reason: n.Reason}
|
||||||
if strings.Contains(slug, "/") {
|
if strings.Contains(slug, "/") {
|
||||||
opts.Path = slug
|
opts.Path = slug
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -60,7 +60,6 @@ type CaptureContext struct {
|
|||||||
Classification string // caller-declared level token ("" = unspecified)
|
Classification string // caller-declared level token ("" = unspecified)
|
||||||
Principal string // server-derived (auth); audit identity
|
Principal string // server-derived (auth); audit identity
|
||||||
Origin Zone // server-derived trust zone; the I1 gate input
|
Origin Zone // server-derived trust zone; the I1 gate input
|
||||||
AuthPath string // server-derived: "static" or "jwt"; stamped as note provenance (#145)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Insight is one piece of session knowledge bound for the brain. A
|
// Insight is one piece of session knowledge bound for the brain. A
|
||||||
|
|||||||
@@ -40,11 +40,6 @@ type Note struct {
|
|||||||
Wing string
|
Wing string
|
||||||
Hall string
|
Hall string
|
||||||
Reason string
|
Reason string
|
||||||
// Provenance (hyperguild#145) is stamped by the server from the
|
|
||||||
// authenticated principal, never taken from Content's frontmatter.
|
|
||||||
// Origin is "static", "jwt" or "unknown"; empty means "unknown".
|
|
||||||
Origin string
|
|
||||||
WrittenBy string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// BrainStore is the brain persistence port — the shared implementation of
|
// BrainStore is the brain persistence port — the shared implementation of
|
||||||
|
|||||||
@@ -142,7 +142,7 @@ func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt,
|
|||||||
var landed []string
|
var landed []string
|
||||||
|
|
||||||
for i, ins := range in.Insights {
|
for i, ins := range in.Insights {
|
||||||
res, item, err := s.persistInsight(ctx, ins, in.Context)
|
res, item, err := s.persistInsight(ctx, ins)
|
||||||
receipt.Insights = append(receipt.Insights, res)
|
receipt.Insights = append(receipt.Insights, res)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
receipt.Errors = append(receipt.Errors, ItemError{Item: fmt.Sprintf("insight[%d]", i), Error: err.Error()})
|
receipt.Errors = append(receipt.Errors, ItemError{Item: fmt.Sprintf("insight[%d]", i), Error: err.Error()})
|
||||||
@@ -278,11 +278,8 @@ func escalationReason(escalators []string) string {
|
|||||||
return "required by: " + strings.Join(escalators, ", ")
|
return "required by: " + strings.Join(escalators, ", ")
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Service) persistInsight(ctx context.Context, ins Insight, cc CaptureContext) (InsightResult, string, error) {
|
func (s *Service) persistInsight(ctx context.Context, ins Insight) (InsightResult, string, error) {
|
||||||
note := Note{
|
note := Note{Content: ins.Text, Wing: ins.Wing, Hall: ins.Hall, Filename: brain.Sanitise(firstLine(ins.Text))}
|
||||||
Content: ins.Text, Wing: ins.Wing, Hall: ins.Hall, Filename: brain.Sanitise(firstLine(ins.Text)),
|
|
||||||
Origin: cc.AuthPath, WrittenBy: cc.Principal,
|
|
||||||
}
|
|
||||||
var ref Ref
|
var ref Ref
|
||||||
var err error
|
var err error
|
||||||
if ins.SupersedeSlug != "" {
|
if ins.SupersedeSlug != "" {
|
||||||
|
|||||||
@@ -104,10 +104,6 @@ func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
// caller may have tried to put in the body.
|
// caller may have tried to put in the body.
|
||||||
in.Context.Principal = principal
|
in.Context.Principal = principal
|
||||||
in.Context.Origin = h.resolver.Resolve(principal, viaStatic)
|
in.Context.Origin = h.resolver.Resolve(principal, viaStatic)
|
||||||
in.Context.AuthPath = "jwt"
|
|
||||||
if viaStatic {
|
|
||||||
in.Context.AuthPath = "static"
|
|
||||||
}
|
|
||||||
|
|
||||||
rec, err := h.svc.Capture(r.Context(), in)
|
rec, err := h.svc.Capture(r.Context(), in)
|
||||||
switch {
|
switch {
|
||||||
|
|||||||
@@ -43,26 +43,6 @@ var DefaultRules = []Rule{
|
|||||||
// JSON, so both segments begin with "eyJ".
|
// JSON, so both segments begin with "eyJ".
|
||||||
{Name: "jwt", RE: regexp.MustCompile(`eyJ[A-Za-z0-9_\-]{8,}\.eyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}`)},
|
{Name: "jwt", RE: regexp.MustCompile(`eyJ[A-Za-z0-9_\-]{8,}\.eyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}`)},
|
||||||
{Name: "postgres-uri-with-password", RE: regexp.MustCompile(`postgres(?:ql)?://[^:\s/]+:[^@\s/]+@`)},
|
{Name: "postgres-uri-with-password", RE: regexp.MustCompile(`postgres(?:ql)?://[^:\s/]+:[^@\s/]+@`)},
|
||||||
// Any credential embedded in a URL's userinfo, which is what a git credential
|
|
||||||
// store holds: `https://<user>:<token>@host`. The postgres rule above is the
|
|
||||||
// same shape narrowed to one scheme, and is kept ahead of this one only so a
|
|
||||||
// DSN reports the more specific name -- same ordering convention as
|
|
||||||
// authorization-header before bearer-token.
|
|
||||||
//
|
|
||||||
// Added after this shape passed ALL rules and was therefore ingested rather
|
|
||||||
// than dropped (hyperguild#142). It reached ~/.claude/projects -- this
|
|
||||||
// watcher's own input -- because a test suite in mathias/infra executed a
|
|
||||||
// script that configures git while inheriting the operator's real $HOME, read
|
|
||||||
// koala's live credential store, and printed it. Two credentials were rotated
|
|
||||||
// (infra#466). The scrubber is the control that decides whether that class of
|
|
||||||
// mistake is survivable, and for this shape it was failing open.
|
|
||||||
//
|
|
||||||
// The {8,} floor on the secret half is what keeps `https://host:443/path` and
|
|
||||||
// `https://user:@host` out; the `[^:\s/@]+` user half stops a host:port pair
|
|
||||||
// being read as userinfo. Both directions are covered by
|
|
||||||
// TestScrub_URLUserinfoCredential, because a rule that dropped every URL with
|
|
||||||
// a colon in it would discard most of the signal this watcher exists to keep.
|
|
||||||
{Name: "url-userinfo-credential", RE: regexp.MustCompile(`[a-zA-Z][a-zA-Z0-9+.\-]*://[^:\s/@]+:[^@\s/]{8,}@`)},
|
|
||||||
{Name: "private-key", RE: regexp.MustCompile(`-----BEGIN[^-]*PRIVATE KEY-----`)},
|
{Name: "private-key", RE: regexp.MustCompile(`-----BEGIN[^-]*PRIVATE KEY-----`)},
|
||||||
{Name: "ssh-key", RE: regexp.MustCompile(`ssh-(?:rsa|ed25519|ecdsa)\s+[A-Za-z0-9+/=]{40,}`)},
|
{Name: "ssh-key", RE: regexp.MustCompile(`ssh-(?:rsa|ed25519|ecdsa)\s+[A-Za-z0-9+/=]{40,}`)},
|
||||||
{Name: "github-pat", RE: regexp.MustCompile(`\b(?:ghp|gho|ghu|ghr|gha)_[A-Za-z0-9]{30,}\b`)},
|
{Name: "github-pat", RE: regexp.MustCompile(`\b(?:ghp|gho|ghu|ghr|gha)_[A-Za-z0-9]{30,}\b`)},
|
||||||
|
|||||||
@@ -46,64 +46,6 @@ func TestScrub_PoisonedFixtures(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// A credential embedded in a URL's userinfo is the shape a git credential store
|
|
||||||
// holds -- `https://<user>:<token>@host` -- and it walked past every rule until
|
|
||||||
// hyperguild#142. There was a rule for `postgres://user:pass@` and nothing for the
|
|
||||||
// general form, so `~/.git-credentials` content was ingested rather than dropped.
|
|
||||||
//
|
|
||||||
// Found the hard way: a test suite in mathias/infra executed a script that
|
|
||||||
// configures git while inheriting the operator's real $HOME, read koala's live
|
|
||||||
// credential store, and printed two live credentials into a session transcript.
|
|
||||||
// ~/.claude/projects is this watcher's own input, so the scrubber was the last
|
|
||||||
// line of defence for that mistake and it failed open. Two credentials were
|
|
||||||
// rotated (infra#466).
|
|
||||||
//
|
|
||||||
// The lesson is not "do not read the credential file" -- that mistake will recur.
|
|
||||||
// It is that this shape must fail closed, because the scrubber is what decides
|
|
||||||
// whether such a mistake is survivable.
|
|
||||||
func TestScrub_URLUserinfoCredential(t *testing.T) {
|
|
||||||
// Every secret below is INVENTED. None is or was a real credential.
|
|
||||||
mustScrub := []struct {
|
|
||||||
name string
|
|
||||||
content string
|
|
||||||
}{
|
|
||||||
{"git-credential-store-user", "https://mathias:0123456789abcdef0123456789abcdef01234567@git.d-ma.be"},
|
|
||||||
{"git-credential-store-oauth2", "https://oauth2:0123456789abcdef0123456789abcdef01234567@git.d-ma.be"},
|
|
||||||
{"in-context-payload-line", `payload line: https://mathias:0123456789abcdef0123456789abcdef01234567@git.d-ma.be`},
|
|
||||||
{"clone-url-in-argv", "git clone https://oauth2:abcdefghijklmnopqrstuvwx@git.d-ma.be/mathias/infra.git"},
|
|
||||||
{"http-not-https", "http://admin:supersecretvalue123@10.0.1.20/api"},
|
|
||||||
{"git-plus-ssh-scheme", "git+ssh://deploy:abcdefghijklmnop@example.com/repo"},
|
|
||||||
{"redis-scheme", "redis://default:abcdefghijklmnopqrst@10.0.1.20:6379/0"},
|
|
||||||
}
|
|
||||||
for _, tc := range mustScrub {
|
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
|
||||||
assert.NotEmpty(t, Scrub(tc.content),
|
|
||||||
"a credential in URL userinfo must be dropped, not ingested")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// The other direction matters as much: a rule that scrubs every URL with a
|
|
||||||
// colon in it would drop most of the signal this watcher exists to keep.
|
|
||||||
mustNotScrub := []struct {
|
|
||||||
name string
|
|
||||||
content string
|
|
||||||
}{
|
|
||||||
{"host-with-port", "cloning https://git.d-ma.be:443/mathias/infra.git"},
|
|
||||||
{"postgres-no-password", "postgres://localhost:5432/brain"},
|
|
||||||
{"url-with-colon-in-query", "GET https://example.com/path?filter=a:b&x=1"},
|
|
||||||
{"plain-url", "see https://git.d-ma.be/mathias/infra/issues/466"},
|
|
||||||
{"userinfo-no-secret", "https://user:@example.com"},
|
|
||||||
{"format-string-placeholder", `printf 'https://oauth2:%s@%s\n' "$TOKEN" "$HOST"`},
|
|
||||||
{"scheme-relative-time", "duration https://example.com took 00:03:22"},
|
|
||||||
}
|
|
||||||
for _, tc := range mustNotScrub {
|
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
|
||||||
assert.Empty(t, Scrub(tc.content),
|
|
||||||
"this is not a credential and must pass through")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestScrub_CleanContentPassesThrough(t *testing.T) {
|
func TestScrub_CleanContentPassesThrough(t *testing.T) {
|
||||||
cases := []string{
|
cases := []string{
|
||||||
"",
|
"",
|
||||||
|
|||||||
@@ -66,7 +66,7 @@ func Fix(doc []byte) ([]byte, bool, error) {
|
|||||||
return doc, false, fmt.Errorf("frontmatter: quoting did not repair the block: %w", err)
|
return doc, false, fmt.Errorf("frontmatter: quoting did not repair the block: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, rest, _ := Split(doc)
|
_, rest, _ := split(doc)
|
||||||
out := []byte(fence + "\n")
|
out := []byte(fence + "\n")
|
||||||
out = append(out, fixed...)
|
out = append(out, fixed...)
|
||||||
out = append(out, []byte(fence)...)
|
out = append(out, []byte(fence)...)
|
||||||
@@ -122,25 +122,14 @@ func needsQuoting(value string) bool {
|
|||||||
|
|
||||||
// block returns the raw YAML between the opening and closing fences.
|
// block returns the raw YAML between the opening and closing fences.
|
||||||
func block(doc []byte) ([]byte, bool) {
|
func block(doc []byte) ([]byte, bool) {
|
||||||
b, _, ok := Split(doc)
|
b, _, ok := split(doc)
|
||||||
return b, ok
|
return b, ok
|
||||||
}
|
}
|
||||||
|
|
||||||
// Split returns doc's frontmatter block and everything from the closing
|
// split returns the frontmatter block and everything from the closing fence
|
||||||
// fence onward (the closing fence itself excluded, its trailing newline
|
// onward (the closing fence itself excluded, its trailing newline included).
|
||||||
// included), or ok=false if doc has no properly-fenced frontmatter.
|
func split(doc []byte) (block, rest []byte, ok bool) {
|
||||||
//
|
s := string(doc)
|
||||||
// The canonical fence detector for this codebase (hyperguild#138): three
|
|
||||||
// independent reimplementations of this same logic each grew their own gap
|
|
||||||
// through separate bug-fixing before being consolidated here —
|
|
||||||
// internal/mcp's needed a CRLF/BOM guard added after both silently defeated
|
|
||||||
// its own HasPrefix check, and internal/api's closing-fence match had no
|
|
||||||
// trailing-newline requirement, so a document ending in "\n---extra" with no
|
|
||||||
// real newline after the fence was treated as validly closed. Both classes
|
|
||||||
// are closed here, once, for every caller.
|
|
||||||
func Split(doc []byte) (block, rest []byte, ok bool) {
|
|
||||||
s := strings.TrimPrefix(string(doc), "\xef\xbb\xbf")
|
|
||||||
s = strings.ReplaceAll(s, "\r\n", "\n")
|
|
||||||
if !strings.HasPrefix(s, fence+"\n") {
|
if !strings.HasPrefix(s, fence+"\n") {
|
||||||
return nil, nil, false
|
return nil, nil, false
|
||||||
}
|
}
|
||||||
@@ -150,18 +139,5 @@ func Split(doc []byte) (block, rest []byte, ok bool) {
|
|||||||
if idx < 0 {
|
if idx < 0 {
|
||||||
return nil, nil, false
|
return nil, nil, false
|
||||||
}
|
}
|
||||||
// What follows the closing fence must be either a newline (a real body
|
return []byte(body[:idx+1]), []byte(body[idx+1+len(fence):]), true
|
||||||
// follows) or nothing at all (the fence is the literal end of the
|
|
||||||
// document — no body, not a bypass attempt). Anything else, e.g.
|
|
||||||
// "\n---extra", is not a valid close: an earlier version of this check
|
|
||||||
// accepted it, which is exactly the looser match hyperguild#138 closes.
|
|
||||||
after := idx + 1 + len(fence)
|
|
||||||
switch {
|
|
||||||
case after == len(body):
|
|
||||||
return []byte(body[:idx+1]), nil, true
|
|
||||||
case body[after] == '\n':
|
|
||||||
return []byte(body[:idx+1]), []byte(body[after:]), true
|
|
||||||
default:
|
|
||||||
return nil, nil, false
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -168,9 +168,9 @@ func TestFix_PreservesBodyExactly(t *testing.T) {
|
|||||||
require.True(t, changed)
|
require.True(t, changed)
|
||||||
assert.True(t, len(got) > 0)
|
assert.True(t, len(got) > 0)
|
||||||
|
|
||||||
_, rest, ok := Split([]byte(doc))
|
_, rest, ok := split([]byte(doc))
|
||||||
require.True(t, ok)
|
require.True(t, ok)
|
||||||
_, gotRest, ok := Split(got)
|
_, gotRest, ok := split(got)
|
||||||
require.True(t, ok)
|
require.True(t, ok)
|
||||||
assert.Equal(t, string(rest), string(gotRest), "the body must not be touched")
|
assert.Equal(t, string(rest), string(gotRest), "the body must not be touched")
|
||||||
}
|
}
|
||||||
@@ -186,55 +186,3 @@ func TestFix_RefusesToReturnAnUnparseableResult(t *testing.T) {
|
|||||||
assert.False(t, changed)
|
assert.False(t, changed)
|
||||||
assert.Equal(t, doc, string(got), "an unfixable document must come back byte-identical")
|
assert.Equal(t, doc, string(got), "an unfixable document must come back byte-identical")
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestSplit_NormalizesCRLFBeforeFenceDetection is hyperguild#138: a note
|
|
||||||
// whose fence lines use CRLF ("---\r\n") failed strings.HasPrefix(doc,
|
|
||||||
// "---\n") on the precondition read in internal/mcp's independent
|
|
||||||
// reimplementation -- encoding alone, no adversarial input, silently
|
|
||||||
// defeated fence detection there before this consolidation. Split must
|
|
||||||
// never repeat that.
|
|
||||||
func TestSplit_NormalizesCRLFBeforeFenceDetection(t *testing.T) {
|
|
||||||
doc := "---\r\ntitle: 'n'\r\nhalf_life: durable\r\n---\r\n\r\nbody\r\n"
|
|
||||||
block, rest, ok := Split([]byte(doc))
|
|
||||||
require.True(t, ok)
|
|
||||||
assert.Contains(t, string(block), "half_life: durable")
|
|
||||||
assert.Contains(t, string(rest), "body")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSplit_StripsALeadingUTF8BOM is the same mechanism one layer earlier:
|
|
||||||
// a leading BOM (Notepad's default "UTF-8" save, PowerShell's pre-v6
|
|
||||||
// Out-File/Set-Content) defeats the same HasPrefix check before CRLF
|
|
||||||
// normalization even applies.
|
|
||||||
func TestSplit_StripsALeadingUTF8BOM(t *testing.T) {
|
|
||||||
doc := "\xef\xbb\xbf---\ntitle: 'n'\n---\n\nbody\n"
|
|
||||||
block, rest, ok := Split([]byte(doc))
|
|
||||||
require.True(t, ok)
|
|
||||||
assert.Contains(t, string(block), "title: 'n'")
|
|
||||||
assert.Contains(t, string(rest), "body")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSplit_RequiresANewlineAfterTheClosingFence closes the looser match
|
|
||||||
// internal/api's independent reimplementation carried (strings.Index(body,
|
|
||||||
// "\n"+fence) with no trailing-newline requirement): a document ending in
|
|
||||||
// "\n---extra" with no newline separating the fence from what follows it is
|
|
||||||
// not a valid closing fence and must not be treated as one.
|
|
||||||
func TestSplit_RequiresANewlineAfterTheClosingFence(t *testing.T) {
|
|
||||||
doc := "---\ntitle: 'n'\n---extra-not-a-real-close"
|
|
||||||
_, _, ok := Split([]byte(doc))
|
|
||||||
assert.False(t, ok, "a fence line not followed by a newline is not a valid closing fence")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSplit_AcceptsAClosingFenceThatIsTheLastBytesOfTheDocument is a
|
|
||||||
// regression pin found in review of hyperguild#138: the fix for the
|
|
||||||
// "\n---extra" bypass above must not also reject the legitimate case of a
|
|
||||||
// closing fence with nothing after it at all (no trailing newline because
|
|
||||||
// there's no body, not because a bypass is being attempted) -- this
|
|
||||||
// package's own private split() accepted that shape before the
|
|
||||||
// consolidation, and cmd/frontmatterlint's Lint/Fix relied on it.
|
|
||||||
func TestSplit_AcceptsAClosingFenceThatIsTheLastBytesOfTheDocument(t *testing.T) {
|
|
||||||
doc := "---\ntitle: 'n'\n---"
|
|
||||||
block, rest, ok := Split([]byte(doc))
|
|
||||||
require.True(t, ok, "a closing fence at literal end-of-file, with nothing after it, is valid -- there is no body to require a newline before")
|
|
||||||
assert.Contains(t, string(block), "title: 'n'")
|
|
||||||
assert.Equal(t, "", string(rest))
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -1,47 +0,0 @@
|
|||||||
package mcp
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"log/slog"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// logToolCall emits one structured record per tools/call (hyperguild#144,
|
|
||||||
// OWASP MCP08): who called which tool, against which wing/hall, and how it
|
|
||||||
// ended. Brain content is never copied into the record — only its size — so
|
|
||||||
// the audit trail does not become a second, unscrubbed copy of the brain.
|
|
||||||
func logToolCall(ctx context.Context, name string, args json.RawMessage, start time.Time, err error) {
|
|
||||||
origin, principal := provenanceFrom(ctx)
|
|
||||||
attrs := []any{
|
|
||||||
"tool", name,
|
|
||||||
"principal", principal,
|
|
||||||
"origin", origin,
|
|
||||||
"duration_ms", time.Since(start).Milliseconds(),
|
|
||||||
}
|
|
||||||
var a struct {
|
|
||||||
Wing string `json:"wing"`
|
|
||||||
Hall string `json:"hall"`
|
|
||||||
Content string `json:"content"`
|
|
||||||
}
|
|
||||||
if json.Unmarshal(args, &a) == nil {
|
|
||||||
if a.Wing != "" {
|
|
||||||
attrs = append(attrs, "wing", a.Wing)
|
|
||||||
}
|
|
||||||
if a.Hall != "" {
|
|
||||||
attrs = append(attrs, "hall", a.Hall)
|
|
||||||
}
|
|
||||||
if a.Content != "" {
|
|
||||||
attrs = append(attrs, "content_bytes", len(a.Content))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
outcome := "ok"
|
|
||||||
if err != nil {
|
|
||||||
outcome = err.Error()
|
|
||||||
if len(outcome) > 200 {
|
|
||||||
outcome = outcome[:200]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
attrs = append(attrs, "outcome", outcome)
|
|
||||||
slog.InfoContext(ctx, "mcp tool call", attrs...)
|
|
||||||
}
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
package mcp_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"log/slog"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// captureToolCallLog routes slog to a buffer for fn and returns the decoded
|
|
||||||
// "mcp tool call" records.
|
|
||||||
func captureToolCallLog(t *testing.T, fn func()) []map[string]any {
|
|
||||||
t.Helper()
|
|
||||||
var buf bytes.Buffer
|
|
||||||
prev := slog.Default()
|
|
||||||
slog.SetDefault(slog.New(slog.NewJSONHandler(&buf, nil)))
|
|
||||||
t.Cleanup(func() { slog.SetDefault(prev) })
|
|
||||||
fn()
|
|
||||||
var out []map[string]any
|
|
||||||
for _, l := range strings.Split(strings.TrimSpace(buf.String()), "\n") {
|
|
||||||
var rec map[string]any
|
|
||||||
if json.Unmarshal([]byte(l), &rec) == nil && rec["msg"] == "mcp tool call" {
|
|
||||||
out = append(out, rec)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
// hyperguild#144: brain_query/brain_read left no record of who read what.
|
|
||||||
func TestToolsCallLogsOneRecordWithToolPrincipalOriginAndScope(t *testing.T) {
|
|
||||||
srv, _, _, _ := govServer(t)
|
|
||||||
recs := captureToolCallLog(t, func() {
|
|
||||||
authCall(t, srv, "Bearer "+capStaticTok, "brain_query",
|
|
||||||
map[string]any{"query": "anything", "wing": "homelab", "hall": "facts"})
|
|
||||||
})
|
|
||||||
require.Len(t, recs, 1)
|
|
||||||
r := recs[0]
|
|
||||||
assert.Equal(t, "brain_query", r["tool"])
|
|
||||||
assert.Equal(t, "local-cli", r["principal"])
|
|
||||||
assert.Equal(t, "static", r["origin"])
|
|
||||||
assert.Equal(t, "homelab", r["wing"])
|
|
||||||
assert.Equal(t, "facts", r["hall"])
|
|
||||||
assert.Equal(t, "ok", r["outcome"])
|
|
||||||
assert.Contains(t, r, "duration_ms")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestToolsCallWithoutDerivablePrincipalLogsOriginUnknown(t *testing.T) {
|
|
||||||
srv, _, _, _ := govServer(t)
|
|
||||||
recs := captureToolCallLog(t, func() {
|
|
||||||
authCall(t, srv, "", "brain_query", map[string]any{"query": "x"})
|
|
||||||
})
|
|
||||||
require.Len(t, recs, 1)
|
|
||||||
assert.Equal(t, "unknown", recs[0]["origin"])
|
|
||||||
assert.Equal(t, "", recs[0]["principal"])
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestToolsCallLogsErrorOutcome(t *testing.T) {
|
|
||||||
srv, _, _, _ := govServer(t)
|
|
||||||
recs := captureToolCallLog(t, func() {
|
|
||||||
authCall(t, srv, "Bearer "+capStaticTok, "brain_query", map[string]any{})
|
|
||||||
})
|
|
||||||
require.Len(t, recs, 1)
|
|
||||||
assert.Contains(t, recs[0]["outcome"], "query is required")
|
|
||||||
}
|
|
||||||
|
|
||||||
// The audit line must not become a second copy of brain content.
|
|
||||||
func TestToolsCallLogOmitsContentButRecordsItsSize(t *testing.T) {
|
|
||||||
const sentinel = "SENTINEL-BRAIN-CONTENT-91c2"
|
|
||||||
srv, _, _, _ := govServer(t)
|
|
||||||
recs := captureToolCallLog(t, func() {
|
|
||||||
authCall(t, srv, "Bearer "+capStaticTok, "brain_write", map[string]any{
|
|
||||||
"content": "# t\n\n" + sentinel, "wing": "testing", "hall": "facts", "filename": "audit-sentinel",
|
|
||||||
})
|
|
||||||
})
|
|
||||||
require.Len(t, recs, 1)
|
|
||||||
raw, _ := json.Marshal(recs[0])
|
|
||||||
assert.NotContains(t, string(raw), sentinel)
|
|
||||||
assert.EqualValues(t, len("# t\n\n"+sentinel), recs[0]["content_bytes"])
|
|
||||||
}
|
|
||||||
@@ -255,14 +255,11 @@ func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.Raw
|
|||||||
// one implementation (capture uses the same store). The read-after-write
|
// one implementation (capture uses the same store). The read-after-write
|
||||||
// handle {id, path, content_hash} comes back from the store; path is kept
|
// handle {id, path, content_hash} comes back from the store; path is kept
|
||||||
// for backward compatibility.
|
// for backward compatibility.
|
||||||
origin, writtenBy := provenanceFrom(ctx)
|
|
||||||
ref, err := s.store.Write(ctx, capture.Note{
|
ref, err := s.store.Write(ctx, capture.Note{
|
||||||
Content: a.Content,
|
Content: a.Content,
|
||||||
Filename: a.Filename,
|
Filename: a.Filename,
|
||||||
Wing: a.Wing,
|
Wing: a.Wing,
|
||||||
Hall: a.Hall,
|
Hall: a.Hall,
|
||||||
Origin: origin,
|
|
||||||
WrittenBy: writtenBy,
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -307,14 +304,11 @@ func (s *Server) brainUpdate(ctx context.Context, args json.RawMessage) (json.Ra
|
|||||||
if a.Path != "" {
|
if a.Path != "" {
|
||||||
slug = a.Path
|
slug = a.Path
|
||||||
}
|
}
|
||||||
origin, writtenBy := provenanceFrom(ctx)
|
|
||||||
ref, err := s.store.Update(ctx, slug, capture.Note{
|
ref, err := s.store.Update(ctx, slug, capture.Note{
|
||||||
Content: a.Content,
|
Content: a.Content,
|
||||||
Wing: a.Wing,
|
Wing: a.Wing,
|
||||||
Hall: a.Hall,
|
Hall: a.Hall,
|
||||||
Reason: a.Reason,
|
Reason: a.Reason,
|
||||||
Origin: origin,
|
|
||||||
WrittenBy: writtenBy,
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
|
|||||||
@@ -1,124 +0,0 @@
|
|||||||
package mcp_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// resultText decodes the tool result payload of a tools/call response.
|
|
||||||
func resultText(t *testing.T, resp map[string]any) map[string]any {
|
|
||||||
t.Helper()
|
|
||||||
require.NotContains(t, resp, "error", "tool call failed: %v", resp["error"])
|
|
||||||
content := resp["result"].(map[string]any)["content"].([]any)
|
|
||||||
var out map[string]any
|
|
||||||
require.NoError(t, json.Unmarshal([]byte(content[0].(map[string]any)["text"].(string)), &out))
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func readNote(t *testing.T, dir, rel string) string {
|
|
||||||
t.Helper()
|
|
||||||
b, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
|
|
||||||
require.NoError(t, err)
|
|
||||||
return string(b)
|
|
||||||
}
|
|
||||||
|
|
||||||
func frontmatterOf(note string) string {
|
|
||||||
parts := strings.SplitN(note, "---\n", 3)
|
|
||||||
if len(parts) < 3 {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
return parts[1]
|
|
||||||
}
|
|
||||||
|
|
||||||
// hyperguild#145: a brain entry records which principal wrote it, so a reader
|
|
||||||
// can tell a local-CLI note from one a connector session wrote after reading
|
|
||||||
// an untrusted page.
|
|
||||||
func TestBrainWriteStampsPrincipalProvenance(t *testing.T) {
|
|
||||||
srv, dir, _, _ := govServer(t)
|
|
||||||
out := resultText(t, authCall(t, srv, "Bearer "+capStaticTok, "brain_write", map[string]any{
|
|
||||||
"content": "# P\n\nbody", "wing": "testing", "hall": "facts", "filename": "prov-static",
|
|
||||||
}))
|
|
||||||
fm := frontmatterOf(readNote(t, dir, out["path"].(string)))
|
|
||||||
assert.Contains(t, fm, "origin: static\n")
|
|
||||||
assert.Contains(t, fm, "written_by: local-cli\n")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestBrainWriteWithoutPrincipalStampsOriginUnknown(t *testing.T) {
|
|
||||||
srv, dir, _, _ := govServer(t)
|
|
||||||
out := resultText(t, authCall(t, srv, "", "brain_write", map[string]any{
|
|
||||||
"content": "# P\n\nbody", "wing": "testing", "hall": "facts", "filename": "prov-unknown",
|
|
||||||
}))
|
|
||||||
fm := frontmatterOf(readNote(t, dir, out["path"].(string)))
|
|
||||||
assert.Contains(t, fm, "origin: unknown\n")
|
|
||||||
assert.NotContains(t, fm, "written_by:")
|
|
||||||
}
|
|
||||||
|
|
||||||
// The control: a caller cannot claim a more trusted origin by putting it in
|
|
||||||
// its own frontmatter.
|
|
||||||
func TestBrainWriteOverwritesCallerSuppliedProvenance(t *testing.T) {
|
|
||||||
srv, dir, _, _ := govServer(t)
|
|
||||||
forged := "---\norigin: static\nwritten_by: local-cli\n---\n# F\n\nbody"
|
|
||||||
out := resultText(t, authCall(t, srv, "", "brain_write", map[string]any{
|
|
||||||
"content": forged, "wing": "testing", "hall": "facts", "filename": "prov-forged",
|
|
||||||
}))
|
|
||||||
fm := frontmatterOf(readNote(t, dir, out["path"].(string)))
|
|
||||||
assert.Contains(t, fm, "origin: unknown\n")
|
|
||||||
assert.NotContains(t, fm, "origin: static")
|
|
||||||
assert.NotContains(t, fm, "written_by: local-cli")
|
|
||||||
assert.Equal(t, 1, strings.Count(fm, "origin:"), "exactly one origin line")
|
|
||||||
}
|
|
||||||
|
|
||||||
// brain_update replaces the content wholesale, so provenance follows the
|
|
||||||
// last writer — and a forged fragment must not survive the merge either.
|
|
||||||
func TestBrainUpdateRestampsProvenanceAndIgnoresForgedFragment(t *testing.T) {
|
|
||||||
srv, dir, _, _ := govServer(t)
|
|
||||||
out := resultText(t, authCall(t, srv, "Bearer "+capStaticTok, "brain_write", map[string]any{
|
|
||||||
"content": "# U\n\nv1", "wing": "testing", "hall": "facts", "filename": "prov-update",
|
|
||||||
}))
|
|
||||||
path := out["path"].(string)
|
|
||||||
resultText(t, authCall(t, srv, "", "brain_update", map[string]any{
|
|
||||||
"path": path, "content": "---\norigin: static\nwritten_by: local-cli\n---\n# U\n\nv2",
|
|
||||||
}))
|
|
||||||
fm := frontmatterOf(readNote(t, dir, path))
|
|
||||||
assert.Contains(t, fm, "origin: unknown\n")
|
|
||||||
assert.NotContains(t, fm, "written_by: local-cli")
|
|
||||||
assert.Equal(t, 1, strings.Count(fm, "origin:"))
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestBrainQueryReturnsProvenance(t *testing.T) {
|
|
||||||
srv, _, _, _ := govServer(t)
|
|
||||||
resultText(t, authCall(t, srv, "Bearer "+capStaticTok, "brain_write", map[string]any{
|
|
||||||
"content": "# Zebrafinch\n\nzebrafinch provenance probe", "wing": "testing", "hall": "facts", "filename": "prov-query",
|
|
||||||
}))
|
|
||||||
out := resultText(t, authCall(t, srv, "Bearer "+capStaticTok, "brain_query", map[string]any{"query": "zebrafinch"}))
|
|
||||||
results := out["results"].([]any)
|
|
||||||
require.NotEmpty(t, results)
|
|
||||||
hit := results[0].(map[string]any)
|
|
||||||
assert.Equal(t, "static", hit["origin"])
|
|
||||||
assert.Equal(t, "local-cli", hit["written_by"])
|
|
||||||
}
|
|
||||||
|
|
||||||
// capture is the door a claude.ai connector session uses, so it is the one
|
|
||||||
// whose provenance matters most: a JWT principal must be stamped as such.
|
|
||||||
func TestCaptureStampsJWTProvenance(t *testing.T) {
|
|
||||||
srv, brainDir := captureServer(t, capFakeValidator{subject: "koala-cli"}, []string{"koala-cli"})
|
|
||||||
resp := captureCall(t, srv, "Bearer jwt-token", map[string]any{
|
|
||||||
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
|
|
||||||
"insights": []map[string]any{{"text": "Capture provenance probe", "wing": "hyperguild", "hall": "facts"}},
|
|
||||||
})
|
|
||||||
require.Nil(t, resp["error"], "got error: %v", resp["error"])
|
|
||||||
matches, err := filepath.Glob(filepath.Join(brainDir, "wiki/hyperguild/facts/*.md"))
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.Len(t, matches, 1)
|
|
||||||
b, err := os.ReadFile(matches[0])
|
|
||||||
require.NoError(t, err)
|
|
||||||
fm := frontmatterOf(string(b))
|
|
||||||
assert.Contains(t, fm, "origin: jwt\n")
|
|
||||||
assert.Contains(t, fm, "written_by: koala-cli\n")
|
|
||||||
}
|
|
||||||
@@ -16,7 +16,6 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
||||||
@@ -230,9 +229,7 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
ctx = withPrincipal(ctx, principal, viaStatic)
|
ctx = withPrincipal(ctx, principal, viaStatic)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
start := time.Now()
|
|
||||||
out, err := s.handleCall(ctx, p.Name, p.Arguments)
|
out, err := s.handleCall(ctx, p.Name, p.Arguments)
|
||||||
logToolCall(ctx, p.Name, p.Arguments, start, err)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rpcErr = &rpcError{Code: -32000, Message: err.Error()}
|
rpcErr = &rpcError{Code: -32000, Message: err.Error()}
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -90,7 +90,6 @@ func (s *Server) brainCapture(ctx context.Context, args json.RawMessage) (json.R
|
|||||||
// Principal and origin are server-derived — never taken from the body.
|
// Principal and origin are server-derived — never taken from the body.
|
||||||
in.Context.Principal = info.principal
|
in.Context.Principal = info.principal
|
||||||
in.Context.Origin = s.capture.resolver.Resolve(info.principal, info.viaStatic)
|
in.Context.Origin = s.capture.resolver.Resolve(info.principal, info.viaStatic)
|
||||||
in.Context.AuthPath, _ = provenanceFrom(ctx)
|
|
||||||
|
|
||||||
rec, err := s.capture.svc.Capture(ctx, in)
|
rec, err := s.capture.svc.Capture(ctx, in)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -100,17 +99,3 @@ func (s *Server) brainCapture(ctx context.Context, args json.RawMessage) (json.R
|
|||||||
}
|
}
|
||||||
return json.Marshal(rec)
|
return json.Marshal(rec)
|
||||||
}
|
}
|
||||||
|
|
||||||
// provenanceFrom maps the authenticated principal in ctx to the provenance
|
|
||||||
// stamped on a brain write (hyperguild#145). No principal → "unknown", never
|
|
||||||
// silence: an unstamped note must not read as trusted.
|
|
||||||
func provenanceFrom(ctx context.Context) (origin, writtenBy string) {
|
|
||||||
info, ok := ctx.Value(principalKey).(principalInfo)
|
|
||||||
if !ok {
|
|
||||||
return "unknown", ""
|
|
||||||
}
|
|
||||||
if info.viaStatic {
|
|
||||||
return "static", info.principal
|
|
||||||
}
|
|
||||||
return "jwt", info.principal
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -13,7 +13,6 @@ import (
|
|||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/claim"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/claim"
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/frontmatter"
|
|
||||||
"github.com/mathiasbq/hyperguild/ingestion/internal/notes"
|
"github.com/mathiasbq/hyperguild/ingestion/internal/notes"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -332,20 +331,27 @@ func checkClaimEnvelope(brainDir, path, oldStr, newStr string) error {
|
|||||||
// frontmatterBlock returns the raw text between doc's opening and closing
|
// frontmatterBlock returns the raw text between doc's opening and closing
|
||||||
// `---` fences, or ok=false if doc has no (properly closed) fence.
|
// `---` fences, or ok=false if doc has no (properly closed) fence.
|
||||||
func frontmatterBlock(doc string) (block string, ok bool) {
|
func frontmatterBlock(doc string) (block string, ok bool) {
|
||||||
// hyperguild#138: fence detection itself is frontmatter.Split, shared
|
// Round 4 found a note with CRLF fence lines ("---\r\n") silently
|
||||||
// with internal/api and internal/frontmatter — the CRLF/BOM guards that
|
// defeats HasPrefix(doc, "---\n") on the precondition read — not an
|
||||||
// used to live here (added piecemeal across rounds 4 and 5 of #123's
|
// adversarial old_str/new_str, just an encoding difference — which
|
||||||
// review) are now the shared parser's job, so every caller inherits a
|
// makes checkClaimEnvelope no-op entirely. Round 5 found the same
|
||||||
// fix instead of needing bug-for-bug parity edits.
|
// mechanism one layer earlier: a leading UTF-8 BOM (a live, plausible
|
||||||
b, _, ok := frontmatter.Split([]byte(doc))
|
// byte sequence — Notepad's default "UTF-8" save and PowerShell's
|
||||||
if !ok {
|
// pre-v6 Out-File/Set-Content both emit one) defeats the same prefix
|
||||||
|
// check before CRLF normalization even applies. Strip both, here, so
|
||||||
|
// every caller (hadFence/hasFence detection and frontmatterScalars)
|
||||||
|
// agrees regardless of encoding.
|
||||||
|
doc = strings.TrimPrefix(doc, "\xef\xbb\xbf")
|
||||||
|
doc = strings.ReplaceAll(doc, "\r\n", "\n")
|
||||||
|
if !strings.HasPrefix(doc, "---\n") {
|
||||||
return "", false
|
return "", false
|
||||||
}
|
}
|
||||||
// Split's block keeps the "\n" immediately before the closing fence
|
rest := doc[len("---\n"):]
|
||||||
// (by design — see its own doc comment); this function never included
|
end := strings.Index(rest, "\n---\n")
|
||||||
// it. Split guarantees ok=true only when "\n---\n" matched, so this is
|
if end < 0 {
|
||||||
// always exactly one byte, never ambiguous.
|
return "", false
|
||||||
return strings.TrimSuffix(string(b), "\n"), true
|
}
|
||||||
|
return rest[:end], true
|
||||||
}
|
}
|
||||||
|
|
||||||
// frontmatterScalars extracts top-level "key: value" scalar pairs from
|
// frontmatterScalars extracts top-level "key: value" scalar pairs from
|
||||||
|
|||||||
@@ -48,11 +48,6 @@ type Result struct {
|
|||||||
// (infra#72). Read from frontmatter when present, otherwise
|
// (infra#72). Read from frontmatter when present, otherwise
|
||||||
// inferred from the parent directory.
|
// inferred from the parent directory.
|
||||||
Tier string `json:"tier,omitempty"`
|
Tier string `json:"tier,omitempty"`
|
||||||
// Origin/WrittenBy surface write provenance (hyperguild#145) so a
|
|
||||||
// reader can discount an entry a connector session wrote. Omitted for
|
|
||||||
// notes written before provenance existed — never fabricated.
|
|
||||||
Origin string `json:"origin,omitempty"`
|
|
||||||
WrittenBy string `json:"written_by,omitempty"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// tierWeight maps the DIKW tier to a score multiplier applied right
|
// tierWeight maps the DIKW tier to a score multiplier applied right
|
||||||
@@ -151,7 +146,6 @@ func QueryContext(ctx context.Context, brainDir string, opts QueryOptions) ([]Re
|
|||||||
rel = filepath.ToSlash(rel)
|
rel = filepath.ToSlash(rel)
|
||||||
wing, hall := extractWingHall(string(content), rel)
|
wing, hall := extractWingHall(string(content), rel)
|
||||||
tier := extractTier(string(content), rel)
|
tier := extractTier(string(content), rel)
|
||||||
origin, writtenBy := extractProvenance(string(content))
|
|
||||||
results = append(results, Result{
|
results = append(results, Result{
|
||||||
Path: rel,
|
Path: rel,
|
||||||
Title: extractTitle(string(content), d.Name()),
|
Title: extractTitle(string(content), d.Name()),
|
||||||
@@ -160,9 +154,6 @@ func QueryContext(ctx context.Context, brainDir string, opts QueryOptions) ([]Re
|
|||||||
Wing: wing,
|
Wing: wing,
|
||||||
Hall: hall,
|
Hall: hall,
|
||||||
Tier: tier,
|
Tier: tier,
|
||||||
|
|
||||||
Origin: origin,
|
|
||||||
WrittenBy: writtenBy,
|
|
||||||
})
|
})
|
||||||
return nil
|
return nil
|
||||||
})
|
})
|
||||||
@@ -281,7 +272,6 @@ func hydrate(brainDir, relPath string) (Result, error) {
|
|||||||
}
|
}
|
||||||
wing, hall := extractWingHall(string(content), relPath)
|
wing, hall := extractWingHall(string(content), relPath)
|
||||||
tier := extractTier(string(content), relPath)
|
tier := extractTier(string(content), relPath)
|
||||||
origin, writtenBy := extractProvenance(string(content))
|
|
||||||
return Result{
|
return Result{
|
||||||
Path: relPath,
|
Path: relPath,
|
||||||
Title: extractTitle(string(content), filepath.Base(relPath)),
|
Title: extractTitle(string(content), filepath.Base(relPath)),
|
||||||
@@ -289,45 +279,9 @@ func hydrate(brainDir, relPath string) (Result, error) {
|
|||||||
Wing: wing,
|
Wing: wing,
|
||||||
Hall: hall,
|
Hall: hall,
|
||||||
Tier: tier,
|
Tier: tier,
|
||||||
|
|
||||||
Origin: origin,
|
|
||||||
WrittenBy: writtenBy,
|
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// extractProvenance reads origin/written_by from the leading frontmatter
|
|
||||||
// block only — a body line that happens to start with "origin:" is not
|
|
||||||
// provenance.
|
|
||||||
func extractProvenance(content string) (origin, writtenBy string) {
|
|
||||||
scanner := bufio.NewScanner(strings.NewReader(content))
|
|
||||||
inFrontmatter := false
|
|
||||||
for scanner.Scan() {
|
|
||||||
line := scanner.Text()
|
|
||||||
if strings.TrimSpace(line) == "---" {
|
|
||||||
if !inFrontmatter {
|
|
||||||
inFrontmatter = true
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
break
|
|
||||||
}
|
|
||||||
if !inFrontmatter {
|
|
||||||
return "", ""
|
|
||||||
}
|
|
||||||
key, val, ok := strings.Cut(line, ":")
|
|
||||||
if !ok || strings.HasPrefix(line, " ") || strings.HasPrefix(line, "\t") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
v := strings.Trim(strings.TrimSpace(val), `"'`)
|
|
||||||
switch strings.TrimSpace(key) {
|
|
||||||
case "origin":
|
|
||||||
origin = v
|
|
||||||
case "written_by":
|
|
||||||
writtenBy = v
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return origin, writtenBy
|
|
||||||
}
|
|
||||||
|
|
||||||
// resolveRoots returns the directories to walk for the given wing/hall
|
// resolveRoots returns the directories to walk for the given wing/hall
|
||||||
// filters. Validates hall against the closed vocabulary when set.
|
// filters. Validates hall against the closed vocabulary when set.
|
||||||
func resolveRoots(brainDir, wing, hall string) ([]string, error) {
|
func resolveRoots(brainDir, wing, hall string) ([]string, error) {
|
||||||
|
|||||||
@@ -58,7 +58,7 @@ if [ "$FORCE" = false ]; then
|
|||||||
while IFS= read -r _match; do
|
while IFS= read -r _match; do
|
||||||
_lineno="${_match%%:*}"
|
_lineno="${_match%%:*}"
|
||||||
_content="${_match#*:}"
|
_content="${_match#*:}"
|
||||||
_token=$(head -1 < <(grep -o '{{[^}]*}}' <<<"$_content"))
|
_token=$(printf '%s' "$_content" | grep -o '{{[^}]*}}' | head -1)
|
||||||
echo " $PROJECT_FILE:$_lineno: unfilled placeholder $_token" >&2
|
echo " $PROJECT_FILE:$_lineno: unfilled placeholder $_token" >&2
|
||||||
done <<< "$_placeholders"
|
done <<< "$_placeholders"
|
||||||
echo "" >&2
|
echo "" >&2
|
||||||
|
|||||||
Reference in New Issue
Block a user