Compare commits
17
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ce365eee44 | ||
|
|
60a8801a1d | ||
|
|
54237af7ad | ||
|
|
b6ceccca26 | ||
|
|
36a45068ab | ||
|
|
055af368cd | ||
|
|
eb9e3e69a9 | ||
|
|
b7cf0efb9d | ||
|
|
125c99bb46 | ||
|
|
3e537bd582 | ||
|
|
4e6fc7aeee | ||
|
|
9a232ad444 | ||
|
|
3da0eca9b4 | ||
|
|
7d1213451c | ||
|
|
7c9293f4ed | ||
|
|
6e0155a2ab | ||
|
|
1cea2c9f78 |
@@ -78,6 +78,14 @@ These rules apply to every task across every project, regardless of harness.
|
||||
- Surface the brain entries and issue numbers in the closing summary so the
|
||||
trail is auditable.
|
||||
|
||||
## CAD sprint patterns
|
||||
|
||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||
- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
|
||||
- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
|
||||
|
||||
Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
|
||||
|
||||
## Default stack
|
||||
|
||||
| Layer | Default | Fallback | Last resort |
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
# Presence of this file marks this repo as opt-in for headless dispatch.
|
||||
# See dispatch#3.
|
||||
@@ -0,0 +1,63 @@
|
||||
# gitleaks config for the hyperguild repo (infra#39 — leak prevention pass,
|
||||
# Phase 3 checklist item: "gitleaks pre-commit hook in infra AND hyperguild").
|
||||
#
|
||||
# Ported from mathias/infra's .gitleaks.toml (2026-08-04) — same homelab
|
||||
# token-shape rules, minus the SOPS/searxng allowlists infra needed (this
|
||||
# repo doesn't use SOPS).
|
||||
|
||||
title = "hyperguild gitleaks config"
|
||||
|
||||
[extend]
|
||||
useDefault = true
|
||||
|
||||
# --- Homelab-specific rules -------------------------------------------------
|
||||
|
||||
[[rules]]
|
||||
id = "homelab-static-bearer"
|
||||
description = "Homelab MCP/LLM static bearer or API key assigned a long literal value"
|
||||
regex = '''(?i)\b(DMABE_[A-Z0-9_]+|[A-Z0-9_]*MCP_TOKEN|ROUTING_MCP_TOKEN|INFRA_MCP_TOKEN|BRAIN_MCP_TOKEN|GITEA_MCP_TOKEN|LITELLM_MASTER_KEY|LITELLM_SALT_KEY|DMABE_LLMAPI_KEY|BRAIN_PG_DSN)\s*[:=]\s*['"]?([A-Za-z0-9/_+.\-]{16,})['"]?'''
|
||||
keywords = ["dmabe_", "mcp_token", "litellm_master_key", "litellm_salt_key", "llmapi_key", "brain_pg_dsn"]
|
||||
[[rules.allowlists]]
|
||||
description = "Env indirection is not a literal secret"
|
||||
regexes = [
|
||||
'''os\.environ''',
|
||||
'''valueFrom''',
|
||||
'''secretKeyRef''',
|
||||
'''\$\{?[A-Za-z_][A-Za-z0-9_]*\}?''',
|
||||
'''REDACTED''',
|
||||
'''<[A-Z_]+>''',
|
||||
]
|
||||
|
||||
[[rules]]
|
||||
id = "homelab-authorization-bearer"
|
||||
description = "Hardcoded Authorization: Bearer header"
|
||||
regex = '''(?i)authorization['"]?\s*[:=]\s*['"]?bearer\s+([A-Za-z0-9/_+.\-=]{16,})'''
|
||||
keywords = ["authorization", "bearer"]
|
||||
[[rules.allowlists]]
|
||||
description = "Env indirection is not a literal secret"
|
||||
regexes = [
|
||||
'''\$\{?[A-Za-z_][A-Za-z0-9_]*\}?''',
|
||||
'''os\.environ''',
|
||||
'''REDACTED''',
|
||||
'''<[A-Z_]+>''',
|
||||
]
|
||||
|
||||
# --- Global allowlist: claudewatcher's own scrubber test fixtures ------------
|
||||
# ingestion/internal/claudewatcher/{scrubber,watcher}_test.go deliberately
|
||||
# contain fake secret-shaped literals to test that the scrubber detects and
|
||||
# redacts them. Verified 2026-08-04: all 9 findings here are test fixtures
|
||||
# (github-pat, jwt, generic-api-key, homelab-authorization-bearer rules) plus
|
||||
# 1 doc finding that was gitleaks matching the literal placeholder word
|
||||
# "REDACTED" in a plan doc — not a real secret in either case.
|
||||
[[allowlists]]
|
||||
description = "claudewatcher scrubber test fixtures — deliberately fake secrets"
|
||||
paths = [
|
||||
'''ingestion/internal/claudewatcher/scrubber_test\.go$''',
|
||||
'''ingestion/internal/claudewatcher/watcher_test\.go$''',
|
||||
]
|
||||
|
||||
[[allowlists]]
|
||||
description = "Literal placeholder word REDACTED matched as if it were a token (verified 2026-08-04: extracted Secret == 'REDACTED' exactly, gitleaks' curl-auth-header rule matched the placeholder text itself, not a real credential)"
|
||||
condition = "AND"
|
||||
paths = ['''docs/superpowers/plans/2026-04-22-phase4-attempt-wiring\.md$''']
|
||||
regexes = ['''REDACTED''']
|
||||
@@ -73,6 +73,14 @@ These rules apply to every task across every project, regardless of harness.
|
||||
- Surface the brain entries and issue numbers in the closing summary so the
|
||||
trail is auditable.
|
||||
|
||||
## CAD sprint patterns
|
||||
|
||||
Two CAD sprint archetypes (defs: brain `wiki/homelab/decisions/continuous-agentic-development-cad-concept-2026-06-16`):
|
||||
- **Inception Sprint** — birth a new repo. Oath-gated close; tag only once every clause holds. Brain: `.../inception-sprint-and-oath`.
|
||||
- **Sweep Sprint** — clear an existing backlog. Parallel read-only research per issue → sequential one-issue-per-commit execution. Brain: `.../sweep-sprint-and-oath`.
|
||||
|
||||
Shared: decisions that are yours, not the agent's, surface as `DECISION NEEDED` (question/options/recommendation/blocks) — ask live if you're present, else comment and wait, never act past it. Live-breaking regressions get `git revert`, never forward-fixed; host-network work skips even read-only research (solo, synchronous, you present). Rule 6's closing ritual only fires once every Oath clause genuinely holds — blocked = descoped-and-tracked, never marked done.
|
||||
|
||||
## Default stack
|
||||
|
||||
| Layer | Default | Fallback | Last resort |
|
||||
|
||||
@@ -5,6 +5,11 @@ Instead of letting Claude Code do whatever it wants, hyperguild enforces structu
|
||||
workflows (TDD red/green/refactor), logs every session, and accumulates learnings
|
||||
into a searchable brain.
|
||||
|
||||
**This repo is also where `brain-mcp` lives** — the MCP server at
|
||||
`brain-mcp.d-ma.be` that `brain_query`, `brain_write`, `brain_read`, `brain_edit`
|
||||
and `capture` are served from. It is not a separate deployable; see
|
||||
[Where brain-mcp lives](#where-brain-mcp-lives).
|
||||
|
||||
## Hypothesis
|
||||
|
||||
> We believe routing skill tasks through local models, backed by brain context,
|
||||
@@ -20,7 +25,7 @@ usage, the fast-model routing path should be reconsidered.
|
||||
|
||||
**hyperguild = Claude Code + MCP.** This is a supervisor for Claude Code sessions specifically.
|
||||
For multi-agent orchestration (OpenCode + LiteLLM, executor/reviewer pipelines), see
|
||||
[agentsquad](http://gitea.d-ma.be/mathias/agentsquad) — a separate harness for a different
|
||||
[agentsquad](https://git.d-ma.be/mathias/agentsquad) — a separate harness for a different
|
||||
orchestration model. Skills (mathias/skills) are shared between both.
|
||||
|
||||
## How it works
|
||||
@@ -44,6 +49,73 @@ brain/
|
||||
└── knowledge/ — legacy flat notes (migration pending: hyperguild#22)
|
||||
```
|
||||
|
||||
## Where brain-mcp lives
|
||||
|
||||
`brain-mcp` is served from **`ingestion/internal/mcp`** in this repo, at
|
||||
**`brain-mcp.d-ma.be`**. It is a component *inside* the `ingestion` service, not
|
||||
a standalone deployable — one pod in the `supervisor` namespace serves the MCP
|
||||
endpoint, the REST API, and the claudewatcher together.
|
||||
|
||||
| You are looking for | It is here |
|
||||
|---|---|
|
||||
| The MCP tool surface (`brain_query`, `brain_write`, `brain_read`, `brain_edit`, `capture`, …) | `ingestion/internal/mcp` |
|
||||
| The HTTP/REST API (`/query`, `/write`, `/ingest`) | `ingestion/internal/api` |
|
||||
| Session ingestion from Claude Code transcripts | `ingestion/internal/claudewatcher` |
|
||||
| The brain *content* (notes, wings, halls) | [`mathias/brain`](https://git.d-ma.be/mathias/brain) — a separate repo |
|
||||
| Deployment manifests | `config/supervisor` |
|
||||
|
||||
### Why it is not its own repo
|
||||
|
||||
Asked and decided in [#91](https://git.d-ma.be/mathias/hyperguild/issues/91)
|
||||
(part 2). `internal/mcp` imports 17 of ingestion's other internal packages and
|
||||
has exactly one importer — it is the MCP facade over the service's own
|
||||
internals, so "move it out" resolves to renaming `ingestion`, publishing 17
|
||||
packages as a shared library, or maintaining a shim. The discoverability problem
|
||||
that prompted the question is answered by naming it here instead.
|
||||
|
||||
Full reasoning, including the reversal trigger: brain
|
||||
`wiki/hyperguild/decisions/brain-mcp-stays-in-ingestion-fix-discoverability-by-name`.
|
||||
|
||||
## Ingest revises, it does not overwrite
|
||||
|
||||
Re-ingesting a source **merges into the page already on disk** rather than
|
||||
replacing it. Previously the write was unconditional, so a second ingest of the
|
||||
same source discarded everything accumulated since — including hand-written
|
||||
sections.
|
||||
|
||||
The merge is deliberately asymmetric in the existing page's favour:
|
||||
|
||||
| On the page | In the fresh extraction | Result |
|
||||
|---|---|---|
|
||||
| a section | the same section | **the on-disk version wins** |
|
||||
| — | a new section | added |
|
||||
| a bullet list (`Sources`, `Key Claims`, …) | more bullets | unioned, no duplicates |
|
||||
| an append section (`Open Questions`, `Updates`, …) | more lines | only genuinely new lines appended |
|
||||
| frontmatter | frontmatter | **the on-disk frontmatter wins** |
|
||||
|
||||
So an extraction can add to a note but never silently rewrite what a human put
|
||||
there, which is what makes it safe to run on every ingest rather than behind a
|
||||
review gate.
|
||||
|
||||
Re-ingesting unchanged content is **byte-identical** — no diff, so brain-sync
|
||||
does not commit noise that hides real edits.
|
||||
|
||||
Two fields are **generator-owned** and do get refreshed on a revise:
|
||||
`last_updated` and `source_hash`. Everything else on disk wins. The boundary is
|
||||
an explicit allowlist, so a field nobody has thought about stays human-owned.
|
||||
|
||||
## Re-ingesting an unchanged source is skipped
|
||||
|
||||
Ingest stores `source_hash` — sha256 of the ingested **body**, frontmatter
|
||||
excluded — on the source page. A later ingest of the same source compares it
|
||||
first and skips extraction entirely when it matches, reporting the skip rather
|
||||
than doing it silently.
|
||||
|
||||
Body-only is deliberate: a re-crawl that stamps a new fetch date has not changed
|
||||
the source, and treating that as drift would re-extract the corpus every time a
|
||||
crawler ran. A missing or unreadable hash always means "extract" — a missed skip
|
||||
costs one extraction, a wrong skip loses an update.
|
||||
|
||||
## Phase 1 tools (available now)
|
||||
|
||||
| Tool | What it does |
|
||||
@@ -58,7 +130,7 @@ brain/
|
||||
| `tier` | Returns the current connectivity tier (1=cloud, 2=LAN, 3=offline) |
|
||||
|
||||
> **Note:** `tdd_red/green/refactor` and `spec` were retired in Plan 7 (2026-05-12).
|
||||
> They are now SKILL.md files in [mathias/skills](http://gitea.d-ma.be/mathias/skills).
|
||||
> They are now SKILL.md files in [mathias/skills](https://git.d-ma.be/mathias/skills).
|
||||
|
||||
## Connect a project
|
||||
|
||||
@@ -147,7 +219,7 @@ The routing pod selects models per skill call based on historical pass rate:
|
||||
|
||||
## Open issues
|
||||
|
||||
See [issues](http://gitea.d-ma.be/mathias/hyperguild/issues) — key open items:
|
||||
See [issues](https://git.d-ma.be/mathias/hyperguild/issues) — key open items:
|
||||
|
||||
- **#25** — skills platform overhaul (audit first, then lazy loading + brain feedback loop)
|
||||
- **#24** — reduce context burn from skill listing
|
||||
|
||||
@@ -101,12 +101,48 @@ tasks:
|
||||
- task: lint
|
||||
- task: test
|
||||
- task: vet
|
||||
- task: security:gitleaks
|
||||
|
||||
# ── Security ─────────────────────────────────────────────
|
||||
security:gitleaks:
|
||||
desc: Scan the working tree for secrets (gitleaks, fail-closed; skipped if gitleaks absent)
|
||||
dir: '{{.ROOT_DIR}}'
|
||||
cmds:
|
||||
- |
|
||||
GL="$(command -v gitleaks || true)"
|
||||
[ -z "$GL" ] && [ -x "$(go env GOPATH 2>/dev/null)/bin/gitleaks" ] && GL="$(go env GOPATH)/bin/gitleaks"
|
||||
if [ -z "$GL" ]; then
|
||||
echo "⚠ gitleaks not installed — skipping secret scan (CI enforces it)."
|
||||
echo " Install: go install github.com/zricethezav/gitleaks/v8@latest"
|
||||
exit 0
|
||||
fi
|
||||
"$GL" detect --no-git --redact --config .gitleaks.toml --source .
|
||||
|
||||
security:gitleaks:history:
|
||||
desc: "One-time FULL-HISTORY secret audit (infra#39 rotation pass; not a per-push gate)"
|
||||
dir: '{{.ROOT_DIR}}'
|
||||
cmds:
|
||||
- |
|
||||
GL="$(command -v gitleaks || true)"
|
||||
[ -z "$GL" ] && [ -x "$(go env GOPATH 2>/dev/null)/bin/gitleaks" ] && GL="$(go env GOPATH)/bin/gitleaks"
|
||||
if [ -z "$GL" ]; then
|
||||
echo "gitleaks not installed: go install github.com/zricethezav/gitleaks/v8@latest" >&2
|
||||
exit 2
|
||||
fi
|
||||
echo "Scanning FULL git history (redacted). Known historical leaks are expected"
|
||||
echo "until the infra#39 rotation pass completes — triage against the rotation list."
|
||||
"$GL" detect --redact --config .gitleaks.toml
|
||||
|
||||
lint:
|
||||
cmds:
|
||||
- golangci-lint run ./...
|
||||
- cd ingestion && golangci-lint run ./...
|
||||
|
||||
lint:frontmatter:
|
||||
desc: Report brain notes whose YAML frontmatter does not parse (BRAIN=path, FIX=1 repairs in place)
|
||||
cmds:
|
||||
- cd ingestion && go run ./cmd/frontmatterlint {{if .FIX}}-fix{{end}} {{.BRAIN | default "../../brain"}}
|
||||
|
||||
test:
|
||||
cmds:
|
||||
- go test -race -count=1 ./...
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
// Command frontmatterlint reports markdown notes whose YAML frontmatter does
|
||||
// not parse, and optionally repairs the ones that only need quoting.
|
||||
//
|
||||
// go run ./cmd/frontmatterlint ../../brain # report
|
||||
// go run ./cmd/frontmatterlint -fix ../../brain # repair in place
|
||||
//
|
||||
// Exit codes: 0 clean (or everything repaired), 1 findings remain, 2 the scan
|
||||
// itself failed.
|
||||
package main
|
||||
|
||||
import (
|
||||
"flag"
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/frontmatter"
|
||||
)
|
||||
|
||||
func main() {
|
||||
fix := flag.Bool("fix", false, "repair files that quoting can rescue, in place")
|
||||
flag.Parse()
|
||||
|
||||
root := "."
|
||||
if flag.NArg() > 0 {
|
||||
root = flag.Arg(0)
|
||||
}
|
||||
|
||||
findings, err := frontmatter.Scan(root, *fix)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "frontmatterlint:", err)
|
||||
os.Exit(2)
|
||||
}
|
||||
|
||||
var unresolved int
|
||||
for _, f := range findings {
|
||||
if f.Repaired {
|
||||
fmt.Printf("repaired %s\n", f.Path)
|
||||
continue
|
||||
}
|
||||
unresolved++
|
||||
fmt.Printf("BROKEN %s: %v\n", f.Path, f.Err)
|
||||
}
|
||||
|
||||
if unresolved > 0 {
|
||||
fmt.Fprintf(os.Stderr, "\n%d file(s) with unparseable frontmatter", unresolved)
|
||||
if !*fix {
|
||||
fmt.Fprint(os.Stderr, " — re-run with -fix")
|
||||
}
|
||||
fmt.Fprintln(os.Stderr)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("✓ frontmatter: %d file(s) repaired, none broken\n", len(findings))
|
||||
}
|
||||
@@ -32,6 +32,7 @@ import (
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/reranker"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/startupretry"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/vectorstore"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/watcher"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/webhook"
|
||||
@@ -143,6 +144,16 @@ func envInt(key string, fallback int) int {
|
||||
return fallback
|
||||
}
|
||||
|
||||
// buildNotifier returns the ntfy notifier, or a nil interface when
|
||||
// BRAIN_NTFY_URL is unset. A nil *interface* (not a typed-nil) is required so
|
||||
// DegradingSink/Reconcile and the governance observer skip it cleanly.
|
||||
func buildNotifier() audit.Notifier {
|
||||
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
||||
return n
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// buildAuditSink selects the capture audit sink. When BRAIN_LOKI_URL is
|
||||
// set it builds the classification-aware DegradingSink (loki central +
|
||||
// durable file buffer + optional ntfy) and starts the reconcile loop;
|
||||
@@ -160,12 +171,7 @@ func buildAuditSink(ctx context.Context, brainDir string, logger *slog.Logger) c
|
||||
logger.Error("capture audit buffer init", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
// Keep notifier as a nil interface (not a typed-nil) when unconfigured
|
||||
// so DegradingSink/Reconcile skip it cleanly.
|
||||
var notifier audit.Notifier
|
||||
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
|
||||
notifier = n
|
||||
}
|
||||
notifier := buildNotifier()
|
||||
reconcileInterval := time.Duration(envInt("BRAIN_AUDIT_RECONCILE_INTERVAL", 60)) * time.Second
|
||||
audit.StartReconcile(ctx, central, buffer, notifier, reconcileInterval)
|
||||
logger.Info("capture audit: loki+buffer sink", "loki", lokiURL, "reconcile_s", int(reconcileInterval.Seconds()))
|
||||
@@ -314,6 +320,36 @@ func main() {
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
|
||||
// Built before the watcher, the graph backfill and session ingestion, and
|
||||
// retried rather than fatal (#83). Two separate reasons:
|
||||
//
|
||||
// - On a cold cluster boot this pod can beat dex to serving and gets a
|
||||
// 502 from ingress. Nothing is misconfigured; it simply started first.
|
||||
// Exiting there meant CrashLoopBackOff redoing the backfill on every
|
||||
// attempt, with exponential backoff stretching the brain outage.
|
||||
// - Constructing it first means a genuine config error (wrong issuer,
|
||||
// bad audience) fails in seconds instead of after all that work.
|
||||
//
|
||||
// The budget stays bounded: retrying forever would turn a misconfiguration
|
||||
// into a process that looks alive and never serves.
|
||||
jwtValidator, err := startupretry.Do(ctx,
|
||||
startupretry.Policy{Attempts: 8, Base: time.Second, Max: 30 * time.Second},
|
||||
func(ctx context.Context) (*chassisauth.JWTValidator, error) {
|
||||
v, err := chassisauth.NewJWTValidator(ctx, os.Getenv("DEX_ISSUER_URL"), os.Getenv("MCP_AUDIENCE"))
|
||||
if err != nil {
|
||||
logger.Warn("oidc discovery not ready, retrying", "err", err)
|
||||
}
|
||||
return v, err
|
||||
})
|
||||
if err != nil {
|
||||
logger.Error("build jwt validator", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if jwtValidator != nil {
|
||||
logger.Info("jwt auth enabled", "issuer", os.Getenv("DEX_ISSUER_URL"))
|
||||
}
|
||||
|
||||
if watchInterval > 0 {
|
||||
watcher.Start(ctx, watcher.Config{
|
||||
BrainDir: brainDir,
|
||||
@@ -418,15 +454,6 @@ func main() {
|
||||
if webhookHandler != nil {
|
||||
mux.Handle("POST /webhooks/brain-sync", webhookHandler)
|
||||
}
|
||||
jwtValidator, err := chassisauth.NewJWTValidator(ctx, os.Getenv("DEX_ISSUER_URL"), os.Getenv("MCP_AUDIENCE"))
|
||||
if err != nil {
|
||||
logger.Error("build jwt validator", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if jwtValidator != nil {
|
||||
logger.Info("jwt auth enabled", "issuer", os.Getenv("DEX_ISSUER_URL"))
|
||||
}
|
||||
|
||||
// Resource-metadata URL is only emitted on 401 when Dex OAuth is
|
||||
// configured. Static-Bearer-only deployments leave this empty so
|
||||
// clients never see an OAuth challenge.
|
||||
@@ -457,12 +484,8 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
auditSink := buildAuditSink(ctx, brainDir, logger)
|
||||
// The Gitea client also satisfies SummaryWriter (#66): session
|
||||
// summaries are written to mathias/ai-sessions over the same API
|
||||
// token. nil only if a future tracker impl lacks file writes.
|
||||
summaryWriter, _ := tracker.(capture.SummaryWriter)
|
||||
captureSvc := capture.NewService(
|
||||
mcpSrv.BrainStore(), tracker, summaryWriter, classCfg, auditSink)
|
||||
mcpSrv.BrainStore(), tracker, classCfg, auditSink)
|
||||
sovereign := splitList(os.Getenv("BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS"))
|
||||
resolver := capturehttp.NewOriginResolver(sovereign)
|
||||
captureH := capturehttp.New(captureSvc, jwtValidator, mcpToken, "local-cli", resolver)
|
||||
@@ -472,7 +495,13 @@ func main() {
|
||||
// the existing /mcp OAuth connector. mcpSrv is already wrapped above;
|
||||
// WithCapture mutates the same instance, so the tool appears live.
|
||||
mcpSrv.WithCapture(captureSvc, jwtValidator, mcpToken, "local-cli", resolver)
|
||||
logger.Info("capture enabled (REST + MCP tool)", "sovereign_principals", len(sovereign))
|
||||
// hyperguild#97: a governance-file rewrite gets the same trail a
|
||||
// capture already gets, plus an out-of-band alert. Wired here because
|
||||
// brain_edit_root needs the capture plumbing anyway — that is what
|
||||
// re-derives the static principal its gate checks.
|
||||
mcpSrv.WithGovernanceObserver(auditSink, buildNotifier())
|
||||
logger.Info("capture enabled (REST + MCP tool)", "sovereign_principals", len(sovereign),
|
||||
"governance_observer", true)
|
||||
} else {
|
||||
logger.Info("capture endpoint disabled (BRAIN_GITEA_TOKEN unset)")
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/claim"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/extract"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
|
||||
@@ -152,15 +153,24 @@ func writeHallNote(brainDir string, opts WriteNoteOptions) (string, error) {
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
|
||||
return "", fmt.Errorf("create hall dir: %w", err)
|
||||
}
|
||||
|
||||
existingFields, body := splitFrontmatter(opts.Content)
|
||||
existingByKey := make(map[string]frontmatterField, len(existingFields))
|
||||
for _, f := range existingFields {
|
||||
existingByKey[f.key] = f
|
||||
}
|
||||
|
||||
// Validate the claim-property envelope BEFORE creating the directory, so a
|
||||
// rejected write leaves nothing behind — not even an empty hall dir
|
||||
// (brain#21). Fail closed: a value outside the enum is worse than an absent
|
||||
// one, because a lint skips what it cannot parse and the note then reads as
|
||||
// classified while nothing acts on it.
|
||||
if err := claim.ValidateEnvelope(scalarValues(existingFields)); err != nil {
|
||||
return "", fmt.Errorf("claim envelope: %w", err)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
|
||||
return "", fmt.Errorf("create hall dir: %w", err)
|
||||
}
|
||||
emitted := make(map[string]bool, 6)
|
||||
|
||||
var fm strings.Builder
|
||||
@@ -218,6 +228,25 @@ func writeHallNote(brainDir string, opts WriteNoteOptions) (string, error) {
|
||||
return filepath.ToSlash(rel), nil
|
||||
}
|
||||
|
||||
// scalarValues flattens frontmatter fields to key -> scalar value for
|
||||
// validation. Multi-line values (lists, nested blocks) yield the text after
|
||||
// the colon on the first line, which is empty for a list — correct here,
|
||||
// since every claim-property field is a scalar enum.
|
||||
func scalarValues(fields []frontmatterField) map[string]string {
|
||||
out := make(map[string]string, len(fields))
|
||||
for _, f := range fields {
|
||||
if len(f.lines) == 0 {
|
||||
continue
|
||||
}
|
||||
_, value, found := strings.Cut(f.lines[0], ":")
|
||||
if !found {
|
||||
continue
|
||||
}
|
||||
out[f.key] = strings.TrimSpace(value)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// frontmatterField is one top-level YAML key from a frontmatter block,
|
||||
// along with its raw line and any indented continuation lines (e.g. a
|
||||
// bulleted list value spanning multiple lines).
|
||||
|
||||
@@ -440,3 +440,60 @@ func TestIngestPath_Directory(t *testing.T) {
|
||||
require.True(t, ok, "pages must be an array")
|
||||
assert.NotEmpty(t, pagesSlice)
|
||||
}
|
||||
|
||||
// --- brain#21: claim-property envelope on the write path -------------------
|
||||
|
||||
func TestWriteNote_PreservesClaimEnvelopeVerbatim(t *testing.T) {
|
||||
// writeHallNote already passes unknown keys through; this pins that
|
||||
// behaviour for the envelope specifically, so a future refactor of the
|
||||
// merge loop cannot silently drop it.
|
||||
dir := t.TempDir()
|
||||
|
||||
rel, err := api.WriteNote(dir, api.WriteNoteOptions{
|
||||
Content: "---\nhalf_life: durable\napplies: universal\nclaim_provenance: asserted\n---\n\nA durable claim.\n",
|
||||
Wing: "homelab",
|
||||
Hall: "decisions",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(got), "half_life: durable")
|
||||
assert.Contains(t, string(got), "applies: universal")
|
||||
assert.Contains(t, string(got), "claim_provenance: asserted")
|
||||
}
|
||||
|
||||
func TestWriteNote_RejectsInvalidClaimEnvelopeAndWritesNothing(t *testing.T) {
|
||||
// Fail closed: a value outside the enum must not land. A lint skips what
|
||||
// it cannot parse, so an unvalidated write produces a note that reads as
|
||||
// classified while nothing acts on it.
|
||||
dir := t.TempDir()
|
||||
|
||||
_, err := api.WriteNote(dir, api.WriteNoteOptions{
|
||||
Content: "---\nhalf_life: forever\nclaim_provenance: asserted\n---\n\nBody.\n",
|
||||
Filename: "bad-envelope",
|
||||
Wing: "homelab",
|
||||
Hall: "facts",
|
||||
})
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "half_life")
|
||||
|
||||
_, statErr := os.Stat(filepath.Join(dir, "wiki/homelab/facts/bad-envelope.md"))
|
||||
assert.True(t, os.IsNotExist(statErr), "a rejected write must leave no file behind")
|
||||
}
|
||||
|
||||
func TestWriteNote_RejectsClaimFieldWithoutProvenance(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
|
||||
_, err := api.WriteNote(dir, api.WriteNoteOptions{
|
||||
Content: "---\napplies: homelab\n---\n\nBody.\n",
|
||||
Filename: "no-provenance",
|
||||
Wing: "homelab",
|
||||
Hall: "facts",
|
||||
})
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "claim_provenance")
|
||||
|
||||
_, statErr := os.Stat(filepath.Join(dir, "wiki/homelab/facts/no-provenance.md"))
|
||||
assert.True(t, os.IsNotExist(statErr))
|
||||
}
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
// Package capture is the Clean-Architecture use-case for the uniform
|
||||
// capture capability (issue #49/#51): persist a finished session's
|
||||
// valuable output — insights → brain, action items → Gitea tickets,
|
||||
// optional summary → ai-sessions — with one invocation, identical core
|
||||
// behaviour across every harness.
|
||||
// valuable output — insights → brain, action items → Gitea tickets —
|
||||
// with one invocation, identical core behaviour across every harness.
|
||||
//
|
||||
// This package is pure orchestration. It depends only on ports
|
||||
// (interfaces) and plain entities — no HTTP, no live Gitea, no embedding
|
||||
@@ -83,19 +82,11 @@ type Ticket struct {
|
||||
Body string
|
||||
}
|
||||
|
||||
// Summary is an optional session summary bound for ai-sessions.
|
||||
type Summary struct {
|
||||
Title string
|
||||
Body string
|
||||
ReposTouched []string
|
||||
}
|
||||
|
||||
// CaptureInput is the whole capture request.
|
||||
type CaptureInput struct {
|
||||
Context CaptureContext
|
||||
Insights []Insight
|
||||
Tickets []Ticket
|
||||
Summary *Summary
|
||||
DryRun bool
|
||||
}
|
||||
|
||||
@@ -117,12 +108,6 @@ type TicketResult struct {
|
||||
OK bool `json:"ok"`
|
||||
}
|
||||
|
||||
// SummaryResult is the summary outcome in the receipt.
|
||||
type SummaryResult struct {
|
||||
Path string `json:"path,omitempty"`
|
||||
OK bool `json:"ok"`
|
||||
}
|
||||
|
||||
// ItemError pins a failure to a specific request item for the partial
|
||||
// receipt. Item is a stable locator like "insight[1]" or "ticket[0]".
|
||||
type ItemError struct {
|
||||
@@ -136,7 +121,6 @@ type ItemError struct {
|
||||
type CaptureReceipt struct {
|
||||
Insights []InsightResult `json:"insights"`
|
||||
Tickets []TicketResult `json:"tickets"`
|
||||
Summary *SummaryResult `json:"summary,omitempty"`
|
||||
Errors []ItemError `json:"errors"`
|
||||
EffectiveClassification string `json:"effective_classification,omitempty"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
|
||||
@@ -69,11 +69,6 @@ type IssueTracker interface {
|
||||
CommentIssue(ctx context.Context, repo string, number int, body string) (IssueRef, error)
|
||||
}
|
||||
|
||||
// SummaryWriter is the ai-sessions summary port.
|
||||
type SummaryWriter interface {
|
||||
WriteFile(ctx context.Context, repo, path, content string) error
|
||||
}
|
||||
|
||||
// ClassificationPolicy derives a target's sensitivity (model C). The
|
||||
// "stricter wins" combination of declared vs derived is use-case policy
|
||||
// and lives in the service, so the port stays minimal. Satisfied by
|
||||
|
||||
@@ -2,8 +2,6 @@ package capture
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -14,22 +12,19 @@ import (
|
||||
|
||||
// Service is the CaptureSession use-case. It depends only on ports.
|
||||
type Service struct {
|
||||
brain BrainStore
|
||||
issues IssueTracker
|
||||
summaries SummaryWriter
|
||||
policy ClassificationPolicy
|
||||
audit AuditSink
|
||||
brain BrainStore
|
||||
issues IssueTracker
|
||||
policy ClassificationPolicy
|
||||
audit AuditSink
|
||||
|
||||
// now is the clock, injectable for deterministic summary paths and
|
||||
// audit timestamps in tests.
|
||||
// now is the clock, injectable for deterministic audit timestamps in
|
||||
// tests.
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
// NewService constructs a Service from its ports. summaries may be nil
|
||||
// when no summary persistence is wired; a CaptureInput with a Summary
|
||||
// then fails that item rather than panicking.
|
||||
func NewService(b BrainStore, tr IssueTracker, sw SummaryWriter, p ClassificationPolicy, a AuditSink) *Service {
|
||||
return &Service{brain: b, issues: tr, summaries: sw, policy: p, audit: a, now: time.Now}
|
||||
// NewService constructs a Service from its ports.
|
||||
func NewService(b BrainStore, tr IssueTracker, p ClassificationPolicy, a AuditSink) *Service {
|
||||
return &Service{brain: b, issues: tr, policy: p, audit: a, now: time.Now}
|
||||
}
|
||||
|
||||
var validActions = map[string]bool{"create": true, "close": true, "comment": true}
|
||||
@@ -88,7 +83,7 @@ func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt,
|
||||
declared, _ = classification.ParseLevel(in.Context.Classification)
|
||||
}
|
||||
|
||||
effective, securityEvents := s.resolveClassification(declared, in)
|
||||
effective, securityEvents, escalators := s.resolveClassification(declared, in)
|
||||
|
||||
// Server-derived origin governs the I1 gate; a caller-asserted harness
|
||||
// label that names a different zone is descriptive-only and logged as a
|
||||
@@ -112,8 +107,8 @@ func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt,
|
||||
Items: nil, // refused before any write
|
||||
SecurityEvents: append(securityEvents, "I1 refusal: confidential capture via us-nexus origin"),
|
||||
}, AuditCentral)
|
||||
return CaptureReceipt{}, fmt.Errorf("%w: effective classification confidential through %s origin",
|
||||
ErrSovereigntyRefused, in.Context.Origin)
|
||||
return CaptureReceipt{}, fmt.Errorf("%w: effective classification confidential through %s origin (%s)",
|
||||
ErrSovereigntyRefused, in.Context.Origin, escalationReason(escalators))
|
||||
}
|
||||
|
||||
receipt := CaptureReceipt{
|
||||
@@ -131,9 +126,6 @@ func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt,
|
||||
for _, tk := range in.Tickets {
|
||||
receipt.Tickets = append(receipt.Tickets, TicketResult{Repo: tk.Repo, Action: tk.Action, Number: tk.Number, OK: true})
|
||||
}
|
||||
if in.Summary != nil {
|
||||
receipt.Summary = &SummaryResult{Path: s.summaryPath(in.Context, in.Summary), OK: true}
|
||||
}
|
||||
return receipt, nil
|
||||
}
|
||||
|
||||
@@ -169,16 +161,6 @@ func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt,
|
||||
landed = append(landed, fmt.Sprintf("ticket:%s#%d", tk.Repo, res.Number))
|
||||
}
|
||||
|
||||
if in.Summary != nil {
|
||||
res, err := s.persistSummary(ctx, in.Context, in.Summary)
|
||||
receipt.Summary = &res
|
||||
if err != nil {
|
||||
receipt.Errors = append(receipt.Errors, ItemError{Item: "summary", Error: err.Error()})
|
||||
} else {
|
||||
landed = append(landed, "summary:"+res.Path)
|
||||
}
|
||||
}
|
||||
|
||||
// I5: persist the request-level audit record of exactly what landed,
|
||||
// using the outcome reserved before the writes. AuditBuffered surfaces
|
||||
// the degraded (locally-buffered) state on the receipt.
|
||||
@@ -239,15 +221,28 @@ func (s *Service) validate(in CaptureInput) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveClassification computes the effective level (stricter of
|
||||
// declared and every target's derived level) and collects a security
|
||||
// event whenever the caller under-declared relative to a target floor.
|
||||
func (s *Service) resolveClassification(declared classification.Level, in CaptureInput) (classification.Level, []string) {
|
||||
// resolveClassification computes the effective level (stricter of declared
|
||||
// and every target's derived level), collects a security event whenever the
|
||||
// caller under-declared relative to a target floor, and names the targets
|
||||
// that carry the strictest level.
|
||||
//
|
||||
// That third return is what makes an I1 refusal actionable. A caller sending
|
||||
// twenty tickets gets one refusal; without the offending target it cannot
|
||||
// tell which repo tripped the gate, and infra#265 shows what happens next —
|
||||
// the caller concluded the tickets ARRAY was broken, built a fallback of
|
||||
// twenty individual calls, and filed a bug against the wrong thing.
|
||||
func (s *Service) resolveClassification(declared classification.Level, in CaptureInput) (classification.Level, []string, []string) {
|
||||
effective := declared
|
||||
var events []string
|
||||
derivedFor := map[string]classification.Level{}
|
||||
var order []string
|
||||
consider := func(kind classification.TargetKind, name string) {
|
||||
derived := s.policy.Derive(classification.Target{Kind: kind, Name: name})
|
||||
effective = classification.Stricter(effective, derived)
|
||||
if _, seen := derivedFor[name]; !seen {
|
||||
derivedFor[name] = derived
|
||||
order = append(order, name)
|
||||
}
|
||||
if declared < derived {
|
||||
events = append(events, fmt.Sprintf("classification under-declared: declared=%s target=%s(%s) derived=%s",
|
||||
declared, name, kindString(kind), derived))
|
||||
@@ -259,12 +254,28 @@ func (s *Service) resolveClassification(declared classification.Level, in Captur
|
||||
for _, tk := range in.Tickets {
|
||||
consider(classification.RepoTarget, tk.Repo)
|
||||
}
|
||||
if in.Summary != nil {
|
||||
for _, repo := range in.Summary.ReposTouched {
|
||||
consider(classification.RepoTarget, repo)
|
||||
|
||||
// Only targets that actually reach the effective level are named: at
|
||||
// twenty tickets, listing the innocent ones is noise that hides the one
|
||||
// that matters.
|
||||
var escalators []string
|
||||
for _, name := range order {
|
||||
if derivedFor[name] == effective && effective > declared {
|
||||
escalators = append(escalators, name)
|
||||
}
|
||||
}
|
||||
return effective, events
|
||||
return effective, events, escalators
|
||||
}
|
||||
|
||||
// escalationReason renders why the effective classification landed where it
|
||||
// did, for an operator reading a refusal. Naming the caller's own declaration
|
||||
// matters as much as naming a target: "you asked for this" and "a repo you
|
||||
// touched required it" call for different fixes.
|
||||
func escalationReason(escalators []string) string {
|
||||
if len(escalators) == 0 {
|
||||
return "declared by the caller; no target required it"
|
||||
}
|
||||
return "required by: " + strings.Join(escalators, ", ")
|
||||
}
|
||||
|
||||
func (s *Service) persistInsight(ctx context.Context, ins Insight) (InsightResult, string, error) {
|
||||
@@ -309,60 +320,6 @@ func (s *Service) persistTicket(ctx context.Context, tk Ticket) (TicketResult, e
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func (s *Service) persistSummary(ctx context.Context, c CaptureContext, sum *Summary) (SummaryResult, error) {
|
||||
if s.summaries == nil {
|
||||
return SummaryResult{OK: false}, fmt.Errorf("no summary writer configured")
|
||||
}
|
||||
path := s.summaryPath(c, sum)
|
||||
content := s.renderSummary(c, sum)
|
||||
repo := "ai-sessions"
|
||||
if err := s.summaries.WriteFile(ctx, repo, path, content); err != nil {
|
||||
return SummaryResult{Path: path, OK: false}, err
|
||||
}
|
||||
return SummaryResult{Path: path, OK: true}, nil
|
||||
}
|
||||
|
||||
// summaryPath builds summaries/<harness>/<YYYY-MM>/<date>-<slug>-<ref8>.md.
|
||||
// The ref8 disambiguator is derived from the session_ref (or the title
|
||||
// when no ref is present) so distinct sessions never collide.
|
||||
func (s *Service) summaryPath(c CaptureContext, sum *Summary) string {
|
||||
t := s.now().UTC()
|
||||
slug := brain.Sanitise(sum.Title)
|
||||
if slug == "" {
|
||||
slug = "summary"
|
||||
}
|
||||
seed := c.SessionRef
|
||||
if seed == "" {
|
||||
seed = sum.Title + sum.Body
|
||||
}
|
||||
sum8 := shortHash(seed)
|
||||
return fmt.Sprintf("summaries/%s/%s/%s-%s-%s.md",
|
||||
brain.Sanitise(c.Harness), t.Format("2006-01"), t.Format("2006-01-02"), slug, sum8)
|
||||
}
|
||||
|
||||
// renderSummary stamps fidelity + session metadata into frontmatter so the
|
||||
// richer-fidelity-supersedes-thinner collision rule has the data it needs.
|
||||
func (s *Service) renderSummary(c CaptureContext, sum *Summary) string {
|
||||
var b strings.Builder
|
||||
b.WriteString("---\n")
|
||||
fmt.Fprintf(&b, "title: %s\n", sum.Title)
|
||||
fmt.Fprintf(&b, "harness: %s\n", c.Harness)
|
||||
if c.SessionRef != "" {
|
||||
fmt.Fprintf(&b, "session_ref: %s\n", c.SessionRef)
|
||||
}
|
||||
fmt.Fprintf(&b, "fidelity: %s\n", c.Fidelity)
|
||||
fmt.Fprintf(&b, "captured_at: %s\n", s.now().UTC().Format(time.RFC3339))
|
||||
if len(sum.ReposTouched) > 0 {
|
||||
fmt.Fprintf(&b, "repos_touched: [%s]\n", strings.Join(sum.ReposTouched, ", "))
|
||||
}
|
||||
b.WriteString("---\n\n")
|
||||
b.WriteString(sum.Body)
|
||||
if !strings.HasSuffix(sum.Body, "\n") {
|
||||
b.WriteByte('\n')
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
func kindString(k classification.TargetKind) string {
|
||||
if k == classification.RepoTarget {
|
||||
return "repo"
|
||||
@@ -381,8 +338,3 @@ func firstLine(s string) string {
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func shortHash(s string) string {
|
||||
sum := sha256.Sum256([]byte(s))
|
||||
return hex.EncodeToString(sum[:])[:8]
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package capture
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -59,12 +60,18 @@ type fakeTracker struct {
|
||||
closed []int
|
||||
comments []int
|
||||
err error
|
||||
// failOnTitle makes CreateIssue fail for one specific title, so a
|
||||
// partial-failure receipt can be exercised without failing every item.
|
||||
failOnTitle string
|
||||
}
|
||||
|
||||
func (f *fakeTracker) CreateIssue(_ context.Context, repo, title, _ string) (IssueRef, error) {
|
||||
if f.err != nil {
|
||||
return IssueRef{}, f.err
|
||||
}
|
||||
if f.failOnTitle != "" && title == f.failOnTitle {
|
||||
return IssueRef{}, errors.New("tracker refused this one")
|
||||
}
|
||||
f.created = append(f.created, repo+":"+title)
|
||||
return IssueRef{Repo: repo, Number: 100 + len(f.created), URL: "https://git/" + repo + "/issues/x"}, nil
|
||||
}
|
||||
@@ -85,24 +92,11 @@ func (f *fakeTracker) CommentIssue(_ context.Context, repo string, number int, _
|
||||
return IssueRef{Repo: repo, Number: number}, nil
|
||||
}
|
||||
|
||||
type fakeSummary struct {
|
||||
paths []string
|
||||
content []string
|
||||
err error
|
||||
}
|
||||
|
||||
func (f *fakeSummary) WriteFile(_ context.Context, _, path, content string) error {
|
||||
if f.err != nil {
|
||||
return f.err
|
||||
}
|
||||
f.paths = append(f.paths, path)
|
||||
f.content = append(f.content, content)
|
||||
return nil
|
||||
}
|
||||
|
||||
// fakePolicy derives from an explicit map; default Internal so tests pin
|
||||
// behaviour without depending on the real defaulting.
|
||||
type fakePolicy struct{ tags map[string]classification.Level }
|
||||
type fakePolicy struct {
|
||||
tags map[string]classification.Level
|
||||
}
|
||||
|
||||
func (p fakePolicy) Derive(t classification.Target) classification.Level {
|
||||
if lvl, ok := p.tags[t.Name]; ok {
|
||||
@@ -135,8 +129,8 @@ func (f *fakeAudit) Record(_ context.Context, e AuditEntry, _ AuditOutcome) erro
|
||||
|
||||
// --- helpers ---
|
||||
|
||||
func newSvc(b BrainStore, tr IssueTracker, sw SummaryWriter, p ClassificationPolicy, a AuditSink) *Service {
|
||||
s := NewService(b, tr, sw, p, a)
|
||||
func newSvc(b BrainStore, tr IssueTracker, p ClassificationPolicy, a AuditSink) *Service {
|
||||
s := NewService(b, tr, p, a)
|
||||
s.now = func() time.Time { return time.Date(2026, 6, 22, 12, 0, 0, 0, time.UTC) }
|
||||
return s
|
||||
}
|
||||
@@ -151,7 +145,7 @@ func TestCaptureHappyPath(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{}
|
||||
svc := newSvc(b, tr, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -180,7 +174,7 @@ func TestCaptureHappyPath(t *testing.T) {
|
||||
|
||||
func TestCaptureSupersedeNotDuplicate(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
|
||||
svc := newSvc(b, &fakeTracker{}, fakePolicy{}, &fakeAudit{})
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -197,7 +191,7 @@ func TestCaptureValidationFailClosed(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{}
|
||||
svc := newSvc(b, tr, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
_, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -216,7 +210,7 @@ func TestCaptureValidationFailClosed(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestCaptureValidationRejectsBadTicket(t *testing.T) {
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, fakePolicy{}, &fakeAudit{})
|
||||
_, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
Tickets: []Ticket{{Repo: "hyperguild", Action: "frobnicate"}}, // bad action
|
||||
@@ -239,7 +233,7 @@ func TestCapturePartialFailureBestEffort(t *testing.T) {
|
||||
}}
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{}
|
||||
svc := newSvc(b, tr, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -264,7 +258,7 @@ func TestCaptureDryRunWritesNothing(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{}
|
||||
svc := newSvc(b, tr, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -287,7 +281,7 @@ func TestCaptureStricterClassificationWins(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
au := &fakeAudit{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
|
||||
svc := newSvc(b, &fakeTracker{}, nil, pol, au)
|
||||
svc := newSvc(b, &fakeTracker{}, pol, au)
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Classification = "internal"
|
||||
@@ -306,7 +300,7 @@ func TestCaptureCallerRaisingSensitivityHonoured(t *testing.T) {
|
||||
// Caller declares confidential; target internal → effective confidential, NOT a security event.
|
||||
au := &fakeAudit{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"hyperguild": classification.Internal}}
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, pol, au)
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, pol, au)
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Classification = "confidential"
|
||||
@@ -319,26 +313,6 @@ func TestCaptureCallerRaisingSensitivityHonoured(t *testing.T) {
|
||||
assert.Empty(t, au.entries[0].SecurityEvents, "raising sensitivity is honoured, not flagged")
|
||||
}
|
||||
|
||||
func TestCaptureSummaryPathAndFidelity(t *testing.T) {
|
||||
sw := &fakeSummary{}
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, sw, fakePolicy{}, &fakeAudit{})
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Fidelity = "transcript-parse"
|
||||
ctx.SessionRef = "abc123def456"
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: ctx,
|
||||
Summary: &Summary{Title: "Session Wrap", Body: "did stuff", ReposTouched: []string{"hyperguild"}},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.NotNil(t, rec.Summary)
|
||||
assert.True(t, rec.Summary.OK)
|
||||
require.Len(t, sw.paths, 1)
|
||||
assert.True(t, strings.HasPrefix(sw.paths[0], "summaries/claude-code/2026-06/"), "path: %s", sw.paths[0])
|
||||
assert.Contains(t, sw.paths[0], "session-wrap")
|
||||
assert.Contains(t, sw.content[0], "fidelity: transcript-parse", "fidelity stamped in frontmatter")
|
||||
}
|
||||
|
||||
// --- I1 sovereignty gate (#53) ---
|
||||
|
||||
func TestCaptureRefusesConfidentialViaUSNexus(t *testing.T) {
|
||||
@@ -346,7 +320,7 @@ func TestCaptureRefusesConfidentialViaUSNexus(t *testing.T) {
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
|
||||
svc := newSvc(b, tr, nil, pol, au)
|
||||
svc := newSvc(b, tr, pol, au)
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Classification = "confidential"
|
||||
@@ -368,7 +342,7 @@ func TestCaptureRefusesConfidentialViaUSNexus(t *testing.T) {
|
||||
func TestCaptureAllowsConfidentialViaSovereign(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
|
||||
svc := newSvc(b, &fakeTracker{}, nil, pol, &fakeAudit{})
|
||||
svc := newSvc(b, &fakeTracker{}, pol, &fakeAudit{})
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Classification = "confidential"
|
||||
@@ -387,7 +361,7 @@ func TestCaptureAssertedLabelIgnoredAndLogged(t *testing.T) {
|
||||
// us-nexus; confidential ⇒ refused, and the discrepancy is a security event.
|
||||
au := &fakeAudit{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, pol, au)
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, pol, au)
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Harness = "sovereign-soil" // asserted
|
||||
@@ -407,7 +381,7 @@ func TestCaptureAssertedLabelIgnoredAndLogged(t *testing.T) {
|
||||
func TestCaptureInternalViaUSNexusAllowed(t *testing.T) {
|
||||
// us-nexus origin is fine for non-confidential data.
|
||||
b := &fakeBrain{}
|
||||
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
|
||||
svc := newSvc(b, &fakeTracker{}, fakePolicy{}, &fakeAudit{})
|
||||
ctx := baseCtx()
|
||||
ctx.Origin = ZoneUSNexus // internal classification, so gate doesn't fire
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
@@ -426,7 +400,7 @@ func TestCaptureRefusesWhenAuditReserveFails(t *testing.T) {
|
||||
b := &fakeBrain{}
|
||||
tr := &fakeTracker{}
|
||||
au := &fakeAudit{reserveErr: errors.New("central sink unreachable")}
|
||||
svc := newSvc(b, tr, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
_, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -443,7 +417,7 @@ func TestCaptureFlagsLocallyBufferedAudit(t *testing.T) {
|
||||
// proceeds and the receipt flags the degraded audit state.
|
||||
b := &fakeBrain{}
|
||||
au := &fakeAudit{reserveMode: AuditBuffered}
|
||||
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, au)
|
||||
svc := newSvc(b, &fakeTracker{}, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -458,7 +432,7 @@ func TestCaptureFlagsLocallyBufferedAudit(t *testing.T) {
|
||||
func TestCaptureDryRunSkipsAuditGate(t *testing.T) {
|
||||
// dry_run must not even probe the audit sink (writes nothing anywhere).
|
||||
au := &fakeAudit{reserveErr: errors.New("would refuse")}
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, fakePolicy{}, au)
|
||||
svc := newSvc(&fakeBrain{}, &fakeTracker{}, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
@@ -469,3 +443,98 @@ func TestCaptureDryRunSkipsAuditGate(t *testing.T) {
|
||||
assert.True(t, rec.DryRun)
|
||||
assert.Empty(t, au.entries)
|
||||
}
|
||||
|
||||
// ---- hyperguild#91 part 1 -------------------------------------------------
|
||||
//
|
||||
// The reported symptom was "capture with a tickets array fails" (infra#265:
|
||||
// 20 tickets errored, 5 errored, caller fell back to 20 individual
|
||||
// issue_create calls). The array was never the failing dimension — every
|
||||
// ticket targeted a client repo, and the I1 gate refuses that through a
|
||||
// us-nexus origin at ANY length. The refusal did not say which target
|
||||
// escalated the classification, so it read as a malfunction.
|
||||
|
||||
func TestCapture_TicketsArrayWorksAtLengthOne(t *testing.T) {
|
||||
b, tr, au := &fakeBrain{}, &fakeTracker{}, &fakeAudit{}
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
Tickets: []Ticket{{Repo: "hyperguild", Action: "create", Title: "one", Body: "b"}},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
require.Len(t, rec.Tickets, 1)
|
||||
assert.True(t, rec.Tickets[0].OK)
|
||||
assert.Len(t, tr.created, 1)
|
||||
assert.Empty(t, rec.Errors)
|
||||
}
|
||||
|
||||
func TestCapture_TicketsArrayWorksAtLengthMany(t *testing.T) {
|
||||
b, tr, au := &fakeBrain{}, &fakeTracker{}, &fakeAudit{}
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
in := CaptureInput{Context: baseCtx()}
|
||||
for i := range 20 {
|
||||
in.Tickets = append(in.Tickets, Ticket{
|
||||
Repo: "hyperguild", Action: "create", Title: fmt.Sprintf("t%02d", i), Body: "b",
|
||||
})
|
||||
}
|
||||
|
||||
rec, err := svc.Capture(context.Background(), in)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, rec.Tickets, 20, "one receipt entry per ticket")
|
||||
for i, r := range rec.Tickets {
|
||||
assert.True(t, r.OK, "ticket %d failed: %+v", i, rec.Errors)
|
||||
}
|
||||
assert.Len(t, tr.created, 20)
|
||||
assert.Empty(t, rec.Errors)
|
||||
}
|
||||
|
||||
func TestCapture_OneFailingTicketDoesNotSinkTheRest(t *testing.T) {
|
||||
// infra#265 asked explicitly: "if ticket 3 of 5 fails, what happened to
|
||||
// 1 and 2?" They land, and the receipt says so per item.
|
||||
b, au := &fakeBrain{}, &fakeAudit{}
|
||||
tr := &fakeTracker{failOnTitle: "boom"}
|
||||
svc := newSvc(b, tr, fakePolicy{}, au)
|
||||
|
||||
rec, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: baseCtx(),
|
||||
Tickets: []Ticket{
|
||||
{Repo: "hyperguild", Action: "create", Title: "ok-1", Body: "b"},
|
||||
{Repo: "hyperguild", Action: "create", Title: "boom", Body: "b"},
|
||||
{Repo: "hyperguild", Action: "create", Title: "ok-3", Body: "b"},
|
||||
},
|
||||
})
|
||||
require.NoError(t, err, "a per-item failure is a partial receipt, not a call error")
|
||||
require.Len(t, rec.Tickets, 3)
|
||||
assert.True(t, rec.Tickets[0].OK)
|
||||
assert.False(t, rec.Tickets[1].OK)
|
||||
assert.True(t, rec.Tickets[2].OK)
|
||||
require.Len(t, rec.Errors, 1)
|
||||
assert.Contains(t, rec.Errors[0].Item, "ticket[1]")
|
||||
}
|
||||
|
||||
func TestCapture_I1RefusalNamesWhatEscalatedTheClassification(t *testing.T) {
|
||||
// The actual defect behind infra#265. Without the offending target in
|
||||
// the message, a caller with 20 tickets cannot tell which repo tripped
|
||||
// the gate — and reasonably concludes the array itself is broken.
|
||||
b, tr, au := &fakeBrain{}, &fakeTracker{}, &fakeAudit{}
|
||||
pol := fakePolicy{tags: map[string]classification.Level{"trucker-ecommerce": classification.Confidential}}
|
||||
svc := newSvc(b, tr, pol, au)
|
||||
|
||||
ctx := baseCtx()
|
||||
ctx.Origin = ZoneUSNexus
|
||||
_, err := svc.Capture(context.Background(), CaptureInput{
|
||||
Context: ctx,
|
||||
Tickets: []Ticket{
|
||||
{Repo: "hyperguild", Action: "create", Title: "fine", Body: "b"},
|
||||
{Repo: "trucker-ecommerce", Action: "create", Title: "client work", Body: "b"},
|
||||
},
|
||||
})
|
||||
|
||||
require.ErrorIs(t, err, ErrSovereigntyRefused)
|
||||
assert.Contains(t, err.Error(), "trucker-ecommerce",
|
||||
"the refusal must name the target that escalated the classification")
|
||||
assert.NotContains(t, err.Error(), "hyperguild",
|
||||
"only the offending target, or the message is noise at 20 tickets")
|
||||
assert.Empty(t, tr.created, "refused before any write")
|
||||
}
|
||||
|
||||
@@ -55,7 +55,6 @@ type request struct {
|
||||
Context contextBody `json:"context"`
|
||||
Insights []insightBody `json:"insights"`
|
||||
Tickets []ticketBody `json:"tickets"`
|
||||
Summary *summaryBody `json:"summary,omitempty"`
|
||||
DryRun bool `json:"dry_run"`
|
||||
}
|
||||
|
||||
@@ -82,12 +81,6 @@ type ticketBody struct {
|
||||
Body string `json:"body,omitempty"`
|
||||
}
|
||||
|
||||
type summaryBody struct {
|
||||
Title string `json:"title"`
|
||||
Body string `json:"body"`
|
||||
ReposTouched []string `json:"repos_touched,omitempty"`
|
||||
}
|
||||
|
||||
// ServeHTTP authenticates, derives origin, runs the use-case, and maps the
|
||||
// result to an HTTP status.
|
||||
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -183,11 +176,6 @@ func (b request) toInput() capture.CaptureInput {
|
||||
Repo: t.Repo, Action: t.Action, Number: t.Number, Title: t.Title, Body: t.Body,
|
||||
})
|
||||
}
|
||||
if b.Summary != nil {
|
||||
in.Summary = &capture.Summary{
|
||||
Title: b.Summary.Title, Body: b.Summary.Body, ReposTouched: b.Summary.ReposTouched,
|
||||
}
|
||||
}
|
||||
return in
|
||||
}
|
||||
|
||||
@@ -204,9 +192,6 @@ func statusFor(rec capture.CaptureReceipt) int {
|
||||
for _, t := range rec.Tickets {
|
||||
count(&ok, &fail, t.OK)
|
||||
}
|
||||
if rec.Summary != nil {
|
||||
count(&ok, &fail, rec.Summary.OK)
|
||||
}
|
||||
switch {
|
||||
case fail == 0:
|
||||
return http.StatusOK
|
||||
|
||||
@@ -49,7 +49,7 @@ func newHandler(t *testing.T, v capturehttp.Validator, tr capture.IssueTracker,
|
||||
t.Helper()
|
||||
cfg, err := classification.Load(t.TempDir())
|
||||
require.NoError(t, err)
|
||||
svc := capture.NewService(brainstore.New(t.TempDir()), tr, nil, cfg, audit.NewSlogSink(nil))
|
||||
svc := capture.NewService(brainstore.New(t.TempDir()), tr, cfg, audit.NewSlogSink(nil))
|
||||
return capturehttp.New(svc, v, staticTok, "local-cli", capturehttp.NewOriginResolver(sovereign))
|
||||
}
|
||||
|
||||
@@ -190,7 +190,7 @@ func (refusingAudit) Record(context.Context, capture.AuditEntry, capture.AuditOu
|
||||
func TestAuditUnavailableIs503(t *testing.T) {
|
||||
cfg, err := classification.Load(t.TempDir())
|
||||
require.NoError(t, err)
|
||||
svc := capture.NewService(brainstore.New(t.TempDir()), fakeTracker{}, nil, cfg, refusingAudit{})
|
||||
svc := capture.NewService(brainstore.New(t.TempDir()), fakeTracker{}, cfg, refusingAudit{})
|
||||
h := capturehttp.New(svc, nil, staticTok, "local-cli", capturehttp.NewOriginResolver(nil))
|
||||
|
||||
rr := do(t, h, "Bearer "+staticTok, internalReq())
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
// Package claim validates the claim-property frontmatter that says when and
|
||||
// where a note's claim holds, and who decided that (brain#21).
|
||||
//
|
||||
// Two families were designed together and land separately. This is the
|
||||
// applicability envelope:
|
||||
//
|
||||
// half_life: volatile | seasonal | durable — how fast it decays
|
||||
// applies: local | homelab | universal — how widely it holds
|
||||
// claim_provenance: asserted | inferred — who set the above
|
||||
//
|
||||
// The strength family (confidence, contested) follows once the ingest-time
|
||||
// contradiction detector exists to populate it.
|
||||
//
|
||||
// Why the envelope is validated rather than merely accepted: a value outside
|
||||
// the enum is worse than an absent one. A lint skips what it cannot parse, so
|
||||
// the note reads as classified while nothing acts on it — the field becomes
|
||||
// decoration that suppresses the very review it was added to trigger.
|
||||
//
|
||||
// Why `applies` and not `scope`: `scope:` has been in use since brain#5 for
|
||||
// glossary terms (`homelab-core`, `project:<repo>`) — a disjoint value space
|
||||
// on the same key. The shipped schema keeps the name; this one takes another.
|
||||
package claim
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Field names, exported so writers and lints agree on the spelling.
|
||||
const (
|
||||
FieldHalfLife = "half_life"
|
||||
FieldApplies = "applies"
|
||||
FieldProvenance = "claim_provenance"
|
||||
)
|
||||
|
||||
var (
|
||||
halfLives = []string{"volatile", "seasonal", "durable"}
|
||||
appliesTo = []string{"local", "homelab", "universal"}
|
||||
provenances = []string{"asserted", "inferred"}
|
||||
)
|
||||
|
||||
// ValidateEnvelope checks the envelope fields of one note's frontmatter.
|
||||
//
|
||||
// Absent fields are valid: the envelope is opt-in for new writes, and a
|
||||
// corpus-wide backfill is deliberately out of scope (brain#21) — inferring
|
||||
// 1113 notes' properties in bulk has the same failure mode as the extraction
|
||||
// pass that produced 28 wrong wikilinks out of 67.
|
||||
func ValidateEnvelope(fm map[string]string) error {
|
||||
halfLife, hasHalfLife := present(fm, FieldHalfLife)
|
||||
applies, hasApplies := present(fm, FieldApplies)
|
||||
prov, hasProv := present(fm, FieldProvenance)
|
||||
|
||||
if hasHalfLife {
|
||||
if err := oneOf(FieldHalfLife, halfLife, halfLives); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if hasApplies {
|
||||
if err := oneOf(FieldApplies, applies, appliesTo); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if hasProv {
|
||||
if err := oneOf(FieldProvenance, prov, provenances); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
// The field and its provenance travel together or not at all. A value
|
||||
// whose author is unknown cannot be trusted differently from one Mathias
|
||||
// asserted, which is how an inferred guess hardens into accepted fact.
|
||||
switch {
|
||||
case (hasHalfLife || hasApplies) && !hasProv:
|
||||
return fmt.Errorf("%s is required when %s or %s is set",
|
||||
FieldProvenance, FieldHalfLife, FieldApplies)
|
||||
case hasProv && !hasHalfLife && !hasApplies:
|
||||
return fmt.Errorf("%s is set but neither %s nor %s is",
|
||||
FieldProvenance, FieldHalfLife, FieldApplies)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// present reports a trimmed field value and whether it was set to anything
|
||||
// non-empty. A key written with an empty value is treated as absent rather
|
||||
// than as an error: that is a caller emitting a placeholder, not asserting.
|
||||
func present(fm map[string]string, key string) (string, bool) {
|
||||
v := strings.TrimSpace(fm[key])
|
||||
return v, v != ""
|
||||
}
|
||||
|
||||
func oneOf(field, value string, allowed []string) error {
|
||||
for _, a := range allowed {
|
||||
if a == value {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("%s: %q is not one of %s", field, value, strings.Join(allowed, ", "))
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
// Package claim_test covers the claim-property envelope (brain#21).
|
||||
package claim_test
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/claim"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestValidateEnvelope(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
fm map[string]string
|
||||
wantErr string
|
||||
}{
|
||||
{
|
||||
name: "absent is fine — the fields are opt-in on new writes",
|
||||
fm: map[string]string{"wing": "homelab", "hall": "facts"},
|
||||
},
|
||||
{
|
||||
name: "complete envelope with provenance",
|
||||
fm: map[string]string{
|
||||
"half_life": "durable", "applies": "universal", "claim_provenance": "asserted",
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "inferred provenance is equally valid — it just must be stated",
|
||||
fm: map[string]string{
|
||||
"half_life": "volatile", "applies": "local", "claim_provenance": "inferred",
|
||||
},
|
||||
},
|
||||
{
|
||||
// A value nobody can act on is worse than no value: a lint
|
||||
// silently skips it, so the note looks classified and isn't.
|
||||
name: "unknown half_life value",
|
||||
fm: map[string]string{"half_life": "forever", "claim_provenance": "asserted"},
|
||||
wantErr: `half_life: "forever" is not one of volatile, seasonal, durable`,
|
||||
},
|
||||
{
|
||||
name: "unknown applies value",
|
||||
fm: map[string]string{"applies": "galaxy", "claim_provenance": "asserted"},
|
||||
wantErr: `applies: "galaxy" is not one of local, homelab, universal`,
|
||||
},
|
||||
{
|
||||
// brain#21: a value a model inferred must never be
|
||||
// indistinguishable from one Mathias asserted. Writing the field
|
||||
// without saying who set it is exactly that laundering.
|
||||
name: "envelope field without provenance",
|
||||
fm: map[string]string{"half_life": "durable"},
|
||||
wantErr: "claim_provenance is required when half_life or applies is set",
|
||||
},
|
||||
{
|
||||
name: "applies without provenance",
|
||||
fm: map[string]string{"applies": "homelab"},
|
||||
wantErr: "claim_provenance is required when half_life or applies is set",
|
||||
},
|
||||
{
|
||||
name: "unknown provenance value",
|
||||
fm: map[string]string{"half_life": "durable", "claim_provenance": "probably"},
|
||||
wantErr: `claim_provenance: "probably" is not one of asserted, inferred`,
|
||||
},
|
||||
{
|
||||
// Provenance alone says who set fields that aren't there.
|
||||
name: "provenance without any envelope field",
|
||||
fm: map[string]string{"claim_provenance": "asserted"},
|
||||
wantErr: "claim_provenance is set but neither half_life nor applies is",
|
||||
},
|
||||
{
|
||||
// The glossary wing has used scope: homelab-core since #5. The
|
||||
// envelope deliberately does NOT claim that key (brain#21).
|
||||
name: "glossary scope is untouched",
|
||||
fm: map[string]string{"scope": "homelab-core", "term": "koala"},
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := claim.ValidateEnvelope(tt.fm)
|
||||
if tt.wantErr == "" {
|
||||
assert.NoError(t, err)
|
||||
return
|
||||
}
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), tt.wantErr)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
// Package frontmatter lints and repairs the YAML frontmatter block of a
|
||||
// markdown note.
|
||||
//
|
||||
// Notes written directly — hand-authored, or promoted from brain/raw/ — do not
|
||||
// pass through pipeline.BuildPages, so nothing quotes their scalars. A title
|
||||
// containing an unescaped ": " then produces a block that yaml.Unmarshal
|
||||
// rejects with "mapping values are not allowed here", and every consumer that
|
||||
// reads frontmatter silently skips the note.
|
||||
//
|
||||
// Fix is deliberately conservative: it only adds quoting, never reorders or
|
||||
// rewrites, and it refuses to return a document that still does not parse.
|
||||
package frontmatter
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
const fence = "---"
|
||||
|
||||
// Lint reports whether doc's frontmatter block parses as YAML.
|
||||
//
|
||||
// A document with no frontmatter block, or with an unterminated one, is not an
|
||||
// error: there is nothing claiming to be YAML.
|
||||
func Lint(doc []byte) error {
|
||||
b, ok := block(doc)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
var v any
|
||||
if err := yaml.Unmarshal(b, &v); err != nil {
|
||||
return fmt.Errorf("frontmatter: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Fix quotes unquoted scalars whose value breaks YAML parsing and returns the
|
||||
// repaired document.
|
||||
//
|
||||
// It returns (doc, false, nil) when the document already parses, and
|
||||
// (doc, false, err) when quoting cannot rescue it — an unfixable document is
|
||||
// always returned byte-identical, never half-repaired.
|
||||
func Fix(doc []byte) ([]byte, bool, error) {
|
||||
if err := Lint(doc); err == nil {
|
||||
return doc, false, nil
|
||||
}
|
||||
|
||||
b, ok := block(doc)
|
||||
if !ok {
|
||||
// Lint only fails when there is a block, so this is unreachable in
|
||||
// practice; return the original rather than guess.
|
||||
return doc, false, nil
|
||||
}
|
||||
|
||||
fixed := quoteScalars(b)
|
||||
if bytes.Equal(fixed, b) {
|
||||
return doc, false, fmt.Errorf("frontmatter: not repairable by quoting: %w", Lint(doc))
|
||||
}
|
||||
|
||||
var v any
|
||||
if err := yaml.Unmarshal(fixed, &v); err != nil {
|
||||
return doc, false, fmt.Errorf("frontmatter: quoting did not repair the block: %w", err)
|
||||
}
|
||||
|
||||
_, rest, _ := split(doc)
|
||||
out := []byte(fence + "\n")
|
||||
out = append(out, fixed...)
|
||||
out = append(out, []byte(fence)...)
|
||||
out = append(out, rest...)
|
||||
return out, true, nil
|
||||
}
|
||||
|
||||
// mappingLine matches "key: value" at any indentation, capturing the prefix so
|
||||
// it can be re-emitted untouched.
|
||||
//
|
||||
// Sequence items ("- value") are deliberately not handled: "- foo: bar" is
|
||||
// valid YAML, so a document containing one does not reach Fix in the first
|
||||
// place. See the Lint table for why that gap is left open.
|
||||
var mappingLine = regexp.MustCompile(`^(\s*[A-Za-z_][\w.-]*:[ \t]+)(\S.*)$`)
|
||||
|
||||
// quoteScalars single-quotes every unquoted scalar value in the block that
|
||||
// contains a colon. Values that are already quoted, or that carry YAML syntax
|
||||
// this package has no business rewriting, are left alone.
|
||||
func quoteScalars(block []byte) []byte {
|
||||
lines := strings.Split(string(block), "\n")
|
||||
for i, line := range lines {
|
||||
m := mappingLine.FindStringSubmatch(line)
|
||||
if m == nil {
|
||||
continue
|
||||
}
|
||||
prefix, value := m[1], m[2]
|
||||
if !needsQuoting(value) {
|
||||
continue
|
||||
}
|
||||
lines[i] = prefix + "'" + strings.ReplaceAll(value, "'", "''") + "'"
|
||||
}
|
||||
return []byte(strings.Join(lines, "\n"))
|
||||
}
|
||||
|
||||
// needsQuoting reports whether a scalar value has to be quoted to load as the
|
||||
// string it plainly is.
|
||||
//
|
||||
// The test is YAML's own parser rather than a pattern: ask whether the value
|
||||
// stands alone as the right-hand side of a mapping. Heuristics get this wrong
|
||||
// in both directions — `title: "Foo" — bar` opens with a quote but is not
|
||||
// quoted, and a value may contain a colon inside quotes and be perfectly fine.
|
||||
//
|
||||
// Anything that already parses is left alone, whatever it parses into — a flow
|
||||
// collection, a number, a bool. It is not broken, and quoting would change its
|
||||
// type.
|
||||
func needsQuoting(value string) bool {
|
||||
if strings.HasPrefix(value, "|") || strings.HasPrefix(value, ">") {
|
||||
return false // block scalar; its content is not parsed as YAML
|
||||
}
|
||||
var probe map[string]any
|
||||
return yaml.Unmarshal([]byte("v: "+value+"\n"), &probe) != nil
|
||||
}
|
||||
|
||||
// block returns the raw YAML between the opening and closing fences.
|
||||
func block(doc []byte) ([]byte, bool) {
|
||||
b, _, ok := split(doc)
|
||||
return b, ok
|
||||
}
|
||||
|
||||
// split returns the frontmatter block and everything from the closing fence
|
||||
// onward (the closing fence itself excluded, its trailing newline included).
|
||||
func split(doc []byte) (block, rest []byte, ok bool) {
|
||||
s := string(doc)
|
||||
if !strings.HasPrefix(s, fence+"\n") {
|
||||
return nil, nil, false
|
||||
}
|
||||
body := s[len(fence)+1:]
|
||||
|
||||
idx := strings.Index(body, "\n"+fence)
|
||||
if idx < 0 {
|
||||
return nil, nil, false
|
||||
}
|
||||
return []byte(body[:idx+1]), []byte(body[idx+1+len(fence):]), true
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
// ingestion/internal/frontmatter/frontmatter_test.go
|
||||
package frontmatter
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func TestLint(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
doc string
|
||||
wantErr bool
|
||||
}{
|
||||
{
|
||||
name: "no frontmatter block is not an error",
|
||||
doc: "# Just a heading\n\nSome body text.\n",
|
||||
},
|
||||
{
|
||||
name: "quoted title parses",
|
||||
doc: "---\ntitle: 'Headless dispatch runs: Bash is denied'\n---\n\nbody\n",
|
||||
},
|
||||
{
|
||||
name: "plain title parses",
|
||||
doc: "---\ntitle: A simple title\nwing: homelab\n---\n\nbody\n",
|
||||
},
|
||||
{
|
||||
name: "unquoted colon in title does not parse",
|
||||
doc: "---\ntitle: Headless dispatch runs: Bash is denied — use gitea MCP\nwing: homelab\n---\n\nbody\n",
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
// Known gap, deliberately not fixed here: this parses, but the
|
||||
// item becomes map[string]any{"Headless dispatch": "denied"}
|
||||
// rather than a string. It is a silent type corruption, not a
|
||||
// parse failure, so repairing it would rewrite documents that
|
||||
// currently load — out of scope for a lint about broken YAML.
|
||||
name: "unquoted colon in a list item parses as a mapping, not an error",
|
||||
doc: "---\ntitle: Fine\naliases:\n - Headless dispatch: denied\n---\n\nbody\n",
|
||||
},
|
||||
{
|
||||
name: "unterminated frontmatter is not treated as frontmatter",
|
||||
doc: "---\ntitle: no closing fence\n\nbody\n",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
err := Lint([]byte(tt.doc))
|
||||
if tt.wantErr {
|
||||
require.Error(t, err)
|
||||
return
|
||||
}
|
||||
require.NoError(t, err)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFix(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
doc string
|
||||
wantChanged bool
|
||||
wantKey string
|
||||
wantValue string
|
||||
}{
|
||||
{
|
||||
name: "quotes an unquoted title containing a colon",
|
||||
doc: "---\ntitle: Headless dispatch runs: Bash is denied — use gitea MCP\nwing: homelab\n---\n\nbody\n",
|
||||
wantChanged: true,
|
||||
wantKey: "title",
|
||||
wantValue: "Headless dispatch runs: Bash is denied — use gitea MCP",
|
||||
},
|
||||
{
|
||||
// Parses already (see the Lint table), so Fix must leave it alone
|
||||
// rather than rewrite a document that loads.
|
||||
name: "leaves a colon-bearing list item alone because it already parses",
|
||||
doc: "---\ntitle: Fine\naliases:\n - Headless dispatch: denied\n---\n\nbody\n",
|
||||
wantChanged: false,
|
||||
wantKey: "title",
|
||||
wantValue: "Fine",
|
||||
},
|
||||
{
|
||||
name: "escapes an embedded single quote when quoting",
|
||||
doc: "---\ntitle: Don't do this: it breaks\n---\n\nbody\n",
|
||||
wantChanged: true,
|
||||
wantKey: "title",
|
||||
wantValue: "Don't do this: it breaks",
|
||||
},
|
||||
{
|
||||
// The value opens with a double quote and then keeps going after
|
||||
// the closing one, so YAML reads it as a quoted scalar followed by
|
||||
// junk. Treating "starts with a quote" as "already quoted" leaves
|
||||
// this broken forever.
|
||||
name: "quotes a value that only looks quoted",
|
||||
doc: "---\ntitle: \"Evaluation Driven Generation\" — a TDD-style scaffold\ntags: [a, b]\n---\n\nbody\n",
|
||||
wantChanged: true,
|
||||
wantKey: "title",
|
||||
wantValue: "\"Evaluation Driven Generation\" — a TDD-style scaffold",
|
||||
},
|
||||
{
|
||||
name: "leaves an already-valid document untouched",
|
||||
doc: "---\ntitle: 'Already quoted: fine'\nwing: homelab\n---\n\nbody\n",
|
||||
wantChanged: false,
|
||||
wantKey: "title",
|
||||
wantValue: "Already quoted: fine",
|
||||
},
|
||||
{
|
||||
name: "leaves a document with no frontmatter untouched",
|
||||
doc: "# Heading\n\nbody: with a colon\n",
|
||||
wantChanged: false,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got, changed, err := Fix([]byte(tt.doc))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, tt.wantChanged, changed)
|
||||
|
||||
if !tt.wantChanged {
|
||||
assert.Equal(t, tt.doc, string(got), "unchanged docs must be returned byte-identical")
|
||||
}
|
||||
|
||||
// Whatever came back must parse — that is the whole point.
|
||||
require.NoError(t, Lint(got), "Fix must not return a document that still fails to parse")
|
||||
|
||||
if tt.wantKey == "" {
|
||||
return
|
||||
}
|
||||
block, ok := block(got)
|
||||
require.True(t, ok)
|
||||
var m map[string]any
|
||||
require.NoError(t, yaml.Unmarshal(block, &m))
|
||||
assert.Equal(t, tt.wantValue, m[tt.wantKey], "the value must survive quoting unchanged")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFix_LeavesStructuredValuesStructured(t *testing.T) {
|
||||
// Only the broken scalar may be touched. A flow sequence on a neighbouring
|
||||
// line still has to load as a sequence afterwards, not as a string.
|
||||
doc := "---\ntitle: Broken: here\ntags: [prompting, llm]\ncount: 3\n---\n\nbody\n"
|
||||
|
||||
got, changed, err := Fix([]byte(doc))
|
||||
require.NoError(t, err)
|
||||
require.True(t, changed)
|
||||
|
||||
b, ok := block(got)
|
||||
require.True(t, ok)
|
||||
var m map[string]any
|
||||
require.NoError(t, yaml.Unmarshal(b, &m))
|
||||
|
||||
assert.Equal(t, "Broken: here", m["title"])
|
||||
assert.Equal(t, []any{"prompting", "llm"}, m["tags"])
|
||||
assert.Equal(t, 3, m["count"])
|
||||
}
|
||||
|
||||
func TestFix_PreservesBodyExactly(t *testing.T) {
|
||||
body := "\nThe body has: colons, 'quotes', and\n---\na horizontal rule.\n"
|
||||
doc := "---\ntitle: Broken: title\n---" + body
|
||||
|
||||
got, changed, err := Fix([]byte(doc))
|
||||
require.NoError(t, err)
|
||||
require.True(t, changed)
|
||||
assert.True(t, len(got) > 0)
|
||||
|
||||
_, rest, ok := split([]byte(doc))
|
||||
require.True(t, ok)
|
||||
_, gotRest, ok := split(got)
|
||||
require.True(t, ok)
|
||||
assert.Equal(t, string(rest), string(gotRest), "the body must not be touched")
|
||||
}
|
||||
|
||||
func TestFix_RefusesToReturnAnUnparseableResult(t *testing.T) {
|
||||
// A frontmatter block that quoting cannot rescue: the structure itself is
|
||||
// broken, not a scalar. Fix must return the original and an error rather
|
||||
// than a half-repaired document.
|
||||
doc := "---\ntitle: fine\n\tbad: tab-indented mapping\n - stray list item\n---\n\nbody\n"
|
||||
|
||||
got, changed, err := Fix([]byte(doc))
|
||||
require.Error(t, err)
|
||||
assert.False(t, changed)
|
||||
assert.Equal(t, doc, string(got), "an unfixable document must come back byte-identical")
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
// ingestion/internal/frontmatter/scan.go
|
||||
package frontmatter
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// Finding is one markdown file whose frontmatter does not parse.
|
||||
type Finding struct {
|
||||
Path string // relative to the scanned root
|
||||
Err error // why it does not parse, or why the repair was refused
|
||||
Repaired bool // true only when fix was requested and the file was rewritten
|
||||
}
|
||||
|
||||
// Scan walks root for *.md files whose frontmatter fails to parse.
|
||||
//
|
||||
// With fix set, each finding that quoting can rescue is rewritten in place;
|
||||
// anything quoting cannot rescue is reported and left byte-identical. A file
|
||||
// that cannot be read or decoded is reported as a finding rather than aborting
|
||||
// the walk — one poisoned note must not hide the other findings.
|
||||
func Scan(root string, fix bool) ([]Finding, error) {
|
||||
var found []Finding
|
||||
|
||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() || !strings.EqualFold(filepath.Ext(path), ".md") {
|
||||
return nil
|
||||
}
|
||||
|
||||
rel, relErr := filepath.Rel(root, path)
|
||||
if relErr != nil {
|
||||
rel = path
|
||||
}
|
||||
|
||||
doc, readErr := os.ReadFile(path)
|
||||
if readErr != nil {
|
||||
found = append(found, Finding{Path: rel, Err: readErr})
|
||||
return nil
|
||||
}
|
||||
if !utf8.Valid(doc) {
|
||||
found = append(found, Finding{Path: rel, Err: fmt.Errorf("not valid UTF-8")})
|
||||
return nil
|
||||
}
|
||||
if lintErr := Lint(doc); lintErr == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
if !fix {
|
||||
found = append(found, Finding{Path: rel, Err: Lint(doc)})
|
||||
return nil
|
||||
}
|
||||
|
||||
repaired, changed, fixErr := Fix(doc)
|
||||
if fixErr != nil || !changed {
|
||||
found = append(found, Finding{Path: rel, Err: fixErr})
|
||||
return nil
|
||||
}
|
||||
if writeErr := writeFilePreservingMode(path, repaired); writeErr != nil {
|
||||
found = append(found, Finding{Path: rel, Err: writeErr})
|
||||
return nil
|
||||
}
|
||||
found = append(found, Finding{Path: rel, Repaired: true})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("scan %s: %w", root, err)
|
||||
}
|
||||
return found, nil
|
||||
}
|
||||
|
||||
func writeFilePreservingMode(path string, content []byte) error {
|
||||
mode := fs.FileMode(0o644)
|
||||
if info, err := os.Stat(path); err == nil {
|
||||
mode = info.Mode().Perm()
|
||||
}
|
||||
return os.WriteFile(path, content, mode)
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
// ingestion/internal/frontmatter/scan_test.go
|
||||
package frontmatter
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func writeTree(t *testing.T, files map[string]string) string {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
for name, content := range files {
|
||||
path := filepath.Join(root, name)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
|
||||
require.NoError(t, os.WriteFile(path, []byte(content), 0o644))
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
const (
|
||||
brokenDoc = "---\ntitle: Broken: title here\nwing: homelab\n---\n\nbody\n"
|
||||
validDoc = "---\ntitle: 'Fine: quoted'\n---\n\nbody\n"
|
||||
)
|
||||
|
||||
func TestScan_ReportsOnlyBrokenFiles(t *testing.T) {
|
||||
root := writeTree(t, map[string]string{
|
||||
"wiki/a/broken.md": brokenDoc,
|
||||
"wiki/a/valid.md": validDoc,
|
||||
"wiki/b/plain.md": "# no frontmatter\n\nbody: with colon\n",
|
||||
"wiki/a/notes.txt": brokenDoc, // not markdown — must be ignored
|
||||
})
|
||||
|
||||
found, err := Scan(root, false)
|
||||
require.NoError(t, err)
|
||||
|
||||
require.Len(t, found, 1)
|
||||
assert.Equal(t, "wiki/a/broken.md", found[0].Path)
|
||||
assert.False(t, found[0].Repaired)
|
||||
assert.Error(t, found[0].Err)
|
||||
|
||||
// Dry run must not touch the file.
|
||||
got, err := os.ReadFile(filepath.Join(root, "wiki/a/broken.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, brokenDoc, string(got))
|
||||
}
|
||||
|
||||
func TestScan_FixWritesRepairedFilesAndLeavesTheRestAlone(t *testing.T) {
|
||||
root := writeTree(t, map[string]string{
|
||||
"wiki/a/broken.md": brokenDoc,
|
||||
"wiki/a/valid.md": validDoc,
|
||||
})
|
||||
|
||||
found, err := Scan(root, true)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, found, 1)
|
||||
assert.True(t, found[0].Repaired)
|
||||
assert.NoError(t, found[0].Err)
|
||||
|
||||
repaired, err := os.ReadFile(filepath.Join(root, "wiki/a/broken.md"))
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, Lint(repaired), "the repaired file must parse")
|
||||
assert.Contains(t, string(repaired), "title: 'Broken: title here'")
|
||||
assert.Contains(t, string(repaired), "wing: homelab", "untouched keys must survive")
|
||||
|
||||
untouched, err := os.ReadFile(filepath.Join(root, "wiki/a/valid.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, validDoc, string(untouched))
|
||||
|
||||
// Second pass finds nothing — the repair is idempotent and complete.
|
||||
again, err := Scan(root, true)
|
||||
require.NoError(t, err)
|
||||
assert.Empty(t, again)
|
||||
}
|
||||
|
||||
func TestScan_ReportsUnfixableWithoutWriting(t *testing.T) {
|
||||
unfixable := "---\ntitle: fine\n\tbad: tab-indented\n - stray\n---\n\nbody\n"
|
||||
root := writeTree(t, map[string]string{"wiki/x.md": unfixable})
|
||||
|
||||
found, err := Scan(root, true)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, found, 1)
|
||||
assert.False(t, found[0].Repaired)
|
||||
assert.Error(t, found[0].Err)
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(root, "wiki/x.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, unfixable, string(got), "an unfixable file must be left exactly as it was")
|
||||
}
|
||||
|
||||
func TestScan_ReportsUndecodableFilesInsteadOfFailingTheRun(t *testing.T) {
|
||||
root := writeTree(t, map[string]string{"wiki/ok.md": validDoc})
|
||||
require.NoError(t, os.WriteFile(filepath.Join(root, "wiki/bad-bytes.md"),
|
||||
[]byte{'-', '-', '-', '\n', 0xff, 0xfe, '\n', '-', '-', '-', '\n'}, 0o644))
|
||||
|
||||
found, err := Scan(root, false)
|
||||
require.NoError(t, err, "one unreadable file must not abort the whole scan")
|
||||
require.Len(t, found, 1)
|
||||
assert.Equal(t, "wiki/bad-bytes.md", found[0].Path)
|
||||
}
|
||||
@@ -165,7 +165,7 @@ func TestBrainUpdateReembedsNewContent(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
|
||||
callResult(t, toolCall(t, srv, "brain_write", map[string]any{
|
||||
"content": "# Note\n\nthe OLD distinctive payload\n",
|
||||
"content": "# Note\n\nthe OLD distinctive payload\n",
|
||||
"filename": "n", "wing": "a", "hall": "facts",
|
||||
}))
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/extract"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/graphsync"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/notes"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/session"
|
||||
@@ -97,6 +98,8 @@ func (s *Server) tools() []map[string]any {
|
||||
"slug": str("optional target slug; defaults to filename minus date prefix"),
|
||||
}),
|
||||
},
|
||||
noteToolDescriptors()[0],
|
||||
noteToolDescriptors()[1],
|
||||
{
|
||||
"name": "brain_tunnel",
|
||||
"description": "Create an explicit bidirectional [[wikilink]] between two notes in different wings. Idempotent.",
|
||||
@@ -105,13 +108,6 @@ func (s *Server) tools() []map[string]any {
|
||||
"target": str("path of target note (must be in a different wing)"),
|
||||
}),
|
||||
},
|
||||
{
|
||||
"name": "brain_index",
|
||||
"description": "Regenerate _index.md (Map of Content) for one or all wings under brain/wiki/. Auto-called after brain_write with wing+hall.",
|
||||
"inputSchema": schema([]string{}, map[string]any{
|
||||
"wing": str("optional wing to index; if absent, rebuilds every wing"),
|
||||
}),
|
||||
},
|
||||
{
|
||||
"name": "brain_ingest_raw",
|
||||
"description": "Ingest pre-structured pages into the brain wiki, bypassing the LLM extraction step.",
|
||||
@@ -327,6 +323,15 @@ func (s *Server) brainGet(ctx context.Context, args json.RawMessage) (json.RawMe
|
||||
if target == "" {
|
||||
return nil, fmt.Errorf("id or path is required")
|
||||
}
|
||||
// Containment: the store resolves any brain-relative path, and
|
||||
// parseFrontmatter returns the WHOLE file as body when there is no
|
||||
// frontmatter fence — so without this, brain_get reads verbatim session
|
||||
// transcripts under archive/ and sessions/, which BM25 never indexes and
|
||||
// which nobody therefore thinks of as reachable. Same allowlist as
|
||||
// brain_read (brain#4).
|
||||
if _, err := notes.Resolve(s.brainDir, target, notes.ScopeNotes); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
note, err := s.store.Get(ctx, target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
|
||||
)
|
||||
|
||||
// pendingNudgeThreshold is how old a raw/ note has to be before brain_context
|
||||
// mentions the queue. Two weeks: long enough that a note written this sprint
|
||||
// does not nag, short enough that four items cannot sit unnoticed for three
|
||||
// months again (#82).
|
||||
const pendingNudgeThreshold = 14 * 24 * time.Hour
|
||||
|
||||
// PendingNudge is the in-session reminder that the promote/reject queue has
|
||||
// items waiting. It is advisory: a count, an age, and how to look.
|
||||
type PendingNudge struct {
|
||||
Pending int `json:"pending"` // everything in the queue
|
||||
Aging int `json:"aging"` // those past the threshold
|
||||
Oldest string `json:"oldest"` // filename of the oldest pending note
|
||||
OldestDays int `json:"oldest_days"` // its age, in whole days
|
||||
Hint string `json:"hint"`
|
||||
}
|
||||
|
||||
// pendingNudge summarises the raw/ review queue, or returns nil when there is
|
||||
// nothing worth saying.
|
||||
//
|
||||
// The trigger is the missing half of the queue (#82): brain_pending has always
|
||||
// existed, and four real items still sat unreviewed from 2026-04-23 to
|
||||
// 2026-07-15 because nothing ever prompted anyone to call it. The system
|
||||
// should be what remembers the queue exists.
|
||||
//
|
||||
// Threshold-gated on purpose. A notice on every context call is one people
|
||||
// learn to skip, which reproduces the original failure with extra steps.
|
||||
//
|
||||
// It never promotes, rejects, or reorders anything. Age is a reason to ask a
|
||||
// human, not a verdict — an old note may be exactly right to promote once
|
||||
// someone finally looks at it.
|
||||
func pendingNudge(brainDir string, now time.Time, threshold time.Duration) *PendingNudge {
|
||||
pending, err := api.ListPending(brainDir)
|
||||
if err != nil || len(pending) == 0 {
|
||||
// brain_context is a hot path; a broken queue degrades to silence
|
||||
// rather than failing the call that asked for project context.
|
||||
return nil
|
||||
}
|
||||
|
||||
var aging int
|
||||
var oldestName string
|
||||
var oldestAge time.Duration
|
||||
|
||||
for _, p := range pending {
|
||||
created, perr := time.Parse(time.RFC3339, p.CreatedAt)
|
||||
if perr != nil {
|
||||
continue // an unparseable timestamp is not evidence of age
|
||||
}
|
||||
age := now.Sub(created)
|
||||
if age >= threshold {
|
||||
aging++
|
||||
}
|
||||
if age > oldestAge {
|
||||
oldestAge, oldestName = age, p.Filename
|
||||
}
|
||||
}
|
||||
|
||||
if aging == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &PendingNudge{
|
||||
Pending: len(pending),
|
||||
Aging: aging,
|
||||
Oldest: oldestName,
|
||||
OldestDays: int(oldestAge.Hours() / 24),
|
||||
Hint: "call brain_pending to review; nothing is promoted or rejected automatically",
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// writePending drops a raw/ note whose created_at is age old.
|
||||
func writePending(t *testing.T, brainDir, name string, age time.Duration, now time.Time) {
|
||||
t.Helper()
|
||||
raw := filepath.Join(brainDir, "raw")
|
||||
require.NoError(t, os.MkdirAll(raw, 0o755))
|
||||
body := "---\ncreated_at: " + now.Add(-age).UTC().Format(time.RFC3339) + "\n---\n\nnote body\n"
|
||||
require.NoError(t, os.WriteFile(filepath.Join(raw, name), []byte(body), 0o644))
|
||||
}
|
||||
|
||||
func TestPendingNudge_SilentWhenNothingIsPending(t *testing.T) {
|
||||
// No queue, no nudge. A notice that fires every session is one people
|
||||
// learn to skip, which is how the queue got ignored for three months in
|
||||
// the first place (#82).
|
||||
got := pendingNudge(t.TempDir(), time.Now(), 14*24*time.Hour)
|
||||
assert.Nil(t, got)
|
||||
}
|
||||
|
||||
func TestPendingNudge_SilentWhileEverythingIsBelowTheThreshold(t *testing.T) {
|
||||
now := time.Now()
|
||||
dir := t.TempDir()
|
||||
writePending(t, dir, "fresh.md", 2*24*time.Hour, now)
|
||||
|
||||
got := pendingNudge(dir, now, 14*24*time.Hour)
|
||||
assert.Nil(t, got, "a two-day-old note is not aging")
|
||||
}
|
||||
|
||||
func TestPendingNudge_ReportsCountAndOldestAgeOnceSomethingAges(t *testing.T) {
|
||||
now := time.Now()
|
||||
dir := t.TempDir()
|
||||
writePending(t, dir, "old.md", 100*24*time.Hour, now)
|
||||
writePending(t, dir, "older.md", 120*24*time.Hour, now)
|
||||
writePending(t, dir, "fresh.md", 1*24*time.Hour, now)
|
||||
|
||||
got := pendingNudge(dir, now, 14*24*time.Hour)
|
||||
require.NotNil(t, got)
|
||||
|
||||
// All three are pending; the threshold decides whether to speak, not
|
||||
// what to count. Someone opening the queue sees the whole queue.
|
||||
assert.Equal(t, 3, got.Pending)
|
||||
assert.Equal(t, 2, got.Aging)
|
||||
assert.Equal(t, 120, got.OldestDays)
|
||||
assert.Equal(t, "older.md", got.Oldest)
|
||||
}
|
||||
|
||||
func TestPendingNudge_NeverDecidesAnything(t *testing.T) {
|
||||
// Aging is a trigger to ask a human, never a verdict. The nudge must
|
||||
// carry no action and must leave the queue exactly as it found it.
|
||||
now := time.Now()
|
||||
dir := t.TempDir()
|
||||
writePending(t, dir, "ancient.md", 365*24*time.Hour, now)
|
||||
|
||||
before, err := os.ReadDir(filepath.Join(dir, "raw"))
|
||||
require.NoError(t, err)
|
||||
|
||||
got := pendingNudge(dir, now, 14*24*time.Hour)
|
||||
require.NotNil(t, got)
|
||||
|
||||
after, err := os.ReadDir(filepath.Join(dir, "raw"))
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, after, len(before), "the nudge must not promote or reject anything")
|
||||
assert.Contains(t, got.Hint, "brain_pending", "the nudge should say how to look, not act")
|
||||
}
|
||||
|
||||
func TestPendingNudge_SurvivesAnUnreadableQueue(t *testing.T) {
|
||||
// brain_context is a hot path; a broken raw/ dir must degrade to silence
|
||||
// rather than failing the whole context call.
|
||||
dir := t.TempDir()
|
||||
require.NoError(t, os.WriteFile(filepath.Join(dir, "raw"), []byte("not a directory"), 0o644))
|
||||
|
||||
assert.NotPanics(t, func() {
|
||||
assert.Nil(t, pendingNudge(dir, time.Now(), 14*24*time.Hour))
|
||||
})
|
||||
}
|
||||
|
||||
func TestBrainContext_CarriesTheNudgeWithoutFailingWhenTheQueueIsEmpty(t *testing.T) {
|
||||
// The wiring, not the summary: a nudge that never reaches a caller is the
|
||||
// same as no nudge, and brain_context must still work with no queue.
|
||||
dir := t.TempDir()
|
||||
srv := NewServer(dir, nil, nil, nil)
|
||||
|
||||
raw, err := srv.brainContext(context.Background(),
|
||||
json.RawMessage(`{"project_root":"/home/mathias/dev/AGENTS/hyperguild"}`))
|
||||
require.NoError(t, err)
|
||||
|
||||
var empty map[string]any
|
||||
require.NoError(t, json.Unmarshal(raw, &empty))
|
||||
assert.NotContains(t, empty, "pending_review", "silence when nothing is aging")
|
||||
|
||||
writePending(t, dir, "ancient.md", 200*24*time.Hour, time.Now())
|
||||
|
||||
raw, err = srv.brainContext(context.Background(),
|
||||
json.RawMessage(`{"project_root":"/home/mathias/dev/AGENTS/hyperguild"}`))
|
||||
require.NoError(t, err)
|
||||
|
||||
var withQueue map[string]any
|
||||
require.NoError(t, json.Unmarshal(raw, &withQueue))
|
||||
nudge, ok := withQueue["pending_review"].(map[string]any)
|
||||
require.True(t, ok, "the nudge must reach the caller: %v", withQueue)
|
||||
assert.Equal(t, float64(1), nudge["aging"])
|
||||
assert.Equal(t, "ancient.md", nudge["oldest"])
|
||||
}
|
||||
@@ -1,9 +1,14 @@
|
||||
// Package mcp implements an MCP HTTP handler for the ingestion service.
|
||||
// Exposed tools: brain_query, brain_write, brain_update, brain_get,
|
||||
// brain_pending, brain_promote, brain_index, brain_tunnel, brain_ingest,
|
||||
// brain_pending, brain_promote, brain_tunnel, brain_ingest,
|
||||
// brain_ingest_raw, brain_answer, brain_classify, brain_graph,
|
||||
// brain_context, session_log, and capture (the #55 relay tool, registered
|
||||
// only when WithCapture is set).
|
||||
//
|
||||
// brain_index dispatches but is deliberately absent from tools/list: it is
|
||||
// auto-called after every write, so surfacing it as a caller verb is
|
||||
// cognitive load with no decision behind it (ADR-0012 rule 4, #78). It stays
|
||||
// callable for maintenance — de-surfacing hides the verb, not the operation.
|
||||
package mcp
|
||||
|
||||
import (
|
||||
@@ -44,6 +49,8 @@ type rpcError struct {
|
||||
// Server handles MCP JSON-RPC over HTTP for the ingestion service.
|
||||
type Server struct {
|
||||
brainDir string
|
||||
govAudit capture.AuditSink
|
||||
govNotify govNotifier
|
||||
pipeline pipeline.Config
|
||||
llm pipeline.CompleteFunc
|
||||
answerLLM pipeline.CompleteFunc // nil = brain_answer and brain_classify unavailable
|
||||
@@ -270,6 +277,15 @@ func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessa
|
||||
return s.brainPromote(ctx, args)
|
||||
case "capture":
|
||||
return s.brainCapture(ctx, args)
|
||||
case "brain_read":
|
||||
return s.brainRead(ctx, args)
|
||||
case "brain_edit":
|
||||
return s.brainEdit(ctx, args)
|
||||
case "brain_edit_root":
|
||||
// Dispatches but is intentionally absent from tools/list — see
|
||||
// tools_notes.go for why a governance-file editor must not be
|
||||
// discoverable by a model.
|
||||
return s.brainEditRoot(ctx, args)
|
||||
case "brain_index":
|
||||
return s.brainIndex(ctx, args)
|
||||
case "brain_tunnel":
|
||||
|
||||
@@ -59,11 +59,43 @@ func TestServerToolsList(t *testing.T) {
|
||||
assert.ElementsMatch(t, []string{
|
||||
"brain_query", "brain_write", "brain_update", "brain_get",
|
||||
"brain_pending", "brain_promote",
|
||||
"brain_index", "brain_tunnel",
|
||||
"brain_read", "brain_edit",
|
||||
"brain_tunnel",
|
||||
"brain_ingest_raw", "brain_ingest",
|
||||
"brain_answer", "brain_classify", "brain_graph", "brain_context",
|
||||
"session_log",
|
||||
}, names)
|
||||
|
||||
// brain_index is auto-called after every brain_write, so surfacing it as
|
||||
// a caller verb is cognitive load with no decision behind it (ADR-0012
|
||||
// rule 4, #78). The capability stays reachable — see the admin test below.
|
||||
assert.NotContains(t, names, "brain_index")
|
||||
|
||||
// brain_edit_root is de-surfaced for a sharper reason than brain_index:
|
||||
// brain content is ingested from the outside world and reaches a model's
|
||||
// context, so a listed governance-file editor would be a durable
|
||||
// prompt-injection path — one poisoned page rewriting the CLAUDE.md a
|
||||
// .dispatch-allow agent later loads as instructions.
|
||||
assert.NotContains(t, names, "brain_edit_root")
|
||||
}
|
||||
|
||||
func TestBrainIndexStaysCallableForAdminUseAfterDeSurfacing(t *testing.T) {
|
||||
// De-surfacing hides the verb from the model-facing list; it must not
|
||||
// remove the operation. A maintenance caller still needs to rebuild a
|
||||
// Map of Content by hand.
|
||||
srv := mcp.NewServer(t.TempDir(), nil, nil, nil)
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
|
||||
"jsonrpc": "2.0", "id": 3, "method": "tools/call",
|
||||
"params": map[string]any{"name": "brain_index", "arguments": map[string]any{}},
|
||||
}))
|
||||
rr := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rr, req)
|
||||
|
||||
require.Equal(t, http.StatusOK, rr.Code)
|
||||
var resp map[string]any
|
||||
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
|
||||
assert.NotContains(t, resp, "error", "brain_index must still dispatch: %v", resp["error"])
|
||||
}
|
||||
|
||||
func TestServerNotificationGetsNoBody(t *testing.T) {
|
||||
|
||||
@@ -31,7 +31,7 @@ func captureToolDescriptor() map[string]any {
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"text": str("the insight body"), "wing": str("brain wing"),
|
||||
"hall": str("brain hall (facts/decisions/failures/hypotheses/sources)"),
|
||||
"hall": str("brain hall (facts/decisions/failures/hypotheses/sources)"),
|
||||
"supersede_slug": str("optional: slug of a prior note to revise in place instead of creating"),
|
||||
},
|
||||
"required": []string{"text", "wing", "hall"},
|
||||
@@ -41,7 +41,7 @@ func captureToolDescriptor() map[string]any {
|
||||
"properties": map[string]any{
|
||||
"repo": str("gitea repo (owner is always mathias)"), "action": str("create|close|comment"),
|
||||
"number": map[string]any{"type": "integer", "description": "issue number (close/comment)"},
|
||||
"title": str("issue title (create)"), "body": str("issue/comment body"),
|
||||
"title": str("issue title (create)"), "body": str("issue/comment body"),
|
||||
},
|
||||
"required": []string{"repo", "action"},
|
||||
}
|
||||
@@ -51,24 +51,20 @@ func captureToolDescriptor() map[string]any {
|
||||
"context": map[string]any{
|
||||
"type": "object",
|
||||
"properties": map[string]any{
|
||||
"harness": str("descriptive harness label (telemetry only, never a gate input)"),
|
||||
"harness": str("descriptive harness label (telemetry only, never a gate input)"),
|
||||
"session_ref": str("optional session reference"), "fidelity": str("live-capture|transcript-parse|agent-runlog"),
|
||||
"actor": str("acting user/agent"), "classification": str("caller-declared sensitivity: public|internal|confidential"),
|
||||
},
|
||||
},
|
||||
"insights": map[string]any{"type": "array", "items": insightItem},
|
||||
"tickets": map[string]any{"type": "array", "items": ticketItem},
|
||||
"summary": map[string]any{"type": "object", "properties": map[string]any{
|
||||
"title": str("summary title"), "body": str("summary body"),
|
||||
"repos_touched": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
|
||||
}},
|
||||
"dry_run": map[string]any{"type": "boolean", "description": "validate + return the would-be receipt, write nothing"},
|
||||
"dry_run": map[string]any{"type": "boolean", "description": "validate + return the would-be receipt, write nothing"},
|
||||
},
|
||||
}
|
||||
b, _ := json.Marshal(schema)
|
||||
return map[string]any{
|
||||
"name": "capture",
|
||||
"description": "Persist a session's value uniformly: insights → brain (write or supersede), action items → Gitea tickets, optional summary → ai-sessions. The relay door for MCP-native harnesses. Origin is server-derived from your authenticated identity; confidential captures through a us-nexus surface are refused (I1). Returns a partial-aware receipt.",
|
||||
"description": "Persist a session's value uniformly: insights → brain (write or supersede), action items → Gitea tickets. The relay door for MCP-native harnesses. Origin is server-derived from your authenticated identity; confidential captures through a us-nexus surface are refused (I1). Returns a partial-aware receipt.",
|
||||
"inputSchema": json.RawMessage(b),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -49,7 +49,7 @@ func captureServer(t *testing.T, validator capturehttp.Validator, sovereign []st
|
||||
brainDir := t.TempDir()
|
||||
cfg, err := classification.Load(brainDir)
|
||||
require.NoError(t, err)
|
||||
svc := capture.NewService(brainstore.New(brainDir), capFakeTracker{}, nil, cfg, audit.NewSlogSink(nil))
|
||||
svc := capture.NewService(brainstore.New(brainDir), capFakeTracker{}, cfg, audit.NewSlogSink(nil))
|
||||
srv := mcp.NewServer(brainDir, nil, nil, nil)
|
||||
srv.WithCapture(svc, validator, capStaticTok, "local-cli", capturehttp.NewOriginResolver(sovereign))
|
||||
return srv, brainDir
|
||||
|
||||
@@ -8,8 +8,10 @@ import (
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/textutil"
|
||||
)
|
||||
|
||||
// brainContextArgs is the input shape of brain_context. project_root is
|
||||
@@ -155,17 +157,21 @@ func (s *Server) brainContext(ctx context.Context, args json.RawMessage) (json.R
|
||||
entries = entries[:limit]
|
||||
}
|
||||
|
||||
return json.Marshal(map[string]any{"entries": entries})
|
||||
out := map[string]any{"entries": entries}
|
||||
// The queue nudge rides along with a call the session already makes, so
|
||||
// noticing an aging note costs nobody a decision to go looking (#82).
|
||||
// Advisory only, and absent entirely when there is nothing to say.
|
||||
if n := pendingNudge(s.brainDir, time.Now(), pendingNudgeThreshold); n != nil {
|
||||
out["pending_review"] = n
|
||||
}
|
||||
return json.Marshal(out)
|
||||
}
|
||||
|
||||
// truncateExcerpt clamps an already-stripped excerpt to maxLen characters
|
||||
// without re-running the frontmatter parser. The ellipsis suffix matches
|
||||
// the convention used in search.excerpt.
|
||||
func truncateExcerpt(s string, maxLen int) string {
|
||||
if len(s) <= maxLen {
|
||||
return s
|
||||
}
|
||||
return s[:maxLen] + "…"
|
||||
return textutil.Truncate(s, maxLen)
|
||||
}
|
||||
|
||||
// readExcerpt loads a doc relative to brainDir, strips its frontmatter,
|
||||
@@ -185,10 +191,7 @@ func readExcerpt(brainDir, relPath string, maxLen int) string {
|
||||
if len(parts) == 3 {
|
||||
body = strings.TrimSpace(parts[2])
|
||||
}
|
||||
if len(body) > maxLen {
|
||||
return body[:maxLen] + "…"
|
||||
}
|
||||
return body
|
||||
return textutil.Truncate(body, maxLen)
|
||||
}
|
||||
|
||||
// max1 returns the maximum of n and 1, used to guard against divide-by-zero
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
package mcp
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/notes"
|
||||
)
|
||||
|
||||
// noteToolDescriptors are the model-facing halves of brain#4.
|
||||
//
|
||||
// brain_edit_root is deliberately absent: it dispatches but is not listed, the
|
||||
// same de-surfacing ADR-0012 rule 4 applies to brain_index (#78). Hiding the
|
||||
// verb matters here for a specific reason — brain content is ingested from the
|
||||
// outside world and reaches a model's context, so a listed governance-file
|
||||
// editor is a durable prompt-injection path: one poisoned page could rewrite
|
||||
// the CLAUDE.md that a .dispatch-allow agent later obeys. A human who knows
|
||||
// the tool exists can still call it; a model reading tools/list cannot
|
||||
// discover it.
|
||||
func noteToolDescriptors() []map[string]any {
|
||||
str := func(d string) map[string]any { return map[string]any{"type": "string", "description": d} }
|
||||
schema := func(required []string, props map[string]any) json.RawMessage {
|
||||
b, _ := json.Marshal(map[string]any{"type": "object", "required": required, "properties": props})
|
||||
return b
|
||||
}
|
||||
return []map[string]any{
|
||||
{
|
||||
"name": "brain_read",
|
||||
"description": "Return a note's raw markdown verbatim — no synthesis, no excerpting. " +
|
||||
"Use this to obtain the exact text before brain_edit; never build old_str from a " +
|
||||
"brain_query excerpt, which is truncated and normalised and will not match.",
|
||||
"inputSchema": schema([]string{"path"}, map[string]any{
|
||||
"path": str("brain-relative path under knowledge/ or wiki/, e.g. wiki/homelab/facts/x.md"),
|
||||
}),
|
||||
},
|
||||
{
|
||||
"name": "brain_edit",
|
||||
"description": "Replace exactly one occurrence of old_str with new_str in a note, leaving " +
|
||||
"every other byte identical. Fails loudly if old_str is absent or matches more than " +
|
||||
"once — extend old_str until it is unique. Does NOT stamp updated_at or supersedes " +
|
||||
"(brain_update does); it is a byte-level patch, not a metadata operation.",
|
||||
"inputSchema": schema([]string{"path", "old_str", "new_str"}, map[string]any{
|
||||
"path": str("brain-relative path under knowledge/ or wiki/"),
|
||||
"old_str": str("exact text to replace; must occur exactly once"),
|
||||
"new_str": str("replacement text; empty string deletes old_str"),
|
||||
}),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// --- hyperguild#97: governance-rewrite observability -----------------------
|
||||
|
||||
// govNotifier raises an out-of-band alert. Declared here rather than importing
|
||||
// internal/audit so the MCP package keeps depending on capture alone.
|
||||
type govNotifier interface {
|
||||
Notify(ctx context.Context, msg string) error
|
||||
}
|
||||
|
||||
// WithGovernanceObserver attaches the audit sink and notifier that observe a
|
||||
// successful brain_edit_root. Both are optional; either may be nil.
|
||||
func (s *Server) WithGovernanceObserver(sink capture.AuditSink, n govNotifier) {
|
||||
s.govAudit, s.govNotify = sink, n
|
||||
}
|
||||
|
||||
type brainReadArgs struct {
|
||||
Path string `json:"path"`
|
||||
}
|
||||
|
||||
func (s *Server) brainRead(_ context.Context, args json.RawMessage) (json.RawMessage, error) {
|
||||
var a brainReadArgs
|
||||
if err := json.Unmarshal(args, &a); err != nil {
|
||||
return nil, fmt.Errorf("parse args: %w", err)
|
||||
}
|
||||
content, err := notes.Read(s.brainDir, a.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return json.Marshal(map[string]any{"path": a.Path, "content": string(content)})
|
||||
}
|
||||
|
||||
type brainEditArgs struct {
|
||||
Path string `json:"path"`
|
||||
OldStr string `json:"old_str"`
|
||||
NewStr string `json:"new_str"`
|
||||
}
|
||||
|
||||
// brainEdit is the model-facing editor: notes only.
|
||||
//
|
||||
// Deliberately NOT audited (#97). A note edit is already fully reconstructible
|
||||
// from git history, which carries the same before/after bytes plus a diff.
|
||||
// Governance files get the extra trail because they change what later agents
|
||||
// obey — the asymmetry is about instruction-carrying content, not about how
|
||||
// hard the change is to reconstruct. Auditing every note edit would also bury
|
||||
// the rare governance record under routine traffic, which is how an alert
|
||||
// stops being read.
|
||||
func (s *Server) brainEdit(_ context.Context, args json.RawMessage) (json.RawMessage, error) {
|
||||
_, out, err := s.editWithScope(args, notes.ScopeNotes)
|
||||
return out, err
|
||||
}
|
||||
|
||||
// brainEditRoot edits a governance file (CLAUDE.md). De-surfaced AND gated on
|
||||
// the authenticated principal: only the static local-CLI token may call it, so
|
||||
// a claude.ai-origin session — the one that may have just read an untrusted
|
||||
// page — cannot rewrite the instructions later agents load.
|
||||
func (s *Server) brainEditRoot(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
|
||||
info, ok := ctx.Value(principalKey).(principalInfo)
|
||||
if !ok || !info.viaStatic {
|
||||
return nil, fmt.Errorf("brain_edit_root requires the local static principal; " +
|
||||
"a governance file is not editable from a model-facing origin")
|
||||
}
|
||||
res, out, err := s.editWithScope(args, notes.ScopeRoot)
|
||||
if err != nil {
|
||||
// A refusal is not a rewrite. Recording or alerting on one would cry
|
||||
// wolf, and the gate already logs its own denials.
|
||||
return out, err
|
||||
}
|
||||
// A durable instruction rewrite should be loud, not silent.
|
||||
slog.Warn("brain_edit_root applied", "principal", info.principal, "result", string(out))
|
||||
s.observeGovernanceEdit(ctx, info.principal, res)
|
||||
return out, err
|
||||
}
|
||||
|
||||
// observeGovernanceEdit records and alerts on a rewrite that already landed.
|
||||
//
|
||||
// Best-effort by construction: the bytes are written and brain-sync will commit
|
||||
// them within the quarter hour, so failing the call here would report a
|
||||
// non-rewrite to the caller while the rewrite sits on disk. A failure to
|
||||
// observe is therefore logged at ERROR — the one case where the trail itself is
|
||||
// what broke, and the only remaining signal.
|
||||
func (s *Server) observeGovernanceEdit(ctx context.Context, principal string, res notes.Result) {
|
||||
// Path and hashes only. The notification leaves the cluster for a phone
|
||||
// via a third-party topic; a hash identifies the rewrite, the bytes would
|
||||
// leak whatever was written into an instruction file.
|
||||
msg := fmt.Sprintf("brain_edit_root applied\npath=%s\nprincipal=%s\nbefore=%s\nafter=%s",
|
||||
res.Path, principal, res.PrevContentHash, res.ContentHash)
|
||||
|
||||
if s.govNotify != nil {
|
||||
if err := s.govNotify.Notify(ctx, msg); err != nil {
|
||||
slog.Error("brain_edit_root notify failed", "path", res.Path, "err", err)
|
||||
}
|
||||
}
|
||||
if s.govAudit == nil {
|
||||
return
|
||||
}
|
||||
// Internal, not Confidential: a governance file is not client data, and
|
||||
// Confidential would make DegradingSink REFUSE to record while loki is
|
||||
// down rather than buffer — losing exactly the record that matters most.
|
||||
outcome, err := s.govAudit.Reserve(ctx, classification.Internal)
|
||||
if err != nil {
|
||||
slog.Error("brain_edit_root audit reserve failed", "path", res.Path, "err", err)
|
||||
return
|
||||
}
|
||||
entry := capture.AuditEntry{
|
||||
Timestamp: time.Now().UTC(),
|
||||
Principal: principal,
|
||||
Actor: "brain_edit_root",
|
||||
Harness: "mcp",
|
||||
EffectiveClassification: classification.Internal.String(),
|
||||
Items: []string{res.Path},
|
||||
SecurityEvents: []string{fmt.Sprintf(
|
||||
"governance file rewritten: %s before=%s after=%s",
|
||||
res.Path, res.PrevContentHash, res.ContentHash)},
|
||||
}
|
||||
if err := s.govAudit.Record(ctx, entry, outcome); err != nil {
|
||||
slog.Error("brain_edit_root audit record failed", "path", res.Path, "err", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) editWithScope(args json.RawMessage, scope notes.Scope) (notes.Result, json.RawMessage, error) {
|
||||
var a brainEditArgs
|
||||
if err := json.Unmarshal(args, &a); err != nil {
|
||||
return notes.Result{}, nil, fmt.Errorf("parse args: %w", err)
|
||||
}
|
||||
res, err := notes.Edit(s.brainDir, a.Path, a.OldStr, a.NewStr, scope)
|
||||
if err != nil {
|
||||
return notes.Result{}, nil, err
|
||||
}
|
||||
|
||||
// Reindex only when the edit could have changed what the index reads.
|
||||
// BuildWingIndex consumes title/created_at from the frontmatter block;
|
||||
// BM25 walks the filesystem live and embeddings are mtime-driven, so an
|
||||
// unconditional rebuild would buy nothing and dirty a second file —
|
||||
// _index.md being precisely the class brain-sync already needs a bespoke
|
||||
// conflict resolver for.
|
||||
if wing := wingOf(res.Path); wing != "" && touchesFrontmatter(a.OldStr, a.NewStr) {
|
||||
if ierr := brain.BuildWingIndex(s.brainDir, wing); ierr != nil {
|
||||
// Best-effort, matching every other index hook here: the write
|
||||
// already landed and will be committed, so failing the call would
|
||||
// make the caller retry an edit whose old_str no longer exists.
|
||||
res.ReindexError = ierr.Error()
|
||||
}
|
||||
}
|
||||
out, merr := json.Marshal(res)
|
||||
return res, out, merr
|
||||
}
|
||||
|
||||
// wingOf returns the wing for a wiki path, or "" for knowledge/ and root.
|
||||
func wingOf(rel string) string {
|
||||
parts := strings.Split(rel, "/")
|
||||
if len(parts) < 3 || parts[0] != "wiki" {
|
||||
return ""
|
||||
}
|
||||
return parts[1]
|
||||
}
|
||||
|
||||
// touchesFrontmatter reports whether either side of the edit mentions a field
|
||||
// the wing index reads.
|
||||
func touchesFrontmatter(oldStr, newStr string) bool {
|
||||
for _, s := range []string{oldStr, newStr} {
|
||||
if strings.Contains(s, "title:") || strings.Contains(s, "created_at:") || strings.Contains(s, "---") {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,297 @@
|
||||
// ingestion/internal/mcp/tools_notes_test.go
|
||||
package mcp_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// ---- brain#4: brain_read / brain_edit through the JSON-RPC surface --------
|
||||
|
||||
func newBrainRepo(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(dir, ".git"), 0o755))
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(dir, "wiki/homelab/facts"), 0o755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(dir, "wiki/homelab/facts/n.md"),
|
||||
[]byte("---\ntitle: 'n'\n---\n\nalpha\nbeta\n"), 0o644))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(dir, "CLAUDE.md"), []byte("# rules\n"), 0o644))
|
||||
return dir
|
||||
}
|
||||
|
||||
func call(t *testing.T, srv *mcp.Server, name string, args map[string]any) map[string]any {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
|
||||
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
|
||||
"params": map[string]any{"name": name, "arguments": args},
|
||||
}))
|
||||
rr := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rr, req)
|
||||
var resp map[string]any
|
||||
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
|
||||
return resp
|
||||
}
|
||||
|
||||
func TestBrainReadAndEdit_RoundTripThroughJSONRPC(t *testing.T) {
|
||||
// The Go functions are tested in internal/notes; this pins the transport,
|
||||
// because a tool that works in-process and is unreachable over JSON-RPC
|
||||
// is indistinguishable from a working one in a unit test.
|
||||
dir := newBrainRepo(t)
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
|
||||
resp := call(t, srv, "brain_read", map[string]any{"path": "wiki/homelab/facts/n.md"})
|
||||
require.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
|
||||
resp = call(t, srv, "brain_edit", map[string]any{
|
||||
"path": "wiki/homelab/facts/n.md", "old_str": "beta", "new_str": "BETA",
|
||||
})
|
||||
require.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(dir, "wiki/homelab/facts/n.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "---\ntitle: 'n'\n---\n\nalpha\nBETA\n", string(got),
|
||||
"only the matched bytes may change")
|
||||
}
|
||||
|
||||
func TestBrainEdit_RefusesGovernanceFilesFromTheModelFacingTool(t *testing.T) {
|
||||
dir := newBrainRepo(t)
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
|
||||
resp := call(t, srv, "brain_edit", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "# rules", "new_str": "# rewritten",
|
||||
})
|
||||
require.Contains(t, resp, "error", "brain_edit must not reach CLAUDE.md")
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(dir, "CLAUDE.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "# rules\n", string(got))
|
||||
}
|
||||
|
||||
func TestBrainEditRoot_RefusedWithoutTheLocalStaticPrincipal(t *testing.T) {
|
||||
// No principal in context is the model-facing case (and the default in
|
||||
// this test server): the privileged door must stay shut.
|
||||
dir := newBrainRepo(t)
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
|
||||
resp := call(t, srv, "brain_edit_root", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "# rules", "new_str": "# rewritten",
|
||||
})
|
||||
require.Contains(t, resp, "error")
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(dir, "CLAUDE.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "# rules\n", string(got), "governance file must be untouched")
|
||||
}
|
||||
|
||||
func TestBrainGet_CannotReachSessionTranscripts(t *testing.T) {
|
||||
// Pre-existing hole, not introduced here: api.ReadNote resolves any
|
||||
// brain-relative path and parseFrontmatter returns the WHOLE file as body
|
||||
// when there is no frontmatter fence. archive/ holds verbatim Claude
|
||||
// session transcripts (up to 500 KB) that BM25 never indexes, so they are
|
||||
// invisible to search and trivially readable by path — and CLAUDE.md
|
||||
// documents the layout.
|
||||
dir := newBrainRepo(t)
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(dir, "archive/claude-sessions/koala"), 0o755))
|
||||
require.NoError(t, os.WriteFile(
|
||||
filepath.Join(dir, "archive/claude-sessions/koala/session-1.md"),
|
||||
[]byte("verbatim transcript with tool output\n"), 0o644))
|
||||
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
resp := call(t, srv, "brain_get", map[string]any{"path": "archive/claude-sessions/koala/session-1.md"})
|
||||
|
||||
require.Contains(t, resp, "error", "brain_get must not read outside knowledge/ and wiki/")
|
||||
assert.NotContains(t, string(mustJSON(t, resp)), "verbatim transcript")
|
||||
}
|
||||
|
||||
func TestBrainGet_StillReadsNotes(t *testing.T) {
|
||||
dir := newBrainRepo(t)
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
|
||||
resp := call(t, srv, "brain_get", map[string]any{"path": "wiki/homelab/facts/n.md"})
|
||||
assert.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
}
|
||||
|
||||
func mustJSON(t *testing.T, v any) []byte {
|
||||
t.Helper()
|
||||
b, err := json.Marshal(v)
|
||||
require.NoError(t, err)
|
||||
return b
|
||||
}
|
||||
|
||||
// ---- hyperguild#97: a governance rewrite must be observed, not just gated ----
|
||||
|
||||
// govSink captures audit records. Governance edits are rare, so the whole
|
||||
// history is kept and asserted on rather than sampled.
|
||||
type govSink struct {
|
||||
entries []capture.AuditEntry
|
||||
outcomes []capture.AuditOutcome
|
||||
recErr error
|
||||
}
|
||||
|
||||
func (s *govSink) Reserve(context.Context, classification.Level) (capture.AuditOutcome, error) {
|
||||
return capture.AuditCentral, nil
|
||||
}
|
||||
|
||||
func (s *govSink) Record(_ context.Context, e capture.AuditEntry, o capture.AuditOutcome) error {
|
||||
s.entries = append(s.entries, e)
|
||||
s.outcomes = append(s.outcomes, o)
|
||||
return s.recErr
|
||||
}
|
||||
|
||||
type govNotifier struct {
|
||||
msgs []string
|
||||
err error
|
||||
}
|
||||
|
||||
func (n *govNotifier) Notify(_ context.Context, msg string) error {
|
||||
n.msgs = append(n.msgs, msg)
|
||||
return n.err
|
||||
}
|
||||
|
||||
// govServer wires capture (which is what re-derives the static principal) plus
|
||||
// the #97 observer, and returns the brain dir so the test can read the file.
|
||||
func govServer(t *testing.T) (*mcp.Server, string, *govSink, *govNotifier) {
|
||||
t.Helper()
|
||||
dir := newBrainRepo(t)
|
||||
cfg, err := classification.Load(dir)
|
||||
require.NoError(t, err)
|
||||
svc := capture.NewService(brainstore.New(dir), capFakeTracker{}, cfg, audit.NewSlogSink(nil))
|
||||
|
||||
srv := mcp.NewServer(dir, nil, nil, nil)
|
||||
srv.WithCapture(svc, nil, capStaticTok, "local-cli", capturehttp.NewOriginResolver(nil))
|
||||
sink, notifier := &govSink{}, &govNotifier{}
|
||||
srv.WithGovernanceObserver(sink, notifier)
|
||||
return srv, dir, sink, notifier
|
||||
}
|
||||
|
||||
// authCall is call() with an Authorization header, so the static principal is
|
||||
// present and brain_edit_root's gate opens.
|
||||
func authCall(t *testing.T, srv *mcp.Server, authz, name string, args map[string]any) map[string]any {
|
||||
t.Helper()
|
||||
req := httptest.NewRequest(http.MethodPost, "/mcp", body(t, map[string]any{
|
||||
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
|
||||
"params": map[string]any{"name": name, "arguments": args},
|
||||
}))
|
||||
if authz != "" {
|
||||
req.Header.Set("Authorization", authz)
|
||||
}
|
||||
rr := httptest.NewRecorder()
|
||||
srv.ServeHTTP(rr, req)
|
||||
var resp map[string]any
|
||||
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
|
||||
return resp
|
||||
}
|
||||
|
||||
func sha256hex(s string) string {
|
||||
sum := sha256.Sum256([]byte(s))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
func TestBrainEditRoot_RecordsAnAuditEntryNamingPrincipalPathAndBothHashes(t *testing.T) {
|
||||
// The gate proves a rewrite was ALLOWED. Only a record proves one HAPPENED
|
||||
// — and brain-sync will commit it within 15 minutes authored `brain-sync`,
|
||||
// so git history cannot tell it apart from routine note traffic.
|
||||
srv, dir, sink, _ := govServer(t)
|
||||
|
||||
resp := authCall(t, srv, "Bearer "+capStaticTok, "brain_edit_root", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "# rules", "new_str": "# rewritten",
|
||||
})
|
||||
require.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
|
||||
require.Len(t, sink.entries, 1, "exactly one audit record per governance rewrite")
|
||||
e := sink.entries[0]
|
||||
assert.Equal(t, "local-cli", e.Principal)
|
||||
assert.Contains(t, e.Items, "CLAUDE.md", "the record must name what was rewritten")
|
||||
|
||||
joined := strings.Join(e.SecurityEvents, " ")
|
||||
assert.Contains(t, joined, sha256hex("# rules\n"), "before-hash must be recorded")
|
||||
assert.Contains(t, joined, sha256hex("# rewritten\n"), "after-hash must be recorded")
|
||||
assert.NotZero(t, e.Timestamp)
|
||||
|
||||
got, err := os.ReadFile(filepath.Join(dir, "CLAUDE.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "# rewritten\n", string(got))
|
||||
}
|
||||
|
||||
func TestBrainEditRoot_NotifiesWithPathAndHashesButNeverContent(t *testing.T) {
|
||||
srv, _, _, notifier := govServer(t)
|
||||
|
||||
resp := authCall(t, srv, "Bearer "+capStaticTok, "brain_edit_root", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "# rules", "new_str": "# rewritten",
|
||||
})
|
||||
require.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
|
||||
require.Len(t, notifier.msgs, 1)
|
||||
msg := notifier.msgs[0]
|
||||
assert.Contains(t, msg, "CLAUDE.md")
|
||||
assert.Contains(t, msg, "local-cli")
|
||||
assert.Contains(t, msg, sha256hex("# rules\n"))
|
||||
assert.Contains(t, msg, sha256hex("# rewritten\n"))
|
||||
|
||||
// The notification travels to a phone through a third-party topic. A hash
|
||||
// identifies a rewrite; the bytes would leak whatever was written.
|
||||
assert.NotContains(t, msg, "# rewritten", "no file content in the notification body")
|
||||
assert.NotContains(t, msg, "# rules")
|
||||
}
|
||||
|
||||
func TestBrainEditRoot_RefusedProducesNoRecordNoNotificationNoWrite(t *testing.T) {
|
||||
srv, dir, sink, notifier := govServer(t)
|
||||
|
||||
// No Authorization header → no static principal → the gate stays shut.
|
||||
resp := authCall(t, srv, "", "brain_edit_root", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "# rules", "new_str": "# rewritten",
|
||||
})
|
||||
require.Contains(t, resp, "error")
|
||||
|
||||
assert.Empty(t, sink.entries, "a refusal is not a rewrite — recording one would cry wolf")
|
||||
assert.Empty(t, notifier.msgs)
|
||||
got, err := os.ReadFile(filepath.Join(dir, "CLAUDE.md"))
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "# rules\n", string(got))
|
||||
}
|
||||
|
||||
func TestBrainEditRoot_FailedEditProducesNoRecordNoNotification(t *testing.T) {
|
||||
// Authorised but no match: the door opened and nothing went through it.
|
||||
srv, _, sink, notifier := govServer(t)
|
||||
|
||||
resp := authCall(t, srv, "Bearer "+capStaticTok, "brain_edit_root", map[string]any{
|
||||
"path": "CLAUDE.md", "old_str": "absent-from-the-file", "new_str": "x",
|
||||
})
|
||||
require.Contains(t, resp, "error")
|
||||
|
||||
assert.Empty(t, sink.entries)
|
||||
assert.Empty(t, notifier.msgs)
|
||||
}
|
||||
|
||||
func TestBrainEdit_OnOrdinaryNotesIsNotAudited(t *testing.T) {
|
||||
// The decided asymmetry (#97): note edits are covered by git history, which
|
||||
// carries the same before/after bytes with a diff. Governance files get the
|
||||
// extra trail because they change what later agents obey, not because they
|
||||
// are harder to reconstruct.
|
||||
srv, _, sink, notifier := govServer(t)
|
||||
|
||||
resp := authCall(t, srv, "Bearer "+capStaticTok, "brain_edit", map[string]any{
|
||||
"path": "wiki/homelab/facts/n.md", "old_str": "beta", "new_str": "BETA",
|
||||
})
|
||||
require.NotContains(t, resp, "error", "%v", resp["error"])
|
||||
|
||||
assert.Empty(t, sink.entries, "note edits must not fill the governance audit trail")
|
||||
assert.Empty(t, notifier.msgs)
|
||||
}
|
||||
@@ -0,0 +1,282 @@
|
||||
package notes
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// maxNoteBytes caps what Read will hand back. Generous for knowledge/ and
|
||||
// wiki/ (the largest note today is far under it) and small enough that a
|
||||
// pathological file cannot flood a caller's context.
|
||||
const maxNoteBytes = 256 << 10
|
||||
|
||||
// Errors callers distinguish.
|
||||
var (
|
||||
ErrNoMatch = errors.New("notes: old_str not found")
|
||||
ErrAmbiguous = errors.New("notes: old_str matches more than once")
|
||||
ErrNotUTF8 = errors.New("notes: content is not valid UTF-8")
|
||||
ErrGitBusy = errors.New("notes: a git operation is in progress")
|
||||
ErrTooLarge = errors.New("notes: file exceeds the readable size limit")
|
||||
ErrDegenerate = errors.New("notes: edit is degenerate")
|
||||
ErrInConflict = errors.New("notes: file contains unresolved conflict markers")
|
||||
)
|
||||
|
||||
// pathLocks serialises edits per file within this process. The flock below
|
||||
// covers the cross-process case (two pods share the hostPath during a rolling
|
||||
// update, and brain-fix/brain-enrich write the same tree).
|
||||
var pathLocks sync.Map // string -> *sync.Mutex
|
||||
|
||||
// Result is what a successful edit reports.
|
||||
type Result struct {
|
||||
Path string `json:"path"`
|
||||
// PrevContentHash is the sha256 of the file as it was before this edit.
|
||||
// Paired with ContentHash it identifies a rewrite in an audit trail or an
|
||||
// alert without carrying any of the bytes (hyperguild#97).
|
||||
PrevContentHash string `json:"prev_content_hash"`
|
||||
ContentHash string `json:"content_hash"`
|
||||
BytesBefore int `json:"bytes_before"`
|
||||
BytesAfter int `json:"bytes_after"`
|
||||
ReindexError string `json:"reindex_error,omitempty"`
|
||||
}
|
||||
|
||||
// Read returns a note's bytes verbatim.
|
||||
//
|
||||
// No frontmatter parsing, no excerpting, no normalisation: the entire purpose
|
||||
// is to give a caller an exact basis for an edit. An excerpt from brain_query
|
||||
// would be truncated and normalised, and every old_str built from one would
|
||||
// miss.
|
||||
func Read(brainDir, path string) ([]byte, error) {
|
||||
t, err := Resolve(brainDir, path, ScopeNotes)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return readContained(t)
|
||||
}
|
||||
|
||||
// ReadScoped is Read for a caller already holding the privileged scope.
|
||||
func ReadScoped(brainDir, path string, scope Scope) ([]byte, error) {
|
||||
t, err := Resolve(brainDir, path, scope)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return readContained(t)
|
||||
}
|
||||
|
||||
func readContained(t Target) ([]byte, error) {
|
||||
fi, err := os.Stat(t.Abs)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("notes: stat %s: %w", t.Rel, err)
|
||||
}
|
||||
if fi.Size() > maxNoteBytes {
|
||||
return nil, fmt.Errorf("%w: %s is %d bytes (limit %d)", ErrTooLarge, t.Rel, fi.Size(), maxNoteBytes)
|
||||
}
|
||||
b, err := os.ReadFile(t.Abs)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("notes: read %s: %w", t.Rel, err)
|
||||
}
|
||||
// The MCP response marshals this into a JSON string, and encoding/json
|
||||
// substitutes U+FFFD for invalid UTF-8 — silently returning different
|
||||
// bytes than the file holds, which defeats "verbatim" entirely.
|
||||
if !utf8.Valid(b) {
|
||||
return nil, fmt.Errorf("%w: %s", ErrNotUTF8, t.Rel)
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
// Edit replaces exactly one occurrence of oldStr with newStr, in place.
|
||||
//
|
||||
// Fails loudly and writes nothing on: zero matches, more than one match
|
||||
// (counted with overlap), degenerate arguments, invalid UTF-8, a git operation
|
||||
// in flight, or unresolved conflict markers in the target.
|
||||
func Edit(brainDir, path, oldStr, newStr string, scope Scope) (Result, error) {
|
||||
t, err := Resolve(brainDir, path, scope)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if oldStr == "" {
|
||||
return Result{}, fmt.Errorf("%w: old_str is empty", ErrDegenerate)
|
||||
}
|
||||
if oldStr == newStr {
|
||||
return Result{}, fmt.Errorf("%w: old_str equals new_str", ErrDegenerate)
|
||||
}
|
||||
if !utf8.ValidString(newStr) {
|
||||
return Result{}, fmt.Errorf("%w: new_str", ErrNotUTF8)
|
||||
}
|
||||
// brain#2 guard: refuse to write to a tree the sync does not commit from.
|
||||
if err := assertSyncSource(brainDir); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if err := assertGitIdle(brainDir); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
|
||||
mu := lockFor(t.Abs)
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
|
||||
unlock, err := flockFile(t.Abs)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
defer unlock()
|
||||
|
||||
// Read INSIDE the lock: the match count must be evaluated against the
|
||||
// bytes we are about to overwrite, or a concurrent edit is silently lost
|
||||
// while both callers are told they succeeded.
|
||||
before, err := readContained(t)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if hasConflictMarkers(before) {
|
||||
return Result{}, fmt.Errorf("%w: %s", ErrInConflict, t.Rel)
|
||||
}
|
||||
|
||||
switch n := countOverlapping(string(before), oldStr); {
|
||||
case n == 0:
|
||||
return Result{}, fmt.Errorf("%w in %s (0 matches)%s", ErrNoMatch, t.Rel, crlfHint(string(before), oldStr))
|
||||
case n > 1:
|
||||
return Result{}, fmt.Errorf("%w in %s (%d matches) — extend old_str until it is unique",
|
||||
ErrAmbiguous, t.Rel, n)
|
||||
}
|
||||
|
||||
after := strings.Replace(string(before), oldStr, newStr, 1)
|
||||
if err := writeAtomic(t.Abs, []byte(after)); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
|
||||
sum := sha256.Sum256([]byte(after))
|
||||
prev := sha256.Sum256(before)
|
||||
return Result{
|
||||
Path: t.Rel,
|
||||
PrevContentHash: hex.EncodeToString(prev[:]),
|
||||
ContentHash: hex.EncodeToString(sum[:]),
|
||||
BytesBefore: len(before),
|
||||
BytesAfter: len(after),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// countOverlapping counts occurrences allowing overlap. bytes.Count does not:
|
||||
// "aa" occurs once in "aaa" by its reckoning, so a Count-based uniqueness
|
||||
// check accepts an ambiguous edit as unique.
|
||||
func countOverlapping(s, sub string) int {
|
||||
var n, from int
|
||||
for {
|
||||
i := strings.Index(s[from:], sub)
|
||||
if i < 0 {
|
||||
return n
|
||||
}
|
||||
n++
|
||||
from += i + 1
|
||||
}
|
||||
}
|
||||
|
||||
// crlfHint reports whether the caller's old_str would have matched if line
|
||||
// endings agreed. It never normalises — that would break byte-identity — but
|
||||
// an opaque "no match" is unfixable from the caller's side, and a stripped \r
|
||||
// is the likeliest cause.
|
||||
func crlfHint(content, oldStr string) string {
|
||||
if strings.Contains(content, "\r\n") && !strings.Contains(oldStr, "\r\n") {
|
||||
if strings.Contains(strings.ReplaceAll(content, "\r\n", "\n"), oldStr) {
|
||||
return " — the file uses CRLF line endings and old_str does not; include the \\r"
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func hasConflictMarkers(b []byte) bool {
|
||||
s := string(b)
|
||||
return strings.Contains(s, "\n<<<<<<< ") || strings.HasPrefix(s, "<<<<<<< ")
|
||||
}
|
||||
|
||||
// assertGitIdle refuses while brain-sync (or brain-fix, or a human) has a git
|
||||
// operation in flight. An edit landing between `git merge` and its abort is
|
||||
// either hard-reset away or committed as a wrongly resolved conflict — and the
|
||||
// caller was told it succeeded.
|
||||
func assertGitIdle(brainDir string) error {
|
||||
for _, marker := range []string{"MERGE_HEAD", "REBASE_HEAD", "CHERRY_PICK_HEAD", "index.lock"} {
|
||||
if _, err := os.Stat(filepath.Join(brainDir, ".git", marker)); err == nil {
|
||||
return fmt.Errorf("%w (.git/%s present) — retry after the sync settles", ErrGitBusy, marker)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func lockFor(abs string) *sync.Mutex {
|
||||
v, _ := pathLocks.LoadOrStore(abs, &sync.Mutex{})
|
||||
return v.(*sync.Mutex)
|
||||
}
|
||||
|
||||
// flockFile takes an advisory lock covering other processes sharing the
|
||||
// hostPath — during a RollingUpdate two ingestion pods do.
|
||||
func flockFile(abs string) (func(), error) {
|
||||
f, err := os.OpenFile(abs+".lock", os.O_CREATE|os.O_RDWR, 0o644)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("notes: lock %s: %w", filepath.Base(abs), err)
|
||||
}
|
||||
if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
|
||||
_ = f.Close()
|
||||
return nil, fmt.Errorf("notes: flock %s: %w", filepath.Base(abs), err)
|
||||
}
|
||||
return func() {
|
||||
_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
|
||||
_ = f.Close()
|
||||
_ = os.Remove(abs + ".lock")
|
||||
}, nil
|
||||
}
|
||||
|
||||
// writeAtomic replaces the file via a same-directory temp + rename, then
|
||||
// fsyncs the directory so the rename survives a node crash.
|
||||
//
|
||||
// Same directory because a rename across the hostPath boundary fails with
|
||||
// EXDEV; the ".tmp" suffix because the tree's .gitignore already covers it, so
|
||||
// a leaked temp is not committed by `git add -A`.
|
||||
func writeAtomic(abs string, content []byte) error {
|
||||
dir := filepath.Dir(abs)
|
||||
tmp, err := os.CreateTemp(dir, filepath.Base(abs)+".*.tmp")
|
||||
if err != nil {
|
||||
return fmt.Errorf("notes: temp file: %w", err)
|
||||
}
|
||||
tmpName := tmp.Name()
|
||||
defer func() { _ = os.Remove(tmpName) }() // no-op once renamed
|
||||
|
||||
if _, err := tmp.Write(content); err != nil {
|
||||
_ = tmp.Close()
|
||||
return fmt.Errorf("notes: write temp: %w", err)
|
||||
}
|
||||
if err := tmp.Sync(); err != nil {
|
||||
_ = tmp.Close()
|
||||
return fmt.Errorf("notes: sync temp: %w", err)
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
return fmt.Errorf("notes: close temp: %w", err)
|
||||
}
|
||||
// CreateTemp makes 0600; every other writer of this tree uses 0644 and
|
||||
// other uids read it.
|
||||
if err := os.Chmod(tmpName, 0o644); err != nil {
|
||||
return fmt.Errorf("notes: chmod temp: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmpName, abs); err != nil {
|
||||
return fmt.Errorf("notes: rename: %w", err)
|
||||
}
|
||||
return fsyncDir(dir)
|
||||
}
|
||||
|
||||
func fsyncDir(dir string) error {
|
||||
d, err := os.Open(dir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("notes: open dir: %w", err)
|
||||
}
|
||||
defer func() { _ = d.Close() }()
|
||||
if err := d.Sync(); err != nil {
|
||||
return fmt.Errorf("notes: fsync dir: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,256 @@
|
||||
package notes
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func seed(t *testing.T, dir, rel, content string) string {
|
||||
t.Helper()
|
||||
abs := filepath.Join(dir, rel)
|
||||
if err := os.MkdirAll(filepath.Dir(abs), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(abs, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return abs
|
||||
}
|
||||
|
||||
func read(t *testing.T, abs string) string {
|
||||
t.Helper()
|
||||
b, err := os.ReadFile(abs)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func TestRead_ReturnsVerbatimBytes(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
const body = "---\ntitle: 'x'\n---\n\nbody with double spaces\r\nand a CRLF line\nno trailing newline"
|
||||
seed(t, dir, "knowledge/verbatim.md", body)
|
||||
|
||||
got, err := Read(dir, "knowledge/verbatim.md")
|
||||
if err != nil {
|
||||
t.Fatalf("Read: %v", err)
|
||||
}
|
||||
if string(got) != body {
|
||||
t.Errorf("Read returned %q, want the file byte-for-byte", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRead_RefusesNonUTF8RatherThanCorruptingIt(t *testing.T) {
|
||||
// The response is marshalled into a JSON string, and encoding/json
|
||||
// replaces invalid UTF-8 with U+FFFD. Returning it would mean every
|
||||
// old_str the caller derives from that region can never match, and
|
||||
// pasting it back would write the corruption to disk permanently.
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/bad.md", "ok \xff\xfe broken\n")
|
||||
|
||||
if _, err := Read(dir, "knowledge/bad.md"); err == nil {
|
||||
t.Fatal("Read accepted invalid UTF-8; it must refuse rather than silently substitute")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_ReplacesExactlyOnceAndLeavesEveryOtherByteAlone(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
const body = "---\ntitle: 'x'\n---\n\nalpha\nbeta\ngamma\n"
|
||||
abs := seed(t, dir, "knowledge/n.md", body)
|
||||
|
||||
res, err := Edit(dir, "knowledge/n.md", "beta", "BETA", ScopeNotes)
|
||||
if err != nil {
|
||||
t.Fatalf("Edit: %v", err)
|
||||
}
|
||||
want := strings.Replace(body, "beta", "BETA", 1)
|
||||
if got := read(t, abs); got != want {
|
||||
t.Errorf("file = %q, want %q", got, want)
|
||||
}
|
||||
if res.ContentHash == "" {
|
||||
t.Error("result should carry a content hash for read-after-write confirmation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_RefusesAmbiguousAndAbsentMatchesWithoutWriting(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
const body = "alpha\nalpha\n"
|
||||
abs := seed(t, dir, "knowledge/n.md", body)
|
||||
|
||||
cases := []struct{ name, old, want string }{
|
||||
{"two matches", "alpha", "2"},
|
||||
{"no match", "delta", "0"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
_, err := Edit(dir, "knowledge/n.md", tc.old, "x", ScopeNotes)
|
||||
if err == nil {
|
||||
t.Fatal("want a loud refusal")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.want) {
|
||||
t.Errorf("error must state the match count, got %v", err)
|
||||
}
|
||||
if got := read(t, abs); got != body {
|
||||
t.Errorf("file changed on a refused edit: %q", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_CountsOverlappingMatches(t *testing.T) {
|
||||
// bytes.Count counts NON-overlapping occurrences: "aa" in "aaa" is 1, so
|
||||
// a Count-based uniqueness check calls an ambiguous edit unique.
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "aaa\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "aa", "b", ScopeNotes); err == nil {
|
||||
t.Fatal("overlapping matches must be reported as ambiguous")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_RejectsDegenerateArguments(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "alpha\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "", "x", ScopeNotes); err == nil {
|
||||
t.Error("empty old_str must be rejected")
|
||||
}
|
||||
if _, err := Edit(dir, "knowledge/n.md", "alpha", "alpha", ScopeNotes); err == nil {
|
||||
t.Error("a no-op edit must be rejected rather than burning a sync cycle")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_DeletionIsLegal(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
abs := seed(t, dir, "knowledge/n.md", "keep\ndrop\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "drop\n", "", ScopeNotes); err != nil {
|
||||
t.Fatalf("Edit: %v", err)
|
||||
}
|
||||
if got := read(t, abs); got != "keep\n" {
|
||||
t.Errorf("file = %q, want %q", got, "keep\n")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_ReportsWhenOnlyLineEndingsDiffer(t *testing.T) {
|
||||
// The likeliest real failure: the caller's old_str lost its \r. Never
|
||||
// normalise silently — that breaks byte-identity — but say so, because an
|
||||
// opaque "no match" is unfixable from the caller's side.
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "alpha\r\nbeta\r\n")
|
||||
|
||||
_, err := Edit(dir, "knowledge/n.md", "alpha\nbeta", "x", ScopeNotes)
|
||||
if err == nil {
|
||||
t.Fatal("want a refusal")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "line ending") {
|
||||
t.Errorf("error should point at CRLF, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_RefusesWhileGitIsMidOperation(t *testing.T) {
|
||||
// brain-sync runs `git merge -X ours origin/main`, and aborts on conflict
|
||||
// with a hard reset. An edit landing inside that window is either erased
|
||||
// or committed as a wrongly-resolved conflict, and the caller was told it
|
||||
// succeeded.
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "alpha\n")
|
||||
|
||||
for _, marker := range []string{"MERGE_HEAD", "index.lock", "REBASE_HEAD"} {
|
||||
t.Run(marker, func(t *testing.T) {
|
||||
p := filepath.Join(dir, ".git", marker)
|
||||
if err := os.WriteFile(p, []byte("x"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() { _ = os.Remove(p) }()
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "alpha", "beta", ScopeNotes); err == nil {
|
||||
t.Errorf("edit accepted while %s exists", marker)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_RefusesAFileHoldingConflictMarkers(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "<<<<<<< HEAD\nours\n=======\ntheirs\n>>>>>>> origin/main\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "ours", "x", ScopeNotes); err == nil {
|
||||
t.Fatal("editing one side of an unresolved conflict must be refused")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_ConcurrentEditsBothApply(t *testing.T) {
|
||||
// Read-modify-write with no lock loses one update AND reports success for
|
||||
// both. old_str uniqueness does not protect against it: uniqueness is
|
||||
// evaluated against the stale read.
|
||||
dir := newBrain(t)
|
||||
abs := seed(t, dir, "knowledge/n.md", "alpha\nbeta\n")
|
||||
|
||||
var wg sync.WaitGroup
|
||||
errs := make([]error, 2)
|
||||
for i, pair := range [][2]string{{"alpha", "ALPHA"}, {"beta", "BETA"}} {
|
||||
wg.Add(1)
|
||||
go func(i int, old, new string) {
|
||||
defer wg.Done()
|
||||
_, errs[i] = Edit(dir, "knowledge/n.md", old, new, ScopeNotes)
|
||||
}(i, pair[0], pair[1])
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
for i, err := range errs {
|
||||
if err != nil {
|
||||
t.Fatalf("edit %d failed: %v", i, err)
|
||||
}
|
||||
}
|
||||
if got := read(t, abs); got != "ALPHA\nBETA\n" {
|
||||
t.Errorf("file = %q, want both edits applied", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_LeavesNoTemporaryFileBehind(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "alpha\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "alpha", "beta", ScopeNotes); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entries, err := os.ReadDir(filepath.Join(dir, "knowledge"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, e := range entries {
|
||||
if e.Name() != "note.md" && e.Name() != "n.md" {
|
||||
t.Errorf("stray file left in the tree: %q (git add -A would commit it)", e.Name())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_PreservesReadableMode(t *testing.T) {
|
||||
// Other uids write this tree (brain-fix, brain-enrich). os.CreateTemp
|
||||
// yields 0600 and rename preserves it, which would break them silently.
|
||||
dir := newBrain(t)
|
||||
abs := seed(t, dir, "knowledge/n.md", "alpha\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "alpha", "beta", ScopeNotes); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fi, err := os.Stat(abs)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if fi.Mode().Perm() != 0o644 {
|
||||
t.Errorf("mode = %v, want 0644", fi.Mode().Perm())
|
||||
}
|
||||
}
|
||||
|
||||
func TestEdit_RefusesNonUTF8Replacement(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
seed(t, dir, "knowledge/n.md", "alpha\n")
|
||||
|
||||
if _, err := Edit(dir, "knowledge/n.md", "alpha", "bad \xff", ScopeNotes); err == nil {
|
||||
t.Fatal("new_str with invalid UTF-8 must be refused, not written")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,201 @@
|
||||
// Package notes provides containment-checked read and surgical edit of brain
|
||||
// notes on the live tree that brain-sync commits from.
|
||||
//
|
||||
// Everything here is defensive for one reason: brain content is ingested from
|
||||
// the outside world, that content reaches a model's context via brain_query /
|
||||
// brain_context, and these primitives are callable by that model. A path bug
|
||||
// is therefore not a path bug — it is an outside author choosing a file.
|
||||
package notes
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Scope is which part of the tree a caller is allowed to touch.
|
||||
type Scope int
|
||||
|
||||
const (
|
||||
// ScopeNotes is the model-facing scope: markdown under knowledge/ and
|
||||
// wiki/ only.
|
||||
ScopeNotes Scope = iota
|
||||
// ScopeRoot admits the named governance files and NOTHING else. It exists so
|
||||
// there is an MCP path to CLAUDE.md at all — the absence of one is what
|
||||
// pushed a session to edit via the gitea REST API on 2026-06-16, which
|
||||
// never reached the live tree and wedged brain-sync for hours. It is NOT
|
||||
// reachable from the model-facing toolset; see tools_notes.go.
|
||||
ScopeRoot
|
||||
)
|
||||
|
||||
// rootFiles is an exact-name allowlist, deliberately not a "root *.md" glob.
|
||||
//
|
||||
// Two reasons. log.md is gitignored, so an edit to it applies to the live tree
|
||||
// and never reaches gitea — the exact shape of brain#2, admitted by a glob. And
|
||||
// a glob auto-admits whatever governance file appears next (AGENTS.md, a
|
||||
// bootstrap doc) with nobody reviewing that decision.
|
||||
var rootFiles = map[string]bool{"CLAUDE.md": true}
|
||||
|
||||
// noteDirs are the two trees that hold notes.
|
||||
var noteDirs = []string{"knowledge/", "wiki/"}
|
||||
|
||||
// deniedPrefixes are refused in every scope, independent of the allowlist.
|
||||
//
|
||||
// A static backstop rather than a consequence of the allowlist being right:
|
||||
// .git/hooks/post-commit is executed by brain-sync inside a pod holding the
|
||||
// Gitea push credentials, and archive/ + sessions/ hold verbatim session
|
||||
// transcripts that BM25 never indexes and so nobody thinks of as reachable.
|
||||
var deniedPrefixes = []string{".git/", "archive/", "sessions/", "raw/", "training-data/", ".obsidian/"}
|
||||
|
||||
// Errors are sentinels so callers can distinguish a refusal from an I/O fault.
|
||||
var (
|
||||
ErrOutsideBrain = errors.New("notes: path escapes the brain dir")
|
||||
ErrNotAllowed = errors.New("notes: path is not editable through this tool")
|
||||
ErrNotSyncSource = errors.New("notes: brain dir is not the sync source")
|
||||
)
|
||||
|
||||
// Target is a resolved, contained path.
|
||||
type Target struct {
|
||||
Rel string // brain-relative, cleaned
|
||||
Abs string // absolute, symlink-checked
|
||||
}
|
||||
|
||||
// Resolve canonicalises path, then checks it against the scope's allowlist,
|
||||
// then re-checks containment after symlink evaluation.
|
||||
//
|
||||
// Order matters and is the point: a prefix test on the raw argument accepts
|
||||
// "knowledge/../.git/hooks/post-commit", because Clean only collapses it
|
||||
// afterwards.
|
||||
func Resolve(brainDir, path string, scope Scope) (Target, error) {
|
||||
// Note: the sync-source assertion lives on the WRITE path (Edit), not
|
||||
// here. brain#2 was an edit landing where the sync could not commit it; a
|
||||
// read from such a tree is merely a read.
|
||||
|
||||
// 1. Canonicalise. Rooting at "/" makes Clean collapse every ".." that
|
||||
// would otherwise escape, and ToSlash normalises separators first.
|
||||
clean := strings.TrimPrefix(filepath.Clean("/"+filepath.ToSlash(path)), "/")
|
||||
if clean == "" || clean == "." {
|
||||
return Target{}, fmt.Errorf("%w: empty path", ErrNotAllowed)
|
||||
}
|
||||
|
||||
// 2. Static denials, before any allowlist reasoning.
|
||||
for _, d := range deniedPrefixes {
|
||||
if strings.HasPrefix(clean+"/", d) {
|
||||
return Target{}, fmt.Errorf("%w: %q is not note content", ErrNotAllowed, clean)
|
||||
}
|
||||
}
|
||||
if strings.HasPrefix(filepath.Base(clean), ".") {
|
||||
return Target{}, fmt.Errorf("%w: dotfiles are not editable", ErrNotAllowed)
|
||||
}
|
||||
|
||||
// 3. Scope allowlist.
|
||||
if err := allow(clean, scope); err != nil {
|
||||
return Target{}, err
|
||||
}
|
||||
|
||||
// 4. Containment, lexically.
|
||||
abs := filepath.Join(brainDir, filepath.FromSlash(clean))
|
||||
if !within(brainDir, abs) {
|
||||
return Target{}, fmt.Errorf("%w: %q", ErrOutsideBrain, path)
|
||||
}
|
||||
|
||||
// 5. Containment again, after resolving symlinks. The tree is Syncthing
|
||||
// managed, so a link can appear without anyone adding one on purpose;
|
||||
// a symlinked note or directory would otherwise read and write outside
|
||||
// the brain entirely.
|
||||
if err := assertNoSymlinkEscape(brainDir, abs); err != nil {
|
||||
return Target{}, err
|
||||
}
|
||||
|
||||
return Target{Rel: clean, Abs: abs}, nil
|
||||
}
|
||||
|
||||
// allow enforces the scope's allowlist. The two scopes are DISJOINT, not
|
||||
// nested: the privileged root scope reaches governance files and nothing else,
|
||||
// so the tool that carries the higher privilege carries the smaller reach.
|
||||
func allow(clean string, scope Scope) error {
|
||||
if scope == ScopeRoot {
|
||||
if !rootFiles[clean] {
|
||||
return fmt.Errorf("%w: %q is not a governance file", ErrNotAllowed, clean)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !isNotePath(clean) {
|
||||
return fmt.Errorf("%w: %q is outside knowledge/ and wiki/", ErrNotAllowed, clean)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func isNotePath(clean string) bool {
|
||||
var inDir bool
|
||||
for _, d := range noteDirs {
|
||||
if strings.HasPrefix(clean, d) {
|
||||
inDir = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !inDir || filepath.Ext(clean) != ".md" {
|
||||
return false
|
||||
}
|
||||
// _index.md is generated per wing, and brain-sync's conflict resolver
|
||||
// deliberately resolves it in origin's favour — an edit here is reverted
|
||||
// by design, so accepting one would be a lie.
|
||||
return filepath.Base(clean) != "_index.md"
|
||||
}
|
||||
|
||||
func within(root, abs string) bool {
|
||||
rel, err := filepath.Rel(root, abs)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator))
|
||||
}
|
||||
|
||||
// assertNoSymlinkEscape resolves the deepest existing ancestor of abs and
|
||||
// requires it to still be inside brainDir. The file itself need not exist
|
||||
// (a create is legal); its directory chain must be real and contained.
|
||||
func assertNoSymlinkEscape(brainDir, abs string) error {
|
||||
if fi, err := os.Lstat(abs); err == nil && fi.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("%w: %q is a symlink", ErrNotAllowed, abs)
|
||||
}
|
||||
|
||||
realRoot, err := filepath.EvalSymlinks(brainDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("notes: resolve brain dir: %w", err)
|
||||
}
|
||||
|
||||
dir := filepath.Dir(abs)
|
||||
for {
|
||||
resolved, err := filepath.EvalSymlinks(dir)
|
||||
if err == nil {
|
||||
if !within(realRoot, resolved) {
|
||||
return fmt.Errorf("%w: %q resolves outside the brain dir", ErrOutsideBrain, dir)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !os.IsNotExist(err) {
|
||||
return fmt.Errorf("notes: resolve %q: %w", dir, err)
|
||||
}
|
||||
parent := filepath.Dir(dir)
|
||||
if parent == dir {
|
||||
return fmt.Errorf("%w: no existing ancestor for %q", ErrOutsideBrain, abs)
|
||||
}
|
||||
dir = parent
|
||||
}
|
||||
}
|
||||
|
||||
// assertSyncSource refuses a brain dir that is not a git working tree.
|
||||
//
|
||||
// brain#2 was an edit landing on a directory the sync does not commit from.
|
||||
// That cannot be prevented by review — a wrong INGEST_BRAIN_DIR in a future
|
||||
// manifest reintroduces it while every unit test still passes — so it is
|
||||
// asserted at runtime, the same check brain-sync itself makes before running.
|
||||
func assertSyncSource(brainDir string) error {
|
||||
if fi, err := os.Stat(filepath.Join(brainDir, ".git")); err != nil || !fi.IsDir() {
|
||||
return fmt.Errorf("%w: %q has no .git, so an edit there would never be committed",
|
||||
ErrNotSyncSource, brainDir)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
package notes
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// newBrain lays out a tree shaped like the live one: a git dir, note dirs,
|
||||
// and the non-note areas that must never be reachable.
|
||||
func newBrain(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
for _, d := range []string{
|
||||
".git/hooks", "knowledge", "wiki/homelab/facts", "archive/claude-sessions/koala",
|
||||
"sessions", "raw", ".obsidian",
|
||||
} {
|
||||
if err := os.MkdirAll(filepath.Join(dir, d), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write := func(p, s string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(dir, p), []byte(s), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("CLAUDE.md", "# instructions\n")
|
||||
write("log.md", "ignored scratch\n")
|
||||
write("knowledge/note.md", "# note\n")
|
||||
write("wiki/homelab/facts/f.md", "# fact\n")
|
||||
write("wiki/homelab/_index.md", "# index\n")
|
||||
write("archive/claude-sessions/koala/session-1.md", "verbatim transcript\n")
|
||||
write("sessions/s.jsonl", "{}\n")
|
||||
write(".git/hooks/post-commit", "#!/bin/sh\n")
|
||||
return dir
|
||||
}
|
||||
|
||||
func TestResolve_AcceptsNotePaths(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
for _, p := range []string{"knowledge/note.md", "wiki/homelab/facts/f.md"} {
|
||||
got, err := Resolve(dir, p, ScopeNotes)
|
||||
if err != nil {
|
||||
t.Errorf("Resolve(%q): %v", p, err)
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(got.Abs, dir) {
|
||||
t.Errorf("Resolve(%q).Abs = %q, want inside %q", p, got.Abs, dir)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolve_RefusesEscapesAndNonNoteAreas(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
|
||||
// Canonicalisation must happen BEFORE the allowlist check. A prefix test
|
||||
// on the raw argument accepts every one of these: "knowledge/..%2f" style
|
||||
// inputs collapse to .git, and .git/hooks/post-commit is executed by
|
||||
// brain-sync inside a pod holding the Gitea push credentials.
|
||||
refuse := []string{
|
||||
"knowledge/../.git/hooks/post-commit",
|
||||
"wiki/../.git/config",
|
||||
"./knowledge/../.git/config",
|
||||
"knowledge//../.git/config",
|
||||
"../../etc/passwd",
|
||||
"/etc/passwd",
|
||||
".git/config",
|
||||
"archive/claude-sessions/koala/session-1.md", // verbatim transcripts
|
||||
"sessions/s.jsonl",
|
||||
"raw/x.md",
|
||||
".obsidian/workspace.json",
|
||||
"log.md", // gitignored: an edit here would never sync (brain#2)
|
||||
"wiki/homelab/_index.md", // generated; brain-sync resolves it in origin's favour
|
||||
"knowledge/note.txt", // not markdown
|
||||
"CLAUDE.md", // root file — not reachable in the notes scope
|
||||
}
|
||||
for _, p := range refuse {
|
||||
if _, err := Resolve(dir, p, ScopeNotes); err == nil {
|
||||
t.Errorf("Resolve(%q) = nil error, want refusal", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolve_RootScopeAdmitsOnlyNamedFiles(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
|
||||
if _, err := Resolve(dir, "CLAUDE.md", ScopeRoot); err != nil {
|
||||
t.Errorf("Resolve(CLAUDE.md, ScopeRoot): %v", err)
|
||||
}
|
||||
// log.md is the reason this is an exact-name list and not "root *.md":
|
||||
// it is gitignored, so an edit would apply locally and never reach gitea.
|
||||
for _, p := range []string{"log.md", "AGENTS.md", "knowledge/note.md", ".git/config"} {
|
||||
if _, err := Resolve(dir, p, ScopeRoot); err == nil {
|
||||
t.Errorf("Resolve(%q, ScopeRoot) = nil error, want refusal", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolve_RefusesSymlinkedTargets(t *testing.T) {
|
||||
dir := newBrain(t)
|
||||
outside := filepath.Join(t.TempDir(), "secret.md")
|
||||
if err := os.WriteFile(outside, []byte("secret\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// The tree is Syncthing-managed, so a symlink can appear without anyone
|
||||
// adding one deliberately. Containment must survive it.
|
||||
if err := os.Symlink(outside, filepath.Join(dir, "knowledge/link.md")); err != nil {
|
||||
t.Skipf("symlinks unavailable: %v", err)
|
||||
}
|
||||
if _, err := Resolve(dir, "knowledge/link.md", ScopeNotes); err == nil {
|
||||
t.Error("Resolve followed a symlink out of the brain dir")
|
||||
}
|
||||
|
||||
if err := os.Symlink(filepath.Dir(outside), filepath.Join(dir, "wiki/escape")); err != nil {
|
||||
t.Skipf("symlinks unavailable: %v", err)
|
||||
}
|
||||
if _, err := Resolve(dir, "wiki/escape/secret.md", ScopeNotes); err == nil {
|
||||
t.Error("Resolve followed a symlinked directory out of the brain dir")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolve_RefusesWhenTheTreeIsNotTheSyncSource(t *testing.T) {
|
||||
// brain#2 was "wrote to a directory the sync does not commit from". No
|
||||
// code review prevents that recurring — a wrong INGEST_BRAIN_DIR in a
|
||||
// future manifest reintroduces it and every unit test still passes. So
|
||||
// assert the property at runtime, the way brain-sync itself does.
|
||||
dir := t.TempDir()
|
||||
if err := os.MkdirAll(filepath.Join(dir, "knowledge"), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "knowledge/note.md"), []byte("x"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Reads are fine anywhere; the guard belongs to writes.
|
||||
if _, err := Resolve(dir, "knowledge/note.md", ScopeNotes); err != nil {
|
||||
t.Fatalf("Resolve should not require .git for a read: %v", err)
|
||||
}
|
||||
_, err := Edit(dir, "knowledge/note.md", "x", "y", ScopeNotes)
|
||||
if err == nil {
|
||||
t.Fatal("Edit accepted a tree with no .git — the write would never be committed")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "sync source") {
|
||||
t.Errorf("error should name the cause, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -76,6 +76,9 @@ func buildFrontmatter(rp RawPage, date string) string {
|
||||
}
|
||||
fmt.Fprintf(&sb, "date_ingested: %s\n", date)
|
||||
fmt.Fprintf(&sb, "last_updated: %s\n", date)
|
||||
if rp.SourceHash != "" {
|
||||
fmt.Fprintf(&sb, "source_hash: %s\n", rp.SourceHash)
|
||||
}
|
||||
if rp.Source != "" {
|
||||
fmt.Fprintf(&sb, "source: %s\n", yamlScalar(rp.Source))
|
||||
}
|
||||
|
||||
@@ -20,9 +20,13 @@ type RawPage struct {
|
||||
// ingested content's own frontmatter (see parseContentFrontmatter) — never
|
||||
// set by the LLM. json:"-" keeps them immune to same-named keys the LLM
|
||||
// might emit. Only meaningful for Type == "source".
|
||||
Source string `json:"-"`
|
||||
Author string `json:"-"`
|
||||
Published string `json:"-"`
|
||||
// SourceHash is sha256 of the ingested BODY (frontmatter excluded), stamped
|
||||
// so the next ingest of the same source can tell "unchanged" from "edited"
|
||||
// without re-running extraction (#105).
|
||||
SourceHash string `json:"-"`
|
||||
Source string `json:"-"`
|
||||
Author string `json:"-"`
|
||||
Published string `json:"-"`
|
||||
}
|
||||
|
||||
// ParseRawPages parses LLM output as a JSON array of RawPage objects.
|
||||
@@ -112,6 +116,11 @@ type sourceMeta struct {
|
||||
Source string
|
||||
Author string
|
||||
Published string
|
||||
// BodyHash is sha256 of the content with any leading frontmatter removed.
|
||||
// Body-only on purpose: a re-crawl that stamps a new fetch date has not
|
||||
// changed the source, and treating that as drift would re-extract the
|
||||
// whole corpus every time a crawler ran (#105).
|
||||
BodyHash string
|
||||
}
|
||||
|
||||
// parseContentFrontmatter extracts source/author/published from a leading
|
||||
@@ -154,6 +163,7 @@ func applySourceMeta(pages []RawPage, meta sourceMeta) {
|
||||
if pages[i].Type != "source" {
|
||||
continue
|
||||
}
|
||||
pages[i].SourceHash = meta.BodyHash
|
||||
pages[i].Source = meta.Source
|
||||
pages[i].Author = meta.Author
|
||||
pages[i].Published = meta.Published
|
||||
|
||||
@@ -3,6 +3,8 @@ package pipeline
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -26,6 +28,11 @@ type Config struct {
|
||||
type Result struct {
|
||||
Pages []string // relative paths written (or would-be written in dry-run)
|
||||
Warnings []string
|
||||
// Skipped reports that the source was byte-identical to the last ingest,
|
||||
// so no extraction ran and nothing was written. Reported rather than
|
||||
// silent: a run that does nothing must say so, or "the pipeline is fine"
|
||||
// and "the pipeline stopped working" look the same (#105, #107).
|
||||
Skipped bool
|
||||
}
|
||||
|
||||
// Run ingests content and writes structured wiki pages to brainDir/wiki/.
|
||||
@@ -43,6 +50,18 @@ func Run(ctx context.Context, cfg Config, brainDir, content, source string, dryR
|
||||
|
||||
sourceSlug := wiki.Slug(source)
|
||||
date := time.Now().UTC().Format("2006-01-02")
|
||||
|
||||
// Drift check BEFORE any extraction: an unchanged source costs a full LLM
|
||||
// pass over every chunk to produce a result that merges to nothing (#105).
|
||||
// The merge is already idempotent (#103), so this is cost and signal, not
|
||||
// correctness — which is why it is safe for it to be an early return.
|
||||
bodyHash := hashBody(content)
|
||||
if !dryRun && bodyHash != "" && bodyHash == storedSourceHash(brainDir, sourceSlug) {
|
||||
return Result{Skipped: true, Warnings: []string{
|
||||
fmt.Sprintf("source %q unchanged since last ingest (hash %s) — extraction skipped", source, bodyHash[:12]),
|
||||
}}, nil
|
||||
}
|
||||
|
||||
chunks := Chunk(content, cfg.ChunkSize)
|
||||
|
||||
var allRaw []RawPage
|
||||
@@ -59,7 +78,9 @@ func Run(ctx context.Context, cfg Config, brainDir, content, source string, dryR
|
||||
allWarnings = append(allWarnings, warnings...)
|
||||
}
|
||||
|
||||
applySourceMeta(allRaw, parseContentFrontmatter(content))
|
||||
meta := parseContentFrontmatter(content)
|
||||
meta.BodyHash = bodyHash
|
||||
applySourceMeta(allRaw, meta)
|
||||
|
||||
return buildAndWrite(allRaw, sourceSlug, date, brainDir, source, inventory, allWarnings, dryRun)
|
||||
}
|
||||
@@ -96,7 +117,11 @@ func buildAndWrite(rawPages []RawPage, sourceSlug, date, brainDir, source string
|
||||
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
|
||||
return Result{}, fmt.Errorf("mkdir for %s: %w", page.Path, err)
|
||||
}
|
||||
if err := os.WriteFile(dest, []byte(page.Content), 0o644); err != nil {
|
||||
content, err := reviseOrCreate(dest, page)
|
||||
if err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if err := os.WriteFile(dest, []byte(content), 0o644); err != nil {
|
||||
return Result{}, fmt.Errorf("write %s: %w", page.Path, err)
|
||||
}
|
||||
}
|
||||
@@ -115,6 +140,75 @@ func buildAndWrite(rawPages []RawPage, sourceSlug, date, brainDir, source string
|
||||
return Result{Pages: written, Warnings: warnings}, nil
|
||||
}
|
||||
|
||||
// hashBody returns sha256 of content with any leading frontmatter block
|
||||
// removed, or "" for empty content.
|
||||
//
|
||||
// Body-only on purpose: a re-crawl that stamps a new fetch date has not changed
|
||||
// the source, and treating that as drift would re-extract the corpus every time
|
||||
// a crawler ran (#105).
|
||||
func hashBody(content string) string {
|
||||
body := content
|
||||
if strings.HasPrefix(body, "---\n") {
|
||||
if _, rest, found := strings.Cut(body[4:], "\n---"); found {
|
||||
body = strings.TrimPrefix(rest, "\n")
|
||||
}
|
||||
}
|
||||
if strings.TrimSpace(body) == "" {
|
||||
return ""
|
||||
}
|
||||
sum := sha256.Sum256([]byte(body))
|
||||
return hex.EncodeToString(sum[:])
|
||||
}
|
||||
|
||||
// storedSourceHash reads source_hash from the source page a previous ingest
|
||||
// wrote, or "" when there is no such page or no such field.
|
||||
//
|
||||
// Absence always means "extract". That is the fail-safe direction: a missed
|
||||
// skip costs one extraction, while a wrong skip loses an update silently — and
|
||||
// silence is the failure mode this repo keeps paying for.
|
||||
func storedSourceHash(brainDir, sourceSlug string) string {
|
||||
raw, err := os.ReadFile(filepath.Join(brainDir, "wiki", "sources", sourceSlug+".md"))
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
for _, line := range strings.Split(string(raw), "\n") {
|
||||
if key, value, found := strings.Cut(line, ":"); found && strings.TrimSpace(key) == "source_hash" {
|
||||
return strings.TrimSpace(value)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// reviseOrCreate returns the bytes to write for one extracted page: merged into
|
||||
// whatever is already on disk, or the page as-is when the path is new.
|
||||
//
|
||||
// This is the compile half of compile-on-ingest. Before it, the write was an
|
||||
// unconditional os.WriteFile, so re-ingesting a source replaced its page
|
||||
// wholesale and discarded everything accumulated since — including human edits
|
||||
// (hyperguild#43: "re-ingest overwrites wholesale").
|
||||
//
|
||||
// The merge is deliberately asymmetric in the existing page's favour:
|
||||
// wiki.Merge keeps the on-disk frontmatter and the on-disk version of any
|
||||
// section both sides carry, adds sections only the extraction produced, and
|
||||
// unions bullet lists. So a freshly extracted page can add to a note but never
|
||||
// silently rewrite what a human put there — which is what makes it safe to run
|
||||
// this on every ingest rather than behind a review gate.
|
||||
//
|
||||
// Known limitation: because the on-disk frontmatter wins, `last_updated` keeps
|
||||
// its first-ingest date on a revised page. Left deliberately — the alternative
|
||||
// is this function editing frontmatter it does not own, and no lint reads that
|
||||
// field today.
|
||||
func reviseOrCreate(dest string, page wiki.Page) (string, error) {
|
||||
existing, err := os.ReadFile(dest)
|
||||
switch {
|
||||
case os.IsNotExist(err):
|
||||
return page.Content, nil
|
||||
case err != nil:
|
||||
return "", fmt.Errorf("read existing %s: %w", page.Path, err)
|
||||
}
|
||||
return wiki.Merge(wiki.Page{Path: page.Path, Content: string(existing)}, page).Content, nil
|
||||
}
|
||||
|
||||
// mergeAll deduplicates pages by path, merging content from later occurrences.
|
||||
func mergeAll(pages []wiki.Page) []wiki.Page {
|
||||
order := make([]string, 0, len(pages))
|
||||
|
||||
@@ -171,3 +171,125 @@ func mustJSON(v any) string {
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
|
||||
// --- B1: compile-on-ingest — revise the existing page, never clobber it -----
|
||||
|
||||
func TestRunRaw_MergesIntoAnExistingPageInsteadOfOverwritingIt(t *testing.T) {
|
||||
// Before this, buildAndWrite did an unconditional os.WriteFile: re-ingesting
|
||||
// a source replaced its page wholesale, discarding anything accumulated
|
||||
// since — including human edits. hyperguild#43 recorded the symptom as
|
||||
// "re-ingest overwrites wholesale".
|
||||
dir := t.TempDir()
|
||||
page := filepath.Join(dir, "wiki", "sources", "shape-up.md")
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(page), 0o755))
|
||||
require.NoError(t, os.WriteFile(page, []byte(`---
|
||||
title: 'Shape Up'
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
A book about shaping product work.
|
||||
|
||||
## My Take
|
||||
|
||||
Read this before the next planning cycle.
|
||||
`), 0o644))
|
||||
|
||||
_, err := RunRaw(dir, "shape-up", []RawPage{{
|
||||
Title: "Shape Up", Type: "source", Subtype: "book", Domain: "product-strategy",
|
||||
Content: "## Summary\n\nA book about shaping product work.\n\n## Key Claims\n\n- Appetite beats estimation.\n",
|
||||
}}, false)
|
||||
require.NoError(t, err)
|
||||
|
||||
got, err := os.ReadFile(page)
|
||||
require.NoError(t, err)
|
||||
s := string(got)
|
||||
|
||||
assert.Contains(t, s, "## My Take", "a section the extraction never saw must survive the merge")
|
||||
assert.Contains(t, s, "Read this before the next planning cycle.")
|
||||
assert.Contains(t, s, "Appetite beats estimation.", "newly extracted material must still land")
|
||||
}
|
||||
|
||||
func TestRunRaw_ReIngestingUnchangedContentIsByteIdentical(t *testing.T) {
|
||||
// Compile-on-ingest runs on every ingest, so "nothing changed" must produce
|
||||
// no diff at all. Otherwise every re-ingest dirties the tree, brain-sync
|
||||
// commits it, and the git history fills with noise that hides real edits.
|
||||
dir := t.TempDir()
|
||||
raw := []RawPage{{
|
||||
Title: "Shape Up", Type: "source", Subtype: "book", Domain: "product-strategy",
|
||||
Content: "## Summary\n\nA book about shaping product work.\n\n## Open Questions\n\n- Does appetite scale?\n",
|
||||
}}
|
||||
|
||||
_, err := RunRaw(dir, "shape-up", raw, false)
|
||||
require.NoError(t, err)
|
||||
first, err := os.ReadFile(filepath.Join(dir, "wiki", "sources", "shape-up.md"))
|
||||
require.NoError(t, err)
|
||||
|
||||
_, err = RunRaw(dir, "shape-up", raw, false)
|
||||
require.NoError(t, err)
|
||||
second, err := os.ReadFile(filepath.Join(dir, "wiki", "sources", "shape-up.md"))
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, string(first), string(second),
|
||||
"re-ingesting unchanged content must leave the page byte-identical")
|
||||
}
|
||||
|
||||
// --- #105: source drift — an unchanged re-ingest must not re-extract --------
|
||||
|
||||
func TestRun_SkipsExtractionWhenTheSourceIsUnchanged(t *testing.T) {
|
||||
// #103 made a re-ingest harmless (byte-identical merge). It is still not
|
||||
// free: the full LLM extraction runs over every chunk to produce a result
|
||||
// that merges to nothing. This is the cost half.
|
||||
dir := t.TempDir()
|
||||
var calls int
|
||||
cfg := Config{Complete: func(_ context.Context, _, _ string) (string, error) {
|
||||
calls++
|
||||
return `[{"title":"Shape Up","type":"source","subtype":"book","domain":"product","content":"## Summary\n\nA book.\n"}]`, nil
|
||||
}}
|
||||
|
||||
content := "---\nsource: https://example.com/shape-up\n---\n\nThe body of the article.\n"
|
||||
|
||||
first, err := Run(context.Background(), cfg, dir, content, "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, 1, calls, "first ingest must extract")
|
||||
assert.False(t, first.Skipped)
|
||||
|
||||
second, err := Run(context.Background(), cfg, dir, content, "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 1, calls, "unchanged content must not be extracted again")
|
||||
assert.True(t, second.Skipped, "the skip must be reported, not silent")
|
||||
}
|
||||
|
||||
func TestRun_ReExtractsWhenTheSourceBodyChanged(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
var calls int
|
||||
cfg := Config{Complete: func(_ context.Context, _, _ string) (string, error) {
|
||||
calls++
|
||||
return `[{"title":"Shape Up","type":"source","subtype":"book","domain":"product","content":"## Summary\n\nA book.\n"}]`, nil
|
||||
}}
|
||||
|
||||
_, err := Run(context.Background(), cfg, dir, "---\nsource: x\n---\n\nversion one\n", "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
_, err = Run(context.Background(), cfg, dir, "---\nsource: x\n---\n\nversion two — the source was edited\n", "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, 2, calls, "changed content must be re-extracted")
|
||||
}
|
||||
|
||||
func TestRun_FrontmatterOnlyChangeIsNotDrift(t *testing.T) {
|
||||
// The hash covers the body, so re-fetching a page whose metadata moved
|
||||
// (a re-crawl stamping a new date) is not treated as the source changing.
|
||||
dir := t.TempDir()
|
||||
var calls int
|
||||
cfg := Config{Complete: func(_ context.Context, _, _ string) (string, error) {
|
||||
calls++
|
||||
return `[{"title":"Shape Up","type":"source","subtype":"book","domain":"product","content":"## Summary\n\nA book.\n"}]`, nil
|
||||
}}
|
||||
|
||||
_, err := Run(context.Background(), cfg, dir, "---\nsource: x\nfetched: monday\n---\n\nsame body\n", "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
_, err = Run(context.Background(), cfg, dir, "---\nsource: x\nfetched: tuesday\n---\n\nsame body\n", "shape-up", false)
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, 1, calls, "only the body counts as the source")
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
"strings"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/textutil"
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/vectorstore"
|
||||
)
|
||||
|
||||
@@ -427,8 +428,5 @@ func excerpt(content string, maxLen int) string {
|
||||
if len(parts) == 3 {
|
||||
body = strings.TrimSpace(parts[2])
|
||||
}
|
||||
if len(body) > maxLen {
|
||||
return body[:maxLen] + "…"
|
||||
}
|
||||
return body
|
||||
return textutil.Truncate(body, maxLen)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
// Package startupretry retries a startup dependency that may not be serving
|
||||
// yet, instead of exiting on its first failure.
|
||||
//
|
||||
// It exists for the cold-boot ordering race (#83): on a cluster restart the
|
||||
// ingestion pod can reach OIDC discovery before dex is answering, gets a 502
|
||||
// from ingress, and dies — after having already connected postgres, backfilled
|
||||
// the graph and ingested session batches, all of which the restart redoes.
|
||||
// Nothing is misconfigured in that scenario; the pod simply started first.
|
||||
//
|
||||
// The budget is bounded on purpose. Retrying forever would turn a genuine
|
||||
// misconfiguration (wrong issuer URL, bad audience) into a process that looks
|
||||
// healthy and never serves, which is harder to diagnose than a crash loop.
|
||||
package startupretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Policy bounds the retry: at most Attempts calls, with the delay doubling
|
||||
// from Base and capped at Max.
|
||||
type Policy struct {
|
||||
Attempts int // total calls, including the first; < 1 means 1
|
||||
Base time.Duration // delay before the second attempt
|
||||
Max time.Duration // ceiling for the doubling delay
|
||||
}
|
||||
|
||||
// Do calls fn until it succeeds, the attempt budget is spent, or ctx ends.
|
||||
//
|
||||
// The returned error wraps the last failure, so callers keep the diagnosis
|
||||
// they would have had without the retry.
|
||||
func Do[T any](ctx context.Context, p Policy, fn func(context.Context) (T, error)) (T, error) {
|
||||
var zero T
|
||||
|
||||
attempts := p.Attempts
|
||||
if attempts < 1 {
|
||||
attempts = 1
|
||||
}
|
||||
|
||||
delay := p.Base
|
||||
var lastErr error
|
||||
|
||||
for i := range attempts {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return zero, fmt.Errorf("startupretry: %w", err)
|
||||
}
|
||||
|
||||
v, err := fn(ctx)
|
||||
if err == nil {
|
||||
return v, nil
|
||||
}
|
||||
lastErr = err
|
||||
|
||||
if i == attempts-1 {
|
||||
break
|
||||
}
|
||||
|
||||
if delay <= 0 {
|
||||
delay = time.Second
|
||||
}
|
||||
if p.Max > 0 && delay > p.Max {
|
||||
delay = p.Max
|
||||
}
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return zero, fmt.Errorf("startupretry: %w", ctx.Err())
|
||||
case <-time.After(delay):
|
||||
}
|
||||
delay *= 2
|
||||
}
|
||||
|
||||
return zero, fmt.Errorf("startupretry: gave up after %d attempts: %w", attempts, lastErr)
|
||||
}
|
||||
@@ -0,0 +1,144 @@
|
||||
// ingestion/internal/startupretry/startupretry_test.go
|
||||
package startupretry
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func fastPolicy(attempts int) Policy {
|
||||
return Policy{Attempts: attempts, Base: time.Millisecond, Max: 2 * time.Millisecond}
|
||||
}
|
||||
|
||||
func TestDo_ReturnsImmediatelyOnFirstSuccess(t *testing.T) {
|
||||
calls := 0
|
||||
got, err := Do(context.Background(), fastPolicy(5), func(context.Context) (string, error) {
|
||||
calls++
|
||||
return "ready", nil
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("Do: %v", err)
|
||||
}
|
||||
if got != "ready" {
|
||||
t.Errorf("got %q, want %q", got, "ready")
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Errorf("calls = %d, want 1", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDo_SurvivesTransientFailuresThenSucceeds(t *testing.T) {
|
||||
// The observed cold-boot race: dex is not serving yet, ingress answers
|
||||
// 502 for the first few attempts, then discovery works.
|
||||
calls := 0
|
||||
got, err := Do(context.Background(), fastPolicy(5), func(context.Context) (string, error) {
|
||||
calls++
|
||||
if calls < 4 {
|
||||
return "", errors.New("oidc discovery: status 502")
|
||||
}
|
||||
return "ready", nil
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("Do returned %v — a transient 502 must not be fatal", err)
|
||||
}
|
||||
if got != "ready" {
|
||||
t.Errorf("got %q, want %q", got, "ready")
|
||||
}
|
||||
if calls != 4 {
|
||||
t.Errorf("calls = %d, want 4", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDo_GivesUpAfterTheAttemptBudgetAndReportsTheLastError(t *testing.T) {
|
||||
// A genuinely broken issuer must still fail — retrying forever would turn
|
||||
// a config error into a silent hang, which is worse than crash-looping.
|
||||
last := errors.New("oidc discovery: status 404")
|
||||
calls := 0
|
||||
|
||||
_, err := Do(context.Background(), fastPolicy(3), func(context.Context) (string, error) {
|
||||
calls++
|
||||
return "", last
|
||||
})
|
||||
|
||||
if !errors.Is(err, last) {
|
||||
t.Fatalf("err = %v, want it to wrap the last failure", err)
|
||||
}
|
||||
if calls != 3 {
|
||||
t.Errorf("calls = %d, want 3 (the attempt budget)", calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDo_StopsPromptlyWhenTheContextIsCancelled(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
calls := 0
|
||||
|
||||
done := make(chan struct{})
|
||||
var err error
|
||||
go func() {
|
||||
defer close(done)
|
||||
_, err = Do(ctx, Policy{Attempts: 100, Base: 50 * time.Millisecond, Max: time.Second},
|
||||
func(context.Context) (string, error) {
|
||||
calls++
|
||||
if calls == 1 {
|
||||
cancel()
|
||||
}
|
||||
return "", errors.New("still down")
|
||||
})
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("Do did not return after context cancellation")
|
||||
}
|
||||
if !errors.Is(err, context.Canceled) {
|
||||
t.Errorf("err = %v, want context.Canceled", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDo_BacksOffAndIsCappedByMax(t *testing.T) {
|
||||
var delays []time.Duration
|
||||
p := Policy{Attempts: 5, Base: 10 * time.Millisecond, Max: 25 * time.Millisecond}
|
||||
|
||||
last := time.Now()
|
||||
_, _ = Do(context.Background(), p, func(context.Context) (string, error) {
|
||||
now := time.Now()
|
||||
delays = append(delays, now.Sub(last))
|
||||
last = now
|
||||
return "", errors.New("down")
|
||||
})
|
||||
|
||||
// delays[0] is the first call (no wait); the rest are the sleeps.
|
||||
if len(delays) != 5 {
|
||||
t.Fatalf("attempts = %d, want 5", len(delays))
|
||||
}
|
||||
if delays[1] >= delays[2] {
|
||||
t.Errorf("backoff is not growing: %v then %v", delays[1], delays[2])
|
||||
}
|
||||
for i, d := range delays[1:] {
|
||||
if d > 2*p.Max {
|
||||
t.Errorf("delay %d = %v, exceeds the Max cap %v with slack", i+1, d, p.Max)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPolicy_ZeroValueStillMakesOneAttempt(t *testing.T) {
|
||||
// A caller that forgets to configure the policy must not silently get a
|
||||
// no-op that returns a nil validator and a nil error.
|
||||
calls := 0
|
||||
_, err := Do(context.Background(), Policy{}, func(context.Context) (string, error) {
|
||||
calls++
|
||||
return "", errors.New("down")
|
||||
})
|
||||
|
||||
if calls != 1 {
|
||||
t.Errorf("calls = %d, want 1", calls)
|
||||
}
|
||||
if err == nil {
|
||||
t.Error("err = nil, want the failure surfaced")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
// Package textutil holds small text helpers shared across the ingestion
|
||||
// service.
|
||||
package textutil
|
||||
|
||||
import "unicode/utf8"
|
||||
|
||||
// Ellipsis marks a value that was cut short.
|
||||
const Ellipsis = "…"
|
||||
|
||||
// Truncate shortens s to at most limit CHARACTERS, appending an ellipsis when
|
||||
// anything was dropped.
|
||||
//
|
||||
// It exists because four call sites did `s[:limit] + "…"` (#93). Both halves
|
||||
// of that are byte operations: len() counts bytes and a slice cuts bytes, so
|
||||
// on Swedish text — where å ä ö are two bytes and an em dash is three — the
|
||||
// cut lands mid-character and leaves an orphaned lead byte. The result is
|
||||
// invalid UTF-8.
|
||||
//
|
||||
// That failed loudly in exactly one place, wiki/index.md, which is written
|
||||
// raw to disk and then rejected by every strict decoder. The other three fed
|
||||
// MCP responses, where encoding/json silently substitutes U+FFFD — so those
|
||||
// had been quietly eating the last character of truncated excerpts with no
|
||||
// error anywhere.
|
||||
//
|
||||
// The marker is appended only when content was actually dropped: an ellipsis
|
||||
// on complete text is a lie about the content.
|
||||
func Truncate(s string, limit int) string {
|
||||
if limit <= 0 {
|
||||
return ""
|
||||
}
|
||||
if utf8.RuneCountInString(s) <= limit {
|
||||
return s
|
||||
}
|
||||
|
||||
// Walk runes rather than converting to []rune: the input can be a whole
|
||||
// note body, and this runs on the read path.
|
||||
count := 0
|
||||
for i := range s {
|
||||
if count == limit {
|
||||
return s[:i] + Ellipsis
|
||||
}
|
||||
count++
|
||||
}
|
||||
return s + Ellipsis
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
// ingestion/internal/textutil/truncate_test.go
|
||||
package textutil_test
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode/utf8"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/textutil"
|
||||
)
|
||||
|
||||
func TestTruncate_ShortStringsAreUntouched(t *testing.T) {
|
||||
for _, s := range []string{"", "kort", "räv", "a — b"} {
|
||||
if got := textutil.Truncate(s, 100); got != s {
|
||||
t.Errorf("Truncate(%q, 100) = %q, want it unchanged", s, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate_NeverCutsInsideACharacter(t *testing.T) {
|
||||
// The property, not an example: whatever the limit, the result must be
|
||||
// valid UTF-8. Byte-slicing Swedish text (å ä ö) or an em dash produces
|
||||
// an orphaned lead byte, which is what corrupted wiki/index.md (#93).
|
||||
s := strings.Repeat("räv — ål — ökäd ", 20)
|
||||
|
||||
for limit := 0; limit <= len(s)+5; limit++ {
|
||||
got := textutil.Truncate(s, limit)
|
||||
if !utf8.ValidString(got) {
|
||||
t.Fatalf("Truncate(s, %d) produced invalid UTF-8: %q", limit, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate_CountsCharactersNotBytes(t *testing.T) {
|
||||
// "räv" is 4 bytes, 3 runes. A byte-based limit of 3 would cut inside the
|
||||
// "ä"; a rune-based one returns it whole.
|
||||
if got := textutil.Truncate("räv", 3); got != "räv" {
|
||||
t.Errorf("Truncate(\"räv\", 3) = %q, want the whole word", got)
|
||||
}
|
||||
|
||||
got := textutil.Truncate("räven springer", 5)
|
||||
if utf8.RuneCountInString(got) != 6 { // 5 runes + the ellipsis
|
||||
t.Errorf("Truncate(...) = %q, want 5 runes plus the marker", got)
|
||||
}
|
||||
if !strings.HasSuffix(got, "…") {
|
||||
t.Errorf("Truncate(...) = %q, want it marked as truncated", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate_MarksOnlyWhenSomethingWasDropped(t *testing.T) {
|
||||
// An ellipsis on text that was not cut is a lie about the content.
|
||||
if got := textutil.Truncate("räv", 10); strings.HasSuffix(got, "…") {
|
||||
t.Errorf("Truncate(%q, 10) = %q, want no truncation marker", "räv", got)
|
||||
}
|
||||
if got := textutil.Truncate("räv", 3); strings.HasSuffix(got, "…") {
|
||||
t.Errorf("exact-length input must not be marked: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTruncate_NonPositiveLimitYieldsNothingRatherThanPanicking(t *testing.T) {
|
||||
for _, limit := range []int{0, -1, -100} {
|
||||
if got := textutil.Truncate("räven", limit); got != "" {
|
||||
t.Errorf("Truncate(_, %d) = %q, want empty", limit, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/mathiasbq/hyperguild/ingestion/internal/textutil"
|
||||
)
|
||||
|
||||
// RebuildIndex writes brain/wiki/index.md from the current wiki contents.
|
||||
@@ -62,10 +64,7 @@ func pageFirstSentence(brainDir string, e Entry) string {
|
||||
if line == "" || strings.HasPrefix(line, "#") {
|
||||
continue
|
||||
}
|
||||
if len(line) > 100 {
|
||||
return line[:100] + "…"
|
||||
}
|
||||
return line
|
||||
return textutil.Truncate(line, 100)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -34,7 +34,7 @@ type section struct {
|
||||
// all others keep a's version. Sections in b not present in a are appended.
|
||||
func Merge(a, b Page) Page {
|
||||
fmA, secsA := parseSections(a.Content)
|
||||
_, secsB := parseSections(b.Content)
|
||||
fmB, secsB := parseSections(b.Content)
|
||||
|
||||
idx := make(map[string]int, len(secsA))
|
||||
for i, s := range secsA {
|
||||
@@ -53,11 +53,60 @@ func Merge(a, b Page) Page {
|
||||
case bulletSections[sB.heading]:
|
||||
secsA[i].content = mergeBullets(sA.content, sB.content)
|
||||
case appendSections[sB.heading]:
|
||||
secsA[i].content = strings.TrimRight(sA.content, "\n") + "\n\n" + strings.TrimLeft(sB.content, "\n")
|
||||
secsA[i].content = appendNew(sA.content, sB.content)
|
||||
}
|
||||
}
|
||||
|
||||
return Page{Path: a.Path, Content: rebuildContent(fmA, secsA)}
|
||||
return Page{Path: a.Path, Content: rebuildContent(refreshOwned(fmA, fmB), secsA)}
|
||||
}
|
||||
|
||||
// generatorOwned names the frontmatter fields the ingest pipeline may refresh
|
||||
// on a page it is revising. Everything else on disk wins.
|
||||
//
|
||||
// #103 made the on-disk frontmatter win wholesale, which is what stops a
|
||||
// regenerating writer from overwriting human-set metadata — a title someone
|
||||
// fixed, or the claim-property envelope (brain#21). The cost was that fields
|
||||
// the generator legitimately owns froze at their first-ingest values: when it
|
||||
// last ran, and what it last saw.
|
||||
//
|
||||
// An explicit allowlist rather than a carve-out: the ownership boundary is
|
||||
// nameable, and a field nobody has thought about stays human-owned by default,
|
||||
// which is the safe direction.
|
||||
var generatorOwned = map[string]bool{
|
||||
"last_updated": true,
|
||||
"source_hash": true,
|
||||
}
|
||||
|
||||
// refreshOwned returns a's frontmatter with the generator-owned fields taken
|
||||
// from b where b supplies them. Fields absent from b keep a's value, so a
|
||||
// generator that stops emitting a field cannot blank it.
|
||||
func refreshOwned(a, b string) string {
|
||||
bVals := make(map[string]string)
|
||||
for _, line := range strings.Split(b, "\n") {
|
||||
key, value, found := strings.Cut(line, ":")
|
||||
if !found || !generatorOwned[strings.TrimSpace(key)] {
|
||||
continue
|
||||
}
|
||||
bVals[strings.TrimSpace(key)] = strings.TrimSpace(value)
|
||||
}
|
||||
if len(bVals) == 0 {
|
||||
return a
|
||||
}
|
||||
|
||||
out := make([]string, 0, len(strings.Split(a, "\n")))
|
||||
for _, line := range strings.Split(a, "\n") {
|
||||
key, _, found := strings.Cut(line, ":")
|
||||
k := strings.TrimSpace(key)
|
||||
if found && generatorOwned[k] {
|
||||
if v, ok := bVals[k]; ok {
|
||||
out = append(out, k+": "+v)
|
||||
delete(bVals, k)
|
||||
continue
|
||||
}
|
||||
}
|
||||
out = append(out, line)
|
||||
}
|
||||
return strings.Join(out, "\n")
|
||||
}
|
||||
|
||||
func parseSections(markdown string) (frontmatter string, sections []section) {
|
||||
@@ -96,15 +145,64 @@ func parseSections(markdown string) (frontmatter string, sections []section) {
|
||||
return
|
||||
}
|
||||
|
||||
// rebuildContent serialises sections back into a page.
|
||||
//
|
||||
// Each section body is normalised to exactly one trailing newline and no
|
||||
// leading blank line. Without that, a section keeps whatever blank lines it was
|
||||
// parsed with and this function adds its own, so every merge pass grows the page
|
||||
// by a blank line — invisible in a diff viewer, fatal to byte-stability. Compile-on-ingest
|
||||
// merges on every run, so "merging changes nothing when nothing changed" has to
|
||||
// hold at the byte level, not just the content level.
|
||||
func rebuildContent(frontmatter string, sections []section) string {
|
||||
var sb strings.Builder
|
||||
sb.WriteString(frontmatter)
|
||||
for _, sec := range sections {
|
||||
fmt.Fprintf(&sb, "\n## %s\n\n%s", sec.heading, sec.content)
|
||||
body := strings.Trim(sec.content, "\n")
|
||||
fmt.Fprintf(&sb, "\n## %s\n\n%s\n", sec.heading, body)
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
// appendNew concatenates the lines of b that a does not already carry.
|
||||
//
|
||||
// Append sections used to concatenate wholesale, which is correct exactly once.
|
||||
// Compile-on-ingest merges a freshly extracted page into the one already on
|
||||
// disk, so an unchanged source re-ingested would grow "Open Questions" and
|
||||
// "Updates" on every run until the page is unreadable — and the drift guard
|
||||
// that would otherwise catch it (hashing the source body) does not exist yet,
|
||||
// so idempotency has to hold here rather than upstream.
|
||||
//
|
||||
// Line-level rather than block-level: the common case is a re-ingest carrying
|
||||
// one genuinely new line alongside several it already has. A block-level
|
||||
// "skip if identical" would duplicate the old lines every time that happened.
|
||||
//
|
||||
// The cost is that a line repeated verbatim on purpose collapses to one. For
|
||||
// these sections — questions, updates, evolving notes — a repeat is noise
|
||||
// rather than signal, and the failure is conservative: the page stops growing
|
||||
// instead of growing wrongly.
|
||||
func appendNew(a, b string) string {
|
||||
seen := make(map[string]bool)
|
||||
for _, line := range strings.Split(a, "\n") {
|
||||
if t := strings.TrimSpace(line); t != "" {
|
||||
seen[t] = true
|
||||
}
|
||||
}
|
||||
|
||||
var add []string
|
||||
for _, line := range strings.Split(b, "\n") {
|
||||
t := strings.TrimSpace(line)
|
||||
if t == "" || seen[t] {
|
||||
continue
|
||||
}
|
||||
seen[t] = true
|
||||
add = append(add, line)
|
||||
}
|
||||
if len(add) == 0 {
|
||||
return a // nothing new — leave the section byte-identical
|
||||
}
|
||||
return strings.TrimRight(a, "\n") + "\n\n" + strings.Join(add, "\n") + "\n"
|
||||
}
|
||||
|
||||
func mergeBullets(a, b string) string {
|
||||
seen := make(map[string]bool)
|
||||
var lines []string
|
||||
|
||||
@@ -53,3 +53,93 @@ func TestMerge_KeepsFrontmatterFromA(t *testing.T) {
|
||||
assert.Contains(t, got.Content, "title: A")
|
||||
assert.NotContains(t, got.Content, "title: B")
|
||||
}
|
||||
|
||||
// --- brain#21/B1: merging must be safe to run again -------------------------
|
||||
|
||||
func TestMerge_AppendSectionDoesNotDuplicateContentAlreadyPresent(t *testing.T) {
|
||||
// Compile-on-ingest merges a freshly extracted page into the one already
|
||||
// on disk. Bullet sections union, so they are naturally idempotent — but
|
||||
// append sections concatenate, so re-ingesting an unchanged source would
|
||||
// grow "Open Questions" on every run until the page is unreadable.
|
||||
page := Page{Path: "wiki/sources/x.md", Content: `---
|
||||
title: X
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Does this converge?
|
||||
`}
|
||||
|
||||
once := Merge(page, page)
|
||||
twice := Merge(once, page)
|
||||
|
||||
assert.Equal(t, 1, strings.Count(once.Content, "Does this converge?"),
|
||||
"merging a page with itself must not duplicate an append section")
|
||||
assert.Equal(t, once.Content, twice.Content,
|
||||
"merge must be idempotent — a third pass may not keep growing the page")
|
||||
}
|
||||
|
||||
func TestMerge_AppendSectionStillAccumulatesGenuinelyNewContent(t *testing.T) {
|
||||
// The dedup must not turn append sections into keep-a-only sections:
|
||||
// new material is exactly what compile-on-ingest exists to add.
|
||||
existing := Page{Path: "wiki/sources/x.md", Content: `---
|
||||
title: X
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Does this converge?
|
||||
`}
|
||||
fresh := Page{Path: "wiki/sources/x.md", Content: `---
|
||||
title: X
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
- What happens under load?
|
||||
`}
|
||||
|
||||
got := Merge(existing, fresh).Content
|
||||
assert.Contains(t, got, "Does this converge?", "existing question must survive")
|
||||
assert.Contains(t, got, "What happens under load?", "new question must be added")
|
||||
}
|
||||
|
||||
func TestMerge_GeneratorOwnedFrontmatterFieldsAreRefreshed(t *testing.T) {
|
||||
// #103 made on-disk frontmatter win wholesale, which is what protects
|
||||
// human-set fields from a regenerating writer. The cost was that fields the
|
||||
// generator legitimately owns — when it last ran, what it last saw — froze
|
||||
// at their first-ingest values. An explicit owned-set is the boundary:
|
||||
// named fields the generator may refresh, everything else human-owned.
|
||||
existing := Page{Path: "wiki/sources/x.md", Content: `---
|
||||
title: 'X'
|
||||
last_updated: 2026-01-01
|
||||
source_hash: aaaa
|
||||
half_life: durable
|
||||
claim_provenance: asserted
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Original.
|
||||
`}
|
||||
fresh := Page{Path: "wiki/sources/x.md", Content: `---
|
||||
title: 'Renamed by the model'
|
||||
last_updated: 2026-08-22
|
||||
source_hash: bbbb
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Rewritten by the model.
|
||||
`}
|
||||
|
||||
got := Merge(existing, fresh).Content
|
||||
|
||||
assert.Contains(t, got, "last_updated: 2026-08-22", "generator owns last_updated")
|
||||
assert.Contains(t, got, "source_hash: bbbb", "generator owns source_hash")
|
||||
assert.Contains(t, got, "title: 'X'", "a human-set title must survive regeneration")
|
||||
assert.Contains(t, got, "half_life: durable", "the claim envelope must survive regeneration")
|
||||
assert.Contains(t, got, "claim_provenance: asserted")
|
||||
assert.Contains(t, got, "Original.", "an existing section still wins over the model's rewrite")
|
||||
assert.NotContains(t, got, "Renamed by the model")
|
||||
}
|
||||
|
||||
@@ -42,34 +42,35 @@ These actions are **carried into the Phase 4 capture call** as `tickets[]` rathe
|
||||
|
||||
## Phase 4 — Capture (one uniform call)
|
||||
|
||||
Persist the session via a **single `capture` call** (the `brain:capture` MCP tool, live on the Claude.ai connector). Capture owns the writes server-side — insights → brain, action items → Gitea tickets, summary → ai-sessions — plus the I1 sovereignty gate, the I5 audit record, and the supersession/read-after-write discipline. The skill's job is to *assemble the payload*, not to write each store itself. Do NOT fall back to separate `gitea:file_write_branch` + `brain_write` steps unless `capture` is unreachable (see fallback below).
|
||||
Persist the session via a **single `capture` call** (the `brain:capture` MCP tool, live on the Claude.ai connector). Capture owns the writes server-side — insights → brain, action items → Gitea tickets — plus the I1 sovereignty gate, the I5 audit record, and the supersession/read-after-write discipline. The skill's job is to *assemble the payload*, not to write each store itself. Do NOT fall back to separate `gitea:file_write_branch` + `brain_write` steps unless `capture` is unreachable (see fallback below).
|
||||
|
||||
**Note:** capture no longer writes a session summary anywhere (the `summary` → `ai-sessions` write path was removed — it produced a frontmatter shape `ai-sessions`' own pipeline couldn't parse, silently invisible to that repo's own audit tooling; see `ai-sessions#13`). If the session's decisions/artifacts are worth a durable narrative beyond the `insights[]` this skill writes to the brain, that's a separate, explicit call — not something this skill does implicitly.
|
||||
|
||||
**Assemble one payload:**
|
||||
|
||||
- **`insights[]`** — the generalizable learnings from Phase 1 (decisions/failures worth re-reading). Each: `{text, wing, hall}`; add `supersede_slug` to revise a prior note in place instead of creating a duplicate. `hall` ∈ facts/decisions/failures/hypotheses/sources.
|
||||
- **`tickets[]`** — the issue actions from Phase 3: `{repo, action, ...}` where action ∈ create/close/comment. Owner is always `mathias` (server-forced).
|
||||
- **`summary`** — `{title, body, repos_touched}`. Capture writes it to `ai-sessions` and stamps `fidelity` in frontmatter. Body stays reconstructable: one-paragraph summary, decisions, key artifacts, open threads.
|
||||
- **`context`** — `{harness: "claudeai-chat", session_ref: <chatid8-or-slug>, fidelity: "live-capture", actor: "mathias", classification: <see gate below>}`.
|
||||
|
||||
**THE CLASSIFICATION GATE (read before calling — this is where capture refuses).**
|
||||
Capture computes an **effective classification = the strictest across EVERY target it touches** (each insight's `wing`, each ticket's `repo`, and every entry in `summary.repos_touched`), then refuses if that effective level is `confidential` and the origin is us-nexus (claude.ai is us-nexus). Levels come from `classification.yaml` at the brain root (source of truth, #67), with the code defaults as the floor: `hyperguild`/`homelab` → internal; `client-*` → confidential; **anything untagged → confidential (fail-safe)**.
|
||||
Capture computes an **effective classification = the strictest across EVERY target it touches** (each insight's `wing`, each ticket's `repo`), then refuses if that effective level is `confidential` and the origin is us-nexus (claude.ai is us-nexus). Levels come from `classification.yaml` at the brain root (source of truth, #67), with the code defaults as the floor: `hyperguild`/`homelab` → internal; `client-*` → confidential; **anything untagged → confidential (fail-safe)**.
|
||||
- **Tagged `internal` today** (safe through claude.ai): wings `hyperguild`, `homelab`; repos `brain`, `ai-sessions`, `infra`, `hyperguild`, `homelab`, `tapir`, `agentsquad`, `jepa-fx-risk`, `swedsl`. Treat `classification.yaml` as authoritative — this list is a hint, not gospel.
|
||||
- Declare `context.classification: "internal"` for normal homelab work.
|
||||
- `summary.repos_touched`, insight `wing`s, and ticket `repo`s are classification INPUTS, not free-form metadata — every target must resolve `internal` or the whole capture escalates to `confidential` and the gate refuses via claude.ai. Listing the central homelab repos (incl. `brain`/`ai-sessions`) is now fine; they're tagged. The summary always lands in `ai-sessions` (internal), so the summary path itself never escalates.
|
||||
- Insight `wing`s and ticket `repo`s are classification INPUTS, not free-form metadata — every target must resolve `internal` or the whole capture escalates to `confidential` and the gate refuses via claude.ai. Listing the central homelab repos (incl. `brain`/`ai-sessions`) is now fine; they're tagged.
|
||||
- If a session genuinely touched **`client-*` or otherwise-untagged** material, it cannot be captured through claude.ai — note that in the verdict rather than trying to force it.
|
||||
|
||||
**GATE — dry-run first, then execute.**
|
||||
1. Call `capture` with `dry_run: true`. It validates the whole payload and returns the would-be receipt + `effective_classification`, writing nothing.
|
||||
2. **STOP. Show the dry-run receipt** (effective classification, the insights/tickets/summary that would land) and get explicit confirmation.
|
||||
2. **STOP. Show the dry-run receipt** (effective classification, the insights/tickets that would land) and get explicit confirmation.
|
||||
3. On confirmation, call `capture` again with `dry_run: false`. Read the returned receipt: it is partial-aware (`errors[]`, per-item `ok`). Report exactly what landed.
|
||||
|
||||
If `capture` is **unreachable** (tool not on the connector — e.g. a session that started before a deploy; a tool-list refresh usually fixes it): say so. Only then fall back to the legacy inline path (`gitea:file_write_branch` summary + `brain_write`/`brain_update` + `brain_get` confirm), and note in the verdict that the I5 audit record was NOT produced.
|
||||
If `capture` is **unreachable** (tool not on the connector — e.g. a session that started before a deploy; a tool-list refresh usually fixes it): say so. Only then fall back to the legacy inline path (`brain_write`/`brain_update` + `brain_get` confirm), and note in the verdict that the I5 audit record was NOT produced.
|
||||
|
||||
## Phase 5 — Verdict
|
||||
|
||||
Deliver a final "safe to archive" verdict in the chat. Either:
|
||||
|
||||
- **SAFE TO ARCHIVE** — list what landed from the capture receipt (issues closed/filed with numbers, summary path, brain note ids/paths) so the trail is auditable. Then list anything still in the user's queue (e.g. a PR awaiting their merge, a decision owed next session).
|
||||
- **SAFE TO ARCHIVE** — list what landed from the capture receipt (issues closed/filed with numbers, brain note ids/paths) so the trail is auditable. Then list anything still in the user's queue (e.g. a PR awaiting their merge, a decision owed next session).
|
||||
- **NOT YET** — name the specific gate that wasn't passed, the capture refusal reason, or the per-item error from the receipt, and what to do about it.
|
||||
|
||||
Never claim safe-to-archive if the capture refused, any receipt item errored, or a gated confirmation was declined. The verdict is the skill's contract: if it says safe, the session can be lost without losing the work.
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
**Status:** Decisions resolved 2026-06-22 (§4). Ready for implementation scoping. `capture` is a
|
||||
privileged cross-harness write path touching brain + Gitea + ai-sessions.
|
||||
**Superseded 2026-07-27:** the ai-sessions summary write path specified below was removed post-ship
|
||||
(see `specs/capture-implementation-report.md`). `capture` now touches brain + Gitea only.
|
||||
**Tracks:** hyperguild #49.
|
||||
**Governed by:** `infra/docs/architecture/01-invariants.md` (I1–I5), the admissibility test in
|
||||
`00-synthesis-model.md`, and the distributed-consolidation shape mandated by
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
# Capture capability — implementation report (as-built)
|
||||
|
||||
**Status:** Shipped 2026-06-23, tagged `v0.11.0`. Epic hyperguild #49 (sub-issues #50–#55) closed.
|
||||
**Superseded 2026-07-27:** the `summary` → `ai-sessions` write path documented below was removed.
|
||||
It wrote a frontmatter shape (`title`/`harness`/`fidelity`/`captured_at`/`repos_touched`) that
|
||||
`ai-sessions`' own extract/audit pipeline couldn't parse — invisible to that repo's own audit
|
||||
tooling and bypassing its redaction/completeness gates (`ai-sessions#13`). `capture` now persists
|
||||
insights → brain and action items → Gitea tickets only. This document is kept as the historical
|
||||
as-built record of what shipped in #49; treat every `summary`/ai-sessions reference below as
|
||||
retired, not current behaviour.
|
||||
**Spec:** `specs/capture-bdd-spec.md` (the design contract this implements).
|
||||
**Governed by:** `infra/docs/architecture/01-invariants.md` (I1–I5) + the I2 acceptance ledger entry in `infra/docs/security-baseline.md`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user