Compare commits

...
71 Commits
Author SHA1 Message Date
mathiasandClaude Opus 4.8 fcbd1072b6 fix(lint): check fmt.Fprintf errors in webhook handler (infra#190)
CI / Lint / Test / Vet (push) Successful in 16s
CI / Mirror to GitHub (push) Has been skipped
errcheck flagged two unchecked fmt.Fprintf return values in
internal/webhook/webhook.go, failing CI (run 310/311) for the whole
'trigger brain-sync on Gitea push' feature -- so no new ingestion image was
ever built, and the deployed image predates this feature entirely even
though infra's manifest (secret, RBAC, env wiring) was already live.

Both writes are best-effort informational text after WriteHeader has already
committed the status code -- a failed write here only happens on client
disconnect and nothing depends on it succeeding, so explicitly discard
(_, _ =) rather than log-and-continue noise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 08:29:39 +02:00
mathias 3b7706b358 chore(context): re-sync derived adapters (skills-source paragraph) 2026-07-09 08:28:54 +02:00
mathias 394a227877 feat(webhook): trigger brain-sync on Gitea push instead of 15-min poll
CI / Mirror to GitHub (push) Has been skipped
CI / Lint / Test / Vet (push) Failing after 8s
Adds POST /webhooks/brain-sync: verifies Gitea's HMAC-SHA256 signature,
checks the push is to mathias/brain main, then creates a one-off Job from
the existing brain-sync CronJob's template (same script the 15-min poll
already runs, just triggered on-demand). Off by default -- opt in via
GITEA_WEBHOOK_SECRET, since it needs Job-create RBAC in the "brain"
namespace a fresh deploy won't have.

10 new tests (internal/webhook), including a fake-clientset reactor to
simulate server-side GenerateName expansion, which the plain fake tracker
doesn't do on its own.

Needs (follow-up, infra repo): RBAC granting ingestion's ServiceAccount
get on cronjobs/brain-sync + create on jobs in the brain namespace, the
GITEA_WEBHOOK_SECRET env, and the actual Gitea webhook registration.
2026-07-07 22:00:56 +02:00
mathias 0f84ab5eda fix(ingest): route raw claude-session dumps to non-indexed archive, not wiki
CI / Lint / Test / Vet (push) Successful in 11s
CI / Mirror to GitHub (push) Has been skipped
claudeSink wrote every raw transcript into brain/wiki/claude-sessions/facts/
(BM25-indexed), duplicating what ai-sessions' curated summaries already
cover and out-ranking them in search (177,818 vs 45,335 BM25 weight on the
same query, per ai-sessions#10 investigation). Raw dumps now land in
brain/archive/claude-sessions/<host>/ — kept for deep lookups, never
indexed, never deleted.

Refs: ai-sessions#10
2026-07-07 11:36:01 +02:00
mathias 5b57843346 fix(lint): revert redundant Write conversion (staticcheck S1016)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
writeRequest and WriteNoteOptions have matching fields again now that
both carry SourceType, so the explicit field-by-field literal added
in ee1204d is flagged as a redundant conversion. Back to the plain
type conversion. Caught by task check (golangci-lint), not run
locally before the previous push -- plain go test passed, task check
(which adds -race and golangci-lint) didn't.
2026-07-04 14:13:16 +02:00
mathias ee1204d76b fix(api): default hall=facts source_type to internal (brain-gardener#7)
CI / Lint / Test / Vet (push) Failing after 5s
CI / Mirror to GitHub (push) Has been skipped
The unsourced-check fix (source_type: internal) only covered
claudewatcher's writes. 41 residual findings on the audit's next run
were manually-authored hall=facts entries written via brain_write/
capture -- also first-party observations, just a different write path,
with no way to mark them internal short of hand-editing every entry.

writeHallNote now defaults source_type to internal whenever hall ==
"facts" and the caller didn't set it explicitly. An explicit
source_type: external (now threaded through the /write HTTP endpoint)
opts a genuinely citation-needing entry back out.
2026-07-04 14:09:20 +02:00
mathias 6d58336ce2 fix(claudewatcher): tag session-dump notes source_type: internal
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
brain-gardener#5: the audit's unsourced check was flagging 59 raw
claudewatcher session dumps as high-severity hallucination risk,
because it can't distinguish a first-party observation (a session
transcript) from an external claim needing citation.

Adds WriteNoteOptions.SourceType, emitted as source_type: <value> in
the wing/hall frontmatter route. claudeSink (the claudewatcher sink)
now always sets source_type: internal on the notes it writes.
2026-07-04 13:53:08 +02:00
mathias 0785f14220 Merge pull request 'Consolidate to single harness: icebox cmd/routing, keep cmd/hyperguild + ingestion (#75)' (#76) from feat/consolidate-single-harness into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-07-01 16:27:06 +00:00
mathiasandClaude Opus 4.8 a7db0dd00d feat(mode): brain+gitea in all modes; drop iceboxed routing entry (#75)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
Step 7 of the consolidation: reflect gitea-mcp as an available connection
(Gitea-as-audit-trail is an invariant going forward) and stop emitting the
now-iceboxed routing pod.

- cmd/hyperguild mode.go: every mode (cloud/client-local/sovereign) now lists
  brain + gitea; removed the client-local `routing` entry (koala:30310, the
  iceboxed pod). Tests updated to the new contract.
- .mcp.json: add gitea-mcp (git-mcp.d-ma.be, Bearer ${GITEA_MCP_TOKEN}).
- cmd/hyperguild/README: modes doc updated (brain+gitea, no routing).

Refs #75.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 13:27:31 +02:00
mathiasandClaude Opus 4.8 fb59c390e2 refactor(harness): icebox cmd/routing path — consolidate to single harness (#75)
Consolidate to the hyperguild-alone harness that ran the infra#170 loop-1
experiment (cmd/hyperguild + brain-mcp; routing/injection machinery off).
Verified cmd/hyperguild has zero transitive dep on internal/routing or
cmd/routing's packages (imports only internal/tier). Removed the routing
path, which is dormant from the live session (.mcp.json wires only brain-mcp)
and entangled with the survivorship-biased pass-rate router (infra#174).

Iceboxed (recoverable at tag icebox/cmd-routing-2026-07-01 / branch
icebox/cmd-routing, commit 00e5f62):
- cmd/routing/, internal/routing/
- internal/skills/{review,debug,retrospective,trainer,project}/
  (each imported solely by cmd/routing — verified)
- Dockerfile.routing
- cd.yml: routing image build + infra bump + Flux-wait/rollout-verify
  (ingestion build/deploy unchanged)

Kept: cmd/hyperguild, ingestion/, internal/tier (used by cmd/hyperguild +
skills/org), internal/skills/{brain,org,sessionlog} (per #75; already orphaned).

Regression: root module 21→14 pkgs (−7 = exactly the removed set, no other
breakage); ingestion 24/24 unchanged. See ICEBOX.md. Refs #75, infra#170, #174.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 13:22:26 +02:00
mathiasandClaude Opus 4.8 00e5f62c8e docs(runbook): correct flywheel (nil→local) + instrumentation-fixed note
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
Two corrections after the #73 investigation: (1) the router routes cold
(nil pass-rate) calls to the LOCAL fast tier, not cloud — the earlier
"pay in on cloud" description was wrong (per policy.go). (2) Note that
pass-rate logging was broken until v0.11.1 (#73) and is now verified;
reset the window to 2026-06-30→07-14. Refs #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:16:03 +02:00
mathiasandClaude Opus 4.8 b9d03316fd fix(ingestion): migrate mcp-chassis import to git.d-ma.be path (#74)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Has been skipped
Unblocks CD: the build's go mod download failed because mcp-chassis was
imported via the old gitea.d-ma.be module path, which the renamed server no
longer serves a matching go-import meta tag for. Point at the renamed module
git.d-ma.be/mathias/mcp-chassis v0.2.0 (mcp-chassis 69d389d) + go mod tidy.
Fresh download now resolves; ingestion builds + tests green.

This restores the normal CI path so the routing pass-rate fix (da9bdc4,
#73) builds and deploys through gitea Actions. Refs #74, #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 21:39:41 +02:00
mathiasandClaude Opus 4.8 da9bdc4cbb fix(routing): repair pass-rate instrumentation (3 bugs) — #73, #35
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
The #35 data gate could never fill: a real review call routed cleanly to
qwen36 but /pass-rate stayed total:0 under every key. Root cause was three
independent defects in the session_log path, each alone fatal:

- A: a successful routed call logged final_status "skip", never "pass".
  /pass-rate computes pass/(pass+fail) and skips count as neither, so the
  >=0.90 gate was mathematically unreachable. Success now logs "pass".
- B: every record was written under skill "_routing", so /pass-rate?skill=
  review|debug (what #35 measures) always read zero. Now uses the real
  e.Skill; routing decisions stay groupable via session_id "_routing".
- C: the session_log POST to the bearer-gated ingestion /mcp carried no
  Authorization header → silent 401, swallowed by best-effort logging
  (the documented mcpclient-empty-token-silent-401 footgun). Logger now
  takes a token (BRAIN_MCP_TOKEN) and sets the bearer when non-empty.

Tests rewritten to assert correct behavior (they had encoded the bugs:
"skip" on success, "_routing" skill). New test covers the auth header and
the empty-token path.

Infra (BRAIN_MCP_TOKEN ExternalSecret + env on the routing deployment) and
redeploy follow separately. Refs #73, #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 08:36:18 +02:00
mathiasandClaude Opus 4.8 dcb9ff4a56 docs(runbook): how to exercise review/debug traffic for the pass-rate gate
CI / Lint / Test / Vet (push) Successful in 20s
CI / Mirror to GitHub (push) Successful in 3s
The #35 data gate stays at zero because pass-rate only accrues from review/
debug calls *through the routing pod* — Crush, cloud chat, and the local
.skills all bypass it. Document the connect → route → flywheel steps, the
endpoints (koala:30310/mcp + routing-mcp.d-ma.be), the cold-start behavior
(nil pass-rate routes cloud until passes accrue past the 0.90 floor, then
local qwen36 activates), the 50-invocation / 2026-07-10 target, and the
Berget fallback. Refs #35.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:41:07 +02:00
mathias 0454527b83 Merge pull request 'feat: brain_pending + brain_promote — close the raw→wiki curation loop (#38)' (#70) from feat/brain-pending-promote into main
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
2026-06-26 14:49:27 +00:00
mathiasandClaude Opus 4.8 ef11864121 feat(mcp): brain_pending + brain_promote tools + REST routes (#38)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Wires the curation primitives into the MCP surface (all three sites:
tools() descriptors, handleCall dispatch, package doc) and registers the
GET /pending + POST /promote REST routes. brain_promote additionally
re-indexes the promoted note into the graph (best-effort), matching the
other write paths. brain_pending is the human-review-queue complement to
the agent-facing brain_write.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:34:15 +02:00
mathiasandClaude Opus 4.8 14b04a25cb feat(brain): ListPending + PromoteNote — raw→wiki curation primitives (#38)
Closes the human curation loop: list brain/raw/ notes awaiting review
(oldest-first, with excerpt) and promote one into brain/wiki/<wing>/<hall>/.

PromoteNote rewrites frontmatter (sets wing/hall/promoted_at, preserves
created_at + custom fields via the existing frontmatter editor), rebuilds
the wing _index, and runs auto-tunnel. Atomic from the caller's view:
hall/wing/slug/collision validation happens before any fs change, and the
source is deleted only after the destination write succeeds (write-then-
delete, never move) — a collision or write failure leaves raw/ intact.
Default slug = filename minus the YYYY-MM-DD- prefix.

Also exposes GET /pending + POST /promote for shell scripts (bad
hall/collision/missing-source → 400, not 500).

Scope note: operates on raw/ (the retrospective-skill review queue) per
this issue's spec. The knowledge/ legacy pile is #22's bulk-migration
concern, not this ongoing queue.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:34:15 +02:00
mathias 66a9b8e725 Merge pull request 'docs(close-session): refresh classification gate post-#67 (#62)' (#69) from fix/close-session-classification-refresh into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
2026-06-26 14:28:48 +00:00
mathiasandClaude Opus 4.8 9f8fb9c138 docs(close-session): refresh classification gate post-#67 (#62)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
The capture-routing in 723dab5 (#62 minimal) carried pre-#67 prose: it
warned "no populated classification.yaml yet" and steered repos_touched
away from brain/ai-sessions as if they'd escalate to confidential. #67
tagged the homelab repos internal, so that guidance is now wrong and
over-restrictive — listing the central repos is fine, and the summary's
fixed ai-sessions target no longer escalates the call. Point at
classification.yaml as the source of truth; reserve the refusal warning
for client-*/untagged material. Completes #62's veneer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 16:25:57 +02:00
mathias d39a18dd69 Merge pull request 'fix: wire ai-sessions SummaryWriter into the capture relay (#66)' (#68) from fix/capture-summarywriter into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-23 15:22:42 +00:00
mathiasandClaude Opus 4.8 5288554338 fix(gitea): WriteFile creates via POST, updates via PUT (real contents API)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
A live token-scope probe against mathias/ai-sessions revealed gitea's
contents API uses POST to create and PUT (sha required) to update — the
first impl always PUT'd, so creating a new summary 422'd "[SHA]: Required".
The httptest mock had the same wrong assumption. Pick the method by
whether the file exists (GET sha). Token confirmed contents:write
(push:true) by the probe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 17:22:37 +02:00
mathiasandClaude Opus 4.8 2368564523 fix(capture): wire ai-sessions SummaryWriter into the relay (#66)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
The deployed CaptureService was constructed with a nil SummaryWriter, so
a capture carrying a summary block returned the partial-failure
"no summary writer configured" (surfaced in the 2026-06-23 claude.ai
dogfood). The Gitea client already reaches mathias/* over BRAIN_GITEA_TOKEN
and now implements SummaryWriter, so inject it (type-asserted from the
tracker) — no new credential, no manifest change. Session summaries now
write to mathias/ai-sessions at summaries/<harness>/<YYYY-MM>/...

Reuses the existing token deliberately; assumes it carries contents:write
scope on ai-sessions (it already does issue writes for the tracker). If
the token is issue-scoped only, the live write 422s — a token-scope widen,
not a code fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 10:47:00 +02:00
mathiasandClaude Opus 4.8 0e28b2125b feat(gitea): WriteFile contents-API upsert — satisfies SummaryWriter (#66)
Adds Client.WriteFile (Gitea contents API) so the Gitea client also
implements capture.SummaryWriter. Upserts: a GET resolves the current
blob sha so an existing file is updated (the richer-fidelity-supersedes
rule for re-captured sessions) rather than 422'd. Owner stays the fixed
const; token only in the Authorization header (no leak — regression
tested). Refactors the HTTP path into a shared request() helper so the
contents flow can branch on 404 without it being an error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 10:47:00 +02:00
mathias 723dab51ae feat(close-session): route closeout through the capture tool (#62 minimal)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
Collapse Phases 4 (summary commit) + 5 (brain note) into a single
capture-driven Phase 4. The skill now assembles one brain:capture payload
(insights + tickets + summary + context) and dry-runs-then-executes;
capture owns the brain/gitea/ai-sessions writes, the I1 gate, the I5
audit record, and the supersession/read-after-write discipline server-side.

Adds the classification-gate lesson learned dogfooding capture this
session: effective classification = strictest across ALL targets
(wings, ticket repos, repos_touched); untagged repos (brain, ai-sessions)
fail-safe to confidential and get refused via claude.ai, so repos_touched
must stay to internal-default repos. Legacy inline path kept only as the
capture-unreachable fallback. Staleness prose removed (server owns it now).

Partial of #62; harness-token + harvest-adapter + fidelity-supersession
work remain.
2026-06-23 06:27:57 +00:00
mathiasandClaude Opus 4.8 06e21c019e docs(capture): as-built implementation report for the #49 epic
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
Records what shipped (v0.11.0): architecture, sub-issue→PR map, I1–I5
compliance, the operational env reference, test coverage, and the
deferred follow-ups. Companion to specs/capture-bdd-spec.md (the design
contract) for onboarding + future audit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 07:41:24 +02:00
mathias 76514215f4 Merge pull request 'feat: capture relay — MCP capture tool for non-library harnesses (#55, capture 49f)' (#61) from feat/capture-mcp-relay into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Has been skipped
2026-06-23 05:26:55 +00:00
mathiasandClaude Opus 4.8 7cf5bc221d feat(mcp): capture relay tool — MCP door for non-library harnesses (#55)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Adds the `capture` MCP tool: the #55 relay for harnesses that cannot run
the use-case in-process (claude.ai Chat/Cowork/Design, Crush, Pi, LLM
Council). They reach it through the existing /mcp OAuth connector.

- Thin: forwards to the SAME CaptureService as POST /capture; holds no
  state and retains nothing beyond the I5 audit record. The containment
  properties accepted in infra security-baseline (I2 ledger) hold by
  construction.
- Per-principal: ServeHTTP re-derives the caller's principal from the
  Bearer header (the chassis middleware gates but discards it) and stashes
  it in context; the tool resolves the trust-zone origin from it. A
  caller-asserted harness/origin in the body is ignored — origin is
  server-derived, so the I1 confidential refusal still fires for us-nexus
  callers (claude.ai), and sovereign-allowlisted JWT principals pass.
- Registered only when WithCapture is wired (all three sites: tools(),
  handleCall, package doc); main wires REST + MCP from the same service,
  resolver, and credentials.

Tests: listed-only-when-wired, forwards-via-static-principal,
confidential-via-us-nexus-refused, confidential-via-sovereign-allowed,
unauthenticated-rejected, caller-cannot-forge-origin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 00:21:11 +02:00
mathiasandClaude Opus 4.8 f78a5474a5 refactor(capturehttp): export Authenticate + DecodeRequest for reuse (#55)
Lifts the Bearer principal-derivation and the request→CaptureInput decode
out of the REST handler into exported package funcs, so the MCP capture
tool (#55 relay) reuses the exact same auth precedence and wire shape —
one implementation, not two. No behaviour change to POST /capture.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 00:21:11 +02:00
mathias c307b72bd5 Merge pull request 'feat: I5 audit path + classification-aware degradation (#54, capture 49e)' (#60) from feat/capture-audit-degradation into main
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 21:57:52 +00:00
mathiasandClaude Opus 4.8 38a2e91002 feat(capturehttp): 503 on audit-unavailable; wire degrading sink (#54)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
- Map capture.ErrAuditUnavailable → HTTP 503 (audit substrate down /
  confidential unauditable / floor).
- main: buildAuditSink selects the DegradingSink (loki central + durable
  file buffer under brainDir + optional ntfy) when BRAIN_LOKI_URL is set
  and starts the reconcile loop; else the plain slog sink. Notifier kept
  as a nil interface (not typed-nil) when unconfigured so the sink and
  reconcile skip it cleanly.

Env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN,
BRAIN_AUDIT_RECONCILE_INTERVAL (default 60s). Buffer at
<brain>/.audit-buffer/capture.jsonl.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathiasandClaude Opus 4.8 77f5e06d6b feat(audit): DegradingSink + durable buffer + loki/ntfy + reconcile (#54)
The classification-aware I5 audit path (§4.4):

- DegradingSink.Reserve: central up → AuditCentral; central down +
  confidential → refuse (no buffer); central down + internal/public +
  buffer writable → AuditBuffered; central down + buffer unwritable →
  floor refuse. Record executes the reserved outcome and, when buffered,
  fires an ntfy alert.
- FileBuffer: durable JSONL buffer that survives process restart; Confirm
  rewrites the file without a record, so a buffered record is cleared ONLY
  after its central write is confirmed.
- LokiCentral: /ready probe + /loki/api/v1/push (full audit entry as the
  structured line). NtfyNotifier: degraded-state alerts; token only in the
  auth header, never logged (regression-tested).
- Reconcile + StartReconcile: replay buffered records to central on
  recovery, confirm-then-clear per record; a failed push keeps the record
  buffered (no loss). SlogSink updated to the two-phase shape (always
  central, never fails) — the default when no loki endpoint is set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathiasandClaude Opus 4.8 202212e8d5 feat(capture): two-phase classification-aware AuditSink port (#54)
Splits the audit port into Reserve (before any write) + Record (after),
so "confidential + sink-down → refuse before any write" is literally true
even though the audit record — which lists what landed — can only be
written afterwards.

- AuditSink.Reserve(ctx, level) → AuditOutcome | error. The error path
  refuses the capture before writing: confidential + central sink down,
  or the all-tiers floor (nothing can record).
- AuditSink.Record(ctx, entry, outcome) persists per the reserved outcome.
- Service: I5 gate runs after the I1 gate and after the dry-run
  short-circuit (dry-run never probes the sink). AuditBuffered surfaces on
  the receipt. New ErrAuditUnavailable sentinel (→ HTTP 503).

The tier→behaviour decision lives in the sink impl (#54's DegradingSink),
not the service — the service just honours Reserve's verdict.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:54:24 +02:00
mathias b7938d4636 Merge pull request 'feat: POST /capture REST adapter + OAuth2 + I1 sovereignty gate (#53, capture 49d)' (#59) from feat/capture-rest-i1-gate into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 21:44:14 +00:00
mathiasandClaude Opus 4.8 a1997838b0 feat(capturehttp): POST /capture REST adapter + OAuth2 + origin resolver (#53)
CI / Lint / Test / Vet (pull_request) Successful in 12s
CI / Mirror to GitHub (pull_request) Has been skipped
The HTTP door for the capture capability. Thin: authenticate → derive
trust-zone origin → decode → capture.Service → map receipt to status.

- Auth mirrors the chassis Bearer precedence (static token wins, then Dex
  JWT) but returns the resolved principal + auth path, which the chassis
  middleware hides — capture needs the principal to derive the origin.
  Depends on a small Validator interface (the chassis *JWTValidator
  satisfies it) so the JWT/origin path is testable without a live JWKS.
- OriginResolver maps principal → trust zone: static-token caller and
  allowlisted JWT subjects → sovereign; every other principal → us-nexus
  (fail safe, so the I1 gate refuses confidential by default). Principal
  and origin are server-set on the input, overwriting any body the caller
  sent.
- HTTP status: 200 all-ok / dry-run, 207 partial, 502 all-failed, 403 on
  the I1 refusal, 400 on fail-closed validation.
- Wired in main behind the same static+JWT credentials as /mcp, reusing
  the MCP server's graph-wired brain store (one implementation) and the
  classification tags (#50). Mounts only when a Gitea tracker is
  configured. Sovereign JWT principals via BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathiasandClaude Opus 4.8 77680c7445 feat(audit): minimal slog AuditSink for capture I5 (#53)
Emits the request-level audit record to structured logs (scraped by the
existing alloy/loki substrate) and surfaces security events at warn
level. Placeholder behind the AuditSink interface — the classification-
aware loki+buffer+reconcile sink (confidential fails closed, internal
degrades) lands in #54 and replaces this without touching callers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathiasandClaude Opus 4.8 d7a842f356 feat(capture): I1 sovereignty gate + server-derived origin (#53)
Adds the trust-zone Origin to CaptureContext and the I1 gate to the
use-case: a confidential effective classification through a us-nexus
origin is refused before ANY write (ErrSovereigntyRefused), and the
refusal is itself audited. A caller-asserted harness label that names a
different zone than the server-derived origin is logged as a security
event — context.Harness is descriptive-only, never a gate input.

The gate triggers only on an explicit ZoneUSNexus, so the unset default
(ZoneUnknown) can never make it fire on caller-controllable input; the
REST adapter always sets a concrete zone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:42:18 +02:00
mathias aad90f2dfe Merge pull request 'feat: Gitea IssueTracker client + inject into brain server (#52, capture 49c)' (#58) from feat/capture-gitea-tracker into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 21:32:02 +00:00
mathias 07fca9ee73 Merge pull request 'feat: CaptureService use-case + BrainStore extraction (#51, capture 49b)' (#57) from feat/capture-service into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Has been cancelled
2026-06-22 21:31:47 +00:00
mathias f6bf9b5f57 Merge pull request 'feat: capture classification taxonomy + per-wing/repo tags (#50, capture 49a)' (#56) from feat/capture-classification into main
CI / Lint / Test / Vet (push) Has been cancelled
CI / Mirror to GitHub (push) Has been cancelled
2026-06-22 21:31:18 +00:00
mathiasandClaude Opus 4.8 6606b38a76 feat(gitea): IssueTracker client + inject into brain server (#52)
Implements the IssueTracker port as a real Gitea REST client (#49c) — the
new outbound dependency the brain server gains for capture.

- CreateIssue / CommentIssue / CloseIssue(+optional closing comment) over
  the Gitea API. Owner is the const "mathias", never caller-supplied, so
  a caller cannot redirect a write to another owner's repo.
- Token read once at construction (BRAIN_GITEA_TOKEN), held in the struct,
  travels only in the Authorization header — never logged or in argv.
  Error messages carry status + truncated body, never the token
  (regression-tested). gitea.New returns nil when URL or token is unset,
  so missing config = tracker disabled via one nil check.
- Injected into the MCP server behind the capture.IssueTracker interface
  via WithIssueTracker (constructor injection, swappable/testable); main
  wires it from BRAIN_GITEA_URL (default https://git.d-ma.be) +
  BRAIN_GITEA_TOKEN. Consumed by the capture use-case in #53.

Tests use httptest transports: create (owner+auth header asserted),
comment, close with/without comment, error path that proves the token
never leaks into an error string.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:27:29 +02:00
mathiasandClaude Opus 4.8 4cfc98de56 refactor(capture): CloseIssue carries a closing comment (#52)
#52's IssueTracker spec is CloseIssue(repo, number, comment). Refine the
#51 port signature to match and have the service pass the ticket body as
the closing comment (empty ⇒ close only). Keeps the close-with-comment
flow first-class rather than forcing two separate ticket items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:27:29 +02:00
mathiasandClaude Opus 4.8 0ac165cca3 feat(brainstore): shared BrainStore impl; re-point MCP handlers (#51)
Extracts the #45 write/update/get logic + the wiki upkeep that must
accompany a write (wing _index rebuild, cross-wing auto-tunnel, graph
re-index) into a single concrete brainstore.Store implementing
capture.BrainStore. The MCP brain_write/brain_update/brain_get handlers
are re-pointed at it, so there is one implementation, not two — the DRY
payoff #51 is named for. capture and MCP now share the exact same brain
write path and read-after-write contract.

The Server gains a *brainstore.Store, constructed in NewServer and given
the graph store in WithGraph. Embedding refresh stays out-of-band
(mtime-driven vectorstore.Sync), unchanged. Existing MCP brain_update/
brain_get/brain_write tests pass unmodified — behaviour and the
{id, path, content_hash} response contract are preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:21:43 +02:00
mathiasandClaude Opus 4.8 43f92e3102 feat(capture): CaptureService use-case + ports + entities (#51)
The Clean-Architecture core of the capture capability (#49b). Pure
orchestration over ports — no HTTP, no live Gitea, no audit I/O — fully
unit-tested against fakes before any adapter exists.

- Ports: BrainStore (#45 write/update/get), IssueTracker, SummaryWriter,
  ClassificationPolicy (satisfied by #50's classification.Config),
  AuditSink. Entities: Insight, Ticket, Summary, CaptureContext,
  CaptureInput, CaptureReceipt.
- CaptureService.Capture: validate-before-write (fail-closed), resolve
  effective classification (stricter of declared vs target-derived;
  under-declaration logged as a security event), orchestrate insights
  (write/supersede) → tickets → summary best-effort, emit a request-level
  audit record of exactly what landed, return a partial-aware receipt.
- dry_run short-circuits after validation, writes nothing (not even audit).

Out of scope here, layered on later: the I1 origin sovereignty gate (#53,
needs the server-derived principal) and the classification-aware audit
degradation/refusal (#54). "Effective" is folded into the service as
classification.Stricter rather than a port method — the stricter-wins
rule is use-case policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 23:21:43 +02:00
mathiasandClaude Opus 4.8 98cfae595c feat(classification): data-sensitivity taxonomy + per-wing/repo tags (#50)
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Implements the I1-prerequisite from #49/#50: the classification taxonomy
and the per-wing / per-repo tagging the capture server reads to derive a
target's sensitivity.

- Levels public < internal < confidential, ordered so "stricter wins"
  (spec §4.1 model C) is a plain max via Stricter().
- Tags read from an optional classification.yaml at the brain root
  (wings:/repos: maps). Absent file → defaults-only, not an error.
- Defaulting: client-* → confidential; hyperguild/homelab → internal;
  everything else → confidential. Fail-safe-to-strictest is the
  load-bearing property: a missing tag never silently downgrades.
- Config.Derive(Target) is the function the use-case calls; Wing/Repo
  are the per-kind helpers. ParseLevel rejects unknown tokens; a bad
  level in the config file is a hard load error.

Central classification.yaml (not _index.md frontmatter, not gitea repo
topics): classifying a repo needs no live Gitea client, so #50 has no
dependency on the tracker work (#52); it's auditable in one place; and
it avoids BuildWingIndex clobbering a wing's regenerated _index.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 22:42:05 +02:00
mathias 2a595b5a92 docs(capture): make Q4 audit-sink-down posture classification-aware (#49)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
Reconsidered Q4: instead of one global degrade-and-warn, the posture now
inherits from effective classification (Q1):
- confidential + audit-sink-down -> hard-refuse (no buffer; removes the
  buffer-integrity question for confidential data)
- internal/public + audit-sink-down -> degrade-and-warn + durable local
  buffer + ntfy + reconcile-on-recovery
- floor (all tiers): refuse if nothing can record the audit
Updated the I5 Gherkin scenarios + obligations row to match. Couples Q4
to the Q1 classification spine -> one coherent sensitivity model.
2026-06-22 20:20:40 +00:00
mathias b7a2cc5fdf docs(capture): resolve §4 open questions into binding decisions (#49)
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Q1 classification trust: model (C) — caller declares, server cross-checks
  target tag, stricter wins, mismatch logged. Needs a classification
  taxonomy + per-wing/repo tags (prerequisite, sub-task of #49).
Q2 harness origin: server-derived from authenticated principal;
  context.harness is descriptive-only, never a gate input.
Q3 central relay: ships in v1 (needed for claude.ai/Crush/Pi/LLM Council)
  + I2 security-baseline ledger entry is v1 work.
Q4 audit-sink-down: degrade-and-warn + durable local buffer + reconcile
  on recovery; refuse only if NOTHING can record the audit.

Updated the I1 sovereignty + I5 auditability Gherkin scenarios to match;
added classification-mismatch and origin-spoofing scenarios.
2026-06-22 20:17:07 +00:00
mathias db638cca11 docs(capture): add use-case + BDD spec for the capture capability (#49)
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 3s
Pre-build spec for the uniform cross-harness capture path. Grounds the
design in the homelab invariants (I1-I5, now canonical on infra main):
- I1 sovereignty gate: refuse confidential capture via us-nexus harness
- I2: distributed-library form (no high-degree observer node) is
  admissible; central relay needs a security-baseline ledger entry
- I5: capture is a privileged write path, must emit audit records
Gherkin scenarios cover the happy path, the sovereignty refusal,
supersession + staleness discipline (#45/#47), fail-closed validation,
best-effort partial-failure receipts, dry-run, and fidelity supersession.
Four open questions flagged for review (classification trust is the
highest-risk one).
2026-06-22 17:25:03 +00:00
mathias 38579598e0 feat(skills): add close-session workflow skill
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Disciplined end-of-session closeout for Claude.ai chats: harvest →
ground-truth gitea → confirm issue actions → commit session summary →
brain orientation note → safe-to-archive verdict.

Finalized against current infra (git.d-ma.be, koala:30401 LiteLLM,
Authentik) and the brain_update/brain_get verbs. Phase 5 uses
supersede-by-slug + brain_get read-after-write with the batch
no-semantic-query-after-supersede discipline. session_log/brain_tunnel
inlined (now callable from claude.ai). Summary frontmatter is the
reduced live-capture schema with fidelity:live-capture to distinguish
from batch-export summaries.
2026-06-22 15:57:02 +00:00
mathias d6fa92b176 Merge pull request 'chore(context): drop unsupported cursor + aider adapters' (#48) from chore/drop-cursor-aider-adapters into main
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 09:03:59 +00:00
mathias 9173f9058d chore(context): remove generated .aider.conventions.md
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
Aider is not a supported harness; the generator no longer emits this.
Note: generator no longer writes .aider.conf.yml either.
2026-06-22 09:01:39 +00:00
mathias 3e84a41fed chore(context): remove generated .cursorrules
Cursor is not a supported harness; the generator no longer emits this.
2026-06-22 09:01:29 +00:00
mathias 0e0571c7da chore(context): stop policing cursor/aider adapters in task check
Drop the context:sync:cursor task and remove .cursorrules +
.aider.conventions.md from the drift-guard file list in `check`, now
that the generator no longer emits them.
2026-06-22 09:01:19 +00:00
mathias 7a27cf71a2 chore(context): drop cursor + aider adapters from generator
Cursor and Aider are not supported harnesses. Remove generate_cursor
and generate_aider, their no-arg calls, and their case arms. The active
adapters are claude (CLAUDE.md), agents (AGENTS.md), and system-prompt
(.context/system-prompt.txt).
2026-06-22 09:00:55 +00:00
mathias 63df6d3283 Merge pull request 'feat: brain_update supersede verb + brain_get / brain_write read-after-write handle (#45)' (#46) from feat/brain-update-supersede into main
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-06-22 08:59:28 +00:00
mathiasandClaude Opus 4.8 f04b03e07e chore(context): re-sync derived adapters after root rule-0 update
CI / Lint / Test / Vet (pull_request) Successful in 13s
CI / Mirror to GitHub (pull_request) Has been skipped
context-sync regenerated the adapters from the updated root AGENT.md
(rule 0 pre-task ritual + TDD constraint). The committed adapters had
drifted; this is the documented `task check` remedy, not a content
change in this repo.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:16 +02:00
mathiasandClaude Opus 4.8 6c61f93146 feat(mcp): register brain_update + brain_get, extend brain_write handle
Wires the #45 verbs into the MCP surface (all three sites: tools()
descriptors, handleCall dispatch, package doc comment).

- brain_update: supersede-by-slug or full path; rebuilds wing _index and
  re-tunnels cross-wing matches against the new body (idempotent,
  best-effort), re-indexes the graph, returns {id, path, content_hash,
  superseded}.
- brain_get: fetch by id or path (both are the brain-relative handle);
  returns {id, path, content_hash, frontmatter, body}.
- brain_write: return contract extended from {path} to {id, path,
  content_hash} — path kept for backward compat — so the create path
  also yields a stable handle.

id == relPath; content_hash == sha256 of the file bytes. Tests cover the
supersede happy path, missing-target error + no-create, get by id/path,
write handle, and an end-to-end re-embed test that drives the real
vectorstore.Sync re-index after an update.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:10 +02:00
mathiasandClaude Opus 4.8 95a69fc2c1 feat(brain): add UpdateNote/ReadNote supersede primitives + frontmatter editor
Implements the api-layer half of #45. UpdateNote supersedes a note in
place (whole-note body replace, frontmatter re-stamp: updated_at,
supersedes=prior content hash, supersede_reason), preserving created_at,
wing, hall, and any custom fields. Never creates — a missing target is
an error so callers fall back to brain_write. ReadNote is the read-after-
write primitive (frontmatter + body + content_hash). ContentHash is the
sha256 handle that round-trips write/update → get.

Frontmatter is edited via a line-preserving ordered editor rather than a
yaml.v3 round-trip, which would reorder keys and strip comments — the
brain writes flat key:value frontmatter by hand.

Embeddings are not refreshed here: the rewritten file's mtime advances,
which the out-of-band vectorstore.Sync ticker uses to re-embed it — the
same mechanism brain_write relies on.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 08:25:00 +02:00
mathiasandClaude Opus 4.8 bb8bc0478c chore(cd): use git.d-ma.be for SSH alias + INFRA_REPO (post-rename consistency)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:19:42 +02:00
mathiasandClaude Opus 4.8 a961a3c064 fix(vectorstore): hard-split oversized heading-less chunks
A doc with no headings and no blank-line paragraphs (JSON-lines, e.g.
wiki/telos/decisions/human-intent-column.md) survived both chunk passes whole
and was sent to nomic-embed over its context window → 'input length exceeds
the context length' (400, the steady embed errors=1). Add a final hard-split
pass (line then UTF-8 rune boundaries) so no chunk exceeds maxBytes. TDD.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 00:19:42 +02:00
mathiasandClaude Opus 4.8 bec28f9014 fix(cd): point registry/patch/verify refs at git.d-ma.be after host rename
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
The gitea.d-ma.be→git.d-ma.be rename moved the infra deployment manifests, but
cd.yml still sed-patched 'gitea.d-ma.be/mathias/ingestion:' — which no longer
matches, so the patch produced no change and 'git commit' failed under set -e,
wedging all deploys (image stuck at e8dbcf6). Repoint the registry image refs,
the infra-patch sed patterns, and the rollout-verify EXPECTED values to
git.d-ma.be (same registry backend). SSH alias + INFRA_REPO left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:22:07 +02:00
mathiasandClaude Opus 4.8 b62ac57382 fix(brain_answer): reranker is a filter, not a gate — fall back to BM25 when it keeps nothing
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 3s
The Qwen3-Reranker is a web-search cross-encoder. Against conversational /
personal-intent queries (e.g. 'what am I optimizing toward?') it scores every
candidate as 'no', so brain_answer collapsed to 'No relevant content found'
even though BM25 had retrieved on-topic notes (incl. the telos wing). Treat
the reranker as a filter: when it keeps zero results, fall back to the
BM25/vector ordering (capped to the no-reranker depth of 10). Refs brain#11.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 22:53:02 +02:00
mathiasandClaude Opus 4.8 aa918388b9 docs(brain): add two-column intent merge scaffold
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Drop-in unified findings doc for the brain-MCP intent study. Locks the
shared row schema + closed intent vocab both columns must conform to.
Agent column filled from agent-intent-column.jsonl (46 acts, 37%
mismatch); human column left as PENDING cells + <<SYNTH>> blocks so the
Claude.ai-history analysis merges in without re-deriving structure.

Pre-seeds the cross-consumer divergence questions: agent mismatch is
write-side-heavy (supersede + verify-landed); hypothesis is human
mismatch is read-side-heavy (semantic + answer) — interface may fail the
two consumers at opposite ends.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 20:45:24 +02:00
mathiasandClaude Opus 4.8 e8dbcf6eef docs(brain): add agent-consumer brain-MCP intent analysis
CI / Lint / Test / Vet (push) Successful in 14s
CI / Mirror to GitHub (push) Successful in 4s
Agent-consumer column of the two-part brain intent↔interface study.
Reconstructs the knowledge-act behind every brain MCP call in the
Claude Code agent transcripts on koala (the only corpus with brain
calls; brain/sessions and agentsquad eval logs carry none).

46 distinct knowledge-acts, 37% interface mismatch, 0% intent_unclear.
Headline gap: no update/supersede verb → agents blind re-write same
slug (5x); no read-after-write → lexical re-query of own note (4x);
lexical-only reads → semantic-as-keyword-stuffing chains (3x);
brain_answer hedged with parallel brain_query.

Canonical schema brain-intent-extraction.md absent on host; vocab
reconstructed, every row tagged schema_source=reconstructed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 19:44:59 +02:00
mathiasandClaude Opus 4.8 0eeb1df4a2 fix(claudewatcher): scrub bare 1Password service-account tokens (ops_)
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
A ~/.zshrc read surfaced OP_SERVICE_ACCOUNT_TOKEN into a transcript. The
_TOKEN= form was already caught by homelab-env-token, but a bare ops_<b64>
value was not. Add an op-service-account rule (ordered early). Tests cover
both env-assigned and bare forms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 00:28:47 +02:00
mathiasandClaude Opus 4.8 9febb1bba1 fix(claudewatcher): harden secret scrubber against \b evasion + add JWT
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
A shell mangle that glued a key to a preceding word ('yes'+'sk-...') had
no word boundary, so the leading \b in the openai-sk rule failed to match
and a LiteLLM master key leaked past the scrubber into ingest (2026-06-11).

- openai-sk: drop leading \b, match sk- shape anywhere ({32,} floor keeps
  short task-/disk- words clean).
- add jwt rule for bare header.payload.sig tokens (no Bearer prefix).
- regression tests: the exact yessk- evasion, standalone sk-, bare JWT,
  plus clean-content guards for task-/disk-.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 08:55:33 +02:00
mathias 5dc247b994 fix(ci): quote "on" key so Gitea parses workflow triggers
CI / Lint / Test / Vet (push) Successful in 17s
CI / Mirror to GitHub (push) Successful in 4s
Bare on: parses as YAML boolean true (Norway problem); Gitea then ignores the triggers and silently skips jobs. Quoting forces the string key.
2026-06-03 08:38:46 +02:00
mathias 2125558196 docs: extend harness boundary decision to cover Crush as third harness
CI / Mirror to GitHub (push) Successful in 4s
CI / Lint / Test / Vet (push) Successful in 12s
2026-05-28 11:38:36 +00:00
mathias 2beaac2feb docs: add 2026-05-28 decisions — harness boundary + field benchmark definition
CI / Lint / Test / Vet (push) Successful in 12s
CI / Mirror to GitHub (push) Successful in 3s
2026-05-28 10:31:41 +00:00
mathias 525811bc1a docs: add hypothesis statement and harness boundary clarification
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-05-28 10:30:39 +00:00
mathias bad0581623 merge: client-name scrubber rule (refs hyperguild#27)
CI / Lint / Test / Vet (push) Successful in 13s
CI / Mirror to GitHub (push) Successful in 4s
2026-05-26 07:10:05 +02:00
mathias a94b860c2e feat(claudewatcher): client-name guard via RegisterRule + env
Pre-rollout guard. Source code stays clean — client identities come
from CLAUDE_INGEST_CLIENT_BLOCK env (sourced from a SOPS-encrypted k8s
secret in infra repo). Env value is a regex alternation; main wraps
it with `(?i)\b(...)\b` so word-boundary matching avoids false hits
inside longer identifiers (e.g. "Sebastian" doesn't trigger on "SEB").

DefaultRules (credential shapes) still take precedence so any leak
that's BOTH a client mention AND a credential shape logs as the
credential — strictly more dangerous, points triage at the right
thing. Tests cover precedence + case variations + word-boundary
respect + invalid-pattern rejection.

Refs: infra#73 Track E.1 pre-rollout grill (option B).

Bump-Type: minor
2026-05-26 07:10:05 +02:00
102 changed files with 7230 additions and 3716 deletions
-315
View File
@@ -1,315 +0,0 @@
# Agent context — Mathias workspace
<!-- Canonical root context for all AI coding agents.
Lives at: ~/dev/.context/AGENT.md
Applies to every project under ~/dev/ unless overridden.
Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
Project-level context in .context/PROJECT.md layers on top of this. -->
## Who I am
I'm Mathias, a digital product manager and technology consultant based in Sweden.
I build software, research emerging tech, and deliver consulting engagements
for clients under NDA. I work across AI/ML, financial automation, web applications,
and climate/sustainability tech.
## How I work with agents
- I think like a product manager — I care about *why* before *how*
- I want agents to be opinionated and push back, not just execute blindly
- I prefer concise responses; skip ceremony and get to the point
- When I say "build this", I mean production-quality with tests, not a demo
- Ask me before making irreversible changes or adding heavy dependencies
- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
## Behavior rules
These rules apply to every task across every project, regardless of harness.
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
Think before coding; if the problem is unclear, ask or state assumptions before acting.
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
files, and formatting alone. Diffs should be small and reviewable.
4. **Goal-driven execution.** Define clear success criteria up front for every task.
Loop — implement, verify, refine — until those criteria are met. Don't claim
completion without evidence (tests pass, command output, observed behavior).
5. **Trunk-Based Development — commit directly to main.** Every commit is one
logical change (one tool, one fix, one test) with passing tests. Main is always
deployable. Never create long-lived feature branches.
**Exception — parallel agents on same repo:** If another agent is known to be
actively working on the same repo simultaneously, create a short-lived branch
(`agent/<description>`), finish the task, and merge to main within the same
session. Do not leave agent branches open between sessions.
**Exception — external contributor or client four-eyes requirement:** Use
PR flow only when a human reviewer outside the project is required. Document
the reason in PROJECT.md.
## Default stack
| Layer | Default | Fallback | Last resort |
|-------|---------|----------|-------------|
| Language | Go | Python | TypeScript, Java, C |
| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
| Build | Task (taskfile.dev) | Make | — |
| Containers | Docker Compose (dev), k3s (prod) | — | — |
| DB | PostgreSQL + sqlc | SQLite | — |
| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
| Logging | slog (structured) | — | — |
| Testing | Table-driven, testify | — | — |
| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
Exploratory: Rust, Zig — I'll tell you when I want these.
## Code conventions
- **Go style**: golines, gofumpt, golangci-lint
- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return
- **Naming**: stdlib conventions, no stuttering
- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
one logical change per commit, CI is the quality gate
- **Never**: long-lived feature branches, PRs for solo work, direct push without
passing `task check` locally first
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
## Infrastructure
Three machines on Tailscale:
| Machine | Role | Key specs |
|---------|------|-----------|
| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
| iguana | Services, builds | M2 Ultra Mac |
| flamingo | Daily driver, edge | Mac mini, ~/dev is here |
- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
- **Orchestration**: k3s cluster across all three machines
- **Networking**: Tailscale mesh
## Project landscape
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
Organized in thematic folders:
| Folder | Focus | Count |
|--------|-------|-------|
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
### Key active projects
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
## Knowledge base — actively use it
A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
reference material — query it actively, not just when explicitly told.**
### When to query (treat as a reflex)
- **Before** starting a non-trivial task — search for prior art with the symptom
AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
- **When debugging** — search for the error string, the stack frame, the affected
service. Past you may have already paid this tax.
- **Before adopting** a pattern, library, framework, or model name — check if it
was tried and rejected, or what the integration footguns are.
- **When making architectural decisions** — search for the domain + "ADR" or
"decision" to find prior reasoning before re-deriving it.
- **When a recommendation feels novel** — challenge yourself: "has this been
documented?" The brain often has it.
### When to write
After you discover something that **future-you would forget** and that **isn't
recoverable from the code, git log, or PR description alone**:
- Bugs whose root cause is non-obvious and generalisable beyond this project.
- Framework / library / model-name quirks that bit you and would bite anyone.
- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
DON'T write project status, sprint progress, PR summaries, or "what I did this
session" — those rot fast and the originals are in git/gitea anyway. Brain
entries that age well are about *why*, *how to avoid*, and *what to do when*.
### How to access (per harness)
| Harness | Query | Write |
|---------|-------|-------|
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://gitea.d-ma.be/mathias/hyperguild``knowledge/` and `wiki/` markdown files |
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
on the koala k3s cluster; don't hardcode local-only model names into the
berget URL (see knowledge entry on namespace mismatches).
### Quick reflex checks
If you find yourself about to say any of these out loud, you owe yourself a brain query first:
- "I think the issue might be..."
- "Let me try X and see..."
- "I'll just write a script to..."
- "This is probably a new bug..."
- "Has anyone done this before?" — *yes, probably, go check.*
## Client work rules
When working on a project tagged with a client name:
1. Never send code, data, or context to cloud APIs — use local models only
2. Never reference other client projects or their data
3. Keep all artifacts within the client's git org / directory
4. Treat everything as confidential unless told otherwise
## Harness-agnostic principles
This context is designed to work with any AI coding tool:
- Claude Code, Cursor, Aider, Open WebUI, Charmbracelet Mods/Crush
- Pi Coding Agent, Mistral Vibe, Antigravity
- Any tool that accepts a system prompt or reads a markdown context file
The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
## How context propagates
Canonical sources of truth:
- Universal: `~/dev/.context/AGENT.md` (this file)
- Project: `<repo>/.context/PROJECT.md` (per-repo)
Derived files (committed, regenerated by `task context:sync`):
- `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.aider.conventions.md`,
`.context/system-prompt.txt`
Workflow:
1. Edit a canonical file. Run `task context:sync`. Commit canonical and
derived together. Push.
2. On any other host, `git pull` brings both. Claude Code (tree-walking)
uses `CLAUDE.md`; Crush / Pi / Antigravity (cwd-only) use `AGENTS.md`;
Cursor uses `.cursorrules`; Aider uses `.aider.conventions.md`.
3. `task check` runs `context:sync` then asserts `git status --porcelain`
is empty over the derived files (catches both modified-tracked drift
and missing-untracked adapters). A drift fails the check with a
message telling you to stage the regenerated files.
Behavior rules in this file and per-project rules in `PROJECT.md` apply
unconditionally on every host, every harness.
## Engineering Skills
Shared engineering skills are available in `~/dev/.skills/`. Load on demand via the index.
See `~/dev/.skills/SKILLS_INDEX.md` for the full list with descriptions and "use when" triggers.
Key skills:
- **TDD**: always write tests first — load `tdd` skill
- **Code Review**: load `code-review` skill before any review
- **SOLID/Clean Code**: load `solid` or `clean-code` skill for design work
- **Problem first**: load `problem-analysis` skill before coding non-trivial features
---
# Project context
<!-- Canonical project context. Edit this, run `task context:sync`.
Root agent context from ~/dev/.context/AGENT.md is automatically
prepended for harnesses that don't walk the directory tree. -->
## Identity
- **Name**: supervisor
- **Owner**: Mathias
- **Client**: personal
- **Repo**:
- **Status**: active
## Stack
- **Primary language**: Go
- **UI layer**: HTMX + Templ (when applicable)
- **Fallback languages**: Python, TypeScript (justify in PR if used)
- **Build**: Task (taskfile.dev), not Make
- **Containers**: Docker (compose for dev, k3s for deploy)
- **Target infra**: koala (GPU workloads), iguana (services), flamingo (edge)
## Conventions
### Code style
- Go: follow `golines`, `gofumpt`, `golangci-lint` with project config
- Tests: table-driven, in `_test.go` next to source, `testify` for assertions
- Errors: wrap with `fmt.Errorf("operation: %w", err)`, no naked returns
- Naming: stdlib conventions, no stuttering (`http.Client` not `http.HTTPClient`)
### Architecture preferences
- Prefer standard library over frameworks (net/http over gin/echo)
- Dependency injection via constructor functions, not containers
- Configuration via environment variables, parsed at startup into a typed struct
- Structured logging via `slog`
### Git
- Conventional commits: `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`
- Branch naming: `feat/short-description`, `fix/short-description`
- PRs: one concern per PR, description explains *why* not *what*
### Security
- No secrets in code, ever — use env vars or SOPS-encrypted files
- Client data never leaves local network unless explicitly cleared
- Dependencies: audit with `govulncheck` before adding
## MCP endpoints
Two MCP servers are live, both reachable over Tailscale and via HTTPS domain:
- **`brain`** at `https://brain-mcp.d-ma.be/mcp` (NodePort `koala:30330`) —
`brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`,
`brain_answer`, `brain_classify`, `session_log`. Hosted by the ingestion
service. Auth: Dex JWT (claude.ai OAuth) or static `BRAIN_MCP_TOKEN`.
- **`routing`** at `http://koala:30310/mcp` — Mode 2 routing pod. Advertises
`review`, `debug`, `retrospective`, `trainer`; per-call routes to local model
or Claude based on brain `/pass-rate`. Bearer auth via `ROUTING_MCP_TOKEN`
(opt-in). Only `mode client-local` registers this endpoint.
The supervisor MCP (`koala:30320`) was retired in Plan 7 (2026-05-12). Its
skill workers (`tdd`, `spec`) are now SKILL.md files; routed skills moved to
the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
and tags. Both require `BRAIN_LLM_PRIMARY_URL` to be set in the ingestion pod.
## Agent instructions
When acting as a coding agent on this project:
1. Read this file and all `SKILL.md` files in `.skills/` before starting work
2. Run `task check` before committing (lint + test + vet)
3. If unsure about a convention, check `DECISIONS.md` or ask
4. Never modify files outside the project root without explicit permission
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
+54 -8
View File
@@ -32,6 +32,14 @@ and climate/sustainability tech.
These rules apply to every task across every project, regardless of harness.
0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
- **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
- **Load the relevant skill** — see trigger table in *Engineering Skills* below.
- **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
- **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
**TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
Think before coding; if the problem is unclear, ask or state assumptions before acting.
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
@@ -54,6 +62,22 @@ These rules apply to every task across every project, regardless of harness.
PR flow only when a human reviewer outside the project is required. Document
the reason in PROJECT.md.
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
the code is not the end of the task; capturing it is. Run this unprompted:
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
actual last tag — stated versions in docs drift stale.
- **Push** main and the tag (CI is the gate).
- **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
the reusable patterns and the footguns that would bite anyone again, never
project status. See *Knowledge base — when to write* below.
- **File discovered-but-deferred work as tracker issues** on the project's own
repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
"out of scope, recorded" rot in a commit message; make it a ticket with a
source pointer.
- Surface the brain entries and issue numbers in the closing summary so the
trail is auditable.
## Default stack
| Layer | Default | Fallback | Last resort |
@@ -83,6 +107,26 @@ Exploratory: Rust, Zig — I'll tell you when I want these.
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
## Secret handling (every harness, every command)
Tool output is persisted: terminal → `~/.claude/projects` transcripts →
claudewatcher → brain/wiki → gitea history. A secret printed once is
searchable forever, and clearing it means rotating the key. So:
1. **Never print, echo, log, or transform a secret to inspect it.** No
`base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
to defeat `op run`'s output masking (it masks raw values; base64 hides them
from the mask — that exact trick leaked a key on 2026-06-11).
2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
*inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
in a command's argv (it lands in the tool call and the transcript).
3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set` —
never `${X:-...}` (returns the value when set) and never echo a substring of it.
4. **Cross-host secrets:** run the secret-consuming command on the host that has
the secret; do not forward a raw key over ssh argv/stdout.
5. If a secret does leak into output, say so immediately and flag it for rotation —
don't bury it.
## Infrastructure
Three machines on Tailscale:
@@ -162,7 +206,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://gitea.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
@@ -224,15 +268,17 @@ unconditionally on every host, every harness.
## Engineering Skills
Shared engineering skills are available in `~/dev/.skills/`. Load on demand via the index.
Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Clone it to `~/dev/skills/` and run `SKILLS_CHECKOUT_DIR="$PWD" bash install.sh` there to wire every skill into your harnesses (Claude Code, Crush, Antigravity, Mistral Vibe) as native, on-demand skills. (Use `install.sh`, not `task install` — the latter is currently broken, skills#7.) Load at task start — not "on demand" but on schedule, before writing code. Browse `~/dev/skills/SKILLS_INDEX.md` for the full list.
See `~/dev/.skills/SKILLS_INDEX.md` for the full list with descriptions and "use when" triggers.
**Skill trigger table — load before starting, not after getting stuck:**
Key skills:
- **TDD**: always write tests first — load `tdd` skill
- **Code Review**: load `code-review` skill before any review
- **SOLID/Clean Code**: load `solid` or `clean-code` skill for design work
- **Problem first**: load `problem-analysis` skill before coding non-trivial features
| Task type | Load |
|-----------|------|
| Any feature or bug fix | `tdd` |
| Refactor or design | `clean-code` or `solid` |
| Debug | `problem-analysis` |
| Review code or PRs | `code-review` |
| Frame a problem before coding | `problem-analysis` |
---
-318
View File
@@ -1,318 +0,0 @@
# Cursor rules — auto-generated
# Do not edit. Run: task context:sync
# Agent context — Mathias workspace
<!-- Canonical root context for all AI coding agents.
Lives at: ~/dev/.context/AGENT.md
Applies to every project under ~/dev/ unless overridden.
Run `task context:sync` from ~/dev/ to regenerate harness-specific files.
Project-level context in .context/PROJECT.md layers on top of this. -->
## Who I am
I'm Mathias, a digital product manager and technology consultant based in Sweden.
I build software, research emerging tech, and deliver consulting engagements
for clients under NDA. I work across AI/ML, financial automation, web applications,
and climate/sustainability tech.
## How I work with agents
- I think like a product manager — I care about *why* before *how*
- I want agents to be opinionated and push back, not just execute blindly
- I prefer concise responses; skip ceremony and get to the point
- When I say "build this", I mean production-quality with tests, not a demo
- Ask me before making irreversible changes or adding heavy dependencies
- I work with confidential client data — never send it to cloud APIs unless I explicitly say it's OK
## Behavior rules
These rules apply to every task across every project, regardless of harness.
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
Think before coding; if the problem is unclear, ask or state assumptions before acting.
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
speculative, no "while we're here" cleanups, no premature abstractions. Simplicity first.
3. **Surgical changes.** Touch only what the task requires. Leave unrelated code,
files, and formatting alone. Diffs should be small and reviewable.
4. **Goal-driven execution.** Define clear success criteria up front for every task.
Loop — implement, verify, refine — until those criteria are met. Don't claim
completion without evidence (tests pass, command output, observed behavior).
5. **Trunk-Based Development — commit directly to main.** Every commit is one
logical change (one tool, one fix, one test) with passing tests. Main is always
deployable. Never create long-lived feature branches.
**Exception — parallel agents on same repo:** If another agent is known to be
actively working on the same repo simultaneously, create a short-lived branch
(`agent/<description>`), finish the task, and merge to main within the same
session. Do not leave agent branches open between sessions.
**Exception — external contributor or client four-eyes requirement:** Use
PR flow only when a human reviewer outside the project is required. Document
the reason in PROJECT.md.
## Default stack
| Layer | Default | Fallback | Last resort |
|-------|---------|----------|-------------|
| Language | Go | Python | TypeScript, Java, C |
| UI | HTMX + Templ | Server-rendered HTML | React (only if SPA is justified) |
| Build | Task (taskfile.dev) | Make | — |
| Containers | Docker Compose (dev), k3s (prod) | — | — |
| DB | PostgreSQL + sqlc | SQLite | — |
| Search | pgvector (vector), BM25 | Qdrant (when >1M vectors or hybrid retrieval) | — |
| Logging | slog (structured) | — | — |
| Testing | Table-driven, testify | — | — |
| Agents (Go) | google.golang.org/adk + pkg/litellm adapter | — | — |
Exploratory: Rust, Zig — I'll tell you when I want these.
## Code conventions
- **Go style**: golines, gofumpt, golangci-lint
- **Errors**: `fmt.Errorf("operation: %w", err)` — never naked, never log-and-return
- **Naming**: stdlib conventions, no stuttering
- **Architecture**: prefer stdlib over frameworks, constructor injection, env-var config parsed into typed structs
- **Git**: conventional commits (`feat:`, `fix:`, `chore:`), commit directly to main,
one logical change per commit, CI is the quality gate
- **Never**: long-lived feature branches, PRs for solo work, direct push without
passing `task check` locally first
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
## Infrastructure
Three machines on Tailscale:
| Machine | Role | Key specs |
|---------|------|-----------|
| koala | GPU inference, heavy compute | RTX 5070, runs k3s + llama-swap + shared postgres18/pgvector |
| iguana | Services, builds | M2 Ultra Mac |
| flamingo | Daily driver, edge | Mac mini, ~/dev is here |
- **Model routing**: LiteLLM in front of llama-swap (local) + cloud APIs (when permitted)
- **Orchestration**: k3s cluster across all three machines
- **Networking**: Tailscale mesh
## Project landscape
All development repos live at `~/dev/` (softlink from `~/Documents/local-dev/`).
Organized in thematic folders:
| Folder | Focus | Count |
|--------|-------|-------|
| `GO/` | Go web frameworks, API integrations, learning projects | ~10 |
| `AI/` | ML research, AI frameworks (FinRL, DSPy, crawl4ai) | ~6 |
| `AGENTS/` | Autonomous agents, coding agents, MCP servers, infra | ~15 |
| `QKX/` | Invoice processing, financial automation, payment systems | ~13 |
| `XT/` | Climate data, sustainability (Klimatkollen, Garbo) | ~2 |
See `~/dev/PROJECT_SUMMARY.md` for detailed descriptions of each project.
### Key active projects
- **super-koala** (`AGENTS/`) — multi-component agent stack with LangGraph, DSPy, MCP
- **azure-tiger** (`QKX/`) — invoice extraction → ISO 20022 payment instructions
- **gocrwl** (`AGENTS/`) — Go web crawler with containerized deployment
- **koala-ai-stack** (`AGENTS/`) — local AI server infrastructure management
- **klimatkollen** (`XT/`) — Swedish municipal climate data platform
## Knowledge base — actively use it
A persistent brain (BM25 search + LLM-synthesised Q&A) survives across sessions,
hosts, and harnesses. It holds 100+ hard-won entries: infra incident postmortems,
Go pitfalls, framework gotchas, design principles, ADRs. **It is not optional
reference material — query it actively, not just when explicitly told.**
### When to query (treat as a reflex)
- **Before** starting a non-trivial task — search for prior art with the symptom
AND the system component ("how did we solve X in Y?"). 5 seconds beats 5 hours.
- **When debugging** — search for the error string, the stack frame, the affected
service. Past you may have already paid this tax.
- **Before adopting** a pattern, library, framework, or model name — check if it
was tried and rejected, or what the integration footguns are.
- **When making architectural decisions** — search for the domain + "ADR" or
"decision" to find prior reasoning before re-deriving it.
- **When a recommendation feels novel** — challenge yourself: "has this been
documented?" The brain often has it.
### When to write
After you discover something that **future-you would forget** and that **isn't
recoverable from the code, git log, or PR description alone**:
- Bugs whose root cause is non-obvious and generalisable beyond this project.
- Framework / library / model-name quirks that bit you and would bite anyone.
- Design principles validated under fire (e.g. "every `_get` needs a `_list`").
- Postmortems for incidents: what broke, why, how diagnosed, what to do next time.
DON'T write project status, sprint progress, PR summaries, or "what I did this
session" — those rot fast and the originals are in git/gitea anyway. Brain
entries that age well are about *why*, *how to avoid*, and *what to do when*.
### How to access (per harness)
| Harness | Query | Write |
|---------|-------|-------|
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://gitea.d-ma.be/mathias/hyperguild` → `knowledge/` and `wiki/` markdown files |
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
fallback. Both are configurable in the `supervisor/ingestion-deployment.yaml`
on the koala k3s cluster; don't hardcode local-only model names into the
berget URL (see knowledge entry on namespace mismatches).
### Quick reflex checks
If you find yourself about to say any of these out loud, you owe yourself a brain query first:
- "I think the issue might be..."
- "Let me try X and see..."
- "I'll just write a script to..."
- "This is probably a new bug..."
- "Has anyone done this before?" — *yes, probably, go check.*
## Client work rules
When working on a project tagged with a client name:
1. Never send code, data, or context to cloud APIs — use local models only
2. Never reference other client projects or their data
3. Keep all artifacts within the client's git org / directory
4. Treat everything as confidential unless told otherwise
## Harness-agnostic principles
This context is designed to work with any AI coding tool:
- Claude Code, Cursor, Aider, Open WebUI, Charmbracelet Mods/Crush
- Pi Coding Agent, Mistral Vibe, Antigravity
- Any tool that accepts a system prompt or reads a markdown context file
The canonical source is always `.context/AGENT.md` (root) and `.context/PROJECT.md` (per-project).
Derived files are committed (see *How context propagates* below) so a `git pull` on any host yields full agent context with no setup.
## How context propagates
Canonical sources of truth:
- Universal: `~/dev/.context/AGENT.md` (this file)
- Project: `<repo>/.context/PROJECT.md` (per-repo)
Derived files (committed, regenerated by `task context:sync`):
- `CLAUDE.md`, `AGENTS.md`, `.cursorrules`, `.aider.conventions.md`,
`.context/system-prompt.txt`
Workflow:
1. Edit a canonical file. Run `task context:sync`. Commit canonical and
derived together. Push.
2. On any other host, `git pull` brings both. Claude Code (tree-walking)
uses `CLAUDE.md`; Crush / Pi / Antigravity (cwd-only) use `AGENTS.md`;
Cursor uses `.cursorrules`; Aider uses `.aider.conventions.md`.
3. `task check` runs `context:sync` then asserts `git status --porcelain`
is empty over the derived files (catches both modified-tracked drift
and missing-untracked adapters). A drift fails the check with a
message telling you to stage the regenerated files.
Behavior rules in this file and per-project rules in `PROJECT.md` apply
unconditionally on every host, every harness.
## Engineering Skills
Shared engineering skills are available in `~/dev/.skills/`. Load on demand via the index.
See `~/dev/.skills/SKILLS_INDEX.md` for the full list with descriptions and "use when" triggers.
Key skills:
- **TDD**: always write tests first — load `tdd` skill
- **Code Review**: load `code-review` skill before any review
- **SOLID/Clean Code**: load `solid` or `clean-code` skill for design work
- **Problem first**: load `problem-analysis` skill before coding non-trivial features
---
# Project context
<!-- Canonical project context. Edit this, run `task context:sync`.
Root agent context from ~/dev/.context/AGENT.md is automatically
prepended for harnesses that don't walk the directory tree. -->
## Identity
- **Name**: supervisor
- **Owner**: Mathias
- **Client**: personal
- **Repo**:
- **Status**: active
## Stack
- **Primary language**: Go
- **UI layer**: HTMX + Templ (when applicable)
- **Fallback languages**: Python, TypeScript (justify in PR if used)
- **Build**: Task (taskfile.dev), not Make
- **Containers**: Docker (compose for dev, k3s for deploy)
- **Target infra**: koala (GPU workloads), iguana (services), flamingo (edge)
## Conventions
### Code style
- Go: follow `golines`, `gofumpt`, `golangci-lint` with project config
- Tests: table-driven, in `_test.go` next to source, `testify` for assertions
- Errors: wrap with `fmt.Errorf("operation: %w", err)`, no naked returns
- Naming: stdlib conventions, no stuttering (`http.Client` not `http.HTTPClient`)
### Architecture preferences
- Prefer standard library over frameworks (net/http over gin/echo)
- Dependency injection via constructor functions, not containers
- Configuration via environment variables, parsed at startup into a typed struct
- Structured logging via `slog`
### Git
- Conventional commits: `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`
- Branch naming: `feat/short-description`, `fix/short-description`
- PRs: one concern per PR, description explains *why* not *what*
### Security
- No secrets in code, ever — use env vars or SOPS-encrypted files
- Client data never leaves local network unless explicitly cleared
- Dependencies: audit with `govulncheck` before adding
## MCP endpoints
Two MCP servers are live, both reachable over Tailscale and via HTTPS domain:
- **`brain`** at `https://brain-mcp.d-ma.be/mcp` (NodePort `koala:30330`) —
`brain_query`, `brain_write`, `brain_ingest`, `brain_ingest_raw`,
`brain_answer`, `brain_classify`, `session_log`. Hosted by the ingestion
service. Auth: Dex JWT (claude.ai OAuth) or static `BRAIN_MCP_TOKEN`.
- **`routing`** at `http://koala:30310/mcp` — Mode 2 routing pod. Advertises
`review`, `debug`, `retrospective`, `trainer`; per-call routes to local model
or Claude based on brain `/pass-rate`. Bearer auth via `ROUTING_MCP_TOKEN`
(opt-in). Only `mode client-local` registers this endpoint.
The supervisor MCP (`koala:30320`) was retired in Plan 7 (2026-05-12). Its
skill workers (`tdd`, `spec`) are now SKILL.md files; routed skills moved to
the routing pod; brain tools moved to the brain MCP.
The brain HTTP REST API (`/query`, `/write`, `/ingest`, `/ingest-raw`,
`/ingest-path`, `/backfill-refs`, `/pass-rate`) remains available on port 3300
for shell scripts and non-MCP clients.
`brain_answer(query)` performs BM25 retrieval + LLM synthesis (berget.ai
gemma4:31b → iguana fallback). `brain_classify(text)` infers doc type, title,
and tags. Both require `BRAIN_LLM_PRIMARY_URL` to be set in the ingestion pod.
## Agent instructions
When acting as a coding agent on this project:
1. Read this file and all `SKILL.md` files in `.skills/` before starting work
2. Run `task check` before committing (lint + test + vet)
3. If unsure about a convention, check `DECISIONS.md` or ask
4. Never modify files outside the project root without explicit permission
5. When adding a dependency, explain why in the commit message
6. For client projects: never send code or context to cloud APIs — use local models via LiteLLM
+9 -68
View File
@@ -1,6 +1,6 @@
name: cd
on:
"on":
workflow_run:
workflows: ["CI"]
types: [completed]
@@ -13,9 +13,8 @@ jobs:
if: ${{ github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'push' }}
environment: staging
env:
INGESTION_IMAGE: gitea.d-ma.be/mathias/ingestion
ROUTING_IMAGE: gitea.d-ma.be/mathias/routing
INFRA_REPO: git@gitea.d-ma.be:mathias/infra.git
INGESTION_IMAGE: git.d-ma.be/mathias/ingestion
INFRA_REPO: git@git.d-ma.be:mathias/infra.git
BUILDKIT_HOST: unix:///run/buildkit/buildkitd.sock
steps:
- name: Checkout
@@ -41,28 +40,6 @@ jobs:
echo "Built and pushed ${INGESTION_IMAGE}:${IMAGE_TAG}"
- name: Build and push routing image
run: |
set -e
trap 'rm -f /tmp/routing-image.tar' EXIT
IMAGE_TAG="${{ github.sha }}"
echo "Building ${ROUTING_IMAGE}:${IMAGE_TAG}"
buildctl --addr "${BUILDKIT_HOST}" build \
--frontend dockerfile.v0 \
--local context=. \
--local dockerfile=. \
--opt filename=Dockerfile.routing \
--opt build-arg:VERSION="${IMAGE_TAG}" \
--output type=oci,dest=/tmp/routing-image.tar
skopeo copy \
oci-archive:/tmp/routing-image.tar \
docker://${ROUTING_IMAGE}:${IMAGE_TAG} \
--dest-creds "${{ secrets.REGISTRY_CREDS }}"
echo "Built and pushed ${ROUTING_IMAGE}:${IMAGE_TAG}"
- name: Update infra repo
run: |
set -e
@@ -71,28 +48,24 @@ jobs:
mkdir -p ~/.ssh
echo "${{ secrets.INFRA_DEPLOY_KEY }}" > ~/.ssh/infra_deploy_key
chmod 600 ~/.ssh/infra_deploy_key
printf 'Host gitea.d-ma.be\n HostName 127.0.0.1\n Port 30022\n StrictHostKeyChecking no\n' >> ~/.ssh/config
printf 'Host git.d-ma.be\n HostName 127.0.0.1\n Port 30022\n StrictHostKeyChecking no\n' >> ~/.ssh/config
GIT_SSH_COMMAND="ssh -i ~/.ssh/infra_deploy_key -o IdentitiesOnly=yes" \
git clone "${INFRA_REPO}" /tmp/infra-update
cd /tmp/infra-update
sed -i "s|gitea.d-ma.be/mathias/ingestion:.*|gitea.d-ma.be/mathias/ingestion:${IMAGE_TAG}|" \
sed -i "s|git.d-ma.be/mathias/ingestion:.*|git.d-ma.be/mathias/ingestion:${IMAGE_TAG}|" \
"k3s/apps/supervisor/ingestion-deployment.yaml"
sed -i "s|gitea.d-ma.be/mathias/routing:.*|gitea.d-ma.be/mathias/routing:${IMAGE_TAG}|" \
"k3s/apps/routing/deployment.yaml"
git config user.email "cd-bot@d-ma.be"
git config user.name "CD Bot"
git add "k3s/apps/supervisor/ingestion-deployment.yaml" \
"k3s/apps/routing/deployment.yaml"
git commit -m "chore(deploy): ingestion+routing → ${IMAGE_TAG}"
git add "k3s/apps/supervisor/ingestion-deployment.yaml"
git commit -m "chore(deploy): ingestion → ${IMAGE_TAG}"
GIT_SSH_COMMAND="ssh -i ~/.ssh/infra_deploy_key -o IdentitiesOnly=yes" \
git push
echo "Infra repo updated: ingestion+routing → ${IMAGE_TAG}"
echo "Infra repo updated: ingestion → ${IMAGE_TAG}"
- name: Trigger Flux reconcile (immediate)
run: |
@@ -103,7 +76,7 @@ jobs:
- name: Wait for Flux to apply new ingestion image
run: |
EXPECTED="gitea.d-ma.be/mathias/ingestion:${{ github.sha }}"
EXPECTED="git.d-ma.be/mathias/ingestion:${{ github.sha }}"
for i in $(seq 1 60); do
CURRENT=$(kubectl get deploy ingestion -n supervisor \
-o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null || echo "")
@@ -132,35 +105,3 @@ jobs:
kubectl describe pods -n supervisor -l app=ingestion | tail -40
exit 1
}
- name: Wait for Flux to apply new routing image
run: |
EXPECTED="gitea.d-ma.be/mathias/routing:${{ github.sha }}"
for i in $(seq 1 60); do
CURRENT=$(kubectl get deploy routing -n routing \
-o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null || echo "")
if [ "$CURRENT" = "$EXPECTED" ]; then
echo "✓ Flux applied routing image after ${i}s"
break
fi
sleep 1
done
kubectl get deploy routing -n routing \
-o jsonpath='{.spec.template.spec.containers[0].image}' \
| grep -qx "$EXPECTED" \
|| { echo "✗ Flux did not apply routing image within 60s"; exit 1; }
- name: Verify routing rollout
run: |
kubectl rollout status deployment/routing \
--namespace routing \
--timeout=120s \
|| {
echo "── pod status ──"
kubectl get pods -n routing -o wide
echo "── events ──"
kubectl get events -n routing --sort-by='.lastTimestamp' | tail -20
echo "── describe ──"
kubectl describe pods -n routing -l app=routing | tail -40
exit 1
}
+1 -1
View File
@@ -1,6 +1,6 @@
name: CI
on:
"on":
push:
branches: [main]
tags: ["v*"]
+7
View File
@@ -6,6 +6,13 @@
"headers": {
"Authorization": "Bearer ${BRAIN_MCP_TOKEN}"
}
},
"gitea": {
"type": "http",
"url": "https://git-mcp.d-ma.be/mcp",
"headers": {
"Authorization": "Bearer ${GITEA_MCP_TOKEN}"
}
}
}
}
+54 -8
View File
@@ -27,6 +27,14 @@ and climate/sustainability tech.
These rules apply to every task across every project, regardless of harness.
0. **Pre-task ritual — before ANY implementation (non-negotiable).** Run this before writing a single line:
- **Query the brain** (`brain_query`) for the domain + symptom. If the result changes your approach, surface it before acting. 5 seconds beats 5 hours.
- **Load the relevant skill** — see trigger table in *Engineering Skills* below.
- **Write the failing test first.** Name the test before the function. If the target is untestable (e.g. `main()` wiring), extract the logic into a testable function first. No implementation without a red test.
- **State the observable success criterion** — what specific behavior, output, or passing test proves this is done?
**TDD is non-negotiable.** "Tests pass" is not proof of correctness — only proof the tests ran. Write tests that would catch the bug before writing code that fixes it.
1. **No assumptions.** Don't hide confusion — surface it. Surface tradeoffs explicitly.
Think before coding; if the problem is unclear, ask or state assumptions before acting.
2. **Minimum viable code.** Solve with the smallest change that works. Nothing
@@ -49,6 +57,22 @@ These rules apply to every task across every project, regardless of harness.
PR flow only when a human reviewer outside the project is required. Document
the reason in PROJECT.md.
6. **Close the loop — every substantive task ends with the same ritual.** Shipping
the code is not the end of the task; capturing it is. Run this unprompted:
- **Tag + bump SemVer** on the change (annotated tag; minor for a feature or
new/changed ADR, patch for a fix; docs in the same commit). Check the repo's
actual last tag — stated versions in docs drift stale.
- **Push** main and the tag (CI is the gate).
- **Persist generalizable learnings to the brain** (`brain_write`, wing/hall) —
the reusable patterns and the footguns that would bite anyone again, never
project status. See *Knowledge base — when to write* below.
- **File discovered-but-deferred work as tracker issues** on the project's own
repo — token-budget gaps, recorded ADR limitations, v2 follow-ups. Don't let
"out of scope, recorded" rot in a commit message; make it a ticket with a
source pointer.
- Surface the brain entries and issue numbers in the closing summary so the
trail is auditable.
## Default stack
| Layer | Default | Fallback | Last resort |
@@ -78,6 +102,26 @@ Exploratory: Rust, Zig — I'll tell you when I want these.
- **Security**: no secrets in code, govulncheck before adding deps, SOPS for encrypted config
- **Dependencies**: prefer stdlib. testify, slog, templ, sqlc, google.golang.org/adk (agent projects only) are pre-approved; anything else needs justification in the commit message
## Secret handling (every harness, every command)
Tool output is persisted: terminal → `~/.claude/projects` transcripts →
claudewatcher → brain/wiki → gitea history. A secret printed once is
searchable forever, and clearing it means rotating the key. So:
1. **Never print, echo, log, or transform a secret to inspect it.** No
`base64`/`xxd`/`cat` of a key, and never pipe a secret through a transform
to defeat `op run`'s output masking (it masks raw values; base64 hides them
from the mask — that exact trick leaked a key on 2026-06-11).
2. **Secrets stay in the subprocess.** Reference them only as env vars consumed
*inside* `op run --env-file ~/.op-env -- <cmd>`. Never place a literal secret
in a command's argv (it lands in the tool call and the transcript).
3. **Existence check without revealing the value:** `[ -n "$X" ] && echo set`
never `${X:-...}` (returns the value when set) and never echo a substring of it.
4. **Cross-host secrets:** run the secret-consuming command on the host that has
the secret; do not forward a raw key over ssh argv/stdout.
5. If a secret does leak into output, say so immediately and flag it for rotation —
don't bury it.
## Infrastructure
Three machines on Tailscale:
@@ -157,7 +201,7 @@ entries that age well are about *why*, *how to avoid*, and *what to do when*.
| **Claude Code, Claude Desktop** | `brain_query` (BM25), `brain_answer` (LLM-synth + sources) MCP tools | `brain_write` MCP tool |
| **Crush, Pi, Antigravity, other MCP-capable** | same MCP server: `ingestion-brain` (via the `mcp__*_brain__*` namespace once authenticated) | same |
| **Anything HTTP-only (curl, scripts)** | `POST https://brain-mcp.d-ma.be/query` with `{"query":"..."}` (auth via `BRAIN_MCP_TOKEN`) | `POST .../write` with `{"content":"...","filename":"..."}` |
| **Browser / human inspection** | `https://gitea.d-ma.be/mathias/hyperguild``knowledge/` and `wiki/` markdown files |
| **Browser / human inspection** | `https://git.d-ma.be/mathias/hyperguild``knowledge/` and `wiki/` markdown files |
- **Scoping**: defaults to `public` collection; client projects filter to `{client}` + `public`.
- **Routing**: brain_answer's LLM uses berget.ai as primary, iguana ollama as
@@ -219,15 +263,17 @@ unconditionally on every host, every harness.
## Engineering Skills
Shared engineering skills are available in `~/dev/.skills/`. Load on demand via the index.
Shared engineering skills live in the **`mathias/skills`** repo (`git.d-ma.be/mathias/skills`). Clone it to `~/dev/skills/` and run `SKILLS_CHECKOUT_DIR="$PWD" bash install.sh` there to wire every skill into your harnesses (Claude Code, Crush, Antigravity, Mistral Vibe) as native, on-demand skills. (Use `install.sh`, not `task install` — the latter is currently broken, skills#7.) Load at task start — not "on demand" but on schedule, before writing code. Browse `~/dev/skills/SKILLS_INDEX.md` for the full list.
See `~/dev/.skills/SKILLS_INDEX.md` for the full list with descriptions and "use when" triggers.
**Skill trigger table — load before starting, not after getting stuck:**
Key skills:
- **TDD**: always write tests first — load `tdd` skill
- **Code Review**: load `code-review` skill before any review
- **SOLID/Clean Code**: load `solid` or `clean-code` skill for design work
- **Problem first**: load `problem-analysis` skill before coding non-trivial features
| Task type | Load |
|-----------|------|
| Any feature or bug fix | `tdd` |
| Refactor or design | `clean-code` or `solid` |
| Debug | `problem-analysis` |
| Review code or PRs | `code-review` |
| Frame a problem before coding | `problem-analysis` |
---
+68
View File
@@ -4,6 +4,74 @@ Record *why* things are the way they are. Future-you will thank present-you.
---
## 2026-05-28 — three active harnesses: hyperguild, agentsquad, Crush (extends earlier boundary decision)
**Context:** After wiring Crush to LiteLLM in May 2026, there are now three active harnesses.
The earlier boundary decision only covered hyperguild vs agentsquad. Crush's role was undefined.
**Decision:** Three harnesses, three distinct roles, shared skills layer.
| Harness | Engine | Primary use | Brain MCP? | Routing pod? | Skills? |
|---------|--------|-------------|------------|--------------|---------|
| **hyperguild** | Claude Code + MCP | Disciplined solo coding sessions, TDD/review/debug workflows | Yes | Yes | Yes (SKILL.md) |
| **agentsquad** | OpenCode + LiteLLM | Multi-agent task execution, executor/reviewer pipelines | No | No (own routing) | Yes (SKILL.md) |
| **Crush** | Charmbracelet TUI + LiteLLM | Interactive local coding, quick iterations on flamingo | No (not yet) | No (direct LiteLLM) | Yes (SKILL.md) |
**Crush specifics (as of 2026-05-28):**
- Config: `~/.config/crush/crush.json` on flamingo (see brain: `homelab/facts/crush-litellm-wiring-2026-05`)
- Connects directly to LiteLLM at `http://koala:4000/v1/` using `sk-local-123`
- Auth type: `openai-compat` (not `openai`)
- Does NOT go through the routing pod — model selection is manual in the Crush UI
- Brain MCP not wired — Crush has no MCP client capability today; revisit if Crush adds MCP support
**Shared across all three:**
- `mathias/skills` — any SKILL.md file works in all three harnesses
- LiteLLM proxy on koala (`http://koala:4000/v1/`) — Crush and agentsquad both route through it; hyperguild does too for local model calls
**Consequences:** No consolidation needed. crush.json must be kept in sync when litellm_config.yaml model names change. The `crush.json` canonical location is `~/.config/crush/crush.json` on flamingo — not yet tracked in a dotfiles repo (track as tech debt).
---
## 2026-05-28 — "field benchmark" for local models = pass-rate at scale (supersedes GOTTH eval suite)
**Context:** The GOTTH eval suite (45 offline prompts across 5 categories) was replaced by
a "field benchmark" in May 2026, but the replacement was never defined concretely.
**Decision:** The field benchmark is per-skill pass rate over real routing pod usage,
collected automatically by `internal/routing/passrate.go` and exposed at:
```
GET /pass-rate?skill=<name>&window=<duration>
```
No separate eval suite. No synthetic prompts. The benchmark runs itself once the routing
pod receives real traffic. Target: 30-day rolling window per skill, reviewed monthly.
**Bootstrap note:** With no session history, `passrate.go` returns `nil` and the router
defaults to the thinking model for every call. The fast-model path activates only after
real pass-rate data accumulates. Seed with real usage — do not pre-populate.
**Consequences:** Zero maintenance overhead for the benchmark. The tradeoff is that results
are only meaningful after ~2 weeks of real usage, and skills that are rarely invoked will
have statistically thin pass-rate data. Revisit if a skill has fewer than 20 calls in 30 days.
---
## 2026-05-28 — brain injection in skill handlers: review is done, others unverified
**Context:** The April 2026 scope reset listed "brain_query injection into skill handlers"
as the top priority. As of 2026-05-28, `internal/skills/review/handlers.go` calls
`brain.Query(ctx, ...)` before dispatching to the LLM — confirmed in code review.
Status of debug, retrospective, and trainer handlers is unverified.
**Decision:** Treat review as the reference implementation. Verify debug, retrospective,
trainer against the same pattern before shipping new skill work. Tracked in issue #32.
**Consequences:** The April concern may be stale for review. A one-pass audit of the other
three skill handlers closes this fully.
---
## 2026-04-08 — AGENTS.md as cross-tool standard, not CLAUDE.md
**Context**: Multiple tools (Crush, Pi, Antigravity) read `AGENTS.md` natively. Claude Code reads `CLAUDE.md`. Building on `CLAUDE.md` as the primary format locks into one vendor.
-30
View File
@@ -1,30 +0,0 @@
# syntax=docker/dockerfile:1
# ── Build stage ───────────────────────────────────────────────────────────────
FROM golang:1.26-bookworm AS builder
ARG VERSION=dev
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
go build -trimpath -ldflags="-s -w -X main.version=${VERSION}" \
-o /out/routing ./cmd/routing
# ── Runtime stage ─────────────────────────────────────────────────────────────
FROM gcr.io/distroless/base-debian12
COPY --from=builder /out/routing /usr/local/bin/routing
COPY config/ /app/config/
ENV SUPERVISOR_CONFIG_DIR=/app/config/supervisor
ENV ROUTING_PORT=3210
EXPOSE 3210
USER 65532:65532
ENTRYPOINT ["/usr/local/bin/routing"]
+49
View File
@@ -0,0 +1,49 @@
# Icebox — retired code (recoverable, not destroyed)
Per issue #75 (consolidate to a single harness), the routing-pod path was removed
from the live tree. It is **preserved and recoverable**, not deleted without trace.
## What was iceboxed (2026-07-01, issue #75)
| Path | Why |
|------|-----|
| `cmd/routing/` | The routing MCP-server binary; every skill call was wrapped through the broken pass-rate router (`wrap(skillName)`). |
| `internal/routing/` | Router / Fetcher / Policy / pass-rate. Signal is survivorship-biased and unusable as-is (infra#174). |
| `internal/skills/{review,debug,retrospective,trainer,project}/` | Skill handlers usable **only** through `cmd/routing` (verified: each imported solely by `cmd/routing`). |
| `Dockerfile.routing` | Built `cmd/routing` exclusively. |
| `.gitea/workflows/cd.yml` (routing steps only) | Removed the routing image build + infra image-bump + Flux-wait/rollout-verify for routing; **ingestion build/deploy is unchanged**. |
## Why
The live minimal harness is `cmd/hyperguild` + `brain-mcp` (+ `gitea-mcp` available) —
that is what ran the infra#170 loop-1 experiment (routing/injection machinery off) and
closed it twice. `cmd/hyperguild` has **zero** transitive dependency on `internal/routing`
or `cmd/routing`'s packages (it imports only `internal/tier`). The routing pass-rate signal
is being retired, not resurrected (fresh start, per infra#174).
## How to recover
Everything above is preserved at commit `00e5f62` under:
- **tag** `icebox/cmd-routing-2026-07-01`
- **branch** `icebox/cmd-routing`
```bash
# inspect
git checkout icebox/cmd-routing-2026-07-01
# restore specific packages onto a branch
git checkout icebox/cmd-routing-2026-07-01 -- cmd/routing internal/routing \
internal/skills/review internal/skills/debug internal/skills/retrospective \
internal/skills/trainer internal/skills/project Dockerfile.routing
```
## Deliberately NOT touched here (separate scope)
- **Live k8s routing deployment** (`infra` repo, `k3s/apps/routing/`) still runs its last
image; CD no longer rebuilds/redeploys it. Tearing down that deployment is a separate
infra-repo task.
- **`internal/skills/{brain,org,sessionlog}/`** — kept per #75; already had no importer
(orphaned before this cut), harmless, compile + test green.
- **`config/supervisor/{review,debug,retrospective,trainer-*}.md`** — routing skill prompts,
now orphaned data; left in place (not code, no build impact).
+72 -49
View File
@@ -5,14 +5,31 @@ Instead of letting Claude Code do whatever it wants, hyperguild enforces structu
workflows (TDD red/green/refactor), logs every session, and accumulates learnings
into a searchable brain.
## Hypothesis
> We believe routing skill tasks through local models, backed by brain context,
> produces measurably better outcomes than raw Claude Code alone —
> measurable by per-skill pass rate over rolling 30-day windows
> (available at `GET /pass-rate?skill=<name>&window=30d` on the brain pod).
This is the falsifiable claim the routing pod and pass-rate infrastructure exist to test.
If per-skill pass rates don't improve over baseline (all-cloud) after 30 days of real
usage, the fast-model routing path should be reconsidered.
## Harness
**hyperguild = Claude Code + MCP.** This is a supervisor for Claude Code sessions specifically.
For multi-agent orchestration (OpenCode + LiteLLM, executor/reviewer pipelines), see
[agentsquad](http://gitea.d-ma.be/mathias/agentsquad) — a separate harness for a different
orchestration model. Skills (mathias/skills) are shared between both.
## How it works
```
Your Claude Code session (in any project)
│ MCP over HTTP (Tailscale)
├──▶ supervisor :3200 (NodePort 30320 on koala) — skill workers: tdd, debug, spec, …
├──▶ routing :3210 (NodePort 30310 on koala) — Mode 2 only: review, debug, retrospective, trainer
├──▶ routing :3210 (NodePort 30310 on koala) — review, debug, retrospective, trainer
└──▶ brain :3300 (NodePort 30330 on koala) — brain_query, brain_write, brain_ingest, session_log
└─ also serves the legacy REST endpoints (/query, /write, /ingest, …)
@@ -20,34 +37,28 @@ Your Claude Code session (in any project)
brain/
├── sessions/ — JSONL log, one file per session_id
├── wiki/ — searchable knowledge (full-text)
│ ├── concepts/
│ ├── entities/
│ └── sources/
── raw/ — retrospective output, staged for review
└── training-data/ — SFT/DPO/RL data (Phase 2)
├── wiki/ — searchable knowledge (wing/hall layout)
│ ├── homelab/
│ ├── claude-sessions/
│ └── ...
── knowledge/ — legacy flat notes (migration pending: hyperguild#22)
```
## Phase 1 tools (available now)
| Tool | What it does |
|------|-------------|
| `tdd_red` | Writes a failing test for a spec, verifies it fails |
| `tdd_green` | Writes the minimal implementation to make tests pass |
| `tdd_refactor` | Cleans up implementation while keeping tests green |
| `session_log` | Appends a structured entry to the session JSONL log |
| `retrospective` | Reads the session log, identifies novel learnings, writes to brain/raw/ |
| `retrospective` | Reads the session log, identifies novel learnings, writes to brain |
| `review` | Structured code review via local model, brain-context injected |
| `debug` | Hypothesis-driven debugging via local model |
| `brain_query` | Full-text search over brain/wiki/ |
| `brain_write` | Writes a note to brain/raw/ (with optional YAML frontmatter) |
| `brain_write` | Writes a note to brain (with wing/hall routing) |
| `brain_answer` | BM25 + LLM synthesis — Q&A over brain corpus |
| `tier` | Returns the current connectivity tier (1=cloud, 2=LAN, 3=offline) |
## Start the servers
```bash
# Requires goreman: go install github.com/mattn/goreman@latest
task start # starts ingestion (:3300) + supervisor (:3200) via goreman
task stop # kills both by port
```
> **Note:** `tdd_red/green/refactor` and `spec` were retired in Plan 7 (2026-05-12).
> They are now SKILL.md files in [mathias/skills](http://gitea.d-ma.be/mathias/skills).
## Connect a project
@@ -56,9 +67,9 @@ Create `.mcp.json` in your project root:
```json
{
"mcpServers": {
"supervisor": {
"routing": {
"type": "http",
"url": "http://koala:30320/mcp"
"url": "http://koala:30310/mcp"
},
"brain": {
"type": "http",
@@ -68,33 +79,29 @@ Create `.mcp.json` in your project root:
}
```
Two MCP servers are exposed today, both reachable over Tailscale:
Two MCP servers are exposed, both reachable over Tailscale:
- **`supervisor`** at `koala:30320` — skill workers (`tdd_red/green/refactor`,
`review`, `debug`, `spec`, `retrospective`, `trainer`, `tier`).
- **`routing`** at `koala:30310` — skill workers (`review`, `debug`, `retrospective`, `trainer`).
Routes each call to fast local model or thinking model based on per-skill pass rate.
- **`brain`** at `koala:30330` — knowledge access (`brain_query`, `brain_write`,
`brain_ingest`, `brain_ingest_raw`) and `session_log`. Hosted by the ingestion
service directly, no separate pod.
`brain_ingest`, `brain_ingest_raw`, `brain_answer`, `brain_classify`) and `session_log`.
No local binary or stdio shim is required — Claude Code talks to both via HTTP.
Open Claude Code in your project — run `/mcp` to confirm both servers are listed.
## A typical TDD session
## A typical session
```
1. Call tdd_red → spec in, failing test file out
2. Call tdd_green → test path in, implementation out
3. Call tdd_refactor → impl + test in, cleaned code out
4. Call session_log → log each phase result
5. Call retrospective → extracts learnings → brain/raw/
6. Review brain/raw/, move worthy notes to brain/wiki/concepts/
7. Future sessions: call brain_query to retrieve relevant context
1. Call review → brain context injected + local model review → findings
2. Call session_log → log each phase result
3. Call retrospective → extracts learnings → brain
4. Future sessions: call brain_query / brain_answer to retrieve relevant context
```
## Tier detection
The supervisor probes connectivity at call time:
The routing pod probes connectivity at call time:
| Tier | Label | Condition |
|------|-------|-----------|
@@ -102,31 +109,47 @@ The supervisor probes connectivity at call time:
| 2 | lan-only | Can reach LiteLLM but not Anthropic |
| 3 | airplane | No external connectivity |
## Model routing
The routing pod selects models per skill call based on historical pass rate:
| Pass rate | Decision |
|-----------|----------|
| ≥ 0.90 (FLOOR) | Fast model (`HYPERGUILD_FAST_MODEL`) |
| ≤ 0.70 (CEIL) | Thinking model (`HYPERGUILD_THINKING_MODEL`) |
| between CEIL and FLOOR | Sample band — probabilistic routing |
| nil (no history yet) | Defaults to thinking model |
> **Bootstrap note:** With no session history, all calls route to the thinking model.
> The fast-model path activates only after real pass-rate data accumulates at `/pass-rate`.
> Seed with real usage — don't try to pre-populate.
## Key env vars
| Variable | Default | Purpose |
|----------|---------|---------|
| `INGEST_BRAIN_DIR` | `../brain` | Brain directory for ingestion server |
| `INGEST_PORT` | `3300` | Ingestion server port |
| `SUPERVISOR_CONFIG_DIR` | `./config/supervisor` | Skill discipline files |
| `SUPERVISOR_SESSIONS_DIR` | `./brain/sessions` | JSONL session logs |
| `INGEST_BASE_URL` | `http://localhost:3300` | Supervisor → ingestion |
| `INGEST_BASE_URL` | `http://localhost:3300` | Routing pod → brain |
| `LITELLM_BASE_URL` | — | LiteLLM proxy for Tier 2 model routing |
| `SUPERVISOR_MCP_TOKEN` | — | Optional bearer token for the supervisor MCP HTTP endpoint; when empty, no auth is enforced |
| `ROUTING_PORT` | `3210` | Routing pod's listen port |
| `ROUTING_MCP_TOKEN` | — | Optional bearer token for the routing MCP HTTP endpoint |
| `ROUTING_MCP_TOKEN` | — | Optional bearer token; when empty, no auth enforced |
| `BRAIN_URL` | `http://ingestion.supervisor:3300` | Routing pod → brain (in-cluster) |
| `HYPERGUILD_FAST_MODEL` | `koala/qwen35-9b-fast` | Fast model for high-pass-rate skill calls |
| `HYPERGUILD_THINKING_MODEL` | `iguana/gemma4-26b` | Thinking model for low-pass-rate skill calls |
| `HYPERGUILD_ROUTE_LOCAL_FLOOR` | `0.90` | At/above pass rate, route to fast model |
| `HYPERGUILD_ROUTE_LOCAL_CEIL` | `0.70` | Below pass rate, route to thinking model. Between CEIL and FLOOR is the sample band. |
| `HYPERGUILD_ROUTE_LOCAL_FLOOR` | `0.90` | Fast model threshold |
| `HYPERGUILD_ROUTE_LOCAL_CEIL` | `0.70` | Thinking model threshold |
| `HYPERGUILD_PASS_RATE_TTL_SECONDS` | `60` | Per-skill pass-rate cache TTL |
> **Operator note:** LiteLLM at `LITELLM_BASE_URL` must register both `HYPERGUILD_FAST_MODEL` and `HYPERGUILD_THINKING_MODEL` for routing to do useful work. If a model is missing, LiteLLM returns 4xx, the routing pod's fast route fails, the fail-open retry on the thinking model likely also fails (since both are missing), and the only signal is `final_status: "fail"` on `_routing` entries in the brain.
> **Operator note:** LiteLLM at `LITELLM_BASE_URL` must register both `HYPERGUILD_FAST_MODEL`
> and `HYPERGUILD_THINKING_MODEL`. If a model is missing, the fail-open retry also fails and
> the only signal is `final_status: "fail"` on `_routing` entries in the brain.
## Phase 2 (planned)
## Open issues
- `review` skill — structured code review with iron law enforcement
- `debug` skill — hypothesis-driven debugging sessions
- `spec` skill — generates specs from conversations
- `trainer` — extracts SFT/DPO pairs from session logs for fine-tuning
See [issues](http://gitea.d-ma.be/mathias/hyperguild/issues) — key open items:
- **#25** — skills platform overhaul (audit first, then lazy loading + brain feedback loop)
- **#24** — reduce context burn from skill listing
- **#22** — migrate legacy brain notes to wing/hall layout (one-shot script, low risk)
- **#31** — connect routing-mcp to claude.ai as custom connector
+2 -4
View File
@@ -17,8 +17,6 @@ tasks:
cmds: [bash scripts/context-sync.sh claude]
context:sync:agents:
cmds: [bash scripts/context-sync.sh agents]
context:sync:cursor:
cmds: [bash scripts/context-sync.sh cursor]
# ── Development ────────────────────────────────────────────────────────────
@@ -90,12 +88,12 @@ tasks:
cmds:
- task: context:sync
- cmd: |
drift=$(git status --porcelain -- AGENTS.md CLAUDE.md .cursorrules .aider.conventions.md .context/system-prompt.txt 2>/dev/null)
drift=$(git status --porcelain -- AGENTS.md CLAUDE.md .context/system-prompt.txt 2>/dev/null)
if [ -n "$drift" ]; then
echo "ERROR: derived adapters drifted from canonical context." >&2
echo "$drift" >&2
echo "" >&2
echo "Run: git add AGENTS.md CLAUDE.md .cursorrules .aider.conventions.md .context/system-prompt.txt" >&2
echo "Run: git add AGENTS.md CLAUDE.md .context/system-prompt.txt" >&2
echo " git commit -m 'chore: re-sync context adapters'" >&2
exit 1
fi
@@ -0,0 +1,48 @@
{"_meta":true,"note":"Agent-consumer column of brain-MCP intent analysis. consumer_type fixed=autonomous_agent. CAVEAT: canonical schema file brain-intent-extraction.md is NOT present on this host (koala) — only this session's own task prompt references it. The closed intent vocabulary below was RECONSTRUCTED from the task prompt's framing + brain/schema.md. Re-map intent labels if the canonical vocab differs. schema_source=reconstructed on every row.","closed_intent_vocab":["semantic_retrieval","lexical_lookup","check_prior_art","synthesized_answer","store_new_knowledge","update_or_supersede","ingest_raw_source","verify_write_landed","discover_capability","intent_unclear"],"intent_tool_match_values":["match","mismatch","partial"],"corpus":"~/.claude/projects/*/*.jsonl (Claude Code agent transcripts on koala). brain/sessions/*.jsonl empty. agentsquad docs/eval/*.jsonl are code-review eval results, NOT brain calls. No separate Crush logs found. Zero brain calls appear under any mcp__ name with a human typing the call — all brain acts are agent-initiated (CLAUDE.md reflex), so all qualify as autonomous_agent."}
{"id":"a01","session":"tapir-c","ts":"2026-06-?T15:01:51","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"single pre-task query 'YouTube Data API captions download ownership limitation timedtext adapter Go' — named-entity lexical lookup, fit BM25 well, no reformulation."}
{"id":"a02","session":"tapir","ts":"2026-06-05T21:44:10","tool":"brain_ingest","intent":"ingest_raw_source","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_ingest 14s earlier — tool not ambient, had to be discovered/loaded first.","evidence":"source=tapir-scheduled-discovery-session-2026-06-05, a session learnings dump."}
{"id":"a03","session":"tapir","ts":"2026-06-?T14:00:59","tool":"brain_ingest","intent":"ingest_raw_source","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"source=tapir-rls-identity-bootstrapping, first write of RLS lesson."}
{"id":"a04","session":"tapir","ts":"2026-06-?T14:02:08","tool":"brain_ingest","intent":"update_or_supersede","intent_tool_match":"mismatch","workaround":"RE-INGESTED same source name 'tapir-rls-identity-bootstrapping' 69s later with edited/condensed body. No update/patch/supersede verb exists, so the agent overwrote-by-re-ingest. Whether this dedups or creates a v2 duplicate is opaque to the agent.","observed_friction":"agent revised content within 70s of first write — classic edit-after-write with no edit primitive.","evidence":"two brain_ingest, identical source string, divergent content."}
{"id":"a05","session":"tapir","ts":"2026-06-?T14:56:27","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"postgres-cascade-skips-tables-without-fk.md — distinct new lesson."}
{"id":"a06","session":"tapir","ts":"2026-06-?T21:08:57","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_query — discovery tax again.","evidence":"'Dex passwords.dex.coreos.com CRD ...' keyword-rich, single shot."}
{"id":"a07","session":"AI-infra","ts":"2026-05-?T15:38:03","tool":"brain_query(HTTP-curl)","intent":"discover_capability","intent_tool_match":"mismatch","workaround":"raw `curl -X POST` to brain-mcp endpoint instead of MCP tool. Preceded by two ToolSearch ('brain knowledge memory' then 'brain') that did not yield a usable loaded tool, so agent fell back to HTTP.","observed_friction":"3-step ladder: ToolSearch 'brain knowledge memory' -> ToolSearch 'brain' -> curl. Agent did not know which act maps to which tool name.","evidence":"curl -s -o /tmp/brain-init.txt -w code:%{http_code} -X POST ..."}
{"id":"a08","session":"AI-infra","ts":"2026-05-?T04:46:13","tool":"brain_query(HTTP-curl)","intent":"discover_capability","intent_tool_match":"mismatch","workaround":"hand-set TOKEN=... then curl brain-test endpoint — probing whether the HTTP brain path is reachable/authed at all. MCP path not used.","observed_friction":"agent testing connectivity by hand; MCP auth/availability not trusted.","evidence":"TOKEN=...; curl -s -o /tmp/brain-test ..."}
{"id":"a09","session":"AI-infra","ts":"2026-05-?T05:23:35","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_query (after an earlier 05:03 ToolSearch 'brain ingestion knowledge wiki' that explored layers).","evidence":"'koala machine state RTX 5070 llama-swap' — named-entity recall, fits lexical."}
{"id":"a10","session":"AI-infra","ts":"2026-05-?T05:27:36","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"batch of 3 in ~1s (k3s/flux gitops; llama-swap ai-stack GPU; MCP Dex OAuth claude.ai) — parallel prior-art sweep, all named-entity."}
{"id":"a11","session":"AI-infra","ts":"2026-05-?T07:13:50","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"batch of 4 in ~2s before a debugging session (flux healthCheck; exit 255 restart loop; NVML mismatch; mirror rebase). Named symptoms, lexical fit OK on first pass."}
{"id":"a12","session":"AI-infra","ts":"2026-05-?T07:14:26","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"4 writes in ~35s (flux-healthcheck-stale; exit-255-unknown-reason-not-oom; nvidia-nvml-mismatch; mcp-static-bearer) — answers to the 4 queries just run, captured as lessons. Healthy query->fix->write loop."}
{"id":"a13","session":"AI-infra","ts":"2026-05-?T07:15:25","tool":"brain_query","intent":"verify_write_landed","intent_tool_match":"mismatch","workaround":"25s after writing exit-255-unknown-reason-not-oom.md, re-queried 'exit 255 unknown SIGKILL containerd' — reformulated terms (SIGKILL/containerd not in original query 'exit 255 unknown reason restart loop diagnosis'). Either confirming the fresh write is retrievable or re-searching because first lexical query missed. No read-after-write / get-by-id act exists.","observed_friction":"reformulation chain: 'exit 255 unknown reason restart loop diagnosis' -> 'exit 255 unknown SIGKILL containerd'. Same need, different keywords.","evidence":"query at 07:13:51 vs 07:15:25 bracketing the 07:14:37 write."}
{"id":"a14","session":"AI-infra","ts":"2026-05-?T07:22:55","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"batch of 5 in ~20s before a homelab security audit (piguard/iguana tailscale; unifi UCG firewall; SOPS age; ingress TLS cert-manager; koala UFW iptables). Named-entity sweep."}
{"id":"a15","session":"AI-infra","ts":"2026-05-?T09:27:53","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"audit-shortcut-tls-blocks-zero; policy-audit-mode-blocks-nothing — distinct new audit lessons."}
{"id":"a16","session":"AI-infra","ts":"2026-05-?T09:28:22","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"homelab-security-chains-not-bugs.md FIRST write (worked example: koala 2026-05-13)."}
{"id":"a17","session":"AI-infra","ts":"2026-05-?T10:32:49","tool":"brain_write","intent":"update_or_supersede","intent_tool_match":"mismatch","workaround":"RE-WROTE homelab-security-chains-not-bugs.md ~64min later with a different/expanded worked example (host-user dotfile, over-broad ClusterRole). Same filename, additive revision, no patch/append/supersede verb — agent overwrites and hopes the index replaces rather than duplicates.","observed_friction":"the in-between hour of audit work produced a better example; only way to fold it in was a full re-write of the same slug.","evidence":"two brain_write same filename at 09:28:22 and 10:32:49, divergent worked examples."}
{"id":"a18","session":"AI-infra","ts":"2026-05-?T10:18:25","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"homelab-document-accepted-risk-to-break-audit-cycle.md — distinct."}
{"id":"a19","session":"AI-infra","ts":"2026-05-?T10:32:55","tool":"brain_query","intent":"verify_write_landed","intent_tool_match":"mismatch","workaround":"6s after the homelab-chains re-write, queried 'RBAC MCP cluster pods log chain' — checking the chain reasoning is retrievable / finding the related entry. Read-after-write done via lexical search.","observed_friction":null,"evidence":"query immediately follows the 10:32:49 write."}
{"id":"a20","session":"AI-infra","ts":"2026-05-?T18:57:34","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_write,brain_query — re-discovered tools this session.","evidence":"'extension build pinned version major version upgrade postgres pgvector'."}
{"id":"a21","session":"AI-infra","ts":"2026-05-?T18:57:58","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"extension-version-lags-platform-major-upgrade.md."}
{"id":"a22","session":"AI-infra","ts":"2026-05-?T18:58:06","tool":"brain_query","intent":"verify_write_landed","intent_tool_match":"mismatch","workaround":"8s after writing extension-version-lags, re-queried 'pgvector postgres extension version compile error bump' — reformulated from the 18:57:34 query ('extension build pinned version...'). Lexical re-search to confirm the just-written lesson is findable, with different keyword guess.","observed_friction":"reformulation: 'extension build pinned version major version upgrade postgres pgvector' -> 'pgvector postgres extension version compile error bump'.","evidence":"write at 18:57:58 bracketed by queries 18:57:34 and 18:58:06."}
{"id":"a23","session":"AI-infra","ts":"2026-05-?T18:31:23","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"webfetch-readme-when-image-or-flag-uncertain.md FIRST write."}
{"id":"a24","session":"AI-infra","ts":"2026-05-?T18:34:23","tool":"brain_write","intent":"update_or_supersede","intent_tool_match":"mismatch","workaround":"RE-WROTE webfetch-readme-when-image-or-flag-uncertain.md 3min later, near-identical body. Looks like a retry/overwrite (uncertain the first landed, or minor edit). No idempotent upsert with confirmation, so agent re-fires the write.","observed_friction":"followed 7s later by a brain_query on the same topic ('OSS tool image registry CLI flag webhook path schema drift README pre-flight') — write-write-query, i.e. overwrite then verify-by-search.","evidence":"two brain_write same filename 18:31:23 / 18:34:23, then query 18:34:31."}
{"id":"a25","session":"AI-infra","ts":"2026-05-?T18:34:31","tool":"brain_query","intent":"verify_write_landed","intent_tool_match":"mismatch","workaround":"keyword-stuffed lexical query 'OSS tool image registry CLI flag webhook path schema drift README pre-flight' fired right after the webfetch-readme write — agent dumps every concept token hoping BM25 surfaces its own fresh note. This is semantic intent (find that conceptual lesson) coerced into a bag-of-keywords.","observed_friction":"query is a concatenation of the note's section headings — a tell that the agent is groping lexically for content it knows by meaning.","evidence":"query text mirrors the just-written note's bullet topics."}
{"id":"a26","session":"dev","ts":"2026-06-?T21:18:26","tool":"brain_answer","intent":"synthesized_answer","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_answer.","evidence":"'tapir transcript persistence shared cross-user dedup table RLS isolation ADR-021 ...' -> 22min later a brain_write (acted on the answer). Answer consumed, not re-queried. Healthy."}
{"id":"a27","session":"dev","ts":"2026-06-?T21:40:20","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"tapir-migration-and-rls-test-infra-gotchas, with wing/hall absent here (flat) — see schema-confusion note a40."}
{"id":"a28","session":"dev","ts":"2026-06-?T05:35:04","tool":"brain_answer","intent":"synthesized_answer","intent_tool_match":"partial","workaround":"asked 'gitea MCP not working workaround file issue via API which token ... how to authenticate gitea API' — a how-do-I question. Next brain act (05:39 query) is a different topic (tapir transcript), so the answer was apparently sufficient OR abandoned; ambiguous.","observed_friction":null,"evidence":"brain_answer then unrelated brain_query 4min later."}
{"id":"a29","session":"dev","ts":"2026-06-?T05:39:54","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"'tapir transcript persistence ADR-021 shared non-RLS'."}
{"id":"a30","session":"dev","ts":"2026-06-?T05:57:37","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"gitea-mcp-per-repo-tools-404-and-rest-fallback FIRST write."}
{"id":"a31","session":"dev","ts":"2026-06-?T05:57:55","tool":"brain_write","intent":"update_or_supersede","intent_tool_match":"mismatch","workaround":"RE-WROTE gitea-mcp-per-repo-tools-404-and-rest-fallback 18s later — overwrite/retry of same slug, no upsert confirmation.","observed_friction":"sub-20s gap = almost certainly a content tweak the agent could not express as an edit.","evidence":"two brain_write same filename 05:57:37 / 05:57:55."}
{"id":"a32","session":"dev","ts":"2026-05-?T11:51:47","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"infra-litellm-absorption-2026-05-16.md."}
{"id":"a33","session":"dev","ts":"2026-05-?T12:07:04","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"batch of 3 (litellm rebuild time piguard; docker compose orphaned volumes; prometheus_client ModuleNotFoundError) — lexical, error-string driven."}
{"id":"a34","session":"dev","ts":"2026-05-?T15:08:15","tool":"brain_answer","intent":"synthesized_answer","intent_tool_match":"mismatch","workaround":"THREE brain_answer at 15:08 (moved compose volumes? / pi rebuild time? / litellm ModuleNotFound prometheus) — the SAME three topics queried lexically an hour earlier (12:07) — were IMMEDIATELY followed at 15:09 by THREE brain_query on the same three topics. The agent asked the synthesizer, was unsatisfied, and fell straight back to raw lexical search. Strongest answer->query fallback in the corpus.","observed_friction":"answer/query duplication across one intent: agent hedges by firing both interfaces, trusting neither.","evidence":"15:08 answers vs 15:09 queries, topic-for-topic aligned."}
{"id":"a35","session":"dev","ts":"2026-05-?T15:09:16","tool":"brain_query","intent":"semantic_retrieval","intent_tool_match":"mismatch","workaround":"after the 3 brain_answer calls failed to satisfy, re-issued as lexical brain_query ('moved compose stack to new directory volumes disappeared empty'; 'raspberry pi docker build time arm slow'; 'how to enable prometheus metrics on litellm proxy callback'). The want is meaning-based ('did my volumes move?') but the only retrieval that 'worked' was keyword search — and these are full natural-language sentences crammed into a BM25 box.","observed_friction":"natural-language questions ('how to enable...', 'moved ... disappeared') passed to a lexical query tool — semantic intent, lexical interface.","evidence":"3 queries at 15:09 mirror the 3 answers at 15:08."}
{"id":"a36","session":"dev","ts":"2026-05-?T20:31:50","tool":"brain_answer","intent":"synthesized_answer","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"'What happened with the litellm migration on 2026-05-16?' — episodic recall question, answer fit; no re-query followed."}
{"id":"a37","session":"dev","ts":"2026-05-?T21:07:17","tool":"brain_query","intent":"semantic_retrieval","intent_tool_match":"mismatch","workaround":"FOUR-step reformulation chain over one Go bug: 'bytes.Buffer Bytes Reset aliasing slice sharing' -> 'go buffer reuse map backing array bug' -> [write go-bytes-buffer-bytes-reset-aliasing-trap.md] -> 'go map values all show same content after loop' -> 'bytes.Buffer Bytes returns same data every iteration'. The agent knows the SYMPTOM (all map values identical) and the CAUSE (Bytes() aliasing) but cannot phrase a single lexical query that bridges them — it wants concept retrieval and is forced to brute-force keyword variants.","observed_friction":"4 distinct phrasings of the same bug, two before and two after writing the lesson — also doubles as verify_write_landed on the trailing queries.","evidence":"21:07:17, 21:07:17, (write 21:08:01), 21:08:10, 21:08:19."}
{"id":"a38","session":"dev","ts":"2026-05-?T21:08:01","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"go-bytes-buffer-bytes-reset-aliasing-trap.md."}
{"id":"a39","session":"dev","ts":"2026-05-?T07:54:26","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"mcp-tool-design-get-needs-list-partner.md — a design principle."}
{"id":"a40","session":"dev","ts":"2026-06-?T06:25:00","tool":"brain_query","intent":"check_prior_art","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"'Dex to Authentik migration auth.d-ma.be issuer cutover OIDC subject ...'."}
{"id":"a41","session":"dev","ts":"2026-06-?T06:25:08","tool":"brain_answer","intent":"synthesized_answer","intent_tool_match":"mismatch","workaround":"brain_query (a40) and brain_answer (a41) fired ~8s apart on the SAME intent (Dex->Authentik subject-keyed token orphan). Agent runs lexical search AND synthesized answer in parallel for one question rather than choosing — it cannot predict which interface will return usable knowledge, so it pays both.","observed_friction":"query+answer doublet on one need.","evidence":"06:25:00 query then 06:25:08 answer, same topic."}
{"id":"a42","session":"dev","ts":"2026-06-?T13:48:49","tool":"brain_query","intent":"verify_write_landed","intent_tool_match":"mismatch","workaround":"'authentik cutover validation probe' then 6min later 'authentik cutover post-flip validation' — reformulated pair, likely searching for the agent's own earlier cutover notes / confirming validation steps are recorded. Lexical re-search standing in for recall-my-recent-context.","observed_friction":"reformulation: 'validation probe' -> 'post-flip validation'.","evidence":"13:48:49 and 13:54:56."}
{"id":"a43","session":"dev","ts":"2026-06-?T13:59:17","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":"preceded by ToolSearch select:brain_write.","evidence":"oidc-issuer-host-change-vs-idp-swap-subject."}
{"id":"a44","session":"dev","ts":"2026-06-?T13:59:50","tool":"brain_write","intent":"store_new_knowledge","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"cannot-move-ingress-host-across-namespaces-flux-dryrun FIRST write."}
{"id":"a45","session":"dev","ts":"2026-06-?T14:00:13","tool":"brain_write","intent":"update_or_supersede","intent_tool_match":"mismatch","workaround":"RE-WROTE cannot-move-ingress-host-across-namespaces-flux-dryrun 23s later — overwrite of same slug, no edit/upsert primitive.","observed_friction":"sub-30s gap = content correction expressed as a full re-write.","evidence":"two brain_write same filename 13:59:50 / 14:00:13."}
{"id":"a46","session":"dev","ts":"2026-06-?T13:53:54","tool":"brain_write(HTTP-staged)","intent":"store_new_knowledge","intent_tool_match":"mismatch","workaround":"after `ToolSearch select:mcp__claude_ai_brain__authenticate` (MCP auth flow), the agent staged the entry as `cat > /tmp/brain_entry.json` ({filename:'postgres-force-rls-cross-u...', content}) for a curl write rather than calling brain_write directly — MCP write path was not usable (auth/loading), so it dropped to the HTTP bodge.","observed_friction":"reached for an 'authenticate' tool, then abandoned MCP for hand-built JSON + curl. Matches known pattern: brain/op MCP auth lapses often.","evidence":"ToolSearch authenticate 13:53:27 -> cat /tmp/brain_entry.json 13:53:54."}
{"id":"a47","session":"template-go-agent","ts":"2026-05-?T18:46:26","tool":"BASH(not-a-brain-act)","intent":"intent_unclear","intent_tool_match":"match","workaround":null,"observed_friction":null,"evidence":"'brain_' substring was inside a git commit message body ('agent boundaries, network policy, agent s...'), NOT a brain call. Excluded from knowledge-act analysis; logged for audit completeness."}
@@ -0,0 +1,148 @@
# Agent-Consumer Brain Intent Analysis — koala column
**Consumer:** `autonomous_agent` (all rows). **Host:** koala. **Date:** 2026-06-15.
**Raw rows:** `agent-intent-column.jsonl` (46 real knowledge-acts + 1 excluded false-positive).
## Caveat — canonical schema not on this host
The shared closed-vocabulary file `brain-intent-extraction.md` **does not exist on
koala** — the only reference to it is inside *this task's own prompt*. The intent
vocabulary below was **reconstructed** from the prompt's framing + `brain/schema.md`.
Every row carries `schema_source: reconstructed`. If the canonical vocab differs,
re-map the `intent` field; the `intent_tool_match` / `workaround` / `observed_friction`
evidence stands regardless of label names.
**Reconstructed closed vocab:** `semantic_retrieval`, `lexical_lookup`,
`check_prior_art`, `synthesized_answer`, `store_new_knowledge`, `update_or_supersede`,
`ingest_raw_source`, `verify_write_landed`, `discover_capability`, `intent_unclear`.
## Corpus
- `~/.claude/projects/*/*.jsonl` — Claude Code agent transcripts (98 files). **The only
source with brain calls.**
- `brain/sessions/*.jsonl` — empty (only `.gitkeep`).
- `agentsquad docs/eval/*.jsonl` — code-review eval results, **not** brain calls.
- No separate Crush session logs on this host.
- **Zero** brain calls were human-typed. Every brain act is agent-initiated (the
CLAUDE.md "query as reflex / close-the-loop write" behaviour), so all qualify as
`autonomous_agent`. The human gave the top-level task; the agent chose every brain act.
## 1. Intent histogram, split by `intent_tool_match`
| intent | match | mismatch | partial | total |
|---|---|---|---|---|
| check_prior_art | 10 | 0 | 0 | 10 |
| store_new_knowledge | 14 | 1 | 0 | 15 |
| update_or_supersede | 0 | 5 | 0 | 5 |
| synthesized_answer | 2 | 2 | 1 | 5 |
| verify_write_landed | 0 | 4 | 0 | 4 |
| semantic_retrieval | 0 | 3 | 0 | 3 |
| ingest_raw_source | 2 | 0 | 0 | 2 |
| discover_capability | 0 | 2 | 0 | 2 |
| **total** | **28** | **17** | **1** | **46** |
> Batch note: several rows collapse a same-second fan-out of identical-intent calls
> (a10=3, a11=4, a14=5, a33=3, a34=3, a35=3). Call-level the corpus is ~62 brain calls;
> the table counts the 46 distinct knowledge-acts. Frequency is deliberately *not* the
> point — the mismatch column is.
**37% of agent knowledge-acts (17/46) are interface mismatches.** Every mismatch falls
into one of four intents: `update_or_supersede`, `verify_write_landed`,
`semantic_retrieval`, `discover_capability` — plus one `store` that had to use HTTP.
## 2. Mismatch list, grouped by intent (primary deliverable)
### update_or_supersede → re-write same slug (5/5 mismatch) — HIGHEST VALUE
There is **no update / patch / append / supersede verb**. When an agent improves a note
it already wrote, the only move is to call `brain_write`/`brain_ingest` **again with the
same filename/source** and hope the index replaces rather than duplicates. Observed:
| slug | 1st write | 2nd write | gap | what changed |
|---|---|---|---|---|
| `tapir-rls-identity-bootstrapping` (ingest) | 14:00:59 | 14:02:08 | 69s | condensed body |
| `homelab-security-chains-not-bugs.md` | 09:28:22 | 10:32:49 | 64m | new worked example |
| `webfetch-readme-when-image-or-flag-uncertain.md` | 18:31:23 | 18:34:23 | 3m | near-identical (retry) |
| `gitea-mcp-per-repo-tools-404-and-rest-fallback` | 05:57:37 | 05:57:55 | 18s | content tweak |
| `cannot-move-ingress-host-across-namespaces-flux-dryrun` | 13:59:50 | 14:00:13 | 23s | content tweak |
Sub-30s gaps (3 of 5) read as "I wanted to edit but can only overwrite." The agent has
no way to know whether the second write deduped or created a contradictory v2 — opacity
the brain's own design principle (`mcp-tool-design-get-needs-list-partner.md`, written
*by one of these very agents*) would flag: every `_write` needs a `_get`/`_update` partner.
### verify_write_landed → lexical re-query (4/4 mismatch)
No read-after-write / get-by-id confirmation. After every substantive write, agents
re-query lexically to check the note is retrievable — and *reformulate the keywords*
because they can't predict what BM25 indexed:
- `exit-255` lesson: query `exit 255 unknown reason restart loop diagnosis` → write →
query `exit 255 unknown SIGKILL containerd`.
- `extension-version-lags`: query `extension build pinned version...pgvector` → write →
query `pgvector postgres extension version compile error bump`.
- `webfetch-readme`: write → write → query stuffed with the note's own section headings.
### semantic_retrieval → BM25 keyword-stuffing (3/3 mismatch)
Agent knows the *meaning* but not the *indexed words*, so it brute-forces phrasings of
one need against a lexical tool:
- **4-step chain on one Go bug:** `bytes.Buffer Bytes Reset aliasing slice sharing`
`go buffer reuse map backing array bug` → (write) → `go map values all show same
content after loop``bytes.Buffer Bytes returns same data every iteration`. Symptom
and cause both known; no single lexical query bridges them.
- Natural-language questions (`how to enable prometheus metrics on litellm proxy
callback`, `moved compose stack to new directory volumes disappeared empty`) shoved
into `brain_query`.
### synthesized_answer → fall back to / hedge with brain_query (2 mismatch + 1 partial)
`brain_answer` is frequently **not trusted as terminal**:
- **Strongest signal:** 3× `brain_answer` at 15:08 (compose volumes / pi rebuild time /
litellm ModuleNotFound) → 3× `brain_query` at 15:09 on the *same three topics*. The
agent asked the synthesizer, was unsatisfied, and immediately re-ran raw search.
- Dex→Authentik: `brain_query` and `brain_answer` fired **8s apart on one question** —
the agent pays both interfaces because it can't predict which returns usable knowledge.
- (Counter-examples exist: `brain_answer` for episodic recall — "what happened with the
litellm migration on 2026-05-16?" — was consumed and not re-queried. So `answer`
works for *episodic/temporal* recall, fails for *how-do-I / does-X-hold* reasoning.)
### discover_capability + store-via-HTTP (3 mismatch)
brain tools are **not ambient** — they are deferred and must be `ToolSearch`-loaded each
session. Agents fumble the discovery (`ToolSearch 'brain knowledge memory'` →
`'brain'` → `'brain ingestion knowledge wiki'`) and, when MCP load/auth fails, drop to
**raw `curl` against `brain-mcp` / hand-built `/tmp/brain_entry.json`**. One agent even
`ToolSearch`-ed an `authenticate` tool, then abandoned MCP for the HTTP bodge — matching
the known "brain/op MCP auth lapses too often" footgun.
### Write-interface / layer schema confusion (cross-cutting)
`brain_write` was called with **three different param shapes** in the same corpus:
`{filename, type:"lesson", content}`, `{filename, content}` (no type), and
`{wing:"tapir", hall:"failures", filename, content}` — plus `brain_ingest {source,
content}`. Agents are unsure which verb and which layer (flat slug vs `wing`/`hall`
knowledge routing vs raw ingest) a given knowledge-act maps to. This is the
`knowledge/ vs wiki/` confusion expressed at the parameter level.
## 3. `intent_unclear` rate
**0 / 46 genuine brain acts (0%).** Agent intent is unusually legible because these are
Claude Code transcripts: the surrounding task, the query/filename strings, and the
write content all disambiguate. One row (`a47`) was tagged `intent_unclear` and
**excluded** — its `brain_` substring was inside a git commit message, not a brain call.
Example of the only ambiguity that arose: a `brain_answer` on "gitea MCP not working...
how to authenticate" followed by an unrelated query — can't tell if the answer satisfied
or was abandoned (`partial`, row a28).
## 4. The single biggest intent↔interface gap
**The brain offers one write verb and one lexical read verb, but autonomous agents
perform four distinct knowledge-acts against them — and three of the four have no fitting
interface.** The deepest gap is the **missing update/supersede path**: agents close every
task by writing a lesson (the CLAUDE.md ritual), routinely improve it minutes-to-an-hour
later, and — having no edit primitive — re-write the same slug blind, unable to tell
whether they corrected the entry or forked a contradiction into the index. This compounds
with the lexical-only read side: because there is no `get-by-id` or semantic retrieval,
agents can't even reliably *find their own just-written note* to check it, so they
keyword-stuff reformulated queries and hedge `brain_answer` with parallel `brain_query`.
The interface is built for *append-and-keyword-search*; the agents are trying to
*curate a living, deduplicated knowledge base*, and the seam between those two shows up
as the 5 blind re-writes, 4 read-after-write re-queries, and 3 semantic-as-lexical chains
that dominate the mismatch column.
---
*Evidence-only per task scope — no redesign proposed.*
@@ -0,0 +1,140 @@
# Brain-MCP Intent↔Interface Findings — Unified (two-column merge)
**Status — 2026-06-16**
-**Agent column** filled from `agent-intent-column.jsonl` (46 acts, koala).
-**Human column** = `PENDING`. Drop the Claude.ai-history analysis into
`human-intent-column.jsonl` (same dir, schema below), then fill the `PENDING`
cells and the synthesis blocks marked `<<SYNTH>>`.
- ⚠️ Canonical `brain-intent-extraction.md` still absent on koala. Vocab below is
the **reconstructed** lock both columns must share. If the real file surfaces,
re-map `intent` labels in *both* columns identically before merging.
---
## Shared schema (LOCKED — both columns conform)
Per-call row, JSONL:
| field | values / form | notes |
|---|---|---|
| `id` | `a01..` (agent) / `h01..` (human) | column prefix kept distinct |
| `session` | string | source session/conversation id |
| `ts` | ISO-8601 | best-effort |
| `tool` | brain tool name (+ `(HTTP-curl)` / `(HTTP-staged)` suffix for bodges) | |
| `intent` | closed vocab ↓ | the knowledge-act WANTED |
| `intent_tool_match` | `match` \| `mismatch` \| `partial` | does the called tool fit the want |
| `consumer_type` | `autonomous_agent` \| `human_interactive` | fixed per column |
| `workaround` | string \| null | the bodge when mismatch — **primary signal** |
| `observed_friction` | string \| null | reformulation chains, discovery tax, hedging |
| `evidence` | string | excerpt anchoring the classification |
| `schema_source` | `reconstructed` | flip to `canonical` if real vocab lands |
### Closed intent vocab (LOCKED)
`semantic_retrieval`, `lexical_lookup`, `check_prior_art`, `synthesized_answer`,
`store_new_knowledge`, `update_or_supersede`, `ingest_raw_source`,
`verify_write_landed`, `discover_capability`, `intent_unclear`.
---
## Master comparison — by intent
| intent | agent acts | agent mismatch | human acts | human mismatch | shared gap |
|---|---|---|---|---|---|
| check_prior_art | 10 | 0% | `PENDING` | `PENDING` | — |
| store_new_knowledge | 15 | 7% (1/15) | `PENDING` | `PENDING` | `<<SYNTH>>` |
| update_or_supersede | 5 | **100%** (5/5) | `PENDING` | `PENDING` | `<<SYNTH>>` no edit verb |
| synthesized_answer | 5 | 40% (2/5)+1 partial | `PENDING` | `PENDING` | `<<SYNTH>>` |
| verify_write_landed | 4 | **100%** (4/4) | `PENDING` | `PENDING` | `<<SYNTH>>` no read-after-write |
| semantic_retrieval | 3 | **100%** (3/3) | `PENDING` | `PENDING` | `<<SYNTH>>` lexical-only read |
| ingest_raw_source | 2 | 0% | `PENDING` | `PENDING` | — |
| discover_capability | 2 | **100%** (2/2) | `PENDING` | `PENDING` | agent-specific (ToolSearch/auth)? |
| intent_unclear | 0 | — | `PENDING` | `PENDING` | divergence expected ↓ |
| **TOTAL** | **46** | **37% (17)** | `PENDING` | `PENDING` | |
---
## Per-intent merged findings
### update_or_supersede — agent: 5/5 mismatch (highest value)
**Agent:** no edit/patch/append verb. Agents re-write same slug blind:
`homelab-security-chains-not-bugs.md` (+64m), `tapir-rls-identity-bootstrapping`,
`webfetch-readme...`, `gitea-mcp-per-repo-tools-404...`,
`cannot-move-ingress-host...` — 3 of 5 sub-30s ("wanted edit, got overwrite").
Cannot tell if write deduped or forked a contradiction.
**Human:** `PENDING` — *look for: user editing a prior note, asking "update what I
saved about X", or expressing frustration that an old fact is stale/duplicated.*
**<<SYNTH>>** shared verdict once both filled.
### verify_write_landed — agent: 4/4 mismatch
**Agent:** no `get-by-id`/read-after-write. Agents lexically re-query their own
fresh note with reformulated keywords (`exit 255 unknown reason``...SIGKILL
containerd`; `extension build pinned...``pgvector ...compile error bump`).
**Human:** `PENDING` — *humans may not exhibit this (they trust the write UI
confirmation). If absent in human column, it's an agent-specific gap → flag.*
**<<SYNTH>>**.
### semantic_retrieval — agent: 3/3 mismatch
**Agent:** meaning known, indexed words unknown → BM25 keyword-stuffing. 4-step
chain on one Go `bytes.Buffer` bug; NL questions shoved into `brain_query`.
**Human:** `PENDING` — *humans likely hit this HARDER (they phrase conversationally).
Compare reformulation-chain length agent vs human.*
**<<SYNTH>>** — likely the strongest cross-consumer overlap.
### synthesized_answer — agent: 2 mismatch + 1 partial
**Agent:** `brain_answer` not trusted terminal — 3 answers → 3 same-topic queries
1min later; query+answer fired 8s apart hedging one need. Works for *episodic*
recall, fails for *how-do-I / does-X-hold*.
**Human:** `PENDING` — *humans may prefer `brain_answer` as primary (chat-native).
If human match-rate >> agent, the tool fits humans not agents → key divergence.*
**<<SYNTH>>**.
### store_new_knowledge — agent: 14/15 match
**Agent:** healthy, except 1 HTTP-staged bodge when MCP auth lapsed. Also surfaced
write-schema confusion: 3 param shapes (`{filename,type}` / `{filename}` /
`{wing,hall,filename}`) + `ingest{source}`.
**Human:** `PENDING`*humans rarely write directly; expect low volume.*
**<<SYNTH>>**.
### check_prior_art / ingest_raw_source — agent: 0% mismatch
Lexical fits named-entity recall and raw-source capture. **Human:** `PENDING`.
### discover_capability — agent: 2/2 mismatch (agent-specific)
Brain tools deferred → `ToolSearch`-load each session; auth lapse → `curl` bodge.
**Likely has NO human analog** (humans get ambient connectors). Candidate for
"agent-only gap" bucket. **Human:** `PENDING` to confirm absent.
---
## Cross-consumer divergence — questions to resolve at merge
1. **intent_unclear rate.** Agent = 0% (transcripts self-document). Human expected
higher (conversational, implicit). Big delta = the columns measure legibility
differently, not just intent.
2. **Where does each consumer's mismatch concentrate?** Agent mismatch is
write-side-heavy (supersede + verify-landed = 9/17). Hypothesis: human mismatch
is read-side-heavy (semantic + answer). If true → **the interface fails the two
consumers at opposite ends.**
3. **Agent-only gaps** (`discover_capability`, `verify_write_landed`) vs
**shared gaps** (`semantic_retrieval`, `update_or_supersede`). Shared gaps =
highest-priority evidence; agent-only = harness/auth issues.
---
## Combined headline — `<<SYNTH>>` (fill when human column lands)
> Agent-side draft (to be reconciled with human-side):
> Brain = append + keyword-search; agents want a curated, dedup'd, self-verifying KB.
> Missing update/supersede path + lexical-only reads are the seam. **Open question
> for the merge: do humans hit the same read-side wall, making semantic-retrieval the
> universal gap — or do agents uniquely suffer the write-side (supersede / verify)
> wall that humans sidestep via the chat UI?**
---
## Drop-in checklist (when human column arrives)
1. Place `human-intent-column.jsonl` in this dir; conform to LOCKED schema.
2. Fill every `PENDING` cell in master table + per-intent blocks.
3. Resolve the 3 divergence questions with evidence.
4. Replace each `<<SYNTH>>` with the reconciled verdict; write the combined headline.
5. If canonical vocab surfaced: re-map both columns' `intent`, flip `schema_source`.
6. Commit as `docs(brain): merge human+agent intent columns`.
+8 -9
View File
@@ -112,16 +112,15 @@ Flags:
- `--out PATH` — output file (default `./.mcp.json`)
- `--force` — overwrite an existing file
Modes:
Modes (all list **brain + gitea** — Gitea is the audit-trail invariant of the
consolidated single harness, #75):
- **cloud** — brain MCP only. Claude Code with no routing.
- **client-local** — brain + routing pod. The `routing` entry points at
`koala:30310/mcp` (the routing pod, deployed in Plan 6). The
`X-Hyperguild-Mode: client-local` header is forward-compat for future
modes; the pod treats absent or unknown values as `client-local`.
- **sovereign** — brain only, with a `_mode_note` explaining that this
mode primarily uses Crush + LiteLLM and the `.mcp.json` is a Claude
Code fallback for emergency offline use.
- **cloud** — brain + gitea MCP.
- **client-local** — brain + gitea MCP. (The former `routing` entry pointing at
`koala:30310/mcp` was removed — the routing pod is iceboxed, #75.)
- **sovereign** — brain + gitea, with a `_mode_note` explaining that this mode
primarily uses Crush + LiteLLM and the `.mcp.json` is a Claude Code fallback
for emergency offline use.
## Environment
+26 -19
View File
@@ -59,31 +59,40 @@ func runMode(ctx context.Context, args []string, _ io.Reader, stdout, stderr io.
return nil
}
// giteaMCPURL is the public Gitea MCP endpoint (OAuth via Dex/Authentik). Gitea
// is the audit-trail invariant of the consolidated single harness (#75), so every
// mode lists it as an available connection.
const giteaMCPURL = "https://git-mcp.d-ma.be/mcp"
func brainEntry(brainURL string) map[string]any {
return map[string]any{
"url": brainURL + "/mcp",
"description": "Brain MCP — knowledge query, write, ingestion, session log",
}
}
func giteaEntry() map[string]any {
return map[string]any{
"url": giteaMCPURL,
"description": "Gitea MCP — issues/PRs/repo ops (audit-trail invariant)",
}
}
func modeCloud(brainURL string) map[string]any {
return map[string]any{
"mcpServers": map[string]any{
"brain": map[string]any{
"url": brainURL + "/mcp",
"description": "Brain MCP — knowledge query, write, ingestion, session log",
},
"brain": brainEntry(brainURL),
"gitea": giteaEntry(),
},
}
}
func modeClientLocal(brainURL string) map[string]any {
// The routing pod is iceboxed (#75); the consolidated harness is brain + gitea.
return map[string]any{
"mcpServers": map[string]any{
"brain": map[string]any{
"url": brainURL + "/mcp",
"description": "Brain MCP — knowledge query, write, ingestion, session log",
},
"routing": map[string]any{
"url": "http://koala:30310/mcp",
"description": "Mode 2 routing pod — routes skill calls to LiteLLM/local",
"headers": map[string]any{
"X-Hyperguild-Mode": "client-local",
},
},
"brain": brainEntry(brainURL),
"gitea": giteaEntry(),
},
}
}
@@ -92,10 +101,8 @@ func modeSovereign(brainURL string) map[string]any {
return map[string]any{
"_mode_note": "Sovereign mode primarily uses Crush + LiteLLM. This .mcp.json is provided as Claude Code fallback (e.g. emergency offline editing).",
"mcpServers": map[string]any{
"brain": map[string]any{
"url": brainURL + "/mcp",
"description": "Brain MCP — knowledge query, write, ingestion, session log",
},
"brain": brainEntry(brainURL),
"gitea": giteaEntry(),
},
}
}
+11 -17
View File
@@ -35,11 +35,13 @@ func TestRunMode_Cloud_Default(t *testing.T) {
servers, ok := got["mcpServers"].(map[string]any)
require.True(t, ok, "mcpServers must be a JSON object")
assert.Contains(t, servers, "brain")
assert.Contains(t, servers, "gitea")
assert.NotContains(t, servers, "routing")
assert.NotContains(t, got, "_mode_note")
}
func TestRunMode_ClientLocal_HasRoutingEntry(t *testing.T) {
func TestRunMode_ClientLocal_NoRouting_HasGitea(t *testing.T) {
// #75: the routing pod is iceboxed; the consolidated harness is brain + gitea.
dir := t.TempDir()
outPath := filepath.Join(dir, ".mcp.json")
t.Setenv("BRAIN_URL", "http://koala:30330")
@@ -51,17 +53,11 @@ func TestRunMode_ClientLocal_HasRoutingEntry(t *testing.T) {
got := readJSON(t, outPath)
servers := got["mcpServers"].(map[string]any)
require.Contains(t, servers, "brain")
require.Contains(t, servers, "routing")
routing := servers["routing"].(map[string]any)
assert.NotContains(t, routing, "_routing_pending", "placeholder should be removed once Plan 6 ships")
headers, ok := routing["headers"].(map[string]any)
require.True(t, ok, "routing entry should have headers block")
assert.Equal(t, "client-local", headers["X-Hyperguild-Mode"])
require.Contains(t, servers, "gitea")
assert.NotContains(t, servers, "routing", "routing pod iceboxed (#75)")
}
func TestModeClientLocalHasRoutingHeader(t *testing.T) {
func TestModeGiteaEntryPresentAndWellFormed(t *testing.T) {
tmp := t.TempDir() + "/mcp.json"
out := &bytes.Buffer{}
stderr := &bytes.Buffer{}
@@ -73,13 +69,10 @@ func TestModeClientLocalHasRoutingHeader(t *testing.T) {
require.NoError(t, json.Unmarshal(body, &doc))
servers := doc["mcpServers"].(map[string]any)
routing := servers["routing"].(map[string]any)
assert.Equal(t, "http://koala:30310/mcp", routing["url"])
assert.NotContains(t, routing, "_routing_pending", "placeholder should be removed once Plan 6 ships")
headers, ok := routing["headers"].(map[string]any)
require.True(t, ok, "routing entry should have headers block")
assert.Equal(t, "client-local", headers["X-Hyperguild-Mode"])
require.NotContains(t, servers, "routing")
gitea, ok := servers["gitea"].(map[string]any)
require.True(t, ok, "gitea entry must be present (audit-trail invariant)")
assert.Equal(t, "https://git-mcp.d-ma.be/mcp", gitea["url"])
}
func TestRunMode_Sovereign_HasModeNote(t *testing.T) {
@@ -94,6 +87,7 @@ func TestRunMode_Sovereign_HasModeNote(t *testing.T) {
assert.Contains(t, got, "_mode_note")
servers := got["mcpServers"].(map[string]any)
assert.Contains(t, servers, "brain")
assert.Contains(t, servers, "gitea")
assert.NotContains(t, servers, "routing")
}
-170
View File
@@ -1,170 +0,0 @@
package main
// The internal/skills/{debug,retrospective,review,trainer} packages imported
// below are also imported by cmd/supervisor. Plan 7 (supervisor retirement)
// MUST NOT delete these four packages — the routing pod is their second
// consumer. Plan 7 deletes only internal/skills/{tdd,spec,tier} (the skills
// that don't route to local), the supervisor binary, and supervisor manifests.
// See docs/superpowers/specs/2026-05-04-mode-2-routing-pod-design.md (Constraints).
import (
"context"
"log/slog"
"net/http"
"os"
"time"
"github.com/mathiasbq/supervisor/internal/auth"
"github.com/mathiasbq/supervisor/internal/config"
iexec "github.com/mathiasbq/supervisor/internal/exec"
"github.com/mathiasbq/supervisor/internal/githubclient"
"github.com/mathiasbq/supervisor/internal/mcp"
"github.com/mathiasbq/supervisor/internal/mcpclient"
"github.com/mathiasbq/supervisor/internal/registry"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/mathiasbq/supervisor/internal/skills/debug"
"github.com/mathiasbq/supervisor/internal/skills/project"
"github.com/mathiasbq/supervisor/internal/skills/retrospective"
"github.com/mathiasbq/supervisor/internal/skills/review"
"github.com/mathiasbq/supervisor/internal/skills/trainer"
)
func main() {
logger := slog.New(slog.NewTextHandler(os.Stderr, nil))
slog.SetDefault(logger)
cfg, err := config.LoadRouting()
if err != nil {
logger.Error("config load failed", "err", err)
os.Exit(1)
}
configDir := envOr("SUPERVISOR_CONFIG_DIR", "/app/config/supervisor")
mustRead := func(path string) string {
b, err := os.ReadFile(configDir + "/" + path)
if err != nil {
logger.Error("read prompt failed", "path", path, "err", err)
os.Exit(1)
}
return string(b)
}
llm := iexec.NewLiteLLM(cfg.LiteLLMBaseURL, cfg.LiteLLMAPIKey, 0)
router := &routing.Router{
Fetcher: routing.NewFetcher(cfg.BrainURL, "7d", time.Duration(cfg.PassRateTTLSeconds)*time.Second),
Logger: routing.NewLogger(cfg.BrainURL),
Policy: routing.Policy{Floor: cfg.RouteLocalFloor, Ceil: cfg.RouteLocalCeil},
FastModel: cfg.FastModel,
ThinkingModel: cfg.ThinkingModel,
Complete: llm.Complete,
}
// Skill packages call CompleteFunc(ctx, model, system, user) — no session_id
// or project_root in the signature. Rather than modifying every skill's API
// (and inflating Plan 6's blast radius), the routing pod logs every decision
// under a fixed session_id "_routing". Operators query
// `GET /pass-rate?skill=_routing&window=...` to inspect routing health.
const routingSessionID = "_routing"
wrap := func(skillName string) routing.CompleteFunc {
return func(ctx context.Context, _, system, user string) (string, int64, error) {
// The model param is ignored: the router picks the model based on policy.
return router.Run(ctx, routing.RunInput{
Skill: skillName,
System: system,
User: user,
SessionID: routingSessionID,
ProjectRoot: "",
})
}
}
reg := registry.New()
reg.Register(review.New(review.Config{
SkillPrompt: mustRead("review.md"),
DefaultModel: cfg.FastModel,
CompleteFunc: review.CompleteFunc(wrap("review")),
}))
reg.Register(debug.New(debug.Config{
SkillPrompt: mustRead("debug.md"),
DefaultModel: cfg.FastModel,
CompleteFunc: debug.CompleteFunc(wrap("debug")),
}))
reg.Register(retrospective.New(retrospective.Config{
SkillPrompt: mustRead("retrospective.md"),
DefaultModel: cfg.FastModel,
CompleteFunc: retrospective.CompleteFunc(wrap("retrospective")),
}))
reg.Register(trainer.New(trainer.Config{
ReaderPrompt: mustRead("trainer-reader.md"),
WriterPrompt: mustRead("trainer-writer.md"),
DefaultModel: cfg.FastModel,
CompleteFunc: trainer.CompleteFunc(wrap("trainer")),
}))
if cfg.GiteaMCPURL != "" {
mcpC, err := mcpclient.New(cfg.GiteaMCPURL, cfg.GiteaMCPToken)
if err != nil {
logger.Error("mcpclient init for project_create — GITEA_MCP_URL is set but GITEA_MCP_TOKEN is empty (check routing-secrets)", "err", err)
os.Exit(1)
}
var ghClient *githubclient.Client
if cfg.GitHubPAT != "" {
ghClient = githubclient.New(cfg.GitHubPAT)
}
reg.Register(project.New(project.Config{
Client: mcpC,
GitHub: ghClient,
GiteaOwner: cfg.GiteaOwner,
GitHubOwner: cfg.GitHubOwner,
GitHubPAT: cfg.GitHubPAT,
InfraRepo: cfg.InfraRepo,
}))
logger.Info("project_create registered", "gitea_mcp_url", cfg.GiteaMCPURL,
"gitea_owner", cfg.GiteaOwner, "github_owner", cfg.GitHubOwner,
"infra_repo", cfg.InfraRepo, "github_pat_set", cfg.GitHubPAT != "")
} else {
logger.Info("project_create skipped — GITEA_MCP_URL not set")
}
var validator *auth.Validator
if dexURL := os.Getenv("DEX_ISSUER_URL"); dexURL != "" {
audience := os.Getenv("MCP_AUDIENCE")
v, err := auth.NewValidator(dexURL, audience)
if err != nil {
logger.Error("build jwt validator", "err", err)
os.Exit(1)
}
validator = v
logger.Info("jwt auth enabled", "issuer", dexURL)
}
srv := mcp.NewServer(reg, cfg.MCPAuthToken, validator)
mux := http.NewServeMux()
mux.Handle("/mcp", srv)
mux.HandleFunc("/healthz", func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
})
if dexURL := os.Getenv("DEX_ISSUER_URL"); dexURL != "" {
resourceURL := os.Getenv("MCP_RESOURCE_URL")
mux.HandleFunc("GET /.well-known/oauth-protected-resource",
auth.ProtectedResourceHandler(resourceURL, dexURL))
}
addr := ":" + cfg.Port
logger.Info("routing pod starting", "addr", addr,
"fast", cfg.FastModel, "thinking", cfg.ThinkingModel,
"floor", cfg.RouteLocalFloor, "ceil", cfg.RouteLocalCeil)
if err := http.ListenAndServe(addr, mux); err != nil { //nolint:gosec
logger.Error("server stopped", "err", err)
os.Exit(1)
}
}
func envOr(key, def string) string {
if v := os.Getenv(key); v != "" {
return v
}
return def
}
-135
View File
@@ -1,135 +0,0 @@
package main_test
import (
"context"
"encoding/json"
"io"
"net"
"net/http"
"net/http/httptest"
"os"
"os/exec"
"strconv"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestRoutingPodEndToEnd boots the binary against fake LiteLLM + brain servers,
// calls tools/list and one tools/call, and verifies the brain saw a session_log POST.
func TestRoutingPodEndToEnd(t *testing.T) {
if testing.Short() {
t.Skip("end-to-end binary boot")
}
var brainHits int
llm := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{
"choices": []map[string]any{{"message": map[string]any{"role": "assistant", "content": "stub"}}},
})
}))
defer llm.Close()
brain := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/pass-rate":
brainHits++
_ = json.NewEncoder(w).Encode(map[string]any{"pass_rate": 0.95})
case "/mcp":
brainHits++
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{}})
}
}))
defer brain.Close()
port := freePort(t)
addr := "127.0.0.1:" + port
baseURL := "http://" + addr
bin := buildRouting(t)
cmd := exec.Command(bin)
cmd.Env = []string{
"ROUTING_PORT=" + port,
"LITELLM_BASE_URL=" + llm.URL,
"LITELLM_API_KEY=stub",
"BRAIN_URL=" + brain.URL,
"SUPERVISOR_CONFIG_DIR=../../config/supervisor",
"PATH=" + os.Getenv("PATH"),
"HOME=" + os.Getenv("HOME"),
}
require.NoError(t, cmd.Start())
t.Cleanup(func() { _ = cmd.Process.Kill() })
require.NoError(t, waitForPort(t, addr, 30*time.Second))
resp := mcpCall(t, baseURL+"/mcp", `{"jsonrpc":"2.0","id":1,"method":"tools/list"}`)
assert.Contains(t, resp, `"review"`)
assert.Contains(t, resp, `"debug"`)
assert.Contains(t, resp, `"retrospective"`)
assert.Contains(t, resp, `"trainer"`)
resp = mcpCall(t, baseURL+"/mcp", `{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"review","arguments":{"project_root":"/tmp","files":["README.md"]}}}`)
_ = resp // shape varies by skill; we only need a 200
// Wait briefly for the async session_log to land.
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) && brainHits < 2 {
time.Sleep(50 * time.Millisecond)
}
assert.GreaterOrEqual(t, brainHits, 2, "expected at least one /pass-rate hit and one /mcp session_log hit")
}
func buildRouting(t *testing.T) string {
t.Helper()
bin := t.TempDir() + "/routing"
out, err := exec.Command("go", "build", "-o", bin, "github.com/mathiasbq/supervisor/cmd/routing").CombinedOutput()
require.NoError(t, err, "build failed: %s", out)
return bin
}
func waitForPort(_ *testing.T, addr string, dur time.Duration) error {
deadline := time.Now().Add(dur)
for time.Now().Before(deadline) {
c, err := http.Get("http://" + addr + "/healthz") //nolint:noctx
if err == nil {
_ = c.Body.Close()
return nil
}
conn, err := http.NewRequest(http.MethodPost, "http://"+addr+"/mcp", strings.NewReader(`{}`))
if err == nil {
r, err := http.DefaultClient.Do(conn)
if err == nil {
_ = r.Body.Close()
return nil
}
}
time.Sleep(50 * time.Millisecond)
}
return context.DeadlineExceeded
}
func mcpCall(t *testing.T, url, body string) string {
t.Helper()
r, err := http.Post(url, "application/json", strings.NewReader(body)) //nolint:noctx
require.NoError(t, err)
defer func() { _ = r.Body.Close() }()
raw, err := io.ReadAll(r.Body)
require.NoError(t, err)
return string(raw)
}
// freePort grabs an OS-assigned TCP port and releases it. There is a small
// race window before the subprocess re-binds it, but it is acceptable for
// test isolation against a hardcoded port colliding with another test or
// stray process.
func freePort(t *testing.T) string {
t.Helper()
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
port := l.Addr().(*net.TCPAddr).Port
require.NoError(t, l.Close())
return strconv.Itoa(port)
}
@@ -0,0 +1,104 @@
# Runbook: exercising review/debug traffic to fill the pass-rate dataset
**Why this exists:** the routing pod's local-vs-cloud decision is gated on a
pass-rate history that only accrues from real `review`/`debug` invocations
**through the pod**. Until the dataset has data, the fast (local) path never
activates and the core hypothesis (hyperguild #35) can't be validated. This
runbook is how you spin that flywheel.
## The one trap
Pass-rate accrues **only** when a skill tool is called via the routing pod's MCP
endpoint. These look like they should count but **do not**:
- **Crush** — talks to LiteLLM directly, bypasses the pod. No log.
- **claude.ai web / Claude Desktop without the connector** — no log.
- **Running the local `code-review` / `debug` skills** (`~/dev/.skills`) inline in
a Claude Code session — those are local skills, not the pod's MCP tools. No log.
Only a `tools/call` to the routing pod records a pass/fail.
## Endpoints
| Purpose | URL | Auth |
|---------|-----|------|
| Routing MCP (local, Tailscale) | `http://koala:30310/mcp` | Bearer `ROUTING_MCP_TOKEN` |
| Routing MCP (remote) | `https://routing-mcp.d-ma.be/mcp` | OAuth via `auth.d-ma.be` (audience `claude-ai`) |
| Pass-rate readout | `http://koala:30330/pass-rate?skill=<name>` | none (read-only) |
Tools advertised: **`review`**, **`debug`** (the two the #35 gate measures),
plus `session_log`, `retrospective`, `trainer`.
## Step 1 — connect the routing pod as an MCP server
**Local** (needs the bearer token; keep it out of argv via 1Password):
```bash
op run --env-file ~/.op-env -- \
claude mcp add routing --transport http http://koala:30310/mcp \
--header "Authorization: Bearer $ROUTING_MCP_TOKEN"
```
**Remote** (claude.ai / Claude Desktop): add a custom connector pointing at
`https://routing-mcp.d-ma.be/mcp`; it completes OAuth against `auth.d-ma.be`,
no static token.
Verify: a `tools/list` should return `review`, `debug`, `session_log`,
`retrospective`, `trainer`.
## Step 2 — route real work through it
In normal sessions, invoke the pod's tools instead of reviewing/debugging inline:
- *"Use the **routing** `review` tool on this diff."*
- *"**debug** this failure through the routing pod."*
Each call logs an outcome to ingestion → `/pass-rate` ticks up.
## Step 3 — how routing actually picks the model
Per `internal/routing/policy.go`:
1. pass-rate `nil` (cold) → **local** fast tier. The router defaults to local
from invocation #1, not to cloud — so the fast tier is exercised immediately.
2. pass-rate `>= 0.90` (floor) → **local**; `< 0.70` (ceil) → **cloud/thinking**;
in the `[0.70, 0.90)` band a request-hash bit samples 50/50.
3. On a local execution error the router falls open to the thinking model for
that one call (logged `thinking_fallback`).
So you are not "paying in on cloud" — cold calls already run on the (validated)
local fast tier **`koala/qwen36-35b-a3b`** (Qwen3.6-35B-A3B MTP, promoted
2026-06-29, infra `c66a195`, `HYPERGUILD_FAST_MODEL`). Accumulating passes just
keeps it there once real pass-rate is computed.
> **Instrumentation note (#73, fixed 2026-06-30):** until v0.11.1 the pod logged
> successes as `"skip"` (not `"pass"`), under `skill:"_routing"`, via an
> unauthenticated POST that silently 401'd — so `/pass-rate` stayed at zero no
> matter how much you used it. That's fixed and verified (a real review call now
> moves `/pass-rate?skill=review` 0→1). If you see traffic not registering,
> re-check #73's three failure modes first.
## Target & verification
- **50 logged invocations** across `review` + `debug` within the 14-day window.
The clock restarts **2026-06-30** (the day instrumentation was verified working;
the original 2026-06-26→07-10 window measured broken plumbing) → **kill-date
2026-07-14**, ~4 calls/day (1 already logged from the #73 smoke test).
- Check progress anytime:
```bash
curl -s "http://koala:30330/pass-rate?skill=review"
curl -s "http://koala:30330/pass-rate?skill=debug"
```
- If ~45/day isn't realistic alongside Crush, that is **not** a failure — per
#35 deliverable #1 it's the signal hyperguild isn't on the work critical path,
and the pre-decided **Berget fallback** (`gpt-oss-120b` / `qwen3-32b`) carries
the fast tier instead.
## Refs
- hyperguild #35 — the validation issue (data gate = deliverable #1)
- `docs/multi-model-routing.md` — routing policy
- brain: `wiki/homelab/hypotheses/qwen36-35b-a3b-fast-model-experiment-2026-05-28.md`
- infra `c66a195` — qwen36 promotion; `models.yml` / `llama-swap-configmap.yaml`
+165 -14
View File
@@ -8,15 +8,21 @@ import (
"net/http"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
chassisauth "gitea.d-ma.be/mathias/mcp-chassis/auth"
chassisauth "git.d-ma.be/mathias/mcp-chassis/auth"
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/mathiasbq/hyperguild/ingestion/internal/claudewatcher"
"github.com/mathiasbq/hyperguild/ingestion/internal/embed"
"github.com/mathiasbq/hyperguild/ingestion/internal/gitea"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphsync"
"github.com/mathiasbq/hyperguild/ingestion/internal/llm"
@@ -28,12 +34,33 @@ import (
"github.com/mathiasbq/hyperguild/ingestion/internal/search"
"github.com/mathiasbq/hyperguild/ingestion/internal/vectorstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/watcher"
"github.com/mathiasbq/hyperguild/ingestion/internal/webhook"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
"k8s.io/client-go/tools/clientcmd"
)
// claudeSink converts each claudewatcher.Batch into one wiki note under
// brain/wiki/claude-sessions/facts/. v1 emits one note per session
// keyed by host + session id; classifier-driven hall routing is a
// follow-up (hyperguild#27 v2).
// kubeClient builds an in-cluster Kubernetes client (falls back to
// $KUBECONFIG for local dev/testing against a real cluster).
func kubeClient() (kubernetes.Interface, error) {
if cfg, err := rest.InClusterConfig(); err == nil {
return kubernetes.NewForConfig(cfg)
}
rules := clientcmd.NewDefaultClientConfigLoadingRules()
cc := clientcmd.NewNonInteractiveDeferredLoadingClientConfig(rules, &clientcmd.ConfigOverrides{})
cfg, err := cc.ClientConfig()
if err != nil {
return nil, fmt.Errorf("kube config (no in-cluster, no kubeconfig): %w", err)
}
return kubernetes.NewForConfig(cfg)
}
// claudeSink converts each claudewatcher.Batch into a raw session dump
// under brain/archive/claude-sessions/<host>/. Deliberately NOT a wiki
// note (api.WriteNote / brain/wiki/) — raw full transcripts out-ranked
// curated ai-sessions summaries in BM25 (177,818 vs 45,335 on the same
// query) and duplicated content already summarized elsewhere. Kept for
// deep lookups, never indexed. See ai-sessions#10.
type claudeSink struct {
brainDir string
logger *slog.Logger
@@ -61,16 +88,14 @@ func (s *claudeSink) Ingest(ctx context.Context, b claudewatcher.Batch) error {
sb.WriteString("\n\n")
}
slug := "session-" + b.Host + "-" + b.SessionID
if _, err := api.WriteNote(s.brainDir, api.WriteNoteOptions{
Filename: slug,
Wing: "claude-sessions",
Hall: "facts",
Type: "source",
Domain: b.ProjectID,
Content: sb.String(),
}); err != nil {
return fmt.Errorf("write claude session note: %w", err)
dest := filepath.Join(s.brainDir, "archive", "claude-sessions", b.Host, slug+".md")
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
return fmt.Errorf("create claude-sessions archive dir: %w", err)
}
if err := os.WriteFile(dest, []byte(sb.String()), 0o644); err != nil {
return fmt.Errorf("write claude session archive: %w", err)
}
s.logger.Debug("claude session archived (non-indexed)", "path", dest)
return nil
}
@@ -118,6 +143,47 @@ func envInt(key string, fallback int) int {
return fallback
}
// buildAuditSink selects the capture audit sink. When BRAIN_LOKI_URL is
// set it builds the classification-aware DegradingSink (loki central +
// durable file buffer + optional ntfy) and starts the reconcile loop;
// otherwise it falls back to a plain slog sink. The buffer lives under the
// brain dir so it survives process restarts.
func buildAuditSink(ctx context.Context, brainDir string, logger *slog.Logger) capture.AuditSink {
lokiURL := os.Getenv("BRAIN_LOKI_URL")
central := audit.NewLokiCentral(lokiURL)
if central == nil {
logger.Info("capture audit: slog sink (BRAIN_LOKI_URL unset)")
return audit.NewSlogSink(logger)
}
buffer, err := audit.NewFileBuffer(filepath.Join(brainDir, ".audit-buffer", "capture.jsonl"))
if err != nil {
logger.Error("capture audit buffer init", "err", err)
os.Exit(1)
}
// Keep notifier as a nil interface (not a typed-nil) when unconfigured
// so DegradingSink/Reconcile skip it cleanly.
var notifier audit.Notifier
if n := audit.NewNtfyNotifier(os.Getenv("BRAIN_NTFY_URL"), os.Getenv("BRAIN_NTFY_TOKEN")); n != nil {
notifier = n
}
reconcileInterval := time.Duration(envInt("BRAIN_AUDIT_RECONCILE_INTERVAL", 60)) * time.Second
audit.StartReconcile(ctx, central, buffer, notifier, reconcileInterval)
logger.Info("capture audit: loki+buffer sink", "loki", lokiURL, "reconcile_s", int(reconcileInterval.Seconds()))
return audit.NewDegradingSink(central, buffer, notifier)
}
// splitList parses a comma-separated env value into a trimmed,
// empty-free slice. Used for the capture sovereign-principal allowlist.
func splitList(v string) []string {
var out []string
for _, p := range strings.Split(v, ",") {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
// systemHostname returns os.Hostname() with a "unknown" fallback so the
// caller never has to handle the rare error path.
func systemHostname() string {
@@ -175,6 +241,15 @@ func main() {
logger.Info("brain reranker configured", "url", rerankURL, "model", rerankModel)
}
// Gitea ticket tracker for the capture capability (#52). Token via env
// only — never logged or in argv. Both vars must be set to enable it;
// gitea.New returns nil otherwise, leaving ticket integration off.
giteaURL := envOr("BRAIN_GITEA_URL", "https://git.d-ma.be")
if tracker := gitea.New(giteaURL, os.Getenv("BRAIN_GITEA_TOKEN")); tracker != nil {
mcpSrv = mcpSrv.WithIssueTracker(tracker)
logger.Info("brain gitea tracker configured", "url", giteaURL)
}
// Hybrid retrieval (pgvector + nomic-embed-text). Both env vars must
// be set together for the path to wire on; otherwise BM25-only.
var vectorStore *vectorstore.PGStore
@@ -256,6 +331,19 @@ func main() {
logger.Error("CLAUDE_SESSIONS_DIR set but BRAIN_PG_DSN missing — claudewatcher needs the cursor table")
os.Exit(1)
}
// Client-name guard. The env value is a regex alternation
// (e.g. "SEB|Mastercard"); we wrap it with word boundaries
// and case-insensitive flag so substrings inside longer
// identifiers don't false-match. Sourced from a SOPS secret
// so client identities never live in source.
if clientBlock := os.Getenv("CLAUDE_INGEST_CLIENT_BLOCK"); clientBlock != "" {
pattern := `(?i)\b(` + clientBlock + `)\b`
if err := claudewatcher.RegisterRule("client-name", pattern); err != nil {
logger.Error("claudewatcher client-block rule invalid", "err", err)
os.Exit(1)
}
logger.Info("claudewatcher client-block guard registered")
}
cursorStore, cerr := claudewatcher.NewCursorStore(ctx, pgDSN)
if cerr != nil {
logger.Error("claudewatcher cursor init", "err", cerr)
@@ -283,6 +371,29 @@ func main() {
logger.Info("claudewatcher started",
"sessions_dir", claudeDir, "host", host, "interval", interval)
}
// Gitea push webhook -> on-demand brain-sync Job, instead of waiting up
// to 15 minutes for the next CronJob poll. Off by default (opt in via
// GITEA_WEBHOOK_SECRET) since it needs Job-create RBAC in the "brain"
// namespace that a fresh deploy won't have granted yet.
var webhookHandler *webhook.Handler
if webhookSecret := os.Getenv("GITEA_WEBHOOK_SECRET"); webhookSecret != "" {
kc, err := kubeClient()
if err != nil {
logger.Error("brain-sync webhook: kube client", "err", err)
os.Exit(1)
}
webhookHandler = &webhook.Handler{
Secret: webhookSecret,
Clientset: kc,
Namespace: envOr("BRAIN_SYNC_NAMESPACE", "brain"),
CronJobName: envOr("BRAIN_SYNC_CRONJOB", "brain-sync"),
WatchRepo: envOr("BRAIN_SYNC_WATCH_REPO", "mathias/brain"),
Logger: logger,
}
logger.Info("brain-sync webhook enabled", "namespace", webhookHandler.Namespace, "cronjob", webhookHandler.CronJobName)
}
if vectorStore != nil {
embedSyncInterval := envInt("BRAIN_EMBED_SYNC_INTERVAL", 300)
vectorstore.StartSync(ctx, brainDir, vectorStore,
@@ -300,8 +411,13 @@ func main() {
mux.HandleFunc("POST /ingest-path", h.IngestPath)
mux.HandleFunc("POST /ingest-raw", h.IngestRaw)
mux.HandleFunc("POST /backfill-refs", h.BackfillRefs)
mux.HandleFunc("GET /pending", h.Pending)
mux.HandleFunc("POST /promote", h.Promote)
mux.HandleFunc("POST /backfill-embeddings", h.BackfillEmbeddings)
mux.HandleFunc("GET /pass-rate", h.PassRate)
if webhookHandler != nil {
mux.Handle("POST /webhooks/brain-sync", webhookHandler)
}
jwtValidator, err := chassisauth.NewJWTValidator(ctx, os.Getenv("DEX_ISSUER_URL"), os.Getenv("MCP_AUDIENCE"))
if err != nil {
logger.Error("build jwt validator", "err", err)
@@ -326,6 +442,41 @@ func main() {
mux.Handle("/mcp", chassisauth.BearerMiddleware(mcpToken, jwtValidator, "brain", resourceMetadataURL, mcpSrv))
// POST /capture (#53/#54): the uniform capture REST door. Needs a ticket
// tracker to file action items, so it only mounts when Gitea is
// configured. It reuses the MCP server's graph-wired brain store (one
// implementation), the classification tags for the I1 gate, and a
// classification-aware audit sink (loki + durable buffer + ntfy when
// BRAIN_LOKI_URL is set, else a plain slog sink). The handler does its
// own auth (static + JWT) because it needs the principal to derive the
// trust-zone origin — the chassis middleware hides it.
if tracker := mcpSrv.IssueTracker(); tracker != nil {
classCfg, cerr := classification.Load(brainDir)
if cerr != nil {
logger.Error("load classification config", "err", cerr)
os.Exit(1)
}
auditSink := buildAuditSink(ctx, brainDir, logger)
// The Gitea client also satisfies SummaryWriter (#66): session
// summaries are written to mathias/ai-sessions over the same API
// token. nil only if a future tracker impl lacks file writes.
summaryWriter, _ := tracker.(capture.SummaryWriter)
captureSvc := capture.NewService(
mcpSrv.BrainStore(), tracker, summaryWriter, classCfg, auditSink)
sovereign := splitList(os.Getenv("BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS"))
resolver := capturehttp.NewOriginResolver(sovereign)
captureH := capturehttp.New(captureSvc, jwtValidator, mcpToken, "local-cli", resolver)
mux.Handle("POST /capture", captureH)
// Same use-case behind the MCP `capture` tool (#55 relay) so MCP-native
// harnesses (claude.ai, Crush, Pi, LLM Council) reach capture through
// the existing /mcp OAuth connector. mcpSrv is already wrapped above;
// WithCapture mutates the same instance, so the tool appears live.
mcpSrv.WithCapture(captureSvc, jwtValidator, mcpToken, "local-cli", resolver)
logger.Info("capture enabled (REST + MCP tool)", "sovereign_principals", len(sovereign))
} else {
logger.Info("capture endpoint disabled (BRAIN_GITEA_TOKEN unset)")
}
// Opt-in OAuth 2.0 client_credentials flow for claude.ai's custom-MCP
// integration UI, which has no static-Bearer field. Setting both
// OAUTH_CLIENT_ID and OAUTH_CLIENT_SECRET enables the token exchange;
+38
View File
@@ -0,0 +1,38 @@
// ingestion/cmd/server/main_test.go
package main
import (
"context"
"log/slog"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/mathiasbq/hyperguild/ingestion/internal/claudewatcher"
)
func TestClaudeSink_IngestWritesToNonIndexedArchiveNotWiki(t *testing.T) {
dir := t.TempDir()
sink := &claudeSink{brainDir: dir, logger: slog.New(slog.NewTextHandler(os.Stderr, nil))}
err := sink.Ingest(context.Background(), claudewatcher.Batch{
Host: "koala",
FilePath: "/host-home-claude/projects/-home-mathias-dev/abc.jsonl",
SessionID: "abc",
ProjectID: "-home-mathias-dev",
Turns: []claudewatcher.Turn{
{Type: "assistant", Content: "did a thing"},
},
})
require.NoError(t, err)
got, err := os.ReadFile(filepath.Join(dir, "archive", "claude-sessions", "koala", "session-koala-abc.md"))
require.NoError(t, err, "raw session dump must land in the non-indexed archive")
assert.Contains(t, string(got), "did a thing")
_, err = os.Stat(filepath.Join(dir, "wiki", "claude-sessions"))
assert.True(t, os.IsNotExist(err), "raw transcripts must never land under wiki/ (ai-sessions#10 — BM25 pollution)")
}
+49 -6
View File
@@ -3,29 +3,72 @@ module github.com/mathiasbq/hyperguild/ingestion
go 1.26.1
require (
github.com/lestrrat-go/jwx/v2 v2.1.6
github.com/stretchr/testify v1.11.1
k8s.io/api v0.31.3
k8s.io/apimachinery v0.31.3
k8s.io/client-go v0.31.3
)
require (
gitea.d-ma.be/mathias/mcp-chassis v0.1.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.4 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/imdario/mergo v0.3.6 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/lestrrat-go/jwx/v2 v2.1.6 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/rogpeppe/go-internal v1.15.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/x448/float16 v0.8.4 // indirect
golang.org/x/net v0.26.0 // indirect
golang.org/x/oauth2 v0.21.0 // indirect
golang.org/x/term v0.28.0 // indirect
golang.org/x/time v0.3.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 // indirect
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)
require (
git.d-ma.be/mathias/mcp-chassis v0.2.0
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect
github.com/jackc/pgx/v5 v5.9.2
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/lestrrat-go/blackmagic v1.0.3 // indirect
github.com/lestrrat-go/httpcc v1.0.1 // indirect
github.com/lestrrat-go/httprc v1.0.6 // indirect
github.com/lestrrat-go/iter v1.0.2 // indirect
github.com/lestrrat-go/option v1.0.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/segmentio/asm v1.2.0 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/sys v0.31.0 // indirect
golang.org/x/text v0.29.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gopkg.in/yaml.v3 v3.0.1
)
+143 -5
View File
@@ -1,12 +1,47 @@
gitea.d-ma.be/mathias/mcp-chassis v0.1.0 h1:8RXO34+n7Vu8HnUMagars6fc4oemqRpMu7MVtjaj4qY=
gitea.d-ma.be/mathias/mcp-chassis v0.1.0/go.mod h1:ajbLlwr2L7FAN3TBU39KucZkKJM02wTbKbDKDEW2YvE=
git.d-ma.be/mathias/mcp-chassis v0.2.0 h1:6fLmb7xqRa2nNVWsHaUbbfbArgDXJw/gDhb09clBIjo=
git.d-ma.be/mathias/mcp-chassis v0.2.0/go.mod h1:Ks7EK2UnGAN0H3rJjKUxUagX8/ZBdtLrOlcUbv0RwH8=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc=
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40=
github.com/emicklei/go-restful/v3 v3.11.0 h1:rAQeMHw1c7zTmncogyy8VvRZwtkmkZ4FxERmMY4rD+g=
github.com/emicklei/go-restful/v3 v3.11.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc=
github.com/fxamacker/cbor/v2 v2.7.0 h1:iM5WgngdRBanHcxugY4JySA0nk1wZorNOpTgCMedv5E=
github.com/fxamacker/cbor/v2 v2.7.0/go.mod h1:pxXPTn3joSm21Gbwsv0w9OSA2y1HFR9qXEeXQVeNoDQ=
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE=
github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs=
github.com/go-openapi/jsonreference v0.20.2 h1:3sVjiK66+uXK/6oQ8xgcRKcFgQ5KXa2KvnJRumpMGbE=
github.com/go-openapi/jsonreference v0.20.2/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k=
github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-openapi/swag v0.22.4 h1:QLMzNJnMGPRNDCbySlcj1x01tzU8/9LTTL9hZZZogBU=
github.com/go-openapi/swag v0.22.4/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA=
github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/gnostic-models v0.6.8 h1:yo/ABAfM5IMRsS1VnXjTBvUb61tFIHozhlYvRgGre9I=
github.com/google/gnostic-models v0.6.8/go.mod h1:5n7qKqH0f5wFt+aWF8CW6pZLLNOfYuF5OpfBSENuI8U=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20240525223248-4bfdf5a9a2af h1:kmjWCqn2qkEml422C2Rrd27c3VGxi6a/6HNq8QmHRKM=
github.com/google/pprof v0.0.0-20240525223248-4bfdf5a9a2af/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
@@ -15,6 +50,19 @@ github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw=
github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/lestrrat-go/blackmagic v1.0.3 h1:94HXkVLxkZO9vJI/w2u1T0DAoprShFd13xtnSINtDWs=
github.com/lestrrat-go/blackmagic v1.0.3/go.mod h1:6AWFyKNNj0zEXQYfTMPfZrAXUWUfTIZ5ECEUEJaijtw=
github.com/lestrrat-go/httpcc v1.0.1 h1:ydWCStUeJLkpYyjLDHihupbn2tYmZ7m22BGkcvZZrIE=
@@ -27,28 +75,118 @@ github.com/lestrrat-go/jwx/v2 v2.1.6 h1:hxM1gfDILk/l5ylers6BX/Eq1m/pnxe9NBwW6lVf
github.com/lestrrat-go/jwx/v2 v2.1.6/go.mod h1:Y722kU5r/8mV7fYDifjug0r8FK8mZdw0K0GpJw/l8pU=
github.com/lestrrat-go/option v1.0.1 h1:oAzP2fvZGQKWkvHa1/SAcFolBEca1oN+mQ7eooNBEYU=
github.com/lestrrat-go/option v1.0.1/go.mod h1:5ZHFbivi4xwXxhxY9XHDe2FHo6/Z7WWmtT7T5nBBp3I=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.19.0 h1:9Cnnf7UHo57Hy3k6/m5k3dRfGTMXGvxhHFvkDTCTpvA=
github.com/onsi/ginkgo/v2 v2.19.0/go.mod h1:rlwLi9PilAFJ8jCg9UE1QP6VBpd6/xj3SRC0d6TU0To=
github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw=
github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.15.0 h1:D0RCU5rMAp+SpgkiNdrjfJ+LX4J1M32V2NeCY7EJ6hc=
github.com/rogpeppe/go-internal v1.15.0/go.mod h1:DrUVZyrJU+txYW5/1kwtXQSMFio52ZOxX7yM1VHvnxs=
github.com/segmentio/asm v1.2.0 h1:9BQrFxC+YOHJlTlHGkTrFWf59nbL3XnCoFLTwDCI7ys=
github.com/segmentio/asm v1.2.0/go.mod h1:BqMnlJP91P8d+4ibuonYZw9mfnzI9HfxselHZr5aAcs=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/x448/float16 v0.8.4 h1:qLwI1I70+NjRFUR3zs1JPUCgaCXSh3SW62uAKT1mSBM=
github.com/x448/float16 v0.8.4/go.mod h1:14CWIYCyZA/cWjXOioeEpHeN/83MdbZDRQHoFcYsOfg=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc=
golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
golang.org/x/oauth2 v0.21.0 h1:tsimM75w1tF/uws5rbeHzIWxEqElMehnc+iW793zsZs=
golang.org/x/oauth2 v0.21.0/go.mod h1:XYTD2NtWslqkgxebSiOHnXEap4TF09sJSc7H1sXbhtI=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.28.0 h1:/Ts8HFuMR2E6IP/jlo7QVLZHggjKQbhu/7H0LJFr3Gg=
golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4=
golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/evanphx/json-patch.v4 v4.12.0 h1:n6jtcsulIzXPJaxegRbvFNNrZDjbij7ny3gmSPG+6V4=
gopkg.in/evanphx/json-patch.v4 v4.12.0/go.mod h1:p8EYWUEYMpynmqDbY58zCKCFZw8pRWMG4EsWvDvM72M=
gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc=
gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
k8s.io/api v0.31.3 h1:umzm5o8lFbdN/hIXbrK9oRpOproJO62CV1zqxXrLgk8=
k8s.io/api v0.31.3/go.mod h1:UJrkIp9pnMOI9K2nlL6vwpxRzzEX5sWgn8kGQe92kCE=
k8s.io/apimachinery v0.31.3 h1:6l0WhcYgasZ/wk9ktLq5vLaoXJJr5ts6lkaQzgeYPq4=
k8s.io/apimachinery v0.31.3/go.mod h1:rsPdaZJfTfLsNJSQzNHQvYoTmxhoOEofxtOsF3rtsMo=
k8s.io/client-go v0.31.3 h1:CAlZuM+PH2cm+86LOBemaJI/lQ5linJ6UFxKX/SoG+4=
k8s.io/client-go v0.31.3/go.mod h1:2CgjPUTpv3fE5dNygAr2NcM8nhHzXvxB8KL5gYc3kJs=
k8s.io/klog/v2 v2.130.1 h1:n9Xl7H1Xvksem4KFG4PYbdQCQxqc/tTUyrgXaOhHSzk=
k8s.io/klog/v2 v2.130.1/go.mod h1:3Jpz1GvMt720eyJH1ckRHK1EDfpxISzJ7I9OYgaDtPE=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340 h1:BZqlfIlq5YbRMFko6/PM7FjZpUb45WallggurYhKGag=
k8s.io/kube-openapi v0.0.0-20240228011516-70dd3763d340/go.mod h1:yD4MZYeKMBwQKVht279WycxKyM84kkAx2DPrTXaeb98=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8 h1:pUdcCO1Lk/tbT5ztQWOBi5HBgbBP1J8+AsQnQCKsi8A=
k8s.io/utils v0.0.0-20240711033017-18e509b52bc8/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1 h1:150L+0vs/8DA78h1u02ooW1/fFq/Lwr+sGiqlzvrtq4=
sigs.k8s.io/structured-merge-diff/v4 v4.4.1/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08=
sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=
sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=
+97
View File
@@ -0,0 +1,97 @@
package api
import "strings"
// frontmatter is an ordered, line-preserving view of a note's YAML
// frontmatter block. It deliberately avoids a full YAML round-trip: the
// brain writes flat `key: value` frontmatter by hand, and a yaml.v3
// re-marshal would reorder keys and strip comments. Preserving the
// original lines verbatim keeps brain_update a surgical edit — only the
// keys it manages (updated_at, supersedes, supersede_reason) change.
type frontmatter struct {
lines []fmLine
}
// fmLine is one frontmatter line. For `key: value` lines, key and value
// are populated; for blank lines, comments, or anything that isn't a
// simple scalar pair, key is empty and raw holds the line verbatim.
type fmLine struct {
key string
value string
raw string
}
// parseFrontmatter splits src into its frontmatter block and body. A
// frontmatter block is recognised only when the file opens with a `---`
// fence and a closing `---` fence follows. Otherwise the whole input is
// the body and the returned frontmatter is empty.
func parseFrontmatter(src string) (frontmatter, string) {
var fm frontmatter
if !strings.HasPrefix(src, "---\n") {
return fm, src
}
rest := src[len("---\n"):]
end := strings.Index(rest, "\n---\n")
if end < 0 {
// Opening fence with no closing fence — treat as bodyless content.
return fm, src
}
block := rest[:end]
body := rest[end+len("\n---\n"):]
for _, line := range strings.Split(block, "\n") {
key, val, ok := strings.Cut(line, ":")
key = strings.TrimSpace(key)
if !ok || key == "" || strings.HasPrefix(strings.TrimSpace(line), "#") {
fm.lines = append(fm.lines, fmLine{raw: line})
continue
}
fm.lines = append(fm.lines, fmLine{key: key, value: strings.TrimSpace(val)})
}
return fm, body
}
// get returns the value for key, or "" if absent.
func (f *frontmatter) get(key string) string {
for _, l := range f.lines {
if l.key == key {
return l.value
}
}
return ""
}
// set overrides the value for an existing key in place, or appends a new
// `key: value` line when the key is absent.
func (f *frontmatter) set(key, value string) {
for i := range f.lines {
if f.lines[i].key == key {
f.lines[i].value = value
return
}
}
f.lines = append(f.lines, fmLine{key: key, value: value})
}
// render serialises the frontmatter back into a `---`-fenced block. An
// empty frontmatter renders to the empty string so bodies without a
// header stay header-less.
func (f *frontmatter) render() string {
if len(f.lines) == 0 {
return ""
}
var b strings.Builder
b.WriteString("---\n")
for _, l := range f.lines {
if l.key == "" {
b.WriteString(l.raw)
} else {
b.WriteString(l.key)
b.WriteString(": ")
b.WriteString(l.value)
}
b.WriteByte('\n')
}
b.WriteString("---\n")
return b.String()
}
@@ -0,0 +1,61 @@
package api
import (
"strings"
"testing"
"github.com/stretchr/testify/assert"
)
func TestParseFrontmatterSplitsHeaderAndBody(t *testing.T) {
src := "---\nwing: jepa-fx\nhall: facts\ncreated_at: 2026-01-01T00:00:00Z\n---\n# Title\n\nbody text\n"
fm, body := parseFrontmatter(src)
assert.Equal(t, "jepa-fx", fm.get("wing"))
assert.Equal(t, "facts", fm.get("hall"))
assert.Equal(t, "2026-01-01T00:00:00Z", fm.get("created_at"))
assert.Equal(t, "# Title\n\nbody text\n", body)
}
func TestParseFrontmatterNoHeader(t *testing.T) {
src := "# Just a body\n\nno frontmatter here\n"
fm, body := parseFrontmatter(src)
assert.Empty(t, fm.lines)
assert.Equal(t, src, body)
}
func TestFrontmatterSetOverridesExistingKey(t *testing.T) {
fm, _ := parseFrontmatter("---\nwing: a\nupdated_at: old\n---\nbody\n")
fm.set("updated_at", "new")
assert.Equal(t, "new", fm.get("updated_at"))
// No duplicate key.
assert.Equal(t, 1, strings.Count(fm.render(), "updated_at:"))
}
func TestFrontmatterSetAppendsNewKey(t *testing.T) {
fm, _ := parseFrontmatter("---\nwing: a\n---\nbody\n")
fm.set("supersedes", "abc123")
out := fm.render()
assert.Contains(t, out, "wing: a")
assert.Contains(t, out, "supersedes: abc123")
}
func TestFrontmatterRenderPreservesCustomFields(t *testing.T) {
src := "---\nwing: a\nhall: facts\ncustom_field: keep-me\ntags: [x, y]\n---\nbody\n"
fm, _ := parseFrontmatter(src)
fm.set("updated_at", "2026-06-22T00:00:00Z")
out := fm.render()
assert.Contains(t, out, "custom_field: keep-me")
assert.Contains(t, out, "tags: [x, y]")
assert.Contains(t, out, "updated_at: 2026-06-22T00:00:00Z")
}
func TestFrontmatterRenderRoundTrips(t *testing.T) {
src := "---\nwing: a\nhall: facts\n---\n"
fm, _ := parseFrontmatter(src)
assert.Equal(t, src, fm.render())
}
+59 -12
View File
@@ -51,12 +51,13 @@ type queryRequest struct {
}
type writeRequest struct {
Content string `json:"content"`
Filename string `json:"filename,omitempty"`
Type string `json:"type,omitempty"`
Domain string `json:"domain,omitempty"`
Wing string `json:"wing,omitempty"`
Hall string `json:"hall,omitempty"`
Content string `json:"content"`
Filename string `json:"filename,omitempty"`
Type string `json:"type,omitempty"`
Domain string `json:"domain,omitempty"`
Wing string `json:"wing,omitempty"`
Hall string `json:"hall,omitempty"`
SourceType string `json:"source_type,omitempty"` // "external" opts a hall=facts entry out of the internal default
}
type ingestRequest struct {
@@ -115,12 +116,13 @@ func (h *Handler) Query(w http.ResponseWriter, r *http.Request) {
// When either is empty, the note falls back to brain/knowledge/<filename>
// with optional type/domain frontmatter (legacy behaviour).
type WriteNoteOptions struct {
Content string
Filename string
Type string
Domain string
Wing string
Hall string
Content string
Filename string
Type string
Domain string
Wing string
Hall string
SourceType string // "internal" marks a first-party observation (e.g. claudewatcher) that needs no external citation
}
// WriteNote writes a markdown note into the brain. Returns the path
@@ -165,6 +167,17 @@ func writeHallNote(brainDir string, opts WriteNoteOptions) (string, error) {
if opts.Domain != "" {
fmt.Fprintf(&fm, "domain: %s\n", opts.Domain)
}
sourceType := opts.SourceType
if sourceType == "" && opts.Hall == "facts" {
// Most hall=facts entries are first-party (an eval/benchmark the
// writer ran itself), not external claims — default to internal and
// require an explicit source_type: external opt-out for the rare
// citation-needing entry (brain-gardener#7).
sourceType = "internal"
}
if sourceType != "" {
fmt.Fprintf(&fm, "source_type: %s\n", sourceType)
}
fm.WriteString("---\n")
if err := os.WriteFile(dest, []byte(fm.String()+opts.Content), 0o644); err != nil {
@@ -483,6 +496,40 @@ func (h *Handler) BackfillRefs(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]int{"updated": n})
}
// Pending handles GET /pending — list raw/ notes awaiting promotion.
func (h *Handler) Pending(w http.ResponseWriter, _ *http.Request) {
pending, err := ListPending(h.brainDir)
if err != nil {
h.logger.Error("pending failed", "err", err)
writeError(w, http.StatusInternalServerError, "pending error")
return
}
writeJSON(w, map[string]any{"pending": pending})
}
type promoteRequest struct {
Filename string `json:"filename"`
Wing string `json:"wing"`
Hall string `json:"hall"`
Slug string `json:"slug,omitempty"`
}
// Promote handles POST /promote — move a raw/ note into the wiki. A bad
// hall / collision / missing source is a 400 (caller error), not a 500.
func (h *Handler) Promote(w http.ResponseWriter, r *http.Request) {
var req promoteRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeError(w, http.StatusBadRequest, "invalid JSON")
return
}
rel, err := PromoteNote(h.brainDir, PromoteOptions(req))
if err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
writeJSON(w, map[string]string{"path": rel})
}
func writeJSON(w http.ResponseWriter, v any) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(v) //nolint:errcheck
+68
View File
@@ -118,6 +118,74 @@ func TestWrite_IncludesFrontmatterWhenTypeProvided(t *testing.T) {
assert.Contains(t, string(content), "Some learning.")
}
func TestWriteNote_HallRouteIncludesSourceTypeWhenSet(t *testing.T) {
dir := t.TempDir()
rel, err := api.WriteNote(dir, api.WriteNoteOptions{
Content: "# Claude session abc (koala)\n\nBody.\n",
Filename: "session-koala-abc",
Wing: "claude-sessions",
Hall: "facts",
Type: "source",
SourceType: "internal",
})
require.NoError(t, err)
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
require.NoError(t, err)
assert.Contains(t, string(got), "source_type: internal")
assert.Contains(t, string(got), "wing: claude-sessions")
}
func TestWriteNote_HallFactsDefaultsSourceTypeInternalWhenUnset(t *testing.T) {
dir := t.TempDir()
rel, err := api.WriteNote(dir, api.WriteNoteOptions{
Content: "manually captured fact.\n",
Wing: "agentsquad",
Hall: "facts",
})
require.NoError(t, err)
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
require.NoError(t, err)
// Most hall=facts entries are first-party (an eval/benchmark the agent ran
// itself), not external claims — default to internal, require explicit
// opt-out for the rare case that does need a citation (brain-gardener#7).
assert.Contains(t, string(got), "source_type: internal")
}
func TestWriteNote_HallFactsPreservesExplicitExternalSourceType(t *testing.T) {
dir := t.TempDir()
rel, err := api.WriteNote(dir, api.WriteNoteOptions{
Content: "vendor pricing claim, needs a citation.\n",
Wing: "agentsquad",
Hall: "facts",
SourceType: "external",
})
require.NoError(t, err)
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
require.NoError(t, err)
assert.Contains(t, string(got), "source_type: external")
}
func TestWriteNote_HallRouteOmitsSourceTypeForNonFactsHalls(t *testing.T) {
dir := t.TempDir()
rel, err := api.WriteNote(dir, api.WriteNoteOptions{
Content: "a decision record.\n",
Wing: "agentsquad",
Hall: "decisions",
})
require.NoError(t, err)
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
require.NoError(t, err)
assert.NotContains(t, string(got), "source_type")
}
func TestWrite_GeneratesFilenameIfAbsent(t *testing.T) {
dir, h := setup(t)
body, _ := json.Marshal(map[string]any{"content": "auto name"})
+156
View File
@@ -0,0 +1,156 @@
package api
import (
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
)
// PendingNote describes a raw/ note awaiting human promotion to the wiki.
type PendingNote struct {
Filename string `json:"filename"`
CreatedAt string `json:"created_at"`
SizeBytes int64 `json:"size_bytes"`
Excerpt string `json:"excerpt"`
}
// datePrefix matches a leading YYYY-MM-DD- on a raw filename, stripped when
// deriving the default promoted slug.
var datePrefix = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}-`)
// ListPending returns the notes in brain/raw/ awaiting review, oldest-first
// (natural review order). An absent raw/ dir yields an empty slice, not an
// error. Only .md files are listed; tunnel-candidate files are skipped.
func ListPending(brainDir string) ([]PendingNote, error) {
dir := filepath.Join(brainDir, "raw")
entries, err := os.ReadDir(dir)
if err != nil {
if os.IsNotExist(err) {
return []PendingNote{}, nil
}
return nil, fmt.Errorf("read raw dir: %w", err)
}
out := make([]PendingNote, 0, len(entries))
for _, e := range entries {
if e.IsDir() || !strings.HasSuffix(e.Name(), ".md") || strings.HasPrefix(e.Name(), "tunnel-candidates-") {
continue
}
info, statErr := e.Info()
if statErr != nil {
continue
}
raw, readErr := os.ReadFile(filepath.Join(dir, e.Name()))
if readErr != nil {
continue
}
fm, body := parseFrontmatter(string(raw))
created := fm.get("created_at")
if created == "" {
created = info.ModTime().UTC().Format(time.RFC3339)
}
out = append(out, PendingNote{
Filename: e.Name(),
CreatedAt: created,
SizeBytes: info.Size(),
Excerpt: excerpt(body, 200),
})
}
sort.SliceStable(out, func(i, j int) bool { return out[i].CreatedAt < out[j].CreatedAt })
return out, nil
}
// PromoteOptions identifies a raw note to promote and its wiki destination.
type PromoteOptions struct {
Filename string // basename in brain/raw/
Wing string
Hall string
Slug string // optional; defaults to Filename minus date prefix + .md
}
// PromoteNote moves a note from brain/raw/ into the structured wiki: it
// rewrites frontmatter (sets wing/hall/promoted_at, preserves created_at and
// any custom fields), writes to brain/wiki/<wing>/<hall>/<slug>.md, deletes
// the source, then rebuilds the wing index and runs auto-tunnel detection.
//
// It is atomic from the caller's view: validation (hall, wing, slug,
// collision) happens before any filesystem change, and the source is deleted
// only after the destination write succeeds (write-then-delete, never move).
// Returns the promoted note's path relative to brainDir.
func PromoteNote(brainDir string, opts PromoteOptions) (string, error) {
// Validate filename (basename only — no traversal) before touching fs.
base := filepath.Base(opts.Filename)
if base != opts.Filename || base == "." || base == ".." || strings.ContainsAny(opts.Filename, `/\`) {
return "", fmt.Errorf("invalid filename %q", opts.Filename)
}
slug := opts.Slug
if slug == "" {
slug = datePrefix.ReplaceAllString(strings.TrimSuffix(base, ".md"), "")
}
// NotePath validates hall + wing + slug; do this before reading anything.
dest, err := brain.NotePath(brainDir, opts.Wing, opts.Hall, slug)
if err != nil {
return "", err
}
src := filepath.Join(brainDir, "raw", base)
raw, err := os.ReadFile(src)
if err != nil {
if os.IsNotExist(err) {
return "", fmt.Errorf("pending note %q does not exist in raw/", base)
}
return "", fmt.Errorf("read source: %w", err)
}
// Collision: never silently overwrite an existing promoted note.
if _, statErr := os.Stat(dest); statErr == nil {
rel, _ := filepath.Rel(brainDir, dest)
return "", fmt.Errorf("target %s already exists; choose a different slug", filepath.ToSlash(rel))
}
fm, body := parseFrontmatter(string(raw))
now := time.Now().UTC().Format(time.RFC3339)
fm.set("wing", brain.Sanitise(opts.Wing))
fm.set("hall", opts.Hall)
if fm.get("created_at") == "" {
fm.set("created_at", now)
}
fm.set("promoted_at", now)
if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil {
return "", fmt.Errorf("create wing dir: %w", err)
}
// Write-then-delete: the source survives any write failure.
if err := os.WriteFile(dest, []byte(fm.render()+body), 0o644); err != nil {
return "", fmt.Errorf("write promoted note: %w", err)
}
if err := os.Remove(src); err != nil {
return "", fmt.Errorf("promoted note written but source removal failed: %w", err)
}
rel, _ := filepath.Rel(brainDir, dest)
relSlash := filepath.ToSlash(rel)
// Best-effort wiki upkeep — the note is already promoted.
_ = brain.BuildWingIndex(brainDir, opts.Wing)
_ = brain.AutoTunnel(brainDir, relSlash, body)
return relSlash, nil
}
// excerpt returns the first n runes of s, trimmed, single-spaced.
func excerpt(s string, n int) string {
s = strings.TrimSpace(s)
r := []rune(s)
if len(r) > n {
r = r[:n]
}
return strings.TrimSpace(string(r))
}
+121
View File
@@ -0,0 +1,121 @@
package api
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func writeRaw(t *testing.T, brainDir, name, content string) {
t.Helper()
dir := filepath.Join(brainDir, "raw")
require.NoError(t, os.MkdirAll(dir, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(dir, name), []byte(content), 0o644))
}
func TestListPendingEmptyWhenAbsent(t *testing.T) {
got, err := ListPending(t.TempDir())
require.NoError(t, err, "absent raw/ is not an error")
assert.Empty(t, got)
}
func TestListPendingReturnsOldestFirstWithExcerpt(t *testing.T) {
dir := t.TempDir()
writeRaw(t, dir, "2026-06-02-newer.md", "---\ncreated_at: 2026-06-02T00:00:00Z\n---\nNewer body here.\n")
writeRaw(t, dir, "2026-06-01-older.md", "---\ncreated_at: 2026-06-01T00:00:00Z\n---\nOlder body content.\n")
// non-md ignored
writeRaw(t, dir, "notes.txt", "ignore me")
got, err := ListPending(dir)
require.NoError(t, err)
require.Len(t, got, 2)
assert.Equal(t, "2026-06-01-older.md", got[0].Filename, "oldest first")
assert.Equal(t, "2026-06-02-newer.md", got[1].Filename)
assert.Contains(t, got[0].Excerpt, "Older body content")
assert.NotContains(t, got[0].Excerpt, "---", "excerpt is body, not frontmatter")
assert.Positive(t, got[0].SizeBytes)
}
func TestPromoteHappyPath(t *testing.T) {
dir := t.TempDir()
writeRaw(t, dir, "2026-06-01-lejpa-decision.md",
"---\ncreated_at: 2026-06-01T09:00:00Z\ncustom_field: keep-me\n---\n# LeJEPA\n\nbody.\n")
rel, err := PromoteNote(dir, PromoteOptions{
Filename: "2026-06-01-lejpa-decision.md", Wing: "jepa-fx", Hall: "decisions",
})
require.NoError(t, err)
assert.Equal(t, "wiki/jepa-fx/decisions/lejpa-decision.md", rel, "slug defaults to filename minus date prefix")
// Source deleted.
_, statErr := os.Stat(filepath.Join(dir, "raw", "2026-06-01-lejpa-decision.md"))
assert.True(t, os.IsNotExist(statErr), "source removed after promote")
got, err := os.ReadFile(filepath.Join(dir, filepath.FromSlash(rel)))
require.NoError(t, err)
s := string(got)
assert.Contains(t, s, "wing: jepa-fx")
assert.Contains(t, s, "hall: decisions")
assert.Contains(t, s, "created_at: 2026-06-01T09:00:00Z", "original created_at preserved")
assert.Contains(t, s, "promoted_at:")
assert.Contains(t, s, "custom_field: keep-me", "custom frontmatter preserved")
assert.Contains(t, s, "# LeJEPA")
}
func TestPromoteExplicitSlug(t *testing.T) {
dir := t.TempDir()
writeRaw(t, dir, "2026-06-01-x.md", "body\n")
rel, err := PromoteNote(dir, PromoteOptions{Filename: "2026-06-01-x.md", Wing: "a", Hall: "facts", Slug: "custom-slug"})
require.NoError(t, err)
assert.Equal(t, "wiki/a/facts/custom-slug.md", rel)
}
func TestPromoteInvalidHallErrorsBeforeTouchingFS(t *testing.T) {
dir := t.TempDir()
writeRaw(t, dir, "2026-06-01-x.md", "body\n")
_, err := PromoteNote(dir, PromoteOptions{Filename: "2026-06-01-x.md", Wing: "a", Hall: "garbage"})
require.Error(t, err)
// Source untouched.
_, statErr := os.Stat(filepath.Join(dir, "raw", "2026-06-01-x.md"))
assert.NoError(t, statErr, "invalid hall must not delete or move the source")
}
func TestPromoteMissingSourceErrors(t *testing.T) {
_, err := PromoteNote(t.TempDir(), PromoteOptions{Filename: "ghost.md", Wing: "a", Hall: "facts"})
require.Error(t, err)
}
func TestPromoteSlugCollisionNoOverwrite(t *testing.T) {
dir := t.TempDir()
// Pre-existing target.
dest := filepath.Join(dir, "wiki", "a", "facts", "x.md")
require.NoError(t, os.MkdirAll(filepath.Dir(dest), 0o755))
require.NoError(t, os.WriteFile(dest, []byte("EXISTING\n"), 0o644))
writeRaw(t, dir, "2026-06-01-x.md", "NEW\n")
_, err := PromoteNote(dir, PromoteOptions{Filename: "2026-06-01-x.md", Wing: "a", Hall: "facts"})
require.Error(t, err, "collision must error, not overwrite")
got, _ := os.ReadFile(dest)
assert.Equal(t, "EXISTING\n", string(got), "target not overwritten")
_, statErr := os.Stat(filepath.Join(dir, "raw", "2026-06-01-x.md"))
assert.NoError(t, statErr, "source preserved on collision (atomic: no delete without write)")
}
func TestPromoteRejectsTraversalFilename(t *testing.T) {
_, err := PromoteNote(t.TempDir(), PromoteOptions{Filename: "../escape.md", Wing: "a", Hall: "facts"})
require.Error(t, err)
}
func TestPromoteRebuildsWingIndex(t *testing.T) {
dir := t.TempDir()
writeRaw(t, dir, "2026-06-01-x.md", "---\ntitle: X Note\n---\nbody\n")
_, err := PromoteNote(dir, PromoteOptions{Filename: "2026-06-01-x.md", Wing: "a", Hall: "facts"})
require.NoError(t, err)
idx, err := os.ReadFile(filepath.Join(dir, "wiki", "a", "_index.md"))
require.NoError(t, err, "wing _index regenerated")
assert.Contains(t, string(idx), "x", "promoted note appears in the index")
}
+131
View File
@@ -0,0 +1,131 @@
package api
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
)
// ContentHash returns the lowercase hex sha256 of b. It is the note's
// content_hash handle: brain_write / brain_update return it, brain_get
// recomputes it from the file on disk, and brain_update stamps the prior
// note's hash into the new note's `supersedes` frontmatter.
func ContentHash(b []byte) string {
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])
}
// resolveWithin maps a brainDir-relative path to an absolute path and
// guarantees it does not escape brainDir. Returns the cleaned relPath
// (forward-slashed) and the absolute path.
func resolveWithin(brainDir, relPath string) (rel, abs string, err error) {
clean := filepath.Clean("/" + filepath.ToSlash(relPath))
rel = strings.TrimPrefix(clean, "/")
abs = filepath.Join(brainDir, filepath.FromSlash(rel))
check, err := filepath.Rel(brainDir, abs)
if err != nil || check == ".." || strings.HasPrefix(check, ".."+string(filepath.Separator)) {
return "", "", fmt.Errorf("path %q escapes brain dir", relPath)
}
return rel, abs, nil
}
// UpdateNoteOptions identifies the note to supersede and supplies its new
// body. Path takes precedence; otherwise the target is resolved from
// Wing/Hall/Slug via brain.NotePath.
type UpdateNoteOptions struct {
Path string // brainDir-relative path; takes precedence over wing/hall/slug
Wing string
Hall string
Slug string
Content string // new full body (whole-note replace)
Reason string // optional; stamped as supersede_reason
}
// UpdateNote supersedes an existing note in place. It replaces the body
// with opts.Content, preserves the existing frontmatter (created_at,
// wing, hall, and any custom fields), and stamps updated_at, supersedes
// (the prior content hash), and supersede_reason (when given).
//
// It never creates: if the target does not exist, it returns an error so
// the caller can fall back to brain_write. Returns the note's relPath,
// the new content hash, and the prior content hash.
//
// Embeddings are NOT refreshed here. The rewritten file's mtime advances,
// which the mtime-driven vectorstore.Sync ticker uses to re-embed it on
// its next pass — the same out-of-band mechanism brain_write relies on.
func UpdateNote(brainDir string, opts UpdateNoteOptions) (relPath, contentHash, priorHash string, err error) {
if opts.Content == "" {
return "", "", "", fmt.Errorf("content is required")
}
var rel string
if opts.Path != "" {
rel = opts.Path
} else {
full, perr := brain.NotePath(brainDir, opts.Wing, opts.Hall, opts.Slug)
if perr != nil {
return "", "", "", perr
}
rel, _ = filepath.Rel(brainDir, full)
rel = filepath.ToSlash(rel)
}
rel, abs, err := resolveWithin(brainDir, rel)
if err != nil {
return "", "", "", err
}
prior, err := os.ReadFile(abs)
if err != nil {
if os.IsNotExist(err) {
return "", "", "", fmt.Errorf("note %q does not exist: use brain_write to create", rel)
}
return "", "", "", fmt.Errorf("read target: %w", err)
}
priorHash = ContentHash(prior)
fm, _ := parseFrontmatter(string(prior))
fm.set("updated_at", time.Now().UTC().Format(time.RFC3339))
fm.set("supersedes", priorHash)
if opts.Reason != "" {
fm.set("supersede_reason", opts.Reason)
}
out := []byte(fm.render() + opts.Content)
if err := os.WriteFile(abs, out, 0o644); err != nil {
return "", "", "", fmt.Errorf("write: %w", err)
}
return rel, ContentHash(out), priorHash, nil
}
// ReadNote reads the note at the brainDir-relative relPath and returns
// its parsed frontmatter, body, and content hash. It is the read-after-
// write primitive behind brain_get: the hash it returns equals the hash
// brain_write / brain_update returned for the same bytes.
func ReadNote(brainDir, relPath string) (fm map[string]string, body, contentHash string, err error) {
_, abs, err := resolveWithin(brainDir, relPath)
if err != nil {
return nil, "", "", err
}
raw, err := os.ReadFile(abs)
if err != nil {
if os.IsNotExist(err) {
return nil, "", "", fmt.Errorf("note %q does not exist", relPath)
}
return nil, "", "", fmt.Errorf("read note: %w", err)
}
parsed, body := parseFrontmatter(string(raw))
fm = make(map[string]string, len(parsed.lines))
for _, l := range parsed.lines {
if l.key != "" {
fm[l.key] = l.value
}
}
return fm, body, ContentHash(raw), nil
}
+130
View File
@@ -0,0 +1,130 @@
package api
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// seedNote writes a note directly to disk and returns its relPath.
func seedNote(t *testing.T, brainDir, rel, content string) string {
t.Helper()
full := filepath.Join(brainDir, filepath.FromSlash(rel))
require.NoError(t, os.MkdirAll(filepath.Dir(full), 0o755))
require.NoError(t, os.WriteFile(full, []byte(content), 0o644))
return rel
}
func TestUpdateNoteSupersedesAndStamps(t *testing.T) {
brainDir := t.TempDir()
rel := seedNote(t, brainDir, "wiki/jepa-fx/facts/val-vol.md",
"---\nwing: jepa-fx\nhall: facts\ncreated_at: 2026-01-01T00:00:00Z\ncustom: keep-me\n---\n# Old\n\nold body\n")
relPath, hash, priorHash, err := UpdateNote(brainDir, UpdateNoteOptions{
Path: rel,
Content: "# New\n\nnew body\n",
Reason: "facts changed",
})
require.NoError(t, err)
assert.Equal(t, rel, relPath)
assert.NotEmpty(t, hash)
assert.NotEmpty(t, priorHash)
assert.NotEqual(t, hash, priorHash)
got, err := os.ReadFile(filepath.Join(brainDir, filepath.FromSlash(rel)))
require.NoError(t, err)
s := string(got)
// Body replaced.
assert.Contains(t, s, "# New")
assert.NotContains(t, s, "old body")
// Prior fields preserved.
assert.Contains(t, s, "wing: jepa-fx")
assert.Contains(t, s, "hall: facts")
assert.Contains(t, s, "created_at: 2026-01-01T00:00:00Z")
assert.Contains(t, s, "custom: keep-me")
// Supersession stamped.
assert.Contains(t, s, "updated_at:")
assert.Contains(t, s, "supersedes: "+priorHash)
assert.Contains(t, s, "supersede_reason: facts changed")
}
func TestUpdateNoteResolvesByWingHallSlug(t *testing.T) {
brainDir := t.TempDir()
seedNote(t, brainDir, "wiki/jepa-fx/facts/val-vol.md",
"---\nwing: jepa-fx\nhall: facts\n---\nold\n")
relPath, _, _, err := UpdateNote(brainDir, UpdateNoteOptions{
Wing: "jepa-fx", Hall: "facts", Slug: "val-vol",
Content: "new\n",
})
require.NoError(t, err)
assert.Equal(t, "wiki/jepa-fx/facts/val-vol.md", relPath)
}
func TestUpdateNoteErrorsOnMissingAndDoesNotCreate(t *testing.T) {
brainDir := t.TempDir()
_, _, _, err := UpdateNote(brainDir, UpdateNoteOptions{
Wing: "jepa-fx", Hall: "facts", Slug: "ghost",
Content: "x\n",
})
require.Error(t, err)
assert.Contains(t, err.Error(), "does not exist")
// No file created.
_, statErr := os.Stat(filepath.Join(brainDir, "wiki/jepa-fx/facts/ghost.md"))
assert.True(t, os.IsNotExist(statErr), "missing-target update must not create a note")
}
func TestUpdateNoteRejectsTraversal(t *testing.T) {
brainDir := t.TempDir()
_, _, _, err := UpdateNote(brainDir, UpdateNoteOptions{
Path: "../escape.md",
Content: "x\n",
})
require.Error(t, err)
}
func TestReadNoteReturnsFrontmatterBodyHash(t *testing.T) {
brainDir := t.TempDir()
rel := seedNote(t, brainDir, "wiki/jepa-fx/facts/n.md",
"---\nwing: jepa-fx\nhall: facts\n---\n# Body\n\ntext\n")
fm, body, hash, err := ReadNote(brainDir, rel)
require.NoError(t, err)
assert.Equal(t, "jepa-fx", fm["wing"])
assert.Equal(t, "facts", fm["hall"])
assert.Equal(t, "# Body\n\ntext\n", body)
// Hash matches ContentHash of the raw bytes on disk (round-trip).
raw, _ := os.ReadFile(filepath.Join(brainDir, filepath.FromSlash(rel)))
assert.Equal(t, ContentHash(raw), hash)
}
func TestReadNoteRejectsTraversal(t *testing.T) {
brainDir := t.TempDir()
_, _, _, err := ReadNote(brainDir, "../../etc/passwd")
require.Error(t, err)
}
func TestUpdateThenReadRoundTripsHash(t *testing.T) {
brainDir := t.TempDir()
rel := seedNote(t, brainDir, "wiki/a/facts/n.md", "---\nwing: a\nhall: facts\n---\nold\n")
_, hash, _, err := UpdateNote(brainDir, UpdateNoteOptions{Path: rel, Content: "new\n"})
require.NoError(t, err)
_, _, readHash, err := ReadNote(brainDir, rel)
require.NoError(t, err)
assert.Equal(t, hash, readHash, "update content_hash must round-trip through ReadNote")
}
func TestContentHashStable(t *testing.T) {
assert.Equal(t, ContentHash([]byte("abc")), ContentHash([]byte("abc")))
assert.NotEqual(t, ContentHash([]byte("abc")), ContentHash([]byte("abd")))
assert.True(t, strings.HasPrefix(ContentHash([]byte("")), "")) // hex, non-panicking
}
+167
View File
@@ -0,0 +1,167 @@
package audit
import (
"bufio"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"os"
"path/filepath"
"sync"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
)
// FileBuffer is a durable, restart-surviving audit buffer backed by a
// JSONL file: one {id, entry} record per line. It is the internal/public
// tier fallback when loki is unreachable. Confirm rewrites the file
// without the confirmed record, so a record is cleared only after its
// central write is confirmed.
//
// Access is serialised by a mutex; the buffer is low-throughput (only
// written during a loki outage), so a whole-file rewrite on Confirm is
// acceptable and keeps the on-disk format trivially correct.
type FileBuffer struct {
path string
mu sync.Mutex
}
type bufferLine struct {
ID string `json:"id"`
Entry capture.AuditEntry `json:"entry"`
}
// NewFileBuffer returns a buffer backed by path. The parent directory is
// created if needed. The file itself is created lazily on first Append.
func NewFileBuffer(path string) (*FileBuffer, error) {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return nil, fmt.Errorf("create buffer dir: %w", err)
}
return &FileBuffer{path: path}, nil
}
// Writable reports whether the buffer file can be appended to. It probes
// by opening the file for append (creating it if absent) — the same
// operation Append performs — so Reserve's check matches Append's reality.
func (b *FileBuffer) Writable() error {
b.mu.Lock()
defer b.mu.Unlock()
f, err := os.OpenFile(b.path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return err
}
return f.Close()
}
// Append durably writes one audit record. The ID is derived from the
// content + timestamp so it is stable and unique per record.
func (b *FileBuffer) Append(e capture.AuditEntry) error {
b.mu.Lock()
defer b.mu.Unlock()
line := bufferLine{ID: recordID(e), Entry: e}
data, err := json.Marshal(line)
if err != nil {
return fmt.Errorf("marshal buffer line: %w", err)
}
f, err := os.OpenFile(b.path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o644)
if err != nil {
return err
}
defer func() { _ = f.Close() }()
if _, err := f.Write(append(data, '\n')); err != nil {
return err
}
return f.Sync()
}
// Pending reads all buffered records. A missing file means none.
func (b *FileBuffer) Pending() ([]Buffered, error) {
b.mu.Lock()
defer b.mu.Unlock()
return b.readAllLocked()
}
func (b *FileBuffer) readAllLocked() ([]Buffered, error) {
f, err := os.Open(b.path)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, err
}
defer func() { _ = f.Close() }()
var out []Buffered
sc := bufio.NewScanner(f)
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for sc.Scan() {
raw := sc.Bytes()
if len(raw) == 0 {
continue
}
var l bufferLine
if err := json.Unmarshal(raw, &l); err != nil {
return nil, fmt.Errorf("parse buffer line: %w", err)
}
out = append(out, Buffered(l))
}
return out, sc.Err()
}
// Confirm removes a single record after its central write is confirmed, by
// rewriting the file without it. Unknown IDs are a no-op.
func (b *FileBuffer) Confirm(id string) error {
b.mu.Lock()
defer b.mu.Unlock()
all, err := b.readAllLocked()
if err != nil {
return err
}
tmp := b.path + ".tmp"
f, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o644)
if err != nil {
return err
}
w := bufio.NewWriter(f)
kept := 0
for _, rec := range all {
if rec.ID == id {
continue
}
data, _ := json.Marshal(bufferLine(rec))
if _, err := w.Write(append(data, '\n')); err != nil {
_ = f.Close()
return err
}
kept++
}
if err := w.Flush(); err != nil {
_ = f.Close()
return err
}
if err := f.Sync(); err != nil {
_ = f.Close()
return err
}
if err := f.Close(); err != nil {
return err
}
// Empty buffer → remove the file entirely so Pending sees nothing.
if kept == 0 {
_ = os.Remove(tmp)
return os.Remove(b.path)
}
return os.Rename(tmp, b.path)
}
// recordID is a stable per-record identifier: sha256 of the principal,
// timestamp, and item list. Distinct captures never collide; the same
// buffered record always hashes the same.
func recordID(e capture.AuditEntry) string {
h := sha256.New()
_, _ = fmt.Fprintf(h, "%s|%s|%v|%s", e.Principal, e.Timestamp.UTC().Format("2006-01-02T15:04:05.000000000Z07:00"), e.Items, e.SessionRef)
return hex.EncodeToString(h.Sum(nil))[:16]
}
+96
View File
@@ -0,0 +1,96 @@
package audit
import (
"context"
"fmt"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
)
// Central is the central audit substrate (loki). Ready is a cheap
// reachability probe used by the pre-write reserve; Push writes a record.
type Central interface {
Ready(ctx context.Context) error
Push(ctx context.Context, e capture.AuditEntry) error
}
// Buffer is the durable local fallback for internal/public-tier records
// when the central sink is unreachable. It must survive process restart.
type Buffer interface {
// Writable reports whether the buffer can currently be appended to.
Writable() error
Append(e capture.AuditEntry) error
// Pending returns buffered records awaiting reconciliation, each with a
// stable ID used to Confirm (delete) it after a confirmed central write.
Pending() ([]Buffered, error)
Confirm(id string) error
}
// Buffered is a buffered audit record plus its stable buffer ID.
type Buffered struct {
ID string
Entry capture.AuditEntry
}
// Notifier raises an out-of-band alert (ntfy) about a degraded state.
type Notifier interface {
Notify(ctx context.Context, msg string) error
}
// DegradingSink is the classification-aware AuditSink (§4.4):
//
// - central reachable → AuditCentral (all tiers).
// - central down + confidential → refuse (no buffer): confidential must
// be centrally auditable at write time.
// - central down + internal/public + buffer writable → AuditBuffered.
// - central down + (confidential, or buffer not writable) → refuse (floor).
//
// The decision is made in Reserve, before any write; Record then executes it.
type DegradingSink struct {
central Central
buffer Buffer
notifier Notifier
}
// NewDegradingSink wires the central sink, durable buffer, and notifier.
func NewDegradingSink(central Central, buffer Buffer, notifier Notifier) *DegradingSink {
return &DegradingSink{central: central, buffer: buffer, notifier: notifier}
}
// Reserve decides, before any write, how the capture will be audited — or
// returns an error to refuse it.
func (d *DegradingSink) Reserve(ctx context.Context, level classification.Level) (capture.AuditOutcome, error) {
if err := d.central.Ready(ctx); err == nil {
return capture.AuditCentral, nil
}
// Central sink is down.
if level == classification.Confidential {
return 0, fmt.Errorf("confidential capture requires the central audit sink, which is unreachable")
}
if err := d.buffer.Writable(); err != nil {
// Floor: neither central nor local buffer can record the audit.
return 0, fmt.Errorf("audit floor: central sink down and local buffer unwritable: %w", err)
}
return capture.AuditBuffered, nil
}
// Record persists the entry per the reserved outcome. For AuditBuffered it
// also fires the degraded-state alert.
func (d *DegradingSink) Record(ctx context.Context, e capture.AuditEntry, outcome capture.AuditOutcome) error {
switch outcome {
case capture.AuditBuffered:
if err := d.buffer.Append(e); err != nil {
return fmt.Errorf("buffer audit record: %w", err)
}
// Best-effort alert; the record is already durably buffered.
if d.notifier != nil {
_ = d.notifier.Notify(ctx, fmt.Sprintf(
"capture audit BUFFERED LOCALLY (loki unreachable) — principal=%s class=%s items=%d",
e.Principal, e.EffectiveClassification, len(e.Items)))
}
return nil
default:
return d.central.Push(ctx, e)
}
}
+191
View File
@@ -0,0 +1,191 @@
package audit_test
import (
"context"
"errors"
"path/filepath"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// --- fakes ---
type fakeCentral struct {
down bool
pushed []capture.AuditEntry
pushErr error
}
func (f *fakeCentral) Ready(context.Context) error {
if f.down {
return errors.New("loki down")
}
return nil
}
func (f *fakeCentral) Push(_ context.Context, e capture.AuditEntry) error {
if f.pushErr != nil {
return f.pushErr
}
f.pushed = append(f.pushed, e)
return nil
}
type fakeNotifier struct{ msgs []string }
func (f *fakeNotifier) Notify(_ context.Context, msg string) error {
f.msgs = append(f.msgs, msg)
return nil
}
// unwritableBuffer always reports it cannot be written (floor condition).
type unwritableBuffer struct{}
func (unwritableBuffer) Writable() error { return errors.New("disk full") }
func (unwritableBuffer) Append(capture.AuditEntry) error { return errors.New("disk full") }
func (unwritableBuffer) Pending() ([]audit.Buffered, error) { return nil, nil }
func (unwritableBuffer) Confirm(string) error { return nil }
func newFileBuffer(t *testing.T) *audit.FileBuffer {
t.Helper()
b, err := audit.NewFileBuffer(filepath.Join(t.TempDir(), "audit-buffer.jsonl"))
require.NoError(t, err)
return b
}
func entry(principal string) capture.AuditEntry {
return capture.AuditEntry{Principal: principal, EffectiveClassification: "internal", Items: []string{"insight:x"}}
}
// --- Reserve: classification-aware decision ---
func TestReserveCentralUpGrantsCentral(t *testing.T) {
d := audit.NewDegradingSink(&fakeCentral{}, newFileBuffer(t), &fakeNotifier{})
for _, lvl := range []classification.Level{classification.Public, classification.Internal, classification.Confidential} {
out, err := d.Reserve(context.Background(), lvl)
require.NoError(t, err)
assert.Equal(t, capture.AuditCentral, out)
}
}
func TestReserveConfidentialSinkDownRefuses(t *testing.T) {
d := audit.NewDegradingSink(&fakeCentral{down: true}, newFileBuffer(t), &fakeNotifier{})
_, err := d.Reserve(context.Background(), classification.Confidential)
require.Error(t, err, "confidential + sink down → refuse, no buffer")
}
func TestReserveInternalSinkDownBuffers(t *testing.T) {
d := audit.NewDegradingSink(&fakeCentral{down: true}, newFileBuffer(t), &fakeNotifier{})
out, err := d.Reserve(context.Background(), classification.Internal)
require.NoError(t, err)
assert.Equal(t, capture.AuditBuffered, out)
}
func TestReserveFloorRefusesWhenNothingCanRecord(t *testing.T) {
d := audit.NewDegradingSink(&fakeCentral{down: true}, unwritableBuffer{}, &fakeNotifier{})
_, err := d.Reserve(context.Background(), classification.Internal)
require.Error(t, err, "central down AND buffer unwritable → floor refuse")
}
// --- Record: executes the reserved outcome ---
func TestRecordCentralPushes(t *testing.T) {
c := &fakeCentral{}
d := audit.NewDegradingSink(c, newFileBuffer(t), &fakeNotifier{})
require.NoError(t, d.Record(context.Background(), entry("p"), capture.AuditCentral))
assert.Len(t, c.pushed, 1)
}
func TestRecordBufferedAppendsAndNotifies(t *testing.T) {
buf := newFileBuffer(t)
nt := &fakeNotifier{}
d := audit.NewDegradingSink(&fakeCentral{down: true}, buf, nt)
require.NoError(t, d.Record(context.Background(), entry("p"), capture.AuditBuffered))
pending, err := buf.Pending()
require.NoError(t, err)
assert.Len(t, pending, 1)
assert.NotEmpty(t, nt.msgs, "degraded state alerts via ntfy")
}
// --- FileBuffer durability + Confirm ---
func TestFileBufferSurvivesRestart(t *testing.T) {
path := filepath.Join(t.TempDir(), "buf.jsonl")
b1, err := audit.NewFileBuffer(path)
require.NoError(t, err)
require.NoError(t, b1.Append(entry("p1")))
require.NoError(t, b1.Append(entry("p2")))
// "restart": a fresh FileBuffer over the same file sees the records.
b2, err := audit.NewFileBuffer(path)
require.NoError(t, err)
pending, err := b2.Pending()
require.NoError(t, err)
assert.Len(t, pending, 2)
}
func TestFileBufferConfirmRemovesOnlyThatRecord(t *testing.T) {
buf := newFileBuffer(t)
require.NoError(t, buf.Append(entry("keep")))
require.NoError(t, buf.Append(entry("drop")))
pending, _ := buf.Pending()
require.Len(t, pending, 2)
var dropID string
for _, p := range pending {
if p.Entry.Principal == "drop" {
dropID = p.ID
}
}
require.NoError(t, buf.Confirm(dropID))
after, _ := buf.Pending()
require.Len(t, after, 1)
assert.Equal(t, "keep", after[0].Entry.Principal)
}
// --- Reconcile ---
func TestReconcileReplaysAndClearsOnlyAfterConfirmedWrite(t *testing.T) {
buf := newFileBuffer(t)
require.NoError(t, buf.Append(entry("a")))
require.NoError(t, buf.Append(entry("b")))
c := &fakeCentral{} // up
nt := &fakeNotifier{}
n, err := audit.Reconcile(context.Background(), c, buf, nt)
require.NoError(t, err)
assert.Equal(t, 2, n)
assert.Len(t, c.pushed, 2, "buffered records replayed to central")
pending, _ := buf.Pending()
assert.Empty(t, pending, "buffer cleared after confirmed central writes")
}
func TestReconcileNoopWhenCentralDown(t *testing.T) {
buf := newFileBuffer(t)
require.NoError(t, buf.Append(entry("a")))
n, err := audit.Reconcile(context.Background(), &fakeCentral{down: true}, buf, &fakeNotifier{})
require.NoError(t, err)
assert.Equal(t, 0, n)
pending, _ := buf.Pending()
assert.Len(t, pending, 1, "records stay buffered while central is down")
}
func TestReconcileKeepsRecordWhenPushFails(t *testing.T) {
buf := newFileBuffer(t)
require.NoError(t, buf.Append(entry("a")))
// Ready ok but Push fails → record must remain buffered (not lost).
c := &fakeCentral{pushErr: errors.New("push rejected")}
n, err := audit.Reconcile(context.Background(), c, buf, &fakeNotifier{})
require.NoError(t, err)
assert.Equal(t, 0, n)
pending, _ := buf.Pending()
assert.Len(t, pending, 1)
}
+99
View File
@@ -0,0 +1,99 @@
package audit
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strconv"
"strings"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
)
// LokiCentral pushes capture audit records to a Grafana Loki instance via
// its push API, and probes readiness via /ready. It is the central audit
// substrate behind DegradingSink.
type LokiCentral struct {
baseURL string
labels map[string]string
http *http.Client
}
// NewLokiCentral constructs a LokiCentral for the given base URL (e.g.
// http://loki:3100). Returns nil when baseURL is empty so callers can
// treat missing config as "no central sink" with a single nil check.
func NewLokiCentral(baseURL string) *LokiCentral {
if baseURL == "" {
return nil
}
return &LokiCentral{
baseURL: strings.TrimRight(baseURL, "/"),
labels: map[string]string{"service": "brain-capture", "kind": "audit"},
http: &http.Client{Timeout: 10 * time.Second},
}
}
// Ready probes Loki's readiness endpoint.
func (l *LokiCentral) Ready(ctx context.Context) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, l.baseURL+"/ready", nil)
if err != nil {
return err
}
resp, err := l.http.Do(req)
if err != nil {
return fmt.Errorf("loki not ready: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("loki not ready: status %d", resp.StatusCode)
}
return nil
}
// pushPayload is the Loki push API body: one stream, one entry whose line
// is the JSON-encoded audit record.
type pushPayload struct {
Streams []lokiStream `json:"streams"`
}
type lokiStream struct {
Stream map[string]string `json:"stream"`
Values [][2]string `json:"values"`
}
// Push writes one audit record to Loki as a structured log line.
func (l *LokiCentral) Push(ctx context.Context, e capture.AuditEntry) error {
line, err := json.Marshal(e)
if err != nil {
return fmt.Errorf("marshal audit entry: %w", err)
}
ts := e.Timestamp
if ts.IsZero() {
ts = time.Now()
}
body, err := json.Marshal(pushPayload{Streams: []lokiStream{{
Stream: l.labels,
Values: [][2]string{{strconv.FormatInt(ts.UTC().UnixNano(), 10), string(line)}},
}}})
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
l.baseURL+"/loki/api/v1/push", bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
resp, err := l.http.Do(req)
if err != nil {
return fmt.Errorf("loki push: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("loki push: status %d", resp.StatusCode)
}
return nil
}
@@ -0,0 +1,86 @@
package audit_test
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLokiReadyAndPush(t *testing.T) {
var pushBody string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/ready":
w.WriteHeader(http.StatusOK)
case "/loki/api/v1/push":
b, _ := io.ReadAll(r.Body)
pushBody = string(b)
w.WriteHeader(http.StatusNoContent)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer srv.Close()
c := audit.NewLokiCentral(srv.URL)
require.NotNil(t, c)
require.NoError(t, c.Ready(context.Background()))
err := c.Push(context.Background(), capture.AuditEntry{
Principal: "koala-cli", EffectiveClassification: "internal", Items: []string{"insight:x"},
})
require.NoError(t, err)
assert.Contains(t, pushBody, "streams")
assert.Contains(t, pushBody, "koala-cli", "audit entry serialised into the loki line")
}
func TestLokiReadyFailsWhenDown(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusServiceUnavailable)
}))
defer srv.Close()
require.Error(t, audit.NewLokiCentral(srv.URL).Ready(context.Background()))
}
func TestLokiNilWhenUnconfigured(t *testing.T) {
assert.Nil(t, audit.NewLokiCentral(""))
}
func TestNtfyNotify(t *testing.T) {
var gotBody, gotAuth string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
gotBody = string(b)
gotAuth = r.Header.Get("Authorization")
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
n := audit.NewNtfyNotifier(srv.URL, "ntfy-token")
require.NotNil(t, n)
require.NoError(t, n.Notify(context.Background(), "audit buffered locally"))
assert.Contains(t, gotBody, "audit buffered locally")
assert.Equal(t, "Bearer ntfy-token", gotAuth)
}
func TestNtfyDoesNotLeakTokenOnError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
}))
defer srv.Close()
err := audit.NewNtfyNotifier(srv.URL, "secret-token").Notify(context.Background(), "x")
require.Error(t, err)
assert.False(t, strings.Contains(err.Error(), "secret-token"), "token must not leak into errors")
}
func TestNtfyNilWhenUnconfigured(t *testing.T) {
assert.Nil(t, audit.NewNtfyNotifier("", "tok"))
}
+55
View File
@@ -0,0 +1,55 @@
package audit
import (
"context"
"fmt"
"net/http"
"strings"
"time"
)
// NtfyNotifier posts alerts to an ntfy topic URL. Used to surface a
// degraded audit state (records buffered locally during a loki outage).
type NtfyNotifier struct {
topicURL string
token string
http *http.Client
}
// NewNtfyNotifier constructs a notifier for the given ntfy topic URL
// (e.g. https://ntfy.sh/my-topic). token is an optional bearer for
// protected ntfy instances; it is held here and only sent in the
// Authorization header, never logged. Returns nil when topicURL is empty.
func NewNtfyNotifier(topicURL, token string) *NtfyNotifier {
if topicURL == "" {
return nil
}
return &NtfyNotifier{
topicURL: strings.TrimRight(topicURL, "/"),
token: token,
http: &http.Client{Timeout: 10 * time.Second},
}
}
// Notify posts a message to the ntfy topic.
func (n *NtfyNotifier) Notify(ctx context.Context, msg string) error {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, n.topicURL, strings.NewReader(msg))
if err != nil {
return err
}
req.Header.Set("Title", "brain-capture audit degraded")
req.Header.Set("Priority", "high")
req.Header.Set("Tags", "warning,brain")
if n.token != "" {
req.Header.Set("Authorization", "Bearer "+n.token)
}
resp, err := n.http.Do(req)
if err != nil {
return fmt.Errorf("ntfy notify: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("ntfy notify: status %d", resp.StatusCode)
}
return nil
}
+65
View File
@@ -0,0 +1,65 @@
package audit
import (
"context"
"fmt"
"log/slog"
"time"
)
// Reconcile replays locally-buffered audit records to the central sink
// when it is reachable again. A record is removed from the buffer ONLY
// after its central write is confirmed, so a crash mid-reconcile re-plays
// rather than loses. Returns the number of records reconciled.
//
// A no-op (0, nil) when the central sink is still unreachable or the
// buffer is empty.
func Reconcile(ctx context.Context, central Central, buffer Buffer, notifier Notifier) (int, error) {
if err := central.Ready(ctx); err != nil {
return 0, nil // still down; try again next tick
}
pending, err := buffer.Pending()
if err != nil {
return 0, fmt.Errorf("read buffer: %w", err)
}
reconciled := 0
for _, rec := range pending {
if err := central.Push(ctx, rec.Entry); err != nil {
// Central went away mid-drain; stop and keep the rest buffered.
break
}
if err := buffer.Confirm(rec.ID); err != nil {
return reconciled, fmt.Errorf("confirm buffered record %s: %w", rec.ID, err)
}
reconciled++
}
if reconciled > 0 && notifier != nil {
_ = notifier.Notify(ctx, fmt.Sprintf("reconciled %d buffered capture audit record(s) to loki", reconciled))
}
return reconciled, nil
}
// StartReconcile runs Reconcile on a ticker until ctx is cancelled. It is
// the recovery half of the degrade-and-buffer path; pair it with a
// DegradingSink sharing the same buffer + central.
func StartReconcile(ctx context.Context, central Central, buffer Buffer, notifier Notifier, interval time.Duration) {
if interval <= 0 {
interval = time.Minute
}
go func() {
t := time.NewTicker(interval)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
if n, err := Reconcile(ctx, central, buffer, notifier); err != nil {
slog.Warn("audit reconcile failed", "err", err)
} else if n > 0 {
slog.Info("audit reconcile", "reconciled", n)
}
}
}
}()
}
+56
View File
@@ -0,0 +1,56 @@
// Package audit provides AuditSink implementations for the capture
// capability (I5). This file ships the minimal slog-backed sink used in
// #53: it emits the request-level audit record to structured logs, which
// the alloy/loki substrate already scrapes. The classification-aware
// degradation/refusal sink (confidential fails closed, internal buffers +
// reconciles) lands in #54 and replaces this behind the same interface.
package audit
import (
"context"
"log/slog"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
)
// SlogSink records audit entries to an slog.Logger. It never fails and is
// always centrally available, so its Reserve always grants AuditCentral —
// it does not exercise the I5 degradation/floor. That is DegradingSink's
// job (loki + durable buffer). SlogSink is the default for deployments
// without a loki endpoint configured. A nil logger ⇒ slog.Default().
type SlogSink struct {
logger *slog.Logger
}
// NewSlogSink constructs a SlogSink. nil logger ⇒ slog.Default().
func NewSlogSink(logger *slog.Logger) *SlogSink {
if logger == nil {
logger = slog.Default()
}
return &SlogSink{logger: logger}
}
// Reserve always grants central recording — slog is always available.
func (s *SlogSink) Reserve(_ context.Context, _ classification.Level) (capture.AuditOutcome, error) {
return capture.AuditCentral, nil
}
// Record emits the audit entry at info level. Security events, when
// present, are logged at warn level so they surface independently of the
// routine audit stream.
func (s *SlogSink) Record(_ context.Context, e capture.AuditEntry, _ capture.AuditOutcome) error {
s.logger.Info("capture audit",
"principal", e.Principal,
"actor", e.Actor,
"harness", e.Harness,
"session_ref", e.SessionRef,
"classification", e.EffectiveClassification,
"items", e.Items,
"ts", e.Timestamp,
)
for _, ev := range e.SecurityEvents {
s.logger.Warn("capture security event", "principal", e.Principal, "event", ev)
}
return nil
}
+41
View File
@@ -0,0 +1,41 @@
package audit_test
import (
"bytes"
"context"
"log/slog"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestSlogSinkRecordsEntryAndSecurityEvents(t *testing.T) {
var buf bytes.Buffer
sink := audit.NewSlogSink(slog.New(slog.NewTextHandler(&buf, nil)))
err := sink.Record(context.Background(), capture.AuditEntry{
Principal: "koala-cli",
Harness: "claude-code",
EffectiveClassification: "confidential",
Items: []string{"insight:wiki/a/facts/x.md"},
SecurityEvents: []string{"asserted-vs-derived origin mismatch"},
}, capture.AuditCentral)
require.NoError(t, err)
out := buf.String()
assert.Contains(t, out, "capture audit")
assert.Contains(t, out, "koala-cli")
assert.Contains(t, out, "confidential")
assert.Contains(t, out, "capture security event")
assert.Contains(t, out, "asserted-vs-derived origin mismatch")
}
func TestSlogSinkNilLoggerDefaults(t *testing.T) {
// nil logger must not panic.
require.NotPanics(t, func() {
_ = audit.NewSlogSink(nil).Record(context.Background(), capture.AuditEntry{}, capture.AuditCentral)
})
}
+129
View File
@@ -0,0 +1,129 @@
// Package brainstore is the concrete BrainStore: the single shared
// implementation of the #45 write/update/get verbs, used by BOTH the MCP
// handlers and the capture use-case so there is one implementation, not
// two (the Clean-Architecture / DRY payoff of #51).
//
// It composes the file-level primitives in package api (WriteNote,
// UpdateNote, ReadNote — the read-after-write contract) with the wiki
// upkeep that must accompany a write: wing _index rebuild, cross-wing
// auto-tunnel, and graph re-index. Embedding refresh is intentionally
// out-of-band (mtime-driven vectorstore.Sync) and not triggered here —
// see the brain note on out-of-band sync.
package brainstore
import (
"context"
"log/slog"
"strings"
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphsync"
)
// Store implements capture.BrainStore against a brain directory on disk,
// optionally re-indexing each write into the knowledge graph.
type Store struct {
brainDir string
graph graphsync.Store // nil = graph re-index disabled
}
// New constructs a Store bound to brainDir with graph indexing disabled.
func New(brainDir string) *Store {
return &Store{brainDir: brainDir}
}
// WithGraph enables graph re-index on every write/update. nil disables it.
func (s *Store) WithGraph(g graphsync.Store) *Store {
s.graph = g
return s
}
// Write creates a brain note and returns its read-after-write handle.
func (s *Store) Write(ctx context.Context, n capture.Note) (capture.Ref, error) {
relPath, err := api.WriteNote(s.brainDir, api.WriteNoteOptions{
Content: n.Content,
Filename: n.Filename,
Type: n.Type,
Domain: n.Domain,
Wing: n.Wing,
Hall: n.Hall,
})
if err != nil {
return capture.Ref{}, err
}
s.wikiUpkeep(relPath, n.Wing, n.Content)
s.indexInGraph(ctx, "brain_write", relPath)
_, _, hash, _ := api.ReadNote(s.brainDir, relPath)
return capture.Ref{ID: relPath, Path: relPath, ContentHash: hash}, nil
}
// Update supersedes an existing note in place. slug may be a bare slug
// (resolved against n.Wing/n.Hall) or a full brain-relative path (when it
// contains a slash). It never creates — a missing target is an error.
func (s *Store) Update(ctx context.Context, slug string, n capture.Note) (capture.Ref, error) {
opts := api.UpdateNoteOptions{Content: n.Content, Reason: n.Reason}
if strings.Contains(slug, "/") {
opts.Path = slug
} else {
opts.Wing, opts.Hall, opts.Slug = n.Wing, n.Hall, slug
}
relPath, hash, _, err := api.UpdateNote(s.brainDir, opts)
if err != nil {
return capture.Ref{}, err
}
if wing := wingFromRelPath(relPath); wing != "" {
s.wikiUpkeep(relPath, wing, n.Content)
}
s.indexInGraph(ctx, "brain_update", relPath)
return capture.Ref{ID: relPath, Path: relPath, ContentHash: hash, Superseded: true}, nil
}
// Get fetches a note by id/path — the read-after-write confirmation
// primitive (a direct fetch, never a semantic query).
func (s *Store) Get(_ context.Context, id string) (capture.StoredNote, error) {
fm, body, hash, err := api.ReadNote(s.brainDir, id)
if err != nil {
return capture.StoredNote{}, err
}
return capture.StoredNote{ID: id, Path: id, ContentHash: hash, Frontmatter: fm, Body: body}, nil
}
// wikiUpkeep rebuilds the wing _index and re-tunnels cross-wing matches
// when a note lands in the structured wiki. Both are best-effort: the
// note is already written, so a failure here is logged, not propagated.
func (s *Store) wikiUpkeep(relPath, wing, content string) {
if wing == "" {
return
}
if err := brain.BuildWingIndex(s.brainDir, wing); err != nil {
slog.Warn("brainstore: auto-index failed", "wing", wing, "err", err)
}
if err := brain.AutoTunnel(s.brainDir, relPath, content); err != nil {
slog.Warn("brainstore: auto-tunnel failed", "src", relPath, "err", err)
}
}
// indexInGraph re-indexes a written doc into the graph, best-effort.
func (s *Store) indexInGraph(ctx context.Context, op, relPath string) {
if s.graph == nil || relPath == "" {
return
}
if err := graphsync.IndexDoc(ctx, s.graph, s.brainDir, relPath); err != nil {
slog.Warn(op+": graph index failed", "path", relPath, "err", err)
}
}
// wingFromRelPath extracts the wing from a structured wiki path
// (wiki/<wing>/<hall>/<slug>.md). Returns "" for legacy/non-wiki paths.
func wingFromRelPath(relPath string) string {
parts := strings.Split(relPath, "/")
if len(parts) >= 4 && parts[0] == "wiki" {
return parts[1]
}
return ""
}
@@ -0,0 +1,85 @@
package brainstore_test
import (
"context"
"os"
"path/filepath"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestStoreWriteReturnsHandle(t *testing.T) {
dir := t.TempDir()
s := brainstore.New(dir)
ref, err := s.Write(context.Background(), capture.Note{
Content: "# X\n\nbody\n", Filename: "x", Wing: "a", Hall: "facts",
})
require.NoError(t, err)
assert.Equal(t, "wiki/a/facts/x.md", ref.Path)
assert.Equal(t, ref.Path, ref.ID)
assert.NotEmpty(t, ref.ContentHash)
assert.False(t, ref.Superseded)
_, err = os.Stat(filepath.Join(dir, "wiki/a/facts/x.md"))
require.NoError(t, err)
}
func TestStoreUpdateSupersedes(t *testing.T) {
dir := t.TempDir()
s := brainstore.New(dir)
_, err := s.Write(context.Background(), capture.Note{
Content: "old\n", Filename: "n", Wing: "a", Hall: "facts",
})
require.NoError(t, err)
ref, err := s.Update(context.Background(), "n", capture.Note{
Content: "new\n", Wing: "a", Hall: "facts", Reason: "changed",
})
require.NoError(t, err)
assert.True(t, ref.Superseded)
assert.Equal(t, "wiki/a/facts/n.md", ref.Path)
got, _ := os.ReadFile(filepath.Join(dir, "wiki/a/facts/n.md"))
assert.Contains(t, string(got), "new")
assert.Contains(t, string(got), "supersede_reason: changed")
}
func TestStoreUpdateByFullPath(t *testing.T) {
dir := t.TempDir()
s := brainstore.New(dir)
_, err := s.Write(context.Background(), capture.Note{Content: "old\n", Filename: "n", Wing: "a", Hall: "facts"})
require.NoError(t, err)
ref, err := s.Update(context.Background(), "wiki/a/facts/n.md", capture.Note{Content: "fresh\n"})
require.NoError(t, err)
assert.Equal(t, "wiki/a/facts/n.md", ref.Path)
}
func TestStoreUpdateMissingErrors(t *testing.T) {
dir := t.TempDir()
s := brainstore.New(dir)
_, err := s.Update(context.Background(), "ghost", capture.Note{Content: "x\n", Wing: "a", Hall: "facts"})
require.Error(t, err)
_, statErr := os.Stat(filepath.Join(dir, "wiki/a/facts/ghost.md"))
assert.True(t, os.IsNotExist(statErr), "update must not create")
}
func TestStoreGetRoundTripsHash(t *testing.T) {
dir := t.TempDir()
s := brainstore.New(dir)
ref, err := s.Write(context.Background(), capture.Note{
Content: "# Body\n\ntext\n", Filename: "n", Wing: "a", Hall: "facts",
})
require.NoError(t, err)
note, err := s.Get(context.Background(), ref.ID)
require.NoError(t, err)
assert.Equal(t, ref.ContentHash, note.ContentHash, "write→get hash round-trips")
assert.Equal(t, "a", note.Frontmatter["wing"])
assert.Contains(t, note.Body, "# Body")
}
+148
View File
@@ -0,0 +1,148 @@
// Package capture is the Clean-Architecture use-case for the uniform
// capture capability (issue #49/#51): persist a finished session's
// valuable output — insights → brain, action items → Gitea tickets,
// optional summary → ai-sessions — with one invocation, identical core
// behaviour across every harness.
//
// This package is pure orchestration. It depends only on ports
// (interfaces) and plain entities — no HTTP, no live Gitea, no embedding
// or audit I/O. The real adapters are wired in #52 (Gitea tracker), #53
// (REST + I1 origin gate), and #54/#55 (audit path + relay). The I1
// sovereignty refusal and the classification-aware audit degradation are
// deliberately NOT here — those need the server-derived principal origin
// (#53) and the loki/buffer machinery (#54). What lives here is everything
// testable against fakes: validation, effective-classification resolution
// (stricter wins), best-effort orchestration, and the partial receipt.
package capture
// Zone is the trust zone a capture originates from, server-derived from
// the authenticated principal (spec §4.2 / I1). It is NEVER taken from
// caller input — context.Harness is descriptive telemetry only.
type Zone int
const (
// ZoneUnknown means the origin was not set. The REST adapter always
// sets a concrete zone; the service treats Unknown as "not gated" (only
// an explicit ZoneUSNexus triggers the I1 refusal) so the gate can
// never fire on a caller-controllable default.
ZoneUnknown Zone = iota
// ZoneSovereign is sovereign soil (homelab / Tailscale CLI callers).
ZoneSovereign
// ZoneUSNexus is a non-sovereign US-jurisdiction surface (e.g.
// claude.ai). Confidential captures through it are refused (I1).
ZoneUSNexus
)
// String renders the zone for audit/refusal messages.
func (z Zone) String() string {
switch z {
case ZoneSovereign:
return "sovereign-soil"
case ZoneUSNexus:
return "us-nexus"
default:
return "unknown"
}
}
// CaptureContext is the per-session metadata accompanying a capture.
//
// Classification is the caller-declared sensitivity (model C, spec §4.1):
// the server independently derives the target's classification and gates
// on the stricter of the two. Principal and Origin are server-derived from
// the authenticated identity (the REST adapter populates them); they are
// never caller-asserted. Harness is descriptive telemetry only — never a
// gate input.
type CaptureContext struct {
Harness string
SessionRef string
Fidelity string
Actor string
Classification string // caller-declared level token ("" = unspecified)
Principal string // server-derived (auth); audit identity
Origin Zone // server-derived trust zone; the I1 gate input
}
// Insight is one piece of session knowledge bound for the brain. A
// non-empty SupersedeSlug routes to Update (revise in place); otherwise
// Write (create).
type Insight struct {
Text string
Wing string
Hall string
SupersedeSlug string
}
// Ticket is one action item bound for a Gitea repo. Owner is always the
// operator (set by the tracker adapter), never carried here.
type Ticket struct {
Repo string
Action string // create | close | comment
Number int // required for close/comment
Title string // required for create
Body string
}
// Summary is an optional session summary bound for ai-sessions.
type Summary struct {
Title string
Body string
ReposTouched []string
}
// CaptureInput is the whole capture request.
type CaptureInput struct {
Context CaptureContext
Insights []Insight
Tickets []Ticket
Summary *Summary
DryRun bool
}
// InsightResult is the per-insight outcome in the receipt.
type InsightResult struct {
ID string `json:"id,omitempty"`
Path string `json:"path,omitempty"`
ContentHash string `json:"content_hash,omitempty"`
Superseded bool `json:"superseded"`
OK bool `json:"ok"`
}
// TicketResult is the per-ticket outcome in the receipt.
type TicketResult struct {
Repo string `json:"repo"`
Number int `json:"number,omitempty"`
Action string `json:"action"`
URL string `json:"url,omitempty"`
OK bool `json:"ok"`
}
// SummaryResult is the summary outcome in the receipt.
type SummaryResult struct {
Path string `json:"path,omitempty"`
OK bool `json:"ok"`
}
// ItemError pins a failure to a specific request item for the partial
// receipt. Item is a stable locator like "insight[1]" or "ticket[0]".
type ItemError struct {
Item string `json:"item"`
Error string `json:"error"`
}
// CaptureReceipt is the structured, partial-aware result. Per-item ok
// flags plus a flat Errors list make partial success explicit; the
// caller never has to infer what landed.
type CaptureReceipt struct {
Insights []InsightResult `json:"insights"`
Tickets []TicketResult `json:"tickets"`
Summary *SummaryResult `json:"summary,omitempty"`
Errors []ItemError `json:"errors"`
EffectiveClassification string `json:"effective_classification,omitempty"`
DryRun bool `json:"dry_run"`
// AuditBuffered is true when the central audit sink was unreachable and
// this capture's audit record was written to the durable local buffer
// instead (internal/public tier). Surfaces the degraded state to the
// caller per §4.4.
AuditBuffered bool `json:"audit_buffered,omitempty"`
}
+126
View File
@@ -0,0 +1,126 @@
package capture
import (
"context"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
)
// Ref is the read-after-write handle returned by a brain write/update —
// the #45 contract. ContentHash lets the caller confirm what landed
// without a re-query; for an Update, Superseded is true.
type Ref struct {
ID string
Path string
ContentHash string
Superseded bool
}
// StoredNote is a brain note fetched by Get: the read-after-write
// confirmation primitive (a direct fetch, never a semantic query).
type StoredNote struct {
ID string
Path string
ContentHash string
Frontmatter map[string]string
Body string
}
// Note is the brain-write payload. It carries both the wing/hall taxonomy
// and the legacy type/domain fields so a single BrainStore serves both
// capture insights and the existing MCP brain_write surface. Reason is
// the supersede rationale, used only by Update.
type Note struct {
Content string
Filename string
Wing string
Hall string
Type string
Domain string
Reason string
}
// BrainStore is the brain persistence port — the shared implementation of
// the #45 write/update/get verbs that both the MCP handlers and capture
// call, so there is one implementation, not two. The read-after-write +
// staleness discipline lives behind this interface so no caller carries
// the rule.
type BrainStore interface {
Write(ctx context.Context, n Note) (Ref, error)
Update(ctx context.Context, slug string, n Note) (Ref, error)
Get(ctx context.Context, id string) (StoredNote, error)
}
// IssueRef identifies a ticket touched by the tracker.
type IssueRef struct {
Repo string
Number int
URL string
}
// IssueTracker is the Gitea ticket port. The implementation (#52) always
// scopes to owner "mathias"; the port deliberately omits owner.
type IssueTracker interface {
CreateIssue(ctx context.Context, repo, title, body string) (IssueRef, error)
// CloseIssue closes an issue, optionally posting a closing comment
// first (empty comment ⇒ close only).
CloseIssue(ctx context.Context, repo string, number int, comment string) (IssueRef, error)
CommentIssue(ctx context.Context, repo string, number int, body string) (IssueRef, error)
}
// SummaryWriter is the ai-sessions summary port.
type SummaryWriter interface {
WriteFile(ctx context.Context, repo, path, content string) error
}
// ClassificationPolicy derives a target's sensitivity (model C). The
// "stricter wins" combination of declared vs derived is use-case policy
// and lives in the service, so the port stays minimal. Satisfied by
// classification.Config (#50).
type ClassificationPolicy interface {
Derive(target classification.Target) classification.Level
}
// AuditEntry is the request-level audit record (I5): who/what captured
// what, when, via which principal. SecurityEvents carries anomalies such
// as a caller under-declaring sensitivity relative to the target floor.
type AuditEntry struct {
Timestamp time.Time
Principal string
Actor string
Harness string
SessionRef string
EffectiveClassification string
Items []string
SecurityEvents []string
}
// AuditOutcome is how a capture's audit record was (or will be) persisted.
type AuditOutcome int
const (
// AuditCentral means the record goes to the central sink (loki).
AuditCentral AuditOutcome = iota
// AuditBuffered means the central sink was unreachable and the record
// is written to a durable local buffer for later reconciliation
// (internal/public tier only).
AuditBuffered
)
// AuditSink is the two-phase, classification-aware audit port (I5, §4.4).
//
// Reserve runs BEFORE any write and decides whether the capture can be
// audited at its effective classification: it returns the outcome to use,
// or an error to refuse the capture before anything is written
// (confidential + central sink down → refuse; the all-tiers floor when
// nothing can record → refuse). Record runs AFTER the writes and persists
// the final entry per the reserved outcome.
//
// Splitting reserve from record is what lets "confidential + sink-down →
// refuse before any write" be literally true while the record itself
// (which lists what landed) is necessarily written afterwards.
type AuditSink interface {
Reserve(ctx context.Context, level classification.Level) (AuditOutcome, error)
Record(ctx context.Context, e AuditEntry, outcome AuditOutcome) error
}
+388
View File
@@ -0,0 +1,388 @@
package capture
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"strings"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
)
// Service is the CaptureSession use-case. It depends only on ports.
type Service struct {
brain BrainStore
issues IssueTracker
summaries SummaryWriter
policy ClassificationPolicy
audit AuditSink
// now is the clock, injectable for deterministic summary paths and
// audit timestamps in tests.
now func() time.Time
}
// NewService constructs a Service from its ports. summaries may be nil
// when no summary persistence is wired; a CaptureInput with a Summary
// then fails that item rather than panicking.
func NewService(b BrainStore, tr IssueTracker, sw SummaryWriter, p ClassificationPolicy, a AuditSink) *Service {
return &Service{brain: b, issues: tr, summaries: sw, policy: p, audit: a, now: time.Now}
}
var validActions = map[string]bool{"create": true, "close": true, "comment": true}
// ErrSovereigntyRefused is returned when the I1 gate refuses a capture
// (confidential effective classification through a us-nexus origin). The
// REST adapter maps it to HTTP 403. Callers test with errors.Is.
var ErrSovereigntyRefused = fmt.Errorf("capture refused by I1 sovereignty gate")
// ErrAuditUnavailable is returned when the I5 audit gate refuses a capture
// before any write: a confidential capture whose central audit sink is
// unreachable, or the all-tiers floor where nothing can record the audit.
// The REST adapter maps it to HTTP 503. Callers test with errors.Is.
var ErrAuditUnavailable = fmt.Errorf("capture refused: audit substrate unavailable")
// assertedZoneMismatch returns a security-event string when the caller's
// harness label asserts a trust zone that contradicts the server-derived
// origin. A harness label that names no zone (the normal case, e.g.
// "claude-code") returns "". The label is never used as a gate input —
// this only flags the discrepancy for the audit trail.
func assertedZoneMismatch(harness string, derived Zone) string {
var asserted Zone
switch strings.ToLower(strings.TrimSpace(harness)) {
case "sovereign-soil", "sovereign":
asserted = ZoneSovereign
case "us-nexus", "usnexus":
asserted = ZoneUSNexus
default:
return "" // no zone claim
}
if asserted != derived {
return fmt.Sprintf("asserted-vs-derived origin mismatch: harness asserted %s, principal resolves to %s",
asserted, derived)
}
return ""
}
// Capture runs the use-case: validate (fail-closed), resolve effective
// classification (stricter of declared vs target-derived), then persist
// insights → tickets → summary best-effort, emit an audit record, and
// return a partial-aware receipt.
//
// A validation failure returns a non-nil error with nothing written. A
// per-item execution failure is recorded in the receipt (no rollback);
// the call still returns a nil error so the caller gets the partial
// receipt. The I1 origin gate and audit-down degradation are layered on
// by #53/#54 around this core.
func (s *Service) Capture(ctx context.Context, in CaptureInput) (CaptureReceipt, error) {
if err := s.validate(in); err != nil {
return CaptureReceipt{}, err
}
declared := classification.Public // unspecified ⇒ lowest ⇒ target floor governs
if in.Context.Classification != "" {
// Already validated parseable.
declared, _ = classification.ParseLevel(in.Context.Classification)
}
effective, securityEvents := s.resolveClassification(declared, in)
// Server-derived origin governs the I1 gate; a caller-asserted harness
// label that names a different zone is descriptive-only and logged as a
// security event (spec §4.2: a control keyed on attacker-suppliable
// input is not a control).
if ev := assertedZoneMismatch(in.Context.Harness, in.Context.Origin); ev != "" {
securityEvents = append(securityEvents, ev)
}
// I1 sovereignty gate: a confidential capture through a us-nexus origin
// is refused before ANY write. The refusal itself is audited (best
// effort) — refusals must be reconstructable too.
if effective == classification.Confidential && in.Context.Origin == ZoneUSNexus {
_ = s.audit.Record(ctx, AuditEntry{
Timestamp: s.now().UTC(),
Principal: in.Context.Principal,
Actor: in.Context.Actor,
Harness: in.Context.Harness,
SessionRef: in.Context.SessionRef,
EffectiveClassification: effective.String(),
Items: nil, // refused before any write
SecurityEvents: append(securityEvents, "I1 refusal: confidential capture via us-nexus origin"),
}, AuditCentral)
return CaptureReceipt{}, fmt.Errorf("%w: effective classification confidential through %s origin",
ErrSovereigntyRefused, in.Context.Origin)
}
receipt := CaptureReceipt{
Errors: []ItemError{},
EffectiveClassification: effective.String(),
DryRun: in.DryRun,
}
if in.DryRun {
// Would-be receipt: mark planned items ok, write nothing (not even
// audit — dry_run touches nothing).
for range in.Insights {
receipt.Insights = append(receipt.Insights, InsightResult{OK: true})
}
for _, tk := range in.Tickets {
receipt.Tickets = append(receipt.Tickets, TicketResult{Repo: tk.Repo, Action: tk.Action, Number: tk.Number, OK: true})
}
if in.Summary != nil {
receipt.Summary = &SummaryResult{Path: s.summaryPath(in.Context, in.Summary), OK: true}
}
return receipt, nil
}
// I5 audit gate: decide BEFORE any write whether this capture can be
// audited at its effective classification. Confidential + central sink
// down → refuse here, before writing anything; the all-tiers floor
// (nothing can record) likewise refuses. Internal/public degrade to the
// durable local buffer (signalled by AuditBuffered).
outcome, err := s.audit.Reserve(ctx, effective)
if err != nil {
return CaptureReceipt{}, fmt.Errorf("%w: %v", ErrAuditUnavailable, err)
}
var landed []string
for i, ins := range in.Insights {
res, item, err := s.persistInsight(ctx, ins)
receipt.Insights = append(receipt.Insights, res)
if err != nil {
receipt.Errors = append(receipt.Errors, ItemError{Item: fmt.Sprintf("insight[%d]", i), Error: err.Error()})
continue
}
landed = append(landed, item)
}
for i, tk := range in.Tickets {
res, err := s.persistTicket(ctx, tk)
receipt.Tickets = append(receipt.Tickets, res)
if err != nil {
receipt.Errors = append(receipt.Errors, ItemError{Item: fmt.Sprintf("ticket[%d]", i), Error: err.Error()})
continue
}
landed = append(landed, fmt.Sprintf("ticket:%s#%d", tk.Repo, res.Number))
}
if in.Summary != nil {
res, err := s.persistSummary(ctx, in.Context, in.Summary)
receipt.Summary = &res
if err != nil {
receipt.Errors = append(receipt.Errors, ItemError{Item: "summary", Error: err.Error()})
} else {
landed = append(landed, "summary:"+res.Path)
}
}
// I5: persist the request-level audit record of exactly what landed,
// using the outcome reserved before the writes. AuditBuffered surfaces
// the degraded (locally-buffered) state on the receipt.
if err := s.audit.Record(ctx, AuditEntry{
Timestamp: s.now().UTC(),
Principal: in.Context.Principal,
Actor: in.Context.Actor,
Harness: in.Context.Harness,
SessionRef: in.Context.SessionRef,
EffectiveClassification: effective.String(),
Items: landed,
SecurityEvents: securityEvents,
}, outcome); err != nil {
receipt.Errors = append(receipt.Errors, ItemError{Item: "audit", Error: err.Error()})
}
if outcome == AuditBuffered {
receipt.AuditBuffered = true
}
return receipt, nil
}
// validate enforces fail-closed structural validity over the whole
// request before any write. A bad declared classification, an invalid
// wing/hall, an empty insight, or a malformed ticket aborts the capture
// with nothing written.
func (s *Service) validate(in CaptureInput) error {
if in.Context.Classification != "" {
if _, err := classification.ParseLevel(in.Context.Classification); err != nil {
return fmt.Errorf("context.classification: %w", err)
}
}
for i, ins := range in.Insights {
if strings.TrimSpace(ins.Text) == "" {
return fmt.Errorf("insight[%d]: text is required", i)
}
if strings.TrimSpace(ins.Wing) == "" {
return fmt.Errorf("insight[%d]: wing is required", i)
}
if !brain.IsValidHall(ins.Hall) {
return fmt.Errorf("insight[%d]: invalid hall %q", i, ins.Hall)
}
}
for i, tk := range in.Tickets {
if strings.TrimSpace(tk.Repo) == "" {
return fmt.Errorf("ticket[%d]: repo is required", i)
}
if !validActions[tk.Action] {
return fmt.Errorf("ticket[%d]: invalid action %q (want create/close/comment)", i, tk.Action)
}
if tk.Action == "create" && strings.TrimSpace(tk.Title) == "" {
return fmt.Errorf("ticket[%d]: create requires a title", i)
}
if (tk.Action == "close" || tk.Action == "comment") && tk.Number <= 0 {
return fmt.Errorf("ticket[%d]: %s requires an issue number", i, tk.Action)
}
}
return nil
}
// resolveClassification computes the effective level (stricter of
// declared and every target's derived level) and collects a security
// event whenever the caller under-declared relative to a target floor.
func (s *Service) resolveClassification(declared classification.Level, in CaptureInput) (classification.Level, []string) {
effective := declared
var events []string
consider := func(kind classification.TargetKind, name string) {
derived := s.policy.Derive(classification.Target{Kind: kind, Name: name})
effective = classification.Stricter(effective, derived)
if declared < derived {
events = append(events, fmt.Sprintf("classification under-declared: declared=%s target=%s(%s) derived=%s",
declared, name, kindString(kind), derived))
}
}
for _, ins := range in.Insights {
consider(classification.WingTarget, ins.Wing)
}
for _, tk := range in.Tickets {
consider(classification.RepoTarget, tk.Repo)
}
if in.Summary != nil {
for _, repo := range in.Summary.ReposTouched {
consider(classification.RepoTarget, repo)
}
}
return effective, events
}
func (s *Service) persistInsight(ctx context.Context, ins Insight) (InsightResult, string, error) {
note := Note{Content: ins.Text, Wing: ins.Wing, Hall: ins.Hall, Filename: brain.Sanitise(firstLine(ins.Text))}
var ref Ref
var err error
if ins.SupersedeSlug != "" {
note.Reason = "superseded via capture"
ref, err = s.brain.Update(ctx, ins.SupersedeSlug, note)
} else {
ref, err = s.brain.Write(ctx, note)
}
if err != nil {
return InsightResult{OK: false, Superseded: ins.SupersedeSlug != ""}, "", err
}
return InsightResult{
ID: ref.ID, Path: ref.Path, ContentHash: ref.ContentHash,
Superseded: ref.Superseded, OK: true,
}, "insight:" + ref.ID, nil
}
func (s *Service) persistTicket(ctx context.Context, tk Ticket) (TicketResult, error) {
res := TicketResult{Repo: tk.Repo, Action: tk.Action, Number: tk.Number}
var ref IssueRef
var err error
switch tk.Action {
case "create":
ref, err = s.issues.CreateIssue(ctx, tk.Repo, tk.Title, tk.Body)
case "close":
ref, err = s.issues.CloseIssue(ctx, tk.Repo, tk.Number, tk.Body)
case "comment":
ref, err = s.issues.CommentIssue(ctx, tk.Repo, tk.Number, tk.Body)
}
if err != nil {
return res, err
}
if ref.Number != 0 {
res.Number = ref.Number
}
res.URL = ref.URL
res.OK = true
return res, nil
}
func (s *Service) persistSummary(ctx context.Context, c CaptureContext, sum *Summary) (SummaryResult, error) {
if s.summaries == nil {
return SummaryResult{OK: false}, fmt.Errorf("no summary writer configured")
}
path := s.summaryPath(c, sum)
content := s.renderSummary(c, sum)
repo := "ai-sessions"
if err := s.summaries.WriteFile(ctx, repo, path, content); err != nil {
return SummaryResult{Path: path, OK: false}, err
}
return SummaryResult{Path: path, OK: true}, nil
}
// summaryPath builds summaries/<harness>/<YYYY-MM>/<date>-<slug>-<ref8>.md.
// The ref8 disambiguator is derived from the session_ref (or the title
// when no ref is present) so distinct sessions never collide.
func (s *Service) summaryPath(c CaptureContext, sum *Summary) string {
t := s.now().UTC()
slug := brain.Sanitise(sum.Title)
if slug == "" {
slug = "summary"
}
seed := c.SessionRef
if seed == "" {
seed = sum.Title + sum.Body
}
sum8 := shortHash(seed)
return fmt.Sprintf("summaries/%s/%s/%s-%s-%s.md",
brain.Sanitise(c.Harness), t.Format("2006-01"), t.Format("2006-01-02"), slug, sum8)
}
// renderSummary stamps fidelity + session metadata into frontmatter so the
// richer-fidelity-supersedes-thinner collision rule has the data it needs.
func (s *Service) renderSummary(c CaptureContext, sum *Summary) string {
var b strings.Builder
b.WriteString("---\n")
fmt.Fprintf(&b, "title: %s\n", sum.Title)
fmt.Fprintf(&b, "harness: %s\n", c.Harness)
if c.SessionRef != "" {
fmt.Fprintf(&b, "session_ref: %s\n", c.SessionRef)
}
fmt.Fprintf(&b, "fidelity: %s\n", c.Fidelity)
fmt.Fprintf(&b, "captured_at: %s\n", s.now().UTC().Format(time.RFC3339))
if len(sum.ReposTouched) > 0 {
fmt.Fprintf(&b, "repos_touched: [%s]\n", strings.Join(sum.ReposTouched, ", "))
}
b.WriteString("---\n\n")
b.WriteString(sum.Body)
if !strings.HasSuffix(sum.Body, "\n") {
b.WriteByte('\n')
}
return b.String()
}
func kindString(k classification.TargetKind) string {
if k == classification.RepoTarget {
return "repo"
}
return "wing"
}
func firstLine(s string) string {
s = strings.TrimSpace(s)
if i := strings.IndexByte(s, '\n'); i >= 0 {
s = s[:i]
}
s = strings.TrimLeft(s, "# ")
if len(s) > 60 {
s = s[:60]
}
return s
}
func shortHash(s string) string {
sum := sha256.Sum256([]byte(s))
return hex.EncodeToString(sum[:])[:8]
}
+471
View File
@@ -0,0 +1,471 @@
package capture
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// --- fakes ---
type fakeBrain struct {
writes []Note
updates []Note
gets []string
failOn func(Note) error // nil = always succeed
hashSeq int
}
func (f *fakeBrain) ref(prefix string, n Note, superseded bool) Ref {
f.hashSeq++
path := "wiki/" + n.Wing + "/" + n.Hall + "/" + n.Filename + ".md"
return Ref{ID: path, Path: path, ContentHash: prefix + string(rune('0'+f.hashSeq)), Superseded: superseded}
}
func (f *fakeBrain) Write(_ context.Context, n Note) (Ref, error) {
if f.failOn != nil {
if err := f.failOn(n); err != nil {
return Ref{}, err
}
}
f.writes = append(f.writes, n)
return f.ref("w", n, false), nil
}
func (f *fakeBrain) Update(_ context.Context, slug string, n Note) (Ref, error) {
if f.failOn != nil {
if err := f.failOn(n); err != nil {
return Ref{}, err
}
}
n.Filename = slug
f.updates = append(f.updates, n)
return f.ref("u", n, true), nil
}
func (f *fakeBrain) Get(_ context.Context, id string) (StoredNote, error) {
f.gets = append(f.gets, id)
return StoredNote{ID: id, Path: id}, nil
}
type fakeTracker struct {
created []string
closed []int
comments []int
err error
}
func (f *fakeTracker) CreateIssue(_ context.Context, repo, title, _ string) (IssueRef, error) {
if f.err != nil {
return IssueRef{}, f.err
}
f.created = append(f.created, repo+":"+title)
return IssueRef{Repo: repo, Number: 100 + len(f.created), URL: "https://git/" + repo + "/issues/x"}, nil
}
func (f *fakeTracker) CloseIssue(_ context.Context, repo string, number int, _ string) (IssueRef, error) {
if f.err != nil {
return IssueRef{}, f.err
}
f.closed = append(f.closed, number)
return IssueRef{Repo: repo, Number: number}, nil
}
func (f *fakeTracker) CommentIssue(_ context.Context, repo string, number int, _ string) (IssueRef, error) {
if f.err != nil {
return IssueRef{}, f.err
}
f.comments = append(f.comments, number)
return IssueRef{Repo: repo, Number: number}, nil
}
type fakeSummary struct {
paths []string
content []string
err error
}
func (f *fakeSummary) WriteFile(_ context.Context, _, path, content string) error {
if f.err != nil {
return f.err
}
f.paths = append(f.paths, path)
f.content = append(f.content, content)
return nil
}
// fakePolicy derives from an explicit map; default Internal so tests pin
// behaviour without depending on the real defaulting.
type fakePolicy struct{ tags map[string]classification.Level }
func (p fakePolicy) Derive(t classification.Target) classification.Level {
if lvl, ok := p.tags[t.Name]; ok {
return lvl
}
return classification.Internal
}
type fakeAudit struct {
entries []AuditEntry
err error // Record error
reserveErr error // Reserve error (refuse before write)
reserveMode AuditOutcome
}
func (f *fakeAudit) Reserve(_ context.Context, _ classification.Level) (AuditOutcome, error) {
if f.reserveErr != nil {
return 0, f.reserveErr
}
return f.reserveMode, nil
}
func (f *fakeAudit) Record(_ context.Context, e AuditEntry, _ AuditOutcome) error {
if f.err != nil {
return f.err
}
f.entries = append(f.entries, e)
return nil
}
// --- helpers ---
func newSvc(b BrainStore, tr IssueTracker, sw SummaryWriter, p ClassificationPolicy, a AuditSink) *Service {
s := NewService(b, tr, sw, p, a)
s.now = func() time.Time { return time.Date(2026, 6, 22, 12, 0, 0, 0, time.UTC) }
return s
}
func baseCtx() CaptureContext {
return CaptureContext{Harness: "claude-code", Actor: "mathias", Principal: "mathias", Classification: "internal"}
}
// --- scenarios ---
func TestCaptureHappyPath(t *testing.T) {
b := &fakeBrain{}
tr := &fakeTracker{}
au := &fakeAudit{}
svc := newSvc(b, tr, nil, fakePolicy{}, au)
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{
{Text: "a", Wing: "hyperguild", Hall: "decisions", SupersedeSlug: ""},
{Text: "b", Wing: "hyperguild", Hall: "facts"},
},
Tickets: []Ticket{{Repo: "hyperguild", Action: "create", Title: "do x", Body: "y"}},
})
require.NoError(t, err)
require.Len(t, rec.Insights, 2)
for _, r := range rec.Insights {
assert.True(t, r.OK)
assert.NotEmpty(t, r.ContentHash, "read-after-write hash returned")
}
require.Len(t, rec.Tickets, 1)
assert.True(t, rec.Tickets[0].OK)
assert.Equal(t, 2, len(b.writes))
assert.Empty(t, rec.Errors)
// Audit emitted naming principal/harness + items that landed.
require.Len(t, au.entries, 1)
assert.Equal(t, "mathias", au.entries[0].Principal)
assert.Equal(t, "claude-code", au.entries[0].Harness)
assert.Len(t, au.entries[0].Items, 3)
}
func TestCaptureSupersedeNotDuplicate(t *testing.T) {
b := &fakeBrain{}
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{{Text: "revised", Wing: "hyperguild", Hall: "facts", SupersedeSlug: "prior-note"}},
})
require.NoError(t, err)
assert.Empty(t, b.writes, "supersede must not create")
require.Len(t, b.updates, 1)
assert.Equal(t, "prior-note", b.updates[0].Filename)
assert.True(t, rec.Insights[0].Superseded)
}
func TestCaptureValidationFailClosed(t *testing.T) {
b := &fakeBrain{}
tr := &fakeTracker{}
au := &fakeAudit{}
svc := newSvc(b, tr, nil, fakePolicy{}, au)
_, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{
{Text: "ok", Wing: "hyperguild", Hall: "facts"},
{Text: "bad", Wing: "hyperguild", Hall: "garbage-hall"}, // invalid hall
},
Tickets: []Ticket{{Repo: "hyperguild", Action: "create", Title: "t"}},
})
require.Error(t, err)
// Nothing written anywhere.
assert.Empty(t, b.writes)
assert.Empty(t, b.updates)
assert.Empty(t, tr.created)
assert.Empty(t, au.entries)
}
func TestCaptureValidationRejectsBadTicket(t *testing.T) {
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
_, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Tickets: []Ticket{{Repo: "hyperguild", Action: "frobnicate"}}, // bad action
})
require.Error(t, err)
_, err = svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Tickets: []Ticket{{Repo: "hyperguild", Action: "close"}}, // close needs number
})
require.Error(t, err)
}
func TestCapturePartialFailureBestEffort(t *testing.T) {
b := &fakeBrain{failOn: func(n Note) error {
if strings.Contains(n.Content, "FAIL") {
return errors.New("disk full")
}
return nil
}}
tr := &fakeTracker{}
au := &fakeAudit{}
svc := newSvc(b, tr, nil, fakePolicy{}, au)
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{
{Text: "good one", Wing: "hyperguild", Hall: "facts"},
{Text: "FAIL here", Wing: "hyperguild", Hall: "facts"},
},
Tickets: []Ticket{{Repo: "hyperguild", Action: "create", Title: "t"}},
})
require.NoError(t, err, "partial failure is not a request-level error")
assert.True(t, rec.Insights[0].OK)
assert.False(t, rec.Insights[1].OK)
assert.True(t, rec.Tickets[0].OK, "ticket still persisted; no rollback")
require.Len(t, rec.Errors, 1)
assert.Equal(t, "insight[1]", rec.Errors[0].Item)
// Audit reflects exactly what landed: 1 insight + 1 ticket.
require.Len(t, au.entries, 1)
assert.Len(t, au.entries[0].Items, 2)
}
func TestCaptureDryRunWritesNothing(t *testing.T) {
b := &fakeBrain{}
tr := &fakeTracker{}
au := &fakeAudit{}
svc := newSvc(b, tr, nil, fakePolicy{}, au)
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
DryRun: true,
Insights: []Insight{{Text: "a", Wing: "hyperguild", Hall: "facts"}},
Tickets: []Ticket{{Repo: "hyperguild", Action: "create", Title: "t"}},
})
require.NoError(t, err)
assert.True(t, rec.DryRun)
assert.Len(t, rec.Insights, 1)
assert.True(t, rec.Insights[0].OK, "would-be receipt marks planned items ok")
// Nothing written anywhere, including audit.
assert.Empty(t, b.writes)
assert.Empty(t, tr.created)
assert.Empty(t, au.entries)
}
func TestCaptureStricterClassificationWins(t *testing.T) {
// Caller declares internal; target wing tagged confidential → effective confidential + security event.
b := &fakeBrain{}
au := &fakeAudit{}
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
svc := newSvc(b, &fakeTracker{}, nil, pol, au)
ctx := baseCtx()
ctx.Classification = "internal"
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "client-seb", Hall: "facts"}},
})
require.NoError(t, err)
assert.Equal(t, "confidential", rec.EffectiveClassification)
require.Len(t, au.entries, 1)
assert.NotEmpty(t, au.entries[0].SecurityEvents, "under-declaration logged as security event")
assert.Equal(t, "confidential", au.entries[0].EffectiveClassification)
}
func TestCaptureCallerRaisingSensitivityHonoured(t *testing.T) {
// Caller declares confidential; target internal → effective confidential, NOT a security event.
au := &fakeAudit{}
pol := fakePolicy{tags: map[string]classification.Level{"hyperguild": classification.Internal}}
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, pol, au)
ctx := baseCtx()
ctx.Classification = "confidential"
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "hyperguild", Hall: "facts"}},
})
require.NoError(t, err)
assert.Equal(t, "confidential", rec.EffectiveClassification)
assert.Empty(t, au.entries[0].SecurityEvents, "raising sensitivity is honoured, not flagged")
}
func TestCaptureSummaryPathAndFidelity(t *testing.T) {
sw := &fakeSummary{}
svc := newSvc(&fakeBrain{}, &fakeTracker{}, sw, fakePolicy{}, &fakeAudit{})
ctx := baseCtx()
ctx.Fidelity = "transcript-parse"
ctx.SessionRef = "abc123def456"
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Summary: &Summary{Title: "Session Wrap", Body: "did stuff", ReposTouched: []string{"hyperguild"}},
})
require.NoError(t, err)
require.NotNil(t, rec.Summary)
assert.True(t, rec.Summary.OK)
require.Len(t, sw.paths, 1)
assert.True(t, strings.HasPrefix(sw.paths[0], "summaries/claude-code/2026-06/"), "path: %s", sw.paths[0])
assert.Contains(t, sw.paths[0], "session-wrap")
assert.Contains(t, sw.content[0], "fidelity: transcript-parse", "fidelity stamped in frontmatter")
}
// --- I1 sovereignty gate (#53) ---
func TestCaptureRefusesConfidentialViaUSNexus(t *testing.T) {
b := &fakeBrain{}
tr := &fakeTracker{}
au := &fakeAudit{}
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
svc := newSvc(b, tr, nil, pol, au)
ctx := baseCtx()
ctx.Classification = "confidential"
ctx.Origin = ZoneUSNexus
_, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "client-seb", Hall: "facts"}},
})
require.Error(t, err)
assert.ErrorIs(t, err, ErrSovereigntyRefused)
// Refused before any write.
assert.Empty(t, b.writes)
assert.Empty(t, tr.created)
// Refusal is audited.
require.Len(t, au.entries, 1)
assert.Empty(t, au.entries[0].Items, "no items landed on refusal")
}
func TestCaptureAllowsConfidentialViaSovereign(t *testing.T) {
b := &fakeBrain{}
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
svc := newSvc(b, &fakeTracker{}, nil, pol, &fakeAudit{})
ctx := baseCtx()
ctx.Classification = "confidential"
ctx.Origin = ZoneSovereign
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "client-seb", Hall: "facts"}},
})
require.NoError(t, err)
assert.True(t, rec.Insights[0].OK)
assert.Len(t, b.writes, 1)
}
func TestCaptureAssertedLabelIgnoredAndLogged(t *testing.T) {
// Caller asserts harness "sovereign-soil" but principal resolves to
// us-nexus; confidential ⇒ refused, and the discrepancy is a security event.
au := &fakeAudit{}
pol := fakePolicy{tags: map[string]classification.Level{"client-seb": classification.Confidential}}
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, pol, au)
ctx := baseCtx()
ctx.Harness = "sovereign-soil" // asserted
ctx.Origin = ZoneUSNexus // server-derived
ctx.Classification = "confidential"
_, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "client-seb", Hall: "facts"}},
})
require.ErrorIs(t, err, ErrSovereigntyRefused)
require.Len(t, au.entries, 1)
joined := strings.Join(au.entries[0].SecurityEvents, " | ")
assert.Contains(t, joined, "asserted-vs-derived origin mismatch")
assert.Contains(t, joined, "I1 refusal")
}
func TestCaptureInternalViaUSNexusAllowed(t *testing.T) {
// us-nexus origin is fine for non-confidential data.
b := &fakeBrain{}
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, &fakeAudit{})
ctx := baseCtx()
ctx.Origin = ZoneUSNexus // internal classification, so gate doesn't fire
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: ctx,
Insights: []Insight{{Text: "x", Wing: "hyperguild", Hall: "facts"}},
})
require.NoError(t, err)
assert.True(t, rec.Insights[0].OK)
}
// --- I5 audit gate (#54) ---
func TestCaptureRefusesWhenAuditReserveFails(t *testing.T) {
// Reserve refusing (e.g. confidential + central sink down, or the floor)
// aborts the capture before any write.
b := &fakeBrain{}
tr := &fakeTracker{}
au := &fakeAudit{reserveErr: errors.New("central sink unreachable")}
svc := newSvc(b, tr, nil, fakePolicy{}, au)
_, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{{Text: "x", Wing: "hyperguild", Hall: "facts"}},
})
require.Error(t, err)
assert.ErrorIs(t, err, ErrAuditUnavailable)
assert.Empty(t, b.writes, "nothing written when audit unavailable")
assert.Empty(t, tr.created)
}
func TestCaptureFlagsLocallyBufferedAudit(t *testing.T) {
// Reserve returns AuditBuffered (internal/public, central down) → capture
// proceeds and the receipt flags the degraded audit state.
b := &fakeBrain{}
au := &fakeAudit{reserveMode: AuditBuffered}
svc := newSvc(b, &fakeTracker{}, nil, fakePolicy{}, au)
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
Insights: []Insight{{Text: "x", Wing: "hyperguild", Hall: "facts"}},
})
require.NoError(t, err)
assert.True(t, rec.Insights[0].OK, "capture proceeds on degraded audit")
assert.True(t, rec.AuditBuffered, "receipt flags locally-buffered audit")
require.Len(t, au.entries, 1)
}
func TestCaptureDryRunSkipsAuditGate(t *testing.T) {
// dry_run must not even probe the audit sink (writes nothing anywhere).
au := &fakeAudit{reserveErr: errors.New("would refuse")}
svc := newSvc(&fakeBrain{}, &fakeTracker{}, nil, fakePolicy{}, au)
rec, err := svc.Capture(context.Background(), CaptureInput{
Context: baseCtx(),
DryRun: true,
Insights: []Insight{{Text: "x", Wing: "hyperguild", Hall: "facts"}},
})
require.NoError(t, err, "dry-run does not hit the audit gate")
assert.True(t, rec.DryRun)
assert.Empty(t, au.entries)
}
+232
View File
@@ -0,0 +1,232 @@
// Package capturehttp is the REST adapter for the capture use-case: the
// POST /capture door (#53). It is deliberately thin — authenticate, derive
// the trust-zone origin from the authenticated principal, decode the
// request, call capture.Service, map the receipt to an HTTP status. No
// business logic lives here; the I1 gate, validation, and orchestration
// are all in the use-case.
package capturehttp
import (
"context"
"crypto/subtle"
"encoding/json"
"errors"
"io"
"net/http"
"strings"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
)
// Validator validates a Bearer JWT and returns its subject. The chassis
// *auth.JWTValidator satisfies it (including its nil-receiver "disabled"
// behaviour), and tests can substitute a fake without a live JWKS.
type Validator interface {
Validate(ctx context.Context, rawToken string) (string, error)
}
// Handler serves POST /capture.
type Handler struct {
svc *capture.Service
validator Validator // nil ⇒ JWT auth disabled
staticToken string // "" ⇒ static auth disabled
staticPrincipal string // principal name attributed to static-token callers
resolver OriginResolver
}
// New constructs a capture HTTP handler. staticToken callers are
// attributed to staticPrincipal (a sovereign homelab identity); JWT
// callers are attributed to their token subject.
func New(svc *capture.Service, validator Validator, staticToken, staticPrincipal string, resolver OriginResolver) *Handler {
if staticPrincipal == "" {
staticPrincipal = "local-cli"
}
return &Handler{
svc: svc,
validator: validator,
staticToken: staticToken,
staticPrincipal: staticPrincipal,
resolver: resolver,
}
}
// wire types — the POST /capture request body.
type request struct {
Context contextBody `json:"context"`
Insights []insightBody `json:"insights"`
Tickets []ticketBody `json:"tickets"`
Summary *summaryBody `json:"summary,omitempty"`
DryRun bool `json:"dry_run"`
}
type contextBody struct {
Harness string `json:"harness"`
SessionRef string `json:"session_ref"`
Fidelity string `json:"fidelity"`
Actor string `json:"actor"`
Classification string `json:"classification"`
}
type insightBody struct {
Text string `json:"text"`
Wing string `json:"wing"`
Hall string `json:"hall"`
SupersedeSlug string `json:"supersede_slug,omitempty"`
}
type ticketBody struct {
Repo string `json:"repo"`
Action string `json:"action"`
Number int `json:"number,omitempty"`
Title string `json:"title,omitempty"`
Body string `json:"body,omitempty"`
}
type summaryBody struct {
Title string `json:"title"`
Body string `json:"body"`
ReposTouched []string `json:"repos_touched,omitempty"`
}
// ServeHTTP authenticates, derives origin, runs the use-case, and maps the
// result to an HTTP status.
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
principal, viaStatic, ok := Authenticate(r, h.staticToken, h.staticPrincipal, h.validator)
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
body, err := io.ReadAll(r.Body)
if err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "read body"})
return
}
in, err := DecodeRequest(body)
if err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "invalid JSON"})
return
}
// Principal and origin are server-derived — overwrite anything the
// caller may have tried to put in the body.
in.Context.Principal = principal
in.Context.Origin = h.resolver.Resolve(principal, viaStatic)
rec, err := h.svc.Capture(r.Context(), in)
switch {
case errors.Is(err, capture.ErrSovereigntyRefused):
writeJSON(w, http.StatusForbidden, map[string]string{"error": err.Error()})
return
case errors.Is(err, capture.ErrAuditUnavailable):
// I5 refusal: confidential + audit sink down, or the all-tiers floor.
writeJSON(w, http.StatusServiceUnavailable, map[string]string{"error": err.Error()})
return
case err != nil:
// Pre-write validation failure (fail-closed).
writeJSON(w, http.StatusBadRequest, map[string]string{"error": err.Error()})
return
}
writeJSON(w, statusFor(rec), rec)
}
// Authenticate mirrors the chassis Bearer precedence (static token wins,
// then Dex JWT) and returns the resolved principal plus whether the static
// path was taken — the chassis middleware hides both, and capture (REST or
// MCP) needs them to derive the trust-zone origin. ok is false when no
// credential matched.
func Authenticate(r *http.Request, staticToken, staticPrincipal string, validator Validator) (principal string, viaStatic, ok bool) {
raw, found := strings.CutPrefix(r.Header.Get("Authorization"), "Bearer ")
if !found || raw == "" {
return "", false, false
}
if staticToken != "" && subtle.ConstantTimeCompare([]byte(raw), []byte(staticToken)) == 1 {
return staticPrincipal, true, true
}
if validator != nil {
if sub, err := validator.Validate(r.Context(), raw); err == nil && sub != "" {
return sub, false, true
}
}
return "", false, false
}
// DecodeRequest parses a capture request body into a CaptureInput. Shared
// by the REST adapter and the MCP capture tool so the wire shape has one
// definition. Principal and Origin are NOT set here — the caller sets them
// from the authenticated identity.
func DecodeRequest(data []byte) (capture.CaptureInput, error) {
var b request
if err := json.Unmarshal(data, &b); err != nil {
return capture.CaptureInput{}, err
}
return b.toInput(), nil
}
func (b request) toInput() capture.CaptureInput {
in := capture.CaptureInput{
Context: capture.CaptureContext{
Harness: b.Context.Harness,
SessionRef: b.Context.SessionRef,
Fidelity: b.Context.Fidelity,
Actor: b.Context.Actor,
Classification: b.Context.Classification,
},
DryRun: b.DryRun,
}
for _, i := range b.Insights {
in.Insights = append(in.Insights, capture.Insight{
Text: i.Text, Wing: i.Wing, Hall: i.Hall, SupersedeSlug: i.SupersedeSlug,
})
}
for _, t := range b.Tickets {
in.Tickets = append(in.Tickets, capture.Ticket{
Repo: t.Repo, Action: t.Action, Number: t.Number, Title: t.Title, Body: t.Body,
})
}
if b.Summary != nil {
in.Summary = &capture.Summary{
Title: b.Summary.Title, Body: b.Summary.Body, ReposTouched: b.Summary.ReposTouched,
}
}
return in
}
// statusFor maps a receipt to an HTTP status: 200 all-ok (or dry-run),
// 207 partial, 502 everything-failed.
func statusFor(rec capture.CaptureReceipt) int {
if rec.DryRun {
return http.StatusOK
}
var ok, fail int
for _, i := range rec.Insights {
count(&ok, &fail, i.OK)
}
for _, t := range rec.Tickets {
count(&ok, &fail, t.OK)
}
if rec.Summary != nil {
count(&ok, &fail, rec.Summary.OK)
}
switch {
case fail == 0:
return http.StatusOK
case ok == 0:
return http.StatusBadGateway // every persistence attempt failed
default:
return http.StatusMultiStatus // 207: partial success
}
}
func count(ok, fail *int, isOK bool) {
if isOK {
*ok++
} else {
*fail++
}
}
func writeJSON(w http.ResponseWriter, status int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(v)
}
@@ -0,0 +1,198 @@
package capturehttp_test
import (
"bytes"
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const staticTok = "static-secret"
// fakeValidator stands in for the chassis JWT validator.
type fakeValidator struct {
subject string
err error
}
func (f fakeValidator) Validate(context.Context, string) (string, error) {
return f.subject, f.err
}
type fakeTracker struct{ failCreate bool }
func (f fakeTracker) CreateIssue(context.Context, string, string, string) (capture.IssueRef, error) {
if f.failCreate {
return capture.IssueRef{}, errors.New("gitea down")
}
return capture.IssueRef{Repo: "hyperguild", Number: 1, URL: "https://git/1"}, nil
}
func (fakeTracker) CloseIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func (fakeTracker) CommentIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func newHandler(t *testing.T, v capturehttp.Validator, tr capture.IssueTracker, sovereign []string) *capturehttp.Handler {
t.Helper()
cfg, err := classification.Load(t.TempDir())
require.NoError(t, err)
svc := capture.NewService(brainstore.New(t.TempDir()), tr, nil, cfg, audit.NewSlogSink(nil))
return capturehttp.New(svc, v, staticTok, "local-cli", capturehttp.NewOriginResolver(sovereign))
}
func do(t *testing.T, h *capturehttp.Handler, authz string, body any) *httptest.ResponseRecorder {
t.Helper()
b, _ := json.Marshal(body)
req := httptest.NewRequest(http.MethodPost, "/capture", bytes.NewReader(b))
if authz != "" {
req.Header.Set("Authorization", authz)
}
rr := httptest.NewRecorder()
h.ServeHTTP(rr, req)
return rr
}
func internalReq() map[string]any {
return map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "a fact", "wing": "hyperguild", "hall": "facts"}},
}
}
func TestUnauthorizedWithoutToken(t *testing.T) {
h := newHandler(t, fakeValidator{err: errors.New("no")}, fakeTracker{}, nil)
rr := do(t, h, "", internalReq())
assert.Equal(t, http.StatusUnauthorized, rr.Code)
}
func TestUnauthorizedBadToken(t *testing.T) {
h := newHandler(t, fakeValidator{err: errors.New("bad jwt")}, fakeTracker{}, nil)
rr := do(t, h, "Bearer wrong", internalReq())
assert.Equal(t, http.StatusUnauthorized, rr.Code)
}
func TestHappyPathStaticToken(t *testing.T) {
h := newHandler(t, nil, fakeTracker{}, nil)
rr := do(t, h, "Bearer "+staticTok, map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "a fact", "wing": "hyperguild", "hall": "facts"}},
"tickets": []map[string]any{{"repo": "hyperguild", "action": "create", "title": "t"}},
})
require.Equal(t, http.StatusOK, rr.Code)
var rec capture.CaptureReceipt
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &rec))
assert.True(t, rec.Insights[0].OK)
assert.True(t, rec.Tickets[0].OK)
assert.Empty(t, rec.Errors)
}
func TestConfidentialViaUSNexusRefused(t *testing.T) {
// JWT principal not in the sovereign allowlist ⇒ us-nexus; confidential ⇒ 403.
h := newHandler(t, fakeValidator{subject: "claudeai-oauth-client"}, fakeTracker{}, nil)
rr := do(t, h, "Bearer jwt-token", map[string]any{
"context": map[string]any{"harness": "claudeai-chat", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
assert.Equal(t, http.StatusForbidden, rr.Code)
assert.Contains(t, rr.Body.String(), "sovereignty")
}
func TestConfidentialViaSovereignJWTAllowed(t *testing.T) {
// Same confidential payload, but the principal is allowlisted sovereign ⇒ allowed.
h := newHandler(t, fakeValidator{subject: "koala-cli"}, fakeTracker{}, []string{"koala-cli"})
rr := do(t, h, "Bearer jwt-token", map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
require.Equal(t, http.StatusOK, rr.Code)
}
func TestStaticTokenIsSovereignSoConfidentialAllowed(t *testing.T) {
h := newHandler(t, nil, fakeTracker{}, nil)
rr := do(t, h, "Bearer "+staticTok, map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
assert.Equal(t, http.StatusOK, rr.Code)
}
func TestValidationRejectedBeforeWrite(t *testing.T) {
h := newHandler(t, nil, fakeTracker{}, nil)
rr := do(t, h, "Bearer "+staticTok, map[string]any{
"context": map[string]any{"actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "x", "wing": "hyperguild", "hall": "not-a-hall"}},
})
assert.Equal(t, http.StatusBadRequest, rr.Code)
}
func TestPartialFailureIs207(t *testing.T) {
h := newHandler(t, nil, fakeTracker{failCreate: true}, nil)
rr := do(t, h, "Bearer "+staticTok, map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "ok insight", "wing": "hyperguild", "hall": "facts"}},
"tickets": []map[string]any{{"repo": "hyperguild", "action": "create", "title": "fails"}},
})
assert.Equal(t, http.StatusMultiStatus, rr.Code)
var rec capture.CaptureReceipt
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &rec))
assert.True(t, rec.Insights[0].OK)
assert.False(t, rec.Tickets[0].OK)
assert.Len(t, rec.Errors, 1)
}
func TestDryRunWritesNothing(t *testing.T) {
h := newHandler(t, nil, fakeTracker{}, nil)
rr := do(t, h, "Bearer "+staticTok, map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "a", "wing": "hyperguild", "hall": "facts"}},
"dry_run": true,
})
require.Equal(t, http.StatusOK, rr.Code)
var rec capture.CaptureReceipt
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &rec))
assert.True(t, rec.DryRun)
}
func TestCallerCannotForgeOrigin(t *testing.T) {
// Even if the body tried to assert a sovereign harness, a us-nexus JWT
// principal + confidential ⇒ refused. (Origin is server-derived.)
h := newHandler(t, fakeValidator{subject: "claudeai-oauth-client"}, fakeTracker{}, nil)
rr := do(t, h, "Bearer jwt", map[string]any{
"context": map[string]any{"harness": "sovereign-soil", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
assert.Equal(t, http.StatusForbidden, rr.Code)
}
// refusingAudit refuses at Reserve (e.g. confidential + loki down, or floor).
type refusingAudit struct{}
func (refusingAudit) Reserve(context.Context, classification.Level) (capture.AuditOutcome, error) {
return 0, errors.New("central audit sink unreachable")
}
func (refusingAudit) Record(context.Context, capture.AuditEntry, capture.AuditOutcome) error {
return nil
}
func TestAuditUnavailableIs503(t *testing.T) {
cfg, err := classification.Load(t.TempDir())
require.NoError(t, err)
svc := capture.NewService(brainstore.New(t.TempDir()), fakeTracker{}, nil, cfg, refusingAudit{})
h := capturehttp.New(svc, nil, staticTok, "local-cli", capturehttp.NewOriginResolver(nil))
rr := do(t, h, "Bearer "+staticTok, internalReq())
assert.Equal(t, http.StatusServiceUnavailable, rr.Code)
}
+42
View File
@@ -0,0 +1,42 @@
package capturehttp
import "github.com/mathiasbq/hyperguild/ingestion/internal/capture"
// OriginResolver maps an authenticated principal to its trust zone
// (spec §4.2). The mapping is server-side and never reads caller input.
//
// Rules:
// - The static-token path is a homelab CLI caller on sovereign soil →
// ZoneSovereign.
// - A JWT principal in the sovereign allowlist → ZoneSovereign.
// - Any other JWT principal (e.g. claude.ai's OAuth identity, or any
// unrecognised subject) → ZoneUSNexus.
//
// The default is the strict one: an unknown principal is treated as
// us-nexus so the I1 gate fails safe (refuses confidential), exactly as
// an untagged classification target fails safe to confidential (#50).
type OriginResolver struct {
sovereign map[string]bool
}
// NewOriginResolver builds a resolver whose JWT sovereign principals are
// the given subjects. The static-token caller is always sovereign and
// need not be listed.
func NewOriginResolver(sovereignPrincipals []string) OriginResolver {
m := make(map[string]bool, len(sovereignPrincipals))
for _, p := range sovereignPrincipals {
if p != "" {
m[p] = true
}
}
return OriginResolver{sovereign: m}
}
// Resolve returns the trust zone for a principal. viaStatic is true when
// the static-token auth path was taken.
func (r OriginResolver) Resolve(principal string, viaStatic bool) capture.Zone {
if viaStatic || r.sovereign[principal] {
return capture.ZoneSovereign
}
return capture.ZoneUSNexus
}
@@ -0,0 +1,189 @@
// Package classification defines the data-sensitivity taxonomy and the
// per-wing / per-repo tagging the capture server reads to enforce the I1
// sovereignty gate (issue #50, capture spec §4.1).
//
// The single load-bearing property is fail-safe-to-strictest: a target
// with no explicit tag and no known default classifies as Confidential,
// never as something more permissive. A missing tag must never silently
// downgrade — that would turn the I1 gate into theatre.
//
// Classification is read from an optional classification.yaml at the
// brain root. A central, Flux-reconcilable file is deliberate: it is
// auditable in one place (I2/I5), it does not require a live Gitea client
// to classify a repo (so this package has no dependency on the gitea
// tracker work), and it avoids tagging a wing's _index.md frontmatter —
// which BuildWingIndex regenerates and would clobber.
package classification
import (
"fmt"
"os"
"path/filepath"
"strings"
"gopkg.in/yaml.v3"
)
// Level is a data-sensitivity tier. Higher is stricter, so the "stricter
// wins" rule (spec §4.1 model C) is a plain max.
type Level int
const (
Public Level = iota
Internal
Confidential
)
// String returns the canonical lowercase token for a level.
func (l Level) String() string {
switch l {
case Public:
return "public"
case Internal:
return "internal"
case Confidential:
return "confidential"
default:
return fmt.Sprintf("level(%d)", int(l))
}
}
// ParseLevel parses a level token (case-insensitive, surrounding space
// tolerated). An unknown token is an error — callers must decide what to
// do with bad input rather than have it silently coerced.
func ParseLevel(s string) (Level, error) {
switch strings.ToLower(strings.TrimSpace(s)) {
case "public":
return Public, nil
case "internal":
return Internal, nil
case "confidential":
return Confidential, nil
default:
return Confidential, fmt.Errorf("unknown classification level %q (want public/internal/confidential)", s)
}
}
// Stricter returns the more restrictive of two levels.
func Stricter(a, b Level) Level {
if a > b {
return a
}
return b
}
// TargetKind distinguishes the two kinds of capture destination.
type TargetKind int
const (
WingTarget TargetKind = iota // a brain wing (insights land here)
RepoTarget // a Gitea repo (tickets / summaries land here)
)
// Target names a capture destination to classify.
type Target struct {
Kind TargetKind
Name string
}
// Config holds the explicit per-wing / per-repo classification tags read
// from classification.yaml. Absent entries fall through to the built-in
// defaults in defaultFor. The zero value (no file) is valid and applies
// defaults to everything.
type Config struct {
wings map[string]Level
repos map[string]Level
}
// rawConfig is the on-disk YAML shape: string→string maps, parsed into
// validated levels by Load.
type rawConfig struct {
Wings map[string]string `yaml:"wings"`
Repos map[string]string `yaml:"repos"`
}
// Load reads classification.yaml from brainDir. An absent file is not an
// error — it yields an empty config where every target classifies by the
// built-in defaults. A malformed file, or any unparseable level token in
// it, is a hard error: a classification source the server cannot trust
// must fail loud, not degrade silently.
func Load(brainDir string) (*Config, error) {
cfg := &Config{wings: map[string]Level{}, repos: map[string]Level{}}
data, err := os.ReadFile(filepath.Join(brainDir, "classification.yaml"))
if err != nil {
if os.IsNotExist(err) {
return cfg, nil
}
return nil, fmt.Errorf("read classification.yaml: %w", err)
}
var raw rawConfig
if err := yaml.Unmarshal(data, &raw); err != nil {
return nil, fmt.Errorf("parse classification.yaml: %w", err)
}
for name, lvl := range raw.Wings {
parsed, perr := ParseLevel(lvl)
if perr != nil {
return nil, fmt.Errorf("wing %q: %w", name, perr)
}
cfg.wings[normalise(name)] = parsed
}
for name, lvl := range raw.Repos {
parsed, perr := ParseLevel(lvl)
if perr != nil {
return nil, fmt.Errorf("repo %q: %w", name, perr)
}
cfg.repos[normalise(name)] = parsed
}
return cfg, nil
}
// Derive returns the classification for any target — the function the
// capture use-case calls per item.
func (c *Config) Derive(t Target) Level {
if t.Kind == RepoTarget {
return c.Repo(t.Name)
}
return c.Wing(t.Name)
}
// Wing classifies a brain wing: an explicit tag wins, else defaults.
func (c *Config) Wing(name string) Level {
if lvl, ok := c.wings[normalise(name)]; ok {
return lvl
}
return defaultFor(name)
}
// Repo classifies a Gitea repo: an explicit tag wins, else defaults.
func (c *Config) Repo(name string) Level {
if lvl, ok := c.repos[normalise(name)]; ok {
return lvl
}
return defaultFor(name)
}
// defaultFor applies the built-in defaulting rules when a target has no
// explicit tag:
// - client-* → Confidential (client work is confidential by default)
// - hyperguild / homelab → Internal (the operator's own infra)
// - everything else → Confidential (fail safe to strictest)
func defaultFor(name string) Level {
n := normalise(name)
if strings.HasPrefix(n, "client-") {
return Confidential
}
switch n {
case "hyperguild", "homelab":
return Internal
default:
return Confidential
}
}
// normalise lowercases and trims a wing/repo name so matching and the
// client-* prefix check are case-insensitive.
func normalise(name string) string {
return strings.ToLower(strings.TrimSpace(name))
}
@@ -0,0 +1,112 @@
package classification
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLevelOrderingAndString(t *testing.T) {
assert.True(t, Public < Internal)
assert.True(t, Internal < Confidential)
assert.Equal(t, "public", Public.String())
assert.Equal(t, "internal", Internal.String())
assert.Equal(t, "confidential", Confidential.String())
}
func TestParseLevel(t *testing.T) {
for s, want := range map[string]Level{
"public": Public, "internal": Internal, "confidential": Confidential,
"PUBLIC": Public, " Confidential ": Confidential,
} {
got, err := ParseLevel(s)
require.NoError(t, err, s)
assert.Equal(t, want, got, s)
}
_, err := ParseLevel("secret")
require.Error(t, err, "unknown level must error, not silently default")
_, err = ParseLevel("")
require.Error(t, err)
}
func TestStricterReturnsMax(t *testing.T) {
assert.Equal(t, Confidential, Stricter(Internal, Confidential))
assert.Equal(t, Confidential, Stricter(Confidential, Public))
assert.Equal(t, Internal, Stricter(Public, Internal))
assert.Equal(t, Public, Stricter(Public, Public))
}
func TestLoadAbsentFileIsDefaultsOnly(t *testing.T) {
cfg, err := Load(t.TempDir())
require.NoError(t, err, "absent classification.yaml must not be an error — defaults apply")
require.NotNil(t, cfg)
// Pure defaulting still works.
assert.Equal(t, Internal, cfg.Wing("hyperguild"))
assert.Equal(t, Confidential, cfg.Wing("anything-unknown"))
}
func TestLoadParsesExplicitTags(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "classification.yaml"), []byte(
"wings:\n research-public: public\n hyperguild: confidential\nrepos:\n infra: internal\n research-public: public\n",
), 0o644))
cfg, err := Load(dir)
require.NoError(t, err)
// Explicit tag wins over the built-in default (hyperguild default is internal).
assert.Equal(t, Confidential, cfg.Wing("hyperguild"))
// Explicit public is honoured.
assert.Equal(t, Public, cfg.Wing("research-public"))
assert.Equal(t, Internal, cfg.Repo("infra"))
assert.Equal(t, Public, cfg.Repo("research-public"))
}
func TestLoadRejectsUnknownLevelInFile(t *testing.T) {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, "classification.yaml"),
[]byte("wings:\n x: top-secret\n"), 0o644))
_, err := Load(dir)
require.Error(t, err, "an unparseable level in the config must fail loud, not be ignored")
}
func TestWingDefaulting(t *testing.T) {
cfg, err := Load(t.TempDir())
require.NoError(t, err)
cases := map[string]Level{
"client-seb": Confidential, // client-* → confidential
"client-mastercard": Confidential,
"hyperguild": Internal,
"homelab": Internal,
"jepa-fx": Confidential, // unknown → fail safe to strictest
"": Confidential, // empty → fail safe
}
for wing, want := range cases {
assert.Equal(t, want, cfg.Wing(wing), "wing %q", wing)
}
}
func TestRepoDefaulting(t *testing.T) {
cfg, err := Load(t.TempDir())
require.NoError(t, err)
assert.Equal(t, Confidential, cfg.Repo("client-seb-pipeline"))
assert.Equal(t, Internal, cfg.Repo("hyperguild"))
assert.Equal(t, Confidential, cfg.Repo("some-unknown-repo"), "untagged repo → confidential (fail safe)")
}
func TestDeriveUnifiedTarget(t *testing.T) {
cfg, err := Load(t.TempDir())
require.NoError(t, err)
assert.Equal(t, Internal, cfg.Derive(Target{Kind: WingTarget, Name: "homelab"}))
assert.Equal(t, Confidential, cfg.Derive(Target{Kind: RepoTarget, Name: "client-x"}))
assert.Equal(t, Confidential, cfg.Derive(Target{Kind: WingTarget, Name: "untagged"}))
}
func TestCaseInsensitiveMatching(t *testing.T) {
cfg, err := Load(t.TempDir())
require.NoError(t, err)
assert.Equal(t, Confidential, cfg.Wing("Client-SEB"), "client- prefix match is case-insensitive")
assert.Equal(t, Internal, cfg.Wing("HyperGuild"))
}
+66 -2
View File
@@ -1,6 +1,10 @@
package claudewatcher
import "regexp"
import (
"fmt"
"regexp"
"sync"
)
// Scrubber drops any turn whose content matches a known-bad pattern.
// Fail-closed by design: we'd rather lose signal than ingest credentials
@@ -34,21 +38,74 @@ var DefaultRules = []Rule{
// specific match name in logs.
{Name: "authorization-header", RE: regexp.MustCompile(`(?i)Authorization\s*:\s*[A-Za-z]+\s+\S{8,}`)},
{Name: "bearer-token", RE: regexp.MustCompile(`(?i)Bearer\s+[A-Za-z0-9._\-]{16,}`)},
// JWT (header.payload.sig), e.g. a Dex/OAuth token dumped to stdout
// without a "Bearer " prefix. Both header and payload base64url-encode
// JSON, so both segments begin with "eyJ".
{Name: "jwt", RE: regexp.MustCompile(`eyJ[A-Za-z0-9_\-]{8,}\.eyJ[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}`)},
{Name: "postgres-uri-with-password", RE: regexp.MustCompile(`postgres(?:ql)?://[^:\s/]+:[^@\s/]+@`)},
{Name: "private-key", RE: regexp.MustCompile(`-----BEGIN[^-]*PRIVATE KEY-----`)},
{Name: "ssh-key", RE: regexp.MustCompile(`ssh-(?:rsa|ed25519|ecdsa)\s+[A-Za-z0-9+/=]{40,}`)},
{Name: "github-pat", RE: regexp.MustCompile(`\b(?:ghp|gho|ghu|ghr|gha)_[A-Za-z0-9]{30,}\b`)},
{Name: "openai-sk", RE: regexp.MustCompile(`\bsk-(?:proj-)?[A-Za-z0-9]{32,}\b`)},
// 1Password service-account token (ops_<base64url>). Long, high-value root
// credential; guard the bare value (the _TOKEN= form also hits homelab-env-token).
{Name: "op-service-account", RE: regexp.MustCompile(`\bops_[A-Za-z0-9_\-]{40,}`)},
// No leading \b: a shell mangle can glue the key to a preceding word
// ("yes"+"sk-...") which has no word boundary, and that exact case
// leaked a LiteLLM master key past this rule (2026-06-11). Match the
// sk- shape wherever it appears; the {32,} length floor keeps short
// "task-"/"disk-" words from tripping it.
{Name: "openai-sk", RE: regexp.MustCompile(`sk-(?:proj-)?[A-Za-z0-9]{32,}`)},
{Name: "anthropic-sk", RE: regexp.MustCompile(`\bsk-ant-[A-Za-z0-9_\-]{32,}\b`)},
{Name: "aws-access-key", RE: regexp.MustCompile(`\bAKIA[0-9A-Z]{16}\b`)},
{Name: "homelab-env-token", RE: regexp.MustCompile(`(?i)(?:_TOKEN|_PASSWORD|_API_KEY|_SECRET)\s*[:=]\s*['"]?[A-Za-z0-9._/+\-]{12,}`)},
{Name: "sops-encrypted-marker", RE: regexp.MustCompile(`ENC\[AES256_GCM,data:[A-Za-z0-9+/=]{8,}`)},
}
// extraRules is appended to DefaultRules at process startup via
// RegisterRule. The mutex guards concurrent RegisterRule calls (rare)
// against concurrent Scrub reads (hot path). Scrub takes a read lock
// only when extraRules is non-empty, so steady-state cost is zero
// when no client-name guard is configured.
var (
extraRulesMu sync.RWMutex
extraRules []Rule
)
// RegisterRule appends a runtime-configured regex to the scrubber's
// rule set. Used by main to inject client-name guards from
// CLAUDE_INGEST_CLIENT_BLOCK env var (or equivalent SOPS-encrypted
// secret) without baking client identities into source code.
//
// pattern is compiled as-is — callers wrap with `\b...\b` and case
// flags as needed. Duplicate names are accepted (rules are positional);
// the second registration just fires after the first.
func RegisterRule(name, pattern string) error {
re, err := regexp.Compile(pattern)
if err != nil {
return fmt.Errorf("compile rule %q: %w", name, err)
}
extraRulesMu.Lock()
extraRules = append(extraRules, Rule{Name: name, RE: re})
extraRulesMu.Unlock()
return nil
}
// ResetExtraRules clears every RegisterRule-added rule. Test-only.
func ResetExtraRules() {
extraRulesMu.Lock()
extraRules = nil
extraRulesMu.Unlock()
}
// Scrub reports the first matching rule, or empty when content is clean.
// Empty string is treated as clean. Caller decides what to do on a hit;
// the convention in claudewatcher is to drop the turn entirely and emit
// a slog.Warn naming the rule.
//
// Rule order: DefaultRules first (credential shapes), then runtime
// RegisterRule additions (client-name guards). Credential leaks
// outrank client-name hits in the log because they're strictly more
// dangerous.
func Scrub(content string) string {
if content == "" {
return ""
@@ -58,5 +115,12 @@ func Scrub(content string) string {
return r.Name
}
}
extraRulesMu.RLock()
defer extraRulesMu.RUnlock()
for _, r := range extraRules {
if r.RE.MatchString(content) {
return r.Name
}
}
return ""
}
@@ -25,6 +25,18 @@ func TestScrub_PoisonedFixtures(t *testing.T) {
{"aws-access-key", "AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE", "aws-access-key"},
{"homelab-env", "POSTGRES_PASSWORD=hunter2supersecretvalue", "homelab-env-token"},
{"sops-marker", "value: ENC[AES256_GCM,data:abc123def456,iv:zzz]", "sops-encrypted-marker"},
// Regression: a shell mangle glued the key to a preceding word
// ("yes"+"sk-..."), defeating the leading \b in the sk- rule and
// leaking a LiteLLM master key past the scrubber (2026-06-11).
{"sk-glued-to-word", "master key resolved: yessk-7181ca984603239d8c4819361bf33b94b9c3c07018791868", "openai-sk"},
{"sk-standalone-hex", "sk-7181ca984603239d8c4819361bf33b94b9c3c07018791868", "openai-sk"},
// Bare JWT not preceded by "Bearer" (e.g. a Dex token dumped to stdout).
{"jwt-bare", "token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dQw4w9WgXcQabcdef", "jwt"},
// 1Password service-account token (ops_<base64url>), env-assigned and bare.
// Both hit the dedicated op-service-account rule (ordered before the
// generic homelab-env-token). Guards ~/.zshrc reads etc. (2026-06-14).
{"op-sa-env", "export OP_SERVICE_ACCOUNT_TOKEN=ops_eyJzaWduSW5BZGRyZXNzIjoibXkuMXBhc3N3b3JkLmNvbSJ9", "op-service-account"},
{"op-sa-bare", "ops_eyJzaWduSW5BZGRyZXNzIjoibXkuMXBhc3N3b3JkLmNvbSIsInVzZXJBdXRoIjp7fX0aGVsbG8", "op-service-account"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
@@ -43,6 +55,9 @@ func TestScrub_CleanContentPassesThrough(t *testing.T) {
"file at ~/.ssh/id_ed25519",
"the function Authorization() takes no args",
"comment: see API key in 1Password",
// loosened sk- rule must not trip on short "task-"/"disk-" words
"run task-build then task-test in the pipeline",
"mounted /dev/disk-by-id/wwn-0x5000",
}
for _, c := range cases {
assert.Empty(t, Scrub(c), "expected clean for %q", c)
@@ -55,3 +70,63 @@ func TestScrub_FirstMatchWins(t *testing.T) {
content := "Authorization: Bearer ghp_aBcD1234EfGh5678IjKl9012MnOp3456QrSt"
assert.Equal(t, "authorization-header", Scrub(content))
}
func TestRegisterRule_ClientNameGuard(t *testing.T) {
t.Cleanup(ResetExtraRules)
require := func(err error) {
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
}
require(RegisterRule("client-name", `(?i)\b(SEB|Mastercard)\b`))
// Hits — case variations + word-boundary respect.
for _, hit := range []string{
"mentioned SEB in this commit",
"the Mastercard project deadline",
"working on mastercard scope",
"SEB internal review",
} {
assert.Equal(t, "client-name", Scrub(hit), "should match %q", hit)
}
// Misses — substring within a longer word should NOT match
// thanks to \b. "Sebastian" contains "seb" but \b prevents hit.
for _, miss := range []string{
"Sebastian wrote the docs",
"unrelated text",
"researcher",
"https://example.com/search?seb=1", // 'seb' bounded by ?=, still matches \b
} {
got := Scrub(miss)
if miss == "https://example.com/search?seb=1" {
// `seb=` has word-boundary at '='; this DOES match \bseb\b.
// Accept either outcome; document the tradeoff.
assert.Contains(t, []string{"", "client-name"}, got)
continue
}
assert.Empty(t, got, "should NOT match %q", miss)
}
}
func TestRegisterRule_CredentialsTakePrecedence(t *testing.T) {
t.Cleanup(ResetExtraRules)
require := func(err error) {
if err != nil {
t.Fatalf("unexpected err: %v", err)
}
}
require(RegisterRule("client-name", `\b(SEB)\b`))
// Content matches both a credential rule AND a client rule —
// credential rule wins by ordering, so log triage points at the
// strictly more dangerous leak.
content := "SEB project uses OPENAI_API_KEY=sk-proj-AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHHIIII"
assert.Equal(t, "openai-sk", Scrub(content))
}
func TestRegisterRule_RejectsInvalidPattern(t *testing.T) {
t.Cleanup(ResetExtraRules)
err := RegisterRule("bad", "[unclosed")
assert.Error(t, err)
}
+198
View File
@@ -0,0 +1,198 @@
// Package gitea implements capture.IssueTracker against a Gitea instance
// over its REST API. It is the new outbound dependency the brain server
// gains for the capture capability (#49c/#52): the server otherwise does
// brain-local file ops only.
//
// Owner is hard-coded to the operator and never taken from caller input.
// The API token is read once at construction, held in the struct, and
// never logged or placed in argv — it travels only in the Authorization
// header of outbound requests (AGENTS.md secret-handling).
package gitea
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
)
// owner is the fixed repository owner for every ticket operation. It is a
// constant, not a parameter, so a caller can never redirect a write to
// another owner's repo.
const owner = "mathias"
// Client is a Gitea REST API IssueTracker.
type Client struct {
baseURL string
token string
http *http.Client
}
// New constructs a Client. It returns nil when either baseURL or token is
// empty, so callers can treat missing config as "tracker disabled" with a
// single nil check (mirrors embed.New).
func New(baseURL, token string) *Client {
if baseURL == "" || token == "" {
return nil
}
return &Client{
baseURL: strings.TrimRight(baseURL, "/"),
token: token,
http: &http.Client{Timeout: 15 * time.Second},
}
}
// issueResponse is the subset of a Gitea issue/comment payload we read.
type issueResponse struct {
Number int `json:"number"`
HTMLURL string `json:"html_url"`
}
// CreateIssue opens a new issue under the fixed owner.
func (c *Client) CreateIssue(ctx context.Context, repo, title, body string) (capture.IssueRef, error) {
var out issueResponse
if err := c.do(ctx, http.MethodPost,
fmt.Sprintf("/api/v1/repos/%s/%s/issues", owner, repo),
map[string]any{"title": title, "body": body}, &out); err != nil {
return capture.IssueRef{}, err
}
return capture.IssueRef{Repo: repo, Number: out.Number, URL: out.HTMLURL}, nil
}
// CommentIssue posts a comment on an existing issue.
func (c *Client) CommentIssue(ctx context.Context, repo string, number int, body string) (capture.IssueRef, error) {
var out issueResponse
if err := c.do(ctx, http.MethodPost,
fmt.Sprintf("/api/v1/repos/%s/%s/issues/%d/comments", owner, repo, number),
map[string]any{"body": body}, &out); err != nil {
return capture.IssueRef{}, err
}
return capture.IssueRef{Repo: repo, Number: number, URL: out.HTMLURL}, nil
}
// CloseIssue closes an issue, first posting a closing comment when one is
// given (empty comment ⇒ close only).
func (c *Client) CloseIssue(ctx context.Context, repo string, number int, comment string) (capture.IssueRef, error) {
if strings.TrimSpace(comment) != "" {
if _, err := c.CommentIssue(ctx, repo, number, comment); err != nil {
return capture.IssueRef{}, err
}
}
var out issueResponse
if err := c.do(ctx, http.MethodPatch,
fmt.Sprintf("/api/v1/repos/%s/%s/issues/%d", owner, repo, number),
map[string]any{"state": "closed"}, &out); err != nil {
return capture.IssueRef{}, err
}
return capture.IssueRef{Repo: repo, Number: number, URL: out.HTMLURL}, nil
}
// WriteFile creates or updates a file in repo at path via the Gitea
// contents API — the SummaryWriter port (#66). It upserts: a GET resolves
// the current blob sha (if any) so an existing file is updated rather than
// rejected (the richer-fidelity-supersedes rule for re-captured sessions).
// Owner is the fixed const, like every other call.
func (c *Client) WriteFile(ctx context.Context, repo, path, content string) error {
cpath := fmt.Sprintf("/api/v1/repos/%s/%s/contents/%s", owner, repo, path)
sha, err := c.fileSHA(ctx, cpath)
if err != nil {
return err
}
payload := map[string]any{
"message": "capture: " + path,
"content": base64.StdEncoding.EncodeToString([]byte(content)),
}
// Gitea contents API: POST creates a new file, PUT updates an existing
// one (PUT requires the current sha). Pick by whether the file exists.
method := http.MethodPost
if sha != "" {
method = http.MethodPut
payload["sha"] = sha
}
status, body, err := c.request(ctx, method, cpath, payload)
if err != nil {
return err
}
if status < 200 || status >= 300 {
return fmt.Errorf("gitea %s %s: status %d: %s", method, cpath, status, strings.TrimSpace(string(body)))
}
return nil
}
// fileSHA returns the current blob sha for a contents path, or "" when the
// file does not exist (404). Any other non-2xx is an error.
func (c *Client) fileSHA(ctx context.Context, cpath string) (string, error) {
status, body, err := c.request(ctx, http.MethodGet, cpath, nil)
if err != nil {
return "", err
}
if status == http.StatusNotFound {
return "", nil
}
if status < 200 || status >= 300 {
return "", fmt.Errorf("gitea GET %s: status %d: %s", cpath, status, strings.TrimSpace(string(body)))
}
var meta struct {
SHA string `json:"sha"`
}
if err := json.Unmarshal(body, &meta); err != nil {
return "", fmt.Errorf("gitea GET %s: decode: %w", cpath, err)
}
return meta.SHA, nil
}
// do performs a JSON request against the Gitea API and decodes a 2xx
// response into out. Errors carry the status and a truncated body for
// diagnosis but never the token.
func (c *Client) do(ctx context.Context, method, path string, payload any, out *issueResponse) error {
status, body, err := c.request(ctx, method, path, payload)
if err != nil {
return err
}
if status < 200 || status >= 300 {
return fmt.Errorf("gitea %s %s: status %d: %s", method, path, status, strings.TrimSpace(string(body)))
}
if out != nil && len(body) > 0 {
if err := json.Unmarshal(body, out); err != nil {
return fmt.Errorf("gitea %s %s: decode response: %w", method, path, err)
}
}
return nil
}
// request is the shared HTTP path: marshals an optional JSON payload,
// attaches auth (token only ever in the header), and returns the status +
// body so callers can branch on status (e.g. 404) without it being an
// error. Never logs the token.
func (c *Client) request(ctx context.Context, method, path string, payload any) (int, []byte, error) {
var reader io.Reader
if payload != nil {
reqBody, err := json.Marshal(payload)
if err != nil {
return 0, nil, fmt.Errorf("marshal request: %w", err)
}
reader = bytes.NewReader(reqBody)
}
req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, reader)
if err != nil {
return 0, nil, err
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
req.Header.Set("Authorization", "token "+c.token)
resp, err := c.http.Do(req)
if err != nil {
return 0, nil, fmt.Errorf("gitea %s %s: %w", method, path, err)
}
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8192))
return resp.StatusCode, body, nil
}
+183
View File
@@ -0,0 +1,183 @@
package gitea_test
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/gitea"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const testToken = "super-secret-token-value"
func TestNewNilWhenUnconfigured(t *testing.T) {
assert.Nil(t, gitea.New("", testToken))
assert.Nil(t, gitea.New("https://git.example", ""))
}
func TestCreateIssueForcesOwnerAndAuth(t *testing.T) {
var gotPath, gotAuth, gotBody string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
gotAuth = r.Header.Get("Authorization")
b, _ := io.ReadAll(r.Body)
gotBody = string(b)
assert.Equal(t, http.MethodPost, r.Method)
w.WriteHeader(http.StatusCreated)
_ = json.NewEncoder(w).Encode(map[string]any{"number": 42, "html_url": "https://git.d-ma.be/mathias/hyperguild/issues/42"})
}))
defer srv.Close()
c := gitea.New(srv.URL, testToken)
require.NotNil(t, c)
ref, err := c.CreateIssue(context.Background(), "hyperguild", "Do the thing", "details")
require.NoError(t, err)
assert.Equal(t, "/api/v1/repos/mathias/hyperguild/issues", gotPath, "owner forced to mathias")
assert.Equal(t, "token "+testToken, gotAuth)
assert.Contains(t, gotBody, "Do the thing")
assert.Equal(t, "hyperguild", ref.Repo)
assert.Equal(t, 42, ref.Number)
assert.Contains(t, ref.URL, "/issues/42")
}
func TestCommentIssue(t *testing.T) {
var gotPath string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath = r.URL.Path
w.WriteHeader(http.StatusCreated)
_ = json.NewEncoder(w).Encode(map[string]any{"html_url": "https://git/c/1"})
}))
defer srv.Close()
ref, err := gitea.New(srv.URL, testToken).CommentIssue(context.Background(), "hyperguild", 7, "a comment")
require.NoError(t, err)
assert.Equal(t, "/api/v1/repos/mathias/hyperguild/issues/7/comments", gotPath)
assert.Equal(t, 7, ref.Number)
}
func TestCloseIssueWithComment(t *testing.T) {
var paths []string
var states []string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
paths = append(paths, r.Method+" "+r.URL.Path)
if r.Method == http.MethodPatch {
var body map[string]any
b, _ := io.ReadAll(r.Body)
_ = json.Unmarshal(b, &body)
states = append(states, body["state"].(string))
}
w.WriteHeader(http.StatusOK)
_ = json.NewEncoder(w).Encode(map[string]any{"number": 9, "html_url": "https://git/i/9"})
}))
defer srv.Close()
ref, err := gitea.New(srv.URL, testToken).CloseIssue(context.Background(), "hyperguild", 9, "closing because done")
require.NoError(t, err)
assert.Equal(t, 9, ref.Number)
// Comment posted first, then state PATCHed to closed.
assert.Contains(t, paths, "POST /api/v1/repos/mathias/hyperguild/issues/9/comments")
assert.Contains(t, paths, "PATCH /api/v1/repos/mathias/hyperguild/issues/9")
assert.Equal(t, []string{"closed"}, states)
}
func TestCloseIssueNoComment(t *testing.T) {
var commented bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.HasSuffix(r.URL.Path, "/comments") {
commented = true
}
w.WriteHeader(http.StatusOK)
_ = json.NewEncoder(w).Encode(map[string]any{"number": 3, "html_url": "https://git/i/3"})
}))
defer srv.Close()
_, err := gitea.New(srv.URL, testToken).CloseIssue(context.Background(), "hyperguild", 3, "")
require.NoError(t, err)
assert.False(t, commented, "empty comment ⇒ no comment POST")
}
func TestErrorPathDoesNotLeakToken(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write([]byte("boom"))
}))
defer srv.Close()
_, err := gitea.New(srv.URL, testToken).CreateIssue(context.Background(), "hyperguild", "t", "b")
require.Error(t, err)
assert.NotContains(t, err.Error(), testToken, "token must never appear in an error message")
assert.Contains(t, err.Error(), "500")
}
func TestWriteFileCreatesNewFile(t *testing.T) {
var getPath, postPath, postBody string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
getPath = r.URL.Path
w.WriteHeader(http.StatusNotFound) // file does not exist yet
case http.MethodPost: // gitea contents API: POST = create
postPath = r.URL.Path
b, _ := io.ReadAll(r.Body)
postBody = string(b)
w.WriteHeader(http.StatusCreated)
_ = json.NewEncoder(w).Encode(map[string]any{"content": map[string]any{"html_url": "https://git/x"}})
default:
t.Errorf("create must POST, got %s", r.Method)
}
}))
defer srv.Close()
err := gitea.New(srv.URL, testToken).WriteFile(context.Background(),
"ai-sessions", "summaries/claude-code/2026-06/2026-06-23-x-abcd1234.md", "# Summary\n\nbody\n")
require.NoError(t, err)
assert.Equal(t, "/api/v1/repos/mathias/ai-sessions/contents/summaries/claude-code/2026-06/2026-06-23-x-abcd1234.md", getPath)
assert.Equal(t, getPath, postPath)
// base64 of the content, no sha on create.
assert.Contains(t, postBody, "IyBTdW1tYXJ5") // base64("# Summary")
assert.NotContains(t, postBody, `"sha"`)
}
func TestWriteFileUpdatesExisting(t *testing.T) {
var putBody string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
w.WriteHeader(http.StatusOK)
_ = json.NewEncoder(w).Encode(map[string]any{"sha": "deadbeef"})
case http.MethodPut:
b, _ := io.ReadAll(r.Body)
putBody = string(b)
w.WriteHeader(http.StatusOK)
_ = json.NewEncoder(w).Encode(map[string]any{"content": map[string]any{"html_url": "https://git/x"}})
}
}))
defer srv.Close()
err := gitea.New(srv.URL, testToken).WriteFile(context.Background(), "ai-sessions", "p/x.md", "new")
require.NoError(t, err)
assert.Contains(t, putBody, `"sha":"deadbeef"`, "existing file → update with sha")
}
func TestWriteFileErrorNoTokenLeak(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.WriteHeader(http.StatusNotFound)
return
}
w.WriteHeader(http.StatusUnprocessableEntity)
_, _ = w.Write([]byte("bad"))
}))
defer srv.Close()
err := gitea.New(srv.URL, testToken).WriteFile(context.Background(), "ai-sessions", "p/x.md", "x")
require.Error(t, err)
assert.NotContains(t, err.Error(), testToken)
assert.Contains(t, err.Error(), "422")
}
+204
View File
@@ -0,0 +1,204 @@
package mcp_test
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/mathiasbq/hyperguild/ingestion/internal/vectorstore"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// callResult parses the JSON text payload of a successful tool call.
func callResult(t *testing.T, resp map[string]any) map[string]any {
t.Helper()
require.Nil(t, resp["error"], "tool returned error: %v", resp["error"])
text := resp["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
var out map[string]any
require.NoError(t, json.Unmarshal([]byte(text), &out))
return out
}
func TestBrainUpdateSupersedesExisting(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
// Seed via brain_write so the note carries real frontmatter.
callResult(t, toolCall(t, srv, "brain_write", map[string]any{
"content": "# Old\n\nold body\n", "filename": "val-vol",
"wing": "jepa-fx", "hall": "facts",
}))
out := callResult(t, toolCall(t, srv, "brain_update", map[string]any{
"wing": "jepa-fx", "hall": "facts", "slug": "val-vol",
"content": "# New\n\nnew body\n", "reason": "facts changed",
}))
assert.Equal(t, "wiki/jepa-fx/facts/val-vol.md", out["path"])
assert.Equal(t, out["path"], out["id"])
assert.NotEmpty(t, out["content_hash"])
assert.Equal(t, true, out["superseded"])
got, err := os.ReadFile(filepath.Join(brainDir, "wiki/jepa-fx/facts/val-vol.md"))
require.NoError(t, err)
s := string(got)
assert.Contains(t, s, "# New")
assert.NotContains(t, s, "old body")
assert.Contains(t, s, "wing: jepa-fx")
assert.Contains(t, s, "supersede_reason: facts changed")
assert.Contains(t, s, "supersedes:")
}
func TestBrainUpdateMissingTargetErrorsNoCreate(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
resp := toolCall(t, srv, "brain_update", map[string]any{
"wing": "jepa-fx", "hall": "facts", "slug": "ghost",
"content": "x\n",
})
require.NotNil(t, resp["error"])
assert.Contains(t, resp["error"].(map[string]any)["message"].(string), "does not exist")
_, statErr := os.Stat(filepath.Join(brainDir, "wiki/jepa-fx/facts/ghost.md"))
assert.True(t, os.IsNotExist(statErr))
}
func TestBrainUpdateByFullPath(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
callResult(t, toolCall(t, srv, "brain_write", map[string]any{
"content": "old\n", "filename": "n", "wing": "a", "hall": "facts",
}))
out := callResult(t, toolCall(t, srv, "brain_update", map[string]any{
"slug": "wiki/a/facts/n.md", "content": "fresh\n",
}))
assert.Equal(t, "wiki/a/facts/n.md", out["path"])
}
func TestBrainGetByIDAndPath(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
w := callResult(t, toolCall(t, srv, "brain_write", map[string]any{
"content": "# Body\n\ntext\n", "filename": "n", "wing": "a", "hall": "facts",
}))
id := w["id"].(string)
hash := w["content_hash"].(string)
require.NotEmpty(t, id)
require.NotEmpty(t, hash)
// by id
g1 := callResult(t, toolCall(t, srv, "brain_get", map[string]any{"id": id}))
assert.Equal(t, id, g1["path"])
assert.Equal(t, hash, g1["content_hash"], "content_hash must round-trip write→get")
assert.Contains(t, g1["body"].(string), "# Body")
fm := g1["frontmatter"].(map[string]any)
assert.Equal(t, "a", fm["wing"])
// by path
g2 := callResult(t, toolCall(t, srv, "brain_get", map[string]any{"path": id}))
assert.Equal(t, hash, g2["content_hash"])
}
func TestBrainGetMissingArgsErrors(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil, nil)
resp := toolCall(t, srv, "brain_get", map[string]any{})
require.NotNil(t, resp["error"])
}
func TestBrainWriteReturnsHandle(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
out := callResult(t, toolCall(t, srv, "brain_write", map[string]any{
"content": "# X\n\nbody\n", "filename": "x", "wing": "a", "hall": "facts",
}))
assert.Equal(t, "wiki/a/facts/x.md", out["path"])
assert.Equal(t, out["path"], out["id"])
assert.NotEmpty(t, out["content_hash"])
}
// --- retrieval-reflects-new-content: exercises the real mtime-driven Sync ---
type fakeVecStore struct {
chunks map[string][]float32
deleted []string
}
func (f *fakeVecStore) KnownPathsWithTime(_ context.Context) (map[string]time.Time, error) {
m := make(map[string]time.Time, len(f.chunks))
for p := range f.chunks {
m[p] = time.Unix(0, 0) // always stale → mtime(now) is always newer
}
return m, nil
}
func (f *fakeVecStore) Upsert(_ context.Context, path string, vec []float32) error {
f.chunks[path] = vec
return nil
}
func (f *fakeVecStore) Delete(_ context.Context, path string) error {
delete(f.chunks, path)
f.deleted = append(f.deleted, path)
return nil
}
type fakeEmbedder struct{ seen []string }
func (e *fakeEmbedder) Embed(_ context.Context, text string) ([]float32, error) {
e.seen = append(e.seen, text)
return []float32{1, 0, 0}, nil
}
// TestBrainUpdateReembedsNewContent proves the supersede contract end to
// end against the actual embedding mechanism: brain_update rewrites the
// file, advancing its mtime, and the next vectorstore.Sync pass re-embeds
// the NEW body and drops the stale chunk. No stub of the re-index path.
func TestBrainUpdateReembedsNewContent(t *testing.T) {
brainDir := t.TempDir()
srv := mcp.NewServer(brainDir, nil, nil, nil)
ctx := context.Background()
callResult(t, toolCall(t, srv, "brain_write", map[string]any{
"content": "# Note\n\nthe OLD distinctive payload\n",
"filename": "n", "wing": "a", "hall": "facts",
}))
store := &fakeVecStore{chunks: map[string][]float32{}}
emb := &fakeEmbedder{}
// First sync embeds the original content.
_, err := vectorstore.Sync(ctx, brainDir, store, emb)
require.NoError(t, err)
require.NotEmpty(t, store.chunks)
require.True(t, anyContains(emb.seen, "OLD distinctive payload"))
callResult(t, toolCall(t, srv, "brain_update", map[string]any{
"wing": "a", "hall": "facts", "slug": "n",
"content": "# Note\n\nthe NEW distinctive payload\n",
}))
emb.seen = nil // only watch what the second pass embeds
_, err = vectorstore.Sync(ctx, brainDir, store, emb)
require.NoError(t, err)
assert.True(t, anyContains(emb.seen, "NEW distinctive payload"),
"Sync must re-embed the superseded body; saw %v", emb.seen)
assert.False(t, anyContains(emb.seen, "OLD distinctive payload"),
"the old body must not be re-embedded")
assert.NotEmpty(t, store.deleted, "stale chunks must be deleted before re-embed")
}
func anyContains(ss []string, sub string) bool {
for _, s := range ss {
if strings.Contains(s, sub) {
return true
}
}
return false
}
+159 -13
View File
@@ -11,6 +11,7 @@ import (
"github.com/mathiasbq/hyperguild/ingestion/internal/api"
"github.com/mathiasbq/hyperguild/ingestion/internal/brain"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/extract"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphsync"
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
@@ -38,7 +39,7 @@ func (s *Server) tools() []map[string]any {
return b
}
return []map[string]any{
tools := []map[string]any{
{
"name": "brain_query",
"description": "BM25 full-text search across brain/knowledge/ and brain/wiki/ markdown files. Optionally scope by wing (topic domain) and hall (memory type).",
@@ -61,6 +62,41 @@ func (s *Server) tools() []map[string]any {
"hall": enum("optional memory type (requires wing)", halls...),
}),
},
{
"name": "brain_update",
"description": "Supersede an existing brain note in place: whole-note body replace + frontmatter re-stamp (updated_at, supersedes=prior content hash, supersede_reason). Errors if the target does not exist — use brain_write to create. Returns {id, path, content_hash, superseded}. Prior version recoverable from git.",
"inputSchema": schema([]string{"content"}, map[string]any{
"content": str("new full body (whole-note replace)"),
"slug": str("target note slug within wing/hall, OR a full brain-relative path (e.g. wiki/jepa-fx/facts/x.md)"),
"wing": str("wing of the target (required unless slug/path is a full path)"),
"hall": enum("hall of the target (required unless slug/path is a full path)", halls...),
"path": str("full brain-relative path to the target; takes precedence over slug/wing/hall"),
"reason": str("optional short note on why superseded — stamped into frontmatter"),
}),
},
{
"name": "brain_get",
"description": "Fetch a single brain note by id or path (both are the brain-relative path — the note handle). Returns {id, path, content_hash, frontmatter, body}. Read-after-write confirmation without a lexical re-query.",
"inputSchema": schema([]string{}, map[string]any{
"id": str("note id (brain-relative path) as returned by brain_write/brain_update"),
"path": str("brain-relative path to the note; equivalent to id"),
}),
},
{
"name": "brain_pending",
"description": "List notes in brain/raw/ awaiting human promotion to the wiki, oldest-first. Returns filename, created_at, size_bytes, excerpt. The human-review queue complement to brain_promote.",
"inputSchema": schema([]string{}, map[string]any{}),
},
{
"name": "brain_promote",
"description": "Promote a brain/raw/ note into brain/wiki/<wing>/<hall>/: rewrites frontmatter (sets wing/hall/promoted_at, preserves created_at + custom fields), deletes the source, rebuilds the wing index, runs auto-tunnel. Errors (without touching the fs) on invalid hall or a slug collision. Returns {path}.",
"inputSchema": schema([]string{"filename", "wing", "hall"}, map[string]any{
"filename": str("basename in brain/raw/, e.g. 2026-06-01-lejpa-decision.md"),
"wing": str("target wing, e.g. jepa-fx"),
"hall": enum("target hall", halls...),
"slug": str("optional target slug; defaults to filename minus date prefix"),
}),
},
{
"name": "brain_tunnel",
"description": "Create an explicit bidirectional [[wikilink]] between two notes in different wings. Idempotent.",
@@ -151,6 +187,13 @@ func (s *Server) tools() []map[string]any {
}),
},
}
// The capture relay tool (#55) is advertised only when wired via
// WithCapture — MCP-native harnesses (claude.ai, Crush, Pi, LLM Council)
// reach capture through it.
if s.capture != nil {
tools = append(tools, captureToolDescriptor())
}
return tools
}
type brainQueryArgs struct {
@@ -199,7 +242,11 @@ func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.Raw
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
relPath, err := api.WriteNote(s.brainDir, api.WriteNoteOptions{
// Delegate to the shared BrainStore so write+index+tunnel+graph live in
// one implementation (capture uses the same store). The read-after-write
// handle {id, path, content_hash} comes back from the store; path is kept
// for backward compatibility.
ref, err := s.store.Write(ctx, capture.Note{
Content: a.Content,
Filename: a.Filename,
Type: a.Type,
@@ -210,18 +257,117 @@ func (s *Server) brainWrite(ctx context.Context, args json.RawMessage) (json.Raw
if err != nil {
return nil, err
}
// Auto-regenerate the wing _index.md when the write landed in the
// structured wiki, and auto-tunnel cross-wing matches. Both are
// best-effort: the note is already written.
if a.Wing != "" && a.Hall != "" {
if err := brain.BuildWingIndex(s.brainDir, a.Wing); err != nil {
slog.Warn("brain_write: auto-index failed", "wing", a.Wing, "err", err)
}
if err := brain.AutoTunnel(s.brainDir, relPath, a.Content); err != nil {
slog.Warn("brain_write: auto-tunnel failed", "src", relPath, "err", err)
}
return json.Marshal(map[string]string{"id": ref.ID, "path": ref.Path, "content_hash": ref.ContentHash})
}
type brainUpdateArgs struct {
Slug string `json:"slug,omitempty"`
Wing string `json:"wing,omitempty"`
Hall string `json:"hall,omitempty"`
Path string `json:"path,omitempty"`
Content string `json:"content"`
Reason string `json:"reason,omitempty"`
}
// brainUpdate supersedes an existing note in place: whole-note body
// replace, frontmatter re-stamp (updated_at/supersedes/supersede_reason),
// graph re-index, and wing _index rebuild. It never creates — a missing
// target is an error so the caller can fall back to brain_write.
//
// Embedding re-sync is delegated to the out-of-band vectorstore.Sync
// ticker: the rewritten file's mtime advances, so the next pass re-embeds
// it. This mirrors brain_write, which likewise does not embed in-handler.
func (s *Server) brainUpdate(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainUpdateArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
s.indexInGraph(ctx, "brain_write", relPath)
if a.Content == "" {
return nil, fmt.Errorf("content is required")
}
// path takes precedence over slug; the store treats any slug containing
// a slash as a full brain-relative path (issue #45: "slug ... OR path").
slug := a.Slug
if a.Path != "" {
slug = a.Path
}
ref, err := s.store.Update(ctx, slug, capture.Note{
Content: a.Content,
Wing: a.Wing,
Hall: a.Hall,
Reason: a.Reason,
})
if err != nil {
return nil, err
}
return json.Marshal(map[string]any{
"id": ref.ID, "path": ref.Path, "content_hash": ref.ContentHash, "superseded": ref.Superseded,
})
}
type brainGetArgs struct {
ID string `json:"id,omitempty"`
Path string `json:"path,omitempty"`
}
// brainGet fetches a note by id or path (both are the brainDir-relative
// path — the de-facto handle). Read-only; the create-path read-after-
// write primitive that lets callers confirm a write landed without a
// lexical re-query.
func (s *Server) brainGet(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainGetArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
target := a.Path
if target == "" {
target = a.ID
}
if target == "" {
return nil, fmt.Errorf("id or path is required")
}
note, err := s.store.Get(ctx, target)
if err != nil {
return nil, err
}
return json.Marshal(map[string]any{
"id": note.ID, "path": note.Path, "content_hash": note.ContentHash,
"frontmatter": note.Frontmatter, "body": note.Body,
})
}
// brainPending lists the raw/ review queue (oldest-first).
func (s *Server) brainPending(_ context.Context, _ json.RawMessage) (json.RawMessage, error) {
pending, err := api.ListPending(s.brainDir)
if err != nil {
return nil, err
}
return json.Marshal(map[string]any{"pending": pending})
}
type brainPromoteArgs struct {
Filename string `json:"filename"`
Wing string `json:"wing"`
Hall string `json:"hall"`
Slug string `json:"slug,omitempty"`
}
// brainPromote moves a raw/ note into the structured wiki (frontmatter
// rewrite + index + auto-tunnel, all owned by api.PromoteNote) and then
// re-indexes it into the graph. The human-facing complement to brain_write.
func (s *Server) brainPromote(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
var a brainPromoteArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
relPath, err := api.PromoteNote(s.brainDir, api.PromoteOptions{
Filename: a.Filename, Wing: a.Wing, Hall: a.Hall, Slug: a.Slug,
})
if err != nil {
return nil, err
}
s.indexInGraph(ctx, "brain_promote", relPath)
return json.Marshal(map[string]string{"path": relPath})
}
+58
View File
@@ -332,3 +332,61 @@ func TestSessionLogRequiresSessionID(t *testing.T) {
resp := toolCall(t, srv, "session_log", map[string]any{"skill": "tdd"})
require.NotNil(t, resp["error"])
}
func TestBrainPendingListsRaw(t *testing.T) {
brainDir := t.TempDir()
raw := filepath.Join(brainDir, "raw")
require.NoError(t, os.MkdirAll(raw, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(raw, "2026-06-01-x.md"),
[]byte("---\ncreated_at: 2026-06-01T00:00:00Z\n---\npending body\n"), 0o644))
srv := mcp.NewServer(brainDir, nil, nil, nil)
resp := toolCall(t, srv, "brain_pending", map[string]any{})
require.Nil(t, resp["error"])
text := resp["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
assert.Contains(t, text, "2026-06-01-x.md")
assert.Contains(t, text, "pending body")
}
func TestBrainPendingEmpty(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil, nil)
resp := toolCall(t, srv, "brain_pending", map[string]any{})
require.Nil(t, resp["error"])
text := resp["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
assert.Contains(t, text, `"pending":[]`)
}
func TestBrainPromoteMovesToWiki(t *testing.T) {
brainDir := t.TempDir()
raw := filepath.Join(brainDir, "raw")
require.NoError(t, os.MkdirAll(raw, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(raw, "2026-06-01-decision.md"),
[]byte("---\ncreated_at: 2026-06-01T00:00:00Z\n---\n# D\n\nbody\n"), 0o644))
srv := mcp.NewServer(brainDir, nil, nil, nil)
resp := toolCall(t, srv, "brain_promote", map[string]any{
"filename": "2026-06-01-decision.md", "wing": "jepa-fx", "hall": "decisions",
})
require.Nil(t, resp["error"], "got: %v", resp["error"])
text := resp["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
assert.Contains(t, text, "wiki/jepa-fx/decisions/decision.md")
_, err := os.Stat(filepath.Join(brainDir, "wiki/jepa-fx/decisions/decision.md"))
require.NoError(t, err)
_, srcErr := os.Stat(filepath.Join(raw, "2026-06-01-decision.md"))
assert.True(t, os.IsNotExist(srcErr), "source removed")
}
func TestBrainPromoteInvalidHallErrors(t *testing.T) {
brainDir := t.TempDir()
raw := filepath.Join(brainDir, "raw")
require.NoError(t, os.MkdirAll(raw, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(raw, "x.md"), []byte("body\n"), 0o644))
srv := mcp.NewServer(brainDir, nil, nil, nil)
resp := toolCall(t, srv, "brain_promote", map[string]any{
"filename": "x.md", "wing": "a", "hall": "garbage",
})
require.NotNil(t, resp["error"])
_, srcErr := os.Stat(filepath.Join(raw, "x.md"))
assert.NoError(t, srcErr, "source untouched on validation error")
}
+98 -5
View File
@@ -1,7 +1,9 @@
// Package mcp implements an MCP HTTP handler for the ingestion service.
// Exposed tools: brain_query, brain_write, brain_index, brain_tunnel,
// brain_ingest, brain_ingest_raw, brain_answer, brain_classify,
// brain_graph, brain_context, session_log.
// Exposed tools: brain_query, brain_write, brain_update, brain_get,
// brain_pending, brain_promote, brain_index, brain_tunnel, brain_ingest,
// brain_ingest_raw, brain_answer, brain_classify, brain_graph,
// brain_context, session_log, and capture (the #55 relay tool, registered
// only when WithCapture is set).
package mcp
import (
@@ -10,6 +12,9 @@ import (
"fmt"
"net/http"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/graphsync"
"github.com/mathiasbq/hyperguild/ingestion/internal/pipeline"
@@ -46,6 +51,21 @@ type Server struct {
vector search.VectorSearcher // nil = BM25-only retrieval
embedder search.Embedder // nil = BM25-only retrieval
graph graphsync.Store // nil = brain_graph and GraphRAG augmentation disabled
store *brainstore.Store // shared brain write/update/get impl (also used by capture)
tracker capture.IssueTracker // nil = no Gitea ticket integration; wired for capture (#53)
capture *captureDeps // nil = capture MCP tool disabled (#55 relay)
}
// captureDeps holds what the MCP `capture` tool (the #55 relay door for
// MCP-native harnesses like claude.ai) needs: the use-case, the auth bits
// to re-derive the caller's principal from the Bearer header (the chassis
// middleware gates but discards the principal), and the origin resolver.
type captureDeps struct {
svc *capture.Service
validator capturehttp.Validator
staticToken string
staticPrincipal string
resolver capturehttp.OriginResolver
}
// NewServer constructs a Server bound to brainDir. pipelineCfg supplies the
@@ -56,7 +76,13 @@ func NewServer(brainDir string, pipelineCfg *pipeline.Config, llm pipeline.Compl
if pipelineCfg != nil {
cfg = *pipelineCfg
}
return &Server{brainDir: brainDir, pipeline: cfg, llm: llm, answerLLM: answerLLM}
return &Server{
brainDir: brainDir,
pipeline: cfg,
llm: llm,
answerLLM: answerLLM,
store: brainstore.New(brainDir),
}
}
// WithReranker installs an opt-in cross-encoder reranker. When set,
@@ -84,9 +110,55 @@ func (s *Server) WithHybridRetrieval(v search.VectorSearcher, e search.Embedder)
func (s *Server) WithGraph(g *graphstore.PGStore) *Server {
if g == nil {
s.graph = nil
s.store.WithGraph(nil)
return s
}
s.graph = g
s.store.WithGraph(g)
return s
}
// WithIssueTracker injects the Gitea ticket tracker behind the
// capture.IssueTracker interface. nil leaves ticket integration off. The
// use-case (capture) consumes this in #53; it is wired here so the
// dependency is constructed once and stays swappable/testable.
func (s *Server) WithIssueTracker(t capture.IssueTracker) *Server {
s.tracker = t
return s
}
// IssueTracker returns the injected ticket tracker (nil when unconfigured).
func (s *Server) IssueTracker() capture.IssueTracker {
return s.tracker
}
// BrainStore returns the shared brain store (graph-wired once WithGraph
// has run), so the capture use-case writes through the exact same
// implementation as the MCP handlers.
func (s *Server) BrainStore() *brainstore.Store {
return s.store
}
// WithCapture enables the MCP `capture` tool (#55) — the relay door for
// MCP-native harnesses (claude.ai, Crush, Pi, LLM Council) that cannot run
// the use-case in-process. It forwards to the same CaptureService as
// POST /capture, deriving the caller's principal + origin from the same
// auth credentials that gate /mcp. nil svc leaves the tool unregistered.
func (s *Server) WithCapture(svc *capture.Service, validator capturehttp.Validator, staticToken, staticPrincipal string, resolver capturehttp.OriginResolver) *Server {
if svc == nil {
s.capture = nil
return s
}
if staticPrincipal == "" {
staticPrincipal = "local-cli"
}
s.capture = &captureDeps{
svc: svc,
validator: validator,
staticToken: staticToken,
staticPrincipal: staticPrincipal,
resolver: resolver,
}
return s
}
@@ -139,7 +211,18 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
rpcErr = &rpcError{Code: -32602, Message: "invalid params"}
break
}
out, err := s.handleCall(r.Context(), p.Name, p.Arguments)
// Re-derive the authenticated principal from the Bearer header so
// the capture tool can compute the trust-zone origin. The request
// is already gated by BearerMiddleware; this only recovers the
// identity that middleware discards.
ctx := r.Context()
if s.capture != nil {
if principal, viaStatic, ok := capturehttp.Authenticate(
r, s.capture.staticToken, s.capture.staticPrincipal, s.capture.validator); ok {
ctx = withPrincipal(ctx, principal, viaStatic)
}
}
out, err := s.handleCall(ctx, p.Name, p.Arguments)
if err != nil {
rpcErr = &rpcError{Code: -32000, Message: err.Error()}
break
@@ -177,6 +260,16 @@ func (s *Server) handleCall(ctx context.Context, name string, args json.RawMessa
return s.brainQuery(ctx, args)
case "brain_write":
return s.brainWrite(ctx, args)
case "brain_update":
return s.brainUpdate(ctx, args)
case "brain_get":
return s.brainGet(ctx, args)
case "brain_pending":
return s.brainPending(ctx, args)
case "brain_promote":
return s.brainPromote(ctx, args)
case "capture":
return s.brainCapture(ctx, args)
case "brain_index":
return s.brainIndex(ctx, args)
case "brain_tunnel":
+24 -1
View File
@@ -2,12 +2,14 @@ package mcp_test
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -55,7 +57,9 @@ func TestServerToolsList(t *testing.T) {
names = append(names, t.(map[string]any)["name"].(string))
}
assert.ElementsMatch(t, []string{
"brain_query", "brain_write", "brain_index", "brain_tunnel",
"brain_query", "brain_write", "brain_update", "brain_get",
"brain_pending", "brain_promote",
"brain_index", "brain_tunnel",
"brain_ingest_raw", "brain_ingest",
"brain_answer", "brain_classify", "brain_graph", "brain_context",
"session_log",
@@ -92,3 +96,22 @@ func TestServerUnknownMethodReturnsError(t *testing.T) {
assert.Equal(t, float64(-32601), errObj["code"])
assert.Contains(t, errObj["message"].(string), "unknown/method")
}
type stubTracker struct{}
func (stubTracker) CreateIssue(context.Context, string, string, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func (stubTracker) CloseIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func (stubTracker) CommentIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func TestWithIssueTrackerInjects(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil, nil)
assert.Nil(t, srv.IssueTracker(), "tracker is off by default")
srv = srv.WithIssueTracker(stubTracker{})
assert.NotNil(t, srv.IssueTracker(), "tracker injected behind the interface")
}
+16 -3
View File
@@ -77,9 +77,22 @@ func (s *Server) brainAnswer(ctx context.Context, args json.RawMessage) (json.Ra
return nil, fmt.Errorf("search: %w", err)
}
if s.reranker != nil && len(results) > 0 {
results, err = rerankResults(ctx, s.reranker, a.Query, results, 5)
if err != nil {
return nil, fmt.Errorf("rerank: %w", err)
reranked, rerr := rerankResults(ctx, s.reranker, a.Query, results, 5)
if rerr != nil {
return nil, fmt.Errorf("rerank: %w", rerr)
}
// The reranker is a filter, not a gate. The Qwen3-Reranker is a
// web-search cross-encoder: against a conversational / personal-
// intent query ("what am I optimizing toward?") it scores even
// on-topic notes as "no", which would collapse the whole answer to
// "no relevant content" despite BM25 having retrieved relevant
// content. When the reranker keeps nothing, fall back to the
// BM25/vector ordering (capped to the no-reranker depth) rather
// than returning an empty answer.
if len(reranked) > 0 {
results = reranked
} else if len(results) > 10 {
results = results[:10]
}
}
if len(results) == 0 {
@@ -98,6 +98,42 @@ func TestBrainAnswer_RerankerFiltersBeforeLLM(t *testing.T) {
assert.NotContains(t, sawSources, "noise.md")
}
func TestBrainAnswer_RerankerKeepsNone_FallsBackToBM25(t *testing.T) {
brainDir := brainDirWithContent(t) // test.md BM25-matches "pass-rate logging"
// Reranker rejects every candidate ("no" to all) — models a
// web-search cross-encoder facing a conversational / personal-intent
// query, which is exactly when it wrongly scores on-topic notes as
// irrelevant. The answer must still synthesize from the BM25 hits, not
// collapse to "no relevant content".
rrSrv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"response": "no", "done": true})
}))
defer rrSrv.Close()
var sawSources string
llm := func(_ context.Context, _, user string) (string, error) {
sawSources = user
return "fallback answer", nil
}
srv := mcp.NewServer(brainDir, nil, nil, llm).
WithReranker(reranker.New(rrSrv.URL, "qwen3"))
ts := httptest.NewServer(srv)
defer ts.Close()
rpc := callTool(t, ts, "brain_answer", map[string]any{"query": "pass-rate logging"})
require.Nil(t, rpc["error"])
content := rpc["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
var result map[string]any
require.NoError(t, json.Unmarshal([]byte(content), &result))
assert.Equal(t, "fallback answer", result["answer"])
assert.NotEmpty(t, result["sources"], "reranker keeping nothing must fall back to BM25, not empty")
assert.Contains(t, sawSources, "test.md")
}
func TestBrainAnswer_NoLLM(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil, nil)
ts := httptest.NewServer(srv)
+105
View File
@@ -0,0 +1,105 @@
package mcp
import (
"context"
"encoding/json"
"fmt"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
)
// principalKey is the context key under which the authenticated principal
// (re-derived in ServeHTTP) is stashed for the capture tool.
type principalKeyT struct{}
var principalKey principalKeyT
type principalInfo struct {
principal string
viaStatic bool
}
func withPrincipal(ctx context.Context, principal string, viaStatic bool) context.Context {
return context.WithValue(ctx, principalKey, principalInfo{principal: principal, viaStatic: viaStatic})
}
// captureToolDescriptor is the tools/list entry for the capture relay.
// Appended only when WithCapture has wired the tool.
func captureToolDescriptor() map[string]any {
str := func(d string) map[string]any { return map[string]any{"type": "string", "description": d} }
insightItem := map[string]any{
"type": "object",
"properties": map[string]any{
"text": str("the insight body"), "wing": str("brain wing"),
"hall": str("brain hall (facts/decisions/failures/hypotheses/sources)"),
"supersede_slug": str("optional: slug of a prior note to revise in place instead of creating"),
},
"required": []string{"text", "wing", "hall"},
}
ticketItem := map[string]any{
"type": "object",
"properties": map[string]any{
"repo": str("gitea repo (owner is always mathias)"), "action": str("create|close|comment"),
"number": map[string]any{"type": "integer", "description": "issue number (close/comment)"},
"title": str("issue title (create)"), "body": str("issue/comment body"),
},
"required": []string{"repo", "action"},
}
schema := map[string]any{
"type": "object",
"properties": map[string]any{
"context": map[string]any{
"type": "object",
"properties": map[string]any{
"harness": str("descriptive harness label (telemetry only, never a gate input)"),
"session_ref": str("optional session reference"), "fidelity": str("live-capture|transcript-parse|agent-runlog"),
"actor": str("acting user/agent"), "classification": str("caller-declared sensitivity: public|internal|confidential"),
},
},
"insights": map[string]any{"type": "array", "items": insightItem},
"tickets": map[string]any{"type": "array", "items": ticketItem},
"summary": map[string]any{"type": "object", "properties": map[string]any{
"title": str("summary title"), "body": str("summary body"),
"repos_touched": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
}},
"dry_run": map[string]any{"type": "boolean", "description": "validate + return the would-be receipt, write nothing"},
},
}
b, _ := json.Marshal(schema)
return map[string]any{
"name": "capture",
"description": "Persist a session's value uniformly: insights → brain (write or supersede), action items → Gitea tickets, optional summary → ai-sessions. The relay door for MCP-native harnesses. Origin is server-derived from your authenticated identity; confidential captures through a us-nexus surface are refused (I1). Returns a partial-aware receipt.",
"inputSchema": json.RawMessage(b),
}
}
// brainCapture is the MCP capture tool: the #55 relay for MCP-native
// harnesses. It re-uses the same CaptureService, principal-derivation, and
// origin resolver as POST /capture — only the transport differs. It holds
// no state and retains nothing beyond the I5 audit record.
func (s *Server) brainCapture(ctx context.Context, args json.RawMessage) (json.RawMessage, error) {
if s.capture == nil {
return nil, fmt.Errorf("capture tool not configured")
}
info, ok := ctx.Value(principalKey).(principalInfo)
if !ok || info.principal == "" {
// No authenticated principal ⇒ cannot derive origin ⇒ cannot gate.
return nil, fmt.Errorf("capture requires an authenticated principal")
}
in, err := capturehttp.DecodeRequest(args)
if err != nil {
return nil, fmt.Errorf("invalid capture request: %w", err)
}
// Principal and origin are server-derived — never taken from the body.
in.Context.Principal = info.principal
in.Context.Origin = s.capture.resolver.Resolve(info.principal, info.viaStatic)
rec, err := s.capture.svc.Capture(ctx, in)
if err != nil {
// Surface I1/I5 refusals and validation failures verbatim; errors.Is
// markers (ErrSovereigntyRefused / ErrAuditUnavailable) ride in the message.
return nil, err
}
return json.Marshal(rec)
}
@@ -0,0 +1,150 @@
package mcp_test
import (
"bytes"
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/mathiasbq/hyperguild/ingestion/internal/audit"
"github.com/mathiasbq/hyperguild/ingestion/internal/brainstore"
"github.com/mathiasbq/hyperguild/ingestion/internal/capture"
"github.com/mathiasbq/hyperguild/ingestion/internal/capturehttp"
"github.com/mathiasbq/hyperguild/ingestion/internal/classification"
"github.com/mathiasbq/hyperguild/ingestion/internal/mcp"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const capStaticTok = "cap-static-tok"
type capFakeTracker struct{}
func (capFakeTracker) CreateIssue(context.Context, string, string, string) (capture.IssueRef, error) {
return capture.IssueRef{Repo: "hyperguild", Number: 1, URL: "https://git/1"}, nil
}
func (capFakeTracker) CloseIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
func (capFakeTracker) CommentIssue(context.Context, string, int, string) (capture.IssueRef, error) {
return capture.IssueRef{}, nil
}
type capFakeValidator struct {
subject string
err error
}
func (v capFakeValidator) Validate(context.Context, string) (string, error) {
return v.subject, v.err
}
func captureServer(t *testing.T, validator capturehttp.Validator, sovereign []string) (*mcp.Server, string) {
t.Helper()
brainDir := t.TempDir()
cfg, err := classification.Load(brainDir)
require.NoError(t, err)
svc := capture.NewService(brainstore.New(brainDir), capFakeTracker{}, nil, cfg, audit.NewSlogSink(nil))
srv := mcp.NewServer(brainDir, nil, nil, nil)
srv.WithCapture(svc, validator, capStaticTok, "local-cli", capturehttp.NewOriginResolver(sovereign))
return srv, brainDir
}
func captureCall(t *testing.T, srv http.Handler, authz string, args map[string]any) map[string]any {
t.Helper()
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0", "id": 1, "method": "tools/call",
"params": map[string]any{"name": "capture", "arguments": args},
})
req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(body))
if authz != "" {
req.Header.Set("Authorization", authz)
}
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
var resp map[string]any
require.NoError(t, json.Unmarshal(rr.Body.Bytes(), &resp))
return resp
}
func TestCaptureToolListedWhenWired(t *testing.T) {
srv, _ := captureServer(t, nil, nil)
body, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(body))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.Contains(t, rr.Body.String(), `"capture"`)
}
func TestCaptureToolNotListedByDefault(t *testing.T) {
srv := mcp.NewServer(t.TempDir(), nil, nil, nil) // no WithCapture
body, _ := json.Marshal(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
req := httptest.NewRequest(http.MethodPost, "/mcp", bytes.NewReader(body))
rr := httptest.NewRecorder()
srv.ServeHTTP(rr, req)
assert.NotContains(t, rr.Body.String(), `"capture"`)
}
func TestCaptureToolForwardsViaStaticPrincipal(t *testing.T) {
srv, brainDir := captureServer(t, nil, nil)
resp := captureCall(t, srv, "Bearer "+capStaticTok, map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "internal"},
"insights": []map[string]any{{"text": "a fact", "wing": "hyperguild", "hall": "facts"}},
"tickets": []map[string]any{{"repo": "hyperguild", "action": "create", "title": "t"}},
})
require.Nil(t, resp["error"], "got error: %v", resp["error"])
text := resp["result"].(map[string]any)["content"].([]any)[0].(map[string]any)["text"].(string)
var rec capture.CaptureReceipt
require.NoError(t, json.Unmarshal([]byte(text), &rec))
assert.True(t, rec.Insights[0].OK)
assert.True(t, rec.Tickets[0].OK)
// Forwarded to the real brain store.
_, statErr := os.Stat(filepath.Join(brainDir, "wiki/hyperguild/facts"))
require.NoError(t, statErr)
}
func TestCaptureToolRefusesConfidentialViaUSNexus(t *testing.T) {
// JWT principal not in the sovereign allowlist ⇒ us-nexus origin.
srv, _ := captureServer(t, capFakeValidator{subject: "claudeai-oauth"}, nil)
resp := captureCall(t, srv, "Bearer jwt-token", map[string]any{
"context": map[string]any{"harness": "claudeai-chat", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
require.NotNil(t, resp["error"])
assert.Contains(t, resp["error"].(map[string]any)["message"].(string), "sovereignty")
}
func TestCaptureToolAllowsConfidentialViaSovereignJWT(t *testing.T) {
srv, _ := captureServer(t, capFakeValidator{subject: "koala-cli"}, []string{"koala-cli"})
resp := captureCall(t, srv, "Bearer jwt-token", map[string]any{
"context": map[string]any{"harness": "claude-code", "actor": "mathias", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
assert.Nil(t, resp["error"], "sovereign JWT principal should be allowed: %v", resp["error"])
}
func TestCaptureToolRejectsUnauthenticated(t *testing.T) {
srv, _ := captureServer(t, capFakeValidator{err: errors.New("no jwt")}, nil)
resp := captureCall(t, srv, "", map[string]any{ // no Authorization
"context": map[string]any{"harness": "x", "classification": "internal"},
"insights": []map[string]any{{"text": "a", "wing": "hyperguild", "hall": "facts"}},
})
require.NotNil(t, resp["error"])
assert.Contains(t, resp["error"].(map[string]any)["message"].(string), "authenticated principal")
}
func TestCaptureToolCallerCannotForgeOrigin(t *testing.T) {
// Body asserts sovereign harness, but the us-nexus JWT principal governs.
srv, _ := captureServer(t, capFakeValidator{subject: "claudeai-oauth"}, nil)
resp := captureCall(t, srv, "Bearer jwt", map[string]any{
"context": map[string]any{"harness": "sovereign-soil", "classification": "confidential"},
"insights": []map[string]any{{"text": "secret", "wing": "client-seb", "hall": "facts"}},
})
require.NotNil(t, resp["error"])
assert.Contains(t, resp["error"].(map[string]any)["message"].(string), "sovereignty")
}
+65
View File
@@ -3,6 +3,7 @@ package vectorstore
import (
"fmt"
"strings"
"unicode/utf8"
)
// NumberedChunk pairs a chunk's body with the storage path it will use
@@ -66,6 +67,70 @@ func ChunkMarkdown(content string, maxBytes int) []string {
}
out = append(out, splitAtParagraphs(s, maxBytes)...)
}
// Final guarantee: no chunk exceeds maxBytes. A single heading-less,
// paragraph-less block (JSON-lines, minified content) survives the two
// passes above whole — splitAtParagraphs emits an over-budget paragraph
// rather than truncating prose. Hard-split any such chunk at line/rune
// boundaries so the embedder never rejects an over-context chunk.
final := make([]string, 0, len(out))
for _, c := range out {
if len(c) <= maxBytes {
final = append(final, c)
continue
}
final = append(final, hardSplit(c, maxBytes)...)
}
return final
}
// hardSplit slices s into pieces no larger than maxBytes, breaking at line
// boundaries where possible and otherwise mid-line at a UTF-8 rune boundary.
// Last resort for content that has neither headings nor blank-line paragraphs.
func hardSplit(s string, maxBytes int) []string {
var out []string
var cur strings.Builder
flush := func() {
if cur.Len() > 0 {
out = append(out, cur.String())
cur.Reset()
}
}
for _, line := range strings.SplitAfter(s, "\n") {
if line == "" {
continue
}
if len(line) > maxBytes {
flush()
out = append(out, runeSplit(line, maxBytes)...)
continue
}
if cur.Len() > 0 && cur.Len()+len(line) > maxBytes {
flush()
}
cur.WriteString(line)
}
flush()
return out
}
// runeSplit slices s into <=maxBytes pieces without splitting a UTF-8 rune.
func runeSplit(s string, maxBytes int) []string {
var out []string
for len(s) > maxBytes {
cut := maxBytes
for cut > 0 && !utf8.RuneStart(s[cut]) {
cut--
}
if cut == 0 { // single rune wider than the budget; emit it whole
cut = maxBytes
}
out = append(out, s[:cut])
s = s[cut:]
}
if len(s) > 0 {
out = append(out, s)
}
return out
}
@@ -30,6 +30,23 @@ func TestChunkMarkdown_SplitsAtHeadings(t *testing.T) {
}
}
func TestChunkMarkdown_HardSplitsHeadinglessOversizedBlock(t *testing.T) {
// A document with no headings and no blank-line paragraph breaks (e.g.
// JSON-lines like wiki/telos/decisions/human-intent-column.md). The old
// chunker emitted it as one over-budget chunk → nomic-embed returned
// "input length exceeds the context length" (400). Every chunk must now
// fit the budget, with no content lost.
maxBytes := 200
src := strings.Repeat("x", 1000) // one 1000-byte blob, no headings, no \n\n
out := vectorstore.ChunkMarkdown(src, maxBytes)
require.Greater(t, len(out), 1, "oversized blob must be split")
for i, c := range out {
assert.LessOrEqual(t, len(c), maxBytes, "chunk %d over budget: %d bytes", i, len(c))
}
assert.Equal(t, 1000, strings.Count(strings.Join(out, ""), "x"), "no content lost")
}
func TestChunkMarkdown_FurtherSplitsOversizedSection(t *testing.T) {
// One H2 section with 4 paragraphs of ~80 chars each, limit 100.
src := "## big\n\n" +
+109
View File
@@ -0,0 +1,109 @@
// Package webhook triggers an on-demand brain-sync Job when Gitea pushes to
// mathias/brain, instead of waiting for the next 15-minute CronJob poll.
package webhook
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
batchv1 "k8s.io/api/batch/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
)
// VerifySignature checks a Gitea webhook's X-Gitea-Signature header: a
// hex-encoded HMAC-SHA256 of the raw request body, keyed by the shared
// webhook secret. Constant-time compare — timing must not leak how much of
// the signature matched.
func VerifySignature(payload []byte, signatureHeader, secret string) bool {
if signatureHeader == "" {
return false
}
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(payload)
expected := hex.EncodeToString(mac.Sum(nil))
return hmac.Equal([]byte(expected), []byte(signatureHeader))
}
// pushEvent is the subset of Gitea's push webhook payload this handler needs.
type pushEvent struct {
Ref string `json:"ref"`
Repo struct {
FullName string `json:"full_name"`
} `json:"repository"`
}
// TriggerJobFromCronJob reads the named CronJob's job template and creates a
// new, uniquely-named Job from it — the same thing `kubectl create job
// --from=cronjob/<name>` does. Reuses the CronJob's already-tested script
// rather than re-implementing sync logic here.
func TriggerJobFromCronJob(ctx context.Context, cs kubernetes.Interface, namespace, cronJobName string) (string, error) {
cj, err := cs.BatchV1().CronJobs(namespace).Get(ctx, cronJobName, metav1.GetOptions{})
if err != nil {
return "", fmt.Errorf("get cronjob %s/%s: %w", namespace, cronJobName, err)
}
job := &batchv1.Job{
ObjectMeta: metav1.ObjectMeta{
GenerateName: cronJobName + "-webhook-",
Namespace: namespace,
Annotations: map[string]string{
"triggered-by": "brain-webhook",
},
},
Spec: cj.Spec.JobTemplate.Spec,
}
created, err := cs.BatchV1().Jobs(namespace).Create(ctx, job, metav1.CreateOptions{})
if err != nil {
return "", fmt.Errorf("create job from cronjob %s/%s: %w", namespace, cronJobName, err)
}
return created.Name, nil
}
// Handler is the HTTP handler for Gitea's push webhook on mathias/brain.
type Handler struct {
Secret string
Clientset kubernetes.Interface
Namespace string // e.g. "brain"
CronJobName string // e.g. "brain-sync"
WatchRepo string // e.g. "mathias/brain"
Logger *slog.Logger
}
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
body, err := io.ReadAll(r.Body)
if err != nil {
http.Error(w, "bad body", http.StatusBadRequest)
return
}
sig := r.Header.Get("X-Gitea-Signature")
if !VerifySignature(body, sig, h.Secret) {
http.Error(w, "bad signature", http.StatusUnauthorized)
return
}
var ev pushEvent
if err := json.Unmarshal(body, &ev); err != nil {
http.Error(w, "bad payload", http.StatusBadRequest)
return
}
if ev.Repo.FullName != h.WatchRepo || ev.Ref != "refs/heads/main" {
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprintf(w, "ignored: repo=%s ref=%s", ev.Repo.FullName, ev.Ref)
return
}
jobName, err := TriggerJobFromCronJob(r.Context(), h.Clientset, h.Namespace, h.CronJobName)
if err != nil {
h.Logger.Error("webhook: trigger job failed", "err", err)
http.Error(w, "trigger failed", http.StatusInternalServerError)
return
}
h.Logger.Info("webhook: triggered brain-sync job", "job", jobName)
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprintf(w, "triggered %s", jobName)
}
+222
View File
@@ -0,0 +1,222 @@
package webhook
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
batchv1 "k8s.io/api/batch/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/client-go/kubernetes/fake"
k8stesting "k8s.io/client-go/testing"
)
// newFakeClientset simulates the real API server's GenerateName expansion,
// which the plain fake clientset tracker does not do on its own -- without
// this, every created Job keeps an empty Name (a fake-clientset limitation,
// not real API server behavior).
func newFakeClientset(objects ...runtime.Object) *fake.Clientset {
cs := fake.NewSimpleClientset(objects...)
cs.PrependReactor("create", "jobs", func(action k8stesting.Action) (bool, runtime.Object, error) {
createAction := action.(k8stesting.CreateAction)
job, ok := createAction.GetObject().(*batchv1.Job)
if ok && job.Name == "" && job.GenerateName != "" {
job.Name = job.GenerateName + "test0001"
}
return false, nil, nil // not "handled" -- let the default reactor store it
})
return cs
}
func sign(t *testing.T, payload []byte, secret string) string {
t.Helper()
mac := hmac.New(sha256.New, []byte(secret))
mac.Write(payload)
return hex.EncodeToString(mac.Sum(nil))
}
func testLogger() *slog.Logger {
return slog.New(slog.NewTextHandler(os.Stderr, nil))
}
func fakeCronJob(namespace, name string) *batchv1.CronJob {
return &batchv1.CronJob{
ObjectMeta: metav1.ObjectMeta{Name: name, Namespace: namespace},
Spec: batchv1.CronJobSpec{
JobTemplate: batchv1.JobTemplateSpec{
Spec: batchv1.JobSpec{
Template: corev1.PodTemplateSpec{
Spec: corev1.PodSpec{
Containers: []corev1.Container{
{Name: "sync", Image: "alpine/git:v2.47.2"},
},
RestartPolicy: corev1.RestartPolicyNever,
},
},
},
},
},
}
}
// --------------------------------------------------------------------------- #
// VerifySignature
// --------------------------------------------------------------------------- #
func TestVerifySignature_AcceptsCorrectHMAC(t *testing.T) {
payload := []byte(`{"ref":"refs/heads/main"}`)
secret := "s3cret"
sig := sign(t, payload, secret)
assert.True(t, VerifySignature(payload, sig, secret))
}
func TestVerifySignature_RejectsWrongSecret(t *testing.T) {
payload := []byte(`{"ref":"refs/heads/main"}`)
sig := sign(t, payload, "right-secret")
assert.False(t, VerifySignature(payload, sig, "wrong-secret"))
}
func TestVerifySignature_RejectsTamperedPayload(t *testing.T) {
secret := "s3cret"
sig := sign(t, []byte(`{"ref":"refs/heads/main"}`), secret)
assert.False(t, VerifySignature([]byte(`{"ref":"refs/heads/evil"}`), sig, secret))
}
func TestVerifySignature_RejectsEmptySignature(t *testing.T) {
assert.False(t, VerifySignature([]byte("payload"), "", "secret"))
}
// --------------------------------------------------------------------------- #
// TriggerJobFromCronJob
// --------------------------------------------------------------------------- #
func TestTriggerJobFromCronJob_CreatesJobMatchingTemplate(t *testing.T) {
cs := newFakeClientset(fakeCronJob("brain", "brain-sync"))
jobName, err := TriggerJobFromCronJob(context.Background(), cs, "brain", "brain-sync")
require.NoError(t, err)
assert.NotEmpty(t, jobName)
jobs, err := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
require.NoError(t, err)
require.Len(t, jobs.Items, 1)
assert.Equal(t, "alpine/git:v2.47.2", jobs.Items[0].Spec.Template.Spec.Containers[0].Image)
}
func TestTriggerJobFromCronJob_ErrorsWhenCronJobMissing(t *testing.T) {
cs := fake.NewSimpleClientset()
_, err := TriggerJobFromCronJob(context.Background(), cs, "brain", "brain-sync")
assert.Error(t, err)
}
// --------------------------------------------------------------------------- #
// Handler.ServeHTTP
// --------------------------------------------------------------------------- #
func newTestHandler(cs *fake.Clientset) *Handler {
return &Handler{
Secret: "s3cret",
Clientset: cs,
Namespace: "brain",
CronJobName: "brain-sync",
WatchRepo: "mathias/brain",
Logger: testLogger(),
}
}
func pushPayload(t *testing.T, repo, ref string) []byte {
t.Helper()
body := map[string]any{
"ref": ref,
"repository": map[string]any{
"full_name": repo,
},
}
b, err := json.Marshal(body)
require.NoError(t, err)
return b
}
func TestHandler_RejectsMissingSignature(t *testing.T) {
cs := fake.NewSimpleClientset(fakeCronJob("brain", "brain-sync"))
h := newTestHandler(cs)
payload := pushPayload(t, "mathias/brain", "refs/heads/main")
req := httptest.NewRequest(http.MethodPost, "/webhooks/brain-sync", bytes.NewReader(payload))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
assert.Equal(t, http.StatusUnauthorized, rec.Code)
jobs, _ := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
assert.Empty(t, jobs.Items, "must not trigger a job on an unsigned request")
}
func TestHandler_RejectsWrongSignature(t *testing.T) {
cs := fake.NewSimpleClientset(fakeCronJob("brain", "brain-sync"))
h := newTestHandler(cs)
payload := pushPayload(t, "mathias/brain", "refs/heads/main")
req := httptest.NewRequest(http.MethodPost, "/webhooks/brain-sync", bytes.NewReader(payload))
req.Header.Set("X-Gitea-Signature", sign(t, payload, "not-the-real-secret"))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
assert.Equal(t, http.StatusUnauthorized, rec.Code)
jobs, _ := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
assert.Empty(t, jobs.Items)
}
func TestHandler_IgnoresOtherRepos(t *testing.T) {
cs := fake.NewSimpleClientset(fakeCronJob("brain", "brain-sync"))
h := newTestHandler(cs)
payload := pushPayload(t, "mathias/some-other-repo", "refs/heads/main")
req := httptest.NewRequest(http.MethodPost, "/webhooks/brain-sync", bytes.NewReader(payload))
req.Header.Set("X-Gitea-Signature", sign(t, payload, "s3cret"))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
assert.Equal(t, http.StatusOK, rec.Code)
jobs, _ := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
assert.Empty(t, jobs.Items, "must not trigger for a push to an unrelated repo")
}
func TestHandler_IgnoresNonMainBranch(t *testing.T) {
cs := fake.NewSimpleClientset(fakeCronJob("brain", "brain-sync"))
h := newTestHandler(cs)
payload := pushPayload(t, "mathias/brain", "refs/heads/some-feature-branch")
req := httptest.NewRequest(http.MethodPost, "/webhooks/brain-sync", bytes.NewReader(payload))
req.Header.Set("X-Gitea-Signature", sign(t, payload, "s3cret"))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
assert.Equal(t, http.StatusOK, rec.Code)
jobs, _ := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
assert.Empty(t, jobs.Items, "must not trigger for a push to a non-main branch")
}
func TestHandler_TriggersJobOnValidMainPush(t *testing.T) {
cs := fake.NewSimpleClientset(fakeCronJob("brain", "brain-sync"))
h := newTestHandler(cs)
payload := pushPayload(t, "mathias/brain", "refs/heads/main")
req := httptest.NewRequest(http.MethodPost, "/webhooks/brain-sync", bytes.NewReader(payload))
req.Header.Set("X-Gitea-Signature", sign(t, payload, "s3cret"))
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
assert.Equal(t, http.StatusOK, rec.Code)
jobs, err := cs.BatchV1().Jobs("brain").List(context.Background(), metav1.ListOptions{})
require.NoError(t, err)
assert.Len(t, jobs.Items, 1, "a valid push to main on the watched repo must trigger exactly one job")
}
+2
View File
@@ -14,6 +14,7 @@ type RoutingConfig struct {
LiteLLMBaseURL string // LITELLM_BASE_URL, default https://llm-api.d-ma.be
LiteLLMAPIKey string // LITELLM_API_KEY
BrainURL string // BRAIN_URL, default http://ingestion.supervisor:3300
BrainMCPToken string // BRAIN_MCP_TOKEN, bearer for the auth-gated ingestion /mcp (session_log)
FastModel string // HYPERGUILD_FAST_MODEL, default koala/qwen35-9b-fast
ThinkingModel string // HYPERGUILD_THINKING_MODEL, default iguana/gemma4-26b
// RouteLocalFloor and RouteLocalCeil intentionally invert the usual
@@ -44,6 +45,7 @@ func LoadRouting() (RoutingConfig, error) {
LiteLLMBaseURL: envOr("LITELLM_BASE_URL", "https://llm-api.d-ma.be"),
LiteLLMAPIKey: os.Getenv("LITELLM_API_KEY"),
BrainURL: envOr("BRAIN_URL", "http://ingestion.supervisor:3300"),
BrainMCPToken: os.Getenv("BRAIN_MCP_TOKEN"),
FastModel: envOr("HYPERGUILD_FAST_MODEL", "koala/qwen35-9b-fast"),
ThinkingModel: envOr("HYPERGUILD_THINKING_MODEL", "iguana/gemma4-26b"),
}
-21
View File
@@ -1,21 +0,0 @@
package routing
import (
"crypto/sha256"
"encoding/binary"
)
// CanonicalHash returns a deterministic 64-bit hash of (system, user).
// Used to make sample-band routing decisions reproducible: identical input
// strings produce the same hash on every call, independent of process state.
//
// Inputs are joined with a 0x00 byte separator before hashing — distinguishes
// (system="ab", user="cd") from (system="abcd", user="").
func CanonicalHash(system, user string) uint64 {
h := sha256.New()
h.Write([]byte(system))
h.Write([]byte{0})
h.Write([]byte(user))
sum := h.Sum(nil)
return binary.BigEndian.Uint64(sum[:8])
}
-46
View File
@@ -1,46 +0,0 @@
package routing_test
import (
"testing"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/stretchr/testify/assert"
)
func TestCanonicalHashDeterministic(t *testing.T) {
a := routing.CanonicalHash("system one", "user one")
b := routing.CanonicalHash("system one", "user one")
assert.Equal(t, a, b, "same inputs must produce same hash")
}
func TestCanonicalHashDistinguishesInputs(t *testing.T) {
cases := [][2]string{
{"sys", "user"},
{"sys", "user2"},
{"sys2", "user"},
{"", "system\x00user"}, // separator collision attempt
{"system\x00user", ""},
}
seen := make(map[uint64]bool)
for _, c := range cases {
h := routing.CanonicalHash(c[0], c[1])
assert.False(t, seen[h], "collision on %v", c)
seen[h] = true
}
}
func TestCanonicalHashLowBitDistribution(t *testing.T) {
// Sanity check: across 1000 distinct inputs, low-bit split is roughly even.
zeros, ones := 0, 0
for i := 0; i < 1000; i++ {
h := routing.CanonicalHash("sys", string(rune('a'+(i%26)))+string(rune(i)))
if h&1 == 0 {
zeros++
} else {
ones++
}
}
// Allow ±15% deviation from 500/500. Tighter would be flaky on real data.
assert.InDelta(t, 500, zeros, 150)
assert.InDelta(t, 500, ones, 150)
}
-79
View File
@@ -1,79 +0,0 @@
package routing
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"time"
)
// LogEntry describes a single routing decision to log via the brain MCP.
type LogEntry struct {
SessionID string
Skill string // the original skill the call routed (e.g., "review")
Decision string // "local" or "thinking" or "thinking_fallback"
Message string // free-form, e.g. "model=qwen35, pass_rate=0.94"
ProjectRoot string
DurationMs int64
Failed bool // true → final_status: "fail"; false → "skip"
}
// Logger posts session_log entries to a brain MCP at BrainURL + /mcp.
type Logger struct {
BrainURL string
HTTP *http.Client
}
// NewLogger creates a Logger with a 2-second HTTP timeout.
func NewLogger(brainURL string) *Logger {
return &Logger{
BrainURL: brainURL,
HTTP: &http.Client{Timeout: 2 * time.Second},
}
}
// LogDecision posts a session_log MCP call. Errors are returned but the caller
// MUST NOT block real work on them — logging is best-effort.
func (l *Logger) LogDecision(ctx context.Context, e LogEntry) error {
status := "skip"
if e.Failed {
status = "fail"
}
payload := map[string]any{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": map[string]any{
"name": "session_log",
"arguments": map[string]any{
"session_id": e.SessionID,
"skill": "_routing",
"phase": "decide",
"final_status": status,
"message": fmt.Sprintf("%s: %s — %s", e.Skill, e.Decision, e.Message),
"duration_ms": e.DurationMs,
"project_root": e.ProjectRoot,
},
},
}
body, err := json.Marshal(payload)
if err != nil {
return fmt.Errorf("log: marshal: %w", err)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, l.BrainURL+"/mcp", bytes.NewReader(body))
if err != nil {
return fmt.Errorf("log: build request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
resp, err := l.HTTP.Do(req)
if err != nil {
return fmt.Errorf("log: request: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("log: server returned status %d", resp.StatusCode)
}
return nil
}
-81
View File
@@ -1,81 +0,0 @@
package routing_test
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestLoggerLogDecision(t *testing.T) {
var captured map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, http.MethodPost, r.Method)
assert.Equal(t, "/mcp", r.URL.Path)
body, _ := io.ReadAll(r.Body)
require.NoError(t, json.Unmarshal(body, &captured))
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{"content": []map[string]any{{"type": "text", "text": "ok"}}}})
}))
defer srv.Close()
l := routing.NewLogger(srv.URL)
err := l.LogDecision(context.Background(), routing.LogEntry{
SessionID: "sess-1",
Skill: "review",
Decision: "local",
Message: "model=qwen35, pass_rate=0.94",
ProjectRoot: "/home/x/proj",
DurationMs: 1234,
Failed: false,
})
require.NoError(t, err)
params := captured["params"].(map[string]any)
assert.Equal(t, "tools/call", captured["method"])
assert.Equal(t, "session_log", params["name"])
args := params["arguments"].(map[string]any)
assert.Equal(t, "_routing", args["skill"])
assert.Equal(t, "decide", args["phase"])
assert.Equal(t, "skip", args["final_status"])
assert.Contains(t, args["message"].(string), "review: local")
assert.Equal(t, "sess-1", args["session_id"])
assert.Equal(t, "/home/x/proj", args["project_root"])
assert.Equal(t, float64(1234), args["duration_ms"])
}
func TestLoggerLogFailure(t *testing.T) {
var captured map[string]any
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
_ = json.Unmarshal(body, &captured)
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{}})
}))
defer srv.Close()
l := routing.NewLogger(srv.URL)
err := l.LogDecision(context.Background(), routing.LogEntry{
SessionID: "s", Skill: "debug", Decision: "local", Message: "litellm down", Failed: true,
})
require.NoError(t, err)
args := captured["params"].(map[string]any)["arguments"].(map[string]any)
assert.Equal(t, "fail", args["final_status"])
}
func TestLoggerSurfacesUpstreamError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "down", http.StatusBadGateway)
}))
defer srv.Close()
l := routing.NewLogger(srv.URL)
err := l.LogDecision(context.Background(), routing.LogEntry{Skill: "x", SessionID: "y", Decision: "local"})
require.Error(t, err)
}
-85
View File
@@ -1,85 +0,0 @@
package routing
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"sync"
"time"
)
// Fetcher reads /pass-rate from the brain pod with a per-skill TTL cache.
type Fetcher struct {
BaseURL string
Window string
TTL time.Duration
HTTP *http.Client
mu sync.Mutex
cache map[string]cachedRate
}
type cachedRate struct {
value *float64
at time.Time
}
type passRateResponse struct {
PassRate *float64 `json:"pass_rate"`
}
// NewFetcher returns a Fetcher that calls baseURL + /pass-rate with the
// given window string. If ttl is zero, defaults to 60 seconds. The HTTP
// client uses a 1-second total timeout.
func NewFetcher(baseURL, window string, ttl time.Duration) *Fetcher {
if ttl == 0 {
ttl = 60 * time.Second
}
return &Fetcher{
BaseURL: baseURL,
Window: window,
TTL: ttl,
HTTP: &http.Client{Timeout: time.Second},
cache: make(map[string]cachedRate),
}
}
// Get returns the pass rate for the named skill, or nil if no data exists,
// or an error if the brain is unreachable. Caches successful results.
func (f *Fetcher) Get(ctx context.Context, skill string) (*float64, error) {
f.mu.Lock()
if c, ok := f.cache[skill]; ok && time.Since(c.at) < f.TTL {
v := c.value
f.mu.Unlock()
return v, nil
}
f.mu.Unlock()
u := fmt.Sprintf("%s/pass-rate?skill=%s&window=%s",
f.BaseURL, url.QueryEscape(skill), url.QueryEscape(f.Window))
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return nil, fmt.Errorf("passrate: build request: %w", err)
}
resp, err := f.HTTP.Do(req)
if err != nil {
return nil, fmt.Errorf("passrate: request: %w", err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("passrate: server returned status %d", resp.StatusCode)
}
var body passRateResponse
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
return nil, fmt.Errorf("passrate: decode: %w", err)
}
f.mu.Lock()
f.cache[skill] = cachedRate{value: body.PassRate, at: time.Now()}
f.mu.Unlock()
return body.PassRate, nil
}
-94
View File
@@ -1,94 +0,0 @@
package routing_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestFetcherGetReturnsPassRate(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
assert.Equal(t, http.MethodGet, r.Method)
assert.Equal(t, "/pass-rate", r.URL.Path)
assert.Equal(t, "tdd", r.URL.Query().Get("skill"))
assert.Equal(t, "7d", r.URL.Query().Get("window"))
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]any{"skill": "tdd", "pass_rate": 0.94})
}))
defer srv.Close()
f := routing.NewFetcher(srv.URL, "7d", time.Minute)
pr, err := f.Get(context.Background(), "tdd")
require.NoError(t, err)
require.NotNil(t, pr)
assert.InDelta(t, 0.94, *pr, 1e-9)
}
func TestFetcherGetReturnsNilWhenNoData(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"skill": "novel", "pass_rate": nil})
}))
defer srv.Close()
f := routing.NewFetcher(srv.URL, "7d", time.Minute)
pr, err := f.Get(context.Background(), "novel")
require.NoError(t, err)
assert.Nil(t, pr)
}
func TestFetcherCachesWithinTTL(t *testing.T) {
var calls int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&calls, 1)
_ = json.NewEncoder(w).Encode(map[string]any{"pass_rate": 0.5})
}))
defer srv.Close()
f := routing.NewFetcher(srv.URL, "7d", time.Minute)
for i := 0; i < 5; i++ {
_, err := f.Get(context.Background(), "tdd")
require.NoError(t, err)
}
assert.Equal(t, int32(1), atomic.LoadInt32(&calls), "should hit upstream once and serve four times from cache")
}
func TestFetcherFetchesAgainAfterTTLExpires(t *testing.T) {
var calls int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
atomic.AddInt32(&calls, 1)
_ = json.NewEncoder(w).Encode(map[string]any{"pass_rate": 0.5})
}))
defer srv.Close()
// Tight TTL so the test stays fast.
f := routing.NewFetcher(srv.URL, "7d", 5*time.Millisecond)
_, err := f.Get(context.Background(), "tdd")
require.NoError(t, err)
assert.Equal(t, int32(1), atomic.LoadInt32(&calls))
// Sleep past TTL, then a second Get should hit upstream again.
time.Sleep(15 * time.Millisecond)
_, err = f.Get(context.Background(), "tdd")
require.NoError(t, err)
assert.Equal(t, int32(2), atomic.LoadInt32(&calls), "expected fresh upstream call after TTL expiry")
}
func TestFetcherSurfacesUpstreamError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "boom", http.StatusInternalServerError)
}))
defer srv.Close()
f := routing.NewFetcher(srv.URL, "7d", time.Minute)
pr, err := f.Get(context.Background(), "tdd")
require.Error(t, err)
assert.Nil(t, pr)
}
-47
View File
@@ -1,47 +0,0 @@
package routing
// Decision is the route picked for a single skill call.
type Decision int
const (
DecideLocal Decision = iota
DecideClaude
)
func (d Decision) String() string {
if d == DecideLocal {
return "local"
}
return "claude"
}
// Policy holds the floor/ceil thresholds for routing decisions.
//
// Rules (in order):
//
// 1. passRate == nil → DecideLocal (default-to-local for cost-routable skills)
// 2. *passRate >= Floor → DecideLocal (trust local)
// 3. *passRate < Ceil → DecideClaude (don't trust local)
// 4. otherwise (sample band) → requestHash low bit picks: 0=local, 1=claude
type Policy struct {
Floor float64
Ceil float64
}
// Decide returns the routing decision for a single call.
// requestHash is consulted only when passRate is in the sample band [Ceil, Floor).
func (p Policy) Decide(passRate *float64, requestHash uint64) Decision {
if passRate == nil {
return DecideLocal
}
if *passRate >= p.Floor {
return DecideLocal
}
if *passRate < p.Ceil {
return DecideClaude
}
if requestHash&1 == 0 {
return DecideLocal
}
return DecideClaude
}
-36
View File
@@ -1,36 +0,0 @@
package routing_test
import (
"testing"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/stretchr/testify/assert"
)
func ptr(f float64) *float64 { return &f }
func TestPolicyDecide(t *testing.T) {
p := routing.Policy{Floor: 0.9, Ceil: 0.7}
cases := []struct {
name string
passRate *float64
hash uint64
want routing.Decision
}{
{"null pass rate → local", nil, 0, routing.DecideLocal},
{"null pass rate, hash irrelevant → local", nil, 0xDEADBEEF, routing.DecideLocal},
{"at floor → local", ptr(0.9), 0, routing.DecideLocal},
{"above floor → local", ptr(0.95), 0, routing.DecideLocal},
{"below ceil → claude", ptr(0.5), 0, routing.DecideClaude},
{"at ceil → sample-band even-hash → local", ptr(0.7), 0, routing.DecideLocal},
{"sample band, even hash → local", ptr(0.8), 2, routing.DecideLocal},
{"sample band, odd hash → claude", ptr(0.8), 3, routing.DecideClaude},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.want, p.Decide(tc.passRate, tc.hash))
})
}
}
-84
View File
@@ -1,84 +0,0 @@
package routing
import (
"context"
"fmt"
"log/slog"
)
// CompleteFunc matches the signature used by every skill package's Config.
type CompleteFunc func(ctx context.Context, model, system, user string) (string, int64, error)
// RunInput captures the per-call inputs the dispatch wrapper needs.
type RunInput struct {
Skill string
System string
User string
SessionID string
ProjectRoot string
}
// Router composes a pass-rate fetcher, a decision policy, a session logger,
// and a LiteLLM client. Skill packages receive Router.Run as their CompleteFunc.
type Router struct {
Fetcher *Fetcher
Logger *Logger
Policy Policy
FastModel string
ThinkingModel string
Complete CompleteFunc
}
// Run executes one skill call: decides local vs claude, calls LiteLLM, logs the
// decision. On local-side error, falls open by retrying once on the Claude model.
func (r *Router) Run(ctx context.Context, in RunInput) (string, int64, error) {
pr, ferr := r.Fetcher.Get(ctx, in.Skill)
if ferr != nil {
slog.Warn("router: pass-rate unreachable, defaulting to local", "skill", in.Skill, "err", ferr)
pr = nil
}
hash := CanonicalHash(in.System, in.User)
decision := r.Policy.Decide(pr, hash)
model := r.ThinkingModel
if decision == DecideLocal {
model = r.FastModel
}
out, ms, err := r.Complete(ctx, model, in.System, in.User)
if lerr := r.Logger.LogDecision(ctx, LogEntry{
SessionID: in.SessionID,
Skill: in.Skill,
Decision: decision.String(),
Message: fmt.Sprintf("model=%s, pass_rate=%s", model, formatPassRate(pr)),
ProjectRoot: in.ProjectRoot,
DurationMs: ms,
Failed: err != nil,
}); lerr != nil {
slog.Warn("router: log decision failed", "skill", in.Skill, "err", lerr)
}
if err != nil && decision == DecideLocal {
slog.Warn("router: fast failed, falling open to thinking model", "skill", in.Skill, "err", err)
out, ms, err = r.Complete(ctx, r.ThinkingModel, in.System, in.User)
if lerr := r.Logger.LogDecision(ctx, LogEntry{
SessionID: in.SessionID,
Skill: in.Skill,
Decision: "thinking_fallback",
Message: fmt.Sprintf("model=%s, after-fast-error", r.ThinkingModel),
ProjectRoot: in.ProjectRoot,
DurationMs: ms,
Failed: err != nil,
}); lerr != nil {
slog.Warn("router: log decision failed", "skill", in.Skill, "err", lerr)
}
}
return out, ms, err
}
func formatPassRate(pr *float64) string {
if pr == nil {
return "null"
}
return fmt.Sprintf("%.2f", *pr)
}
-136
View File
@@ -1,136 +0,0 @@
package routing_test
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
"github.com/mathiasbq/supervisor/internal/routing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
type fakeLLM struct {
mu sync.Mutex
calls []struct{ Model, System, User string }
resp string
err error
errOn string // if non-empty, only the named model errors
}
func (f *fakeLLM) Complete(_ context.Context, model, system, user string) (string, int64, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, struct{ Model, System, User string }{model, system, user})
if f.errOn == model {
return "", 0, f.err
}
if f.err != nil && f.errOn == "" {
return "", 0, f.err
}
return f.resp, 100, nil
}
func newRouter(t *testing.T, llm *fakeLLM, passRate float64) (*routing.Router, *httptest.Server, *httptest.Server) {
t.Helper()
brain := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/pass-rate":
_ = json.NewEncoder(w).Encode(map[string]any{"pass_rate": passRate})
case "/mcp":
_ = json.NewEncoder(w).Encode(map[string]any{"jsonrpc": "2.0", "id": 1, "result": map[string]any{}})
}
}))
t.Cleanup(brain.Close)
r := &routing.Router{
Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute),
Logger: routing.NewLogger(brain.URL),
Policy: routing.Policy{Floor: 0.9, Ceil: 0.7},
FastModel: "koala/qwen35-9b-fast",
ThinkingModel: "iguana/gemma4-26b",
Complete: llm.Complete,
}
return r, brain, brain
}
func TestRouterRoutesLocalAtHighPassRate(t *testing.T) {
llm := &fakeLLM{resp: "ok"}
r, _, _ := newRouter(t, llm, 0.95)
out, _, err := r.Run(context.Background(), routing.RunInput{
Skill: "review", System: "sys", User: "user", SessionID: "s1", ProjectRoot: "/p",
})
require.NoError(t, err)
assert.Equal(t, "ok", out)
llm.mu.Lock()
defer llm.mu.Unlock()
require.Len(t, llm.calls, 1)
assert.Equal(t, "koala/qwen35-9b-fast", llm.calls[0].Model)
}
func TestRouterRoutesThinkingAtLowPassRate(t *testing.T) {
llm := &fakeLLM{resp: "ok"}
r, _, _ := newRouter(t, llm, 0.3)
_, _, err := r.Run(context.Background(), routing.RunInput{
Skill: "review", System: "sys", User: "user", SessionID: "s2",
})
require.NoError(t, err)
llm.mu.Lock()
defer llm.mu.Unlock()
require.Len(t, llm.calls, 1)
assert.Equal(t, "iguana/gemma4-26b", llm.calls[0].Model)
}
func TestRouterFailsOpenFastErrorToThinking(t *testing.T) {
llm := &fakeLLM{resp: "ok-after-fallback", err: errors.New("fast boom"), errOn: "koala/qwen35-9b-fast"}
r, _, _ := newRouter(t, llm, 0.95) // would route fast
out, _, err := r.Run(context.Background(), routing.RunInput{
Skill: "review", System: "sys", User: "user", SessionID: "s3",
})
require.NoError(t, err)
assert.Equal(t, "ok-after-fallback", out)
llm.mu.Lock()
defer llm.mu.Unlock()
require.Len(t, llm.calls, 2)
assert.Equal(t, "koala/qwen35-9b-fast", llm.calls[0].Model)
assert.Equal(t, "iguana/gemma4-26b", llm.calls[1].Model)
}
func TestRouterDefaultsToFastWhenBrainUnreachable(t *testing.T) {
// Brain returns 500 → fetcher errors → router treats pass rate as nil → fast.
brain := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "down", http.StatusInternalServerError)
}))
defer brain.Close()
llm := &fakeLLM{resp: "ok"}
r := &routing.Router{
Fetcher: routing.NewFetcher(brain.URL, "7d", time.Minute),
Logger: routing.NewLogger(brain.URL),
Policy: routing.Policy{Floor: 0.9, Ceil: 0.7},
FastModel: "koala/qwen35-9b-fast",
ThinkingModel: "iguana/gemma4-26b",
Complete: llm.Complete,
}
_, _, err := r.Run(context.Background(), routing.RunInput{
Skill: "review", System: "sys", User: "user", SessionID: "s4",
})
require.NoError(t, err)
llm.mu.Lock()
defer llm.mu.Unlock()
require.Len(t, llm.calls, 1)
assert.Equal(t, "koala/qwen35-9b-fast", llm.calls[0].Model)
}
-80
View File
@@ -1,80 +0,0 @@
package routing_test
import (
"context"
"encoding/json"
"os"
"sort"
"testing"
"github.com/mathiasbq/supervisor/internal/registry"
"github.com/mathiasbq/supervisor/internal/skills/debug"
"github.com/mathiasbq/supervisor/internal/skills/retrospective"
"github.com/mathiasbq/supervisor/internal/skills/review"
"github.com/mathiasbq/supervisor/internal/skills/trainer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestToolsListMatchesSupervisorSnapshot pins the four routed skills' tool
// definitions to the supervisor's current advertisement. A deliberate schema
// change must be reflected here by updating testdata/tools_list.snapshot.json.
func TestToolsListMatchesSupervisorSnapshot(t *testing.T) {
complete := func(_ context.Context, _, _, _ string) (string, int64, error) {
return "", 0, nil
}
reg := registry.New()
reg.Register(review.New(review.Config{
SkillPrompt: "stub",
DefaultModel: "stub",
CompleteFunc: complete,
}))
reg.Register(debug.New(debug.Config{
SkillPrompt: "stub",
DefaultModel: "stub",
CompleteFunc: complete,
}))
reg.Register(retrospective.New(retrospective.Config{
SkillPrompt: "stub",
DefaultModel: "stub",
CompleteFunc: complete,
}))
reg.Register(trainer.New(trainer.Config{
ReaderPrompt: "stub",
WriterPrompt: "stub",
DefaultModel: "stub",
CompleteFunc: complete,
}))
wanted := map[string]bool{
"review": true,
"debug": true,
"retrospective": true,
"trainer": true,
}
var routed []registry.ToolDef
for _, td := range reg.Tools() {
if wanted[td.Name] {
routed = append(routed, td)
}
}
sort.Slice(routed, func(i, j int) bool { return routed[i].Name < routed[j].Name })
got, err := json.MarshalIndent(routed, "", " ")
require.NoError(t, err)
want, err := os.ReadFile("testdata/tools_list.snapshot.json")
require.NoError(t, err)
// Normalize both via re-encode so whitespace differences don't dominate.
var gotV, wantV any
require.NoError(t, json.Unmarshal(got, &gotV))
require.NoError(t, json.Unmarshal(want, &wantV))
gotN, _ := json.MarshalIndent(gotV, "", " ")
wantN, _ := json.MarshalIndent(wantV, "", " ")
assert.Equal(t, string(wantN), string(gotN),
"tool advertisement drifted from supervisor snapshot — update testdata/tools_list.snapshot.json deliberately if the schema change is intentional")
}
-97
View File
@@ -1,97 +0,0 @@
[
{
"name": "debug",
"description": "Consult a local model to analyse an error and return hypotheses ordered by likelihood, each with a concrete verification step.",
"inputSchema": {
"properties": {
"context": {
"type": "string"
},
"error": {
"type": "string"
},
"model": {
"type": "string"
},
"project_root": {
"type": "string"
},
"session_id": {
"type": "string"
}
},
"required": [
"project_root",
"error"
],
"type": "object"
}
},
{
"name": "retrospective",
"description": "Consult a local model to analyse a completed session and identify what is novel or worth preserving as organizational knowledge.",
"inputSchema": {
"type": "object",
"required": [
"session_id"
],
"properties": {
"session_id": {
"type": "string"
},
"model": {
"type": "string"
}
}
}
},
{
"name": "review",
"description": "Consult a local model for a structured code review of the specified files. Returns findings with severity levels.",
"inputSchema": {
"properties": {
"context": {
"type": "string"
},
"files": {
"items": {
"type": "string"
},
"type": "array"
},
"model": {
"type": "string"
},
"project_root": {
"type": "string"
},
"session_id": {
"type": "string"
}
},
"required": [
"project_root",
"files"
],
"type": "object"
}
},
{
"name": "trainer",
"description": "Consult a local model to identify learning moments from a session log and suggest knowledge to preserve in the brain.",
"inputSchema": {
"properties": {
"model": {
"type": "string"
},
"session_id": {
"type": "string"
}
},
"required": [
"session_id"
],
"type": "object"
}
}
]
-82
View File
@@ -1,82 +0,0 @@
// internal/skills/debug/handlers.go
package debug
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/mathiasbq/supervisor/internal/brain"
"github.com/mathiasbq/supervisor/internal/session"
)
type debugArgs struct {
ProjectRoot string `json:"project_root"`
Error string `json:"error"`
Context string `json:"context"`
Model string `json:"model"`
SessionID string `json:"session_id"`
}
// Handle dispatches the MCP tool call to the appropriate handler.
func (s *Skill) Handle(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
if tool != "debug" {
return nil, fmt.Errorf("unknown tool: %s", tool)
}
var a debugArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.ProjectRoot == "" {
return nil, fmt.Errorf("project_root is required")
}
if a.Error == "" {
return nil, fmt.Errorf("error is required")
}
model := a.Model
if model == "" {
model = s.cfg.DefaultModel
}
brainCtx, _ := brain.Query(ctx, s.cfg.IngestBaseURL, a.Error+" "+a.Context, 3)
task := fmt.Sprintf(
"phase: debug\nproject_root: %s\nerror: %s\ncontext: %s\nmodel: %s",
a.ProjectRoot, a.Error, a.Context, model,
)
task = session.PrependHistory(s.cfg.SessionsDir, a.SessionID, "debug", task)
if brainCtx != "" {
task = brainCtx + "\n---\n\n" + task
}
if s.cfg.CompleteFunc == nil {
return nil, fmt.Errorf("no executor configured")
}
t0 := time.Now()
text, dur, err := s.cfg.CompleteFunc(ctx, model, s.cfg.SkillPrompt, task)
if err != nil {
return nil, err
}
if a.SessionID != "" && s.cfg.SessionsDir != "" {
msg := text
if len(msg) > 200 {
msg = msg[:200]
}
_ = session.Append(s.cfg.SessionsDir, a.SessionID, session.Entry{
SessionID: a.SessionID,
Timestamp: time.Now(),
Skill: "debug",
Phase: "debug",
ProjectRoot: a.ProjectRoot,
FinalStatus: "ok",
ModelUsed: model,
DurationMs: time.Since(t0).Milliseconds(),
Message: msg,
})
}
return json.Marshal(map[string]any{"text": text, "model": model, "duration_ms": dur})
}
-53
View File
@@ -1,53 +0,0 @@
// internal/skills/debug/handlers_test.go
package debug_test
import (
"context"
"encoding/json"
"testing"
"github.com/mathiasbq/supervisor/internal/skills/debug"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestDebugToolRegistered(t *testing.T) {
sk := debug.New(debug.Config{SkillPrompt: "debug rules"})
names := make([]string, 0)
for _, tool := range sk.Tools() {
names = append(names, tool.Name)
}
assert.Contains(t, names, "debug")
}
func TestDebugRequiresProjectRoot(t *testing.T) {
sk := debug.New(debug.Config{SkillPrompt: "d"})
_, err := sk.Handle(context.Background(), "debug", json.RawMessage(`{"error":"panic: nil pointer"}`))
assert.ErrorContains(t, err, "project_root")
}
func TestDebugRequiresError(t *testing.T) {
sk := debug.New(debug.Config{SkillPrompt: "d"})
_, err := sk.Handle(context.Background(), "debug", json.RawMessage(`{"project_root":"/tmp"}`))
assert.ErrorContains(t, err, "error")
}
func TestDebugCallsCompleteFunc(t *testing.T) {
var capturedTask string
fakeFn := func(_ context.Context, _, _, user string) (string, int64, error) {
capturedTask = user
return "HYPOTHESIS 1 (high): nil map access. Verify: go test ./...", 90, nil
}
sk := debug.New(debug.Config{SkillPrompt: "debug rules", CompleteFunc: fakeFn, SessionsDir: t.TempDir()})
out, err := sk.Handle(context.Background(), "debug", json.RawMessage(
`{"project_root":"/tmp/proj","error":"panic: nil pointer dereference at foo.go:42","context":"occurs on startup"}`,
))
require.NoError(t, err)
assert.Contains(t, capturedTask, "panic: nil pointer dereference")
assert.Contains(t, capturedTask, "occurs on startup")
var result map[string]any
require.NoError(t, json.Unmarshal(out, &result))
assert.Contains(t, result["text"], "nil map access")
}
-55
View File
@@ -1,55 +0,0 @@
// internal/skills/debug/skill.go
package debug
import (
"context"
"encoding/json"
"github.com/mathiasbq/supervisor/internal/registry"
)
// CompleteFunc is the function used to call a local model.
type CompleteFunc func(ctx context.Context, model, system, user string) (string, int64, error)
// Config holds dependencies for the debug skill.
type Config struct {
SkillPrompt string
DefaultModel string
CompleteFunc CompleteFunc
SessionsDir string
IngestBaseURL string
}
// Skill implements the debug MCP tool.
type Skill struct{ cfg Config }
// New creates a new debug Skill.
func New(cfg Config) *Skill { return &Skill{cfg: cfg} }
// Name returns the skill identifier.
func (s *Skill) Name() string { return "debug" }
// Tools returns the MCP tool definitions for this skill.
func (s *Skill) Tools() []registry.ToolDef {
schema := func(required []string, props map[string]any) json.RawMessage {
b, _ := json.Marshal(map[string]any{"type": "object", "required": required, "properties": props})
return b
}
str := map[string]any{"type": "string"}
return []registry.ToolDef{
{
Name: "debug",
Description: "Consult a local model to analyse an error and return hypotheses ordered by likelihood, each with a concrete verification step.",
InputSchema: schema(
[]string{"project_root", "error"},
map[string]any{
"project_root": str,
"error": str,
"context": str,
"model": str,
"session_id": str,
},
),
},
}
}
-297
View File
@@ -1,297 +0,0 @@
package project
import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"github.com/mathiasbq/supervisor/internal/githubclient"
"github.com/mathiasbq/supervisor/internal/mcpclient"
)
type createArgs struct {
Name string `json:"name"`
Description string `json:"description"`
Hypothesis string `json:"hypothesis"`
Folder string `json:"folder"`
Stack string `json:"stack"`
Private bool `json:"private"`
MirrorToGitHub bool `json:"mirror_to_github,omitempty"`
}
type createResult struct {
GiteaURL string `json:"gitea_url"`
GitHubURL string `json:"github_url"`
IssueURL string `json:"issue_url"`
NextSteps string `json:"next_steps"`
// Reached records the steps that completed. Populated on partial failure
// so callers can resume manually instead of guessing what already ran.
Reached []string `json:"reached,omitempty"`
// FailedStep is non-empty when a downstream gitea-mcp call returned an
// error; the error itself is surfaced via the JSON-RPC error response,
// this field tells the operator which step it happened in.
FailedStep string `json:"failed_step,omitempty"`
}
func errUnknownTool(name string) error { return fmt.Errorf("unknown tool: %s", name) }
// step names — must match what we surface in failed_step / reached.
const (
stepCreateRepo = "create_repo"
stepCreateGitHub = "create_github_repo"
stepMirror = "mirror"
stepInfraCommit = "infra_commit"
stepIssue = "issue"
)
func (s *Skill) handleCreate(ctx context.Context, raw json.RawMessage) (json.RawMessage, error) {
var args createArgs
if err := json.Unmarshal(raw, &args); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if err := validate(args); err != nil {
return nil, err
}
tmpl := templateFor(args.Stack)
giteaURL := fmt.Sprintf("http://gitea.d-ma.be/%s/%s", s.cfg.GiteaOwner, args.Name)
res := createResult{
GiteaURL: giteaURL,
}
if args.MirrorToGitHub {
res.GitHubURL = fmt.Sprintf("https://github.com/%s/%s", s.cfg.GitHubOwner, args.Name)
}
// Step 1: create_project_from_template. If the repo already exists,
// gitea-mcp returns -32003 Conflict; we treat that as idempotent success
// and continue to the next steps so re-running self-heals partial runs.
existed, err := s.callCreateRepo(ctx, args, tmpl)
if err != nil {
return marshalPartial(res, stepCreateRepo, err)
}
res.Reached = append(res.Reached, stepCreateRepo)
// Steps 2+3 are skipped when MirrorToGitHub is false. Default per
// infra ADR (Gitea as true master, GitHub as optional opt-in): keep
// client / business-logic / personal repos Gitea-only. Set
// `mirror_to_github: true` for open-source projects that want a
// public GitHub mirror (hyperguild, gitea-mcp, template-*).
if args.MirrorToGitHub {
// Step 2: create empty GitHub repo. Gitea's push-mirror cannot push
// to a non-existent remote, so the destination must exist before
// step 3 configures the mirror. Skipped when GitHub client is unset
// (degraded mode — see Config.GitHub doc).
if s.cfg.GitHub != nil {
if err := s.callCreateGitHubRepo(ctx, args); err != nil && !errors.Is(err, githubclient.ErrAlreadyExists) {
return marshalPartial(res, stepCreateGitHub, err)
}
res.Reached = append(res.Reached, stepCreateGitHub)
}
// Step 3: configure push mirror to GitHub. Idempotent: if a mirror with
// the same remote already exists, gitea-mcp returns Conflict; we swallow it.
if err := s.callMirror(ctx, args.Name); err != nil {
if !isConflict(err) {
return marshalPartial(res, stepMirror, err)
}
}
res.Reached = append(res.Reached, stepMirror)
}
// Step 3: commit staging namespace manifest to infra repo. Done before
// the issue so the staging env is reconciling by the time the issue lands.
if err := s.callInfraCommit(ctx, args.Name); err != nil {
if !isConflict(err) {
return marshalPartial(res, stepInfraCommit, err)
}
}
res.Reached = append(res.Reached, stepInfraCommit)
// Step 4: open the experiment-brief issue on the new repo.
issueURL, err := s.callIssue(ctx, args, existed)
if err != nil {
return marshalPartial(res, stepIssue, err)
}
res.IssueURL = issueURL
res.Reached = append(res.Reached, stepIssue)
folder := args.Folder
if folder == "" {
folder = "."
}
res.NextSteps = fmt.Sprintf(
"cd ~/dev/%s/%s && task new-project -- %s personal %s %s && git remote add origin http://gitea.d-ma.be/%s/%s.git && git push -u origin main",
folder, args.Name, args.Name, folder, args.Stack, s.cfg.GiteaOwner, args.Name,
)
return marshalResult(res)
}
// callCreateRepo invokes create_project_from_template. Returns (existed, err)
// where existed=true means the destination was already present and we should
// treat it as a no-op success (idempotency).
func (s *Skill) callCreateRepo(ctx context.Context, args createArgs, template string) (bool, error) {
var out struct {
HTMLURL string `json:"html_url"`
}
err := s.cfg.Client.CallTool(ctx, "create_project_from_template", map[string]any{
"owner": s.cfg.GiteaOwner,
"name": args.Name,
"description": args.Description,
"private": args.Private,
"template_name": template,
}, &out)
if err == nil {
return false, nil
}
if isConflict(err) {
return true, nil
}
return false, err
}
// callCreateGitHubRepo creates the empty destination repo on GitHub.
// auto_init=false in githubclient so first push from gitea doesn't conflict
// with an auto-generated README.
func (s *Skill) callCreateGitHubRepo(ctx context.Context, args createArgs) error {
_, err := s.cfg.GitHub.CreateRepo(ctx, args.Name, args.Description, args.Private)
return err
}
// callMirror configures the push mirror to GitHub.
func (s *Skill) callMirror(ctx context.Context, name string) error {
remote := fmt.Sprintf("https://github.com/%s/%s.git", s.cfg.GitHubOwner, name)
return s.cfg.Client.CallTool(ctx, "repo_mirror_push", map[string]any{
"owner": s.cfg.GiteaOwner,
"name": name,
"action": "add",
"remote_address": remote,
"remote_username": s.cfg.GitHubOwner,
"remote_password": s.cfg.GitHubPAT,
"interval": "8h0m0s",
"sync_on_commit": true,
}, nil)
}
// callInfraCommit writes the staging namespace manifest directly to infra
// main. Flux reconciles within ~60s. See DECISIONS.md 2026-05-18.
func (s *Skill) callInfraCommit(ctx context.Context, name string) error {
manifest := stagingNamespaceManifest(name, time.Now().UTC().Format(time.RFC3339))
return s.cfg.Client.CallTool(ctx, "file_write_branch", map[string]any{
"owner": s.cfg.GiteaOwner,
"name": s.cfg.InfraRepo,
"path": fmt.Sprintf("k3s/staging/%s/namespace.yaml", name),
"content": manifest,
"branch": "main",
"message": fmt.Sprintf("feat(staging): add namespace for %s\n\nGenerated by hyperguild project_create.", name),
}, nil)
}
// callIssue opens the experiment-brief issue on the newly-created repo.
// existed=true (repo pre-existed) still posts a new brief — repeated runs
// can intentionally restate intent without colliding.
func (s *Skill) callIssue(ctx context.Context, args createArgs, existed bool) (string, error) {
body := experimentBrief(args, existed)
var out struct {
HTMLURL string `json:"html_url"`
}
err := s.cfg.Client.CallTool(ctx, "issue_create", map[string]any{
"owner": s.cfg.GiteaOwner,
"name": args.Name,
"title": "experiment brief: " + args.Description,
"body": body,
}, &out)
if err != nil {
return "", err
}
return out.HTMLURL, nil
}
func stagingNamespaceManifest(name, createdAt string) string {
return fmt.Sprintf(`apiVersion: v1
kind: Namespace
metadata:
name: staging-%s
labels:
managed-by: hyperguild
project: %s
created-at: "%s"
`, name, name, createdAt)
}
func experimentBrief(args createArgs, existed bool) string {
var b strings.Builder
b.WriteString("## Hypothesis\n\n")
b.WriteString(args.Hypothesis)
b.WriteString("\n\n## Description\n\n")
b.WriteString(args.Description)
b.WriteString("\n\n## Stack\n\n`")
b.WriteString(args.Stack)
b.WriteString("`\n\n## Provisioning\n\n")
b.WriteString("- Repo created from `template-")
b.WriteString(args.Stack)
b.WriteString("` on Gitea.\n")
if args.MirrorToGitHub {
b.WriteString("- Push-mirror configured to GitHub.\n")
} else {
b.WriteString("- Gitea-only (no GitHub mirror — set `mirror_to_github: true` to opt in).\n")
}
b.WriteString("- Staging namespace manifest committed to infra repo.\n\n")
if existed {
b.WriteString("> Note: this repo already existed when `project_create` ran — provisioning steps were re-applied idempotently.\n")
}
return b.String()
}
func validate(args createArgs) error {
if args.Name == "" {
return errors.New("name is required")
}
if args.Description == "" {
return errors.New("description is required")
}
if args.Hypothesis == "" {
return errors.New("hypothesis is required")
}
if args.Stack != "go-agent" && args.Stack != "go-web" {
return fmt.Errorf("stack must be go-agent or go-web, got %q", args.Stack)
}
return nil
}
func templateFor(stack string) string {
switch stack {
case "go-agent":
return "template-go-agent"
default:
return "template-go-web"
}
}
func isConflict(err error) bool {
var me *mcpclient.Error
if errors.As(err, &me) && me.Code == -32003 {
return true
}
return false
}
func marshalResult(r createResult) (json.RawMessage, error) {
b, err := json.Marshal(r)
if err != nil {
return nil, fmt.Errorf("marshal result: %w", err)
}
return b, nil
}
func marshalPartial(r createResult, step string, inner error) (json.RawMessage, error) {
r.FailedStep = step
b, _ := json.Marshal(r)
return b, fmt.Errorf("project_create step %q failed: %w", step, inner)
}
-419
View File
@@ -1,419 +0,0 @@
package project_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"github.com/mathiasbq/supervisor/internal/githubclient"
"github.com/mathiasbq/supervisor/internal/mcpclient"
"github.com/mathiasbq/supervisor/internal/skills/project"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// fakeGitHub captures POST /user/repos calls.
type fakeGitHub struct {
mu sync.Mutex
Calls []map[string]any
ReturnError int // 0 = 201 Created, 422 = already exists, etc.
}
func (g *fakeGitHub) handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var args map[string]any
_ = json.NewDecoder(r.Body).Decode(&args)
g.mu.Lock()
g.Calls = append(g.Calls, args)
code := g.ReturnError
g.mu.Unlock()
switch code {
case 0:
w.WriteHeader(http.StatusCreated)
_, _ = w.Write([]byte(`{"full_name":"mathiasb/x","html_url":"https://github.com/mathiasb/x","clone_url":"https://github.com/mathiasb/x.git"}`))
case 422:
w.WriteHeader(http.StatusUnprocessableEntity)
_, _ = w.Write([]byte(`{"errors":[{"message":"name already exists on this account"}]}`))
default:
w.WriteHeader(code)
_, _ = w.Write([]byte(`{"message":"boom"}`))
}
})
}
// fakeGiteaMCP implements just enough of the JSON-RPC tools/call surface
// to drive project_create end-to-end without an actual gitea-mcp server.
type fakeGiteaMCP struct {
mu sync.Mutex
// Recorded calls in order.
Calls []recordedCall
// Per-tool response. Default is a generic success object.
Responses map[string]any
// Per-tool error response, takes precedence over Responses.
Errors map[string]rpcErr
}
type rpcErr struct {
Code int
Message string
}
type recordedCall struct {
Tool string
Args map[string]any
}
func (f *fakeGiteaMCP) handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
var req struct {
ID int `json:"id"`
Params json.RawMessage `json:"params"`
}
_ = json.NewDecoder(r.Body).Decode(&req)
var p struct {
Name string `json:"name"`
Arguments json.RawMessage `json:"arguments"`
}
_ = json.Unmarshal(req.Params, &p)
var args map[string]any
_ = json.Unmarshal(p.Arguments, &args)
f.mu.Lock()
f.Calls = append(f.Calls, recordedCall{Tool: p.Name, Args: args})
errResp, hasErr := f.Errors[p.Name]
var resp any
if r, ok := f.Responses[p.Name]; ok {
resp = r
} else {
resp = map[string]any{"html_url": "http://gitea.example/" + p.Name}
}
f.mu.Unlock()
w.Header().Set("Content-Type", "application/json")
if hasErr {
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0",
"id": req.ID,
"error": map[string]any{"code": errResp.Code, "message": errResp.Message},
})
_, _ = w.Write(body)
return
}
respText, _ := json.Marshal(resp)
body, _ := json.Marshal(map[string]any{
"jsonrpc": "2.0",
"id": req.ID,
"result": map[string]any{
"content": []map[string]any{{"type": "text", "text": string(respText)}},
},
})
_, _ = w.Write(body)
})
}
func newSkill(t *testing.T, f *fakeGiteaMCP) (*project.Skill, *fakeGitHub) {
t.Helper()
srv := httptest.NewServer(f.handler())
t.Cleanup(srv.Close)
gh := &fakeGitHub{}
ghSrv := httptest.NewServer(gh.handler())
t.Cleanup(ghSrv.Close)
return project.New(project.Config{
Client: mustClient(t, srv.URL),
GitHub: githubclient.New("ghp_test").WithBaseURL(ghSrv.URL),
GiteaOwner: "mathias",
GitHubOwner: "mathiasb",
GitHubPAT: "ghp_test",
InfraRepo: "infra",
}), gh
}
// newSkillNoGitHub builds a skill with the GitHub client unset — degraded
// mode where the github-repo-creation step is skipped.
func newSkillNoGitHub(t *testing.T, f *fakeGiteaMCP) *project.Skill {
t.Helper()
srv := httptest.NewServer(f.handler())
t.Cleanup(srv.Close)
return project.New(project.Config{
Client: mustClient(t, srv.URL),
GiteaOwner: "mathias",
GitHubOwner: "mathiasb",
InfraRepo: "infra",
})
}
// mustClient builds an mcpclient against an httptest server. Uses a
// non-empty dummy token because httptest servers don't enforce bearer
// auth, but mcpclient.New now requires non-empty token (see #13).
func mustClient(t *testing.T, url string) *mcpclient.Client {
t.Helper()
c, err := mcpclient.New(url, "test-token")
require.NoError(t, err)
return c
}
// happyArgs returns the minimal valid request. With the Gitea-as-true-master
// ADR shipped, this defaults to Gitea-only (mirror_to_github omitted = false).
// Tests that need the full Gitea + GitHub mirror flow use mirroredArgs().
func happyArgs() json.RawMessage {
return json.RawMessage(`{
"name":"my-experiment",
"description":"One-line desc",
"hypothesis":"We believe X produces Y",
"folder":"AGENTS",
"stack":"go-agent",
"private":true
}`)
}
// mirroredArgs is happyArgs + mirror_to_github=true — the explicit opt-in
// path. Equivalent to the pre-ADR default.
func mirroredArgs() json.RawMessage {
return json.RawMessage(`{
"name":"my-experiment",
"description":"One-line desc",
"hypothesis":"We believe X produces Y",
"folder":"AGENTS",
"stack":"go-agent",
"private":true,
"mirror_to_github":true
}`)
}
func TestProjectCreate_HappyPath(t *testing.T) {
f := &fakeGiteaMCP{
Responses: map[string]any{
"issue_create": map[string]any{"html_url": "http://gitea.d-ma.be/mathias/my-experiment/issues/1"},
},
}
skill, gh := newSkill(t, f)
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.NoError(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment", res["gitea_url"])
assert.Equal(t, "https://github.com/mathiasb/my-experiment", res["github_url"])
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment/issues/1", res["issue_url"])
assert.Contains(t, res["next_steps"], "cd ~/dev/AGENTS/my-experiment")
assert.Contains(t, res["next_steps"], "git remote add origin")
// All 4 gitea-mcp calls in order.
require.Len(t, f.Calls, 4)
assert.Equal(t, "create_project_from_template", f.Calls[0].Tool)
assert.Equal(t, "repo_mirror_push", f.Calls[1].Tool)
assert.Equal(t, "file_write_branch", f.Calls[2].Tool)
assert.Equal(t, "issue_create", f.Calls[3].Tool)
// GitHub repo created between create_project_from_template and mirror.
require.Len(t, gh.Calls, 1)
assert.Equal(t, "my-experiment", gh.Calls[0]["name"])
assert.Equal(t, true, gh.Calls[0]["private"])
assert.Equal(t, false, gh.Calls[0]["auto_init"])
// template selection wired from stack
assert.Equal(t, "template-go-agent", f.Calls[0].Args["template_name"])
// mirror config
assert.Equal(t, "add", f.Calls[1].Args["action"])
assert.Equal(t, "https://github.com/mathiasb/my-experiment.git", f.Calls[1].Args["remote_address"])
assert.Equal(t, "ghp_test", f.Calls[1].Args["remote_password"])
// infra commit path
assert.Equal(t, "k3s/staging/my-experiment/namespace.yaml", f.Calls[2].Args["path"])
assert.Contains(t, f.Calls[2].Args["content"], "name: staging-my-experiment")
assert.Contains(t, f.Calls[2].Args["content"], "managed-by: hyperguild")
// PAT must NOT appear in the response
assert.NotContains(t, string(out), "ghp_test")
// reached records the github step too.
reached := res["reached"].([]any)
assert.Equal(t, []any{"create_repo", "create_github_repo", "mirror", "infra_commit", "issue"}, reached)
}
func TestProjectCreate_GitHubExists_Idempotent(t *testing.T) {
f := &fakeGiteaMCP{
Responses: map[string]any{
"issue_create": map[string]any{"html_url": "http://gitea.d-ma.be/mathias/my-experiment/issues/1"},
},
}
skill, gh := newSkill(t, f)
gh.ReturnError = 422 // already exists
_, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.NoError(t, err, "422 already-exists should be idempotent")
require.Len(t, f.Calls, 4, "all gitea steps still run despite github 422")
}
func TestProjectCreate_GitHubFails(t *testing.T) {
f := &fakeGiteaMCP{}
skill, gh := newSkill(t, f)
gh.ReturnError = 401 // bad PAT
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.Error(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "create_github_repo", res["failed_step"])
assert.Equal(t, []any{"create_repo"}, res["reached"])
require.Len(t, f.Calls, 1, "mirror + later steps must not run when github creation fails")
}
func TestProjectCreate_NoGitHubClient_DegradedMode(t *testing.T) {
f := &fakeGiteaMCP{
Responses: map[string]any{
"issue_create": map[string]any{"html_url": "http://gitea.d-ma.be/mathias/my-experiment/issues/1"},
},
}
skill := newSkillNoGitHub(t, f)
// Use mirroredArgs so we exercise the GitHub-mirror path. With the
// GitHub client nil, the create_github_repo step is skipped but the
// mirror step still attempts to configure the push-mirror remote
// (degraded mode preserves the prior contract for opted-in projects).
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.NoError(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
// reached does NOT include create_github_repo when client is nil.
reached := res["reached"].([]any)
assert.Equal(t, []any{"create_repo", "mirror", "infra_commit", "issue"}, reached)
}
func TestProjectCreate_Idempotent_RepoExists(t *testing.T) {
f := &fakeGiteaMCP{
Errors: map[string]rpcErr{
"create_project_from_template": {Code: -32003, Message: "already exists"},
},
Responses: map[string]any{
"issue_create": map[string]any{"html_url": "http://gitea.d-ma.be/mathias/my-experiment/issues/1"},
},
}
skill, _ := newSkill(t, f)
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.NoError(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment", res["gitea_url"])
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment/issues/1", res["issue_url"])
// Still ran all 4 gitea-mcp steps; idempotent flow falls through.
require.Len(t, f.Calls, 4)
}
func TestProjectCreate_MirrorFails(t *testing.T) {
f := &fakeGiteaMCP{
Errors: map[string]rpcErr{
"repo_mirror_push": {Code: -32000, Message: "github unreachable"},
},
}
skill, _ := newSkill(t, f)
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.Error(t, err)
assert.Contains(t, err.Error(), `"mirror" failed`)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "mirror", res["failed_step"])
reached := res["reached"].([]any)
assert.Equal(t, []any{"create_repo", "create_github_repo"}, reached)
// Steps 1 (create) + 2 (mirror attempt) reached gitea; github made 1 call.
require.Len(t, f.Calls, 2)
}
func TestProjectCreate_InfraCommitFails(t *testing.T) {
f := &fakeGiteaMCP{
Errors: map[string]rpcErr{
"file_write_branch": {Code: -32000, Message: "write rejected"},
},
}
skill, _ := newSkill(t, f)
out, err := skill.Handle(context.Background(), "project_create", mirroredArgs())
require.Error(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "infra_commit", res["failed_step"])
reached := res["reached"].([]any)
assert.Equal(t, []any{"create_repo", "create_github_repo", "mirror"}, reached)
require.Len(t, f.Calls, 3)
}
func TestProjectCreate_ValidationErrors(t *testing.T) {
f := &fakeGiteaMCP{}
skill, _ := newSkill(t, f)
cases := []struct {
name string
body string
want string
}{
{"missing name", `{"description":"d","hypothesis":"h","stack":"go-agent"}`, "name"},
{"missing description", `{"name":"x","hypothesis":"h","stack":"go-agent"}`, "description"},
{"missing hypothesis", `{"name":"x","description":"d","stack":"go-agent"}`, "hypothesis"},
{"bad stack", `{"name":"x","description":"d","hypothesis":"h","stack":"python"}`, "stack"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
_, err := skill.Handle(context.Background(), "project_create", json.RawMessage(tc.body))
require.Error(t, err)
assert.True(t, strings.Contains(err.Error(), tc.want), "want %q in %v", tc.want, err)
})
}
assert.Empty(t, f.Calls, "no upstream calls should occur on validation failure")
}
func TestProjectCreate_DefaultSkipsGitHubMirror(t *testing.T) {
// Default (mirror_to_github omitted) skips create_github_repo + mirror
// per the Gitea-as-true-master ADR. Gitea repo + staging namespace
// + issue still run; github_url is empty in the response.
f := &fakeGiteaMCP{
Responses: map[string]any{
"issue_create": map[string]any{"html_url": "http://gitea.d-ma.be/mathias/my-experiment/issues/1"},
},
}
skill, gh := newSkill(t, f)
out, err := skill.Handle(context.Background(), "project_create", happyArgs())
require.NoError(t, err)
var res map[string]any
require.NoError(t, json.Unmarshal(out, &res))
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment", res["gitea_url"])
assert.Equal(t, "", res["github_url"], "github_url must be empty when mirror not opted in")
assert.Equal(t, "http://gitea.d-ma.be/mathias/my-experiment/issues/1", res["issue_url"])
// 3 gitea-mcp calls: template create, staging file write, issue. NO mirror call.
require.Len(t, f.Calls, 3)
assert.Equal(t, "create_project_from_template", f.Calls[0].Tool)
assert.Equal(t, "file_write_branch", f.Calls[1].Tool)
assert.Equal(t, "issue_create", f.Calls[2].Tool)
// Zero GitHub API calls.
assert.Empty(t, gh.Calls, "no GitHub repo created when mirror_to_github is false")
// reached lists the Gitea-only path.
reached := res["reached"].([]any)
assert.Equal(t, []any{"create_repo", "infra_commit", "issue"}, reached)
// experiment-brief body reflects Gitea-only provisioning.
require.Contains(t, f.Calls[2].Args["body"], "Gitea-only")
require.NotContains(t, f.Calls[2].Args["body"], "Push-mirror configured")
}
func TestProjectCreate_UnknownTool(t *testing.T) {
f := &fakeGiteaMCP{}
skill, _ := newSkill(t, f)
_, err := skill.Handle(context.Background(), "nope", happyArgs())
require.Error(t, err)
}
-109
View File
@@ -1,109 +0,0 @@
// Package project implements the `project_create` MCP tool: a single-call
// pipeline that creates a Gitea repo from a template, configures push-mirror
// to GitHub, commits a staging namespace manifest to the infra repo, and
// opens an experiment-brief issue on the new repo. See hyperguild gitea
// issue #10 for the design.
package project
import (
"context"
"encoding/json"
"github.com/mathiasbq/supervisor/internal/githubclient"
"github.com/mathiasbq/supervisor/internal/mcpclient"
"github.com/mathiasbq/supervisor/internal/registry"
)
// Config holds the orchestration dependencies for the project skill.
type Config struct {
// Client talks to the gitea-mcp server. project_create makes
// sequential calls (create_project_from_template, repo_mirror_push,
// file_write_branch, issue_create) through this client.
Client *mcpclient.Client
// GitHub is the client used to create the empty destination repo on
// GitHub before the push-mirror is configured. Gitea's push-mirror
// cannot push to a non-existent remote, so this step is mandatory
// when GitHubPAT is set. Pass nil to skip github repo creation
// entirely (degraded mode — mirror config will land but the actual
// sync to github will fail until the repo exists).
GitHub *githubclient.Client
// GiteaOwner is the org/user that owns the new repo and the infra repo
// the namespace manifest is committed to (typically "mathias").
GiteaOwner string
// GitHubOwner is the GitHub org/user the push-mirror targets
// (typically "mathiasb").
GitHubOwner string
// GitHubPAT is the personal access token used as the push-mirror
// password and to create the destination repo on GitHub. Must have
// `repo` scope. Never logged.
GitHubPAT string
// InfraRepo is the name of the infra repo on Gitea where the
// k3s/staging/<name>/namespace.yaml manifest gets committed
// (typically "infra").
InfraRepo string
}
// Skill exposes project_create as an MCP tool.
type Skill struct{ cfg Config }
// New constructs the project Skill.
func New(cfg Config) *Skill { return &Skill{cfg: cfg} }
// Name returns the skill identifier.
func (s *Skill) Name() string { return "project" }
// Tools returns the MCP tool definitions for this skill.
func (s *Skill) Tools() []registry.ToolDef {
schema, _ := json.Marshal(map[string]any{
"type": "object",
"properties": map[string]any{
"name": map[string]any{
"type": "string",
"pattern": `^[a-z][a-z0-9-]{1,38}[a-z0-9]$`,
"description": "Lowercase repo name. 3-40 chars, must start with a letter.",
},
"description": map[string]any{"type": "string"},
"hypothesis": map[string]any{"type": "string"},
"folder": map[string]any{
"type": "string",
"description": "Informational only — appears in next_steps. Example: AGENTS, AI, QKX.",
},
"stack": map[string]any{
"type": "string",
"enum": []string{"go-agent", "go-web"},
"description": "Selects template-go-agent or template-go-web.",
},
"private": map[string]any{"type": "boolean"},
"mirror_to_github": map[string]any{
"type": "boolean",
"description": "Default false. When true, also create an empty GitHub repo " +
"and configure a push-mirror from Gitea. Opt-in per the Gitea-as-true-master " +
"ADR — only set true for open-source projects (hyperguild, gitea-mcp, template-*). " +
"Never set true for client projects, business logic, or personal experiments.",
},
},
"required": []string{"name", "description", "hypothesis", "stack"},
})
return []registry.ToolDef{
{
Name: "project_create",
Description: "Bootstrap a new project: Gitea repo from template, staging namespace manifest, " +
"experiment-brief issue. Optionally mirrors to GitHub when `mirror_to_github: true` " +
"(default false). Idempotent — re-running with an existing repo returns the existing URLs.",
InputSchema: schema,
},
}
}
// Handle dispatches the tool call.
func (s *Skill) Handle(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
if tool != "project_create" {
return nil, errUnknownTool(tool)
}
return s.handleCreate(ctx, args)
}
-76
View File
@@ -1,76 +0,0 @@
// internal/skills/retrospective/handlers.go
package retrospective
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/mathiasbq/supervisor/internal/session"
)
type retroArgs struct {
SessionID string `json:"session_id"`
Model string `json:"model,omitempty"`
}
// Handle dispatches the retrospective tool call.
func (s *Skill) Handle(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
if tool != "retrospective" {
return nil, fmt.Errorf("unknown retrospective tool: %s", tool)
}
var a retroArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.SessionID == "" {
return nil, fmt.Errorf("session_id is required")
}
model := a.Model
if model == "" {
model = s.cfg.DefaultModel
}
entries, err := session.Read(s.cfg.SessionsDir, a.SessionID)
if err != nil {
return nil, fmt.Errorf("read session log: %w", err)
}
logJSON, err := json.MarshalIndent(entries, "", " ")
if err != nil {
return nil, fmt.Errorf("marshal session log: %w", err)
}
taskPrompt := fmt.Sprintf(
"SESSION_ID: %s\n\nSESSION_LOG:\n%s\n\nReview this session log. Identify what is novel or worth preserving as organizational knowledge. Provide structured insights.",
a.SessionID, string(logJSON),
)
if s.cfg.CompleteFunc == nil {
return nil, fmt.Errorf("no executor configured")
}
t0 := time.Now()
text, dur, err := s.cfg.CompleteFunc(ctx, model, s.cfg.SkillPrompt, taskPrompt)
if err != nil {
return nil, fmt.Errorf("retrospective model: %w", err)
}
msg := text
if len(msg) > 200 {
msg = msg[:200]
}
_ = session.Append(s.cfg.SessionsDir, a.SessionID, session.Entry{
SessionID: a.SessionID,
Timestamp: time.Now(),
Skill: "retrospective",
Phase: "retrospective",
FinalStatus: "ok",
ModelUsed: model,
DurationMs: time.Since(t0).Milliseconds(),
Message: msg,
})
return json.Marshal(map[string]any{"text": text, "model": model, "duration_ms": dur})
}
@@ -1,41 +0,0 @@
// internal/skills/retrospective/handlers_test.go
package retrospective_test
import (
"context"
"encoding/json"
"testing"
"github.com/mathiasbq/supervisor/internal/skills/retrospective"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestHandle_Retrospective_RequiresSessionID(t *testing.T) {
s := retrospective.New(retrospective.Config{})
_, err := s.Handle(context.Background(), "retrospective", json.RawMessage(`{}`))
assert.Error(t, err)
assert.Contains(t, err.Error(), "session_id")
}
func TestHandle_Retrospective_BuildsPromptWithSessionLog(t *testing.T) {
var capturedTask string
s := retrospective.New(retrospective.Config{
SkillPrompt: "retrospective discipline",
DefaultModel: "ollama/test",
SessionsDir: t.TempDir(),
CompleteFunc: func(_ context.Context, _, _, user string) (string, int64, error) {
capturedTask = user
return "Key insight: the team resolved a tricky nil pointer issue via careful logging.", 75, nil
},
})
args, _ := json.Marshal(map[string]string{"session_id": "empty-session"})
out, err := s.Handle(context.Background(), "retrospective", args)
require.NoError(t, err)
var result map[string]any
require.NoError(t, json.Unmarshal(out, &result))
assert.Contains(t, result["text"], "nil pointer")
assert.Contains(t, capturedTask, "empty-session")
}
-49
View File
@@ -1,49 +0,0 @@
// internal/skills/retrospective/skill.go
package retrospective
import (
"context"
"encoding/json"
"github.com/mathiasbq/supervisor/internal/registry"
)
// CompleteFunc is the function used to call a local model.
type CompleteFunc func(ctx context.Context, model, system, user string) (string, int64, error)
// Config holds retrospective skill configuration.
type Config struct {
SkillPrompt string
DefaultModel string
SessionsDir string
CompleteFunc CompleteFunc
}
// Skill implements registry.Skill for the retrospective tool.
type Skill struct {
cfg Config
}
// New constructs a retrospective Skill.
func New(cfg Config) *Skill { return &Skill{cfg: cfg} }
// Name returns the skill name.
func (s *Skill) Name() string { return "retrospective" }
// Tools returns the MCP tool definitions.
func (s *Skill) Tools() []registry.ToolDef {
return []registry.ToolDef{
{
Name: "retrospective",
Description: "Consult a local model to analyse a completed session and identify what is novel or worth preserving as organizational knowledge.",
InputSchema: json.RawMessage(`{
"type": "object",
"required": ["session_id"],
"properties": {
"session_id": {"type": "string"},
"model": {"type": "string"}
}
}`),
},
}
}
-83
View File
@@ -1,83 +0,0 @@
// internal/skills/review/handlers.go
package review
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/mathiasbq/supervisor/internal/brain"
"github.com/mathiasbq/supervisor/internal/session"
)
type reviewArgs struct {
ProjectRoot string `json:"project_root"`
Files []string `json:"files"`
Context string `json:"context"`
Model string `json:"model"`
SessionID string `json:"session_id"`
}
// Handle dispatches the MCP tool call to the appropriate handler.
func (s *Skill) Handle(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
if tool != "review" {
return nil, fmt.Errorf("unknown tool: %s", tool)
}
var a reviewArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.ProjectRoot == "" {
return nil, fmt.Errorf("project_root is required")
}
if len(a.Files) == 0 {
return nil, fmt.Errorf("files is required")
}
model := a.Model
if model == "" {
model = s.cfg.DefaultModel
}
brainCtx, _ := brain.Query(ctx, s.cfg.IngestBaseURL, strings.Join(a.Files, " ")+" "+a.Context, 3)
task := fmt.Sprintf(
"phase: review\nproject_root: %s\nfiles: %s\ncontext: %s\nmodel: %s",
a.ProjectRoot, strings.Join(a.Files, ", "), a.Context, model,
)
task = session.PrependHistory(s.cfg.SessionsDir, a.SessionID, "review", task)
if brainCtx != "" {
task = brainCtx + "\n---\n\n" + task
}
if s.cfg.CompleteFunc == nil {
return nil, fmt.Errorf("no executor configured")
}
t0 := time.Now()
text, dur, err := s.cfg.CompleteFunc(ctx, model, s.cfg.SkillPrompt, task)
if err != nil {
return nil, err
}
if a.SessionID != "" && s.cfg.SessionsDir != "" {
msg := text
if len(msg) > 200 {
msg = msg[:200]
}
_ = session.Append(s.cfg.SessionsDir, a.SessionID, session.Entry{
SessionID: a.SessionID,
Timestamp: time.Now(),
Skill: "review",
Phase: "review",
ProjectRoot: a.ProjectRoot,
FinalStatus: "ok",
ModelUsed: model,
DurationMs: time.Since(t0).Milliseconds(),
Message: msg,
})
}
return json.Marshal(map[string]any{"text": text, "model": model, "duration_ms": dur})
}
-53
View File
@@ -1,53 +0,0 @@
// internal/skills/review/handlers_test.go
package review_test
import (
"context"
"encoding/json"
"testing"
"github.com/mathiasbq/supervisor/internal/skills/review"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestReviewToolRegistered(t *testing.T) {
sk := review.New(review.Config{SkillPrompt: "review rules"})
names := make([]string, 0)
for _, tool := range sk.Tools() {
names = append(names, tool.Name)
}
assert.Contains(t, names, "review")
}
func TestReviewRequiresProjectRoot(t *testing.T) {
sk := review.New(review.Config{SkillPrompt: "r"})
_, err := sk.Handle(context.Background(), "review", json.RawMessage(`{"files":["main.go"]}`))
assert.ErrorContains(t, err, "project_root")
}
func TestReviewRequiresFiles(t *testing.T) {
sk := review.New(review.Config{SkillPrompt: "r"})
_, err := sk.Handle(context.Background(), "review", json.RawMessage(`{"project_root":"/tmp"}`))
assert.ErrorContains(t, err, "files")
}
func TestReviewCallsCompleteFunc(t *testing.T) {
var capturedTask string
fakeFn := func(_ context.Context, _, _, user string) (string, int64, error) {
capturedTask = user
return "2 warnings found: missing error handling at line 42", 80, nil
}
sk := review.New(review.Config{SkillPrompt: "review rules", CompleteFunc: fakeFn, SessionsDir: t.TempDir()})
out, err := sk.Handle(context.Background(), "review", json.RawMessage(
`{"project_root":"/tmp/proj","files":["internal/foo/foo.go"],"context":"PR: add Foo helper"}`,
))
require.NoError(t, err)
assert.Contains(t, capturedTask, "internal/foo/foo.go")
assert.Contains(t, capturedTask, "PR: add Foo helper")
var result map[string]any
require.NoError(t, json.Unmarshal(out, &result))
assert.Contains(t, result["text"], "2 warnings found")
}
-55
View File
@@ -1,55 +0,0 @@
// internal/skills/review/skill.go
package review
import (
"context"
"encoding/json"
"github.com/mathiasbq/supervisor/internal/registry"
)
// CompleteFunc is the function used to call a local model.
type CompleteFunc func(ctx context.Context, model, system, user string) (string, int64, error)
// Config holds dependencies for the review skill.
type Config struct {
SkillPrompt string
DefaultModel string
CompleteFunc CompleteFunc
SessionsDir string
IngestBaseURL string
}
// Skill implements the review MCP tool.
type Skill struct{ cfg Config }
// New creates a new review Skill.
func New(cfg Config) *Skill { return &Skill{cfg: cfg} }
// Name returns the skill identifier.
func (s *Skill) Name() string { return "review" }
// Tools returns the MCP tool definitions for this skill.
func (s *Skill) Tools() []registry.ToolDef {
schema := func(required []string, props map[string]any) json.RawMessage {
b, _ := json.Marshal(map[string]any{"type": "object", "required": required, "properties": props})
return b
}
str := map[string]any{"type": "string"}
return []registry.ToolDef{
{
Name: "review",
Description: "Consult a local model for a structured code review of the specified files. Returns findings with severity levels.",
InputSchema: schema(
[]string{"project_root", "files"},
map[string]any{
"project_root": str,
"files": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"context": str,
"model": str,
"session_id": str,
},
),
},
}
}
-87
View File
@@ -1,87 +0,0 @@
// internal/skills/trainer/handlers.go
package trainer
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/mathiasbq/supervisor/internal/session"
)
type trainArgs struct {
SessionID string `json:"session_id"`
Model string `json:"model"`
}
// Handle dispatches the MCP tool call to the trainer handler.
func (s *Skill) Handle(ctx context.Context, tool string, args json.RawMessage) (json.RawMessage, error) {
if tool != "trainer" {
return nil, fmt.Errorf("unknown tool: %s", tool)
}
var a trainArgs
if err := json.Unmarshal(args, &a); err != nil {
return nil, fmt.Errorf("parse args: %w", err)
}
if a.SessionID == "" {
return nil, fmt.Errorf("session_id is required")
}
if s.cfg.CompleteFunc == nil {
return nil, fmt.Errorf("no executor configured")
}
model := a.Model
if model == "" {
model = s.cfg.DefaultModel
}
entries, err := session.Read(s.cfg.SessionsDir, a.SessionID)
if err != nil {
return nil, fmt.Errorf("read session log: %w", err)
}
// ── Step 1: Reader ────────────────────────────────────────────────────────
history := session.FormatHistory(entries, "")
readerTask := fmt.Sprintf(
"role: reader\nsession_id: %s\nbrain_dir: %s\n\n%s",
a.SessionID, s.cfg.BrainDir, history,
)
readerText, _, err := s.cfg.CompleteFunc(ctx, model, s.cfg.ReaderPrompt, readerTask)
if err != nil {
return nil, fmt.Errorf("reader: %w", err)
}
// ── Step 2: Writer (receives reader output) ───────────────────────────────
t0 := time.Now()
writerTask := fmt.Sprintf(
"role: writer\nsession_id: %s\nbrain_dir: %s\n\nreader_analysis:\n%s",
a.SessionID, s.cfg.BrainDir, readerText,
)
writerText, dur, err := s.cfg.CompleteFunc(ctx, model, s.cfg.WriterPrompt, writerTask)
if err != nil {
return nil, fmt.Errorf("writer: %w", err)
}
msg := writerText
if len(msg) > 200 {
msg = msg[:200]
}
_ = session.Append(s.cfg.SessionsDir, a.SessionID, session.Entry{
SessionID: a.SessionID,
Timestamp: time.Now(),
Skill: "trainer",
Phase: "trainer",
FinalStatus: "ok",
ModelUsed: model,
DurationMs: time.Since(t0).Milliseconds(),
Message: msg,
})
return json.Marshal(map[string]any{
"reader_analysis": readerText,
"writer_output": writerText,
"model": model,
"duration_ms": dur,
})
}
-73
View File
@@ -1,73 +0,0 @@
// internal/skills/trainer/handlers_test.go
package trainer_test
import (
"context"
"encoding/json"
"testing"
"github.com/mathiasbq/supervisor/internal/session"
"github.com/mathiasbq/supervisor/internal/skills/trainer"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestTrainerToolRegistered(t *testing.T) {
sk := trainer.New(trainer.Config{ReaderPrompt: "r", WriterPrompt: "w"})
names := make([]string, 0)
for _, tool := range sk.Tools() {
names = append(names, tool.Name)
}
assert.Contains(t, names, "trainer")
}
func TestTrainerRequiresSessionID(t *testing.T) {
sk := trainer.New(trainer.Config{ReaderPrompt: "r", WriterPrompt: "w"})
_, err := sk.Handle(context.Background(), "trainer", json.RawMessage(`{}`))
assert.ErrorContains(t, err, "session_id")
}
func TestTrainerCallsReaderThenWriter(t *testing.T) {
sessDir := t.TempDir()
require.NoError(t, session.Append(sessDir, "sess-1", session.Entry{
SessionID: "sess-1", Skill: "tdd", Phase: "red", FinalStatus: "ok",
Message: "wrote failing test", FilePath: "internal/foo/foo_test.go",
}))
callCount := 0
var readerTask, writerTask string
fakeFn := func(_ context.Context, _, sys, user string) (string, int64, error) {
callCount++
if callCount == 1 {
// reader call
readerTask = user
return "1 sft candidate found: first-pass clean TDD", 60, nil
}
// writer call
writerTask = user
return "written 1 knowledge entry to brain/knowledge/tdd-patterns.md", 70, nil
}
sk := trainer.New(trainer.Config{
ReaderPrompt: "reader rules",
WriterPrompt: "writer rules",
CompleteFunc: fakeFn,
SessionsDir: sessDir,
BrainDir: t.TempDir(),
})
out, err := sk.Handle(context.Background(), "trainer", json.RawMessage(`{"session_id":"sess-1"}`))
require.NoError(t, err)
assert.Equal(t, 2, callCount, "complete must be called exactly twice: reader then writer")
assert.Contains(t, readerTask, "role: reader")
assert.Contains(t, readerTask, "sess-1")
assert.Contains(t, readerTask, "wrote failing test")
assert.Contains(t, writerTask, "role: writer")
assert.Contains(t, writerTask, "sft candidate")
var result map[string]any
require.NoError(t, json.Unmarshal(out, &result))
assert.Contains(t, result["reader_analysis"], "sft candidate")
assert.Contains(t, result["writer_output"], "knowledge entry")
}
-52
View File
@@ -1,52 +0,0 @@
// internal/skills/trainer/skill.go
package trainer
import (
"context"
"encoding/json"
"github.com/mathiasbq/supervisor/internal/registry"
)
// CompleteFunc is the function used to call a local model.
type CompleteFunc func(ctx context.Context, model, system, user string) (string, int64, error)
// Config holds dependencies for the trainer skill.
type Config struct {
ReaderPrompt string
WriterPrompt string
DefaultModel string
CompleteFunc CompleteFunc
SessionsDir string
BrainDir string // root of brain/ directory
}
// Skill implements the trainer MCP tool.
type Skill struct{ cfg Config }
// New creates a new trainer Skill.
func New(cfg Config) *Skill { return &Skill{cfg: cfg} }
// Name returns the skill identifier.
func (s *Skill) Name() string { return "trainer" }
// Tools returns the MCP tool definitions for this skill.
func (s *Skill) Tools() []registry.ToolDef {
schema := func(required []string, props map[string]any) json.RawMessage {
b, _ := json.Marshal(map[string]any{"type": "object", "required": required, "properties": props})
return b
}
return []registry.ToolDef{
{
Name: "trainer",
Description: "Consult a local model to identify learning moments from a session log and suggest knowledge to preserve in the brain.",
InputSchema: schema(
[]string{"session_id"},
map[string]any{
"session_id": map[string]any{"type": "string"},
"model": map[string]any{"type": "string"},
},
),
},
}
}
+3 -32
View File
@@ -40,10 +40,9 @@ if [ -n "$ROOT_CONTEXT" ] && [ -f "$ROOT_CONTEXT" ]; then
echo " Root context: $ROOT_CONTEXT"
else
# No reachable root AGENT.md — common in CI's clean checkout. The root+project
# adapters (AGENTS.md, .cursorrules, .aider.conventions.md, system-prompt.txt)
# require the root context to regenerate correctly, so we skip them entirely
# and only regenerate CLAUDE.md (which is project-only and inherits root via
# tree walk in Claude Code itself).
# adapters (AGENTS.md, system-prompt.txt) require the root context to
# regenerate correctly, so we skip them entirely and only regenerate CLAUDE.md
# (which is project-only and inherits root via tree walk in Claude Code itself).
echo " No root AGENT.md found — regenerating CLAUDE.md only"
echo "Syncing project context from $PROJECT_FILE..."
cat "$PROJECT_FILE" > CLAUDE.md
@@ -78,30 +77,6 @@ generate_agents() {
echo " → AGENTS.md (root + project; Crush, Pi, Antigravity)"
}
# ── Cursor ───────────────────────────────────────────────────
generate_cursor() {
{
echo "# Cursor rules — auto-generated"
echo "# Do not edit. Run: task context:sync"
echo ""
root_block
cat "$PROJECT_FILE"
} > .cursorrules
echo " → .cursorrules (root + project)"
}
# ── Aider ────────────────────────────────────────────────────
generate_aider() {
{ root_block; cat "$PROJECT_FILE"; } > .aider.conventions.md
if [ ! -f .aider.conf.yml ]; then
cat > .aider.conf.yml << 'YAML'
read: .aider.conventions.md
auto-commits: false
YAML
fi
echo " → .aider.conventions.md (root + project)"
}
# ── Generic system prompt (Open WebUI, Mods, etc.) ──────────
generate_system_prompt() {
{
@@ -142,8 +117,6 @@ echo "Syncing project context from $PROJECT_FILE..."
if [ $# -eq 0 ]; then
generate_claude
generate_agents
generate_cursor
generate_aider
generate_system_prompt
generate_mcp
else
@@ -151,8 +124,6 @@ else
case "$adapter" in
claude) generate_claude ;;
agents) generate_agents ;;
cursor) generate_cursor ;;
aider) generate_aider ;;
prompt|system|openwebui|owui|generic) generate_system_prompt ;;
mcp) generate_mcp ;;
*) echo "Unknown adapter: $adapter" ;;
+79
View File
@@ -0,0 +1,79 @@
---
name: close-session
description: Disciplined end-of-session closeout for a Claude.ai chat before archiving it. Harvests the session's decisions, artifacts, and open threads and durably persists them to the brain MCP and the right Gitea repo so nothing is lost when context resets. Use this whenever the user signals they are wrapping up — phrases like "close this out", "let's wrap up", "before I archive", "session retro", "capture this before I go", "did we lose anything", or any end-of-session/handoff cue — even if they don't say the word "close". Also use when the user explicitly asks to retro, archive, or hand off a working session.
---
# close-session
Capture a finishing Claude.ai work session into durable storage before the chat is archived and its context is lost. The goal is simple and load-bearing: **after this runs, a fresh session (or another agent) can reconstruct what was decided, what was shipped, and what is still open — without the original chat.**
This skill is **batch**: one session in, findings out, done. Run the phases in order. Stop at any confirmation gate that says STOP.
## Operating constraints (read first)
- **Gitea owner is always `mathias`.** Never guess another owner.
- **Ground-truth at HEAD before acting.** Issue bodies and doc references rot — stale hostnames, retired services, moved endpoints. Before closing/commenting on any issue, `gitea:issue_get` it fresh. Before asserting an infra fact, verify it; do not copy it from memory or from a stale issue body.
- **Current infra truths** (verify rather than trust, but these are the known-good baseline): Gitea is `git.d-ma.be` (not `gitea.d-ma.be`). LiteLLM is `http://koala:30401/v1/` (public `https://llm-api.d-ma.be`); piguard runs NGINX Proxy Manager only — never reference `piguard:4000` or `koala:4000`. Identity provider is Authentik (Dex migration complete).
- **Side-effects need a confirmation gate.** Closing issues and capturing to brain/Gitea/ai-sessions are real writes. Surface exactly what will happen and get a clear yes before doing it. Reads are free; writes are gated.
- **Never fabricate.** If the session didn't produce a decision worth persisting, say so and skip that write. An empty-but-honest closeout beats an invented one.
## Phase 1 — Harvest
Reconstruct what actually happened this session from the conversation itself. Produce, in working memory:
- **Decisions taken** — what was decided and the reasoning, not just the outcome.
- **Artifacts produced** — issues filed/closed, PRs opened/merged, files committed, brain notes written, ADRs. Capture identifiers (issue numbers, PR numbers, paths, commit SHAs) as you go.
- **Open threads** — what was deferred, what's blocked, what the next session should pick up.
- **Generalizable learnings** — reusable patterns or footguns that would bite anyone again (these are brain-worthy; project status is not).
Be honest about fidelity: a long session compresses harder at the start than the end. Flag anything you're reconstructing rather than certain of.
## Phase 2 — Ground-truth Gitea state
For every repo touched this session, get its true current state before proposing any change. `gitea:repo_status` (owner `mathias`) gives branches + open PRs + protection in one call. For each issue you intend to close, comment on, or reference: `gitea:issue_get` it fresh and compare to what the session assumed. Note any drift (closed-already, body rotted, renamed) — you'll surface it in Phase 3.
Do not write anything in this phase. This is the read pass.
## Phase 3 — Plan the issue changes
Decide the issue actions: which to close (with closing comment), which to file (discovered-but-deferred work — token-budget gaps, recorded limitations, v2 follow-ups), which to comment on. Include the exact title/body for any new issue and the closing rationale for any close.
These actions are **carried into the Phase 4 capture call** as `tickets[]` rather than executed here with direct `gitea:issue_*` calls — routing them through capture puts each one into the I5 audit record. (Closing an issue that needs a separate explanatory comment first is the one case to do directly; otherwise prefer the capture path.)
## Phase 4 — Capture (one uniform call)
Persist the session via a **single `capture` call** (the `brain:capture` MCP tool, live on the Claude.ai connector). Capture owns the writes server-side — insights → brain, action items → Gitea tickets, summary → ai-sessions — plus the I1 sovereignty gate, the I5 audit record, and the supersession/read-after-write discipline. The skill's job is to *assemble the payload*, not to write each store itself. Do NOT fall back to separate `gitea:file_write_branch` + `brain_write` steps unless `capture` is unreachable (see fallback below).
**Assemble one payload:**
- **`insights[]`** — the generalizable learnings from Phase 1 (decisions/failures worth re-reading). Each: `{text, wing, hall}`; add `supersede_slug` to revise a prior note in place instead of creating a duplicate. `hall` ∈ facts/decisions/failures/hypotheses/sources.
- **`tickets[]`** — the issue actions from Phase 3: `{repo, action, ...}` where action ∈ create/close/comment. Owner is always `mathias` (server-forced).
- **`summary`** — `{title, body, repos_touched}`. Capture writes it to `ai-sessions` and stamps `fidelity` in frontmatter. Body stays reconstructable: one-paragraph summary, decisions, key artifacts, open threads.
- **`context`** — `{harness: "claudeai-chat", session_ref: <chatid8-or-slug>, fidelity: "live-capture", actor: "mathias", classification: <see gate below>}`.
**THE CLASSIFICATION GATE (read before calling — this is where capture refuses).**
Capture computes an **effective classification = the strictest across EVERY target it touches** (each insight's `wing`, each ticket's `repo`, and every entry in `summary.repos_touched`), then refuses if that effective level is `confidential` and the origin is us-nexus (claude.ai is us-nexus). Levels come from `classification.yaml` at the brain root (source of truth, #67), with the code defaults as the floor: `hyperguild`/`homelab` → internal; `client-*` → confidential; **anything untagged → confidential (fail-safe)**.
- **Tagged `internal` today** (safe through claude.ai): wings `hyperguild`, `homelab`; repos `brain`, `ai-sessions`, `infra`, `hyperguild`, `homelab`, `tapir`, `agentsquad`, `jepa-fx-risk`, `swedsl`. Treat `classification.yaml` as authoritative — this list is a hint, not gospel.
- Declare `context.classification: "internal"` for normal homelab work.
- `summary.repos_touched`, insight `wing`s, and ticket `repo`s are classification INPUTS, not free-form metadata — every target must resolve `internal` or the whole capture escalates to `confidential` and the gate refuses via claude.ai. Listing the central homelab repos (incl. `brain`/`ai-sessions`) is now fine; they're tagged. The summary always lands in `ai-sessions` (internal), so the summary path itself never escalates.
- If a session genuinely touched **`client-*` or otherwise-untagged** material, it cannot be captured through claude.ai — note that in the verdict rather than trying to force it.
**GATE — dry-run first, then execute.**
1. Call `capture` with `dry_run: true`. It validates the whole payload and returns the would-be receipt + `effective_classification`, writing nothing.
2. **STOP. Show the dry-run receipt** (effective classification, the insights/tickets/summary that would land) and get explicit confirmation.
3. On confirmation, call `capture` again with `dry_run: false`. Read the returned receipt: it is partial-aware (`errors[]`, per-item `ok`). Report exactly what landed.
If `capture` is **unreachable** (tool not on the connector — e.g. a session that started before a deploy; a tool-list refresh usually fixes it): say so. Only then fall back to the legacy inline path (`gitea:file_write_branch` summary + `brain_write`/`brain_update` + `brain_get` confirm), and note in the verdict that the I5 audit record was NOT produced.
## Phase 5 — Verdict
Deliver a final "safe to archive" verdict in the chat. Either:
- **SAFE TO ARCHIVE** — list what landed from the capture receipt (issues closed/filed with numbers, summary path, brain note ids/paths) so the trail is auditable. Then list anything still in the user's queue (e.g. a PR awaiting their merge, a decision owed next session).
- **NOT YET** — name the specific gate that wasn't passed, the capture refusal reason, or the per-item error from the receipt, and what to do about it.
Never claim safe-to-archive if the capture refused, any receipt item errored, or a gated confirmation was declined. The verdict is the skill's contract: if it says safe, the session can be lost without losing the work.
## Why the gate and the single-call shape matter
The whole point is durability across a context reset. The capture call is the one place a wrong payload would silently corrupt the record (close the wrong issue, escalate to a refusal, commit a half-truth), which is why it is dry-run-then-confirm. Routing everything through one `capture` keeps the supersession discipline, the read-after-write confirmation, and the I5 audit trail server-side — the skill never has to carry those rules itself, and every closeout is uniformly audited. Get the payload and the classification right and the skill does what it promises — nothing important is lost when the chat goes away.

Some files were not shown because too many files have changed in this diff Show More