Only /mcp was wrapped in BearerMiddleware; /query, /write, /ingest*,
/promote, /pending and the rest on the same mux had no auth, and
infra#517 found them on the internet (mitigated there by narrowing the
ingress). They are still open on the tailnet and LAN NodePort 30330.
mountREST puts the REST routes on their own mux, mounted at "/" behind
the same bearer middleware and credential as /mcp. Routes registered on
the outer mux stay explicitly public (/mcp has its own auth, /capture,
the HMAC webhook, OAuth metadata, /metrics); anything else, including a
route added later, is authenticated by default.
The hyperguild CLI (cmd/hyperguild and internal/brain) now sends
BRAIN_MCP_TOKEN, which it never did. brain-ui and brain-ingest-ntfy
already send theirs.
Refs #149
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(server): say at startup when alerting is unconfigured (#107)
WARN once at startup when BRAIN_NTFY_URL is unset, naming what will not fire (brain_edit_root governance rewrites, capture audit degradation). Rebase of PR #108 (never CI-tested) onto current main, no functional change to alertingWarnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
CLAUDE_INGEST_SKIP_PROJECTS takes filepath.Match globs tested against
the project directory NAME, and a matching directory is skipped during
the directory WALK — so its files are never opened and their content
never enters the process. That is a stronger property than filtering
after reading, and the difference is observable: without SkipDir,
WalkDir lists the directory's entries, and an unreadable one aborts the
entire walk, silently stopping ingestion for every other project.
Globs rather than exact names because a project directory is derived
from the session's working directory, so a git worktree yields an
ephemeral name no exact-match list could keep up with.
An unparseable pattern refuses to run. filepath.Match returns an error
ALONGSIDE a false match, so a bad pattern checked per-file would ingest
the directory while the log complained — the failure nobody notices.
Validated once, up front, in both entry points.
This complements the client-name scrubber rather than replacing it. The
scrubber drops individual turns whose content matches a client
identity; this excludes whole workspaces whose content is confidential
in ways no pattern reliably recognises. Learnings from excluded
sessions are captured deliberately through the session-close ritual
instead.
Recorded caveat: the project directory reflects the session's working
directory, NOT the repositories it touches. A session run from ~/dev
that reads an excluded repo is not excluded by this. The scrubber
remains the control for content; this is a control for workspaces.
A mutation test earned the strongest assertion here: the original tests
could not distinguish SkipDir from filtering after the walk, because
neither opens files. The unreadable-directory test can.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
llm.Router was primary-then-fallback (berget -> iguana ollama). Both
have failed together at least once in production (berget 5xx outage +
a cold fallback model exceeding the shared timeout budget), leaving
brain_answer/brain_classify fully down.
Router gains an optional Tertiary *Client, tried only if both Primary
and Fallback fail. Nil-safe — existing 2-tier deployments (no
BRAIN_LLM_TERTIARY_URL) behave exactly as before.
main.go wires it from BRAIN_LLM_TERTIARY_URL/MODEL + GEMINI_API_KEY,
pointed at Google's OpenAI-compatible Gemini endpoint
(generativelanguage.googleapis.com/v1beta/openai/) — llm.Client already
speaks OpenAI-shaped chat-completions, so no new client type needed.
Tests: TestRouter_FallsBackToTertiaryWhenPrimaryAndFallbackFail,
TestRouter_TertiaryNotCalledWhenFallbackSucceeds, TestRouter_AllThreeFail,
TestRouter_NilTertiary (2-tier backward-compat). Full suite green
(both modules, -race), golangci-lint clean.
Deployment env vars (infra repo, separate commit — GitOps, TBD there
too): BRAIN_LLM_TERTIARY_URL, BRAIN_LLM_TERTIARY_MODEL, GEMINI_API_KEY
secretRef.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
brain_edit_root was gated but unobserved: on success it emitted a slog.Warn
and nothing else. That proves a rewrite was allowed, not that one happened —
and brain-sync commits it within 15 minutes authored `brain-sync`, so git
history cannot tell a CLAUDE.md rewrite apart from routine note traffic.
A successful brain_edit_root now emits an ntfy notification and an AuditEntry
through the same sink a capture already uses. notes.Result gains
PrevContentHash so both ends of the change are identifiable.
Path and hashes only in the notification body. Reserved at Internal, not
Confidential, because Confidential makes DegradingSink refuse to record while
loki is down rather than buffer. Observation is best-effort and never fails
the call; a failure to observe logs at ERROR.
Ordinary brain_edit on knowledge//wiki/ is deliberately not audited — git
history already carries the same before/after bytes with a diff.
Closes#97.
Co-authored-by: Mathias <mthbqv@gmail.com>
On a cluster restart this pod can beat dex to serving, gets a 502 from
ingress, and exits — after connecting postgres, backfilling 794 graph
nodes and ingesting three session batches, all of which the restart
redoes. Nothing was misconfigured; it started first.
The validator now retries with bounded exponential backoff (8 attempts,
1s doubling to 30s) and is built immediately after ctx, before the
watcher and the backfill. That second part is the other half of the
issue: a genuine config error now fails in seconds rather than after all
that work.
Bounded on purpose. Retrying forever would convert a wrong issuer URL
into a process that looks alive and never serves, which is harder to
diagnose than a crash loop.
startupretry.Do is generic and tested directly: transient-then-success,
budget exhaustion surfacing the last error via errors.Is, prompt return
on context cancellation, growing-and-capped delays, and a zero-value
policy still making one attempt rather than silently succeeding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeAp5LTscnz2W7Ubt4eJ6m
Hand-authored and promoted notes skip pipeline.BuildPages, so nothing
quotes their scalars. A title carrying an unescaped ": " yields a block
yaml.Unmarshal rejects, and every consumer that reads frontmatter drops
the note silently — 18 files repo-wide today, up from the 8 in #90.
frontmatter.Lint reports it; frontmatter.Fix quotes only what quoting can
rescue and refuses to return a document that still fails to parse, so an
unfixable file comes back byte-identical rather than half-repaired. Scan
walks a tree and reports an unreadable file as a finding instead of
aborting, so one poisoned note cannot hide the rest.
Sequence items are deliberately left alone: "- foo: bar" is valid YAML
that silently becomes a map, which is a type corruption rather than a
parse failure, and repairing it would rewrite notes that currently load.
task lint:frontmatter # report
task lint:frontmatter FIX=1 # repair in place
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BeAp5LTscnz2W7Ubt4eJ6m
ai-sessions#13: capture's `summary` field wrote a frontmatter shape
(title/harness/fidelity/captured_at/repos_touched) that ai-sessions'
own extract/audit pipeline can't parse -- silently invisible to that
repo's own audit tooling, and bypassing its redaction + Stage2
completeness gates by construction. Only 5 files ever landed this way
over 3 weeks; the summary capability's whole value (speed) fights
ai-sessions' whole value (redacted, audited, complete), so kill it
rather than build a second parse branch.
capture now persists insights -> brain and action items -> Gitea
tickets only. Removes Summary/SummaryResult/SummaryWriter and all
wiring (service, REST body, MCP tool schema); close-session updated to
stop assembling a summary payload. specs/capture-*.md kept as historical
record with a superseded note -- the feature shipped and is documented,
just no longer current behaviour.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPdSHbp9Utb2hPFm9wDG59
Adds POST /webhooks/brain-sync: verifies Gitea's HMAC-SHA256 signature,
checks the push is to mathias/brain main, then creates a one-off Job from
the existing brain-sync CronJob's template (same script the 15-min poll
already runs, just triggered on-demand). Off by default -- opt in via
GITEA_WEBHOOK_SECRET, since it needs Job-create RBAC in the "brain"
namespace a fresh deploy won't have.
10 new tests (internal/webhook), including a fake-clientset reactor to
simulate server-side GenerateName expansion, which the plain fake tracker
doesn't do on its own.
Needs (follow-up, infra repo): RBAC granting ingestion's ServiceAccount
get on cronjobs/brain-sync + create on jobs in the brain namespace, the
GITEA_WEBHOOK_SECRET env, and the actual Gitea webhook registration.
claudeSink wrote every raw transcript into brain/wiki/claude-sessions/facts/
(BM25-indexed), duplicating what ai-sessions' curated summaries already
cover and out-ranking them in search (177,818 vs 45,335 BM25 weight on the
same query, per ai-sessions#10 investigation). Raw dumps now land in
brain/archive/claude-sessions/<host>/ — kept for deep lookups, never
indexed, never deleted.
Refs: ai-sessions#10
brain-gardener#5: the audit's unsourced check was flagging 59 raw
claudewatcher session dumps as high-severity hallucination risk,
because it can't distinguish a first-party observation (a session
transcript) from an external claim needing citation.
Adds WriteNoteOptions.SourceType, emitted as source_type: <value> in
the wing/hall frontmatter route. claudeSink (the claudewatcher sink)
now always sets source_type: internal on the notes it writes.
Unblocks CD: the build's go mod download failed because mcp-chassis was
imported via the old gitea.d-ma.be module path, which the renamed server no
longer serves a matching go-import meta tag for. Point at the renamed module
git.d-ma.be/mathias/mcp-chassis v0.2.0 (mcp-chassis 69d389d) + go mod tidy.
Fresh download now resolves; ingestion builds + tests green.
This restores the normal CI path so the routing pass-rate fix (da9bdc4,
#73) builds and deploys through gitea Actions. Refs #74, #73, #35.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires the curation primitives into the MCP surface (all three sites:
tools() descriptors, handleCall dispatch, package doc) and registers the
GET /pending + POST /promote REST routes. brain_promote additionally
re-indexes the promoted note into the graph (best-effort), matching the
other write paths. brain_pending is the human-review-queue complement to
the agent-facing brain_write.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deployed CaptureService was constructed with a nil SummaryWriter, so
a capture carrying a summary block returned the partial-failure
"no summary writer configured" (surfaced in the 2026-06-23 claude.ai
dogfood). The Gitea client already reaches mathias/* over BRAIN_GITEA_TOKEN
and now implements SummaryWriter, so inject it (type-asserted from the
tracker) — no new credential, no manifest change. Session summaries now
write to mathias/ai-sessions at summaries/<harness>/<YYYY-MM>/...
Reuses the existing token deliberately; assumes it carries contents:write
scope on ai-sessions (it already does issue writes for the tracker). If
the token is issue-scoped only, the live write 422s — a token-scope widen,
not a code fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds the `capture` MCP tool: the #55 relay for harnesses that cannot run
the use-case in-process (claude.ai Chat/Cowork/Design, Crush, Pi, LLM
Council). They reach it through the existing /mcp OAuth connector.
- Thin: forwards to the SAME CaptureService as POST /capture; holds no
state and retains nothing beyond the I5 audit record. The containment
properties accepted in infra security-baseline (I2 ledger) hold by
construction.
- Per-principal: ServeHTTP re-derives the caller's principal from the
Bearer header (the chassis middleware gates but discards it) and stashes
it in context; the tool resolves the trust-zone origin from it. A
caller-asserted harness/origin in the body is ignored — origin is
server-derived, so the I1 confidential refusal still fires for us-nexus
callers (claude.ai), and sovereign-allowlisted JWT principals pass.
- Registered only when WithCapture is wired (all three sites: tools(),
handleCall, package doc); main wires REST + MCP from the same service,
resolver, and credentials.
Tests: listed-only-when-wired, forwards-via-static-principal,
confidential-via-us-nexus-refused, confidential-via-sovereign-allowed,
unauthenticated-rejected, caller-cannot-forge-origin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Map capture.ErrAuditUnavailable → HTTP 503 (audit substrate down /
confidential unauditable / floor).
- main: buildAuditSink selects the DegradingSink (loki central + durable
file buffer under brainDir + optional ntfy) when BRAIN_LOKI_URL is set
and starts the reconcile loop; else the plain slog sink. Notifier kept
as a nil interface (not typed-nil) when unconfigured so the sink and
reconcile skip it cleanly.
Env: BRAIN_LOKI_URL, BRAIN_NTFY_URL, BRAIN_NTFY_TOKEN,
BRAIN_AUDIT_RECONCILE_INTERVAL (default 60s). Buffer at
<brain>/.audit-buffer/capture.jsonl.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The HTTP door for the capture capability. Thin: authenticate → derive
trust-zone origin → decode → capture.Service → map receipt to status.
- Auth mirrors the chassis Bearer precedence (static token wins, then Dex
JWT) but returns the resolved principal + auth path, which the chassis
middleware hides — capture needs the principal to derive the origin.
Depends on a small Validator interface (the chassis *JWTValidator
satisfies it) so the JWT/origin path is testable without a live JWKS.
- OriginResolver maps principal → trust zone: static-token caller and
allowlisted JWT subjects → sovereign; every other principal → us-nexus
(fail safe, so the I1 gate refuses confidential by default). Principal
and origin are server-set on the input, overwriting any body the caller
sent.
- HTTP status: 200 all-ok / dry-run, 207 partial, 502 all-failed, 403 on
the I1 refusal, 400 on fail-closed validation.
- Wired in main behind the same static+JWT credentials as /mcp, reusing
the MCP server's graph-wired brain store (one implementation) and the
classification tags (#50). Mounts only when a Gitea tracker is
configured. Sovereign JWT principals via BRAIN_CAPTURE_SOVEREIGN_PRINCIPALS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements the IssueTracker port as a real Gitea REST client (#49c) — the
new outbound dependency the brain server gains for capture.
- CreateIssue / CommentIssue / CloseIssue(+optional closing comment) over
the Gitea API. Owner is the const "mathias", never caller-supplied, so
a caller cannot redirect a write to another owner's repo.
- Token read once at construction (BRAIN_GITEA_TOKEN), held in the struct,
travels only in the Authorization header — never logged or in argv.
Error messages carry status + truncated body, never the token
(regression-tested). gitea.New returns nil when URL or token is unset,
so missing config = tracker disabled via one nil check.
- Injected into the MCP server behind the capture.IssueTracker interface
via WithIssueTracker (constructor injection, swappable/testable); main
wires it from BRAIN_GITEA_URL (default https://git.d-ma.be) +
BRAIN_GITEA_TOKEN. Consumed by the capture use-case in #53.
Tests use httptest transports: create (owner+auth header asserted),
comment, close with/without comment, error path that proves the token
never leaks into an error string.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-rollout guard. Source code stays clean — client identities come
from CLAUDE_INGEST_CLIENT_BLOCK env (sourced from a SOPS-encrypted k8s
secret in infra repo). Env value is a regex alternation; main wraps
it with `(?i)\b(...)\b` so word-boundary matching avoids false hits
inside longer identifiers (e.g. "Sebastian" doesn't trigger on "SEB").
DefaultRules (credential shapes) still take precedence so any leak
that's BOTH a client mention AND a credential shape logs as the
credential — strictly more dangerous, points triage at the right
thing. Tests cover precedence + case variations + word-boundary
respect + invalid-pattern rejection.
Refs: infra#73 Track E.1 pre-rollout grill (option B).
Bump-Type: minor
Opt-in by setting CLAUDE_SESSIONS_DIR to the ~/.claude/projects path.
When set, the server starts claudewatcher.Watch in a goroutine that
ticks every CLAUDE_INGEST_INTERVAL seconds (default 60). Requires
BRAIN_PG_DSN for the cursor table — fail-fast if missing.
Each Batch becomes one wiki note at:
brain/wiki/claude-sessions/facts/session-<host>-<session_id>.md
with frontmatter type=source + domain=<project basename>. Per-turn
content capped at 2000 chars (full transcripts stay in
~/.claude/projects already); the brain entry is a digest, not a
mirror.
CLAUDE_INGEST_HOST overrides the os.Hostname()-derived host label,
useful when multiple ingestion pods consume the same DSN from
different machines.
Closes hyperguild#27.
Bump-Type: minor
Commit 2 of Track A. Service stays a no-op until BRAIN_GRAPH_ENABLED=
true; flipping it on creates the schema (idempotent), starts indexing
every successful write, and optionally backfills the existing brain
dir.
- internal/graphsync: best-effort wrapper around graph.Extract +
graphstore. IndexDoc reads docPath under brainDir, parses, upserts
entity + replaces edges. BackfillFromBrainDir walks wiki/ +
knowledge/. Both are no-ops on nil store so callers wire
unconditionally.
- mcp.Server gains WithGraph builder + graphsync.Store field.
brain_write, brain_ingest, brain_ingest_raw, brain_tunnel call
indexInGraph after success — failures slog.Warn but never
propagate (graph is augmentation, not correctness).
- cmd/server gates the wiring on BRAIN_GRAPH_ENABLED=true (default
off so first rollout doesn't surprise). BRAIN_GRAPH_BACKFILL=true
triggers a one-shot walk of the brain dir on boot.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Second port of the MCP chassis (gitea-mcp was first, commit 658f4ba).
Closes the chassis-adoption loop on the two highest-LOC consumers.
Changes:
- Drop ingestion/internal/auth/ entirely (jwt.go + jwt_test.go +
protected_resource.go + protected_resource_test.go) — chassis provides
JWTValidator + ProtectedResourceHandler with identical semantics.
- Drop ingestion/internal/mcp/auth.go (BearerAuth function, ~65 LOC)
and the integration test auth_test.go (~200 LOC) — chassis
BearerMiddleware replaces it. Static-Bearer-or-Dex-JWT precedence and
RFC 9728 resource_metadata challenge behavior preserved 1:1.
- cmd/server/main.go: import chassis as `chassisauth`, rewire the three
call sites. Use realm="brain" in the BearerMiddleware call so a 401
challenge identifies the resource as the brain MCP.
OAuth client_credentials handler (ingestion/internal/oauth) stays —
chassis v0.1.0 covers only the JWT path; OAuth flow is a candidate for
chassis v0.2.0 once a second MCP needs it (rule of three).
Net delta: -~330 LOC of duplicated auth code; +1 import; +1 GOPRIVATE
env requirement on dev machines (documented in the spike handoff
2026-05-22-mcp-chassis-spike.md).
task check green (lint + test + vet + govulncheck).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes infra#50.
Adds an internal/metrics package with a hand-rolled Prometheus
exposition layer (stdlib + sync/atomic only — no new dep) and wraps the
HTTP mux with a timing middleware. Every request emits one observation
on the `brain_query_duration_seconds` histogram labeled by
`path` (request Pattern, low cardinality) and `status` (2xx/3xx/4xx/5xx).
Dependency choice: hand-rolled rather than github.com/prometheus/client_golang
because the surface needed is small (one histogram + bucket constants)
and the repo CLAUDE.md keeps deps stdlib + jwx + testify only. ~150 LOC
of code + tests is cheaper than the chart of transitive prometheus deps.
Endpoints:
- GET /metrics — OpenMetrics text exposition, no auth (cluster-internal)
Wire format pinned by tests in internal/metrics/metrics_test.go. The
ServiceMonitor that drives the kube-prometheus-stack scrape lives in
infra/k3s/apps/supervisor/ (separate commit on mathias/infra).
After this image deploys, the canary alert from
docs/superpowers/specs/2026-05-homelab-architecture-review.md becomes
wireable:
histogram_quantile(0.95,
sum(rate(brain_query_duration_seconds_bucket[5m])) by (le))
> 1.5 * histogram_quantile(0.95,
sum(rate(brain_query_duration_seconds_bucket[5m] offset 7d)) by (le))
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous "crude redaction" — pgDSN[:strings.IndexByte(pgDSN+"@", '@')] —
sliced up to the `@` character, which sits *after* the password in a
postgres URL, so the log line included the password in plaintext (caught
on first activation, 2026-05-18 startup log).
Use url.Parse + URL.Redacted() instead. Falls back to "postgres://***"
if parsing fails — we never log a raw DSN.
Wires nomic-embed-text (iguana ollama) + pgvector on the shared
postgres18 into brain_query / brain_answer via Reciprocal Rank Fusion.
Pure BM25 stays the default; setting BRAIN_PG_DSN and BRAIN_EMBED_URL
together opts in. Setting one without the other is misconfiguration →
exit 1.
New packages:
- internal/embed
Client.Embed(ctx, text) → []float32 via POST {URL}/api/embed.
Defaults to nomic-embed-text:latest (768 dim). nil-on-empty-URL so
callers gate on a single nil check.
- internal/vectorstore
PGStore wraps a pgxpool against postgres18. Init creates
brain_embeddings(path PK, vector(768), updated_at) + HNSW cosine
index idempotently. Upsert / Delete / Search / KnownPaths.
Sync(brainDir, store, embedder) diffs brain/wiki/ against the store
and upserts new files / deletes removed ones; StartSync runs it on
a ticker (default 300s). Integration tests gated by BRAIN_PG_TEST_DSN.
- scripts/brain-embeddings-init.sql
One-time DBA setup: brain DB, brain_app role, vector extension,
GRANTs. Idempotent.
Search layer:
- search.QueryOptions gains Vector + Embedder fields.
- QueryContext is the cancellable variant; Query stays for callers.
- When both are set, BM25 (top-N) and pgvector (top-4N) candidates
merge via Reciprocal Rank Fusion (k=60, Cormack et al. 2009 — no
tuning knob, robust to scale differences between rankers).
- Vector-only hits are hydrated from disk so callers see uniform
Result records (path, title, excerpt, wing, hall, score).
- Wing/hall filters still apply to vector candidates via path-prefix.
- On embedder/vector errors the search falls back to BM25 — embedding
outage degrades quality but doesn't take the brain offline.
MCP wiring:
- mcp.Server.WithHybridRetrieval(v, e) opt-in setter, same shape as
WithReranker.
- brainQuery and brainAnswer pass the wired vector/embedder through
to search.QueryContext.
REST:
- POST /backfill-embeddings drives Sync synchronously. Returns
{added, deleted, errors[]}. 503 when feature is unconfigured.
cmd/server/main.go:
- BRAIN_PG_DSN + BRAIN_EMBED_URL together enable hybrid; one alone
→ exit 1.
- vectorAdapter bridges *PGStore (returns []Hit) to
search.VectorSearcher (which takes []VectorHit) without either
package importing the other.
- BRAIN_EMBED_SYNC_INTERVAL (default 300s) controls the background
Sync ticker.
Backend pivot from Qdrant to pgvector recorded in DECISIONS.md
2026-05-18 (supersedes 2026-04-08): postgres18 already runs in
databases/ ns, Qdrant was never deployed, one engine beats two.
Dependency: github.com/jackc/pgx/v5 — modern, native pgvector via
parametric vector literals.
Tests:
- embed.Client: empty-URL nil, request shape, dimension, upstream
error propagation, empty-text rejection.
- vectorstore.PGStore: dimension validation (unit); upsert/search/
KnownPaths (integration, BRAIN_PG_TEST_DSN-gated).
- vectorstore.Sync: adds new files, skips known, deletes
disappeared, skips _index.md, no-op when nil, collects embedder
errors.
- search.Query: hybrid promotes vector-only hits via RRF; falls
back to BM25 on embedder error.
Closes hyperguild#8.
Adds an opt-in cross-encoder rerank step between BM25 retrieval and LLM
synthesis. With BRAIN_RERANKER_URL set, brain_answer retrieves BM25
top-20, scores each excerpt against the query via Qwen3-Reranker on
Ollama, drops the "no" answers, and forwards up to 5 surviving sources
to the LLM. Unset, behaviour is unchanged (BM25 top-10 → LLM).
The reranker is a *filter*, not a re-ranker: Qwen3-Reranker emits a
binary yes/no token under its native chat template, and ties within the
"yes" set are broken by BM25 rank — what got retrieved first stays
ahead.
New package ingestion/internal/reranker:
- Client with URL, Model, HTTP fields.
- New(url, model) returns nil on empty url so callers can treat
"feature disabled" as a single nil check.
- Score(ctx, query, docs) issues one /api/generate call per doc using
the Qwen3-Reranker yes/no chat template (verbatim, because the model
was trained on this exact wording). Parses the first non-think token.
Wiring:
- mcp.Server gains a WithReranker fluent setter to keep NewServer
signature stable.
- brain_answer's BM25 limit jumps to 20 only when a reranker is wired,
to give the filter something to do.
- cmd/server/main.go reads BRAIN_RERANKER_URL (+ optional
BRAIN_RERANKER_MODEL, default dengcao/Qwen3-Reranker-0.6B:F16).
Tests cover: nil-on-empty-url, ordered yes/no scoring, request shape
(model, prompt contents, yes/no template), ambiguous response → 0,
empty doc slice, upstream-error propagation, plus an end-to-end
brain_answer integration that proves only the relevant note reaches the
LLM when noise.md is rejected.
Closes hyperguild#7.
Adds a minimal RFC 8414 + RFC 6749 client_credentials flow so claude.ai's
custom-MCP integration (no static-Bearer field in the UI) can exchange a
client_id + client_secret pair for the existing BRAIN_MCP_TOKEN and use
it as a Bearer on /mcp. No JWTs, no refresh, no expiry — the rest of
the auth middleware is unchanged.
New package ingestion/internal/oauth:
- MetadataHandler(issuer): serves /.well-known/oauth-authorization-server
with grant_types=[client_credentials] and both
token_endpoint_auth_methods (post + basic).
- TokenHandler(cfg): serves /oauth/token. Validates client_id and
client_secret via constant-time compare; returns BRAIN_MCP_TOKEN as
access_token. RFC 6749 §5.2 error JSON on bad grant / bad creds.
Wiring in cmd/server/main.go: opt-in by setting both OAUTH_CLIENT_ID and
OAUTH_CLIENT_SECRET. Setting only one is misconfiguration → exit 1.
Mounts both endpoints with no auth; MCP_RESOURCE_URL supplies the
issuer.
Also pivots issue #8's vector backend from Qdrant to pgvector (see
DECISIONS.md 2026-05-18) — Qdrant was never deployed and postgres18 with
pgvector already runs as the project default; supersedes 2026-04-08 for
this use case.
Tests cover post-auth, basic-auth, wrong secret, bad grant, GET
rejection, malformed Basic header, and Basic without colon.
Closes hyperguild#5.
Reorders BearerAuth so a valid BRAIN_MCP_TOKEN match wins instantly and
never emits WWW-Authenticate. Adds RFC 9728 resource_metadata challenge
header on 401 (only when MCP_RESOURCE_URL is configured) so claude.ai's
OAuth-discovery path still works.
Why: claude CLI on koala/flamingo with `.mcp.json` `Authorization: Bearer
$BRAIN_MCP_TOKEN` was being kicked into RFC 7591 dynamic client
registration against Dex (static-only) and dying. Cause was the auth
middleware running JWT validation first and emitting an OAuth challenge
on the fall-through 401 even when the caller had a valid static token.
Inverting the precedence and gating the challenge on resourceMetadataURL
keeps the LAN/Tailscale CLI path silent and only invites OAuth discovery
on actually-unauthenticated requests.
Regression guards in the test file:
- valid static Bearer 200 has no WWW-Authenticate
- 401 with resourceMetadataURL set carries the challenge
- 401 with empty resourceMetadataURL emits no challenge
Closes hyperguild#9 in code. Live verification (claude CLI on koala
listing brain tools) blocked on ingestion image rebuild + redeploy.
Adds a two-dimensional address (wing, hall) to brain notes. A wing is a
topic domain (e.g. jepa-fx, hyperguild); a hall is one of a closed
vocabulary of memory types (facts, decisions, failures, hypotheses,
sources). Notes route to brain/wiki/<wing>/<hall>/<slug>.md with
wing/hall/created_at YAML frontmatter, making the directory a valid
Obsidian vault.
Changes:
- new package ingestion/internal/brain (NotePath, ValidHalls, Sanitise,
BuildWingIndex, BuildAllWingIndexes)
- api.WriteNote refactored to WriteNoteOptions; wing+hall routes to
brain/wiki/, otherwise falls back to brain/knowledge/ (legacy)
- search.Query → QueryOptions with optional Wing/Hall filtering; Result
carries wing/hall extracted from frontmatter or path segments
- MCP tools brain_write and brain_query gain optional wing/hall params
(hall enum-validated); new brain_index tool regenerates _index.md MOC
- POST /index REST endpoint mirrors brain_index
- brain_write auto-rebuilds the wing's _index.md after a wing+hall write
- scripts/migrate-brain-halls.sh migrates flat brain/wiki/{concepts,entities}/
into the new layout (dry-run by default, --commit applies)
All existing tests pass; new tests cover wing/hall write routing, scope
filtering, invalid hall rejection, _index.md generation, and migration
script paths.
Closes hyperguild#1.
Adds two new LLM-backed MCP tools to the ingestion service:
- brain_answer(query): BM25 retrieval + LLM synthesis → answer + sources
- brain_classify(text): classifies doc into type/title/tags via LLM
Adds llm.Router for primary→fallback routing (berget.ai → iguana).
Wired via BRAIN_LLM_PRIMARY_URL/BRAIN_LLM_FALLBACK_URL env vars;
no-op when unset so existing deployments are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds the route entry alongside the existing POST routes. Note: this
is the brain HTTP REST API's first GET endpoint — it follows REST
semantics for pure reads, while the legacy POST routes (query, write,
ingest, etc.) all take JSON bodies. Future read endpoints SHOULD use
GET; future write endpoints continue with POST.
The ingestion server now exposes both REST and MCP on the same port
(3300). MCP shares brainDir, pipeline config, and LLM client with the
REST handlers — single source of process state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Allows callers to provide pre-structured RawPage data directly, bypassing the
LLM extraction step. The pipeline still handles slug computation, frontmatter,
link canonicalization, source back-references, and dedup — only the extraction
is skipped. Useful when a more capable model or manual curation produces the
structured data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Start background watcher on startup when INGEST_WATCH_INTERVAL > 0
- Procfile: add INGEST_WATCH_INTERVAL=30 and INGEST_SVC_URL for supervisor
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wires pipeline.Run into the HTTP layer so callers can ingest raw text
or files/directories without touching the filesystem directly. Rewrites
main.go to parse LLM and watcher env vars and build pipeline.Config.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mathiasandClaude Opus 5.5