mathias/hyperguild
1
0
Fork 0
Code Issues 30 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

feat(brain-mcp): read-only enumeration surface over HTTP (list/get/graph) — unblocks brain-audit's real read backend #71

New Issue
Open
opened 2026-06-26 21:40:47 +00:00 by mathias · 2 comments
mathias commented 2026-06-26 21:40:47 +00:00
Owner
Copy Link
Copy Source

Why

mathias/brain-gardener Phase-1 brain-audit (issue #2) needs to read the entire brain corpus (every entry, every wing) to compute graph-health checks — orphans, broken [[wikilinks]], dup pairs, dangling supersede chains. The current brain-mcp surfaces can't enumerate that:

  • HTTP /query = BM25 search (top-N matches), not "list all".
  • brain_get = single path. MCP brain_query = search, same ceiling.

So there is no read path that returns the full set of entries. brain-audit's spine is blocked on it.

Decision context (from brain-gardener#2, comment 1770)

To not block the Phase-1 milestone, brain-audit will ship an interim filesystem reader over a read-only checkout of git.d-ma.be/mathias/brain (wiki/<wing>/<hall>/<slug>.md + knowledge/<slug>.md), behind a brain.Reader interface. That's strictly read-only, gives the full corpus + frontmatter, and satisfies both brain-audit invariants (no brain-write verb; no internet egress).

This issue is the proper long-term backend that drops in behind the same brain.Reader interface once it exists.

Proposed scope — strictly read-only

Add to the brain-mcp HTTP surface (ingestion/), authed like /query:

  1. /list (or brain_list) — enumerate all entry paths, optionally filtered by wing/hall. Paginated. Returns {path, content_hash, frontmatter} without bodies (cheap manifest).
  2. /get over HTTP — already exists as brain_get on MCP; expose the same read on HTTP so non-MCP Go consumers can fetch {path, frontmatter, body, content_hash}.
  3. /graph over HTTP — expose the existing brain_graph neighbors/subgraph ops so wikilink-graph checks don't have to reconstruct edges client-side.

Hard constraint: read-only. No write/update/tunnel verbs on this surface. brain-audit's invariant #1 (no brain-write path in the binary) means the consumer must not even be able to mutate through this API.

Open questions / feedback wanted

  • Is the interim FS reader acceptable as the Phase-1 backend, with this HTTP surface as a fast-follow — or do you want brain-audit to wait for this before going end-to-end? (My lean: ship FS reader now, this later; the brain.Reader seam makes the swap a one-file change.)
  • Does this belong in ingestion/ (the brain-mcp server) or is there a cleaner home given the #43 brain-interface analysis?
  • /list without bodies + lazy /get per node, or a single streaming full-dump endpoint? The audit reads everything, so a bulk dump might be simpler than N+1 gets — but breaks the per-node access pattern other consumers want.

Related: #43 (brain interface mismatch), #47 (read-after-write semantics — same read surface).

## Why `mathias/brain-gardener` Phase-1 `brain-audit` ([issue #2](https://git.d-ma.be/mathias/brain-gardener/issues/2)) needs to read the **entire brain corpus** (every entry, every wing) to compute graph-health checks — orphans, broken `[[wikilinks]]`, dup pairs, dangling supersede chains. The current brain-mcp surfaces can't enumerate that: - HTTP `/query` = BM25 **search** (top-N matches), not "list all". - `brain_get` = single path. MCP `brain_query` = search, same ceiling. So there is no read path that returns the full set of entries. brain-audit's spine is blocked on it. ## Decision context (from brain-gardener#2, comment 1770) To not block the Phase-1 milestone, brain-audit will ship an **interim filesystem reader** over a read-only checkout of `git.d-ma.be/mathias/brain` (`wiki/<wing>/<hall>/<slug>.md` + `knowledge/<slug>.md`), behind a `brain.Reader` interface. That's strictly read-only, gives the full corpus + frontmatter, and satisfies both brain-audit invariants (no brain-write verb; no internet egress). **This issue is the proper long-term backend** that drops in behind the same `brain.Reader` interface once it exists. ## Proposed scope — strictly read-only Add to the brain-mcp HTTP surface (`ingestion/`), authed like `/query`: 1. **`/list`** (or `brain_list`) — enumerate all entry paths, optionally filtered by wing/hall. Paginated. Returns `{path, content_hash, frontmatter}` without bodies (cheap manifest). 2. **`/get`** over HTTP — already exists as `brain_get` on MCP; expose the same read on HTTP so non-MCP Go consumers can fetch `{path, frontmatter, body, content_hash}`. 3. **`/graph`** over HTTP — expose the existing `brain_graph` neighbors/subgraph ops so wikilink-graph checks don't have to reconstruct edges client-side. **Hard constraint:** read-only. No write/update/tunnel verbs on this surface. brain-audit's invariant #1 (no brain-write path in the binary) means the consumer must not even be *able* to mutate through this API. ## Open questions / feedback wanted - Is the **interim FS reader acceptable** as the Phase-1 backend, with this HTTP surface as a fast-follow — or do you want brain-audit to wait for this before going end-to-end? (My lean: ship FS reader now, this later; the `brain.Reader` seam makes the swap a one-file change.) - Does this belong in `ingestion/` (the brain-mcp server) or is there a cleaner home given the #43 brain-interface analysis? - `/list` without bodies + lazy `/get` per node, or a single streaming full-dump endpoint? The audit reads everything, so a bulk dump might be simpler than N+1 gets — but breaks the per-node access pattern other consumers want. Related: #43 (brain interface mismatch), #47 (read-after-write semantics — same read surface).
mathias commented 2026-06-26 21:44:56 +00:00
Author
Owner
Copy Link
Copy Source

Approved scope. Ship FS reader now; this is the fast-follow.

Three answers:

1. Interim FS reader — yes, acceptable; do NOT wait on this issue.

The brain.Reader seam makes the swap a one-file change, so blocking the Phase-1 end-to-end milestone on a different repo's new endpoint would be self-inflicted. Ship the FS reader now (verified viable against mathias/brain HEAD — 674 md, correct wiki/<wing>/<hall>/ shape). This HTTP surface drops in behind the same interface later.

2. Home — ingestion/ is right, but sequence after #43.

It's the brain-mcp server, so that's the correct module. One dependency check: if #43 (brain-interface mismatch) is about to relocate or restructure that surface, build #71 after #43's structural call — don't bake new endpoints into a module that's about to move. Not a block, just a one-line check before you start.

3. Bulk-dump vs N+1 — do both; it's not actually a fork.

  • /list → bodiless manifest {path, content_hash, frontmatter}, paginated, wing/hall filter. Cheap.
  • bulk /get accepting a set of paths (or a streaming full-dump) → the audit's read-everything path.
  • single /get stays for per-node consumers.

The audit uses /list + bulk-fetch; other consumers use single /get. Bonus from the manifest's content_hash: the audit can skip-unchanged across runs later (incremental audits).

Hard constraint (non-negotiable)

The entire surface must be physically incapable of mutation — not merely "exposes no write," but no write/update/tunnel verb reachable through this auth scope at all. brain-audit's invariant #1 (no brain-write path in the binary) must hold even against a compromised or buggy consumer. If a write verb is reachable through the same credential, this surface fails its purpose.

No rush — FS reader is the unblock; this lands when #43 settles.

## Approved scope. Ship FS reader now; this is the fast-follow. Three answers: ### 1. Interim FS reader — **yes, acceptable; do NOT wait on this issue.** The `brain.Reader` seam makes the swap a one-file change, so blocking the Phase-1 end-to-end milestone on a different repo's new endpoint would be self-inflicted. Ship the FS reader now (verified viable against `mathias/brain` HEAD — 674 md, correct `wiki/<wing>/<hall>/` shape). This HTTP surface drops in behind the same interface later. ### 2. Home — **`ingestion/` is right, but sequence after #43.** It's the brain-mcp server, so that's the correct module. One dependency check: if #43 (brain-interface mismatch) is about to relocate or restructure that surface, build #71 *after* #43's structural call — don't bake new endpoints into a module that's about to move. Not a block, just a one-line check before you start. ### 3. Bulk-dump vs N+1 — **do both; it's not actually a fork.** - **`/list`** → bodiless manifest `{path, content_hash, frontmatter}`, paginated, wing/hall filter. Cheap. - **bulk `/get`** accepting a set of paths (or a streaming full-dump) → the audit's read-everything path. - **single `/get`** stays for per-node consumers. The audit uses `/list` + bulk-fetch; other consumers use single `/get`. Bonus from the manifest's `content_hash`: the audit can skip-unchanged across runs later (incremental audits). ### Hard constraint (non-negotiable) The entire surface must be **physically incapable of mutation** — not merely "exposes no write," but **no write/update/tunnel verb reachable through this auth scope at all**. brain-audit's invariant #1 (no brain-write path in the binary) must hold even against a compromised or buggy consumer. If a write verb is reachable through the same credential, this surface fails its purpose. No rush — FS reader is the unblock; this lands when #43 settles.
mathias commented 2026-08-22 06:42:23 +00:00
Author
Owner
Copy Link
Copy Source

Re-gated: unblocked, no consumer waiting. Optimisation, not a dependency.

Checked against live state during the ADR-0009 sprint (2026-08-22). This issue's premise no longer holds.

It opens with "brain-audit's spine is blocked on it." It is not. brain-gardener#2 shipped and closed, and brain-audit is running in production today: three worklist commits authored brain-audit CI on 2026-08-22, latest reports/2026-08-22-audit.json carrying 1668 findings across all eight checks (ul-drift 1046, orphan 569, stale-date 28, malformed 10, broken-link 8, thin 4, deindex 2, ul-gap 1). Orphan and broken-link are exactly the whole-corpus graph checks this issue said could not be computed without an enumeration API.

It shipped on the interim filesystem reader over a read-only checkout — the fallback this issue itself proposed, behind the brain.Reader seam. So the lean recorded here ("ship FS reader now, this later; the seam makes the swap a one-file change") was taken, and it worked.

Status

Open, re-gated as an optimisation. Deliberately not closed: the design is sound and the seam it targets still exists. But nothing is waiting on it, so it should not read as blocking work.

What would make it actionable again — any one of:

  • A consumer that cannot use a filesystem checkout. A pod without the brain hostPath, or an out-of-cluster reader, would need the HTTP surface. brain-audit has the mount; a future consumer might not.
  • The FS reader becoming a correctness problem rather than a shape problem — e.g. audit reading a checkout that lags the live tree and reporting stale findings. Worth watching, since brain-sync commits every 15 minutes and the audit reads git, not the live tree. Not currently observed, and I did not test for it.
  • The /list manifest being needed by something other than audit (a UI, a search index rebuild, brain-ui).

One answer this issue asked for, now available

Does this belong in ingestion/ (the brain-mcp server) or is there a cleaner home given the #43 brain-interface analysis?

ingestion/ — settled by ADR-0009 and hyperguild#91 part 2. brain-mcp stays inside ingestion; there is no separate home to move a read surface to.

Two design notes for whenever it is picked up, both from work that postdates this issue:

  • It must land as recall/get shapes, not new mechanism-named endpoints. ADR-0012 governs; a /list verb added today would be a fresh rule-1 violation. get with a manifest mode, or recall with an enumerate strategy, fits the set designed on #80.
  • Strictly read-only remains the right constraint, and is now doubly motivated: brain-audit's invariant #1, plus ADR-0009's authority split, which puts curation-state transitions behind a human principal. A read surface must not become a place where that gate can be sidestepped.

Refs: brain-gardener#2 (closed, live), #43, #80, #91, ADR-0009 (infra PR #296), ADR-0012.

## Re-gated: unblocked, no consumer waiting. Optimisation, not a dependency. Checked against live state during the ADR-0009 sprint (2026-08-22). **This issue's premise no longer holds.** It opens with *"brain-audit's spine is blocked on it."* It is not. brain-gardener#2 shipped and closed, and brain-audit is running in production today: three worklist commits authored `brain-audit CI` on 2026-08-22, latest `reports/2026-08-22-audit.json` carrying 1668 findings across all eight checks (`ul-drift` 1046, `orphan` 569, `stale-date` 28, `malformed` 10, `broken-link` 8, `thin` 4, `deindex` 2, `ul-gap` 1). Orphan and broken-link are exactly the whole-corpus graph checks this issue said could not be computed without an enumeration API. It shipped on the interim filesystem reader over a read-only checkout — the fallback this issue itself proposed, behind the `brain.Reader` seam. So the lean recorded here (*"ship FS reader now, this later; the seam makes the swap a one-file change"*) was taken, and it worked. ### Status **Open, re-gated as an optimisation.** Deliberately not closed: the design is sound and the seam it targets still exists. But nothing is waiting on it, so it should not read as blocking work. What would make it actionable again — any one of: - A consumer that **cannot** use a filesystem checkout. A pod without the brain hostPath, or an out-of-cluster reader, would need the HTTP surface. brain-audit has the mount; a future consumer might not. - The FS reader becoming a **correctness** problem rather than a shape problem — e.g. audit reading a checkout that lags the live tree and reporting stale findings. Worth watching, since brain-sync commits every 15 minutes and the audit reads git, not the live tree. **Not currently observed**, and I did not test for it. - The `/list` manifest being needed by something other than audit (a UI, a search index rebuild, brain-ui). ### One answer this issue asked for, now available > *Does this belong in `ingestion/` (the brain-mcp server) or is there a cleaner home given the #43 brain-interface analysis?* `ingestion/` — settled by ADR-0009 and hyperguild#91 part 2. brain-mcp stays inside `ingestion`; there is no separate home to move a read surface to. Two design notes for whenever it is picked up, both from work that postdates this issue: - **It must land as `recall`/`get` shapes, not new mechanism-named endpoints.** ADR-0012 governs; a `/list` verb added today would be a fresh rule-1 violation. `get` with a manifest mode, or `recall` with an enumerate strategy, fits the set designed on #80. - **Strictly read-only remains the right constraint**, and is now doubly motivated: brain-audit's invariant #1, plus ADR-0009's authority split, which puts curation-state transitions behind a human principal. A read surface must not become a place where that gate can be sidestepped. Refs: brain-gardener#2 (closed, live), #43, #80, #91, ADR-0009 (infra PR #296), ADR-0012.
mathias added the risk/mediumsize/mtype/feat labels 2026-08-27 05:25:42 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
agent/ready
Meets the agent-ready contract; authorizes unattended execution
 blocked/client
Waiting on the client
 blocked/supplier
Waiting on a supplier
 blocked/upstream
Waiting on an upstream project or vendor
 risk/high
Irreversible, security-relevant, or control plane
 risk/low
Reversible, local, test-covered
 risk/medium
Touches shared state, deploys or data
 size/l
Too big - split it
 size/m
Under two days
 size/s
Under half a day
 size/xs
Under an hour
 status:dispatch
status:dispatched
status:review
status:reviewing
type/chore
Maintenance, no behaviour change
 type/decision
Human decision required; never agent/ready
 type/docs
Documentation only
 type/feat
New capability
 type/fix
Corrects broken behaviour
 type/ops
Infrastructure, deploy, runtime
 type/research
Investigate; output is a finding, not code
 type/test
Test coverage only
No labels risk/medium size/m type/feat
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate-bot mathias
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#71