Brain interface: intent↔interface mismatch analysis (problem statement, evidence-backed) #43

Open
opened 2026-06-15 17:57:14 +00:00 by mathias · 0 comments
Owner

What this is

An evidence-backed problem statement for the brain interface, produced by a two-column intent analysis of real usage (autonomous agents + human-in-the-loop). It is deliberately not a redesign — the verb signatures, consumer split, and layer model are a separate, deliberate step left to the human. This issue exists so the finding doesn't evaporate before that step.

Evidence

  • Two-column analysis doc: mathias/brain branch analysis/intent-two-column, analysis/2026-06-15-brain-intent-two-column-analysis.md (commit 2ea5644).
  • Agent column: koala session logs, brain/sessions/analysis/agent-intent-column.{jsonl,md} (commit e8dbcf6).
  • Human column: Claude.ai conversation history, May–Jun 2026.

Method: classify each brain call by what it was trying to do (knowledge-act), not which tool it called; find intent↔interface mismatches.

Headline finding

Both columns, independently, ~37–38% mismatch with 0% genuine intent_unclear. Usage is not chaotic — the muddiness is in the interface, not the behaviour. The brain today is append + keyword-search: good at create and lexical read, missing update, missing promotion-as-operation, and serving semantic intent with a lexical (BM25) tool.

The structural root: the interface was never split by consumer, so the mechanism-named tools (brain_write, brain_query, brain_ingest, …) read as purpose-less — each serves whichever consumer happens to call it, with no statement of which knowledge-act it's for. And each consumer is missing a different verb.

The gaps (by consumer)

Shared — highest priority:

  • update / supersede — the seam. Agents 5/5 mismatch (blind-rewrite same slug → duplicates/contradictions). Humans 5+ (manual cherry-pick; brain PR #1 closed because content existed at two paths with no reconcile; re-ingest overwrites wholesale). The one act both consumers need and neither has. Highest-value gap. Note: distinct from #23 (embedding re-sync on edit) — this is a missing write verb, not an embedding-staleness fix.
  • Semantic vs lexical read. Both bend BM25 toward meaning-based intent. A true semantic read is missing.
  • Layer routing. knowledge/ vs wiki/ is a curation-state distinction (unreviewed vs promoted) with no verb to move a note across it — a noun with no verb. Both columns flag brain_ingest_raw ignoring the path prefix.

Agent-only:

  • verify_write_landed — 4/4 mismatch. No read-after-write; agents lexical-re-query their own just-written note to confirm it landed. Humans never hit this.
  • synthesized_answer. brain_answer distrusted — replaced by parallel brain_query.

Human-only:

  • curate / promote raw→wiki. Top human friction. Already tracked — see #38 (brain_pending + brain_promote). Not re-specced here.

Relationship to existing issues

  • #38 — specs the promote half (human curation gap). This issue does not duplicate it; it frames the broader interface problem it's one part of. (#37 was a duplicate of #38 — now closed.)
  • #45brain_update supersede-by-slug write verb + read-after-write (brain_get, brain_write handle). The implementation issue for the two highest-priority gaps named here.
  • #23 (closed) — embedding re-sync on edit; adjacent but not the update/supersede write verb.
  • #25 — skills/MCP-boundary overhaul; the MCP-vs-CLI verdict per server should account for the verb gaps named here.
  • #28 (closed) — shipped brain_context.

What's NOT in this issue

No verb signatures. No consumer-split design. No layer-model redesign. Those are the deliberate next step (notebook-first). This issue is the durable handle for the problem, with evidence attached.

Suggested done-when (for the problem-framing, not the fix)

  • Human reviews the two-column evidence doc and confirms/adjusts the gap list. (2026-06-22 — gap list confirmed; no adjustments.)
  • Decision recorded on whether update/supersede and verify_write_landed become tracked implementation issues. (2026-06-22 — both YES. update/supersede#45 as supersede-by-slug whole-note replace, git as history layer. verify_write_landed → folded into #45 as brain_get + brain_write handle rather than its own issue.)
  • #37/#38 deduped and confirmed as the promote half. (2026-06-22 — #37 closed as duplicate; #38 retained.)
  • Redesign step scheduled (separate issue / ADR) once the verb set and consumer split are decided.

Decision log — 2026-06-22 review session

Worked the done-when sequence end to end:

  1. #37/#38 deduped#37 closed (strictly weaker), #38 kept as canonical promote-half.
  2. update/supersede tracked — decision: YES. Shape = supersede-by-slug, whole-note replace (not patch, not append-tombstone) — agents regenerate whole notes, so patch-addressing solves a non-problem. Provenance = git is the history mechanism (no sidecar, no tombstone); superseded content recoverable but not queryable; frontmatter stamps the supersession.
  3. verify_write_landed tracked — decision: YES, but folded into the same issue (#45) as brain_get(id|path) + extending brain_write to return a handle. Not its own issue — it's a return-value change plus one getter.
  4. #45 filed with the above as the implementation target.

Still open: the redesign step (consumer-split / layer-model) below.

## What this is An evidence-backed **problem statement** for the brain interface, produced by a two-column intent analysis of real usage (autonomous agents + human-in-the-loop). It is deliberately *not* a redesign — the verb signatures, consumer split, and layer model are a separate, deliberate step left to the human. This issue exists so the finding doesn't evaporate before that step. ## Evidence - Two-column analysis doc: `mathias/brain` branch `analysis/intent-two-column`, `analysis/2026-06-15-brain-intent-two-column-analysis.md` (commit `2ea5644`). - Agent column: koala session logs, `brain/sessions/analysis/agent-intent-column.{jsonl,md}` (commit `e8dbcf6`). - Human column: Claude.ai conversation history, May–Jun 2026. Method: classify each brain call by *what it was trying to do* (knowledge-act), not which tool it called; find intent↔interface mismatches. ## Headline finding Both columns, independently, ~37–38% mismatch with **0% genuine `intent_unclear`**. Usage is not chaotic — the muddiness is in the interface, not the behaviour. The brain today is **append + keyword-search**: good at create and lexical read, missing update, missing promotion-as-operation, and serving semantic intent with a lexical (BM25) tool. The structural root: the interface was never **split by consumer**, so the mechanism-named tools (`brain_write`, `brain_query`, `brain_ingest`, …) read as purpose-less — each serves whichever consumer happens to call it, with no statement of which knowledge-act it's for. And each consumer is missing a *different* verb. ## The gaps (by consumer) **Shared — highest priority:** - **`update / supersede` — the seam.** Agents 5/5 mismatch (blind-rewrite same slug → duplicates/contradictions). Humans 5+ (manual cherry-pick; brain PR #1 closed because content existed at two paths with no reconcile; re-ingest overwrites wholesale). The one act both consumers need and neither has. **Highest-value gap.** Note: distinct from #23 (embedding re-sync on edit) — this is a missing *write verb*, not an embedding-staleness fix. - **Semantic vs lexical read.** Both bend BM25 toward meaning-based intent. A true semantic read is missing. - **Layer routing.** knowledge/ vs wiki/ is a *curation-state* distinction (unreviewed vs promoted) with **no verb to move a note across it** — a noun with no verb. Both columns flag `brain_ingest_raw` ignoring the path prefix. **Agent-only:** - **`verify_write_landed` — 4/4 mismatch.** No read-after-write; agents lexical-re-query their own just-written note to confirm it landed. Humans never hit this. - **`synthesized_answer`.** `brain_answer` distrusted — replaced by parallel `brain_query`. **Human-only:** - **`curate / promote raw→wiki`.** Top human friction. **Already tracked — see #38** (`brain_pending` + `brain_promote`). Not re-specced here. ## Relationship to existing issues - **#38** — specs the *promote* half (human curation gap). This issue does **not** duplicate it; it frames the broader interface problem it's one part of. (#37 was a duplicate of #38 — now closed.) - **#45** — `brain_update` supersede-by-slug write verb + read-after-write (`brain_get`, `brain_write` handle). The implementation issue for the two highest-priority gaps named here. - **#23** (closed) — embedding re-sync on edit; adjacent but not the `update/supersede` write verb. - **#25** — skills/MCP-boundary overhaul; the MCP-vs-CLI verdict per server should account for the verb gaps named here. - **#28** (closed) — shipped `brain_context`. ## What's NOT in this issue No verb signatures. No consumer-split design. No layer-model redesign. Those are the deliberate next step (notebook-first). This issue is the durable handle for the problem, with evidence attached. ## Suggested done-when (for the problem-framing, not the fix) - [x] Human reviews the two-column evidence doc and confirms/adjusts the gap list. *(2026-06-22 — gap list confirmed; no adjustments.)* - [x] Decision recorded on whether `update/supersede` and `verify_write_landed` become tracked implementation issues. *(2026-06-22 — both YES. `update/supersede` → #45 as supersede-by-slug whole-note replace, git as history layer. `verify_write_landed` → folded into #45 as `brain_get` + `brain_write` handle rather than its own issue.)* - [x] #37/#38 deduped and confirmed as the promote half. *(2026-06-22 — #37 closed as duplicate; #38 retained.)* - [ ] Redesign step scheduled (separate issue / ADR) once the verb set and consumer split are decided. --- ## Decision log — 2026-06-22 review session Worked the done-when sequence end to end: 1. **#37/#38 deduped** — #37 closed (strictly weaker), #38 kept as canonical promote-half. 2. **`update/supersede` tracked** — decision: YES. Shape = supersede-by-slug, whole-note replace (not patch, not append-tombstone) — agents regenerate whole notes, so patch-addressing solves a non-problem. Provenance = git is the history mechanism (no sidecar, no tombstone); superseded content recoverable but not queryable; frontmatter stamps the supersession. 3. **`verify_write_landed` tracked** — decision: YES, but folded into the same issue (#45) as `brain_get(id|path)` + extending `brain_write` to return a handle. Not its own issue — it's a return-value change plus one getter. 4. **#45 filed** with the above as the implementation target. Still open: the redesign step (consumer-split / layer-model) below.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/hyperguild#43