[handover] Brain-MCP intent↔interface study — agent column + merge scaffold ready for review #44

New Issue

2026-06-16T18:48:41Z

mathias commented

2026-06-16 18:48:41 +00:00

Status: agent-consumer column complete, ready for review; awaiting human column to merge

Two-part study of how the brain MCP is actually used — intent↔interface mismatch, not call-frequency. This issue hands over the agent-consumer column (run on koala) + the merge scaffold for review.

Deliverables (all on `main`, `brain/sessions/analysis/`)

file	commit	what
`agent-intent-column.jsonl`	`e8dbcf6`	46 per-call rows, reconstructed-vocab intent classification
`agent-intent-column.md`	`e8dbcf6`	histogram, mismatch list, headline gap
`intent-interface-findings.md`	`aa91838`	unified two-column merge scaffold (schema + vocab locked)

Headline numbers (agent side)

46 distinct knowledge-acts, 37% interface mismatch (17/46), 0% intent_unclear.
Corpus = Claude Code agent transcripts (~/.claude/projects) — the only source with brain calls. brain/sessions/*.jsonl empty; agentsquad eval logs are not brain calls; no Crush logs. All calls agent-initiated → all autonomous_agent.

Primary finding — 4 mismatch clusters

update_or_supersede (5/5 mismatch) — highest value. No edit/patch verb → agents blind-re-write the same slug (3 of 5 within 30s). Can't tell if a re-write deduped or forked a contradiction.
verify_write_landed (4/4). No read-after-write/get-by-id → agents lexically re-query their own fresh note with reformulated keywords.
semantic_retrieval (3/3). Meaning known, indexed words unknown → BM25 keyword-stuffing; a 4-step reformulation chain on one Go bug.
synthesized_answer (2/5 + 1 partial). brain_answer not trusted terminal — 3 answers → 3 same-topic queries 1 min later; query+answer fired 8s apart hedging one need. Works for episodic recall, fails for how-do-I.

Plus: write-schema confusion (3 param shapes + ingest), and HTTP-curl / /tmp/brain_entry.json bodges when MCP auth lapsed.

Caveat (needs reviewer awareness)

Canonical brain-intent-extraction.md is not present on koala — only this task's prompt references it. The closed intent vocabulary was reconstructed from the prompt + brain/schema.md; every row tagged schema_source=reconstructed. If the canonical file surfaces, re-map intent in both columns identically before merging. Evidence (workaround / observed_friction) stands regardless of label names.

Open dependency — blocks the merge

Human-consumer column (Claude.ai conversation history) not delivered to koala. Searched local repo, git pull, whole host, gitea remote — only the agent column exists. Drop human-intent-column.jsonl (LOCKED schema in the scaffold) into brain/sessions/analysis/ to unblock.

Review asks

Validate the reconstructed intent vocab vs the canonical schema (if it exists elsewhere).
Sanity-check the agent-side mismatch classifications, esp. update_or_supersede and verify_write_landed (the 100%-mismatch intents).
Confirm the divergence hypothesis pre-seeded in the scaffold: agent mismatch is write-side-heavy (supersede + verify = 9/17); is human mismatch read-side-heavy (semantic + answer)? If so → the interface fails the two consumers at opposite ends.

Scope was evidence-only, no redesign — kept to that.

🤖 Generated with Claude Code

## Status: agent-consumer column complete, ready for review; awaiting human column to merge Two-part study of how the brain MCP is *actually* used — intent↔interface mismatch, not call-frequency. This issue hands over the **agent-consumer column** (run on koala) + the **merge scaffold** for review. ### Deliverables (all on `main`, `brain/sessions/analysis/`) | file | commit | what | |---|---|---| | `agent-intent-column.jsonl` | `e8dbcf6` | 46 per-call rows, reconstructed-vocab intent classification | | `agent-intent-column.md` | `e8dbcf6` | histogram, mismatch list, headline gap | | `intent-interface-findings.md` | `aa91838` | unified two-column merge scaffold (schema + vocab locked) | ### Headline numbers (agent side) - **46 distinct knowledge-acts**, **37% interface mismatch (17/46)**, **0% intent_unclear**. - Corpus = Claude Code agent transcripts (`~/.claude/projects`) — the *only* source with brain calls. `brain/sessions/*.jsonl` empty; agentsquad eval logs are not brain calls; no Crush logs. All calls agent-initiated → all `autonomous_agent`. ### Primary finding — 4 mismatch clusters 1. **update_or_supersede (5/5 mismatch) — highest value.** No edit/patch verb → agents blind-re-write the same slug (3 of 5 within 30s). Can't tell if a re-write deduped or forked a contradiction. 2. **verify_write_landed (4/4).** No read-after-write/get-by-id → agents lexically re-query their own fresh note with reformulated keywords. 3. **semantic_retrieval (3/3).** Meaning known, indexed words unknown → BM25 keyword-stuffing; a 4-step reformulation chain on one Go bug. 4. **synthesized_answer (2/5 + 1 partial).** `brain_answer` not trusted terminal — 3 answers → 3 same-topic queries 1 min later; query+answer fired 8s apart hedging one need. Works for episodic recall, fails for how-do-I. Plus: write-schema confusion (3 param shapes + `ingest`), and HTTP-curl / `/tmp/brain_entry.json` bodges when MCP auth lapsed. ### Caveat (needs reviewer awareness) Canonical `brain-intent-extraction.md` is **not present on koala** — only this task's prompt references it. The closed intent vocabulary was **reconstructed** from the prompt + `brain/schema.md`; every row tagged `schema_source=reconstructed`. If the canonical file surfaces, re-map `intent` in both columns identically before merging. **Evidence (workaround / observed_friction) stands regardless of label names.** ### Open dependency — blocks the merge - [ ] **Human-consumer column** (Claude.ai conversation history) not delivered to koala. Searched local repo, `git pull`, whole host, gitea remote — only the agent column exists. Drop `human-intent-column.jsonl` (LOCKED schema in the scaffold) into `brain/sessions/analysis/` to unblock. ### Review asks 1. Validate the reconstructed intent vocab vs the canonical schema (if it exists elsewhere). 2. Sanity-check the agent-side mismatch classifications, esp. `update_or_supersede` and `verify_write_landed` (the 100%-mismatch intents). 3. Confirm the divergence hypothesis pre-seeded in the scaffold: **agent mismatch is write-side-heavy (supersede + verify = 9/17); is human mismatch read-side-heavy (semantic + answer)?** If so → the interface fails the two consumers at opposite ends. Scope was **evidence-only, no redesign** — kept to that. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mathias commented

2026-06-16 18:59:22 +00:00

Human column delivered — merge complete

human-intent-column.md produced from claude.ai session history analysis (15 knowledge-acts across 6 sessions, 2026-06-15/16). Stored in brain: wiki/telos/decisions/human-intent-column.md. Unified merged findings: wiki/homelab/decisions/intent-interface-findings-merged-2026-06-16.md.

Human column headline numbers

15 distinct knowledge-acts, 60% mismatch (9/15), 0% intent_unclear (Claude mediates intent classification, structurally suppressing ambiguity)
Corpus: claude.ai sessions — the only surface with human-initiated brain calls

Divergence hypothesis — RESOLVED

Agent mismatch is write-side-heavy (53% of mismatches are update_or_supersede + verify_write_landed). Human mismatch is more balanced (56% write-side, 44% read-side) but both consumers hit the same four intents at 100% mismatch rate.

The sharper divergence confirmed: brain_answer does not synthesize the telos wing. All 2 human synthesized_answer attempts returned "No relevant content found" — even after TELOS wing was established and indexed. brain_query wing=telos works; brain_answer cross-wing synthesis does not pick it up. This is the highest-impact human gap because it makes the entire intention substrate invisible to natural-language queries, defeating TELOS purpose.

The four universal gaps (both consumers, priority order)

No edit/patch primitive (update_or_supersede: agent 5/5, human 3/3) — every iterative knowledge session hits this
brain_answer doesn't synthesize telos wing (synthesized_answer: human 2/2 failed) — defeats TELOS as intention substrate
No get-by-id/read-after-write (verify_write_landed: agent 4/4, human 1/1) — agents and humans can't confirm writes landed
BM25 fails conceptual queries (semantic_retrieval: agent 3/3, human 2/2) — Claude hides human reformulation chains; true friction rate undercounted

Additional human-only findings

Layer confusion (knowledge/ vs wiki/): risk-tier gate on brain_ingest_raw forces downgrade to knowledge/ for humans without approval-loop control; content intended as stable reference lands in session-derived storage silently
Bulk-write gap: TELOS establishment required 9 sequential brain_write calls for one logical multi-file operation
Within-wing tunnel blocked: brain_tunnel requires cross-wing; CAD concept → session-dispatch link (both homelab wing) silently skipped

Proposed interface additions (evidence-grounded)

brain_patch slug section_header new_content — targeted edit, eliminates update_or_supersede mismatch
brain_get slug — read by exact path, eliminates verify_write_landed mismatch
Fix brain_answer telos wing synthesis scope — likely a synthesis index bug, highest human impact
brain_write_many [{...}] — bulk write for multi-file operations
Within-wing tunnel (or relax brain_tunnel cross-wing requirement)

Closing this issue — merge complete. Follow-up work belongs in brain#4 (brain_read/brain_edit root enabler) and a new brain issue for the telos wing synthesis bug.

## Human column delivered — merge complete `human-intent-column.md` produced from claude.ai session history analysis (15 knowledge-acts across 6 sessions, 2026-06-15/16). Stored in brain: `wiki/telos/decisions/human-intent-column.md`. Unified merged findings: `wiki/homelab/decisions/intent-interface-findings-merged-2026-06-16.md`. ### Human column headline numbers - **15 distinct knowledge-acts**, **60% mismatch (9/15)**, **0% intent_unclear** (Claude mediates intent classification, structurally suppressing ambiguity) - Corpus: claude.ai sessions — the only surface with human-initiated brain calls ### Divergence hypothesis — RESOLVED Agent mismatch is write-side-heavy (53% of mismatches are update_or_supersede + verify_write_landed). Human mismatch is more balanced (56% write-side, 44% read-side) but both consumers hit the same four intents at 100% mismatch rate. The sharper divergence confirmed: **brain_answer does not synthesize the telos wing**. All 2 human synthesized_answer attempts returned "No relevant content found" — even after TELOS wing was established and indexed. brain_query wing=telos works; brain_answer cross-wing synthesis does not pick it up. This is the highest-impact human gap because it makes the entire intention substrate invisible to natural-language queries, defeating TELOS purpose. ### The four universal gaps (both consumers, priority order) 1. **No edit/patch primitive** (update_or_supersede: agent 5/5, human 3/3) — every iterative knowledge session hits this 2. **brain_answer doesn't synthesize telos wing** (synthesized_answer: human 2/2 failed) — defeats TELOS as intention substrate 3. **No get-by-id/read-after-write** (verify_write_landed: agent 4/4, human 1/1) — agents and humans can't confirm writes landed 4. **BM25 fails conceptual queries** (semantic_retrieval: agent 3/3, human 2/2) — Claude hides human reformulation chains; true friction rate undercounted ### Additional human-only findings - **Layer confusion** (knowledge/ vs wiki/): risk-tier gate on brain_ingest_raw forces downgrade to knowledge/ for humans without approval-loop control; content intended as stable reference lands in session-derived storage silently - **Bulk-write gap**: TELOS establishment required 9 sequential brain_write calls for one logical multi-file operation - **Within-wing tunnel blocked**: brain_tunnel requires cross-wing; CAD concept → session-dispatch link (both homelab wing) silently skipped ### Proposed interface additions (evidence-grounded) - `brain_patch slug section_header new_content` — targeted edit, eliminates update_or_supersede mismatch - `brain_get slug` — read by exact path, eliminates verify_write_landed mismatch - Fix brain_answer telos wing synthesis scope — likely a synthesis index bug, highest human impact - `brain_write_many [{...}]` — bulk write for multi-file operations - Within-wing tunnel (or relax brain_tunnel cross-wing requirement) Closing this issue — merge complete. Follow-up work belongs in brain#4 (brain_read/brain_edit root enabler) and a new brain issue for the telos wing synthesis bug.

mathias closed this issue

2026-06-16 18:59:30 +00:00

mathias referenced this issue

2026-06-22 06:06:16 +00:00

feat: brain_update — supersede-by-slug write verb (+ read-after-write: brain_get, brain_write handle) #45

mathias referenced this issue

2026-08-22 06:41:05 +00:00

design(brain): remember + recall consolidations under ADR-0009/0012 (blocked on 0009 design) #80

mathias referenced this issue

2026-08-23 05:38:31 +00:00

Semantic vs lexical retrieval — the gap ADR-0009 structured but did not close #113

mathias referenced this issue