Extraction/canonicalization mass-attaches spurious [[mission]] link to unrelated tunnel-candidates notes #88
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mathias/hyperguild#88
Reference in New Issue
Block a user
Problem
4 separate, topically-unrelated notes each got a
[[mission]]link (→wiki/telos/decisions/mission.md) added with zero real connection:wiki/sources/tunnel-candidates-2026-07-19.md,-07-20.md,-07-22.md,-07-25.md— SSO decisions, BGP tunnel notes, security allowlist comparisons, none about TELOS mission.Local-LLM semantic-relevance check flagged all 4 independently as hallucinated/no topical connection. Same target, same source-note type, recurring — one systemic cause, not 4 independent hallucinations.
Where to look
Two candidate mechanisms in
ingestion/internal/pipeline/:links.go'sCanonicalizeLinks— if the LLM output a bare[[mission]](plain-link form) as a generic/filler mention,canonicalizeContent's title-lookup would resolve it against the inventory's "Mission" title and silently canonicalize it into a real link, rather than questioning whether the mention belongs there at all.schema.md/BuildPrompt) — if the schema's example output or instructions make "mission" look like a default/expected related-concept slot, the LLM may be pattern-matching it in from few-shot bias ontunnel-candidates-shaped inputs specifically.Scope
Acceptance criteria
tunnel-candidates-*-shaped fixture with no mission-related content does not get[[mission]]attached.Root cause was neither of the two mechanisms this issue named. Investigated both directly:
[[Mission]]becausewiki/concepts/mission.mdgenuinely exists as a concept page titled "Mission." Not a bug; it did its job.systemPrompt/BuildPrompt.Actual mechanism, found by tracing where these
tunnel-candidates-*.mdnotes come from: they're not normal ingested sources — they'reinternal/brain/tunnel.go'sAutoTunnel/DetectTunnelsoutput.DetectTunnelsdoes a naive case-insensitive whole-word substring match of every wiki note's title against source content; "Mission" (a generic English word) fuzzy-matches unrelated content constantly, and those fuzzy (non-exact) matches get logged tobrain/raw/tunnel-candidates-YYYY-MM-DD.mdas a human-review queue (brain_tunnelto promote).The bug:
internal/watcher/watcher.go'sprocessDirwalksbrain/raw/and feeds every.mdfile to the LLM extraction pipeline — with no exclusion fortunnel-candidates-*.md. So the watcher re-ingested AutoTunnel's own review-queue file as if it were external source material. The LLM, reading a log that literally contains(term: "mission")entries, reasonably surfaced "mission" as a term in its summary — which CanonicalizeLinks then correctly turned into a real link, because it correctly is a real page, just not a relevant one.Fixed in hyperguild@f005548:
processDirnow skipstunnel-candidates-*filenames, mirroring the exclusionapi.ListPendingalready applied. Regression test reproduces the original bug (asserts the LLMCompletefunc is never called for a tunnel-candidates file) and confirms the fix.Removed the 4 existing bad links in
mathias/brain@95af3b7— de-linked "mission" to plain text with a note explaining the fuzzy match, and trimmed one derivative hallucinated sentence in the 07-25 note that asserted a "broader mission" connection built on the bad link.