Brain search requires Postgres, so there is no offline or portable path — decide whether that matters #122
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mathias/hyperguild#122
Reference in New Issue
Block a user
Surfaced by the LEANN research spike, 2026-09-22. LEANN itself was evaluated and rejected — it saves ~94 MB against a 97 MB index and costs sub-10ms → under 2s on the query path. Brain:
leann-evaluated-and-rejected-it-optimises-a-resource-this-estate-has-in-surplus.But the evaluation exposed a property of the current design that nobody has stated as a decision:
Every brain query needs koala
ingestion/internal/search/search.gohybridMerge()embeds the query and runs a pgvector search againstbrain_embeddingsonpostgres18-0. BM25 needs the brain checkout; the dense half needs the database.So brain retrieval is available exactly when koala is, and only from something that can reach it. There is no read-only, self-contained copy that works on a laptop with no network.
Why it is worth deciding rather than assuming
The brain is the estate's memory, and the two situations where you most want memory are the two where this design does not have it:
docs/backup-recovery.mdbare-metal rebuild path is exactly when prior incident notes are most valuable, and that is when they are least reachable./var/lib/supervisor/brainwould fit anywhere; the retrieval layer is what does not travel.Today's mitigation is implicit: the markdown is in git, so
grepworks. That is a real fallback and may be sufficient — but it is a fallback nobody has decided to rely on, and it loses both ranking and the dense half.Options
git grepis the offline path. Cheapest, and honest if written down.pip installand a directory, no Postgres. Trigger 2 of the spike's re-open conditions.brain_querythat degrades to BM25 when Postgres is unreachable is a smaller change than (2) and recovers most of the value.Option 3 looks like the best ratio, and it is worth noting that it is mostly already there — the BM25 path exists and runs sub-10ms; what is missing is a way to invoke it without the dense half, and a decision that doing so is acceptable rather than broken.
Not urgent
Nothing is failing. This is a resilience property that was never chosen — filed so it is a decision with a record rather than an assumption that surfaces during an outage.
Refs
ingestion/internal/search/search.go—hybridMerge()ingestion/internal/vectorstore/pg.go— the pgvector pathadr-pgvector-over-qdrant— the original choice, taken on operational simplicity and still correctdocs/backup-recovery.md— the rebuild path where this bites hardestCreated via git-mcp on behalf of @mathias
Cross-reference: the LEANN spike that surfaced this is recorded as signals#96, and in the brain as
wiki/homelab/decisions/leann-evaluated-and-rejected-it-optimises-a-resource-this-estate-has-in-surplus.Worth restating the relationship, because it is easy to read this issue as "adopt LEANN" and it is not:
LEANN was rejected — it saves ~94 MB against a 97 MB index and moves the query path from sub-10ms to under 2 seconds. This issue is the one thing that survived the evaluation: the observation that
hybridMerge()needspostgres18-0, so brain retrieval is available exactly when koala is.That is a property of the current design, not an argument for LEANN's. Option 3 on this issue — BM25-only degraded mode — needs no new index technology at all, and the sparse path already exists and already runs sub-10ms. LEANN only re-enters if option 2 is chosen and a portable dense index turns out to be worth its latency, which is a separate question nobody has asked yet.
Created via git-mcp on behalf of @mathias
Third input to the tiering question, and it is the strongest argument yet for keeping the corpora separate — from the maintenance side rather than the retrieval side.
Asked whether brain-gardener / brain-fix could manage both tiers. They cannot, and the reason generalises past this repo.
ADR-0002 selects one finding per run, uniformly at random, no priority scoring, hourly. So:
Merging the corpora would starve brain gardening 100×, and nothing would alert: the CronJob still runs hourly, still commits, still reports success. The metric that would reveal it — fixes applied per corpus — does not exist, because there has only ever been one corpus.
So the cost of putting archives in the brain's store is not only retrieval dilution (signals#96). It is that the repair loop degrades in proportion to archive size, invisibly.
Filed as brain-gardener#198. It does not block this issue — if the answer here is option 1 or 3 (accept, or BM25-only offline mode), no archive corpus is created and #198 never fires. It becomes live only if archives are ingested into the same store.
Created via git-mcp on behalf of @mathias