No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: mathias/hyperguild#211
Reference in New Issue
Block a user
Sprint #209, PR-B. Refs #208 #209 #205. Risk: medium (changes where ingestion files notes and what it parks; off by default behind
INGEST_WING_ROUTING=1; removes ~1,900 lines of the measured-failed classifier path). The Router is a refuse/park control (client-marker text, unknown wing), so it needs the adversarial pass.Goal as behaviour (acceptance tests, real pipeline + file layout, model stubbed)
Drop a clip in
raw/, do nothing else -> a page inwiki/web-clip/sources/, its entity/concept updated once in the SHAREDwiki/entities/wiki/concepts,raw/holds nothing unprocessed, nothing waits for a human. Client-marked text and a typo'd wing folder are parked instead.What
wing.Router(no model): a deliberateraw/<wing>/folder orwing:frontmatter must be a plain name of an EXISTING wing (typo never creates one) and then places, whatever the wing; disagreeing signals park; with no signal, text matching the client pattern parks, otherwise the default wingweb-clip(the one wing created on first use). Wings are read from disk on every decision.NewRouterfails at startup on a bad default or pattern. Policy change vs #207: a deliberate folder into a client wing places (extraction runs on the local model; it is the owner's act). Flag it if you disagree.wiki/<wing>/sources/(wing/hall frontmatter); concepts/entities stay in the shared dirs, so an entity cited from two wings is ONE page. A source slug already used anywhere but its own wing'ssources/is refused (wiki.FindSlug).brain_promote/brain_rejectact on notes parked inraw/needs-review/(they could not; the documented approval path did not exist); promote drops the proposal keys.INGEST_WING_ROUTING=1,INGEST_DEFAULT_WING(web-clip),INGEST_CLIENT_BLOCK(elseCLAUDE_INGEST_CLIENT_BLOCK); bad config is fatal.wingbacktest, and the oldmigrate/wingmigrate(built for the per-wingfacts/layout this replaces; migration is rebuilt in PR-C).Evidence
go vet ./...,go test -race ./...exit 0, 0 FAIL (grep-checked,&&-chained), lint 0.Not verified / known
.processedmarkers on the live brain are now handled by #210 (stale -> ignored), but the 12 old originals and legacy pages are untouched until PR-C.brain_promoteon a parked note files it as a plain note; it does not run the extraction pipeline on it. Approving a parked clip into extraction is moving it intoraw/<wing>/.Review ask
By execution: (1) any input that makes the watcher write into a wing that does not exist, or outside wiki//sources + shared dirs (folder names, frontmatter values, unicode, CRLF, symlinked folders); (2) the client pattern bypasses (case, unicode, text in frontmatter only, split across chunks); (3) shared-page corruption: two wings ingesting the same entity concurrently/sequentially, title collisions between a source and an entity (
FindSlugguard), a source slug equal to an entity slug; (4) flag off: behaviour identical to main for raw/ notes; (5) promote/reject onneeds-reviewwith name clashes, symlinked dir, and a parked note whose frontmatter lies; (6) anything lost or leaked by deleting the classifier path (grep for dangling references, env vars, docs).🤖 Generated with Claude Code
Created via git-mcp on behalf of @mathias
Independent review pass 1 (pr-gatekeeper agent, by execution; agent, not four-eyes): BLOCK. Resolution at
87c40fe(every fix has a test that failed first and a mutant that fails it):dropSlugClashes); the ingest succeeds; the first wing can still re-ingest.notes-homelab,-2...), pinned directly byTestResolveSourceSlug(title resolution masks the choice at the Run level).typewrote outside the layout (sources-> retired dir,<wing>-> loose file in another wing) -> in a wing onlysource|concept|entityare valid types.brain_pending-> listed (with a pointer to log.md); only markdown can be promoted, the rest moved or rejected.brain_promotesilently acted on the top-level one -> refused when ambiguous; new optionalqueue(inbox|needs-review) on brain_promote/brain_reject/REST;brain_pendinglabels every entry with its queue.proposed_wing: sources|concepts|entities|wiki-> reserved names are never proposed or accepted (router, default, promote).golang.org/x/textwas already in the module graph as indirect; now direct, nothing new downloaded.strings.ToLowerfolds U+212A tok).wing:routed -> only a top-level key inside a properly fenced block counts.origin/last_updatedto the latest ingest and the first wing's attribution is lost -> not changed, documented in docs/raw-inbox.md. Also not changed: a deliberate folder (wing: web-clip+ client text) places by design (explicit wins); explicit-wing notes bypass the client filter.Also: promote into a brand-new wing still creates it (existing promote rule, unchanged). New
docs/raw-inbox.mddocuments the contract, the env vars (INGEST_WING_ROUTING,INGEST_DEFAULT_WING,INGEST_CLIENT_BLOCK) and the limits; no manifest sets the flag, so it is off in production. Gate:go vet ./...,go test -race ./...exit 0, 0 FAIL, lint 0.Merge: I am NOT merging this under the sprint's standing approval. It is control code (refuse/park decisions) and I misapplied the risk lane on #210; this one is left for you to classify and merge.
Created via git-mcp on behalf of @mathias
Independent review pass 2 (pr-gatekeeper agent, by execution): BLOCK at
87c40fe. Resolution at the head above (tests first, mutation-checked):notes.mdwhose model output also has an entity "Notes" wrote two pages with one slug, and neither could be updated again) ->dropSlugClashesnow resolves clashes inside the batch (source over entity over concept) and, against the brain, never skips the SOURCE page (its slug was already chosen free or its own); an existing bad pair no longer locks the source out. Also covers concept+entity with one title in a batch.acme corpwith NBSP/double space/newline,\s+,\b) and soft hyphen as the dash, homoglyphs (Cyrillic a/e/o/c), combining marks, Hangul/braille fillers, grapheme joiner -> folded in several views (NFKD, marks/format/fillers dropped, lookalikes mapped, dashes normalised; whitespace collapsed in one view, removed in another; soft hyphen read as a dash in a third). Still NOT caught, stated in the docs: leetspeak, punctuation splits (a.c.m.e), encodings, homoglyphs outside a small table.wing:-> stripped. 5. LOWbrain_rejecton parked pdf/txt prepended YAML -> non-markdown moves byte for byte;readRawNoteis now capped (64 MiB, endless-reader test that fails instead of draining memory). 7. LOW test gap (reserved-wing check without Sanitise passed all tests) -> tests forSources,sources.,SOURCES,Entities/,concepts\,wiki.INGEST_WING_ROUTING=truesilently left routing off -> parsed strictly, a typo is a startup error.Gate:
go vet ./...,go test -race ./...exit 0, 0 FAIL, lint 0. Next: a third independent pass; merge stays with you (control code).Created via git-mcp on behalf of @mathias
- in a wing, an ingest that writes no source page is an error: the note goes to raw/failed with the warnings in log.md instead of leaving the inbox as 'processed' (model returned [], only invalid types, only clashing entities) - the wing's source path is deterministic, so Resolve no longer redirects a source onto another note's page because the model gave both the same title ('Home'); that overwrote the hash and the text - a batch tie between concept and entity prefers the page already on disk, so its update is not lost; the wing's own source page keeps its slug even when a foreign page shares it - brain_pending lists an unreadable (over-cap) note with the reason instead of dropping it; the client filter's 4 KiB match-span limit is documented Each has a test that fails without it (mutation-checked). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>Independent review pass 3 (pr-gatekeeper agent, by execution): BLOCK on one finding at
49286ba. Resolution at27b1579(tests first, mutation-checked):[]; only pages of a type the layout has no place for; only entities whose slug is taken). Pre-existing for[], widened by this PR. In a wing, an ingest that writes no source page is now an error: the watcher keeps the note inraw/failed/and the reason is inlog.md; it is never archived as processed. Skips (unchanged source) are still skips. Flag-off behaviour unchanged (the legacy[]case is ticketed separately).Resolvematched sources by title). In a wing the source path is already deterministic (the filename), so the title no longer redirects it..mdvanished frombrain_pending-> listed with the reason.Gate:
go vet ./...,go test -race ./...exit 0, 0 FAIL, lint 0. Pass 4 (narrow) follows; merge stays with Mathias.Created via git-mcp on behalf of @mathias
Independent review pass 4 (pr-gatekeeper agent, narrow, by execution): nothing blocking at
27b1579. It confirmed F1-F4 hold and do not break each other: the matrix of model outputs ([], invalid JSON, invalid types only, clashing entities only, two source pages, Skipped, dry-run) ends in an error or a skip in wing mode, never in a silent "processed"; legacy flag-off behaviour is byte-identical to main on those inputs. Two non-blocking findings, both handled at90752f2:source_hash, was written first; the next ingest of identical content was a hash skip). In a wing the source page is now written LAST; test fails without it (a failed batch leaves no source page, and the retry is a real ingest).Everything else the reviewers found and I chose not to fix this sprint is collected in #212 with sources. Gate:
go vet ./...,go test -race ./...exit 0, 0 FAIL, lint 0.Review history for the record: four independent agent passes (not four-eyes) at 49286ba-era heads: pass 1 BLOCK (10 findings), pass 2 BLOCK (in-batch slug clash, filter patterns, memory), pass 3 BLOCK (nothing-ingested archived as processed; title remap merging notes), pass 4 clear. Every fix has a failing-first test and was mutation-checked; several of my own tests passed under their mutant and were rebuilt. Merge is yours: control code, and I misapplied the risk lane once already in this sprint (#210).
Created via git-mcp on behalf of @mathias
Merged by me via the Gitea API with the owner token, on Mathias's explicit yes for this PR (chat, 2026-10-20: "Du mergar åt mig via API:t"). The MCP merge tool refuses above risk/low by design; this bypass is recorded here and in #212 item 15. Head
90752f2, CI green on that commit. The routing is OFF untilINGEST_WING_ROUTINGis set on the ingestion deployment (nothing in infra sets it). Next, in order: deploy,legacymovedry run shown to Mathias before-apply, then the infra change to enable routing (prepared as an issue/draft, not applied).Created via git-mcp on behalf of @mathias