mathiasandClaude Sonnet 5.5
27b1579b80
- in a wing, an ingest that writes no source page is an error: the note goes to
raw/failed with the warnings in log.md instead of leaving the inbox as
'processed' (model returned [], only invalid types, only clashing entities)
- the wing's source path is deterministic, so Resolve no longer redirects a
source onto another note's page because the model gave both the same title
('Home'); that overwrote the hash and the text
- a batch tie between concept and entity prefers the page already on disk, so
its update is not lost; the wing's own source page keeps its slug even when a
foreign page shares it
- brain_pending lists an unreadable (over-cap) note with the reason instead of
dropping it; the client filter's 4 KiB match-span limit is documented
Each has a test that fails without it (mutation-checked).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
3.9 KiB
The raw/ inbox
raw/ is an inbox: only unprocessed material sits there. Syncthing shares it
with the machines you drop files from. (hyperguild#205, #208, #209)
What happens to a file you drop
| You drop | Result |
|---|---|
raw/clip.md (no folder, no wing:) |
extracted and filed as a source in wiki/web-clip/sources/; its concepts and entities update the shared wiki/concepts/ and wiki/entities/ pages; the original moves to raw/processed/<date>/, so raw/ is empty again |
raw/<wing>/clip.md or wing: <wing> in the frontmatter |
the same, with the source in wiki/<wing>/sources/. The wing must already exist; a typo is parked, never created. This is your deliberate choice and is not second-guessed |
| a note whose text matches the client-content pattern, with no folder | parked in raw/needs-review/, not extracted |
a folder or wing: naming an unknown, reserved or non-ASCII wing, or two disagreeing signals |
parked in raw/needs-review/ |
| a note over the size cap, or one that cannot be archived | stays in raw/, reported once, not retried until it changes |
| a note the model returned nothing usable for (no source page could be written) | fails: the note is kept in raw/failed/ and the reason is in log.md; it is never archived as processed |
Concepts and entities are shared across wings because their slug is the graph's
identity. A source whose slug is taken gets a wing-qualified one (notes-homelab).
Parked notes
brain_pending lists them with queue: needs-review, a proposed wing and the
reason. To act on one: brain_promote / brain_reject (pass queue when the
same name also sits in the inbox), or move the file into raw/<wing>/ to have it
extracted. Parked pdf/txt files are listed but can only be moved or rejected.
Switching it on
Routing is off by default (the retired layout is used). Set on the ingestion deployment:
| Variable | Meaning |
|---|---|
INGEST_WING_ROUTING |
1/true enables the rules above; unset or 0/false is off; any other value stops the server at startup |
INGEST_DEFAULT_WING |
wing for unsignalled notes (default web-clip; must be a plain name, not sources/concepts/entities/wiki) |
INGEST_CLIENT_BLOCK |
case-insensitive regexp over note text; a match parks an unsignalled note. Falls back to CLAUDE_INGEST_CLIENT_BLOCK. Matched after folding fullwidth letters, removing zero-width characters, normalising dashes and removing whitespace. Patterns with spaces, \\s or \\b work (whitespace is collapsed in one view and removed in another). It catches what it names and nothing else: leetspeak, punctuation splits (a.c.m.e), encodings and homoglyphs outside a small Cyrillic/Greek table are not caught. Large notes are checked in 1 MiB chunks |
A bad default or pattern stops the server at startup. A wing created while the server runs is usable without a restart.
Known limits
-
The client filter works on 1 MiB chunks that overlap by 4 KiB. A match that can span more than 4 KiB (crafted padding between two words of a multi-word pattern, exactly at a chunk edge) is not guaranteed to be found; this affects notes over 1 MiB only. On a 64 MiB note the check takes about ten seconds.
-
A pattern that matches the empty string (
.*,x*) parks every unsignalled note. That fails closed and only the operator's own configuration can do it. -
A deliberate folder into a sensitive wing places the note there, and the shared concept/entity pages then cite that source by title. Client material does not belong in the brain at all; the pattern only guards the unsignalled lane.
-
A merge into a shared page refreshes
last_updatedandoriginto the latest ingest; the first wing's attribution is not kept. -
A marker file (
<note>.processed) is only written when a processed original cannot be removed. One at least as new as its note holds the note (and says so once); an older or future-dated one is ignored and removed.