Files
hyperguild/docs/raw-inbox.md
T
mathiasandClaude Sonnet 5.5 87c40fe364
CI / Lint / Test / Vet (pull_request) Successful in 1m2s
CI / Mirror to GitHub (pull_request) Skipped
fix(api,watcher): name the queue, list parked non-markdown, reserved wings, top-level-only frontmatter key; document the inbox (review of #211)
- brain_promote/brain_reject take an optional queue (inbox | needs-review); a
  name present in both is refused instead of silently acting on the top-level
  note; brain_pending labels every entry with its queue
- parked pdf/txt files are listed (they were invisible); only markdown can be
  promoted, the others are moved or rejected
- promote refuses sources/concepts/entities/wiki as a wing
- frontmatterWing reads only a top-level key inside a properly fenced block
- docs/raw-inbox.md: the contract, the three env vars, the known limits
Each behaviour has a test that fails without it (mutation-checked).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 20:34:52 +02:00

2.9 KiB

The raw/ inbox

raw/ is an inbox: only unprocessed material sits there. Syncthing shares it with the machines you drop files from. (hyperguild#205, #208, #209)

What happens to a file you drop

You drop Result
raw/clip.md (no folder, no wing:) extracted and filed as a source in wiki/web-clip/sources/; its concepts and entities update the shared wiki/concepts/ and wiki/entities/ pages; the original moves to raw/processed/<date>/, so raw/ is empty again
raw/<wing>/clip.md or wing: <wing> in the frontmatter the same, with the source in wiki/<wing>/sources/. The wing must already exist; a typo is parked, never created. This is your deliberate choice and is not second-guessed
a note whose text matches the client-content pattern, with no folder parked in raw/needs-review/, not extracted
a folder or wing: naming an unknown, reserved or non-ASCII wing, or two disagreeing signals parked in raw/needs-review/
a note over the size cap, or one that cannot be archived stays in raw/, reported once, not retried until it changes

Concepts and entities are shared across wings because their slug is the graph's identity. A source whose slug is taken gets a wing-qualified one (notes-homelab).

Parked notes

brain_pending lists them with queue: needs-review, a proposed wing and the reason. To act on one: brain_promote / brain_reject (pass queue when the same name also sits in the inbox), or move the file into raw/<wing>/ to have it extracted. Parked pdf/txt files are listed but can only be moved or rejected.

Switching it on

Routing is off by default (the retired layout is used). Set on the ingestion deployment:

Variable Meaning
INGEST_WING_ROUTING=1 enable the rules above
INGEST_DEFAULT_WING wing for unsignalled notes (default web-clip; must be a plain name, not sources/concepts/entities/wiki)
INGEST_CLIENT_BLOCK case-insensitive regexp over note text; a match parks an unsignalled note. Falls back to CLAUDE_INGEST_CLIENT_BLOCK. Matched after folding fullwidth letters, removing zero-width characters, normalising dashes and removing whitespace. It catches what it names and nothing else

A bad default or pattern stops the server at startup. A wing created while the server runs is usable without a restart.

Known limits

  • A deliberate folder into a sensitive wing places the note there, and the shared concept/entity pages then cite that source by title. Client material does not belong in the brain at all; the pattern only guards the unsignalled lane.
  • A merge into a shared page refreshes last_updated and origin to the latest ingest; the first wing's attribution is not kept.
  • A marker file (<note>.processed) is only written when a processed original cannot be removed. One at least as new as its note holds the note (and says so once); an older or future-dated one is ignored and removed.