The ingest pipeline read wiki/sources/<slug>.md and schema.md (both go
into the LLM prompt), the pages it revises, and the pages it adds
source refs to, with os.ReadFile; BackfillRefs listed wiki/sources with
os.ReadDir. A symlinked file's content reached the prompt and the
written pages (1.6M leaks in the race), and a FIFO hung the ingest.
classification.Load (the I1 sovereignty levels) read
classification.yaml the same way and hung startup on a FIFO.
- pipeline: readSourcePage, loadSchema, reviseOrCreate, injectSourceRefs
read through readBrainFile (brainfs ReadRegular); reviseOrCreate takes
the brain dir and reads page.Path, a missing page is still created and
a non-regular one is an error. BackfillRefs lists with ReadDirRegular
and reads with ReadRegular.
- classification.Load: read through brainfs; a symlinked or special
classification.yaml is a hard error (as a malformed one is); an absent
file or brain dir still yields the default config.
Tests use in-brain targets and the [[slug|title]] link form the pipeline
actually follows (bare [[slug]] links are ignored by it, which made a
first draft of the backfill test vacuous).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes#144 (audit line per tools/call) and #145 (origin/written_by stamped on every model-reachable write, surfaced in brain_query/brain_get).
Passed pr-gatekeeper after four rounds of adversarial review, verified by execution: #147
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
Textual-diff conflict detection only (semantic contradiction detection via brain-gardener Judge deferred, confirmed by Mathias, comment 12594). Includes canonicalBlock whitespace-run normalization fix for the disagreement comparison.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
fix(wiki): give the preamble its own field instead of overloading heading=="" (#103)
Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820).
Co-authored-by: Mathias <mthbqv@gmail.com>
pipeline.Run parses source/author/published from the raw content's own
frontmatter and applies them deterministically to source-type RawPages
after LLM extraction — never relies on the LLM to copy them through.
buildFrontmatter now emits all three (source pages only) when present.
Backfill of the 46 already-affected wiki/sources/*.md notes is a
separate concern per the issue, not done here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
Allows callers to provide pre-structured RawPage data directly, bypassing the
LLM extraction step. The pipeline still handles slug computation, frontmatter,
link canonicalization, source back-references, and dedup — only the extraction
is skipped. Useful when a more capable model or manual curation produces the
structured data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Strips slug authority from the LLM. The new RawPage type carries only
{title, type, subtype, domain, content} — no paths or frontmatter.
Pipeline will derive slugs deterministically (Task 4).
pipeline.go gets a temporary bridge stub (TODO task4) to keep the
package compiling between tasks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CLAUDE.md has a specific meaning in the Claude Code ecosystem (agent
instructions). The wiki schema for the ingestion pipeline should live
in schema.md to avoid confusion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mathiasandClaude Opus 5.5