#103 made ingest merge into the page already on disk instead of
clobbering it, but the extraction itself stayed page-blind: the model
wrote every page as if fresh, and Merge reconciled the two texts
afterwards by heading. It could restate a claim but never refine it,
and had no way to know what already existed.
BuildPrompt now accepts the source page's current on-disk content
(when one exists) and asks the model to revise it explicitly: add
what's missing, strengthen what the source now supports, leave the
rest alone and reuse its wording verbatim so unchanged claims are
recognized as unchanged. Scoped to the source page specifically —
its path (wiki/sources/<sourceSlug>.md) is deterministic before
extraction runs, unlike a concept/entity page's path, which only
exists once the model has produced a title.
The deterministic merge in wiki.Merge is unchanged and stays the
backstop: an on-disk section the model doesn't touch survives
untouched regardless of what the prompt asks for, so a model that
ignores the instruction still cannot destroy human content.
Closes hyperguild#106 (first half; heading normalization follows).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fix(wiki): give the preamble its own field instead of overloading heading=="" (#103)
Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820).
Co-authored-by: Mathias <mthbqv@gmail.com>
pipeline.Run parses source/author/published from the raw content's own
frontmatter and applies them deterministically to source-type RawPages
after LLM extraction — never relies on the LLM to copy them through.
buildFrontmatter now emits all three (source pages only) when present.
Backfill of the 46 already-affected wiki/sources/*.md notes is a
separate concern per the issue, not done here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mathiasandClaude Sonnet 5