4 Commits

Include renames

Author	SHA1	Message	Date
mathias	9345a0a867	feat(pipeline): report when a re-ingest disagrees with the page on disk (#104) ... The asymmetry that makes unattended ingest safe (#103) also discarded a real signal. When a fresh extraction produced a different version of a section already on disk, the on-disk version won and the model's version went in the bin without a word — so "the source is unchanged" and "the source changed its mind and we ignored it" were indistinguishable. wiki.MergeReport now returns the headings where the incoming version disagreed and lost, and the ingest run reports them as warnings, which AppendLog already writes into log.md. This is the simplified answer to contradiction flagging at ingest. The issue proposed calling brain-gardener's Judge from the ingest path, which needs either a shared Go module across two repos or an HTTP service that does not exist — and puts an LLM call on the write path, where a timeout must neither fail the ingest nor silently mark nothing as contested. None of that is necessary to get the signal. The merge already computes the disagreement; it was throwing it away. Surfacing it costs nothing, needs no model, no cue-word matcher, no cross-repo dependency, and is deterministic. Deliberately textual disagreement, not semantic contradiction. Deciding whether a disagreement is a genuine contradiction stays with the Judge-gated enrich pass that already runs hourly and is approval-gated by design (brain-gardener#175). Detection here, adjudication there. Merge is kept as a thin wrapper over MergeReport so existing callers are untouched. Refs #104, #103, brain-gardener#175, brain#21.	2026-09-23 06:51:49 +02:00
mathias	a5a328fea5	feat(pipeline): skip extraction when the source has not changed (#105) ... Ingest now stamps source_hash — sha256 of the ingested body, frontmatter excluded — onto the source page, and compares it before doing any work. An unchanged source returns Skipped with no LLM call at all. This is cost and signal, not correctness: #103 already made a re-ingest byte-identical, so a repeated run was harmless but not free — a full extraction over every chunk to produce a result that merges to nothing. Building this first would have put idempotency in the guard rather than in the write, which is the wrong dependency direction. Body-only hashing on purpose. A re-crawl that stamps a new fetch date has not changed the source, and treating that as drift would re-extract the whole corpus every time a crawler ran. A missing or unreadable hash always means "extract": a missed skip costs one extraction, a wrong skip loses an update silently, and silence is the failure mode this repo keeps paying for. The skip is reported (Result.Skipped plus a warning naming the hash), not silent — a run that does nothing must say so, or "the pipeline is fine" and "the pipeline stopped working" look identical (#107). Storing the hash required settling who owns which frontmatter field. #103 made the on-disk block win wholesale, which protects human-set metadata from a regenerating writer but froze fields the generator legitimately owns. wiki.Merge now refreshes an explicit generatorOwned allowlist — last_updated and source_hash — and keeps everything else on disk. An allowlist rather than a carve-out, so a field nobody has thought about stays human-owned by default. That also retires the last_updated limitation #103 shipped with. Refs #105, #103, #107, brain#21.	2026-09-22 23:45:25 +02:00
mathias	f450a6c84f	feat(pipeline): ingest revises the existing page instead of clobbering it (#103) ... fix(wiki): give the preamble its own field instead of overloading heading=="" (#103) Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820). Co-authored-by: Mathias <mthbqv@gmail.com>	2026-09-22 21:43:04 +00:00
mathias	ae5a4d04f0	feat(ingestion): add wiki page merge logic	2026-04-22 22:28:55 +02:00