#130 - feat(pipeline): page-aware extraction — feed the current page into the revision prompt - hyperguild

Conversation 1 Commits 3 Files Changed 5

+247 -13

3 Commits

Author	SHA1	Message	Date
mathiasandClaude Sonnet 5	28d9f40733	fix(wiki): first-occurrence wins when heading normalization collides pre-existing duplicates CI / Lint / Test / Vet (pull_request) Successful in 37s Details CI / Mirror to GitHub (pull_request) Skipped Details Review of PR #130 found a real bug in the new normalizeHeading matching: idx was built with idx[normalizeHeading(s.heading)] = i, plain map assignment while iterating secsA — last write wins. A page that already carries two on-disk sections differing only by case or whitespace (real, pre-existing corpus drift, the exact pattern #106 itself cites) never collided in idx before this normalization existed, since it keyed on the exact string. After normalization they collide, and every future update silently, deterministically routes to whichever duplicate happens to sort last in the file — the first one is frozen forever, never receiving an update again, with nothing to say so. Same bug class parseSections's own doc comment already warns about (two keys sharing one map slot means one loses lookup visibility), reintroduced here. Fixed by keeping only the first occurrence's index when a collision happens while building idx — deterministic, and matches "earliest is canonical," the same reasoning behind every other on-disk-wins default in this file. New test reproduces the exact review finding with a positional assertion (the fresh bullet must land adjacent to the first occurrence's own content, not the second) — a Contains-only version of this test would have passed even with the bug present, since nothing is literally deleted, a bullet just silently routes to the wrong of two still-present sections. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-23 07:34:49 +02:00
mathiasandClaude Sonnet 5	3564498a1b	fix(wiki): normalise heading casing/whitespace before matching in Merge CI / Lint / Test / Vet (pull_request) Successful in 36s Details CI / Mirror to GitHub (pull_request) Skipped Details Page-aware extraction (previous commit) asks the model to revise the page it's given, but nothing guarantees it reproduces a heading's exact on-disk text. Before this, Merge matched sections by exact string equality, so "Key Claims" vs "Key claims" (or trailing whitespace) silently created a parallel section instead of updating the original — the drift #106 calls out by name. Merge's idx map and the bulletSections/appendSections strategy lookup now key on normalizeHeading (trim + lowercase), so casing/whitespace drift matches the same section. The on-disk heading's original text (secsA[i].heading) is untouched and is what gets written out — the normalized form is used only to decide "is this the same section," never to rewrite a human's heading casing. Touches wiki/merge.go, which is explicitly out of casual-edit scope per today's review history on this file: justified because heading normalization is #106's own checklist item and has to hook into exactly this matching logic — there's nowhere else to put it. Scope kept minimal: the two lookup sites only, no other behavior in this file changed. bulletSectionsNorm/appendSectionsNorm exist because matching a section by its normalized key but then classifying its merge strategy by exact-string lookup would silently fall through to "keep a's version" for a case-drifted bullet/append heading — dropping b's content instead of unioning or appending it. Closes hyperguild#106. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-23 07:28:11 +02:00
mathiasandClaude Sonnet 5	9fd9a0b920	feat(pipeline): feed the current source page into the extraction prompt #103 made ingest merge into the page already on disk instead of clobbering it, but the extraction itself stayed page-blind: the model wrote every page as if fresh, and Merge reconciled the two texts afterwards by heading. It could restate a claim but never refine it, and had no way to know what already existed. BuildPrompt now accepts the source page's current on-disk content (when one exists) and asks the model to revise it explicitly: add what's missing, strengthen what the source now supports, leave the rest alone and reuse its wording verbatim so unchanged claims are recognized as unchanged. Scoped to the source page specifically — its path (wiki/sources/<sourceSlug>.md) is deterministic before extraction runs, unlike a concept/entity page's path, which only exists once the model has produced a title. The deterministic merge in wiki.Merge is unchanged and stays the backstop: an on-disk section the model doesn't touch survives untouched regardless of what the prompt asks for, so a model that ignores the instruction still cannot destroy human content. Closes hyperguild#106 (first half; heading normalization follows). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-23 07:27:57 +02:00

Author

SHA1

Message

Date

mathiasandClaude Sonnet 5

28d9f40733

fix(wiki): first-occurrence wins when heading normalization collides pre-existing duplicates

CI / Lint / Test / Vet (pull_request) Successful in 37s

Details

CI / Mirror to GitHub (pull_request) Skipped

Details

Review of PR #130 found a real bug in the new normalizeHeading matching:
idx was built with idx[normalizeHeading(s.heading)] = i, plain map
assignment while iterating secsA — last write wins. A page that already
carries two on-disk sections differing only by case or whitespace (real,
pre-existing corpus drift, the exact pattern #106 itself cites) never
collided in idx before this normalization existed, since it keyed on the
exact string. After normalization they collide, and every future update
silently, deterministically routes to whichever duplicate happens to
sort last in the file — the first one is frozen forever, never
receiving an update again, with nothing to say so. Same bug class
parseSections's own doc comment already warns about (two keys sharing
one map slot means one loses lookup visibility), reintroduced here.

Fixed by keeping only the first occurrence's index when a collision
happens while building idx — deterministic, and matches "earliest is
canonical," the same reasoning behind every other on-disk-wins default
in this file.

New test reproduces the exact review finding with a positional
assertion (the fresh bullet must land adjacent to the first
occurrence's own content, not the second) — a Contains-only version of
this test would have passed even with the bug present, since nothing is
literally deleted, a bullet just silently routes to the wrong of two
still-present sections.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-23 07:34:49 +02:00

mathiasandClaude Sonnet 5

3564498a1b

fix(wiki): normalise heading casing/whitespace before matching in Merge

CI / Lint / Test / Vet (pull_request) Successful in 36s

Details

CI / Mirror to GitHub (pull_request) Skipped

Details

Page-aware extraction (previous commit) asks the model to revise the
page it's given, but nothing guarantees it reproduces a heading's
exact on-disk text. Before this, Merge matched sections by exact
string equality, so "Key Claims" vs "Key claims" (or trailing
whitespace) silently created a parallel section instead of updating
the original — the drift #106 calls out by name.

Merge's idx map and the bulletSections/appendSections strategy lookup
now key on normalizeHeading (trim + lowercase), so casing/whitespace
drift matches the same section. The on-disk heading's original text
(secsA[i].heading) is untouched and is what gets written out — the
normalized form is used only to decide "is this the same section,"
never to rewrite a human's heading casing.

Touches wiki/merge.go, which is explicitly out of casual-edit scope
per today's review history on this file: justified because heading
normalization is #106's own checklist item and has to hook into
exactly this matching logic — there's nowhere else to put it. Scope
kept minimal: the two lookup sites only, no other behavior in this
file changed. bulletSectionsNorm/appendSectionsNorm exist because
matching a section by its normalized key but then classifying its
merge strategy by exact-string lookup would silently fall through to
"keep a's version" for a case-drifted bullet/append heading — dropping
b's content instead of unioning or appending it.

Closes hyperguild#106.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-23 07:28:11 +02:00

mathiasandClaude Sonnet 5

9fd9a0b920

feat(pipeline): feed the current source page into the extraction prompt

#103 made ingest merge into the page already on disk instead of
clobbering it, but the extraction itself stayed page-blind: the model
wrote every page as if fresh, and Merge reconciled the two texts
afterwards by heading. It could restate a claim but never refine it,
and had no way to know what already existed.

BuildPrompt now accepts the source page's current on-disk content
(when one exists) and asks the model to revise it explicitly: add
what's missing, strengthen what the source now supports, leave the
rest alone and reuse its wording verbatim so unchanged claims are
recognized as unchanged. Scoped to the source page specifically —
its path (wiki/sources/<sourceSlug>.md) is deterministic before
extraction runs, unlike a concept/entity page's path, which only
exists once the model has produced a title.

The deterministic merge in wiki.Merge is unchanged and stays the
backstop: an on-disk section the model doesn't touch survives
untouched regardless of what the prompt asks for, so a model that
ignores the instruction still cannot destroy human content.

Closes hyperguild#106 (first half; heading normalization follows).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

2026-09-23 07:27:57 +02:00

feat(pipeline): page-aware extraction — feed the current page into the revision prompt #130