fix(wiki): stop Merge from silently deleting on-disk content it can't parse
CI / Lint / Test / Vet (pull_request) Successful in 32s
CI / Mirror to GitHub (pull_request) Skipped

Review of PR #103 found a real data-loss bug: parseSections only captured
text between `## ` headings. Content before the first heading — an intro
paragraph, or, if the page had no headings at all, the entire body — was
dropped on the floor, not carried into any section. Merge then had
nothing of `a` to keep there, so the fresh extraction's sections silently
became the whole output wherever that content used to be. Exactly the
failure this file exists to prevent, just re-triggered by a page shape
the parser didn't expect instead of by every re-ingest.

Fixed by treating pre-first-heading content as a section with an empty
heading. Merge's switch never matches "", so it falls through to the
same on-disk-wins default every other unlisted section already gets —
no new merge path, just no longer discarding this one's input.

Second, separate finding from the same review: mergeBullets/appendNew
deduped on the raw trimmed line, so two re-extractions of the same
logical bullet with different internal whitespace (a real possibility —
the extractor is an LLM, not a fixed serializer) produced a duplicate
instead of a no-op, undermining the idempotency this file's own doc
comments claim. Fixed by comparing a whitespace-canonicalized key
(strings.Fields + single-space join) while still emitting the original
line's text.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-22 23:34:11 +02:00
co-authored by Claude Sonnet 5
parent 60a8801a1d
commit 9abe4369f2
2 changed files with 95 additions and 16 deletions
+53
View File
@@ -54,6 +54,59 @@ func TestMerge_KeepsFrontmatterFromA(t *testing.T) {
assert.NotContains(t, got.Content, "title: B")
}
// --- hyperguild#103 review: a page with content outside any `## ` section ---
// must never lose that content on merge. parseSections used to only capture
// text between `## ` headings — anything before the first heading (a common,
// ordinary shape for a hand-written note: an intro paragraph before the first
// section) or, worse, an entire page with no `## ` headings at all (a legacy
// note, a stub, any hand-edit that doesn't follow the section convention)
// vanished silently. Merge would then fill the page entirely from the fresh
// extraction — the opposite of "never silently rewrite what a human put
// there," the guarantee this whole file exists to provide.
func TestMerge_PreservesOnDiskContentWithNoSectionHeadings(t *testing.T) {
a := Page{Path: "wiki/x.md", Content: "---\ntitle: X\n---\n\nJust some free text a human wrote. No section headings at all.\n"}
b := Page{Path: "wiki/x.md", Content: "---\ntitle: X\n---\n\n## Definition\n\nFresh extraction's take.\n"}
got := Merge(a, b)
assert.Contains(t, got.Content, "Just some free text a human wrote",
"on-disk content with no ## headings must survive the merge, not be replaced wholesale by the fresh extraction")
}
func TestMerge_PreservesPreambleBeforeFirstHeading(t *testing.T) {
a := Page{Path: "wiki/x.md", Content: "---\ntitle: X\n---\n\nAn intro paragraph before the first heading.\n\n## Definition\n\nA thing.\n"}
b := Page{Path: "wiki/x.md", Content: "---\ntitle: X\n---\n\n## Definition\n\nA thing.\n"}
got := Merge(a, b)
assert.Contains(t, got.Content, "An intro paragraph before the first heading",
"text before the first ## heading is ordinary page content, not something the parser is entitled to drop")
}
// --- hyperguild#103 review: dedup must be whitespace-insensitive -----------
// Page content is produced by an LLM per run, so byte-identical re-extraction
// of "the same" logical line is not guaranteed — different bullet spacing or
// trailing whitespace on an otherwise-identical line must still dedup, or
// idempotency only holds when the extractor happens to reproduce whitespace
// exactly, which is not a real guarantee.
func TestMerge_AppendSectionDedupsAcrossWhitespaceVariants(t *testing.T) {
a := Page{Path: "p.md", Content: "---\ntitle: X\n---\n\n## Open Questions\n\n- Does this converge?\n"}
b := Page{Path: "p.md", Content: "---\ntitle: X\n---\n\n## Open Questions\n\n- Does this converge? \n"}
got := Merge(a, b)
assert.Equal(t, 1, strings.Count(got.Content, "Does this converge?"),
"same logical line with different whitespace must dedup, not duplicate")
}
func TestMerge_BulletSectionDedupsAcrossWhitespaceVariants(t *testing.T) {
a := Page{Path: "p.md", Content: "---\ntitle: X\n---\n\n## Related Concepts\n\n- [[bar|Bar]]\n"}
b := Page{Path: "p.md", Content: "---\ntitle: X\n---\n\n## Related Concepts\n\n- [[bar|Bar]]\n"}
got := Merge(a, b)
assert.Equal(t, 1, strings.Count(got.Content, "[[bar|Bar]]"),
"same logical bullet with different whitespace must dedup, not duplicate")
}
// --- brain#21/B1: merging must be safe to run again -------------------------
func TestMerge_AppendSectionDoesNotDuplicateContentAlreadyPresent(t *testing.T) {