Commit Graph
9 Commits
Author SHA1 Message Date
mathias 719eb8c2d6 feat(pipeline): report when a re-ingest disagrees with the page on disk (#104)
CI / Lint / Test / Vet (pull_request) Successful in 32s
CI / Mirror to GitHub (pull_request) Skipped
The asymmetry that makes unattended ingest safe (#103) also discarded a
real signal. When a fresh extraction produced a different version of a
section already on disk, the on-disk version won and the model's version
went in the bin without a word — so "the source is unchanged" and "the
source changed its mind and we ignored it" were indistinguishable.

wiki.MergeReport now returns the headings where the incoming version
disagreed and lost, and the ingest run reports them as warnings, which
AppendLog already writes into log.md.

This is the simplified answer to contradiction flagging at ingest. The
issue proposed calling brain-gardener's Judge from the ingest path, which
needs either a shared Go module across two repos or an HTTP service that
does not exist — and puts an LLM call on the write path, where a timeout
must neither fail the ingest nor silently mark nothing as contested.

None of that is necessary to get the signal. The merge already computes
the disagreement; it was throwing it away. Surfacing it costs nothing,
needs no model, no cue-word matcher, no cross-repo dependency, and is
deterministic.

Deliberately textual disagreement, not semantic contradiction. Deciding
whether a disagreement is a genuine contradiction stays with the
Judge-gated enrich pass that already runs hourly and is approval-gated by
design (brain-gardener#175). Detection here, adjudication there.

Merge is kept as a thin wrapper over MergeReport so existing callers are
untouched.

Refs #104, #103, brain-gardener#175, brain#21.
2026-09-23 07:11:53 +02:00
mathias b340f86a45 feat(pipeline): skip extraction when the source has not changed (#105) (#126)
CI / Lint / Test / Vet (push) Successful in 40s
CI / Mirror to GitHub (push) Successful in 4s
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-23 05:09:30 +00:00
mathias f450a6c84f feat(pipeline): ingest revises the existing page instead of clobbering it (#103)
CI / Lint / Test / Vet (push) Successful in 32s
CI / Mirror to GitHub (push) Successful in 5s
fix(wiki): give the preamble its own field instead of overloading heading=="" (#103)

Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820).

Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-22 21:43:04 +00:00
mathiasandClaude Sonnet 5 6ad275b505 fix(ingest): thread source/author/published frontmatter through extraction (#85)
CI / Lint / Test / Vet (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
pipeline.Run parses source/author/published from the raw content's own
frontmatter and applies them deterministically to source-type RawPages
after LLM extraction — never relies on the LLM to copy them through.
buildFrontmatter now emits all three (source pages only) when present.

Backfill of the 46 already-affected wiki/sources/*.md notes is a
separate concern per the issue, not done here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
2026-07-26 23:42:33 +02:00
mathiasandClaude Sonnet 4.6 de35d4dbb0 feat(pipeline): wire ParseRawPages+BuildPages+CanonicalizeLinks into Run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:07:33 +02:00
mathiasandClaude Sonnet 4.6 a620f6cb01 fix(pipeline): guard empty-title bridge + skip stale integration tests until task4
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 18:46:07 +02:00
mathias 3607920601 fix(lint): resolve all errcheck violations in ingestion module
cd / Build and deploy (push) Successful in 10s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-23 16:20:59 +02:00
mathiasandClaude Sonnet 4.6 04fefe8e9c fix(ingestion): wrap naked error returns and harden mustJSON helper
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:51:19 +02:00
mathiasandClaude Sonnet 4.6 103f4d90bf feat(ingestion): add pipeline orchestrator with prompt builder
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:45:19 +02:00