Commit Graph
27 Commits
Author SHA1 Message Date
mathiasandClaude Opus 5.5 a6da93332d feat(brain): audit-log every tools/call; server-owned write provenance (#147)
CI / Lint / Test / Vet (push) Successful in 39s
CI / Mirror to GitHub (push) Successful in 4s
Closes #144 (audit line per tools/call) and #145 (origin/written_by stamped on every model-reachable write, surfaced in brain_query/brain_get).

Passed pr-gatekeeper after four rounds of adversarial review, verified by execution: #147

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-10-01 21:06:36 +00:00
mathias 9a37881441 feat(pipeline): feed the current source page into the extraction prompt (#106) (#132)
CI / Lint / Test / Vet (push) Successful in 39s
CI / Mirror to GitHub (push) Successful in 4s
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-23 05:45:47 +00:00
mathiasandClaude Sonnet 5 cdafdf6e0f fix(wiki): surface merge conflicts as textual-diff warnings (closes #104)
CI / Lint / Test / Vet (push) Successful in 33s
CI / Mirror to GitHub (push) Successful in 4s
Textual-diff conflict detection only (semantic contradiction detection via brain-gardener Judge deferred, confirmed by Mathias, comment 12594). Includes canonicalBlock whitespace-run normalization fix for the disagreement comparison.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-23 05:22:27 +00:00
mathias b340f86a45 feat(pipeline): skip extraction when the source has not changed (#105) (#126)
CI / Lint / Test / Vet (push) Successful in 40s
CI / Mirror to GitHub (push) Successful in 4s
Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-23 05:09:30 +00:00
mathias f450a6c84f feat(pipeline): ingest revises the existing page instead of clobbering it (#103)
CI / Lint / Test / Vet (push) Successful in 32s
CI / Mirror to GitHub (push) Successful in 5s
fix(wiki): give the preamble its own field instead of overloading heading=="" (#103)

Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820).

Co-authored-by: Mathias <mthbqv@gmail.com>
2026-09-22 21:43:04 +00:00
mathiasandClaude Sonnet 5 6d014c1d0f fix(ingest): CanonicalizeLinks strips wing:/wiki/ prefix and repairs path-style links (#87)
CI / Mirror to GitHub (push) Has been skipped
CI / Lint / Test / Vet (push) Failing after 2s
plainLinkRE matches any [[...]] without a pipe, including path-prefixed
forms like [[wing:homelab/failures/foo]] or [[wiki/agentsquad/decisions/bar]]
that the LLM extraction step occasionally emits. Title-lookup against
titleToSlug always fails for these (they're paths, not titles), so they
were silently left broken.

Before falling to the unknown-wikilink warning, strip a known wing:/wiki/
root prefix and emit the clean wing/hall/slug path directly — matching
the brain-graph path-style link convention.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
2026-07-27 14:32:31 +02:00
mathiasandClaude Sonnet 5 6ad275b505 fix(ingest): thread source/author/published frontmatter through extraction (#85)
CI / Lint / Test / Vet (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
pipeline.Run parses source/author/published from the raw content's own
frontmatter and applies them deterministically to source-type RawPages
after LLM extraction — never relies on the LLM to copy them through.
buildFrontmatter now emits all three (source pages only) when present.

Backfill of the 46 already-affected wiki/sources/*.md notes is a
separate concern per the issue, not done here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
2026-07-26 23:42:33 +02:00
mathiasandClaude Opus 4.6 0a70d9e972 feat(pipeline): add POST /ingest-raw for direct batch ingestion without LLM
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Has been skipped
Allows callers to provide pre-structured RawPage data directly, bypassing the
LLM extraction step. The pipeline still handles slug computation, frontmatter,
link canonicalization, source back-references, and dedup — only the extraction
is skipped. Useful when a more capable model or manual curation produces the
structured data.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 11:15:59 +02:00
mathiasandClaude Sonnet 4.6 3e9a648115 fix(pipeline): repair invalid JSON escape sequences from LLM output before parsing
CI / Lint / Test / Vet (push) Successful in 11s
CI / Mirror to GitHub (push) Has been skipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 22:04:27 +02:00
mathiasandClaude Sonnet 4.6 923a665365 fix(pipeline): skip RawPages with empty title in BuildPages instead of producing broken paths
CI / Lint / Test / Vet (push) Successful in 9s
CI / Mirror to GitHub (push) Has been skipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:55:37 +02:00
mathiasandClaude Sonnet 4.6 537aebc302 feat(pipeline): update system prompt for new LLM JSON contract (no slugs)
- Change prompt to reflect new output format: title, type, subtype, domain, content
- Remove slug/path generation responsibility from LLM — pipeline now handles it
- Wikilinks change from [[slug|Display Name]] to [[Display Name]] only
- LLM no longer includes frontmatter or paths in output

docs(schema): update LLM output format and wikilink convention for Level 3

- Specify JSON schema: title, type, subtype, domain, content fields
- Remove frontmatter requirements from schema output (handled by pipeline)
- Simplify wikilink format to [[Display Name]] — no slug or pipe
- Pipeline now responsible for slug generation and frontmatter construction

These changes shift slug/frontmatter generation from LLM to pipeline,
reducing cognitive load on the model and improving control over output.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:45:21 +02:00
mathiasandClaude Sonnet 4.6 de35d4dbb0 feat(pipeline): wire ParseRawPages+BuildPages+CanonicalizeLinks into Run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 19:07:33 +02:00
mathias 26855f69b0 feat(pipeline): add CanonicalizeLinks — convert [[Display Name]] to [[slug|Display Name]] 2026-04-23 18:59:10 +02:00
mathias a7b363d589 fix(pipeline): quote YAML scalar fields in buildFrontmatter to prevent injection 2026-04-23 18:56:39 +02:00
mathias 7b57051af8 feat(pipeline): add BuildPages — compute slugs/paths/frontmatter from RawPage 2026-04-23 18:50:37 +02:00
mathiasandClaude Sonnet 4.6 a620f6cb01 fix(pipeline): guard empty-title bridge + skip stale integration tests until task4
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 18:46:07 +02:00
mathiasandClaude Sonnet 4.6 26b5636b43 feat(pipeline): replace ParsePages with ParseRawPages + RawPage type
Strips slug authority from the LLM. The new RawPage type carries only
{title, type, subtype, domain, content} — no paths or frontmatter.
Pipeline will derive slugs deterministically (Task 4).

pipeline.go gets a temporary bridge stub (TODO task4) to keep the
package compiling between tasks.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-23 18:41:33 +02:00
mathias 1605624668 feat(pipeline): add POST /backfill-refs endpoint to retroactively inject source back-references 2026-04-23 16:50:00 +02:00
mathias 3c2bd9268c feat(pipeline): wire source back-reference injection into Run 2026-04-23 16:36:22 +02:00
mathias 29727ec2a5 feat(pipeline): inject source back-references into concept and entity pages 2026-04-23 16:35:47 +02:00
mathias 3607920601 fix(lint): resolve all errcheck violations in ingestion module
cd / Build and deploy (push) Successful in 10s
CI / Lint / Test / Vet (push) Successful in 10s
CI / Mirror to GitHub (push) Successful in 3s
2026-04-23 16:20:59 +02:00
mathias 53e46781b1 feat(pipeline): resolve proposed pages against inventory before writing 2026-04-23 16:00:31 +02:00
mathias e9b5cc401c feat(pipeline): add fuzzy entity resolution to prevent slug proliferation 2026-04-23 15:59:36 +02:00
mathiasandClaude Sonnet 4.6 1b0706f270 chore(brain): rename CLAUDE.md to schema.md for clarity
CLAUDE.md has a specific meaning in the Claude Code ecosystem (agent
instructions). The wiki schema for the ingestion pipeline should live
in schema.md to avoid confusion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 23:06:32 +02:00
mathiasandClaude Sonnet 4.6 04fefe8e9c fix(ingestion): wrap naked error returns and harden mustJSON helper
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:51:19 +02:00
mathiasandClaude Sonnet 4.6 103f4d90bf feat(ingestion): add pipeline orchestrator with prompt builder
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 22:45:19 +02:00
mathias 9b11719481 feat(ingestion): add content chunking and LLM JSON output parser 2026-04-22 22:37:14 +02:00