The source page carries source_hash, which the drift check trusts. Written first,
a batch that failed midway left a hash behind, so re-ingesting the same content
was skipped and the rest of the batch was lost for good. In a wing it is now
written last. An over-cap note in brain_pending says promote/reject cannot act
on it and to delete or move it by hand. Mutation-checked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- in a wing, an ingest that writes no source page is an error: the note goes to
raw/failed with the warnings in log.md instead of leaving the inbox as
'processed' (model returned [], only invalid types, only clashing entities)
- the wing's source path is deterministic, so Resolve no longer redirects a
source onto another note's page because the model gave both the same title
('Home'); that overwrote the hash and the text
- a batch tie between concept and entity prefers the page already on disk, so
its update is not lost; the wing's own source page keeps its slug even when a
foreign page shares it
- brain_pending lists an unreadable (over-cap) note with the reason instead of
dropping it; the client filter's 4 KiB match-span limit is documented
Each has a test that fails without it (mutation-checked).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dropSlugClashes only compared against what was on disk, so a clip named like an
entity the model extracted wrote both pages and neither could be updated again.
In-batch winners: source over entity over concept. Against the brain, only
entities/concepts are skipped; the source's slug was already chosen free or its
own by resolveSourceSlug, so an existing bad pair no longer locks the source out.
Mutation-checked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- a source whose slug is taken by anything but this wing's own page gets a
wing-qualified, stable slug (talk-homelab) instead of failing the ingest
- an extracted entity/concept whose slug belongs to a different page is skipped
with a warning, so one page can no longer lock a source out of re-ingest
- in a wing only source/concept/entity are valid page types; the model (and so
the clip) can no longer choose wiki/<type>/ directories
Mutation-checked; stable-slug choice pinned directly because title resolution
masks it at the Run level.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
FindSlug returned the first hit, so a wing's own page of a name masked another
page of the same name (the retired dirs are searched before the wings, and wings
alphabetically). FindSlugs returns every match and callers refuse any that is
not their own. Found by the legacy-move planner's own test.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
With Config.Wing a source goes to wiki/<wing>/sources/ (wing/hall frontmatter);
concepts and entities stay in the shared wiki/concepts and wiki/entities, so an
entity cited from two wings is ONE page with both sources (the per-wing facts/
layout duplicated it). A source slug already used anywhere but its own wing's
sources/ is refused (slug is the graph identity). wiki.FindSlug added.
Mutation-checked: collision guard, shared inventory, per-wing source path.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reconciled with v0.25.0 semantics:
- markers are only the fallback when the original cannot be removed, and are
now written through the brain root; the refused set (size+mtime keyed) replaces
the path-only 'stuck' set
- #195's tests updated for inbox zero (archive written, original retired, no
marker), and the planted-marker-link test keeps its point: nothing is ever
written through the link, and the note is processed once
- the 'cannot retire' test now counts archive copies; it counted LLM calls,
which pipeline.Run's source-hash check hides (mutation: removing the
remember step left it green)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sources -> wiki/<wing>/sources, concepts/entities -> wiki/<wing>/facts with
kind: frontmatter; inventory, title resolution and back-references are
confined to the target wing. Empty Wing keeps the retired layout unchanged.
Mutation-checked: widening the inventory beyond the wing fails the
cross-wing back-reference test; a non-plain wing name is rejected.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Review round 2 on PR #195: the marker writes had no test, so reverting
them to by-path writes, or the refusal to log-only, stayed green. The
link is planted from inside the fake model call, between the walk's
Lstat and the marker write, for both markers and both an outside and
an in-brain relative target; red on all four flips. The marker-entry
skip is now checked for .failed too.
Also pins cleanRel's acceptance of a backslash as a filename byte in
brainfs itself, and stops the buildAndWrite control claiming writes it
does not count.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 1 on PR #195:
- A refused marker or archive write left no marker, so the note was
reprocessed every interval (archive copies piling up; an LLM call per
round on the failure path). The marker check now uses Lstat (any entry
at the marker path counts, so a dangling marker link skips the note
instead of looping), and a note whose archive copy or marker write is
refused is remembered (path, size, mtime) and not retried until it
changes.
- cleanRel no longer refuses a backslash: it is a legal filename byte on
Linux, not a traversal, and a raw note named with one was never
archived or marked.
- Tests now reach every converted writer: the log.md append (real
raw/failed so the log is reached, symlinked log.md), a dangling marker
link, the refused-retry loop with a positive control, a backslash name,
a buildAndWrite swap race, and the claude-session archive with in-brain
and outside symlinked archive dirs. The stale lint allowlist entry for
appendWatcherLog is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The remaining writers created or rewrote brain files by path:
- pipeline buildAndWrite: os.MkdirAll + os.WriteFile of every page.
- BackfillRefs: os.WriteFile of each concept page; a page swapped for a
symlink after the read was written through (the race test overwrote an
in-brain target before this change).
- watcher: the raw/processed and raw/failed archive copies (os.Create),
the .processed/.failed markers and the log.md entry (os.OpenFile).
- cmd/server: the claude-session archive.
All now go through brainfs PutRegular/AppendRegular (new package-level
PutFile/AppendFile for one-off writers: the watcher's call-site lint bans
.Open calls in that file so raw/ reads stay on OpenRegular).
Stacked on #189 (needs PutRegular/AppendRegular).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ingest pipeline readers (readSourcePage and loadSchema feed the LLM prompt; reviseOrCreate; injectSourceRefs; BackfillRefs list and reads) and classification.Load read through brainfs: symlinked or special files no longer reach the prompt, written pages or the I1 sovereignty config, and a FIFO no longer hangs ingest or startup. A symlinked/special classification.yaml is a hard error; an absent one still yields defaults.
Refs #157. Adversarial review: BLOCKED, BLOCKED, PASS. Follow-ups: #158 (write paths), #179 (FIFO tripwire).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
Closes#144 (audit line per tools/call) and #145 (origin/written_by stamped on every model-reachable write, surfaced in brain_query/brain_get).
Passed pr-gatekeeper after four rounds of adversarial review, verified by execution: #147
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
Textual-diff conflict detection only (semantic contradiction detection via brain-gardener Judge deferred, confirmed by Mathias, comment 12594). Includes canonicalBlock whitespace-run normalization fix for the disagreement comparison.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Mathias <mthbqv@gmail.com>
fix(wiki): give the preamble its own field instead of overloading heading=="" (#103)
Round 3 review: verified structurally that the preamble is no longer in the sections slice or the idx map, so it cannot collide with a degenerate `## ` heading. Flip-tested by reverting to the heading=="" model and confirming TestMerge_DegenerateEmptyHeadingDoesNotCollideWithPreamble goes red. Adversarial pass (multiple degenerate headings, frontmatter-only page, unicode around the ## boundary, 3x self-merge idempotency with preamble+degenerate) all held. CI green on 09aeaa9 (run 7820).
Co-authored-by: Mathias <mthbqv@gmail.com>
plainLinkRE matches any [[...]] without a pipe, including path-prefixed
forms like [[wing:homelab/failures/foo]] or [[wiki/agentsquad/decisions/bar]]
that the LLM extraction step occasionally emits. Title-lookup against
titleToSlug always fails for these (they're paths, not titles), so they
were silently left broken.
Before falling to the unknown-wikilink warning, strip a known wing:/wiki/
root prefix and emit the clean wing/hall/slug path directly — matching
the brain-graph path-style link convention.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
pipeline.Run parses source/author/published from the raw content's own
frontmatter and applies them deterministically to source-type RawPages
after LLM extraction — never relies on the LLM to copy them through.
buildFrontmatter now emits all three (source pages only) when present.
Backfill of the 46 already-affected wiki/sources/*.md notes is a
separate concern per the issue, not done here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A
Allows callers to provide pre-structured RawPage data directly, bypassing the
LLM extraction step. The pipeline still handles slug computation, frontmatter,
link canonicalization, source back-references, and dedup — only the extraction
is skipped. Useful when a more capable model or manual curation produces the
structured data.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Change prompt to reflect new output format: title, type, subtype, domain, content
- Remove slug/path generation responsibility from LLM — pipeline now handles it
- Wikilinks change from [[slug|Display Name]] to [[Display Name]] only
- LLM no longer includes frontmatter or paths in output
docs(schema): update LLM output format and wikilink convention for Level 3
- Specify JSON schema: title, type, subtype, domain, content fields
- Remove frontmatter requirements from schema output (handled by pipeline)
- Simplify wikilink format to [[Display Name]] — no slug or pipe
- Pipeline now responsible for slug generation and frontmatter construction
These changes shift slug/frontmatter generation from LLM to pipeline,
reducing cognitive load on the model and improving control over output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Strips slug authority from the LLM. The new RawPage type carries only
{title, type, subtype, domain, content} — no paths or frontmatter.
Pipeline will derive slugs deterministically (Task 4).
pipeline.go gets a temporary bridge stub (TODO task4) to keep the
package compiling between tasks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CLAUDE.md has a specific meaning in the Claude Code ecosystem (agent
instructions). The wiki schema for the ingestion pipeline should live
in schema.md to avoid confusion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds prompt.go (BuildPrompt + systemPrompt) and pipeline.go (Run, Config,
Result, mergeAll) that wire chunking, LLM calls, parse, merge, index rebuild,
and log append into a single ingestion pipeline. Includes integration tests
covering write, dry-run, and duplicate-path merge scenarios.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mathiasandClaude Sonnet 5.5