Every wiki/sources/*.md note produced by ingestion/internal/pipeline.Run is missing source:, author:, published: frontmatter — even though the raw clipping passed in as content has them. Checked all 46 existing source notes: 0 have a source: field. Confirmed on a fresh test article ingested today (2026-07-26): meet-ornith-the-agentic-coding-model-that-runs-entirely-on-your-laptop.md's raw copy has source:/author:/published:, the generated wiki page has none of it (only title/type/domain/date_ingested/last_updated/aliases).
Result: no wiki source note is traceable back to its origin URL without manually opening the raw archive.
Where it lives
pipeline.Run (pipeline.go) → BuildPrompt(schema, source, chunk, inventory) → LLM Complete → ParseRawPages(output) → BuildPages(rawPages, sourceSlug, date). The source param passed into Run is just used as a slug basis (wiki.Slug(source)), not threaded through as structured metadata — and the extraction prompt apparently isn't told to carry source/author/published from the input's own frontmatter into its output.
Scope
Parse source/author/published out of the raw content's frontmatter before/alongside the LLM extraction call, and have BuildPages inject them into the generated wiki/sources/*.md frontmatter unchanged (don't rely on the LLM to faithfully copy them through — treat as deterministic passthrough, not an extraction task).
Acceptance criteria
Ingesting raw content with source:/author:/published: in its frontmatter produces a wiki source note carrying all three fields verbatim.
Regression test: fixture raw content with these fields → assert present in generated frontmatter.
Existing 46 files are a separate backfill concern, not in scope here (raw copies are on koala at /var/lib/supervisor/brain/raw/processed/**, cross-referenceable by title if a backfill is wanted later).
## Problem
Every `wiki/sources/*.md` note produced by `ingestion/internal/pipeline.Run` is missing `source:`, `author:`, `published:` frontmatter — even though the raw clipping passed in as `content` has them. Checked all 46 existing source notes: **0 have a `source:` field.** Confirmed on a fresh test article ingested today (2026-07-26): `meet-ornith-the-agentic-coding-model-that-runs-entirely-on-your-laptop.md`'s raw copy has `source:`/`author:`/`published:`, the generated wiki page has none of it (only `title/type/domain/date_ingested/last_updated/aliases`).
Result: no wiki source note is traceable back to its origin URL without manually opening the raw archive.
## Where it lives
`pipeline.Run` (`pipeline.go`) → `BuildPrompt(schema, source, chunk, inventory)` → LLM `Complete` → `ParseRawPages(output)` → `BuildPages(rawPages, sourceSlug, date)`. The `source` param passed into `Run` is just used as a slug basis (`wiki.Slug(source)`), not threaded through as structured metadata — and the extraction prompt apparently isn't told to carry `source`/`author`/`published` from the input's own frontmatter into its output.
## Scope
Parse `source`/`author`/`published` out of the raw content's frontmatter before/alongside the LLM extraction call, and have `BuildPages` inject them into the generated `wiki/sources/*.md` frontmatter unchanged (don't rely on the LLM to faithfully copy them through — treat as deterministic passthrough, not an extraction task).
## Acceptance criteria
- Ingesting raw content with `source:`/`author:`/`published:` in its frontmatter produces a wiki source note carrying all three fields verbatim.
- Regression test: fixture raw content with these fields → assert present in generated frontmatter.
- Existing 46 files are a separate backfill concern, not in scope here (raw copies are on koala at `/var/lib/supervisor/brain/raw/processed/**`, cross-referenceable by title if a backfill is wanted later).
Fixed in 6ad275b. pipeline.Run now parses source/author/published out of the raw content's own frontmatter (parseContentFrontmatter) and applies them deterministically to source-type RawPages post-extraction (applySourceMeta) — never relies on the LLM to copy them through (fields are json:"-" on RawPage, immune to LLM-controlled JSON). buildFrontmatter emits all three, source pages only, when present. Regression tests added at both the BuildPages and Run level. Full go test ./... green.
Backfill of the 46 existing notes left out of scope per the issue — raw copies are on koala at /var/lib/supervisor/brain/raw/processed/** if you want that done separately.
Fixed in 6ad275b. `pipeline.Run` now parses `source`/`author`/`published` out of the raw content's own frontmatter (`parseContentFrontmatter`) and applies them deterministically to source-type `RawPage`s post-extraction (`applySourceMeta`) — never relies on the LLM to copy them through (fields are `json:"-"` on `RawPage`, immune to LLM-controlled JSON). `buildFrontmatter` emits all three, source pages only, when present. Regression tests added at both the `BuildPages` and `Run` level. Full `go test ./...` green.
Backfill of the 46 existing notes left out of scope per the issue — raw copies are on koala at `/var/lib/supervisor/brain/raw/processed/**` if you want that done separately.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Problem
Every
wiki/sources/*.mdnote produced byingestion/internal/pipeline.Runis missingsource:,author:,published:frontmatter — even though the raw clipping passed in ascontenthas them. Checked all 46 existing source notes: 0 have asource:field. Confirmed on a fresh test article ingested today (2026-07-26):meet-ornith-the-agentic-coding-model-that-runs-entirely-on-your-laptop.md's raw copy hassource:/author:/published:, the generated wiki page has none of it (onlytitle/type/domain/date_ingested/last_updated/aliases).Result: no wiki source note is traceable back to its origin URL without manually opening the raw archive.
Where it lives
pipeline.Run(pipeline.go) →BuildPrompt(schema, source, chunk, inventory)→ LLMComplete→ParseRawPages(output)→BuildPages(rawPages, sourceSlug, date). Thesourceparam passed intoRunis just used as a slug basis (wiki.Slug(source)), not threaded through as structured metadata — and the extraction prompt apparently isn't told to carrysource/author/publishedfrom the input's own frontmatter into its output.Scope
Parse
source/author/publishedout of the raw content's frontmatter before/alongside the LLM extraction call, and haveBuildPagesinject them into the generatedwiki/sources/*.mdfrontmatter unchanged (don't rely on the LLM to faithfully copy them through — treat as deterministic passthrough, not an extraction task).Acceptance criteria
source:/author:/published:in its frontmatter produces a wiki source note carrying all three fields verbatim./var/lib/supervisor/brain/raw/processed/**, cross-referenceable by title if a backfill is wanted later).Fixed in
6ad275b.pipeline.Runnow parsessource/author/publishedout of the raw content's own frontmatter (parseContentFrontmatter) and applies them deterministically to source-typeRawPages post-extraction (applySourceMeta) — never relies on the LLM to copy them through (fields arejson:"-"onRawPage, immune to LLM-controlled JSON).buildFrontmatteremits all three, source pages only, when present. Regression tests added at both theBuildPagesandRunlevel. Fullgo test ./...green.Backfill of the 46 existing notes left out of scope per the issue — raw copies are on koala at
/var/lib/supervisor/brain/raw/processed/**if you want that done separately.