8 Commits

Include renames

Author	SHA1	Message	Date
mathiasandClaude Sonnet 5	f0055483a3	fix(watcher): never re-ingest AutoTunnel's own tunnel-candidates log (#88) ... Root cause was neither of the two mechanisms the issue named (CanonicalizeLinks over-canonicalizing, or a schema/prompt bias) — CanonicalizeLinks correctly resolved [[Mission]] because wiki/concepts/mission.md genuinely exists; the extraction prompt has no "mission" bias either. The real bug: watcher.processDir walks brain/raw/ and feeds every .md/.txt/.pdf file to pipeline.Run as source material to extract, with no exclusion for tunnel-candidates-*.md — AutoTunnel's own fuzzy-match human-review queue (logFuzzyCandidates writes it into that same raw/ directory). The watcher's next poll re-ingested the raw candidate log itself, and the LLM naturally surfaced "mission" as a wikilink because the log literally lists `(term: "mission")` entries in its content — a generic-word title match DetectTunnels correctly flagged as fuzzy (not auto-tunneled) but which still leaked downstream once treated as real source material. api.ListPending already excludes tunnel-candidates-* when listing raw/ for promotion; processDir gets the same exclusion. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Roq1ajWKR5f1hG5Df9wC6A	2026-07-27 14:37:50 +02:00
mathiasandClaude Sonnet 4.6	de35d4dbb0	feat(pipeline): wire ParseRawPages+BuildPages+CanonicalizeLinks into Run ... Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-23 19:07:33 +02:00
mathias	3607920601	fix(lint): resolve all errcheck violations in ingestion module	2026-04-23 16:20:59 +02:00
mathias	a6c39e8691	feat: PDF extraction and fuzzy entity resolution ... - New extract package: Text() dispatcher for .md/.txt passthrough and PDF extraction via pdftotext subprocess - wiki.Entry gains Aliases []string, loaded from YAML frontmatter - Fuzzy entity resolution in pipeline: normalizes titles (lowercase, strip articles, collapse hyphens) and matches proposed pages against existing inventory slugs and aliases to prevent proliferation - Watcher and API handler now use extract.Text() instead of os.ReadFile - Dockerfile: apk add poppler-utils in Alpine runtime stage	2026-04-23 16:03:02 +02:00
mathias	2975eadc87	feat(watcher,api): use extract.Text() for file reading — fixes PDF ingestion	2026-04-23 16:01:36 +02:00
mathiasandClaude Sonnet 4.6	6928907d79	fix(watcher): copy files instead of moving them, leave originals for Obsidian ... Files dropped into brain/raw/ are now copied to processed/ or failed/ rather than moved. A .processed or .failed marker is written next to the original so the watcher skips it on subsequent polls without deleting it. This keeps Syncthing-synced Obsidian vaults intact after ingestion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-23 14:47:50 +02:00
mathiasandClaude Sonnet 4.6	2f4b577131	fix(ingestion): address code review issues in api and watcher packages ... - Strip internal error detail from 500 responses (leak prevention) - Add path containment assertion in /write handler - Use Go 1.22 method-prefixed mux routes for automatic 405 responses - Clarify watch_interval log when watcher not yet wired - Consolidate validation tests into table-driven TestIngest_Validation - Watcher: return nil after successful quarantine to avoid double-logging - Watcher: append timestamp suffix to processed dest if file already exists Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-22 22:59:39 +02:00
mathiasandClaude Sonnet 4.6	78531bb238	feat(ingestion): add background file watcher for brain/raw/ ... Polls brain/raw/ on a configurable ticker, derives human-readable source names from filenames, runs the pipeline, and moves files to processed/YYYY-MM-DD/ on success or failed/ on error with a log.md entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-04-22 22:54:03 +02:00