4 Commits

Include renames

Author	SHA1	Message	Date
mathiasandClaude Opus 5.5	87c80e4e82	fix(ingestion): close the PR #156 review findings on raw/ reads ... - Reverting the ingest call sites to extract.Text(path) stayed green; a test now fails if anything outside internal/extract calls extract.Text( (only TextIn, through an os.Root, may be used). - The watcher symlink test passed for a watcher that ingested nothing, or stopped at the first symlink (SkipAll: one synced link halting all ingestion). It now plants a real note after the symlink and asserts exactly one ingest, and the symlinked-raw refusal has its own test. - A FIFO swapped in between Lstat and Open hung open(2) forever. extract.OpenRegular opens non-blocking and checks the type on the handle; readRawNote, TextIn and the watcher's archive copy use it. - The symlinked-root test covers pending, promote, reject, resolve and ReadIngestFile, and is named for exactly those. - The hard-link residual is documented where it is accepted. Out of scope, filed: #157 (symlinks elsewhere in the Syncthing-shared brain dir reach search and other readers), #158 (writes under raw/ follow symlinks). Refs #153 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-03 16:59:08 +02:00
mathiasandClaude Opus 5.5	8a91feb02b	fix(ingestion): every brain/raw/ reader goes through an os.Root; symlinks are inert ... raw/ is writable via Syncthing (a sendreceive folder shared with flamingo, symlinks synced) and by the brain CronJobs, and several readers followed symlinks out of it: - /pending and brain_pending showed a symlink target's contents; brain_promote would copy it into wiki/ (then brain-sync to Gitea); brain_reject would copy it into raw/rejected/. - The watcher, which needs no API call at all, ingested symlinked files (WalkDir yields them; extract.Text followed them). Not in the issue; found while fixing it. - Ingest checked a path then reopened it by name (TOCTOU). OpenRawRoot opens brain/raw as an os.Root and refuses a raw/ that is itself a symlink. ListPending, PromoteNote and RejectNote read through it and refuse anything not a regular file. extract.TextIn reads through a root (temp copy for pdf/docmark); /ingest-path, brain_ingest and the watcher use it, and the watcher archives via the root too, so a file swapped for a symlink after a check still cannot lead out. Refs #153 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-03 16:47:43 +02:00
mathiasandClaude Opus 4.8	6520c2fc4b	feat(ingest): route DOCX/XLSX/PPTX/images through docmark (ADR-0013) ... Wires the sovereign docmark MCP server (git.d-ma.be/mathias/docmark) as the converter for formats ingestion previously couldn't handle at all -- DOCX, XLSX, PPTX, PNG/JPG. A single self-contained tools/call POST (docmark runs stateless_http, no session handshake needed) does the conversion. - internal/extract/docmark.go: extractViaDocmark(path) -- reads DOCMARK_URL + DOCMARK_BEARER_TOKEN from env, base64-encodes the file, calls docmark's convert_to_markdown tool, surfaces tool/HTTP errors with context. - internal/extract/extract.go: routes .docx/.xlsx/.pptx/.png/.jpg/.jpeg to it. - internal/api/handler.go: isSupportedExtension() replaces the static supportedExtensions map -- the new extensions are gated on DOCMARK_URL being set, checked per-request. When DOCMARK_URL is unset (e.g. this repo's current deployed state), behavior is byte-for-byte unchanged from today: same 400/silent-skip as before. Zero risk until the env var is actually wired in infra. TDD throughout: docmark.go tested via httptest mock (success, not-configured, tool-error, HTTP-error, all 6 new extensions route correctly); handler-level test proves the DOCMARK_URL gate (docx 400 when unset, 200 when a working mock is configured). Full task check green (fmt/vet/govulncheck/all tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-09 08:46:03 +02:00
mathias	43a46d07e5	feat(extract): add Text() dispatcher with md/txt passthrough	2026-04-23 15:45:20 +02:00