Review round 1 on PR #195:
- A refused marker or archive write left no marker, so the note was
reprocessed every interval (archive copies piling up; an LLM call per
round on the failure path). The marker check now uses Lstat (any entry
at the marker path counts, so a dangling marker link skips the note
instead of looping), and a note whose archive copy or marker write is
refused is remembered (path, size, mtime) and not retried until it
changes.
- cleanRel no longer refuses a backslash: it is a legal filename byte on
Linux, not a traversal, and a raw note named with one was never
archived or marked.
- Tests now reach every converted writer: the log.md append (real
raw/failed so the log is reached, symlinked log.md), a dangling marker
link, the refused-retry loop with a positive control, a backslash name,
a buildAndWrite swap race, and the claude-session archive with in-brain
and outside symlinked archive dirs. The stale lint allowlist entry for
appendWatcherLog is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found via a real live-cluster end-to-end test (POST /ingest-path with a real
.docx against the deployed ingestion + docmark): 'decode docmark response:
invalid character e looking for beginning of value'. docmark's Streamable-HTTP
transport frames every response as SSE (event: message\r\ndata: {...}\r\n\r\n,
Content-Type: text/event-stream) -- stateless_http=True removes the need for
an initialize handshake/session ID, it does NOT change the wire framing to
bare JSON. My original client + its own test mock both assumed bare JSON,
so the unit tests passed while the real call failed.
sseDataPayload() extracts the data: line before JSON-unmarshaling; falls back
to the raw body if no SSE framing is present (forward-compatible). Test mock
(docmark_test.go) now emits the SAME framing the live server actually sends,
confirmed by directly probing docmark's live response before writing the fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires the sovereign docmark MCP server (git.d-ma.be/mathias/docmark) as the
converter for formats ingestion previously couldn't handle at all -- DOCX,
XLSX, PPTX, PNG/JPG. A single self-contained tools/call POST (docmark runs
stateless_http, no session handshake needed) does the conversion.
- internal/extract/docmark.go: extractViaDocmark(path) -- reads DOCMARK_URL +
DOCMARK_BEARER_TOKEN from env, base64-encodes the file, calls docmark's
convert_to_markdown tool, surfaces tool/HTTP errors with context.
- internal/extract/extract.go: routes .docx/.xlsx/.pptx/.png/.jpg/.jpeg to it.
- internal/api/handler.go: isSupportedExtension() replaces the static
supportedExtensions map -- the new extensions are gated on DOCMARK_URL being
set, checked per-request. When DOCMARK_URL is unset (e.g. this repo's
current deployed state), behavior is byte-for-byte unchanged from today:
same 400/silent-skip as before. Zero risk until the env var is actually
wired in infra.
TDD throughout: docmark.go tested via httptest mock (success, not-configured,
tool-error, HTTP-error, all 6 new extensions route correctly); handler-level
test proves the DOCMARK_URL gate (docx 400 when unset, 200 when a working
mock is configured). Full task check green (fmt/vet/govulncheck/all tests).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mathiasandClaude Opus 5.5