Commit Graph
2 Commits
Author SHA1 Message Date
mathiasandClaude Opus 4.8 9bdab1c48c fix(ingest): parse docmark's SSE-framed response, not bare JSON
CI / Lint / Test / Vet (push) Successful in 16s
CI / Mirror to GitHub (push) Has been skipped
Found via a real live-cluster end-to-end test (POST /ingest-path with a real
.docx against the deployed ingestion + docmark): 'decode docmark response:
invalid character e looking for beginning of value'. docmark's Streamable-HTTP
transport frames every response as SSE (event: message\r\ndata: {...}\r\n\r\n,
Content-Type: text/event-stream) -- stateless_http=True removes the need for
an initialize handshake/session ID, it does NOT change the wire framing to
bare JSON. My original client + its own test mock both assumed bare JSON,
so the unit tests passed while the real call failed.

sseDataPayload() extracts the data: line before JSON-unmarshaling; falls back
to the raw body if no SSE framing is present (forward-compatible). Test mock
(docmark_test.go) now emits the SAME framing the live server actually sends,
confirmed by directly probing docmark's live response before writing the fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 08:53:24 +02:00
mathiasandClaude Opus 4.8 6520c2fc4b feat(ingest): route DOCX/XLSX/PPTX/images through docmark (ADR-0013)
CI / Lint / Test / Vet (push) Successful in 16s
CI / Mirror to GitHub (push) Successful in 4s
Wires the sovereign docmark MCP server (git.d-ma.be/mathias/docmark) as the
converter for formats ingestion previously couldn't handle at all -- DOCX,
XLSX, PPTX, PNG/JPG. A single self-contained tools/call POST (docmark runs
stateless_http, no session handshake needed) does the conversion.

- internal/extract/docmark.go: extractViaDocmark(path) -- reads DOCMARK_URL +
  DOCMARK_BEARER_TOKEN from env, base64-encodes the file, calls docmark's
  convert_to_markdown tool, surfaces tool/HTTP errors with context.
- internal/extract/extract.go: routes .docx/.xlsx/.pptx/.png/.jpg/.jpeg to it.
- internal/api/handler.go: isSupportedExtension() replaces the static
  supportedExtensions map -- the new extensions are gated on DOCMARK_URL being
  set, checked per-request. When DOCMARK_URL is unset (e.g. this repo's
  current deployed state), behavior is byte-for-byte unchanged from today:
  same 400/silent-skip as before. Zero risk until the env var is actually
  wired in infra.

TDD throughout: docmark.go tested via httptest mock (success, not-configured,
tool-error, HTTP-error, all 6 new extensions route correctly); handler-level
test proves the DOCMARK_URL gate (docx 400 when unset, 200 when a working
mock is configured). Full task check green (fmt/vet/govulncheck/all tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 08:46:03 +02:00