diff --git a/CLAUDE.md b/CLAUDE.md index cf4a939..bc1a8cd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,8 +50,11 @@ These caused real mistakes that were caught and corrected; the corrections are l - **No global cross-tenant *video* table** — videos stay per-user (data-model). Transcripts ARE shared since ADR-021 (public caption content, keyed by `(provider, provider_video_id)`, non-RLS) so re-analysis never re-fetches; the *videos* half of cross-tenant dedup stays a Future C concern. -- **No audio-download + speech-to-text in the core path** — captions-first (ADR-007). STT is a - deferred, bounded optional component. +- **No YouTube audio-download + speech-to-text in the core path** — captions-first (ADR-007). + That rejection stands. ADR-033 superseded the STT deferral **for user-supplied audio only**: + an authenticated user may upload a recording and get a *private* transcript (own RLS table, + never the shared `transcripts`). It is off by default (`TAPIR_UPLOADS_ENABLED`) and does not + open the door to scraping audio. - **No public SaaS / sign-up / billing / Google OAuth verification at scale** — Future C, deferred behind the Stage 0 gate (ADR-008). @@ -97,7 +100,7 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes ( - Adapters present under `internal/adapters/`: `youtube` (captions-first `VideoSource`, timedtext/InnerTube acquisition per ADR-010), `summarizer` + `llm` (the copied AI router, now a resilient endpoint chain — local primary → local fallback → external worst-case, - parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–015), + parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–019), `secrets` (file-backed `SecretStore`). The brain HTTP sink (ADR-005) is the remaining optional sink. - Stage 1 is open (ADR-012): multi-user with **DB-enforced** isolation — Postgres RLS `FORCE`d @@ -109,6 +112,28 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes ( (`TestTranscriptsTableIsSharedNotRLS`). The engine reads stored transcripts before any caption fetch (`usecase.resolveTranscript`), so re-analysis — re-summarize, paste-a-URL, onboarding burst — never re-touches YouTube. Per-user summaries/videos stay RLS-scoped. +- Audio upload (ADR-033, migration 017): `internal/uploads` (validation by **probing**, intake, + one-at-a-time in-process worker), `internal/adapters/audio` (ffprobe/ffmpeg), and + `internal/adapters/transcribe` (gateway client + iguana→berget chain). Transcripts are private + in `uploaded_transcripts`; the audio is never retained; chat reuses `chat.Service`. The + isolation test pins the **exact set** of non-RLS tables (`TestRowSecurityOffTablesAreExactlyTheKnownSet`). + The runtime image does not yet contain ffmpeg — that is a pending decision recorded in ADR-033. + Two-engine reconciliation (ADR-034, migration 019, off unless `TAPIR_RECONCILE_BASE_MODEL`): + `internal/reconcile` (word diff, disputes, a local-model adjudicator) and + `transcribe.Ensemble`; every model must be a local alias, enforced at start-up. + Progress and logging (ADR-035, migration 020): `internal/progress.Tracker` is the ONE writer of a + job's phases (saved to the row for the page, logged by the same call); engines and the reconciler + report through `progress.From(ctx)`, so `uploads.Transcriber` is unchanged, and a transcriber may + implement `uploads.Planner` to declare its own phases. The page tells running / quiet / lost from + two timestamps (`alive_at` heartbeat vs `progress_at`) and never shows a percentage that does not + come from real counts. **Nothing logged may carry transcript text, words from it, or the spool path.** + Errors reach a log, a row or a page as `errkind.Of(err)` (a kind such as `timeout` or `http_502`), + never `err.Error()`: error text can hold a provider's response body, the spool path or transcript + words (ADR-035 decision 8). Local-only for anything that carries transcript text: `config.IsLocalModel`. + `TAPIR_SHOT_DIR=… go test -run TestRenderUploadProgressStatesForReview ./internal/web` renders every + state for a screenshot review. + **Lint caveat:** `task check` does not fail on lint (#41: the Taskfile `|| echo` swallows + findings) — run `golangci-lint run ./...` yourself until that is fixed. - `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch watch→summarize), `serve` (the HTMX+Templ web reader/writer under `internal/web`, a new transport over the unchanged engine/ports — ADR-003). `tapir env` prints config. diff --git a/DECISIONS.md b/DECISIONS.md index 44eb2b2..dae1596 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -123,7 +123,9 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path. ## ADR-007 — Captions-first; audio-download + speech-to-text deferred **Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API -`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand. +`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand +**for YouTube**. STT deferral superseded **for user-supplied audio only** by ADR-033; the +YouTube audio-download rejection is unchanged. **Context.** YouTube's Data API does not expose transcripts. Options: official captions (clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey, @@ -1314,6 +1316,480 @@ roll back. No schema, no structural change. --- +## ADR-033 — Upload audio, get a private Swedish transcript, discuss it (user-supplied STT) + +**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #33 (audio-only subset of +#28). **Supersedes ADR-007's speech-to-text deferral for user-supplied audio only.** The +YouTube audio-download rejection in ADR-007 **stands untouched**. Builds on ADR-012 (isolation), +ADR-018 (single replica), ADR-021 (**the constraint**), ADR-022 (endpoint chain), ADR-025 +(honest status), ADR-027 (chat). ADR-014 item 4 ("Whisper reconsideration is a future ADR, +gated on caption data") is a different question — whether to transcribe YouTube videos with no +captions — and this ADR does not answer it. Measurements: `docs/research/audio-length-ceiling-2026-10-01.md`. + +**Context.** The maintainer has a concrete need: a Swedish Teams voice-only export, 30–90 +minutes, to upload and then *discuss* (what was decided, what is open, who owes what). Every +piece exists — a Whisper endpoint on the gateway (infra#274), chat-with-transcript (ADR-027) — +and nothing connects them. + +ADR-007 deferred STT for three reasons: yt-dlp fragility, ToS-grey scraping, and GPU contention +with the JEPA PoC. Upload-STT shares **none** of them: the user owns the file, nothing is +scraped, there is no caption rate gate, and the contention moved from koala to iguana. What it +does share is landing Whisper in the codebase, which ADR-007 deferred — so a new ADR is required, +and ADR-017's process note records the one time a guardrail-reversing change shipped without one. + +**The constraint that shapes everything.** ADR-021 made `transcripts` a shared, **non-RLS** +table on the explicit grounds that it holds only public caption content and nothing +user-identifying. A recorded internal meeting is the opposite: private third-party content owned +by one user. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse +would leak across tenants **and pass CI green**. + +**What was measured, not argued** (research doc above): +- **No transcription ceiling up to 90 minutes.** One 174 MB request through the LiteLLM NodePort + took 440 s (~12× realtime). The first failure is the public ingress `llm-api.d-ma.be` + returning `413` above roughly 8–10 MB (~5 min of WAV) — not whisper, and not a Metal OOM. + The 8:20 failure recorded in #33 is consistent with that cap. +- **`response_format=srt` survives LiteLLM, wrapped** in a JSON envelope (`{"text": ""}`). +- **The two backends differ.** `iguana/kb-whisper` returns whole-second segments; + `berget/kb-whisper` returns sub-second times and a different `language` token. +- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.** + #33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does + not retrieve; it stuffs the transcript into the prompt and cuts the tail. Measured against + the offered models: `phi4-mini` 400s at ~13k tokens; `gemma4-26b` is right to 90,000 chars and + silently drops the head at 120,000. + +**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on +iguana; and the Metal OOM, deliberately not provoked. (Whether the offered chat models can hold +an hour of Swedish *was* an unknown here; it is now measured, see decision 8.) + +**Decision.** + +1. **Audio is what is used; the transcript is the only thing kept.** An authenticated user + uploads a recording; Tapir transcribes its audio and stores the transcript. The audio itself + is deleted after transcription (temp file, removed in a `defer`, swept at start-up) — no + custody problem to solve, and nothing to hand to #31. **A file that also carries a video + track is accepted: its audio is extracted, the video is discarded, and the user is told so** + (see 4). *Amended 2026-10-01, the same day this ADR was drafted: the first draft rejected + video and named #28. The maintainer's first real recording — a Teams meeting with the camera + off — is still an `.mp4` with a video track, so rejecting video rejected the primary use + case.* This keeps no video and analyses none: playback, keyframes and what-is-shown analysis + remain #28. YouTube audio-download stays rejected. No public/anonymous upload (ADR-008). + +2. **Private store: a separate RLS-scoped table, `uploaded_transcripts`** (resolves #28's + decision 1). `user_id NOT NULL REFERENCES users ON DELETE CASCADE`, `ENABLE` + `FORCE ROW LEVEL + SECURITY`, the same `tapir.current_user_id` policy as every other user-owned table. It carries + the transcript text and its timestamped segments (re-transcribing 90 minutes costs ~7 + minutes; discarding the timestamps is irreversible), `status`, the `endpoint` that served + it, `audio_seconds`, and a typed `error_kind`. **Not** a `user_id` column on `transcripts` + with a partial policy, even though that would reuse more code: ADR-021's exception is + precisely that *that table holds no private row*, the isolation test asserts it has no + `user_id`, and a partial policy would turn "single exception" into "single exception, with a + private lane inside it" — the shape under which a future query drops the policy clause and + leaks. Two tables keep both boundaries independently assertable. + +3. **The isolation test asserts both boundaries and the set.** (a) User A cannot read, update or + delete user B's uploaded transcript; no GUC set → deny-all. (b) `transcripts` is still + non-RLS with no `user_id`. (c) **The set of public-schema tables with row security off is + asserted exactly** — so a new table added without RLS fails CI instead of passing it. (c) is + new; the existing test only checks the named tables. + +4. **Validate by probing, reject clearly, and say what was ignored.** Format, duration and + what a file contains cannot be read from a filename or MIME type — a Teams recording is an + `.mp4` whether or not the camera was on. The upload is probed (`ffprobe`): **accept any + container with an audio stream, whether or not it also has a video stream** (an mp3's cover + art counts as neither). Whether a real video track was present is recorded on the row + (`had_video`) and the transcript page says so plainly — *this file contained a video track; + only its audio was used and the video was discarded* — so the user is never left to wonder + what happened to the picture. Reject a file with **no** audio stream (including a screen + recording with no microphone) and an unreadable one. Limits, all configurable, all rejected + synchronously with a specific message rather than by timeout: **200 MB** upload and + **90 minutes** duration. 90 minutes is *the longest length measured*, not a ceiling that was + found; raising it is a measurement, not an edit. The size cap now matters more: a video file + is much larger than its audio and is spooled whole before the audio is extracted, so a long + recording with screen sharing can exceed 200 MB while its audio is a few tens of MB. + +5. **Transcode server-side, always, to 16 kHz mono PCM WAV** — even when the input already is, + rather than assuming. whisper-server's `--convert` is rejected (its `--tmp-dir` is created at + process start and rots under macOS periodic cleanup). The transcoder sits behind a port; the + tests use a fake. **This puts `ffmpeg` and `ffprobe` in the runtime image**, which is + distroless-static today. **Provisional: copy static ffmpeg/ffprobe binaries into the existing + distroless image** (no shell, no package manager retained). This is a heavy new dependency + and is **the maintainer's call** — see Open items. + +6. **Transcription adapter via the gateway, never the public hostname.** A `Transcriber` port; + the implementation calls `POST {TAPIR_GATEWAY_URL}/audio/transcriptions` where the gateway + URL is the in-cluster NodePort (`koala:30401/v1`). Primary `iguana/kb-whisper`, fallback + `berget/kb-whisper` (immune to iguana contention, which can wedge permanently — infra#273). + - **Fallback triggers on any primary failure** (transport error, timeout, non-2xx). It never + branches on the body: `{"error":"failed to process audio"}` has had at least five distinct + causes. + - **`response_format=verbose_json`**, normalised to one segment shape: times clamped to the + audio duration (the final segment can end past it), one language token. Not `srt`: the + envelope adds a parse step and iguana's segment timing is the same either way. + - **No chunking in v1.** Nothing failed to 90 minutes (measured). Chunk-and-stitch is the + answer *if* the cap is raised or smaller requests become necessary — see triggers. + - **The serving endpoint is recorded per row**, and surfaced, because the two backends + differ in timestamp granularity: which one served a transcript is a quality fact. + - **Berget is a cloud endpoint.** On fallback the audio of a private meeting leaves the + homelab. #33 asks for this fallback; it is configurable (`TAPIR_TRANSCRIBE_FALLBACK_MODEL`, + empty disables), recorded per row, and called out in Open items rather than defaulted + silently. + - Swedish only (`language=sv`). Language detection and routing (#28 decision 2) are not + touched. + - `-H 'Expect:'`, multipart field `file` — encoded in the adapter and its tests, not rediscovered. + +7. **Jobs run in-process, one at a time (ADR-018 stands).** Transcription is minutes (~7.5 min + for 90) but a single goroutine behind a one-slot semaphore: two simultaneous 90-minute jobs on + iguana were not measured, so concurrency is 1 until they are. No worker, no new deployable, + no second replica. Honest status per ADR-025: `queued → transcribing → done | failed`, polled + by the same `/status` pattern, every terminal state a rendered card, never a stuck spinner. + Audio is not retained, so a restart mid-job **cannot resume**: at start-up any + `queued`/`transcribing` row is marked `failed` with `error_kind = interrupted`, and the user + re-uploads. The single-replica assumption gains a second dependent (the semaphore is + process-local) and is recorded here. + +8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers + look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism + as ADR-027's summary gate, **before any model is called**. ADR-027's decisions (ephemeral, + switchable models, trust-the-model) carry over unchanged. **Two things must differ, and both + were measured** (`docs/research/chat-context-over-long-transcripts-2026-10-01.md`): + - **The transcript budget.** The 18,000-char caption default cuts a one-hour meeting to its + first third. Uploaded transcripts take their own `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`, + default **90,000** — the measured ceiling of the best offered model (`iguana/gemma4-26b`), + which answers correctly there but **silently loses the head of a 120,000-char transcript** + with no error and no truncation flag. (An earlier draft of this ADR proposed 120,000 as a + guess; the measurement refuted it.) The mechanism is a per-request `chat.Request.MaxChars`. + - **The opening model.** The offered list leads with the summarizer default `koala/phi4-mini`, + whose 8,192-token context returns HTTP 400 on an hour of speech (~13,000 tokens). Chat over + an upload opens on `TAPIR_UPLOAD_CHAT_MODEL` (default `iguana/gemma4-26b`); the switcher + lists **local models only** (amended by ADR-035 after the adversarial review found it listed + the cloud model too: the whole private transcript goes to the one chosen), and picking + `phi4-mini` on a long recording fails loudly and inline. A per-model budget is not built. + Past ~90,000 chars (≈100 minutes) no offered model can answer by stuffing; see triggers. + +9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py` + hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection; + per-transcript deletion UX (account deletion cascades; see Open items); retention of audio. + +**On the sequencing question #28 asks of itself** — *is this the thing that best serves the +Stage-0 gate, or the most interesting thing to build?* Recorded answer: it proceeds **ahead of +#32** (the demand test) as the maintainer's explicit, knowing call (#28, comment of 2026-10-01), +on a concrete need rather than speculation. It is not a return-nudge (ADR-020/027): it adds +nothing that prompts a user to come back, so it does not contaminate the unprompted-return signal +ADR-016 measures. The honest risk is the one ADR-016 names: it is larger than anything ADR-020 +deferred. The scope cut — chat first, no rich analysis, no video — is what keeps it bounded. + +**Consequences.** +- **Positive.** The user's request is met end to end with existing parts. ADR-021's exception stays + singular and now provably so. The measured ceiling removes chunking from v1. No audio is stored. +- **Negative / watch.** + - Tapir's runtime image gains `ffmpeg`: a larger image and a new CVE surface to patch. + - iguana becomes load-bearing for a user-facing feature (#28 decision 3; infra#273 is still the + dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e. + private audio leaving the homelab, silently unless the per-row endpoint is surfaced. + - A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it. + - Chat over a long transcript is bounded by model context, not by anything in Tapir, and the + failure at the edge is *silent* for gemma (the head is dropped, no error). The 90,000 default + sits just inside the measured limit and is only as good as that single measurement. + - **The default chat model for uploads is a large ollama model on iguana — the same host as + `iguana/kb-whisper`.** infra#273 is exactly that a large resident ollama model can Metal-OOM + whisper-server permanently. Chatting with a transcript can therefore load the model that + wedges the transcriber. Not solved here; the berget fallback limits the blast radius. + - Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`). + - Accepting video means large files are spooled before their audio is extracted, and ffmpeg + demuxes (but, with `-vn`, does not decode) video it then throws away. A client-side audio + extraction would avoid the upload cost; it is not built. + +**Validation / resolution triggers** (each observable by someone other than the author): +- *Chunking*: build it when the duration cap is raised above 90 min, or when a measured + request at the current cap fails with whisper-side (not ingress) errors. +- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback, + swap the primary/fallback order and revisit infra#273. +- *Retrieval chat*: build chunked retrieval when a recording's transcript exceeds the measured + 90,000-char ceiling (≈100 minutes) — i.e. the moment the duration cap is raised above that, or a + real recording turns out denser than the fixture — or when the offered model list changes and + the ceiling is re-measured lower. +- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one + replica, ADR-018 is revisited. +- *The decision itself*: if the maintainer has not used the feature unprompted within the + ADR-016 window, it was the interesting thing, not the useful one — remove it. + +**Open items this ADR creates.** +- **DECIDED (Mathias, 2026-10-02): option A.** Static ffmpeg and ffprobe 7.0.2 are copied into the + distroless image, pinned by version and sha256 (Dockerfile in this PR; #34). Measured at that time: + **about 160 MB** added, not the 70 MB first estimated (80 MB per binary); the build is **GPLv3**; + the checksum proves the bytes reviewed, not the publisher; upstream's `releases/` URL moves on, so + vendoring the verified tarball is the durable fix (not done). Uploads stay off until the deployment + sets `TAPIR_UPLOADS_ENABLED`. *The question as it was put:* *Options:* (A) static ffmpeg/ffprobe copied + into distroless **(recommended — the only option that satisfies "accept m4a/mp3/opus/mp4, + validate duration, reject video"; pure-Go decoders cover wav/mp3 only)**; (B) accept WAV + only, convert client-side (fails "I just want to upload it"); (C) a transcode sidecar + (second deployable, ADR-018's reasoning against). *Blocks:* the Dockerfile slice, not the + tests, which use the port. +- **DECIDED (Mathias, 2026-10-02): option A, enabled by default (#35).** Scope: the audio + transcription fallback only. Reconciliation text and chat over a recording never go to a cloud + model (ADR-034, ADR-035). *The question as it was put:* *Options:* (A) enabled by default as + #33 specifies, with the serving endpoint shown **(recommended, matches the issue)**; (B) + disabled by default, a failed iguana job fails honestly and the user retries. *Blocks:* + the production default only; the adapter ships either way. +- Verify Tapir's ingress `proxy-body-size` against the 200 MB cap (`infra`). +- Per-transcript deletion is not built; account deletion cascades. A private-content store + without a delete button is a gap worth a ticket, not a silent omission. + +**Reversibility.** Additive: one table (down-migration drops it), one adapter, one upload +surface, one image change. Removing the upload route removes the feature; `transcripts`, `videos` +and ADR-021 are untouched. + +--- + +## ADR-034 — Transcribe with two engines and let a local model reconcile their disagreements + +**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #40 (decision B) and #33. +Builds on ADR-033 (private upload transcripts), ADR-022 (endpoint chain, a pattern reused), ADR-018 +(single replica), ADR-025 (honest status). Evidence: +`docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md` and +`docs/research/local-model-adjudicator-sweep-2026-10-01.md`. + +**Context.** On the first real one-hour recording `iguana/kb-whisper` and `iguana/pianissimo` were +a near tie on accuracy (the speaker's blind verdicts on the 30 meaning-changing disagreements: +pianissimo 17, kb-whisper 13) and they **err in different places**: pianissimo is better on terms and +numbers, kb-whisper on ordinary words; kb-whisper's errors garble, pianissimo's read as plausible. +Pianissimo is ~6x faster (40 s vs 4m08s for 63.7 min) and runs through the same adapter. Two engines +that disagree in ~380 places per hour, with a ~92% word agreement, is information a single-engine +pipeline throws away. Separately, eleven local models were scored on adjudicating those disagreements +against the speaker's answers: **`gemma4-31b` 97% and `gemma4-26b` 92%** passed the pre-set bar +(>= 80% accuracy, >= 85% order-swap consistency, <= 10% unparseable), nothing else did. + +**Unknowns, stated.** The adjudicator evidence is 30 items from one speaker and one domain, chosen by +Opus as decidable; the other ~350 disagreements (fillers, compounds, small words) have no ground +truth. Whether a *merged* transcript is better than the better single engine has not been measured. +Reconcile time per hour is estimated, not measured. `gemma4-31b`'s slowness was Low Power Mode on +iguana: with it off the 31B decodes at 22.3 tok/s, not 7.9 (2026-10-02, same-machine before/after on +a different synthetic prompt; see the research note). The reconciler default `gemma4-26b` was chosen +on the slow numbers and is reopened: 31B scored 97% against 92%. + +**Decision.** + +1. **Two engines, one WAV, both local, one after the other.** Each upload is transcribed by + `iguana/pianissimo` and `iguana/kb-whisper` from the same converted WAV, sequentially (concurrency + stays 1, ADR-033). Each engine fails independently; the berget fallback (ADR-033) applies only when + **both** fail. Off by default: with no second engine configured nothing changes. + +2. **Align by word, with pianissimo as the base text.** Both texts are tokenised (lower case, letters + and digits only) and diffed word by word (Myers); neighbouring differences separated by a single + equal word are merged into one **dispute**. The base is pianissimo: sentence-level prose with + punctuation and millisecond timestamps, where kb-whisper's output is wrapped fragments. **This + choice is about readability and timing, not accuracy — accuracy was a tie — so it is the cheaper + mistake, not a measured winner.** A replaced span takes the alternative's own words. + +3. **Resolve the trivial ones without a model.** A dispute that is identical once spaces are removed + (a compound split or joined) keeps the base; so does a dispute that is only a short filler word + on one side. Everything else is adjudicated. (Opus's classification of 380 disputes found ~46 + compound and ~26 punctuation-only residue, ~65 filler.) + +4. **A local model adjudicates the rest, bounded.** Each dispute goes to a local chat model (default + `iguana/gemma4-26b`; `gemma4-31b` is configurable and scored higher but is ~5x slower) with ~100 + words of context either side and the two readings, in the wording that was measured. The reading + order alternates by dispute so position bias averages out. The verdict is *base*, *alternative* or + *unsure*; **unsure keeps the base and is flagged**. An optional order-check mode asks twice with the + order swapped and treats disagreement as unsure; it is **off by default** because it doubles the time + and the measured consistency of the passing models is 93%. + - **The model must be local.** Its alias must begin `koala/` or `iguana/`; a cloud alias is a + start-up configuration error. Private meeting content never goes to a cloud model for this step. + - **Bounded impact.** The reconciler can only choose between two readings the engines already + produced; it cannot write new text. Speech that says "answer 2" can at worst flip one dispute. + +5. **Reconciliation never fails a job.** One engine fails → its text is used, recorded as + single-engine. The adjudicator is down (three consecutive errors) → the base text is used + unreconciled, recorded as skipped with the reason, and the remaining disputes are not retried one + timeout at a time. Both engines fail → ADR-033's fallback path. + +6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed + segments, untouched (**amended by ADR-035: they now carry the merged words, timing stays the + base's**): `content` is authoritative for chat, `segments` only for time. `endpoint` + lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same + RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute + mark and both readings**, so the user can check them against the audio. The transcript page says in + plain words what was done — "reconciled from two transcripts: N differences, K uncertain" — and + lists the uncertain spots. A reconciled transcript is never presented as simply "the" transcript. + +7. **Configuration.** `TAPIR_RECONCILE_BASE_MODEL` (empty = off; the engine whose text is the base, + e.g. `iguana/pianissimo` — `TAPIR_TRANSCRIBE_MODEL` stays the other engine, e.g. kb-whisper), + `TAPIR_RECONCILE_MODEL` (the adjudicator; required when a base engine is set), + `TAPIR_RECONCILE_CHECK_ORDER` (off). All three models must be local aliases (`koala/` or `iguana/`): + a cloud alias is a start-up error, and the base engine must differ from the other one. The default + job timeout rises from 30 to 60 minutes: ~4 min of kb-whisper and ~40 s of pianissimo per hour of + audio, plus the adjudication. + +8. **Not decided here.** A cloud adjudicator (rejected: private content); tuning the prompt; making + `gemma4-31b` the default; a "reconciling" progress state (the page says "transcribing" for the whole + job); feeding the uncertain spots to chat; whether pianissimo should also be a middle tier in the + fallback chain (#40, option A). + +**Consequences.** +- **Positive.** Uses the finding that the engines err differently; stays local end to end; degrades + to today's behaviour at every failure; and is honest on the page about what it did. +- **Negative / watch.** + - Job time grows: roughly 300 adjudicated disputes x ~2 s on `gemma4-26b` ~ 10 minutes per hour of + audio — an estimate, not a measurement; double with order-check on, ~5x with `gemma4-31b`. + - Two gemma-class calls sit on iguana beside whisper. During the sweep a 17-20 GB model was resident + beside whisper-server without wedging it, far below the 52 GB that did (infra#273); the hazard is + reduced, not removed. + - A merged transcript can be wrong where both engines are wrong, and where the adjudicator picks + the wrong one (it was right 92% on a hand-picked set, ~97% for 31b). + - Reading the page takes more words: the reconciliation summary is new surface to keep honest. + +**First live smoke (2026-10-01, synthetic audio, 3 disputes):** all 3 were changed to the +alternative's reading; against the known script 2 were right and **1 was wrong** (a name replaced by a +pronoun, a correct base reading lost). That is one run of three, not a measurement, but it is the failure +this design must expect: the reconciler can overwrite a correct reading, which is why changed spots, not +only uncertain ones, are listed on the page, and why the decision below is a blind check on a real +recording. + +**Validation / resolution triggers.** +- **The test that decides it:** re-upload the same real recording after merge; the speaker judges, blind, + ~30 disputes where the reconciled text differs from pianissimo alone. Keep the feature only if the + reconciled reading is right at least 85% of the time **and** no worse than the better single engine. + If not, switch it off (config) and keep the single-engine path. + **Decision (Mathias, 2026-10-02), on the first result below: keep it as an opt-in feature**, reading + the bar as 95% of decided items, with the second condition to be answered on the next real run. + For deployment the reconciler is `iguana/gemma4-31b` (97% against the speaker's answers, about 22 + minutes for the 63.6-minute recording with Low Power Mode off). A cloud reconciler was raised and is + **not adopted**: it would send private transcript text to a cloud API on every upload; recorded as + an evaluation question in #45. + **First result, 2026-10-02, before merge** (`docs/research/reconcile-blind-check-2026-10-02.md`): 30 + random replacement spots on the real 63.6-minute recording, judged blind by Opus from text (not by the + speaker, who delegated it): 21 right, 1 worse than pianissimo, 3 neither reading right, 5 undecidable. + That is 95% of decided items but 70% of all 30; the bar did not say how ties count, so it is **met on + one reading and missed on the other**, and the speaker has not ruled. The second condition, no worse + than the better single engine, is **not answered**: kept-base spots are not stored, so kb-whisper alone + cannot be compared. Left `Proposed`; the feature stays off by default. +- Reconcile time above 25 minutes per hour of audio: drop to a smaller adjudicator or skip more + disputes automatically. +- A different recording or speaker in which the engines agree far less (or far more): re-measure before + trusting the 92%/97% figures. + +**Reversibility.** Off by default and additive: the second engine, the reconciler and the column are +unused when the settings are empty; the down-migration drops the column; the single-engine path is +unchanged. + +--- + +## ADR-035 — Show an upload's progress as real phases, log the same transitions, and keep every judged spot + +**Status:** Proposed (2026-10-02). Decider: Mathias (pending). Amends ADR-034 decision 6 (segments). +Closes #42, #43 and #44. + +**Context (measured, not assumed).** The first real run (a 63.6-minute recording, 21 min 49 s) exposed +three gaps. (1) The page said only `queued`/`transcribing`/`done`; during 22 minutes nothing told the +user whether the job was working, hung or dead, and after a crash the row would say `transcribing` +until the next start. (2) The server log had no line about the job between start-up and the final +line, so an operator saw it only from outside (spool files, iguana CPU). (3) Evaluating the result +showed that spots the model *kept* as the base are counted but not stored (so ADR-034's "no worse than +the better single engine" cannot be measured after the fact, and the audio is not retained), insertions +and deletions cannot be placed without context, and the saved `segments` still carry the base engine's +wording while only `content` is merged. + +**Decision.** + +1. **Phases are data on the row.** A new `progress` JSONB column (migration 020, same RLS-scoped table) + holds an ordered list of steps `{key, label, state, started_at, finished_at, done, total}` with state + `pending|running|done|failed|skipped`. The plan is declared up front so the page shows what is coming: + received, preparing audio, then the transcriber's own steps (two-engine: engine A, engine B, + comparing, deciding n of N; single: transcribing), then saving. +2. **One source for the page and the log.** An `internal/progress` Tracker is the only writer. Every + transition is saved to the row and logged by the same call, so the UI and the log cannot disagree. + Engines and the reconciler report through a context-carried `Reporter` (no-op when absent), so the + `Transcriber` interface does not change. +3. **Alive is not the same as progressing, and the page says which it knows.** A heartbeat goroutine, + independent of any engine call, stamps `alive_at` every 10 s; `progress_at` moves only on a real + transition or advance. The page derives: *running*; *quiet* (the deciding step has not advanced for + 5 minutes, so the model is probably waiting or slow); *lost* (`alive_at` older than 45 s, so the + service stopped reporting and may have restarted or crashed). The two engine calls are single blocking + requests, so inside them the page shows elapsed time only: **no percentage is ever shown that does not + come from real counts.** Only the deciding step has a bar. +4. **The page.** A server-rendered step list in tapir's existing tokens and mono font, swapped in place + by the existing HTMX poll (every 2 s while unfinished, none once done or failed). Finished steps + collapse to one line with their duration; the running step has a CSS-only spinner that respects + `prefers-reduced-motion`; a failed step and a lost service are visibly different from running. + No new JavaScript, framework or stylesheet. +5. **Logs.** One INFO line per phase transition and engine call (upload id, endpoint, elapsed), the + reconcile counts, `adjudicated n/N` at most once a minute, a WARN when an engine soft-fails or the + breaker trips, and a "still in phase X" line every 5 minutes. **Ids, counts, durations, endpoints and + error kinds only: never transcript text, disputed words or the filename**, asserted by a test. +6. **Every judged spot is stored.** `reconciliation.contested` keeps each model-decided spot with its + verdict (changed, kept base, uncertain) and a few words of left context, bounded at 2,000. The page + still lists only changed and uncertain spots. This makes ADR-034's second condition measurable on the + next real run. +7. **Segments carry the merged words.** The reconciled words replace the base's in the saved `segments`; + timing stays the base's. If the segments cannot be aligned to the merged text (a word wrapped across + a segment edge), they stay the base's and the result says so in the log. This replaces ADR-034 + decision 6's "segments stay untouched". + +8. **Hardening from the adversarial review** (`skills:pr-gatekeeper`, head `7029dd1`: BLOCK; recorded on + PR #39). Each finding was reproduced by a failing test first. + - **Chat over a recording offers and accepts local models only** (it listed `berget/mistral-small`); + `TAPIR_UPLOAD_CHAT_MODEL` must be local at start-up; with no local model, nothing is sent. + - **A model alias counts as local only if it is one plain name after `koala/` or `iguana/`** + (`koala/../berget/x`, `koala//x`, `koala/x?host=…` were accepted by a bare prefix check). + - **An error reaches a log line, the row or the page as a kind** (`timeout`, `canceled`, `http_502`, + `network`, `error`), never its text, which can hold a provider's response body, the spool path or + a word from the transcript (`internal/errkind`). The stored reconcile reason and an engine's + failure reason are kinds; the page no longer shows provider text. + - **Every local engine call has its own deadline** (`TAPIR_TRANSCRIBE_ENGINE_TIMEOUT`, default 20 + minutes: two wedged engines in sequence then leave 20 of the job's 60 minutes for a fallback), + so a wedged engine fails while the job lives and the fallback can run; the job's own + deadline or a cancel still never reaches the cloud. + - **The duration limit is checked against what is converted**, not the file's header: ffmpeg is + capped at the limit plus a minute, a longer result fails the job (`audio_too_long`), and a result + under 80% of the claimed length and more than two minutes short is **transcribed and noted** on + the audio step ("the audio is 20 min long; the file claimed 1 h 19 min"), not refused. The second + adversarial review showed a complete variable-bitrate MP3 whose header over-estimated its length + by 4x was wrongly refused by the first version of this check; a note keeps a truncated file + visible without turning away real recordings. + - **The queue is bounded**, 20 waiting in total and 3 per user, and its room is checked before the + upload is spooled (not after 200 MB and an ffprobe run); a full queue is a 429 with a typed reason, + never a stuck row or a server error. + - **The configured upload chat model is callable.** It is added to the shared chat service's model + list when uploads are on and it is local, because the service re-maps a model it does not offer + (a valid `iguana/gemma4-31b` was being silently swapped for `koala/phi4-mini`). The video chat's own switcher + lists that local model too while uploads are on. + - **A client that hangs up mid-upload is not an error**: the handler logs INFO and writes nothing + (rendering the error page ran a database read on a cancelled context and logged a false ERROR). The + 400 "cut off" answer is keyed on a marker for errors from *reading the client's body* + (`uploads.ErrClientRead`), not on `io.ErrUnexpectedEOF`, which is also what a database drop looks like. + - **A note is not hidden**: a finished upload's record opens by default when any step carries a note. + - **Bad input is a 4xx**: a non-UUID id is a 404, a malformed or cut-off multipart body is a 400 + whose text is not logged, and a title with control bytes, bidirectional controls or invalid UTF-8 is + cleaned (emoji joiners, Swedish letters and variation selectors are kept), not a 500. + - **The image was built and inspected** (buildah on koala, 2026-10-02): the first build FAILED + (`tar: xz: Cannot exec`, the golang image has no xz); fixed, rebuilt at 180 MB, ffmpeg and ffprobe + run as `nonroot` in distroless. CI does not build the image on a pull request + (`if: github.event_name != 'pull_request'`), so the first CI build of it is the merge to main. + Not done, filed as follow-ups: a vendored ffmpeg tarball, CSRF cookie review, ffprobe CPU and + stream-count limits, mov external references, `alive_at` clock skew, a hung database blocking the + tracker, an unbounded gateway response body, a user-facing notice for a recording that is partial but + inside the tolerance. + +**Alternatives rejected.** Server-sent events or a websocket: a long-lived connection through the +ingress, with buffering to get wrong, for a page that changes every few seconds at best; polling is what +the page already does. A percentage from the engine calls: there is no data behind it. A separate +progress table: progress belongs to the row and inherits its RLS. Logging only through the UI: an +operator has no page. + +**Consequences.** Positive: running, quiet, lost and failed are distinguishable by the user and the +operator; the next blind check can answer ADR-034's second condition. Negative / watch: two more writes +per transition and one per 10 s per running job (one job at a time, so negligible); a heartbeat proves +the process is alive, not that an engine is making progress inside one request, which only the job +timeout (60 min) bounds. The stored spot list grows to a few hundred small objects per hour of audio. + +**Validation.** A fake engine held open past the thresholds must change the page (quiet, lost); a log +capture on a synthetic transcript must contain none of its words; a screenshot review of the running, +quiet, lost, failed and done states against the existing pages (the visual claims tests cannot assert). +Revisit if the heartbeat write ever shows in the database's load, or if a longer recording cap makes +five-minute quiet detection too coarse. + +--- + ## Rejected alternatives Approaches considered during the 2026-06-02 planning + grill session and **deliberately not @@ -1338,6 +1814,11 @@ maps to the ADR that settles it. | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 | | k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 | | Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 | +| Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 | +| whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 | +| Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 | +| A cloud model (e.g. Opus) as the adjudicator | It scored 30/30 but would send private meeting content to a cloud API; local gemma4-26b/31b passed the same bar | ADR-034 | +| Pick one engine globally (pianissimo only, or kb-whisper only) | A near tie on accuracy with different error profiles; choosing one discards the information in their disagreement | ADR-034 | If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — not a silent reversal. diff --git a/Dockerfile b/Dockerfile index bd385ab..3fb2f60 100644 --- a/Dockerfile +++ b/Dockerfile @@ -16,12 +16,52 @@ COPY . . # no libc. Trim symbols/DWARF to shrink the layer. RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags='-s -w' -o /out/tapir ./cmd/tapir +# ── ffmpeg ────────────────────────────────────────────────────────────────── +# Audio upload probes files with ffprobe and converts them with ffmpeg (ADR-033, +# #34 option A). Static builds, so they run in the distroless image below. +# +# Pinned by version AND sha256: a changed upstream file fails the build instead of +# shipping. The checksum proves these are the bytes that were reviewed (2026-10-02; +# the publisher's md5 matched too). It does NOT prove authenticity: the publisher +# signs nothing we can check. amd64 only (the cluster is one amd64 node). +# +# These builds are GPLv3 (--enable-gpl --enable-version3). The service is private and +# the image is not distributed; the licence text is copied into the image anyway. +# +# The release URL moves on when upstream releases a newer version, so the pinned +# version is tried in releases/ and then old-releases/. If neither has it the build +# fails; vendoring the verified tarball is the durable fix. +FROM golang:1.26 AS ffmpeg + +ARG FFMPEG_VERSION=7.0.2 +ARG FFMPEG_SHA256=abda8d77ce8309141f83ab8edf0596834087c52467f6badf376a6a2a4c87cf67 + +# xz is not in the golang image (found by building it: `tar: xz: Cannot exec`). +RUN set -eu; \ + apt-get update; \ + apt-get install -y --no-install-recommends xz-utils; \ + rm -rf /var/lib/apt/lists/*; \ + mkdir -p /out; \ + for dir in releases old-releases; do \ + curl -fsSL -o /out/ffmpeg.tar.xz "https://johnvansickle.com/ffmpeg/${dir}/ffmpeg-${FFMPEG_VERSION}-amd64-static.tar.xz" && break; \ + done; \ + echo "${FFMPEG_SHA256} /out/ffmpeg.tar.xz" | sha256sum -c -; \ + tar -xJf /out/ffmpeg.tar.xz -C /out --strip-components=1 --wildcards '*/ffmpeg' '*/ffprobe' '*/GPLv3.txt'; \ + rm /out/ffmpeg.tar.xz + # ── runtime ───────────────────────────────────────────────────────────────── # distroless static + nonroot: no shell, no package manager, runs as uid 65532. # ca-certificates are bundled, which the OIDC/HTTPS clients need. FROM gcr.io/distroless/static-debian12:nonroot COPY --from=build /out/tapir /tapir +COPY --from=ffmpeg /out/ffmpeg /out/ffprobe /usr/local/bin/ +COPY --from=ffmpeg /out/GPLv3.txt /usr/share/doc/ffmpeg/GPLv3.txt + +# Where the service finds them. Uploads stay OFF until TAPIR_UPLOADS_ENABLED is set +# (a deployment decision, not an image one). +ENV TAPIR_FFMPEG_BIN=/usr/local/bin/ffmpeg \ + TAPIR_FFPROBE_BIN=/usr/local/bin/ffprobe EXPOSE 8080 USER nonroot:nonroot diff --git a/cmd/tapir/e2e_upload_test.go b/cmd/tapir/e2e_upload_test.go new file mode 100644 index 0000000..683e483 --- /dev/null +++ b/cmd/tapir/e2e_upload_test.go @@ -0,0 +1,209 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "mime/multipart" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "strings" + "testing" + "time" + + embeddedpostgres "github.com/fergusstrange/embedded-postgres" + "github.com/jackc/pgx/v5/pgxpool" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// TestE2E_UploadTranscribeAndChat is the end-to-end check for tapir#33: upload a +// real audio file, watch it become a transcript through the REAL ffprobe, ffmpeg +// and gateway, then ask a question and get a grounded answer from a REAL model. +// It is opt-in — it needs real binaries and the live gateway — and skipped unless +// TAPIR_E2E_AUDIO is set: +// +// TAPIR_E2E_AUDIO=/path/meeting.m4a TAPIR_E2E_FFMPEG=/path/ffmpeg \ +// TAPIR_E2E_FFPROBE=/path/ffprobe TAPIR_E2E_QUESTION='Vem ansvarar för virket?' \ +// TAPIR_E2E_EXPECT=anders op run --env-file ~/.op-env -- \ +// go test ./cmd/tapir -run E2E_Upload -v -count=1 +// +// Use SYNTHETIC audio only: this repo is public, and a fallback to berget sends +// the audio to a cloud endpoint (#33). +func TestE2E_UploadTranscribeAndChat(t *testing.T) { + audioPath := os.Getenv("TAPIR_E2E_AUDIO") + if audioPath == "" { + t.Skip("set TAPIR_E2E_AUDIO (and friends) to run the live end-to-end check") + } + key := os.Getenv("LITELLM_MASTER_KEY") + require.NotEmpty(t, key, "LITELLM_MASTER_KEY must be set (via op run)") + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + + // A private embedded Postgres: nothing here may touch a real database. + port := uint32(55000 + os.Getpid()%1000) + rt := filepath.Join(os.TempDir(), fmt.Sprintf("tapir-epg-e2e-%d", os.Getpid())) + pg := embeddedpostgres.NewDatabase(embeddedpostgres.DefaultConfig().Port(port). + RuntimePath(rt).DataPath(filepath.Join(rt, "data")).BinariesPath(filepath.Join(rt, "bin")). + CachePath(filepath.Join(os.TempDir(), "tapir-epg-cache"))) + require.NoError(t, pg.Start()) + t.Cleanup(func() { _ = pg.Stop(); _ = os.RemoveAll(rt) }) + dsn := fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port) + + st, err := store.New(ctx, dsn) + require.NoError(t, err) + t.Cleanup(st.Close) + raw, err := pgxpool.New(ctx, dsn) + require.NoError(t, err) + t.Cleanup(raw.Close) + const uid = "11111111-1111-1111-1111-111111111111" + _, err = raw.Exec(ctx, `INSERT INTO users (id) VALUES ($1)`, uid) + require.NoError(t, err) + _, err = raw.Exec(ctx, `INSERT INTO user_identities (dex_subject, user_id) VALUES ('e2e', $1)`, uid) + require.NoError(t, err) + + cfg, err := config.Load() + require.NoError(t, err) + cfg.GatewayKey = key + cfg.UploadsEnabled = true + cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool") + if os.Getenv("TAPIR_E2E_RECONCILE") == "1" { + // Two engines and a local model, as ADR-034 describes. + cfg.ReconcileBaseModel = envOrDefault("TAPIR_E2E_BASE_MODEL", "iguana/pianissimo") + cfg.ReconcileModel = envOrDefault("TAPIR_E2E_RECONCILE_MODEL", "iguana/gemma4-26b") + } + cfg.FFmpegBin = envOrDefault("TAPIR_E2E_FFMPEG", "ffmpeg") + cfg.FFprobeBin = envOrDefault("TAPIR_E2E_FFPROBE", "ffprobe") + + intake, worker, err := buildUploads(ctx, cfg, st, quietLog()) + require.NoError(t, err) + go worker.Run(ctx) + app := &web.App{ + Store: st, Identity: st, Auth: web.StubAuth{U: web.User{Subject: "e2e"}}, + Chat: buildChat(cfg), + } + wireUploads(app, cfg, intake) // the same wiring cmdServe uses + srv := httptest.NewServer(app.Router()) + t.Cleanup(srv.Close) + noRedirect := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + + // 1. Upload. + f, err := os.Open(audioPath) + require.NoError(t, err) + defer func() { _ = f.Close() }() + pr, pw := io.Pipe() + mw := multipart.NewWriter(pw) + go func() { + _ = mw.WriteField("title", "E2E recording") + fw, _ := mw.CreateFormFile("file", filepath.Base(audioPath)) + _, _ = io.Copy(fw, f) + _ = mw.Close() + _ = pw.Close() + }() + req, err := http.NewRequest(http.MethodPost, srv.URL+"/uploads", pr) + require.NoError(t, err) + req.Header.Set("Content-Type", mw.FormDataContentType()) + resp, err := noRedirect.Do(req) + require.NoError(t, err) + _ = resp.Body.Close() + require.Equal(t, http.StatusSeeOther, resp.StatusCode, "an accepted upload redirects to its page") + loc := resp.Header.Get("Location") + require.True(t, strings.HasPrefix(loc, "/uploads/"), loc) + t.Logf("accepted -> %s", loc) + + // 2. Wait for the transcript: unfinished states poll, terminal states do not. + var page string + require.Eventually(t, func() bool { + r, err := http.Get(srv.URL + loc + "/status") + if err != nil { + return false + } + b, _ := io.ReadAll(r.Body) + _ = r.Body.Close() + page = string(b) + return !strings.Contains(page, "hx-trigger") + }, 12*time.Minute, 3*time.Second, "the upload never reached a terminal state") + require.Contains(t, page, "Transcribed", "terminal state was not 'done': %.300s", page) + + var status, endpoint, content string + var secs float64 + var hadVideo bool + require.NoError(t, raw.QueryRow(ctx, + `SELECT status, endpoint, content, audio_seconds, had_video FROM uploaded_transcripts WHERE user_id = $1`, uid). + Scan(&status, &endpoint, &content, &secs, &hadVideo)) + t.Logf("done: endpoint=%s audio_s=%.1f transcript_chars=%d had_video=%v", endpoint, secs, len(content), hadVideo) + if os.Getenv("TAPIR_E2E_RECONCILE") == "1" { + var recJSON string + require.NoError(t, raw.QueryRow(ctx, + `SELECT COALESCE(reconciliation::text, 'NULL') FROM uploaded_transcripts WHERE user_id = $1`, uid).Scan(&recJSON)) + require.NotEqual(t, "NULL", recJSON, "a two-engine run must record its reconciliation") + t.Logf("reconciliation: %.600s", recJSON) + assert.Contains(t, endpoint, "reconciled by", "every contributor is named in the endpoint") + assert.Contains(t, page, "Reconciled from two transcripts") + } + if os.Getenv("TAPIR_E2E_HAD_VIDEO") == "1" { + // A Teams meeting with the camera off: a video file whose audio is what matters. + assert.True(t, hadVideo, "the file had a video track and the row must say so") + assert.Contains(t, page, "Only its audio was used", "and the user must be told") + } + assert.Equal(t, "done", status) + assert.NotEmpty(t, endpoint) + assert.NotEmpty(t, content) + + // ADR-035: a finished job's phases are all closed, and it was reporting while it ran. + var progJSON string + var progressAt, aliveAt *time.Time + require.NoError(t, raw.QueryRow(ctx, + `SELECT COALESCE(progress::text, '[]'), progress_at, alive_at FROM uploaded_transcripts WHERE user_id = $1`, uid). + Scan(&progJSON, &progressAt, &aliveAt)) + var steps []domain.ProgressStep + require.NoError(t, json.Unmarshal([]byte(progJSON), &steps)) + require.NotEmpty(t, steps, "the job recorded no phases") + for _, st := range steps { + assert.Contains(t, []domain.StepState{domain.StepDone, domain.StepSkipped}, st.State, + "phase %q was left %q after the job finished", st.Key, st.State) + } + assert.Equal(t, "save", steps[len(steps)-1].Key) + assert.NotNil(t, progressAt, "progress_at must have been stamped") + t.Logf("phases: %d, heartbeat stamped: %v", len(steps), aliveAt != nil) + entries, err := os.ReadDir(cfg.UploadTmpDir) + require.NoError(t, err) + assert.Empty(t, entries, "the audio must be deleted once transcribed") + + // 3. Ask a question; the answer must come from the transcript. + q := envOrDefault("TAPIR_E2E_QUESTION", "Who is responsible for the timber order?") + chatResp, err := http.PostForm(srv.URL+loc+"/chat", url.Values{"question": {q}}) + require.NoError(t, err) + answerPage, _ := io.ReadAll(chatResp.Body) + _ = chatResp.Body.Close() + require.Equal(t, http.StatusOK, chatResp.StatusCode) + t.Logf("question: %s", q) + if want := os.Getenv("TAPIR_E2E_EXPECT"); want != "" { + assert.Contains(t, strings.ToLower(string(answerPage)), strings.ToLower(want), + "the answer should be grounded in the transcript") + } + t.Logf("answer page excerpt: %.600s", afterMarker(string(answerPage), `class="chat-turn chat-a"`)) +} + +func envOrDefault(k, d string) string { + if v := os.Getenv(k); v != "" { + return v + } + return d +} + +func afterMarker(s, m string) string { + if i := strings.Index(s, m); i >= 0 { + return s[i:] + } + return "(no answer turn found)" +} diff --git a/cmd/tapir/main.go b/cmd/tapir/main.go index 95de372..b94fe5d 100644 --- a/cmd/tapir/main.go +++ b/cmd/tapir/main.go @@ -207,6 +207,26 @@ func cmdServe(ctx context.Context, log *slog.Logger) error { log.Info("web chat enabled (stored-transcript only)", "models", chatModels(cfg)) } + // Audio upload -> private transcript (ADR-033). Off unless TAPIR_UPLOADS_ENABLED: + // the image must carry ffmpeg first. The worker is IN-PROCESS and one job at a + // time, so it inherits the single-replica assumption below (its queue is + // process-local); it stops with the same ctx as the server. + intake, worker, err := buildUploads(ctx, cfg, st, log) + if err != nil { + return err + } + wireUploads(app, cfg, intake) + if intake != nil { + go worker.Run(ctx) + log.Info("audio upload enabled", "model", cfg.TranscribeModel, "fallback", cfg.TranscribeFallbackModel, + "max_mb", cfg.UploadMaxBytes>>20, "max_minutes", int(cfg.UploadMaxDuration.Minutes())) + log.Warn("audio upload assumes a SINGLE replica: the transcription queue is process-local (ADR-033, ADR-018)") + if cfg.TranscribeFallbackModel != "" { + log.Warn("audio upload fallback is a CLOUD endpoint: if the primary fails, private audio leaves the homelab", + "fallback", cfg.TranscribeFallbackModel) + } + } + // User onboarding is handled by the IdP (Authentik invite flow), not Tapir — // the Dex local-password provisioning path was removed (ADR-019). An // authenticated subject with no Tapir user is routed to /register. diff --git a/cmd/tapir/processor.go b/cmd/tapir/processor.go index dc5b483..980ddd4 100644 --- a/cmd/tapir/processor.go +++ b/cmd/tapir/processor.go @@ -134,9 +134,30 @@ func chatModels(cfg config.Config) []string { add(cfg.SummarizerModel) add(cfg.FallbackModel) add(cfg.CloudFallbackModel) + // The model chat over an uploaded recording opens on must be one the shared service can + // call: it re-maps a model it does not offer, which silently swapped a configured 31B for + // phi4-mini, whose context cannot hold an hour of speech. Local only, and only when + // uploads are on, so the video chat is unchanged otherwise (second adversarial review). + if cfg.UploadsEnabled && config.IsLocalModel(cfg.UploadChatModel) { + add(cfg.UploadChatModel) + } return models } +// uploadChatModels is the chat switcher for a PRIVATE uploaded recording: the same +// ordered set as chatModels, local models only. The whole transcript (up to 90,000 +// characters) is sent to the one chosen, so a cloud model the video chat offers must not +// be offered here; with none local, none is offered (ADR-035 review). +func uploadChatModels(cfg config.Config) []string { + var local []string + for _, m := range chatModels(cfg) { + if config.IsLocalModel(m) { + local = append(local, m) + } + } + return local +} + // buildChat wires the per-video chat service (ADR-027): a Completer factory over // the SAME LiteLLM gateway the summarizer uses (a different alias per model, not a // second client config) and the same transcript-truncation budget. It returns nil diff --git a/cmd/tapir/uploads.go b/cmd/tapir/uploads.go new file mode 100644 index 0000000..bda8a19 --- /dev/null +++ b/cmd/tapir/uploads.go @@ -0,0 +1,142 @@ +package main + +import ( + "context" + "fmt" + "log/slog" + "net/http" + "os" + "path/filepath" + "time" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" + "git.d-ma.be/mathias/tapir/internal/adapters/llm" + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/reconcile" + "git.d-ma.be/mathias/tapir/internal/uploads" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// uploadsStore is the store surface the upload wiring needs. *store.Store +// satisfies it. +type uploadsStore interface { + uploads.Store + uploads.WorkerStore + FailInterruptedUploads(ctx context.Context) (int, error) +} + +// buildUploads wires audio upload and private transcription (ADR-033). It returns +// nils when uploads are disabled — never a typed-nil Intake, which would mount the +// /uploads routes over nothing. When enabled it, in order: prepares a private spool +// directory, deletes any audio a crashed run left in it, fails the rows such a run +// left queued or transcribing (the audio is not retained, so they can never +// resume), and only then builds the intake and the worker. The caller runs the +// worker. The fallback endpoint is a cloud service; an empty +// TAPIR_TRANSCRIBE_FALLBACK_MODEL keeps private audio on the homelab. +func buildUploads(ctx context.Context, cfg config.Config, st uploadsStore, log *slog.Logger) (*uploads.Intake, *uploads.Worker, error) { + if !cfg.UploadsEnabled { + return nil, nil, nil + } + if cfg.GatewayURL == "" { + return nil, nil, fmt.Errorf("uploads: TAPIR_UPLOADS_ENABLED is set but there is no TAPIR_GATEWAY_URL to transcribe through") + } + + if err := os.MkdirAll(cfg.UploadTmpDir, 0o700); err != nil { + return nil, nil, fmt.Errorf("uploads: spool dir: %w", err) + } + removeSpooledAudio(cfg.UploadTmpDir, log) + + n, err := st.FailInterruptedUploads(ctx) + if err != nil { + return nil, nil, fmt.Errorf("uploads: fail interrupted rows: %w", err) + } + if n > 0 { + log.Warn("uploads: marked interrupted by a restart (audio is not retained; they must be re-uploaded)", "count", n) + } + + worker := uploads.NewWorker(st, newConverter(cfg), buildTranscriber(cfg), + cfg.UploadTmpDir, cfg.TranscribeTimeout, uploads.WithMaxDuration(cfg.UploadMaxDuration)) + intake := uploads.NewIntake(st, &audio.FFprobe{Bin: cfg.FFprobeBin}, worker, + uploadLimits(cfg), cfg.UploadTmpDir) + return intake, worker, nil +} + +func uploadLimits(cfg config.Config) uploads.Limits { + return uploads.Limits{MaxBytes: cfg.UploadMaxBytes, MaxDuration: cfg.UploadMaxDuration} +} + +// removeSpooledAudio deletes the spool files this process family creates +// (upload-*, wav-*) and nothing else. +func removeSpooledAudio(dir string, log *slog.Logger) { + for _, pat := range []string{"upload-*", "wav-*"} { + matches, _ := filepath.Glob(filepath.Join(dir, pat)) + for _, m := range matches { + if err := os.Remove(m); err != nil { + log.Warn("uploads: remove leftover audio", "path", m, "err", err) + } + } + } +} + +// wireUploads connects a built Intake to the web App: the routes, the limits the +// form states, and the chat settings for an uploaded recording. A nil intake +// leaves the App untouched — assigning a nil *Intake to the Uploader interface +// would produce a non-nil interface and mount the routes over nothing. +func wireUploads(app *web.App, cfg config.Config, in *uploads.Intake) { + if in == nil { + return + } + app.Uploads = in + app.UploadLimits = uploadLimits(cfg) + app.UploadChatMaxChars = cfg.MaxUploadedTranscriptChars + app.UploadChatModel = cfg.UploadChatModel + app.UploadChatModels = uploadChatModels(cfg) +} + +// newConverter is the ffmpeg adapter capped at the configured maximum duration, so a file +// whose header lies about its length cannot become hours of WAV (ADR-035 review). +func newConverter(cfg config.Config) *audio.FFmpeg { + return &audio.FFmpeg{Bin: cfg.FFmpegBin, MaxSeconds: int(cfg.UploadMaxDuration.Seconds())} +} + +// adjudicatorTimeout bounds one adjudication call. A call takes ~2 s on gemma4-26b and ~10 s +// on 31b; this is the wait before an unresponsive model is counted as an error. +const adjudicatorTimeout = 2 * time.Minute + +// buildTranscriber builds what the worker transcribes with. By default that is the +// single-engine chain (the primary, then the optional cloud fallback). When +// TAPIR_RECONCILE_BASE_MODEL is set it is the two-engine ensemble (ADR-034): the base +// engine and the primary run on the same WAV and a LOCAL model reconciles them. Config +// has already refused any non-local model, so nothing here can send private audio or +// text to a cloud model for reconciliation; the cloud fallback is unchanged. +func buildTranscriber(cfg config.Config) uploads.Transcriber { + // No overall client timeout: a 90 minute recording takes minutes. The deadline is + // the worker's per-job context. + client := &http.Client{} + // Each LOCAL engine call gets its own deadline: a wedged engine does not answer, and + // without one it used the whole job before the fallback could run. + local := func(model string) transcribe.Backend { + return transcribe.WithTimeout(transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, model, client), cfg.TranscribeEngineTimeout) + } + primary := local(cfg.TranscribeModel) + var fallback transcribe.Backend + if cfg.TranscribeFallbackModel != "" && cfg.TranscribeFallbackModel != cfg.TranscribeModel { + fallback = transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeFallbackModel, client) + } + if cfg.ReconcileBaseModel == "" { + return transcribe.NewChain(primary, fallback) + } + + base := local(cfg.ReconcileBaseModel) + // Temperature 0 and a short reply: the adjudicator's accuracy was measured that way. + chat := llm.New(cfg.GatewayURL, cfg.GatewayKey, cfg.ReconcileModel, adjudicatorTimeout, + llm.WithTemperature(0), llm.WithMaxTokens(200)) + var adj reconcile.Adjudicator = reconcile.NewLLMAdjudicator(chat) + if cfg.ReconcileCheckOrder { + adj = reconcile.NewOrderChecked(adj) + } + return transcribe.NewEnsemble(base, primary, fallback, adj, reconcile.Options{ + BaseEngine: cfg.ReconcileBaseModel, AltEngine: cfg.TranscribeModel, Reconciler: cfg.ReconcileModel, + }) +} diff --git a/cmd/tapir/uploads_chat_models_test.go b/cmd/tapir/uploads_chat_models_test.go new file mode 100644 index 0000000..cc749de --- /dev/null +++ b/cmd/tapir/uploads_chat_models_test.go @@ -0,0 +1,85 @@ +package main + +import ( + "reflect" + "testing" + + "git.d-ma.be/mathias/tapir/internal/config" +) + +// The chat over an uploaded recording sends the whole private transcript to the model +// the user picks, so the switcher for it may offer local models only (ADR-035 review). + +func TestUploadChatModels_NeverOffersACloudModelButKeepsTheLocalOrder(t *testing.T) { + var cfg config.Config + cfg.SummarizerModel = "koala/phi4-mini" + cfg.FallbackModel = "iguana/gemma4-26b" + cfg.CloudFallbackModel = "berget/mistral-small" + + got := uploadChatModels(cfg) + + want := []string{"koala/phi4-mini", "iguana/gemma4-26b"} + if !reflect.DeepEqual(got, want) { + t.Errorf("uploadChatModels = %v, want %v: the cloud model that the video chat offers must not be offered for a private recording", got, want) + } +} + +func TestUploadChatModels_ACloudPrimaryIsDroppedToo(t *testing.T) { + var cfg config.Config + cfg.SummarizerModel = "berget/mistral-small" + cfg.FallbackModel = "iguana/gemma4-26b" + + got := uploadChatModels(cfg) + + if !reflect.DeepEqual(got, []string{"iguana/gemma4-26b"}) { + t.Errorf("uploadChatModels = %v", got) + } +} + +func TestUploadChatModels_NoLocalModelMeansNoneOfferedNotACloudDefault(t *testing.T) { + var cfg config.Config + cfg.SummarizerModel = "berget/mistral-small" + + if got := uploadChatModels(cfg); len(got) != 0 { + t.Errorf("uploadChatModels = %v, want none", got) + } +} + +// A valid local TAPIR_UPLOAD_CHAT_MODEL that is not in the video chat's chain used to be +// accepted by config and then silently ignored: the shared chat service re-maps a model it +// does not offer, so the user got koala/phi4-mini, whose 8,192-token context cannot hold an +// hour of speech (second adversarial review). + +func TestChatModels_AnUploadChatModelOutsideTheChainIsOfferedWhenUploadsAreOn(t *testing.T) { + var cfg config.Config + cfg.SummarizerModel = "koala/phi4-mini" + cfg.FallbackModel = "iguana/gemma4-26b" + cfg.UploadsEnabled = true + cfg.UploadChatModel = "iguana/gemma4-31b" + + service := chatModels(cfg) + upload := uploadChatModels(cfg) + + if !reflect.DeepEqual(service, []string{"koala/phi4-mini", "iguana/gemma4-26b", "iguana/gemma4-31b"}) { + t.Errorf("chatModels = %v: the shared service must be able to call the configured upload model", service) + } + if !reflect.DeepEqual(upload, service) { + t.Errorf("uploadChatModels = %v, want %v", upload, service) + } +} + +func TestChatModels_NothingIsAddedWhenUploadsAreOffOrTheModelIsNotLocal(t *testing.T) { + var cfg config.Config + cfg.SummarizerModel = "koala/phi4-mini" + cfg.UploadChatModel = "iguana/gemma4-31b" + + if got := chatModels(cfg); !reflect.DeepEqual(got, []string{"koala/phi4-mini"}) { + t.Errorf("uploads off: chatModels = %v, want the video chain unchanged", got) + } + + cfg.UploadsEnabled = true + cfg.UploadChatModel = "berget/mistral-small" + if got := chatModels(cfg); !reflect.DeepEqual(got, []string{"koala/phi4-mini"}) { + t.Errorf("a cloud upload chat model was added to the service: %v", got) + } +} diff --git a/cmd/tapir/uploads_converter_test.go b/cmd/tapir/uploads_converter_test.go new file mode 100644 index 0000000..a8bde82 --- /dev/null +++ b/cmd/tapir/uploads_converter_test.go @@ -0,0 +1,26 @@ +package main + +import ( + "testing" + "time" + + "git.d-ma.be/mathias/tapir/internal/config" +) + +// The converter is told the configured maximum, so a file whose header lies about its +// length cannot become hours of WAV (adversarial review F2). + +func TestNewConverter_IsCappedAtTheConfiguredMaximumDuration(t *testing.T) { + var cfg config.Config + cfg.FFmpegBin = "/usr/local/bin/ffmpeg" + cfg.UploadMaxDuration = 90 * time.Minute + + c := newConverter(cfg) + + if c.Bin != "/usr/local/bin/ffmpeg" { + t.Errorf("Bin = %q", c.Bin) + } + if c.MaxSeconds != 5400 { + t.Errorf("MaxSeconds = %v, want 5400", c.MaxSeconds) + } +} diff --git a/cmd/tapir/uploads_test.go b/cmd/tapir/uploads_test.go new file mode 100644 index 0000000..d96aeb2 --- /dev/null +++ b/cmd/tapir/uploads_test.go @@ -0,0 +1,196 @@ +package main + +import ( + "context" + "errors" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// fakeUploadsStore satisfies the store surface buildUploads needs, and counts the +// start-up sweep. +type fakeUploadsStore struct { + sweeps int + sweepErr error + failedKinds []domain.UploadErrorKind +} + +func (f *fakeUploadsStore) CreateUpload(context.Context, string, string, bool) (string, error) { + return "id", nil +} + +func (f *fakeUploadsStore) FailUpload(_ context.Context, _, _ string, k domain.UploadErrorKind) error { + f.failedKinds = append(f.failedKinds, k) + return nil +} + +func (f *fakeUploadsStore) StartTranscribing(context.Context, string, string) error { return nil } + +func (f *fakeUploadsStore) CompleteUpload(context.Context, string, string, domain.Transcription) error { + return nil +} + +func (f *fakeUploadsStore) SaveProgress(context.Context, string, string, []domain.ProgressStep, time.Time) error { + return nil +} + +func (f *fakeUploadsStore) Heartbeat(context.Context, string, string) error { return nil } + +func (f *fakeUploadsStore) FailInterruptedUploads(context.Context) (int, error) { + f.sweeps++ + return 2, f.sweepErr +} + +func uploadCfg(t *testing.T, enabled bool) config.Config { + t.Helper() + var cfg config.Config + cfg.GatewayURL = "http://gateway.invalid/v1" + cfg.UploadsEnabled = enabled + cfg.UploadMaxBytes = 200 << 20 + cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool") + cfg.TranscribeModel = "iguana/kb-whisper" + cfg.TranscribeFallbackModel = "berget/kb-whisper" + return cfg +} + +func TestBuildUploads_DisabledMountsNothingAndSweepsNothing(t *testing.T) { + st := &fakeUploadsStore{} + + in, w, err := buildUploads(context.Background(), uploadCfg(t, false), st, quietLog()) + + require.NoError(t, err) + assert.Nil(t, in, "a typed-nil Intake must never reach web.App.Uploads: it would mount the routes over nothing") + assert.Nil(t, w) + assert.Zero(t, st.sweeps) +} + +func TestBuildUploads_EnabledFailsInterruptedRowsBeforeAnythingElseStarts(t *testing.T) { + st := &fakeUploadsStore{} + + in, w, err := buildUploads(context.Background(), uploadCfg(t, true), st, quietLog()) + + require.NoError(t, err) + require.NotNil(t, in) + require.NotNil(t, w) + assert.Equal(t, 1, st.sweeps, + "audio is not retained, so rows left queued/transcribing by a restart can never resume: fail them at start-up, not never") +} + +func TestBuildUploads_SweepFailureStopsStartup(t *testing.T) { + st := &fakeUploadsStore{sweepErr: errors.New("db down")} + + _, _, err := buildUploads(context.Background(), uploadCfg(t, true), st, quietLog()) + + require.Error(t, err) + assert.Contains(t, err.Error(), "db down") +} + +func TestBuildUploads_CreatesTheSpoolDirectoryPrivately(t *testing.T) { + cfg := uploadCfg(t, true) + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + require.NoError(t, err) + info, err := os.Stat(cfg.UploadTmpDir) + require.NoError(t, err) + assert.True(t, info.IsDir()) + assert.Equal(t, os.FileMode(0o700), info.Mode().Perm(), "spooled audio is private third-party content") +} + +func TestBuildUploads_EnabledWithoutAGatewayIsAnError(t *testing.T) { + cfg := uploadCfg(t, true) + cfg.GatewayURL = "" + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + assert.Error(t, err, "uploads on with nowhere to transcribe must fail loudly at start-up, not at the first upload") +} + +func TestBuildUploads_RemovesAudioLeftBehindByACrash(t *testing.T) { + cfg := uploadCfg(t, true) + require.NoError(t, os.MkdirAll(cfg.UploadTmpDir, 0o700)) + leftovers := []string{"upload-123", "wav-456"} + for _, n := range leftovers { + require.NoError(t, os.WriteFile(filepath.Join(cfg.UploadTmpDir, n), []byte("audio"), 0o600)) + } + unrelated := filepath.Join(cfg.UploadTmpDir, "someone-elses-file") + require.NoError(t, os.WriteFile(unrelated, []byte("keep"), 0o600)) + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + require.NoError(t, err) + for _, n := range leftovers { + _, err := os.Stat(filepath.Join(cfg.UploadTmpDir, n)) + assert.True(t, os.IsNotExist(err), "%s: audio is never retained, so a spooled file from a crashed run must not survive start-up", n) + } + _, err = os.Stat(unrelated) + assert.NoError(t, err, "only our own spool files are removed") +} + +func TestWireUploads_SetsEverythingTheWebAppNeedsForUploadsAndChat(t *testing.T) { + cfg := uploadCfg(t, true) + cfg.UploadMaxDuration = 90 * 60 * 1e9 + cfg.MaxUploadedTranscriptChars = 90000 + cfg.UploadChatModel = "iguana/gemma4-26b" + in, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + require.NoError(t, err) + app := &web.App{} + + wireUploads(app, cfg, in) + + assert.NotNil(t, app.Uploads) + assert.Equal(t, cfg.UploadMaxBytes, app.UploadLimits.MaxBytes, "the form states the limits Intake enforces") + assert.Equal(t, cfg.UploadMaxDuration, app.UploadLimits.MaxDuration) + assert.Equal(t, 90000, app.UploadChatMaxChars, + "without this the chat would use the 18,000-char caption default and cut an hour to its first third") + assert.Equal(t, "iguana/gemma4-26b", app.UploadChatModel) +} + +func TestWireUploads_NilIntakeLeavesTheAppUntouched(t *testing.T) { + app := &web.App{} + + wireUploads(app, uploadCfg(t, false), nil) + + assert.Nil(t, app.Uploads, "a nil *Intake must not become a non-nil interface that mounts the routes") + assert.Zero(t, app.UploadChatMaxChars) +} + +func TestBuildTranscriber_DefaultIsTheSingleEngineChainAsBefore(t *testing.T) { + cfg := uploadCfg(t, true) + + tr := buildTranscriber(cfg) + + assert.IsType(t, &transcribe.Chain{}, tr, "with no base engine configured nothing changes") +} + +func TestBuildTranscriber_ABaseEngineTurnsOnTheEnsemble(t *testing.T) { + cfg := uploadCfg(t, true) + cfg.ReconcileBaseModel = "iguana/pianissimo" + cfg.ReconcileModel = "iguana/gemma4-26b" + + tr := buildTranscriber(cfg) + + assert.IsType(t, &transcribe.Ensemble{}, tr) +} + +func TestBuildUploads_WiresTheEnsembleWhenConfigured(t *testing.T) { + cfg := uploadCfg(t, true) + cfg.ReconcileBaseModel = "iguana/pianissimo" + cfg.ReconcileModel = "iguana/gemma4-26b" + + in, w, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + require.NoError(t, err) + assert.NotNil(t, in) + assert.NotNil(t, w) +} diff --git a/cmd/tapir/uploads_timeout_test.go b/cmd/tapir/uploads_timeout_test.go new file mode 100644 index 0000000..f39fc70 --- /dev/null +++ b/cmd/tapir/uploads_timeout_test.go @@ -0,0 +1,78 @@ +package main + +import ( + "context" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// The wiring of the per-engine deadline: a gateway whose primary model never answers +// must still reach the fallback model inside the job's own deadline (adversarial review +// F4). The fake gateway answers per model from the multipart `model` field. + +func TestBuildTranscriber_AWedgedPrimaryReachesTheFallbackBeforeTheJobDeadline(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _ = r.ParseMultipartForm(1 << 20) + if r.FormValue("model") == "iguana/kb-whisper" { + <-r.Context().Done() // wedged: never answers + return + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"text":"hej","duration":1,"segments":[{"start":0,"end":1,"text":"hej"}]}`)) + })) + defer srv.Close() + cfg := uploadCfg(t, true) + cfg.GatewayURL = srv.URL + cfg.TranscribeModel = "iguana/kb-whisper" + cfg.TranscribeFallbackModel = "berget/kb-whisper" + cfg.TranscribeEngineTimeout = 150 * time.Millisecond + wav := filepath.Join(t.TempDir(), "a.wav") + require.NoError(t, os.WriteFile(wav, []byte("RIFF....WAVEfmt "), 0o600)) + job, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + + got, err := buildTranscriber(cfg).Transcribe(job, wav) + + require.NoError(t, err, "the wedged primary used the whole job and the fallback never ran") + assert.Equal(t, "berget/kb-whisper", got.Endpoint) +} + +func TestBuildTranscriber_AWedgedBaseEngineIsASoftFailureInTheEnsemble(t *testing.T) { + // The same deadline must guard the ensemble's BASE engine, not only the chain's primary + // (second adversarial review: the wiring was tested for the chain alone). + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _ = r.ParseMultipartForm(1 << 20) + if r.FormValue("model") == "iguana/pianissimo" { + <-r.Context().Done() + return + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"text":"hej","duration":1,"segments":[{"start":0,"end":1,"text":"hej"}]}`)) + })) + defer srv.Close() + cfg := uploadCfg(t, true) + cfg.GatewayURL = srv.URL + cfg.TranscribeModel = "iguana/kb-whisper" + cfg.ReconcileBaseModel = "iguana/pianissimo" + cfg.ReconcileModel = "iguana/gemma4-31b" + cfg.TranscribeEngineTimeout = 150 * time.Millisecond + wav := filepath.Join(t.TempDir(), "a.wav") + require.NoError(t, os.WriteFile(wav, []byte("RIFF....WAVEfmt "), 0o600)) + job, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + + got, err := buildTranscriber(cfg).Transcribe(job, wav) + + require.NoError(t, err, "a wedged base engine must not use up the job") + assert.Equal(t, "hej", got.Text) + require.NotNil(t, got.Reconciliation) + assert.Equal(t, "single_engine", string(got.Reconciliation.Status)) + assert.Contains(t, got.Reconciliation.Reason, "timeout") +} diff --git a/cmd/tapir/uploads_worker_limit_test.go b/cmd/tapir/uploads_worker_limit_test.go new file mode 100644 index 0000000..1be8f32 --- /dev/null +++ b/cmd/tapir/uploads_worker_limit_test.go @@ -0,0 +1,39 @@ +package main + +import ( + "context" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// The worker buildUploads returns enforces the configured maximum on what was really +// converted (adversarial review F2). A stub ffmpeg writes a sparse 100-minute WAV however +// short the file claimed to be; the gateway is never reached because the job fails first. + +func TestBuildUploads_TheWorkerRefusesAudioLongerThanTheConfiguredMaximum(t *testing.T) { + dir := t.TempDir() + stub := filepath.Join(dir, "ffmpeg") + require.NoError(t, os.WriteFile(stub, []byte("#!/bin/sh\nfor a; do last=\"$a\"; done\ntruncate -s 192000044 \"$last\"\n"), 0o700)) + cfg := uploadCfg(t, true) + cfg.FFmpegBin = stub + cfg.UploadMaxDuration = 90 * time.Minute + cfg.TranscribeTimeout = time.Minute + st := &fakeUploadsStore{} + _, w, err := buildUploads(context.Background(), cfg, st, quietLog()) + require.NoError(t, err) + src := filepath.Join(cfg.UploadTmpDir, "upload-x") + require.NoError(t, os.WriteFile(src, []byte("audio"), 0o600)) + + w.Process(context.Background(), uploads.Job{UserID: "u", UploadID: "id", Path: src, Duration: 10 * time.Second}) + + assert.Equal(t, []domain.UploadErrorKind{domain.UploadErrTooLong}, st.failedKinds, + "a file that claimed 10 s but converted to 100 minutes must fail, not be sent to an engine") +} diff --git a/docs/data-model.md b/docs/data-model.md index f290124..a644782 100644 --- a/docs/data-model.md +++ b/docs/data-model.md @@ -18,7 +18,12 @@ only opaque references to them; the secret material lives in ESO/1Password (ADR- not LLM re-summarization but a rate-gated, reputation-risky caption fetch (ADR-010/014), which is paid per re-fetch regardless of user count — so persisting public caption content once and sharing it strictly beats the coupling it removes. Everything else each user owns is - self-contained; `rls_test.go` proves transcripts is the single exception. + self-contained; `rls_test.go` proves transcripts is the single exception for *content*. + **Uploaded-audio transcripts are the opposite of public caption content** (private, owned by + one user) and therefore live in their own RLS-scoped `uploaded_transcripts` table, never in + `transcripts` (ADR-033). The isolation test now also pins the **exact set** of tables with row + security off (`transcripts`, `user_identities`, `invitations`, `schema_migrations`), so a new + table added without RLS fails CI. - **Secrets by reference only.** Tables hold a `secret_ref` (opaque string/UUID resolved via the `SecretStore` port), never tokens or keys. - **The brain sink is just a delivery target.** No brain-specific tables. Whether a summary @@ -41,6 +46,7 @@ erDiagram VIDEO ||--o| SUMMARY : "has at most one" SUMMARY ||--o{ SINK_DELIVERY : "delivered via" USER ||--o{ CHANNEL_ERROR : "reports unavailable channels" + USER ||--o{ UPLOADED_TRANSCRIPT : "uploads (private, ADR-033)" USER { uuid id PK @@ -136,12 +142,32 @@ erDiagram timestamptz first_seen timestamptz last_seen } + UPLOADED_TRANSCRIPT { + uuid id PK + uuid user_id FK "-> USER, ON DELETE CASCADE, NOT NULL; RLS ENABLE+FORCE (migration 017)" + text title + text status "queued | transcribing | done | failed" + text content "nullable until done; the transcript text" + jsonb segments "nullable until done; [{start,end,text}] in seconds" + text endpoint "which backend served it: iguana/... | berget/... (berget = audio left the homelab)" + float audio_seconds + text error_kind "typed: interrupted | transcription_failed — never a provider string" + bool had_video "the file carried a video track; only its audio was used (migration 018)" + jsonb reconciliation "NULL for a single engine; else status, exact counts and a bounded list of every judged spot (migration 019, ADR-034; all spots stored, ADR-035)" + jsonb progress "the job's ordered phases: key, label, state, times, done/total; NULL until the job starts (migration 020, ADR-035)" + timestamptz progress_at "when a phase last really changed or advanced (migration 020)" + timestamptz alive_at "heartbeat from the running job, every ~10 s; not progress (migration 020)" + timestamptz created_at + timestamptz finished_at + } ``` `SUMMARY_ACTION` has `UNIQUE (user_id, video_id, action)`; `VIDEO_CONNECTION` has `UNIQUE (user_id, provider)` (one connection per provider — reconnect upserts in place). RLS (`ENABLE` + `FORCE`) is on **every solid user-owned table above** — `users`, `videos`, -`transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. +`summaries`, `summary_actions`, `video_connections`, `channel_errors`, `uploaded_transcripts`. +(`transcripts` is deliberately **not** in this list: since ADR-021 it is shared public caption +content with no `user_id`.) `sink_deliveries` is RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally **not** RLS'd (auth plumbing). See the *Isolation invariant* section for the mechanism. @@ -197,6 +223,14 @@ mechanism. the most recent discovery pass. Upserted per scheduler pass (`last_seen` refreshed each run); surfaced on the account page as a warning. Cascades on user deletion. Primary key is `(user_id, channel_id)`. FORCE RLS'd. +- **UPLOADED_TRANSCRIPTS** (migration 017, ADR-033) — the **private** transcript of audio a user + uploaded. Deliberately a separate table from the shared `transcripts`: that one is non-RLS on the + grounds that it holds only public caption content, and a recorded meeting is the opposite. + `user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing → + done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK + constraints). `reconciliation` (migration 019, ADR-034) is NULL when one engine produced the text, so the page can never claim a second opinion that was not there; otherwise it holds the status (`reconciled` / `single_engine` / `skipped`), exact counts (differences, settled automatically, decided by the model, changed, uncertain, model errors) and up to 2,000 judged spots (ADR-035: every one the model decided, kept or changed or unsure, each a position in seconds, both readings, which was used, and the few words before it; the page lists only the changed and uncertain ones). `progress`, `progress_at` and `alive_at` (migration 020, ADR-035) let the page show the job's phases and tell a running job from a quiet one from a lost service: `alive_at` is a heartbeat from a goroutine independent of any engine call, `progress_at` moves only when work really moved; the saved `segments` carry the reconciled words with the base engine's timing. The spots hold fragments of the private transcript, which is why they live in this RLS-scoped row. `had_video` (migration 018) records that the file carried a video track — a Teams meeting with the camera off is still a video file — so the page can say only its audio was used and the video was discarded. `endpoint` records which backend served it — `berget/...` means the audio left the + homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90 + minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion. - **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the Stage-0 gate query (VISION §Stage 0: "returned and used in ≥2 distinct weeks"). @@ -209,7 +243,10 @@ enforcement dormant); **ADR-012 opened Stage 1 and turned enforcement on in the Enforcement is **Postgres Row-Level Security** (migration `003_rls.up.sql`): - RLS is `ENABLE`d **and** `FORCE`d on every user-owned table — `users`, `videos`, - `transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. + `summaries`, `summary_actions`, `video_connections`, `channel_errors`, and (migration 017) + `uploaded_transcripts`. `transcripts` is **not** one of them: ADR-021 made it shared public + content, and migration 003's original per-user `transcripts` policy was dropped with that + table in migration 015. `FORCE` is load-bearing: the app connects as the table **owner** (`tapir` role), and owners bypass RLS unless forced. - Each policy keys off the per-request GUC `tapir.current_user_id`, set transaction-locally by diff --git a/docs/homelab-integration.md b/docs/homelab-integration.md index 1b805a6..6537d96 100644 --- a/docs/homelab-integration.md +++ b/docs/homelab-integration.md @@ -12,6 +12,28 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-06), - **Off-cluster (host / NodePort):** `http://koala:30401/v1/` — service `litellm-nodeport` (`4000:30401`). - NOTE: the earlier `31234` was **wrong** — `31234` is the `llama-swap` NodePort (`8080:31234`), not LiteLLM. Use `30401` for off-cluster LiteLLM. `confirm` resolved. +- **Speech-to-text (ADR-033):** the same gateway serves `POST /v1/audio/transcriptions` for + `iguana/kb-whisper` (primary, local) and `berget/kb-whisper` (fallback, EU cloud — private audio + leaves the homelab if it is used). **Call it via the NodePort / in-cluster address above, NOT + `https://llm-api.d-ma.be`:** that public ingress (openresty) returns `413` for bodies over + roughly 8–10 MB (~5 minutes of 16 kHz WAV), while the NodePort took 174 MB (90 minutes) in one + request (measured 2026-10-01, `docs/research/audio-length-ceiling-2026-10-01.md`). Send multipart + field `file`, `response_format=verbose_json`, no `Expect: 100-continue`. iguana's whisper-server + can wedge permanently after a Metal OOM (infra#273); never branch on its + `{"error":"failed to process audio"}` string — it has had at least five causes. Tapir's knobs: + `TAPIR_UPLOADS_ENABLED` (off by default), `TAPIR_UPLOAD_MAX_MB` (200), `TAPIR_UPLOAD_MAX_MINUTES` + (90), `TAPIR_TRANSCRIBE_MODEL`, `TAPIR_TRANSCRIBE_FALLBACK_MODEL` (empty disables the cloud + fallback), `TAPIR_TRANSCRIBE_TIMEOUT` (60m, the whole job), `TAPIR_TRANSCRIBE_ENGINE_TIMEOUT` (20m, one engine call: a wedged engine then fails while the job lives and the fallback can run), `TAPIR_UPLOAD_TMP_DIR`, `TAPIR_FFMPEG_BIN`, + `TAPIR_FFPROBE_BIN`, `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS` (90000 — the measured ceiling of + `iguana/gemma4-26b`, which silently drops the head above it), `TAPIR_UPLOAD_CHAT_MODEL` + (`iguana/gemma4-26b`; must be a local `koala/` or `iguana/` alias, since the whole private transcript is sent to it; `koala/phi4-mini`'s 8,192-token context 400s on an hour of speech). `ffmpeg` + `ffprobe` are in the runtime image (static, pinned by sha256; #34 decided A on 2026-10-02); Tapir's own ingress body limit for a 200 MB upload is unverified (infra). + **Two-engine mode (ADR-034), off unless `TAPIR_RECONCILE_BASE_MODEL` is set:** it runs + `iguana/pianissimo` (the base text) and `TAPIR_TRANSCRIBE_MODEL` (kb-whisper) on the same audio, then a + LOCAL chat model (`TAPIR_RECONCILE_MODEL`) chooses between their disagreements. **Deployment choice (2026-10-02): `iguana/gemma4-31b`** (97% against a speaker's blind answers; 92% for `iguana/gemma4-26b`). With iguana's macOS **Low Power Mode off** the 31B decodes at 22 tok/s and reconciled a 63.6-minute recording in 21 min 49 s end to end; with it on it ran at 8 tok/s. Check `pmset -g | grep lowpowermode` before blaming a model or an engine for slowness. + `TAPIR_RECONCILE_CHECK_ORDER=true` asks twice per dispute with the order swapped (doubles the time). + Every model must start `koala/` or `iguana/`: a cloud alias is a start-up error. The default job timeout is + 60 minutes. `iguana/pianissimo` is KlangAI's Swedish ASR (a Parakeet/FastConformer on MLX, served by a small + FastAPI shell on iguana:8102) and speaks the same `/v1/audio/transcriptions` API. - **Auth key:** treat as a secret via the SecretStore port; do not hardcode in committed code. NOTE: the documented `sk-local-123` returned `401` on `30401/v1/models` (2026-06-02) — it is stale or the gateway now enforces the `LITELLM_MASTER_KEY` (1Password `op://HomeLab/LITELLM_MASTER_KEY`). diff --git a/docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md b/docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md new file mode 100644 index 0000000..6a51e23 --- /dev/null +++ b/docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md @@ -0,0 +1,81 @@ +# kb-whisper vs pianissimo on one real hour of Swedish speech (tapir#33, 2026-10-01) + +One recording, one speaker: a 63.7-minute Teams session of the maintainer dictating project +notes (domain-heavy, English loanwords and product names mixed into Swedish). **No transcript +text and no domain terms are recorded here**: this repo is public and the content is private. +Only aggregate numbers. + +Backends, both through the LiteLLM gateway and Tapir's own adapter, unchanged: +`iguana/kb-whisper` (whisper.cpp, KB's Swedish fine-tune) and `iguana/pianissimo` (KlangAI +Swedish Parakeet/FastConformer-TDT, MLX 8-bit, served by a small FastAPI shell on iguana:8102). + +## Result: a near tie on accuracy, a large difference in speed + +| | kb-whisper | pianissimo | +|---|---|---| +| Wall time for 63.7 min | 4m08s (~15x realtime) | **40 s (~95x realtime)** | +| Segments | 1,302 (timed pieces, wrapped at ~60 chars) | 434 (sentences) | +| Words (normalised) | 6,606 | 6,510 | +| Punctuation | light, every segment ends in a full stop | heavier (138 vs 82 commas) | +| Repetition loops | none | none | + +Word-level agreement between them: **92.4%** as first computed, **93.1%** after the segment +rejoin fix below. 380 aligned disagreement spans, median one word each. + +**Blind A/B by the speaker** (the only ground truth available), on the 30 spans a model +(Claude Opus 5.5) had picked as changing meaning, shuffled and unlabelled: **pianissimo 17, +kb-whisper 13**, no "neither", no "can't tell". + +| Category of the 30 | pianissimo | kb-whisper | +|---|---|---| +| Technical term / English loanword | 8 | 4 | +| Number / date / unit | 4 | 1 | +| Name / proper noun | 3 | 3 | +| Function word / small word | 1 | 2 | +| Other (everyday words) | 1 | 3 | + +The reading: pianissimo is better on terminology and numbers, kb-whisper on ordinary Swedish +words; names are even. Their errors differ in kind: kb-whisper's tend to be garbled words, a +couple were wrong numbers; pianissimo's tend to be fluent substitutions that read as plausible +(a wrong institution, an invented person, a dropped sign), which a reader or an LLM is less +likely to catch. + +Opus's own lean agreed with the speaker's blind answer on **30 of 30**. That is not a measure +of Opus over all 380 spans: it chose spans it could decide, and every one of the 30 carried a +lean. Its full classification of the 380 (counts only): function/small word 81, filler or +false start 65, technical term 58, compound split/join 46, other 39, name 28, casing or +punctuation residue 26, inflection 23, number/date/unit 10, content omission or insertion +(>= 3 words) 4. + +## A bug this comparison found in Tapir (fixed) + +kb-whisper's output is wrapped at ~60 characters and **a wrap can fall inside a word**; the +continuation segment has no leading space, the response's `text` field joins segments with +newlines, and a segment can be a lone `.`. Tapir stored that joined text, so chat received +broken words. Measured on real gateway output: 7% of segments (96/1,332) in a synthetic hour, +42/1,302 lines in the real one. berget's segments are whole sentences with no leading space, so +the two need different joins. `transcribe.joinSegments` now rebuilds the text from the segments. +Checked against the real transcript with pianissimo's vocabulary as an independent yardstick: +36 spans changed, 31 now entirely known words, words unknown to pianissimo 224 -> 177. +Five rejoined spans still contain an unknown word (a name or an unlucky glue): small residual +risk. Earlier tests used short sentences and could not see this. + +## What this does not show + +- **One recording, one speaker, one domain.** No claim about other voices, accents, crosstalk, + or room noise. 17 vs 13 is not a significant difference. +- **The other ~350 spans** were not judged by the speaker. They are mostly spelling, compounds, + fillers and small words; Opus's counts there are a model's judgement, not ground truth. +- Errors **both** models share in text they agree on are invisible to this method. +- Speed was measured once each, on an otherwise idle iguana; pianissimo's 40 s includes the + gateway hop and a 122 MB WAV upload. +- Whether pianissimo survives the infra#273 Metal-OOM contention the way whisper does not was + not tested; both run on iguana. + +## What it suggests for Tapir (not decided, filed as an issue) + +Pianissimo is a **second local backend on the same host, through the same adapter**. That +makes it a natural middle tier ahead of the cloud fallback (kb-whisper -> pianissimo -> +berget), so a wedged whisper no longer sends private audio off the homelab. At 40 s an hour it +is also cheap enough to run **both** on every upload and give chat both texts, or flag where +they disagree; the maintainer's blind answers show they err in different places. diff --git a/docs/research/audio-length-ceiling-2026-10-01.md b/docs/research/audio-length-ceiling-2026-10-01.md new file mode 100644 index 0000000..d03df5a --- /dev/null +++ b/docs/research/audio-length-ceiling-2026-10-01.md @@ -0,0 +1,110 @@ +# Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01) + +Question (#33, "the main technical risk"): at what recording length does Swedish +transcription through the LiteLLM gateway first fail, and with what failure mode? +Measured **before** any chunking design, with iguana not holding a large ollama model, +so the result is the ceiling and not the Metal OOM of infra#273. + +## Answer + +**No transcription ceiling was found at or below 90 minutes.** A single request carrying +90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing +that actually fails is **not whisper**: it is the public gateway ingress, which rejects the +upload with `413` at roughly 5.5 minutes of WAV. + +| Route | First failure | Failure mode | +|---|---|---| +| `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly | +| `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — | +| `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — | + +The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a +30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the +size-dependent form described, a body-size cap explains it without one. (The OOM +mechanism in infra#273 is separately real; this run did not provoke it.) + +The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the +ingress configuration, which lives in `infra`. "About 10 MB" is an inference. + +## Method + +- **Audio.** No known-good Swedish recording is available that is safe to use: the only real + one is a private recording (#31 custody still open) and was + deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char + five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`). + Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe). + Nothing here is a real transcript; none of it is committed. +- **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`, + `response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only + through `op run` and never on argv. +- **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident: + nothing, or `nomic-embed-text` (0.37 GB). Never a large model. + +## Results (`iguana/kb-whisper`, NodePort) + +| Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars | +|---|---|---|---|---|---|---| +| 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — | +| 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — | +| 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 | +| 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 | +| 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 | +| 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 | +| 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 | +| 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 | + +- Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is + a minutes-long job, not a request/response one (see ADR-033 on ADR-018). +- Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are + monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop + hallucination appeared across 90 minutes. This is a weak check — the input is a repeated + loop and synthetic voice — not evidence of accuracy. +- **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration` + 330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`. + +## Findings that change the design + +1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the + gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the + gateway, never the host" and named the public URL; the gateway is right, that URL is not. + `TAPIR_GATEWAY_URL` already points at `koala:30401/v1`. +2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON + `{"text": "", "usage": …}`, not a bare SRT body — a client that parses the body + as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside + an envelope. +3. **The two backends return different timestamp granularity.** `iguana/kb-whisper` + segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real + sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`). + `verbose_json` and the default JSON were byte-identical on iguana. An adapter that + falls back between the two must normalise both, and "which endpoint served this" is + a quality fact, not just a log line. +4. **The existing chat path cannot answer about most of a long recording.** At + 828 chars/min (this fixture; real speech is plausibly a little denser), + `TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute + transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping + the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of + a one-hour meeting get an answer from a transcript that does not contain it. #33 says + chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and + truncates**, and its ceiling is the model context. See ADR-033. +5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The + chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is + not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces + smaller requests — see ADR-033's resolution triggers. + +## What this did not measure + +- **Metal OOM under contention.** Deliberately avoided. A big resident ollama model + during a long transcription is still the known way to wedge iguana permanently. +- **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and + speaker overlap in a Teams export are untested. Speed is probably representative; + accuracy is not claimed. +- **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper + without a server-side transcode is not established here (ADR-033 treats it as needing one). +- **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were + not tried. +- **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not + size-capped at meeting length for 22 minutes; not proven at 90. +- **Time-of-day / other tenants.** One afternoon, one run per length. + +Scripts were throwaway shell/Python in a session scratchpad and are not committed; the +method above reproduces them. diff --git a/docs/research/chat-context-over-long-transcripts-2026-10-01.md b/docs/research/chat-context-over-long-transcripts-2026-10-01.md new file mode 100644 index 0000000..b9fc97d --- /dev/null +++ b/docs/research/chat-context-over-long-transcripts-2026-10-01.md @@ -0,0 +1,66 @@ +# Chat over a long transcript — what the offered models really hold (tapir#33, 2026-10-01) + +Question: ADR-033 reuses `chat.Service`, which stuffs the transcript into the prompt and +cuts the tail. #33 assumed chat "retrieves rather than generates" and so has no length +ceiling. It does not retrieve, so its ceiling is the model's context. What is it, for the +models chat actually offers? + +## Answer + +| Model (via the NodePort gateway) | Result | Limit found | +|---|---|---| +| `koala/phi4-mini` | correct to 30,000 chars; **HTTP 400 at 50,000** (13,332 tokens vs an 8,192-token context) | ~22,000 chars in production, where the client adds `max_tokens: 1500` | +| `iguana/gemma4-26b` | correct at 18k, 50k and **90,000 chars** (26.5 k tokens); **at 120,000 chars it silently loses the head** | **90,000 chars** | +| `berget/mistral-small` | answered "the transcript does not contain it" for a fact **at the end** of an 18,000-char and a 50,000-char transcript | unreliable here, one run each — see caveats | + +The dangerous one is gemma at 120,000 chars: **no error, no truncation flag** — the reported +prompt tokens fell from 26,548 (90 k chars) to 16,387 (120 k chars), and the model answered +"it does not say who the chair is" about a sentence at the very start of the transcript. The +serving layer truncated the prompt from the front. `chat.Service` reports `Truncated: false` +because *it* did not cut anything. + +## What this changes + +1. **The budget for an uploaded transcript is 90,000 chars, not a round 120,000.** That is the + best offered model's measured limit, ~100 minutes of speech at the fixture's density. + `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS` defaults to it. +2. **Chat over an upload must not open on the summarizer default.** The offered list is + `[SummarizerModel, FallbackModel, CloudFallbackModel]`, so `koala/phi4-mini` is first — and + 400s on an hour of speech. `TAPIR_UPLOAD_CHAT_MODEL` (default `iguana/gemma4-26b`) is what + the upload chat opens with; the switcher still lists every model. +3. **A per-model budget does not exist yet.** Someone who switches the dropdown to + `phi4-mini` on a long recording gets a loud HTTP 400, shown inline as "couldn't answer". + That fails honestly but unhelpfully. Filed as a follow-up. +4. **The "retrieval chat" trigger in ADR-033 is now concrete:** any recording past 90,000 chars + (≈100 min) cannot be answered by stuffing, for any offered model. + +## Method + +`POST /v1/chat/completions` through `http://koala:30401/v1`, `temperature: 0`, +`max_tokens: 200` (production sends 1500 for chat via the summary token cap), the same system +framing as `chat.Service`. The transcript is the 60-minute synthetic fixture from +`audio-length-ceiling-2026-10-01.md` truncated/repeated to N chars, with one unique sentence +planted either at the **end** ("lagret på Kvarngatan ska tömmas … av Pelle", question: who/when) +or at the **head** ("kvällens ordförande är Ingrid", question: who chairs). Needle at the +end alone would have missed the gemma failure — a serving layer that truncates from the front +keeps the tail, so the end-needle passed at 120,000 chars; only the head-needle exposed it. +Grounded = the answer contains the planted fact. + +## What this did not measure + +- **Real speech**: the fixture is a repeated 82-second script, so retrieval "difficulty" is + artificially easy except for the planted needle. One question per run, one run per cell. +- **`berget/mistral-small`'s failure**: it missed an end-needle at 18,000 chars where the + others succeeded. One run; could be the model, the prompt framing, or position bias. Not + investigated; do not read it as a ranking. +- **Latency under load**, **concurrent** chat while whisper is transcribing, and in particular + the hazard below. +- **`koala/qwen36-35b-a3b` or any frontier model** — not in the offered list. + +## A hazard this surfaces and does not solve + +`iguana/gemma4-26b` is a large ollama model on the same host as `iguana/kb-whisper`. infra#273 +is precisely that a large resident ollama model can Metal-OOM whisper-server permanently. Making +gemma the default *chat* model for uploads means chatting with a transcript can load the model +that wedges the transcriber. The primary-fails-over-to-berget design limits the blast radius but +does not remove it. Recorded in ADR-033 and filed. diff --git a/docs/research/local-model-adjudicator-sweep-2026-10-01.md b/docs/research/local-model-adjudicator-sweep-2026-10-01.md new file mode 100644 index 0000000..d541171 --- /dev/null +++ b/docs/research/local-model-adjudicator-sweep-2026-10-01.md @@ -0,0 +1,85 @@ +# Can a local model adjudicate two ASR transcripts? (tapir#33, 2026-10-01) + +Question: when `iguana/kb-whisper` and `iguana/pianissimo` disagree about a word or phrase, +can a **local** model say which reading is right, well enough to merge the two transcripts? +Ground truth: the speaker's own blind answers on 30 disagreements of one real recording +(see `asr-kb-whisper-vs-pianissimo-2026-10-01.md`; no transcript text is recorded here, +this repo is public). Reference: Claude Opus 5.5 agreed with the speaker on all 30. + +## Method + +- **Items:** 30 disagreement spans that Opus had picked as changing meaning, the speaker's + verdict as truth (pianissimo right 17, kb-whisper right 13). +- **Prompt:** one fixed English instruction (two transcriptions of Swedish speech differ at + ONE marked spot; say which reading the speaker most likely said; reply `ANSWER: 1|2|UNSURE` + plus one sentence). Each item was asked in **both reading orders** and in **two contexts**: + narrow (+-8 words) and wide (+-100 words). 120 calls per model, temperature 0. +- **Models:** 11, all local (`koala/*`, `iguana/*`): nothing left the homelab. Run one iguana + model at a time, each unloaded afterwards, to avoid the memory pressure behind infra#273; + whisper was confirmed healthy afterwards. +- **Pass criteria, fixed before the run:** wide context, strict accuracy >= 80% (an + unparseable or "unsure" answer counts as wrong), order-swap consistency >= 85%, + unparseable <= 10%. + +## Results (wide context, 60 calls per model) + +| Model | Correct | 95% CI | Same pick when order swapped | Unparseable | s/call | Pass | +|---|---|---|---|---|---|---| +| `iguana/gemma4-31b` | 58 = 97% | 89-99% | 93% | 0 | 10.4 | **yes** | +| `iguana/gemma4-26b` | 55 = 92% | 82-96% | 93% | 0 | 2.0 | **yes** | +| `iguana/ornith-35b` | 51 = 85% | 74-92% | 77% | 0 | 2.5 | no (consistency) | +| `koala/phi4-14b` | 50 = 83% | 72-91% | 80% | 0 | 1.0 | no (consistency) | +| `koala/qwen36-35b-a3b` | 49 = 82% | 70-89% | 83% | 0 | 3.3 | no (consistency) | +| `iguana/granite4` | 47 = 78% | 66-87% | 83% | 0 | 4.6 | no | +| `iguana/qwen3-14b` | 44 = 73% | 61-83% | 87% | 0 | 4.4 | no (accuracy) | +| `iguana/gpt-oss-20b` | 40 = 67% | 54-77% | 100% | 17 | 11.2 | no (unparseable) | +| `koala/phi4-mini` | 37 = 62% | 49-73% | 24% | 1 | 0.5 | no | +| `iguana/gemma4-26b-think` | 23 = 38% | 27-51% | 100% | 37 | 15.3 | no (unparseable) | +| `koala/qwen35-9b-fast` | 0 | n/a | n/a | 60 | 7.2 | no (no output) | + +Narrow context only (+-8 words) costs 6-12 points for the models that pass: gemma4-31b 92%, +gemma4-26b 83%, ornith-35b 80%. Context is not decoration: Opus also used evidence from +elsewhere in the recording (an amount stated earlier, a term that recurs). + +Position bias: the passing models pick "Reading 1" 47-51% of the time. `phi4-mini` picks it 15% +and flips its answer when the order is swapped 76% of the time: it carries almost no signal. + +## Reading it + +- **Two models pass:** the gemma4 pair. Both clear the bar and neither shows position bias. + 26b is ~5x faster; their intervals overlap, so 30 items cannot separate 92% from 97%. +- **The thinking models were not tested fairly.** `gpt-oss-20b` and `gemma4-26b-think` spend + the 600-token budget reasoning and often never reach the answer line. When gemma4-26b-think + did answer, it was right 23 of 23; gpt-oss-20b 93% of the answered. A larger budget would + make them eligible; at 15 s a call it would not make them practical. + +## What this does not show + +- **Thirty items, one recording, one speaker, one domain.** The 30 were chosen by Opus as + decidable, so they overstate how easy the typical disagreement is; ~350 others are mostly + spelling, compounds and fillers, which no ground truth covers. +- Opus agreeing 30/30 is not a measure of Opus over all 380 disagreements. +- Accuracy was measured at temperature 0 on one prompt; no prompt tuning was tried. +- The speaker's answers are the only truth; they are one person's recollection of what they said. + +## Why gemma4-31b is slow, and what was ruled out + +On iguana (M2 Ultra, 64 GB) the dense 31B runs at **103 tok/s prefill, 7.9 tok/s decode**; the +26B MoE at 641 and 39.6. Ruled out as the cause: memory pressure (no swap, 45% free, model 100% +on GPU), context length (32k vs 4k made no difference), and the **inference engine** — MLX 4-bit +of the same model measured 76 tok/s prefill and 8.6 decode, the same within noise. (Terse +output saves only ~3.6 s of ~9.3 s because prefill dominates.) Two unrelated engines at the same +speed points at the machine: ~140 GB/s effective memory bandwidth on a part rated for 800, and +**iguana had macOS Low Power Mode on** (`pmset` `lowpowermode 1`). A first MLX benchmark was +wrong because `mlx_lm.server` caches prompts; the figures above are from the corrected run. + +**Confirmed 2026-10-02.** The owner turned Low Power Mode off (`lowpowermode 0`, read back over +ssh) and `gemma4:31b` was re-measured with nothing else resident: **217 tok/s prefill, 22.3 +tok/s decode** (4 runs, spread 22.2-22.6), against 103 and 7.9 with it on, so 2.1x and 2.8x. The +slowness was the power mode, not the engine or the model. Caveats on the comparison: the new run +used a different, synthetic Swedish prompt (~700 words, the original items held real transcript +text and were deleted) and the model answered in 26-43 tokens, not the ~200 of the first run, so +it is a same-machine before/after, not a controlled A/B. `gemma4:26b` and the MLX figures were +not re-measured. The 31B's 97% judge accuracy is therefore usable at roughly a third of the +latency the sweep implied; the 26B-vs-31B choice in ADR-034 was made on the slow numbers and is +reopened by this. diff --git a/docs/research/reconcile-blind-check-2026-10-02.md b/docs/research/reconcile-blind-check-2026-10-02.md new file mode 100644 index 0000000..e0b6e22 --- /dev/null +++ b/docs/research/reconcile-blind-check-2026-10-02.md @@ -0,0 +1,68 @@ +# Two-engine reconcile, first check on a real recording (tapir#33, ADR-034, 2026-10-02) + +Aggregate numbers only. This repo is public and the recording is private (#31 custody is open), so no +transcript text, no disputed words and no per-item reading appears here. + +## What was run + +One real 63.6-minute Teams recording of one speaker dictating project notes (the video track, camera +off, was discarded). Build: PR #39 head `7f8e367`. Everything local: `iguana/pianissimo` as base, +`iguana/kb-whisper` as the second engine, `iguana/gemma4-31b` as reconciler (iguana Low Power Mode +off), cloud fallbacks off, a throwaway Postgres. + +| Measure | Value | +|---|---| +| Job time, upload to saved | 21 min 49 s for 63.6 min of audio (20.6 min per hour of audio) | +| Differences between the engines | 362 | +| Settled automatically (compound splits, up to two filler words) | 80 | +| Decided by the model | 282 | +| Of those: alternative reading used / base kept / uncertain | 130 / 151 / 1 | +| Model call errors | 0 | + +20.6 min per hour is inside the 25-minute resolution trigger in ADR-034, so no smaller adjudicator is +needed on speed grounds. The earlier estimate was about 10 min per hour on gemma4-26b; it was not +measured and this run used the 31B. + +## Blind check + +30 spots drawn at random (fixed seed) from the 89 replacement spots that could be placed in the base +text, across the whole recording (0:44 to 60:57). The two readings were shown as x and y in random +order, with the key kept apart from the judge. Judge: Claude Opus, text only, no audio, at the +speaker's request ("I trust Opus"). The speaker did not judge these items themselves. + +| Verdict | Items | +|---|---| +| The reconciler's reading (the alternative) is right | 21 | +| The base reading (pianissimo) is right, so the change made it worse | 1 | +| Neither reading is right | 3 | +| Cannot tell from text (compound, hyphen and number-format differences) | 5 | + +- Right of the items with a verdict: 21 of 22, 95% (95% interval 78% to 99%). +- Right of all 30, counting "neither" and "cannot tell" as not right: 21 of 30, 70% (52% to 83%). +- Right of the 25 that were not "cannot tell": 21 of 25, 84% (65% to 94%). +- Net effect against the base on these 30: 21 better, 1 worse, 8 unchanged in quality. + +## Reading the result + +ADR-034's bar was "right at least 85% of the time". It did not say how ties count, and that was a +flaw in the bar, not in the data. The literal reading (all 30) gives 70% and **does not meet** the bar; +counting only decided items gives 95% and meets it. Neither is a hidden pass or a hidden failure; both +are reported. What is not in doubt: on a random sample the reconciler improved the text far more often +than it harmed it (21 to 1), and the 8 unchanged cases are mostly spellings the text cannot settle. + +## What this did not measure + +- **Better single engine.** The second ADR-034 condition, "no worse than the better single engine", is + **not answered**. Only changed spots are stored, not the 151 where the model kept pianissimo, so the + reconciler cannot be compared with kb-whisper alone. The 21-to-1 result is against pianissimo. +- **The speaker as judge.** Opus judged from text. It cannot hear the audio, so it can be wrong where + the right word is audible but not inferable (items such as a mis-heard number or a name). Earlier, + Opus and the speaker agreed 30 of 30, but on items chosen as decidable; that does not transfer to a + random sample. +- **Insertions and deletions.** 42 of the 131 stored spots were not sampled: 39 are an added or + removed word, which cannot be placed in the base text, and 3 are the uncertain spot or could not be + located. The 39 are mostly small words ("och", "det"), so this sample says little about them. +- **One recording, one speaker, 30 items.** Wide intervals, and neighbouring spots (three pairs within + seconds of each other) are not independent. +- **Segments keep the base text.** The saved timed segments hold pianissimo's wording, not the merged + text; only the stored full text is merged. Found while building this sample; not yet an issue. diff --git a/docs/use-cases/audio_upload.feature b/docs/use-cases/audio_upload.feature new file mode 100644 index 0000000..e45c465 --- /dev/null +++ b/docs/use-cases/audio_upload.feature @@ -0,0 +1,258 @@ +Feature: Upload an audio recording, get a private Swedish transcript, discuss it + As someone with a recorded meeting on my computer + I want to upload it and then ask questions about what was said + So that I can find out what was decided, what is open and who owes what + + # ADR-033, tapir#33. The load-bearing constraint: an uploaded transcript is + # PRIVATE content, so it lives in its own RLS-scoped store and never in the + # shared public-caption `transcripts` table (ADR-021). Rejection is synchronous, + # the audio is never retained, and a failed job ends in a terminal state with a + # message — never a spinner (ADR-025). A file with a video track (a Teams meeting + # with the camera off is one) is accepted: only its audio is used and the user is + # told. Video playback, keyframes and analysis of what is shown remain #28. + + Background: + Given I am a registered user with audio upload enabled + + Scenario: The upload form states the limits up front + When I open the upload page + Then I see the size limit and the length limit before I choose a file + + Scenario: An accepted recording is queued and I am taken to its page + When I upload an audio file within the limits + Then it is recorded as queued for me + And I am redirected to its page + + Scenario: A recording with no title is named after its file + When I upload an audio file and give no title + Then its title is the filename without its extension + + Scenario: A file with a video track is accepted and I am told the video was ignored + When I upload a recording that also has a video track + Then it is accepted and queued like any other + And its page says only its audio was used and the video was discarded + And it says so while queued, while transcribing, when done and when failed + + Scenario: A video file with no audio is rejected for having no audio + When I upload a video that has no audio track + Then it is rejected with a message that it has no audio + And nothing is recorded + + Scenario: A recording over the length limit is rejected with both lengths + When I upload a recording longer than the limit + Then it is rejected stating the recording's length and the limit + And nothing is recorded + + Scenario: A file over the size limit is rejected as too large + When I upload a file larger than the size limit + Then it is rejected with 413 and a message about the limit + And nothing is recorded + + Scenario: Cover art on an audio file is not a video track + When I upload an mp3 whose only video stream is attached cover art + Then it is treated as audio and no video notice is shown + + Scenario: A file that is not media is rejected as unreadable + When I upload a file ffprobe cannot read + Then it is rejected as the file's fault + + Scenario: A broken ffprobe is reported as our fault, not as a bad file + When the prober itself fails + Then the upload is an error rather than a rejection of my file + + Scenario: A recording becomes a transcript and the audio is deleted + Given I have uploaded a recording + When it has been transcribed + Then the transcript is stored privately for me + And the audio is deleted + + Scenario: The endpoint that served a transcript is recorded and shown + Given a recording was transcribed by the fallback endpoint + When I view its page + Then it names the fallback endpoint, because that means the audio left the homelab + + Scenario: The local endpoint is tried first and the fallback only on failure + When the primary transcription endpoint fails for any reason + Then the fallback endpoint transcribes it + But when the primary succeeds the fallback is never contacted + + Scenario: A failed transcription ends in a message, never a spinner + When every transcription endpoint fails + Then the upload is failed with a typed reason + And its page shows that reason and stops polling + + Scenario: A restart fails the jobs it interrupted + Given the service restarted while a recording was queued or transcribing + When it starts again + Then those uploads are failed as interrupted + And any audio left in the spool is deleted + + Scenario: Only one recording is transcribed at a time + When several recordings are queued + Then they are transcribed one after another + + Scenario: My uploads are invisible to other users + Given another user has uploaded a recording + When I ask for it by its id + Then I get 404 and no part of its title or text + And the chat for it calls no model + + Scenario: I can ask about my recording and get a grounded answer + Given my recording has been transcribed + When I ask a question about it + Then the answer comes from my transcript through the same chat as a video's + + Scenario: A long recording is not cut to its first third + Given my recording's transcript is longer than the caption default + When I ask a question about it + Then the transcript is sent with the upload budget, not the 18,000-character caption default + + # ADR-034, two engines and a local reconciler (off unless TAPIR_RECONCILE_BASE_MODEL is set) + + Scenario: Two engines transcribe the audio and their disagreements are reconciled + Given two-engine transcription is enabled + When a recording is transcribed + Then both engines run on the same audio, the fast one first + And a local model chooses between their disagreements + And the endpoint names every engine and the model that took part + + Scenario: Trivial disagreements never reach the model + When the engines differ only by a compound written as one word or two, or by a filler word + Then the base reading is kept without a model call + + Scenario: Differences close together are one decision + When two differences are separated by a single shared word + Then they are asked about as one dispute + + Scenario: An unsure or changed reading is flagged for me + When the model changes a reading or cannot decide + Then the page lists the spot with a minute mark, both readings and which was used + And an unsure spot keeps the base reading and is marked uncertain + + Scenario: The model cannot be a cloud model + When the reconciliation model or base engine is not a koala/ or iguana/ alias + Then the service refuses to start + And a cloud alias that merely contains a local host name is refused too + + Scenario: One engine failing does not fail the upload + When one of the two engines fails + Then the other engine's transcript is used + And the page says only one engine produced it and why + And the cloud fallback is not used + + Scenario: Only when both local engines fail does the cloud fallback run + When both local engines fail and a fallback is configured + Then the fallback transcribes it and the row records that it did + + Scenario: A model that is down gives an unreconciled transcript, not a failed job + When the reconciliation model fails three times in a row + Then the base transcript is kept unreconciled with the reason + And the page says it could not be reconciled + + Scenario: A cancelled job never reaches the cloud + When the job is cancelled during either engine + Then neither the second engine nor the cloud fallback is called + + Scenario: A single-engine transcript claims no second opinion + When only one engine was configured + Then the page does not mention reconciliation + + # ADR-035: progress, the log, and the judged spots + + Scenario: I can see which phase my upload is in + Given a recording is being transcribed + When I open its page + Then it lists every phase in order, the finished ones with how long each took + And the running phase is marked, with how long it has run + And the page updates in place every two seconds until the job ends + + Scenario: A bar appears only where the count is real + When the deciding phase has decided 142 of 282 differences + Then the page shows a bar with "142 of 282" + And an engine call, which is one request with nothing to count, never gets a bar or a percentage + + Scenario: A slow engine call is not mistaken for a dead service + When an engine call has run for 25 minutes and the service is still reporting + Then the page shows the elapsed time and says nothing is wrong + + Scenario: A service that stopped reporting is called lost + When nothing has been heard from the service for longer than the heartbeat allows + Then the page says the service stopped reporting and may have restarted or crashed + And the running phase carries a warning mark and no spinner + And the page keeps checking + + Scenario: A model that stops advancing is called quiet + When the deciding phase has not advanced for five minutes and the service is still reporting + Then the page says nothing new has happened for that long and that the service is still running + And it does not say the service is lost + + Scenario: A failed job shows where it stopped + When a job fails + Then the failed phase is marked and what never started stays pending + And the page stops checking + + Scenario: A finished job keeps its record of what was done + When a job is done + Then the page keeps a collapsed list of the phases with how long each took + + Scenario: The heartbeat runs on its own while an engine call is stuck + When an engine call does not return + Then the row is still stamped alive every few seconds + And no heartbeat outlives its job + + Scenario: The log names every phase and never the transcript + When a job runs + Then the log has a line per phase in the order it ran, each naming the upload + And no line contains transcript text or the spool path + + Scenario: Every judged spot is stored but only the ones to check are listed + When the model has changed, kept and been unsure about spots + Then every one is stored with its verdict and the words before it + And the page lists only the changed and uncertain ones, with the words before each + + Scenario: The saved segments carry the merged words + When the model changes a word + Then the segment containing it carries the reconciled word + And the segment's timing is unchanged + + # ADR-035 decision 8: hardening from the adversarial review + + Scenario: Chat about a recording is offered with local models only + When I open the chat for my recording + Then the model list holds only models that run on the homelab + And a cloud model the shared chat service knows is not in it + + Scenario: A forged cloud model is never sent the transcript + When a chat request names a cloud model + Then a local model answers instead, and nothing was sent to the cloud one + + Scenario: A wedged engine does not use up the whole job + When the primary engine never answers + Then it fails at its own deadline while the job is still alive + And the fallback runs, unless the job itself was cancelled or timed out + + Scenario: A file that lies about its length is refused + When a file claims to be short but converts to more than the limit + Then the job fails with a plain explanation and no engine receives the audio + + Scenario: A recording much shorter than its file claimed is transcribed and noted + When a file converts to under 80 percent of the length it claimed and more than two minutes short + Then it is transcribed anyway and its audio step says how long the audio really is + And a complete recording whose header over-estimated its length is not refused + + Scenario: A full queue is a retry, not an error + When too many recordings are already waiting + Then the upload is answered as busy, its row is failed honestly and its audio is deleted + + Scenario: One user cannot fill the queue for everyone + When one user has several recordings waiting + Then further uploads from that user are refused and other users are not affected + + Scenario: Errors are shown as kinds, never their text + When a model or an engine fails + Then the log, the stored reason and the page carry a kind such as timeout or http_502 + And never the provider's response, a file path or a word from the transcript + + Scenario: Input that cannot be right is a client error + When the id is not an id, or a header or title holds control bytes + Then the answer is a 4xx or the title is cleaned, never a server error diff --git a/internal/adapters/audio/ffmpeg.go b/internal/adapters/audio/ffmpeg.go new file mode 100644 index 0000000..a78908a --- /dev/null +++ b/internal/adapters/audio/ffmpeg.go @@ -0,0 +1,62 @@ +package audio + +import ( + "context" + "fmt" + "os" + "os/exec" + "strconv" + "strings" +) + +// FFmpeg converts any audio container to the strict format whisper wants: 16 kHz +// mono signed 16-bit PCM WAV (ADR-033). It ALWAYS transcodes, even when the input +// already is that format — a Teams export often is, and the transcode is then +// nearly free; assuming it would be the bug. This replaces whisper-server's +// --convert, whose --tmp-dir is created at process start and rots under macOS +// periodic cleanup. +type FFmpeg struct { + // Bin is the ffmpeg executable ("ffmpeg" resolves via PATH). + Bin string + // MaxSeconds caps the converted length when positive. The duration limit is a claim + // in the file's header and can be lied about (a 2 h file claiming 10 s converted to a + // 120-minute WAV); this bounds what a lie can cost. A minute of slack is added so a + // file at the limit is never cut, and the worker measures the result. + MaxSeconds int +} + +// lengthSlackSeconds is added to MaxSeconds so ffmpeg's rounding never clips a file that +// is really at the limit. +const lengthSlackSeconds = 60 + +// ToWAV writes src, converted, to dst. Video streams are dropped (-vn): by now the +// file has already been probed as audio-only, but a cover-art stream must not end +// up in the WAV. +func (f *FFmpeg) ToWAV(ctx context.Context, src, dst string) error { + bin := f.Bin + if bin == "" { + bin = "ffmpeg" + } + // Input restrictions go BEFORE -i: placed after it they would apply to the output. + args := append([]string{"-y", "-v", "error"}, inputRestrictions...) + args = append(args, "-i", src, "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le") + if f.MaxSeconds > 0 { + // After -i: an OUTPUT cap. Before it, it would limit what is read, not written. + args = append(args, "-t", strconv.Itoa(f.MaxSeconds+lengthSlackSeconds)) + } + args = append(args, "-f", "wav", dst) + cmd := exec.CommandContext(ctx, bin, args...) + cmd.WaitDelay = killGrace + if out, err := cmd.CombinedOutput(); err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return fmt.Errorf("audio: ffmpeg: %w", ctxErr) + } + return fmt.Errorf("audio: ffmpeg: %w: %s", err, strings.TrimSpace(string(out))) + } + // A zero exit that wrote nothing is still a failure: never hand whisper a + // missing file and let the error surface three layers later. + if info, err := os.Stat(dst); err != nil || info.Size() == 0 { + return fmt.Errorf("audio: ffmpeg wrote no output at %s", dst) + } + return nil +} diff --git a/internal/adapters/audio/ffmpeg_test.go b/internal/adapters/audio/ffmpeg_test.go new file mode 100644 index 0000000..57205a5 --- /dev/null +++ b/internal/adapters/audio/ffmpeg_test.go @@ -0,0 +1,111 @@ +package audio_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" +) + +func TestFFmpegToWAV_AsksForSixteenKilohertzMonoPCMAlways(t *testing.T) { + // Always transcode, even when the input already is 16 kHz mono: a Teams export + // often is, and the point is to not ASSUME (ADR-033). Records the argv. + rec := filepath.Join(t.TempDir(), "argv") + bin := stub(t, `printf '%s\n' "$@" > `+rec+` +for a; do last="$a"; done +printf data > "$last"`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "out.wav")) + + require.NoError(t, err) + raw, err := os.ReadFile(rec) + require.NoError(t, err) + argv := strings.Split(strings.TrimSpace(string(raw)), "\n") + joined := " " + strings.Join(argv, " ") + " " + for _, want := range []string{" -ac 1 ", " -ar 16000 ", " -c:a pcm_s16le ", " -vn ", " -i /spool/in "} { + assert.Contains(t, joined, want) + } + assert.True(t, strings.HasSuffix(strings.TrimSpace(joined), "out.wav"), "the output path is the last argument") +} + +func TestFFmpegToWAV_NonzeroExitIsAnErrorCarryingStderr(t *testing.T) { + bin := stub(t, `echo "moov atom not found" >&2; exit 1`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/in", filepath.Join(t.TempDir(), "o.wav")) + + require.Error(t, err) + assert.Contains(t, err.Error(), "moov atom not found") +} + +func TestFFmpegToWAV_ZeroExitWithNoOutputIsAnError(t *testing.T) { + bin := stub(t, `exit 0`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/in", filepath.Join(t.TempDir(), "never-written.wav")) + + require.Error(t, err) +} + +func TestFFmpegToWAV_MissingBinaryIsAnError(t *testing.T) { + err := (&audio.FFmpeg{Bin: filepath.Join(t.TempDir(), "no-ffmpeg")}).ToWAV(context.Background(), "/in", "/out.wav") + + require.Error(t, err) + assert.False(t, errors.Is(err, context.Canceled)) +} + +func TestFFmpegToWAV_RestrictsWhatTheFileMayMakeFFmpegOpen(t *testing.T) { + rec := filepath.Join(t.TempDir(), "argv") + bin := stub(t, `printf '%s\n' "$@" > `+rec+` +for a; do last="$a"; done +printf data > "$last"`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "o.wav")) + + require.NoError(t, err) + args := argvOf(t, rec) + pw, fw, in := indexOf(args, "-protocol_whitelist"), indexOf(args, "-format_whitelist"), indexOf(args, "-i") + require.GreaterOrEqual(t, pw, 0, "argv: %v", args) + require.GreaterOrEqual(t, fw, 0, "argv: %v", args) + assert.Equal(t, "file,pipe", args[pw+1]) + assert.Equal(t, audio.AllowedDemuxers, args[fw+1]) + assert.Less(t, pw, in, "input options must come BEFORE -i or ffmpeg applies them to the output") + assert.Less(t, fw, in) +} + +func TestFFmpegToWAV_PassesAnOutputLengthCapAfterTheInputWhenAMaximumIsSet(t *testing.T) { + // The duration limit is a CLAIM in the file's header; a file that lies about it can + // convert to hours of audio (adversarial review F2: a 2 h file claiming 10 s became a + // 120-minute WAV). The cap is an OUTPUT option, after -i, with a minute of slack. + rec := filepath.Join(t.TempDir(), "argv") + bin := stub(t, `printf '%s\n' "$@" > `+rec+` +for a; do last="$a"; done +printf data > "$last"`) + + err := (&audio.FFmpeg{Bin: bin, MaxSeconds: 5400}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "out.wav")) + + require.NoError(t, err) + raw, err := os.ReadFile(rec) + require.NoError(t, err) + joined := " " + strings.Join(strings.Split(strings.TrimSpace(string(raw)), "\n"), " ") + " " + assert.Contains(t, joined, " -t 5460 ", "limit plus a minute of slack") + assert.Less(t, strings.Index(joined, " -i /spool/in "), strings.Index(joined, " -t 5460 "), "-t after -i is an output cap; before it, an input one") +} + +func TestFFmpegToWAV_PassesNoLengthCapWhenNoMaximumIsSet(t *testing.T) { + rec := filepath.Join(t.TempDir(), "argv") + bin := stub(t, `printf '%s\n' "$@" > `+rec+` +for a; do last="$a"; done +printf data > "$last"`) + + require.NoError(t, (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "out.wav"))) + + raw, err := os.ReadFile(rec) + require.NoError(t, err) + assert.NotContains(t, " "+strings.ReplaceAll(string(raw), "\n", " ")+" ", " -t ") +} diff --git a/internal/adapters/audio/ffprobe.go b/internal/adapters/audio/ffprobe.go new file mode 100644 index 0000000..5b75b08 --- /dev/null +++ b/internal/adapters/audio/ffprobe.go @@ -0,0 +1,109 @@ +// Package audio holds the adapters that inspect and convert user-supplied audio +// with ffmpeg's tools (ADR-033). They sit behind small ports so the use case is +// tested with fakes and only this package knows a subprocess is involved. +package audio + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os/exec" + "strconv" + "time" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// FFprobe implements uploads.Prober by running ffprobe on the spooled file. +type FFprobe struct { + // Bin is the ffprobe executable ("ffprobe" resolves via PATH). + Bin string +} + +// AllowedDemuxers are the input formats an upload may be. The input is +// attacker-supplied, and a file's CONTENT can tell ffmpeg to open other files or +// URLs (HLS and concat playlists, image2 sequences, network demuxers). Recent +// ffmpeg refuses the ones tried (7.0.2: HLS without a .m3u8 name is not sniffed, +// concat rejects unsafe paths), but that is version luck; this list is not. It +// holds the containers real recordings use and none of the playlist or network +// formats. "mov" covers .mp4/.m4a/.3gp; "matroska" covers .webm. +const AllowedDemuxers = "wav,mov,mp4,m4a,3gp,mp3,ogg,matroska,webm,flac,aac,opus,aiff,amr,asf" + +// inputRestrictions are ffmpeg/ffprobe INPUT options and must precede the input. +// Local files and pipes only: no network, no other protocols. +var inputRestrictions = []string{"-protocol_whitelist", "file,pipe", "-format_whitelist", AllowedDemuxers} + +// killGrace bounds how long Probe waits for a cancelled ffprobe's pipes to close. +const killGrace = 2 * time.Second + +// Probe runs ffprobe and classifies the outcome. A non-zero exit means ffprobe +// looked at the file and could not read it: that is the FILE's fault and returns +// uploads.ErrUnreadable. A missing binary, a cancelled context or unparsable +// output is OURS and is returned as an ordinary error, so it is never reported to +// the user as their file being bad. +func (f *FFprobe) Probe(ctx context.Context, path string) (uploads.Probe, error) { + bin := f.Bin + if bin == "" { + bin = "ffprobe" + } + args := append([]string{"-v", "error"}, inputRestrictions...) + args = append(args, "-print_format", "json", "-show_format", "-show_streams", path) + cmd := exec.CommandContext(ctx, bin, args...) + cmd.WaitDelay = killGrace + out, err := cmd.Output() + if err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return uploads.Probe{}, fmt.Errorf("audio: ffprobe: %w", ctxErr) + } + var exit *exec.ExitError + if errors.As(err, &exit) { + return uploads.Probe{}, fmt.Errorf("%w: ffprobe: %s", uploads.ErrUnreadable, exit.Stderr) + } + return uploads.Probe{}, fmt.Errorf("audio: run ffprobe: %w", err) + } + p, err := ParseProbe(out) + if err != nil { + return uploads.Probe{}, fmt.Errorf("audio: ffprobe output: %w", err) + } + return p, nil +} + +// probeJSON is the subset of `ffprobe -print_format json` output that matters. +type probeJSON struct { + Streams []struct { + CodecType string `json:"codec_type"` + Disposition struct { + AttachedPic int `json:"attached_pic"` + } `json:"disposition"` + } `json:"streams"` + Format struct { + Duration string `json:"duration"` + } `json:"format"` +} + +// ParseProbe turns ffprobe's JSON into a uploads.Probe. A video stream that is +// only attached cover art (an mp3/m4a with an album picture) is not video. A +// missing or "N/A" duration yields zero, which uploads.Validate rejects as +// unreadable rather than accepting an empty recording. +func ParseProbe(data []byte) (uploads.Probe, error) { + var j probeJSON + if err := json.Unmarshal(data, &j); err != nil { + return uploads.Probe{}, fmt.Errorf("decode: %w", err) + } + var p uploads.Probe + for _, s := range j.Streams { + switch s.CodecType { + case "audio": + p.HasAudio = true + case "video": + if s.Disposition.AttachedPic == 0 { + p.HasVideo = true + } + } + } + if secs, err := strconv.ParseFloat(j.Format.Duration, 64); err == nil && secs > 0 { + p.Duration = time.Duration(secs * float64(time.Second)) + } + return p, nil +} diff --git a/internal/adapters/audio/ffprobe_test.go b/internal/adapters/audio/ffprobe_test.go new file mode 100644 index 0000000..214320b --- /dev/null +++ b/internal/adapters/audio/ffprobe_test.go @@ -0,0 +1,186 @@ +package audio_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +func TestParseProbe(t *testing.T) { + tests := []struct { + name string + json string + want uploads.Probe + }{ + { + name: "audio-only m4a", + json: `{"streams":[{"codec_type":"audio","disposition":{"attached_pic":0}}],"format":{"duration":"2701.5"}}`, + want: uploads.Probe{Duration: 2701*time.Second + 500*time.Millisecond, HasAudio: true}, + }, + { + name: "mp4 with a real video stream", + json: `{"streams":[{"codec_type":"video","disposition":{"attached_pic":0}},{"codec_type":"audio"}],"format":{"duration":"240.0"}}`, + want: uploads.Probe{Duration: 240 * time.Second, HasAudio: true, HasVideo: true}, + }, + { + name: "mp3 whose only video stream is attached cover art is NOT video", + json: `{"streams":[{"codec_type":"audio"},{"codec_type":"video","disposition":{"attached_pic":1}}],"format":{"duration":"60"}}`, + want: uploads.Probe{Duration: 60 * time.Second, HasAudio: true}, + }, + { + name: "no streams at all", + json: `{"streams":[],"format":{"duration":"10"}}`, + want: uploads.Probe{Duration: 10 * time.Second}, + }, + { + name: "duration absent means zero, which Validate rejects as unreadable", + json: `{"streams":[{"codec_type":"audio"}],"format":{}}`, + want: uploads.Probe{HasAudio: true}, + }, + { + name: "duration reported as N/A means zero", + json: `{"streams":[{"codec_type":"audio"}],"format":{"duration":"N/A"}}`, + want: uploads.Probe{HasAudio: true}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := audio.ParseProbe([]byte(tt.json)) + + require.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} + +func TestParseProbe_GarbageIsAnError(t *testing.T) { + _, err := audio.ParseProbe([]byte("not json")) + assert.Error(t, err) +} + +// stub writes an executable shell script standing in for ffprobe. This is a fake +// at the process boundary: the real binary is not on every host (it is not on +// koala), and what is under test is how Probe drives and interprets a process. +func stub(t *testing.T, body string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "ffprobe") + require.NoError(t, os.WriteFile(p, []byte("#!/bin/sh\n"+body+"\n"), 0o755)) + return p +} + +func TestFFprobeProbe_ParsesWhatTheProcessPrints(t *testing.T) { + bin := stub(t, `echo '{"streams":[{"codec_type":"audio"}],"format":{"duration":"90"}}'`) + + got, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/ignored") + + require.NoError(t, err) + assert.Equal(t, uploads.Probe{Duration: 90 * time.Second, HasAudio: true}, got) +} + +func TestFFprobeProbe_PassesTheFilePathAsTheLastArgument(t *testing.T) { + out := filepath.Join(t.TempDir(), "args") + bin := stub(t, `for a; do last="$a"; done; printf '%s' "$last" > `+out+` +echo '{"streams":[{"codec_type":"audio"}],"format":{"duration":"1"}}'`) + + _, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/spool/upload-123") + + require.NoError(t, err) + got, err := os.ReadFile(out) + require.NoError(t, err) + assert.Equal(t, "/spool/upload-123", string(got)) +} + +func TestFFprobeProbe_NonzeroExitMeansTheFileIsUnreadable(t *testing.T) { + bin := stub(t, `echo "Invalid data found when processing input" >&2; exit 1`) + + _, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/x") + + assert.ErrorIs(t, err, uploads.ErrUnreadable) +} + +func TestFFprobeProbe_MissingBinaryIsOurFaultNotAnUnreadableFile(t *testing.T) { + _, err := (&audio.FFprobe{Bin: filepath.Join(t.TempDir(), "no-such-ffprobe")}).Probe(context.Background(), "/x") + + require.Error(t, err) + assert.False(t, errors.Is(err, uploads.ErrUnreadable), + "a missing ffprobe must not be reported to the user as their file being bad") +} + +func TestFFprobeProbe_StopsWhenTheContextIsCancelled(t *testing.T) { + bin := stub(t, `sleep 30`) + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + + start := time.Now() + _, err := (&audio.FFprobe{Bin: bin}).Probe(ctx, "/x") + + require.Error(t, err) + assert.Less(t, time.Since(start), 5*time.Second) + assert.False(t, errors.Is(err, uploads.ErrUnreadable), "a timeout is not a verdict on the file") +} + +// argvStub records the arguments it was called with into a file and then behaves as +// a successful ffprobe, so a test can assert exactly what the process was asked. +func argvStub(t *testing.T) (bin, argvFile string) { + t.Helper() + argvFile = filepath.Join(t.TempDir(), "argv") + bin = stub(t, `printf '%s\n' "$@" > `+argvFile+` +echo '{"streams":[{"codec_type":"audio"}],"format":{"duration":"1"}}'`) + return bin, argvFile +} + +func argvOf(t *testing.T, file string) []string { + t.Helper() + raw, err := os.ReadFile(file) + require.NoError(t, err) + return strings.Split(strings.TrimSpace(string(raw)), "\n") +} + +func indexOf(args []string, want string) int { + for i, a := range args { + if a == want { + return i + } + } + return -1 +} + +func TestFFprobeProbe_RestrictsWhatTheFileMayMakeFFmpegOpen(t *testing.T) { + // The input is attacker-supplied. ffmpeg can be told by a file's CONTENT to open + // other files or URLs (HLS and concat playlists); recent versions refuse by + // default, but that is version luck. Pin it: local files and pipes only, and + // only the demuxers real audio recordings use. + bin, argvFile := argvStub(t) + + _, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/spool/upload-1") + + require.NoError(t, err) + args := argvOf(t, argvFile) + pw := indexOf(args, "-protocol_whitelist") + require.GreaterOrEqual(t, pw, 0, "argv: %v", args) + assert.Equal(t, "file,pipe", args[pw+1]) + fw := indexOf(args, "-format_whitelist") + require.GreaterOrEqual(t, fw, 0, "argv: %v", args) + assert.Equal(t, audio.AllowedDemuxers, args[fw+1]) + assert.Equal(t, "/spool/upload-1", args[len(args)-1], "the path stays the last argument") +} + +func TestAllowedDemuxers_ExcludeThePlaylistFormats(t *testing.T) { + for _, bad := range []string{"concat", "hls", "applehttp", "dash", "rtsp", "sdp", "image2"} { + assert.NotContains(t, strings.Split(audio.AllowedDemuxers, ","), bad, + "%s can make ffmpeg open other files or URLs named inside the upload", bad) + } + for _, good := range []string{"wav", "mov", "mp4", "m4a", "mp3", "ogg", "matroska", "flac"} { + assert.Contains(t, strings.Split(audio.AllowedDemuxers, ","), good) + } +} diff --git a/internal/adapters/audio/hostile_live_test.go b/internal/adapters/audio/hostile_live_test.go new file mode 100644 index 0000000..35a0958 --- /dev/null +++ b/internal/adapters/audio/hostile_live_test.go @@ -0,0 +1,32 @@ +package audio_test + +import ( + "context" + "os" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// TestFFprobeLive_HostilePlaylistsAreRefusedAndRealAudioIsNot runs the REAL ffprobe +// (opt-in: TAPIR_LIVE_FFPROBE) on crafted concat/HLS files and on a real m4a. +func TestFFprobeLive_HostilePlaylistsAreRefusedAndRealAudioIsNot(t *testing.T) { + bin := os.Getenv("TAPIR_LIVE_FFPROBE") + if bin == "" { + t.Skip("set TAPIR_LIVE_FFPROBE (and TAPIR_LIVE_HOSTILE_DIR, TAPIR_LIVE_M4A) to run against the real ffprobe") + } + p := &audio.FFprobe{Bin: bin} + for _, name := range []string{"up-concat", "up-hls"} { + _, err := p.Probe(context.Background(), os.Getenv("TAPIR_LIVE_HOSTILE_DIR")+"/"+name) + assert.ErrorIs(t, err, uploads.ErrUnreadable, name) + } + got, err := p.Probe(context.Background(), os.Getenv("TAPIR_LIVE_M4A")) + require.NoError(t, err) + assert.True(t, got.HasAudio) + assert.False(t, got.HasVideo) + assert.InDelta(t, 82.5, got.Duration.Seconds(), 1) +} diff --git a/internal/adapters/chat/chat.go b/internal/adapters/chat/chat.go index 8cd0610..a774260 100644 --- a/internal/adapters/chat/chat.go +++ b/internal/adapters/chat/chat.go @@ -41,6 +41,11 @@ type Request struct { Transcript string History []Turn Question string + // MaxChars, when positive, overrides the Service's transcript budget for this + // request. The Service default is sized for YouTube captions (~22 minutes of + // speech); an uploaded recording is much longer, and truncation keeps only the + // head (ADR-033). Zero or negative means "use the Service default". + MaxChars int } // Reply is the model's answer plus whether the transcript was bounded to fit the @@ -113,7 +118,11 @@ func (s *Service) Answer(ctx context.Context, req Request) (Reply, error) { model = s.DefaultModel("") } - transcript, truncated := truncate(req.Transcript, s.maxChars) + budget := s.maxChars + if req.MaxChars > 0 { + budget = req.MaxChars + } + transcript, truncated := truncate(req.Transcript, budget) system := buildSystem(transcript, truncated) user := buildUser(req.History, req.Question) diff --git a/internal/adapters/chat/maxchars_test.go b/internal/adapters/chat/maxchars_test.go new file mode 100644 index 0000000..87f1275 --- /dev/null +++ b/internal/adapters/chat/maxchars_test.go @@ -0,0 +1,52 @@ +package chat + +import ( + "context" + "strings" + "testing" +) + +// A Request may carry its own transcript budget. The Service's cap (18,000 chars by +// default) is sized for YouTube captions; an uploaded recording is much longer +// (~50,000 chars an hour), and keeping only the head of it would answer questions +// about the last two-thirds of a meeting from text that does not contain them +// (ADR-033). The per-request value overrides; zero means "the service default". + +func TestAnswerMaxCharsOverridesTheServiceCap(t *testing.T) { + tests := []struct { + name string + serviceCap int + requestCap int + transcriptLen int + wantKept int + wantTruncated bool + }{ + {"request cap above the service cap keeps the whole transcript", 10, 100, 50, 50, false}, + {"request cap below the service cap bounds tighter", 100, 10, 50, 10, true}, + {"zero request cap falls back to the service cap", 20, 0, 50, 20, true}, + {"negative request cap is treated as unset", 20, -1, 50, 20, true}, + {"request cap exceeded is reported as truncated", 10, 30, 50, 30, true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := &factory{answer: "answer"} + s := New(f.make, []string{"koala/phi4-mini"}, tt.serviceCap) + + reply, err := s.Answer(context.Background(), Request{ + Model: "koala/phi4-mini", + Transcript: strings.Repeat("x", tt.transcriptLen), + Question: "what was decided?", + MaxChars: tt.requestCap, + }) + if err != nil { + t.Fatalf("Answer: %v", err) + } + if got := strings.Count(f.used.lastSystem, "x"); got != tt.wantKept { + t.Errorf("transcript chars sent = %d, want %d", got, tt.wantKept) + } + if reply.Truncated != tt.wantTruncated { + t.Errorf("Truncated = %v, want %v", reply.Truncated, tt.wantTruncated) + } + }) + } +} diff --git a/internal/adapters/llm/client.go b/internal/adapters/llm/client.go index f2fc1a6..33a3688 100644 --- a/internal/adapters/llm/client.go +++ b/internal/adapters/llm/client.go @@ -25,12 +25,16 @@ import ( // docs/homelab-integration.md. const defaultMaxTokens = 8192 +// defaultTemperature is what every caller got before WithTemperature existed. +const defaultTemperature = 0.2 + // Client calls an OpenAI-compatible chat completions endpoint. type Client struct { baseURL string apiKey string model string maxTokens int + temp float64 httpClient *http.Client usageHook func(model string, prompt, completion int) } @@ -51,6 +55,35 @@ func WithMaxTokens(n int) Option { } } +// maxErrorBody bounds how much of a failing response ends up in an error message. +const maxErrorBody = 512 + +// HTTPError is a non-200 answer from the gateway. Its text keeps the form callers that +// log it already rely on, with the body bounded; its Code is what code may branch on and +// what errkind turns into a kind. The body is provider text and can echo the prompt, so +// nothing private may persist or display Error() (ADR-035 review F3). +type HTTPError struct { + Code int + Body string +} + +func (e *HTTPError) Error() string { + body := e.Body + if len(body) > maxErrorBody { + body = body[:maxErrorBody] + "…" + } + return fmt.Sprintf("LLM returned %d: %s", e.Code, body) +} + +// HTTPStatus implements errkind.HTTPStatuser. +func (e *HTTPError) HTTPStatus() int { return e.Code } + +// WithTemperature overrides the sampling temperature. Zero IS a valid value and is sent +// as zero: the reconciler's accuracy was measured at temperature 0 and must run there. +func WithTemperature(t float64) Option { + return func(c *Client) { c.temp = t } +} + // WithUsageHook registers a callback fired after a successful completion with the // model and the prompt/completion token counts from the response usage block. It // keeps this copied, stdlib-only package (ADR-004) decoupled from metrics: the @@ -66,6 +99,7 @@ func New(baseURL, apiKey, model string, timeout time.Duration, opts ...Option) * apiKey: apiKey, model: model, maxTokens: defaultMaxTokens, + temp: defaultTemperature, httpClient: &http.Client{Timeout: timeout}, } for _, opt := range opts { @@ -105,7 +139,7 @@ func (c *Client) Complete(ctx context.Context, system, user string) (string, err {Role: "system", Content: system}, {Role: "user", Content: user}, }, - Temperature: 0.2, + Temperature: c.temp, MaxTokens: c.maxTokens, } b, err := json.Marshal(body) @@ -155,7 +189,7 @@ func (c *Client) Complete(ctx context.Context, system, user string) (string, err return "", fmt.Errorf("read response: %w", err) } if resp.StatusCode != http.StatusOK { - return "", fmt.Errorf("LLM returned %d: %s", resp.StatusCode, out) + return "", &HTTPError{Code: resp.StatusCode, Body: string(out)} } var cr chatResponse diff --git a/internal/adapters/llm/status_error_test.go b/internal/adapters/llm/status_error_test.go new file mode 100644 index 0000000..8e9b68c --- /dev/null +++ b/internal/adapters/llm/status_error_test.go @@ -0,0 +1,63 @@ +package llm + +import ( + "context" + "errors" + "net/http" + "net/http/httptest" + "strings" + "testing" + "time" + + "git.d-ma.be/mathias/tapir/internal/errkind" +) + +// A non-200 from the gateway is an HTTP status error: its kind can be logged and stored +// without its body, which can echo the prompt (adversarial review F3). + +func failingGateway(t *testing.T, status int, body string) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(status) + _, _ = w.Write([]byte(body)) + })) + t.Cleanup(srv.Close) + return srv +} + +func TestClient_ANon200IsAnHTTPStatusErrorWhoseKindCarriesNoBody(t *testing.T) { + srv := failingGateway(t, http.StatusBadRequest, `{"error":"prompt was: Anders ansvarar för virket"}`) + + _, err := New(srv.URL, "", "m", 5*time.Second).Complete(context.Background(), "s", "u") + + if err == nil { + t.Fatal("expected an error") + } + if got := errkind.Of(err); got != "http_400" { + t.Errorf("errkind.Of = %q, want http_400", got) + } + var he *HTTPError + if !errors.As(err, &he) || he.Code != http.StatusBadRequest { + t.Errorf("error is not an *HTTPError with the status code: %v", err) + } +} + +func TestClient_TheErrorTextIsUnchangedForCallersThatLogIt(t *testing.T) { + srv := failingGateway(t, http.StatusBadGateway, "upstream down") + + _, err := New(srv.URL, "", "m", 5*time.Second).Complete(context.Background(), "s", "u") + + if err == nil || err.Error() != "LLM returned 502: upstream down" { + t.Errorf("error text = %v: callers that log it already rely on this form", err) + } +} + +func TestClient_AnEnormousErrorBodyIsBoundedInTheErrorText(t *testing.T) { + srv := failingGateway(t, http.StatusInternalServerError, strings.Repeat("x", 100_000)) + + _, err := New(srv.URL, "", "m", 5*time.Second).Complete(context.Background(), "s", "u") + + if err == nil || len(err.Error()) > 1024 { + t.Errorf("error text is %d bytes: a gateway's whole body must not become an error message", len(err.Error())) + } +} diff --git a/internal/adapters/llm/temperature_test.go b/internal/adapters/llm/temperature_test.go new file mode 100644 index 0000000..7a227e7 --- /dev/null +++ b/internal/adapters/llm/temperature_test.go @@ -0,0 +1,57 @@ +package llm + +import ( + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "testing" + "time" +) + +// capturedTemperature runs one Complete against a fake gateway and returns the +// "temperature" field the request carried, and whether it was present at all. +func capturedTemperature(t *testing.T, opts ...Option) (float64, bool) { + t.Helper() + var body map[string]any + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + raw, _ := io.ReadAll(r.Body) + _ = json.Unmarshal(raw, &body) + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(map[string]any{"choices": []map[string]any{{"message": map[string]any{"role": "assistant", "content": "ok"}}}}) + })) + defer srv.Close() + + if _, err := New(srv.URL, "", "m", 5*time.Second, opts...).Complete(context.Background(), "s", "u"); err != nil { + t.Fatalf("Complete: %v", err) + } + v, ok := body["temperature"].(float64) + return v, ok +} + +func TestClient_DefaultTemperatureIsUnchanged(t *testing.T) { + got, ok := capturedTemperature(t) + + if !ok || got != 0.2 { + t.Fatalf("default temperature = %v (present %v), want 0.2: existing callers must not change behaviour", got, ok) + } +} + +func TestClient_WithTemperatureZeroSendsExactlyZero(t *testing.T) { + // The reconciler's accuracy (92%/97%) was measured at temperature 0. Zero must be + // SENT, not treated as "unset" and replaced by the default. + got, ok := capturedTemperature(t, WithTemperature(0)) + + if !ok || got != 0 { + t.Fatalf("temperature = %v (present %v), want an explicit 0", got, ok) + } +} + +func TestClient_WithTemperatureSendsTheGivenValue(t *testing.T) { + got, ok := capturedTemperature(t, WithTemperature(0.7)) + + if !ok || got != 0.7 { + t.Fatalf("temperature = %v (present %v), want 0.7", got, ok) + } +} diff --git a/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql b/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql new file mode 100644 index 0000000..8040680 --- /dev/null +++ b/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql @@ -0,0 +1,3 @@ +-- Drops the private uploaded-transcript store and every transcript in it. There +-- is no revert of the data: the original audio was never kept (ADR-033). +DROP TABLE uploaded_transcripts; diff --git a/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql b/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql new file mode 100644 index 0000000..bdd311a --- /dev/null +++ b/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql @@ -0,0 +1,48 @@ +-- Migration 017: uploaded_transcripts — PRIVATE transcripts of audio a user +-- uploaded themselves (ADR-033, tapir#33). +-- +-- This is deliberately NOT the shared `transcripts` table (migration 015, +-- ADR-021). That table is non-RLS on the explicit grounds that it holds only +-- public caption content and nothing user-identifying. A recorded meeting is the +-- opposite: private third-party content owned by exactly one user. Reusing +-- `transcripts` — or adding a user_id to it with a partial policy — would put a +-- private row in the one table the isolation test asserts is shared, and would +-- pass CI green. A separate table keeps ADR-021's exception singular. +-- +-- Row-level security follows migration 003 exactly: ENABLE + FORCE (the app +-- connects as the table owner, which bypasses RLS unless forced), and a policy +-- keyed off the per-request GUC tapir.current_user_id. With no GUC set the +-- predicate is NULL and the table is deny-all. +-- +-- The original audio is never stored (ADR-033): this row is all that is kept. +-- segments holds the timestamped cues; re-transcribing 90 minutes costs ~7 +-- minutes, so discarding the timestamps would be irreversible. +CREATE TABLE uploaded_transcripts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, + title TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'queued' + CHECK (status IN ('queued', 'transcribing', 'done', 'failed')), + content TEXT, + segments JSONB, + endpoint TEXT, -- which backend served it (iguana | berget); a quality fact + audio_seconds DOUBLE PRECISION, + error_kind TEXT, -- typed, never a provider's error string (ADR-033) + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + finished_at TIMESTAMPTZ, + CONSTRAINT uploaded_transcripts_done_has_content + CHECK (status <> 'done' OR content IS NOT NULL), + CONSTRAINT uploaded_transcripts_failed_has_error_kind + CHECK (status <> 'failed' OR error_kind IS NOT NULL) +); + +CREATE INDEX idx_uploaded_transcripts_user ON uploaded_transcripts (user_id, created_at DESC); + +ALTER TABLE uploaded_transcripts ENABLE ROW LEVEL SECURITY; +ALTER TABLE uploaded_transcripts FORCE ROW LEVEL SECURITY; +CREATE POLICY uploaded_transcripts_isolation ON uploaded_transcripts + FOR ALL + USING (user_id = current_setting('tapir.current_user_id', true)::uuid); + +COMMENT ON TABLE uploaded_transcripts IS + 'PRIVATE per-user transcripts of user-supplied audio. RLS-scoped (ADR-012). Never merge into the shared transcripts table (ADR-021, ADR-033).'; diff --git a/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.down.sql b/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.down.sql new file mode 100644 index 0000000..8e08fad --- /dev/null +++ b/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.down.sql @@ -0,0 +1 @@ +ALTER TABLE uploaded_transcripts DROP COLUMN had_video; diff --git a/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.up.sql b/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.up.sql new file mode 100644 index 0000000..6a560e3 --- /dev/null +++ b/internal/adapters/store/migrations/018_uploaded_transcripts_had_video.up.sql @@ -0,0 +1,9 @@ +-- Migration 018: record whether an uploaded file carried a video track (ADR-033, +-- amended 2026-10-01). +-- +-- A Teams meeting with the camera off is still an .mp4 with a video track, so video +-- files are ACCEPTED: only the audio is extracted and the video is discarded. This +-- flag lets the transcript page say so — "this file contained a video track; only its +-- audio was used" — so the user is never left to wonder what happened to the picture. +-- It outlives the job: completing or failing an upload does not clear it. +ALTER TABLE uploaded_transcripts ADD COLUMN had_video BOOLEAN NOT NULL DEFAULT FALSE; diff --git a/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.down.sql b/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.down.sql new file mode 100644 index 0000000..46eba75 --- /dev/null +++ b/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.down.sql @@ -0,0 +1 @@ +ALTER TABLE uploaded_transcripts DROP COLUMN reconciliation; diff --git a/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.up.sql b/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.up.sql new file mode 100644 index 0000000..433870a --- /dev/null +++ b/internal/adapters/store/migrations/019_uploaded_transcripts_reconciliation.up.sql @@ -0,0 +1,12 @@ +-- Migration 019: how a merged transcript came about (ADR-034). +-- +-- When two engines transcribed the same audio and a local model chose between their +-- disagreements, this records the status, the exact counts, and a bounded list of the +-- contested spots (changed or unsure, each with a position and both readings) so the +-- page can say what was done and the user can check the uncertain spots against the +-- audio. NULL means a single engine: no second opinion, and the page must not claim one. +-- +-- It lives on the already-FORCE'd, RLS-scoped uploaded_transcripts, so the exact-set +-- isolation test is unaffected. The contested spots hold fragments of the private +-- transcript, which is why they are here and nowhere else. +ALTER TABLE uploaded_transcripts ADD COLUMN reconciliation JSONB; diff --git a/internal/adapters/store/migrations/020_uploaded_transcripts_progress.down.sql b/internal/adapters/store/migrations/020_uploaded_transcripts_progress.down.sql new file mode 100644 index 0000000..f80bcd2 --- /dev/null +++ b/internal/adapters/store/migrations/020_uploaded_transcripts_progress.down.sql @@ -0,0 +1,4 @@ +ALTER TABLE uploaded_transcripts + DROP COLUMN alive_at, + DROP COLUMN progress_at, + DROP COLUMN progress; diff --git a/internal/adapters/store/migrations/020_uploaded_transcripts_progress.up.sql b/internal/adapters/store/migrations/020_uploaded_transcripts_progress.up.sql new file mode 100644 index 0000000..40d373e --- /dev/null +++ b/internal/adapters/store/migrations/020_uploaded_transcripts_progress.up.sql @@ -0,0 +1,15 @@ +-- Migration 020: where an upload job is, and whether the service is still reporting (ADR-035). +-- +-- progress the job's ordered phases as JSON (key, label, state, times, done/total). +-- progress_at when a phase last REALLY changed or advanced. +-- alive_at a heartbeat stamped every few seconds by a goroutine independent of any +-- engine call. The two differ on purpose: alive says the service is still +-- reporting, progress says work is moving, and a page that cannot tell a +-- crashed service from a slow model is the failure this exists to end. +-- +-- All three are NULL until the job starts. They sit on the already-FORCE'd, RLS-scoped +-- uploaded_transcripts, so the exact-set isolation test is unaffected. +ALTER TABLE uploaded_transcripts + ADD COLUMN progress JSONB, + ADD COLUMN progress_at TIMESTAMPTZ, + ADD COLUMN alive_at TIMESTAMPTZ; diff --git a/internal/adapters/store/uploaded_transcripts.go b/internal/adapters/store/uploaded_transcripts.go new file mode 100644 index 0000000..71d289b --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts.go @@ -0,0 +1,249 @@ +package store + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "time" + + "github.com/jackc/pgx/v5" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrUploadNotFound is returned when an uploaded transcript does not exist OR is +// not the caller's. The two are deliberately indistinguishable: a non-owner learns +// nothing about whether someone else's upload exists (ADR-033). +var ErrUploadNotFound = errors.New("store: uploaded transcript not found") + +// Uploaded transcripts are PRIVATE per-user content, stored in their own +// RLS-scoped table (migration 017) — never in the shared `transcripts` (ADR-021). +// Every method goes through withUser, so row-level security confines it to the +// caller's rows; each statement ALSO names user_id in its predicate, so the +// isolation does not rest on a single mechanism and stays testable under the +// superuser connection the unit tests use. + +// CreateUpload records a new upload in the queued state and returns its id. +// hadVideo records that the file carried a video track (only its audio is used); +// it is kept for the life of the row so the page can say so. +func (s *Store) CreateUpload(ctx context.Context, userID, title string, hadVideo bool) (string, error) { + var id string + if err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + return tx.QueryRow(ctx, + `INSERT INTO uploaded_transcripts (user_id, title, had_video) VALUES ($1, $2, $3) RETURNING id`, + userID, title, hadVideo).Scan(&id) + }); err != nil { + return "", fmt.Errorf("store: create upload: %w", err) + } + return id, nil +} + +// StartTranscribing moves an upload to transcribing. +func (s *Store) StartTranscribing(ctx context.Context, userID, id string) error { + return s.updateUpload(ctx, userID, id, "start transcribing", + `UPDATE uploaded_transcripts SET status = 'transcribing' + WHERE id = $1 AND user_id = $2`) +} + +// CompleteUpload stores the finished transcription and moves the upload to done. +func (s *Store) CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error { + segs, err := json.Marshal(t.Segments) + if err != nil { + return fmt.Errorf("store: complete upload: marshal segments: %w", err) + } + // A nil Reconciliation must be SQL NULL, not the JSON text "null": the page decides + // whether to claim a second opinion by whether this column is NULL. + var rec []byte + if t.Reconciliation != nil { + if rec, err = json.Marshal(t.Reconciliation); err != nil { + return fmt.Errorf("store: complete upload: marshal reconciliation: %w", err) + } + } + return s.updateUpload(ctx, userID, id, "complete upload", + `UPDATE uploaded_transcripts + SET status = 'done', content = $3, segments = $4, endpoint = $5, + audio_seconds = $6, reconciliation = $7, error_kind = NULL, finished_at = NOW() + WHERE id = $1 AND user_id = $2`, + t.Text, segs, t.Endpoint, t.AudioSeconds, rec) +} + +// SaveProgress records the job's phases and when something last really changed +// (ADR-035). It replaces the previous snapshot: the Tracker holds the whole plan. +func (s *Store) SaveProgress(ctx context.Context, userID, id string, steps []domain.ProgressStep, at time.Time) error { + b, err := json.Marshal(steps) + if err != nil { + return fmt.Errorf("store: save progress: marshal: %w", err) + } + return s.updateUpload(ctx, userID, id, "save progress", + `UPDATE uploaded_transcripts SET progress = $3, progress_at = $4 + WHERE id = $1 AND user_id = $2`, b, at) +} + +// Heartbeat stamps the upload as alive now. It does not touch progress_at: alive says +// the service is still reporting, not that work is moving (ADR-035). +func (s *Store) Heartbeat(ctx context.Context, userID, id string) error { + return s.updateUpload(ctx, userID, id, "heartbeat", + `UPDATE uploaded_transcripts SET alive_at = NOW() + WHERE id = $1 AND user_id = $2`) +} + +// FailUpload moves an upload to failed with a typed reason. +func (s *Store) FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error { + return s.updateUpload(ctx, userID, id, "fail upload", + `UPDATE uploaded_transcripts + SET status = 'failed', error_kind = $3, finished_at = NOW() + WHERE id = $1 AND user_id = $2`, + string(kind)) +} + +// updateUpload runs one UPDATE keyed on ($1 = id, $2 = userID) plus extra args, +// and maps "no row touched" to ErrUploadNotFound — which is what a write to another +// user's upload looks like under RLS (the row is invisible, not forbidden). +func (s *Store) updateUpload(ctx context.Context, userID, id, op, sql string, args ...any) error { + all := append([]any{id, userID}, args...) + return s.withUser(ctx, userID, func(tx pgx.Tx) error { + ct, err := tx.Exec(ctx, sql, all...) + if err != nil { + return fmt.Errorf("store: %s: %w", op, err) + } + if ct.RowsAffected() == 0 { + return ErrUploadNotFound + } + return nil + }) +} + +// GetUpload returns one of the caller's uploads, including the transcript text. +func (s *Store) GetUpload(ctx context.Context, userID, id string) (*domain.UploadedTranscript, error) { + var u domain.UploadedTranscript + err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + row := tx.QueryRow(ctx, uploadSelect+` WHERE id = $1 AND user_id = $2`, id, userID) + return scanUpload(row, &u, true) + }) + if errors.Is(err, pgx.ErrNoRows) { + return nil, ErrUploadNotFound + } + if err != nil { + return nil, fmt.Errorf("store: get upload: %w", err) + } + return &u, nil +} + +// ListUploads returns the caller's uploads, newest first, WITHOUT transcript text +// or segments: a list view must not drag every transcript along. +func (s *Store) ListUploads(ctx context.Context, userID string) ([]domain.UploadedTranscript, error) { + var out []domain.UploadedTranscript + err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + rows, err := tx.Query(ctx, uploadSelect+` WHERE user_id = $1 ORDER BY created_at DESC, id`, userID) + if err != nil { + return err + } + defer rows.Close() + for rows.Next() { + var u domain.UploadedTranscript + if err := scanUpload(rows, &u, false); err != nil { + return err + } + out = append(out, u) + } + return rows.Err() + }) + if err != nil { + return nil, fmt.Errorf("store: list uploads: %w", err) + } + return out, nil +} + +// FailInterruptedUploads is the start-up sweep. The audio of an upload is never +// retained (ADR-033), so a job that was queued or transcribing when the process +// stopped can never resume; leaving it would be a stuck spinner. Every such row, +// for every user, becomes failed/interrupted and the user re-uploads. It +// enumerates users from the un-RLS'd identity map and scopes each pass with +// withUser, exactly as scheduled discovery does (ADR-018). Returns rows failed. +func (s *Store) FailInterruptedUploads(ctx context.Context) (int, error) { + users, err := s.ListAllUsers(ctx) + if err != nil { + return 0, fmt.Errorf("store: fail interrupted uploads: %w", err) + } + total := 0 + for _, u := range users { + err := s.withUser(ctx, u.UserID, func(tx pgx.Tx) error { + ct, err := tx.Exec(ctx, + `UPDATE uploaded_transcripts + SET status = 'failed', error_kind = $2, finished_at = NOW() + WHERE user_id = $1 AND status IN ('queued', 'transcribing')`, + u.UserID, string(domain.UploadErrInterrupted)) + total += int(ct.RowsAffected()) + return err + }) + if err != nil { + return total, fmt.Errorf("store: fail interrupted uploads: user %s: %w", u.UserID, err) + } + } + return total, nil +} + +const uploadSelect = `SELECT id, user_id, title, status, content, segments, endpoint, + audio_seconds, error_kind, had_video, reconciliation, progress, progress_at, alive_at, + created_at, finished_at FROM uploaded_transcripts` + +// scanUpload scans one uploadSelect row. withText controls whether the transcript +// body and segments are kept (GetUpload) or dropped (ListUploads). +func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error { + var ( + status, errKind, endpoint, content *string + segments, reconciliation, progress []byte + seconds *float64 + finished, progressAt, aliveAt *time.Time + ) + if err := row.Scan(&u.ID, &u.UserID, &u.Title, &status, &content, &segments, &endpoint, + &seconds, &errKind, &u.HadVideo, &reconciliation, &progress, &progressAt, &aliveAt, + &u.CreatedAt, &finished); err != nil { + return err + } + u.Status = domain.UploadStatus(deref(status)) + u.Endpoint = deref(endpoint) + u.ErrorKind = domain.UploadErrorKind(deref(errKind)) + if seconds != nil { + u.AudioSeconds = *seconds + } + if finished != nil { + u.FinishedAt = *finished + } + if !withText { + return nil + } + if len(progress) > 0 { + if err := json.Unmarshal(progress, &u.Progress); err != nil { + return fmt.Errorf("unmarshal progress: %w", err) + } + } + if progressAt != nil { + u.ProgressAt = *progressAt + } + if aliveAt != nil { + u.AliveAt = *aliveAt + } + if len(reconciliation) > 0 { + var r domain.Reconciliation + if err := json.Unmarshal(reconciliation, &r); err != nil { + return fmt.Errorf("unmarshal reconciliation: %w", err) + } + u.Reconciliation = &r + } + u.Content = deref(content) + if len(segments) > 0 { + if err := json.Unmarshal(segments, &u.Segments); err != nil { + return fmt.Errorf("unmarshal segments: %w", err) + } + } + return nil +} + +func deref(s *string) string { + if s == nil { + return "" + } + return *s +} diff --git a/internal/adapters/store/uploaded_transcripts_isolation_test.go b/internal/adapters/store/uploaded_transcripts_isolation_test.go new file mode 100644 index 0000000..d8bd6f0 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts_isolation_test.go @@ -0,0 +1,196 @@ +package store_test + +import ( + "context" + "sort" + "testing" + + "github.com/jackc/pgx/v5/pgxpool" + "github.com/stretchr/testify/require" +) + +// Isolation proof for ADR-033: an uploaded transcript is PRIVATE content owned by +// one user, so it lives in its own RLS-scoped table — never in the shared, +// non-RLS `transcripts` (ADR-021). These tests drive the database directly (raw +// SQL as the non-superuser `app` role) so they observe row-level security itself, +// not any application-layer WHERE clause. The superuser pool is used only to seed +// and to read pg_catalog; it bypasses RLS and so must never be the pool whose +// answers the assertions rest on. + +// seedUploadedTranscript inserts one finished upload for userID as the superuser +// (RLS bypassed so both users' rows land regardless of any GUC) and returns its id. +// The text is synthetic — this repo is public, and a real transcript must never +// appear in it (#33). +func seedUploadedTranscript(t *testing.T, super *pgxpool.Pool, userID, text string) string { + t.Helper() + ctx := context.Background() + _, err := super.Exec(ctx, `INSERT INTO users (id) VALUES ($1) ON CONFLICT DO NOTHING`, userID) + require.NoError(t, err) + var id string + require.NoError(t, super.QueryRow(ctx, + `INSERT INTO uploaded_transcripts (user_id, title, status, content) + VALUES ($1, 'synthetic fixture', 'done', $2) RETURNING id`, + userID, text).Scan(&id)) + return id +} + +func TestUploadedTranscriptsRLS_ScopedReadsSeeOnlyOwnRows(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + seedUploadedTranscript(t, super, userB, "text owned by B") + app := appPool(t, super) + + require.Equal(t, 0, scopedCount(t, app, "", "uploaded_transcripts"), + "no tapir.current_user_id must be deny-all: this proves RLS is live, not bypassed") + require.Equal(t, 1, scopedCount(t, app, userA, "uploaded_transcripts"), + "user A must see exactly its own upload") + require.Equal(t, 1, scopedCount(t, app, userB, "uploaded_transcripts"), + "user B must see exactly its own upload") + + // Count alone cannot distinguish "A's row" from "B's row" — read the content. + var got string + ctx := context.Background() + tx, err := app.Begin(ctx) + require.NoError(t, err) + defer tx.Rollback(ctx) //nolint:errcheck + _, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA) + require.NoError(t, err) + require.NoError(t, tx.QueryRow(ctx, `SELECT content FROM uploaded_transcripts`).Scan(&got)) + require.Equal(t, "text owned by A", got, "A's scoped read must return A's text, never B's") +} + +func TestUploadedTranscriptsRLS_CrossUserWritesAffectZeroRows(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + bID := seedUploadedTranscript(t, super, userB, "text owned by B") + app := appPool(t, super) + + require.Equal(t, int64(0), scopedRowsAffected(t, app, userA, + `UPDATE uploaded_transcripts SET content = 'hacked' WHERE id = $1`, bID), + "user A scoped UPDATE must touch zero of B's uploads") + require.Equal(t, int64(0), scopedRowsAffected(t, app, userA, + `DELETE FROM uploaded_transcripts WHERE id = $1`, bID), + "user A scoped DELETE must touch zero of B's uploads") + + var content string + require.NoError(t, super.QueryRow(context.Background(), + `SELECT content FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&content)) + require.Equal(t, "text owned by B", content, "B's upload must survive A's writes unchanged") +} + +func TestUploadedTranscriptsRLS_InsertForAnotherUserIsRejected(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + _, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, userB) + require.NoError(t, err) + app := appPool(t, super) + + ctx := context.Background() + tx, err := app.Begin(ctx) + require.NoError(t, err) + defer tx.Rollback(ctx) //nolint:errcheck + _, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA) + require.NoError(t, err) + + // Scoped as A, plant a row that claims to belong to B. The policy's WITH CHECK + // (defaulting to USING under FOR ALL) must refuse it. + _, err = tx.Exec(ctx, + `INSERT INTO uploaded_transcripts (user_id, title, status, content) + VALUES ($1, 'planted', 'done', 'planted')`, userB) + require.Error(t, err, "a row scoped as A must not be insertable on behalf of B") + require.Contains(t, err.Error(), "row-level security") +} + +func TestUploadedTranscripts_RowSecurityIsEnabledForcedAndKeyedByUser(t *testing.T) { + newStore(t) + super := rawPool(t) + ctx := context.Background() + + var enabled, forced bool + require.NoError(t, super.QueryRow(ctx, + `SELECT relrowsecurity, relforcerowsecurity FROM pg_class + WHERE oid = 'public.uploaded_transcripts'::regclass`).Scan(&enabled, &forced)) + require.True(t, enabled, "uploaded_transcripts must have row-level security enabled") + require.True(t, forced, + "FORCE is what binds the table-owner role tapir; without it the policy is dead in production") + + var userIDNotNull bool + require.NoError(t, super.QueryRow(ctx, + `SELECT is_nullable = 'NO' FROM information_schema.columns + WHERE table_name = 'uploaded_transcripts' AND column_name = 'user_id'`).Scan(&userIDNotNull)) + require.True(t, userIDNotNull, "an upload with no owner would match no policy and be unreachable") +} + +func TestUploadedTranscripts_DeleteUserRemovesTheirUploads(t *testing.T) { + s := newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + bID := seedUploadedTranscript(t, super, userB, "text owned by B") + + require.NoError(t, s.DeleteUser(context.Background(), userA)) + + var aRows, bRows int + ctx := context.Background() + require.NoError(t, super.QueryRow(ctx, + `SELECT count(*) FROM uploaded_transcripts WHERE user_id = $1`, userA).Scan(&aRows)) + require.NoError(t, super.QueryRow(ctx, + `SELECT count(*) FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&bRows)) + require.Equal(t, 0, aRows, "deleting a user must remove their private transcripts (ON DELETE CASCADE)") + require.Equal(t, 1, bRows, "deleting A must not touch B's transcripts") +} + +// rowSecurityOffTables are the public-schema tables that deliberately have row +// security OFF. The list is exact on purpose: a NEW table added without RLS must +// fail this test and force a decision, instead of passing CI by omission — the +// failure mode ADR-033 names, where a naive reuse "leaks private content across +// tenants and passes CI green". +// +// - transcripts: shared public caption content, the single exception for CONTENT +// (ADR-021). It is the one that must never gain a private row. +// - user_identities: the Dex-subject -> user_id map, read at login before any +// user is known (migration 004). Holds no user content. +// - invitations: exists before the user does, so there is no user to scope by; +// the single-use token is the capability (migration 009). +// - schema_migrations: golang-migrate bookkeeping. +// +// #33 describes `transcripts` as "the only non-RLS table". That is true of content +// and false of tables; this test asserts the real set. +var rowSecurityOffTables = []string{ + "invitations", + "schema_migrations", + "transcripts", + "user_identities", +} + +func TestRowSecurityOffTablesAreExactlyTheKnownSet(t *testing.T) { + newStore(t) + super := rawPool(t) + + rows, err := super.Query(context.Background(), + `SELECT c.relname FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p') AND NOT c.relrowsecurity + ORDER BY c.relname`) + require.NoError(t, err) + defer rows.Close() + var got []string + for rows.Next() { + var name string + require.NoError(t, rows.Scan(&name)) + got = append(got, name) + } + require.NoError(t, rows.Err()) + + want := append([]string{}, rowSecurityOffTables...) + sort.Strings(want) + require.Equal(t, want, got, + "every public table must have row security ON unless it is in rowSecurityOffTables; "+ + "adding a table here is a decision that needs an ADR") +} diff --git a/internal/adapters/store/uploaded_transcripts_progress_test.go b/internal/adapters/store/uploaded_transcripts_progress_test.go new file mode 100644 index 0000000..3456df2 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts_progress_test.go @@ -0,0 +1,137 @@ +package store_test + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// Progress and heartbeat of an upload job (ADR-035, migration 020). Like the other +// upload store tests these run as the superuser, so the cross-user cases prove the +// statement's own user_id predicate; the database-enforced isolation is covered by the +// RLS tests, and migration 020 adds columns to the already-FORCE'd table, not a table. + +func startedUpload(t *testing.T, s *store.Store) string { + t.Helper() + seedUsers(t, userA, userB) + id, err := s.CreateUpload(context.Background(), userA, "m", false) + require.NoError(t, err) + return id +} + +func TestUploadProgress_AFreshUploadHasNoProgressAndNoHeartbeat(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + + got, err := s.GetUpload(context.Background(), userA, id) + + require.NoError(t, err) + assert.Empty(t, got.Progress) + assert.True(t, got.ProgressAt.IsZero()) + assert.True(t, got.AliveAt.IsZero()) +} + +func TestUploadProgress_StepsAndTheirTimesRoundTripInOrder(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC) + steps := []domain.ProgressStep{ + {Key: "audio", Label: "Preparing audio", State: domain.StepDone, StartedAt: at, FinishedAt: at.Add(40 * time.Second)}, + {Key: "decide", Label: "Deciding", State: domain.StepRunning, StartedAt: at.Add(time.Minute), Done: 12, Total: 282}, + {Key: "save", Label: "Saving", State: domain.StepPending}, + } + + require.NoError(t, s.SaveProgress(ctx, userA, id, steps, at.Add(2*time.Minute))) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + require.Len(t, got.Progress, 3) + assert.Equal(t, []string{"audio", "decide", "save"}, []string{got.Progress[0].Key, got.Progress[1].Key, got.Progress[2].Key}) + assert.Equal(t, domain.StepRunning, got.Progress[1].State) + assert.Equal(t, 12, got.Progress[1].Done) + assert.Equal(t, 282, got.Progress[1].Total) + assert.True(t, got.Progress[0].FinishedAt.Equal(at.Add(40*time.Second)), "a step's finish time must survive the round trip") + assert.True(t, got.Progress[2].StartedAt.IsZero(), "a pending step has no start time, not a zero date") + assert.True(t, got.ProgressAt.Equal(at.Add(2*time.Minute))) +} + +func TestUploadProgress_AHeartbeatMovesAliveAtAndNotProgressAt(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC) + require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "audio", Label: "a", State: domain.StepRunning}}, at)) + before := time.Now().Add(-time.Second) + + require.NoError(t, s.Heartbeat(ctx, userA, id)) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.True(t, got.AliveAt.After(before), "the heartbeat must stamp the row with now") + assert.True(t, got.ProgressAt.Equal(at), "alive is not progress: a heartbeat must not move progress_at") +} + +func TestUploadProgress_ASecondSaveReplacesTheFirst(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC) + require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepRunning}}, at)) + + require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepDone}, {Key: "b", Label: "B", State: domain.StepRunning}}, at.Add(time.Minute))) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + require.Len(t, got.Progress, 2) + assert.Equal(t, domain.StepDone, got.Progress[0].State) +} + +func TestUploadProgress_ByAnotherUserChangesNothing(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + + errSave := s.SaveProgress(ctx, userB, id, []domain.ProgressStep{{Key: "x", Label: "X", State: domain.StepDone}}, time.Now()) + errBeat := s.Heartbeat(ctx, userB, id) + + assert.ErrorIs(t, errSave, store.ErrUploadNotFound) + assert.ErrorIs(t, errBeat, store.ErrUploadNotFound) + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Empty(t, got.Progress, "B's write must not have reached A's upload") + assert.True(t, got.AliveAt.IsZero()) +} + +func TestUploadProgress_ListDoesNotDragItAlong(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepRunning}}, time.Now())) + + rows, err := s.ListUploads(ctx, userA) + + require.NoError(t, err) + require.Len(t, rows, 1) + assert.Empty(t, rows[0].Progress, "the list view needs a status, not every row's phases") +} + +func TestUploadProgress_SurvivesCompletion(t *testing.T) { + s := newStore(t) + id := startedUpload(t, s) + ctx := context.Background() + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepDone}}, time.Now())) + + require.NoError(t, s.CompleteUpload(ctx, userA, id, domain.Transcription{Text: "x", Endpoint: "e"})) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + require.Len(t, got.Progress, 1, "a finished upload keeps its record of what was done, and how long each step took") +} diff --git a/internal/adapters/store/uploaded_transcripts_test.go b/internal/adapters/store/uploaded_transcripts_test.go new file mode 100644 index 0000000..efa1948 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts_test.go @@ -0,0 +1,286 @@ +package store_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// These exercise the Store API over uploaded_transcripts. The test Store connects +// as the superuser, which BYPASSES row-level security — so the cross-user cases +// here prove the query's own user_id predicate (defence in depth), while the +// database-enforced guarantee is proven separately, as the non-superuser app +// role, in uploaded_transcripts_isolation_test.go. Text is synthetic; this repo is +// public and no real transcript may appear in it (#33). + +func seedUsers(t *testing.T, ids ...string) { + t.Helper() + super := rawPool(t) + resetDB(t, super) + for _, id := range ids { + _, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, id) + require.NoError(t, err) + } +} + +func TestCreateUpload_StartsQueuedAndIsOwnedByTheCreator(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + + id, err := s.CreateUpload(ctx, userA, "Teams export", false) + require.NoError(t, err) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, id, got.ID) + assert.Equal(t, userA, got.UserID) + assert.Equal(t, "Teams export", got.Title) + assert.Equal(t, domain.UploadQueued, got.Status) + assert.Empty(t, got.Content) + assert.True(t, got.FinishedAt.IsZero(), "an unfinished upload has no finish time") +} + +func TestUploadLifecycle_CompletedKeepsTextSegmentsEndpointAndDuration(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m", false) + require.NoError(t, err) + + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + mid, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadTranscribing, mid.Status) + + want := domain.Transcription{ + Text: "God morgon allihopa. Vi går igenom läget.", + Segments: []domain.Segment{ + {Start: 0.0, End: 1.5, Text: "God morgon allihopa."}, + {Start: 1.5, End: 4.25, Text: "Vi går igenom läget."}, + }, + Endpoint: "iguana/kb-whisper", + AudioSeconds: 4.25, + } + require.NoError(t, s.CompleteUpload(ctx, userA, id, want)) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadDone, got.Status) + assert.Equal(t, want.Text, got.Content) + assert.Equal(t, want.Segments, got.Segments, "segment times must survive the JSONB round trip") + assert.Equal(t, "iguana/kb-whisper", got.Endpoint, "the serving endpoint is a quality fact and is recorded per row") + assert.InDelta(t, 4.25, got.AudioSeconds, 1e-9) + assert.False(t, got.FinishedAt.IsZero()) + assert.Empty(t, got.ErrorKind) +} + +func TestUploadLifecycle_FailedRecordsTypedKindAndNoContent(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + + require.NoError(t, s.FailUpload(ctx, userA, id, domain.UploadErrTranscriptionFailed)) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadFailed, got.Status) + assert.Equal(t, domain.UploadErrTranscriptionFailed, got.ErrorKind) + assert.Empty(t, got.Content) + assert.False(t, got.FinishedAt.IsZero()) +} + +func TestGetUpload_AnotherUsersIDIsNotFound(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "A's meeting", false) + require.NoError(t, err) + + _, err = s.GetUpload(ctx, userB, id) + + assert.ErrorIs(t, err, store.ErrUploadNotFound, + "a non-owner must get not-found, not forbidden: existence is not disclosed") +} + +func TestUploadWrites_ByAnotherUserChangeNothing(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "A's meeting", false) + require.NoError(t, err) + + assert.ErrorIs(t, s.StartTranscribing(ctx, userB, id), store.ErrUploadNotFound) + assert.ErrorIs(t, s.CompleteUpload(ctx, userB, id, domain.Transcription{Text: "planted"}), store.ErrUploadNotFound) + assert.ErrorIs(t, s.FailUpload(ctx, userB, id, domain.UploadErrTranscriptionFailed), store.ErrUploadNotFound) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadQueued, got.Status, "B's writes must leave A's upload exactly as it was") + assert.Empty(t, got.Content) +} + +func TestListUploads_ReturnsOnlyTheCallersNewestFirstWithoutTranscriptText(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + first, err := s.CreateUpload(ctx, userA, "first", false) + require.NoError(t, err) + second, err := s.CreateUpload(ctx, userA, "second", false) + require.NoError(t, err) + _, err = s.CreateUpload(ctx, userB, "B's", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, first)) + require.NoError(t, s.CompleteUpload(ctx, userA, first, domain.Transcription{Text: "long text body", Endpoint: "e"})) + + got, err := s.ListUploads(ctx, userA) + + require.NoError(t, err) + require.Len(t, got, 2, "A must see only A's uploads") + assert.Equal(t, second, got[0].ID, "newest first") + assert.Equal(t, first, got[1].ID) + assert.Empty(t, got[1].Content, "a list row must not drag the whole transcript along") + assert.Equal(t, domain.UploadDone, got[1].Status) +} + +func TestFailInterruptedUploads_FailsOnlyUnfinishedRowsForEveryUser(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + super := rawPool(t) + ctx := context.Background() + for subject, uid := range map[string]string{"sub-a": userA, "sub-b": userB} { + _, err := super.Exec(ctx, `INSERT INTO user_identities (dex_subject, user_id) VALUES ($1, $2)`, subject, uid) + require.NoError(t, err) + } + queued, err := s.CreateUpload(ctx, userA, "queued", false) + require.NoError(t, err) + running, err := s.CreateUpload(ctx, userB, "running", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userB, running)) + done, err := s.CreateUpload(ctx, userA, "done", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, done)) + require.NoError(t, s.CompleteUpload(ctx, userA, done, domain.Transcription{Text: "kept", Endpoint: "e"})) + + n, err := s.FailInterruptedUploads(ctx) + + require.NoError(t, err) + assert.Equal(t, 2, n, "the queued and the transcribing row, across both users") + for _, c := range []struct{ user, id string }{{userA, queued}, {userB, running}} { + got, err := s.GetUpload(ctx, c.user, c.id) + require.NoError(t, err) + assert.Equal(t, domain.UploadFailed, got.Status) + assert.Equal(t, domain.UploadErrInterrupted, got.ErrorKind, + "audio is not retained, so a restart cannot resume: the honest state is failed/interrupted") + } + kept, err := s.GetUpload(ctx, userA, done) + require.NoError(t, err) + assert.Equal(t, domain.UploadDone, kept.Status, "a finished transcript must survive the sweep") + assert.Equal(t, "kept", kept.Content) +} + +func TestCreateUpload_RecordsWhetherTheFileHadAVideoTrack(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + withVideo, err := s.CreateUpload(ctx, userA, "camera off", true) + require.NoError(t, err) + audioOnly, err := s.CreateUpload(ctx, userA, "voice only", false) + require.NoError(t, err) + + gotVideo, err := s.GetUpload(ctx, userA, withVideo) + require.NoError(t, err) + gotAudio, err := s.GetUpload(ctx, userA, audioOnly) + require.NoError(t, err) + listed, err := s.ListUploads(ctx, userA) + require.NoError(t, err) + + assert.True(t, gotVideo.HadVideo, "the page must be able to say the video was ignored") + assert.False(t, gotAudio.HadVideo) + byID := map[string]bool{} + for _, u := range listed { + byID[u.ID] = u.HadVideo + } + assert.Equal(t, map[string]bool{withVideo: true, audioOnly: false}, byID, "the list carries it too") +} + +func TestUploadHadVideo_SurvivesCompletionAndFailure(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + done, err := s.CreateUpload(ctx, userA, "d", true) + require.NoError(t, err) + failed, err := s.CreateUpload(ctx, userA, "f", true) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, done)) + require.NoError(t, s.CompleteUpload(ctx, userA, done, domain.Transcription{Text: "x", Endpoint: "e"})) + require.NoError(t, s.FailUpload(ctx, userA, failed, domain.UploadErrTranscriptionFailed)) + + for _, id := range []string{done, failed} { + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.True(t, got.HadVideo, "completing or failing must not clear it: the notice outlives the job") + } +} + +func TestUploadLifecycle_ReconciliationRoundTripsWithItsContestedSpots(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + want := &domain.Reconciliation{ + Status: domain.ReconcileDone, Base: "iguana/pianissimo", Alt: "iguana/kb-whisper", Reconciler: "iguana/gemma4-26b", + Disputes: 380, AutoResolved: 90, Adjudicated: 285, Changed: 120, Unsure: 5, Errors: 0, + Contested: []domain.ContestedSpot{ + {Seconds: 2210.5, Base: "Axel", Alt: "Alex", Used: "alt"}, + {Seconds: 3400, Base: "A+2", Alt: "en A", Used: "base", Unsure: true}, + }, + } + + require.NoError(t, s.CompleteUpload(ctx, userA, id, domain.Transcription{Text: "x", Endpoint: "e", Reconciliation: want})) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, want, got.Reconciliation) +} + +func TestUploadLifecycle_ASingleEngineTranscriptHasNoReconciliation(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + require.NoError(t, s.CompleteUpload(ctx, userA, id, domain.Transcription{Text: "x", Endpoint: "e"})) + + got, err := s.GetUpload(ctx, userA, id) + + require.NoError(t, err) + assert.Nil(t, got.Reconciliation, "no reconciliation is not an empty one: the page must not claim a second opinion") +} + +func TestUploadLifecycle_ASkippedReconciliationKeepsItsReason(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m", false) + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + rec := &domain.Reconciliation{Status: domain.ReconcileSkipped, Reason: "the model failed 3 times in a row: gateway down", Errors: 3} + + require.NoError(t, s.CompleteUpload(ctx, userA, id, domain.Transcription{Text: "x", Endpoint: "e", Reconciliation: rec})) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, rec, got.Reconciliation) +} diff --git a/internal/adapters/transcribe/chain.go b/internal/adapters/transcribe/chain.go new file mode 100644 index 0000000..68df323 --- /dev/null +++ b/internal/adapters/transcribe/chain.go @@ -0,0 +1,53 @@ +package transcribe + +import ( + "context" + "errors" + "fmt" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// Backend is one transcription endpoint. *Gateway satisfies it; Name is the value +// recorded as the serving endpoint. +type Backend interface { + Name() string + Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) +} + +// Chain tries a primary (local) backend and, if it fails for ANY reason, a +// fallback. The fallback exists because iguana can wedge permanently on a Metal +// OOM (infra#273) — which is also why a fallback is a data-egress decision: the +// berget endpoint is a cloud service, so the result records which one served it. +type Chain struct { + primary Backend + fallback Backend // nil disables the fallback +} + +// NewChain builds a Chain. A nil fallback means a primary failure is the result. +func NewChain(primary, fallback Backend) *Chain { + return &Chain{primary: primary, fallback: fallback} +} + +// Transcribe returns the first backend's success. It falls back on any primary +// error — transport, timeout, non-2xx, empty transcript — and never inspects a +// response body to decide. A cancelled context stops the chain: a job the user or +// shutdown cancelled must not ship its audio to a cloud endpoint as a last act. +func (c *Chain) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) { + t, err := c.primary.Transcribe(ctx, wavPath) + if err == nil { + t.Endpoint = c.primary.Name() + return t, nil + } + if c.fallback == nil || ctx.Err() != nil { + return domain.Transcription{}, fmt.Errorf("%s: %w", c.primary.Name(), err) + } + t, ferr := c.fallback.Transcribe(ctx, wavPath) + if ferr != nil { + return domain.Transcription{}, errors.Join( + fmt.Errorf("%s: %w", c.primary.Name(), err), + fmt.Errorf("%s: %w", c.fallback.Name(), ferr)) + } + t.Endpoint = c.fallback.Name() + return t, nil +} diff --git a/internal/adapters/transcribe/chain_test.go b/internal/adapters/transcribe/chain_test.go new file mode 100644 index 0000000..dc9af18 --- /dev/null +++ b/internal/adapters/transcribe/chain_test.go @@ -0,0 +1,99 @@ +package transcribe_test + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +type fakeBackend struct { + name string + text string + err error + calls int +} + +func (f *fakeBackend) Name() string { return f.name } +func (f *fakeBackend) Transcribe(_ context.Context, _ string) (domain.Transcription, error) { + f.calls++ + if f.err != nil { + return domain.Transcription{}, f.err + } + return domain.Transcription{Text: f.text, Endpoint: f.name}, nil +} + +func TestChain_PrimarySuccessNeverTouchesTheFallback(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", text: "primary text"} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "fallback text"} + + got, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "primary text", got.Text) + assert.Equal(t, "iguana/kb-whisper", got.Endpoint) + assert.Zero(t, fallback.calls, "audio must not leave the homelab when the local endpoint worked") +} + +func TestChain_AnyPrimaryFailureFallsBackAndRecordsTheFallbackEndpoint(t *testing.T) { + tests := []struct { + name string + err error + }{ + {"server error", &transcribe.StatusError{Code: 500}}, + {"gateway timeout", &transcribe.StatusError{Code: 504}}, + {"client error", &transcribe.StatusError{Code: 400}}, + {"transport failure", errors.New("dial tcp: connection refused")}, + {"empty transcript", transcribe.ErrEmptyTranscript}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: tt.err} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "fallback text"} + + got, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "fallback text", got.Text) + assert.Equal(t, "berget/kb-whisper", got.Endpoint, + "the row must say berget served it: that means the audio left the homelab") + }) + } +} + +func TestChain_BothFailingReportsBothErrors(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: errors.New("iguana down")} + fallback := &fakeBackend{name: "berget/kb-whisper", err: errors.New("berget down")} + + _, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.Error(t, err) + assert.Contains(t, err.Error(), "iguana down") + assert.Contains(t, err.Error(), "berget down") +} + +func TestChain_WithoutAFallbackAPrimaryFailureIsTheResult(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: errors.New("iguana down")} + + _, err := transcribe.NewChain(primary, nil).Transcribe(context.Background(), "/a.wav") + + require.Error(t, err) + assert.Contains(t, err.Error(), "iguana down") +} + +func TestChain_ACancelledJobDoesNotFallBack(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + primary := &fakeBackend{name: "iguana/kb-whisper", err: context.Canceled} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "x"} + + _, err := transcribe.NewChain(primary, fallback).Transcribe(ctx, "/a.wav") + + assert.ErrorIs(t, err, context.Canceled) + assert.Zero(t, fallback.calls, "a cancelled job must not ship the audio to a cloud endpoint") +} diff --git a/internal/adapters/transcribe/ensemble.go b/internal/adapters/transcribe/ensemble.go new file mode 100644 index 0000000..262e5fd --- /dev/null +++ b/internal/adapters/transcribe/ensemble.go @@ -0,0 +1,114 @@ +package transcribe + +import ( + "context" + "errors" + "fmt" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" + "git.d-ma.be/mathias/tapir/internal/progress" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// Ensemble transcribes the same WAV with two engines, one after the other, and has a +// local model reconcile their disagreements (ADR-034). It satisfies uploads.Transcriber, +// so the worker neither knows nor cares that two engines were involved. +// +// It degrades in steps and never turns a recoverable problem into a failed job: +// one engine down → the other's text, said so on the result; the model down → the +// base text unreconciled; both engines down → the cloud fallback, if configured, and +// recorded as such. A cancelled job stops everything and never reaches the cloud. +type Ensemble struct { + base, alt Backend + fallback Backend // nil = no cloud fallback + adj reconcile.Adjudicator + opts reconcile.Options +} + +// NewEnsemble builds an Ensemble. base is the engine whose text stands where the two +// disagree and is the faster one, so it runs first; alt supplies the other reading. +func NewEnsemble(base, alt, fallback Backend, adj reconcile.Adjudicator, opts reconcile.Options) *Ensemble { + return &Ensemble{base: base, alt: alt, fallback: fallback, adj: adj, opts: opts} +} + +// Plan implements uploads.Planner: the phases this ensemble reports, so the page can +// show them before they run. The fallback is a phase only when one is configured. +func (e *Ensemble) Plan() []progress.Planned { + plan := []progress.Planned{ + {Key: "engine_a", Label: "Transcribing with " + e.base.Name()}, + {Key: "engine_b", Label: "Transcribing with " + e.alt.Name()}, + {Key: "compare", Label: "Comparing the two transcripts"}, + {Key: "decide", Label: "Deciding the differences"}, + } + if e.fallback != nil { + plan = append(plan, progress.Planned{Key: "fallback", Label: "Cloud fallback (" + e.fallback.Name() + ")"}) + } + return plan +} + +// Transcribe implements uploads.Transcriber. Each phase is reported through the +// progress.Reporter in ctx (a no-op when nobody listens). +func (e *Ensemble) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) { + rep := progress.From(ctx) + b, berr := e.run(ctx, rep, "engine_a", e.base, wavPath) + if err := ctx.Err(); err != nil { + return domain.Transcription{}, err + } + a, aerr := e.run(ctx, rep, "engine_b", e.alt, wavPath) + if err := ctx.Err(); err != nil { + return domain.Transcription{}, err + } + + switch { + case berr == nil && aerr == nil: + out, err := reconcile.Reconcile(ctx, b, a, e.adj, e.opts) + if err != nil { + return domain.Transcription{}, err + } + out.Endpoint = fmt.Sprintf("%s + %s, reconciled by %s", e.base.Name(), e.alt.Name(), e.opts.Reconciler) + if rc := out.Reconciliation; rc != nil { + rep.Note("reconciled", "status", string(rc.Status), "disputes", rc.Disputes, "auto_resolved", rc.AutoResolved, + "adjudicated", rc.Adjudicated, "changed", rc.Changed, "unsure", rc.Unsure, "errors", rc.Errors) + } + return out, nil + case berr == nil: + return e.single(b, e.base.Name(), fmt.Sprintf("%s failed (%s)", e.alt.Name(), errkind.Of(aerr))), nil + case aerr == nil: + return e.single(a, e.alt.Name(), fmt.Sprintf("%s failed (%s)", e.base.Name(), errkind.Of(berr))), nil + } + + both := errors.Join(fmt.Errorf("%s: %w", e.base.Name(), berr), fmt.Errorf("%s: %w", e.alt.Name(), aerr)) + if e.fallback == nil { + return domain.Transcription{}, both + } + f, ferr := e.run(ctx, rep, "fallback", e.fallback, wavPath) + if ferr != nil { + return domain.Transcription{}, errors.Join(both, fmt.Errorf("%s: %w", e.fallback.Name(), ferr)) + } + return e.single(f, e.fallback.Name(), "both local engines failed"), nil +} + +// run is one engine call, reported as a phase. A job that was stopped is not a broken +// engine, so a cancelled context is never reported as a failed phase. +func (e *Ensemble) run(ctx context.Context, rep progress.Reporter, key string, be Backend, wav string) (domain.Transcription, error) { + rep.Begin(key, 0) + t, err := be.Transcribe(ctx, wav) + switch { + case err == nil: + rep.Done(key) + case ctx.Err() == nil: + rep.Fail(key, err) + } + return t, err +} + +// single is a transcript from one engine only: nothing to reconcile, and the result +// says why, because a reader should know the second opinion was missing. +func (e *Ensemble) single(t domain.Transcription, engine, reason string) domain.Transcription { + t.Endpoint = engine + t.Reconciliation = &domain.Reconciliation{ + Status: domain.ReconcileSingle, Reason: reason, Base: e.opts.BaseEngine, Alt: e.opts.AltEngine, Reconciler: e.opts.Reconciler, + } + return t +} diff --git a/internal/adapters/transcribe/ensemble_progress_test.go b/internal/adapters/transcribe/ensemble_progress_test.go new file mode 100644 index 0000000..4c779cf --- /dev/null +++ b/internal/adapters/transcribe/ensemble_progress_test.go @@ -0,0 +1,162 @@ +package transcribe_test + +import ( + "context" + "errors" + "strings" + "sync" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/progress" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// What the Ensemble reports to the page and the log (ADR-035). The reporter here is a +// recording double carried in the context, exactly as the worker's Tracker is. + +type recReporter struct { + mu sync.Mutex + events []string + notes []map[string]any +} + +func (r *recReporter) add(e string) { + r.mu.Lock() + defer r.mu.Unlock() + r.events = append(r.events, e) +} + +func (r *recReporter) Begin(key string, _ int) { r.add("begin:" + key) } +func (r *recReporter) Advance(key string, _ int) { r.add("advance:" + key) } +func (r *recReporter) Done(key string) { r.add("done:" + key) } +func (r *recReporter) Fail(key string, _ error) { r.add("fail:" + key) } +func (r *recReporter) Note(msg string, kv ...any) { + m := map[string]any{"msg": msg} + for i := 0; i+1 < len(kv); i += 2 { + m[kv[i].(string)] = kv[i+1] + } + r.mu.Lock() + r.notes = append(r.notes, m) + r.mu.Unlock() + r.add("note:" + msg) +} + +func (r *recReporter) joined() string { + r.mu.Lock() + defer r.mu.Unlock() + return strings.Join(r.events, ",") +} + +func TestEnsemblePlan_DeclaresBothEnginesAndTheReconcileStepsByName(t *testing.T) { + r := newEnsemble("a", "a", nil, nil, false, &verdictAdj{}) + + plan := r.e.Plan() + + require.Len(t, plan, 4) + assert.Equal(t, []string{"engine_a", "engine_b", "compare", "decide"}, []string{plan[0].Key, plan[1].Key, plan[2].Key, plan[3].Key}) + assert.Contains(t, plan[0].Label, "iguana/pianissimo", "the page says which engine is running") + assert.Contains(t, plan[1].Label, "iguana/kb-whisper") +} + +func TestEnsemblePlan_TheCloudFallbackIsAStepOnlyWhenOneIsConfigured(t *testing.T) { + with := newEnsemble("a", "a", nil, nil, true, &verdictAdj{}).e.Plan() + without := newEnsemble("a", "a", nil, nil, false, &verdictAdj{}).e.Plan() + + assert.Equal(t, "fallback", with[len(with)-1].Key) + assert.Len(t, without, 4, "no fallback configured: the page must not promise one") +} + +func TestEnsembleProgress_BothEnginesAreReportedInTheOrderTheyRun(t *testing.T) { + rep := &recReporter{} + r := newEnsemble("Totalt 1015 kronor idag.", "Totalt 1050 kronor idag.", nil, nil, false, &verdictAdj{v: reconcile.VerdictAlt}) + + _, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav") + + require.NoError(t, err) + got := rep.joined() + a, aDone := strings.Index(got, "begin:engine_a"), strings.Index(got, "done:engine_a") + b, bDone := strings.Index(got, "begin:engine_b"), strings.Index(got, "done:engine_b") + require.True(t, a >= 0 && aDone > a && b > aDone && bDone > b, "events out of order: %s", got) + assert.NotContains(t, got, "fail:") +} + +func TestEnsembleProgress_AnEngineThatFailsIsShownFailedAndTheOtherStillRuns(t *testing.T) { + rep := &recReporter{} + r := newEnsemble("text", "text", errors.New("pianissimo down"), nil, false, &verdictAdj{}) + + got, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav") + + require.NoError(t, err, "one engine down is a degraded result, not a failed job") + assert.Equal(t, domain.ReconcileSingle, got.Reconciliation.Status) + ev := rep.joined() + assert.Contains(t, ev, "fail:engine_a") + assert.Contains(t, ev, "done:engine_b") + assert.NotContains(t, ev, "begin:compare", "nothing to compare with one engine") +} + +func TestEnsembleProgress_TheFallbackIsReportedOnlyWhenBothEnginesFailed(t *testing.T) { + rep := &recReporter{} + r := newEnsemble("t", "t", errors.New("a down"), errors.New("b down"), true, &verdictAdj{}) + + _, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav") + + require.NoError(t, err) + ev := rep.joined() + assert.Contains(t, ev, "fail:engine_a") + assert.Contains(t, ev, "fail:engine_b") + assert.Contains(t, ev, "begin:fallback") + assert.Contains(t, ev, "done:fallback") + + rep2 := &recReporter{} + ok := newEnsemble("x", "x", nil, nil, true, &verdictAdj{}) + _, err = ok.e.Transcribe(progress.With(context.Background(), rep2), "/a.wav") + require.NoError(t, err) + assert.NotContains(t, rep2.joined(), "fallback", "a healthy run never touches the cloud step") +} + +// cancellingBackend ends the job in the middle of its own call, like a shutdown. +type cancellingBackend struct { + name string + cancel context.CancelFunc +} + +func (c cancellingBackend) Name() string { return c.name } +func (c cancellingBackend) Transcribe(ctx context.Context, _ string) (domain.Transcription, error) { + c.cancel() + return domain.Transcription{}, ctx.Err() +} + +func TestEnsembleProgress_AShutdownMidEngineIsNotReportedAsAFailedEngine(t *testing.T) { + rep := &recReporter{} + ctx, cancel := context.WithCancel(progress.With(context.Background(), rep)) + calls := &[]string{} + alt := &orderedBackend{name: "iguana/kb-whisper", text: "x", calls: calls} + e := transcribe.NewEnsemble(cancellingBackend{name: "iguana/pianissimo", cancel: cancel}, alt, nil, &verdictAdj{}, ensembleOpts) + + _, err := e.Transcribe(ctx, "/a.wav") + + require.ErrorIs(t, err, context.Canceled) + assert.NotContains(t, rep.joined(), "fail:", "a stopped job is not a broken engine: the page must not blame one") + assert.Empty(t, *calls, "and the second engine never starts") +} + +func TestEnsembleProgress_TheReconcileCountsAreNotedForTheOperator(t *testing.T) { + rep := &recReporter{} + r := newEnsemble("Totalt 1015 kronor idag.", "Totalt 1050 kronor idag.", nil, nil, false, &verdictAdj{v: reconcile.VerdictAlt}) + + _, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav") + + require.NoError(t, err) + require.Len(t, rep.notes, 1) + n := rep.notes[0] + assert.Equal(t, "reconciled", n["msg"]) + assert.Equal(t, 1, n["disputes"]) + assert.Equal(t, 1, n["changed"]) + assert.Equal(t, 0, n["unsure"]) + assert.Equal(t, 0, n["errors"]) +} diff --git a/internal/adapters/transcribe/ensemble_reason_test.go b/internal/adapters/transcribe/ensemble_reason_test.go new file mode 100644 index 0000000..4777bd7 --- /dev/null +++ b/internal/adapters/transcribe/ensemble_reason_test.go @@ -0,0 +1,36 @@ +package transcribe_test + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// When one engine is down the page says so and why; the "why" is a kind, not the +// engine's error text, which can carry provider output (adversarial review F3). + +func TestEnsemble_AFailedEnginesReasonIsAKindNotItsErrorText(t *testing.T) { + r := newEnsemble("text", "text", nil, errors.New("kb-whisper said: zebrakaka secret words"), false, &verdictAdj{v: reconcile.VerdictAlt}) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.NotContains(t, got.Reconciliation.Reason, "zebrakaka") + assert.Contains(t, got.Reconciliation.Reason, "iguana/kb-whisper failed", "which engine failed is still said") +} + +func TestEnsemble_AnHTTPFailureShowsItsStatusCodeAsTheKind(t *testing.T) { + r := newEnsemble("text", "text", &transcribe.StatusError{Code: 502, Snippet: "zebrakaka"}, nil, false, &verdictAdj{}) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Contains(t, got.Reconciliation.Reason, "http_502") + assert.NotContains(t, got.Reconciliation.Reason, "zebrakaka") +} diff --git a/internal/adapters/transcribe/ensemble_test.go b/internal/adapters/transcribe/ensemble_test.go new file mode 100644 index 0000000..f9feca4 --- /dev/null +++ b/internal/adapters/transcribe/ensemble_test.go @@ -0,0 +1,206 @@ +package transcribe_test + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// orderedBackend records, in a shared slice, when it was called. +type orderedBackend struct { + name string + text string + err error + calls *[]string +} + +func (o *orderedBackend) Name() string { return o.name } +func (o *orderedBackend) Transcribe(_ context.Context, _ string) (domain.Transcription, error) { + *o.calls = append(*o.calls, o.name) + if o.err != nil { + return domain.Transcription{}, o.err + } + return domain.Transcription{Text: o.text, Endpoint: o.name, AudioSeconds: 10}, nil +} + +type verdictAdj struct { + v reconcile.Verdict + err error + calls int +} + +func (a *verdictAdj) Adjudicate(context.Context, reconcile.Request) (reconcile.Verdict, error) { + a.calls++ + return a.v, a.err +} + +var ensembleOpts = reconcile.Options{Reconciler: "iguana/gemma4-26b", BaseEngine: "iguana/pianissimo", AltEngine: "iguana/kb-whisper"} + +type rig struct { + e *transcribe.Ensemble + calls *[]string + adj *verdictAdj +} + +func newEnsemble(baseText, altText string, baseErr, altErr error, withFallback bool, adj *verdictAdj) *rig { + calls := &[]string{} + base := &orderedBackend{name: "iguana/pianissimo", text: baseText, err: baseErr, calls: calls} + alt := &orderedBackend{name: "iguana/kb-whisper", text: altText, err: altErr, calls: calls} + var fb transcribe.Backend + if withFallback { + fb = &orderedBackend{name: "berget/kb-whisper", text: "fallback text", calls: calls} + } + return &rig{e: transcribe.NewEnsemble(base, alt, fb, adj, ensembleOpts), calls: calls, adj: adj} +} + +func TestEnsemble_BothEnginesRunInOrderAndTheirDisagreementIsReconciled(t *testing.T) { + adj := &verdictAdj{v: reconcile.VerdictAlt} + r := newEnsemble("Totalt 1015 kronor idag.", "Totalt 1050 kronor idag.", nil, nil, true, adj) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, []string{"iguana/pianissimo", "iguana/kb-whisper"}, *r.calls, + "the 40-second engine runs first, so a failure there is known early; the fallback is not touched") + assert.Equal(t, "Totalt 1050 kronor idag.", got.Text) + assert.Equal(t, "iguana/pianissimo + iguana/kb-whisper, reconciled by iguana/gemma4-26b", got.Endpoint, + "every contributor is recorded: which engines touched the audio is a quality and a privacy fact") + require.NotNil(t, got.Reconciliation) + assert.Equal(t, domain.ReconcileDone, got.Reconciliation.Status) +} + +func TestEnsemble_IdenticalTextsNeverCallTheModel(t *testing.T) { + adj := &verdictAdj{v: reconcile.VerdictAlt} + r := newEnsemble("Vi går igenom läget.", "Vi går igenom läget.", nil, nil, false, adj) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Zero(t, adj.calls) + assert.Equal(t, "Vi går igenom läget.", got.Text) +} + +func TestEnsemble_WhenOneEngineFailsTheOtherIsUsedAndItIsSaidSo(t *testing.T) { + tests := []struct { + name string + baseErr error + altErr error + wantText string + wantEndpoint string + wantReasonHas string + }{ + {"pianissimo down", errors.New("pianissimo wedged"), nil, "alt text", "iguana/kb-whisper", "iguana/pianissimo failed"}, + {"kb-whisper down", nil, errors.New("metal oom"), "base text", "iguana/pianissimo", "iguana/kb-whisper failed"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + adj := &verdictAdj{v: reconcile.VerdictAlt} + r := newEnsemble("base text", "alt text", tt.baseErr, tt.altErr, true, adj) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, tt.wantText, got.Text) + assert.Equal(t, tt.wantEndpoint, got.Endpoint) + assert.Zero(t, adj.calls, "there is nothing to reconcile") + require.NotNil(t, got.Reconciliation) + assert.Equal(t, domain.ReconcileSingle, got.Reconciliation.Status) + assert.Contains(t, got.Reconciliation.Reason, tt.wantReasonHas) + assert.NotContains(t, *r.calls, "berget/kb-whisper", "one local engine working means audio does not leave the homelab") + }) + } +} + +func TestEnsemble_OnlyWhenBothLocalEnginesFailDoesTheCloudFallbackRun(t *testing.T) { + r := newEnsemble("", "", errors.New("a"), errors.New("b"), true, &verdictAdj{}) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "fallback text", got.Text) + assert.Equal(t, "berget/kb-whisper", got.Endpoint, "the row must say berget served it: the audio left the homelab") + assert.Equal(t, domain.ReconcileSingle, got.Reconciliation.Status) + assert.Contains(t, got.Reconciliation.Reason, "both local engines failed") +} + +func TestEnsemble_BothFailingWithNoFallbackReportsBothErrors(t *testing.T) { + r := newEnsemble("", "", errors.New("pianissimo down"), errors.New("whisper down"), false, &verdictAdj{}) + + _, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.Error(t, err) + assert.Contains(t, err.Error(), "pianissimo down") + assert.Contains(t, err.Error(), "whisper down") +} + +func TestEnsemble_ACancelledJobNeverReachesTheCloud(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + r := newEnsemble("", "", context.Canceled, context.Canceled, true, &verdictAdj{}) + + _, err := r.e.Transcribe(ctx, "/a.wav") + + assert.ErrorIs(t, err, context.Canceled) + assert.NotContains(t, *r.calls, "berget/kb-whisper", "a cancelled job must not ship its audio to a cloud endpoint as a last act") +} + +func TestEnsemble_AModelThatIsDownGivesTheBaseTextUnreconciledNotAFailedJob(t *testing.T) { + adj := &verdictAdj{err: errors.New("gateway down")} + // Four separate disagreements, each separated by two shared words, so the + // breaker trips after three consecutive failures. + r := newEnsemble("a1 s t b1 s t c1 s t d1", "a2 s t b2 s t c2 s t d2", nil, nil, false, adj) + + got, err := r.e.Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "a1 s t b1 s t c1 s t d1", got.Text) + assert.Equal(t, domain.ReconcileSkipped, got.Reconciliation.Status) + assert.Equal(t, 3, adj.calls) +} + +// cancelling is a backend that cancels the job's context while it runs, the way a +// shutdown arrives in the middle of a long transcription. +type cancelling struct { + name string + cancel context.CancelFunc + calls *[]string +} + +func (c *cancelling) Name() string { return c.name } +func (c *cancelling) Transcribe(_ context.Context, _ string) (domain.Transcription, error) { + *c.calls = append(*c.calls, c.name) + c.cancel() + return domain.Transcription{}, context.Canceled +} + +func TestEnsemble_AShutdownDuringTheFirstEngineStopsBeforeTheSecondOrTheCloud(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + calls := &[]string{} + base := &cancelling{name: "iguana/pianissimo", cancel: cancel, calls: calls} + alt := &orderedBackend{name: "iguana/kb-whisper", text: "alt", calls: calls} + fb := &orderedBackend{name: "berget/kb-whisper", text: "fb", calls: calls} + + _, err := transcribe.NewEnsemble(base, alt, fb, &verdictAdj{}, ensembleOpts).Transcribe(ctx, "/a.wav") + + assert.ErrorIs(t, err, context.Canceled) + assert.Equal(t, []string{"iguana/pianissimo"}, *calls, "no second engine and no cloud once the job is cancelled") +} + +func TestEnsemble_AShutdownDuringTheSecondEngineIsAnErrorNotASingleEngineSuccess(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + calls := &[]string{} + base := &orderedBackend{name: "iguana/pianissimo", text: "base", calls: calls} + alt := &cancelling{name: "iguana/kb-whisper", cancel: cancel, calls: calls} + + got, err := transcribe.NewEnsemble(base, alt, nil, &verdictAdj{}, ensembleOpts).Transcribe(ctx, "/a.wav") + + assert.ErrorIs(t, err, context.Canceled, "a cancelled job must not be recorded as 'kb-whisper failed'") + assert.Empty(t, got.Text) +} diff --git a/internal/adapters/transcribe/gateway.go b/internal/adapters/transcribe/gateway.go new file mode 100644 index 0000000..106798a --- /dev/null +++ b/internal/adapters/transcribe/gateway.go @@ -0,0 +1,188 @@ +// Package transcribe turns a WAV file into a timed Swedish transcript by calling +// the LiteLLM gateway's OpenAI-compatible /audio/transcriptions endpoint +// (ADR-033). A Gateway is one model alias; a Chain tries a local one and falls +// back to a second. +package transcribe + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "mime/multipart" + "net/http" + "os" + "strings" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrEmptyTranscript is returned when the endpoint answered 200 but with no text. +// An empty transcript is not a success: chat over it could only answer "nothing". +var ErrEmptyTranscript = errors.New("transcribe: endpoint returned an empty transcript") + +// StatusError is a non-2xx answer from the gateway. Nothing may branch on the +// body — {"error":"failed to process audio"} has had at least five distinct +// causes — so only the status code is exposed for logic; Snippet is for logs. +type StatusError struct { + Code int + Snippet string +} + +// HTTPStatus implements errkind.HTTPStatuser: the code, never the snippet. +func (e *StatusError) HTTPStatus() int { return e.Code } + +func (e *StatusError) Error() string { + return fmt.Sprintf("transcribe: gateway returned HTTP %d: %s", e.Code, e.Snippet) +} + +// Gateway transcribes through one model alias on the gateway. baseURL is the +// gateway's /v1 root — the in-cluster NodePort, not the public hostname, whose +// ingress rejects bodies over ~8-10 MB with 413 (docs/research/audio-length-ceiling). +type Gateway struct { + baseURL string + key string + model string + client *http.Client +} + +// NewGateway builds a Gateway. The client should carry no overall Timeout: a 90 +// minute recording takes minutes, so the deadline belongs to the caller's context. +func NewGateway(baseURL, key, model string, client *http.Client) *Gateway { + return &Gateway{baseURL: strings.TrimRight(baseURL, "/"), key: key, model: model, client: client} +} + +// Name is the model alias; it is what gets recorded as the serving endpoint. +func (g *Gateway) Name() string { return g.model } + +// Transcribe uploads the WAV at path and returns the normalised transcription. +// The body is streamed from disk with a COMPUTED Content-Length: a 90 minute WAV +// is ~170 MB, so it is neither buffered in memory nor sent chunked. +func (g *Gateway) Transcribe(ctx context.Context, path string) (domain.Transcription, error) { + f, err := os.Open(path) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: open audio: %w", err) + } + defer func() { _ = f.Close() }() + info, err := f.Stat() + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: stat audio: %w", err) + } + + var prefix bytes.Buffer + mw := multipart.NewWriter(&prefix) + for _, kv := range [][2]string{{"model", g.model}, {"response_format", "verbose_json"}, {"language", "sv"}} { + if err := mw.WriteField(kv[0], kv[1]); err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + } + // The multipart field is `file`; `audio_file` is rejected as "Invalid request". + if _, err := mw.CreateFormFile("file", "audio.wav"); err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + suffix := []byte("\r\n--" + mw.Boundary() + "--\r\n") + + body := io.MultiReader(bytes.NewReader(prefix.Bytes()), f, bytes.NewReader(suffix)) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, g.baseURL+"/audio/transcriptions", body) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + req.ContentLength = int64(prefix.Len()) + info.Size() + int64(len(suffix)) + req.Header.Set("Content-Type", mw.FormDataContentType()) + req.Header.Set("Authorization", "Bearer "+g.key) + // No "Expect: 100-continue": it trips the proxy for any body over 1 KB. + + resp, err := g.client.Do(req) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err) + } + defer func() { _ = resp.Body.Close() }() + if resp.StatusCode < 200 || resp.StatusCode > 299 { + snip, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return domain.Transcription{}, &StatusError{Code: resp.StatusCode, Snippet: strings.TrimSpace(string(snip))} + } + + t, err := parseVerbose(resp.Body) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err) + } + t.Endpoint = g.model + return t, nil +} + +// verboseJSON is the subset of the verbose_json response used. Both backends +// return it; they differ in segment granularity (iguana: whole seconds and +// sub-sentence pieces, berget: sub-second sentences), in the language token, and in +// whether a segment's text starts with a space. +type verboseJSON struct { + Text string `json:"text"` + Duration float64 `json:"duration"` + Segments []struct { + Start float64 `json:"start"` + End float64 `json:"end"` + Text string `json:"text"` + } `json:"segments"` +} + +// parseVerbose normalises a verbose_json body. Whisper's final window can end past +// the audio (9.0 s for 4.25 s of audio was observed), so segment ends are clamped +// to the reported duration. Text is rebuilt from the segments, not taken from the +// response's `text` field: see joinSegments. +func parseVerbose(r io.Reader) (domain.Transcription, error) { + var v verboseJSON + if err := json.NewDecoder(r).Decode(&v); err != nil { + return domain.Transcription{}, fmt.Errorf("decode response: %w", err) + } + raw := make([]string, 0, len(v.Segments)) + for _, s := range v.Segments { + raw = append(raw, s.Text) + } + text := joinSegments(raw) + if text == "" { + text = collapseSpace(v.Text) + } + if text == "" { + return domain.Transcription{}, ErrEmptyTranscript + } + t := domain.Transcription{Text: text, AudioSeconds: v.Duration} + for _, s := range v.Segments { + seg := domain.Segment{Start: s.Start, End: s.End, Text: strings.TrimSpace(s.Text)} + if v.Duration > 0 { + seg.End = min(seg.End, v.Duration) + seg.Start = min(seg.Start, seg.End) + } + t.Segments = append(t.Segments, seg) + } + return t, nil +} + +// joinSegments rebuilds the running text from raw (untrimmed) segment texts. +// +// whisper.cpp wraps its output at ~60 characters and a wrap can land INSIDE a word +// ("nyt" | "t godkännande"): a segment that begins with a space starts a new word, +// one that does not continues the previous one, and the response's own `text` +// field joins them with newlines, which breaks the word. So when any segment +// carries a leading space the raw texts are concatenated as they are. Backends +// that return whole sentences with no leading space (berget) are joined with a +// space instead — concatenating those would glue sentences together. +func joinSegments(raw []string) string { + leadingSpace := false + for _, s := range raw { + if strings.HasPrefix(s, " ") { + leadingSpace = true + break + } + } + sep := " " + if leadingSpace { + sep = "" + } + return collapseSpace(strings.Join(raw, sep)) +} + +// collapseSpace turns every run of whitespace (including newlines) into one space. +func collapseSpace(s string) string { + return strings.Join(strings.Fields(s), " ") +} diff --git a/internal/adapters/transcribe/gateway_test.go b/internal/adapters/transcribe/gateway_test.go new file mode 100644 index 0000000..97c655e --- /dev/null +++ b/internal/adapters/transcribe/gateway_test.go @@ -0,0 +1,257 @@ +package transcribe_test + +import ( + "bytes" + "context" + "fmt" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" +) + +// The httptest server stands in for the LiteLLM gateway: it is the process +// boundary, so a fake here is legitimate. The JSON it returns mirrors what the +// real gateway returned for iguana/kb-whisper and berget/kb-whisper on 2026-10-01 +// (docs/research/audio-length-ceiling-2026-10-01.md). All text is synthetic. + +const verboseJSON = `{ + "text": " God morgon allihopa.\n Vi går igenom läget.\n", + "language": "swedish", + "duration": 4.25, + "segments": [ + {"id": 0, "start": 0.0, "end": 1.0, "text": " God morgon allihopa."}, + {"id": 1, "start": 1.0, "end": 9.0, "text": " Vi går igenom läget."} + ] +}` + +func wavFile(t *testing.T, content string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "a.wav") + require.NoError(t, os.WriteFile(p, []byte(content), 0o600)) + return p +} + +type seen struct { + method, path, auth, expect, contentType string + contentLength int64 + transferEncoding []string + fields map[string]string + file string + fileField string + bodyLen int +} + +// serve starts a gateway fake that records the request it received and replies +// with status and body. +func serve(t *testing.T, status int, reply string, got *seen) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if got != nil { + got.method, got.path = r.Method, r.URL.Path + got.auth = r.Header.Get("Authorization") + got.expect = r.Header.Get("Expect") + got.contentType = r.Header.Get("Content-Type") + got.contentLength = r.ContentLength + got.transferEncoding = r.TransferEncoding + raw, _ := io.ReadAll(r.Body) + got.bodyLen = len(raw) + r.Body = io.NopCloser(bytes.NewReader(raw)) + require.NoError(t, r.ParseMultipartForm(32<<20)) + got.fields = map[string]string{} + for k, v := range r.MultipartForm.Value { + got.fields[k] = v[0] + } + for field, fhs := range r.MultipartForm.File { + got.fileField = field + f, _ := fhs[0].Open() + b, _ := io.ReadAll(f) + got.file = string(b) + } + } + w.WriteHeader(status) + _, _ = io.WriteString(w, reply) + })) + t.Cleanup(srv.Close) + return srv +} + +func TestGateway_SendsTheRequestTheGatewayExpects(t *testing.T) { + var got seen + srv := serve(t, 200, verboseJSON, &got) + g := transcribe.NewGateway(srv.URL+"/v1", "secret-key", "iguana/kb-whisper", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "RIFF-synthetic-pcm")) + + require.NoError(t, err) + assert.Equal(t, http.MethodPost, got.method) + assert.Equal(t, "/v1/audio/transcriptions", got.path) + assert.Equal(t, "Bearer secret-key", got.auth) + assert.Equal(t, "file", got.fileField, "the multipart field is `file`; `audio_file` is rejected as 'Invalid request'") + assert.Equal(t, "RIFF-synthetic-pcm", got.file) + assert.Equal(t, "iguana/kb-whisper", got.fields["model"]) + assert.Equal(t, "verbose_json", got.fields["response_format"]) + assert.Equal(t, "sv", got.fields["language"]) + assert.Empty(t, got.expect, "no Expect: 100-continue: it trips the proxy for any body over 1 KB") +} + +func TestGateway_SendsAKnownContentLengthNotChunked(t *testing.T) { + var got seen + srv := serve(t, 200, verboseJSON, &got) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "0123456789")) + + require.NoError(t, err) + assert.Empty(t, got.transferEncoding, "a 90-minute WAV is ~170 MB: stream it with a computed length, not chunked and not buffered") + assert.Equal(t, int64(got.bodyLen), got.contentLength) +} + +func TestGateway_NormalisesTheVerboseJSONResponse(t *testing.T) { + srv := serve(t, 200, verboseJSON, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "iguana/kb-whisper", srv.Client()) + + got, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + require.NoError(t, err) + assert.Equal(t, "God morgon allihopa. Vi går igenom läget.", got.Text) + assert.Equal(t, "iguana/kb-whisper", got.Endpoint, "the serving endpoint is part of the result") + assert.InDelta(t, 4.25, got.AudioSeconds, 1e-9) + assert.Equal(t, []domain.Segment{ + {Start: 0, End: 1, Text: "God morgon allihopa."}, + // The final segment ended at 9.0 s for 4.25 s of audio: whisper's last + // window can run past the end. Clamped to the audio duration. + {Start: 1, End: 4.25, Text: "Vi går igenom läget."}, + }, got.Segments) +} + +func TestGateway_ErrorsDoNotDependOnTheBody(t *testing.T) { + // {"error":"failed to process audio"} has had at least five distinct causes, so + // no code may branch on it. Two different bodies must be treated identically. + for _, reply := range []string{`{"error":"failed to process audio"}`, `{"error":"something else entirely"}`} { + srv := serve(t, 500, reply, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + var se *transcribe.StatusError + require.ErrorAs(t, err, &se) + assert.Equal(t, 500, se.Code) + } +} + +func TestGateway_AnEmptyTranscriptIsAFailureNotASuccess(t *testing.T) { + srv := serve(t, 200, `{"text":" ","language":"swedish","duration":3,"segments":[]}`, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + assert.ErrorIs(t, err, transcribe.ErrEmptyTranscript) +} + +func TestGateway_MalformedJSONIsAnError(t *testing.T) { + srv := serve(t, 200, `not json`, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + assert.Error(t, err) +} + +func TestGateway_MissingAudioFileIsAnError(t *testing.T) { + g := transcribe.NewGateway("http://127.0.0.1:1/v1", "k", "m", http.DefaultClient) + + _, err := g.Transcribe(context.Background(), filepath.Join(t.TempDir(), "nope.wav")) + + assert.Error(t, err) +} + +// whisper.cpp wraps its output at ~60 characters, and a wrap can land INSIDE a word: +// the continuation segment has no leading space, the `text` field joins segments +// with newlines, and a segment can be a lone ".". Measured on real gateway output +// 2026-10-01: 7% of a one-hour recording's segments were such fragments. Joining by +// newline hands the chat model broken words ("nyt\nt"), so Text is rebuilt from the +// segments: a segment that begins with a space starts a new word, one that does not +// continues the previous one. +func TestGateway_RejoinsWordsWhisperCppSplitAcrossSegments(t *testing.T) { + reply := `{ + "text": " Vi ligger just nu 12 procent över den ursprungliga kalkylen\n.\n Det kräver ett nyt\nt godkännande från beställaren.\n Framförallt på arbetspl\natsen.\n", + "language": "swedish", "duration": 20, + "segments": [ + {"start": 0, "end": 4, "text": " Vi ligger just nu 12 procent över den ursprungliga kalkylen"}, + {"start": 4, "end": 5, "text": "."}, + {"start": 5, "end": 8, "text": " Det kräver ett nyt"}, + {"start": 8, "end": 11, "text": "t godkännande från beställaren."}, + {"start": 11, "end": 14, "text": " Framförallt på arbetspl"}, + {"start": 14, "end": 16, "text": "atsen."} + ]}` + srv := serve(t, 200, reply, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "iguana/kb-whisper", srv.Client()) + + got, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + require.NoError(t, err) + assert.Equal(t, "Vi ligger just nu 12 procent över den ursprungliga kalkylen. "+ + "Det kräver ett nytt godkännande från beställaren. Framförallt på arbetsplatsen.", got.Text) + assert.Len(t, got.Segments, 6, "the timed segments are kept as the backend gave them (trimmed)") + assert.Equal(t, ".", got.Segments[1].Text) +} + +func TestGateway_SentenceLevelSegmentsWithNoLeadingSpaceAreJoinedWithASpace(t *testing.T) { + // berget/kb-whisper returns whole sentences with no leading space; rebuilding by + // concatenation would glue them ("allihopa!Vi"). + reply := `{"text": "God morgon allihopa!\nVi går igenom läget.", "language": "sv", "duration": 5, + "segments": [ + {"start": 0.05, "end": 1.2, "text": "God morgon allihopa!"}, + {"start": 1.6, "end": 4.2, "text": "Vi går igenom läget."} + ]}` + srv := serve(t, 200, reply, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "berget/kb-whisper", srv.Client()) + + got, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + require.NoError(t, err) + assert.Equal(t, "God morgon allihopa! Vi går igenom läget.", got.Text) +} + +func TestGateway_WithNoSegmentsTheTextFieldIsUsedWithWhitespaceCollapsed(t *testing.T) { + srv := serve(t, 200, `{"text": " Hej allihopa.\n Vi börjar. ", "duration": 3, "segments": []}`, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + got, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + require.NoError(t, err) + assert.Equal(t, "Hej allihopa. Vi börjar.", got.Text) +} + +func TestStatusError_ExposesItsCodeAsAKindAndNotItsSnippet(t *testing.T) { + err := fmt.Errorf("transcribe: %w", &transcribe.StatusError{Code: 502, Snippet: "echoed: Anders ansvarar"}) + + assert.Equal(t, "http_502", errkind.Of(err)) +} + +func TestGateway_AnErrorBodyIsBoundedToTwoHundredBytesInTheStatusError(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte(strings.Repeat("x", 100_000))) + })) + defer srv.Close() + wav := filepath.Join(t.TempDir(), "a.wav") + require.NoError(t, os.WriteFile(wav, []byte("RIFF....WAVEfmt "), 0o600)) + + _, err := transcribe.NewGateway(srv.URL, "", "iguana/kb-whisper", srv.Client()).Transcribe(context.Background(), wav) + + var se *transcribe.StatusError + require.ErrorAs(t, err, &se) + assert.LessOrEqual(t, len(se.Snippet), 200, "a gateway's whole body must not become part of an error") +} diff --git a/internal/adapters/transcribe/live_test.go b/internal/adapters/transcribe/live_test.go new file mode 100644 index 0000000..fc48bab --- /dev/null +++ b/internal/adapters/transcribe/live_test.go @@ -0,0 +1,49 @@ +package transcribe_test + +import ( + "context" + "net/http" + "os" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" +) + +// TestGateway_LiveGateway is an opt-in check against the REAL gateway, because +// the httptest fake above can only prove what we think the gateway accepts. It +// is skipped unless TAPIR_LIVE_WAV names a 16 kHz mono WAV and the gateway +// coordinates are set: +// +// TAPIR_LIVE_WAV=/path/a.wav TAPIR_LIVE_GATEWAY_URL=http://koala:30401/v1 \ +// TAPIR_LIVE_MODEL=iguana/kb-whisper op run --env-file ~/.op-env -- \ +// go test ./internal/adapters/transcribe -run LiveGateway -v +// +// Use SYNTHETIC audio only: this repo is public and the gateway may be a cloud +// endpoint (#33). +func TestGateway_LiveGateway(t *testing.T) { + wav := os.Getenv("TAPIR_LIVE_WAV") + url := os.Getenv("TAPIR_LIVE_GATEWAY_URL") + if wav == "" || url == "" { + t.Skip("set TAPIR_LIVE_WAV and TAPIR_LIVE_GATEWAY_URL to run against the real gateway") + } + model := os.Getenv("TAPIR_LIVE_MODEL") + if model == "" { + model = "iguana/kb-whisper" + } + key := os.Getenv("LITELLM_MASTER_KEY") + require.NotEmpty(t, key, "LITELLM_MASTER_KEY must be set (via op run)") + + ctx, cancel := context.WithTimeout(context.Background(), 25*time.Minute) + defer cancel() + got, err := transcribe.NewGateway(url, key, model, &http.Client{}).Transcribe(ctx, wav) + + require.NoError(t, err) + assert.NotEmpty(t, got.Text) + assert.NotEmpty(t, got.Segments) + assert.Equal(t, model, got.Endpoint) + t.Logf("endpoint=%s audio_s=%.1f segments=%d chars=%d", got.Endpoint, got.AudioSeconds, len(got.Segments), len(got.Text)) +} diff --git a/internal/adapters/transcribe/timeout.go b/internal/adapters/transcribe/timeout.go new file mode 100644 index 0000000..fee7b1b --- /dev/null +++ b/internal/adapters/transcribe/timeout.go @@ -0,0 +1,33 @@ +package transcribe + +import ( + "context" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// WithTimeout gives every call to b its own deadline. A wedged engine does not answer at +// all (iguana after a Metal OOM, infra#273), and with only the job's deadline it used up +// the whole job before a fallback could run -- and Chain then refuses the fallback because +// the job context has expired. With its own deadline the call fails while the job is still +// alive, which is exactly when the fallback, or the other engine, can take over. +// +// A zero or negative d returns b unchanged. The wrapper is invisible: Name is b's. +func WithTimeout(b Backend, d time.Duration) Backend { + if d <= 0 { + return b + } + return timeoutBackend{Backend: b, d: d} +} + +type timeoutBackend struct { + Backend + d time.Duration +} + +func (t timeoutBackend) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) { + ctx, cancel := context.WithTimeout(ctx, t.d) + defer cancel() + return t.Backend.Transcribe(ctx, wavPath) +} diff --git a/internal/adapters/transcribe/timeout_test.go b/internal/adapters/transcribe/timeout_test.go new file mode 100644 index 0000000..fed4489 --- /dev/null +++ b/internal/adapters/transcribe/timeout_test.go @@ -0,0 +1,104 @@ +package transcribe_test + +import ( + "context" + "errors" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// A wedged engine (iguana after a Metal OOM, infra#273) does not answer at all. With only +// the job's deadline, the primary used up the whole job before any fallback could run, and +// the chain then refused the fallback because the job context had expired. Each engine call +// gets its own deadline (adversarial review F4). + +type hangingBackend struct{ name string } + +func (h hangingBackend) Name() string { return h.name } +func (hangingBackend) Transcribe(ctx context.Context, _ string) (domain.Transcription, error) { + <-ctx.Done() + return domain.Transcription{}, ctx.Err() +} + +func TestWithTimeout_AHungBackendFailsAtItsOwnDeadlineWhileTheJobIsStillAlive(t *testing.T) { + job, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + b := transcribe.WithTimeout(hangingBackend{name: "iguana/kb-whisper"}, 40*time.Millisecond) + + start := time.Now() + _, err := b.Transcribe(job, "/a.wav") + + require.Error(t, err) + assert.Equal(t, "timeout", errkind.Of(err)) + assert.Less(t, time.Since(start), 2*time.Second, "it must give up at its own deadline, not wait for the job's") + assert.NoError(t, job.Err(), "the job context is untouched, which is what lets a fallback run") + assert.Equal(t, "iguana/kb-whisper", b.Name(), "the wrapper is invisible in the recorded endpoint") +} + +func TestWithTimeout_ASuccessPassesStraightThrough(t *testing.T) { + calls := &[]string{} + inner := &orderedBackend{name: "iguana/kb-whisper", text: "hej", calls: calls} + + got, err := transcribe.WithTimeout(inner, time.Minute).Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "hej", got.Text) +} + +func TestWithTimeout_ZeroMeansNoDeadlineOfItsOwn(t *testing.T) { + inner := &orderedBackend{name: "x", text: "hej", calls: &[]string{}} + + assert.Same(t, inner, transcribe.WithTimeout(inner, 0), "a zero timeout must not wrap or change anything") +} + +func TestChain_AHungPrimaryReachesTheFallbackWhenItHasItsOwnDeadline(t *testing.T) { + calls := &[]string{} + fb := &orderedBackend{name: "berget/kb-whisper", text: "fallback text", calls: calls} + c := transcribe.NewChain(transcribe.WithTimeout(hangingBackend{name: "iguana/kb-whisper"}, 40*time.Millisecond), fb) + job, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + + got, err := c.Transcribe(job, "/a.wav") + + require.NoError(t, err, "a wedged primary is exactly what the fallback exists for") + assert.Equal(t, "berget/kb-whisper", got.Endpoint) +} + +func TestChain_AJobThatEndedWhileThePrimaryHungStillNeverReachesTheFallback(t *testing.T) { + calls := &[]string{} + fb := &orderedBackend{name: "berget/kb-whisper", text: "fallback text", calls: calls} + c := transcribe.NewChain(transcribe.WithTimeout(hangingBackend{name: "iguana/kb-whisper"}, time.Minute), fb) + job, cancel := context.WithTimeout(context.Background(), 40*time.Millisecond) + defer cancel() + + _, err := c.Transcribe(job, "/a.wav") + + require.Error(t, err) + assert.Empty(t, *calls, "the job's own deadline or a cancel must still stop the audio from reaching a cloud endpoint") +} + +func TestEnsemble_AHungEngineIsASoftFailureAndTheOtherEnginesTextIsUsed(t *testing.T) { + calls := &[]string{} + alt := &orderedBackend{name: "iguana/kb-whisper", text: "text from the other engine", calls: calls} + e := transcribe.NewEnsemble( + transcribe.WithTimeout(hangingBackend{name: "iguana/pianissimo"}, 40*time.Millisecond), alt, nil, + &verdictAdj{v: reconcile.VerdictAlt}, ensembleOpts) + job, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + + got, err := e.Transcribe(job, "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "text from the other engine", got.Text) + assert.Equal(t, domain.ReconcileSingle, got.Reconciliation.Status) + assert.Contains(t, got.Reconciliation.Reason, "timeout") + assert.False(t, errors.Is(err, context.DeadlineExceeded)) +} diff --git a/internal/config/config.go b/internal/config/config.go index 01f01f5..9b43d10 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -20,6 +20,9 @@ import ( // Config is Tapir's fully-resolved runtime configuration. type Config struct { + // UploadConfig is the audio-upload and private-transcription settings (ADR-033). + UploadConfig + // UserID is the Tapir user the run operates as. Stage 0 has exactly one. // It must be a UUID: it keys the UUID user_id/video_id columns (data-model). UserID string @@ -320,6 +323,10 @@ func Load() (Config, error) { } c.OnboardMaxVideoSeconds = onboardMax + if c.UploadConfig, err = loadUploadConfig(); err != nil { + return Config{}, err + } + return c, nil } diff --git a/internal/config/uploads.go b/internal/config/uploads.go new file mode 100644 index 0000000..809c320 --- /dev/null +++ b/internal/config/uploads.go @@ -0,0 +1,197 @@ +package config + +import ( + "fmt" + "os" + "regexp" + "strconv" + "strings" + "time" +) + +// UploadConfig holds the settings for audio upload and private transcription +// (ADR-033). It is embedded in Config, so its fields read as c.UploadsEnabled etc. +type UploadConfig struct { + // UploadsEnabled mounts the /uploads routes and starts the worker. Off by + // default: the runtime image must contain ffmpeg and ffprobe first (ADR-033). + UploadsEnabled bool + // UploadMaxBytes caps one upload. UploadMaxDuration caps one recording; 90 + // minutes is the longest length MEASURED, not a ceiling that was found, so + // raising it is a measurement, not an edit. + UploadMaxBytes int64 + UploadMaxDuration time.Duration + // TranscribeModel is the primary gateway alias; TranscribeFallbackModel is the + // fallback, tried on ANY primary failure. The fallback is a cloud endpoint + // (berget): set it EMPTY to keep private audio on the homelab. + TranscribeModel string + TranscribeFallbackModel string + // TranscribeTimeout bounds one job (conversion + transcription); a 90 minute + // recording took ~7.5 minutes on iguana. + TranscribeTimeout time.Duration + // TranscribeEngineTimeout bounds ONE engine call, so a wedged engine fails while the + // job is still alive and the fallback (or the other engine) can take over. The job + // timeout alone cannot do that (ADR-035 review). + TranscribeEngineTimeout time.Duration + // UploadTmpDir is where uploads are spooled and converted. Audio is deleted + // after transcription, so this holds at most the in-flight files. + UploadTmpDir string + FFmpegBin string + FFprobeBin string + // MaxUploadedTranscriptChars bounds the transcript sent to a chat model for an + // UPLOADED recording. The 18,000-char default for captions is ~22 minutes of + // speech; an hour of meeting is ~50,000 chars. MEASURED 2026-10-01 against the + // real models: iguana/gemma4-26b answers correctly to 90,000 chars but silently + // drops the head of a 120,000-char transcript, so 90,000 is the ceiling of the + // best model, not a round number. + MaxUploadedTranscriptChars int + // UploadChatModel is the model chat over an uploaded recording opens with. It is + // NOT the summarizer default: koala/phi4-mini has an 8,192-token context and + // returns HTTP 400 on an hour of speech (~13,000 tokens). + UploadChatModel string + // ReconcileBaseModel turns on two-engine transcription (ADR-034): the engine whose + // TEXT is the base of the merged transcript (e.g. iguana/pianissimo). TranscribeModel + // stays the other engine (e.g. kb-whisper). Empty = off. + ReconcileBaseModel string + // ReconcileModel is the LOCAL chat model that chooses between the two engines' + // disagreements. Required when ReconcileBaseModel is set. + ReconcileModel string + // ReconcileCheckOrder asks the model twice per dispute with the reading order + // swapped and keeps only agreement. It doubles the time; off by default. + ReconcileCheckOrder bool +} + +const ( + defaultUploadMaxMB = 200 + defaultUploadMaxMinutes = 90 + defaultTranscribeModel = "iguana/kb-whisper" + defaultTranscribeFallbackModel = "berget/kb-whisper" + defaultTranscribeTimeout = 60 * time.Minute + defaultTranscribeEngineTimeout = 20 * time.Minute + defaultMaxUploadedChars = 90000 + defaultUploadChatModel = "iguana/gemma4-26b" +) + +func loadUploadConfig() (UploadConfig, error) { + u := UploadConfig{ + TranscribeModel: envOr("TAPIR_TRANSCRIBE_MODEL", defaultTranscribeModel), + TranscribeFallbackModel: lookupOr("TAPIR_TRANSCRIBE_FALLBACK_MODEL", defaultTranscribeFallbackModel), + UploadTmpDir: envOr("TAPIR_UPLOAD_TMP_DIR", os.TempDir()), + FFmpegBin: envOr("TAPIR_FFMPEG_BIN", "ffmpeg"), + FFprobeBin: envOr("TAPIR_FFPROBE_BIN", "ffprobe"), + UploadChatModel: envOr("TAPIR_UPLOAD_CHAT_MODEL", defaultUploadChatModel), + ReconcileBaseModel: os.Getenv("TAPIR_RECONCILE_BASE_MODEL"), + ReconcileModel: os.Getenv("TAPIR_RECONCILE_MODEL"), + } + + if v := os.Getenv("TAPIR_UPLOADS_ENABLED"); v != "" { + on, err := strconv.ParseBool(v) + if err != nil { + return UploadConfig{}, fmt.Errorf("config: TAPIR_UPLOADS_ENABLED=%q: %w", v, err) + } + u.UploadsEnabled = on + } + + mb, err := positiveInt("TAPIR_UPLOAD_MAX_MB", defaultUploadMaxMB) + if err != nil { + return UploadConfig{}, err + } + u.UploadMaxBytes = int64(mb) << 20 + + mins, err := positiveInt("TAPIR_UPLOAD_MAX_MINUTES", defaultUploadMaxMinutes) + if err != nil { + return UploadConfig{}, err + } + u.UploadMaxDuration = time.Duration(mins) * time.Minute + + timeout, err := durationOr("TAPIR_TRANSCRIBE_TIMEOUT", defaultTranscribeTimeout) + if err != nil { + return UploadConfig{}, err + } + if timeout <= 0 { + return UploadConfig{}, fmt.Errorf("config: TAPIR_TRANSCRIBE_TIMEOUT must be positive, got %v", timeout) + } + u.TranscribeTimeout = timeout + + engineTimeout, err := durationOr("TAPIR_TRANSCRIBE_ENGINE_TIMEOUT", defaultTranscribeEngineTimeout) + if err != nil { + return UploadConfig{}, err + } + if engineTimeout <= 0 { + return UploadConfig{}, fmt.Errorf("config: TAPIR_TRANSCRIBE_ENGINE_TIMEOUT must be positive, got %v", engineTimeout) + } + u.TranscribeEngineTimeout = engineTimeout + + if v := os.Getenv("TAPIR_RECONCILE_CHECK_ORDER"); v != "" { + on, err := strconv.ParseBool(v) + if err != nil { + return UploadConfig{}, fmt.Errorf("config: TAPIR_RECONCILE_CHECK_ORDER=%q: %w", v, err) + } + u.ReconcileCheckOrder = on + } + if err := u.validateReconcile(); err != nil { + return UploadConfig{}, err + } + if !IsLocalModel(u.UploadChatModel) { + return UploadConfig{}, fmt.Errorf("config: TAPIR_UPLOAD_CHAT_MODEL=%q is not a local model (koala/ or iguana/): the whole private transcript is sent to it", u.UploadChatModel) + } + + chars, err := intOr("TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", defaultMaxUploadedChars) + if err != nil { + return UploadConfig{}, err + } + if chars < 0 { + chars = 0 + } + u.MaxUploadedTranscriptChars = chars + return u, nil +} + +// positiveInt is intOr that refuses zero and negatives: a zero upload limit would +// reject every file, which is a misconfiguration to fail on, not to run with. +func positiveInt(key string, fallback int) (int, error) { + n, err := intOr(key, fallback) + if err != nil { + return 0, err + } + if n <= 0 { + return 0, fmt.Errorf("config: %s must be positive, got %d", key, n) + } + return n, nil +} + +// localAlias is the shape of a model alias that runs on the homelab: a local host +// prefix, then one plain name. A bare prefix check let through aliases that are shaped +// like an escape (koala/../berget/x, koala//x, koala/x?host=berget): whether the +// gateway would route them elsewhere is unknown, so they are refused (ADR-035 review). +var localAlias = regexp.MustCompile(`^(koala|iguana)/[A-Za-z0-9][A-Za-z0-9._:-]*$`) + +// IsLocalModel reports whether alias is a local model. Private meeting content may be +// sent to these and to nothing else: reconciliation (ADR-034) and chat over an uploaded +// recording (ADR-035) are both held to it, and a cloud alias is refused at start-up +// rather than discovered later. +func IsLocalModel(alias string) bool { + return localAlias.MatchString(alias) && !strings.Contains(alias, "..") +} + +// validateReconcile checks the two-engine settings. They are inert while no base +// engine is set; once one is, the setup must be complete and entirely local. +func (u UploadConfig) validateReconcile() error { + if u.ReconcileBaseModel == "" { + return nil + } + if u.ReconcileModel == "" { + return fmt.Errorf("config: TAPIR_RECONCILE_BASE_MODEL is set but TAPIR_RECONCILE_MODEL (the adjudicator) is not") + } + for _, m := range []struct{ name, alias string }{ + {"TAPIR_RECONCILE_BASE_MODEL", u.ReconcileBaseModel}, + {"TAPIR_RECONCILE_MODEL", u.ReconcileModel}, + } { + if !IsLocalModel(m.alias) { + return fmt.Errorf("config: %s=%q is not a local model (koala/ or iguana/): private meeting content must not leave the homelab for reconciliation", m.name, m.alias) + } + } + if u.ReconcileBaseModel == u.TranscribeModel { + return fmt.Errorf("config: TAPIR_RECONCILE_BASE_MODEL and TAPIR_TRANSCRIBE_MODEL are both %q: two engines must differ", u.TranscribeModel) + } + return nil +} diff --git a/internal/config/uploads_local_test.go b/internal/config/uploads_local_test.go new file mode 100644 index 0000000..de594c9 --- /dev/null +++ b/internal/config/uploads_local_test.go @@ -0,0 +1,101 @@ +package config + +import ( + "testing" + "time" +) + +// What counts as a LOCAL model alias, and where that is demanded (ADR-034, ADR-035 review). +// A private transcript may be sent to a local model and to nothing else; the check is a +// guard against a misconfiguration or a crafted alias, so it must be strict about shape, +// not just about a prefix. + +func TestIsLocalModel_AcceptsRealLocalAliasesAndNothingShapedLikeAnEscape(t *testing.T) { + tests := []struct { + alias string + want bool + }{ + {"koala/phi4-mini", true}, + {"iguana/gemma4-26b", true}, + {"iguana/gemma4-31b-think", true}, + {"koala/qwen36-35b-a3b", true}, + {"iguana/kb-whisper", true}, + {"iguana/hf.co-model_v1:q4", true}, + {"", false}, + {"koala/", false}, + {"iguana/", false}, + {"gemma4-26b", false}, + {"berget/mistral-small", false}, + {"berget/koala/x", false}, + {"xkoala/x", false}, + {"Koala/x", false}, + {" koala/x", false}, + {"koala/x ", false}, + {"koala/x y", false}, + {"koala/x\n", false}, + {"koala/../berget/kb", false}, + {"koala//berget/mistral-small", false}, + {"koala/x/y", false}, + {"koala/x?host=berget", false}, + {"koala/x#frag", false}, + {"koala/x@evil", false}, + {"koalа/x", false}, // Cyrillic а + } + for _, tt := range tests { + t.Run(tt.alias, func(t *testing.T) { + if got := IsLocalModel(tt.alias); got != tt.want { + t.Errorf("IsLocalModel(%q) = %v, want %v", tt.alias, got, tt.want) + } + }) + } +} + +func TestLoad_UploadChatModelMustBeLocalBecauseTheWholeTranscriptIsSentToIt(t *testing.T) { + for _, alias := range []string{"berget/mistral-small", "gemini/gemini-2.5-pro", "gemma4-26b", "koala/../berget/kb"} { + t.Run(alias, func(t *testing.T) { + clearUploadEnv(t) + setEnv(t, map[string]string{"TAPIR_UPLOAD_CHAT_MODEL": alias}) + + if _, err := Load(); err == nil { + t.Errorf("Load accepted TAPIR_UPLOAD_CHAT_MODEL=%q: a private transcript of up to 90,000 characters would be sent to it", alias) + } + }) + } +} + +func TestLoad_ALocalUploadChatModelIsAccepted(t *testing.T) { + clearUploadEnv(t) + setEnv(t, map[string]string{"TAPIR_UPLOAD_CHAT_MODEL": "iguana/gemma4-31b"}) + + cfg, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.UploadChatModel != "iguana/gemma4-31b" { + t.Errorf("UploadChatModel = %q", cfg.UploadChatModel) + } +} + +func TestLoad_TheEngineTimeoutDefaultsToTwentyMinutesAndMustBePositive(t *testing.T) { + clearUploadEnv(t) + cfg, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + if cfg.TranscribeEngineTimeout != 20*time.Minute { + t.Errorf("TranscribeEngineTimeout = %v, want 20m: a 90-minute recording takes about 8 minutes on kb-whisper, and two wedged engines in sequence must leave the job time to reach a fallback", cfg.TranscribeEngineTimeout) + } + + setEnv(t, map[string]string{"TAPIR_TRANSCRIBE_ENGINE_TIMEOUT": "7m"}) + cfg, err = Load() + if err != nil || cfg.TranscribeEngineTimeout != 7*time.Minute { + t.Errorf("override: %v, %v", cfg.TranscribeEngineTimeout, err) + } + + for _, bad := range []string{"0", "-1m", "soon"} { + setEnv(t, map[string]string{"TAPIR_TRANSCRIBE_ENGINE_TIMEOUT": bad}) + if _, err := Load(); err == nil { + t.Errorf("Load accepted TAPIR_TRANSCRIBE_ENGINE_TIMEOUT=%q", bad) + } + } +} diff --git a/internal/config/uploads_test.go b/internal/config/uploads_test.go new file mode 100644 index 0000000..745bc99 --- /dev/null +++ b/internal/config/uploads_test.go @@ -0,0 +1,217 @@ +package config + +import ( + "os" + "testing" + "time" +) + +// clearUploadEnv unsets every upload knob so a test observes the default, not the +// host environment. +func clearUploadEnv(t *testing.T) { + t.Helper() + for _, k := range []string{ + "TAPIR_UPLOADS_ENABLED", "TAPIR_UPLOAD_MAX_MB", "TAPIR_UPLOAD_MAX_MINUTES", + "TAPIR_TRANSCRIBE_MODEL", "TAPIR_TRANSCRIBE_FALLBACK_MODEL", "TAPIR_TRANSCRIBE_TIMEOUT", "TAPIR_TRANSCRIBE_ENGINE_TIMEOUT", + "TAPIR_UPLOAD_TMP_DIR", "TAPIR_FFMPEG_BIN", "TAPIR_FFPROBE_BIN", + "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", "TAPIR_UPLOAD_CHAT_MODEL", + "TAPIR_RECONCILE_BASE_MODEL", "TAPIR_RECONCILE_MODEL", "TAPIR_RECONCILE_CHECK_ORDER", + } { + unset(t, k) + } +} + +func TestLoad_UploadDefaults(t *testing.T) { + clearUploadEnv(t) + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.UploadsEnabled { + t.Error("uploads must be OFF by default: the runtime image needs ffmpeg first (ADR-033)") + } + if c.UploadMaxBytes != 200<<20 { + t.Errorf("UploadMaxBytes = %d, want 200 MB", c.UploadMaxBytes) + } + if c.UploadMaxDuration != 90*time.Minute { + t.Errorf("UploadMaxDuration = %v, want 90m (the longest length measured, not a found ceiling)", c.UploadMaxDuration) + } + if c.TranscribeModel != "iguana/kb-whisper" { + t.Errorf("TranscribeModel = %q", c.TranscribeModel) + } + if c.TranscribeFallbackModel != "berget/kb-whisper" { + t.Errorf("TranscribeFallbackModel = %q", c.TranscribeFallbackModel) + } + if c.TranscribeTimeout != 60*time.Minute { + t.Errorf("TranscribeTimeout = %v, want 60m (ADR-034: two engines plus ~300 adjudications)", c.TranscribeTimeout) + } + if c.ReconcileBaseModel != "" || c.ReconcileModel != "" || c.ReconcileCheckOrder { + t.Errorf("reconciliation must be OFF by default: %q %q %v", c.ReconcileBaseModel, c.ReconcileModel, c.ReconcileCheckOrder) + } + if c.UploadTmpDir != os.TempDir() { + t.Errorf("UploadTmpDir = %q, want os.TempDir()", c.UploadTmpDir) + } + if c.FFmpegBin != "ffmpeg" || c.FFprobeBin != "ffprobe" { + t.Errorf("bins = %q, %q", c.FFmpegBin, c.FFprobeBin) + } + if c.MaxUploadedTranscriptChars != 90000 { + t.Errorf("MaxUploadedTranscriptChars = %d, want 90000: iguana/gemma4-26b answered correctly at 90,000 chars "+ + "but silently dropped the head of a 120,000-char transcript (docs/research/chat-context)", c.MaxUploadedTranscriptChars) + } + if c.UploadChatModel != "iguana/gemma4-26b" { + t.Errorf("UploadChatModel = %q: koala/phi4-mini has an 8,192-token context and answers HTTP 400 on an hour of speech", c.UploadChatModel) + } +} + +func TestLoad_UploadChatModelOverride(t *testing.T) { + clearUploadEnv(t) + t.Setenv("TAPIR_UPLOAD_CHAT_MODEL", "koala/qwen36-35b-a3b") + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.UploadChatModel != "koala/qwen36-35b-a3b" { + t.Errorf("UploadChatModel = %q", c.UploadChatModel) + } +} + +func TestLoad_UploadValuesAreParsed(t *testing.T) { + clearUploadEnv(t) + setEnv(t, map[string]string{ + "TAPIR_UPLOADS_ENABLED": "true", + "TAPIR_UPLOAD_MAX_MB": "50", + "TAPIR_UPLOAD_MAX_MINUTES": "30", + "TAPIR_TRANSCRIBE_MODEL": "koala/whisper", + "TAPIR_TRANSCRIBE_TIMEOUT": "10m", + "TAPIR_UPLOAD_TMP_DIR": "/var/spool/tapir", + "TAPIR_FFMPEG_BIN": "/opt/ffmpeg", + "TAPIR_FFPROBE_BIN": "/opt/ffprobe", + "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS": "60000", + }) + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if !c.UploadsEnabled || c.UploadMaxBytes != 50<<20 || c.UploadMaxDuration != 30*time.Minute || + c.TranscribeModel != "koala/whisper" || c.TranscribeTimeout != 10*time.Minute || + c.UploadTmpDir != "/var/spool/tapir" || c.FFmpegBin != "/opt/ffmpeg" || + c.FFprobeBin != "/opt/ffprobe" || c.MaxUploadedTranscriptChars != 60000 { + t.Errorf("values not applied: %+v", c) + } +} + +func TestLoad_EmptyTranscribeFallbackDisablesIt(t *testing.T) { + clearUploadEnv(t) + t.Setenv("TAPIR_TRANSCRIBE_FALLBACK_MODEL", "") + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.TranscribeFallbackModel != "" { + t.Errorf("TranscribeFallbackModel = %q: an explicit empty value must disable the cloud fallback, "+ + "so private audio can be kept on the homelab", c.TranscribeFallbackModel) + } +} + +func TestLoad_UploadRejectsInvalidValues(t *testing.T) { + tests := []struct{ key, val string }{ + {"TAPIR_UPLOADS_ENABLED", "maybe"}, + {"TAPIR_UPLOAD_MAX_MB", "lots"}, + {"TAPIR_UPLOAD_MAX_MB", "0"}, + {"TAPIR_UPLOAD_MAX_MB", "-5"}, + {"TAPIR_UPLOAD_MAX_MINUTES", "0"}, + {"TAPIR_UPLOAD_MAX_MINUTES", "forever"}, + {"TAPIR_TRANSCRIBE_TIMEOUT", "soon"}, + {"TAPIR_TRANSCRIBE_TIMEOUT", "0s"}, + } + for _, tt := range tests { + t.Run(tt.key+"="+tt.val, func(t *testing.T) { + clearUploadEnv(t) + t.Setenv(tt.key, tt.val) + + if _, err := Load(); err == nil { + t.Errorf("Load accepted %s=%q; a zero or negative limit would reject every upload", tt.key, tt.val) + } + }) + } +} + +func TestLoad_ReconcileSettingsAreParsed(t *testing.T) { + clearUploadEnv(t) + setEnv(t, map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", + "TAPIR_RECONCILE_MODEL": "iguana/gemma4-31b", + "TAPIR_RECONCILE_CHECK_ORDER": "true", + }) + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.ReconcileBaseModel != "iguana/pianissimo" || c.ReconcileModel != "iguana/gemma4-31b" || !c.ReconcileCheckOrder { + t.Errorf("not applied: %+v", c.UploadConfig) + } +} + +func TestLoad_ReconcileRefusesAnUnsafeOrIncompleteSetup(t *testing.T) { + tests := []struct { + name string + env map[string]string + }{ + {"a base engine with no adjudicator", map[string]string{"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo"}}, + {"a cloud adjudicator: private meeting content must not reach it", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "berget/mistral-small", + }}, + {"a cloud adjudicator from another provider", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "gemini/gemini-2.5-pro", + }}, + {"an adjudicator alias with no host prefix", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "gemma4-26b", + }}, + {"a cloud alias that merely contains a local host name", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "berget/iguana/gemma4-26b", + }}, + {"a cloud base engine", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "berget/kb-whisper", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b", + }}, + {"the same engine twice", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/kb-whisper", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b", + }}, + {"a bad order-check flag", map[string]string{ + "TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b", "TAPIR_RECONCILE_CHECK_ORDER": "perhaps", + }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + clearUploadEnv(t) + setEnv(t, tt.env) + + if _, err := Load(); err == nil { + t.Errorf("Load accepted %v", tt.env) + } + }) + } +} + +func TestLoad_AnAdjudicatorWithoutABaseEngineIsIgnoredNotAnError(t *testing.T) { + clearUploadEnv(t) + t.Setenv("TAPIR_RECONCILE_MODEL", "iguana/gemma4-26b") + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.ReconcileBaseModel != "" { + t.Errorf("reconciliation must stay off without a base engine") + } +} diff --git a/internal/domain/domain.go b/internal/domain/domain.go index 5cf988b..548b51d 100644 --- a/internal/domain/domain.go +++ b/internal/domain/domain.go @@ -111,3 +111,161 @@ type Summary struct { FallbackUsed bool CreatedAt time.Time } + +// Segment is one timed span of a transcription. Times are seconds from the start +// of the audio. +type Segment struct { + Start float64 `json:"start"` + End float64 `json:"end"` + Text string `json:"text"` +} + +// Transcription is the result of transcribing user-supplied audio (ADR-033): +// the full text, its timed segments, which backend served it, and how long the +// audio was. Endpoint is recorded because the backends differ in timestamp +// granularity — which one served a transcript is a quality fact. +type Transcription struct { + Text string + Segments []Segment + Endpoint string + AudioSeconds float64 + // Reconciliation is set when two engines transcribed the same audio and a local + // model chose between their disagreements (ADR-034). Nil for a single engine. + Reconciliation *Reconciliation +} + +// ReconcileStatus says what became of an attempt to merge two transcripts. +type ReconcileStatus string + +const ( + // ReconcileDone: both engines produced text and the disagreements were resolved. + ReconcileDone ReconcileStatus = "reconciled" + // ReconcileSingle: only one engine produced text, so there was nothing to merge. + ReconcileSingle ReconcileStatus = "single_engine" + // ReconcileSkipped: both engines produced text but the model could not be used; + // the base text stands, unreconciled. + ReconcileSkipped ReconcileStatus = "skipped" +) + +// ContestedSpot is a place the engines disagreed that the model judged: it changed it, +// kept the base's reading, or could not decide. Every judged spot is stored, not only +// the ones a reader should check, because the audio is not retained and the kept ones +// are what a later evaluation needs (ADR-035). The page lists only the changed and +// uncertain ones. +type ContestedSpot struct { + Seconds float64 `json:"seconds"` // roughly where in the recording + Base string `json:"base"` // what the base engine heard + Alt string `json:"alt"` // what the other engine heard + Used string `json:"used"` // "base" or "alt": the reading now in the text + Unsure bool `json:"unsure"` // the model could not decide; the base was kept + Before string `json:"before,omitempty"` // the few words just before the spot, to find it by +} + +// Reconciliation records how a merged transcript came about, so the page can say so +// plainly and the user can check the uncertain spots. The counts are exact; Contested +// is bounded. +type Reconciliation struct { + Status ReconcileStatus `json:"status"` + Reason string `json:"reason,omitempty"` // why single_engine or skipped + Base string `json:"base"` // engine whose text is the base + Alt string `json:"alt"` + Reconciler string `json:"reconciler"` + Disputes int `json:"disputes"` // places the engines disagreed + AutoResolved int `json:"auto_resolved"` // trivial: compounds, filler words + Adjudicated int `json:"adjudicated"` // decided by the model + Changed int `json:"changed"` // the alternative's reading was used + Unsure int `json:"unsure"` // base kept, flagged + Errors int `json:"errors"` // model calls that failed + Contested []ContestedSpot `json:"contested,omitempty"` +} + +// StepState is where one phase of an upload job stands (ADR-035). +type StepState string + +const ( + StepPending StepState = "pending" + StepRunning StepState = "running" + StepDone StepState = "done" + StepFailed StepState = "failed" + // StepSkipped is a step that never ran because the job finished without it, for + // example the second engine's step when the first engine's text was used alone. + StepSkipped StepState = "skipped" +) + +// ProgressStep is one named phase of an upload job. Total is zero for a phase with no +// count; only a phase whose work is really countable carries one (ADR-035: no +// percentage that does not come from real counts). +type ProgressStep struct { + Key string `json:"key"` + Label string `json:"label"` + State StepState `json:"state"` + StartedAt time.Time `json:"started_at,omitzero"` + FinishedAt time.Time `json:"finished_at,omitzero"` + Done int `json:"done,omitempty"` + Total int `json:"total,omitempty"` + // Note is a short plain remark about the phase that the reader should see even though + // it did not fail, for example that the audio is much shorter than the file claimed. + // Durations and counts only: never transcript text, a name or a path. + Note string `json:"note,omitempty"` +} + +// UploadStatus is the state of an uploaded-audio transcription job. +type UploadStatus string + +const ( + UploadQueued UploadStatus = "queued" + UploadTranscribing UploadStatus = "transcribing" + UploadDone UploadStatus = "done" + UploadFailed UploadStatus = "failed" +) + +// UploadErrorKind is the typed reason an upload failed. It is deliberately NOT a +// provider's error string: the whisper server's "failed to process audio" has had +// at least five distinct causes, so nothing may branch on it (ADR-033). +type UploadErrorKind string + +const ( + // UploadErrInterrupted: the process restarted mid-job. The audio is not + // retained, so the job cannot resume; the user re-uploads. + UploadErrInterrupted UploadErrorKind = "interrupted" + // UploadErrTranscriptionFailed: every configured transcription endpoint failed. + UploadErrTranscriptionFailed UploadErrorKind = "transcription_failed" + // UploadErrTooLong: the audio, once converted, is longer than the limit. The limit is + // first checked against the file's header, which a hostile or damaged file can lie + // about (ADR-035 review). + UploadErrTooLong UploadErrorKind = "audio_too_long" + // UploadErrBusy: too many recordings were already waiting, so this one was not + // queued. The user's to retry later (ADR-035 review). + UploadErrBusy UploadErrorKind = "queue_full" +) + +// UploadedTranscript is a user's PRIVATE transcript of audio they uploaded. It is +// never stored in, or merged with, the shared public-caption Transcript (ADR-021). +// Content and Segments are empty until Status is UploadDone. +type UploadedTranscript struct { + ID string + UserID string + Title string + Status UploadStatus + Content string + Segments []Segment + Endpoint string + AudioSeconds float64 + ErrorKind UploadErrorKind + // HadVideo records that the uploaded file carried a video track. Only its audio + // was used and the video was discarded; the page says so. It is set at upload + // and never cleared (ADR-033). + HadVideo bool + // Reconciliation is how a merged transcript came about, or nil for a single + // engine (ADR-034). Set by Get, not by List. + Reconciliation *Reconciliation + // Progress is the job's phases, in order. Set by Get, not by List (ADR-035). + Progress []ProgressStep + // ProgressAt is when a phase last really changed or advanced; AliveAt is the + // heartbeat. They differ on purpose: alive says the service is still reporting, + // progress says work is moving (ADR-035). Both are zero until the job starts. + ProgressAt time.Time + AliveAt time.Time + CreatedAt time.Time + FinishedAt time.Time // zero until the job reaches done or failed +} diff --git a/internal/errkind/errkind.go b/internal/errkind/errkind.go new file mode 100644 index 0000000..f1d719d --- /dev/null +++ b/internal/errkind/errkind.go @@ -0,0 +1,51 @@ +// Package errkind reduces an error to a short kind that is safe to log, store and show. +// +// An error's own text can hold a provider's response body (a gateway that echoes the +// prompt), a file path (ffmpeg names the spool file), or a word from a private +// transcript. None of that may reach a log line, a database column or a page about a +// private recording (ADR-033: "typed, never a provider's string"; ADR-035). Of returns +// only a fixed vocabulary, so it can never carry any. +package errkind + +import ( + "context" + "errors" + "fmt" + "net" +) + +// HTTPStatuser is implemented by errors that are a non-2xx answer from a service. +type HTTPStatuser interface{ HTTPStatus() int } + +// SQLStater is implemented by database errors that carry a SQLSTATE (pgx's PgError does). +// The five-character code is a fixed vocabulary; the error's message is not: Postgres quotes +// the offending value, which for a transcript row is transcript text. +type SQLStater interface{ SQLState() string } + +// Of is the kind of err: "timeout", "canceled", "http_", "sql_", "network", +// or "error". +// It is empty for nil. +func Of(err error) string { + if err == nil { + return "" + } + var hs HTTPStatuser + var ss SQLStater + var ne net.Error + switch { + case errors.Is(err, context.DeadlineExceeded): + return "timeout" + case errors.Is(err, context.Canceled): + return "canceled" + case errors.As(err, &hs): + return fmt.Sprintf("http_%d", hs.HTTPStatus()) + case errors.As(err, &ss) && ss.SQLState() != "": + return "sql_" + ss.SQLState() + case errors.As(err, &ne): + if ne.Timeout() { + return "timeout" + } + return "network" + } + return "error" +} diff --git a/internal/errkind/errkind_test.go b/internal/errkind/errkind_test.go new file mode 100644 index 0000000..d9bb6d6 --- /dev/null +++ b/internal/errkind/errkind_test.go @@ -0,0 +1,82 @@ +package errkind_test + +import ( + "context" + "errors" + "fmt" + "net" + "net/url" + "testing" + + "github.com/stretchr/testify/assert" + + "git.d-ma.be/mathias/tapir/internal/errkind" +) + +// Of reduces an error to a short kind that is safe to log, store and show: it never +// carries the error's own text, because that text can hold a provider's response body, a +// file path or a word from a private transcript (ADR-035 review F3, F5). + +type statusErr struct{ code int } + +func (e statusErr) Error() string { + return fmt.Sprintf("gateway said: %d and echoed the transcript", e.code) +} +func (e statusErr) HTTPStatus() int { return e.code } + +type timeoutNet struct{} + +func (timeoutNet) Error() string { return "i/o timeout to 10.0.0.1" } +func (timeoutNet) Timeout() bool { return true } +func (timeoutNet) Temporary() bool { return true } + +func TestOf_ReducesAnErrorToAKindAndNeverItsText(t *testing.T) { + tests := []struct { + name string + err error + want string + }{ + {"nothing", nil, ""}, + {"a deadline", context.DeadlineExceeded, "timeout"}, + {"a wrapped deadline", fmt.Errorf("transcribe: %w", context.DeadlineExceeded), "timeout"}, + {"a cancellation", fmt.Errorf("job: %w", context.Canceled), "canceled"}, + {"an HTTP status", statusErr{code: 502}, "http_502"}, + {"a wrapped HTTP status", fmt.Errorf("model: %w", statusErr{code: 400}), "http_400"}, + {"a network timeout", timeoutNet{}, "timeout"}, + {"a network failure", &net.OpError{Op: "dial", Err: errors.New("connection refused")}, "network"}, + {"a url error wrapping a refusal", &url.Error{Op: "Post", URL: "http://gateway/secret-path", Err: errors.New("refused")}, "network"}, + {"anything else", errors.New("moov atom not found in /spool/upload-555"), "error"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + assert.Equal(t, tt.want, errkind.Of(tt.err)) + }) + } +} + +type sqlErr struct{} + +func (sqlErr) Error() string { + return `ERROR: invalid input syntax for type json: "Anders ansvarar för virket"` +} +func (sqlErr) SQLState() string { return "22P02" } + +func TestOf_ADatabaseErrorIsReducedToItsSQLStateNeverItsMessage(t *testing.T) { + // A Postgres error can quote the offending value, which for a transcript row is + // transcript text. The five-character SQLSTATE is a fixed vocabulary (fourth review). + err := fmt.Errorf("store: complete upload: %w", sqlErr{}) + + got := errkind.Of(err) + + assert.Equal(t, "sql_22P02", got) + assert.NotContains(t, got, "Anders") +} + +func TestOf_AKindCarriesNoWordFromTheError(t *testing.T) { + err := fmt.Errorf("convert /spool/upload-555: %w", errors.New("Anders ansvarar för virket")) + + got := errkind.Of(err) + + assert.NotContains(t, got, "Anders") + assert.NotContains(t, got, "spool") +} diff --git a/internal/progress/tracker.go b/internal/progress/tracker.go new file mode 100644 index 0000000..71c3d2f --- /dev/null +++ b/internal/progress/tracker.go @@ -0,0 +1,284 @@ +// Package progress tracks the phases of one upload job (ADR-035). A Tracker is the +// only writer of a job's progress: every transition is saved to the job's row and +// logged by the same call, so what the page shows and what the log says cannot +// disagree. The engines and the reconciler report through a Reporter carried in the +// context, so the Transcriber interface stays what it was. +package progress + +import ( + "context" + "log/slog" + "sync" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" +) + +// Planned is one phase of a job, declared up front so the page can show what is coming. +type Planned struct { + Key, Label string +} + +// Reporter is what a long-running component reports to. Begin starts a phase (total +// is the number of countable units, 0 when there is no honest count), Advance says how +// many are done, Done closes the phase. Fail closes a phase that did not work while +// the job goes on (an engine that was down, the other one's text used instead) and +// says why in the log. Note is a log line for the operator that costs no write. +type Reporter interface { + Begin(key string, total int) + Advance(key string, done int) + Done(key string) + Fail(key string, err error) + Note(msg string, kv ...any) +} + +// Sink is where a job's progress is persisted. Both calls are best effort: progress +// is for people, so a failed write is logged and never stops the job. +type Sink interface { + SaveProgress(ctx context.Context, steps []domain.ProgressStep, at time.Time) error + Beat(ctx context.Context) error +} + +const ( + defaultMinSave = 2 * time.Second + logAdvanceEach = time.Minute +) + +// Option tunes a Tracker. +type Option func(*Tracker) + +// WithClock fixes the clock, for tests. +func WithClock(now func() time.Time) Option { return func(t *Tracker) { t.now = now } } + +// WithMinSave sets the least time between two saves caused by Advance alone. Begin, +// Done, Fail and Finish always save. +func WithMinSave(d time.Duration) Option { return func(t *Tracker) { t.minSave = d } } + +// Tracker implements Reporter for one job. +type Tracker struct { + ctx context.Context + sink Sink + log *slog.Logger + now func() time.Time + minSave time.Duration + + mu sync.Mutex + steps []domain.ProgressStep + lastAdvanceSave time.Time + lastAdvanceLog time.Time +} + +// NewTracker plans a job's phases. ctx is used for the saves, so it should outlive the +// job's own timeout: a job that timed out still gets its failure recorded. +func NewTracker(ctx context.Context, sink Sink, log *slog.Logger, plan []Planned, opts ...Option) *Tracker { + t := &Tracker{ctx: ctx, sink: sink, log: log, now: time.Now, minSave: defaultMinSave} + for _, o := range opts { + o(t) + } + for _, p := range plan { + t.steps = append(t.steps, domain.ProgressStep{Key: p.Key, Label: p.Label, State: domain.StepPending}) + } + return t +} + +// Start saves the plan, every phase pending. +func (t *Tracker) Start() { + t.mu.Lock() + defer t.mu.Unlock() + t.log.Info("job started", "steps", len(t.steps)) + t.save() +} + +// Begin implements Reporter. +func (t *Tracker) Begin(key string, total int) { + t.mu.Lock() + defer t.mu.Unlock() + s := t.find(key) + if s == nil { + return + } + s.State, s.StartedAt, s.FinishedAt, s.Done, s.Total = domain.StepRunning, t.now(), time.Time{}, 0, total + t.log.Info("step started", "step", key, "total", total) + t.save() +} + +// Advance implements Reporter. The count is always kept; it is saved at most once per +// minimum interval and logged at most once a minute, so a counted phase does not turn +// into a write and a log line per unit. +func (t *Tracker) Advance(key string, done int) { + t.mu.Lock() + defer t.mu.Unlock() + s := t.find(key) + if s == nil { + return + } + s.Done = done + now := t.now() + if t.lastAdvanceLog.IsZero() || now.Sub(t.lastAdvanceLog) >= logAdvanceEach { + t.lastAdvanceLog = now + t.log.Info("step progress", "step", key, "done", done, "total", s.Total) + } + if t.lastAdvanceSave.IsZero() || now.Sub(t.lastAdvanceSave) >= t.minSave { + t.lastAdvanceSave = now + t.save() + } +} + +// Done implements Reporter. +func (t *Tracker) Done(key string) { + t.mu.Lock() + defer t.mu.Unlock() + s := t.find(key) + if s == nil { + return + } + t.close(s, domain.StepDone) + t.log.Info("step finished", "step", key, "elapsed", s.FinishedAt.Sub(s.StartedAt)) + t.save() +} + +// Fail implements Reporter: one phase failed and the job carries on. +func (t *Tracker) Fail(key string, err error) { + t.mu.Lock() + defer t.mu.Unlock() + s := t.find(key) + if s == nil { + return + } + t.close(s, domain.StepFailed) + t.log.Warn("step failed", "step", key, "elapsed", s.FinishedAt.Sub(s.StartedAt), "kind", errkind.Of(err)) + t.save() +} + +// Note implements Reporter. +func (t *Tracker) Note(msg string, kv ...any) { t.log.Info(msg, kv...) } + +// FailRunning marks the phase that was running as failed, because the whole job +// failed. What never started stays pending, which is what shows the user where the +// job stopped. +func (t *Tracker) FailRunning() { + t.mu.Lock() + defer t.mu.Unlock() + for i := range t.steps { + if t.steps[i].State == domain.StepRunning { + t.close(&t.steps[i], domain.StepFailed) + t.log.Warn("step failed", "step", t.steps[i].Key, "elapsed", t.steps[i].FinishedAt.Sub(t.steps[i].StartedAt)) + } + } + t.save() +} + +// Finish closes the job: a phase still running is done, a phase that never ran is +// skipped. +func (t *Tracker) Finish() { + t.mu.Lock() + defer t.mu.Unlock() + for i := range t.steps { + switch t.steps[i].State { + case domain.StepRunning: + t.close(&t.steps[i], domain.StepDone) + case domain.StepPending: + t.steps[i].State = domain.StepSkipped + } + } + t.log.Info("job finished") + t.save() +} + +// Annotate attaches a short plain note to a phase and saves it at once. It is for a +// remark the reader should see although nothing failed. Durations and counts only: a note +// is shown on a page about a private recording. +func (t *Tracker) Annotate(key, note string) { + t.mu.Lock() + defer t.mu.Unlock() + s := t.find(key) + if s == nil { + return + } + s.Note = note + t.save() +} + +// Heartbeat stamps the job's row every beatEvery and logs which phase is still running +// every stillEvery, until ctx ends. It runs on its own, independent of any engine call: +// a heartbeat that stopped means the process is gone, not that a request is slow. +func (t *Tracker) Heartbeat(ctx context.Context, beatEvery, stillEvery time.Duration) { + beat, still := time.NewTicker(beatEvery), time.NewTicker(stillEvery) + defer beat.Stop() + defer still.Stop() + for { + select { + case <-ctx.Done(): + return + case <-beat.C: + if err := t.sink.Beat(ctx); err != nil && ctx.Err() == nil { + t.log.Warn("progress: heartbeat not saved", "err", err) + } + case <-still.C: + t.logStill() + } + } +} + +func (t *Tracker) logStill() { + t.mu.Lock() + defer t.mu.Unlock() + for _, s := range t.steps { + if s.State == domain.StepRunning { + t.log.Info("still running", "step", s.Key, "elapsed", t.now().Sub(s.StartedAt), "done", s.Done, "total", s.Total) + return + } + } +} + +// close ends a phase. Callers hold t.mu. +func (t *Tracker) close(s *domain.ProgressStep, state domain.StepState) { + s.State, s.FinishedAt = state, t.now() + if s.StartedAt.IsZero() { + s.StartedAt = s.FinishedAt + } +} + +// find returns the phase with this key, or nil. Callers hold t.mu. +func (t *Tracker) find(key string) *domain.ProgressStep { + for i := range t.steps { + if t.steps[i].Key == key { + return &t.steps[i] + } + } + return nil +} + +// save persists a copy of the phases. Callers hold t.mu. A failed write is logged and +// dropped: progress is cosmetic and must never fail the job it describes. +func (t *Tracker) save() { + snapshot := append([]domain.ProgressStep(nil), t.steps...) + if err := t.sink.SaveProgress(t.ctx, snapshot, t.now()); err != nil && t.ctx.Err() == nil { + t.log.Warn("progress: not saved", "err", err) + } +} + +type ctxKey struct{} + +// With returns a context that carries r. +func With(ctx context.Context, r Reporter) context.Context { + return context.WithValue(ctx, ctxKey{}, r) +} + +// From returns the Reporter carried by ctx, or one that does nothing, so a component +// can always report without checking whether anyone is listening. +func From(ctx context.Context) Reporter { + if r, ok := ctx.Value(ctxKey{}).(Reporter); ok { + return r + } + return noop{} +} + +type noop struct{} + +func (noop) Begin(string, int) {} +func (noop) Advance(string, int) {} +func (noop) Done(string) {} +func (noop) Fail(string, error) {} +func (noop) Note(string, ...any) {} diff --git a/internal/progress/tracker_test.go b/internal/progress/tracker_test.go new file mode 100644 index 0000000..52b00ee --- /dev/null +++ b/internal/progress/tracker_test.go @@ -0,0 +1,402 @@ +package progress_test + +import ( + "context" + "errors" + "log/slog" + "strings" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/progress" +) + +// fakeSink records every saved snapshot and every heartbeat. +type fakeSink struct { + mu sync.Mutex + saves [][]domain.ProgressStep + at []time.Time + beats int + err error +} + +func (f *fakeSink) SaveProgress(_ context.Context, steps []domain.ProgressStep, at time.Time) error { + f.mu.Lock() + defer f.mu.Unlock() + f.saves = append(f.saves, append([]domain.ProgressStep(nil), steps...)) + f.at = append(f.at, at) + return f.err +} + +func (f *fakeSink) Beat(context.Context) error { + f.mu.Lock() + defer f.mu.Unlock() + f.beats++ + return f.err +} + +func (f *fakeSink) last(t *testing.T) []domain.ProgressStep { + t.Helper() + f.mu.Lock() + defer f.mu.Unlock() + require.NotEmpty(t, f.saves, "nothing was saved") + return f.saves[len(f.saves)-1] +} + +func (f *fakeSink) saveCount() int { + f.mu.Lock() + defer f.mu.Unlock() + return len(f.saves) +} + +func (f *fakeSink) beatCount() int { + f.mu.Lock() + defer f.mu.Unlock() + return f.beats +} + +// syncBuf is a log sink safe to read while the heartbeat goroutine is still writing. +type syncBuf struct { + mu sync.Mutex + b strings.Builder +} + +func (s *syncBuf) Write(p []byte) (int, error) { + s.mu.Lock() + defer s.mu.Unlock() + return s.b.Write(p) +} + +func (s *syncBuf) String() string { + s.mu.Lock() + defer s.mu.Unlock() + return s.b.String() +} + +// clock is a hand-wound clock so durations and throttling are exact. +type clock struct{ t time.Time } + +func (c *clock) now() time.Time { return c.t } +func (c *clock) tick(d time.Duration) { c.t = c.t.Add(d) } +func newClock() *clock { return &clock{t: time.Date(2026, 10, 2, 8, 0, 0, 0, time.UTC)} } + +func plan() []progress.Planned { + return []progress.Planned{{Key: "audio", Label: "Preparing audio"}, {Key: "decide", Label: "Deciding"}, {Key: "save", Label: "Saving"}} +} + +func newTracker(t *testing.T) (*progress.Tracker, *fakeSink, *clock, *syncBuf) { + t.Helper() + sink, clk, buf := &fakeSink{}, newClock(), &syncBuf{} + log := slog.New(slog.NewTextHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug})) + return progress.NewTracker(context.Background(), sink, log, plan(), progress.WithClock(clk.now), progress.WithMinSave(2*time.Second)), sink, clk, buf +} + +func stateOf(steps []domain.ProgressStep, key string) domain.StepState { + for _, s := range steps { + if s.Key == key { + return s.State + } + } + return "" +} + +func TestTracker_PlanIsShownUpFrontInOrderAllPending(t *testing.T) { + tr, sink, _, _ := newTracker(t) + + tr.Start() + + steps := sink.last(t) + require.Len(t, steps, 3) + require.Equal(t, []string{"audio", "decide", "save"}, []string{steps[0].Key, steps[1].Key, steps[2].Key}) + for _, s := range steps { + require.Equal(t, domain.StepPending, s.State, s.Key) + } + require.Equal(t, "Preparing audio", steps[0].Label) +} + +func TestTracker_BeginRunsAStepAndDoneRecordsItsDuration(t *testing.T) { + tr, sink, clk, _ := newTracker(t) + tr.Start() + + tr.Begin("audio", 0) + require.Equal(t, domain.StepRunning, stateOf(sink.last(t), "audio")) + clk.tick(90 * time.Second) + tr.Done("audio") + + s := sink.last(t)[0] + require.Equal(t, domain.StepDone, s.State) + require.Equal(t, 90*time.Second, s.FinishedAt.Sub(s.StartedAt), "the finished step must carry how long it took") +} + +func TestTracker_AdvanceCountsAreSavedAndThrottled(t *testing.T) { + tr, sink, clk, _ := newTracker(t) + tr.Start() + tr.Begin("decide", 282) + base := sink.saveCount() + + tr.Advance("decide", 1) + clk.tick(500 * time.Millisecond) + tr.Advance("decide", 2) + clk.tick(500 * time.Millisecond) + tr.Advance("decide", 3) + + require.Equal(t, base+1, sink.saveCount(), "advances inside the minimum interval must not each hit the database") + clk.tick(2 * time.Second) + tr.Advance("decide", 4) + require.Equal(t, base+2, sink.saveCount()) + got := sink.last(t)[1] + require.Equal(t, 4, got.Done) + require.Equal(t, 282, got.Total) +} + +func TestTracker_DoneFlushesTheFinalCountEvenInsideTheThrottle(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + tr.Begin("decide", 10) + tr.Advance("decide", 9) + + tr.Advance("decide", 10) + tr.Done("decide") + + got := sink.last(t)[1] + require.Equal(t, domain.StepDone, got.State) + require.Equal(t, 10, got.Done, "a finished step must show its full count, not the last throttled one") +} + +func TestTracker_FinishSkipsWhatNeverRanAndClosesWhatDid(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + tr.Done("audio") + tr.Begin("decide", 5) + + tr.Finish() + + steps := sink.last(t) + require.Equal(t, domain.StepDone, stateOf(steps, "audio")) + require.Equal(t, domain.StepDone, stateOf(steps, "decide"), "a step still running when the job completes is closed") + require.Equal(t, domain.StepSkipped, stateOf(steps, "save"), "a step that never ran is skipped, not left pending") +} + +func TestTracker_FailRunningMarksTheRunningStepFailedAndLeavesTheRestPending(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + tr.Done("audio") + tr.Begin("decide", 5) + + tr.FailRunning() + + steps := sink.last(t) + require.Equal(t, domain.StepDone, stateOf(steps, "audio")) + require.Equal(t, domain.StepFailed, stateOf(steps, "decide")) + require.Equal(t, domain.StepPending, stateOf(steps, "save"), "what never started stays pending: it shows where the job stopped") +} + +func TestTracker_AKeyedFailMarksOneStepFailedLogsItsKindAndTheJobGoesOn(t *testing.T) { + tr, sink, _, buf := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + + tr.Fail("audio", errors.New("gateway said no")) + tr.Begin("decide", 5) + + steps := sink.last(t) + require.Equal(t, domain.StepFailed, stateOf(steps, "audio")) + require.Equal(t, domain.StepRunning, stateOf(steps, "decide"), "a soft failure must not stop the next step from running") + require.Contains(t, buf.String(), "level=WARN") + require.NotContains(t, buf.String(), "gateway said no", "an error's text can carry provider output or a path: the log gets its kind") + require.Contains(t, buf.String(), "kind=error") + require.Contains(t, buf.String(), "step=audio") +} + +func TestTracker_NoteIsALogLineAndNeverASave(t *testing.T) { + tr, sink, _, buf := newTracker(t) + tr.Start() + before := sink.saveCount() + + tr.Note("reconciled", "disputes", 362, "auto_resolved", 80) + + require.Equal(t, before, sink.saveCount(), "a note is for the operator; it must not cost a database write") + require.Contains(t, buf.String(), "reconciled") + require.Contains(t, buf.String(), "disputes=362") + require.Contains(t, buf.String(), "auto_resolved=80") +} + +func TestTracker_AnUnknownKeyIsIgnored(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + before := sink.saveCount() + + require.NotPanics(t, func() { + tr.Begin("no-such-step", 0) + tr.Advance("no-such-step", 1) + tr.Done("no-such-step") + tr.Fail("no-such-step", errors.New("x")) + }) + require.Equal(t, before, sink.saveCount()) +} + +func TestTracker_ASinkFailureIsLoggedAndNeverStopsTheJob(t *testing.T) { + tr, sink, _, buf := newTracker(t) + sink.err = errors.New("db down") + + require.NotPanics(t, func() { + tr.Start() + tr.Begin("audio", 0) + tr.Done("audio") + }) + require.Contains(t, buf.String(), "level=WARN", "a lost progress write is worth a WARN") + require.Contains(t, buf.String(), "db down") +} + +func TestTracker_EveryTransitionIsLoggedByTheSameCall(t *testing.T) { + tr, _, clk, buf := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + clk.tick(3 * time.Second) + tr.Done("audio") + tr.Begin("decide", 5) + clk.tick(2 * time.Second) + tr.FailRunning() + tr.Finish() + + out := buf.String() + for _, want := range []string{ + `msg="job started"`, + `msg="step started" step=audio`, + `msg="step finished" step=audio elapsed=3s`, + `msg="step started" step=decide total=5`, + `msg="step failed" step=decide elapsed=2s`, + `msg="job finished"`, + } { + require.Contains(t, out, want, "a transition with no log line is invisible to the operator") + } +} + +func TestTracker_AKeyedFailIsSavedAtOnceNotOnTheNextTransition(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + before := sink.saveCount() + + tr.Fail("audio", errors.New("down")) + + require.Equal(t, before+1, sink.saveCount(), "a soft-failed phase must show as failed on the page now") + require.Equal(t, domain.StepFailed, stateOf(sink.last(t), "audio")) +} + +// retainingSink keeps the very slice it is handed, like a sink that queues it for later. +type retainingSink struct{ got [][]domain.ProgressStep } + +func (r *retainingSink) SaveProgress(_ context.Context, steps []domain.ProgressStep, _ time.Time) error { + r.got = append(r.got, steps) + return nil +} +func (r *retainingSink) Beat(context.Context) error { return nil } + +func TestTracker_ASnapshotHandedToTheSinkIsNotChangedByLaterTransitions(t *testing.T) { + sink := &retainingSink{} + tr := progress.NewTracker(context.Background(), sink, slog.New(slog.NewTextHandler(&syncBuf{}, nil)), plan(), + progress.WithClock(newClock().now)) + tr.Start() + first := sink.got[0] + + tr.Begin("audio", 0) + tr.Done("audio") + + require.Equal(t, domain.StepPending, first[0].State, "a sink that keeps a snapshot must not see it change under it") +} + +func TestTracker_AdvanceIsLoggedAtMostOncePerMinute(t *testing.T) { + tr, _, clk, buf := newTracker(t) + tr.Start() + tr.Begin("decide", 100) + + for i := 1; i <= 30; i++ { + clk.tick(1 * time.Second) + tr.Advance("decide", i) + } + require.LessOrEqual(t, strings.Count(buf.String(), "step progress"), 1, + "30 seconds of progress is at most one log line, not thirty") + clk.tick(61 * time.Second) + tr.Advance("decide", 31) + require.Equal(t, 2, strings.Count(buf.String(), "step progress")) + require.Contains(t, buf.String(), "done=31") + require.Contains(t, buf.String(), "total=100") +} + +func TestTracker_HeartbeatBeatsOnItsOwnUntilStopped(t *testing.T) { + tr, sink, _, _ := newTracker(t) + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + + go func() { + defer close(done) + tr.Heartbeat(ctx, 5*time.Millisecond, time.Hour) + }() + + require.Eventually(t, func() bool { return sink.beatCount() >= 3 }, 2*time.Second, 5*time.Millisecond, + "a heartbeat that does not depend on any engine call is the point") + cancel() + select { + case <-done: + case <-time.After(2 * time.Second): + t.Fatal("Heartbeat did not stop when its context ended") + } +} + +func TestTracker_HeartbeatSaysWhichStepIsStillRunningAndForHowLong(t *testing.T) { + tr, _, clk, buf := newTracker(t) + tr.Start() + tr.Begin("decide", 100) + clk.tick(7 * time.Minute) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + go tr.Heartbeat(ctx, time.Hour, 5*time.Millisecond) + + require.Eventually(t, func() bool { return strings.Contains(buf.String(), "still running") }, 2*time.Second, 5*time.Millisecond) + require.Contains(t, buf.String(), "step=decide") + require.Contains(t, buf.String(), "elapsed=7m0s") +} + +func TestFrom_WithoutAReporterReturnsOneThatDoesNothing(t *testing.T) { + r := progress.From(context.Background()) + + require.NotNil(t, r) + require.NotPanics(t, func() { + r.Begin("x", 3) + r.Advance("x", 1) + r.Done("x") + r.Fail("x", errors.New("e")) + r.Note("n", "k", 1) + }) +} + +func TestWithAndFrom_ACarriedReporterIsTheOneReturned(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + ctx := progress.With(context.Background(), tr) + + progress.From(ctx).Begin("audio", 0) + + require.Equal(t, domain.StepRunning, stateOf(sink.last(t), "audio")) +} + +func TestTracker_AnnotateAttachesANoteToAStepAndSavesItAtOnce(t *testing.T) { + tr, sink, _, _ := newTracker(t) + tr.Start() + tr.Begin("audio", 0) + before := sink.saveCount() + + tr.Annotate("audio", "20 min of the 79 the file claimed") + + require.Equal(t, before+1, sink.saveCount()) + require.Equal(t, "20 min of the 79 the file claimed", sink.last(t)[0].Note) + require.NotPanics(t, func() { tr.Annotate("no-such-step", "x") }) +} diff --git a/internal/reconcile/adjudicate.go b/internal/reconcile/adjudicate.go new file mode 100644 index 0000000..6959a60 --- /dev/null +++ b/internal/reconcile/adjudicate.go @@ -0,0 +1,119 @@ +package reconcile + +import ( + "context" + "fmt" + "regexp" + "strings" +) + +// Verdict is an adjudicator's decision about one dispute. The zero value is +// VerdictUnsure: with no decision the base reading stands and the spot is flagged. +type Verdict int + +const ( + VerdictUnsure Verdict = iota + VerdictBase + VerdictAlt +) + +// Request is one dispute as the adjudicator sees it: the surrounding words, the two +// readings, and which of them is shown as "Reading 1". Alternating AltFirst between +// disputes averages out position bias. +type Request struct { + Before, After string + Base, Alt string + AltFirst bool +} + +// Adjudicator decides which reading of a dispute the speaker most likely said. +type Adjudicator interface { + Adjudicate(ctx context.Context, r Request) (Verdict, error) +} + +// Completer is the chat surface the LLM adjudicator needs. *llm.Client satisfies it. +type Completer interface { + Complete(ctx context.Context, system, user string) (string, error) +} + +// systemPrompt is the wording that was MEASURED (docs/research/ +// local-model-adjudicator-sweep-2026-10-01.md): gemma4-26b 92% and gemma4-31b 97% against +// the speaker's blind answers. Changing it invalidates those figures. +const systemPrompt = "You compare two automatic transcriptions of the same Swedish speech. A single person is dictating project notes " + + "(finance and payments jargon, with English terms mixed into Swedish). The two transcriptions differ at ONE marked spot. " + + "Decide which reading is what the speaker most likely actually said, using the surrounding text and ordinary Swedish. " + + "Reply with exactly one line, either 'ANSWER: 1', 'ANSWER: 2' or 'ANSWER: UNSURE', then one short sentence of reasoning." + +var answerRe = regexp.MustCompile(`(?i)ANSWER:\s*(1|2|UNSURE)\b`) + +// LLMAdjudicator asks a chat model. It never invents an answer: a reply it cannot read +// is VerdictUnsure, and a transport error is returned as an error so the caller can tell +// "the model is down" from "the model is unsure". +type LLMAdjudicator struct{ c Completer } + +// NewLLMAdjudicator wraps a Completer. +func NewLLMAdjudicator(c Completer) *LLMAdjudicator { return &LLMAdjudicator{c: c} } + +// Adjudicate implements Adjudicator. +func (a *LLMAdjudicator) Adjudicate(ctx context.Context, r Request) (Verdict, error) { + first, second := r.Base, r.Alt + if r.AltFirst { + first, second = r.Alt, r.Base + } + user := fmt.Sprintf("Text before the spot:\n%s\n\n[[ THE SPOT ]]\n\nText after the spot:\n%s\n\nReading 1: %s\nReading 2: %s\n", + r.Before, r.After, describe(first), describe(second)) + reply, err := a.c.Complete(ctx, systemPrompt, user) + if err != nil { + return VerdictUnsure, fmt.Errorf("reconcile: adjudicate: %w", err) + } + m := answerRe.FindStringSubmatch(reply) + if m == nil { + return VerdictUnsure, nil + } + switch strings.ToUpper(m[1]) { + case "1": + if r.AltFirst { + return VerdictAlt, nil + } + return VerdictBase, nil + case "2": + if r.AltFirst { + return VerdictBase, nil + } + return VerdictAlt, nil + } + return VerdictUnsure, nil +} + +func describe(reading string) string { + if strings.TrimSpace(reading) == "" { + return "(nothing: the words are simply absent)" + } + return reading +} + +// OrderChecked asks the inner adjudicator twice with the reading order swapped and +// keeps a verdict only when both agree. A model that flips with the order is showing +// position bias, not judgement, so disagreement is VerdictUnsure. It doubles the time. +type OrderChecked struct{ inner Adjudicator } + +// NewOrderChecked wraps an adjudicator. +func NewOrderChecked(inner Adjudicator) *OrderChecked { return &OrderChecked{inner: inner} } + +// Adjudicate implements Adjudicator. +func (o *OrderChecked) Adjudicate(ctx context.Context, r Request) (Verdict, error) { + r.AltFirst = false + v1, err := o.inner.Adjudicate(ctx, r) + if err != nil { + return VerdictUnsure, err + } + r.AltFirst = true + v2, err := o.inner.Adjudicate(ctx, r) + if err != nil { + return VerdictUnsure, err + } + if v1 == v2 { + return v1, nil + } + return VerdictUnsure, nil +} diff --git a/internal/reconcile/adjudicate_test.go b/internal/reconcile/adjudicate_test.go new file mode 100644 index 0000000..7f73fba --- /dev/null +++ b/internal/reconcile/adjudicate_test.go @@ -0,0 +1,147 @@ +package reconcile_test + +import ( + "context" + "errors" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +type fakeCompleter struct { + replies []string + err error + systems []string + users []string +} + +func (f *fakeCompleter) Complete(_ context.Context, system, user string) (string, error) { + f.systems = append(f.systems, system) + f.users = append(f.users, user) + if f.err != nil { + return "", f.err + } + i := len(f.users) - 1 + if i >= len(f.replies) { + i = len(f.replies) - 1 + } + return f.replies[i], nil +} + +var req = reconcile.Request{Before: "grannarna ska då", After: "De ska kunna måla", Base: "målade", Alt: "mölade"} + +func TestLLMAdjudicator_SendsTheMeasuredPromptWithTheReadingsInTheRequestedOrder(t *testing.T) { + fc := &fakeCompleter{replies: []string{"ANSWER: 1"}} + a := reconcile.NewLLMAdjudicator(fc) + + _, err := a.Adjudicate(context.Background(), req) + + require.NoError(t, err) + require.Len(t, fc.users, 1) + assert.Contains(t, fc.systems[0], "two automatic transcriptions of the same Swedish speech") + assert.Contains(t, fc.systems[0], "ANSWER: 1") + u := fc.users[0] + assert.Contains(t, u, "Text before the spot:\ngrannarna ska då") + assert.Contains(t, u, "[[ THE SPOT ]]") + assert.Contains(t, u, "Text after the spot:\nDe ska kunna måla") + assert.Contains(t, u, "Reading 1: målade\nReading 2: mölade", "base first when AltFirst is false") +} + +func TestLLMAdjudicator_AltFirstSwapsWhichReadingIsNumberOne(t *testing.T) { + fc := &fakeCompleter{replies: []string{"ANSWER: 1"}} + r := req + r.AltFirst = true + + _, err := reconcile.NewLLMAdjudicator(fc).Adjudicate(context.Background(), r) + + require.NoError(t, err) + assert.Contains(t, fc.users[0], "Reading 1: mölade\nReading 2: målade") +} + +func TestLLMAdjudicator_MapsTheAnswerBackToBaseOrAltWhicheverOrderWasShown(t *testing.T) { + tests := []struct { + name string + altFirst bool + reply string + want reconcile.Verdict + }{ + {"1 with base first is base", false, "ANSWER: 1", reconcile.VerdictBase}, + {"2 with base first is alt", false, "ANSWER: 2", reconcile.VerdictAlt}, + {"1 with alt first is alt", true, "ANSWER: 1", reconcile.VerdictAlt}, + {"2 with alt first is base", true, "ANSWER: 2", reconcile.VerdictBase}, + {"unsure is unsure", false, "ANSWER: UNSURE", reconcile.VerdictUnsure}, + {"case and trailing reasoning are tolerated", false, "answer: 2\nThe second reading fits the context.", reconcile.VerdictAlt}, + {"reasoning before the answer line is tolerated", true, "Hmm.\nANSWER: 2", reconcile.VerdictBase}, + {"no answer line is unsure, not a guess", false, "I think the first one.", reconcile.VerdictUnsure}, + {"an empty reply is unsure", false, "", reconcile.VerdictUnsure}, + {"an out of range number is unsure", false, "ANSWER: 3", reconcile.VerdictUnsure}, + {"a number that merely starts with 1 is not 1", false, "ANSWER: 12", reconcile.VerdictUnsure}, + {"a word that merely starts with 2 is not 2", false, "ANSWER: 2nd option", reconcile.VerdictUnsure}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := req + r.AltFirst = tt.altFirst + + got, err := reconcile.NewLLMAdjudicator(&fakeCompleter{replies: []string{tt.reply}}).Adjudicate(context.Background(), r) + + require.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} + +func TestLLMAdjudicator_AnEmptyReadingIsDescribedNotLeftBlank(t *testing.T) { + fc := &fakeCompleter{replies: []string{"ANSWER: 1"}} + r := req + r.Base = "" + + _, err := reconcile.NewLLMAdjudicator(fc).Adjudicate(context.Background(), r) + + require.NoError(t, err) + assert.Contains(t, fc.users[0], "Reading 1: (nothing: the words are simply absent)") +} + +func TestLLMAdjudicator_ACompleterErrorIsReturnedNotTurnedIntoAVerdict(t *testing.T) { + boom := errors.New("gateway down") + + _, err := reconcile.NewLLMAdjudicator(&fakeCompleter{err: boom}).Adjudicate(context.Background(), req) + + assert.ErrorIs(t, err, boom, "the caller must be able to tell 'the model is down' from 'the model is unsure'") +} + +func TestOrderChecked_AsksTwiceWithTheOrderSwappedAndKeepsOnlyAgreement(t *testing.T) { + tests := []struct { + name string + replies []string // for [base-first call, alt-first call] + want reconcile.Verdict + }{ + {"both orders pick the base", []string{"ANSWER: 1", "ANSWER: 2"}, reconcile.VerdictBase}, + {"both orders pick the alternative", []string{"ANSWER: 2", "ANSWER: 1"}, reconcile.VerdictAlt}, + {"it flips with the order, so it is position bias, not a judgement", []string{"ANSWER: 1", "ANSWER: 1"}, reconcile.VerdictUnsure}, + {"one order unsure is unsure", []string{"ANSWER: 1", "ANSWER: UNSURE"}, reconcile.VerdictUnsure}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + fc := &fakeCompleter{replies: tt.replies} + + got, err := reconcile.NewOrderChecked(reconcile.NewLLMAdjudicator(fc)).Adjudicate(context.Background(), req) + + require.NoError(t, err) + assert.Equal(t, tt.want, got) + require.Len(t, fc.users, 2) + assert.True(t, strings.Contains(fc.users[0], "Reading 1: målade"), "first call shows the base first") + assert.True(t, strings.Contains(fc.users[1], "Reading 1: mölade"), "second call shows the alternative first") + }) + } +} + +func TestOrderChecked_AnErrorOnEitherCallIsAnError(t *testing.T) { + _, err := reconcile.NewOrderChecked(reconcile.NewLLMAdjudicator(&fakeCompleter{err: errors.New("down")})).Adjudicate(context.Background(), req) + + assert.Error(t, err) +} diff --git a/internal/reconcile/diff.go b/internal/reconcile/diff.go new file mode 100644 index 0000000..b558f85 --- /dev/null +++ b/internal/reconcile/diff.go @@ -0,0 +1,187 @@ +package reconcile + +// OpKind is the kind of one run in an edit script. +type OpKind int + +const ( + // Equal: A[AStart:AEnd] and B[BStart:BEnd] are the same words. + Equal OpKind = iota + // Replace: different words on both sides. + Replace + // Delete: words only in A. + Delete + // Insert: words only in B. + Insert +) + +// Op is one run of the edit script. The ops of a Diff tile both inputs: each op starts +// where the previous one ended, in A and in B. +type Op struct { + Kind OpKind + AStart, AEnd int + BStart, BEnd int +} + +// maxEdits bounds the Myers search. Memory grows with the square of the edit +// distance, so two texts that share almost nothing (one engine produced garbage) +// are reported as one Replace instead of spending gigabytes proving it. A real +// hour disagrees in roughly 400 places, i.e. about 800 edits, far below this. +const maxEdits = 3000 + +// Diff returns a minimal edit script from a to b (Myers, O(ND)). Adjacent deletions +// and insertions are reported as one Replace. +func Diff(a, b []string) []Op { + pre := 0 + for pre < len(a) && pre < len(b) && a[pre] == b[pre] { + pre++ + } + suf := 0 + for suf < len(a)-pre && suf < len(b)-pre && a[len(a)-1-suf] == b[len(b)-1-suf] { + suf++ + } + midA, midB := a[pre:len(a)-suf], b[pre:len(b)-suf] + + steps, ok := myers(midA, midB) + if !ok { + steps = steps[:0] + for range midA { + steps = append(steps, stepDel) + } + for range midB { + steps = append(steps, stepIns) + } + } + + var ops []Op + ai, bi := 0, 0 + emit := func(k OpKind, da, db int) { + if da == 0 && db == 0 { + return + } + if k == Equal && len(ops) > 0 && ops[len(ops)-1].Kind == Equal { + ops[len(ops)-1].AEnd += da + ops[len(ops)-1].BEnd += db + } else { + ops = append(ops, Op{Kind: k, AStart: ai, AEnd: ai + da, BStart: bi, BEnd: bi + db}) + } + ai += da + bi += db + } + emit(Equal, pre, pre) + for i := 0; i < len(steps); { + if steps[i] == stepEq { + j := i + for j < len(steps) && steps[j] == stepEq { + j++ + } + emit(Equal, j-i, j-i) + i = j + continue + } + dels, inss := 0, 0 + for i < len(steps) && steps[i] != stepEq { + if steps[i] == stepDel { + dels++ + } else { + inss++ + } + i++ + } + switch { + case dels > 0 && inss > 0: + emit(Replace, dels, inss) + case dels > 0: + emit(Delete, dels, 0) + default: + emit(Insert, 0, inss) + } + } + emit(Equal, suf, suf) + return ops +} + +type step int8 + +const ( + stepEq step = iota + stepDel + stepIns +) + +// myers returns one step per element of the edit script, in order. ok is false when +// the edit distance exceeds maxEdits. +func myers(a, b []string) ([]step, bool) { + n, m := len(a), len(b) + if n == 0 && m == 0 { + return nil, true + } + limit := min(n+m, maxEdits) + off := limit + 1 + v := make([]int32, 2*limit+3) + var trace [][]int32 // trace[d] is v as it stood at the start of round d, for k in [-d, d] + + for d := 0; d <= limit; d++ { + snap := make([]int32, 2*d+1) + for k := -d; k <= d; k++ { + snap[k+d] = v[off+k] + } + trace = append(trace, snap) + for k := -d; k <= d; k += 2 { + var x int + if k == -d || (k != d && v[off+k-1] < v[off+k+1]) { + x = int(v[off+k+1]) + } else { + x = int(v[off+k-1]) + 1 + } + y := x - k + for x < n && y < m && a[x] == b[y] { + x++ + y++ + } + v[off+k] = int32(x) + if x >= n && y >= m { + return backtrack(trace, n, m, d), true + } + } + } + return nil, false +} + +func backtrack(trace [][]int32, n, m, d int) []step { + var rev []step + x, y := n, m + for ; d >= 0; d-- { + snap := trace[d] + at := func(k int) int { return int(snap[k+d]) } + k := x - y + var prevK int + if k == -d || (k != d && at(k-1) < at(k+1)) { + prevK = k + 1 + } else { + prevK = k - 1 + } + prevX := 0 + if d > 0 { + prevX = at(prevK) + } + prevY := prevX - prevK + for x > prevX && y > prevY { + rev = append(rev, stepEq) + x-- + y-- + } + if d > 0 { + if x == prevX { + rev = append(rev, stepIns) + } else { + rev = append(rev, stepDel) + } + } + x, y = prevX, prevY + } + out := make([]step, len(rev)) + for i, s := range rev { + out[len(rev)-1-i] = s + } + return out +} diff --git a/internal/reconcile/diff_test.go b/internal/reconcile/diff_test.go new file mode 100644 index 0000000..c75204e --- /dev/null +++ b/internal/reconcile/diff_test.go @@ -0,0 +1,139 @@ +package reconcile_test + +import ( + "math/rand" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +func words(s string) []string { return strings.Fields(s) } + +// apply rebuilds b from a using the ops, copying equal runs from a and the +// replacement/insertion runs from b. If Diff is right this must equal b. +func apply(a, b []string, ops []reconcile.Op) []string { + var out []string + for _, o := range ops { + switch o.Kind { + case reconcile.Equal: + out = append(out, a[o.AStart:o.AEnd]...) + default: + out = append(out, b[o.BStart:o.BEnd]...) + } + } + return out +} + +func TestDiff_Cases(t *testing.T) { + tests := []struct { + name string + a, b string + want []reconcile.Op + }{ + {"identical", "a b c", "a b c", []reconcile.Op{{Kind: reconcile.Equal, AStart: 0, AEnd: 3, BStart: 0, BEnd: 3}}}, + {"both empty", "", "", nil}, + {"one word replaced", "a b c", "a x c", []reconcile.Op{ + {Kind: reconcile.Equal, AStart: 0, AEnd: 1, BStart: 0, BEnd: 1}, + {Kind: reconcile.Replace, AStart: 1, AEnd: 2, BStart: 1, BEnd: 2}, + {Kind: reconcile.Equal, AStart: 2, AEnd: 3, BStart: 2, BEnd: 3}, + }}, + {"word only in b", "a c", "a b c", []reconcile.Op{ + {Kind: reconcile.Equal, AStart: 0, AEnd: 1, BStart: 0, BEnd: 1}, + {Kind: reconcile.Insert, AStart: 1, AEnd: 1, BStart: 1, BEnd: 2}, + {Kind: reconcile.Equal, AStart: 1, AEnd: 2, BStart: 2, BEnd: 3}, + }}, + {"word only in a", "a b c", "a c", []reconcile.Op{ + {Kind: reconcile.Equal, AStart: 0, AEnd: 1, BStart: 0, BEnd: 1}, + {Kind: reconcile.Delete, AStart: 1, AEnd: 2, BStart: 1, BEnd: 1}, + {Kind: reconcile.Equal, AStart: 2, AEnd: 3, BStart: 1, BEnd: 2}, + }}, + {"nothing in common", "a b", "x y z", []reconcile.Op{ + {Kind: reconcile.Replace, AStart: 0, AEnd: 2, BStart: 0, BEnd: 3}, + }}, + {"a is empty", "", "x y", []reconcile.Op{{Kind: reconcile.Insert, AStart: 0, AEnd: 0, BStart: 0, BEnd: 2}}}, + {"b is empty", "x y", "", []reconcile.Op{{Kind: reconcile.Delete, AStart: 0, AEnd: 2, BStart: 0, BEnd: 0}}}, + {"adjacent delete and insert read as one replace", "a b c d", "a x y d", []reconcile.Op{ + {Kind: reconcile.Equal, AStart: 0, AEnd: 1, BStart: 0, BEnd: 1}, + {Kind: reconcile.Replace, AStart: 1, AEnd: 3, BStart: 1, BEnd: 3}, + {Kind: reconcile.Equal, AStart: 3, AEnd: 4, BStart: 3, BEnd: 4}, + }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + assert.Equal(t, tt.want, reconcile.Diff(words(tt.a), words(tt.b))) + }) + } +} + +// lcsLen is a plain dynamic-programming reference. Myers must find an edit script +// whose equal runs add up to exactly this: a diff that is merely valid but not +// minimal would turn real agreement into disputes the adjudicator is then asked about. +func lcsLen(a, b []string) int { + prev := make([]int, len(b)+1) + for i := 1; i <= len(a); i++ { + cur := make([]int, len(b)+1) + for j := 1; j <= len(b); j++ { + if a[i-1] == b[j-1] { + cur[j] = prev[j-1] + 1 + } else { + cur[j] = max(prev[j], cur[j-1]) + } + } + prev = cur + } + return prev[len(b)] +} + +func TestDiff_RebuildsBAndKeepsEveryCommonWordOnRandomInputs(t *testing.T) { + rng := rand.New(rand.NewSource(7)) + alphabet := []string{"a", "b", "c", "d", "e", "f"} + gen := func(n int) []string { + out := make([]string, n) + for i := range out { + out[i] = alphabet[rng.Intn(len(alphabet))] + } + return out + } + for i := 0; i < 400; i++ { + a, b := gen(rng.Intn(25)), gen(rng.Intn(25)) + + ops := reconcile.Diff(a, b) + + require.Equal(t, append([]string{}, b...), append([]string{}, apply(a, b, ops)...), "a=%v b=%v ops=%v", a, b, ops) + common := 0 + prevA, prevB := 0, 0 + for _, o := range ops { + require.Equal(t, prevA, o.AStart, "ops must tile a without gaps: %v", ops) + require.Equal(t, prevB, o.BStart, "ops must tile b without gaps: %v", ops) + prevA, prevB = o.AEnd, o.BEnd + if o.Kind == reconcile.Equal { + common += o.AEnd - o.AStart + require.Equal(t, a[o.AStart:o.AEnd], b[o.BStart:o.BEnd], "an Equal op must really be equal") + } + } + require.Equal(t, len(a), prevA) + require.Equal(t, len(b), prevB) + require.Equal(t, lcsLen(a, b), common, "the diff must be minimal: a=%v b=%v", a, b) + } +} + +func TestDiff_HandlesAnHoursWorthOfWordsFast(t *testing.T) { + rng := rand.New(rand.NewSource(3)) + a := make([]string, 6500) + for i := range a { + a[i] = string(rune('a'+rng.Intn(26))) + string(rune('a'+rng.Intn(26))) + } + b := append([]string{}, a...) + for i := 0; i < 380; i++ { // ~380 disputes, as measured on a real hour + b[rng.Intn(len(b))] = "zz" + string(rune('a'+rng.Intn(26))) + } + + ops := reconcile.Diff(a, b) + + assert.NotEmpty(t, ops) + assert.Equal(t, append([]string{}, b...), append([]string{}, apply(a, b, ops)...)) +} diff --git a/internal/reconcile/disputes.go b/internal/reconcile/disputes.go new file mode 100644 index 0000000..27b0be5 --- /dev/null +++ b/internal/reconcile/disputes.go @@ -0,0 +1,100 @@ +package reconcile + +import "strings" + +// Dispute is one place where the two transcripts disagree. BaseStart:BaseEnd and +// AltStart:AltEnd are token ranges into the two Tokenize results; an empty range +// means that engine heard nothing there. Base and Alt are the two readings as each +// engine wrote them, so a replaced span takes the alternative's own spelling. +type Dispute struct { + BaseStart, BaseEnd int + AltStart, AltEnd int + Base, Alt string +} + +// FindDisputes aligns base against alt word by word and returns the disagreements. +// Differences separated by a single shared word are merged into one dispute: for a +// reader that is one decision, and asking about "b" and "d" separately in "b c d" +// hides that "c" sits between them. Two shared words keep disputes apart. +func FindDisputes(base, alt []Token) []Dispute { + a, b := keysOf(base), keysOf(alt) + ops := Diff(a, b) + var out []Dispute + for i := 0; i < len(ops); i++ { + if ops[i].Kind == Equal { + continue + } + cur := ops[i] + for i+2 < len(ops) && ops[i+1].Kind == Equal && ops[i+1].AEnd-ops[i+1].AStart == 1 && ops[i+2].Kind != Equal { + cur.AEnd, cur.BEnd = ops[i+2].AEnd, ops[i+2].BEnd + i += 2 + } + out = append(out, Dispute{ + BaseStart: cur.AStart, BaseEnd: cur.AEnd, + AltStart: cur.BStart, AltEnd: cur.BEnd, + Base: join(base[cur.AStart:cur.AEnd]), + Alt: join(alt[cur.BStart:cur.BEnd]), + }) + } + return out +} + +// fillers are Swedish discourse particles. A dispute that is only one or two of them +// on one side changes no meaning, so it is resolved without asking a model. The list +// is deliberately short and leaves out words that can be an answer ("nej") or that +// carry content. +var fillers = map[string]bool{ + "ju": true, "då": true, "liksom": true, "alltså": true, "så": true, "ja": true, + "eh": true, "ehm": true, "öh": true, "mm": true, "hm": true, "typ": true, "va": true, "asså": true, +} + +// maxFillerRun is how many filler words may be waved through together. +const maxFillerRun = 2 + +// AutoResolve reports whether a dispute needs no model, in which case the base +// reading is kept: either the two sides are the same letters once spaces are removed +// (a compound written as one word or two), or one side is empty and the other is a +// short run of filler words. +func AutoResolve(d Dispute) bool { + baseKeys, altKeys := keysOfText(d.Base), keysOfText(d.Alt) + if strings.Join(baseKeys, "") == strings.Join(altKeys, "") { + return true + } + return (len(baseKeys) == 0 && onlyFillers(altKeys)) || (len(altKeys) == 0 && onlyFillers(baseKeys)) +} + +func onlyFillers(keys []string) bool { + if len(keys) == 0 || len(keys) > maxFillerRun { + return false + } + for _, k := range keys { + if !fillers[k] { + return false + } + } + return true +} + +// ContextAround returns up to window words of the base before and after the dispute, +// as the base wrote them. +func ContextAround(base []Token, d Dispute, window int) (before, after string) { + return join(base[max(0, d.BaseStart-window):d.BaseStart]), join(base[d.BaseEnd:min(len(base), d.BaseEnd+window)]) +} + +func keysOf(ts []Token) []string { + out := make([]string, len(ts)) + for i, t := range ts { + out[i] = t.Key + } + return out +} + +func keysOfText(s string) []string { return keysOf(Tokenize(s)) } + +func join(ts []Token) string { + parts := make([]string, len(ts)) + for i, t := range ts { + parts[i] = t.Text + } + return strings.Join(parts, " ") +} diff --git a/internal/reconcile/disputes_test.go b/internal/reconcile/disputes_test.go new file mode 100644 index 0000000..1c87ed4 --- /dev/null +++ b/internal/reconcile/disputes_test.go @@ -0,0 +1,131 @@ +package reconcile_test + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +func disputes(base, alt string) []reconcile.Dispute { + return reconcile.FindDisputes(reconcile.Tokenize(base), reconcile.Tokenize(alt)) +} + +func TestFindDisputes_NoneWhenTheTextsAgreeWordForWord(t *testing.T) { + assert.Empty(t, disputes("Vi går igenom läget.", "vi går igenom läget"), + "casing and punctuation are not disagreements") +} + +func TestFindDisputes_OneReplacedWordIsOneDisputeCarryingBothReadings(t *testing.T) { + got := disputes("Vi ska målade grannarna.", "Vi ska mölade grannarna.") + + require.Len(t, got, 1) + d := got[0] + assert.Equal(t, "målade", d.Base) + assert.Equal(t, "mölade", d.Alt) + assert.Equal(t, 2, d.BaseStart) + assert.Equal(t, 3, d.BaseEnd) + assert.Equal(t, 2, d.AltStart) + assert.Equal(t, 3, d.AltEnd) +} + +func TestFindDisputes_ReadingsKeepTheirOwnSpellingAndPunctuation(t *testing.T) { + got := disputes("Det blir A+2, alltså.", "Det blir A 2, alltså.") + + require.Len(t, got, 1) + assert.Equal(t, "A+2,", got[0].Base) + assert.Equal(t, "A 2,", got[0].Alt, "the alternative's words replace the span as the engine wrote them") +} + +func TestFindDisputes_AWordOnlyOneEngineHeardHasAnEmptyOtherSide(t *testing.T) { + got := disputes("Man får tillbaka ett förslag.", "Man får tillbaka ett coolt förslag.") + + require.Len(t, got, 1) + assert.Equal(t, "", got[0].Base) + assert.Equal(t, "coolt", got[0].Alt) + assert.Equal(t, got[0].BaseStart, got[0].BaseEnd, "an insertion has an empty base range at its position") + assert.Equal(t, 4, got[0].BaseStart) +} + +func TestFindDisputes_TwoDifferencesSeparatedByOneSharedWordAreOneDispute(t *testing.T) { + // "x ... y" around a single agreed word is one decision for the reader, not two. + got := disputes("a b c d e", "a x c y e") + + require.Len(t, got, 1) + assert.Equal(t, "b c d", got[0].Base) + assert.Equal(t, "x c y", got[0].Alt) +} + +func TestFindDisputes_TwoSharedWordsKeepTheDisputesApart(t *testing.T) { + got := disputes("a b c d e f", "a x c d y f") + + require.Len(t, got, 2) + assert.Equal(t, "b", got[0].Base) + assert.Equal(t, "e", got[1].Base) +} + +func TestFindDisputes_ChainsOfCloseDifferencesMergeIntoOne(t *testing.T) { + got := disputes("a b c d e f g", "a x c y e z g") + + require.Len(t, got, 1) + assert.Equal(t, "b c d e f", got[0].Base) +} + +func TestFindDisputes_LeavesIndicesThatAddressBothTokenSlices(t *testing.T) { + base := reconcile.Tokenize("Hej Anders och Maria") + alt := reconcile.Tokenize("Hej Sanders och Maria") + + got := reconcile.FindDisputes(base, alt) + + require.Len(t, got, 1) + assert.Equal(t, "Anders", base[got[0].BaseStart].Text) + assert.Equal(t, "Sanders", alt[got[0].AltStart].Text) +} + +func TestAutoResolve(t *testing.T) { + tests := []struct { + name string + base, alt string + resolved bool + }{ + {"a compound written as two words or one", "i takrenns delen", "i takrennsdelen", true}, + {"a compound with a hyphen", "takrenns-delen", "takrenns delen", true}, + {"a filler only the alternative heard", "vi ska ju betala", "vi ska betala", true}, + {"a filler only the base heard", "vi ska betala", "vi ska ju betala", true}, + {"two fillers in a row", "vi ska ja ju betala", "vi ska betala", true}, + {"a different word is for the model", "vi ska målade", "vi ska mölade", false}, + {"a different number is for the model", "totalt 1050 kronor", "totalt 2050 kronor", false}, + {"a content word only one side heard is for the model", "ett förslag", "ett coolt förslag", false}, + {"three filler words are too many to wave through", "vi ska ju då liksom betala", "vi ska betala", false}, + {"a filler swapped for a content word is for the model", "vi ska ju betala", "vi ska nu betala", false}, + {"nej is an answer, not a filler", "men nej det går inte", "men det går inte", false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := disputes(tt.base, tt.alt) + require.Len(t, got, 1, "the fixture must be exactly one dispute") + + assert.Equal(t, tt.resolved, reconcile.AutoResolve(got[0])) + }) + } +} + +func TestContextAround_TakesWordsFromTheBaseOnBothSidesAndStopsAtTheEdges(t *testing.T) { + base := reconcile.Tokenize("ett två tre fyra fem sex sju åtta nio tio") + d := reconcile.Dispute{BaseStart: 4, BaseEnd: 6} // "fem sex" + + before, after := reconcile.ContextAround(base, d, 3) + + assert.Equal(t, "två tre fyra", before) + assert.Equal(t, "sju åtta nio", after) + + b2, a2 := reconcile.ContextAround(base, reconcile.Dispute{BaseStart: 0, BaseEnd: 1}, 3) + assert.Equal(t, "", b2) + assert.Equal(t, "två tre fyra", a2) + + b3, a3 := reconcile.ContextAround(base, reconcile.Dispute{BaseStart: 9, BaseEnd: 10}, 3) + assert.Equal(t, "sju åtta nio", b3) + assert.Equal(t, "", a3) +} diff --git a/internal/reconcile/reconcile.go b/internal/reconcile/reconcile.go new file mode 100644 index 0000000..35048e1 --- /dev/null +++ b/internal/reconcile/reconcile.go @@ -0,0 +1,241 @@ +package reconcile + +import ( + "context" + "fmt" + "strings" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" + "git.d-ma.be/mathias/tapir/internal/progress" +) + +// MaxContested is the default bound on the stored list of judged spots. A 90-minute +// recording has a few hundred, so this is a guard against a runaway, not a budget. The +// counts on the Reconciliation are always exact; only the list is capped. +const MaxContested = 2000 + +// beforeWords is how many words of left context a stored spot carries: enough to find +// the place again, small enough to stay out of the way. +const beforeWords = 8 + +// Options configure Reconcile. Zero Window and BreakAfter take their defaults. +type Options struct { + // Window is how many words of the base to show on each side of a dispute. 100 is + // what was measured (docs/research/local-model-adjudicator-sweep-2026-10-01.md). + Window int + // BreakAfter is how many consecutive model errors stop the reconciliation, so a + // model that is down costs three timeouts, not one per dispute. + BreakAfter int + // MaxContested caps the stored judged spots; zero takes the package default. + MaxContested int + // BaseEngine, AltEngine and Reconciler are recorded on the result. + BaseEngine, AltEngine, Reconciler string +} + +// Reconcile merges two transcripts of the same speech. The base text stands except +// where the model chooses the alternative's reading of a dispute; trivial disputes +// (a compound split, a filler word) are resolved without the model. It never fails a +// transcription: if the model cannot be used the base text comes back unreconciled +// and the result says why. The only error it returns is a cancelled context, which +// is the job stopping, not a verdict on the model. +func Reconcile(ctx context.Context, base, alt domain.Transcription, adj Adjudicator, o Options) (domain.Transcription, error) { + window, breakAfter, maxSpots := o.Window, o.BreakAfter, o.MaxContested + if window <= 0 { + window = 100 + } + if breakAfter <= 0 { + breakAfter = 3 + } + if maxSpots <= 0 { + maxSpots = MaxContested + } + rep := progress.From(ctx) + + rep.Begin("compare", 0) + bt, at := Tokenize(base.Text), Tokenize(alt.Text) + ds := FindDisputes(bt, at) + rep.Done("compare") + + // The deciding phase counts what the MODEL decides: the trivial disputes are settled + // without it, so counting them would leave the bar short of its end. + deciding := 0 + for _, d := range ds { + if !AutoResolve(d) { + deciding++ + } + } + rec := &domain.Reconciliation{ + Status: domain.ReconcileDone, Base: o.BaseEngine, Alt: o.AltEngine, Reconciler: o.Reconciler, Disputes: len(ds), + } + seconds := positioner(base, len(bt)) + spot := func(d Dispute, used string, unsure bool) { + before, _ := ContextAround(bt, d, window) + if w := strings.Fields(before); len(w) > beforeWords { + before = strings.Join(w[len(w)-beforeWords:], " ") + } + if len(rec.Contested) < maxSpots { + rec.Contested = append(rec.Contested, domain.ContestedSpot{ + Seconds: seconds(d.BaseStart), Base: d.Base, Alt: d.Alt, Used: used, Unsure: unsure, Before: before, + }) + } + } + + rep.Begin("decide", deciding) + verdicts := make([]Verdict, len(ds)) + consecutive, handled := 0, 0 + for i, d := range ds { + if AutoResolve(d) { + rec.AutoResolved++ + verdicts[i] = VerdictBase + continue + } + if err := ctx.Err(); err != nil { + return domain.Transcription{}, err + } + before, after := ContextAround(bt, d, window) + v, err := adj.Adjudicate(ctx, Request{Before: before, After: after, Base: d.Base, Alt: d.Alt, AltFirst: i%2 == 1}) + if err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return domain.Transcription{}, ctxErr + } + rec.Errors++ + consecutive++ + if consecutive >= breakAfter { + rep.Fail("decide", err) + return skipped(base, rec, fmt.Sprintf("the model failed %d times in a row (%s)", consecutive, errkind.Of(err))), nil + } + v = VerdictUnsure + } else { + consecutive = 0 + rec.Adjudicated++ + } + verdicts[i] = v + switch v { + case VerdictAlt: + rec.Changed++ + spot(d, "alt", false) + case VerdictUnsure: + rec.Unsure++ + spot(d, "base", true) + default: + spot(d, "base", false) + } + handled++ + rep.Advance("decide", handled) + } + rep.Done("decide") + + out := base + out.Text, out.Segments = merge(base, bt, at, ds, verdicts) + out.Reconciliation = rec + return out, nil +} + +// merge applies the verdicts: the base's words stand except where the alternative's +// reading was chosen. It returns the merged text and the base's segments carrying the +// merged words (timing is the base's). Segments are rewritten only when their words line +// up with the text; a word wrapped across a segment edge makes them differ, and guessing +// would put words in the wrong place, so then the base's segments are returned as they +// were (ADR-035). The base's own segments are never edited in place. +func merge(base domain.Transcription, bt, at []Token, ds []Dispute, verdicts []Verdict) (string, []domain.Segment) { + segOf, aligned := segmentIndex(base.Segments, len(bt)) + var perSeg [][]string + var changed map[int]bool + if aligned { + perSeg, changed = make([][]string, len(base.Segments)), map[int]bool{} + } + + var parts []string + keep := func(from, to int) { + for i := from; i < to; i++ { + parts = append(parts, bt[i].Text) + if aligned { + perSeg[segOf[i]] = append(perSeg[segOf[i]], bt[i].Text) + } + } + } + pos := 0 + for i, d := range ds { + keep(pos, d.BaseStart) + if verdicts[i] == VerdictAlt { + for _, t := range at[d.AltStart:d.AltEnd] { + parts = append(parts, t.Text) + } + if aligned { + anchor := min(d.BaseStart, len(bt)-1) + for _, t := range at[d.AltStart:d.AltEnd] { + perSeg[segOf[anchor]] = append(perSeg[segOf[anchor]], t.Text) + } + changed[segOf[anchor]] = true + for k := d.BaseStart; k < d.BaseEnd; k++ { + changed[segOf[k]] = true + } + } + } else { + keep(d.BaseStart, d.BaseEnd) + } + pos = d.BaseEnd + } + keep(pos, len(bt)) + + if !aligned { + return strings.Join(parts, " "), base.Segments + } + segs := append([]domain.Segment(nil), base.Segments...) + for k := range changed { + segs[k].Text = strings.Join(perSeg[k], " ") + } + return strings.Join(parts, " "), segs +} + +// segmentIndex maps each base token to the segment it came from. aligned is false when +// there are no segments or their word count is not the text's. +func segmentIndex(segs []domain.Segment, tokens int) (segOf []int, aligned bool) { + if len(segs) == 0 || tokens == 0 { + return nil, false + } + segOf = make([]int, 0, tokens) + for k, s := range segs { + for range Tokenize(s.Text) { + segOf = append(segOf, k) + } + } + return segOf, len(segOf) == tokens +} + +// skipped is the base text, none of the partial changes, and the reason. +func skipped(base domain.Transcription, rec *domain.Reconciliation, reason string) domain.Transcription { + out := base + out.Reconciliation = &domain.Reconciliation{ + Status: domain.ReconcileSkipped, Reason: reason, Base: rec.Base, Alt: rec.Alt, Reconciler: rec.Reconciler, + Disputes: rec.Disputes, AutoResolved: rec.AutoResolved, Errors: rec.Errors, + } + return out +} + +// positioner maps a token index of the base to the second it starts at: through the +// segments when there are any, proportionally otherwise. +func positioner(base domain.Transcription, tokens int) func(int) float64 { + if len(base.Segments) == 0 { + return func(i int) float64 { + if tokens == 0 { + return 0 + } + return float64(i) / float64(tokens) * base.AudioSeconds + } + } + starts := make([]int, len(base.Segments)) + n := 0 + for i, s := range base.Segments { + starts[i] = n + n += len(Tokenize(s.Text)) + } + return func(i int) float64 { + k := 0 + for k+1 < len(starts) && starts[k+1] <= i { + k++ + } + return base.Segments[k].Start + } +} diff --git a/internal/reconcile/reconcile_progress_test.go b/internal/reconcile/reconcile_progress_test.go new file mode 100644 index 0000000..aa13dab --- /dev/null +++ b/internal/reconcile/reconcile_progress_test.go @@ -0,0 +1,116 @@ +package reconcile_test + +import ( + "context" + "errors" + "fmt" + "strings" + "sync" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/progress" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// What Reconcile reports while it works (ADR-035): the comparison, then the deciding +// phase with the real number of model decisions as its total. + +type repRec struct { + mu sync.Mutex + events []string +} + +func (r *repRec) add(e string) { + r.mu.Lock() + defer r.mu.Unlock() + r.events = append(r.events, e) +} + +func (r *repRec) Begin(key string, total int) { r.add(fmt.Sprintf("begin:%s:%d", key, total)) } +func (r *repRec) Advance(key string, done int) { r.add(fmt.Sprintf("advance:%s:%d", key, done)) } +func (r *repRec) Done(key string) { r.add("done:" + key) } +func (r *repRec) Fail(key string, _ error) { r.add("fail:" + key) } +func (r *repRec) Note(string, ...any) {} +func (r *repRec) joined() string { + r.mu.Lock() + defer r.mu.Unlock() + return strings.Join(r.events, ",") +} +func withRep(r *repRec) context.Context { return progress.With(context.Background(), r) } +func reconcileWith(t *testing.T, ctx context.Context, base, alt string, adj reconcile.Adjudicator) error { + t.Helper() + _, err := reconcile.Reconcile(ctx, tr(base), tr(alt), adj, opts) + return err +} + +// two compound/filler disputes the model is never asked about, then two it is. +const ( + progBase = "i takrenns delen ska vi ju betala totalt 1015 kronor till Axel idag" + progAlt = "i takrennsdelen ska vi betala totalt 1050 kronor till Alex idag" +) + +func TestReconcileProgress_TheDecidingPhaseTotalIsTheModelDecisionsNotTheDisputes(t *testing.T) { + rep := &repRec{} + + require.NoError(t, reconcileWith(t, withRep(rep), progBase, progAlt, always(reconcile.VerdictAlt))) + + assert.Equal(t, + "begin:compare:0,done:compare,begin:decide:2,advance:decide:1,advance:decide:2,done:decide", + rep.joined(), + "4 disputes, 2 settled without the model: the bar counts the 2 the model decides, so it can reach its end") +} + +func TestReconcileProgress_TextsThatAgreeStillClosePhasesSoNothingIsLeftRunning(t *testing.T) { + rep := &repRec{} + + require.NoError(t, reconcileWith(t, withRep(rep), "samma text hela vägen", "samma text hela vägen", always(reconcile.VerdictAlt))) + + assert.Equal(t, "begin:compare:0,done:compare,begin:decide:0,done:decide", rep.joined()) +} + +func TestReconcileProgress_AModelErrorStillCountsAsADecisionTaken(t *testing.T) { + rep := &repRec{} + adj := &scripted{fn: func(i int, _ reconcile.Request) (reconcile.Verdict, error) { + if i == 0 { + return reconcile.VerdictBase, errors.New("one timeout") + } + return reconcile.VerdictAlt, nil + }} + + require.NoError(t, reconcileWith(t, withRep(rep), progBase, progAlt, adj)) + + assert.Contains(t, rep.joined(), "advance:decide:1,advance:decide:2", "a spot that errored was still handled; the bar must move past it") +} + +func TestReconcileProgress_TheBreakerTrippingShowsTheDecidingPhaseFailedNotDone(t *testing.T) { + rep := &repRec{} + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { + return reconcile.VerdictBase, errors.New("model down") + }} + small := opts + small.BreakAfter = 2 + + _, err := reconcile.Reconcile(withRep(rep), tr(progBase), tr(progAlt), adj, small) + + require.NoError(t, err, "a model that is down degrades the result; it does not fail the job") + assert.Contains(t, rep.joined(), "fail:decide") + assert.NotContains(t, rep.joined(), "done:decide") +} + +func TestReconcileProgress_AShutdownIsNeitherDoneNorFailed(t *testing.T) { + rep := &repRec{} + ctx, cancel := context.WithCancel(withRep(rep)) + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { + cancel() + return reconcile.VerdictAlt, nil + }} + + err := reconcileWith(t, ctx, progBase, progAlt, adj) + + require.ErrorIs(t, err, context.Canceled) + assert.NotContains(t, rep.joined(), "fail:decide") + assert.NotContains(t, rep.joined(), "done:decide") +} diff --git a/internal/reconcile/reconcile_reason_test.go b/internal/reconcile/reconcile_reason_test.go new file mode 100644 index 0000000..d8ab642 --- /dev/null +++ b/internal/reconcile/reconcile_reason_test.go @@ -0,0 +1,33 @@ +package reconcile_test + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// The reason a reconciliation was skipped is stored and shown on a page about a private +// recording, so it carries a kind, never the model's error text, which can echo the +// prompt (adversarial review F3). + +func TestReconcile_TheBreakerReasonCarriesAKindNotTheModelsText(t *testing.T) { + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { + return reconcile.VerdictBase, errors.New("gateway echoed the prompt: zebrakaka secret words") + }} + small := opts + small.BreakAfter = 2 + + out, err := reconcile.Reconcile(context.Background(), tr(progBase), tr(progAlt), adj, small) + + require.NoError(t, err) + require.Equal(t, domain.ReconcileSkipped, out.Reconciliation.Status) + assert.NotContains(t, out.Reconciliation.Reason, "zebrakaka", "provider text must not be stored or shown") + assert.Contains(t, out.Reconciliation.Reason, "2 times in a row", "the reason still says what happened") + assert.Contains(t, out.Reconciliation.Reason, "error", "and carries a kind") +} diff --git a/internal/reconcile/reconcile_spots_test.go b/internal/reconcile/reconcile_spots_test.go new file mode 100644 index 0000000..4dcab61 --- /dev/null +++ b/internal/reconcile/reconcile_spots_test.go @@ -0,0 +1,130 @@ +package reconcile_test + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// Every judged spot is stored with a few words of left context, and the saved segments +// carry the merged words while keeping the base's timing (ADR-035 decisions 6 and 7). + +func TestReconcileSpots_AKeptSpotAndAChangedSpotAreBothStoredWithTheirVerdict(t *testing.T) { + adj := &scripted{fn: func(i int, _ reconcile.Request) (reconcile.Verdict, error) { + if i == 0 { + return reconcile.VerdictBase, nil + } + return reconcile.VerdictAlt, nil + }} + + out := merge(t, tr(progBase), tr(progAlt), adj) + + require.Len(t, out.Reconciliation.Contested, 2) + assert.Equal(t, "base", out.Reconciliation.Contested[0].Used) + assert.Equal(t, "1015", out.Reconciliation.Contested[0].Base) + assert.Equal(t, "alt", out.Reconciliation.Contested[1].Used) + assert.Equal(t, 1, out.Reconciliation.Changed, "the exact counts are unchanged by storing more") +} + +func TestReconcileSpots_AutoResolvedDisputesAreNotStored(t *testing.T) { + out := merge(t, tr("i takrenns delen ska vi ju betala"), tr("i takrennsdelen ska vi betala"), always(reconcile.VerdictAlt)) + + assert.Empty(t, out.Reconciliation.Contested, "compounds and fillers were never judged; they are counted, not listed") +} + +func TestReconcileSpots_ASpotCarriesTheWordsJustBeforeIt(t *testing.T) { + out := merge(t, tr("grannen målar ett vanligt staket mot gatan"), tr("grannen målar ett KORT staket mot gatan"), always(reconcile.VerdictAlt)) + + require.Len(t, out.Reconciliation.Contested, 1) + assert.Equal(t, "grannen målar ett", out.Reconciliation.Contested[0].Before) +} + +func TestReconcileSpots_TheContextIsAFewWordsNotTheWholeMeeting(t *testing.T) { + long := "ett två tre fyra fem sex sju åtta nio tio elva tolv" + out := merge(t, tr(long+" X slut"), tr(long+" Y slut"), always(reconcile.VerdictAlt)) + + require.Len(t, out.Reconciliation.Contested, 1) + assert.Equal(t, "fem sex sju åtta nio tio elva tolv", out.Reconciliation.Contested[0].Before, + "eight words: enough to find the place, small enough to stay out of the way") +} + +func segmented(text string, segs ...domain.Segment) domain.Transcription { + t := tr(text) + t.Segments = segs + return t +} + +func TestReconcileSegments_AChangedWordReplacesTheBaseWordInItsOwnSegmentOnly(t *testing.T) { + base := segmented("ett två tre. fyra fem sex. sju åtta nio.", + domain.Segment{Start: 0, End: 10, Text: "ett två tre."}, + domain.Segment{Start: 10, End: 20, Text: "fyra fem sex."}, + domain.Segment{Start: 20, End: 30, Text: "sju åtta nio."}) + + out := merge(t, base, tr("ett två tre. fyra femm sex. sju åtta nio."), always(reconcile.VerdictAlt)) + + require.Len(t, out.Segments, 3) + assert.Equal(t, "ett två tre.", out.Segments[0].Text, "an untouched segment keeps its exact text") + assert.Equal(t, "fyra femm sex.", out.Segments[1].Text) + assert.Equal(t, "sju åtta nio.", out.Segments[2].Text) + assert.InDelta(t, 10.0, out.Segments[1].Start, 1e-9, "timing stays the base's") + assert.InDelta(t, 20.0, out.Segments[1].End, 1e-9) +} + +func TestReconcileSegments_AWordOnlyTheAlternativeHeardLandsInTheSegmentItBelongsTo(t *testing.T) { + base := segmented("a b c. d e f.", domain.Segment{Start: 0, End: 5, Text: "a b c."}, domain.Segment{Start: 5, End: 10, Text: "d e f."}) + + out := merge(t, base, tr("a b c. d NY e f."), always(reconcile.VerdictAlt)) + + assert.Equal(t, "a b c.", out.Segments[0].Text) + assert.Equal(t, "d NY e f.", out.Segments[1].Text) +} + +func TestReconcileSegments_AWordOnlyTheBaseHeardIsRemovedFromItsSegment(t *testing.T) { + base := segmented("a b c. d X e f.", domain.Segment{Start: 0, End: 5, Text: "a b c."}, domain.Segment{Start: 5, End: 10, Text: "d X e f."}) + + out := merge(t, base, tr("a b c. d e f."), always(reconcile.VerdictAlt)) + + assert.Equal(t, "d e f.", out.Segments[1].Text) +} + +func TestReconcileSegments_AKeptBaseLeavesEverySegmentAsItWas(t *testing.T) { + base := segmented("a x b", domain.Segment{Start: 0, End: 4, Text: "a x b"}) + + out := merge(t, base, tr("a y b"), always(reconcile.VerdictBase)) + + assert.Equal(t, base.Segments, out.Segments) +} + +func TestReconcileSegments_TheCallersSegmentsAreNeverChangedInPlace(t *testing.T) { + segs := []domain.Segment{{Start: 0, End: 4, Text: "a x b"}} + base := segmented("a x b", segs...) + + _ = merge(t, base, tr("a y b"), always(reconcile.VerdictAlt)) + + assert.Equal(t, "a x b", segs[0].Text, "Reconcile builds new segments; it does not edit the base transcription's") +} + +func TestReconcileSegments_SegmentsThatDoNotLineUpWithTheTextAreLeftAsTheBases(t *testing.T) { + // A word wrapped across a segment edge makes the segments' word count differ from + // the joined text's. Guessing would put words in the wrong place, so leave them. + base := segmented("a x b c", domain.Segment{Start: 0, End: 4, Text: "a x"}, domain.Segment{Start: 4, End: 8, Text: "b c d"}) + + out := merge(t, base, tr("a y b c"), always(reconcile.VerdictAlt)) + + assert.Equal(t, base.Segments, out.Segments) + assert.Equal(t, "a y b c", out.Text, "the merged text is unaffected") +} + +func TestReconcileSegments_ASpanThatCrossesASegmentEdgeIsRemovedFromBothSegments(t *testing.T) { + base := segmented("a b c d", domain.Segment{Start: 0, End: 5, Text: "a b"}, domain.Segment{Start: 5, End: 10, Text: "c d"}) + + out := merge(t, base, tr("a X d"), always(reconcile.VerdictAlt)) + + require.Equal(t, "a X d", out.Text) + assert.Equal(t, "a X", out.Segments[0].Text) + assert.Equal(t, "d", out.Segments[1].Text, "'c' was replaced too: it must not linger in the second segment") +} diff --git a/internal/reconcile/reconcile_test.go b/internal/reconcile/reconcile_test.go new file mode 100644 index 0000000..0f26740 --- /dev/null +++ b/internal/reconcile/reconcile_test.go @@ -0,0 +1,308 @@ +package reconcile_test + +import ( + "context" + "errors" + "fmt" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +// scripted is an Adjudicator that answers from a function and records every request. +type scripted struct { + fn func(i int, r reconcile.Request) (reconcile.Verdict, error) + calls []reconcile.Request +} + +func (s *scripted) Adjudicate(_ context.Context, r reconcile.Request) (reconcile.Verdict, error) { + i := len(s.calls) + s.calls = append(s.calls, r) + return s.fn(i, r) +} + +func always(v reconcile.Verdict) *scripted { + return &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { return v, nil }} +} + +func tr(text string) domain.Transcription { + return domain.Transcription{Text: text, AudioSeconds: 100, Endpoint: "e"} +} + +var opts = reconcile.Options{Window: 100, BreakAfter: 3, Reconciler: "iguana/gemma4-26b", BaseEngine: "iguana/pianissimo", AltEngine: "iguana/kb-whisper"} + +func merge(t *testing.T, base, alt domain.Transcription, adj reconcile.Adjudicator) domain.Transcription { + t.Helper() + out, err := reconcile.Reconcile(context.Background(), base, alt, adj, opts) + require.NoError(t, err) + return out +} + +func TestReconcile_IdenticalTextsNeedNoModelAndComeBackUnchanged(t *testing.T) { + adj := always(reconcile.VerdictAlt) + + out := merge(t, tr("Vi går igenom läget."), tr("vi går igenom läget"), adj) + + assert.Equal(t, "Vi går igenom läget.", out.Text, "the base's own spelling and punctuation stand") + assert.Empty(t, adj.calls) + require.NotNil(t, out.Reconciliation) + assert.Equal(t, domain.ReconcileDone, out.Reconciliation.Status) + assert.Zero(t, out.Reconciliation.Disputes) +} + +func TestReconcile_TrivialDisputesAreResolvedWithoutAskingTheModel(t *testing.T) { + adj := always(reconcile.VerdictAlt) + + out := merge(t, tr("i takrenns delen ska vi ju betala"), tr("i takrennsdelen ska vi betala"), adj) + + assert.Empty(t, adj.calls, "a compound split and a filler particle are not worth a model call") + assert.Equal(t, "i takrenns delen ska vi ju betala", out.Text, "the base reading is kept") + assert.Equal(t, 2, out.Reconciliation.Disputes) + assert.Equal(t, 2, out.Reconciliation.AutoResolved) + assert.Zero(t, out.Reconciliation.Adjudicated) +} + +func TestReconcile_AltVerdictReplacesTheSpanWithTheAlternativesOwnWords(t *testing.T) { + out := merge(t, tr("Totalt 1015 kronor. Det ska betalas."), tr("Totalt 1050 kronor. Det ska betalas."), always(reconcile.VerdictAlt)) + + assert.Equal(t, "Totalt 1050 kronor. Det ska betalas.", out.Text) + assert.Equal(t, 1, out.Reconciliation.Changed) + assert.Equal(t, 1, out.Reconciliation.Adjudicated) +} + +func TestReconcile_BaseAndUnsureVerdictsKeepTheBaseButOnlyUnsureIsFlagged(t *testing.T) { + base := tr("Totalt 1015 kronor till Axel.") + alt := tr("Totalt 1050 kronor till Alex.") + + keptBase := merge(t, base, alt, always(reconcile.VerdictBase)) + unsure := merge(t, base, alt, always(reconcile.VerdictUnsure)) + + assert.Equal(t, "Totalt 1015 kronor till Axel.", keptBase.Text) + assert.Zero(t, keptBase.Reconciliation.Unsure) + require.Len(t, keptBase.Reconciliation.Contested, 2, + "every judged spot is stored, kept ones too: without them the reconciler cannot be compared with either engine alone (ADR-035)") + for _, c := range keptBase.Reconciliation.Contested { + assert.Equal(t, "base", c.Used) + assert.False(t, c.Unsure, "a confident 'the base is right' is stored but is not an uncertain spot") + } + assert.Equal(t, "Totalt 1015 kronor till Axel.", unsure.Text, "unsure keeps the base") + assert.Equal(t, 2, unsure.Reconciliation.Unsure) + require.Len(t, unsure.Reconciliation.Contested, 2) + assert.True(t, unsure.Reconciliation.Contested[0].Unsure) + assert.Equal(t, "base", unsure.Reconciliation.Contested[0].Used) +} + +func TestReconcile_ChangedSpotsAreRecordedWithBothReadings(t *testing.T) { + out := merge(t, tr("det blir A+2 imorgon"), tr("det blir en A imorgon"), always(reconcile.VerdictAlt)) + + require.Len(t, out.Reconciliation.Contested, 1) + c := out.Reconciliation.Contested[0] + assert.Equal(t, "A+2", c.Base) + assert.Equal(t, "en A", c.Alt) + assert.Equal(t, "alt", c.Used) + assert.False(t, c.Unsure) +} + +func TestReconcile_AWordOnlyTheAlternativeHeardIsInsertedWhereItBelongs(t *testing.T) { + out := merge(t, tr("Man får tillbaka ett förslag."), tr("Man får tillbaka ett coolt förslag."), always(reconcile.VerdictAlt)) + + assert.Equal(t, "Man får tillbaka ett coolt förslag.", out.Text) +} + +func TestReconcile_AWordOnlyTheBaseHeardCanBeRemoved(t *testing.T) { + out := merge(t, tr("Man får tillbaka ett coolt förslag."), tr("Man får tillbaka ett förslag."), always(reconcile.VerdictAlt)) + + assert.Equal(t, "Man får tillbaka ett förslag.", out.Text) +} + +func TestReconcile_SeveralDisputesAreAppliedIndependentlyAndInOrder(t *testing.T) { + adj := &scripted{fn: func(i int, r reconcile.Request) (reconcile.Verdict, error) { + if r.Alt == "mölade" || r.Alt == "Alex" { + return reconcile.VerdictAlt, nil + } + return reconcile.VerdictBase, nil + }} + + out := merge(t, tr("kunderna målade nu och sen betalas till Axel via ombud med 1050 kronor"), + tr("kunderna mölade nu och sen betalas till Alex via ombud med 2050 kronor"), adj) + + assert.Equal(t, "kunderna mölade nu och sen betalas till Alex via ombud med 1050 kronor", out.Text) + assert.Equal(t, 3, out.Reconciliation.Disputes) + assert.Equal(t, 2, out.Reconciliation.Changed) +} + +func TestReconcile_TheReadingOrderAlternatesFromOneDisputeToTheNext(t *testing.T) { + adj := always(reconcile.VerdictBase) + + merge(t, tr("a x s1 s2 y t1 t2 z d"), tr("a p s1 s2 q t1 t2 r d"), adj) + + require.Len(t, adj.calls, 3) + assert.False(t, adj.calls[0].AltFirst) + assert.True(t, adj.calls[1].AltFirst) + assert.False(t, adj.calls[2].AltFirst, "alternating averages out position bias across a transcript") +} + +func TestReconcile_TheModelGetsTheContextWindowFromTheBase(t *testing.T) { + adj := always(reconcile.VerdictBase) + o := opts + o.Window = 2 + + _, err := reconcile.Reconcile(context.Background(), tr("ett två tre fyra fem sex sju åtta nio"), tr("ett två tre fyra FEMM sex sju åtta nio"), adj, o) + + require.NoError(t, err) + require.Len(t, adj.calls, 1) + assert.Equal(t, "tre fyra", adj.calls[0].Before) + assert.Equal(t, "sex sju", adj.calls[0].After) + assert.Equal(t, "fem", adj.calls[0].Base) + assert.Equal(t, "FEMM", adj.calls[0].Alt) +} + +func TestReconcile_OneModelErrorLeavesThatSpotUnsureAndCarriesOn(t *testing.T) { + boom := errors.New("timeout") + adj := &scripted{fn: func(i int, r reconcile.Request) (reconcile.Verdict, error) { + if i == 0 { + return reconcile.VerdictUnsure, boom + } + return reconcile.VerdictAlt, nil + }} + + out := merge(t, tr("a x s1 s2 y c"), tr("a p s1 s2 q c"), adj) + + assert.Equal(t, "a x s1 s2 q c", out.Text, "the first spot stayed as the base, the second was decided") + assert.Equal(t, 1, out.Reconciliation.Errors) + assert.Equal(t, 1, out.Reconciliation.Unsure) + assert.Equal(t, domain.ReconcileDone, out.Reconciliation.Status) +} + +func TestReconcile_ConsecutiveModelErrorsStopTheWholeReconciliationInsteadOfWaitingOutEveryTimeout(t *testing.T) { + boom := errors.New("gateway down") + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { return reconcile.VerdictUnsure, boom }} + a, b := make([]string, 0, 20), make([]string, 0, 20) + for i := 0; i < 10; i++ { + a = append(a, fmt.Sprintf("w%d", i), "s", "t") + b = append(b, fmt.Sprintf("v%d", i), "s", "t") + } + + out := merge(t, tr(strings.Join(a, " ")), tr(strings.Join(b, " ")), adj) + + assert.Len(t, adj.calls, 3, "three consecutive failures and it stops asking") + assert.Equal(t, strings.Join(a, " "), out.Text, "the base text, with none of the partial changes") + assert.Equal(t, domain.ReconcileSkipped, out.Reconciliation.Status) + assert.Contains(t, out.Reconciliation.Reason, "3 times in a row") + assert.NotContains(t, out.Reconciliation.Reason, "gateway down", "the reason is shown on the page: a kind, not the model's own text") +} + +func TestReconcile_ASuccessBetweenFailuresResetsTheBreaker(t *testing.T) { + boom := errors.New("blip") + adj := &scripted{fn: func(i int, r reconcile.Request) (reconcile.Verdict, error) { + if i%3 == 2 { + return reconcile.VerdictBase, nil + } + return reconcile.VerdictUnsure, boom + }} + a, b := make([]string, 0, 20), make([]string, 0, 20) + for i := 0; i < 9; i++ { + a = append(a, fmt.Sprintf("w%d", i), "s", "t") + b = append(b, fmt.Sprintf("v%d", i), "s", "t") + } + + out := merge(t, tr(strings.Join(a, " ")), tr(strings.Join(b, " ")), adj) + + assert.Len(t, adj.calls, 9, "two failures then a success never reach three in a row") + assert.Equal(t, domain.ReconcileDone, out.Reconciliation.Status) + assert.Equal(t, 6, out.Reconciliation.Errors) +} + +func TestReconcile_ACancelledJobIsAnErrorNotASkippedReconciliation(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { + cancel() + return reconcile.VerdictUnsure, ctx.Err() + }} + + _, err := reconcile.Reconcile(ctx, tr("a x b"), tr("a y b"), adj, opts) + + assert.ErrorIs(t, err, context.Canceled, "a shutdown must not be recorded as 'the model was unavailable'") +} + +func TestReconcile_KeepsTheBaseTimingAndDurationAndNamesTheEngines(t *testing.T) { + base := tr("a x b") + base.Segments = []domain.Segment{{Start: 0, End: 4, Text: "a x b"}} + base.AudioSeconds = 3821.5 + + out := merge(t, base, tr("a y b"), always(reconcile.VerdictAlt)) + + require.Len(t, out.Segments, 1) + assert.InDelta(t, 0.0, out.Segments[0].Start, 1e-9, "timing is the base engine's") + assert.InDelta(t, 4.0, out.Segments[0].End, 1e-9) + assert.InDelta(t, 3821.5, out.AudioSeconds, 1e-9) + assert.Equal(t, "iguana/pianissimo", out.Reconciliation.Base) + assert.Equal(t, "iguana/kb-whisper", out.Reconciliation.Alt) + assert.Equal(t, "iguana/gemma4-26b", out.Reconciliation.Reconciler) +} + +func TestReconcile_ASpotCarriesTheSecondItStartsAt(t *testing.T) { + base := tr("ett två tre. fyra fem sex. sju åtta nio.") + base.Segments = []domain.Segment{ + {Start: 0, End: 10, Text: "ett två tre."}, + {Start: 10, End: 20, Text: "fyra fem sex."}, + {Start: 20, End: 30, Text: "sju åtta nio."}, + } + alt := tr("ett två tre. fyra femm sex. sju åtta nio.") + + out := merge(t, base, alt, always(reconcile.VerdictUnsure)) + + require.Len(t, out.Reconciliation.Contested, 1) + assert.InDelta(t, 10.0, out.Reconciliation.Contested[0].Seconds, 1e-9, "'fem' is in the second segment, which starts at 10 s") +} + +func TestReconcile_WithNoSegmentsASpotIsPlacedProportionally(t *testing.T) { + base := tr("a b c d e f g h i j") + base.AudioSeconds = 100 + alt := tr("a b c d e f g h i J2") + + out := merge(t, base, alt, always(reconcile.VerdictUnsure)) + + require.Len(t, out.Reconciliation.Contested, 1) + assert.InDelta(t, 90.0, out.Reconciliation.Contested[0].Seconds, 1e-9, "token 9 of 10 in 100 s") +} + +func TestReconcile_JudgedSpotsAreCappedButTheCountsAreExact(t *testing.T) { + a, b := make([]string, 0, 600), make([]string, 0, 600) + for i := 0; i < 300; i++ { + a = append(a, fmt.Sprintf("w%d", i), "same", "same2") + b = append(b, fmt.Sprintf("v%d", i), "same", "same2") + } + capped := opts + capped.MaxContested = 50 + + out, err := reconcile.Reconcile(context.Background(), tr(strings.Join(a, " ")), tr(strings.Join(b, " ")), always(reconcile.VerdictUnsure), capped) + + require.NoError(t, err) + assert.Equal(t, 300, out.Reconciliation.Unsure) + assert.Equal(t, 300, out.Reconciliation.Disputes) + assert.Len(t, out.Reconciliation.Contested, 50, "the stored list is bounded; the counts are not") +} + +func TestReconcile_TheDefaultCapIsAFewThousandSoAWholeMeetingIsKept(t *testing.T) { + assert.Equal(t, 2000, reconcile.MaxContested, "a 90-minute recording has a few hundred judged spots; the cap is a guard, not a budget") +} + +func TestReconcile_AVerdictReturnedAlongsideAnErrorIsIgnored(t *testing.T) { + // An adjudicator that returns a verdict AND an error must not have the verdict used: + // the contract is that an error means no decision. + adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) { + return reconcile.VerdictAlt, errors.New("half a reply") + }} + + out := merge(t, tr("a x b"), tr("a y b"), adj) + + assert.Equal(t, "a x b", out.Text, "no decision means the base stands") + assert.Zero(t, out.Reconciliation.Changed) + assert.Equal(t, 1, out.Reconciliation.Unsure) +} diff --git a/internal/reconcile/tokens.go b/internal/reconcile/tokens.go new file mode 100644 index 0000000..30899fa --- /dev/null +++ b/internal/reconcile/tokens.go @@ -0,0 +1,57 @@ +// Package reconcile merges two transcripts of the same speech: it finds where they +// disagree word by word, resolves the trivial disagreements itself, and asks a LOCAL +// model to choose between the two readings of the rest (ADR-034). It depends on +// nothing outside the standard library and the domain; the model, the engines and +// the store are wired in from outside. +package reconcile + +import ( + "strings" + "unicode" +) + +// Token is one word of a transcript. Text is the word as written, with its casing +// and punctuation, so the merged transcript reads naturally; Key is what two engines +// are compared on: lower case, letters and digits only. +type Token struct { + Text string + Key string +} + +// Tokenize splits s on whitespace. A "word" that has no letter or digit (a dash, a +// lone full stop) is not something two engines can disagree about, so it is joined +// to the neighbouring word instead of becoming a token of its own. +func Tokenize(s string) []Token { + var out []Token + pending := "" + for _, w := range strings.Fields(s) { + k := key(w) + if k == "" { + if len(out) > 0 { + out[len(out)-1].Text += " " + w + } else if pending == "" { + pending = w + } else { + pending += " " + w + } + continue + } + text := w + if pending != "" { + text = pending + " " + w + pending = "" + } + out = append(out, Token{Text: text, Key: k}) + } + return out +} + +func key(w string) string { + var b strings.Builder + for _, r := range w { + if unicode.IsLetter(r) || unicode.IsDigit(r) { + b.WriteRune(unicode.ToLower(r)) + } + } + return b.String() +} diff --git a/internal/reconcile/tokens_test.go b/internal/reconcile/tokens_test.go new file mode 100644 index 0000000..1675ba5 --- /dev/null +++ b/internal/reconcile/tokens_test.go @@ -0,0 +1,66 @@ +package reconcile_test + +import ( + "testing" + + "github.com/stretchr/testify/assert" + + "git.d-ma.be/mathias/tapir/internal/reconcile" +) + +func texts(ts []reconcile.Token) []string { + out := make([]string, len(ts)) + for i, t := range ts { + out[i] = t.Text + } + return out +} + +func keys(ts []reconcile.Token) []string { + out := make([]string, len(ts)) + for i, t := range ts { + out[i] = t.Key + } + return out +} + +func TestTokenize_KeepsTheWordAsWrittenAndKeysItWithoutCaseOrPunctuation(t *testing.T) { + got := reconcile.Tokenize("Hej, allihopa! Vi går igenom ÅÄÖ.") + + assert.Equal(t, []string{"Hej,", "allihopa!", "Vi", "går", "igenom", "ÅÄÖ."}, texts(got), + "the original spelling and punctuation are kept for the merged text") + assert.Equal(t, []string{"hej", "allihopa", "vi", "går", "igenom", "åäö"}, keys(got), + "the key is lower case with punctuation removed, Swedish letters lowered correctly") +} + +func TestTokenize_KeysIgnoreHyphensAndPlusSoOnlyWordsDecideEquality(t *testing.T) { + got := reconcile.Tokenize("end-to-end A+2 5 000 kr") + + assert.Equal(t, []string{"endtoend", "a2", "5", "000", "kr"}, keys(got)) +} + +func TestTokenize_APunctuationOnlyWordJoinsThePreviousWordInsteadOfBecomingOne(t *testing.T) { + got := reconcile.Tokenize("Vi går – sen tillbaka . Slut") + + assert.Equal(t, []string{"Vi", "går –", "sen", "tillbaka .", "Slut"}, texts(got), + "a dash or a lone full stop is not a word the engines can disagree about") + assert.Equal(t, []string{"vi", "går", "sen", "tillbaka", "slut"}, keys(got)) +} + +func TestTokenize_LeadingPunctuationOnlyWordJoinsTheNextWord(t *testing.T) { + got := reconcile.Tokenize("— Hej då") + + assert.Equal(t, []string{"— Hej", "då"}, texts(got)) +} + +func TestTokenize_NewlinesAndRunsOfSpaceAreJustSeparators(t *testing.T) { + got := reconcile.Tokenize(" Hej\n allihopa \t Vi ") + + assert.Equal(t, []string{"Hej", "allihopa", "Vi"}, texts(got)) +} + +func TestTokenize_EmptyAndPunctuationOnlyInputGiveNoTokens(t *testing.T) { + assert.Empty(t, reconcile.Tokenize("")) + assert.Empty(t, reconcile.Tokenize(" \n ")) + assert.Empty(t, reconcile.Tokenize("— . !")) +} diff --git a/internal/uploads/intake.go b/internal/uploads/intake.go new file mode 100644 index 0000000..8f5ce13 --- /dev/null +++ b/internal/uploads/intake.go @@ -0,0 +1,161 @@ +package uploads + +import ( + "context" + "errors" + "fmt" + "io" + "os" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrUnreadable is what a Prober returns when the FILE is the problem (not media, +// truncated, damaged), as opposed to the prober itself failing. Accept turns it +// into a user-facing rejection; any other prober error is our fault and surfaces +// as an internal error — a missing ffprobe must not read as "your file is bad". +var ErrUnreadable = errors.New("uploads: file is not readable media") + +// Prober inspects a file on disk. The ffprobe adapter implements it. +type Prober interface { + Probe(ctx context.Context, path string) (Probe, error) +} + +// Store is the slice of the private transcript store the intake needs. +type Store interface { + CreateUpload(ctx context.Context, userID, title string, hadVideo bool) (string, error) + FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error +} + +// Job is one accepted upload awaiting transcription. Path is a temp file the +// receiver owns: it must delete it when finished (the audio is never retained). +type Job struct { + UserID string + UploadID string + Path string + // Duration is what the file CLAIMED when probed. The worker compares it with what + // was really converted; zero means unknown (ADR-035 review). + Duration time.Duration +} + +// Submitter hands accepted jobs to the in-process worker (ADR-033). +type Submitter interface { + Submit(Job) error +} + +// Intake validates an uploaded recording and, if it passes, queues it. +type Intake struct { + store Store + prober Prober + sub Submitter + limits Limits + tmpDir string +} + +// NewIntake wires an Intake. tmpDir is where uploads are spooled; it must exist. +func NewIntake(s Store, p Prober, sub Submitter, l Limits, tmpDir string) *Intake { + return &Intake{store: s, prober: p, sub: sub, limits: l, tmpDir: tmpDir} +} + +// ErrClientRead marks an error that came from READING the client's body: a connection that +// dropped, a body cut short. It is the sender's to retry, unlike a spool, probe or database +// fault, which is ours. It is a marker on the error, not a sentinel that others may wrap: +// io.ErrUnexpectedEOF alone is also what a database drop looks like. +var ErrClientRead = errors.New("uploads: client body read failed") + +// clientReader remembers the last non-EOF error its source returned, so Accept can tell +// a failure reading the client from a failure writing the spool file. +type clientReader struct { + r io.Reader + err error +} + +func (c *clientReader) Read(p []byte) (int, error) { + n, err := c.r.Read(p) + if err != nil && !errors.Is(err, io.EOF) { + c.err = err + } + return n, err +} + +// busyMessage is what a user is told when the queue is full. +const busyMessage = "Too many recordings are waiting to be transcribed. Please try again in a few minutes." + +// Accept spools src to disk (bounded by MaxBytes), probes it, validates it, records +// a queued upload and submits the job. It returns the upload id. A file that fails +// validation yields a *RejectError and creates NO row — rejection is synchronous +// and leaves nothing behind. On success the temp file is handed to the job and the +// worker deletes it; on every failure path Accept deletes it itself. +func (in *Intake) Accept(ctx context.Context, userID, title string, src io.Reader) (string, error) { + // Refuse before spooling when the queue has no room, if the submitter can say so: a full + // queue otherwise cost up to the upload cap on disk, an ffprobe run and a failed row for + // every refused attempt. + if rc, ok := in.sub.(interface{ HasRoom(userID string) bool }); ok && !rc.HasRoom(userID) { + return "", &RejectError{Reason: ReasonBusy, Message: busyMessage} + } + f, err := os.CreateTemp(in.tmpDir, "upload-*") + if err != nil { + return "", fmt.Errorf("uploads: spool: %w", err) + } + path := f.Name() + handedOver := false + defer func() { + if !handedOver { + _ = os.Remove(path) + } + }() + + // Read one byte past the cap so "exactly at the limit" and "over" are told apart. + cr := &clientReader{r: io.LimitReader(src, in.limits.MaxBytes+1)} + n, err := io.Copy(f, cr) + if cerr := f.Close(); err == nil { + err = cerr + } + if err != nil { + if cr.err != nil && errors.Is(err, cr.err) { + return "", fmt.Errorf("%w: %w", ErrClientRead, err) + } + return "", fmt.Errorf("uploads: receive: %w", err) + } + if n > in.limits.MaxBytes { + return "", &RejectError{Reason: ReasonTooLarge, Message: fmt.Sprintf( + "This file is larger than the %d MB limit.", in.limits.MaxBytes>>20)} + } + + probe, err := in.prober.Probe(ctx, path) + if errors.Is(err, ErrUnreadable) { + return "", &RejectError{ + Reason: ReasonUnreadable, + Message: "This file could not be read as audio; it may be damaged or not a recording.", + } + } + if err != nil { + return "", fmt.Errorf("uploads: probe: %w", err) + } + if err := Validate(probe, in.limits); err != nil { + return "", err + } + + id, err := in.store.CreateUpload(ctx, userID, title, probe.HasVideo) + if err != nil { + return "", fmt.Errorf("uploads: create: %w", err) + } + if err := in.sub.Submit(Job{UserID: userID, UploadID: id, Path: path, Duration: probe.Duration}); err != nil { + if errors.Is(err, ErrQueueFull) { + // The user's to retry, not a server fault: refuse it as such, and do not leave + // the row queued -- that is the stuck spinner. + if ferr := in.store.FailUpload(ctx, userID, id, domain.UploadErrBusy); ferr != nil { + return "", fmt.Errorf("uploads: submit: %w (and failing the row: %v)", err, ferr) + } + return "", &RejectError{Reason: ReasonBusy, Message: busyMessage} + } + // A row that will never run must not sit queued: that is the stuck spinner. + if ferr := in.store.FailUpload(ctx, userID, id, domain.UploadErrInterrupted); ferr != nil { + return "", fmt.Errorf("uploads: submit: %w (and failing the row: %v)", err, ferr) + } + return "", fmt.Errorf("uploads: submit: %w", err) + } + handedOver = true + return id, nil +} diff --git a/internal/uploads/intake_busy_test.go b/internal/uploads/intake_busy_test.go new file mode 100644 index 0000000..81cc563 --- /dev/null +++ b/internal/uploads/intake_busy_test.go @@ -0,0 +1,96 @@ +package uploads_test + +import ( + "context" + "errors" + "fmt" + "io" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// A full queue is the user's to retry, not a server fault: a typed rejection, the row +// failed honestly, the audio removed (adversarial review F6). + +func TestAccept_AFullQueueIsABusyRejectionNotAServerErrorAndLeavesNothingBehind(t *testing.T) { + r := newRig(t) + r.sub.err = uploads.ErrQueueFull + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej), "a full queue must be a rejection the web layer can answer with 429: %v", err) + assert.Equal(t, uploads.ReasonBusy, rej.Reason) + assert.Contains(t, rej.Message, "waiting") + assert.Equal(t, domain.UploadErrBusy, r.st.failed["up-1"], "the row must not sit queued: that is the stuck spinner") + assert.Empty(t, r.tmpFiles(t), "the audio is deleted") +} + +// roomSubmitter is a Submitter that can say, before anything is spooled, that there is no room. +type roomSubmitter struct { + fakeSubmitter + room bool +} + +func (r *roomSubmitter) HasRoom(string) bool { return r.room } + +func TestAccept_ANoRoomQueueIsRefusedBeforeAnythingIsSpooledProbedOrStored(t *testing.T) { + // Checking only at Submit meant 200 MB was spooled and ffprobe run for every refused + // attempt, and a failed row was left behind each time (second adversarial review). + r := newRig(t) + full := &roomSubmitter{room: false} + in := uploads.NewIntake(r.st, r.pr, full, uploads.Limits{MaxBytes: 1024, MaxDuration: 90 * time.Minute}, r.tmp) + + _, err := in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej)) + assert.Equal(t, uploads.ReasonBusy, rej.Reason) + assert.Empty(t, r.st.created, "no row for an upload that was never accepted") + assert.Empty(t, r.pr.paths, "ffprobe was never run on it") + assert.Empty(t, r.tmpFiles(t), "nothing was spooled") + assert.Empty(t, full.jobs) +} + +func TestAccept_ASubmitterThatCannotSayStillWorksAsBefore(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.NoError(t, err, "a Submitter without HasRoom is accepted: the check at Submit remains") +} + +// brokenBody is a client body that fails mid-read, like a connection that dropped. +type brokenBody struct{ n int } + +func (b *brokenBody) Read(p []byte) (int, error) { + if b.n > 0 { + b.n = 0 + return copy(p, "partial audio"), nil + } + return 0, io.ErrUnexpectedEOF +} + +func TestAccept_AnErrorReadingTheClientsBodyIsMarkedAsTheClientsAndAWrappedSubmitErrorIsNot(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "t", &brokenBody{n: 1}) + + require.Error(t, err) + assert.ErrorIs(t, err, uploads.ErrClientRead, "a dropped client body is the sender's to retry") + assert.Empty(t, r.tmpFiles(t)) + + // A fault that is ours must NOT carry the marker, even if it wraps the same sentinel. + r2 := newRig(t) + r2.sub.err = fmt.Errorf("db: %w", io.ErrUnexpectedEOF) + _, err = r2.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + require.Error(t, err) + assert.NotErrorIs(t, err, uploads.ErrClientRead) +} diff --git a/internal/uploads/intake_test.go b/internal/uploads/intake_test.go new file mode 100644 index 0000000..4fc8dbd --- /dev/null +++ b/internal/uploads/intake_test.go @@ -0,0 +1,240 @@ +package uploads_test + +import ( + "bytes" + "context" + "errors" + "io" + "os" + "path/filepath" + "strings" + "testing" + "testing/iotest" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +type fakeStore struct { + created []string // titles + hadVideo []bool + failed map[string]domain.UploadErrorKind + nextID string +} + +func (f *fakeStore) CreateUpload(_ context.Context, _, title string, hadVideo bool) (string, error) { + f.created = append(f.created, title) + f.hadVideo = append(f.hadVideo, hadVideo) + return f.nextID, nil +} + +func (f *fakeStore) FailUpload(_ context.Context, _, id string, kind domain.UploadErrorKind) error { + if f.failed == nil { + f.failed = map[string]domain.UploadErrorKind{} + } + f.failed[id] = kind + return nil +} + +type fakeProber struct { + probe uploads.Probe + err error + paths []string + seen []byte // content of the file at probe time +} + +func (f *fakeProber) Probe(_ context.Context, path string) (uploads.Probe, error) { + f.paths = append(f.paths, path) + f.seen, _ = os.ReadFile(path) + return f.probe, f.err +} + +type fakeSubmitter struct { + jobs []uploads.Job + err error +} + +func (f *fakeSubmitter) Submit(j uploads.Job) error { + f.jobs = append(f.jobs, j) + return f.err +} + +type rig struct { + in *uploads.Intake + st *fakeStore + pr *fakeProber + sub *fakeSubmitter + tmp string +} + +func newRig(t *testing.T) *rig { + t.Helper() + r := &rig{ + st: &fakeStore{nextID: "up-1"}, + pr: &fakeProber{probe: uploads.Probe{Duration: 10 * time.Minute, HasAudio: true}}, + sub: &fakeSubmitter{}, + tmp: t.TempDir(), + } + r.in = uploads.NewIntake(r.st, r.pr, r.sub, uploads.Limits{MaxBytes: 1024, MaxDuration: 90 * time.Minute}, r.tmp) + return r +} + +func (r *rig) tmpFiles(t *testing.T) []string { + t.Helper() + ents, err := os.ReadDir(r.tmp) + require.NoError(t, err) + var names []string + for _, e := range ents { + names = append(names, e.Name()) + } + return names +} + +func TestAccept_ValidAudioIsStoredProbedQueuedAndSubmitted(t *testing.T) { + r := newRig(t) + audio := bytes.Repeat([]byte("a"), 500) + + id, err := r.in.Accept(context.Background(), "user-1", "Teams export", bytes.NewReader(audio)) + + require.NoError(t, err) + assert.Equal(t, "up-1", id) + assert.Equal(t, []string{"Teams export"}, r.st.created) + assert.Equal(t, audio, r.pr.seen, "the prober must see the complete uploaded bytes, on disk") + require.Len(t, r.sub.jobs, 1) + job := r.sub.jobs[0] + assert.Equal(t, uploads.Job{UserID: "user-1", UploadID: "up-1", Path: job.Path, Duration: 10 * time.Minute}, job, + "the job carries what the file CLAIMED, so the worker can compare it with what was converted") + assert.Equal(t, filepath.Dir(job.Path), r.tmp, "the temp file lives in the configured directory") + got, err := os.ReadFile(job.Path) + require.NoError(t, err) + assert.Equal(t, audio, got, "the handed-over file still exists: the worker, not Accept, deletes it") +} + +func TestAccept_OversizeBodyIsRejectedAndLeavesNothingBehind(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "big", strings.NewReader(strings.Repeat("a", 1025))) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej)) + assert.Equal(t, uploads.ReasonTooLarge, rej.Reason) + assert.Contains(t, rej.Message, "limit") + assert.Empty(t, r.pr.paths, "a file already over the size cap must not be probed") + assert.Empty(t, r.st.created, "a rejected upload creates no row") + assert.Empty(t, r.sub.jobs) + assert.Empty(t, r.tmpFiles(t), "the partial temp file must be removed") +} + +func TestAccept_BodyExactlyAtTheLimitIsAccepted(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "edge", strings.NewReader(strings.Repeat("a", 1024))) + + require.NoError(t, err) +} + +func TestAccept_ProbeVerdictRejectsWithoutRowAndRemovesTheFile(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + reason uploads.Reason + }{ + {"video with no audio", uploads.Probe{Duration: time.Minute, HasVideo: true}, uploads.ReasonNoAudio}, + {"too long", uploads.Probe{Duration: 2 * time.Hour, HasAudio: true}, uploads.ReasonTooLong}, + {"no audio", uploads.Probe{Duration: time.Minute}, uploads.ReasonNoAudio}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newRig(t) + r.pr.probe = tt.probe + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej)) + assert.Equal(t, tt.reason, rej.Reason) + assert.Empty(t, r.st.created) + assert.Empty(t, r.sub.jobs) + assert.Empty(t, r.tmpFiles(t)) + }) + } +} + +func TestAccept_UnreadableFileIsARejectionButAProberFaultIsAnError(t *testing.T) { + t.Run("ffprobe says the file is not media", func(t *testing.T) { + r := newRig(t) + r.pr.err = uploads.ErrUnreadable + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("not media")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej), "the user's file is the problem: tell them") + assert.Equal(t, uploads.ReasonUnreadable, rej.Reason) + assert.Empty(t, r.tmpFiles(t)) + }) + t.Run("the prober itself broke", func(t *testing.T) { + r := newRig(t) + r.pr.err = errors.New("exec: ffprobe: executable file not found") + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.Error(t, err) + var rej *uploads.RejectError + assert.False(t, errors.As(err, &rej), "our fault must not be reported as the user's bad file") + assert.Empty(t, r.st.created) + assert.Empty(t, r.tmpFiles(t)) + }) +} + +func TestAccept_ClientDisconnectMidUploadLeavesNothingBehind(t *testing.T) { + r := newRig(t) + body := io.MultiReader(strings.NewReader("partial"), iotest.ErrReader(errors.New("client went away"))) + + _, err := r.in.Accept(context.Background(), "user-1", "t", body) + + require.Error(t, err) + assert.Empty(t, r.pr.paths) + assert.Empty(t, r.st.created) + assert.Empty(t, r.tmpFiles(t)) +} + +func TestAccept_SubmitFailureFailsTheRowHonestlyAndRemovesTheFile(t *testing.T) { + r := newRig(t) + r.sub.err = errors.New("worker stopped") + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.Error(t, err) + assert.Equal(t, domain.UploadErrInterrupted, r.st.failed["up-1"], + "a row that will never run must not sit queued: that is the stuck spinner ADR-025 forbids") + assert.Empty(t, r.tmpFiles(t)) +} + +func TestAccept_AFileWithAVideoTrackIsAcceptedAndRecordsThatItHadOne(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + wantVideo bool + }{ + {"audio only", uploads.Probe{Duration: 10 * time.Minute, HasAudio: true}, false}, + {"audio with a video track (a Teams meeting with the camera off)", uploads.Probe{Duration: 10 * time.Minute, HasAudio: true, HasVideo: true}, true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newRig(t) + r.pr.probe = tt.probe + + id, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.NoError(t, err) + assert.Equal(t, "up-1", id) + assert.Equal(t, []bool{tt.wantVideo}, r.st.hadVideo, + "the row must say whether a video track was present, so the page can say it was ignored") + assert.Len(t, r.sub.jobs, 1, "it is queued like any other recording") + }) + } +} diff --git a/internal/uploads/validate.go b/internal/uploads/validate.go new file mode 100644 index 0000000..4251026 --- /dev/null +++ b/internal/uploads/validate.go @@ -0,0 +1,83 @@ +// Package uploads is the use case behind "upload an audio recording" (ADR-033): +// validate the file by PROBING it, keep the transcript private, never retain the +// audio. It depends only on ports (a Prober, a store, a job Submitter) — the +// concrete ffprobe adapter, the web handler and the transcription worker are wired +// from outside. +package uploads + +import ( + "fmt" + "time" +) + +// Limits bound what an upload may be. Both are rejected synchronously, with a +// specific message, rather than by a timeout (ADR-033). +type Limits struct { + MaxBytes int64 + MaxDuration time.Duration +} + +// Probe is what inspecting an uploaded file revealed. It is the input to Validate +// because a name or MIME type cannot say what a container holds: a Teams recording +// is an .mp4 whether or not the camera was on. +type Probe struct { + Duration time.Duration + HasAudio bool + HasVideo bool +} + +// Reason is the machine-checkable class of a rejection. +type Reason string + +const ( + ReasonNoAudio Reason = "no_audio" + ReasonTooLong Reason = "too_long" + ReasonTooLarge Reason = "too_large" + ReasonUnreadable Reason = "unreadable" + // ReasonBusy: the queue is full. Not about the file: the user retries later. + ReasonBusy Reason = "busy" +) + +// RejectError is a synchronous, user-presentable refusal. Message is written for +// the person who uploaded the file; Reason is for code and tests. +type RejectError struct { + Reason Reason + Message string +} + +func (e *RejectError) Error() string { return string(e.Reason) + ": " + e.Message } + +// Validate decides whether a probed file may be transcribed. A video track is NOT +// a reason to refuse: a Teams meeting with the camera off is still an .mp4 with a +// video track, so rejecting video would reject the primary use case. Only the audio +// is used (Probe.HasVideo is recorded so the user can be told the video was +// ignored). What IS refused: no audio at all, an unreadable length, too long. +func Validate(p Probe, l Limits) error { + switch { + case !p.HasAudio: + return &RejectError{Reason: ReasonNoAudio, Message: "This file has no audio track to transcribe."} + case p.Duration <= 0: + return &RejectError{Reason: ReasonUnreadable, Message: "Could not read a length from this file; it may be damaged."} + case p.Duration > l.MaxDuration: + return &RejectError{Reason: ReasonTooLong, Message: fmt.Sprintf( + "This recording is %s long; the limit is %s.", FormatDuration(p.Duration), FormatDuration(l.MaxDuration))} + } + return nil +} + +// FormatDuration renders a length for people: "1 h 42 min", "45 min", "30 s". +func FormatDuration(d time.Duration) string { + d = d.Round(time.Second) + h := int(d / time.Hour) + m := int(d % time.Hour / time.Minute) + switch { + case h > 0 && m > 0: + return fmt.Sprintf("%d h %d min", h, m) + case h > 0: + return fmt.Sprintf("%d h", h) + case m > 0: + return fmt.Sprintf("%d min", m) + default: + return fmt.Sprintf("%d s", int(d/time.Second)) + } +} diff --git a/internal/uploads/validate_test.go b/internal/uploads/validate_test.go new file mode 100644 index 0000000..c5dcd78 --- /dev/null +++ b/internal/uploads/validate_test.go @@ -0,0 +1,93 @@ +package uploads_test + +import ( + "errors" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +var testLimits = uploads.Limits{MaxBytes: 200 << 20, MaxDuration: 90 * time.Minute} + +func TestValidate(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + wantReason uploads.Reason // "" = accepted + wantInMsg []string + }{ + { + name: "audio-only file within the duration limit is accepted", + probe: uploads.Probe{Duration: 45 * time.Minute, HasAudio: true}, + }, + { + name: "a recording exactly at the limit is accepted", + probe: uploads.Probe{Duration: 90 * time.Minute, HasAudio: true}, + }, + { + name: "a recording that also has a video stream is accepted: the audio is what is used", + probe: uploads.Probe{Duration: 4 * time.Minute, HasAudio: true, HasVideo: true}, + }, + { + name: "video with no audio is rejected as having no audio, and says so", + probe: uploads.Probe{Duration: 4 * time.Minute, HasVideo: true}, + wantReason: uploads.ReasonNoAudio, + wantInMsg: []string{"no audio"}, + }, + { + name: "a file with no audio stream is rejected", + probe: uploads.Probe{Duration: 4 * time.Minute}, + wantReason: uploads.ReasonNoAudio, + wantInMsg: []string{"no audio"}, + }, + { + name: "a recording over the limit is rejected, stating both lengths", + probe: uploads.Probe{Duration: 102 * time.Minute, HasAudio: true}, + wantReason: uploads.ReasonTooLong, + wantInMsg: []string{"1 h 42 min", "1 h 30 min"}, + }, + { + name: "zero duration is unreadable, not accepted as an empty recording", + probe: uploads.Probe{HasAudio: true}, + wantReason: uploads.ReasonUnreadable, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := uploads.Validate(tt.probe, testLimits) + + if tt.wantReason == "" { + require.NoError(t, err) + return + } + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej), "want a *RejectError, got %v", err) + assert.Equal(t, tt.wantReason, rej.Reason) + for _, s := range tt.wantInMsg { + assert.Contains(t, rej.Message, s) + } + }) + } +} + +func TestFormatDuration(t *testing.T) { + tests := []struct { + d time.Duration + want string + }{ + {90 * time.Minute, "1 h 30 min"}, + {102 * time.Minute, "1 h 42 min"}, + {45 * time.Minute, "45 min"}, + {2 * time.Hour, "2 h"}, + {30 * time.Second, "30 s"}, + } + for _, tt := range tests { + t.Run(tt.want, func(t *testing.T) { + assert.Equal(t, tt.want, uploads.FormatDuration(tt.d)) + }) + } +} diff --git a/internal/uploads/worker.go b/internal/uploads/worker.go new file mode 100644 index 0000000..8d4b738 --- /dev/null +++ b/internal/uploads/worker.go @@ -0,0 +1,396 @@ +package uploads + +import ( + "context" + "errors" + "fmt" + "log/slog" + "os" + "sync" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" + "git.d-ma.be/mathias/tapir/internal/progress" +) + +// ErrStopped is returned by Submit once the worker has stopped. +var ErrStopped = errors.New("uploads: worker stopped") + +// ErrQueueFull is returned by Submit when the queue, or one user's share of it, is at +// its cap. Every queued job holds up to the upload cap on disk until its turn, so an +// unbounded queue is a disk-filling lever (ADR-035 review). +var ErrQueueFull = errors.New("uploads: queue full") + +// Converter turns the uploaded file into the 16 kHz mono PCM WAV whisper wants. +type Converter interface { + ToWAV(ctx context.Context, src, dst string) error +} + +// Transcriber turns a WAV into a timed transcription. The gateway chain +// (iguana primary, berget fallback) implements it. +type Transcriber interface { + Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) +} + +// Planner is an optional extra on a Transcriber: it declares its own phases (for +// example one per engine) so the page can show them. A Transcriber that does not +// implement it gets a single "transcribing" phase from the worker. Whatever a Planner +// declares it reports itself, through progress.From(ctx) (ADR-035). +type Planner interface { + Plan() []progress.Planned +} + +// WorkerStore is the slice of the private transcript store the worker drives. +type WorkerStore interface { + StartTranscribing(ctx context.Context, userID, id string) error + CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error + FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error + SaveProgress(ctx context.Context, userID, id string, steps []domain.ProgressStep, at time.Time) error + Heartbeat(ctx context.Context, userID, id string) error +} + +// WorkerOption tunes a Worker. +type WorkerOption func(*Worker) + +// WithHeartbeat sets how often a running job stamps its row alive and how often it logs +// which phase is still running. The defaults are 10 s and 5 min (ADR-035). +func WithHeartbeat(beat, still time.Duration) WorkerOption { + return func(w *Worker) { w.beatEvery, w.stillEvery = beat, still } +} + +// WithQueueLimits bounds how many jobs may WAIT: in total, and per user. A job being +// processed is not waiting. The defaults are 20 and 3 (one instance, one job at a time). +func WithQueueLimits(total, perUser int) WorkerOption { + return func(w *Worker) { w.maxQueued, w.maxPerUser = total, perUser } +} + +// WithMaxDuration makes the worker measure the converted audio and fail a job whose real +// length exceeds the limit, or falls far short of what the file claimed. Zero (the +// default) measures nothing. The limit is first checked against the file's header, which +// can lie (ADR-035 review). +func WithMaxDuration(d time.Duration) WorkerOption { return func(w *Worker) { w.maxDuration = d } } + +// WithLogger replaces slog.Default() as the worker's logger. +func WithLogger(l *slog.Logger) WorkerOption { return func(w *Worker) { w.log = l } } + +const ( + defaultMaxQueued = 20 + defaultMaxPerUser = 3 + defaultBeatEvery = 10 * time.Second + defaultStillEvery = 5 * time.Minute +) + +// jobSink binds the store to one job so a progress.Tracker can persist through it. +type jobSink struct { + st WorkerStore + userID, id string +} + +func (s jobSink) SaveProgress(ctx context.Context, steps []domain.ProgressStep, at time.Time) error { + return s.st.SaveProgress(ctx, s.userID, s.id, steps, at) +} + +func (s jobSink) Beat(ctx context.Context) error { return s.st.Heartbeat(ctx, s.userID, s.id) } + +// Worker transcribes accepted uploads in-process, ONE AT A TIME (ADR-033). It is +// not a second deployable: it shares the web process's lifetime, which is why it +// inherits ADR-018's single-replica assumption — the queue is process-local. The +// concurrency of one is deliberate: two simultaneous 90-minute jobs on iguana were +// never measured, and a Metal OOM there latches the whisper backend (infra#273). +type Worker struct { + store WorkerStore + conv Converter + tr Transcriber + tmpDir string + timeout time.Duration + + log *slog.Logger + beatEvery, stillEvery time.Duration + maxDuration time.Duration + maxQueued, maxPerUser int + + mu sync.Mutex + queue []Job + stopped bool + wake chan struct{} +} + +// NewWorker wires a Worker. timeout bounds each job (conversion + transcription); +// a job that outlives it is failed rather than left spinning. +func NewWorker(s WorkerStore, c Converter, t Transcriber, tmpDir string, timeout time.Duration, opts ...WorkerOption) *Worker { + w := &Worker{ + store: s, conv: c, tr: t, tmpDir: tmpDir, timeout: timeout, wake: make(chan struct{}, 1), + beatEvery: defaultBeatEvery, stillEvery: defaultStillEvery, + maxQueued: defaultMaxQueued, maxPerUser: defaultMaxPerUser, + } + for _, o := range opts { + o(w) + } + return w +} + +// HasRoom reports whether a job from this user would be accepted right now. It lets the +// intake refuse BEFORE spooling up to the upload cap and running ffprobe; Submit still +// enforces the same limits atomically, so a race costs at worst one refused submit. +func (w *Worker) HasRoom(userID string) bool { + w.mu.Lock() + defer w.mu.Unlock() + if w.stopped || len(w.queue) >= w.maxQueued { + return false + } + mine := 0 + for _, q := range w.queue { + if q.UserID == userID { + mine++ + } + } + return mine < w.maxPerUser +} + +// Submit queues a job. It never blocks. It satisfies Submitter. +func (w *Worker) Submit(j Job) error { + w.mu.Lock() + defer w.mu.Unlock() + if w.stopped { + return ErrStopped + } + if len(w.queue) >= w.maxQueued { + return ErrQueueFull + } + mine := 0 + for _, q := range w.queue { + if q.UserID == j.UserID { + mine++ + } + } + if mine >= w.maxPerUser { + return ErrQueueFull + } + w.queue = append(w.queue, j) + select { + case w.wake <- struct{}{}: + default: + } + return nil +} + +// Run processes queued jobs until ctx ends, then deletes the audio of anything +// still queued and refuses new jobs. The rows of those jobs stay queued or +// transcribing and are failed as interrupted by the start-up sweep +// (store.FailInterruptedUploads): the audio is not retained, so they cannot resume. +func (w *Worker) Run(ctx context.Context) { + defer w.stop() + for { + job, ok := w.next(ctx) + if !ok { + return + } + w.Process(ctx, job) + } +} + +// next blocks for the next job. Cancellation wins over a non-empty queue: a job +// still queued at shutdown must not be STARTED (it would begin a long conversion +// with an already-cancelled context); stop() deletes its audio instead. +func (w *Worker) next(ctx context.Context) (Job, bool) { + for { + if ctx.Err() != nil { + return Job{}, false + } + w.mu.Lock() + if len(w.queue) > 0 { + j := w.queue[0] + w.queue = w.queue[1:] + w.mu.Unlock() + return j, true + } + w.mu.Unlock() + select { + case <-w.wake: + case <-ctx.Done(): + return Job{}, false + } + } +} + +func (w *Worker) stop() { + w.mu.Lock() + defer w.mu.Unlock() + w.stopped = true + for _, j := range w.queue { + _ = os.Remove(j.Path) + } + w.queue = nil +} + +// Process runs one job: row -> transcribing, convert, transcribe, store. The +// uploaded file and the converted WAV are deleted on every path — the audio is +// never retained (ADR-033). A failure is recorded with a TYPED kind; a shutdown +// mid-job is not a failure and leaves the row for the start-up sweep. +// +// Every phase is reported through one progress.Tracker, which saves it to the row for +// the page and logs it, and a heartbeat stamps the row on its own while the job runs +// (ADR-035). Nothing logged or saved here carries transcript text or the file's name. +func (w *Worker) Process(ctx context.Context, job Job) { + defer func() { _ = os.Remove(job.Path) }() + base := w.log + if base == nil { + base = slog.Default() + } + log := base.With("upload", job.UploadID) + start := time.Now() + + if err := w.store.StartTranscribing(ctx, job.UserID, job.UploadID); err != nil { + log.Error("upload: start transcribing", "kind", errkind.Of(err)) + w.fail(ctx, job, log, domain.UploadErrTranscriptionFailed) + return + } + + plan := []progress.Planned{{Key: "received", Label: "Received"}, {Key: "audio", Label: "Preparing the audio"}} + planner, planned := w.tr.(Planner) + if planned { + plan = append(plan, planner.Plan()...) + } else { + plan = append(plan, progress.Planned{Key: "transcribe", Label: "Transcribing"}) + } + plan = append(plan, progress.Planned{Key: "save", Label: "Saving the transcript"}) + tr := progress.NewTracker(ctx, jobSink{st: w.store, userID: job.UserID, id: job.UploadID}, log, plan) + tr.Start() + tr.Begin("received", 0) + tr.Done("received") + + hbCtx, stopBeat := context.WithCancel(ctx) + hbDone := make(chan struct{}) + go func() { + defer close(hbDone) + tr.Heartbeat(hbCtx, w.beatEvery, w.stillEvery) + }() + defer func() { stopBeat(); <-hbDone }() // no heartbeat outlives its job + + // abort records where the job stopped. A shutdown (the parent context ended) is not + // a failure: the row is left for the start-up sweep, so the step stays as it was. + abort := func(kind domain.UploadErrorKind) { + if ctx.Err() == nil { + tr.FailRunning() + } + w.fail(ctx, job, log, kind) + } + + wav, err := w.tempWAV() + if err != nil { + log.Error("upload: temp wav", "kind", errkind.Of(err)) + abort(domain.UploadErrTranscriptionFailed) + return + } + defer func() { _ = os.Remove(wav) }() + + jobCtx, cancel := context.WithTimeout(ctx, w.timeout) + defer cancel() + tr.Begin("audio", 0) + if err := w.conv.ToWAV(jobCtx, job.Path, wav); err != nil { + log.Error("upload: convert", "kind", errkind.Of(err)) + abort(domain.UploadErrTranscriptionFailed) + return + } + got, tooLong := w.audioLengthProblem(wav) + if tooLong { + abort(domain.UploadErrTooLong) + log.Error("upload: converted audio is over the limit", "audio_s", int(got.Seconds()), "limit_s", int(w.maxDuration.Seconds())) + return + } + if w.maxDuration > 0 && job.Duration > 0 { + if note := shortfallNote(got, job.Duration); note != "" { + log.Warn("upload: audio is much shorter than the file claimed", "audio_s", int(got.Seconds()), "claimed_s", int(job.Duration.Seconds())) + tr.Annotate("audio", note) + } + } + tr.Done("audio") + + if !planned { + tr.Begin("transcribe", 0) + } + t, err := w.tr.Transcribe(progress.With(jobCtx, tr), wav) + if err != nil { + log.Error("upload: transcribe", "kind", errkind.Of(err), "elapsed", time.Since(start).Round(time.Second)) + abort(domain.UploadErrTranscriptionFailed) + return + } + if !planned { + tr.Done("transcribe") + } + + tr.Begin("save", 0) + if err := w.store.CompleteUpload(ctx, job.UserID, job.UploadID, t); err != nil { + log.Error("upload: store transcript", "kind", errkind.Of(err)) + abort(domain.UploadErrTranscriptionFailed) + return + } + tr.Done("save") + tr.Finish() + log.Info("upload transcribed", "endpoint", t.Endpoint, "audio_s", t.AudioSeconds, + "segments", len(t.Segments), "elapsed", time.Since(start).Round(time.Second)) +} + +// fail records a transcription failure — unless the PARENT context ended, which +// is a shutdown, not a failure: the row is left for the start-up sweep. +func (w *Worker) fail(ctx context.Context, job Job, log *slog.Logger, kind domain.UploadErrorKind) { + if ctx.Err() != nil { + return + } + if err := w.store.FailUpload(ctx, job.UserID, job.UploadID, kind); err != nil { + log.Error("upload: record failure", "kind", errkind.Of(err)) + } +} + +func (w *Worker) tempWAV() (string, error) { + f, err := os.CreateTemp(w.tmpDir, "wav-*") + if err != nil { + return "", fmt.Errorf("uploads: temp wav: %w", err) + } + name := f.Name() + if err := f.Close(); err != nil { + _ = os.Remove(name) + return "", fmt.Errorf("uploads: temp wav: %w", err) + } + return name, nil +} + +const ( + // tooLongSlack is how far over the limit the converted audio may be before the claim + // in the file's header is called a lie: ffmpeg rounds. + tooLongSlack = 30 * time.Second + // A file counts as truncated when it converts to under 80% of the length it claimed + // AND more than two minutes short. Probe estimates for variable-bitrate files are + // imprecise; refusing real recordings would be worse than the problem. + incompleteBelow = 0.8 + incompleteShortfall = 2 * time.Minute +) + +// audioLengthProblem measures the converted WAV (16 kHz mono 16-bit PCM is 32,000 bytes a +// second after a 44-byte header) against the limit. It returns the real length and whether +// that is over the limit by more than the tolerance. +func (w *Worker) audioLengthProblem(wav string) (got time.Duration, tooLong bool) { + if w.maxDuration <= 0 { + return 0, false + } + info, err := os.Stat(wav) + if err != nil { + return 0, false + } + got = time.Duration(float64(max(info.Size()-44, 0)) / 32000 * float64(time.Second)) + return got, got > w.maxDuration+tooLongSlack +} + +// shortfallNote is a remark for the audio step when the converted audio is much shorter +// than the file claimed, or "" when it is not. It is a NOTE and never a failure: a +// container header can over-estimate a complete file (a variable-bitrate MP3 without a Xing +// header claimed 79 minutes for 20 real ones), so refusing would turn away real +// recordings, while saying nothing about a truncated one would hide a partial transcript. +func shortfallNote(got, claimed time.Duration) string { + if float64(got) >= incompleteBelow*float64(claimed) || claimed-got <= incompleteShortfall { + return "" + } + return fmt.Sprintf("The audio is %s long; the file claimed %s. If the recording was cut off, upload the complete one.", + FormatDuration(got), FormatDuration(claimed)) +} diff --git a/internal/uploads/worker_duration_test.go b/internal/uploads/worker_duration_test.go new file mode 100644 index 0000000..8d6ede6 --- /dev/null +++ b/internal/uploads/worker_duration_test.go @@ -0,0 +1,187 @@ +package uploads_test + +import ( + "context" + "os" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// The duration limit is a claim in the file's header. The worker measures what was +// really converted and refuses both a recording that is longer than the limit (the +// header lied low) and one that ends far earlier than the file claimed (it was truncated +// and would otherwise yield a partial transcript with no warning) -- adversarial review F2. + +// sizedConverter writes a (sparse) 16 kHz mono PCM WAV of the given length, without +// actually allocating it: 32,000 bytes per second plus a 44-byte header. +type sizedConverter struct{ seconds float64 } + +func (c sizedConverter) ToWAV(_ context.Context, _, dst string) error { + f, err := os.Create(dst) + if err != nil { + return err + } + defer func() { _ = f.Close() }() + return f.Truncate(44 + int64(c.seconds*32000)) +} + +func durationWorker(t *testing.T, r *wrig, wavSeconds float64, max time.Duration) *uploads.Worker { + t.Helper() + opts := []uploads.WorkerOption{} + if max > 0 { + opts = append(opts, uploads.WithMaxDuration(max)) + } + return uploads.NewWorker(r.st, sizedConverter{seconds: wavSeconds}, r.tr, r.tmp, time.Minute, opts...) +} + +func jobWith(r *wrig, t *testing.T, claimed time.Duration) uploads.Job { + j := r.spool(t, "a") + j.Duration = claimed + return j +} + +func TestWorkerDuration_AConvertedLengthOverTheLimitFailsTheJobBeforeAnyEngineSeesIt(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, 100*60, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, 10*time.Second)) + + _, failed := r.st.snapshot() + assert.Equal(t, domain.UploadErrTooLong, failed["a"], "a header that claimed 10 s must not let 100 minutes through") + assert.Empty(t, r.tr.paths, "no engine may be sent audio that is over the limit") + assert.Empty(t, r.leftovers(t), "the audio is still deleted") +} + +func TestWorkerDuration_JustOverTheLimitWithinTheSlackStillTranscribes(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, 90*60+20, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, 90*time.Minute)) + + completed, failed := r.st.snapshot() + assert.Empty(t, failed) + assert.Contains(t, completed, "a", "ffmpeg rounds: 20 s over a 90-minute limit is not a lie") +} + +func audioStepNote(r *wrig) string { + for _, st := range r.st.lastSteps() { + if st.Key == "audio" { + return st.Note + } + } + return "" +} + +func TestWorkerDuration_AFileThatEndsFarEarlierThanItClaimedIsStillTranscribedAndSaysSo(t *testing.T) { + // A container header can over-estimate a complete file (VBR MP3 without a Xing header + // claimed 79 minutes for 20 real ones). Refusing it was wrong; staying silent about a + // truncated one is wrong too. So it is transcribed, and the audio step carries a note. + r := newWRig(t, time.Minute) + w := durationWorker(t, r, 20*60, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, 79*time.Minute)) + + completed, failed := r.st.snapshot() + assert.Empty(t, failed, "a complete recording with a bad header estimate must not be refused") + assert.Contains(t, completed, "a") + note := audioStepNote(r) + assert.Contains(t, note, "20 min", "what was really transcribed") + assert.Contains(t, note, "1 h 19 min", "what the file claimed") + assert.Contains(t, note, "cut off", "and what to do if that is not a surprise") +} + +func TestWorkerDuration_TheShortfallNoteIsPinnedAtEightyPercentAndTwoMinutes(t *testing.T) { + tests := []struct { + name string + claimed time.Duration + wav float64 + noted bool + }{ + {"79 percent of an hour", 60 * time.Minute, 0.79 * 3600, true}, + {"exactly 80 percent of an hour", 60 * time.Minute, 2880, false}, + {"one second under 80 percent of an hour", 60 * time.Minute, 2879, true}, + {"81 percent of an hour", 60 * time.Minute, 0.81 * 3600, false}, + {"exactly two minutes short and under 80 percent", 6 * time.Minute, 4 * 60, false}, + {"two minutes and a second short and under 80 percent", 6 * time.Minute, 4*60 - 1, true}, + {"unknown claimed length", 0, 20 * 60, false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, tt.wav, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, tt.claimed)) + + assert.Equal(t, tt.noted, audioStepNote(r) != "") + completed, _ := r.st.snapshot() + assert.Contains(t, completed, "a", "a note never stops the job") + }) + } +} + +func TestWorkerDuration_TheTooLongToleranceIsExactlyThirtySeconds(t *testing.T) { + tests := []struct { + name string + wav float64 + fail bool + }{ + {"at the limit plus 30 s", 90*60 + 30, false}, + {"at the limit plus 31 s", 90*60 + 31, true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, tt.wav, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, 90*time.Minute)) + + _, failed := r.st.snapshot() + if tt.fail { + assert.Equal(t, domain.UploadErrTooLong, failed["a"]) + } else { + assert.Empty(t, failed) + } + }) + } +} + +func TestWorkerDuration_AnEstimateThatIsMerelyALittleOffIsNotTruncation(t *testing.T) { + tests := []struct { + name string + claimed time.Duration + wav float64 + }{ + {"five percent short", 60 * time.Minute, 57 * 60}, + {"under two minutes short of a long file", 60 * time.Minute, 59 * 60}, + {"unknown claimed length", 0, 20 * 60}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, tt.wav, 90*time.Minute) + + w.Process(context.Background(), jobWith(r, t, tt.claimed)) + + completed, failed := r.st.snapshot() + assert.Empty(t, failed) + assert.Contains(t, completed, "a") + }) + } +} + +func TestWorkerDuration_WithNoMaximumConfiguredNothingIsMeasured(t *testing.T) { + r := newWRig(t, time.Minute) + w := durationWorker(t, r, 100*60, 0) + + w.Process(context.Background(), jobWith(r, t, 0)) + + completed, _ := r.st.snapshot() + assert.Contains(t, completed, "a") + require.NotNil(t, w) +} diff --git a/internal/uploads/worker_progress_test.go b/internal/uploads/worker_progress_test.go new file mode 100644 index 0000000..58faf8c --- /dev/null +++ b/internal/uploads/worker_progress_test.go @@ -0,0 +1,215 @@ +package uploads_test + +import ( + "bytes" + "context" + "errors" + "log/slog" + "strings" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/progress" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// Progress, heartbeat and logging of the worker (ADR-035, #42, #43). + +// plannedTr is a transcriber that declares its own phases and reports them, like the +// two-engine Ensemble does. +type plannedTr struct{ *fakeTranscriber } + +func (plannedTr) Plan() []progress.Planned { + return []progress.Planned{{Key: "engine_a", Label: "Transcribing with A"}, {Key: "engine_b", Label: "Transcribing with B"}} +} + +func (p plannedTr) Transcribe(ctx context.Context, wav string) (domain.Transcription, error) { + r := progress.From(ctx) + r.Begin("engine_a", 0) + r.Done("engine_a") + r.Begin("engine_b", 0) + r.Done("engine_b") + return p.fakeTranscriber.Transcribe(ctx, wav) +} + +func keysAndStates(steps []domain.ProgressStep) []string { + var out []string + for _, s := range steps { + out = append(out, s.Key+":"+string(s.State)) + } + return out +} + +// syncLog is a log sink safe to read while the worker's goroutines write. +type syncLog struct { + mu sync.Mutex + b bytes.Buffer +} + +func (l *syncLog) Write(p []byte) (int, error) { + l.mu.Lock() + defer l.mu.Unlock() + return l.b.Write(p) +} + +func (l *syncLog) String() string { + l.mu.Lock() + defer l.mu.Unlock() + return l.b.String() +} + +func newProgressWorker(t *testing.T, r *wrig, tr uploads.Transcriber, opts ...uploads.WorkerOption) *uploads.Worker { + t.Helper() + return uploads.NewWorker(r.st, r.conv, tr, r.tmp, time.Minute, opts...) +} + +func TestWorkerProgress_ASingleEngineJobShowsEveryPhaseDone(t *testing.T) { + r := newWRig(t, time.Minute) + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Equal(t, []string{"received:done", "audio:done", "transcribe:done", "save:done"}, keysAndStates(r.st.lastSteps())) +} + +func TestWorkerProgress_ATranscriberThatPlansItsOwnPhasesGetsThemBetweenAudioAndSaving(t *testing.T) { + r := newWRig(t, time.Minute) + w := newProgressWorker(t, r, plannedTr{r.tr}) + + w.Process(context.Background(), r.spool(t, "a")) + + assert.Equal(t, []string{"received:done", "audio:done", "engine_a:done", "engine_b:done", "save:done"}, keysAndStates(r.st.lastSteps())) +} + +func TestWorkerProgress_AFailureMarksWhereItStoppedAndLeavesTheRestPending(t *testing.T) { + tests := []struct { + name string + setup func(*wrig) + want []string + }{ + { + "conversion fails", func(r *wrig) { r.conv.err = errors.New("moov atom not found") }, + []string{"received:done", "audio:failed", "transcribe:pending", "save:pending"}, + }, + { + "every engine fails", func(r *wrig) { r.tr.err = errors.New("iguana down") }, + []string{"received:done", "audio:done", "transcribe:failed", "save:pending"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + tt.setup(r) + + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Equal(t, tt.want, keysAndStates(r.st.lastSteps())) + }) + } +} + +func TestWorkerProgress_AShutdownIsNotRecordedAsAFailedStep(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + go func() { defer close(done); r.w.Process(ctx, r.spool(t, "a")) }() + require.Eventually(t, func() bool { + return strings.Contains(strings.Join(keysAndStates(r.st.lastSteps()), ","), "transcribe:running") + }, 2*time.Second, time.Millisecond) + + cancel() + <-done + + assert.Contains(t, keysAndStates(r.st.lastSteps()), "transcribe:running", + "a shutdown leaves the row for the start-up sweep; it must not pose as a failure of the step") +} + +func TestWorkerProgress_TheHeartbeatBeatsWhileAnEngineCallIsStuckAndStopsWhenTheJobEnds(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + r.st.beatDelay = 20 * time.Millisecond + w := newProgressWorker(t, r, r.tr, uploads.WithHeartbeat(5*time.Millisecond, time.Hour)) + done := make(chan struct{}) + go func() { defer close(done); w.Process(context.Background(), r.spool(t, "a")) }() + + require.Eventually(t, func() bool { return r.st.beatCount() >= 3 }, 2*time.Second, 5*time.Millisecond, + "the page tells a slow engine from a dead service by this heartbeat, so it must not wait on the engine") + close(r.tr.block) + <-done + after := r.st.beatCount() + time.Sleep(40 * time.Millisecond) + + assert.Equal(t, after, r.st.beatCount(), "no heartbeat may outlive its job") +} + +func TestWorkerProgress_OnASuccessfulJobTheLogNamesEveryPhaseInOrderAndNeverTheTranscriptOrThePath(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.result = domain.Transcription{Text: "zebrakaka secret meeting words", Endpoint: "iguana/kb-whisper", AudioSeconds: 3} + buf := &syncLog{} + w := newProgressWorker(t, r, r.tr, uploads.WithLogger(slog.New(slog.NewTextHandler(buf, nil)))) + + w.Process(context.Background(), r.spool(t, "a")) + + out := buf.String() + audio := strings.Index(out, "step=audio") + transcribe := strings.Index(out, "step=transcribe") + save := strings.Index(out, "step=save") + require.True(t, audio >= 0 && transcribe > audio && save > transcribe, "phases must be logged in the order they ran:\n%s", out) + assert.Contains(t, out, "upload=a", "every line says which upload it is about") + assert.NotContains(t, out, "zebrakaka", "no transcript text in the log") + assert.NotContains(t, out, r.tmp, "no spool path, which carries the upload's identity, in the log") + assert.Contains(t, out, "upload transcribed", "the job's own summary line") + assert.Contains(t, out, "endpoint=iguana/kb-whisper", "which endpoint served it is a quality fact") +} + +func TestWorkerProgress_AFailureLogsAKindNeverTheErrorTextOrThePath(t *testing.T) { + tests := []struct { + name string + setup func(*wrig) + line string + }{ + {"conversion fails and names the spool file", func(r *wrig) { + r.conv.err = errors.New("Error opening input file " + r.tmp + "/upload-a: Invalid data zebrakaka") + }, "upload: convert"}, + {"every engine fails and the gateway echoed a word", func(r *wrig) { + r.tr.err = errors.New("gateway echoed: zebrakaka secret words") + }, "upload: transcribe"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + tt.setup(r) + buf := &syncLog{} + w := newProgressWorker(t, r, r.tr, uploads.WithLogger(slog.New(slog.NewTextHandler(buf, nil)))) + + w.Process(context.Background(), r.spool(t, "a")) + + out := buf.String() + assert.NotContains(t, out, "zebrakaka", "provider text or file contents in the log") + assert.NotContains(t, out, r.tmp, "the spool path in the log") + assert.Contains(t, out, tt.line, "the failure must be logged at all, not only scrubbed") + assert.Contains(t, out, "kind=error") + assert.Contains(t, out, "upload=a") + }) + } +} + +func TestWorkerProgress_AStoreFailureLogsItsSQLStateNeverTheValueItQuotes(t *testing.T) { + // A Postgres error can quote the offending value; for a transcript row that is the + // transcript (fourth adversarial review). The log gets the SQLSTATE. + r := newWRig(t, time.Minute) + r.st.completeErr = dbErr{} + buf := &syncLog{} + w := newProgressWorker(t, r, r.tr, uploads.WithLogger(slog.New(slog.NewTextHandler(buf, nil)))) + + w.Process(context.Background(), r.spool(t, "a")) + + out := buf.String() + assert.NotContains(t, out, "zebrakaka") + assert.Contains(t, out, "upload: store transcript") + assert.Contains(t, out, "kind=sql_22P02") +} diff --git a/internal/uploads/worker_queue_test.go b/internal/uploads/worker_queue_test.go new file mode 100644 index 0000000..0fa613c --- /dev/null +++ b/internal/uploads/worker_queue_test.go @@ -0,0 +1,108 @@ +package uploads_test + +import ( + "context" + "errors" + "strconv" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// The queue was append-only: 100,000 submits with the worker idle all succeeded, and each +// queued job holds up to the upload cap on disk until its turn (adversarial review F6). + +func queueWorker(t *testing.T, total, perUser int) *uploads.Worker { + t.Helper() + r := newWRig(t, time.Minute) + return uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, time.Minute, uploads.WithQueueLimits(total, perUser)) +} + +func submit(w *uploads.Worker, user string, n int) error { + return w.Submit(uploads.Job{UserID: user, UploadID: user + "-" + strconv.Itoa(n), Path: "/nonexistent"}) +} + +func TestWorkerSubmit_RefusesBeyondTheTotalQueueCap(t *testing.T) { + w := queueWorker(t, 3, 10) + + for i := 0; i < 3; i++ { + require.NoError(t, submit(w, "u"+strconv.Itoa(i), i)) + } + + err := submit(w, "u9", 9) + assert.ErrorIs(t, err, uploads.ErrQueueFull) +} + +func TestWorkerSubmit_OneUserCannotFillTheQueueForEveryoneElse(t *testing.T) { + w := queueWorker(t, 100, 2) + + require.NoError(t, submit(w, "alice", 1)) + require.NoError(t, submit(w, "alice", 2)) + errAlice := submit(w, "alice", 3) + errBob := submit(w, "bob", 1) + + assert.True(t, errors.Is(errAlice, uploads.ErrQueueFull), "a third waiting upload from one user is refused") + assert.NoError(t, errBob, "another user is not affected by alice's queue") +} + +func TestWorkerSubmit_ThereIsADefaultCapSoANeglectedWorkerCannotBeFilledForever(t *testing.T) { + r := newWRig(t, time.Minute) + w := uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, time.Minute) + + refused := 0 + for i := 0; i < 1000; i++ { + if err := submit(w, "u"+strconv.Itoa(i), i); errors.Is(err, uploads.ErrQueueFull) { + refused++ + } + } + + assert.Positive(t, refused, "with the defaults, 1,000 submits to an idle worker must not all be accepted") +} + +func TestWorkerSubmit_AJobThatHasLeftTheQueueFreesItsPlace(t *testing.T) { + r := newWRig(t, time.Minute) + w := uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, time.Minute, uploads.WithQueueLimits(1, 1)) + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + go func() { defer close(done); w.Run(ctx) }() + defer func() { cancel(); <-done }() + + job := r.spool(t, "first") + require.NoError(t, w.Submit(job)) + require.Eventually(t, func() bool { c, _ := r.st.snapshot(); _, ok := c["first"]; return ok }, + 5*time.Second, 5*time.Millisecond, "the first job never completed") + + assert.NoError(t, w.Submit(r.spool(t, "second")), "the cap counts jobs waiting, not jobs ever submitted") +} + +func TestWorkerHasRoom_SaysWhetherASubmitWouldBeAccepted(t *testing.T) { + w := queueWorker(t, 3, 2) + assert.True(t, w.HasRoom("alice")) + require.NoError(t, submit(w, "alice", 1)) + require.NoError(t, submit(w, "alice", 2)) + + assert.False(t, w.HasRoom("alice"), "alice is at her share") + assert.True(t, w.HasRoom("bob"), "bob is not") + require.NoError(t, submit(w, "bob", 1)) + assert.False(t, w.HasRoom("carol"), "the queue as a whole is at its cap") +} + +func TestWorkerHasRoom_AgreesWithSubmitAtEveryStepAndIsFalseOnceStopped(t *testing.T) { + w := queueWorker(t, 2, 1) + for i, user := range []string{"a", "a", "b", "c"} { + room := w.HasRoom(user) + err := submit(w, user, i) + assert.Equal(t, room, err == nil, "step %d (%s): HasRoom=%v but Submit returned %v", i, user, room, err) + } + + r := newWRig(t, time.Minute) + stopped := uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, time.Minute) + ctx, cancel := context.WithCancel(context.Background()) + cancel() + stopped.Run(ctx) + assert.False(t, stopped.HasRoom("a"), "a worker that has stopped has no room") +} diff --git a/internal/uploads/worker_test.go b/internal/uploads/worker_test.go new file mode 100644 index 0000000..04b9abe --- /dev/null +++ b/internal/uploads/worker_test.go @@ -0,0 +1,334 @@ +package uploads_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +type wStore struct { + mu sync.Mutex + started []string + completed map[string]domain.Transcription + failed map[string]domain.UploadErrorKind + startErr error + // completeErr makes CompleteUpload fail, as a database outage would. + completeErr error + saves [][]domain.ProgressStep + beats int + // beatDelay makes each heartbeat write slow, so one is still in flight when a job + // ends -- which is what proves the worker waits for its heartbeat to finish. + beatDelay time.Duration +} + +func newWStore() *wStore { + return &wStore{completed: map[string]domain.Transcription{}, failed: map[string]domain.UploadErrorKind{}} +} + +func (s *wStore) StartTranscribing(_ context.Context, _, id string) error { + s.mu.Lock() + defer s.mu.Unlock() + s.started = append(s.started, id) + return s.startErr +} + +func (s *wStore) CompleteUpload(_ context.Context, _, id string, t domain.Transcription) error { + s.mu.Lock() + defer s.mu.Unlock() + if s.completeErr != nil { + return s.completeErr + } + s.completed[id] = t + return nil +} + +func (s *wStore) FailUpload(_ context.Context, _, id string, k domain.UploadErrorKind) error { + s.mu.Lock() + defer s.mu.Unlock() + s.failed[id] = k + return nil +} + +func (s *wStore) SaveProgress(_ context.Context, _, _ string, steps []domain.ProgressStep, _ time.Time) error { + s.mu.Lock() + defer s.mu.Unlock() + s.saves = append(s.saves, append([]domain.ProgressStep(nil), steps...)) + return nil +} + +func (s *wStore) Heartbeat(context.Context, string, string) error { + time.Sleep(s.beatDelay) + s.mu.Lock() + defer s.mu.Unlock() + s.beats++ + return nil +} + +func (s *wStore) lastSteps() []domain.ProgressStep { + s.mu.Lock() + defer s.mu.Unlock() + if len(s.saves) == 0 { + return nil + } + return s.saves[len(s.saves)-1] +} + +func (s *wStore) beatCount() int { + s.mu.Lock() + defer s.mu.Unlock() + return s.beats +} + +func (s *wStore) snapshot() (map[string]domain.Transcription, map[string]domain.UploadErrorKind) { + s.mu.Lock() + defer s.mu.Unlock() + c := map[string]domain.Transcription{} + f := map[string]domain.UploadErrorKind{} + for k, v := range s.completed { + c[k] = v + } + for k, v := range s.failed { + f[k] = v + } + return c, f +} + +type fakeConverter struct { + err error + srcs []string + dsts []string +} + +func (c *fakeConverter) ToWAV(_ context.Context, src, dst string) error { + c.srcs = append(c.srcs, src) + c.dsts = append(c.dsts, dst) + if c.err != nil { + return c.err + } + return os.WriteFile(dst, []byte("wav"), 0o600) +} + +type fakeTranscriber struct { + result domain.Transcription + err error + paths []string + // block, when non-nil, holds every call until it is closed or ctx ends. + block chan struct{} + running atomic.Int32 + maxSeen atomic.Int32 +} + +func (f *fakeTranscriber) Transcribe(ctx context.Context, wav string) (domain.Transcription, error) { + f.paths = append(f.paths, wav) + n := f.running.Add(1) + defer f.running.Add(-1) + for { + m := f.maxSeen.Load() + if n <= m || f.maxSeen.CompareAndSwap(m, n) { + break + } + } + if f.block != nil { + select { + case <-f.block: + case <-ctx.Done(): + return domain.Transcription{}, ctx.Err() + } + } + return f.result, f.err +} + +type wrig struct { + w *uploads.Worker + st *wStore + conv *fakeConverter + tr *fakeTranscriber + tmp string +} + +func newWRig(t *testing.T, timeout time.Duration) *wrig { + t.Helper() + r := &wrig{ + st: newWStore(), + conv: &fakeConverter{}, + tr: &fakeTranscriber{result: domain.Transcription{Text: "synthetic text", Endpoint: "iguana/kb-whisper", AudioSeconds: 3}}, + tmp: t.TempDir(), + } + r.w = uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, timeout) + return r +} + +// spool writes a fake uploaded file (what Intake would hand over) and returns a Job. +func (r *wrig) spool(t *testing.T, id string) uploads.Job { + t.Helper() + p := filepath.Join(r.tmp, "upload-"+id) + require.NoError(t, os.WriteFile(p, []byte("audio"), 0o600)) + return uploads.Job{UserID: "u1", UploadID: id, Path: p} +} + +func (r *wrig) leftovers(t *testing.T) []string { + t.Helper() + ents, err := os.ReadDir(r.tmp) + require.NoError(t, err) + var n []string + for _, e := range ents { + n = append(n, e.Name()) + } + return n +} + +func TestWorkerProcess_SuccessConvertsTranscribesStoresAndDeletesTheAudio(t *testing.T) { + r := newWRig(t, time.Minute) + job := r.spool(t, "a") + + r.w.Process(context.Background(), job) + + c, f := r.st.snapshot() + assert.Equal(t, []string{"a"}, r.st.started) + assert.Equal(t, "synthetic text", c["a"].Text) + assert.Equal(t, "iguana/kb-whisper", c["a"].Endpoint) + assert.Empty(t, f) + require.Len(t, r.conv.srcs, 1) + assert.Equal(t, job.Path, r.conv.srcs[0], "the converter reads the uploaded file") + assert.Equal(t, r.conv.dsts, r.tr.paths, "whisper is sent the converted WAV, not the original") + assert.Equal(t, r.tmp, filepath.Dir(r.conv.dsts[0])) + assert.Empty(t, r.leftovers(t), "the audio is never retained: both the upload and the WAV are deleted") +} + +func TestWorkerProcess_FailuresAreTypedAndStillDeleteTheAudio(t *testing.T) { + tests := []struct { + name string + setup func(*wrig) + }{ + {"conversion fails", func(r *wrig) { r.conv.err = errors.New("moov atom not found") }}, + {"every transcription endpoint fails", func(r *wrig) { r.tr.err = errors.New("iguana down; berget down") }}, + {"storing the finished transcript fails", func(r *wrig) { r.st.completeErr = errors.New("db down") }}, + {"the temporary WAV cannot be created", func(r *wrig) { + r.w = uploads.NewWorker(r.st, r.conv, r.tr, filepath.Join(r.tmp, "missing"), time.Minute) + }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + tt.setup(r) + job := r.spool(t, "a") + + r.w.Process(context.Background(), job) + + c, f := r.st.snapshot() + assert.Empty(t, c) + assert.Equal(t, domain.UploadErrTranscriptionFailed, f["a"]) + assert.Empty(t, r.leftovers(t)) + }) + } + t.Run("conversion failure never reaches whisper", func(t *testing.T) { + r := newWRig(t, time.Minute) + r.conv.err = errors.New("bad") + + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Empty(t, r.tr.paths) + }) +} + +func TestWorkerProcess_ShutdownMidJobLeavesTheRowForTheStartupSweep(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + job := r.spool(t, "a") + + go func() { r.w.Process(ctx, job); close(done) }() + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + cancel() + <-done + + _, f := r.st.snapshot() + assert.Empty(t, f, "a shutdown is not a transcription failure: the row stays for FailInterruptedUploads to mark interrupted") + assert.Empty(t, r.leftovers(t), "but the audio is still deleted") +} + +func TestWorkerProcess_AJobThatOutlivesItsTimeoutIsFailedNotStuck(t *testing.T) { + r := newWRig(t, 50*time.Millisecond) + r.tr.block = make(chan struct{}) // never released + done := make(chan struct{}) + + go func() { r.w.Process(context.Background(), r.spool(t, "a")); close(done) }() + select { + case <-done: + case <-time.After(5 * time.Second): + t.Fatal("Process did not return: a hung endpoint was not bounded by the job timeout") + } + + _, f := r.st.snapshot() + assert.Equal(t, domain.UploadErrTranscriptionFailed, f["a"], "a hung endpoint must end in failed, not an eternal spinner (ADR-025)") + assert.Empty(t, r.leftovers(t)) +} + +func TestWorkerProcess_StartFailureDropsTheJobAndDeletesTheAudio(t *testing.T) { + r := newWRig(t, time.Minute) + r.st.startErr = errors.New("db down") + + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Empty(t, r.tr.paths, "never transcribe a job whose row could not be moved to transcribing") + _, f := r.st.snapshot() + assert.Contains(t, f, "a") + assert.Empty(t, r.leftovers(t)) +} + +func TestWorkerRun_ProcessesOneJobAtATime(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + go r.w.Run(ctx) + + for _, id := range []string{"a", "b", "c"} { + require.NoError(t, r.w.Submit(r.spool(t, id))) + } + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + time.Sleep(50 * time.Millisecond) // give a (wrong) second job time to start + assert.Equal(t, int32(1), r.tr.running.Load(), "two simultaneous long jobs on iguana were never measured: concurrency is 1") + + close(r.tr.block) + require.Eventually(t, func() bool { c, _ := r.st.snapshot(); return len(c) == 3 }, 3*time.Second, 5*time.Millisecond) + assert.Equal(t, int32(1), r.tr.maxSeen.Load()) +} + +func TestWorkerRun_StoppingDeletesQueuedAudioAndRefusesNewJobs(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + stopped := make(chan struct{}) + go func() { r.w.Run(ctx); close(stopped) }() + require.NoError(t, r.w.Submit(r.spool(t, "a"))) + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + require.NoError(t, r.w.Submit(r.spool(t, "b"))) // queued behind the running job + + cancel() + <-stopped + + assert.Len(t, r.conv.srcs, 1, "a job still queued at shutdown must never be started: only the running job was converted") + assert.Empty(t, r.leftovers(t), "audio of queued jobs is deleted on stop: the rows are swept to interrupted at next start") + assert.ErrorIs(t, r.w.Submit(r.spool(t, "c")), uploads.ErrStopped) +} + +// dbErr is an error shaped like pgx's: a message that quotes a value, and an SQLSTATE. +type dbErr struct{} + +func (dbErr) Error() string { + return `ERROR: invalid input syntax for type json: "zebrakaka secret words"` +} +func (dbErr) SQLState() string { return "22P02" } diff --git a/internal/web/chat.go b/internal/web/chat.go index b8e2d80..71aaa31 100644 --- a/internal/web/chat.go +++ b/internal/web/chat.go @@ -6,6 +6,8 @@ import ( "net/http" "strings" + "github.com/a-h/templ" + "git.d-ma.be/mathias/tapir/internal/adapters/chat" "git.d-ma.be/mathias/tapir/internal/adapters/store" ) @@ -43,6 +45,28 @@ type chatView struct { History []chat.Turn Truncated bool Error string + // PostURL and Subject let the same chat panel serve an UPLOADED recording + // (ADR-033): where the form posts, and what the transcript is called. Empty + // means a YouTube video (/v/{VideoID}/chat, "video"), so existing callers are + // unchanged. + PostURL templ.SafeURL + Subject string +} + +// postURL is where the chat form posts. +func (v chatView) postURL() templ.SafeURL { + if v.PostURL != "" { + return v.PostURL + } + return chatURL(v.VideoID) +} + +// subject is what the transcript is called in the panel's wording. +func (v chatView) subject() string { + if v.Subject != "" { + return v.Subject + } + return "video" } // handleChat renders the chat page for a summarized video (GET). Entry is scoped diff --git a/internal/web/handlers.go b/internal/web/handlers.go index bbf17d0..bd81d03 100644 --- a/internal/web/handlers.go +++ b/internal/web/handlers.go @@ -13,6 +13,7 @@ import ( "git.d-ma.be/mathias/tapir/internal/adapters/store" "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" ) // Store is the read/write surface the web handlers depend on — a narrow port over @@ -53,6 +54,12 @@ type Store interface { // the most recent discovery pass, shown on the account page as a warning. ListChannelErrors(ctx context.Context, userID string) ([]store.ChannelError, error) + // Uploaded-audio transcripts (ADR-033): the caller's own PRIVATE uploads, + // stored apart from the shared public captions. Both are user-scoped; a + // non-owner's id is ErrUploadNotFound. + GetUpload(ctx context.Context, userID, id string) (*domain.UploadedTranscript, error) + ListUploads(ctx context.Context, userID string) ([]domain.UploadedTranscript, error) + // SetTranscriptStatus clears or updates a video's transcript backoff state. // Used by handleRetryNow to clear rate_limited_at before immediate processing. SetTranscriptStatus(ctx context.Context, userID, videoID, status string) error @@ -100,6 +107,26 @@ type App struct { // summary view shows no "dig deeper" affordance. It holds no caption-fetch // dependency, so an enabled chat cannot reach YouTube or the rate gate. Chat Chatter + // Uploads, when non-nil, accepts audio recordings for private transcription + // (ADR-033). Nil = the /uploads routes are not mounted. UploadLimits are the + // same limits the Uploader enforces, shown on the form so they are stated up + // front rather than discovered by failing. + Uploads Uploader + UploadLimits uploads.Limits + // UploadChatMaxChars is the transcript budget for chat over an uploaded + // recording. The caption default (18,000 chars, ~22 minutes of speech) would + // keep only the first third of an hour-long meeting. + UploadChatMaxChars int + // UploadChatModel is the model chat over an uploaded recording opens with. Empty + // means the Chatter's own default. It exists because that default is the + // summarizer's (koala/phi4-mini, 8,192-token context) and cannot hold an hour of + // speech. + UploadChatModel string + // UploadChatModels is the set of models chat over an uploaded recording offers and + // accepts. The whole private transcript goes to the one chosen, so it must hold + // LOCAL models only, and the caller decides that. Nil or empty fails closed: no + // model is offered and none is called (ADR-035 review). + UploadChatModels []string // Processing tracks in-flight immediate summarizations so the status endpoint // shows the animation until the summary lands. The zero value is ready to use. Processing ProcessingSet @@ -166,6 +193,20 @@ func (a *App) Router() http.Handler { app.HandleFunc("GET /v/{videoId}/chat", a.handleChat) app.HandleFunc("POST /v/{videoId}/chat", a.handleChatMessage) } + // Audio upload → private transcript (ADR-033). Mounted only when an Uploader is + // wired; every handler is user-scoped, so another user's upload is a 404. + if a.Uploads != nil { + app.HandleFunc("GET /uploads", a.handleUploadsPage) + app.HandleFunc("POST /uploads", a.handleUploadPost) + app.HandleFunc("GET /uploads/{id}", a.handleUploadDetail) + app.HandleFunc("GET /uploads/{id}/status", a.handleUploadStatus) + // Chat over the uploaded transcript reuses the ADR-027 Chatter; mounted only + // when one is wired. + if a.Chat != nil { + app.HandleFunc("GET /uploads/{id}/chat", a.handleUploadChat) + app.HandleFunc("POST /uploads/{id}/chat", a.handleUploadChatMessage) + } + } app.HandleFunc("GET /register", a.handleRegisterForm) app.HandleFunc("POST /register", a.handleRegister) diff --git a/internal/web/uploads.go b/internal/web/uploads.go new file mode 100644 index 0000000..1558523 --- /dev/null +++ b/internal/web/uploads.go @@ -0,0 +1,532 @@ +package web + +import ( + "context" + "errors" + "fmt" + "io" + "math" + "net/http" + "path/filepath" + "regexp" + "strings" + "time" + "unicode" + "unicode/utf8" + + "github.com/a-h/templ" + + "git.d-ma.be/mathias/tapir/internal/adapters/chat" + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/errkind" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// Uploader accepts an uploaded recording for transcription (ADR-033). +// *uploads.Intake satisfies it. It is the whole of what the handlers know about +// validation, spooling and queueing. +type Uploader interface { + Accept(ctx context.Context, userID, title string, src io.Reader) (id string, err error) +} + +const ( + // uploadOverhead is headroom above the file-size cap for the multipart framing + // and the title field; the cap itself is enforced on the file part by Intake. + uploadOverhead = 1 << 20 + maxTitleRunes = 200 + // uploadPollEvery is how often an unfinished upload re-asks for its status. + uploadPollEvery = "2s" + // serviceSilentAfter is how long a job may go without any signal (heartbeat or + // progress) before the page calls the service lost. The heartbeat is every 10 s, so + // 45 s is four missed beats (ADR-035). + serviceSilentAfter = 45 * time.Second + // countedQuietAfter is how long a COUNTED phase (the deciding one) may go without + // advancing before the page says it is quiet. Each decision has a two-minute timeout, + // so five minutes is more than two in a row. A phase with no count is never quiet. + countedQuietAfter = 5 * time.Minute +) + +// uploadsView is what the /uploads page renders. +type uploadsView struct { + Limits uploads.Limits + Rows []domain.UploadedTranscript + Error string // a synchronous rejection message, empty when none +} + +// handleUploadsPage renders the upload form and the caller's own uploads. +func (a *App) handleUploadsPage(w http.ResponseWriter, r *http.Request) { + userID, ok := a.currentUserID(w, r) + if !ok { + return + } + a.renderUploadsPage(w, r, userID, http.StatusOK, "") +} + +func (a *App) renderUploadsPage(w http.ResponseWriter, r *http.Request, userID string, status int, msg string) { + rows, err := a.Store.ListUploads(r.Context(), userID) + if err != nil { + a.serverError(w, r, "list uploads", err) + return + } + a.renderStatus(w, r, status, UploadsPage(uploadsView{Limits: a.UploadLimits, Rows: rows, Error: msg})) +} + +// handleUploadPost streams the multipart body: the optional title field, then the +// file part, which goes straight to Intake without being buffered in memory. A +// rejection is synchronous — a status code and a message on the upload page, no +// row, no timeout (ADR-033). +func (a *App) handleUploadPost(w http.ResponseWriter, r *http.Request) { + userID, ok := a.currentUserID(w, r) + if !ok { + return + } + r.Body = http.MaxBytesReader(w, r.Body, a.UploadLimits.MaxBytes+uploadOverhead) + mr, err := r.MultipartReader() + if err != nil { + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "Choose an audio file to upload.") + return + } + title := "" + for { + part, err := mr.NextPart() + if errors.Is(err, io.EOF) { + break + } + if err != nil { + var tooBig *http.MaxBytesError + if a.abandoned(w, r) { + return + } + if !errors.As(err, &tooBig) { + // A malformed multipart body (a header with a control byte, a truncated + // part) is the sender's to fix: a 400, not a server error, and the error + // text is never logged: it quotes the headers, which carry the file name. + a.logger().Info("upload rejected", "reason", "malformed") + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "That upload could not be read. Please try again.") + return + } + a.uploadFailure(w, r, userID, err) + return + } + switch part.FormName() { + case "title": + b, err := io.ReadAll(io.LimitReader(part, maxTitleRunes*utf8.UTFMax)) + if err != nil { + if a.abandoned(w, r) { + return + } + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "That upload could not be read. Please try again.") + return + } + title = cleanTitle(string(b)) + case "file": + if title == "" { + title = titleFromFilename(part.FileName()) + } + id, err := a.Uploads.Accept(r.Context(), userID, title, part) + if err != nil { + a.uploadFailure(w, r, userID, err) + return + } + http.Redirect(w, r, "/uploads/"+id, http.StatusSeeOther) + return + } + } + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "Choose an audio file to upload.") +} + +// abandoned reports, logs at INFO and answers 499, because the client has already gone: its request +// context is cancelled, so there is nobody to answer and rendering a page would run its +// database read on a dead context and log a false ERROR. A hang-up mid-upload is routine. +func (a *App) abandoned(w http.ResponseWriter, r *http.Request) bool { + if r.Context().Err() == nil { + return false + } + a.logger().Info("upload abandoned", "reason", "client_gone") + // 499 (the "client closed request" convention): a client that half-closed after sending + // can still read, and an unwritten status would reach it as a false, empty 200. + w.WriteHeader(499) + return true +} + +// uploadFailure maps an Accept (or multipart) error to a response: a validation +// rejection is the user's to fix (422, or 413 for size) and carries its own +// message; anything else is ours. +func (a *App) uploadFailure(w http.ResponseWriter, r *http.Request, userID string, err error) { + if a.abandoned(w, r) { + return + } + var rej *uploads.RejectError + var tooBig *http.MaxBytesError + switch { + case errors.As(err, &rej): + // Logged by reason only: the filename and title are private content. + a.logger().Info("upload rejected", "reason", string(rej.Reason)) + status := http.StatusUnprocessableEntity + switch rej.Reason { + case uploads.ReasonTooLarge: + status = http.StatusRequestEntityTooLarge + case uploads.ReasonBusy: + status = http.StatusTooManyRequests + } + a.renderUploadsPage(w, r, userID, status, rej.Message) + case errors.As(err, &tooBig): + a.logger().Info("upload rejected", "reason", string(uploads.ReasonTooLarge)) + a.renderUploadsPage(w, r, userID, http.StatusRequestEntityTooLarge, + "This file is larger than the upload limit.") + case errors.Is(err, uploads.ErrClientRead): + // An error reading the CLIENT's body: theirs to retry, not a server fault. Only that + // marker counts: io.ErrUnexpectedEOF alone is also what a database drop looks like. + a.logger().Info("upload rejected", "reason", "truncated") + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "The upload was cut off before it finished. Please try again.") + default: + // Not err: it can carry the file's name, a spool path or a database value. + a.logger().Error("upload failed", "kind", errkind.Of(err)) + http.Error(w, "internal error", http.StatusInternalServerError) + } +} + +// handleUploadDetail renders one upload's page (GET /uploads/{id}). The row is +// read through the user-scoped store, so another user's id is a 404 — existence is +// not disclosed, and the title never appears in the response. +func (a *App) handleUploadDetail(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + a.render(w, r, UploadPage(*u, a.Chat != nil, a.now())) +} + +// handleUploadStatus renders the status fragment HTMX polls. Unfinished states +// keep polling; terminal states render without hx-trigger so the page stops +// asking (ADR-025: never a stuck spinner). +func (a *App) handleUploadStatus(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + a.render(w, r, uploadStatus(*u, a.Chat != nil, a.now())) +} + +func (a *App) loadOwnedUpload(w http.ResponseWriter, r *http.Request) (*domain.UploadedTranscript, bool) { + userID, ok := a.currentUserID(w, r) + if !ok { + return nil, false + } + id := r.PathValue("id") + if !uuidShape.MatchString(id) { + // Not an id at all: the same 404 as an unknown or foreign one, never a database + // error that surfaces as a 500 (and no oracle about which ids exist). + http.NotFound(w, r) + return nil, false + } + u, err := a.Store.GetUpload(r.Context(), userID, id) + if errors.Is(err, store.ErrUploadNotFound) { + http.NotFound(w, r) + return nil, false + } + if err != nil { + a.serverError(w, r, "get upload", err) + return nil, false + } + return u, true +} + +// titleFromFilename is the default title: the file's name without its extension. +func titleFromFilename(name string) string { + base := filepath.Base(name) + t := cleanTitle(strings.TrimSuffix(base, filepath.Ext(base))) + if t == "" || t == "." { + return "Untitled recording" + } + return t +} + +// hasNote reports whether any phase carries a note. A note is the only warning for a +// recording that is shorter than its file claimed, so a record that has one is shown open. +func hasNote(steps []domain.ProgressStep) bool { + for _, s := range steps { + if s.Note != "" { + return true + } + } + return false +} + +// uuidShape is the form of an upload id. Anything else cannot be one, so it is a 404. +var uuidShape = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$`) + +// isBidiControl reports the invisible characters that change text direction. Emoji joiners +// (ZWJ, ZWNJ) and variation selectors are not among them and are kept. +func isBidiControl(r rune) bool { + return r >= 0x202A && r <= 0x202E || r >= 0x2066 && r <= 0x2069 || r == 0x200E || r == 0x200F || r == 0x061C +} + +// cleanTitle makes a title safe to store and show: invalid UTF-8 dropped, control +// characters (NUL, newlines, escapes) turned into spaces, runs of space collapsed, trimmed +// and clipped. A title is cosmetic; bad bytes in it must never fail an upload, and Postgres +// refuses NUL in text. +func cleanTitle(s string) string { + s = strings.ToValidUTF8(s, "") + s = strings.Map(func(r rune) rune { + switch { + case isBidiControl(r): + return -1 // dropped: U+202E would reverse the rest of the title on its owner's page + case unicode.IsControl(r): + return ' ' + } + return r + }, s) + return clipRunes(strings.Join(strings.Fields(s), " "), maxTitleRunes) +} + +func clipRunes(s string, n int) string { + if utf8.RuneCountInString(s) <= n { + return s + } + return string([]rune(s)[:n]) +} + +// uploadFailureMessage turns a typed failure kind into words for the user. It is +// keyed on OUR kinds, never on a provider's error string (ADR-033). +func uploadFailureMessage(k domain.UploadErrorKind) string { + switch k { + case domain.UploadErrInterrupted: + return "The server restarted before this recording was transcribed. Please upload it again." + case domain.UploadErrBusy: + return "Too many recordings were waiting to be transcribed when this one was uploaded. Please upload it again in a few minutes." + case domain.UploadErrTooLong: + return "This recording is longer than the limit once its audio is read (the file's header said otherwise). Please upload a shorter recording." + case domain.UploadErrTranscriptionFailed: + return "Transcription failed on every available service. Please upload it again later." + default: + return "This upload failed. Please upload it again." + } +} + +// handleUploadChat renders the chat page for a FINISHED upload (GET). The row is +// read through the user-scoped store, so another user's id is a 404 before any +// model is involved. An unfinished upload has no transcript to ask about yet, so +// it goes back to its status page. +func (a *App) handleUploadChat(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + if u.Status != domain.UploadDone { + http.Redirect(w, r, "/uploads/"+u.ID, http.StatusSeeOther) + return + } + model, _ := a.resolveUploadModel("") + a.render(w, r, UploadChatPage(*u, a.uploadChatView(*u, model, nil, false, ""))) +} + +// handleUploadChatMessage answers one question against the owner's uploaded +// transcript (POST), reusing the ADR-027 Chatter. Nothing is persisted: the prior +// turns ride in the form. The transcript goes to the model with the UPLOAD budget, +// not the caption default. A model failure is inline, not a 500. +func (a *App) handleUploadChatMessage(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + if u.Status != domain.UploadDone { + http.Redirect(w, r, "/uploads/"+u.ID, http.StatusSeeOther) + return + } + if err := r.ParseForm(); err != nil { + http.Error(w, "bad form", http.StatusBadRequest) + return + } + model, ok := a.resolveUploadModel(r.FormValue("model")) + if !ok { + a.logger().Error("upload chat: no local model is configured; nothing was sent", "upload", u.ID) + view := a.uploadChatView(*u, "", nil, false, "No local model is available for chat about a recording.") + a.render(w, r, UploadChatPage(*u, view)) + return + } + history := parseHistory(r.Form["hq"], r.Form["ha"]) + question := strings.TrimSpace(r.FormValue("question")) + + view := a.uploadChatView(*u, model, history, false, "") + if question != "" { + reply, err := a.Chat.Answer(r.Context(), chat.Request{ + Model: model, + Transcript: u.Content, + History: history, + Question: question, + MaxChars: a.UploadChatMaxChars, + }) + if err != nil { + a.logger().Error("upload chat answer", "upload", u.ID, "model", model, "kind", errkind.Of(err)) + view.Error = "That model couldn't answer just now. Try again, or switch models." + } else { + view.History = appendTurn(history, chat.Turn{Question: question, Answer: reply.Answer}) + view.Truncated = reply.Truncated + } + } + if isHTMX(r) { + a.render(w, r, chatPanel(view)) + return + } + a.render(w, r, UploadChatPage(*u, view)) +} + +func (a *App) uploadChatView(u domain.UploadedTranscript, model string, history []chat.Turn, truncated bool, errMsg string) chatView { + return chatView{ + VideoID: u.ID, + Title: u.Title, + Available: true, + Models: a.UploadChatModels, + Selected: model, + History: history, + Truncated: truncated, + Error: errMsg, + PostURL: templ.SafeURL("/uploads/" + u.ID + "/chat"), + Subject: "recording", + } +} + +// maxSpotsShown is how many spots to check the page lists. The stored list holds every +// judged spot and the counts are exact; a page of two hundred "check this" items is not a +// page anyone reads, so the rest are counted, not listed. +const maxSpotsShown = 50 + +// minuteMark renders where in the recording a spot is, as the nearest minute. +func minuteMark(seconds float64) string { + return fmt.Sprintf("~%d min", int(math.Round(seconds/60))) +} + +// shownSpots is the part of the stored list the page renders: the spots a reader should +// check, changed or uncertain, in order, capped. A spot the model confidently kept is +// stored (ADR-035) but is not one to check, so it is not listed. +func shownSpots(r *domain.Reconciliation) []domain.ContestedSpot { + var out []domain.ContestedSpot + for _, c := range r.Contested { + if c.Used == "alt" || c.Unsure { + out = append(out, c) + if len(out) == maxSpotsShown { + break + } + } + } + return out +} + +// moreSpots is how many spots to check are not listed, from the exact counts. +func moreSpots(r *domain.Reconciliation) int { + return max(0, r.Changed+r.Unsure-len(shownSpots(r))) +} + +// jobHealth is what the page can honestly say about a job it is still polling. +type jobHealth string + +const ( + // healthRunning: the service is reporting and work is moving, or there is nothing yet + // to say otherwise. + healthRunning jobHealth = "running" + // healthQuiet: the service is alive but a counted phase has not advanced for a while. + healthQuiet jobHealth = "quiet" + // healthLost: nothing has been heard from the service for too long. It may have + // restarted or crashed; the start-up sweep fails the row once it is back. + healthLost jobHealth = "lost" +) + +// uploadHealth tells running from quiet from lost for an upload being transcribed (the +// page calls it for no other state: a queued or finished job has no heartbeat to judge). The +// two timestamps mean different things (ADR-035): alive_at is the heartbeat, which says +// the service is still reporting; progress_at moves only when work really moved. A +// single blocking engine call has no sub-progress, so only a counted phase can be quiet. +// silent is how long since the signal that decided it. +func uploadHealth(u domain.UploadedTranscript, now time.Time) (health jobHealth, silent time.Duration) { + last := u.AliveAt + if u.ProgressAt.After(last) { + last = u.ProgressAt + } + if last.IsZero() { + return healthRunning, 0 + } + if d := now.Sub(last); d > serviceSilentAfter { + return healthLost, d + } + for _, s := range u.Progress { + if s.State == domain.StepRunning && s.Total > 0 && !u.ProgressAt.IsZero() { + if d := now.Sub(u.ProgressAt); d > countedQuietAfter { + return healthQuiet, d + } + } + } + return healthRunning, 0 +} + +// stepDuration renders how long a phase took or has run: "<1s", "42s", "3m 12s", "1h 02m". +func stepDuration(d time.Duration) string { + switch { + case d < time.Second: + return "<1s" + case d < time.Minute: + return fmt.Sprintf("%ds", int(d.Seconds())) + case d < time.Hour: + return fmt.Sprintf("%dm %02ds", int(d.Minutes()), int(d.Seconds())%60) + default: + return fmt.Sprintf("%dh %02dm", int(d.Hours()), int(d.Minutes())%60) + } +} + +// stepTime is the time shown beside a phase: how long it has run, or how long it took. +func stepTime(s domain.ProgressStep, now time.Time) string { + switch s.State { + case domain.StepRunning: + return stepDuration(now.Sub(s.StartedAt)) + case domain.StepDone, domain.StepFailed: + return stepDuration(s.FinishedAt.Sub(s.StartedAt)) + } + return "" +} + +// stepMark is the glyph at the head of a phase line. +func stepMark(s domain.ProgressStep) string { + switch s.State { + case domain.StepDone: + return "✓" + case domain.StepFailed: + return "✗" + case domain.StepSkipped: + return "–" + } + return "·" +} + +// minutesAgo is a whole number of minutes for the health notes: "6 min", or "under a +// minute" when it is less. +func minutesAgo(d time.Duration) string { + if m := int(d.Minutes()); m >= 1 { + return fmt.Sprintf("%d min", m) + } + return "under a minute" +} + +// resolveUploadModel picks the model for chat over an uploaded recording from the +// offered LOCAL set only: the posted one when it is in the set, else the configured +// default when it is, else the first. ok is false when no model is offered, in which +// case nothing may be sent anywhere (the whole private transcript is the payload). +func (a *App) resolveUploadModel(posted string) (model string, ok bool) { + offered := func(m string) bool { + for _, o := range a.UploadChatModels { + if o == m { + return true + } + } + return false + } + switch { + case posted != "" && offered(posted): + return posted, true + case a.UploadChatModel != "" && offered(a.UploadChatModel): + return a.UploadChatModel, true + case len(a.UploadChatModels) > 0: + return a.UploadChatModels[0], true + } + return "", false +} diff --git a/internal/web/uploads.templ b/internal/web/uploads.templ new file mode 100644 index 0000000..658179a --- /dev/null +++ b/internal/web/uploads.templ @@ -0,0 +1,248 @@ +package web + +import ( + "fmt" + "strconv" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// UploadsPage is the upload form plus the caller's own uploads (ADR-033). +templ UploadsPage(v uploadsView) { + @Layout("Tapir — Upload a recording") { + + } +} + +// UploadPage is one upload's page: its title and the polling status region. +templ UploadPage(u domain.UploadedTranscript, chatEnabled bool, now time.Time) { + @Layout("Tapir — " + u.Title) { + + } +} + +// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the +// page re-asks; done and failed render without it, so nothing polls forever. +templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool, now time.Time) { + switch u.Status { + case domain.UploadDone: +
+ @videoNotice(u.HadVideo) +

+ Transcribed { uploads.FormatDuration(time.Duration(u.AudioSeconds * float64(time.Second))) } + of audio by { u.Endpoint }. +

+ @reconciliationNote(u.Reconciliation) + if len(u.Progress) > 0 { +
+ What was done + @progressSteps(u.Progress, now, false) +
+ } + if chatEnabled { +

Ask about this recording

+ } +
+ Transcript + 
{ u.Content }
+
+
+ case domain.UploadFailed: + + default: +
+ @videoNotice(u.HadVideo) + if u.Status == domain.UploadTranscribing { + {{ health, silent := uploadHealth(u, now) }} + if len(u.Progress) > 0 { + @progressSteps(u.Progress, now, health != healthLost) + @healthNote(health, silent) + } else { +

Transcribing… a long recording takes a few minutes.

+ } + } else { +

Queued — waiting for its turn to be transcribed.

+ } +
+ } +} + +// UploadChatPage is the no-JS full page for chatting with an uploaded transcript: +// the same chat panel as a video's (ADR-027), scoped to this recording. +templ UploadChatPage(u domain.UploadedTranscript, v chatView) { + @Layout("Tapir — Ask about " + u.Title) { + + } +} + +// videoNotice tells the user a video track was present and what became of it. A +// Teams meeting with the camera off is still a video file; it is accepted, only the +// audio is transcribed, and the picture is thrown away — so say so, in every state, +// instead of leaving the user to wonder (ADR-033). +templ videoNotice(hadVideo bool) { + if hadVideo { +

+ This file contained a video track. Only its audio was used; the video was discarded and nothing from it was kept. +

+ } +} + +// reconciliationNote says what was done when two engines transcribed the audio +// (ADR-034), and never claims a second opinion that was not there: nil means a single +// engine and renders nothing. Spots the model changed or was unsure about are listed +// with a minute mark and both readings so the user can check them against the audio. +templ reconciliationNote(r *domain.Reconciliation) { + if r != nil { + switch r.Status { + case domain.ReconcileDone: +
+

+ Reconciled from two transcripts ({ r.Base } and { r.Alt }): + { strconv.Itoa(r.Disputes) } differences, + { strconv.Itoa(r.AutoResolved) } settled automatically, + { strconv.Itoa(r.Adjudicated) } decided by the model ({ r.Reconciler }), + { strconv.Itoa(r.Unsure) } uncertain. +

+ if len(shownSpots(r)) > 0 { +
+ Spots to check against the audio +
    + for _, c := range shownSpots(r) { +
  • + { minuteMark(c.Seconds) }: + if c.Before != "" { + …{ c.Before } + } + { c.Base } / { c.Alt } — + if c.Unsure { + uncertain kept { c.Used } + } else { + changed used { c.Used } + } +
    • + } +
+ if moreSpots(r) > 0 { +

and { strconv.Itoa(moreSpots(r)) } more not listed

+ } +
+ } +
+ case domain.ReconcileSingle: +

Only one engine produced this transcript ({ r.Reason }), so there was no second opinion to reconcile it with.

+ case domain.ReconcileSkipped: +

Two transcripts were made but could not be reconciled ({ r.Reason }); the text shown is { r.Base }'s, unreconciled.

+ } + } +} + +// progressSteps is the job's phases as a log that fills in as it runs: finished ones +// collapse to a line with how long they took, the running one has a spinner and its +// elapsed time, and what has not started is dimmed. live is false when the spinner would +// be a lie (the service is lost, or the job is over), so the running phase shows a +// warning mark instead. Only a counted phase gets a bar (ADR-035). +templ progressSteps(steps []domain.ProgressStep, now time.Time, live bool) { +
    + for _, s := range steps { +
  1. + if s.State == domain.StepRunning && live { + + } else if s.State == domain.StepRunning { + + } else { + + } + { s.Label } + if t := stepTime(s, now); t != "" { + { t } + } + if s.Note != "" { + { s.Note } + } + if s.State == domain.StepRunning && s.Total > 0 { + + { strconv.Itoa(s.Done) } of { strconv.Itoa(s.Total) } + } +
    2. + } +
+} + +// healthNote says so when the job is not simply running: the service stopped reporting, +// or a counted phase has gone quiet. Running says nothing: the spinner does. +templ healthNote(h jobHealth, silent time.Duration) { + switch h { + case healthLost: + + case healthQuiet: +

+ Nothing new for { minutesAgo(silent) }: the model may be slow. The service is still running, + and the job gives up on its own if it takes too long. +

+ } +} diff --git a/internal/web/uploads_badinput_handler_test.go b/internal/web/uploads_badinput_handler_test.go new file mode 100644 index 0000000..b23648f --- /dev/null +++ b/internal/web/uploads_badinput_handler_test.go @@ -0,0 +1,93 @@ +package web_test + +import ( + "context" + "fmt" + "log/slog" + "net/http" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// Input that cannot be right is the caller's to fix: a 4xx, never a 500 (adversarial +// review F7). No cross-tenant oracle either way: unknown and foreign ids look the same. + +func TestUploadRoutes_ANonUUIDIdIs404NotA500(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + for _, path := range []string{"/uploads/not-a-uuid", "/uploads/not-a-uuid/status", "/uploads/not-a-uuid/chat", "/uploads/%00/status", "/uploads/1/status"} { + t.Run(path, func(t *testing.T) { + rec := do(t, app, get(path)) + + assert.Equal(t, http.StatusNotFound, rec.Code) + }) + } +} + +func TestUploadPost_ATitleWithControlCharactersOrInvalidUTF8IsCleanedNotA500(t *testing.T) { + tests := []struct{ name, title, filename string }{ + {"a NUL in the title", "Sprint\x00review", "a.m4a"}, + {"invalid UTF-8 in the title", "Sprint \xff\xfe review", "a.m4a"}, + {"a newline in the title", "Sprint\nreview", "a.m4a"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + app, sub := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, uploadReq(t, tt.title, tt.filename, []byte("synthetic audio bytes"))) + + require.Equal(t, http.StatusSeeOther, rec.Code, "bad bytes in a cosmetic title must not fail the upload: %s", body(t, rec)) + require.Len(t, sub.jobs, 1) + var title string + require.NoError(t, rawPool(t).QueryRow(context.Background(), `SELECT title FROM uploaded_transcripts WHERE id = $1`, sub.jobs[0].UploadID).Scan(&title)) + assert.NotContains(t, title, "\x00") + assert.NotContains(t, title, "\n") + assert.True(t, len(title) > 0, "a title is still shown") + }) + } +} + +func TestUploadPost_AMalformedMultipartHeaderIsA400AndItsTextIsNotLogged(t *testing.T) { + // A control byte in the part's file name makes the multipart body malformed. It is the + // sender's to fix, and the parser's error quotes the header, which carries the name. + app, sub := newUploadApp(t, audioOK, 200<<20) + buf := &lockedBuf{} + app.Log = slog.New(slog.NewTextHandler(buf, nil)) + + rec := do(t, app, uploadReq(t, "", "zebrakaka\x00weekly.m4a", []byte("x"))) + + assert.Equal(t, http.StatusBadRequest, rec.Code) + assert.Empty(t, sub.jobs) + assert.NotContains(t, buf.String(), "zebrakaka", "a file name is private content") +} + +func TestUploadPost_EveryBidirectionalControlCharacterIsStrippedFromATitle(t *testing.T) { + // U+202E reverses the text that follows it, so a title could render reversed on its + // owner's own page (second adversarial review). Every one of the twelve is pinned. + controls := []rune{0x202A, 0x202B, 0x202C, 0x202D, 0x202E, 0x2066, 0x2067, 0x2068, 0x2069, 0x200E, 0x200F, 0x061C} + for _, c := range controls { + t.Run(fmt.Sprintf("U+%04X", c), func(t *testing.T) { + app, sub := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, uploadReq(t, "Sprint"+string(c)+"review", "a.m4a", []byte("x"))) + + require.Equal(t, http.StatusSeeOther, rec.Code) + var title string + require.NoError(t, rawPool(t).QueryRow(context.Background(), `SELECT title FROM uploaded_transcripts WHERE id = $1`, sub.jobs[0].UploadID).Scan(&title)) + assert.Equal(t, "Sprintreview", title, "dropped, not replaced by a space: a space would leave a visible gap") + }) + } +} + +func TestUploadPost_SwedishLettersAndEmojiSurviveTheTitleCleaning(t *testing.T) { + app, sub := newUploadApp(t, audioOK, 200<<20) + + // Includes a ZWJ sequence (woman + ZWJ + laptop) and a ZWNJ: joiners are not direction controls. + rec := do(t, app, uploadReq(t, "Möte åäö 🎙️ 👩\u200d💻 a\u200cb", "a.m4a", []byte("x"))) + + require.Equal(t, http.StatusSeeOther, rec.Code) + var title string + require.NoError(t, rawPool(t).QueryRow(context.Background(), `SELECT title FROM uploaded_transcripts WHERE id = $1`, sub.jobs[0].UploadID).Scan(&title)) + assert.Equal(t, "Möte åäö 🎙️ 👩\u200d💻 a\u200cb", title, "joiners and variation selectors belong to the text") +} diff --git a/internal/web/uploads_busy_handler_test.go b/internal/web/uploads_busy_handler_test.go new file mode 100644 index 0000000..b741dba --- /dev/null +++ b/internal/web/uploads_busy_handler_test.go @@ -0,0 +1,31 @@ +package web_test + +import ( + "net/http" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +func TestUploadPost_ABusyRejectionIs429WithItsMessage(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + app.Uploads = failingUploader{err: &uploads.RejectError{Reason: uploads.ReasonBusy, Message: "Too many recordings are waiting. Try again in a few minutes."}} + + rec := do(t, app, uploadReq(t, "t", "a.m4a", []byte("x"))) + + require.Equal(t, http.StatusTooManyRequests, rec.Code, "the user's to retry: not a 500, not a 422") + assert.Contains(t, body(t, rec), "Too many recordings are waiting") +} + +func TestUploadStatus_ABusyFailureIsExplainedInWords(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + id := seedUpload(t, userID, "m", "failed", "queue_full") + + b := body(t, do(t, app, get("/uploads/"+id+"/status"))) + + assert.Contains(t, b, "waiting") + assert.NotContains(t, b, "This upload failed. Please upload it again.") +} diff --git a/internal/web/uploads_chat_handler_test.go b/internal/web/uploads_chat_handler_test.go new file mode 100644 index 0000000..ab0a125 --- /dev/null +++ b/internal/web/uploads_chat_handler_test.go @@ -0,0 +1,183 @@ +package web_test + +import ( + "errors" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/chat" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// Chat over an uploaded transcript REUSES the ADR-027 chat path (the same +// Chatter, the same panel) — only the lookup differs: the row comes from the +// user-scoped private store, so another user's upload is a 404 and, crucially, no +// model is ever called with it. Transcript text is synthetic; this repo is public. + +const meetingText = "Anders ansvarar för virket. Maria undersöker leverantörer." + +func newUploadChatApp(t *testing.T) (*web.App, *fakeChatter) { + t.Helper() + app, _ := newUploadApp(t, audioOK, 200<<20) + fc := &fakeChatter{ + models: []string{"koala/phi4-mini", "iguana/gemma4-26b"}, + reply: chat.Reply{Answer: "Anders owns the timber order."}, + } + app.Chat = fc + app.UploadChatMaxChars = 120000 + app.UploadChatModels = fc.models + return app, fc +} + +func chatPost(path string, kv url.Values) *http.Request { + req := httptest.NewRequest(http.MethodPost, path, strings.NewReader(kv.Encode())) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + return req +} + +func TestUploadChatPage_OwnerOfAFinishedUploadSeesThePanelPostingToTheUploadRoute(t *testing.T) { + app, _ := newUploadChatApp(t) + id := seedUpload(t, userID, "Roadmap sync", "done", meetingText) + + rec := do(t, app, get("/uploads/"+id+"/chat")) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, `action="/uploads/`+id+`/chat"`, "the form posts to the UPLOAD route, not /v/{id}/chat") + assert.Contains(t, b, "Roadmap sync") + assert.Contains(t, b, "koala/phi4-mini") + assert.Contains(t, b, "recording", "wording says recording, not video") +} + +func TestUploadChatPage_AnUnfinishedUploadRedirectsToItsStatusPage(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "transcribing", "") + + rec := do(t, app, get("/uploads/"+id+"/chat")) + + assert.Equal(t, http.StatusSeeOther, rec.Code) + assert.Equal(t, "/uploads/"+id, rec.Header().Get("Location")) + assert.Zero(t, fc.callCount) +} + +func TestUploadChat_AnotherUsersUploadIs404AndNoModelIsCalled(t *testing.T) { + app, fc := newUploadChatApp(t) + theirs := seedUpload(t, otherUserID, "Their private meeting", "done", meetingText) + + getRec := do(t, app, get("/uploads/"+theirs+"/chat")) + postRec := do(t, app, chatPost("/uploads/"+theirs+"/chat", url.Values{"question": {"what did they say?"}})) + + assert.Equal(t, http.StatusNotFound, getRec.Code) + assert.Equal(t, http.StatusNotFound, postRec.Code) + assert.NotContains(t, body(t, getRec), "Their private meeting") + assert.Zero(t, fc.callCount, "someone else's transcript must never reach a model") +} + +func TestUploadChatPost_SendsTheOwnersTranscriptWithTheUploadBudget(t *testing.T) { + app, fc := newUploadChatApp(t) + app.UploadChatMaxChars = 77777 + id := seedUpload(t, userID, "Roadmap sync", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{ + "question": {"Who owns the timber order?"}, + "model": {"iguana/gemma4-26b"}, + })) + + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, 1, fc.callCount) + assert.Equal(t, meetingText, fc.gotReq.Transcript) + assert.Equal(t, "Who owns the timber order?", fc.gotReq.Question) + assert.Equal(t, "iguana/gemma4-26b", fc.gotReq.Model) + assert.Equal(t, 77777, fc.gotReq.MaxChars, + "an uploaded transcript takes its own budget: the 18,000-char caption default would cut an hour to its first third") + assert.Contains(t, body(t, rec), "Anders owns the timber order.") +} + +func TestUploadChatPost_AnUnofferedModelFallsBackToTheDefault(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "done", meetingText) + + do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}, "model": {"evil/model"}})) + + assert.Equal(t, "koala/phi4-mini", fc.gotReq.Model) +} + +func TestUploadChatPost_UnfinishedUploadCallsNoModel(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "queued", "") + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}})) + + assert.Equal(t, http.StatusSeeOther, rec.Code) + assert.Zero(t, fc.callCount) +} + +func TestUploadChatPost_PriorTurnsRideAlongAndATruncatedReplyIsSaidSo(t *testing.T) { + app, fc := newUploadChatApp(t) + fc.reply = chat.Reply{Answer: "second answer", Truncated: true} + id := seedUpload(t, userID, "m", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{ + "question": {"and who else?"}, + "hq": {"first question"}, + "ha": {"first answer"}, + })) + + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, []chat.Turn{{Question: "first question", Answer: "first answer"}}, fc.gotReq.History) + b := body(t, rec) + assert.Contains(t, b, "first answer") + assert.Contains(t, b, "second answer") + assert.Contains(t, b, "end of the recording", "a long recording cut to a budget must say so, in its own words") +} + +func TestUploadChatPost_AModelFailureIsShownInlineNotAs500(t *testing.T) { + app, fc := newUploadChatApp(t) + fc.err = errors.New("gateway down") + id := seedUpload(t, userID, "m", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}, "hq": {"earlier"}, "ha": {"kept"}})) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, "answer just now", "the failure is inline (the apostrophe is HTML-escaped, so match the tail)") + assert.Contains(t, b, "kept", "the conversation survives a failed turn") +} + +func TestUploadChatRoutes_AreNotMountedWithoutAChatBackend(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + id := seedUpload(t, userID, "m", "done", meetingText) + + assert.Equal(t, http.StatusNotFound, do(t, app, get("/uploads/"+id+"/chat")).Code) +} + +func TestUploadStatus_DoneOffersTheChatLinkOnlyWhenChatIsWired(t *testing.T) { + appWith, _ := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "done", meetingText) + assert.Contains(t, body(t, do(t, appWith, get("/uploads/"+id+"/status"))), `href="/uploads/`+id+`/chat"`) + + appWithout, _ := newUploadApp(t, audioOK, 200<<20) + id2 := seedUpload(t, userID, "m2", "done", meetingText) + assert.NotContains(t, body(t, do(t, appWithout, get("/uploads/"+id2+"/status"))), "/chat") +} + +func TestUploadChat_OpensWithTheUploadChatModelNotTheSummarizerDefault(t *testing.T) { + // koala/phi4-mini is first in the offered list (it is the summarizer default) and + // has an 8,192-token context: an hour of speech returns HTTP 400 there. Chat over + // an upload must open with the model configured for it. + app, fc := newUploadChatApp(t) + app.UploadChatModel = "iguana/gemma4-26b" + id := seedUpload(t, userID, "m", "done", meetingText) + + page := body(t, do(t, app, get("/uploads/"+id+"/chat"))) + do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}})) + + assert.Contains(t, page, `