From feb108def39ce9af4d851705e5d7d25f391d4bc3 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:33:34 +0200 Subject: [PATCH 001/299] docs(research): measure the audio length ceiling before designing chunking (#33) No transcription ceiling found up to 90 min: one 174 MB request through the LiteLLM NodePort transcribed in 440 s. The first real failure is the public gateway ingress rejecting bodies over ~8-10 MB with 413, not whisper or a Metal OOM. Also records that srt arrives wrapped in a JSON envelope, that the two backends differ in timestamp granularity, and that the existing chat path truncates at 18k chars (~22 min of speech). Synthetic Swedish TTS fixture only; no real transcript. Co-Authored-By: Claude Sonnet 5.5 --- .../audio-length-ceiling-2026-10-01.md | 110 ++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 docs/research/audio-length-ceiling-2026-10-01.md diff --git a/docs/research/audio-length-ceiling-2026-10-01.md b/docs/research/audio-length-ceiling-2026-10-01.md new file mode 100644 index 0000000..e9ed2ba --- /dev/null +++ b/docs/research/audio-length-ceiling-2026-10-01.md @@ -0,0 +1,110 @@ +# Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01) + +Question (#33, "the main technical risk"): at what recording length does Swedish +transcription through the LiteLLM gateway first fail, and with what failure mode? +Measured **before** any chunking design, with iguana not holding a large ollama model, +so the result is the ceiling and not the Metal OOM of infra#273. + +## Answer + +**No transcription ceiling was found at or below 90 minutes.** A single request carrying +90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing +that actually fails is **not whisper**: it is the public gateway ingress, which rejects the +upload with `413` at roughly 5.5 minutes of WAV. + +| Route | First failure | Failure mode | +|---|---|---| +| `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly | +| `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — | +| `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — | + +The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a +30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the +size-dependent form described, a body-size cap explains it without one. (The OOM +mechanism in infra#273 is separately real; this run did not provoke it.) + +The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the +ingress configuration, which lives in `infra`. "About 10 MB" is an inference. + +## Method + +- **Audio.** No known-good Swedish recording is available that is safe to use: the only real + one (`IMG_1233.wav`) is the private `/bygge` recording (#31 custody still open) and was + deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char + five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`). + Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe). + Nothing here is a real transcript; none of it is committed. +- **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`, + `response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only + through `op run` and never on argv. +- **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident: + nothing, or `nomic-embed-text` (0.37 GB). Never a large model. + +## Results (`iguana/kb-whisper`, NodePort) + +| Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars | +|---|---|---|---|---|---|---| +| 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — | +| 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — | +| 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 | +| 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 | +| 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 | +| 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 | +| 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 | +| 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 | + +- Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is + a minutes-long job, not a request/response one (see ADR-033 on ADR-018). +- Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are + monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop + hallucination appeared across 90 minutes. This is a weak check — the input is a repeated + loop and synthetic voice — not evidence of accuracy. +- **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration` + 330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`. + +## Findings that change the design + +1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the + gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the + gateway, never the host" and named the public URL; the gateway is right, that URL is not. + `TAPIR_GATEWAY_URL` already points at `koala:30401/v1`. +2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON + `{"text": "", "usage": …}`, not a bare SRT body — a client that parses the body + as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside + an envelope. +3. **The two backends return different timestamp granularity.** `iguana/kb-whisper` + segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real + sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`). + `verbose_json` and the default JSON were byte-identical on iguana. An adapter that + falls back between the two must normalise both, and "which endpoint served this" is + a quality fact, not just a log line. +4. **The existing chat path cannot answer about most of a long recording.** At + 828 chars/min (this fixture; real speech is plausibly a little denser), + `TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute + transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping + the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of + a one-hour meeting get an answer from a transcript that does not contain it. #33 says + chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and + truncates**, and its ceiling is the model context. See ADR-033. +5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The + chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is + not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces + smaller requests — see ADR-033's resolution triggers. + +## What this did not measure + +- **Metal OOM under contention.** Deliberately avoided. A big resident ollama model + during a long transcription is still the known way to wedge iguana permanently. +- **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and + speaker overlap in a Teams export are untested. Speed is probably representative; + accuracy is not claimed. +- **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper + without a server-side transcode is not established here (ADR-033 treats it as needing one). +- **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were + not tried. +- **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not + size-capped at meeting length for 22 minutes; not proven at 90. +- **Time-of-day / other tenants.** One afternoon, one run per length. + +Scripts were throwaway shell/Python in a session scratchpad and are not committed; the +method above reproduces them. -- 2.54.0 From 38b3f1cf3397cbc4073ed4626a200059b4b4545f Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:35:04 +0200 Subject: [PATCH 002/299] docs(adr): ADR-033 upload audio -> private Swedish transcript -> chat (#33) Proposed, decider pending. Supersedes ADR-007's STT deferral for user-supplied audio only; the YouTube audio-download rejection stands. Settles #28 decision 1 as a separate RLS-scoped uploaded_transcripts table and records the measured length ceiling, the ingress 413, and the 18k-char chat truncation that #33 assumed did not exist. Two DECISION NEEDED items (ffmpeg in the runtime image; berget fallback for private audio). Co-Authored-By: Claude Sonnet 5.5 --- DECISIONS.md | 192 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 1 deletion(-) diff --git a/DECISIONS.md b/DECISIONS.md index 44eb2b2..12dc6a9 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -123,7 +123,9 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path. ## ADR-007 — Captions-first; audio-download + speech-to-text deferred **Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API -`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand. +`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand +**for YouTube**. STT deferral superseded **for user-supplied audio only** by ADR-033; the +YouTube audio-download rejection is unchanged. **Context.** YouTube's Data API does not expose transcripts. Options: official captions (clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey, @@ -1314,6 +1316,191 @@ roll back. No schema, no structural change. --- +## ADR-033 — Upload audio, get a private Swedish transcript, discuss it (user-supplied STT) + +**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #33 (audio-only subset of +#28). **Supersedes ADR-007's speech-to-text deferral for user-supplied audio only.** The +YouTube audio-download rejection in ADR-007 **stands untouched**. Builds on ADR-012 (isolation), +ADR-018 (single replica), ADR-021 (**the constraint**), ADR-022 (endpoint chain), ADR-025 +(honest status), ADR-027 (chat). ADR-014 item 4 ("Whisper reconsideration is a future ADR, +gated on caption data") is a different question — whether to transcribe YouTube videos with no +captions — and this ADR does not answer it. Measurements: `docs/research/audio-length-ceiling-2026-10-01.md`. + +**Context.** The maintainer has a concrete need: a Swedish Teams voice-only export, 30–90 +minutes, to upload and then *discuss* (what was decided, what is open, who owes what). Every +piece exists — a Whisper endpoint on the gateway (infra#274), chat-with-transcript (ADR-027) — +and nothing connects them. + +ADR-007 deferred STT for three reasons: yt-dlp fragility, ToS-grey scraping, and GPU contention +with the JEPA PoC. Upload-STT shares **none** of them: the user owns the file, nothing is +scraped, there is no caption rate gate, and the contention moved from koala to iguana. What it +does share is landing Whisper in the codebase, which ADR-007 deferred — so a new ADR is required, +and ADR-017's process note records the one time a guardrail-reversing change shipped without one. + +**The constraint that shapes everything.** ADR-021 made `transcripts` a shared, **non-RLS** +table on the explicit grounds that it holds only public caption content and nothing +user-identifying. A recorded internal meeting is the opposite: private third-party content owned +by one user. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse +would leak across tenants **and pass CI green**. + +**What was measured, not argued** (research doc above): +- **No transcription ceiling up to 90 minutes.** One 174 MB request through the LiteLLM NodePort + took 440 s (~12× realtime). The first failure is the public ingress `llm-api.d-ma.be` + returning `413` above roughly 8–10 MB (~5 min of WAV) — not whisper, and not a Metal OOM. + The 8:20 failure recorded in #33 is consistent with that cap. +- **`response_format=srt` survives LiteLLM, wrapped** in a JSON envelope (`{"text": ""}`). +- **The two backends differ.** `iguana/kb-whisper` returns whole-second segments; + `berget/kb-whisper` returns sub-second times and a different `language` token. +- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.** + #33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does + not retrieve; it stuffs the transcript into the prompt and cuts the tail. + +**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on +iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal +OOM, deliberately not provoked. + +**Decision.** + +1. **Audio only; transcript is the only thing kept.** An authenticated user uploads an audio + file; Tapir transcribes it and stores the transcript. The audio itself is deleted after + transcription (temp file, removed in a `defer`, swept at start-up) — no custody problem to + solve, and nothing to hand to #31. Video is rejected with a message naming #28. YouTube + audio-download stays rejected. No public/anonymous upload (ADR-008). + +2. **Private store: a separate RLS-scoped table, `uploaded_transcripts`** (resolves #28's + decision 1). `user_id NOT NULL REFERENCES users ON DELETE CASCADE`, `ENABLE` + `FORCE ROW LEVEL + SECURITY`, the same `tapir.current_user_id` policy as every other user-owned table. It carries + the transcript text and its timestamped segments (re-transcribing 90 minutes costs ~7 + minutes; discarding the timestamps is irreversible), `status`, the `endpoint` that served + it, `audio_seconds`, and a typed `error_kind`. **Not** a `user_id` column on `transcripts` + with a partial policy, even though that would reuse more code: ADR-021's exception is + precisely that *that table holds no private row*, the isolation test asserts it has no + `user_id`, and a partial policy would turn "single exception" into "single exception, with a + private lane inside it" — the shape under which a future query drops the policy clause and + leaks. Two tables keep both boundaries independently assertable. + +3. **The isolation test asserts both boundaries and the set.** (a) User A cannot read, update or + delete user B's uploaded transcript; no GUC set → deny-all. (b) `transcripts` is still + non-RLS with no `user_id`. (c) **The set of public-schema tables with row security off is + asserted exactly** — so a new table added without RLS fails CI instead of passing it. (c) is + new; the existing test only checks the named tables. + +4. **Validate by probing, reject clearly.** Format and duration cannot be read from a filename + or MIME type — a Teams "voice-only" recording is typically an `.mp4` container with no video + stream, which is indistinguishable from video by extension. The upload is probed + (`ffprobe`): accept any container with an audio stream and **no** video stream; reject video + with "video is #28"; reject unreadable files. Limits, all configurable, all rejected + synchronously with a specific message rather than by timeout: **200 MB** upload and + **90 minutes** duration. 90 minutes is *the longest length measured*, not a ceiling that was + found; raising it is a measurement, not an edit. + +5. **Transcode server-side, always, to 16 kHz mono PCM WAV** — even when the input already is, + rather than assuming. whisper-server's `--convert` is rejected (its `--tmp-dir` is created at + process start and rots under macOS periodic cleanup). The transcoder sits behind a port; the + tests use a fake. **This puts `ffmpeg` and `ffprobe` in the runtime image**, which is + distroless-static today. **Provisional: copy static ffmpeg/ffprobe binaries into the existing + distroless image** (no shell, no package manager retained). This is a heavy new dependency + and is **the maintainer's call** — see Open items. + +6. **Transcription adapter via the gateway, never the public hostname.** A `Transcriber` port; + the implementation calls `POST {TAPIR_GATEWAY_URL}/audio/transcriptions` where the gateway + URL is the in-cluster NodePort (`koala:30401/v1`). Primary `iguana/kb-whisper`, fallback + `berget/kb-whisper` (immune to iguana contention, which can wedge permanently — infra#273). + - **Fallback triggers on any primary failure** (transport error, timeout, non-2xx). It never + branches on the body: `{"error":"failed to process audio"}` has had at least five distinct + causes. + - **`response_format=verbose_json`**, normalised to one segment shape: times clamped to the + audio duration (the final segment can end past it), one language token. Not `srt`: the + envelope adds a parse step and iguana's segment timing is the same either way. + - **No chunking in v1.** Nothing failed to 90 minutes (measured). Chunk-and-stitch is the + answer *if* the cap is raised or smaller requests become necessary — see triggers. + - **The serving endpoint is recorded per row**, and surfaced, because the two backends + differ in timestamp granularity: which one served a transcript is a quality fact. + - **Berget is a cloud endpoint.** On fallback the audio of a private meeting leaves the + homelab. #33 asks for this fallback; it is configurable (`TAPIR_TRANSCRIBE_FALLBACK_MODEL`, + empty disables), recorded per row, and called out in Open items rather than defaulted + silently. + - Swedish only (`language=sv`). Language detection and routing (#28 decision 2) are not + touched. + - `-H 'Expect:'`, multipart field `file` — encoded in the adapter and its tests, not rediscovered. + +7. **Jobs run in-process, one at a time (ADR-018 stands).** Transcription is minutes (~7.5 min + for 90) but a single goroutine behind a one-slot semaphore: two simultaneous 90-minute jobs on + iguana were not measured, so concurrency is 1 until they are. No worker, no new deployable, + no second replica. Honest status per ADR-025: `queued → transcribing → done | failed`, polled + by the same `/status` pattern, every terminal state a rendered card, never a stuck spinner. + Audio is not retained, so a restart mid-job **cannot resume**: at start-up any + `queued`/`transcribing` row is marked `failed` with `error_kind = interrupted`, and the user + re-uploads. The single-replica assumption gains a second dependent (the semaphore is + process-local) and is recorded here. + +8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers + look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism + as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model) + carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour + meeting to its first third, so uploaded transcripts take their own cap, + `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at + measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a + guess until the chat slice measures what the offered models actually accept; it is not + evidence that they can. + +9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py` + hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection; + per-transcript deletion UX (account deletion cascades; see Open items); retention of audio. + +**On the sequencing question #28 asks of itself** — *is this the thing that best serves the +Stage-0 gate, or the most interesting thing to build?* Recorded answer: it proceeds **ahead of +#32** (the demand test) as the maintainer's explicit, knowing call (#28, comment of 2026-10-01), +on a concrete need rather than speculation. It is not a return-nudge (ADR-020/027): it adds +nothing that prompts a user to come back, so it does not contaminate the unprompted-return signal +ADR-016 measures. The honest risk is the one ADR-016 names: it is larger than anything ADR-020 +deferred. The scope cut — chat first, no rich analysis, no video — is what keeps it bounded. + +**Consequences.** +- **Positive.** The user's request is met end to end with existing parts. ADR-021's exception stays + singular and now provably so. The measured ceiling removes chunking from v1. No audio is stored. +- **Negative / watch.** + - Tapir's runtime image gains `ffmpeg`: a larger image and a new CVE surface to patch. + - iguana becomes load-bearing for a user-facing feature (#28 decision 3; infra#273 is still the + dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e. + private audio leaving the homelab, silently unless the per-row endpoint is surfaced. + - A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it. + - Chat over a long transcript is bounded by model context, not by anything in Tapir. + - Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`). + +**Validation / resolution triggers** (each observable by someone other than the author): +- *Chunking*: build it when the duration cap is raised above 90 min, or when a measured + request at the current cap fails with whisper-side (not ingress) errors. +- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback, + swap the primary/fallback order and revisit infra#273. +- *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose + answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap. +- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one + replica, ADR-018 is revisited. +- *The decision itself*: if the maintainer has not used the feature unprompted within the + ADR-016 window, it was the interesting thing, not the useful one — remove it. + +**Open items this ADR creates.** +- **DECISION NEEDED — ffmpeg in the runtime image.** *Options:* (A) static ffmpeg/ffprobe copied + into distroless **(recommended — the only option that satisfies "accept m4a/mp3/opus/mp4, + validate duration, reject video"; pure-Go decoders cover wav/mp3 only)**; (B) accept WAV + only, convert client-side (fails "I just want to upload it"); (C) a transcode sidecar + (second deployable, ADR-018's reasoning against). *Blocks:* the Dockerfile slice, not the + tests, which use the port. +- **DECISION NEEDED — berget fallback for private audio.** *Options:* (A) enabled by default as + #33 specifies, with the serving endpoint shown **(recommended, matches the issue)**; (B) + disabled by default, a failed iguana job fails honestly and the user retries. *Blocks:* + the production default only; the adapter ships either way. +- Verify Tapir's ingress `proxy-body-size` against the 200 MB cap (`infra`). +- Per-transcript deletion is not built; account deletion cascades. A private-content store + without a delete button is a gap worth a ticket, not a silent omission. + +**Reversibility.** Additive: one table (down-migration drops it), one adapter, one upload +surface, one image change. Removing the upload route removes the feature; `transcripts`, `videos` +and ADR-021 are untouched. + +--- + ## Rejected alternatives Approaches considered during the 2026-06-02 planning + grill session and **deliberately not @@ -1338,6 +1525,9 @@ maps to the ADR that settles it. | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 | | k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 | | Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 | +| Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 | +| whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 | +| Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 | If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — not a silent reversal. -- 2.54.0 From 066ae6966a3a3c8b2edef770bfa0cf30edb68510 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:36:22 +0200 Subject: [PATCH 003/299] test(store): isolation tests for the private uploaded_transcripts table (red) Drives RLS directly as the non-superuser app role: deny-all with no scope, own-rows-only scoped reads, zero-row cross-user UPDATE/DELETE, rejected cross-user INSERT, FORCE + NOT NULL user_id, cascade on DeleteUser. Also pins the EXACT set of public tables with row security off, so a new table added without RLS fails CI rather than passing it (ADR-033). The existing test only named transcripts; #33's "the only non-RLS table" is false -- invitations, user_identities and schema_migrations are off too. Red: relation "uploaded_transcripts" does not exist. Set test passes today and guards the migration that follows. Co-Authored-By: Claude Sonnet 5.5 --- .../uploaded_transcripts_isolation_test.go | 196 ++++++++++++++++++ 1 file changed, 196 insertions(+) create mode 100644 internal/adapters/store/uploaded_transcripts_isolation_test.go diff --git a/internal/adapters/store/uploaded_transcripts_isolation_test.go b/internal/adapters/store/uploaded_transcripts_isolation_test.go new file mode 100644 index 0000000..d8bd6f0 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts_isolation_test.go @@ -0,0 +1,196 @@ +package store_test + +import ( + "context" + "sort" + "testing" + + "github.com/jackc/pgx/v5/pgxpool" + "github.com/stretchr/testify/require" +) + +// Isolation proof for ADR-033: an uploaded transcript is PRIVATE content owned by +// one user, so it lives in its own RLS-scoped table — never in the shared, +// non-RLS `transcripts` (ADR-021). These tests drive the database directly (raw +// SQL as the non-superuser `app` role) so they observe row-level security itself, +// not any application-layer WHERE clause. The superuser pool is used only to seed +// and to read pg_catalog; it bypasses RLS and so must never be the pool whose +// answers the assertions rest on. + +// seedUploadedTranscript inserts one finished upload for userID as the superuser +// (RLS bypassed so both users' rows land regardless of any GUC) and returns its id. +// The text is synthetic — this repo is public, and a real transcript must never +// appear in it (#33). +func seedUploadedTranscript(t *testing.T, super *pgxpool.Pool, userID, text string) string { + t.Helper() + ctx := context.Background() + _, err := super.Exec(ctx, `INSERT INTO users (id) VALUES ($1) ON CONFLICT DO NOTHING`, userID) + require.NoError(t, err) + var id string + require.NoError(t, super.QueryRow(ctx, + `INSERT INTO uploaded_transcripts (user_id, title, status, content) + VALUES ($1, 'synthetic fixture', 'done', $2) RETURNING id`, + userID, text).Scan(&id)) + return id +} + +func TestUploadedTranscriptsRLS_ScopedReadsSeeOnlyOwnRows(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + seedUploadedTranscript(t, super, userB, "text owned by B") + app := appPool(t, super) + + require.Equal(t, 0, scopedCount(t, app, "", "uploaded_transcripts"), + "no tapir.current_user_id must be deny-all: this proves RLS is live, not bypassed") + require.Equal(t, 1, scopedCount(t, app, userA, "uploaded_transcripts"), + "user A must see exactly its own upload") + require.Equal(t, 1, scopedCount(t, app, userB, "uploaded_transcripts"), + "user B must see exactly its own upload") + + // Count alone cannot distinguish "A's row" from "B's row" — read the content. + var got string + ctx := context.Background() + tx, err := app.Begin(ctx) + require.NoError(t, err) + defer tx.Rollback(ctx) //nolint:errcheck + _, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA) + require.NoError(t, err) + require.NoError(t, tx.QueryRow(ctx, `SELECT content FROM uploaded_transcripts`).Scan(&got)) + require.Equal(t, "text owned by A", got, "A's scoped read must return A's text, never B's") +} + +func TestUploadedTranscriptsRLS_CrossUserWritesAffectZeroRows(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + bID := seedUploadedTranscript(t, super, userB, "text owned by B") + app := appPool(t, super) + + require.Equal(t, int64(0), scopedRowsAffected(t, app, userA, + `UPDATE uploaded_transcripts SET content = 'hacked' WHERE id = $1`, bID), + "user A scoped UPDATE must touch zero of B's uploads") + require.Equal(t, int64(0), scopedRowsAffected(t, app, userA, + `DELETE FROM uploaded_transcripts WHERE id = $1`, bID), + "user A scoped DELETE must touch zero of B's uploads") + + var content string + require.NoError(t, super.QueryRow(context.Background(), + `SELECT content FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&content)) + require.Equal(t, "text owned by B", content, "B's upload must survive A's writes unchanged") +} + +func TestUploadedTranscriptsRLS_InsertForAnotherUserIsRejected(t *testing.T) { + newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + _, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, userB) + require.NoError(t, err) + app := appPool(t, super) + + ctx := context.Background() + tx, err := app.Begin(ctx) + require.NoError(t, err) + defer tx.Rollback(ctx) //nolint:errcheck + _, err = tx.Exec(ctx, `SELECT set_config('tapir.current_user_id', $1, true)`, userA) + require.NoError(t, err) + + // Scoped as A, plant a row that claims to belong to B. The policy's WITH CHECK + // (defaulting to USING under FOR ALL) must refuse it. + _, err = tx.Exec(ctx, + `INSERT INTO uploaded_transcripts (user_id, title, status, content) + VALUES ($1, 'planted', 'done', 'planted')`, userB) + require.Error(t, err, "a row scoped as A must not be insertable on behalf of B") + require.Contains(t, err.Error(), "row-level security") +} + +func TestUploadedTranscripts_RowSecurityIsEnabledForcedAndKeyedByUser(t *testing.T) { + newStore(t) + super := rawPool(t) + ctx := context.Background() + + var enabled, forced bool + require.NoError(t, super.QueryRow(ctx, + `SELECT relrowsecurity, relforcerowsecurity FROM pg_class + WHERE oid = 'public.uploaded_transcripts'::regclass`).Scan(&enabled, &forced)) + require.True(t, enabled, "uploaded_transcripts must have row-level security enabled") + require.True(t, forced, + "FORCE is what binds the table-owner role tapir; without it the policy is dead in production") + + var userIDNotNull bool + require.NoError(t, super.QueryRow(ctx, + `SELECT is_nullable = 'NO' FROM information_schema.columns + WHERE table_name = 'uploaded_transcripts' AND column_name = 'user_id'`).Scan(&userIDNotNull)) + require.True(t, userIDNotNull, "an upload with no owner would match no policy and be unreachable") +} + +func TestUploadedTranscripts_DeleteUserRemovesTheirUploads(t *testing.T) { + s := newStore(t) + super := rawPool(t) + resetDB(t, super) + seedUploadedTranscript(t, super, userA, "text owned by A") + bID := seedUploadedTranscript(t, super, userB, "text owned by B") + + require.NoError(t, s.DeleteUser(context.Background(), userA)) + + var aRows, bRows int + ctx := context.Background() + require.NoError(t, super.QueryRow(ctx, + `SELECT count(*) FROM uploaded_transcripts WHERE user_id = $1`, userA).Scan(&aRows)) + require.NoError(t, super.QueryRow(ctx, + `SELECT count(*) FROM uploaded_transcripts WHERE id = $1`, bID).Scan(&bRows)) + require.Equal(t, 0, aRows, "deleting a user must remove their private transcripts (ON DELETE CASCADE)") + require.Equal(t, 1, bRows, "deleting A must not touch B's transcripts") +} + +// rowSecurityOffTables are the public-schema tables that deliberately have row +// security OFF. The list is exact on purpose: a NEW table added without RLS must +// fail this test and force a decision, instead of passing CI by omission — the +// failure mode ADR-033 names, where a naive reuse "leaks private content across +// tenants and passes CI green". +// +// - transcripts: shared public caption content, the single exception for CONTENT +// (ADR-021). It is the one that must never gain a private row. +// - user_identities: the Dex-subject -> user_id map, read at login before any +// user is known (migration 004). Holds no user content. +// - invitations: exists before the user does, so there is no user to scope by; +// the single-use token is the capability (migration 009). +// - schema_migrations: golang-migrate bookkeeping. +// +// #33 describes `transcripts` as "the only non-RLS table". That is true of content +// and false of tables; this test asserts the real set. +var rowSecurityOffTables = []string{ + "invitations", + "schema_migrations", + "transcripts", + "user_identities", +} + +func TestRowSecurityOffTablesAreExactlyTheKnownSet(t *testing.T) { + newStore(t) + super := rawPool(t) + + rows, err := super.Query(context.Background(), + `SELECT c.relname FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p') AND NOT c.relrowsecurity + ORDER BY c.relname`) + require.NoError(t, err) + defer rows.Close() + var got []string + for rows.Next() { + var name string + require.NoError(t, rows.Scan(&name)) + got = append(got, name) + } + require.NoError(t, rows.Err()) + + want := append([]string{}, rowSecurityOffTables...) + sort.Strings(want) + require.Equal(t, want, got, + "every public table must have row security ON unless it is in rowSecurityOffTables; "+ + "adding a table here is a decision that needs an ADR") +} -- 2.54.0 From e36c4c1e0ca8690dc690cc0a2ea6d16f545a7008 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:36:50 +0200 Subject: [PATCH 004/299] feat(store): migration 017 -- private RLS-scoped uploaded_transcripts (ADR-033) Separate table, not a user_id on the shared transcripts (ADR-021). ENABLE + FORCE ROW LEVEL SECURITY with the migration-003 policy, user_id NOT NULL ON DELETE CASCADE, typed error_kind, per-row serving endpoint, JSONB segments. CHECK constraints: done => content, failed => error_kind. Up and down are one commit: golang-migrate pairs them. Mutation-checked: dropping FORCE, the policy, or the cascade each turn a test red. FORCE is only observable structurally -- the app role is not the table owner, so no behavioural test can see it. Co-Authored-By: Claude Sonnet 5.5 --- .../017_uploaded_transcripts.down.sql | 3 ++ .../017_uploaded_transcripts.up.sql | 48 +++++++++++++++++++ 2 files changed, 51 insertions(+) create mode 100644 internal/adapters/store/migrations/017_uploaded_transcripts.down.sql create mode 100644 internal/adapters/store/migrations/017_uploaded_transcripts.up.sql diff --git a/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql b/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql new file mode 100644 index 0000000..8040680 --- /dev/null +++ b/internal/adapters/store/migrations/017_uploaded_transcripts.down.sql @@ -0,0 +1,3 @@ +-- Drops the private uploaded-transcript store and every transcript in it. There +-- is no revert of the data: the original audio was never kept (ADR-033). +DROP TABLE uploaded_transcripts; diff --git a/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql b/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql new file mode 100644 index 0000000..bdd311a --- /dev/null +++ b/internal/adapters/store/migrations/017_uploaded_transcripts.up.sql @@ -0,0 +1,48 @@ +-- Migration 017: uploaded_transcripts — PRIVATE transcripts of audio a user +-- uploaded themselves (ADR-033, tapir#33). +-- +-- This is deliberately NOT the shared `transcripts` table (migration 015, +-- ADR-021). That table is non-RLS on the explicit grounds that it holds only +-- public caption content and nothing user-identifying. A recorded meeting is the +-- opposite: private third-party content owned by exactly one user. Reusing +-- `transcripts` — or adding a user_id to it with a partial policy — would put a +-- private row in the one table the isolation test asserts is shared, and would +-- pass CI green. A separate table keeps ADR-021's exception singular. +-- +-- Row-level security follows migration 003 exactly: ENABLE + FORCE (the app +-- connects as the table owner, which bypasses RLS unless forced), and a policy +-- keyed off the per-request GUC tapir.current_user_id. With no GUC set the +-- predicate is NULL and the table is deny-all. +-- +-- The original audio is never stored (ADR-033): this row is all that is kept. +-- segments holds the timestamped cues; re-transcribing 90 minutes costs ~7 +-- minutes, so discarding the timestamps would be irreversible. +CREATE TABLE uploaded_transcripts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, + title TEXT NOT NULL, + status TEXT NOT NULL DEFAULT 'queued' + CHECK (status IN ('queued', 'transcribing', 'done', 'failed')), + content TEXT, + segments JSONB, + endpoint TEXT, -- which backend served it (iguana | berget); a quality fact + audio_seconds DOUBLE PRECISION, + error_kind TEXT, -- typed, never a provider's error string (ADR-033) + created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), + finished_at TIMESTAMPTZ, + CONSTRAINT uploaded_transcripts_done_has_content + CHECK (status <> 'done' OR content IS NOT NULL), + CONSTRAINT uploaded_transcripts_failed_has_error_kind + CHECK (status <> 'failed' OR error_kind IS NOT NULL) +); + +CREATE INDEX idx_uploaded_transcripts_user ON uploaded_transcripts (user_id, created_at DESC); + +ALTER TABLE uploaded_transcripts ENABLE ROW LEVEL SECURITY; +ALTER TABLE uploaded_transcripts FORCE ROW LEVEL SECURITY; +CREATE POLICY uploaded_transcripts_isolation ON uploaded_transcripts + FOR ALL + USING (user_id = current_setting('tapir.current_user_id', true)::uuid); + +COMMENT ON TABLE uploaded_transcripts IS + 'PRIVATE per-user transcripts of user-supplied audio. RLS-scoped (ADR-012). Never merge into the shared transcripts table (ADR-021, ADR-033).'; -- 2.54.0 From 4b557209c8cef73e2393452a7913c52f4e8fee1e Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:37:30 +0200 Subject: [PATCH 005/299] test(store): Store API for uploaded transcripts (red) Lifecycle queued -> transcribing -> done|failed, per-row serving endpoint, segments JSONB round trip, owner-only get/write (not-found, not forbidden), list without transcript text, and the start-up sweep that fails unfinished rows as `interrupted` for every user (audio is not retained, so no resume). Red: CreateUpload and friends are undefined. Co-Authored-By: Claude Sonnet 5.5 --- .../store/uploaded_transcripts_test.go | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 internal/adapters/store/uploaded_transcripts_test.go diff --git a/internal/adapters/store/uploaded_transcripts_test.go b/internal/adapters/store/uploaded_transcripts_test.go new file mode 100644 index 0000000..38d3726 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts_test.go @@ -0,0 +1,188 @@ +package store_test + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// These exercise the Store API over uploaded_transcripts. The test Store connects +// as the superuser, which BYPASSES row-level security — so the cross-user cases +// here prove the query's own user_id predicate (defence in depth), while the +// database-enforced guarantee is proven separately, as the non-superuser app +// role, in uploaded_transcripts_isolation_test.go. Text is synthetic; this repo is +// public and no real transcript may appear in it (#33). + +func seedUsers(t *testing.T, ids ...string) { + t.Helper() + super := rawPool(t) + resetDB(t, super) + for _, id := range ids { + _, err := super.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1)`, id) + require.NoError(t, err) + } +} + +func TestCreateUpload_StartsQueuedAndIsOwnedByTheCreator(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + + id, err := s.CreateUpload(ctx, userA, "Teams export") + require.NoError(t, err) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, id, got.ID) + assert.Equal(t, userA, got.UserID) + assert.Equal(t, "Teams export", got.Title) + assert.Equal(t, domain.UploadQueued, got.Status) + assert.Empty(t, got.Content) + assert.True(t, got.FinishedAt.IsZero(), "an unfinished upload has no finish time") +} + +func TestUploadLifecycle_CompletedKeepsTextSegmentsEndpointAndDuration(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m") + require.NoError(t, err) + + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + mid, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadTranscribing, mid.Status) + + want := domain.Transcription{ + Text: "God morgon allihopa. Vi går igenom läget.", + Segments: []domain.Segment{ + {Start: 0.0, End: 1.5, Text: "God morgon allihopa."}, + {Start: 1.5, End: 4.25, Text: "Vi går igenom läget."}, + }, + Endpoint: "iguana/kb-whisper", + AudioSeconds: 4.25, + } + require.NoError(t, s.CompleteUpload(ctx, userA, id, want)) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadDone, got.Status) + assert.Equal(t, want.Text, got.Content) + assert.Equal(t, want.Segments, got.Segments, "segment times must survive the JSONB round trip") + assert.Equal(t, "iguana/kb-whisper", got.Endpoint, "the serving endpoint is a quality fact and is recorded per row") + assert.InDelta(t, 4.25, got.AudioSeconds, 1e-9) + assert.False(t, got.FinishedAt.IsZero()) + assert.Empty(t, got.ErrorKind) +} + +func TestUploadLifecycle_FailedRecordsTypedKindAndNoContent(t *testing.T) { + s := newStore(t) + seedUsers(t, userA) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "m") + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, id)) + + require.NoError(t, s.FailUpload(ctx, userA, id, domain.UploadErrTranscriptionFailed)) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadFailed, got.Status) + assert.Equal(t, domain.UploadErrTranscriptionFailed, got.ErrorKind) + assert.Empty(t, got.Content) + assert.False(t, got.FinishedAt.IsZero()) +} + +func TestGetUpload_AnotherUsersIDIsNotFound(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "A's meeting") + require.NoError(t, err) + + _, err = s.GetUpload(ctx, userB, id) + + assert.ErrorIs(t, err, store.ErrUploadNotFound, + "a non-owner must get not-found, not forbidden: existence is not disclosed") +} + +func TestUploadWrites_ByAnotherUserChangeNothing(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + id, err := s.CreateUpload(ctx, userA, "A's meeting") + require.NoError(t, err) + + assert.ErrorIs(t, s.StartTranscribing(ctx, userB, id), store.ErrUploadNotFound) + assert.ErrorIs(t, s.CompleteUpload(ctx, userB, id, domain.Transcription{Text: "planted"}), store.ErrUploadNotFound) + assert.ErrorIs(t, s.FailUpload(ctx, userB, id, domain.UploadErrTranscriptionFailed), store.ErrUploadNotFound) + + got, err := s.GetUpload(ctx, userA, id) + require.NoError(t, err) + assert.Equal(t, domain.UploadQueued, got.Status, "B's writes must leave A's upload exactly as it was") + assert.Empty(t, got.Content) +} + +func TestListUploads_ReturnsOnlyTheCallersNewestFirstWithoutTranscriptText(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + ctx := context.Background() + first, err := s.CreateUpload(ctx, userA, "first") + require.NoError(t, err) + second, err := s.CreateUpload(ctx, userA, "second") + require.NoError(t, err) + _, err = s.CreateUpload(ctx, userB, "B's") + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, first)) + require.NoError(t, s.CompleteUpload(ctx, userA, first, domain.Transcription{Text: "long text body", Endpoint: "e"})) + + got, err := s.ListUploads(ctx, userA) + + require.NoError(t, err) + require.Len(t, got, 2, "A must see only A's uploads") + assert.Equal(t, second, got[0].ID, "newest first") + assert.Equal(t, first, got[1].ID) + assert.Empty(t, got[1].Content, "a list row must not drag the whole transcript along") + assert.Equal(t, domain.UploadDone, got[1].Status) +} + +func TestFailInterruptedUploads_FailsOnlyUnfinishedRowsForEveryUser(t *testing.T) { + s := newStore(t) + seedUsers(t, userA, userB) + super := rawPool(t) + ctx := context.Background() + for subject, uid := range map[string]string{"sub-a": userA, "sub-b": userB} { + _, err := super.Exec(ctx, `INSERT INTO user_identities (dex_subject, user_id) VALUES ($1, $2)`, subject, uid) + require.NoError(t, err) + } + queued, err := s.CreateUpload(ctx, userA, "queued") + require.NoError(t, err) + running, err := s.CreateUpload(ctx, userB, "running") + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userB, running)) + done, err := s.CreateUpload(ctx, userA, "done") + require.NoError(t, err) + require.NoError(t, s.StartTranscribing(ctx, userA, done)) + require.NoError(t, s.CompleteUpload(ctx, userA, done, domain.Transcription{Text: "kept", Endpoint: "e"})) + + n, err := s.FailInterruptedUploads(ctx) + + require.NoError(t, err) + assert.Equal(t, 2, n, "the queued and the transcribing row, across both users") + for _, c := range []struct{ user, id string }{{userA, queued}, {userB, running}} { + got, err := s.GetUpload(ctx, c.user, c.id) + require.NoError(t, err) + assert.Equal(t, domain.UploadFailed, got.Status) + assert.Equal(t, domain.UploadErrInterrupted, got.ErrorKind, + "audio is not retained, so a restart cannot resume: the honest state is failed/interrupted") + } + kept, err := s.GetUpload(ctx, userA, done) + require.NoError(t, err) + assert.Equal(t, domain.UploadDone, kept.Status, "a finished transcript must survive the sweep") + assert.Equal(t, "kept", kept.Content) +} -- 2.54.0 From c29c0a7127634c2fcab33c77d50707eb439a8538 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:37:39 +0200 Subject: [PATCH 006/299] feat(domain): types for private uploaded transcripts (ADR-033) Segment, Transcription, UploadStatus, UploadErrorKind and UploadedTranscript. Error kinds are typed, never a provider's string. UploadedTranscript is a distinct type from the shared public-caption Transcript on purpose. Co-Authored-By: Claude Sonnet 5.5 --- internal/domain/domain.go | 59 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/internal/domain/domain.go b/internal/domain/domain.go index 5cf988b..374678f 100644 --- a/internal/domain/domain.go +++ b/internal/domain/domain.go @@ -111,3 +111,62 @@ type Summary struct { FallbackUsed bool CreatedAt time.Time } + +// Segment is one timed span of a transcription. Times are seconds from the start +// of the audio. +type Segment struct { + Start float64 `json:"start"` + End float64 `json:"end"` + Text string `json:"text"` +} + +// Transcription is the result of transcribing user-supplied audio (ADR-033): +// the full text, its timed segments, which backend served it, and how long the +// audio was. Endpoint is recorded because the backends differ in timestamp +// granularity — which one served a transcript is a quality fact. +type Transcription struct { + Text string + Segments []Segment + Endpoint string + AudioSeconds float64 +} + +// UploadStatus is the state of an uploaded-audio transcription job. +type UploadStatus string + +const ( + UploadQueued UploadStatus = "queued" + UploadTranscribing UploadStatus = "transcribing" + UploadDone UploadStatus = "done" + UploadFailed UploadStatus = "failed" +) + +// UploadErrorKind is the typed reason an upload failed. It is deliberately NOT a +// provider's error string: the whisper server's "failed to process audio" has had +// at least five distinct causes, so nothing may branch on it (ADR-033). +type UploadErrorKind string + +const ( + // UploadErrInterrupted: the process restarted mid-job. The audio is not + // retained, so the job cannot resume; the user re-uploads. + UploadErrInterrupted UploadErrorKind = "interrupted" + // UploadErrTranscriptionFailed: every configured transcription endpoint failed. + UploadErrTranscriptionFailed UploadErrorKind = "transcription_failed" +) + +// UploadedTranscript is a user's PRIVATE transcript of audio they uploaded. It is +// never stored in, or merged with, the shared public-caption Transcript (ADR-021). +// Content and Segments are empty until Status is UploadDone. +type UploadedTranscript struct { + ID string + UserID string + Title string + Status UploadStatus + Content string + Segments []Segment + Endpoint string + AudioSeconds float64 + ErrorKind UploadErrorKind + CreatedAt time.Time + FinishedAt time.Time // zero until the job reaches done or failed +} -- 2.54.0 From 836a501576a89d1da6938bd9e15ba38eac4fada1 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:38:25 +0200 Subject: [PATCH 007/299] feat(store): Store API for private uploaded transcripts (ADR-033) Create/Start/Complete/Fail/Get/List plus the start-up sweep that fails unfinished rows as interrupted (audio is not retained, so no resume). Every call goes through withUser AND names user_id in its predicate, so isolation does not rest on RLS alone and stays testable under the superuser test connection; a non-owner gets ErrUploadNotFound, never a distinct forbidden. List drops transcript text. Mutation-checked: removing the user predicate from Get or from the UPDATEs turns the cross-user tests red. Co-Authored-By: Claude Sonnet 5.5 --- .../adapters/store/uploaded_transcripts.go | 199 ++++++++++++++++++ 1 file changed, 199 insertions(+) create mode 100644 internal/adapters/store/uploaded_transcripts.go diff --git a/internal/adapters/store/uploaded_transcripts.go b/internal/adapters/store/uploaded_transcripts.go new file mode 100644 index 0000000..7252761 --- /dev/null +++ b/internal/adapters/store/uploaded_transcripts.go @@ -0,0 +1,199 @@ +package store + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "time" + + "github.com/jackc/pgx/v5" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrUploadNotFound is returned when an uploaded transcript does not exist OR is +// not the caller's. The two are deliberately indistinguishable: a non-owner learns +// nothing about whether someone else's upload exists (ADR-033). +var ErrUploadNotFound = errors.New("store: uploaded transcript not found") + +// Uploaded transcripts are PRIVATE per-user content, stored in their own +// RLS-scoped table (migration 017) — never in the shared `transcripts` (ADR-021). +// Every method goes through withUser, so row-level security confines it to the +// caller's rows; each statement ALSO names user_id in its predicate, so the +// isolation does not rest on a single mechanism and stays testable under the +// superuser connection the unit tests use. + +// CreateUpload records a new upload in the queued state and returns its id. +func (s *Store) CreateUpload(ctx context.Context, userID, title string) (string, error) { + var id string + if err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + return tx.QueryRow(ctx, + `INSERT INTO uploaded_transcripts (user_id, title) VALUES ($1, $2) RETURNING id`, + userID, title).Scan(&id) + }); err != nil { + return "", fmt.Errorf("store: create upload: %w", err) + } + return id, nil +} + +// StartTranscribing moves an upload to transcribing. +func (s *Store) StartTranscribing(ctx context.Context, userID, id string) error { + return s.updateUpload(ctx, userID, id, "start transcribing", + `UPDATE uploaded_transcripts SET status = 'transcribing' + WHERE id = $1 AND user_id = $2`) +} + +// CompleteUpload stores the finished transcription and moves the upload to done. +func (s *Store) CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error { + segs, err := json.Marshal(t.Segments) + if err != nil { + return fmt.Errorf("store: complete upload: marshal segments: %w", err) + } + return s.updateUpload(ctx, userID, id, "complete upload", + `UPDATE uploaded_transcripts + SET status = 'done', content = $3, segments = $4, endpoint = $5, + audio_seconds = $6, error_kind = NULL, finished_at = NOW() + WHERE id = $1 AND user_id = $2`, + t.Text, segs, t.Endpoint, t.AudioSeconds) +} + +// FailUpload moves an upload to failed with a typed reason. +func (s *Store) FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error { + return s.updateUpload(ctx, userID, id, "fail upload", + `UPDATE uploaded_transcripts + SET status = 'failed', error_kind = $3, finished_at = NOW() + WHERE id = $1 AND user_id = $2`, + string(kind)) +} + +// updateUpload runs one UPDATE keyed on ($1 = id, $2 = userID) plus extra args, +// and maps "no row touched" to ErrUploadNotFound — which is what a write to another +// user's upload looks like under RLS (the row is invisible, not forbidden). +func (s *Store) updateUpload(ctx context.Context, userID, id, op, sql string, args ...any) error { + all := append([]any{id, userID}, args...) + return s.withUser(ctx, userID, func(tx pgx.Tx) error { + ct, err := tx.Exec(ctx, sql, all...) + if err != nil { + return fmt.Errorf("store: %s: %w", op, err) + } + if ct.RowsAffected() == 0 { + return ErrUploadNotFound + } + return nil + }) +} + +// GetUpload returns one of the caller's uploads, including the transcript text. +func (s *Store) GetUpload(ctx context.Context, userID, id string) (*domain.UploadedTranscript, error) { + var u domain.UploadedTranscript + err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + row := tx.QueryRow(ctx, uploadSelect+` WHERE id = $1 AND user_id = $2`, id, userID) + return scanUpload(row, &u, true) + }) + if errors.Is(err, pgx.ErrNoRows) { + return nil, ErrUploadNotFound + } + if err != nil { + return nil, fmt.Errorf("store: get upload: %w", err) + } + return &u, nil +} + +// ListUploads returns the caller's uploads, newest first, WITHOUT transcript text +// or segments: a list view must not drag every transcript along. +func (s *Store) ListUploads(ctx context.Context, userID string) ([]domain.UploadedTranscript, error) { + var out []domain.UploadedTranscript + err := s.withUser(ctx, userID, func(tx pgx.Tx) error { + rows, err := tx.Query(ctx, uploadSelect+` WHERE user_id = $1 ORDER BY created_at DESC, id`, userID) + if err != nil { + return err + } + defer rows.Close() + for rows.Next() { + var u domain.UploadedTranscript + if err := scanUpload(rows, &u, false); err != nil { + return err + } + out = append(out, u) + } + return rows.Err() + }) + if err != nil { + return nil, fmt.Errorf("store: list uploads: %w", err) + } + return out, nil +} + +// FailInterruptedUploads is the start-up sweep. The audio of an upload is never +// retained (ADR-033), so a job that was queued or transcribing when the process +// stopped can never resume; leaving it would be a stuck spinner. Every such row, +// for every user, becomes failed/interrupted and the user re-uploads. It +// enumerates users from the un-RLS'd identity map and scopes each pass with +// withUser, exactly as scheduled discovery does (ADR-018). Returns rows failed. +func (s *Store) FailInterruptedUploads(ctx context.Context) (int, error) { + users, err := s.ListAllUsers(ctx) + if err != nil { + return 0, fmt.Errorf("store: fail interrupted uploads: %w", err) + } + total := 0 + for _, u := range users { + err := s.withUser(ctx, u.UserID, func(tx pgx.Tx) error { + ct, err := tx.Exec(ctx, + `UPDATE uploaded_transcripts + SET status = 'failed', error_kind = $2, finished_at = NOW() + WHERE user_id = $1 AND status IN ('queued', 'transcribing')`, + u.UserID, string(domain.UploadErrInterrupted)) + total += int(ct.RowsAffected()) + return err + }) + if err != nil { + return total, fmt.Errorf("store: fail interrupted uploads: user %s: %w", u.UserID, err) + } + } + return total, nil +} + +const uploadSelect = `SELECT id, user_id, title, status, content, segments, endpoint, + audio_seconds, error_kind, created_at, finished_at FROM uploaded_transcripts` + +// scanUpload scans one uploadSelect row. withText controls whether the transcript +// body and segments are kept (GetUpload) or dropped (ListUploads). +func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error { + var ( + status, errKind, endpoint, content *string + segments []byte + seconds *float64 + finished *time.Time + ) + if err := row.Scan(&u.ID, &u.UserID, &u.Title, &status, &content, &segments, &endpoint, + &seconds, &errKind, &u.CreatedAt, &finished); err != nil { + return err + } + u.Status = domain.UploadStatus(deref(status)) + u.Endpoint = deref(endpoint) + u.ErrorKind = domain.UploadErrorKind(deref(errKind)) + if seconds != nil { + u.AudioSeconds = *seconds + } + if finished != nil { + u.FinishedAt = *finished + } + if !withText { + return nil + } + u.Content = deref(content) + if len(segments) > 0 { + if err := json.Unmarshal(segments, &u.Segments); err != nil { + return fmt.Errorf("unmarshal segments: %w", err) + } + } + return nil +} + +func deref(s *string) string { + if s == nil { + return "" + } + return *s +} -- 2.54.0 From f42b8e23fcddc5fba5ad1d9fad12f22226c384de Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:39:20 +0200 Subject: [PATCH 008/299] test(uploads): validation policy -- video, no-audio, too-long, unreadable (red) Probe-based, not extension-based: a Teams voice-only export is typically an .mp4 with no video stream, indistinguishable from video by name (ADR-033). Red: package uploads does not exist. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/validate_test.go | 95 +++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 internal/uploads/validate_test.go diff --git a/internal/uploads/validate_test.go b/internal/uploads/validate_test.go new file mode 100644 index 0000000..b8dc4c6 --- /dev/null +++ b/internal/uploads/validate_test.go @@ -0,0 +1,95 @@ +package uploads_test + +import ( + "errors" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +var testLimits = uploads.Limits{MaxBytes: 200 << 20, MaxDuration: 90 * time.Minute} + +func TestValidate(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + wantReason uploads.Reason // "" = accepted + wantInMsg []string + }{ + { + name: "audio-only file within the duration limit is accepted", + probe: uploads.Probe{Duration: 45 * time.Minute, HasAudio: true}, + }, + { + name: "a recording exactly at the limit is accepted", + probe: uploads.Probe{Duration: 90 * time.Minute, HasAudio: true}, + }, + { + name: "a file with a video stream is rejected and names issue 28", + probe: uploads.Probe{Duration: 4 * time.Minute, HasAudio: true, HasVideo: true}, + wantReason: uploads.ReasonVideo, + wantInMsg: []string{"video", "#28"}, + }, + { + name: "video with no audio is still rejected as video, not as missing audio", + probe: uploads.Probe{Duration: 4 * time.Minute, HasVideo: true}, + wantReason: uploads.ReasonVideo, + wantInMsg: []string{"#28"}, + }, + { + name: "a file with no audio stream is rejected", + probe: uploads.Probe{Duration: 4 * time.Minute}, + wantReason: uploads.ReasonNoAudio, + wantInMsg: []string{"no audio"}, + }, + { + name: "a recording over the limit is rejected, stating both lengths", + probe: uploads.Probe{Duration: 102 * time.Minute, HasAudio: true}, + wantReason: uploads.ReasonTooLong, + wantInMsg: []string{"1 h 42 min", "1 h 30 min"}, + }, + { + name: "zero duration is unreadable, not accepted as an empty recording", + probe: uploads.Probe{HasAudio: true}, + wantReason: uploads.ReasonUnreadable, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + err := uploads.Validate(tt.probe, testLimits) + + if tt.wantReason == "" { + require.NoError(t, err) + return + } + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej), "want a *RejectError, got %v", err) + assert.Equal(t, tt.wantReason, rej.Reason) + for _, s := range tt.wantInMsg { + assert.Contains(t, rej.Message, s) + } + }) + } +} + +func TestFormatDuration(t *testing.T) { + tests := []struct { + d time.Duration + want string + }{ + {90 * time.Minute, "1 h 30 min"}, + {102 * time.Minute, "1 h 42 min"}, + {45 * time.Minute, "45 min"}, + {2 * time.Hour, "2 h"}, + {30 * time.Second, "30 s"}, + } + for _, tt := range tests { + t.Run(tt.want, func(t *testing.T) { + assert.Equal(t, tt.want, uploads.FormatDuration(tt.d)) + }) + } +} -- 2.54.0 From e3e1f17c40a2b7172e5d1c00d433c749dcfa92f7 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:39:51 +0200 Subject: [PATCH 009/299] feat(uploads): validation policy for probed audio uploads (ADR-033) Video is checked first so it is always refused as video; limits and reasons are typed so callers and tests branch on Reason, not message text. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/validate.go | 82 ++++++++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 internal/uploads/validate.go diff --git a/internal/uploads/validate.go b/internal/uploads/validate.go new file mode 100644 index 0000000..8dfea85 --- /dev/null +++ b/internal/uploads/validate.go @@ -0,0 +1,82 @@ +// Package uploads is the use case behind "upload an audio recording" (ADR-033): +// validate the file by PROBING it, keep the transcript private, never retain the +// audio. It depends only on ports (a Prober, a store, a job Submitter) — the +// concrete ffprobe adapter, the web handler and the transcription worker are wired +// from outside. +package uploads + +import ( + "fmt" + "time" +) + +// Limits bound what an upload may be. Both are rejected synchronously, with a +// specific message, rather than by a timeout (ADR-033). +type Limits struct { + MaxBytes int64 + MaxDuration time.Duration +} + +// Probe is what inspecting an uploaded file revealed. It is the input to Validate +// because a name or MIME type cannot say whether a container carries video: a +// Teams voice-only export is typically an .mp4 with no video stream. +type Probe struct { + Duration time.Duration + HasAudio bool + HasVideo bool +} + +// Reason is the machine-checkable class of a rejection. +type Reason string + +const ( + ReasonVideo Reason = "video" + ReasonNoAudio Reason = "no_audio" + ReasonTooLong Reason = "too_long" + ReasonTooLarge Reason = "too_large" + ReasonUnreadable Reason = "unreadable" +) + +// RejectError is a synchronous, user-presentable refusal. Message is written for +// the person who uploaded the file; Reason is for code and tests. +type RejectError struct { + Reason Reason + Message string +} + +func (e *RejectError) Error() string { return string(e.Reason) + ": " + e.Message } + +// Validate decides whether a probed file may be transcribed. Video is checked +// first so a video file is always refused as video, whatever its length. +func Validate(p Probe, l Limits) error { + switch { + case p.HasVideo: + return &RejectError{Reason: ReasonVideo, Message: "This file contains a video track. " + + "Only audio recordings are supported for now; video upload is tracked in issue #28."} + case !p.HasAudio: + return &RejectError{Reason: ReasonNoAudio, Message: "This file has no audio track to transcribe."} + case p.Duration <= 0: + return &RejectError{Reason: ReasonUnreadable, Message: "Could not read a length from this file; it may be damaged."} + case p.Duration > l.MaxDuration: + return &RejectError{Reason: ReasonTooLong, Message: fmt.Sprintf( + "This recording is %s long; the limit is %s.", FormatDuration(p.Duration), FormatDuration(l.MaxDuration))} + } + return nil +} + +// FormatDuration renders a length for people: "1 h 42 min", "45 min", "30 s". +func FormatDuration(d time.Duration) string { + d = d.Round(time.Second) + h := int(d / time.Hour) + m := int(d % time.Hour / time.Minute) + switch { + case h > 0 && m > 0: + return fmt.Sprintf("%d h %d min", h, m) + case h > 0: + return fmt.Sprintf("%d h", h) + case m > 0: + return fmt.Sprintf("%d min", m) + default: + return fmt.Sprintf("%d s", int(d/time.Second)) + } +} -- 2.54.0 From b98e9df46214762511872d66a0aedd05b6ad395f Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:40:03 +0200 Subject: [PATCH 010/299] test(uploads): Accept -- size cap, probe verdicts, cleanup on every failure path (red) Red: uploads.Job, Intake and ErrUnreadable are undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/intake_test.go | 212 ++++++++++++++++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 internal/uploads/intake_test.go diff --git a/internal/uploads/intake_test.go b/internal/uploads/intake_test.go new file mode 100644 index 0000000..0fcafc1 --- /dev/null +++ b/internal/uploads/intake_test.go @@ -0,0 +1,212 @@ +package uploads_test + +import ( + "bytes" + "context" + "errors" + "io" + "os" + "path/filepath" + "strings" + "testing" + "testing/iotest" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +type fakeStore struct { + created []string // titles + failed map[string]domain.UploadErrorKind + nextID string +} + +func (f *fakeStore) CreateUpload(_ context.Context, _, title string) (string, error) { + f.created = append(f.created, title) + return f.nextID, nil +} + +func (f *fakeStore) FailUpload(_ context.Context, _, id string, kind domain.UploadErrorKind) error { + if f.failed == nil { + f.failed = map[string]domain.UploadErrorKind{} + } + f.failed[id] = kind + return nil +} + +type fakeProber struct { + probe uploads.Probe + err error + paths []string + seen []byte // content of the file at probe time +} + +func (f *fakeProber) Probe(_ context.Context, path string) (uploads.Probe, error) { + f.paths = append(f.paths, path) + f.seen, _ = os.ReadFile(path) + return f.probe, f.err +} + +type fakeSubmitter struct { + jobs []uploads.Job + err error +} + +func (f *fakeSubmitter) Submit(j uploads.Job) error { + f.jobs = append(f.jobs, j) + return f.err +} + +type rig struct { + in *uploads.Intake + st *fakeStore + pr *fakeProber + sub *fakeSubmitter + tmp string +} + +func newRig(t *testing.T) *rig { + t.Helper() + r := &rig{ + st: &fakeStore{nextID: "up-1"}, + pr: &fakeProber{probe: uploads.Probe{Duration: 10 * time.Minute, HasAudio: true}}, + sub: &fakeSubmitter{}, + tmp: t.TempDir(), + } + r.in = uploads.NewIntake(r.st, r.pr, r.sub, uploads.Limits{MaxBytes: 1024, MaxDuration: 90 * time.Minute}, r.tmp) + return r +} + +func (r *rig) tmpFiles(t *testing.T) []string { + t.Helper() + ents, err := os.ReadDir(r.tmp) + require.NoError(t, err) + var names []string + for _, e := range ents { + names = append(names, e.Name()) + } + return names +} + +func TestAccept_ValidAudioIsStoredProbedQueuedAndSubmitted(t *testing.T) { + r := newRig(t) + audio := bytes.Repeat([]byte("a"), 500) + + id, err := r.in.Accept(context.Background(), "user-1", "Teams export", bytes.NewReader(audio)) + + require.NoError(t, err) + assert.Equal(t, "up-1", id) + assert.Equal(t, []string{"Teams export"}, r.st.created) + assert.Equal(t, audio, r.pr.seen, "the prober must see the complete uploaded bytes, on disk") + require.Len(t, r.sub.jobs, 1) + job := r.sub.jobs[0] + assert.Equal(t, uploads.Job{UserID: "user-1", UploadID: "up-1", Path: job.Path}, job) + assert.Equal(t, filepath.Dir(job.Path), r.tmp, "the temp file lives in the configured directory") + got, err := os.ReadFile(job.Path) + require.NoError(t, err) + assert.Equal(t, audio, got, "the handed-over file still exists: the worker, not Accept, deletes it") +} + +func TestAccept_OversizeBodyIsRejectedAndLeavesNothingBehind(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "big", strings.NewReader(strings.Repeat("a", 1025))) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej)) + assert.Equal(t, uploads.ReasonTooLarge, rej.Reason) + assert.Contains(t, rej.Message, "limit") + assert.Empty(t, r.pr.paths, "a file already over the size cap must not be probed") + assert.Empty(t, r.st.created, "a rejected upload creates no row") + assert.Empty(t, r.sub.jobs) + assert.Empty(t, r.tmpFiles(t), "the partial temp file must be removed") +} + +func TestAccept_BodyExactlyAtTheLimitIsAccepted(t *testing.T) { + r := newRig(t) + + _, err := r.in.Accept(context.Background(), "user-1", "edge", strings.NewReader(strings.Repeat("a", 1024))) + + require.NoError(t, err) +} + +func TestAccept_ProbeVerdictRejectsWithoutRowAndRemovesTheFile(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + reason uploads.Reason + }{ + {"video", uploads.Probe{Duration: time.Minute, HasAudio: true, HasVideo: true}, uploads.ReasonVideo}, + {"too long", uploads.Probe{Duration: 2 * time.Hour, HasAudio: true}, uploads.ReasonTooLong}, + {"no audio", uploads.Probe{Duration: time.Minute}, uploads.ReasonNoAudio}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newRig(t) + r.pr.probe = tt.probe + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej)) + assert.Equal(t, tt.reason, rej.Reason) + assert.Empty(t, r.st.created) + assert.Empty(t, r.sub.jobs) + assert.Empty(t, r.tmpFiles(t)) + }) + } +} + +func TestAccept_UnreadableFileIsARejectionButAProberFaultIsAnError(t *testing.T) { + t.Run("ffprobe says the file is not media", func(t *testing.T) { + r := newRig(t) + r.pr.err = uploads.ErrUnreadable + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("not media")) + + var rej *uploads.RejectError + require.True(t, errors.As(err, &rej), "the user's file is the problem: tell them") + assert.Equal(t, uploads.ReasonUnreadable, rej.Reason) + assert.Empty(t, r.tmpFiles(t)) + }) + t.Run("the prober itself broke", func(t *testing.T) { + r := newRig(t) + r.pr.err = errors.New("exec: ffprobe: executable file not found") + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.Error(t, err) + var rej *uploads.RejectError + assert.False(t, errors.As(err, &rej), "our fault must not be reported as the user's bad file") + assert.Empty(t, r.st.created) + assert.Empty(t, r.tmpFiles(t)) + }) +} + +func TestAccept_ClientDisconnectMidUploadLeavesNothingBehind(t *testing.T) { + r := newRig(t) + body := io.MultiReader(strings.NewReader("partial"), iotest.ErrReader(errors.New("client went away"))) + + _, err := r.in.Accept(context.Background(), "user-1", "t", body) + + require.Error(t, err) + assert.Empty(t, r.pr.paths) + assert.Empty(t, r.st.created) + assert.Empty(t, r.tmpFiles(t)) +} + +func TestAccept_SubmitFailureFailsTheRowHonestlyAndRemovesTheFile(t *testing.T) { + r := newRig(t) + r.sub.err = errors.New("worker stopped") + + _, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data")) + + require.Error(t, err) + assert.Equal(t, domain.UploadErrInterrupted, r.st.failed["up-1"], + "a row that will never run must not sit queued: that is the stuck spinner ADR-025 forbids") + assert.Empty(t, r.tmpFiles(t)) +} -- 2.54.0 From 1910d3dfb5284b1b033b64ed332744b7fb2d1461 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:40:13 +0200 Subject: [PATCH 011/299] feat(uploads): Intake -- spool, probe, validate, queue; clean up on every failure (ADR-033) Bounded read (cap+1 byte tells 'at the limit' from 'over'); a file that fails validation creates no row; a prober fault is our error, not the user's bad file; a row whose job cannot be submitted is failed, never left queued. Mutation-checked: cap off-by-one, missing cleanup and deleting the handed-over file each turn a test red. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/intake.go | 115 +++++++++++++++++++++++++++++++++++++ 1 file changed, 115 insertions(+) create mode 100644 internal/uploads/intake.go diff --git a/internal/uploads/intake.go b/internal/uploads/intake.go new file mode 100644 index 0000000..d33cba5 --- /dev/null +++ b/internal/uploads/intake.go @@ -0,0 +1,115 @@ +package uploads + +import ( + "context" + "errors" + "fmt" + "io" + "os" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrUnreadable is what a Prober returns when the FILE is the problem (not media, +// truncated, damaged), as opposed to the prober itself failing. Accept turns it +// into a user-facing rejection; any other prober error is our fault and surfaces +// as an internal error — a missing ffprobe must not read as "your file is bad". +var ErrUnreadable = errors.New("uploads: file is not readable media") + +// Prober inspects a file on disk. The ffprobe adapter implements it. +type Prober interface { + Probe(ctx context.Context, path string) (Probe, error) +} + +// Store is the slice of the private transcript store the intake needs. +type Store interface { + CreateUpload(ctx context.Context, userID, title string) (string, error) + FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error +} + +// Job is one accepted upload awaiting transcription. Path is a temp file the +// receiver owns: it must delete it when finished (the audio is never retained). +type Job struct { + UserID string + UploadID string + Path string +} + +// Submitter hands accepted jobs to the in-process worker (ADR-033). +type Submitter interface { + Submit(Job) error +} + +// Intake validates an uploaded recording and, if it passes, queues it. +type Intake struct { + store Store + prober Prober + sub Submitter + limits Limits + tmpDir string +} + +// NewIntake wires an Intake. tmpDir is where uploads are spooled; it must exist. +func NewIntake(s Store, p Prober, sub Submitter, l Limits, tmpDir string) *Intake { + return &Intake{store: s, prober: p, sub: sub, limits: l, tmpDir: tmpDir} +} + +// Accept spools src to disk (bounded by MaxBytes), probes it, validates it, records +// a queued upload and submits the job. It returns the upload id. A file that fails +// validation yields a *RejectError and creates NO row — rejection is synchronous +// and leaves nothing behind. On success the temp file is handed to the job and the +// worker deletes it; on every failure path Accept deletes it itself. +func (in *Intake) Accept(ctx context.Context, userID, title string, src io.Reader) (string, error) { + f, err := os.CreateTemp(in.tmpDir, "upload-*") + if err != nil { + return "", fmt.Errorf("uploads: spool: %w", err) + } + path := f.Name() + handedOver := false + defer func() { + if !handedOver { + _ = os.Remove(path) + } + }() + + // Read one byte past the cap so "exactly at the limit" and "over" are told apart. + n, err := io.Copy(f, io.LimitReader(src, in.limits.MaxBytes+1)) + if cerr := f.Close(); err == nil { + err = cerr + } + if err != nil { + return "", fmt.Errorf("uploads: receive: %w", err) + } + if n > in.limits.MaxBytes { + return "", &RejectError{Reason: ReasonTooLarge, Message: fmt.Sprintf( + "This file is larger than the %d MB limit.", in.limits.MaxBytes>>20)} + } + + probe, err := in.prober.Probe(ctx, path) + if errors.Is(err, ErrUnreadable) { + return "", &RejectError{ + Reason: ReasonUnreadable, + Message: "This file could not be read as audio; it may be damaged or not a recording.", + } + } + if err != nil { + return "", fmt.Errorf("uploads: probe: %w", err) + } + if err := Validate(probe, in.limits); err != nil { + return "", err + } + + id, err := in.store.CreateUpload(ctx, userID, title) + if err != nil { + return "", fmt.Errorf("uploads: create: %w", err) + } + if err := in.sub.Submit(Job{UserID: userID, UploadID: id, Path: path}); err != nil { + // A row that will never run must not sit queued: that is the stuck spinner. + if ferr := in.store.FailUpload(ctx, userID, id, domain.UploadErrInterrupted); ferr != nil { + return "", fmt.Errorf("uploads: submit: %w (and failing the row: %v)", err, ferr) + } + return "", fmt.Errorf("uploads: submit: %w", err) + } + handedOver = true + return id, nil +} -- 2.54.0 From 0ab067c5105853d3da11c2f909222ec16bd29e17 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:40:29 +0200 Subject: [PATCH 012/299] test(audio): ffprobe adapter -- cover art is not video, exit code vs missing binary (red) Red: package audio does not exist. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/audio/ffprobe_test.go | 129 ++++++++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 internal/adapters/audio/ffprobe_test.go diff --git a/internal/adapters/audio/ffprobe_test.go b/internal/adapters/audio/ffprobe_test.go new file mode 100644 index 0000000..609a84c --- /dev/null +++ b/internal/adapters/audio/ffprobe_test.go @@ -0,0 +1,129 @@ +package audio_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +func TestParseProbe(t *testing.T) { + tests := []struct { + name string + json string + want uploads.Probe + }{ + { + name: "audio-only m4a", + json: `{"streams":[{"codec_type":"audio","disposition":{"attached_pic":0}}],"format":{"duration":"2701.5"}}`, + want: uploads.Probe{Duration: 2701*time.Second + 500*time.Millisecond, HasAudio: true}, + }, + { + name: "mp4 with a real video stream", + json: `{"streams":[{"codec_type":"video","disposition":{"attached_pic":0}},{"codec_type":"audio"}],"format":{"duration":"240.0"}}`, + want: uploads.Probe{Duration: 240 * time.Second, HasAudio: true, HasVideo: true}, + }, + { + name: "mp3 whose only video stream is attached cover art is NOT video", + json: `{"streams":[{"codec_type":"audio"},{"codec_type":"video","disposition":{"attached_pic":1}}],"format":{"duration":"60"}}`, + want: uploads.Probe{Duration: 60 * time.Second, HasAudio: true}, + }, + { + name: "no streams at all", + json: `{"streams":[],"format":{"duration":"10"}}`, + want: uploads.Probe{Duration: 10 * time.Second}, + }, + { + name: "duration absent means zero, which Validate rejects as unreadable", + json: `{"streams":[{"codec_type":"audio"}],"format":{}}`, + want: uploads.Probe{HasAudio: true}, + }, + { + name: "duration reported as N/A means zero", + json: `{"streams":[{"codec_type":"audio"}],"format":{"duration":"N/A"}}`, + want: uploads.Probe{HasAudio: true}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, err := audio.ParseProbe([]byte(tt.json)) + + require.NoError(t, err) + assert.Equal(t, tt.want, got) + }) + } +} + +func TestParseProbe_GarbageIsAnError(t *testing.T) { + _, err := audio.ParseProbe([]byte("not json")) + assert.Error(t, err) +} + +// stub writes an executable shell script standing in for ffprobe. This is a fake +// at the process boundary: the real binary is not on every host (it is not on +// koala), and what is under test is how Probe drives and interprets a process. +func stub(t *testing.T, body string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "ffprobe") + require.NoError(t, os.WriteFile(p, []byte("#!/bin/sh\n"+body+"\n"), 0o755)) + return p +} + +func TestFFprobeProbe_ParsesWhatTheProcessPrints(t *testing.T) { + bin := stub(t, `echo '{"streams":[{"codec_type":"audio"}],"format":{"duration":"90"}}'`) + + got, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/ignored") + + require.NoError(t, err) + assert.Equal(t, uploads.Probe{Duration: 90 * time.Second, HasAudio: true}, got) +} + +func TestFFprobeProbe_PassesTheFilePathAsTheLastArgument(t *testing.T) { + out := filepath.Join(t.TempDir(), "args") + bin := stub(t, `for a; do last="$a"; done; printf '%s' "$last" > `+out+` +echo '{"streams":[{"codec_type":"audio"}],"format":{"duration":"1"}}'`) + + _, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/spool/upload-123") + + require.NoError(t, err) + got, err := os.ReadFile(out) + require.NoError(t, err) + assert.Equal(t, "/spool/upload-123", string(got)) +} + +func TestFFprobeProbe_NonzeroExitMeansTheFileIsUnreadable(t *testing.T) { + bin := stub(t, `echo "Invalid data found when processing input" >&2; exit 1`) + + _, err := (&audio.FFprobe{Bin: bin}).Probe(context.Background(), "/x") + + assert.ErrorIs(t, err, uploads.ErrUnreadable) +} + +func TestFFprobeProbe_MissingBinaryIsOurFaultNotAnUnreadableFile(t *testing.T) { + _, err := (&audio.FFprobe{Bin: filepath.Join(t.TempDir(), "no-such-ffprobe")}).Probe(context.Background(), "/x") + + require.Error(t, err) + assert.False(t, errors.Is(err, uploads.ErrUnreadable), + "a missing ffprobe must not be reported to the user as their file being bad") +} + +func TestFFprobeProbe_StopsWhenTheContextIsCancelled(t *testing.T) { + bin := stub(t, `sleep 30`) + ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond) + defer cancel() + + start := time.Now() + _, err := (&audio.FFprobe{Bin: bin}).Probe(ctx, "/x") + + require.Error(t, err) + assert.Less(t, time.Since(start), 5*time.Second) + assert.False(t, errors.Is(err, uploads.ErrUnreadable), "a timeout is not a verdict on the file") +} -- 2.54.0 From f84389028d31d35e083c5553432eac8d4def6de6 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:40:51 +0200 Subject: [PATCH 013/299] feat(audio): ffprobe adapter implementing uploads.Prober (ADR-033) Cover art is not video; non-zero exit is the file's fault (ErrUnreadable), a missing binary / cancelled ctx / bad output is ours. WaitDelay bounds a cancelled probe whose child holds the pipe open. Checked against real ffprobe on iguana: JSON shape matches the fixtures and /etc/hosts exits 1 with 'Invalid data found when processing input'. The unit tests drive a stub process; ffprobe is not installed on koala. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/audio/ffprobe.go | 95 ++++++++++++++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 internal/adapters/audio/ffprobe.go diff --git a/internal/adapters/audio/ffprobe.go b/internal/adapters/audio/ffprobe.go new file mode 100644 index 0000000..d4d27b6 --- /dev/null +++ b/internal/adapters/audio/ffprobe.go @@ -0,0 +1,95 @@ +// Package audio holds the adapters that inspect and convert user-supplied audio +// with ffmpeg's tools (ADR-033). They sit behind small ports so the use case is +// tested with fakes and only this package knows a subprocess is involved. +package audio + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "os/exec" + "strconv" + "time" + + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// FFprobe implements uploads.Prober by running ffprobe on the spooled file. +type FFprobe struct { + // Bin is the ffprobe executable ("ffprobe" resolves via PATH). + Bin string +} + +// killGrace bounds how long Probe waits for a cancelled ffprobe's pipes to close. +const killGrace = 2 * time.Second + +// Probe runs ffprobe and classifies the outcome. A non-zero exit means ffprobe +// looked at the file and could not read it: that is the FILE's fault and returns +// uploads.ErrUnreadable. A missing binary, a cancelled context or unparsable +// output is OURS and is returned as an ordinary error, so it is never reported to +// the user as their file being bad. +func (f *FFprobe) Probe(ctx context.Context, path string) (uploads.Probe, error) { + bin := f.Bin + if bin == "" { + bin = "ffprobe" + } + cmd := exec.CommandContext(ctx, bin, + "-v", "error", "-print_format", "json", "-show_format", "-show_streams", path) + cmd.WaitDelay = killGrace + out, err := cmd.Output() + if err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return uploads.Probe{}, fmt.Errorf("audio: ffprobe: %w", ctxErr) + } + var exit *exec.ExitError + if errors.As(err, &exit) { + return uploads.Probe{}, fmt.Errorf("%w: ffprobe: %s", uploads.ErrUnreadable, exit.Stderr) + } + return uploads.Probe{}, fmt.Errorf("audio: run ffprobe: %w", err) + } + p, err := ParseProbe(out) + if err != nil { + return uploads.Probe{}, fmt.Errorf("audio: ffprobe output: %w", err) + } + return p, nil +} + +// probeJSON is the subset of `ffprobe -print_format json` output that matters. +type probeJSON struct { + Streams []struct { + CodecType string `json:"codec_type"` + Disposition struct { + AttachedPic int `json:"attached_pic"` + } `json:"disposition"` + } `json:"streams"` + Format struct { + Duration string `json:"duration"` + } `json:"format"` +} + +// ParseProbe turns ffprobe's JSON into a uploads.Probe. A video stream that is +// only attached cover art (an mp3/m4a with an album picture) is not video. A +// missing or "N/A" duration yields zero, which uploads.Validate rejects as +// unreadable rather than accepting an empty recording. +func ParseProbe(data []byte) (uploads.Probe, error) { + var j probeJSON + if err := json.Unmarshal(data, &j); err != nil { + return uploads.Probe{}, fmt.Errorf("decode: %w", err) + } + var p uploads.Probe + for _, s := range j.Streams { + switch s.CodecType { + case "audio": + p.HasAudio = true + case "video": + if s.Disposition.AttachedPic == 0 { + p.HasVideo = true + } + } + } + if secs, err := strconv.ParseFloat(j.Format.Duration, 64); err == nil && secs > 0 { + p.Duration = time.Duration(secs * float64(time.Second)) + } + return p, nil +} -- 2.54.0 From e1ed6ea6f481e0b124b7dc0ac908ec3fb10bc889 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:41:42 +0200 Subject: [PATCH 014/299] test(web): upload routes -- form, multipart POST, rejections, owner-only, honest status (red) Real Router + real Intake + real Postgres; only ffprobe and the worker are faked. Covers: the form states the limits up front; accepted audio queues a row and redirects; video / too-long / too-large are rejected synchronously (422/422/413) with a message and NO row; a non-owner gets 404 without the title leaking; unfinished states keep polling and terminal states stop (no stuck spinner); a done transcript names the endpoint that served it. Red: App.Uploads / App.UploadLimits are undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/uploads_handler_test.go | 276 +++++++++++++++++++++++++++ 1 file changed, 276 insertions(+) create mode 100644 internal/web/uploads_handler_test.go diff --git a/internal/web/uploads_handler_test.go b/internal/web/uploads_handler_test.go new file mode 100644 index 0000000..2914e2c --- /dev/null +++ b/internal/web/uploads_handler_test.go @@ -0,0 +1,276 @@ +package web_test + +import ( + "bytes" + "context" + "mime/multipart" + "net/http" + "net/http/httptest" + "os" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/uploads" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// Upload route tests drive the real Router, the real Intake and the real Postgres +// store; only the process boundary (ffprobe) and the transcription worker are +// faked. All audio bytes and transcript text are synthetic — this repo is public +// and no real recording or transcript may appear in it (#33). + +const otherUserID = "22222222-2222-2222-2222-222222222222" + +type uploadProber struct{ probe uploads.Probe } + +func (p uploadProber) Probe(context.Context, string) (uploads.Probe, error) { return p.probe, nil } + +type uploadSubmitter struct{ jobs []uploads.Job } + +func (s *uploadSubmitter) Submit(j uploads.Job) error { s.jobs = append(s.jobs, j); return nil } + +func newUploadApp(t *testing.T, probe uploads.Probe, maxBytes int64) (*web.App, *uploadSubmitter) { + t.Helper() + app := newApp(t) + resetDB(t, rawPool(t)) + sub := &uploadSubmitter{} + limits := uploads.Limits{MaxBytes: maxBytes, MaxDuration: 90 * time.Minute} + s := newStore(t) + app.Uploads = uploads.NewIntake(s, uploadProber{probe: probe}, sub, limits, t.TempDir()) + app.UploadLimits = limits + return app, sub +} + +var audioOK = uploads.Probe{Duration: 45 * time.Minute, HasAudio: true} + +func uploadReq(t *testing.T, title, filename string, content []byte) *http.Request { + t.Helper() + var buf bytes.Buffer + mw := multipart.NewWriter(&buf) + if title != "" { + require.NoError(t, mw.WriteField("title", title)) + } + if filename != "" { + fw, err := mw.CreateFormFile("file", filename) + require.NoError(t, err) + _, err = fw.Write(content) + require.NoError(t, err) + } + require.NoError(t, mw.Close()) + req := httptest.NewRequest(http.MethodPost, "/uploads", &buf) + req.Header.Set("Content-Type", mw.FormDataContentType()) + return req +} + +func seedUpload(t *testing.T, owner, title, status, content string) string { + t.Helper() + p := rawPool(t) + _, err := p.Exec(context.Background(), `INSERT INTO users (id) VALUES ($1) ON CONFLICT DO NOTHING`, owner) + require.NoError(t, err) + var id string + switch status { + case "done": + require.NoError(t, p.QueryRow(context.Background(), + `INSERT INTO uploaded_transcripts (user_id, title, status, content, endpoint, audio_seconds, finished_at) + VALUES ($1, $2, 'done', $3, 'berget/kb-whisper', 2700, NOW()) RETURNING id`, + owner, title, content).Scan(&id)) + case "failed": + require.NoError(t, p.QueryRow(context.Background(), + `INSERT INTO uploaded_transcripts (user_id, title, status, error_kind, finished_at) + VALUES ($1, $2, 'failed', $3, NOW()) RETURNING id`, + owner, title, content).Scan(&id)) // content carries the error_kind here + default: + require.NoError(t, p.QueryRow(context.Background(), + `INSERT INTO uploaded_transcripts (user_id, title, status) VALUES ($1, $2, $3) RETURNING id`, + owner, title, status).Scan(&id)) + } + return id +} + +func get(path string) *http.Request { return httptest.NewRequest(http.MethodGet, path, nil) } + +func TestUploadsPage_ShowsAMultipartFormWithAFileField(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, get("/uploads")) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, `enctype="multipart/form-data"`) + assert.Contains(t, b, `name="file"`, "the multipart field is `file`") + assert.Contains(t, b, "200 MB", "the size limit is stated up front, not discovered by failing") + assert.Contains(t, b, "1 h 30 min", "the length limit is stated up front") +} + +func TestUploadsPage_ListsOnlyTheCallersUploads(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + seedUpload(t, userID, "My weekly sync", "queued", "") + seedUpload(t, otherUserID, "Someone else's board meeting", "queued", "") + + b := body(t, do(t, app, get("/uploads"))) + + assert.Contains(t, b, "My weekly sync") + assert.NotContains(t, b, "Someone else's board meeting") +} + +func TestUploadPost_AcceptedAudioQueuesARowAndRedirectsToIt(t *testing.T) { + app, sub := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, uploadReq(t, "Sprint review", "recording.m4a", []byte("synthetic audio bytes"))) + + require.Equal(t, http.StatusSeeOther, rec.Code) + var id, status, title string + require.NoError(t, rawPool(t).QueryRow(context.Background(), + `SELECT id, status, title FROM uploaded_transcripts WHERE user_id = $1`, userID).Scan(&id, &status, &title)) + assert.Equal(t, "queued", status) + assert.Equal(t, "Sprint review", title) + assert.Equal(t, "/uploads/"+id, rec.Header().Get("Location")) + require.Len(t, sub.jobs, 1) + assert.Equal(t, uploads.Job{UserID: userID, UploadID: id, Path: sub.jobs[0].Path}, sub.jobs[0]) + spooled, err := os.ReadFile(sub.jobs[0].Path) + require.NoError(t, err) + assert.Equal(t, "synthetic audio bytes", string(spooled)) +} + +func TestUploadPost_WithoutATitleUsesTheFilenameMinusItsExtension(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, uploadReq(t, "", "Teams meeting 2026-09-30.mp4", []byte("x"))) + + require.Equal(t, http.StatusSeeOther, rec.Code) + var title string + require.NoError(t, rawPool(t).QueryRow(context.Background(), + `SELECT title FROM uploaded_transcripts WHERE user_id = $1`, userID).Scan(&title)) + assert.Equal(t, "Teams meeting 2026-09-30", title) +} + +func TestUploadPost_RejectionsAreSynchronousWithAMessageAndNoRow(t *testing.T) { + tests := []struct { + name string + probe uploads.Probe + maxBytes int64 + content string + wantStatus int + wantInBody []string + }{ + { + name: "video names issue 28", + probe: uploads.Probe{Duration: time.Minute, HasAudio: true, HasVideo: true}, + maxBytes: 1 << 20, + content: "x", + wantStatus: http.StatusUnprocessableEntity, + wantInBody: []string{"video", "#28"}, + }, + { + name: "too long states both lengths", + probe: uploads.Probe{Duration: 2 * time.Hour, HasAudio: true}, + maxBytes: 1 << 20, + content: "x", + wantStatus: http.StatusUnprocessableEntity, + wantInBody: []string{"2 h", "1 h 30 min"}, + }, + { + name: "too large is 413", + probe: audioOK, + maxBytes: 4, + content: "more than four bytes", + wantStatus: http.StatusRequestEntityTooLarge, + wantInBody: []string{"limit"}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + app, sub := newUploadApp(t, tt.probe, tt.maxBytes) + + rec := do(t, app, uploadReq(t, "t", "r.m4a", []byte(tt.content))) + + assert.Equal(t, tt.wantStatus, rec.Code) + for _, s := range tt.wantInBody { + assert.Contains(t, body(t, rec), s) + } + var n int + require.NoError(t, rawPool(t).QueryRow(context.Background(), + `SELECT count(*) FROM uploaded_transcripts`).Scan(&n)) + assert.Zero(t, n, "a rejected upload leaves no row") + assert.Empty(t, sub.jobs) + }) + } +} + +func TestUploadPost_WithNoFilePartIs400(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + + rec := do(t, app, uploadReq(t, "only a title", "", nil)) + + assert.Equal(t, http.StatusBadRequest, rec.Code) + assert.Contains(t, body(t, rec), "choose") +} + +func TestUploadRoutes_AreNotMountedWithoutAnUploader(t *testing.T) { + app := newApp(t) + resetDB(t, rawPool(t)) + + assert.Equal(t, http.StatusNotFound, do(t, app, get("/uploads")).Code) +} + +func TestUploadPage_OwnerSeesTitleStatusAndNonOwnerGetsNotFound(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + mine := seedUpload(t, userID, "Roadmap sync", "queued", "") + theirs := seedUpload(t, otherUserID, "Their private meeting", "queued", "") + + own := do(t, app, get("/uploads/"+mine)) + require.Equal(t, http.StatusOK, own.Code) + assert.Contains(t, body(t, own), "Roadmap sync") + + for _, path := range []string{"/uploads/" + theirs, "/uploads/" + theirs + "/status"} { + rec := do(t, app, get(path)) + assert.Equal(t, http.StatusNotFound, rec.Code, path) + assert.NotContains(t, body(t, rec), "Their private meeting", "a 404 must not leak the title") + } +} + +func TestUploadStatus_UnfinishedStatesKeepPollingTerminalStatesStop(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + tests := []struct { + name, status, content string + polls bool + wantInBody string + }{ + {"queued", "queued", "", true, "queued"}, + {"transcribing", "transcribing", "", true, "ranscrib"}, + {"done", "done", "Synthetic transcript text.", false, "Synthetic transcript text."}, + {"failed interrupted", "failed", "interrupted", false, "upload it again"}, + {"failed transcription", "failed", "transcription_failed", false, "Transcription failed"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + id := seedUpload(t, userID, "m-"+tt.name, tt.status, tt.content) + + rec := do(t, app, get("/uploads/"+id+"/status")) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, tt.wantInBody) + if tt.polls { + assert.Contains(t, b, `hx-trigger="every`, "an unfinished upload must keep polling") + } else { + assert.NotContains(t, b, "hx-trigger", "a finished upload must not poll forever: no stuck spinner") + } + }) + } +} + +func TestUploadStatus_DoneNamesTheEndpointThatServedIt(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + id := seedUpload(t, userID, "m", "done", "Synthetic transcript text.") + + b := body(t, do(t, app, get("/uploads/"+id+"/status"))) + + assert.Contains(t, b, "berget/kb-whisper", + "which backend served a transcript is a quality fact, and berget means the audio left the homelab") + assert.True(t, strings.Contains(b, "45 min"), "the audio length is shown") +} -- 2.54.0 From 1bc98813fc029ff111ed2db2bc5833737e193b6f Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:42:40 +0200 Subject: [PATCH 015/299] test(web): read the response body once; match the message's real case Two defects in my red tests, found when going green: body() drains the recorder so a second call read empty, and two assertions expected a different case than the user-facing message. The assertions were not weakened. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/uploads_handler_test.go | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/internal/web/uploads_handler_test.go b/internal/web/uploads_handler_test.go index 2914e2c..c43e982 100644 --- a/internal/web/uploads_handler_test.go +++ b/internal/web/uploads_handler_test.go @@ -189,8 +189,9 @@ func TestUploadPost_RejectionsAreSynchronousWithAMessageAndNoRow(t *testing.T) { rec := do(t, app, uploadReq(t, "t", "r.m4a", []byte(tt.content))) assert.Equal(t, tt.wantStatus, rec.Code) + b := body(t, rec) for _, s := range tt.wantInBody { - assert.Contains(t, body(t, rec), s) + assert.Contains(t, b, s) } var n int require.NoError(t, rawPool(t).QueryRow(context.Background(), @@ -207,7 +208,7 @@ func TestUploadPost_WithNoFilePartIs400(t *testing.T) { rec := do(t, app, uploadReq(t, "only a title", "", nil)) assert.Equal(t, http.StatusBadRequest, rec.Code) - assert.Contains(t, body(t, rec), "choose") + assert.Contains(t, body(t, rec), "Choose an audio file") } func TestUploadRoutes_AreNotMountedWithoutAnUploader(t *testing.T) { @@ -240,7 +241,7 @@ func TestUploadStatus_UnfinishedStatesKeepPollingTerminalStatesStop(t *testing.T polls bool wantInBody string }{ - {"queued", "queued", "", true, "queued"}, + {"queued", "queued", "", true, "Queued"}, {"transcribing", "transcribing", "", true, "ranscrib"}, {"done", "done", "Synthetic transcript text.", false, "Synthetic transcript text."}, {"failed interrupted", "failed", "interrupted", false, "upload it again"}, -- 2.54.0 From e12a5e7e8959aed5a42caea0f76b9544496dbe46 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:42:58 +0200 Subject: [PATCH 016/299] feat(web): upload routes -- form, multipart intake, honest status (ADR-033) GET/POST /uploads, GET /uploads/{id}, GET /uploads/{id}/status, mounted only when an Uploader is wired. The multipart body is streamed to Intake, never buffered. Rejections are synchronous: 422 (video, too long, unreadable) or 413 with the user-facing message and no row. Another user's id is a 404 that does not leak the title. Unfinished states poll; done/failed render without hx-trigger so nothing polls forever (ADR-025). A done transcript names the endpoint that served it. handlers.go, uploads.go and the templ pair are one commit: the App fields and Store methods are what make the handlers compile. Mutation-checked: dropping the 413 mapping, the not-found mapping, or letting a done card poll each turn a test red. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/handlers.go | 21 ++ internal/web/uploads.go | 191 +++++++++++++++++ internal/web/uploads.templ | 91 ++++++++ internal/web/uploads_templ.go | 381 ++++++++++++++++++++++++++++++++++ 4 files changed, 684 insertions(+) create mode 100644 internal/web/uploads.go create mode 100644 internal/web/uploads.templ create mode 100644 internal/web/uploads_templ.go diff --git a/internal/web/handlers.go b/internal/web/handlers.go index bbf17d0..b297dce 100644 --- a/internal/web/handlers.go +++ b/internal/web/handlers.go @@ -13,6 +13,7 @@ import ( "git.d-ma.be/mathias/tapir/internal/adapters/store" "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" ) // Store is the read/write surface the web handlers depend on — a narrow port over @@ -53,6 +54,12 @@ type Store interface { // the most recent discovery pass, shown on the account page as a warning. ListChannelErrors(ctx context.Context, userID string) ([]store.ChannelError, error) + // Uploaded-audio transcripts (ADR-033): the caller's own PRIVATE uploads, + // stored apart from the shared public captions. Both are user-scoped; a + // non-owner's id is ErrUploadNotFound. + GetUpload(ctx context.Context, userID, id string) (*domain.UploadedTranscript, error) + ListUploads(ctx context.Context, userID string) ([]domain.UploadedTranscript, error) + // SetTranscriptStatus clears or updates a video's transcript backoff state. // Used by handleRetryNow to clear rate_limited_at before immediate processing. SetTranscriptStatus(ctx context.Context, userID, videoID, status string) error @@ -100,6 +107,12 @@ type App struct { // summary view shows no "dig deeper" affordance. It holds no caption-fetch // dependency, so an enabled chat cannot reach YouTube or the rate gate. Chat Chatter + // Uploads, when non-nil, accepts audio recordings for private transcription + // (ADR-033). Nil = the /uploads routes are not mounted. UploadLimits are the + // same limits the Uploader enforces, shown on the form so they are stated up + // front rather than discovered by failing. + Uploads Uploader + UploadLimits uploads.Limits // Processing tracks in-flight immediate summarizations so the status endpoint // shows the animation until the summary lands. The zero value is ready to use. Processing ProcessingSet @@ -166,6 +179,14 @@ func (a *App) Router() http.Handler { app.HandleFunc("GET /v/{videoId}/chat", a.handleChat) app.HandleFunc("POST /v/{videoId}/chat", a.handleChatMessage) } + // Audio upload → private transcript (ADR-033). Mounted only when an Uploader is + // wired; every handler is user-scoped, so another user's upload is a 404. + if a.Uploads != nil { + app.HandleFunc("GET /uploads", a.handleUploadsPage) + app.HandleFunc("POST /uploads", a.handleUploadPost) + app.HandleFunc("GET /uploads/{id}", a.handleUploadDetail) + app.HandleFunc("GET /uploads/{id}/status", a.handleUploadStatus) + } app.HandleFunc("GET /register", a.handleRegisterForm) app.HandleFunc("POST /register", a.handleRegister) diff --git a/internal/web/uploads.go b/internal/web/uploads.go new file mode 100644 index 0000000..abb8711 --- /dev/null +++ b/internal/web/uploads.go @@ -0,0 +1,191 @@ +package web + +import ( + "context" + "errors" + "io" + "net/http" + "path/filepath" + "strings" + "unicode/utf8" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// Uploader accepts an uploaded recording for transcription (ADR-033). +// *uploads.Intake satisfies it. It is the whole of what the handlers know about +// validation, spooling and queueing. +type Uploader interface { + Accept(ctx context.Context, userID, title string, src io.Reader) (id string, err error) +} + +const ( + // uploadOverhead is headroom above the file-size cap for the multipart framing + // and the title field; the cap itself is enforced on the file part by Intake. + uploadOverhead = 1 << 20 + maxTitleRunes = 200 + // uploadPollEvery is how often an unfinished upload re-asks for its status. + uploadPollEvery = "5s" +) + +// uploadsView is what the /uploads page renders. +type uploadsView struct { + Limits uploads.Limits + Rows []domain.UploadedTranscript + Error string // a synchronous rejection message, empty when none +} + +// handleUploadsPage renders the upload form and the caller's own uploads. +func (a *App) handleUploadsPage(w http.ResponseWriter, r *http.Request) { + userID, ok := a.currentUserID(w, r) + if !ok { + return + } + a.renderUploadsPage(w, r, userID, http.StatusOK, "") +} + +func (a *App) renderUploadsPage(w http.ResponseWriter, r *http.Request, userID string, status int, msg string) { + rows, err := a.Store.ListUploads(r.Context(), userID) + if err != nil { + a.serverError(w, r, "list uploads", err) + return + } + a.renderStatus(w, r, status, UploadsPage(uploadsView{Limits: a.UploadLimits, Rows: rows, Error: msg})) +} + +// handleUploadPost streams the multipart body: the optional title field, then the +// file part, which goes straight to Intake without being buffered in memory. A +// rejection is synchronous — a status code and a message on the upload page, no +// row, no timeout (ADR-033). +func (a *App) handleUploadPost(w http.ResponseWriter, r *http.Request) { + userID, ok := a.currentUserID(w, r) + if !ok { + return + } + r.Body = http.MaxBytesReader(w, r.Body, a.UploadLimits.MaxBytes+uploadOverhead) + mr, err := r.MultipartReader() + if err != nil { + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "Choose an audio file to upload.") + return + } + title := "" + for { + part, err := mr.NextPart() + if errors.Is(err, io.EOF) { + break + } + if err != nil { + a.uploadFailure(w, r, userID, err) + return + } + switch part.FormName() { + case "title": + b, _ := io.ReadAll(io.LimitReader(part, maxTitleRunes*utf8.UTFMax)) + title = clipRunes(strings.TrimSpace(string(b)), maxTitleRunes) + case "file": + if title == "" { + title = titleFromFilename(part.FileName()) + } + id, err := a.Uploads.Accept(r.Context(), userID, title, part) + if err != nil { + a.uploadFailure(w, r, userID, err) + return + } + http.Redirect(w, r, "/uploads/"+id, http.StatusSeeOther) + return + } + } + a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "Choose an audio file to upload.") +} + +// uploadFailure maps an Accept (or multipart) error to a response: a validation +// rejection is the user's to fix (422, or 413 for size) and carries its own +// message; anything else is ours. +func (a *App) uploadFailure(w http.ResponseWriter, r *http.Request, userID string, err error) { + var rej *uploads.RejectError + var tooBig *http.MaxBytesError + switch { + case errors.As(err, &rej): + status := http.StatusUnprocessableEntity + if rej.Reason == uploads.ReasonTooLarge { + status = http.StatusRequestEntityTooLarge + } + a.renderUploadsPage(w, r, userID, status, rej.Message) + case errors.As(err, &tooBig): + a.renderUploadsPage(w, r, userID, http.StatusRequestEntityTooLarge, + "This file is larger than the upload limit.") + default: + a.serverError(w, r, "upload", err) + } +} + +// handleUploadDetail renders one upload's page (GET /uploads/{id}). The row is +// read through the user-scoped store, so another user's id is a 404 — existence is +// not disclosed, and the title never appears in the response. +func (a *App) handleUploadDetail(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + a.render(w, r, UploadPage(*u)) +} + +// handleUploadStatus renders the status fragment HTMX polls. Unfinished states +// keep polling; terminal states render without hx-trigger so the page stops +// asking (ADR-025: never a stuck spinner). +func (a *App) handleUploadStatus(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + a.render(w, r, uploadStatus(*u)) +} + +func (a *App) loadOwnedUpload(w http.ResponseWriter, r *http.Request) (*domain.UploadedTranscript, bool) { + userID, ok := a.currentUserID(w, r) + if !ok { + return nil, false + } + u, err := a.Store.GetUpload(r.Context(), userID, r.PathValue("id")) + if errors.Is(err, store.ErrUploadNotFound) { + http.NotFound(w, r) + return nil, false + } + if err != nil { + a.serverError(w, r, "get upload", err) + return nil, false + } + return u, true +} + +// titleFromFilename is the default title: the file's name without its extension. +func titleFromFilename(name string) string { + base := filepath.Base(name) + t := strings.TrimSuffix(base, filepath.Ext(base)) + if t == "" || t == "." { + return "Untitled recording" + } + return clipRunes(t, maxTitleRunes) +} + +func clipRunes(s string, n int) string { + if utf8.RuneCountInString(s) <= n { + return s + } + return string([]rune(s)[:n]) +} + +// uploadFailureMessage turns a typed failure kind into words for the user. It is +// keyed on OUR kinds, never on a provider's error string (ADR-033). +func uploadFailureMessage(k domain.UploadErrorKind) string { + switch k { + case domain.UploadErrInterrupted: + return "The server restarted before this recording was transcribed. Please upload it again." + case domain.UploadErrTranscriptionFailed: + return "Transcription failed on every available service. Please upload it again later." + default: + return "This upload failed. Please upload it again." + } +} diff --git a/internal/web/uploads.templ b/internal/web/uploads.templ new file mode 100644 index 0000000..ffcb759 --- /dev/null +++ b/internal/web/uploads.templ @@ -0,0 +1,91 @@ +package web + +import ( + "fmt" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// UploadsPage is the upload form plus the caller's own uploads (ADR-033). +templ UploadsPage(v uploadsView) { + @Layout("Tapir — Upload a recording") { + + } +} + +// UploadPage is one upload's page: its title and the polling status region. +templ UploadPage(u domain.UploadedTranscript) { + @Layout("Tapir — " + u.Title) { + + } +} + +// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the +// page re-asks; done and failed render without it, so nothing polls forever. +templ uploadStatus(u domain.UploadedTranscript) { + switch u.Status { + case domain.UploadDone: +
+

+ Transcribed { uploads.FormatDuration(time.Duration(u.AudioSeconds * float64(time.Second))) } + of audio by { u.Endpoint }. +

+
+ Transcript + 
{ u.Content }
+
+
+ case domain.UploadFailed: + + default: +
+ if u.Status == domain.UploadTranscribing { +

Transcribing… a long recording takes a few minutes.

+ } else { +

Queued — waiting for its turn to be transcribed.

+ } +
+ } +} diff --git a/internal/web/uploads_templ.go b/internal/web/uploads_templ.go new file mode 100644 index 0000000..a874853 --- /dev/null +++ b/internal/web/uploads_templ.go @@ -0,0 +1,381 @@ +// Code generated by templ - DO NOT EDIT. + +// templ: version: v0.3.1020 +package web + +//lint:file-ignore SA4006 This context is only used if a nested component is present. + +import "github.com/a-h/templ" +import templruntime "github.com/a-h/templ/runtime" + +import ( + "fmt" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// UploadsPage is the upload form plus the caller's own uploads (ADR-033). +func UploadsPage(v uploadsView) templ.Component { + return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { + return templ_7745c5c3_CtxErr + } + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Var1 := templ.GetChildren(ctx) + if templ_7745c5c3_Var1 == nil { + templ_7745c5c3_Var1 = templ.NopComponent + } + ctx = templ.ClearChildren(ctx) + templ_7745c5c3_Var2 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "

Upload a recording

Audio only. Up to ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var3 string + templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d MB", v.Limits.MaxBytes>>20)) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 17, Col: 67} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, " and ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var4 string + templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(uploads.FormatDuration(v.Limits.MaxDuration)) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 18, Col: 50} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, ". The transcript is private to you; the audio is deleted once it has been transcribed.

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if v.Error != "" { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var5 string + templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(v.Error) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 22, Col: 57} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if len(v.Rows) > 0 { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 7, "

Your recordings

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 13, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) + templ_7745c5c3_Err = Layout("Tapir — Upload a recording").Render(templ.WithChildren(ctx, templ_7745c5c3_Var2), templ_7745c5c3_Buffer) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) +} + +// UploadPage is one upload's page: its title and the polling status region. +func UploadPage(u domain.UploadedTranscript) templ.Component { + return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { + return templ_7745c5c3_CtxErr + } + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Var9 := templ.GetChildren(ctx) + if templ_7745c5c3_Var9 == nil { + templ_7745c5c3_Var9 = templ.NopComponent + } + ctx = templ.ClearChildren(ctx) + templ_7745c5c3_Var10 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 14, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var11 string + templ_7745c5c3_Var11, templ_7745c5c3_Err = templ.JoinStringErrs(u.Title) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 50, Col: 16} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var11)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 15, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = uploadStatus(u).Render(ctx, templ_7745c5c3_Buffer) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 16, "

All recordings

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) + templ_7745c5c3_Err = Layout("Tapir — "+u.Title).Render(templ.WithChildren(ctx, templ_7745c5c3_Var10), templ_7745c5c3_Buffer) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) +} + +// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the +// page re-asks; done and failed render without it, so nothing polls forever. +func uploadStatus(u domain.UploadedTranscript) templ.Component { + return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { + return templ_7745c5c3_CtxErr + } + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Var12 := templ.GetChildren(ctx) + if templ_7745c5c3_Var12 == nil { + templ_7745c5c3_Var12 = templ.NopComponent + } + ctx = templ.ClearChildren(ctx) + switch u.Status { + case domain.UploadDone: + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 17, "

Transcribed ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var13 string + templ_7745c5c3_Var13, templ_7745c5c3_Err = templ.JoinStringErrs(uploads.FormatDuration(time.Duration(u.AudioSeconds * float64(time.Second)))) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 64, Col: 95} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var13)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 18, " of audio by ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var14 string + templ_7745c5c3_Var14, templ_7745c5c3_Err = templ.JoinStringErrs(u.Endpoint) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 65, Col: 48} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var14)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, ".

Transcript
")
+			if templ_7745c5c3_Err != nil {
+				return templ_7745c5c3_Err
+			}
+			var templ_7745c5c3_Var15 string
+			templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinStringErrs(u.Content)
+			if templ_7745c5c3_Err != nil {
+				return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 69, Col: 40}
+			}
+			_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15))
+			if templ_7745c5c3_Err != nil {
+				return templ_7745c5c3_Err
+			}
+			templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + case domain.UploadFailed: + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var16 string + templ_7745c5c3_Var16, templ_7745c5c3_Err = templ.JoinStringErrs(uploadFailureMessage(u.ErrorKind)) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 74, Col: 42} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var16)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + default: + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if u.Status == domain.UploadTranscribing { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "

Transcribing… a long recording takes a few minutes.

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } else { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "

Queued — waiting for its turn to be transcribed.

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + return nil + }) +} + +var _ = templruntime.GeneratedTemplate -- 2.54.0 From bb285d8757b40dfeffa25f7f1eca5130cdf053f3 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:43:51 +0200 Subject: [PATCH 017/299] test(transcribe): gateway client and iguana->berget chain (red) Gateway: field is `file`, no Expect header, a computed Content-Length (not chunked, not buffered -- a 90 min WAV is ~170 MB), verbose_json normalised with the last segment clamped to the audio duration, errors independent of the body, an empty transcript is a failure. Chain: ANY primary failure falls back and the row records the fallback endpoint; a cancelled job never falls back to the cloud. Red: package transcribe does not exist. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/transcribe/chain_test.go | 99 +++++++++++ internal/adapters/transcribe/gateway_test.go | 174 +++++++++++++++++++ 2 files changed, 273 insertions(+) create mode 100644 internal/adapters/transcribe/chain_test.go create mode 100644 internal/adapters/transcribe/gateway_test.go diff --git a/internal/adapters/transcribe/chain_test.go b/internal/adapters/transcribe/chain_test.go new file mode 100644 index 0000000..dc9af18 --- /dev/null +++ b/internal/adapters/transcribe/chain_test.go @@ -0,0 +1,99 @@ +package transcribe_test + +import ( + "context" + "errors" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +type fakeBackend struct { + name string + text string + err error + calls int +} + +func (f *fakeBackend) Name() string { return f.name } +func (f *fakeBackend) Transcribe(_ context.Context, _ string) (domain.Transcription, error) { + f.calls++ + if f.err != nil { + return domain.Transcription{}, f.err + } + return domain.Transcription{Text: f.text, Endpoint: f.name}, nil +} + +func TestChain_PrimarySuccessNeverTouchesTheFallback(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", text: "primary text"} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "fallback text"} + + got, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "primary text", got.Text) + assert.Equal(t, "iguana/kb-whisper", got.Endpoint) + assert.Zero(t, fallback.calls, "audio must not leave the homelab when the local endpoint worked") +} + +func TestChain_AnyPrimaryFailureFallsBackAndRecordsTheFallbackEndpoint(t *testing.T) { + tests := []struct { + name string + err error + }{ + {"server error", &transcribe.StatusError{Code: 500}}, + {"gateway timeout", &transcribe.StatusError{Code: 504}}, + {"client error", &transcribe.StatusError{Code: 400}}, + {"transport failure", errors.New("dial tcp: connection refused")}, + {"empty transcript", transcribe.ErrEmptyTranscript}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: tt.err} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "fallback text"} + + got, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.NoError(t, err) + assert.Equal(t, "fallback text", got.Text) + assert.Equal(t, "berget/kb-whisper", got.Endpoint, + "the row must say berget served it: that means the audio left the homelab") + }) + } +} + +func TestChain_BothFailingReportsBothErrors(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: errors.New("iguana down")} + fallback := &fakeBackend{name: "berget/kb-whisper", err: errors.New("berget down")} + + _, err := transcribe.NewChain(primary, fallback).Transcribe(context.Background(), "/a.wav") + + require.Error(t, err) + assert.Contains(t, err.Error(), "iguana down") + assert.Contains(t, err.Error(), "berget down") +} + +func TestChain_WithoutAFallbackAPrimaryFailureIsTheResult(t *testing.T) { + primary := &fakeBackend{name: "iguana/kb-whisper", err: errors.New("iguana down")} + + _, err := transcribe.NewChain(primary, nil).Transcribe(context.Background(), "/a.wav") + + require.Error(t, err) + assert.Contains(t, err.Error(), "iguana down") +} + +func TestChain_ACancelledJobDoesNotFallBack(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + primary := &fakeBackend{name: "iguana/kb-whisper", err: context.Canceled} + fallback := &fakeBackend{name: "berget/kb-whisper", text: "x"} + + _, err := transcribe.NewChain(primary, fallback).Transcribe(ctx, "/a.wav") + + assert.ErrorIs(t, err, context.Canceled) + assert.Zero(t, fallback.calls, "a cancelled job must not ship the audio to a cloud endpoint") +} diff --git a/internal/adapters/transcribe/gateway_test.go b/internal/adapters/transcribe/gateway_test.go new file mode 100644 index 0000000..eb3b677 --- /dev/null +++ b/internal/adapters/transcribe/gateway_test.go @@ -0,0 +1,174 @@ +package transcribe_test + +import ( + "bytes" + "context" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// The httptest server stands in for the LiteLLM gateway: it is the process +// boundary, so a fake here is legitimate. The JSON it returns mirrors what the +// real gateway returned for iguana/kb-whisper and berget/kb-whisper on 2026-10-01 +// (docs/research/audio-length-ceiling-2026-10-01.md). All text is synthetic. + +const verboseJSON = `{ + "text": " God morgon allihopa.\n Vi går igenom läget.\n", + "language": "swedish", + "duration": 4.25, + "segments": [ + {"id": 0, "start": 0.0, "end": 1.0, "text": " God morgon allihopa."}, + {"id": 1, "start": 1.0, "end": 9.0, "text": " Vi går igenom läget."} + ] +}` + +func wavFile(t *testing.T, content string) string { + t.Helper() + p := filepath.Join(t.TempDir(), "a.wav") + require.NoError(t, os.WriteFile(p, []byte(content), 0o600)) + return p +} + +type seen struct { + method, path, auth, expect, contentType string + contentLength int64 + transferEncoding []string + fields map[string]string + file string + fileField string + bodyLen int +} + +// serve starts a gateway fake that records the request it received and replies +// with status and body. +func serve(t *testing.T, status int, reply string, got *seen) *httptest.Server { + t.Helper() + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if got != nil { + got.method, got.path = r.Method, r.URL.Path + got.auth = r.Header.Get("Authorization") + got.expect = r.Header.Get("Expect") + got.contentType = r.Header.Get("Content-Type") + got.contentLength = r.ContentLength + got.transferEncoding = r.TransferEncoding + raw, _ := io.ReadAll(r.Body) + got.bodyLen = len(raw) + r.Body = io.NopCloser(bytes.NewReader(raw)) + require.NoError(t, r.ParseMultipartForm(32<<20)) + got.fields = map[string]string{} + for k, v := range r.MultipartForm.Value { + got.fields[k] = v[0] + } + for field, fhs := range r.MultipartForm.File { + got.fileField = field + f, _ := fhs[0].Open() + b, _ := io.ReadAll(f) + got.file = string(b) + } + } + w.WriteHeader(status) + _, _ = io.WriteString(w, reply) + })) + t.Cleanup(srv.Close) + return srv +} + +func TestGateway_SendsTheRequestTheGatewayExpects(t *testing.T) { + var got seen + srv := serve(t, 200, verboseJSON, &got) + g := transcribe.NewGateway(srv.URL+"/v1", "secret-key", "iguana/kb-whisper", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "RIFF-synthetic-pcm")) + + require.NoError(t, err) + assert.Equal(t, http.MethodPost, got.method) + assert.Equal(t, "/v1/audio/transcriptions", got.path) + assert.Equal(t, "Bearer secret-key", got.auth) + assert.Equal(t, "file", got.fileField, "the multipart field is `file`; `audio_file` is rejected as 'Invalid request'") + assert.Equal(t, "RIFF-synthetic-pcm", got.file) + assert.Equal(t, "iguana/kb-whisper", got.fields["model"]) + assert.Equal(t, "verbose_json", got.fields["response_format"]) + assert.Equal(t, "sv", got.fields["language"]) + assert.Empty(t, got.expect, "no Expect: 100-continue: it trips the proxy for any body over 1 KB") +} + +func TestGateway_SendsAKnownContentLengthNotChunked(t *testing.T) { + var got seen + srv := serve(t, 200, verboseJSON, &got) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "0123456789")) + + require.NoError(t, err) + assert.Empty(t, got.transferEncoding, "a 90-minute WAV is ~170 MB: stream it with a computed length, not chunked and not buffered") + assert.Equal(t, int64(got.bodyLen), got.contentLength) +} + +func TestGateway_NormalisesTheVerboseJSONResponse(t *testing.T) { + srv := serve(t, 200, verboseJSON, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "iguana/kb-whisper", srv.Client()) + + got, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + require.NoError(t, err) + assert.Equal(t, "God morgon allihopa.\n Vi går igenom läget.", got.Text) + assert.Equal(t, "iguana/kb-whisper", got.Endpoint, "the serving endpoint is part of the result") + assert.InDelta(t, 4.25, got.AudioSeconds, 1e-9) + assert.Equal(t, []domain.Segment{ + {Start: 0, End: 1, Text: "God morgon allihopa."}, + // The final segment ended at 9.0 s for 4.25 s of audio: whisper's last + // window can run past the end. Clamped to the audio duration. + {Start: 1, End: 4.25, Text: "Vi går igenom läget."}, + }, got.Segments) +} + +func TestGateway_ErrorsDoNotDependOnTheBody(t *testing.T) { + // {"error":"failed to process audio"} has had at least five distinct causes, so + // no code may branch on it. Two different bodies must be treated identically. + for _, reply := range []string{`{"error":"failed to process audio"}`, `{"error":"something else entirely"}`} { + srv := serve(t, 500, reply, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + var se *transcribe.StatusError + require.ErrorAs(t, err, &se) + assert.Equal(t, 500, se.Code) + } +} + +func TestGateway_AnEmptyTranscriptIsAFailureNotASuccess(t *testing.T) { + srv := serve(t, 200, `{"text":" ","language":"swedish","duration":3,"segments":[]}`, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + assert.ErrorIs(t, err, transcribe.ErrEmptyTranscript) +} + +func TestGateway_MalformedJSONIsAnError(t *testing.T) { + srv := serve(t, 200, `not json`, nil) + g := transcribe.NewGateway(srv.URL+"/v1", "k", "m", srv.Client()) + + _, err := g.Transcribe(context.Background(), wavFile(t, "x")) + + assert.Error(t, err) +} + +func TestGateway_MissingAudioFileIsAnError(t *testing.T) { + g := transcribe.NewGateway("http://127.0.0.1:1/v1", "k", "m", http.DefaultClient) + + _, err := g.Transcribe(context.Background(), filepath.Join(t.TempDir(), "nope.wav")) + + assert.Error(t, err) +} -- 2.54.0 From cae423750c04f9c9193c2b5d1dcbfe57de605e16 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:44:34 +0200 Subject: [PATCH 018/299] feat(transcribe): gateway client with an iguana->berget fallback chain (ADR-033) Streams the WAV from disk with a computed Content-Length (no buffering, no chunking), posts verbose_json with field `file` and language=sv, clamps the final segment to the audio duration, and treats an empty transcript as a failure. The chain falls back on ANY primary error, records the serving endpoint, and never falls back once the context is cancelled. gateway.go and chain.go are one commit: the chain's tests use the gateway's StatusError and ErrEmptyTranscript, so neither compiles alone. Mutation-checked: field name, clamp, cancel-fallback, content length and the recorded endpoint each turn a test red. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/transcribe/chain.go | 53 +++++++++ internal/adapters/transcribe/gateway.go | 147 ++++++++++++++++++++++++ 2 files changed, 200 insertions(+) create mode 100644 internal/adapters/transcribe/chain.go create mode 100644 internal/adapters/transcribe/gateway.go diff --git a/internal/adapters/transcribe/chain.go b/internal/adapters/transcribe/chain.go new file mode 100644 index 0000000..68df323 --- /dev/null +++ b/internal/adapters/transcribe/chain.go @@ -0,0 +1,53 @@ +package transcribe + +import ( + "context" + "errors" + "fmt" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// Backend is one transcription endpoint. *Gateway satisfies it; Name is the value +// recorded as the serving endpoint. +type Backend interface { + Name() string + Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) +} + +// Chain tries a primary (local) backend and, if it fails for ANY reason, a +// fallback. The fallback exists because iguana can wedge permanently on a Metal +// OOM (infra#273) — which is also why a fallback is a data-egress decision: the +// berget endpoint is a cloud service, so the result records which one served it. +type Chain struct { + primary Backend + fallback Backend // nil disables the fallback +} + +// NewChain builds a Chain. A nil fallback means a primary failure is the result. +func NewChain(primary, fallback Backend) *Chain { + return &Chain{primary: primary, fallback: fallback} +} + +// Transcribe returns the first backend's success. It falls back on any primary +// error — transport, timeout, non-2xx, empty transcript — and never inspects a +// response body to decide. A cancelled context stops the chain: a job the user or +// shutdown cancelled must not ship its audio to a cloud endpoint as a last act. +func (c *Chain) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) { + t, err := c.primary.Transcribe(ctx, wavPath) + if err == nil { + t.Endpoint = c.primary.Name() + return t, nil + } + if c.fallback == nil || ctx.Err() != nil { + return domain.Transcription{}, fmt.Errorf("%s: %w", c.primary.Name(), err) + } + t, ferr := c.fallback.Transcribe(ctx, wavPath) + if ferr != nil { + return domain.Transcription{}, errors.Join( + fmt.Errorf("%s: %w", c.primary.Name(), err), + fmt.Errorf("%s: %w", c.fallback.Name(), ferr)) + } + t.Endpoint = c.fallback.Name() + return t, nil +} diff --git a/internal/adapters/transcribe/gateway.go b/internal/adapters/transcribe/gateway.go new file mode 100644 index 0000000..e885c71 --- /dev/null +++ b/internal/adapters/transcribe/gateway.go @@ -0,0 +1,147 @@ +// Package transcribe turns a WAV file into a timed Swedish transcript by calling +// the LiteLLM gateway's OpenAI-compatible /audio/transcriptions endpoint +// (ADR-033). A Gateway is one model alias; a Chain tries a local one and falls +// back to a second. +package transcribe + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "mime/multipart" + "net/http" + "os" + "strings" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrEmptyTranscript is returned when the endpoint answered 200 but with no text. +// An empty transcript is not a success: chat over it could only answer "nothing". +var ErrEmptyTranscript = errors.New("transcribe: endpoint returned an empty transcript") + +// StatusError is a non-2xx answer from the gateway. Nothing may branch on the +// body — {"error":"failed to process audio"} has had at least five distinct +// causes — so only the status code is exposed for logic; Snippet is for logs. +type StatusError struct { + Code int + Snippet string +} + +func (e *StatusError) Error() string { + return fmt.Sprintf("transcribe: gateway returned HTTP %d: %s", e.Code, e.Snippet) +} + +// Gateway transcribes through one model alias on the gateway. baseURL is the +// gateway's /v1 root — the in-cluster NodePort, not the public hostname, whose +// ingress rejects bodies over ~8-10 MB with 413 (docs/research/audio-length-ceiling). +type Gateway struct { + baseURL string + key string + model string + client *http.Client +} + +// NewGateway builds a Gateway. The client should carry no overall Timeout: a 90 +// minute recording takes minutes, so the deadline belongs to the caller's context. +func NewGateway(baseURL, key, model string, client *http.Client) *Gateway { + return &Gateway{baseURL: strings.TrimRight(baseURL, "/"), key: key, model: model, client: client} +} + +// Name is the model alias; it is what gets recorded as the serving endpoint. +func (g *Gateway) Name() string { return g.model } + +// Transcribe uploads the WAV at path and returns the normalised transcription. +// The body is streamed from disk with a COMPUTED Content-Length: a 90 minute WAV +// is ~170 MB, so it is neither buffered in memory nor sent chunked. +func (g *Gateway) Transcribe(ctx context.Context, path string) (domain.Transcription, error) { + f, err := os.Open(path) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: open audio: %w", err) + } + defer f.Close() + info, err := f.Stat() + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: stat audio: %w", err) + } + + var prefix bytes.Buffer + mw := multipart.NewWriter(&prefix) + for _, kv := range [][2]string{{"model", g.model}, {"response_format", "verbose_json"}, {"language", "sv"}} { + if err := mw.WriteField(kv[0], kv[1]); err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + } + // The multipart field is `file`; `audio_file` is rejected as "Invalid request". + if _, err := mw.CreateFormFile("file", "audio.wav"); err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + suffix := []byte("\r\n--" + mw.Boundary() + "--\r\n") + + body := io.MultiReader(bytes.NewReader(prefix.Bytes()), f, bytes.NewReader(suffix)) + req, err := http.NewRequestWithContext(ctx, http.MethodPost, g.baseURL+"/audio/transcriptions", body) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: build request: %w", err) + } + req.ContentLength = int64(prefix.Len()) + info.Size() + int64(len(suffix)) + req.Header.Set("Content-Type", mw.FormDataContentType()) + req.Header.Set("Authorization", "Bearer "+g.key) + // No "Expect: 100-continue": it trips the proxy for any body over 1 KB. + + resp, err := g.client.Do(req) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err) + } + defer resp.Body.Close() + if resp.StatusCode < 200 || resp.StatusCode > 299 { + snip, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return domain.Transcription{}, &StatusError{Code: resp.StatusCode, Snippet: strings.TrimSpace(string(snip))} + } + + t, err := parseVerbose(resp.Body) + if err != nil { + return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err) + } + t.Endpoint = g.model + return t, nil +} + +// verboseJSON is the subset of the verbose_json response used. Both backends +// return it; they differ in segment granularity (iguana: whole seconds, berget: +// sub-second) and in the language token, neither of which matters here. +type verboseJSON struct { + Text string `json:"text"` + Duration float64 `json:"duration"` + Segments []struct { + Start float64 `json:"start"` + End float64 `json:"end"` + Text string `json:"text"` + } `json:"segments"` +} + +// parseVerbose normalises a verbose_json body. Whisper's final window can end past +// the audio (9.0 s for 4.25 s of audio was observed), so segment ends are clamped +// to the reported duration. +func parseVerbose(r io.Reader) (domain.Transcription, error) { + var v verboseJSON + if err := json.NewDecoder(r).Decode(&v); err != nil { + return domain.Transcription{}, fmt.Errorf("decode response: %w", err) + } + text := strings.TrimSpace(v.Text) + if text == "" { + return domain.Transcription{}, ErrEmptyTranscript + } + t := domain.Transcription{Text: text, AudioSeconds: v.Duration} + for _, s := range v.Segments { + seg := domain.Segment{Start: s.Start, End: s.End, Text: strings.TrimSpace(s.Text)} + if v.Duration > 0 { + seg.End = min(seg.End, v.Duration) + seg.Start = min(seg.Start, seg.End) + } + t.Segments = append(t.Segments, seg) + } + return t, nil +} -- 2.54.0 From 2227cfff7e034da7c9a5a944e308faae58ccce48 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:46:14 +0200 Subject: [PATCH 019/299] test(transcribe): opt-in live check against the real gateway The httptest fake proves only what we THINK the gateway accepts. Skipped unless TAPIR_LIVE_WAV and TAPIR_LIVE_GATEWAY_URL are set. Run once on 2026-10-01 with 11 min of synthetic Swedish (21 MB, above the 8-10 MB ingress cap, via the NodePort): both iguana/kb-whisper (52.8 s) and berget/kb-whisper (35.4 s) accepted the streamed, computed-Content-Length body and returned usable verbose_json. Synthetic audio only; this repo is public. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/transcribe/live_test.go | 49 +++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 internal/adapters/transcribe/live_test.go diff --git a/internal/adapters/transcribe/live_test.go b/internal/adapters/transcribe/live_test.go new file mode 100644 index 0000000..fc48bab --- /dev/null +++ b/internal/adapters/transcribe/live_test.go @@ -0,0 +1,49 @@ +package transcribe_test + +import ( + "context" + "net/http" + "os" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" +) + +// TestGateway_LiveGateway is an opt-in check against the REAL gateway, because +// the httptest fake above can only prove what we think the gateway accepts. It +// is skipped unless TAPIR_LIVE_WAV names a 16 kHz mono WAV and the gateway +// coordinates are set: +// +// TAPIR_LIVE_WAV=/path/a.wav TAPIR_LIVE_GATEWAY_URL=http://koala:30401/v1 \ +// TAPIR_LIVE_MODEL=iguana/kb-whisper op run --env-file ~/.op-env -- \ +// go test ./internal/adapters/transcribe -run LiveGateway -v +// +// Use SYNTHETIC audio only: this repo is public and the gateway may be a cloud +// endpoint (#33). +func TestGateway_LiveGateway(t *testing.T) { + wav := os.Getenv("TAPIR_LIVE_WAV") + url := os.Getenv("TAPIR_LIVE_GATEWAY_URL") + if wav == "" || url == "" { + t.Skip("set TAPIR_LIVE_WAV and TAPIR_LIVE_GATEWAY_URL to run against the real gateway") + } + model := os.Getenv("TAPIR_LIVE_MODEL") + if model == "" { + model = "iguana/kb-whisper" + } + key := os.Getenv("LITELLM_MASTER_KEY") + require.NotEmpty(t, key, "LITELLM_MASTER_KEY must be set (via op run)") + + ctx, cancel := context.WithTimeout(context.Background(), 25*time.Minute) + defer cancel() + got, err := transcribe.NewGateway(url, key, model, &http.Client{}).Transcribe(ctx, wav) + + require.NoError(t, err) + assert.NotEmpty(t, got.Text) + assert.NotEmpty(t, got.Segments) + assert.Equal(t, model, got.Endpoint) + t.Logf("endpoint=%s audio_s=%.1f segments=%d chars=%d", got.Endpoint, got.AudioSeconds, len(got.Segments), len(got.Text)) +} -- 2.54.0 From 3bc33fc34cf8f6dd70b683da55208613f2ba7271 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:46:14 +0200 Subject: [PATCH 020/299] test(audio): ffmpeg transcode to 16 kHz mono PCM, always (red) Red: audio.FFmpeg is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/audio/ffmpeg_test.go | 58 ++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 internal/adapters/audio/ffmpeg_test.go diff --git a/internal/adapters/audio/ffmpeg_test.go b/internal/adapters/audio/ffmpeg_test.go new file mode 100644 index 0000000..1aaf6d1 --- /dev/null +++ b/internal/adapters/audio/ffmpeg_test.go @@ -0,0 +1,58 @@ +package audio_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" +) + +func TestFFmpegToWAV_AsksForSixteenKilohertzMonoPCMAlways(t *testing.T) { + // Always transcode, even when the input already is 16 kHz mono: a Teams export + // often is, and the point is to not ASSUME (ADR-033). Records the argv. + rec := filepath.Join(t.TempDir(), "argv") + bin := stub(t, `printf '%s\n' "$@" > `+rec+`; : > "$(eval echo \${$#})"`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "out.wav")) + + require.NoError(t, err) + raw, err := os.ReadFile(rec) + require.NoError(t, err) + argv := strings.Split(strings.TrimSpace(string(raw)), "\n") + joined := " " + strings.Join(argv, " ") + " " + for _, want := range []string{" -ac 1 ", " -ar 16000 ", " -c:a pcm_s16le ", " -vn ", " -i /spool/in "} { + assert.Contains(t, joined, want) + } + assert.True(t, strings.HasSuffix(strings.TrimSpace(joined), "out.wav"), "the output path is the last argument") +} + +func TestFFmpegToWAV_NonzeroExitIsAnErrorCarryingStderr(t *testing.T) { + bin := stub(t, `echo "moov atom not found" >&2; exit 1`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/in", filepath.Join(t.TempDir(), "o.wav")) + + require.Error(t, err) + assert.Contains(t, err.Error(), "moov atom not found") +} + +func TestFFmpegToWAV_ZeroExitWithNoOutputIsAnError(t *testing.T) { + bin := stub(t, `exit 0`) + + err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/in", filepath.Join(t.TempDir(), "never-written.wav")) + + require.Error(t, err) +} + +func TestFFmpegToWAV_MissingBinaryIsAnError(t *testing.T) { + err := (&audio.FFmpeg{Bin: filepath.Join(t.TempDir(), "no-ffmpeg")}).ToWAV(context.Background(), "/in", "/out.wav") + + require.Error(t, err) + assert.False(t, errors.Is(err, context.Canceled)) +} -- 2.54.0 From 4352446f368747b6eb39f4ea58e998730a52115a Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:46:34 +0200 Subject: [PATCH 021/299] feat(audio): ffmpeg adapter -- always transcode to 16 kHz mono PCM (ADR-033) Replaces whisper-server --convert. A zero exit that wrote nothing is an error. Also fixes the red test's stub, which wrote an empty file and so (correctly) tripped the no-output check. Checked against real ffmpeg on iguana with the same argv: AAC .m4a in, pcm_s16le 16000 Hz mono WAV out, duration preserved. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/audio/ffmpeg.go | 45 ++++++++++++++++++++++++++ internal/adapters/audio/ffmpeg_test.go | 4 ++- 2 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 internal/adapters/audio/ffmpeg.go diff --git a/internal/adapters/audio/ffmpeg.go b/internal/adapters/audio/ffmpeg.go new file mode 100644 index 0000000..ee04971 --- /dev/null +++ b/internal/adapters/audio/ffmpeg.go @@ -0,0 +1,45 @@ +package audio + +import ( + "context" + "fmt" + "os" + "os/exec" + "strings" +) + +// FFmpeg converts any audio container to the strict format whisper wants: 16 kHz +// mono signed 16-bit PCM WAV (ADR-033). It ALWAYS transcodes, even when the input +// already is that format — a Teams export often is, and the transcode is then +// nearly free; assuming it would be the bug. This replaces whisper-server's +// --convert, whose --tmp-dir is created at process start and rots under macOS +// periodic cleanup. +type FFmpeg struct { + // Bin is the ffmpeg executable ("ffmpeg" resolves via PATH). + Bin string +} + +// ToWAV writes src, converted, to dst. Video streams are dropped (-vn): by now the +// file has already been probed as audio-only, but a cover-art stream must not end +// up in the WAV. +func (f *FFmpeg) ToWAV(ctx context.Context, src, dst string) error { + bin := f.Bin + if bin == "" { + bin = "ffmpeg" + } + cmd := exec.CommandContext(ctx, bin, + "-y", "-v", "error", "-i", src, "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le", "-f", "wav", dst) + cmd.WaitDelay = killGrace + if out, err := cmd.CombinedOutput(); err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return fmt.Errorf("audio: ffmpeg: %w", ctxErr) + } + return fmt.Errorf("audio: ffmpeg: %w: %s", err, strings.TrimSpace(string(out))) + } + // A zero exit that wrote nothing is still a failure: never hand whisper a + // missing file and let the error surface three layers later. + if info, err := os.Stat(dst); err != nil || info.Size() == 0 { + return fmt.Errorf("audio: ffmpeg wrote no output at %s", dst) + } + return nil +} diff --git a/internal/adapters/audio/ffmpeg_test.go b/internal/adapters/audio/ffmpeg_test.go index 1aaf6d1..4442c80 100644 --- a/internal/adapters/audio/ffmpeg_test.go +++ b/internal/adapters/audio/ffmpeg_test.go @@ -18,7 +18,9 @@ func TestFFmpegToWAV_AsksForSixteenKilohertzMonoPCMAlways(t *testing.T) { // Always transcode, even when the input already is 16 kHz mono: a Teams export // often is, and the point is to not ASSUME (ADR-033). Records the argv. rec := filepath.Join(t.TempDir(), "argv") - bin := stub(t, `printf '%s\n' "$@" > `+rec+`; : > "$(eval echo \${$#})"`) + bin := stub(t, `printf '%s\n' "$@" > `+rec+` +for a; do last="$a"; done +printf data > "$last"`) err := (&audio.FFmpeg{Bin: bin}).ToWAV(context.Background(), "/spool/in", filepath.Join(t.TempDir(), "out.wav")) -- 2.54.0 From 52417e6b47d89fcdfd82321dc399fb0c0f0cf5ad Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:47:22 +0200 Subject: [PATCH 022/299] test(uploads): in-process transcription worker (red) Success path deletes both the upload and the WAV; failures are typed and still delete the audio; a shutdown mid-job leaves the row for the start-up sweep (it is not a transcription failure); a hung endpoint is failed by the job timeout, never an eternal spinner; strictly one job at a time (two long jobs on iguana were never measured); stopping deletes queued audio and refuses new jobs. Red: uploads.Worker is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/worker_test.go | 275 ++++++++++++++++++++++++++++++++ 1 file changed, 275 insertions(+) create mode 100644 internal/uploads/worker_test.go diff --git a/internal/uploads/worker_test.go b/internal/uploads/worker_test.go new file mode 100644 index 0000000..27e2c11 --- /dev/null +++ b/internal/uploads/worker_test.go @@ -0,0 +1,275 @@ +package uploads_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/domain" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +type wStore struct { + mu sync.Mutex + started []string + completed map[string]domain.Transcription + failed map[string]domain.UploadErrorKind + startErr error +} + +func newWStore() *wStore { + return &wStore{completed: map[string]domain.Transcription{}, failed: map[string]domain.UploadErrorKind{}} +} + +func (s *wStore) StartTranscribing(_ context.Context, _, id string) error { + s.mu.Lock() + defer s.mu.Unlock() + s.started = append(s.started, id) + return s.startErr +} + +func (s *wStore) CompleteUpload(_ context.Context, _, id string, t domain.Transcription) error { + s.mu.Lock() + defer s.mu.Unlock() + s.completed[id] = t + return nil +} + +func (s *wStore) FailUpload(_ context.Context, _, id string, k domain.UploadErrorKind) error { + s.mu.Lock() + defer s.mu.Unlock() + s.failed[id] = k + return nil +} + +func (s *wStore) snapshot() (map[string]domain.Transcription, map[string]domain.UploadErrorKind) { + s.mu.Lock() + defer s.mu.Unlock() + c := map[string]domain.Transcription{} + f := map[string]domain.UploadErrorKind{} + for k, v := range s.completed { + c[k] = v + } + for k, v := range s.failed { + f[k] = v + } + return c, f +} + +type fakeConverter struct { + err error + srcs []string + dsts []string +} + +func (c *fakeConverter) ToWAV(_ context.Context, src, dst string) error { + c.srcs = append(c.srcs, src) + c.dsts = append(c.dsts, dst) + if c.err != nil { + return c.err + } + return os.WriteFile(dst, []byte("wav"), 0o600) +} + +type fakeTranscriber struct { + result domain.Transcription + err error + paths []string + // block, when non-nil, holds every call until it is closed or ctx ends. + block chan struct{} + running atomic.Int32 + maxSeen atomic.Int32 +} + +func (f *fakeTranscriber) Transcribe(ctx context.Context, wav string) (domain.Transcription, error) { + f.paths = append(f.paths, wav) + n := f.running.Add(1) + defer f.running.Add(-1) + for { + m := f.maxSeen.Load() + if n <= m || f.maxSeen.CompareAndSwap(m, n) { + break + } + } + if f.block != nil { + select { + case <-f.block: + case <-ctx.Done(): + return domain.Transcription{}, ctx.Err() + } + } + return f.result, f.err +} + +type wrig struct { + w *uploads.Worker + st *wStore + conv *fakeConverter + tr *fakeTranscriber + tmp string +} + +func newWRig(t *testing.T, timeout time.Duration) *wrig { + t.Helper() + r := &wrig{ + st: newWStore(), + conv: &fakeConverter{}, + tr: &fakeTranscriber{result: domain.Transcription{Text: "synthetic text", Endpoint: "iguana/kb-whisper", AudioSeconds: 3}}, + tmp: t.TempDir(), + } + r.w = uploads.NewWorker(r.st, r.conv, r.tr, r.tmp, timeout) + return r +} + +// spool writes a fake uploaded file (what Intake would hand over) and returns a Job. +func (r *wrig) spool(t *testing.T, id string) uploads.Job { + t.Helper() + p := filepath.Join(r.tmp, "upload-"+id) + require.NoError(t, os.WriteFile(p, []byte("audio"), 0o600)) + return uploads.Job{UserID: "u1", UploadID: id, Path: p} +} + +func (r *wrig) leftovers(t *testing.T) []string { + t.Helper() + ents, err := os.ReadDir(r.tmp) + require.NoError(t, err) + var n []string + for _, e := range ents { + n = append(n, e.Name()) + } + return n +} + +func TestWorkerProcess_SuccessConvertsTranscribesStoresAndDeletesTheAudio(t *testing.T) { + r := newWRig(t, time.Minute) + job := r.spool(t, "a") + + r.w.Process(context.Background(), job) + + c, f := r.st.snapshot() + assert.Equal(t, []string{"a"}, r.st.started) + assert.Equal(t, "synthetic text", c["a"].Text) + assert.Equal(t, "iguana/kb-whisper", c["a"].Endpoint) + assert.Empty(t, f) + require.Len(t, r.conv.srcs, 1) + assert.Equal(t, job.Path, r.conv.srcs[0], "the converter reads the uploaded file") + assert.Equal(t, r.conv.dsts, r.tr.paths, "whisper is sent the converted WAV, not the original") + assert.Equal(t, r.tmp, filepath.Dir(r.conv.dsts[0])) + assert.Empty(t, r.leftovers(t), "the audio is never retained: both the upload and the WAV are deleted") +} + +func TestWorkerProcess_FailuresAreTypedAndStillDeleteTheAudio(t *testing.T) { + tests := []struct { + name string + setup func(*wrig) + }{ + {"conversion fails", func(r *wrig) { r.conv.err = errors.New("moov atom not found") }}, + {"every transcription endpoint fails", func(r *wrig) { r.tr.err = errors.New("iguana down; berget down") }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + r := newWRig(t, time.Minute) + tt.setup(r) + job := r.spool(t, "a") + + r.w.Process(context.Background(), job) + + c, f := r.st.snapshot() + assert.Empty(t, c) + assert.Equal(t, domain.UploadErrTranscriptionFailed, f["a"]) + assert.Empty(t, r.leftovers(t)) + }) + } + t.Run("conversion failure never reaches whisper", func(t *testing.T) { + r := newWRig(t, time.Minute) + r.conv.err = errors.New("bad") + + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Empty(t, r.tr.paths) + }) +} + +func TestWorkerProcess_ShutdownMidJobLeavesTheRowForTheStartupSweep(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan struct{}) + job := r.spool(t, "a") + + go func() { r.w.Process(ctx, job); close(done) }() + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + cancel() + <-done + + _, f := r.st.snapshot() + assert.Empty(t, f, "a shutdown is not a transcription failure: the row stays for FailInterruptedUploads to mark interrupted") + assert.Empty(t, r.leftovers(t), "but the audio is still deleted") +} + +func TestWorkerProcess_AJobThatOutlivesItsTimeoutIsFailedNotStuck(t *testing.T) { + r := newWRig(t, 50*time.Millisecond) + r.tr.block = make(chan struct{}) // never released + + r.w.Process(context.Background(), r.spool(t, "a")) + + _, f := r.st.snapshot() + assert.Equal(t, domain.UploadErrTranscriptionFailed, f["a"], "a hung endpoint must end in failed, not an eternal spinner (ADR-025)") + assert.Empty(t, r.leftovers(t)) +} + +func TestWorkerProcess_StartFailureDropsTheJobAndDeletesTheAudio(t *testing.T) { + r := newWRig(t, time.Minute) + r.st.startErr = errors.New("db down") + + r.w.Process(context.Background(), r.spool(t, "a")) + + assert.Empty(t, r.tr.paths, "never transcribe a job whose row could not be moved to transcribing") + _, f := r.st.snapshot() + assert.Contains(t, f, "a") + assert.Empty(t, r.leftovers(t)) +} + +func TestWorkerRun_ProcessesOneJobAtATime(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + go r.w.Run(ctx) + + for _, id := range []string{"a", "b", "c"} { + require.NoError(t, r.w.Submit(r.spool(t, id))) + } + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + time.Sleep(50 * time.Millisecond) // give a (wrong) second job time to start + assert.Equal(t, int32(1), r.tr.running.Load(), "two simultaneous long jobs on iguana were never measured: concurrency is 1") + + close(r.tr.block) + require.Eventually(t, func() bool { c, _ := r.st.snapshot(); return len(c) == 3 }, 3*time.Second, 5*time.Millisecond) + assert.Equal(t, int32(1), r.tr.maxSeen.Load()) +} + +func TestWorkerRun_StoppingDeletesQueuedAudioAndRefusesNewJobs(t *testing.T) { + r := newWRig(t, time.Minute) + r.tr.block = make(chan struct{}) + ctx, cancel := context.WithCancel(context.Background()) + stopped := make(chan struct{}) + go func() { r.w.Run(ctx); close(stopped) }() + require.NoError(t, r.w.Submit(r.spool(t, "a"))) + require.Eventually(t, func() bool { return r.tr.running.Load() == 1 }, 2*time.Second, 5*time.Millisecond) + require.NoError(t, r.w.Submit(r.spool(t, "b"))) // queued behind the running job + + cancel() + <-stopped + + assert.Empty(t, r.leftovers(t), "audio of queued jobs is deleted on stop: the rows are swept to interrupted at next start") + assert.ErrorIs(t, r.w.Submit(r.spool(t, "c")), uploads.ErrStopped) +} -- 2.54.0 From 3cfc95640e9b5494326b57e87fed4f87799cdda4 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:54:52 +0200 Subject: [PATCH 023/299] test(uploads): a queued job at shutdown must never be started; timeout test fails fast Mutation-testing the worker showed deleting stop()'s cleanup went unnoticed: Process was incidentally deleting the audio of jobs it should never have started. The stop test now asserts only the running job was converted (red on the old code). The timeout test no longer hangs for the 10-minute go test default when the timeout is removed. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/worker_test.go | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/internal/uploads/worker_test.go b/internal/uploads/worker_test.go index 27e2c11..da5235f 100644 --- a/internal/uploads/worker_test.go +++ b/internal/uploads/worker_test.go @@ -218,8 +218,14 @@ func TestWorkerProcess_ShutdownMidJobLeavesTheRowForTheStartupSweep(t *testing.T func TestWorkerProcess_AJobThatOutlivesItsTimeoutIsFailedNotStuck(t *testing.T) { r := newWRig(t, 50*time.Millisecond) r.tr.block = make(chan struct{}) // never released + done := make(chan struct{}) - r.w.Process(context.Background(), r.spool(t, "a")) + go func() { r.w.Process(context.Background(), r.spool(t, "a")); close(done) }() + select { + case <-done: + case <-time.After(5 * time.Second): + t.Fatal("Process did not return: a hung endpoint was not bounded by the job timeout") + } _, f := r.st.snapshot() assert.Equal(t, domain.UploadErrTranscriptionFailed, f["a"], "a hung endpoint must end in failed, not an eternal spinner (ADR-025)") @@ -270,6 +276,7 @@ func TestWorkerRun_StoppingDeletesQueuedAudioAndRefusesNewJobs(t *testing.T) { cancel() <-stopped + assert.Len(t, r.conv.srcs, 1, "a job still queued at shutdown must never be started: only the running job was converted") assert.Empty(t, r.leftovers(t), "audio of queued jobs is deleted on stop: the rows are swept to interrupted at next start") assert.ErrorIs(t, r.w.Submit(r.spool(t, "c")), uploads.ErrStopped) } -- 2.54.0 From a8dadb8ca40c9f4e088133a912ec840f550e5aeb Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:54:52 +0200 Subject: [PATCH 024/299] feat(uploads): in-process transcription worker, one job at a time (ADR-033) Queue -> StartTranscribing -> ffmpeg to 16 kHz WAV -> gateway chain -> CompleteUpload, with a per-job timeout so a hung endpoint ends in failed, not a spinner. The upload and the WAV are deleted on every path (audio is never retained). A shutdown mid-job is not a failure: the row is left for the start-up sweep. Cancellation wins over a non-empty queue, so queued jobs are never started on shutdown; stop() deletes their audio and refuses new jobs. Concurrency is 1 on purpose: two simultaneous long jobs on iguana were never measured, and a Metal OOM there latches whisper (infra#273). The queue is process-local, so ADR-018's single-replica assumption now has a second dependent. Mutation-checked, -race clean: cleanup, shutdown-vs-failure, timeout, concurrency and stop-cleanup each turn a test red. Co-Authored-By: Claude Sonnet 5.5 --- internal/uploads/worker.go | 191 +++++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 internal/uploads/worker.go diff --git a/internal/uploads/worker.go b/internal/uploads/worker.go new file mode 100644 index 0000000..7b32ab0 --- /dev/null +++ b/internal/uploads/worker.go @@ -0,0 +1,191 @@ +package uploads + +import ( + "context" + "errors" + "fmt" + "log/slog" + "os" + "sync" + "time" + + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// ErrStopped is returned by Submit once the worker has stopped. +var ErrStopped = errors.New("uploads: worker stopped") + +// Converter turns the uploaded file into the 16 kHz mono PCM WAV whisper wants. +type Converter interface { + ToWAV(ctx context.Context, src, dst string) error +} + +// Transcriber turns a WAV into a timed transcription. The gateway chain +// (iguana primary, berget fallback) implements it. +type Transcriber interface { + Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) +} + +// WorkerStore is the slice of the private transcript store the worker drives. +type WorkerStore interface { + StartTranscribing(ctx context.Context, userID, id string) error + CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error + FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error +} + +// Worker transcribes accepted uploads in-process, ONE AT A TIME (ADR-033). It is +// not a second deployable: it shares the web process's lifetime, which is why it +// inherits ADR-018's single-replica assumption — the queue is process-local. The +// concurrency of one is deliberate: two simultaneous 90-minute jobs on iguana were +// never measured, and a Metal OOM there latches the whisper backend (infra#273). +type Worker struct { + store WorkerStore + conv Converter + tr Transcriber + tmpDir string + timeout time.Duration + + mu sync.Mutex + queue []Job + stopped bool + wake chan struct{} +} + +// NewWorker wires a Worker. timeout bounds each job (conversion + transcription); +// a job that outlives it is failed rather than left spinning. +func NewWorker(s WorkerStore, c Converter, t Transcriber, tmpDir string, timeout time.Duration) *Worker { + return &Worker{store: s, conv: c, tr: t, tmpDir: tmpDir, timeout: timeout, wake: make(chan struct{}, 1)} +} + +// Submit queues a job. It never blocks. It satisfies Submitter. +func (w *Worker) Submit(j Job) error { + w.mu.Lock() + defer w.mu.Unlock() + if w.stopped { + return ErrStopped + } + w.queue = append(w.queue, j) + select { + case w.wake <- struct{}{}: + default: + } + return nil +} + +// Run processes queued jobs until ctx ends, then deletes the audio of anything +// still queued and refuses new jobs. The rows of those jobs stay queued or +// transcribing and are failed as interrupted by the start-up sweep +// (store.FailInterruptedUploads): the audio is not retained, so they cannot resume. +func (w *Worker) Run(ctx context.Context) { + defer w.stop() + for { + job, ok := w.next(ctx) + if !ok { + return + } + w.Process(ctx, job) + } +} + +// next blocks for the next job. Cancellation wins over a non-empty queue: a job +// still queued at shutdown must not be STARTED (it would begin a long conversion +// with an already-cancelled context); stop() deletes its audio instead. +func (w *Worker) next(ctx context.Context) (Job, bool) { + for { + if ctx.Err() != nil { + return Job{}, false + } + w.mu.Lock() + if len(w.queue) > 0 { + j := w.queue[0] + w.queue = w.queue[1:] + w.mu.Unlock() + return j, true + } + w.mu.Unlock() + select { + case <-w.wake: + case <-ctx.Done(): + return Job{}, false + } + } +} + +func (w *Worker) stop() { + w.mu.Lock() + defer w.mu.Unlock() + w.stopped = true + for _, j := range w.queue { + _ = os.Remove(j.Path) + } + w.queue = nil +} + +// Process runs one job: row -> transcribing, convert, transcribe, store. The +// uploaded file and the converted WAV are deleted on every path — the audio is +// never retained (ADR-033). A failure is recorded with a TYPED kind; a shutdown +// mid-job is not a failure and leaves the row for the start-up sweep. +func (w *Worker) Process(ctx context.Context, job Job) { + defer func() { _ = os.Remove(job.Path) }() + log := slog.Default().With("upload", job.UploadID) + start := time.Now() + + if err := w.store.StartTranscribing(ctx, job.UserID, job.UploadID); err != nil { + log.Error("upload: start transcribing", "err", err) + w.fail(ctx, job, log) + return + } + + wav, err := w.tempWAV() + if err != nil { + log.Error("upload: temp wav", "err", err) + w.fail(ctx, job, log) + return + } + defer func() { _ = os.Remove(wav) }() + + jobCtx, cancel := context.WithTimeout(ctx, w.timeout) + defer cancel() + if err := w.conv.ToWAV(jobCtx, job.Path, wav); err != nil { + log.Error("upload: convert", "err", err) + w.fail(ctx, job, log) + return + } + t, err := w.tr.Transcribe(jobCtx, wav) + if err != nil { + log.Error("upload: transcribe", "err", err, "elapsed", time.Since(start).Round(time.Second)) + w.fail(ctx, job, log) + return + } + if err := w.store.CompleteUpload(ctx, job.UserID, job.UploadID, t); err != nil { + log.Error("upload: store transcript", "err", err) + w.fail(ctx, job, log) + return + } + log.Info("upload transcribed", "endpoint", t.Endpoint, "audio_s", t.AudioSeconds, + "segments", len(t.Segments), "elapsed", time.Since(start).Round(time.Second)) +} + +// fail records a transcription failure — unless the PARENT context ended, which +// is a shutdown, not a failure: the row is left for the start-up sweep. +func (w *Worker) fail(ctx context.Context, job Job, log *slog.Logger) { + if ctx.Err() != nil { + return + } + if err := w.store.FailUpload(ctx, job.UserID, job.UploadID, domain.UploadErrTranscriptionFailed); err != nil { + log.Error("upload: record failure", "err", err) + } +} + +func (w *Worker) tempWAV() (string, error) { + f, err := os.CreateTemp(w.tmpDir, "wav-*") + if err != nil { + return "", fmt.Errorf("uploads: temp wav: %w", err) + } + name := f.Name() + if err := f.Close(); err != nil { + _ = os.Remove(name) + return "", fmt.Errorf("uploads: temp wav: %w", err) + } + return name, nil +} -- 2.54.0 From fb521f2b89450c618679097313f1eec51c28f2bc Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:55:11 +0200 Subject: [PATCH 025/299] test(config): upload and transcription settings (red) Off by default until the image has ffmpeg; 200 MB / 90 min (the longest length measured); iguana primary, berget fallback that an explicit empty value disables; zero or negative limits are refused rather than silently rejecting every upload. Red: the Config fields do not exist. Co-Authored-By: Claude Sonnet 5.5 --- internal/config/uploads_test.go | 123 ++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 internal/config/uploads_test.go diff --git a/internal/config/uploads_test.go b/internal/config/uploads_test.go new file mode 100644 index 0000000..c55d77f --- /dev/null +++ b/internal/config/uploads_test.go @@ -0,0 +1,123 @@ +package config + +import ( + "os" + "testing" + "time" +) + +// clearUploadEnv unsets every upload knob so a test observes the default, not the +// host environment. +func clearUploadEnv(t *testing.T) { + t.Helper() + for _, k := range []string{ + "TAPIR_UPLOADS_ENABLED", "TAPIR_UPLOAD_MAX_MB", "TAPIR_UPLOAD_MAX_MINUTES", + "TAPIR_TRANSCRIBE_MODEL", "TAPIR_TRANSCRIBE_FALLBACK_MODEL", "TAPIR_TRANSCRIBE_TIMEOUT", + "TAPIR_UPLOAD_TMP_DIR", "TAPIR_FFMPEG_BIN", "TAPIR_FFPROBE_BIN", + "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", + } { + unset(t, k) + } +} + +func TestLoad_UploadDefaults(t *testing.T) { + clearUploadEnv(t) + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.UploadsEnabled { + t.Error("uploads must be OFF by default: the runtime image needs ffmpeg first (ADR-033)") + } + if c.UploadMaxBytes != 200<<20 { + t.Errorf("UploadMaxBytes = %d, want 200 MB", c.UploadMaxBytes) + } + if c.UploadMaxDuration != 90*time.Minute { + t.Errorf("UploadMaxDuration = %v, want 90m (the longest length measured, not a found ceiling)", c.UploadMaxDuration) + } + if c.TranscribeModel != "iguana/kb-whisper" { + t.Errorf("TranscribeModel = %q", c.TranscribeModel) + } + if c.TranscribeFallbackModel != "berget/kb-whisper" { + t.Errorf("TranscribeFallbackModel = %q", c.TranscribeFallbackModel) + } + if c.TranscribeTimeout != 30*time.Minute { + t.Errorf("TranscribeTimeout = %v, want 30m (a 90 min recording took ~7.5 min)", c.TranscribeTimeout) + } + if c.UploadTmpDir != os.TempDir() { + t.Errorf("UploadTmpDir = %q, want os.TempDir()", c.UploadTmpDir) + } + if c.FFmpegBin != "ffmpeg" || c.FFprobeBin != "ffprobe" { + t.Errorf("bins = %q, %q", c.FFmpegBin, c.FFprobeBin) + } + if c.MaxUploadedTranscriptChars != 120000 { + t.Errorf("MaxUploadedTranscriptChars = %d, want 120000", c.MaxUploadedTranscriptChars) + } +} + +func TestLoad_UploadValuesAreParsed(t *testing.T) { + clearUploadEnv(t) + setEnv(t, map[string]string{ + "TAPIR_UPLOADS_ENABLED": "true", + "TAPIR_UPLOAD_MAX_MB": "50", + "TAPIR_UPLOAD_MAX_MINUTES": "30", + "TAPIR_TRANSCRIBE_MODEL": "koala/whisper", + "TAPIR_TRANSCRIBE_TIMEOUT": "10m", + "TAPIR_UPLOAD_TMP_DIR": "/var/spool/tapir", + "TAPIR_FFMPEG_BIN": "/opt/ffmpeg", + "TAPIR_FFPROBE_BIN": "/opt/ffprobe", + "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS": "60000", + }) + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if !c.UploadsEnabled || c.UploadMaxBytes != 50<<20 || c.UploadMaxDuration != 30*time.Minute || + c.TranscribeModel != "koala/whisper" || c.TranscribeTimeout != 10*time.Minute || + c.UploadTmpDir != "/var/spool/tapir" || c.FFmpegBin != "/opt/ffmpeg" || + c.FFprobeBin != "/opt/ffprobe" || c.MaxUploadedTranscriptChars != 60000 { + t.Errorf("values not applied: %+v", c) + } +} + +func TestLoad_EmptyTranscribeFallbackDisablesIt(t *testing.T) { + clearUploadEnv(t) + t.Setenv("TAPIR_TRANSCRIBE_FALLBACK_MODEL", "") + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.TranscribeFallbackModel != "" { + t.Errorf("TranscribeFallbackModel = %q: an explicit empty value must disable the cloud fallback, "+ + "so private audio can be kept on the homelab", c.TranscribeFallbackModel) + } +} + +func TestLoad_UploadRejectsInvalidValues(t *testing.T) { + tests := []struct{ key, val string }{ + {"TAPIR_UPLOADS_ENABLED", "maybe"}, + {"TAPIR_UPLOAD_MAX_MB", "lots"}, + {"TAPIR_UPLOAD_MAX_MB", "0"}, + {"TAPIR_UPLOAD_MAX_MB", "-5"}, + {"TAPIR_UPLOAD_MAX_MINUTES", "0"}, + {"TAPIR_UPLOAD_MAX_MINUTES", "forever"}, + {"TAPIR_TRANSCRIBE_TIMEOUT", "soon"}, + {"TAPIR_TRANSCRIBE_TIMEOUT", "0s"}, + } + for _, tt := range tests { + t.Run(tt.key+"="+tt.val, func(t *testing.T) { + clearUploadEnv(t) + t.Setenv(tt.key, tt.val) + + if _, err := Load(); err == nil { + t.Errorf("Load accepted %s=%q; a zero or negative limit would reject every upload", tt.key, tt.val) + } + }) + } +} -- 2.54.0 From b8798dbd5da48abc974fa82fd026842504b23b69 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:55:32 +0200 Subject: [PATCH 026/299] feat(config): upload and transcription settings (ADR-033) Embedded UploadConfig. Off by default until the image carries ffmpeg; 200 MB / 90 min; iguana primary with a berget fallback that an explicit empty TAPIR_TRANSCRIBE_FALLBACK_MODEL disables (private audio then never leaves the homelab); zero or negative limits fail at load instead of rejecting every upload at runtime. uploads.go and the two-line hook in config.go are one commit: the tests need both. Co-Authored-By: Claude Sonnet 5.5 --- internal/config/config.go | 7 +++ internal/config/uploads.go | 110 +++++++++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+) create mode 100644 internal/config/uploads.go diff --git a/internal/config/config.go b/internal/config/config.go index 01f01f5..9b43d10 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -20,6 +20,9 @@ import ( // Config is Tapir's fully-resolved runtime configuration. type Config struct { + // UploadConfig is the audio-upload and private-transcription settings (ADR-033). + UploadConfig + // UserID is the Tapir user the run operates as. Stage 0 has exactly one. // It must be a UUID: it keys the UUID user_id/video_id columns (data-model). UserID string @@ -320,6 +323,10 @@ func Load() (Config, error) { } c.OnboardMaxVideoSeconds = onboardMax + if c.UploadConfig, err = loadUploadConfig(); err != nil { + return Config{}, err + } + return c, nil } diff --git a/internal/config/uploads.go b/internal/config/uploads.go new file mode 100644 index 0000000..a496b20 --- /dev/null +++ b/internal/config/uploads.go @@ -0,0 +1,110 @@ +package config + +import ( + "fmt" + "os" + "strconv" + "time" +) + +// UploadConfig holds the settings for audio upload and private transcription +// (ADR-033). It is embedded in Config, so its fields read as c.UploadsEnabled etc. +type UploadConfig struct { + // UploadsEnabled mounts the /uploads routes and starts the worker. Off by + // default: the runtime image must contain ffmpeg and ffprobe first (ADR-033). + UploadsEnabled bool + // UploadMaxBytes caps one upload. UploadMaxDuration caps one recording; 90 + // minutes is the longest length MEASURED, not a ceiling that was found, so + // raising it is a measurement, not an edit. + UploadMaxBytes int64 + UploadMaxDuration time.Duration + // TranscribeModel is the primary gateway alias; TranscribeFallbackModel is the + // fallback, tried on ANY primary failure. The fallback is a cloud endpoint + // (berget): set it EMPTY to keep private audio on the homelab. + TranscribeModel string + TranscribeFallbackModel string + // TranscribeTimeout bounds one job (conversion + transcription); a 90 minute + // recording took ~7.5 minutes on iguana. + TranscribeTimeout time.Duration + // UploadTmpDir is where uploads are spooled and converted. Audio is deleted + // after transcription, so this holds at most the in-flight files. + UploadTmpDir string + FFmpegBin string + FFprobeBin string + // MaxUploadedTranscriptChars bounds the transcript sent to a chat model for an + // UPLOADED recording. The 18,000-char default for captions is ~22 minutes of + // speech; an hour of meeting is ~50,000 chars. A guess until measured against + // the models chat offers. + MaxUploadedTranscriptChars int +} + +const ( + defaultUploadMaxMB = 200 + defaultUploadMaxMinutes = 90 + defaultTranscribeModel = "iguana/kb-whisper" + defaultTranscribeFallbackModel = "berget/kb-whisper" + defaultTranscribeTimeout = 30 * time.Minute + defaultMaxUploadedChars = 120000 +) + +func loadUploadConfig() (UploadConfig, error) { + u := UploadConfig{ + TranscribeModel: envOr("TAPIR_TRANSCRIBE_MODEL", defaultTranscribeModel), + TranscribeFallbackModel: lookupOr("TAPIR_TRANSCRIBE_FALLBACK_MODEL", defaultTranscribeFallbackModel), + UploadTmpDir: envOr("TAPIR_UPLOAD_TMP_DIR", os.TempDir()), + FFmpegBin: envOr("TAPIR_FFMPEG_BIN", "ffmpeg"), + FFprobeBin: envOr("TAPIR_FFPROBE_BIN", "ffprobe"), + } + + if v := os.Getenv("TAPIR_UPLOADS_ENABLED"); v != "" { + on, err := strconv.ParseBool(v) + if err != nil { + return UploadConfig{}, fmt.Errorf("config: TAPIR_UPLOADS_ENABLED=%q: %w", v, err) + } + u.UploadsEnabled = on + } + + mb, err := positiveInt("TAPIR_UPLOAD_MAX_MB", defaultUploadMaxMB) + if err != nil { + return UploadConfig{}, err + } + u.UploadMaxBytes = int64(mb) << 20 + + mins, err := positiveInt("TAPIR_UPLOAD_MAX_MINUTES", defaultUploadMaxMinutes) + if err != nil { + return UploadConfig{}, err + } + u.UploadMaxDuration = time.Duration(mins) * time.Minute + + timeout, err := durationOr("TAPIR_TRANSCRIBE_TIMEOUT", defaultTranscribeTimeout) + if err != nil { + return UploadConfig{}, err + } + if timeout <= 0 { + return UploadConfig{}, fmt.Errorf("config: TAPIR_TRANSCRIBE_TIMEOUT must be positive, got %v", timeout) + } + u.TranscribeTimeout = timeout + + chars, err := intOr("TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", defaultMaxUploadedChars) + if err != nil { + return UploadConfig{}, err + } + if chars < 0 { + chars = 0 + } + u.MaxUploadedTranscriptChars = chars + return u, nil +} + +// positiveInt is intOr that refuses zero and negatives: a zero upload limit would +// reject every file, which is a misconfiguration to fail on, not to run with. +func positiveInt(key string, fallback int) (int, error) { + n, err := intOr(key, fallback) + if err != nil { + return 0, err + } + if n <= 0 { + return 0, fmt.Errorf("config: %s must be positive, got %d", key, n) + } + return n, nil +} -- 2.54.0 From 3e802adade640be4ed5ea600a3328eec61c74f09 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:55:51 +0200 Subject: [PATCH 027/299] test(cmd): buildUploads wiring -- off by default, sweep first, private spool (red) Red: buildUploads is undefined. Co-Authored-By: Claude Sonnet 5.5 --- cmd/tapir/uploads_test.go | 106 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 106 insertions(+) create mode 100644 cmd/tapir/uploads_test.go diff --git a/cmd/tapir/uploads_test.go b/cmd/tapir/uploads_test.go new file mode 100644 index 0000000..6d33f92 --- /dev/null +++ b/cmd/tapir/uploads_test.go @@ -0,0 +1,106 @@ +package main + +import ( + "context" + "errors" + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/domain" +) + +// fakeUploadsStore satisfies the store surface buildUploads needs, and counts the +// start-up sweep. +type fakeUploadsStore struct { + sweeps int + sweepErr error +} + +func (f *fakeUploadsStore) CreateUpload(context.Context, string, string) (string, error) { + return "id", nil +} + +func (f *fakeUploadsStore) FailUpload(context.Context, string, string, domain.UploadErrorKind) error { + return nil +} + +func (f *fakeUploadsStore) StartTranscribing(context.Context, string, string) error { return nil } + +func (f *fakeUploadsStore) CompleteUpload(context.Context, string, string, domain.Transcription) error { + return nil +} + +func (f *fakeUploadsStore) FailInterruptedUploads(context.Context) (int, error) { + f.sweeps++ + return 2, f.sweepErr +} + +func uploadCfg(t *testing.T, enabled bool) config.Config { + t.Helper() + var cfg config.Config + cfg.GatewayURL = "http://gateway.invalid/v1" + cfg.UploadsEnabled = enabled + cfg.UploadMaxBytes = 200 << 20 + cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool") + cfg.TranscribeModel = "iguana/kb-whisper" + cfg.TranscribeFallbackModel = "berget/kb-whisper" + return cfg +} + +func TestBuildUploads_DisabledMountsNothingAndSweepsNothing(t *testing.T) { + st := &fakeUploadsStore{} + + in, w, err := buildUploads(context.Background(), uploadCfg(t, false), st, quietLog()) + + require.NoError(t, err) + assert.Nil(t, in, "a typed-nil Intake must never reach web.App.Uploads: it would mount the routes over nothing") + assert.Nil(t, w) + assert.Zero(t, st.sweeps) +} + +func TestBuildUploads_EnabledFailsInterruptedRowsBeforeAnythingElseStarts(t *testing.T) { + st := &fakeUploadsStore{} + + in, w, err := buildUploads(context.Background(), uploadCfg(t, true), st, quietLog()) + + require.NoError(t, err) + require.NotNil(t, in) + require.NotNil(t, w) + assert.Equal(t, 1, st.sweeps, + "audio is not retained, so rows left queued/transcribing by a restart can never resume: fail them at start-up, not never") +} + +func TestBuildUploads_SweepFailureStopsStartup(t *testing.T) { + st := &fakeUploadsStore{sweepErr: errors.New("db down")} + + _, _, err := buildUploads(context.Background(), uploadCfg(t, true), st, quietLog()) + + require.Error(t, err) + assert.Contains(t, err.Error(), "db down") +} + +func TestBuildUploads_CreatesTheSpoolDirectoryPrivately(t *testing.T) { + cfg := uploadCfg(t, true) + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + require.NoError(t, err) + info, err := os.Stat(cfg.UploadTmpDir) + require.NoError(t, err) + assert.True(t, info.IsDir()) + assert.Equal(t, os.FileMode(0o700), info.Mode().Perm(), "spooled audio is private third-party content") +} + +func TestBuildUploads_EnabledWithoutAGatewayIsAnError(t *testing.T) { + cfg := uploadCfg(t, true) + cfg.GatewayURL = "" + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + assert.Error(t, err, "uploads on with nowhere to transcribe must fail loudly at start-up, not at the first upload") +} -- 2.54.0 From eb9357a5de6e19ea4f0501783afdf50cd54b2af0 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:56:11 +0200 Subject: [PATCH 028/299] test(cmd): start-up removes spooled audio left by a crash (red) Co-Authored-By: Claude Sonnet 5.5 --- cmd/tapir/uploads_test.go | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/cmd/tapir/uploads_test.go b/cmd/tapir/uploads_test.go index 6d33f92..a18c02e 100644 --- a/cmd/tapir/uploads_test.go +++ b/cmd/tapir/uploads_test.go @@ -104,3 +104,24 @@ func TestBuildUploads_EnabledWithoutAGatewayIsAnError(t *testing.T) { assert.Error(t, err, "uploads on with nowhere to transcribe must fail loudly at start-up, not at the first upload") } + +func TestBuildUploads_RemovesAudioLeftBehindByACrash(t *testing.T) { + cfg := uploadCfg(t, true) + require.NoError(t, os.MkdirAll(cfg.UploadTmpDir, 0o700)) + leftovers := []string{"upload-123", "wav-456"} + for _, n := range leftovers { + require.NoError(t, os.WriteFile(filepath.Join(cfg.UploadTmpDir, n), []byte("audio"), 0o600)) + } + unrelated := filepath.Join(cfg.UploadTmpDir, "someone-elses-file") + require.NoError(t, os.WriteFile(unrelated, []byte("keep"), 0o600)) + + _, _, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog()) + + require.NoError(t, err) + for _, n := range leftovers { + _, err := os.Stat(filepath.Join(cfg.UploadTmpDir, n)) + assert.True(t, os.IsNotExist(err), "%s: audio is never retained, so a spooled file from a crashed run must not survive start-up", n) + } + _, err = os.Stat(unrelated) + assert.NoError(t, err, "only our own spool files are removed") +} -- 2.54.0 From 9b3175db0979816f4c9f11dfe2f9ed9ce880d766 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:56:21 +0200 Subject: [PATCH 029/299] feat(cmd): buildUploads -- spool, sweep, fail interrupted rows, wire the chain (ADR-033) Returns nils when disabled (never a typed-nil Intake). When enabled: private 0700 spool dir, delete audio left by a crash, fail rows a restart left queued or transcribing (the audio is not retained, so they cannot resume), then build the iguana->berget chain, worker and intake. An empty TAPIR_TRANSCRIBE_FALLBACK_MODEL keeps private audio on the homelab. Enabling with no gateway fails at start-up, not at the first upload. Co-Authored-By: Claude Sonnet 5.5 --- cmd/tapir/uploads.go | 85 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 cmd/tapir/uploads.go diff --git a/cmd/tapir/uploads.go b/cmd/tapir/uploads.go new file mode 100644 index 0000000..ebe5339 --- /dev/null +++ b/cmd/tapir/uploads.go @@ -0,0 +1,85 @@ +package main + +import ( + "context" + "fmt" + "log/slog" + "net/http" + "os" + "path/filepath" + + "git.d-ma.be/mathias/tapir/internal/adapters/audio" + "git.d-ma.be/mathias/tapir/internal/adapters/transcribe" + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/uploads" +) + +// uploadsStore is the store surface the upload wiring needs. *store.Store +// satisfies it. +type uploadsStore interface { + uploads.Store + uploads.WorkerStore + FailInterruptedUploads(ctx context.Context) (int, error) +} + +// buildUploads wires audio upload and private transcription (ADR-033). It returns +// nils when uploads are disabled — never a typed-nil Intake, which would mount the +// /uploads routes over nothing. When enabled it, in order: prepares a private spool +// directory, deletes any audio a crashed run left in it, fails the rows such a run +// left queued or transcribing (the audio is not retained, so they can never +// resume), and only then builds the intake and the worker. The caller runs the +// worker. The fallback endpoint is a cloud service; an empty +// TAPIR_TRANSCRIBE_FALLBACK_MODEL keeps private audio on the homelab. +func buildUploads(ctx context.Context, cfg config.Config, st uploadsStore, log *slog.Logger) (*uploads.Intake, *uploads.Worker, error) { + if !cfg.UploadsEnabled { + return nil, nil, nil + } + if cfg.GatewayURL == "" { + return nil, nil, fmt.Errorf("uploads: TAPIR_UPLOADS_ENABLED is set but there is no TAPIR_GATEWAY_URL to transcribe through") + } + + if err := os.MkdirAll(cfg.UploadTmpDir, 0o700); err != nil { + return nil, nil, fmt.Errorf("uploads: spool dir: %w", err) + } + removeSpooledAudio(cfg.UploadTmpDir, log) + + n, err := st.FailInterruptedUploads(ctx) + if err != nil { + return nil, nil, fmt.Errorf("uploads: fail interrupted rows: %w", err) + } + if n > 0 { + log.Warn("uploads: marked interrupted by a restart (audio is not retained; they must be re-uploaded)", "count", n) + } + + // No overall client timeout: a 90 minute recording takes minutes. The deadline + // is the worker's per-job context. + client := &http.Client{} + primary := transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeModel, client) + var fallback transcribe.Backend + if cfg.TranscribeFallbackModel != "" && cfg.TranscribeFallbackModel != cfg.TranscribeModel { + fallback = transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeFallbackModel, client) + } + + worker := uploads.NewWorker(st, &audio.FFmpeg{Bin: cfg.FFmpegBin}, transcribe.NewChain(primary, fallback), + cfg.UploadTmpDir, cfg.TranscribeTimeout) + intake := uploads.NewIntake(st, &audio.FFprobe{Bin: cfg.FFprobeBin}, worker, + uploadLimits(cfg), cfg.UploadTmpDir) + return intake, worker, nil +} + +func uploadLimits(cfg config.Config) uploads.Limits { + return uploads.Limits{MaxBytes: cfg.UploadMaxBytes, MaxDuration: cfg.UploadMaxDuration} +} + +// removeSpooledAudio deletes the spool files this process family creates +// (upload-*, wav-*) and nothing else. +func removeSpooledAudio(dir string, log *slog.Logger) { + for _, pat := range []string{"upload-*", "wav-*"} { + matches, _ := filepath.Glob(filepath.Join(dir, pat)) + for _, m := range matches { + if err := os.Remove(m); err != nil { + log.Warn("uploads: remove leftover audio", "path", m, "err", err) + } + } + } +} -- 2.54.0 From 1c5d85f5c652c06cb3b6bee9214bdb049fd4d2cb Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:56:22 +0200 Subject: [PATCH 030/299] feat(cmd): serve mounts audio upload when enabled (ADR-033) Wires buildUploads into cmdServe: Intake and limits into the web App, the worker on the server's ctx. Logs the two facts an operator must not miss: the queue is process-local (single replica), and a configured fallback is a cloud endpoint that private audio can leave the homelab through. Co-Authored-By: Claude Sonnet 5.5 --- cmd/tapir/main.go | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/cmd/tapir/main.go b/cmd/tapir/main.go index 95de372..4df1cb7 100644 --- a/cmd/tapir/main.go +++ b/cmd/tapir/main.go @@ -207,6 +207,27 @@ func cmdServe(ctx context.Context, log *slog.Logger) error { log.Info("web chat enabled (stored-transcript only)", "models", chatModels(cfg)) } + // Audio upload -> private transcript (ADR-033). Off unless TAPIR_UPLOADS_ENABLED: + // the image must carry ffmpeg first. The worker is IN-PROCESS and one job at a + // time, so it inherits the single-replica assumption below (its queue is + // process-local); it stops with the same ctx as the server. + intake, worker, err := buildUploads(ctx, cfg, st, log) + if err != nil { + return err + } + if intake != nil { + app.Uploads = intake + app.UploadLimits = uploadLimits(cfg) + go worker.Run(ctx) + log.Info("audio upload enabled", "model", cfg.TranscribeModel, "fallback", cfg.TranscribeFallbackModel, + "max_mb", cfg.UploadMaxBytes>>20, "max_minutes", int(cfg.UploadMaxDuration.Minutes())) + log.Warn("audio upload assumes a SINGLE replica: the transcription queue is process-local (ADR-033, ADR-018)") + if cfg.TranscribeFallbackModel != "" { + log.Warn("audio upload fallback is a CLOUD endpoint: if the primary fails, private audio leaves the homelab", + "fallback", cfg.TranscribeFallbackModel) + } + } + // User onboarding is handled by the IdP (Authentik invite flow), not Tapir — // the Dex local-password provisioning path was removed (ADR-019). An // authenticated subject with no Tapir user is routed to /register. -- 2.54.0 From 867afefb088f3dfa6ac0771317b33fc98793ca4c Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:56:51 +0200 Subject: [PATCH 031/299] test(chat): a request may carry its own transcript budget (red) The 18,000-char default is ~22 min of speech and keeps only the head; an uploaded hour is ~50,000 chars. Red: Request.MaxChars is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/chat/maxchars_test.go | 53 +++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 internal/adapters/chat/maxchars_test.go diff --git a/internal/adapters/chat/maxchars_test.go b/internal/adapters/chat/maxchars_test.go new file mode 100644 index 0000000..f3f9541 --- /dev/null +++ b/internal/adapters/chat/maxchars_test.go @@ -0,0 +1,53 @@ +package chat + +import ( + "context" + "strings" + "testing" +) + +// A Request may carry its own transcript budget. The Service's cap (18,000 chars by +// default) is sized for YouTube captions; an uploaded recording is much longer +// (~50,000 chars an hour), and keeping only the head of it would answer questions +// about the last two-thirds of a meeting from text that does not contain them +// (ADR-033). The per-request value overrides; zero means "the service default". + +func TestAnswerMaxCharsOverridesTheServiceCap(t *testing.T) { + tests := []struct { + name string + serviceCap int + requestCap int + transcriptLen int + wantKept int + wantTruncated bool + }{ + {"request cap above the service cap keeps the whole transcript", 10, 100, 50, 50, false}, + {"request cap below the service cap bounds tighter", 100, 10, 50, 10, true}, + {"zero request cap falls back to the service cap", 20, 0, 50, 20, true}, + {"negative request cap is treated as unset", 20, -1, 50, 20, true}, + {"request cap exceeded is reported as truncated", 10, 30, 50, 30, true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := &factory{answer: "answer"} + s := New(f.make, []string{"koala/phi4-mini"}, tt.serviceCap) + + reply, err := s.Answer(context.Background(), Request{ + Model: "koala/phi4-mini", + Transcript: strings.Repeat("x", tt.transcriptLen), + Question: "what was decided?", + MaxChars: tt.requestCap, + }) + + if err != nil { + t.Fatalf("Answer: %v", err) + } + if got := strings.Count(f.used.lastSystem, "x"); got != tt.wantKept { + t.Errorf("transcript chars sent = %d, want %d", got, tt.wantKept) + } + if reply.Truncated != tt.wantTruncated { + t.Errorf("Truncated = %v, want %v", reply.Truncated, tt.wantTruncated) + } + }) + } +} -- 2.54.0 From ec29c5e7618894c67c6e885e32e4c688ecc93e33 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:57:01 +0200 Subject: [PATCH 032/299] style(chat): gofumpt the new test Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/chat/maxchars_test.go | 1 - 1 file changed, 1 deletion(-) diff --git a/internal/adapters/chat/maxchars_test.go b/internal/adapters/chat/maxchars_test.go index f3f9541..87f1275 100644 --- a/internal/adapters/chat/maxchars_test.go +++ b/internal/adapters/chat/maxchars_test.go @@ -38,7 +38,6 @@ func TestAnswerMaxCharsOverridesTheServiceCap(t *testing.T) { Question: "what was decided?", MaxChars: tt.requestCap, }) - if err != nil { t.Fatalf("Answer: %v", err) } -- 2.54.0 From c72d366935f9d7a4b49b51e0640ceb19005ab79e Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:57:01 +0200 Subject: [PATCH 033/299] feat(chat): per-request transcript budget so a long recording is not cut to its head (ADR-033) chat.Request.MaxChars overrides the Service cap when positive. The 18,000-char default (~22 min of speech) suits captions; an uploaded hour is ~50,000 chars and truncation keeps only the head. This is reuse of the existing chat path, not a fork. Co-Authored-By: Claude Sonnet 5.5 --- internal/adapters/chat/chat.go | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/internal/adapters/chat/chat.go b/internal/adapters/chat/chat.go index 8cd0610..a774260 100644 --- a/internal/adapters/chat/chat.go +++ b/internal/adapters/chat/chat.go @@ -41,6 +41,11 @@ type Request struct { Transcript string History []Turn Question string + // MaxChars, when positive, overrides the Service's transcript budget for this + // request. The Service default is sized for YouTube captions (~22 minutes of + // speech); an uploaded recording is much longer, and truncation keeps only the + // head (ADR-033). Zero or negative means "use the Service default". + MaxChars int } // Reply is the model's answer plus whether the transcript was bounded to fit the @@ -113,7 +118,11 @@ func (s *Service) Answer(ctx context.Context, req Request) (Reply, error) { model = s.DefaultModel("") } - transcript, truncated := truncate(req.Transcript, s.maxChars) + budget := s.maxChars + if req.MaxChars > 0 { + budget = req.MaxChars + } + transcript, truncated := truncate(req.Transcript, budget) system := buildSystem(transcript, truncated) user := buildUser(req.History, req.Question) -- 2.54.0 From 10ca45fc0a0c6f1e58c31d400c4f0b8152cfddba Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:57:47 +0200 Subject: [PATCH 034/299] test(web): chat over an uploaded transcript reuses the ADR-027 path (red) Owner sees the panel posting to /uploads/{id}/chat; an unfinished upload redirects to its status; ANOTHER USER'S upload is 404 on GET and POST and no model is ever called with it; the owner's transcript is sent with the upload budget (not the 18,000-char caption default); an unoffered model falls back; a model failure is inline and keeps the conversation; routes are absent without a chat backend; the done card links to chat only when chat is wired. Red: App.UploadChatMaxChars is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/uploads_chat_handler_test.go | 167 ++++++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 internal/web/uploads_chat_handler_test.go diff --git a/internal/web/uploads_chat_handler_test.go b/internal/web/uploads_chat_handler_test.go new file mode 100644 index 0000000..e03d8f5 --- /dev/null +++ b/internal/web/uploads_chat_handler_test.go @@ -0,0 +1,167 @@ +package web_test + +import ( + "errors" + "net/http" + "net/http/httptest" + "net/url" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/chat" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// Chat over an uploaded transcript REUSES the ADR-027 chat path (the same +// Chatter, the same panel) — only the lookup differs: the row comes from the +// user-scoped private store, so another user's upload is a 404 and, crucially, no +// model is ever called with it. Transcript text is synthetic; this repo is public. + +const meetingText = "Anders ansvarar för virket. Maria undersöker leverantörer." + +func newUploadChatApp(t *testing.T) (*web.App, *fakeChatter) { + t.Helper() + app, _ := newUploadApp(t, audioOK, 200<<20) + fc := &fakeChatter{ + models: []string{"koala/phi4-mini", "iguana/gemma4-26b"}, + reply: chat.Reply{Answer: "Anders owns the timber order."}, + } + app.Chat = fc + app.UploadChatMaxChars = 120000 + return app, fc +} + +func chatPost(path string, kv url.Values) *http.Request { + req := httptest.NewRequest(http.MethodPost, path, strings.NewReader(kv.Encode())) + req.Header.Set("Content-Type", "application/x-www-form-urlencoded") + return req +} + +func TestUploadChatPage_OwnerOfAFinishedUploadSeesThePanelPostingToTheUploadRoute(t *testing.T) { + app, _ := newUploadChatApp(t) + id := seedUpload(t, userID, "Roadmap sync", "done", meetingText) + + rec := do(t, app, get("/uploads/"+id+"/chat")) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, `action="/uploads/`+id+`/chat"`, "the form posts to the UPLOAD route, not /v/{id}/chat") + assert.Contains(t, b, "Roadmap sync") + assert.Contains(t, b, "koala/phi4-mini") + assert.Contains(t, b, "recording", "wording says recording, not video") +} + +func TestUploadChatPage_AnUnfinishedUploadRedirectsToItsStatusPage(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "transcribing", "") + + rec := do(t, app, get("/uploads/"+id+"/chat")) + + assert.Equal(t, http.StatusSeeOther, rec.Code) + assert.Equal(t, "/uploads/"+id, rec.Header().Get("Location")) + assert.Zero(t, fc.callCount) +} + +func TestUploadChat_AnotherUsersUploadIs404AndNoModelIsCalled(t *testing.T) { + app, fc := newUploadChatApp(t) + theirs := seedUpload(t, otherUserID, "Their private meeting", "done", meetingText) + + getRec := do(t, app, get("/uploads/"+theirs+"/chat")) + postRec := do(t, app, chatPost("/uploads/"+theirs+"/chat", url.Values{"question": {"what did they say?"}})) + + assert.Equal(t, http.StatusNotFound, getRec.Code) + assert.Equal(t, http.StatusNotFound, postRec.Code) + assert.NotContains(t, body(t, getRec), "Their private meeting") + assert.Zero(t, fc.callCount, "someone else's transcript must never reach a model") +} + +func TestUploadChatPost_SendsTheOwnersTranscriptWithTheUploadBudget(t *testing.T) { + app, fc := newUploadChatApp(t) + app.UploadChatMaxChars = 77777 + id := seedUpload(t, userID, "Roadmap sync", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{ + "question": {"Who owns the timber order?"}, + "model": {"iguana/gemma4-26b"}, + })) + + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, 1, fc.callCount) + assert.Equal(t, meetingText, fc.gotReq.Transcript) + assert.Equal(t, "Who owns the timber order?", fc.gotReq.Question) + assert.Equal(t, "iguana/gemma4-26b", fc.gotReq.Model) + assert.Equal(t, 77777, fc.gotReq.MaxChars, + "an uploaded transcript takes its own budget: the 18,000-char caption default would cut an hour to its first third") + assert.Contains(t, body(t, rec), "Anders owns the timber order.") +} + +func TestUploadChatPost_AnUnofferedModelFallsBackToTheDefault(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "done", meetingText) + + do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}, "model": {"evil/model"}})) + + assert.Equal(t, "koala/phi4-mini", fc.gotReq.Model) +} + +func TestUploadChatPost_UnfinishedUploadCallsNoModel(t *testing.T) { + app, fc := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "queued", "") + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}})) + + assert.Equal(t, http.StatusSeeOther, rec.Code) + assert.Zero(t, fc.callCount) +} + +func TestUploadChatPost_PriorTurnsRideAlongAndATruncatedReplyIsSaidSo(t *testing.T) { + app, fc := newUploadChatApp(t) + fc.reply = chat.Reply{Answer: "second answer", Truncated: true} + id := seedUpload(t, userID, "m", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{ + "question": {"and who else?"}, + "hq": {"first question"}, + "ha": {"first answer"}, + })) + + require.Equal(t, http.StatusOK, rec.Code) + assert.Equal(t, []chat.Turn{{Question: "first question", Answer: "first answer"}}, fc.gotReq.History) + b := body(t, rec) + assert.Contains(t, b, "first answer") + assert.Contains(t, b, "second answer") + assert.Contains(t, b, "end of the recording", "a long recording cut to a budget must say so, in its own words") +} + +func TestUploadChatPost_AModelFailureIsShownInlineNotAs500(t *testing.T) { + app, fc := newUploadChatApp(t) + fc.err = errors.New("gateway down") + id := seedUpload(t, userID, "m", "done", meetingText) + + rec := do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}, "hq": {"earlier"}, "ha": {"kept"}})) + + require.Equal(t, http.StatusOK, rec.Code) + b := body(t, rec) + assert.Contains(t, b, "couldn't answer") + assert.Contains(t, b, "kept", "the conversation survives a failed turn") +} + +func TestUploadChatRoutes_AreNotMountedWithoutAChatBackend(t *testing.T) { + app, _ := newUploadApp(t, audioOK, 200<<20) + id := seedUpload(t, userID, "m", "done", meetingText) + + assert.Equal(t, http.StatusNotFound, do(t, app, get("/uploads/"+id+"/chat")).Code) +} + +func TestUploadStatus_DoneOffersTheChatLinkOnlyWhenChatIsWired(t *testing.T) { + appWith, _ := newUploadChatApp(t) + id := seedUpload(t, userID, "m", "done", meetingText) + assert.Contains(t, body(t, do(t, appWith, get("/uploads/"+id+"/status"))), `href="/uploads/`+id+`/chat"`) + + appWithout, _ := newUploadApp(t, audioOK, 200<<20) + id2 := seedUpload(t, userID, "m2", "done", meetingText) + assert.NotContains(t, body(t, do(t, appWithout, get("/uploads/"+id2+"/status"))), "/chat") +} -- 2.54.0 From a39f58b730fd8e641651517c16835cb2b138901d Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:58:25 +0200 Subject: [PATCH 035/299] test(web): match the HTML-escaped tail of the inline chat error templ escapes the apostrophe in "couldn't", so the assertion could never match the rendered page. A test defect, not a loosening: the failure must still be shown inline with the conversation kept. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/uploads_chat_handler_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/internal/web/uploads_chat_handler_test.go b/internal/web/uploads_chat_handler_test.go index e03d8f5..bc1ce72 100644 --- a/internal/web/uploads_chat_handler_test.go +++ b/internal/web/uploads_chat_handler_test.go @@ -145,7 +145,7 @@ func TestUploadChatPost_AModelFailureIsShownInlineNotAs500(t *testing.T) { require.Equal(t, http.StatusOK, rec.Code) b := body(t, rec) - assert.Contains(t, b, "couldn't answer") + assert.Contains(t, b, "answer just now", "the failure is inline (the apostrophe is HTML-escaped, so match the tail)") assert.Contains(t, b, "kept", "the conversation survives a failed turn") } -- 2.54.0 From 4fdb212cef0815b917f8efedc25cb6c55b4dba74 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:58:25 +0200 Subject: [PATCH 036/299] feat(web): chat with an uploaded transcript, reusing the ADR-027 path (ADR-033) GET/POST /uploads/{id}/chat, mounted only when both an Uploader and a Chatter are wired. The row comes from the user-scoped private store, so another user's upload is a 404 before any model is involved. The owner's transcript goes to the SAME Chatter with App.UploadChatMaxChars instead of the 18,000-char caption default. The chat panel is shared, not forked: chatView gains PostURL/Subject (empty = a YouTube video, so existing callers are unchanged). The done card links to chat only when chat is wired. Mutation-checked: dropping the upload budget, ignoring the done-status guard, or scoping the lookup to the wrong user each turn tests red. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/chat.go | 24 ++ internal/web/handlers.go | 10 + internal/web/uploads.go | 83 ++++- internal/web/uploads.templ | 25 +- internal/web/uploads_templ.go | 178 ++++++++-- internal/web/views.templ | 8 +- internal/web/views_templ.go | 622 ++++++++++++++++++---------------- 7 files changed, 609 insertions(+), 341 deletions(-) diff --git a/internal/web/chat.go b/internal/web/chat.go index b8e2d80..71aaa31 100644 --- a/internal/web/chat.go +++ b/internal/web/chat.go @@ -6,6 +6,8 @@ import ( "net/http" "strings" + "github.com/a-h/templ" + "git.d-ma.be/mathias/tapir/internal/adapters/chat" "git.d-ma.be/mathias/tapir/internal/adapters/store" ) @@ -43,6 +45,28 @@ type chatView struct { History []chat.Turn Truncated bool Error string + // PostURL and Subject let the same chat panel serve an UPLOADED recording + // (ADR-033): where the form posts, and what the transcript is called. Empty + // means a YouTube video (/v/{VideoID}/chat, "video"), so existing callers are + // unchanged. + PostURL templ.SafeURL + Subject string +} + +// postURL is where the chat form posts. +func (v chatView) postURL() templ.SafeURL { + if v.PostURL != "" { + return v.PostURL + } + return chatURL(v.VideoID) +} + +// subject is what the transcript is called in the panel's wording. +func (v chatView) subject() string { + if v.Subject != "" { + return v.Subject + } + return "video" } // handleChat renders the chat page for a summarized video (GET). Entry is scoped diff --git a/internal/web/handlers.go b/internal/web/handlers.go index b297dce..b9b1ed2 100644 --- a/internal/web/handlers.go +++ b/internal/web/handlers.go @@ -113,6 +113,10 @@ type App struct { // front rather than discovered by failing. Uploads Uploader UploadLimits uploads.Limits + // UploadChatMaxChars is the transcript budget for chat over an uploaded + // recording. The caption default (18,000 chars, ~22 minutes of speech) would + // keep only the first third of an hour-long meeting. + UploadChatMaxChars int // Processing tracks in-flight immediate summarizations so the status endpoint // shows the animation until the summary lands. The zero value is ready to use. Processing ProcessingSet @@ -186,6 +190,12 @@ func (a *App) Router() http.Handler { app.HandleFunc("POST /uploads", a.handleUploadPost) app.HandleFunc("GET /uploads/{id}", a.handleUploadDetail) app.HandleFunc("GET /uploads/{id}/status", a.handleUploadStatus) + // Chat over the uploaded transcript reuses the ADR-027 Chatter; mounted only + // when one is wired. + if a.Chat != nil { + app.HandleFunc("GET /uploads/{id}/chat", a.handleUploadChat) + app.HandleFunc("POST /uploads/{id}/chat", a.handleUploadChatMessage) + } } app.HandleFunc("GET /register", a.handleRegisterForm) app.HandleFunc("POST /register", a.handleRegister) diff --git a/internal/web/uploads.go b/internal/web/uploads.go index abb8711..295ced5 100644 --- a/internal/web/uploads.go +++ b/internal/web/uploads.go @@ -9,6 +9,9 @@ import ( "strings" "unicode/utf8" + "github.com/a-h/templ" + + "git.d-ma.be/mathias/tapir/internal/adapters/chat" "git.d-ma.be/mathias/tapir/internal/adapters/store" "git.d-ma.be/mathias/tapir/internal/domain" "git.d-ma.be/mathias/tapir/internal/uploads" @@ -129,7 +132,7 @@ func (a *App) handleUploadDetail(w http.ResponseWriter, r *http.Request) { if !ok { return } - a.render(w, r, UploadPage(*u)) + a.render(w, r, UploadPage(*u, a.Chat != nil)) } // handleUploadStatus renders the status fragment HTMX polls. Unfinished states @@ -140,7 +143,7 @@ func (a *App) handleUploadStatus(w http.ResponseWriter, r *http.Request) { if !ok { return } - a.render(w, r, uploadStatus(*u)) + a.render(w, r, uploadStatus(*u, a.Chat != nil)) } func (a *App) loadOwnedUpload(w http.ResponseWriter, r *http.Request) (*domain.UploadedTranscript, bool) { @@ -189,3 +192,79 @@ func uploadFailureMessage(k domain.UploadErrorKind) string { return "This upload failed. Please upload it again." } } + +// handleUploadChat renders the chat page for a FINISHED upload (GET). The row is +// read through the user-scoped store, so another user's id is a 404 before any +// model is involved. An unfinished upload has no transcript to ask about yet, so +// it goes back to its status page. +func (a *App) handleUploadChat(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + if u.Status != domain.UploadDone { + http.Redirect(w, r, "/uploads/"+u.ID, http.StatusSeeOther) + return + } + a.render(w, r, UploadChatPage(*u, a.uploadChatView(*u, a.Chat.DefaultModel(""), nil, false, ""))) +} + +// handleUploadChatMessage answers one question against the owner's uploaded +// transcript (POST), reusing the ADR-027 Chatter. Nothing is persisted: the prior +// turns ride in the form. The transcript goes to the model with the UPLOAD budget, +// not the caption default. A model failure is inline, not a 500. +func (a *App) handleUploadChatMessage(w http.ResponseWriter, r *http.Request) { + u, ok := a.loadOwnedUpload(w, r) + if !ok { + return + } + if u.Status != domain.UploadDone { + http.Redirect(w, r, "/uploads/"+u.ID, http.StatusSeeOther) + return + } + if err := r.ParseForm(); err != nil { + http.Error(w, "bad form", http.StatusBadRequest) + return + } + model := a.resolveModel(r.FormValue("model"), "") + history := parseHistory(r.Form["hq"], r.Form["ha"]) + question := strings.TrimSpace(r.FormValue("question")) + + view := a.uploadChatView(*u, model, history, false, "") + if question != "" { + reply, err := a.Chat.Answer(r.Context(), chat.Request{ + Model: model, + Transcript: u.Content, + History: history, + Question: question, + MaxChars: a.UploadChatMaxChars, + }) + if err != nil { + a.logger().Error("upload chat answer", "upload", u.ID, "model", model, "err", err) + view.Error = "That model couldn't answer just now. Try again, or switch models." + } else { + view.History = appendTurn(history, chat.Turn{Question: question, Answer: reply.Answer}) + view.Truncated = reply.Truncated + } + } + if isHTMX(r) { + a.render(w, r, chatPanel(view)) + return + } + a.render(w, r, UploadChatPage(*u, view)) +} + +func (a *App) uploadChatView(u domain.UploadedTranscript, model string, history []chat.Turn, truncated bool, errMsg string) chatView { + return chatView{ + VideoID: u.ID, + Title: u.Title, + Available: true, + Models: a.Chat.Models(), + Selected: model, + History: history, + Truncated: truncated, + Error: errMsg, + PostURL: templ.SafeURL("/uploads/" + u.ID + "/chat"), + Subject: "recording", + } +} diff --git a/internal/web/uploads.templ b/internal/web/uploads.templ index ffcb759..b084c10 100644 --- a/internal/web/uploads.templ +++ b/internal/web/uploads.templ @@ -44,11 +44,11 @@ templ UploadsPage(v uploadsView) { } // UploadPage is one upload's page: its title and the polling status region. -templ UploadPage(u domain.UploadedTranscript) { +templ UploadPage(u domain.UploadedTranscript, chatEnabled bool) { @Layout("Tapir — " + u.Title) { } @@ -56,7 +56,7 @@ templ UploadPage(u domain.UploadedTranscript) { // uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the // page re-asks; done and failed render without it, so nothing polls forever. -templ uploadStatus(u domain.UploadedTranscript) { +templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool) { switch u.Status { case domain.UploadDone:
@@ -64,6 +64,9 @@ templ uploadStatus(u domain.UploadedTranscript) { Transcribed { uploads.FormatDuration(time.Duration(u.AudioSeconds * float64(time.Second))) } of audio by { u.Endpoint }.

+ if chatEnabled { +

Ask about this recording

+ }
Transcript 
{ u.Content }
@@ -89,3 +92,19 @@ templ uploadStatus(u domain.UploadedTranscript) {
} } + +// UploadChatPage is the no-JS full page for chatting with an uploaded transcript: +// the same chat panel as a video's (ADR-027), scoped to this recording. +templ UploadChatPage(u domain.UploadedTranscript, v chatView) { + @Layout("Tapir — Ask about " + u.Title) { + + } +} diff --git a/internal/web/uploads_templ.go b/internal/web/uploads_templ.go index a874853..c848d97 100644 --- a/internal/web/uploads_templ.go +++ b/internal/web/uploads_templ.go @@ -5,8 +5,10 @@ package web //lint:file-ignore SA4006 This context is only used if a nested component is present. -import "github.com/a-h/templ" -import templruntime "github.com/a-h/templ/runtime" +import ( + "github.com/a-h/templ" + templruntime "github.com/a-h/templ/runtime" +) import ( "fmt" @@ -173,7 +175,7 @@ func UploadsPage(v uploadsView) templ.Component { } // UploadPage is one upload's page: its title and the polling status region. -func UploadPage(u domain.UploadedTranscript) templ.Component { +func UploadPage(u domain.UploadedTranscript, chatEnabled bool) templ.Component { return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { @@ -223,7 +225,7 @@ func UploadPage(u domain.UploadedTranscript) templ.Component { if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = uploadStatus(u).Render(ctx, templ_7745c5c3_Buffer) + templ_7745c5c3_Err = uploadStatus(u, chatEnabled).Render(ctx, templ_7745c5c3_Buffer) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -243,7 +245,7 @@ func UploadPage(u domain.UploadedTranscript) templ.Component { // uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the // page re-asks; done and failed render without it, so nothing polls forever. -func uploadStatus(u domain.UploadedTranscript) templ.Component { +func uploadStatus(u domain.UploadedTranscript, chatEnabled bool) templ.Component { return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { @@ -292,84 +294,107 @@ func uploadStatus(u domain.UploadedTranscript) templ.Component { if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, ".

Transcript
")
+			templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 19, ".
")
 			if templ_7745c5c3_Err != nil {
 				return templ_7745c5c3_Err
 			}
-			var templ_7745c5c3_Var15 string
-			templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinStringErrs(u.Content)
-			if templ_7745c5c3_Err != nil {
-				return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 69, Col: 40}
+			if chatEnabled {
+				templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "
Ask about this recording
")
+				if templ_7745c5c3_Err != nil {
+					return templ_7745c5c3_Err
+				}
 			}
-			_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15))
-			if templ_7745c5c3_Err != nil {
-				return templ_7745c5c3_Err
-			}
-			templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 20, "
") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - case domain.UploadFailed: - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "

Transcript
")
 			if templ_7745c5c3_Err != nil {
 				return templ_7745c5c3_Err
 			}
 			var templ_7745c5c3_Var16 string
-			templ_7745c5c3_Var16, templ_7745c5c3_Err = templ.JoinStringErrs(uploadFailureMessage(u.ErrorKind))
+			templ_7745c5c3_Var16, templ_7745c5c3_Err = templ.JoinStringErrs(u.Content)
 			if templ_7745c5c3_Err != nil {
-				return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 74, Col: 42}
+				return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 72, Col: 40}
 			}
 			_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var16))
 			if templ_7745c5c3_Err != nil {
 				return templ_7745c5c3_Err
 			}
-			templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - default: - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } var templ_7745c5c3_Var17 string - templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.ResolveAttributeValue("/uploads/" + u.ID + "/status") + templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.JoinStringErrs(uploadFailureMessage(u.ErrorKind)) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 80, Col: 43} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 77, Col: 42} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var17) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var17)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "\" hx-trigger=\"") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 25, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + default: + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "\" hx-trigger=\"") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var19 string + templ_7745c5c3_Var19, templ_7745c5c3_Err = templ.ResolveAttributeValue("every " + uploadPollEvery) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 84, Col: 43} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var19) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "\" hx-swap=\"outerHTML\">") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } if u.Status == domain.UploadTranscribing { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "

Transcribing… a long recording takes a few minutes.

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 29, "

Transcribing… a long recording takes a few minutes.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } else { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "

Queued — waiting for its turn to be transcribed.

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 30, "

Queued — waiting for its turn to be transcribed.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 31, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -378,4 +403,87 @@ func uploadStatus(u domain.UploadedTranscript) templ.Component { }) } +// UploadChatPage is the no-JS full page for chatting with an uploaded transcript: +// the same chat panel as a video's (ADR-027), scoped to this recording. +func UploadChatPage(u domain.UploadedTranscript, v chatView) templ.Component { + return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil { + return templ_7745c5c3_CtxErr + } + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Var20 := templ.GetChildren(ctx) + if templ_7745c5c3_Var20 == nil { + templ_7745c5c3_Var20 = templ.NopComponent + } + ctx = templ.ClearChildren(ctx) + templ_7745c5c3_Var21 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context + templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) + if !templ_7745c5c3_IsBuffer { + defer func() { + templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer) + if templ_7745c5c3_Err == nil { + templ_7745c5c3_Err = templ_7745c5c3_BufErr + } + }() + } + ctx = templ.InitializeContext(ctx) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 32, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var22 string + templ_7745c5c3_Var22, templ_7745c5c3_Err = templ.JoinStringErrs(u.Title) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 101, Col: 16} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var22)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 33, "

Ask about this recording

Answers come only from this recording's transcript, which is private to you.

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = chatPanel(v).Render(ctx, templ_7745c5c3_Buffer) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 34, "

Back to the transcript

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) + templ_7745c5c3_Err = Layout("Tapir — Ask about "+u.Title).Render(templ.WithChildren(ctx, templ_7745c5c3_Var21), templ_7745c5c3_Buffer) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + return nil + }) +} + var _ = templruntime.GeneratedTemplate diff --git a/internal/web/views.templ b/internal/web/views.templ index 602dafd..55d9c9b 100644 --- a/internal/web/views.templ +++ b/internal/web/views.templ @@ -583,7 +583,7 @@ templ chatPanel(v chatView) { } if v.Truncated { -

Working from a bounded portion of a long transcript — answers about the end of the video may be incomplete.

+

Working from a bounded portion of a long transcript — answers about the end of the { v.subject() } may be incomplete.

} if v.Error != "" { @@ -591,8 +591,8 @@ templ chatPanel(v chatView) {
@@ -609,7 +609,7 @@ templ chatPanel(v chatView) { Switch models to compare answers on the same transcript. - + thinking…
diff --git a/internal/web/views_templ.go b/internal/web/views_templ.go index 1a0562b..8f2732f 100644 --- a/internal/web/views_templ.go +++ b/internal/web/views_templ.go @@ -5,8 +5,10 @@ package web //lint:file-ignore SA4006 This context is only used if a nested component is present. -import "github.com/a-h/templ" -import templruntime "github.com/a-h/templ/runtime" +import ( + "github.com/a-h/templ" + templruntime "github.com/a-h/templ/runtime" +) import ( "fmt" @@ -292,7 +294,7 @@ func flashBanner(code string) templ.Component { } ctx = templ.ClearChildren(ctx) if f, ok := flashFor(code); ok { - var templ_7745c5c3_Var9 = []any{"flash", "flash-" + f.Kind} + templ_7745c5c3_Var9 := []any{"flash", "flash-" + f.Kind} templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var9...) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err @@ -824,7 +826,7 @@ func VideoCard(r store.SummaryRow) templ.Component { templ_7745c5c3_Var25 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - var templ_7745c5c3_Var26 = []any{"card", templ.KV("card-pending", !r.Summarized)} + templ_7745c5c3_Var26 := []any{"card", templ.KV("card-pending", !r.Summarized)} templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var26...) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err @@ -2152,148 +2154,174 @@ func chatPanel(v chatView) templ.Component { return templ_7745c5c3_Err } if v.Truncated { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 173, "

Working from a bounded portion of a long transcript — answers about the end of the video may be incomplete.

") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 174, " ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - if v.Error != "" { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 175, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 173, "

Working from a bounded portion of a long transcript — answers about the end of the ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } var templ_7745c5c3_Var88 string - templ_7745c5c3_Var88, templ_7745c5c3_Err = templ.JoinStringErrs(v.Error) + templ_7745c5c3_Var88, templ_7745c5c3_Err = templ.JoinStringErrs(v.subject()) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 589, Col: 48} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 586, Col: 129} } _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var88)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 176, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 174, " may be incomplete.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 177, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var89 string + templ_7745c5c3_Var89, templ_7745c5c3_Err = templ.JoinStringErrs(v.Error) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 589, Col: 48} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var89)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 177, "

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var89)) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 178, " ") + var templ_7745c5c3_Var91 string + templ_7745c5c3_Var91, templ_7745c5c3_Err = templ.ResolveAttributeValue(string(v.postURL())) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 595, Col: 33} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var91) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 180, "\" hx-target=\"#chat-panel\" hx-swap=\"outerHTML\">") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } for _, t := range v.History { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 180, " ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 183, "
thinking…") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 190, "") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 192, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2320,12 +2348,12 @@ func RegisterPage(email, errMsg string) templ.Component { }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Var95 := templ.GetChildren(ctx) - if templ_7745c5c3_Var95 == nil { - templ_7745c5c3_Var95 = templ.NopComponent + templ_7745c5c3_Var97 := templ.GetChildren(ctx) + if templ_7745c5c3_Var97 == nil { + templ_7745c5c3_Var97 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - templ_7745c5c3_Var96 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_Var98 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) if !templ_7745c5c3_IsBuffer { @@ -2337,59 +2365,59 @@ func RegisterPage(email, errMsg string) templ.Component { }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 191, "

Complete your registration

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 193, "

Complete your registration

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } if email != "" { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 192, "

Signed in as ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 194, "

Signed in as ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var97 string - templ_7745c5c3_Var97, templ_7745c5c3_Err = templ.JoinStringErrs(email) + var templ_7745c5c3_Var99 string + templ_7745c5c3_Var99, templ_7745c5c3_Err = templ.JoinStringErrs(email) if templ_7745c5c3_Err != nil { return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 628, Col: 40} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var97)) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var99)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 193, ".

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 195, ".

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 194, "

Choose a display name to finish setting up your Tapir account.

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 196, "

Choose a display name to finish setting up your Tapir account.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } if errMsg != "" { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 195, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 197, "

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var98 string - templ_7745c5c3_Var98, templ_7745c5c3_Err = templ.JoinStringErrs(errMsg) + var templ_7745c5c3_Var100 string + templ_7745c5c3_Var100, templ_7745c5c3_Err = templ.JoinStringErrs(errMsg) if templ_7745c5c3_Err != nil { return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 632, Col: 42} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var98)) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var100)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 196, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 198, "

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 197, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 199, "
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } return nil }) - templ_7745c5c3_Err = Layout("Tapir — Register").Render(templ.WithChildren(ctx, templ_7745c5c3_Var96), templ_7745c5c3_Buffer) + templ_7745c5c3_Err = Layout("Tapir — Register").Render(templ.WithChildren(ctx, templ_7745c5c3_Var98), templ_7745c5c3_Buffer) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2417,12 +2445,12 @@ func AccountPage(displayName, email string, conns []store.Connection, autoSummar }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Var99 := templ.GetChildren(ctx) - if templ_7745c5c3_Var99 == nil { - templ_7745c5c3_Var99 = templ.NopComponent + templ_7745c5c3_Var101 := templ.GetChildren(ctx) + if templ_7745c5c3_Var101 == nil { + templ_7745c5c3_Var101 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - templ_7745c5c3_Var100 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { + templ_7745c5c3_Var102 := templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) { templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W) if !templ_7745c5c3_IsBuffer { @@ -2438,43 +2466,43 @@ func AccountPage(displayName, email string, conns []store.Connection, autoSummar if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 198, "

Account

Display name
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 200, "

Account

Display name
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var101 string - templ_7745c5c3_Var101, templ_7745c5c3_Err = templ.JoinStringErrs(displayNameOr(displayName)) + var templ_7745c5c3_Var103 string + templ_7745c5c3_Var103, templ_7745c5c3_Err = templ.JoinStringErrs(displayNameOr(displayName)) if templ_7745c5c3_Err != nil { return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 656, Col: 36} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var101)) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var103)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 199, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 201, "
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } if email != "" { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 200, "
Signed in as
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 202, "
Signed in as
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var102 string - templ_7745c5c3_Var102, templ_7745c5c3_Err = templ.JoinStringErrs(email) + var templ_7745c5c3_Var104 string + templ_7745c5c3_Var104, templ_7745c5c3_Err = templ.JoinStringErrs(email) if templ_7745c5c3_Err != nil { return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 659, Col: 16} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var102)) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var104)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 201, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 203, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 202, "

Summarization

Automatic summarizes new videos from about the last week as they are discovered. Older videos stay browsable — summarize them on demand. Manual lets you pick which videos to summarize — every new video appears in your list with a Summarize button.

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 204, "

Summarization

Automatic summarizes new videos from about the last week as they are discovered. Older videos stay browsable — summarize them on demand. Manual lets you pick which videos to summarize — every new video appears in your list with a Summarize button.

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2482,178 +2510,178 @@ func AccountPage(displayName, email string, conns []store.Connection, autoSummar if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 203, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 205, "
") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } if len(channelErrors) > 0 { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 204, "

Unavailable channels

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 206, "

Unavailable channels

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var103 string - templ_7745c5c3_Var103, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d channel(s) returned errors on the last discovery pass.", len(channelErrors))) + var templ_7745c5c3_Var105 string + templ_7745c5c3_Var105, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d channel(s) returned errors on the last discovery pass.", len(channelErrors))) if templ_7745c5c3_Err != nil { return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 676, Col: 100} } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var103)) + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var105)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 205, " These may have been deleted or made private on YouTube.

    ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 207, " These may have been deleted or made private on YouTube.

      ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } for _, ce := range channelErrors { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 206, "
    • ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var104 string - templ_7745c5c3_Var104, templ_7745c5c3_Err = templ.JoinStringErrs(ce.ChannelName) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 682, Col: 57} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var104)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 207, " unavailable since ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var105 string - templ_7745c5c3_Var105, templ_7745c5c3_Err = templ.JoinStringErrs(ce.FirstSeen.Format("2006-01-02")) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 684, Col: 89} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var105)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 208, "
      • ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 209, "
") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 210, "

Connected accounts

") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - if len(conns) == 0 { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 211, "

No connected video accounts yet.

") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - } else { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 212, "
    ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - for _, c := range conns { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 213, "
  • ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 208, "
  • ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } var templ_7745c5c3_Var106 string - templ_7745c5c3_Var106, templ_7745c5c3_Err = templ.JoinStringErrs(providerLabel(c.Provider)) + templ_7745c5c3_Var106, templ_7745c5c3_Err = templ.JoinStringErrs(ce.ChannelName) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 699, Col: 64} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 682, Col: 57} } _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var106)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 214, " ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 209, " unavailable since ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - if c.ProviderAccount != "" { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 215, "") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var107 string - templ_7745c5c3_Var107, templ_7745c5c3_Err = templ.JoinStringErrs(c.ProviderAccount) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 701, Col: 49} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var107)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 216, " ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } + var templ_7745c5c3_Var107 string + templ_7745c5c3_Var107, templ_7745c5c3_Err = templ.JoinStringErrs(ce.FirstSeen.Format("2006-01-02")) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 684, Col: 89} } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 217, "") + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var107)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 210, "
    • ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 211, "
") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 212, "

Connected accounts

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + if len(conns) == 0 { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 213, "

No connected video accounts yet.

") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + } else { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 214, "
    ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + for _, c := range conns { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 215, "
  • ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } var templ_7745c5c3_Var108 string - templ_7745c5c3_Var108, templ_7745c5c3_Err = templ.JoinStringErrs(c.Status) + templ_7745c5c3_Var108, templ_7745c5c3_Err = templ.JoinStringErrs(providerLabel(c.Provider)) if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 703, Col: 38} + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 699, Col: 64} } _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var108)) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 218, "
    connected ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 216, " ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - var templ_7745c5c3_Var109 string - templ_7745c5c3_Var109, templ_7745c5c3_Err = templ.JoinStringErrs(c.ConnectedAt.Format("2006-01-02")) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 705, Col: 83} + if c.ProviderAccount != "" { + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 217, "") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var109 string + templ_7745c5c3_Var109, templ_7745c5c3_Err = templ.JoinStringErrs(c.ProviderAccount) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 701, Col: 49} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var109)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 218, " ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var109)) + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 219, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 219, "
    • ") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 220, "
    connected ") + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + var templ_7745c5c3_Var111 string + templ_7745c5c3_Var111, templ_7745c5c3_Err = templ.JoinStringErrs(c.ConnectedAt.Format("2006-01-02")) + if templ_7745c5c3_Err != nil { + return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 705, Col: 83} + } + _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var111)) + if templ_7745c5c3_Err != nil { + return templ_7745c5c3_Err + } + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 221, "
    ") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 221, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 223, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } if !hasYouTube(conns) { - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 222, "

Connect YouTube

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 224, "

Connect YouTube

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 223, "

Delete account

Permanently remove your Tapir account and all of its data — summaries, watch/skip/save actions, and connected accounts. This cannot be undone.

Delete account…

This permanently deletes your account and all data. Are you sure?

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 225, "

Delete account

Permanently remove your Tapir account and all of its data — summaries, watch/skip/save actions, and connected accounts. This cannot be undone.

Delete account…

This permanently deletes your account and all data. Are you sure?

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } return nil }) - templ_7745c5c3_Err = Layout("Tapir — Account").Render(templ.WithChildren(ctx, templ_7745c5c3_Var100), templ_7745c5c3_Buffer) + templ_7745c5c3_Err = Layout("Tapir — Account").Render(templ.WithChildren(ctx, templ_7745c5c3_Var102), templ_7745c5c3_Buffer) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2681,51 +2709,51 @@ func summarizeModeControl(auto bool) templ.Component { }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Var111 := templ.GetChildren(ctx) - if templ_7745c5c3_Var111 == nil { - templ_7745c5c3_Var111 = templ.NopComponent + templ_7745c5c3_Var113 := templ.GetChildren(ctx) + if templ_7745c5c3_Var113 == nil { + templ_7745c5c3_Var113 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 224, "

Current mode: ") - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - var templ_7745c5c3_Var112 string - templ_7745c5c3_Var112, templ_7745c5c3_Err = templ.JoinStringErrs(summarizeModeLabel(auto)) - if templ_7745c5c3_Err != nil { - return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/views.templ`, Line: 743, Col: 53} - } - _, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var112)) - if templ_7745c5c3_Err != nil { - return templ_7745c5c3_Err - } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 225, "

") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 227, "

") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2753,38 +2781,38 @@ func ActionButtons(videoID string, active map[string]bool) templ.Component { }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Var115 := templ.GetChildren(ctx) - if templ_7745c5c3_Var115 == nil { - templ_7745c5c3_Var115 = templ.NopComponent + templ_7745c5c3_Var117 := templ.GetChildren(ctx) + if templ_7745c5c3_Var117 == nil { + templ_7745c5c3_Var117 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 228, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 232, "\" hx-target=\"#action-buttons\" hx-swap=\"outerHTML\">") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2796,7 +2824,7 @@ func ActionButtons(videoID string, active map[string]bool) templ.Component { if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 231, "") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 233, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2804,7 +2832,7 @@ func ActionButtons(videoID string, active map[string]bool) templ.Component { if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 232, "
") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 234, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } @@ -2830,81 +2858,81 @@ func actionButton(verb string, isActive bool) templ.Component { }() } ctx = templ.InitializeContext(ctx) - templ_7745c5c3_Var118 := templ.GetChildren(ctx) - if templ_7745c5c3_Var118 == nil { - templ_7745c5c3_Var118 = templ.NopComponent + templ_7745c5c3_Var120 := templ.GetChildren(ctx) + if templ_7745c5c3_Var120 == nil { + templ_7745c5c3_Var120 = templ.NopComponent } ctx = templ.ClearChildren(ctx) - var templ_7745c5c3_Var119 = []any{"action", templ.KV("active", isActive)} - templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var119...) + templ_7745c5c3_Var121 := []any{"action", templ.KV("active", isActive)} + templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var121...) if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } - templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 233, "") + templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 239, "") if templ_7745c5c3_Err != nil { return templ_7745c5c3_Err } -- 2.54.0 From 3464cdf87652c8a59241ab79b5de5c1584228ca4 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:01:06 +0200 Subject: [PATCH 037/299] test(e2e): opt-in live upload -> transcript -> grounded chat check Real ffprobe/ffmpeg, the live gateway and a real chat model, a private embedded Postgres. Skipped unless TAPIR_E2E_AUDIO is set. Run 2026-10-01 with 82 s of synthetic Swedish as AAC .m4a: - primary: iguana/kb-whisper, 82.6 s audio, 1,186 chars; asked who owns the timber order, the answer named Anders and the 15 November deadline from the transcript; the spool directory was empty afterwards. - primary forced to fail (TAPIR_TRANSCRIBE_MODEL=iguana/does-not-exist): the fallback served it and the row recorded endpoint=berget/kb-whisper. Synthetic audio only; this repo is public. Co-Authored-By: Claude Sonnet 5.5 --- cmd/tapir/e2e_upload_test.go | 170 +++++++++++++++++++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100644 cmd/tapir/e2e_upload_test.go diff --git a/cmd/tapir/e2e_upload_test.go b/cmd/tapir/e2e_upload_test.go new file mode 100644 index 0000000..0cf9ecc --- /dev/null +++ b/cmd/tapir/e2e_upload_test.go @@ -0,0 +1,170 @@ +package main + +import ( + "context" + "fmt" + "io" + "mime/multipart" + "net/http" + "net/http/httptest" + "net/url" + "os" + "path/filepath" + "strings" + "testing" + "time" + + embeddedpostgres "github.com/fergusstrange/embedded-postgres" + "github.com/jackc/pgx/v5/pgxpool" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "git.d-ma.be/mathias/tapir/internal/adapters/store" + "git.d-ma.be/mathias/tapir/internal/config" + "git.d-ma.be/mathias/tapir/internal/web" +) + +// TestE2E_UploadTranscribeAndChat is the end-to-end check for tapir#33: upload a +// real audio file, watch it become a transcript through the REAL ffprobe, ffmpeg +// and gateway, then ask a question and get a grounded answer from a REAL model. +// It is opt-in — it needs real binaries and the live gateway — and skipped unless +// TAPIR_E2E_AUDIO is set: +// +// TAPIR_E2E_AUDIO=/path/meeting.m4a TAPIR_E2E_FFMPEG=/path/ffmpeg \ +// TAPIR_E2E_FFPROBE=/path/ffprobe TAPIR_E2E_QUESTION='Vem ansvarar för virket?' \ +// TAPIR_E2E_EXPECT=anders op run --env-file ~/.op-env -- \ +// go test ./cmd/tapir -run E2E_Upload -v -count=1 +// +// Use SYNTHETIC audio only: this repo is public, and a fallback to berget sends +// the audio to a cloud endpoint (#33). +func TestE2E_UploadTranscribeAndChat(t *testing.T) { + audioPath := os.Getenv("TAPIR_E2E_AUDIO") + if audioPath == "" { + t.Skip("set TAPIR_E2E_AUDIO (and friends) to run the live end-to-end check") + } + key := os.Getenv("LITELLM_MASTER_KEY") + require.NotEmpty(t, key, "LITELLM_MASTER_KEY must be set (via op run)") + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + + // A private embedded Postgres: nothing here may touch a real database. + port := uint32(55000 + os.Getpid()%1000) + rt := filepath.Join(os.TempDir(), fmt.Sprintf("tapir-epg-e2e-%d", os.Getpid())) + pg := embeddedpostgres.NewDatabase(embeddedpostgres.DefaultConfig().Port(port). + RuntimePath(rt).DataPath(filepath.Join(rt, "data")).BinariesPath(filepath.Join(rt, "bin")). + CachePath(filepath.Join(os.TempDir(), "tapir-epg-cache"))) + require.NoError(t, pg.Start()) + t.Cleanup(func() { _ = pg.Stop(); _ = os.RemoveAll(rt) }) + dsn := fmt.Sprintf("postgres://postgres:postgres@localhost:%d/postgres?sslmode=disable", port) + + st, err := store.New(ctx, dsn) + require.NoError(t, err) + t.Cleanup(st.Close) + raw, err := pgxpool.New(ctx, dsn) + require.NoError(t, err) + t.Cleanup(raw.Close) + const uid = "11111111-1111-1111-1111-111111111111" + _, err = raw.Exec(ctx, `INSERT INTO users (id) VALUES ($1)`, uid) + require.NoError(t, err) + _, err = raw.Exec(ctx, `INSERT INTO user_identities (dex_subject, user_id) VALUES ('e2e', $1)`, uid) + require.NoError(t, err) + + cfg, err := config.Load() + require.NoError(t, err) + cfg.GatewayKey = key + cfg.UploadsEnabled = true + cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool") + cfg.FFmpegBin = envOrDefault("TAPIR_E2E_FFMPEG", "ffmpeg") + cfg.FFprobeBin = envOrDefault("TAPIR_E2E_FFPROBE", "ffprobe") + + intake, worker, err := buildUploads(ctx, cfg, st, quietLog()) + require.NoError(t, err) + go worker.Run(ctx) + app := &web.App{ + Store: st, Identity: st, Auth: web.StubAuth{U: web.User{Subject: "e2e"}}, + Uploads: intake, UploadLimits: uploadLimits(cfg), + Chat: buildChat(cfg), UploadChatMaxChars: cfg.MaxUploadedTranscriptChars, + } + srv := httptest.NewServer(app.Router()) + t.Cleanup(srv.Close) + noRedirect := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} + + // 1. Upload. + f, err := os.Open(audioPath) + require.NoError(t, err) + defer f.Close() + pr, pw := io.Pipe() + mw := multipart.NewWriter(pw) + go func() { + _ = mw.WriteField("title", "E2E recording") + fw, _ := mw.CreateFormFile("file", filepath.Base(audioPath)) + _, _ = io.Copy(fw, f) + _ = mw.Close() + _ = pw.Close() + }() + req, err := http.NewRequest(http.MethodPost, srv.URL+"/uploads", pr) + require.NoError(t, err) + req.Header.Set("Content-Type", mw.FormDataContentType()) + resp, err := noRedirect.Do(req) + require.NoError(t, err) + _ = resp.Body.Close() + require.Equal(t, http.StatusSeeOther, resp.StatusCode, "an accepted upload redirects to its page") + loc := resp.Header.Get("Location") + require.True(t, strings.HasPrefix(loc, "/uploads/"), loc) + t.Logf("accepted -> %s", loc) + + // 2. Wait for the transcript: unfinished states poll, terminal states do not. + var page string + require.Eventually(t, func() bool { + r, err := http.Get(srv.URL + loc + "/status") + if err != nil { + return false + } + b, _ := io.ReadAll(r.Body) + _ = r.Body.Close() + page = string(b) + return !strings.Contains(page, "hx-trigger") + }, 12*time.Minute, 3*time.Second, "the upload never reached a terminal state") + require.Contains(t, page, "Transcribed", "terminal state was not 'done': %.300s", page) + + var status, endpoint, content string + var secs float64 + require.NoError(t, raw.QueryRow(ctx, + `SELECT status, endpoint, content, audio_seconds FROM uploaded_transcripts WHERE user_id = $1`, uid). + Scan(&status, &endpoint, &content, &secs)) + t.Logf("done: endpoint=%s audio_s=%.1f transcript_chars=%d", endpoint, secs, len(content)) + assert.Equal(t, "done", status) + assert.NotEmpty(t, endpoint) + assert.NotEmpty(t, content) + entries, err := os.ReadDir(cfg.UploadTmpDir) + require.NoError(t, err) + assert.Empty(t, entries, "the audio must be deleted once transcribed") + + // 3. Ask a question; the answer must come from the transcript. + q := envOrDefault("TAPIR_E2E_QUESTION", "Who is responsible for the timber order?") + chatResp, err := http.PostForm(srv.URL+loc+"/chat", url.Values{"question": {q}}) + require.NoError(t, err) + answerPage, _ := io.ReadAll(chatResp.Body) + _ = chatResp.Body.Close() + require.Equal(t, http.StatusOK, chatResp.StatusCode) + t.Logf("question: %s", q) + if want := os.Getenv("TAPIR_E2E_EXPECT"); want != "" { + assert.Contains(t, strings.ToLower(string(answerPage)), strings.ToLower(want), + "the answer should be grounded in the transcript") + } + t.Logf("answer page excerpt: %.600s", afterMarker(string(answerPage), `class="chat-turn chat-a"`)) +} + +func envOrDefault(k, d string) string { + if v := os.Getenv(k); v != "" { + return v + } + return d +} + +func afterMarker(s, m string) string { + if i := strings.Index(s, m); i >= 0 { + return s[i:] + } + return "(no answer turn found)" +} -- 2.54.0 From f9d6df9f846f8247d0daf3282a54a5a5f23082e0 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:01:20 +0200 Subject: [PATCH 038/299] docs(use-cases): audio_upload.feature -- scenarios for #33 (ADR-033) Co-Authored-By: Claude Sonnet 5.5 --- docs/use-cases/audio_upload.feature | 101 ++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/use-cases/audio_upload.feature diff --git a/docs/use-cases/audio_upload.feature b/docs/use-cases/audio_upload.feature new file mode 100644 index 0000000..4665a37 --- /dev/null +++ b/docs/use-cases/audio_upload.feature @@ -0,0 +1,101 @@ +Feature: Upload an audio recording, get a private Swedish transcript, discuss it + As someone with a recorded meeting on my computer + I want to upload it and then ask questions about what was said + So that I can find out what was decided, what is open and who owes what + + # ADR-033, tapir#33. The load-bearing constraint: an uploaded transcript is + # PRIVATE content, so it lives in its own RLS-scoped store and never in the + # shared public-caption `transcripts` table (ADR-021). Rejection is synchronous, + # the audio is never retained, and a failed job ends in a terminal state with a + # message — never a spinner (ADR-025). Audio only: video is #28. + + Background: + Given I am a registered user with audio upload enabled + + Scenario: The upload form states the limits up front + When I open the upload page + Then I see the size limit and the length limit before I choose a file + + Scenario: An accepted recording is queued and I am taken to its page + When I upload an audio file within the limits + Then it is recorded as queued for me + And I am redirected to its page + + Scenario: A recording with no title is named after its file + When I upload an audio file and give no title + Then its title is the filename without its extension + + Scenario: A file containing video is rejected, naming issue 28 + When I upload a file that has a video track + Then it is rejected with a message that video is tracked in issue 28 + And nothing is recorded + + Scenario: A recording over the length limit is rejected with both lengths + When I upload a recording longer than the limit + Then it is rejected stating the recording's length and the limit + And nothing is recorded + + Scenario: A file over the size limit is rejected as too large + When I upload a file larger than the size limit + Then it is rejected with 413 and a message about the limit + And nothing is recorded + + Scenario: An audio-only MP4 is not mistaken for video + When I upload an mp4 container that has an audio stream and no video stream + Then it is accepted + And an mp3 whose only video stream is cover art is accepted too + + Scenario: A file that is not media is rejected as unreadable + When I upload a file ffprobe cannot read + Then it is rejected as the file's fault + + Scenario: A broken ffprobe is reported as our fault, not as a bad file + When the prober itself fails + Then the upload is an error rather than a rejection of my file + + Scenario: A recording becomes a transcript and the audio is deleted + Given I have uploaded a recording + When it has been transcribed + Then the transcript is stored privately for me + And the audio is deleted + + Scenario: The endpoint that served a transcript is recorded and shown + Given a recording was transcribed by the fallback endpoint + When I view its page + Then it names the fallback endpoint, because that means the audio left the homelab + + Scenario: The local endpoint is tried first and the fallback only on failure + When the primary transcription endpoint fails for any reason + Then the fallback endpoint transcribes it + But when the primary succeeds the fallback is never contacted + + Scenario: A failed transcription ends in a message, never a spinner + When every transcription endpoint fails + Then the upload is failed with a typed reason + And its page shows that reason and stops polling + + Scenario: A restart fails the jobs it interrupted + Given the service restarted while a recording was queued or transcribing + When it starts again + Then those uploads are failed as interrupted + And any audio left in the spool is deleted + + Scenario: Only one recording is transcribed at a time + When several recordings are queued + Then they are transcribed one after another + + Scenario: My uploads are invisible to other users + Given another user has uploaded a recording + When I ask for it by its id + Then I get 404 and no part of its title or text + And the chat for it calls no model + + Scenario: I can ask about my recording and get a grounded answer + Given my recording has been transcribed + When I ask a question about it + Then the answer comes from my transcript through the same chat as a video's + + Scenario: A long recording is not cut to its first third + Given my recording's transcript is longer than the caption default + When I ask a question about it + Then the transcript is sent with the upload budget, not the 18,000-character caption default -- 2.54.0 From 07375f75dec88b07b4645822557cd16dc8854582 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:01:38 +0200 Subject: [PATCH 039/299] test(acceptance): map audio_upload.feature scenarios to their covering tests Co-Authored-By: Claude Sonnet 5.5 --- test/acceptance/scenario_coverage_test.go | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/test/acceptance/scenario_coverage_test.go b/test/acceptance/scenario_coverage_test.go index ccabadb..76bc7b9 100644 --- a/test/acceptance/scenario_coverage_test.go +++ b/test/acceptance/scenario_coverage_test.go @@ -101,6 +101,26 @@ var scenarioCoverage = map[string]string{ "A multi-turn conversation is ephemeral": "TestChatMultiTurnHistoryIsEphemeral", "Chat is reachable only from my own summary view": "TestChatOnlyReachableForOwnVideo", + // audio_upload.feature (ADR-033, #33) + "The upload form states the limits up front": "TestUploadsPage_ShowsAMultipartFormWithAFileField", + "An accepted recording is queued and I am taken to its page": "TestUploadPost_AcceptedAudioQueuesARowAndRedirectsToIt", + "A recording with no title is named after its file": "TestUploadPost_WithoutATitleUsesTheFilenameMinusItsExtension", + "A file containing video is rejected, naming issue 28": "TestUploadPost_RejectionsAreSynchronousWithAMessageAndNoRow", + "A recording over the length limit is rejected with both lengths": "TestUploadPost_RejectionsAreSynchronousWithAMessageAndNoRow", + "A file over the size limit is rejected as too large": "TestUploadPost_RejectionsAreSynchronousWithAMessageAndNoRow", + "An audio-only MP4 is not mistaken for video": "TestParseProbe", + "A file that is not media is rejected as unreadable": "TestAccept_UnreadableFileIsARejectionButAProberFaultIsAnError", + "A broken ffprobe is reported as our fault, not as a bad file": "TestAccept_UnreadableFileIsARejectionButAProberFaultIsAnError", + "A recording becomes a transcript and the audio is deleted": "TestWorkerProcess_SuccessConvertsTranscribesStoresAndDeletesTheAudio", + "The endpoint that served a transcript is recorded and shown": "TestUploadStatus_DoneNamesTheEndpointThatServedIt", + "The local endpoint is tried first and the fallback only on failure": "TestChain_AnyPrimaryFailureFallsBackAndRecordsTheFallbackEndpoint", + "A failed transcription ends in a message, never a spinner": "TestUploadStatus_UnfinishedStatesKeepPollingTerminalStatesStop", + "A restart fails the jobs it interrupted": "TestFailInterruptedUploads_FailsOnlyUnfinishedRowsForEveryUser", + "Only one recording is transcribed at a time": "TestWorkerRun_ProcessesOneJobAtATime", + "My uploads are invisible to other users": "TestUploadChat_AnotherUsersUploadIs404AndNoModelIsCalled", + "I can ask about my recording and get a grounded answer": "TestUploadChatPost_SendsTheOwnersTranscriptWithTheUploadBudget", + "A long recording is not cut to its first third": "TestAnswerMaxCharsOverridesTheServiceCap", + // summarize_new_video.feature "A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered", "A subscribed channel posts a video with no usable transcript": "TestVideoWithNoTranscriptIsSkipped", -- 2.54.0 From 0fc25a24edc55293018643edad5e361ac90f91b4 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:02:05 +0200 Subject: [PATCH 040/299] docs(data-model): uploaded_transcripts, and fix the stale claim that transcripts is RLS'd Adds the private uploaded_transcripts entity and the exact-set isolation rule (ADR-033). Also corrects two lists that still named `transcripts` as an RLS'd table: it has been shared non-RLS since ADR-021 / migration 015, and these lists sit next to the very boundary that matters. Co-Authored-By: Claude Sonnet 5.5 --- docs/data-model.md | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/docs/data-model.md b/docs/data-model.md index f290124..a9e49c2 100644 --- a/docs/data-model.md +++ b/docs/data-model.md @@ -18,7 +18,12 @@ only opaque references to them; the secret material lives in ESO/1Password (ADR- not LLM re-summarization but a rate-gated, reputation-risky caption fetch (ADR-010/014), which is paid per re-fetch regardless of user count — so persisting public caption content once and sharing it strictly beats the coupling it removes. Everything else each user owns is - self-contained; `rls_test.go` proves transcripts is the single exception. + self-contained; `rls_test.go` proves transcripts is the single exception for *content*. + **Uploaded-audio transcripts are the opposite of public caption content** (private, owned by + one user) and therefore live in their own RLS-scoped `uploaded_transcripts` table, never in + `transcripts` (ADR-033). The isolation test now also pins the **exact set** of tables with row + security off (`transcripts`, `user_identities`, `invitations`, `schema_migrations`), so a new + table added without RLS fails CI. - **Secrets by reference only.** Tables hold a `secret_ref` (opaque string/UUID resolved via the `SecretStore` port), never tokens or keys. - **The brain sink is just a delivery target.** No brain-specific tables. Whether a summary @@ -41,6 +46,7 @@ erDiagram VIDEO ||--o| SUMMARY : "has at most one" SUMMARY ||--o{ SINK_DELIVERY : "delivered via" USER ||--o{ CHANNEL_ERROR : "reports unavailable channels" + USER ||--o{ UPLOADED_TRANSCRIPT : "uploads (private, ADR-033)" USER { uuid id PK @@ -136,12 +142,27 @@ erDiagram timestamptz first_seen timestamptz last_seen } + UPLOADED_TRANSCRIPT { + uuid id PK + uuid user_id FK "-> USER, ON DELETE CASCADE, NOT NULL; RLS ENABLE+FORCE (migration 017)" + text title + text status "queued | transcribing | done | failed" + text content "nullable until done; the transcript text" + jsonb segments "nullable until done; [{start,end,text}] in seconds" + text endpoint "which backend served it: iguana/... | berget/... (berget = audio left the homelab)" + float audio_seconds + text error_kind "typed: interrupted | transcription_failed — never a provider string" + timestamptz created_at + timestamptz finished_at + } ``` `SUMMARY_ACTION` has `UNIQUE (user_id, video_id, action)`; `VIDEO_CONNECTION` has `UNIQUE (user_id, provider)` (one connection per provider — reconnect upserts in place). RLS (`ENABLE` + `FORCE`) is on **every solid user-owned table above** — `users`, `videos`, -`transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. +`summaries`, `summary_actions`, `video_connections`, `channel_errors`, `uploaded_transcripts`. +(`transcripts` is deliberately **not** in this list: since ADR-021 it is shared public caption +content with no `user_id`.) `sink_deliveries` is RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally **not** RLS'd (auth plumbing). See the *Isolation invariant* section for the mechanism. @@ -197,6 +218,14 @@ mechanism. the most recent discovery pass. Upserted per scheduler pass (`last_seen` refreshed each run); surfaced on the account page as a warning. Cascades on user deletion. Primary key is `(user_id, channel_id)`. FORCE RLS'd. +- **UPLOADED_TRANSCRIPTS** (migration 017, ADR-033) — the **private** transcript of audio a user + uploaded. Deliberately a separate table from the shared `transcripts`: that one is non-RLS on the + grounds that it holds only public caption content, and a recorded meeting is the opposite. + `user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing → + done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK + constraints). `endpoint` records which backend served it — `berget/...` means the audio left the + homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90 + minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion. - **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the Stage-0 gate query (VISION §Stage 0: "returned and used in ≥2 distinct weeks"). @@ -209,7 +238,10 @@ enforcement dormant); **ADR-012 opened Stage 1 and turned enforcement on in the Enforcement is **Postgres Row-Level Security** (migration `003_rls.up.sql`): - RLS is `ENABLE`d **and** `FORCE`d on every user-owned table — `users`, `videos`, - `transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. + `summaries`, `summary_actions`, `video_connections`, `channel_errors`, and (migration 017) + `uploaded_transcripts`. `transcripts` is **not** one of them: ADR-021 made it shared public + content, and migration 003's original per-user `transcripts` policy was dropped with that + table in migration 015. `FORCE` is load-bearing: the app connects as the table **owner** (`tapir` role), and owners bypass RLS unless forced. - Each policy keys off the per-request GUC `tapir.current_user_id`, set transaction-locally by -- 2.54.0 From 39aeda7c6987fecb890b6732a2b7a8523db100f7 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:02:05 +0200 Subject: [PATCH 041/299] docs(homelab): speech-to-text endpoints, the ingress 413, and the upload knobs Co-Authored-By: Claude Sonnet 5.5 --- docs/homelab-integration.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/homelab-integration.md b/docs/homelab-integration.md index 1b805a6..5104020 100644 --- a/docs/homelab-integration.md +++ b/docs/homelab-integration.md @@ -12,6 +12,20 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-06), - **Off-cluster (host / NodePort):** `http://koala:30401/v1/` — service `litellm-nodeport` (`4000:30401`). - NOTE: the earlier `31234` was **wrong** — `31234` is the `llama-swap` NodePort (`8080:31234`), not LiteLLM. Use `30401` for off-cluster LiteLLM. `confirm` resolved. +- **Speech-to-text (ADR-033):** the same gateway serves `POST /v1/audio/transcriptions` for + `iguana/kb-whisper` (primary, local) and `berget/kb-whisper` (fallback, EU cloud — private audio + leaves the homelab if it is used). **Call it via the NodePort / in-cluster address above, NOT + `https://llm-api.d-ma.be`:** that public ingress (openresty) returns `413` for bodies over + roughly 8–10 MB (~5 minutes of 16 kHz WAV), while the NodePort took 174 MB (90 minutes) in one + request (measured 2026-10-01, `docs/research/audio-length-ceiling-2026-10-01.md`). Send multipart + field `file`, `response_format=verbose_json`, no `Expect: 100-continue`. iguana's whisper-server + can wedge permanently after a Metal OOM (infra#273); never branch on its + `{"error":"failed to process audio"}` string — it has had at least five causes. Tapir's knobs: + `TAPIR_UPLOADS_ENABLED` (off by default), `TAPIR_UPLOAD_MAX_MB` (200), `TAPIR_UPLOAD_MAX_MINUTES` + (90), `TAPIR_TRANSCRIBE_MODEL`, `TAPIR_TRANSCRIBE_FALLBACK_MODEL` (empty disables the cloud + fallback), `TAPIR_TRANSCRIBE_TIMEOUT` (30m), `TAPIR_UPLOAD_TMP_DIR`, `TAPIR_FFMPEG_BIN`, + `TAPIR_FFPROBE_BIN`, `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS` (120000). Needs `ffmpeg` + `ffprobe` + in the runtime image; Tapir's own ingress body limit for a 200 MB upload is unverified (infra). - **Auth key:** treat as a secret via the SecretStore port; do not hardcode in committed code. NOTE: the documented `sk-local-123` returned `401` on `30401/v1/models` (2026-06-02) — it is stale or the gateway now enforces the `LITELLM_MASTER_KEY` (1Password `op://HomeLab/LITELLM_MASTER_KEY`). -- 2.54.0 From 8f16ab671aa4a2c9176a9e29e5d9867d25c85641 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:02:11 +0200 Subject: [PATCH 042/299] docs(claude): audio upload supersedes the STT deferral for user-supplied audio only The 'no audio-download + STT' bullet is narrowed to YouTube (ADR-007 stands); adds the upload packages, the exact-set isolation test and the pending ffmpeg decision to the build state. Co-Authored-By: Claude Sonnet 5.5 --- CLAUDE.md | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index cf4a939..02c5454 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -50,8 +50,11 @@ These caused real mistakes that were caught and corrected; the corrections are l - **No global cross-tenant *video* table** — videos stay per-user (data-model). Transcripts ARE shared since ADR-021 (public caption content, keyed by `(provider, provider_video_id)`, non-RLS) so re-analysis never re-fetches; the *videos* half of cross-tenant dedup stays a Future C concern. -- **No audio-download + speech-to-text in the core path** — captions-first (ADR-007). STT is a - deferred, bounded optional component. +- **No YouTube audio-download + speech-to-text in the core path** — captions-first (ADR-007). + That rejection stands. ADR-033 superseded the STT deferral **for user-supplied audio only**: + an authenticated user may upload a recording and get a *private* transcript (own RLS table, + never the shared `transcripts`). It is off by default (`TAPIR_UPLOADS_ENABLED`) and does not + open the door to scraping audio. - **No public SaaS / sign-up / billing / Google OAuth verification at scale** — Future C, deferred behind the Stage 0 gate (ADR-008). @@ -97,7 +100,7 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes ( - Adapters present under `internal/adapters/`: `youtube` (captions-first `VideoSource`, timedtext/InnerTube acquisition per ADR-010), `summarizer` + `llm` (the copied AI router, now a resilient endpoint chain — local primary → local fallback → external worst-case, - parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–015), + parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–017), `secrets` (file-backed `SecretStore`). The brain HTTP sink (ADR-005) is the remaining optional sink. - Stage 1 is open (ADR-012): multi-user with **DB-enforced** isolation — Postgres RLS `FORCE`d @@ -109,6 +112,12 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes ( (`TestTranscriptsTableIsSharedNotRLS`). The engine reads stored transcripts before any caption fetch (`usecase.resolveTranscript`), so re-analysis — re-summarize, paste-a-URL, onboarding burst — never re-touches YouTube. Per-user summaries/videos stay RLS-scoped. +- Audio upload (ADR-033, migration 017): `internal/uploads` (validation by **probing**, intake, + one-at-a-time in-process worker), `internal/adapters/audio` (ffprobe/ffmpeg), and + `internal/adapters/transcribe` (gateway client + iguana→berget chain). Transcripts are private + in `uploaded_transcripts`; the audio is never retained; chat reuses `chat.Service`. The + isolation test pins the **exact set** of non-RLS tables (`TestRowSecurityOffTablesAreExactlyTheKnownSet`). + The runtime image does not yet contain ffmpeg — that is a pending decision recorded in ADR-033. - `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch watch→summarize), `serve` (the HTMX+Templ web reader/writer under `internal/web`, a new transport over the unchanged engine/ports — ADR-003). `tapir env` prints config. -- 2.54.0 From 4b09dc8f36fdb1c9835204d65c6cfbd25bf19001 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:07:09 +0200 Subject: [PATCH 043/299] test(config): chat defaults for uploads come from measurement, not a guess (red) iguana/gemma4-26b answered correctly at 90,000 chars but SILENTLY dropped the head of a 120,000-char transcript (prompt tokens fell to 16k; 'not in the text'); koala/phi4-mini has an 8,192-token context and answers HTTP 400 at ~13k tokens. So the budget is 90,000 and the upload chat model defaults to gemma4-26b. Red: UploadChatModel is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/config/uploads_test.go | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/internal/config/uploads_test.go b/internal/config/uploads_test.go index c55d77f..3d120bf 100644 --- a/internal/config/uploads_test.go +++ b/internal/config/uploads_test.go @@ -14,7 +14,7 @@ func clearUploadEnv(t *testing.T) { "TAPIR_UPLOADS_ENABLED", "TAPIR_UPLOAD_MAX_MB", "TAPIR_UPLOAD_MAX_MINUTES", "TAPIR_TRANSCRIBE_MODEL", "TAPIR_TRANSCRIBE_FALLBACK_MODEL", "TAPIR_TRANSCRIBE_TIMEOUT", "TAPIR_UPLOAD_TMP_DIR", "TAPIR_FFMPEG_BIN", "TAPIR_FFPROBE_BIN", - "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", + "TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", "TAPIR_UPLOAD_CHAT_MODEL", } { unset(t, k) } @@ -52,8 +52,26 @@ func TestLoad_UploadDefaults(t *testing.T) { if c.FFmpegBin != "ffmpeg" || c.FFprobeBin != "ffprobe" { t.Errorf("bins = %q, %q", c.FFmpegBin, c.FFprobeBin) } - if c.MaxUploadedTranscriptChars != 120000 { - t.Errorf("MaxUploadedTranscriptChars = %d, want 120000", c.MaxUploadedTranscriptChars) + if c.MaxUploadedTranscriptChars != 90000 { + t.Errorf("MaxUploadedTranscriptChars = %d, want 90000: iguana/gemma4-26b answered correctly at 90,000 chars "+ + "but silently dropped the head of a 120,000-char transcript (docs/research/chat-context)", c.MaxUploadedTranscriptChars) + } + if c.UploadChatModel != "iguana/gemma4-26b" { + t.Errorf("UploadChatModel = %q: koala/phi4-mini has an 8,192-token context and answers HTTP 400 on an hour of speech", c.UploadChatModel) + } +} + +func TestLoad_UploadChatModelOverride(t *testing.T) { + clearUploadEnv(t) + t.Setenv("TAPIR_UPLOAD_CHAT_MODEL", "koala/qwen36-35b-a3b") + + c, err := Load() + if err != nil { + t.Fatalf("Load: %v", err) + } + + if c.UploadChatModel != "koala/qwen36-35b-a3b" { + t.Errorf("UploadChatModel = %q", c.UploadChatModel) } } -- 2.54.0 From af66419a44ce8942db4f458ea58fd3d414f37327 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:07:20 +0200 Subject: [PATCH 044/299] feat(config): measured chat budget (90,000) and upload chat model (gemma4-26b) Co-Authored-By: Claude Sonnet 5.5 --- internal/config/uploads.go | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/internal/config/uploads.go b/internal/config/uploads.go index a496b20..ac4942a 100644 --- a/internal/config/uploads.go +++ b/internal/config/uploads.go @@ -33,9 +33,15 @@ type UploadConfig struct { FFprobeBin string // MaxUploadedTranscriptChars bounds the transcript sent to a chat model for an // UPLOADED recording. The 18,000-char default for captions is ~22 minutes of - // speech; an hour of meeting is ~50,000 chars. A guess until measured against - // the models chat offers. + // speech; an hour of meeting is ~50,000 chars. MEASURED 2026-10-01 against the + // real models: iguana/gemma4-26b answers correctly to 90,000 chars but silently + // drops the head of a 120,000-char transcript, so 90,000 is the ceiling of the + // best model, not a round number. MaxUploadedTranscriptChars int + // UploadChatModel is the model chat over an uploaded recording opens with. It is + // NOT the summarizer default: koala/phi4-mini has an 8,192-token context and + // returns HTTP 400 on an hour of speech (~13,000 tokens). + UploadChatModel string } const ( @@ -44,7 +50,8 @@ const ( defaultTranscribeModel = "iguana/kb-whisper" defaultTranscribeFallbackModel = "berget/kb-whisper" defaultTranscribeTimeout = 30 * time.Minute - defaultMaxUploadedChars = 120000 + defaultMaxUploadedChars = 90000 + defaultUploadChatModel = "iguana/gemma4-26b" ) func loadUploadConfig() (UploadConfig, error) { @@ -54,6 +61,7 @@ func loadUploadConfig() (UploadConfig, error) { UploadTmpDir: envOr("TAPIR_UPLOAD_TMP_DIR", os.TempDir()), FFmpegBin: envOr("TAPIR_FFMPEG_BIN", "ffmpeg"), FFprobeBin: envOr("TAPIR_FFPROBE_BIN", "ffprobe"), + UploadChatModel: envOr("TAPIR_UPLOAD_CHAT_MODEL", defaultUploadChatModel), } if v := os.Getenv("TAPIR_UPLOADS_ENABLED"); v != "" { -- 2.54.0 From 4b6ac2a2135fc63f3f98a25d6d03757b0683a8ee Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 17:07:20 +0200 Subject: [PATCH 045/299] test(web): chat over an upload opens with the upload chat model (red) Red: App.UploadChatModel is undefined. Co-Authored-By: Claude Sonnet 5.5 --- internal/web/uploads_chat_handler_test.go | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/internal/web/uploads_chat_handler_test.go b/internal/web/uploads_chat_handler_test.go index bc1ce72..ff6f96f 100644 --- a/internal/web/uploads_chat_handler_test.go +++ b/internal/web/uploads_chat_handler_test.go @@ -165,3 +165,18 @@ func TestUploadStatus_DoneOffersTheChatLinkOnlyWhenChatIsWired(t *testing.T) { id2 := seedUpload(t, userID, "m2", "done", meetingText) assert.NotContains(t, body(t, do(t, appWithout, get("/uploads/"+id2+"/status"))), "/chat") } + +func TestUploadChat_OpensWithTheUploadChatModelNotTheSummarizerDefault(t *testing.T) { + // koala/phi4-mini is first in the offered list (it is the summarizer default) and + // has an 8,192-token context: an hour of speech returns HTTP 400 there. Chat over + // an upload must open with the model configured for it. + app, fc := newUploadChatApp(t) + app.UploadChatModel = "iguana/gemma4-26b" + id := seedUpload(t, userID, "m", "done", meetingText) + + page := body(t, do(t, app, get("/uploads/"+id+"/chat"))) + do(t, app, chatPost("/uploads/"+id+"/chat", url.Values{"question": {"q"}})) + + assert.Contains(t, page, `