From 38b3f1cf3397cbc4073ed4626a200059b4b4545f Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:35:04 +0200 Subject: [PATCH] docs(adr): ADR-033 upload audio -> private Swedish transcript -> chat (#33) Proposed, decider pending. Supersedes ADR-007's STT deferral for user-supplied audio only; the YouTube audio-download rejection stands. Settles #28 decision 1 as a separate RLS-scoped uploaded_transcripts table and records the measured length ceiling, the ingress 413, and the 18k-char chat truncation that #33 assumed did not exist. Two DECISION NEEDED items (ffmpeg in the runtime image; berget fallback for private audio). Co-Authored-By: Claude Sonnet 5.5 --- DECISIONS.md | 192 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 191 insertions(+), 1 deletion(-) diff --git a/DECISIONS.md b/DECISIONS.md index 44eb2b2..12dc6a9 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -123,7 +123,9 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path. ## ADR-007 — Captions-first; audio-download + speech-to-text deferred **Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API -`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand. +`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand +**for YouTube**. STT deferral superseded **for user-supplied audio only** by ADR-033; the +YouTube audio-download rejection is unchanged. **Context.** YouTube's Data API does not expose transcripts. Options: official captions (clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey, @@ -1314,6 +1316,191 @@ roll back. No schema, no structural change. --- +## ADR-033 — Upload audio, get a private Swedish transcript, discuss it (user-supplied STT) + +**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #33 (audio-only subset of +#28). **Supersedes ADR-007's speech-to-text deferral for user-supplied audio only.** The +YouTube audio-download rejection in ADR-007 **stands untouched**. Builds on ADR-012 (isolation), +ADR-018 (single replica), ADR-021 (**the constraint**), ADR-022 (endpoint chain), ADR-025 +(honest status), ADR-027 (chat). ADR-014 item 4 ("Whisper reconsideration is a future ADR, +gated on caption data") is a different question — whether to transcribe YouTube videos with no +captions — and this ADR does not answer it. Measurements: `docs/research/audio-length-ceiling-2026-10-01.md`. + +**Context.** The maintainer has a concrete need: a Swedish Teams voice-only export, 30–90 +minutes, to upload and then *discuss* (what was decided, what is open, who owes what). Every +piece exists — a Whisper endpoint on the gateway (infra#274), chat-with-transcript (ADR-027) — +and nothing connects them. + +ADR-007 deferred STT for three reasons: yt-dlp fragility, ToS-grey scraping, and GPU contention +with the JEPA PoC. Upload-STT shares **none** of them: the user owns the file, nothing is +scraped, there is no caption rate gate, and the contention moved from koala to iguana. What it +does share is landing Whisper in the codebase, which ADR-007 deferred — so a new ADR is required, +and ADR-017's process note records the one time a guardrail-reversing change shipped without one. + +**The constraint that shapes everything.** ADR-021 made `transcripts` a shared, **non-RLS** +table on the explicit grounds that it holds only public caption content and nothing +user-identifying. A recorded internal meeting is the opposite: private third-party content owned +by one user. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse +would leak across tenants **and pass CI green**. + +**What was measured, not argued** (research doc above): +- **No transcription ceiling up to 90 minutes.** One 174 MB request through the LiteLLM NodePort + took 440 s (~12× realtime). The first failure is the public ingress `llm-api.d-ma.be` + returning `413` above roughly 8–10 MB (~5 min of WAV) — not whisper, and not a Metal OOM. + The 8:20 failure recorded in #33 is consistent with that cap. +- **`response_format=srt` survives LiteLLM, wrapped** in a JSON envelope (`{"text": ""}`). +- **The two backends differ.** `iguana/kb-whisper` returns whole-second segments; + `berget/kb-whisper` returns sub-second times and a different `language` token. +- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.** + #33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does + not retrieve; it stuffs the transcript into the prompt and cuts the tail. + +**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on +iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal +OOM, deliberately not provoked. + +**Decision.** + +1. **Audio only; transcript is the only thing kept.** An authenticated user uploads an audio + file; Tapir transcribes it and stores the transcript. The audio itself is deleted after + transcription (temp file, removed in a `defer`, swept at start-up) — no custody problem to + solve, and nothing to hand to #31. Video is rejected with a message naming #28. YouTube + audio-download stays rejected. No public/anonymous upload (ADR-008). + +2. **Private store: a separate RLS-scoped table, `uploaded_transcripts`** (resolves #28's + decision 1). `user_id NOT NULL REFERENCES users ON DELETE CASCADE`, `ENABLE` + `FORCE ROW LEVEL + SECURITY`, the same `tapir.current_user_id` policy as every other user-owned table. It carries + the transcript text and its timestamped segments (re-transcribing 90 minutes costs ~7 + minutes; discarding the timestamps is irreversible), `status`, the `endpoint` that served + it, `audio_seconds`, and a typed `error_kind`. **Not** a `user_id` column on `transcripts` + with a partial policy, even though that would reuse more code: ADR-021's exception is + precisely that *that table holds no private row*, the isolation test asserts it has no + `user_id`, and a partial policy would turn "single exception" into "single exception, with a + private lane inside it" — the shape under which a future query drops the policy clause and + leaks. Two tables keep both boundaries independently assertable. + +3. **The isolation test asserts both boundaries and the set.** (a) User A cannot read, update or + delete user B's uploaded transcript; no GUC set → deny-all. (b) `transcripts` is still + non-RLS with no `user_id`. (c) **The set of public-schema tables with row security off is + asserted exactly** — so a new table added without RLS fails CI instead of passing it. (c) is + new; the existing test only checks the named tables. + +4. **Validate by probing, reject clearly.** Format and duration cannot be read from a filename + or MIME type — a Teams "voice-only" recording is typically an `.mp4` container with no video + stream, which is indistinguishable from video by extension. The upload is probed + (`ffprobe`): accept any container with an audio stream and **no** video stream; reject video + with "video is #28"; reject unreadable files. Limits, all configurable, all rejected + synchronously with a specific message rather than by timeout: **200 MB** upload and + **90 minutes** duration. 90 minutes is *the longest length measured*, not a ceiling that was + found; raising it is a measurement, not an edit. + +5. **Transcode server-side, always, to 16 kHz mono PCM WAV** — even when the input already is, + rather than assuming. whisper-server's `--convert` is rejected (its `--tmp-dir` is created at + process start and rots under macOS periodic cleanup). The transcoder sits behind a port; the + tests use a fake. **This puts `ffmpeg` and `ffprobe` in the runtime image**, which is + distroless-static today. **Provisional: copy static ffmpeg/ffprobe binaries into the existing + distroless image** (no shell, no package manager retained). This is a heavy new dependency + and is **the maintainer's call** — see Open items. + +6. **Transcription adapter via the gateway, never the public hostname.** A `Transcriber` port; + the implementation calls `POST {TAPIR_GATEWAY_URL}/audio/transcriptions` where the gateway + URL is the in-cluster NodePort (`koala:30401/v1`). Primary `iguana/kb-whisper`, fallback + `berget/kb-whisper` (immune to iguana contention, which can wedge permanently — infra#273). + - **Fallback triggers on any primary failure** (transport error, timeout, non-2xx). It never + branches on the body: `{"error":"failed to process audio"}` has had at least five distinct + causes. + - **`response_format=verbose_json`**, normalised to one segment shape: times clamped to the + audio duration (the final segment can end past it), one language token. Not `srt`: the + envelope adds a parse step and iguana's segment timing is the same either way. + - **No chunking in v1.** Nothing failed to 90 minutes (measured). Chunk-and-stitch is the + answer *if* the cap is raised or smaller requests become necessary — see triggers. + - **The serving endpoint is recorded per row**, and surfaced, because the two backends + differ in timestamp granularity: which one served a transcript is a quality fact. + - **Berget is a cloud endpoint.** On fallback the audio of a private meeting leaves the + homelab. #33 asks for this fallback; it is configurable (`TAPIR_TRANSCRIBE_FALLBACK_MODEL`, + empty disables), recorded per row, and called out in Open items rather than defaulted + silently. + - Swedish only (`language=sv`). Language detection and routing (#28 decision 2) are not + touched. + - `-H 'Expect:'`, multipart field `file` — encoded in the adapter and its tests, not rediscovered. + +7. **Jobs run in-process, one at a time (ADR-018 stands).** Transcription is minutes (~7.5 min + for 90) but a single goroutine behind a one-slot semaphore: two simultaneous 90-minute jobs on + iguana were not measured, so concurrency is 1 until they are. No worker, no new deployable, + no second replica. Honest status per ADR-025: `queued → transcribing → done | failed`, polled + by the same `/status` pattern, every terminal state a rendered card, never a stuck spinner. + Audio is not retained, so a restart mid-job **cannot resume**: at start-up any + `queued`/`transcribing` row is marked `failed` with `error_kind = interrupted`, and the user + re-uploads. The single-replica assumption gains a second dependent (the semaphore is + process-local) and is recorded here. + +8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers + look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism + as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model) + carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour + meeting to its first third, so uploaded transcripts take their own cap, + `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at + measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a + guess until the chat slice measures what the offered models actually accept; it is not + evidence that they can. + +9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py` + hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection; + per-transcript deletion UX (account deletion cascades; see Open items); retention of audio. + +**On the sequencing question #28 asks of itself** — *is this the thing that best serves the +Stage-0 gate, or the most interesting thing to build?* Recorded answer: it proceeds **ahead of +#32** (the demand test) as the maintainer's explicit, knowing call (#28, comment of 2026-10-01), +on a concrete need rather than speculation. It is not a return-nudge (ADR-020/027): it adds +nothing that prompts a user to come back, so it does not contaminate the unprompted-return signal +ADR-016 measures. The honest risk is the one ADR-016 names: it is larger than anything ADR-020 +deferred. The scope cut — chat first, no rich analysis, no video — is what keeps it bounded. + +**Consequences.** +- **Positive.** The user's request is met end to end with existing parts. ADR-021's exception stays + singular and now provably so. The measured ceiling removes chunking from v1. No audio is stored. +- **Negative / watch.** + - Tapir's runtime image gains `ffmpeg`: a larger image and a new CVE surface to patch. + - iguana becomes load-bearing for a user-facing feature (#28 decision 3; infra#273 is still the + dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e. + private audio leaving the homelab, silently unless the per-row endpoint is surfaced. + - A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it. + - Chat over a long transcript is bounded by model context, not by anything in Tapir. + - Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`). + +**Validation / resolution triggers** (each observable by someone other than the author): +- *Chunking*: build it when the duration cap is raised above 90 min, or when a measured + request at the current cap fails with whisper-side (not ingress) errors. +- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback, + swap the primary/fallback order and revisit infra#273. +- *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose + answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap. +- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one + replica, ADR-018 is revisited. +- *The decision itself*: if the maintainer has not used the feature unprompted within the + ADR-016 window, it was the interesting thing, not the useful one — remove it. + +**Open items this ADR creates.** +- **DECISION NEEDED — ffmpeg in the runtime image.** *Options:* (A) static ffmpeg/ffprobe copied + into distroless **(recommended — the only option that satisfies "accept m4a/mp3/opus/mp4, + validate duration, reject video"; pure-Go decoders cover wav/mp3 only)**; (B) accept WAV + only, convert client-side (fails "I just want to upload it"); (C) a transcode sidecar + (second deployable, ADR-018's reasoning against). *Blocks:* the Dockerfile slice, not the + tests, which use the port. +- **DECISION NEEDED — berget fallback for private audio.** *Options:* (A) enabled by default as + #33 specifies, with the serving endpoint shown **(recommended, matches the issue)**; (B) + disabled by default, a failed iguana job fails honestly and the user retries. *Blocks:* + the production default only; the adapter ships either way. +- Verify Tapir's ingress `proxy-body-size` against the 200 MB cap (`infra`). +- Per-transcript deletion is not built; account deletion cascades. A private-content store + without a delete button is a gap worth a ticket, not a silent omission. + +**Reversibility.** Additive: one table (down-migration drops it), one adapter, one upload +surface, one image change. Removing the upload route removes the feature; `transcripts`, `videos` +and ADR-021 are untouched. + +--- + ## Rejected alternatives Approaches considered during the 2026-06-02 planning + grill session and **deliberately not @@ -1338,6 +1525,9 @@ maps to the ADR that settles it. | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 | | k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 | | Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 | +| Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 | +| whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 | +| Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 | If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — not a silent reversal.