docs(adr): ADR-033 upload audio -> private Swedish transcript -> chat (#33 )

Proposed, decider pending. Supersedes ADR-007's STT deferral for user-supplied audio only; the YouTube audio-download rejection stands. Settles #28 decision 1 as a separate RLS-scoped uploaded_transcripts table and records the measured length ceiling, the ingress 413, and the 18k-char chat truncation that #33 assumed did not exist. Two DECISION NEEDED items (ffmpeg in the runtime image; berget fallback for private audio). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
docs(research): measure the audio length ceiling before designing chunking (#33 )
2026-10-01 16:35:04 +02:00 · 2026-10-01 16:33:34 +02:00
2 changed files with 301 additions and 1 deletions
@@ -123,7 +123,9 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path.
 ## ADR-007 — Captions-first; audio-download + speech-to-text deferred
 **Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API
-`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand.
+`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand
 **for YouTube**. STT deferral superseded **for user-supplied audio only** by ADR-033; the
 YouTube audio-download rejection is unchanged.
 **Context.** YouTube's Data API does not expose transcripts. Options: official captions
 (clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey,
@@ -1314,6 +1316,191 @@ roll back. No schema, no structural change.
 ---
 ## ADR-033 — Upload audio, get a private Swedish transcript, discuss it (user-supplied STT)
 **Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #33 (audio-only subset of
 #28). **Supersedes ADR-007's speech-to-text deferral for user-supplied audio only.** The
 YouTube audio-download rejection in ADR-007 **stands untouched**. Builds on ADR-012 (isolation),
 ADR-018 (single replica), ADR-021 (**the constraint**), ADR-022 (endpoint chain), ADR-025
 (honest status), ADR-027 (chat). ADR-014 item 4 ("Whisper reconsideration is a future ADR,
 gated on caption data") is a different question — whether to transcribe YouTube videos with no
 captions — and this ADR does not answer it. Measurements: `docs/research/audio-length-ceiling-2026-10-01.md`.
 **Context.** The maintainer has a concrete need: a Swedish Teams voice-only export, 30–90
 minutes, to upload and then *discuss* (what was decided, what is open, who owes what). Every
 piece exists — a Whisper endpoint on the gateway (infra#274), chat-with-transcript (ADR-027) —
 and nothing connects them.
 ADR-007 deferred STT for three reasons: yt-dlp fragility, ToS-grey scraping, and GPU contention
 with the JEPA PoC. Upload-STT shares **none** of them: the user owns the file, nothing is
 scraped, there is no caption rate gate, and the contention moved from koala to iguana. What it
 does share is landing Whisper in the codebase, which ADR-007 deferred — so a new ADR is required,
 and ADR-017's process note records the one time a guardrail-reversing change shipped without one.
 **The constraint that shapes everything.** ADR-021 made `transcripts` a shared, **non-RLS**
 table on the explicit grounds that it holds only public caption content and nothing
 user-identifying. A recorded internal meeting is the opposite: private third-party content owned
 by one user. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse
 would leak across tenants **and pass CI green**.
 **What was measured, not argued** (research doc above):
 - **No transcription ceiling up to 90 minutes.** One 174 MB request through the LiteLLM NodePort
  took 440 s (~12× realtime). The first failure is the public ingress `llm-api.d-ma.be`
  returning `413` above roughly 8–10 MB (~5 min of WAV) — not whisper, and not a Metal OOM.
  The 8:20 failure recorded in #33 is consistent with that cap.
 - **`response_format=srt` survives LiteLLM, wrapped** in a JSON envelope (`{"text": "<srt>"}`).
 - **The two backends differ.** `iguana/kb-whisper` returns whole-second segments;
  `berget/kb-whisper` returns sub-second times and a different `language` token.
 - **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.**
  #33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does
  not retrieve; it stuffs the transcript into the prompt and cuts the tail.
 **Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on
 iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal
 OOM, deliberately not provoked.
 **Decision.**
 1. **Audio only; transcript is the only thing kept.** An authenticated user uploads an audio
   file; Tapir transcribes it and stores the transcript. The audio itself is deleted after
   transcription (temp file, removed in a `defer`, swept at start-up) — no custody problem to
   solve, and nothing to hand to #31. Video is rejected with a message naming #28. YouTube
   audio-download stays rejected. No public/anonymous upload (ADR-008).
 2. **Private store: a separate RLS-scoped table, `uploaded_transcripts`** (resolves #28's
   decision 1). `user_id NOT NULL REFERENCES users ON DELETE CASCADE`, `ENABLE` + `FORCE ROW LEVEL
   SECURITY`, the same `tapir.current_user_id` policy as every other user-owned table. It carries
   the transcript text and its timestamped segments (re-transcribing 90 minutes costs ~7
   minutes; discarding the timestamps is irreversible), `status`, the `endpoint` that served
   it, `audio_seconds`, and a typed `error_kind`. **Not** a `user_id` column on `transcripts`
   with a partial policy, even though that would reuse more code: ADR-021's exception is
   precisely that *that table holds no private row*, the isolation test asserts it has no
   `user_id`, and a partial policy would turn "single exception" into "single exception, with a
   private lane inside it" — the shape under which a future query drops the policy clause and
   leaks. Two tables keep both boundaries independently assertable.
 3. **The isolation test asserts both boundaries and the set.** (a) User A cannot read, update or
   delete user B's uploaded transcript; no GUC set → deny-all. (b) `transcripts` is still
   non-RLS with no `user_id`. (c) **The set of public-schema tables with row security off is
   asserted exactly** — so a new table added without RLS fails CI instead of passing it. (c) is
   new; the existing test only checks the named tables.
 4. **Validate by probing, reject clearly.** Format and duration cannot be read from a filename
   or MIME type — a Teams "voice-only" recording is typically an `.mp4` container with no video
   stream, which is indistinguishable from video by extension. The upload is probed
   (`ffprobe`): accept any container with an audio stream and **no** video stream; reject video
   with "video is #28"; reject unreadable files. Limits, all configurable, all rejected
   synchronously with a specific message rather than by timeout: **200 MB** upload and
   **90 minutes** duration. 90 minutes is *the longest length measured*, not a ceiling that was
   found; raising it is a measurement, not an edit.
 5. **Transcode server-side, always, to 16 kHz mono PCM WAV** — even when the input already is,
   rather than assuming. whisper-server's `--convert` is rejected (its `--tmp-dir` is created at
   process start and rots under macOS periodic cleanup). The transcoder sits behind a port; the
   tests use a fake. **This puts `ffmpeg` and `ffprobe` in the runtime image**, which is
   distroless-static today. **Provisional: copy static ffmpeg/ffprobe binaries into the existing
   distroless image** (no shell, no package manager retained). This is a heavy new dependency
   and is **the maintainer's call** — see Open items.
 6. **Transcription adapter via the gateway, never the public hostname.** A `Transcriber` port;
   the implementation calls `POST {TAPIR_GATEWAY_URL}/audio/transcriptions` where the gateway
   URL is the in-cluster NodePort (`koala:30401/v1`). Primary `iguana/kb-whisper`, fallback
   `berget/kb-whisper` (immune to iguana contention, which can wedge permanently — infra#273).
   - **Fallback triggers on any primary failure** (transport error, timeout, non-2xx). It never
     branches on the body: `{"error":"failed to process audio"}` has had at least five distinct
     causes.
   - **`response_format=verbose_json`**, normalised to one segment shape: times clamped to the
     audio duration (the final segment can end past it), one language token. Not `srt`: the
     envelope adds a parse step and iguana's segment timing is the same either way.
   - **No chunking in v1.** Nothing failed to 90 minutes (measured). Chunk-and-stitch is the
     answer *if* the cap is raised or smaller requests become necessary — see triggers.
   - **The serving endpoint is recorded per row**, and surfaced, because the two backends
     differ in timestamp granularity: which one served a transcript is a quality fact.
   - **Berget is a cloud endpoint.** On fallback the audio of a private meeting leaves the
     homelab. #33 asks for this fallback; it is configurable (`TAPIR_TRANSCRIBE_FALLBACK_MODEL`,
     empty disables), recorded per row, and called out in Open items rather than defaulted
     silently.
   - Swedish only (`language=sv`). Language detection and routing (#28 decision 2) are not
     touched.
   - `-H 'Expect:'`, multipart field `file` — encoded in the adapter and its tests, not rediscovered.
 7. **Jobs run in-process, one at a time (ADR-018 stands).** Transcription is minutes (~7.5 min
   for 90) but a single goroutine behind a one-slot semaphore: two simultaneous 90-minute jobs on
   iguana were not measured, so concurrency is 1 until they are. No worker, no new deployable,
   no second replica. Honest status per ADR-025: `queued → transcribing → done | failed`, polled
   by the same `/status` pattern, every terminal state a rendered card, never a stuck spinner.
   Audio is not retained, so a restart mid-job **cannot resume**: at start-up any
   `queued`/`transcribing` row is marked `failed` with `error_kind = interrupted`, and the user
   re-uploads. The single-replica assumption gains a second dependent (the semaphore is
   process-local) and is recorded here.
 8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers
   look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism
   as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model)
   carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour
   meeting to its first third, so uploaded transcripts take their own cap,
   `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at
   measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a
   guess until the chat slice measures what the offered models actually accept; it is not
   evidence that they can.
 9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py`
   hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection;
   per-transcript deletion UX (account deletion cascades; see Open items); retention of audio.
 **On the sequencing question #28 asks of itself** — *is this the thing that best serves the
 Stage-0 gate, or the most interesting thing to build?* Recorded answer: it proceeds **ahead of
 #32** (the demand test) as the maintainer's explicit, knowing call (#28, comment of 2026-10-01),
 on a concrete need rather than speculation. It is not a return-nudge (ADR-020/027): it adds
 nothing that prompts a user to come back, so it does not contaminate the unprompted-return signal
 ADR-016 measures. The honest risk is the one ADR-016 names: it is larger than anything ADR-020
 deferred. The scope cut — chat first, no rich analysis, no video — is what keeps it bounded.
 **Consequences.**
 - **Positive.** The user's request is met end to end with existing parts. ADR-021's exception stays
  singular and now provably so. The measured ceiling removes chunking from v1. No audio is stored.
 - **Negative / watch.**
  - Tapir's runtime image gains `ffmpeg`: a larger image and a new CVE surface to patch.
  - iguana becomes load-bearing for a user-facing feature (#28 decision 3; infra#273 is still the
    dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e.
    private audio leaving the homelab, silently unless the per-row endpoint is surfaced.
  - A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it.
  - Chat over a long transcript is bounded by model context, not by anything in Tapir.
  - Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`).
 **Validation / resolution triggers** (each observable by someone other than the author):
 - *Chunking*: build it when the duration cap is raised above 90 min, or when a measured
  request at the current cap fails with whisper-side (not ingress) errors.
 - *Berget-first*: if more than a quarter of uploads in any week are served by the fallback,
  swap the primary/fallback order and revisit infra#273.
 - *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose
  answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap.
 - *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one
  replica, ADR-018 is revisited.
 - *The decision itself*: if the maintainer has not used the feature unprompted within the
  ADR-016 window, it was the interesting thing, not the useful one — remove it.
 **Open items this ADR creates.**
 - **DECISION NEEDED — ffmpeg in the runtime image.** *Options:* (A) static ffmpeg/ffprobe copied
  into distroless **(recommended — the only option that satisfies "accept m4a/mp3/opus/mp4,
  validate duration, reject video"; pure-Go decoders cover wav/mp3 only)**; (B) accept WAV
  only, convert client-side (fails "I just want to upload it"); (C) a transcode sidecar
  (second deployable, ADR-018's reasoning against). *Blocks:* the Dockerfile slice, not the
  tests, which use the port.
 - **DECISION NEEDED — berget fallback for private audio.** *Options:* (A) enabled by default as
  #33 specifies, with the serving endpoint shown **(recommended, matches the issue)**; (B)
  disabled by default, a failed iguana job fails honestly and the user retries. *Blocks:*
  the production default only; the adapter ships either way.
 - Verify Tapir's ingress `proxy-body-size` against the 200 MB cap (`infra`).
 - Per-transcript deletion is not built; account deletion cascades. A private-content store
  without a delete button is a gap worth a ticket, not a silent omission.
 **Reversibility.** Additive: one table (down-migration drops it), one adapter, one upload
 surface, one image change. Removing the upload route removes the feature; `transcripts`, `videos`
 and ADR-021 are untouched.
 ---
 ## Rejected alternatives
 Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
@@ -1338,6 +1525,9 @@ maps to the ADR that settles it.
 | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 |
 | k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 |
 | Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 |
 | Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 |
 | whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 |
 | Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 |
 If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
 not a silent reversal.
@@ -0,0 +1,110 @@
 # Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01)
 Question (#33, "the main technical risk"): at what recording length does Swedish
 transcription through the LiteLLM gateway first fail, and with what failure mode?
 Measured **before** any chunking design, with iguana not holding a large ollama model,
 so the result is the ceiling and not the Metal OOM of infra#273.
 ## Answer
 **No transcription ceiling was found at or below 90 minutes.** A single request carrying
 90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing
 that actually fails is **not whisper**: it is the public gateway ingress, which rejects the
 upload with `413` at roughly 5.5 minutes of WAV.
 | Route | First failure | Failure mode |
 |---|---|---|
 | `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly |
 | `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — |
 | `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — |
 The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a
 30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the
 size-dependent form described, a body-size cap explains it without one. (The OOM
 mechanism in infra#273 is separately real; this run did not provoke it.)
 The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the
 ingress configuration, which lives in `infra`. "About 10 MB" is an inference.
 ## Method
 - **Audio.** No known-good Swedish recording is available that is safe to use: the only real
  one (`IMG_1233.wav`) is the private `/bygge` recording (#31 custody still open) and was
  deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char
  five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`).
  Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe).
  Nothing here is a real transcript; none of it is committed.
 - **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`,
  `response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only
  through `op run` and never on argv.
 - **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident:
  nothing, or `nomic-embed-text` (0.37 GB). Never a large model.
 ## Results (`iguana/kb-whisper`, NodePort)
 | Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars |
 |---|---|---|---|---|---|---|
 | 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — |
 | 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — |
 | 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 |
 | 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 |
 | 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 |
 | 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 |
 | 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 |
 | 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 |
 - Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is
  a minutes-long job, not a request/response one (see ADR-033 on ADR-018).
 - Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are
  monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop
  hallucination appeared across 90 minutes. This is a weak check — the input is a repeated
  loop and synthetic voice — not evidence of accuracy.
 - **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration`
  330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`.
 ## Findings that change the design
 1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the
   gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the
   gateway, never the host" and named the public URL; the gateway is right, that URL is not.
   `TAPIR_GATEWAY_URL` already points at `koala:30401/v1`.
 2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON
   `{"text": "<the SRT>", "usage": …}`, not a bare SRT body — a client that parses the body
   as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside
   an envelope.
 3. **The two backends return different timestamp granularity.** `iguana/kb-whisper`
   segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real
   sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`).
   `verbose_json` and the default JSON were byte-identical on iguana. An adapter that
   falls back between the two must normalise both, and "which endpoint served this" is
   a quality fact, not just a log line.
 4. **The existing chat path cannot answer about most of a long recording.** At
   828 chars/min (this fixture; real speech is plausibly a little denser),
   `TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute
   transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping
   the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of
   a one-hour meeting get an answer from a transcript that does not contain it. #33 says
   chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and
   truncates**, and its ceiling is the model context. See ADR-033.
 5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The
   chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is
   not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces
   smaller requests — see ADR-033's resolution triggers.
 ## What this did not measure
 - **Metal OOM under contention.** Deliberately avoided. A big resident ollama model
  during a long transcription is still the known way to wedge iguana permanently.
 - **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and
  speaker overlap in a Teams export are untested. Speed is probably representative;
  accuracy is not claimed.
 - **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper
  without a server-side transcode is not established here (ADR-033 treats it as needing one).
 - **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were
  not tried.
 - **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not
  size-capped at meeting length for 22 minutes; not proven at 90.
 - **Time-of-day / other tenants.** One afternoon, one run per length.
 Scripts were throwaway shell/Python in a session scratchpad and are not committed; the
 method above reproduces them.