From feb108def39ce9af4d851705e5d7d25f391d4bc3 Mon Sep 17 00:00:00 2001 From: Mathias Date: Thu, 1 Oct 2026 16:33:34 +0200 Subject: [PATCH] docs(research): measure the audio length ceiling before designing chunking (#33) No transcription ceiling found up to 90 min: one 174 MB request through the LiteLLM NodePort transcribed in 440 s. The first real failure is the public gateway ingress rejecting bodies over ~8-10 MB with 413, not whisper or a Metal OOM. Also records that srt arrives wrapped in a JSON envelope, that the two backends differ in timestamp granularity, and that the existing chat path truncates at 18k chars (~22 min of speech). Synthetic Swedish TTS fixture only; no real transcript. Co-Authored-By: Claude Sonnet 5.5 --- .../audio-length-ceiling-2026-10-01.md | 110 ++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 docs/research/audio-length-ceiling-2026-10-01.md diff --git a/docs/research/audio-length-ceiling-2026-10-01.md b/docs/research/audio-length-ceiling-2026-10-01.md new file mode 100644 index 0000000..e9ed2ba --- /dev/null +++ b/docs/research/audio-length-ceiling-2026-10-01.md @@ -0,0 +1,110 @@ +# Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01) + +Question (#33, "the main technical risk"): at what recording length does Swedish +transcription through the LiteLLM gateway first fail, and with what failure mode? +Measured **before** any chunking design, with iguana not holding a large ollama model, +so the result is the ceiling and not the Metal OOM of infra#273. + +## Answer + +**No transcription ceiling was found at or below 90 minutes.** A single request carrying +90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing +that actually fails is **not whisper**: it is the public gateway ingress, which rejects the +upload with `413` at roughly 5.5 minutes of WAV. + +| Route | First failure | Failure mode | +|---|---|---| +| `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly | +| `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — | +| `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — | + +The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a +30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the +size-dependent form described, a body-size cap explains it without one. (The OOM +mechanism in infra#273 is separately real; this run did not provoke it.) + +The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the +ingress configuration, which lives in `infra`. "About 10 MB" is an inference. + +## Method + +- **Audio.** No known-good Swedish recording is available that is safe to use: the only real + one (`IMG_1233.wav`) is the private `/bygge` recording (#31 custody still open) and was + deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char + five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`). + Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe). + Nothing here is a real transcript; none of it is committed. +- **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`, + `response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only + through `op run` and never on argv. +- **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident: + nothing, or `nomic-embed-text` (0.37 GB). Never a large model. + +## Results (`iguana/kb-whisper`, NodePort) + +| Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars | +|---|---|---|---|---|---|---| +| 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — | +| 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — | +| 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 | +| 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 | +| 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 | +| 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 | +| 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 | +| 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 | + +- Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is + a minutes-long job, not a request/response one (see ADR-033 on ADR-018). +- Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are + monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop + hallucination appeared across 90 minutes. This is a weak check — the input is a repeated + loop and synthetic voice — not evidence of accuracy. +- **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration` + 330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`. + +## Findings that change the design + +1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the + gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the + gateway, never the host" and named the public URL; the gateway is right, that URL is not. + `TAPIR_GATEWAY_URL` already points at `koala:30401/v1`. +2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON + `{"text": "", "usage": …}`, not a bare SRT body — a client that parses the body + as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside + an envelope. +3. **The two backends return different timestamp granularity.** `iguana/kb-whisper` + segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real + sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`). + `verbose_json` and the default JSON were byte-identical on iguana. An adapter that + falls back between the two must normalise both, and "which endpoint served this" is + a quality fact, not just a log line. +4. **The existing chat path cannot answer about most of a long recording.** At + 828 chars/min (this fixture; real speech is plausibly a little denser), + `TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute + transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping + the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of + a one-hour meeting get an answer from a transcript that does not contain it. #33 says + chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and + truncates**, and its ceiling is the model context. See ADR-033. +5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The + chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is + not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces + smaller requests — see ADR-033's resolution triggers. + +## What this did not measure + +- **Metal OOM under contention.** Deliberately avoided. A big resident ollama model + during a long transcription is still the known way to wedge iguana permanently. +- **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and + speaker overlap in a Teams export are untested. Speed is probably representative; + accuracy is not claimed. +- **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper + without a server-side transcode is not established here (ADR-033 treats it as needing one). +- **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were + not tried. +- **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not + size-capped at meeting length for 22 minutes; not proven at 90. +- **Time-of-day / other tenants.** One afternoon, one run per length. + +Scripts were throwaway shell/Python in a session scratchpad and are not committed; the +method above reproduces them.