tapir/docs/research/audio-length-ceiling-2026-10-01.md

# Audio length ceiling — measured before designing chunking (tapir#33, 2026-10-01)

Question (#33, "the main technical risk"): at what recording length does Swedish
transcription through the LiteLLM gateway first fail, and with what failure mode?
Measured **before** any chunking design, with iguana not holding a large ollama model,
so the result is the ceiling and not the Metal OOM of infra#273.

## Answer

**No transcription ceiling was found at or below 90 minutes.** A single request carrying
90 minutes of 16 kHz mono PCM (174 MB) transcribed in one pass in 440 s. The first thing
that actually fails is **not whisper**: it is the public gateway ingress, which rejects the
upload with `413` at roughly 5.5 minutes of WAV.

| Route | First failure | Failure mode |
|---|---|---|
| `https://llm-api.d-ma.be` (openresty ingress) | between 8.0 MB and 10.5 MB body (~4–5.5 min of WAV) | `413 Request Entity Too Large`, HTML body from `openresty`, returned instantly |
| `http://koala:30401` (LiteLLM NodePort) → `iguana/kb-whisper` | **none up to 90 min / 174 MB** | — |
| `http://koala:30401` → `berget/kb-whisper` | **none up to 22 min / 42 MB** (largest tried) | — |

The ingress limit is the reason the 8:20 / 15 MB recording in #33 "failed repeatedly" and a
30-second slice succeeded. That failure was attributed to a Metal OOM; at least in the
size-dependent form described, a body-size cap explains it without one. (The OOM
mechanism in infra#273 is separately real; this run did not provoke it.)

The exact ingress cap was bracketed (8.0 MB passed, 10.5 MB failed), not read from the
ingress configuration, which lives in `infra`. "About 10 MB" is an inference.

## Method

- **Audio.** No known-good Swedish recording is available that is safe to use: the only real
  one is a private recording (#31 custody still open) and was
  deliberately not touched. Instead: macOS `say -v Alva` (Swedish) on iguana, a 1,171-char
  five-paragraph meeting script → 82.5 s of 16 kHz mono PCM WAV (`ffmpeg -ar 16000 -ac 1`).
  Concatenated ×4, ×8, ×16, ×32, ×44, ×66 with Python `wave` (the issue's own recipe).
  Nothing here is a real transcript; none of it is committed.
- **Request.** `POST /v1/audio/transcriptions`, multipart `file=`, `model=iguana/kb-whisper`,
  `response_format=verbose_json`, `-H 'Expect:'` (per #33's traps), master key supplied only
  through `op run` and never on argv.
- **Confound control.** `ollama /api/ps` on iguana was read before every step. Resident:
  nothing, or `nomic-embed-text` (0.37 GB). Never a large model.

## Results (`iguana/kb-whisper`, NodePort)

| Audio | WAV size | HTTP | Wall | Realtime factor | Segments | Chars |
|---|---|---|---|---|---|---|
| 0:30 | 0.96 MB | 200 | 2.8 s | 11× | — | — |
| 2:30 / 3:20 / 4:10 | 4.8 / 6.4 / 8.0 MB | 200 (ingress) | 12.9 / 16.0 / 19.9 s | 12× | — | — |
| 5:30 (×4) | 10.5 MB | **413 ingress**, 200 NodePort in 27 s | 27 s | 12× | 121 | 4,557 |
| 11:00 (×8) | 21.1 MB | 200 | 53 s | 12× | 241 | 9,094 |
| 22:00 (×16) | 42.3 MB | 200 | 105 s | 13× | 482 | 18,201 |
| 44:01 (×32) | 84.5 MB | 200 | 209 s | 13× | 966 | 36,438 |
| 60:32 (×44) | 116.2 MB | 200 | 291 s | 12× | 1,332 | 50,145 |
| 90:48 (×66) | 174.3 MB | 200 | 440 s | 12× | 2,005 | 75,165 |

- Throughput is flat at ~12× realtime. 90 minutes costs ~7.5 minutes of wall time. This is
  a minutes-long job, not a request/response one (see ADR-033 on ADR-018).
- Segments per source repetition stay at 30.1–30.4 from ×4 to ×66, and segment times are
  monotonic with `last_end` within 0.2 s of `duration`. No drift and no repetition-loop
  hallucination appeared across 90 minutes. This is a weak check — the input is a repeated
  loop and synthetic voice — not evidence of accuracy.
- **Only `iguana/kb-whisper`'s `x4` run reported `last_end` 360.0 s against `duration`
  330.2 s.** Whisper's final segment can end past the audio; clamp to `duration`.

## Findings that change the design

1. **Use the NodePort route, not the public hostname.** Tapir runs in-cluster beside the
   gateway; `llm-api.d-ma.be` is the external door and has a body cap. #33 said "use the
   gateway, never the host" and named the public URL; the gateway is right, that URL is not.
   `TAPIR_GATEWAY_URL` already points at `koala:30401/v1`.
2. **`response_format=srt` survives the LiteLLM hop, but wrapped.** The response is JSON
   `{"text": "<the SRT>", "usage": …}`, not a bare SRT body — a client that parses the body
   as SRT gets a parse error on `{`. #33 question 2 is answered: it passes through, inside
   an envelope.
3. **The two backends return different timestamp granularity.** `iguana/kb-whisper`
   segments are **whole seconds** (`0→1`, `1→4`, `4→7`); `berget/kb-whisper` returns real
   sub-second times (`0.051→1.192`) and a different `language` token (`swedish` vs `sv`).
   `verbose_json` and the default JSON were byte-identical on iguana. An adapter that
   falls back between the two must normalise both, and "which endpoint served this" is
   a quality fact, not just a log line.
4. **The existing chat path cannot answer about most of a long recording.** At
   828 chars/min (this fixture; real speech is plausibly a little denser),
   `TAPIR_MAX_TRANSCRIPT_CHARS`'s default of **18,000** is ~21.7 minutes. A 60-minute
   transcript is 50 k chars, a 90-minute one 75 k. `chat.Service` truncates by keeping
   the head (`internal/adapters/chat/chat.go`), so questions about the last two-thirds of
   a one-hour meeting get an answer from a transcript that does not contain it. #33 says
   chat "retrieves rather than generates" and therefore has no ceiling; it **stuffs and
   truncates**, and its ceiling is the model context. See ADR-033.
5. **Chunking is not needed for v1 on length.** Nothing failed up to 90 minutes. The
   chunking-and-stitching recipe in #33 (offset timestamps, renumber cues, strip `\r`) is
   not built. It stays the answer *if* a longer cap is wanted or the OOM mechanism forces
   smaller requests — see ADR-033's resolution triggers.

## What this did not measure

- **Metal OOM under contention.** Deliberately avoided. A big resident ollama model
  during a long transcription is still the known way to wedge iguana permanently.
- **Real speech.** TTS is clean, single-voice, no crosstalk or room noise. WER and
  speaker overlap in a Teams export are untested. Speed is probably representative;
  accuracy is not claimed.
- **Compressed input.** Only WAV was sent. Whether `m4a`/`mp3`/`opus` can reach whisper
  without a server-side transcode is not established here (ADR-033 treats it as needing one).
- **Concurrency.** One request at a time. Two simultaneous 90-minute jobs on iguana were
  not tried.
- **`berget/kb-whisper` beyond 22 min / 42 MB.** Enough to know the fallback is not
  size-capped at meeting length for 22 minutes; not proven at 90.
- **Time-of-day / other tenants.** One afternoon, one run per length.

Scripts were throwaway shell/Python in a session scratchpad and are not committed; the
method above reproduces them.