Parent: #28. This is the audio-only subset of that epic, scoped to be shippable on its own.
What the user wants
"I have the file on my computer. I just want to upload it somewhere, and it just works."
A Teams voice-only export, Swedish, 30–90 minutes. Upload it, get a transcript, then discuss the contents with the AI — what was decided, what's open, who owes what.
Today the only path is a shell function plus a drag-and-drop into a chat window. Every piece exists; nothing connects them.
Shrinks a lot. Audio is ~30 MB/hour, not 5 GB. No video to keep or discard.
3 — Transcription adapter
Required. Chunking is the real work (see below).
4 — Rich analysis (AVSNITT / ALTERNATIV)
Out of scope — see below.
5 — Synced video playback
Gone entirely. No video.
keyframes
Gone entirely. No visuals.
Cut the rich analysis from v1, deliberately
The stated need is discussion, not a generated document. Tapir already has chat-with-transcript working for YouTube videos — that path takes a transcript and answers questions about it. Point it at an uploaded transcript and the user's actual request is met.
This matters more than it looks: scripts/spike-media/analyze_srt.py:194 hardcodes max_tokens: 6000, and a 4-minute analysis is already ~4,700 output tokens. A 60-minute meeting truncates for every model, and #30 has not run yet. Putting the rich schema in v1 makes the whole feature depend on an unmeasured constant.
Chat-with-transcript has no such ceiling, because it retrieves rather than generates a full document. Ship chat first; the rich analysis is a later slice with #30's numbers attached.
The two things that are actually unknown
1. Length ceiling and chunking
This is the main technical risk and #29 was going to measure it but is unrun. What's known from direct experience:
An 8:20 recording (15 MB WAV) failed repeatedly against iguana:8101; a 30-second slice succeeded.
The root cause was not size. It was a Metal OOM that latches whisper.cpp's backend permanently — ggml_metal_graph_compute: backend is in error state from a previous command buffer failure - recreate the backend to recover. Once it fires, every later request fails at any size until the process restarts.
The trigger is ollama's resident footprint on iguana (qwen3-coder-next at 262k context held 52 GB of 64). See infra#273.
So: measure the real ceiling before designing chunking, and run the measurement when iguana is not holding a large model. Concatenate a known-good WAV ×4, ×8, ×16 and POST each. If it holds at 60 minutes, chunking may not be needed at all for v1.
If chunking is needed, the stitching is a solved shape — offset every timestamp by the chunk start and renumber cues continuously. Strip \r first; whisper.cpp can emit CRLF and a naive index regex silently fails to renumber.
2. Does response_format=srt survive the LiteLLM hop?
infra#274 verified the proxy end to end but with the default JSON response, which returned text plus segments. SRT through the proxy is unverified. Check it; if it doesn't pass through, use verbose_json — the segments carry the timestamps, which is the point.
Use the gateway, never the host
POST https://llm-api.d-ma.be/v1/audio/transcriptions
model=iguana/kb-whisper # Swedish
model=berget/kb-whisper # EU-hosted fallback, immune to iguana contention
Both are registered and verified (infra#274). berget/kb-whisper exists precisely because iguana wedges; the adapter should fall back to it rather than surfacing a failure, and record which endpoint served the request.
Traps already paid for — encode them, don't rediscover
-H 'Expect:' is mandatory. curl adds 100-continue to any body over 1 KB, so every audio upload trips it → curl (52).
The multipart field is file. audio_file returns "Invalid request".
Transcode client-side to strict 16 kHz mono PCM WAV. Do not use whisper-server's --convert — its --tmp-dir is created at process start and macOS periodic cleanup deletes it later, so it rots while the process runs.
{"error":"failed to process audio"} has had at least five distinct causes (missing tmp-dir, ffmpeg off PATH, wrong field name, audio format, Metal OOM). Never branch on that string.
A Teams export's audio is often already 16 kHz mono — the transcode is nearly a no-op, but do it anyway rather than assuming.
Blockers to settle first
ADR before code.#28's decision 1 stands unchanged and unprejudiced: uploaded transcripts must not go in the shared non-RLS transcripts table. ADR-021 made that table non-RLS on the explicit grounds that captions are public content carrying nothing user-identifying; a recorded internal meeting is the opposite. The existing isolation test asserts transcriptsis non-RLS, so a naive reuse would leak private content across tenants and pass CI green.
Repo is public.mathias/tapir is public and a real meeting transcript is private third-party content. #31's custody decision is still open. Nothing in this issue may commit a real transcript — use the synthetic fixtures in scripts/spike-media/fixtures/.
Scope
ADR — supersedes ADR-007's STT deferral for user-supplied media only; leaves the YouTube-audio rejection standing. Settles decision 1.
RLS-scoped store for uploaded transcripts, plus an isolation test asserting both boundaries: uploaded transcripts are user-scoped, and transcripts remains the only non-RLS table.
Authenticated audio upload — format/size/duration validation, rejected clearly rather than by timeout. Audio formats only (m4a, mp3, wav, opus); reject video with a message saying video is #28.
Transcription adapter via LiteLLM, with the berget fallback and the ceiling measured first.
Chat-with-transcript over the uploaded transcript — reuse the existing path, don't fork it.
caveman: me store uploaded transcript scoped to one user, not in the shared public-caption table
Isolation test: user A cannot read user B's uploaded transcript, andtranscripts is still asserted non-RLS
Red-first tests per slice
Real measured ceiling recorded — the length at which transcription first fails, with the failure mode named, or a statement that 60 minutes passes clean
Endpoint actually used is recorded per request (iguana vs berget)
A transcription failure degrades honestly per ADR-025 — never a stuck spinner
End to end: upload a Swedish audio file, get a transcript, ask a question about its contents, get a grounded answer
No real transcript committed to this repo
Refs
#28 (parent), #29 (ceiling — unrun, this issue absorbs the part it needs), #30 (rich analysis — gates the deferred slice), #31 (artifact custody — open)
ADR-007, ADR-012, ADR-021 (the constraint), ADR-018, ADR-025
infra#274 (closed — the live routing and its proof), infra#273 (Metal OOM mechanism)
scripts/spike-media/ — existing toolchain; analyze_srt.py is for the deferred slice, not this one
Parent: #28. **This is the audio-only subset of that epic, scoped to be shippable on its own.**
## What the user wants
> "I have the file on my computer. I just want to upload it somewhere, and it just works."
A Teams voice-only export, Swedish, 30–90 minutes. Upload it, get a transcript, then **discuss the contents with the AI** — what was decided, what's open, who owes what.
Today the only path is a shell function plus a drag-and-drop into a chat window. Every piece exists; nothing connects them.
## Why this is much smaller than #28
Audio-only removes the expensive half:
| #28 slice | Here |
|---|---|
| 1 — ADR + private transcript store | **Required, unchanged.** The highest-risk item. |
| 2 — Upload and custody | Shrinks a lot. Audio is ~30 MB/hour, not 5 GB. No video to keep or discard. |
| 3 — Transcription adapter | Required. Chunking is the real work (see below). |
| 4 — Rich analysis (`AVSNITT` / `ALTERNATIV`) | **Out of scope — see below.** |
| 5 — Synced video playback | Gone entirely. No video. |
| keyframes | Gone entirely. No visuals. |
### Cut the rich analysis from v1, deliberately
The stated need is *discussion*, not a generated document. Tapir already has **chat-with-transcript** working for YouTube videos — that path takes a transcript and answers questions about it. Point it at an uploaded transcript and the user's actual request is met.
This matters more than it looks: `scripts/spike-media/analyze_srt.py:194` hardcodes `max_tokens: 6000`, and a **4-minute** analysis is already ~4,700 output tokens. A 60-minute meeting truncates for every model, and #30 has not run yet. Putting the rich schema in v1 makes the whole feature depend on an unmeasured constant.
Chat-with-transcript has no such ceiling, because it retrieves rather than generates a full document. **Ship chat first; the rich analysis is a later slice with #30's numbers attached.**
## The two things that are actually unknown
### 1. Length ceiling and chunking
This is the main technical risk and #29 was going to measure it but is unrun. What's known from direct experience:
- An 8:20 recording (15 MB WAV) failed repeatedly against `iguana:8101`; a 30-second slice succeeded.
- The root cause was **not** size. It was a Metal OOM that latches whisper.cpp's backend permanently — `ggml_metal_graph_compute: backend is in error state from a previous command buffer failure - recreate the backend to recover`. Once it fires, every later request fails at any size until the process restarts.
- The trigger is ollama's resident footprint on iguana (`qwen3-coder-next` at 262k context held 52 GB of 64). See infra#273.
So: **measure the real ceiling before designing chunking, and run the measurement when iguana is not holding a large model.** Concatenate a known-good WAV ×4, ×8, ×16 and POST each. If it holds at 60 minutes, chunking may not be needed at all for v1.
If chunking is needed, the stitching is a solved shape — offset every timestamp by the chunk start and renumber cues continuously. Strip `\r` first; whisper.cpp can emit CRLF and a naive index regex silently fails to renumber.
### 2. Does `response_format=srt` survive the LiteLLM hop?
infra#274 verified the proxy end to end but with the **default JSON** response, which returned text plus `segments`. SRT through the proxy is unverified. Check it; if it doesn't pass through, use `verbose_json` — the segments carry the timestamps, which is the point.
## Use the gateway, never the host
```
POST https://llm-api.d-ma.be/v1/audio/transcriptions
model=iguana/kb-whisper # Swedish
model=berget/kb-whisper # EU-hosted fallback, immune to iguana contention
```
Both are registered and verified (infra#274). `berget/kb-whisper` exists precisely because iguana wedges; the adapter should fall back to it rather than surfacing a failure, and record which endpoint served the request.
## Traps already paid for — encode them, don't rediscover
- `-H 'Expect:'` is **mandatory**. curl adds `100-continue` to any body over 1 KB, so every audio upload trips it → `curl (52)`.
- The multipart field is `file`. `audio_file` returns "Invalid request".
- Transcode client-side to strict 16 kHz mono PCM WAV. Do **not** use whisper-server's `--convert` — its `--tmp-dir` is created at process start and macOS periodic cleanup deletes it later, so it rots while the process runs.
- `{"error":"failed to process audio"}` has had **at least five distinct causes** (missing tmp-dir, ffmpeg off PATH, wrong field name, audio format, Metal OOM). Never branch on that string.
- A Teams export's audio is often already 16 kHz mono — the transcode is nearly a no-op, but do it anyway rather than assuming.
## Blockers to settle first
**ADR before code.** #28's decision 1 stands unchanged and unprejudiced: uploaded transcripts must **not** go in the shared non-RLS `transcripts` table. ADR-021 made that table non-RLS on the explicit grounds that captions are public content carrying nothing user-identifying; a recorded internal meeting is the opposite. The existing isolation test asserts `transcripts` *is* non-RLS, so a naive reuse would leak private content across tenants **and pass CI green**.
**Repo is public.** `mathias/tapir` is public and a real meeting transcript is private third-party content. #31's custody decision is still open. Nothing in this issue may commit a real transcript — use the synthetic fixtures in `scripts/spike-media/fixtures/`.
## Scope
1. **ADR** — supersedes ADR-007's STT deferral for user-supplied media only; leaves the YouTube-audio rejection standing. Settles decision 1.
2. **RLS-scoped store for uploaded transcripts**, plus an isolation test asserting *both* boundaries: uploaded transcripts are user-scoped, and `transcripts` remains the only non-RLS table.
3. **Authenticated audio upload** — format/size/duration validation, rejected clearly rather than by timeout. Audio formats only (m4a, mp3, wav, opus); reject video with a message saying video is #28.
4. **Transcription adapter** via LiteLLM, with the berget fallback and the ceiling measured first.
5. **Chat-with-transcript over the uploaded transcript** — reuse the existing path, don't fork it.
## Non-goals
- Video, keyframes, synced playback. All #28.
- The rich `AVSNITT`/`ALTERNATIV` analysis. Later slice, gated on #30.
- YouTube audio-download. ADR-007's rejection stands untouched.
- Public or anonymous upload (ADR-008).
## Acceptance
- [ ] ADR accepted before implementation begins
- [ ] `caveman: me store uploaded transcript scoped to one user, not in the shared public-caption table`
- [ ] Isolation test: user A cannot read user B's uploaded transcript, **and** `transcripts` is still asserted non-RLS
- [ ] Red-first tests per slice
- [ ] Real measured ceiling recorded — the length at which transcription first fails, **with the failure mode named**, or a statement that 60 minutes passes clean
- [ ] Endpoint actually used is recorded per request (iguana vs berget)
- [ ] A transcription failure degrades honestly per ADR-025 — never a stuck spinner
- [ ] End to end: upload a Swedish audio file, get a transcript, ask a question about its contents, get a grounded answer
- [ ] No real transcript committed to this repo
## Refs
- #28 (parent), #29 (ceiling — unrun, this issue absorbs the part it needs), #30 (rich analysis — gates the deferred slice), #31 (artifact custody — open)
- ADR-007, ADR-012, ADR-021 (**the constraint**), ADR-018, ADR-025
- infra#274 (closed — the live routing and its proof), infra#273 (Metal OOM mechanism)
- `scripts/spike-media/` — existing toolchain; `analyze_srt.py` is for the deferred slice, not this one
Corrections to this issue, found while building it
Branch feat/33-audio-upload-transcript (PR to follow). Four statements here turned out to be wrong or incomplete; each is recorded in ADR-033 and docs/research/.
"The root cause was not size … a Metal OOM." For the size-dependent failure, it was size. https://llm-api.d-ma.be (openresty) returns 413 for bodies over roughly 8–10 MB (~5 min of WAV); the NodePort koala:30401 took 174 MB / 90 minutes in one request, 440 s, no failure. The 8:20 / 15 MB recording fits the 413. The Metal OOM (infra#273) is separately real; it was not provoked. Tapir must call the NodePort, not the public hostname — this issue named the public URL.
"transcripts remains the only non-RLS table." True of content, false of tables: invitations, user_identities and schema_migrations are non-RLS too. The new isolation test pins the exact set, so a new table added without RLS fails CI.
"Chat-with-transcript … retrieves rather than generates, so has no ceiling." It stuffs the transcript into the prompt and cuts the tail at 18,000 chars (~22 min of speech). Measured: phi4-mini 400s at ~13k tokens; gemma4-26b is right to 90,000 chars and silently loses the head at 120,000. Uploads now open on gemma4-26b with a 90,000-char budget; retrieval is #36.
response_format=srt through LiteLLM is answered: it survives, wrapped in {"text": "<the SRT>"}, and iguana's segments are whole-second. The adapter uses verbose_json instead.
Also: ADR-033 is Proposed, decider pending you — an agent cannot accept it. Two decisions are yours: #34 (ffmpeg in the image; uploads stay off until then) and #35 (berget fallback for private audio). Deferred work: #36, #37, #38.
## Corrections to this issue, found while building it
Branch `feat/33-audio-upload-transcript` (PR to follow). Four statements here turned out to be wrong or incomplete; each is recorded in ADR-033 and `docs/research/`.
1. **"The root cause was not size … a Metal OOM."** For the size-dependent failure, it *was* size. `https://llm-api.d-ma.be` (openresty) returns `413` for bodies over roughly 8–10 MB (~5 min of WAV); the NodePort `koala:30401` took **174 MB / 90 minutes in one request, 440 s, no failure.** The 8:20 / 15 MB recording fits the 413. The Metal OOM (infra#273) is separately real; it was not provoked. **Tapir must call the NodePort, not the public hostname** — this issue named the public URL.
2. **"`transcripts` remains the only non-RLS table."** True of *content*, false of tables: `invitations`, `user_identities` and `schema_migrations` are non-RLS too. The new isolation test pins the exact set, so a new table added without RLS fails CI.
3. **"Chat-with-transcript … retrieves rather than generates, so has no ceiling."** It stuffs the transcript into the prompt and cuts the tail at 18,000 chars (~22 min of speech). Measured: `phi4-mini` 400s at ~13k tokens; `gemma4-26b` is right to 90,000 chars and **silently loses the head at 120,000**. Uploads now open on gemma4-26b with a 90,000-char budget; retrieval is #36.
4. **`response_format=srt` through LiteLLM** is answered: it survives, **wrapped** in `{"text": "<the SRT>"}`, and iguana's segments are whole-second. The adapter uses `verbose_json` instead.
Also: ADR-033 is **Proposed, decider pending you** — an agent cannot accept it. Two decisions are yours: #34 (ffmpeg in the image; uploads stay off until then) and #35 (berget fallback for private audio). Deferred work: #36, #37, #38.
---
_Created via git-mcp on behalf of @mathias_
Scope change: a video track is accepted, the audio is used, the user is told
This issue's scope said "Audio formats only … reject video with a message saying video is #28." The first real recording (a Teams meeting, camera off) was rejected as video: a Teams recording is an .mp4 with a video track whether or not the camera was on, so the rule rejected the primary use case. Decided by the maintainer 2026-10-01.
Now (ADR-033 decisions 1 and 4, amended in place, still Proposed; migration 018; PR #39):
A file with an audio stream is accepted whether or not it also has a video stream. Only the audio is extracted (-vn); the video is discarded and nothing from it is kept.
The row records had_video, and the page says "This file contained a video track. Only its audio was used; the video was discarded" in every state (queued, transcribing, done, failed).
Still rejected: no audio stream (e.g. a screen recording with no mic), unreadable, too long, too large.
Unchanged and still #28: video playback, keyframes, and analysis of what is shown.
Verified with a synthetic camera-off-style H.264+AAC .mp4 through the real pipeline (accepted, had_video=true, transcribed, notice shown, chat answered). Not yet verified on the real file.
## Scope change: a video track is accepted, the audio is used, the user is told
This issue's scope said "Audio formats only … reject video with a message saying video is #28." The first real recording (a Teams meeting, camera off) was rejected as video: a Teams recording is an `.mp4` with a video track whether or not the camera was on, so the rule rejected the primary use case. Decided by the maintainer 2026-10-01.
Now (ADR-033 decisions 1 and 4, amended in place, still Proposed; migration 018; PR #39):
- A file with an audio stream is accepted **whether or not it also has a video stream**. Only the audio is extracted (`-vn`); the video is discarded and nothing from it is kept.
- The row records `had_video`, and the page says "This file contained a video track. Only its audio was used; the video was discarded" in every state (queued, transcribing, done, failed).
- Still rejected: no audio stream (e.g. a screen recording with no mic), unreadable, too long, too large.
- Unchanged and still #28: video playback, keyframes, and analysis of what is shown.
Verified with a synthetic camera-off-style H.264+AAC `.mp4` through the real pipeline (accepted, `had_video=true`, transcribed, notice shown, chat answered). Not yet verified on the real file.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Parent: #28. This is the audio-only subset of that epic, scoped to be shippable on its own.
What the user wants
A Teams voice-only export, Swedish, 30–90 minutes. Upload it, get a transcript, then discuss the contents with the AI — what was decided, what's open, who owes what.
Today the only path is a shell function plus a drag-and-drop into a chat window. Every piece exists; nothing connects them.
Why this is much smaller than #28
Audio-only removes the expensive half:
AVSNITT/ALTERNATIV)Cut the rich analysis from v1, deliberately
The stated need is discussion, not a generated document. Tapir already has chat-with-transcript working for YouTube videos — that path takes a transcript and answers questions about it. Point it at an uploaded transcript and the user's actual request is met.
This matters more than it looks:
scripts/spike-media/analyze_srt.py:194hardcodesmax_tokens: 6000, and a 4-minute analysis is already ~4,700 output tokens. A 60-minute meeting truncates for every model, and #30 has not run yet. Putting the rich schema in v1 makes the whole feature depend on an unmeasured constant.Chat-with-transcript has no such ceiling, because it retrieves rather than generates a full document. Ship chat first; the rich analysis is a later slice with #30's numbers attached.
The two things that are actually unknown
1. Length ceiling and chunking
This is the main technical risk and #29 was going to measure it but is unrun. What's known from direct experience:
iguana:8101; a 30-second slice succeeded.ggml_metal_graph_compute: backend is in error state from a previous command buffer failure - recreate the backend to recover. Once it fires, every later request fails at any size until the process restarts.qwen3-coder-nextat 262k context held 52 GB of 64). See infra#273.So: measure the real ceiling before designing chunking, and run the measurement when iguana is not holding a large model. Concatenate a known-good WAV ×4, ×8, ×16 and POST each. If it holds at 60 minutes, chunking may not be needed at all for v1.
If chunking is needed, the stitching is a solved shape — offset every timestamp by the chunk start and renumber cues continuously. Strip
\rfirst; whisper.cpp can emit CRLF and a naive index regex silently fails to renumber.2. Does
response_format=srtsurvive the LiteLLM hop?infra#274 verified the proxy end to end but with the default JSON response, which returned text plus
segments. SRT through the proxy is unverified. Check it; if it doesn't pass through, useverbose_json— the segments carry the timestamps, which is the point.Use the gateway, never the host
Both are registered and verified (infra#274).
berget/kb-whisperexists precisely because iguana wedges; the adapter should fall back to it rather than surfacing a failure, and record which endpoint served the request.Traps already paid for — encode them, don't rediscover
-H 'Expect:'is mandatory. curl adds100-continueto any body over 1 KB, so every audio upload trips it →curl (52).file.audio_filereturns "Invalid request".--convert— its--tmp-diris created at process start and macOS periodic cleanup deletes it later, so it rots while the process runs.{"error":"failed to process audio"}has had at least five distinct causes (missing tmp-dir, ffmpeg off PATH, wrong field name, audio format, Metal OOM). Never branch on that string.Blockers to settle first
ADR before code. #28's decision 1 stands unchanged and unprejudiced: uploaded transcripts must not go in the shared non-RLS
transcriptstable. ADR-021 made that table non-RLS on the explicit grounds that captions are public content carrying nothing user-identifying; a recorded internal meeting is the opposite. The existing isolation test assertstranscriptsis non-RLS, so a naive reuse would leak private content across tenants and pass CI green.Repo is public.
mathias/tapiris public and a real meeting transcript is private third-party content. #31's custody decision is still open. Nothing in this issue may commit a real transcript — use the synthetic fixtures inscripts/spike-media/fixtures/.Scope
transcriptsremains the only non-RLS table.Non-goals
AVSNITT/ALTERNATIVanalysis. Later slice, gated on #30.Acceptance
caveman: me store uploaded transcript scoped to one user, not in the shared public-caption tabletranscriptsis still asserted non-RLSRefs
scripts/spike-media/— existing toolchain;analyze_srt.pyis for the deferred slice, not this oneCorrections to this issue, found while building it
Branch
feat/33-audio-upload-transcript(PR to follow). Four statements here turned out to be wrong or incomplete; each is recorded in ADR-033 anddocs/research/.https://llm-api.d-ma.be(openresty) returns413for bodies over roughly 8–10 MB (~5 min of WAV); the NodePortkoala:30401took 174 MB / 90 minutes in one request, 440 s, no failure. The 8:20 / 15 MB recording fits the 413. The Metal OOM (infra#273) is separately real; it was not provoked. Tapir must call the NodePort, not the public hostname — this issue named the public URL.transcriptsremains the only non-RLS table." True of content, false of tables:invitations,user_identitiesandschema_migrationsare non-RLS too. The new isolation test pins the exact set, so a new table added without RLS fails CI.phi4-mini400s at ~13k tokens;gemma4-26bis right to 90,000 chars and silently loses the head at 120,000. Uploads now open on gemma4-26b with a 90,000-char budget; retrieval is #36.response_format=srtthrough LiteLLM is answered: it survives, wrapped in{"text": "<the SRT>"}, and iguana's segments are whole-second. The adapter usesverbose_jsoninstead.Also: ADR-033 is Proposed, decider pending you — an agent cannot accept it. Two decisions are yours: #34 (ffmpeg in the image; uploads stay off until then) and #35 (berget fallback for private audio). Deferred work: #36, #37, #38.
Created via git-mcp on behalf of @mathias
Scope change: a video track is accepted, the audio is used, the user is told
This issue's scope said "Audio formats only … reject video with a message saying video is #28." The first real recording (a Teams meeting, camera off) was rejected as video: a Teams recording is an
.mp4with a video track whether or not the camera was on, so the rule rejected the primary use case. Decided by the maintainer 2026-10-01.Now (ADR-033 decisions 1 and 4, amended in place, still Proposed; migration 018; PR #39):
-vn); the video is discarded and nothing from it is kept.had_video, and the page says "This file contained a video track. Only its audio was used; the video was discarded" in every state (queued, transcribing, done, failed).Verified with a synthetic camera-off-style H.264+AAC
.mp4through the real pipeline (accepted,had_video=true, transcribed, notice shown, chat answered). Not yet verified on the real file.Created via git-mcp on behalf of @mathias