Tapir summarizes videos it can reach by URL. It cannot take a video the user has — a phone recording of a site walkthrough, a recorded meeting, a lecture capture. Those are often the videos most worth summarizing, because nobody else will ever caption them.
A working prototype exists at tapir.d-ma.be/bygge: a Swedish construction-site walkthrough (iPhone, 4 min), transcribed with KB-Whisper and analyzed into timestamped sections, decision points, and alternatives, played back in sync with the video. It is entirely static — the transcript and analysis were produced by hand off-cluster and committed as fixtures. This epic is about making that flow real: a logged-in user uploads any video and gets a transcript plus key takeaways back.
Transcription routes by language: KB-Whisper (KBLab/kb-whisper-large) for Swedish, vanilla Whisper for everything else. KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish, which is the difference between a usable transcript and one that needs hand-correction.
⚠️ The constraint that shapes everything: uploaded transcripts are NOT public content
Read this before designing anything.
ADR-021 made transcripts a shared, non-RLS table keyed by (provider, provider_video_id). The justification is explicit and narrow:
It holds only public caption content + the video's public id — nothing user-identifying — and is therefore NOT RLS-scoped … This is the deliberate, single exception to the ADR-012 isolation boundary, and the only one.
An uploaded video's transcript breaks every clause of that. It is private content, owned by one user, that no other user has any right to see. The /bygge prototype is a live example: a named colleague discussing a client's building project.
So uploaded transcripts must NOT go in the transcripts table. They need their own RLS-scoped store — and the existing isolation test will not catch a mistake here, because it asserts transcriptsis non-RLS. A naive "reuse the transcript store" implementation would leak private content across tenants and pass CI green.
This is the single highest-risk item in the epic. Treat it as the first design decision, not an implementation detail.
Prerequisite: an ADR, before any code
ADR-007 deferred speech-to-text; ADR-014 item 4 says the reconsideration is "a future ADR, gated on this data." This epic needs that ADR written and accepted first.
It is not a straight reversal, and the ADR should say so. ADR-007 rejected audio-download from YouTube — yt-dlp fragility, ToS-grey territory, GPU contention. Upload-STT shares none of that: the user owns the file, there is no scraping, no caption rate gate, no ToS surface. What it does share is landing Whisper in the codebase, which ADR-007 deferred. So the ADR supersedes ADR-007's STT deferral for user-supplied media only, and leaves the YouTube-audio rejection standing.
ADR-017's process note records the one time a guardrail-reversing change shipped without an ADR, and why that was bad. Don't repeat it here.
Open decisions the ADR must settle
1. Private transcript storage
Separate RLS-scoped table (uploaded_transcripts or similar), or a user_id column on transcripts with a partial RLS policy? The separate table is cleaner — it keeps ADR-021's exception genuinely singular and unambiguous, and the isolation test can assert both shapes independently. Recommend separate; decide explicitly.
2. Language detection and model routing
Chicken-and-egg: choosing between KB-Whisper and vanilla Whisper requires knowing the language, which normally comes from running Whisper. Options:
Vanilla Whisper's language detection on the first ~30s, then route and transcribe fully. One extra short inference, fully automatic.
Ask the uploader. Zero inference cost, one more form field, and users get it wrong.
Default Swedish (this is a Swedish-first deployment), with a manual override.
Recommend detect-then-route, with an override. State what happens on a mixed-language recording — pick a primary, don't try to switch mid-file.
3. Where inference runs, and what it collides with
KB-Whisper runs on iguana (M2 Ultra) — the same host ADR-022 uses for the gemma4-26b summarizer fallback. A long transcription job and a summarizer fallback now contend for one machine. ADR-007's original GPU-contention objection was about koala; it has moved, not disappeared.
infra#273 (harden iguana for unattended service hosting) becomes a hard dependency. Today whisper-server is started by hand, does not survive reboot, and has no monitoring. That was acceptable when transcription was an occasional manual errand. A user-facing upload feature makes iguana load-bearing, which is exactly the condition #273 says should trigger a revisit.
infra#274 (can LiteLLM front KB-Whisper) is also relevant: if LiteLLM can proxy /v1/audio/transcriptions, Tapir gets routing, auth, and token accounting through the gateway it already uses, and the ADR-022 chain pattern extends naturally to STT endpoints. If it can't, Tapir needs a direct client to a second endpoint type. Resolve#274 before designing the adapter.
4. Long-running jobs vs the single-replica in-process scheduler
ADR-018 put discovery in-process and recorded a load-bearing single-replica assumption. Transcription is minutes, not seconds, and is CPU/GPU-bound elsewhere. Decide whether upload jobs run in the same process (simple, consistent with ADR-018, but a wedged transcription degrades the reading UI — the coupling ADR-018 knowingly accepted at a much smaller cost) or move to a worker. Do not silently multiply replicas; that breaks ADR-018.
5. File custody and retention
A 4-minute iPhone clip is ~330 MB at 11 Mbps; an hour of the same is ~5 GB. Settle:
Ingress body-size limits and upload mechanism (direct POST vs chunked/resumable)
Where bytes live during processing, and whether the video is kept after transcription. Keeping it enables the /bygge-style synced playback; discarding it is far better for storage and for GDPR posture.
Explicit format/size/duration limits, rejected clearly rather than by timeout.
Server-side transcode to 16 kHz mono WAV before inference. Do not rely on whisper-server's --convert; its --tmp-dir must exist on the host and its absence produced a silent failure during the prototype work.
6. The analysis schema
The prototype produces something richer than summary/highlights/takeaways: timestamped sections (AVSNITT), decision points, and alternatives with a stated leaning (ALTERNATIV … Lutar åt: inne). That richer shape is what makes it useful for a walkthrough — but ADR-022 exists because small models already emit malformed JSON on the simple schema. A more complex schema will fail parsing more often.
Decide: does upload analysis reuse the summary schema, or get its own? If its own, the tolerant-parse strategy (ADR-022 §3) needs extending, and the chain's endpoint ordering may need a stronger primary. Relates to #22 (frontier models via LiteLLM) — a structured-output-capable model would make this materially more reliable.
Scope
Ordered so each slice is independently shippable.
Slice 1 — ADR + private transcript store
The ADR above, plus the RLS-scoped store for uploaded transcripts and an isolation test that asserts both boundaries: uploaded transcripts are user-scoped, and transcripts remains the only non-RLS table.
Slice 2 — Upload and custody
Authenticated upload endpoint, size/format validation, storage, a videos row for the upload. Note the data-model wrinkle: videos is keyed by (provider, provider_video_id). An upload has neither — it needs a synthetic provider (upload) and a generated id. Confirm this doesn't break ADR-021's key assumptions or the dedup logic.
Slice 3 — Transcription adapter with language routing
Transcode → detect → route to KB-Whisper or vanilla → store. Chunking for long files, with overlap and SRT stitching. Timestamps are the point, so response_format=srt (or equivalent) rather than plain text.
Slice 4 — Analysis
Structured analysis over the stored transcript, reusing the ADR-022 chain. Schema per decision 6.
Slice 5 — Playback surface
Video plus timestamped analysis, synced. The /bygge page is the working reference for what good looks like; generalize it rather than redesigning.
Non-goals
Audio-download from YouTube. ADR-007's rejection stands untouched. This epic is user-supplied media only, and the ADR must say so explicitly or the next reader will assume the door is open.
Public/anonymous upload. Authenticated users only (ADR-008 — no public SaaS machinery).
Cross-user dedup of uploaded content. Two users uploading the same file get two private transcripts. That is correct, not wasteful.
Real-time/streaming transcription.
Editing or correcting transcripts in-UI.
Honest question about sequencing
The Stage-0 gate (ADR-016) measures whether someone returns unprompted over 3–4 weeks, and ADR-020 deferred search, facets, and read/unread as premature until that loop is validated. This epic is substantially larger than any of those.
The counter-argument is real: /bygge exists because the maintainer had a video he actually wanted analyzed, which is genuine pull rather than speculation, and it serves a use case the caption-only path structurally cannot. But it is worth deciding deliberately rather than by momentum — is this the thing that best serves the gate right now, or the most interesting thing to build? Record the answer in the ADR either way.
Acceptance
ADR accepted before implementation begins, superseding ADR-007's STT deferral for user-supplied media only
caveman: me store uploaded transcript scoped to one user, not in the shared public-caption table
Isolation test: user A cannot read user B's uploaded transcript, andtranscripts is still asserted non-RLS
Red-first tests per slice
Swedish upload routes to KB-Whisper; non-Swedish routes to vanilla — explicit test per branch
Oversized/unsupported uploads rejected with a clear message, never a timeout
A transcription failure degrades honestly (ADR-025's state-aware status), never a stuck spinner
infra#273 closed, or an explicit written decision to proceed without it
infra#274 resolved before the transcription adapter is designed
End-to-end: re-run the /bygge video through the real pipeline and compare against the hand-built fixture
tapir#22 (frontier models), tapir#23–26 (API/MCP — an upload endpoint should not fork their auth)
Prototype: tapir.d-ma.be/bygge
## Problem
Tapir summarizes videos it can reach by URL. It cannot take a video the user *has* — a phone recording of a site walkthrough, a recorded meeting, a lecture capture. Those are often the videos most worth summarizing, because nobody else will ever caption them.
A working prototype exists at `tapir.d-ma.be/bygge`: a Swedish construction-site walkthrough (iPhone, 4 min), transcribed with KB-Whisper and analyzed into timestamped sections, decision points, and alternatives, played back in sync with the video. It is entirely static — the transcript and analysis were produced by hand off-cluster and committed as fixtures. This epic is about making that flow real: **a logged-in user uploads any video and gets a transcript plus key takeaways back.**
Transcription routes by language: **KB-Whisper (`KBLab/kb-whisper-large`) for Swedish, vanilla Whisper for everything else.** KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish, which is the difference between a usable transcript and one that needs hand-correction.
---
## ⚠️ The constraint that shapes everything: uploaded transcripts are NOT public content
**Read this before designing anything.**
ADR-021 made `transcripts` a **shared, non-RLS table** keyed by `(provider, provider_video_id)`. The justification is explicit and narrow:
> It holds **only public caption content + the video's public id** — nothing user-identifying — and is therefore **NOT RLS-scoped** … This is the deliberate, single exception to the ADR-012 isolation boundary, and the only one.
An uploaded video's transcript breaks every clause of that. It is private content, owned by one user, that no other user has any right to see. The `/bygge` prototype is a live example: a named colleague discussing a client's building project.
**So uploaded transcripts must NOT go in the `transcripts` table.** They need their own RLS-scoped store — and the existing isolation test will not catch a mistake here, because it asserts `transcripts` *is* non-RLS. A naive "reuse the transcript store" implementation would leak private content across tenants and pass CI green.
This is the single highest-risk item in the epic. Treat it as the first design decision, not an implementation detail.
---
## Prerequisite: an ADR, before any code
ADR-007 deferred speech-to-text; ADR-014 item 4 says the reconsideration is "a future ADR, gated on this data." This epic needs that ADR written and accepted **first**.
It is not a straight reversal, and the ADR should say so. ADR-007 rejected *audio-download from YouTube* — yt-dlp fragility, ToS-grey territory, GPU contention. Upload-STT shares none of that: the user owns the file, there is no scraping, no caption rate gate, no ToS surface. What it does share is landing Whisper in the codebase, which ADR-007 deferred. So the ADR supersedes ADR-007's STT deferral **for user-supplied media only**, and leaves the YouTube-audio rejection standing.
ADR-017's process note records the one time a guardrail-reversing change shipped without an ADR, and why that was bad. Don't repeat it here.
---
## Open decisions the ADR must settle
### 1. Private transcript storage
Separate RLS-scoped table (`uploaded_transcripts` or similar), or a `user_id` column on `transcripts` with a partial RLS policy? The separate table is cleaner — it keeps ADR-021's exception genuinely singular and unambiguous, and the isolation test can assert both shapes independently. Recommend separate; decide explicitly.
### 2. Language detection and model routing
Chicken-and-egg: choosing between KB-Whisper and vanilla Whisper requires knowing the language, which normally comes from running Whisper. Options:
- Vanilla Whisper's language detection on the first ~30s, then route and transcribe fully. One extra short inference, fully automatic.
- Ask the uploader. Zero inference cost, one more form field, and users get it wrong.
- Default Swedish (this is a Swedish-first deployment), with a manual override.
Recommend detect-then-route, with an override. State what happens on a mixed-language recording — pick a primary, don't try to switch mid-file.
### 3. Where inference runs, and what it collides with
KB-Whisper runs on **iguana** (M2 Ultra) — the same host ADR-022 uses for the `gemma4-26b` summarizer fallback. A long transcription job and a summarizer fallback now contend for one machine. ADR-007's original GPU-contention objection was about koala; it has moved, not disappeared.
**`infra#273` (harden iguana for unattended service hosting) becomes a hard dependency.** Today whisper-server is started by hand, does not survive reboot, and has no monitoring. That was acceptable when transcription was an occasional manual errand. A user-facing upload feature makes iguana load-bearing, which is exactly the condition #273 says should trigger a revisit.
**`infra#274`** (can LiteLLM front KB-Whisper) is also relevant: if LiteLLM can proxy `/v1/audio/transcriptions`, Tapir gets routing, auth, and token accounting through the gateway it already uses, and the ADR-022 chain pattern extends naturally to STT endpoints. If it can't, Tapir needs a direct client to a second endpoint type. **Resolve #274 before designing the adapter.**
### 4. Long-running jobs vs the single-replica in-process scheduler
ADR-018 put discovery in-process and recorded a **load-bearing single-replica assumption**. Transcription is minutes, not seconds, and is CPU/GPU-bound elsewhere. Decide whether upload jobs run in the same process (simple, consistent with ADR-018, but a wedged transcription degrades the reading UI — the coupling ADR-018 knowingly accepted at a much smaller cost) or move to a worker. Do not silently multiply replicas; that breaks ADR-018.
### 5. File custody and retention
A 4-minute iPhone clip is ~330 MB at 11 Mbps; an hour of the same is ~5 GB. Settle:
- Ingress body-size limits and upload mechanism (direct POST vs chunked/resumable)
- Where bytes live during processing, and whether the video is **kept** after transcription. Keeping it enables the `/bygge`-style synced playback; discarding it is far better for storage and for GDPR posture.
- Explicit format/size/duration limits, rejected clearly rather than by timeout.
- Server-side transcode to 16 kHz mono WAV before inference. Do **not** rely on whisper-server's `--convert`; its `--tmp-dir` must exist on the host and its absence produced a silent failure during the prototype work.
### 6. The analysis schema
The prototype produces something richer than `summary`/`highlights`/`takeaways`: timestamped **sections** (`AVSNITT`), **decision points**, and **alternatives** with a stated leaning (`ALTERNATIV` … `Lutar åt: inne`). That richer shape is what makes it useful for a walkthrough — but ADR-022 exists because small models already emit malformed JSON on the *simple* schema. A more complex schema will fail parsing more often.
Decide: does upload analysis reuse the summary schema, or get its own? If its own, the tolerant-parse strategy (ADR-022 §3) needs extending, and the chain's endpoint ordering may need a stronger primary. Relates to **#22** (frontier models via LiteLLM) — a structured-output-capable model would make this materially more reliable.
---
## Scope
Ordered so each slice is independently shippable.
### Slice 1 — ADR + private transcript store
The ADR above, plus the RLS-scoped store for uploaded transcripts and an isolation test that asserts *both* boundaries: uploaded transcripts are user-scoped, and `transcripts` remains the only non-RLS table.
### Slice 2 — Upload and custody
Authenticated upload endpoint, size/format validation, storage, a `videos` row for the upload. Note the data-model wrinkle: `videos` is keyed by `(provider, provider_video_id)`. An upload has neither — it needs a synthetic provider (`upload`) and a generated id. Confirm this doesn't break ADR-021's key assumptions or the dedup logic.
### Slice 3 — Transcription adapter with language routing
Transcode → detect → route to KB-Whisper or vanilla → store. Chunking for long files, with overlap and SRT stitching. Timestamps are the point, so `response_format=srt` (or equivalent) rather than plain text.
### Slice 4 — Analysis
Structured analysis over the stored transcript, reusing the ADR-022 chain. Schema per decision 6.
### Slice 5 — Playback surface
Video plus timestamped analysis, synced. The `/bygge` page is the working reference for what good looks like; generalize it rather than redesigning.
---
## Non-goals
- **Audio-download from YouTube.** ADR-007's rejection stands untouched. This epic is user-supplied media only, and the ADR must say so explicitly or the next reader will assume the door is open.
- Public/anonymous upload. Authenticated users only (ADR-008 — no public SaaS machinery).
- Cross-user dedup of uploaded content. Two users uploading the same file get two private transcripts. That is correct, not wasteful.
- Real-time/streaming transcription.
- Editing or correcting transcripts in-UI.
---
## Honest question about sequencing
The Stage-0 gate (ADR-016) measures whether someone returns unprompted over 3–4 weeks, and ADR-020 deferred search, facets, and read/unread as premature until that loop is validated. This epic is substantially larger than any of those.
The counter-argument is real: `/bygge` exists because the maintainer had a video he actually wanted analyzed, which is genuine pull rather than speculation, and it serves a use case the caption-only path structurally cannot. But it is worth deciding deliberately rather than by momentum — **is this the thing that best serves the gate right now, or the most interesting thing to build?** Record the answer in the ADR either way.
---
## Acceptance
- [ ] ADR accepted before implementation begins, superseding ADR-007's STT deferral for user-supplied media only
- [ ] `caveman: me store uploaded transcript scoped to one user, not in the shared public-caption table`
- [ ] Isolation test: user A cannot read user B's uploaded transcript, **and** `transcripts` is still asserted non-RLS
- [ ] Red-first tests per slice
- [ ] Swedish upload routes to KB-Whisper; non-Swedish routes to vanilla — explicit test per branch
- [ ] Oversized/unsupported uploads rejected with a clear message, never a timeout
- [ ] A transcription failure degrades honestly (ADR-025's state-aware status), never a stuck spinner
- [ ] infra#273 closed, or an explicit written decision to proceed without it
- [ ] infra#274 resolved before the transcription adapter is designed
- [ ] End-to-end: re-run the `/bygge` video through the real pipeline and compare against the hand-built fixture
## Refs
- ADR-007 (STT deferral), ADR-012 (RLS isolation), ADR-021 (shared transcripts — **the constraint**), ADR-022 (endpoint chain), ADR-018 (single-replica), ADR-025 (honest status)
- infra#273 (iguana hardening), infra#274 (LiteLLM STT spike)
- tapir#22 (frontier models), tapir#23–26 (API/MCP — an upload endpoint should not fork their auth)
- Prototype: `tapir.d-ma.be/bygge`
Three time-boxed spikes that reproduce the /bygge result for a file that has never been processed, using only koala and iguana over their existing APIs. They exist to answer this epic's open decisions with measurements instead of argument, and they are explicitly not a first slice of it.
Is the playback page a format that generates, or a one-off?
Slice 5, decision 5
What they deliberately do not touch
No database write, no videos row, no upload endpoint, no ADR. Decision 1 — the RLS-scoped private store — stays fully open and unprejudiced, which is the point: it is this epic's highest-risk item and the last thing that should be settled by a spike's convenience. Nothing here can leak a private transcript because nothing here persists one.
Two things that changed since this epic was written
infra#274 is closed, shape A. LiteLLM v1.83.14-p3 fronts iguana/kb-whisper and the routing is live — verified end to end with a Swedish fixture, not reasoned about. The recommendation was explicit: no pipeline service, no new repo, what remains is a thin client-side wrapper. This epic's decision 3 can drop the "does Tapir need a direct client to a second endpoint type" branch.
infra#273 found the real failure mode, and it is worse than flakiness: a Metal OOM latches whisper.cpp's backend permanently — one out-of-memory and every later request fails for the life of the process, at any size. The trigger is ollama's resident footprint (qwen3-coder-next at 262 k context holds 58 GB of 64). So decision 3's contention is not "a long job and a fallback compete for one machine"; it is any chat request to a large iguana model kills transcription until someone runs launchctl kickstart. #30 reproduces this deliberately, once, to settle it.
On the sequencing question this epic asks itself
The honest answer to "is this the thing that best serves the Stage-0 gate, or the most interesting thing to build" is easier to give after three afternoons than before them — and cheaper. If #30 comes back saying no local model can hold the schema, the epic's cost changes materially and that is worth knowing before the ADR is written, not after.
Also surfaced while filing: the prototype's artifacts (IMG_1233.mov/.wav/.sv.srt/.genomgang.html) are untracked files in an unrelated repo's working directory, and the served copy on koala sits outside git. /bygge currently has no source of truth. #31 closes that.
## Spike filed ahead of the ADR: #29, #30, #31
Three time-boxed spikes that reproduce the `/bygge` result **for a file that has never been processed**, using only koala and iguana over their existing APIs. They exist to answer this epic's open decisions with measurements instead of argument, and they are explicitly *not* a first slice of it.
| | Question | Feeds |
|---|---|---|
| #29 | Does a new file reach a Swedish SRT unattended, through LiteLLM only? | decisions 2, 3, 5 |
| #30 | Can a *local* model produce the rich schema reliably enough to parse? | decisions 3, 6 |
| #31 | Is the playback page a format that generates, or a one-off? | Slice 5, decision 5 |
### What they deliberately do not touch
No database write, no `videos` row, no upload endpoint, no ADR. **Decision 1 — the RLS-scoped private store — stays fully open and unprejudiced**, which is the point: it is this epic's highest-risk item and the last thing that should be settled by a spike's convenience. Nothing here can leak a private transcript because nothing here persists one.
### Two things that changed since this epic was written
**infra#274 is closed, shape A.** LiteLLM `v1.83.14-p3` fronts `iguana/kb-whisper` and the routing is live — verified end to end with a Swedish fixture, not reasoned about. The recommendation was explicit: *no pipeline service, no new repo, what remains is a thin client-side wrapper.* This epic's decision 3 can drop the "does Tapir need a direct client to a second endpoint type" branch.
**infra#273 found the real failure mode**, and it is worse than flakiness: a Metal OOM **latches whisper.cpp's backend permanently** — one out-of-memory and every later request fails for the life of the process, at any size. The trigger is ollama's resident footprint (`qwen3-coder-next` at 262 k context holds 58 GB of 64). So decision 3's contention is not "a long job and a fallback compete for one machine"; it is **any chat request to a large iguana model kills transcription until someone runs `launchctl kickstart`**. #30 reproduces this deliberately, once, to settle it.
### On the sequencing question this epic asks itself
The honest answer to "is this the thing that best serves the Stage-0 gate, or the most interesting thing to build" is easier to give after three afternoons than before them — and cheaper. If #30 comes back saying no local model can hold the schema, the epic's cost changes materially and that is worth knowing before the ADR is written, not after.
Also surfaced while filing: the prototype's artifacts (`IMG_1233.mov/.wav/.sv.srt/.genomgang.html`) are **untracked files in an unrelated repo's working directory**, and the served copy on koala sits outside git. `/bygge` currently has no source of truth. #31 closes that.
Correction to my comment above, and a re-planned spike set
An adversarial review of #29/#30/#31 turned up that the spikes were largely proposing to rebuild work that already existed, and that two of my statements here — and one in this epic's own body — are false.
The toolchain was in tmpfs, one reboot from gone
analyze_srt.py (18.7 KB: SRT parser, frozen prompt, schema, and a four-check validator), build_page.py (the generator), payload.json, a synthetic test fixture, transcode.yaml, and five completed model-comparison runs were sitting in a session scratchpad under /tmp on an Arch host. /tmp is tmpfs.
Recovered and committed: scripts/spike-media/ (489b4bb), defects documented rather than fixed.
What that corrects
Stated
Actual
#28: the transcript and analysis "were produced by hand off-cluster"
Both were scripted. analyze_srt.py is an argparse CLI at temperature=0. Only transcription was interactive.
#30 (orig): produced "with a frontier model in the loop"
_meta.modell = berget/mistral-medium — mid-tier, through your own gateway. #30 was about to measure whether local models can do a thing it believed needed a frontier model.
My comment above: "#31 closes that [source-of-truth gap]"
It cannot, as scoped — its own acceptance said "artifacts gitignored". And mathias/tapir is public, so the real transcript cannot be committed here at all. See #31.
#31 (orig): record a bitrate target, add +faststart
transcode.yaml already had -crf 24 and +faststart. Its scale='min(1280,iw)':-2 is a no-op — the source is 720×1280.
#30 was going to return a false negative and blame the wrong thing
analyze_srt.py:194 hardcodes max_tokens: 6000. The 4-minute reference analysis is ~4,700 output tokens — 78% of budget for four minutes. Past ~6 minutes every model truncates. My scoring mapped truncation onto "malform rate", whose recommendation branch reads escalate to #22. I would have recommended buying frontier models to fix a constant.
Two further metric defects: both checks I specified are precision-only, so an empty analysis scores perfectly on all of them; and the strict-substring citation test scores the reference output at 54/61 (89%), and 2/7 on alternativ, because those citations legitimately stitch non-contiguous passages. A local model behaving identically to mistral-medium would have been graded ungrounded.
The validator already solved this and I did not know: TACKNINGSLUCKA is the recall term (fired on 4 of 5 prior runs), TALET … FINNS INTE I CITATET converts Swedish number-words to digits and caught 45×230 where the transcript said 220 — carried by a verbatim citation and a real timestamp, i.e. invisible to both checks I proposed.
Re-planned: #32 first, then #30, then #29; #31 is a chore
On sequencing, I was wrong above. I argued the honest answer to this epic's own question would be "easier after three afternoons than before them — and cheaper". It is neither. Three feasibility spikes measure what infra#274 and the recovered toolchain already prove, consume roughly the calendar ADR-016's gate measures over, and make "defer this" cost sunk work to say. #32 tests demand instead of restating it, using the artifact that already exists.
DECISION NEEDED — artifact custody
IMG_1233.sv.srt and payload.json are a named person discussing a client's building project, and this repo is public. Options and a recommendation are in #31. Until it is decided the durable copy is at ~/dev/.rescue/bygge-toolchain-2026-08-13/private/ — outside git, which is a stay of execution rather than a plan.
## Correction to my comment above, and a re-planned spike set
An adversarial review of #29/#30/#31 turned up that **the spikes were largely proposing to rebuild work that already existed**, and that two of my statements here — and one in this epic's own body — are false.
### The toolchain was in tmpfs, one reboot from gone
`analyze_srt.py` (18.7 KB: SRT parser, frozen prompt, schema, and a **four-check validator**), `build_page.py` (the generator), `payload.json`, a synthetic test fixture, `transcode.yaml`, and **five completed model-comparison runs** were sitting in a session scratchpad under `/tmp` on an Arch host. `/tmp` is `tmpfs`.
Recovered and committed: `scripts/spike-media/` (`489b4bb`), defects documented rather than fixed.
### What that corrects
| Stated | Actual |
|---|---|
| #28: the transcript and analysis "were produced by hand off-cluster" | Both were scripted. `analyze_srt.py` is an argparse CLI at `temperature=0`. **Only transcription was interactive.** |
| #30 (orig): produced "with a frontier model in the loop" | `_meta.modell` = **`berget/mistral-medium`** — mid-tier, through your own gateway. #30 was about to measure whether local models can do a thing it believed needed a frontier model. |
| My comment above: "#31 closes that [source-of-truth gap]" | It cannot, as scoped — its own acceptance said "artifacts gitignored". And `mathias/tapir` is **public**, so the real transcript cannot be committed here at all. See #31. |
| #31 (orig): record a bitrate target, add `+faststart` | `transcode.yaml` already had `-crf 24` and `+faststart`. Its `scale='min(1280,iw)':-2` is a **no-op** — the source is 720×1280. |
### #30 was going to return a false negative and blame the wrong thing
`analyze_srt.py:194` hardcodes `max_tokens: 6000`. The 4-minute reference analysis is ~4,700 output tokens — **78% of budget for four minutes.** Past ~6 minutes every model truncates. My scoring mapped truncation onto "malform rate", whose recommendation branch reads *escalate to #22*. I would have recommended buying frontier models to fix a constant.
Two further metric defects: both checks I specified are **precision-only**, so an empty analysis scores perfectly on all of them; and the strict-substring citation test scores the **reference output** at 54/61 (89%), and 2/7 on `alternativ`, because those citations legitimately stitch non-contiguous passages. A local model behaving identically to mistral-medium would have been graded ungrounded.
The validator already solved this and I did not know: `TACKNINGSLUCKA` is the recall term (fired on 4 of 5 prior runs), `TALET … FINNS INTE I CITATET` converts Swedish number-words to digits and caught `45×230` where the transcript said `220` — carried by a verbatim citation and a real timestamp, i.e. invisible to both checks I proposed.
### Re-planned: #32 first, then #30, then #29; #31 is a chore
| | | |
|---|---|---|
| **#32** | Does anyone re-open `/bygge`? | An afternoon, zero engineering. **Blocks #29 and #30.** |
| **#30** | Does a *local* model hold the schema? | Runs today against the existing SRT — no new file, no whisper. Now has a control arm and four real metrics. |
| **#29** | Does a new file reach an SRT unattended? | Positive-control fixture instead of a memory pre-flight; deliberate length ceiling; KB-vs-vanilla comparison that can delete most of decision 2. |
| **#31** | Not a spike | A `--title` flag and an encode measurement. |
On sequencing, I was wrong above. I argued the honest answer to this epic's own question would be "easier after three afternoons than before them — and cheaper". It is neither. Three feasibility spikes measure what infra#274 and the recovered toolchain already prove, consume roughly the calendar ADR-016's gate measures over, and make "defer this" cost sunk work to say. **#32 tests demand instead of restating it**, using the artifact that already exists.
### DECISION NEEDED — artifact custody
`IMG_1233.sv.srt` and `payload.json` are a named person discussing a client's building project, and this repo is public. Options and a recommendation are in #31. Until it is decided the durable copy is at `~/dev/.rescue/bygge-toolchain-2026-08-13/private/` — outside git, which is a stay of execution rather than a plan.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Problem
Tapir summarizes videos it can reach by URL. It cannot take a video the user has — a phone recording of a site walkthrough, a recorded meeting, a lecture capture. Those are often the videos most worth summarizing, because nobody else will ever caption them.
A working prototype exists at
tapir.d-ma.be/bygge: a Swedish construction-site walkthrough (iPhone, 4 min), transcribed with KB-Whisper and analyzed into timestamped sections, decision points, and alternatives, played back in sync with the video. It is entirely static — the transcript and analysis were produced by hand off-cluster and committed as fixtures. This epic is about making that flow real: a logged-in user uploads any video and gets a transcript plus key takeaways back.Transcription routes by language: KB-Whisper (
KBLab/kb-whisper-large) for Swedish, vanilla Whisper for everything else. KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish, which is the difference between a usable transcript and one that needs hand-correction.⚠️ The constraint that shapes everything: uploaded transcripts are NOT public content
Read this before designing anything.
ADR-021 made
transcriptsa shared, non-RLS table keyed by(provider, provider_video_id). The justification is explicit and narrow:An uploaded video's transcript breaks every clause of that. It is private content, owned by one user, that no other user has any right to see. The
/byggeprototype is a live example: a named colleague discussing a client's building project.So uploaded transcripts must NOT go in the
transcriptstable. They need their own RLS-scoped store — and the existing isolation test will not catch a mistake here, because it assertstranscriptsis non-RLS. A naive "reuse the transcript store" implementation would leak private content across tenants and pass CI green.This is the single highest-risk item in the epic. Treat it as the first design decision, not an implementation detail.
Prerequisite: an ADR, before any code
ADR-007 deferred speech-to-text; ADR-014 item 4 says the reconsideration is "a future ADR, gated on this data." This epic needs that ADR written and accepted first.
It is not a straight reversal, and the ADR should say so. ADR-007 rejected audio-download from YouTube — yt-dlp fragility, ToS-grey territory, GPU contention. Upload-STT shares none of that: the user owns the file, there is no scraping, no caption rate gate, no ToS surface. What it does share is landing Whisper in the codebase, which ADR-007 deferred. So the ADR supersedes ADR-007's STT deferral for user-supplied media only, and leaves the YouTube-audio rejection standing.
ADR-017's process note records the one time a guardrail-reversing change shipped without an ADR, and why that was bad. Don't repeat it here.
Open decisions the ADR must settle
1. Private transcript storage
Separate RLS-scoped table (
uploaded_transcriptsor similar), or auser_idcolumn ontranscriptswith a partial RLS policy? The separate table is cleaner — it keeps ADR-021's exception genuinely singular and unambiguous, and the isolation test can assert both shapes independently. Recommend separate; decide explicitly.2. Language detection and model routing
Chicken-and-egg: choosing between KB-Whisper and vanilla Whisper requires knowing the language, which normally comes from running Whisper. Options:
Recommend detect-then-route, with an override. State what happens on a mixed-language recording — pick a primary, don't try to switch mid-file.
3. Where inference runs, and what it collides with
KB-Whisper runs on iguana (M2 Ultra) — the same host ADR-022 uses for the
gemma4-26bsummarizer fallback. A long transcription job and a summarizer fallback now contend for one machine. ADR-007's original GPU-contention objection was about koala; it has moved, not disappeared.infra#273(harden iguana for unattended service hosting) becomes a hard dependency. Today whisper-server is started by hand, does not survive reboot, and has no monitoring. That was acceptable when transcription was an occasional manual errand. A user-facing upload feature makes iguana load-bearing, which is exactly the condition #273 says should trigger a revisit.infra#274(can LiteLLM front KB-Whisper) is also relevant: if LiteLLM can proxy/v1/audio/transcriptions, Tapir gets routing, auth, and token accounting through the gateway it already uses, and the ADR-022 chain pattern extends naturally to STT endpoints. If it can't, Tapir needs a direct client to a second endpoint type. Resolve #274 before designing the adapter.4. Long-running jobs vs the single-replica in-process scheduler
ADR-018 put discovery in-process and recorded a load-bearing single-replica assumption. Transcription is minutes, not seconds, and is CPU/GPU-bound elsewhere. Decide whether upload jobs run in the same process (simple, consistent with ADR-018, but a wedged transcription degrades the reading UI — the coupling ADR-018 knowingly accepted at a much smaller cost) or move to a worker. Do not silently multiply replicas; that breaks ADR-018.
5. File custody and retention
A 4-minute iPhone clip is ~330 MB at 11 Mbps; an hour of the same is ~5 GB. Settle:
/bygge-style synced playback; discarding it is far better for storage and for GDPR posture.--convert; its--tmp-dirmust exist on the host and its absence produced a silent failure during the prototype work.6. The analysis schema
The prototype produces something richer than
summary/highlights/takeaways: timestamped sections (AVSNITT), decision points, and alternatives with a stated leaning (ALTERNATIV…Lutar åt: inne). That richer shape is what makes it useful for a walkthrough — but ADR-022 exists because small models already emit malformed JSON on the simple schema. A more complex schema will fail parsing more often.Decide: does upload analysis reuse the summary schema, or get its own? If its own, the tolerant-parse strategy (ADR-022 §3) needs extending, and the chain's endpoint ordering may need a stronger primary. Relates to #22 (frontier models via LiteLLM) — a structured-output-capable model would make this materially more reliable.
Scope
Ordered so each slice is independently shippable.
Slice 1 — ADR + private transcript store
The ADR above, plus the RLS-scoped store for uploaded transcripts and an isolation test that asserts both boundaries: uploaded transcripts are user-scoped, and
transcriptsremains the only non-RLS table.Slice 2 — Upload and custody
Authenticated upload endpoint, size/format validation, storage, a
videosrow for the upload. Note the data-model wrinkle:videosis keyed by(provider, provider_video_id). An upload has neither — it needs a synthetic provider (upload) and a generated id. Confirm this doesn't break ADR-021's key assumptions or the dedup logic.Slice 3 — Transcription adapter with language routing
Transcode → detect → route to KB-Whisper or vanilla → store. Chunking for long files, with overlap and SRT stitching. Timestamps are the point, so
response_format=srt(or equivalent) rather than plain text.Slice 4 — Analysis
Structured analysis over the stored transcript, reusing the ADR-022 chain. Schema per decision 6.
Slice 5 — Playback surface
Video plus timestamped analysis, synced. The
/byggepage is the working reference for what good looks like; generalize it rather than redesigning.Non-goals
Honest question about sequencing
The Stage-0 gate (ADR-016) measures whether someone returns unprompted over 3–4 weeks, and ADR-020 deferred search, facets, and read/unread as premature until that loop is validated. This epic is substantially larger than any of those.
The counter-argument is real:
/byggeexists because the maintainer had a video he actually wanted analyzed, which is genuine pull rather than speculation, and it serves a use case the caption-only path structurally cannot. But it is worth deciding deliberately rather than by momentum — is this the thing that best serves the gate right now, or the most interesting thing to build? Record the answer in the ADR either way.Acceptance
caveman: me store uploaded transcript scoped to one user, not in the shared public-caption tabletranscriptsis still asserted non-RLS/byggevideo through the real pipeline and compare against the hand-built fixtureRefs
tapir.d-ma.be/byggeSpike filed ahead of the ADR: #29, #30, #31
Three time-boxed spikes that reproduce the
/byggeresult for a file that has never been processed, using only koala and iguana over their existing APIs. They exist to answer this epic's open decisions with measurements instead of argument, and they are explicitly not a first slice of it.What they deliberately do not touch
No database write, no
videosrow, no upload endpoint, no ADR. Decision 1 — the RLS-scoped private store — stays fully open and unprejudiced, which is the point: it is this epic's highest-risk item and the last thing that should be settled by a spike's convenience. Nothing here can leak a private transcript because nothing here persists one.Two things that changed since this epic was written
infra#274 is closed, shape A. LiteLLM
v1.83.14-p3frontsiguana/kb-whisperand the routing is live — verified end to end with a Swedish fixture, not reasoned about. The recommendation was explicit: no pipeline service, no new repo, what remains is a thin client-side wrapper. This epic's decision 3 can drop the "does Tapir need a direct client to a second endpoint type" branch.infra#273 found the real failure mode, and it is worse than flakiness: a Metal OOM latches whisper.cpp's backend permanently — one out-of-memory and every later request fails for the life of the process, at any size. The trigger is ollama's resident footprint (
qwen3-coder-nextat 262 k context holds 58 GB of 64). So decision 3's contention is not "a long job and a fallback compete for one machine"; it is any chat request to a large iguana model kills transcription until someone runslaunchctl kickstart. #30 reproduces this deliberately, once, to settle it.On the sequencing question this epic asks itself
The honest answer to "is this the thing that best serves the Stage-0 gate, or the most interesting thing to build" is easier to give after three afternoons than before them — and cheaper. If #30 comes back saying no local model can hold the schema, the epic's cost changes materially and that is worth knowing before the ADR is written, not after.
Also surfaced while filing: the prototype's artifacts (
IMG_1233.mov/.wav/.sv.srt/.genomgang.html) are untracked files in an unrelated repo's working directory, and the served copy on koala sits outside git./byggecurrently has no source of truth. #31 closes that.Correction to my comment above, and a re-planned spike set
An adversarial review of #29/#30/#31 turned up that the spikes were largely proposing to rebuild work that already existed, and that two of my statements here — and one in this epic's own body — are false.
The toolchain was in tmpfs, one reboot from gone
analyze_srt.py(18.7 KB: SRT parser, frozen prompt, schema, and a four-check validator),build_page.py(the generator),payload.json, a synthetic test fixture,transcode.yaml, and five completed model-comparison runs were sitting in a session scratchpad under/tmpon an Arch host./tmpistmpfs.Recovered and committed:
scripts/spike-media/(489b4bb), defects documented rather than fixed.What that corrects
analyze_srt.pyis an argparse CLI attemperature=0. Only transcription was interactive._meta.modell=berget/mistral-medium— mid-tier, through your own gateway. #30 was about to measure whether local models can do a thing it believed needed a frontier model.mathias/tapiris public, so the real transcript cannot be committed here at all. See #31.+faststarttranscode.yamlalready had-crf 24and+faststart. Itsscale='min(1280,iw)':-2is a no-op — the source is 720×1280.#30 was going to return a false negative and blame the wrong thing
analyze_srt.py:194hardcodesmax_tokens: 6000. The 4-minute reference analysis is ~4,700 output tokens — 78% of budget for four minutes. Past ~6 minutes every model truncates. My scoring mapped truncation onto "malform rate", whose recommendation branch reads escalate to #22. I would have recommended buying frontier models to fix a constant.Two further metric defects: both checks I specified are precision-only, so an empty analysis scores perfectly on all of them; and the strict-substring citation test scores the reference output at 54/61 (89%), and 2/7 on
alternativ, because those citations legitimately stitch non-contiguous passages. A local model behaving identically to mistral-medium would have been graded ungrounded.The validator already solved this and I did not know:
TACKNINGSLUCKAis the recall term (fired on 4 of 5 prior runs),TALET … FINNS INTE I CITATETconverts Swedish number-words to digits and caught45×230where the transcript said220— carried by a verbatim citation and a real timestamp, i.e. invisible to both checks I proposed.Re-planned: #32 first, then #30, then #29; #31 is a chore
/bygge?--titleflag and an encode measurement.On sequencing, I was wrong above. I argued the honest answer to this epic's own question would be "easier after three afternoons than before them — and cheaper". It is neither. Three feasibility spikes measure what infra#274 and the recovered toolchain already prove, consume roughly the calendar ADR-016's gate measures over, and make "defer this" cost sunk work to say. #32 tests demand instead of restating it, using the artifact that already exists.
DECISION NEEDED — artifact custody
IMG_1233.sv.srtandpayload.jsonare a named person discussing a client's building project, and this repo is public. Options and a recommendation are in #31. Until it is decided the durable copy is at~/dev/.rescue/bygge-toolchain-2026-08-13/private/— outside git, which is a stay of execution rather than a plan.