Rewritten 2026-08-13 after review. Blocked on #32. Independent of #30 — run #30 first, it is cheaper and more likely to change the ADR.
Question to answer
Transcription itself is proven — infra#274 verified the LiteLLM → iguana/kb-whisper hop end to end. What is unproven is unattendedness: does a new file get to an SRT with no human keystroke between invocation and output.
That is the only leg of the /bygge pipeline that was interactive. The analysis and page steps were already scripts (scripts/spike-media/, committed).
Positive control instead of a memory pre-flight
The original asked for a unit test discriminating "latched backend" from "transcription failure" via the error string, and a pre-flight reading iguana's resident model footprint. Both are wrong.
infra#274 records {"error":"failed to process audio"} appearing for four distinct causes before the wedge was diagnosed — missing --tmp-dir, ffmpeg off PATH, wrong multipart field name, audio format. A test over that string is a green test for a discriminator that provably does not discriminate.
And the memory pre-flight predicts a condition the control observes. It also drags ollama ps and iguana host knowledge into tapir's repo, duplicating infra#273's open watchdog item.
Instead: ship a ~3 s Swedish fixture WAV in scripts/spike-media/fixtures/. POST it before the real file and again after.
fixture before
real file
fixture after
conclusion
exit
fails
—
—
already latched — print launchctl kickstart, do not retry
3
passes
fails
fails
latched during the run (ollama loaded a model mid-job — the #273 window)
3
passes
fails
passes
it is the file
4
passes
passes
—
clean
0
Three fixture-result combinations → three classifications, unit-testable against recorded responses with no model call. This is also exactly infra#273's still-open "health check that transcribes rather than pings".
Find the ceiling deliberately
#28 decision 4 (in-process vs worker) is driven by the epic's own use cases — "a recorded meeting, a lecture capture", i.e. 45–90 minutes. A rate measured on a 4-minute file will be multiplied by 15 and be wrong, because both known failure modes are length-dependent and non-linear: the SIGABRT on large files (infra#273 layer 3 — "restarting yields a healthy process that fails identically on the same file") and the Metal OOM, whose probability rises with allocation size.
IMG_1233.wav is already 16 kHz mono conformant. Concatenate it ×4, ×8, ×16 → 16 / 32 / 64 audio-minutes and POST each. Twenty minutes of wall-clock for the most decision-changing number in the epic.
Settle decision 2 while the audio is already in hand
#28 decision 2 builds detect-then-route, an extra inference pass, an override field and a mixed-language policy — all resting on one inherited, locally unverified claim: "KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish."
If that gap does not appear on your domain — handheld outdoor Swedish, construction jargon, two speakers, wind — decision 2 collapses to "always vanilla, no detection, no routing" and Slice 3 loses its most complex part.
Cheapest decisive form: same IMG_1233.wav through KB-Whisper and through a vanilla-Whisper endpoint on iguana, diff the SRTs, count differences on domain terms (bärlina, nock, bottenplatta, 9,40, 1,80). No WER harness — a diff and a count of material-and-measurement errors settles it. Ground truth is the existing SRT plus your own ears on four minutes.
Traps to encode, not rediscover
-H 'Expect:' mandatory (else curl (52)). Multipart field is file (audio_file → "Invalid request"). Transcode client-side with ffmpeg to strict 16 kHz mono PCM WAV; do not use whisper-server's --convert. Key from the secret store, never inline.
Non-goals
Database writes, videos rows, upload endpoint, ADR. #28 decision 1 (the private RLS-scoped store) stays open and unprejudiced — nothing here persists a transcript.
Chunking with overlap and SRT stitching. Find the ceiling; do not build past it.
Reading iguana's memory state (cut — see above).
Cloud transcription endpoints in the recommendation. The vanilla-vs-KB comparison uses a local vanilla model.
Acceptance
caveman: me turn one new file into timestamped Swedish text, using only machines me already own
The answer to this issue's question is the exit code. Zero human keystrokes between invocation and SRT → exit 0. Any kickstart → exit 3, log records LATCHED.
Output SRT's final cue end is within 2 s of the container duration, and cue count ≥ 15 per audio-minute — an SRT that silently drops the last 40 s must not pass
3 s fixture WAV committed; before/after control implemented; three outcome combinations unit-tested red-first against recorded responses
Timing at 4, 16, 32 and 64 audio-minutes — or the length at which it first fails, with the failure mode named
KB-Whisper vs vanilla diff on the same audio, with a count of material/measurement errors per side, and a recommendation to #28 decision 2
Writeup in docs/research/ alongside whisper-stt-feasibility.md
What changed in this rewrite
Dropped the memory pre-flight and the error-string unit test (both false-green). Added the positive-control fixture, the deliberate ceiling test using audio already on disk, and the KB-vs-vanilla comparison that can delete most of #28 decision 2. Corrected the premise: the /bygge analysis and page steps were not interactive — only transcription was.
infra#274 (closed — shape A, the proof, and the four-causes-one-error-string history), infra#273 (LaunchAgent shipped; Metal OOM root cause; watchdog still open)
**Rewritten 2026-08-13** after review. Blocked on #32. Independent of #30 — run #30 first, it is cheaper and more likely to change the ADR.
## Question to answer
Transcription itself is proven — infra#274 verified the LiteLLM → `iguana/kb-whisper` hop end to end. What is unproven is **unattendedness**: does a new file get to an SRT with no human keystroke between invocation and output.
That is the only leg of the `/bygge` pipeline that was interactive. The analysis and page steps were already scripts (`scripts/spike-media/`, committed).
## Positive control instead of a memory pre-flight
The original asked for a unit test discriminating "latched backend" from "transcription failure" via the error string, and a pre-flight reading iguana's resident model footprint. Both are wrong.
infra#274 records `{"error":"failed to process audio"}` appearing for **four distinct causes** before the wedge was diagnosed — missing `--tmp-dir`, ffmpeg off `PATH`, wrong multipart field name, audio format. A test over that string is a green test for a discriminator that provably does not discriminate.
And the memory pre-flight predicts a condition the control **observes**. It also drags `ollama ps` and iguana host knowledge into tapir's repo, duplicating infra#273's open watchdog item.
**Instead:** ship a ~3 s Swedish fixture WAV in `scripts/spike-media/fixtures/`. POST it before the real file and again after.
| fixture before | real file | fixture after | conclusion | exit |
|---|---|---|---|---|
| fails | — | — | already latched — print `launchctl kickstart`, **do not retry** | 3 |
| passes | fails | fails | latched *during* the run (ollama loaded a model mid-job — the #273 window) | 3 |
| passes | fails | passes | it is the file | 4 |
| passes | passes | — | clean | 0 |
Three fixture-result combinations → three classifications, unit-testable against recorded responses with no model call. This is also exactly infra#273's still-open "health check that transcribes rather than pings".
## Find the ceiling deliberately
#28 decision 4 (in-process vs worker) is driven by the epic's own use cases — "a recorded meeting, a lecture capture", i.e. 45–90 minutes. A rate measured on a 4-minute file will be multiplied by 15 and be wrong, because both known failure modes are length-dependent and non-linear: the SIGABRT on large files (infra#273 layer 3 — *"restarting yields a healthy process that fails identically on the same file"*) and the Metal OOM, whose probability rises with allocation size.
`IMG_1233.wav` is already 16 kHz mono conformant. Concatenate it ×4, ×8, ×16 → 16 / 32 / 64 audio-minutes and POST each. Twenty minutes of wall-clock for the most decision-changing number in the epic.
## Settle decision 2 while the audio is already in hand
#28 decision 2 builds detect-then-route, an extra inference pass, an override field and a mixed-language policy — all resting on one inherited, locally unverified claim: *"KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish."*
If that gap does not appear on **your** domain — handheld outdoor Swedish, construction jargon, two speakers, wind — decision 2 collapses to "always vanilla, no detection, no routing" and Slice 3 loses its most complex part.
Cheapest decisive form: same `IMG_1233.wav` through KB-Whisper and through a vanilla-Whisper endpoint on iguana, diff the SRTs, count differences on domain terms (`bärlina`, `nock`, `bottenplatta`, `9,40`, `1,80`). No WER harness — a diff and a count of material-and-measurement errors settles it. Ground truth is the existing SRT plus your own ears on four minutes.
## Traps to encode, not rediscover
`-H 'Expect:'` mandatory (else `curl (52)`). Multipart field is `file` (`audio_file` → "Invalid request"). Transcode client-side with ffmpeg to strict 16 kHz mono PCM WAV; do **not** use whisper-server's `--convert`. Key from the secret store, never inline.
## Non-goals
- Database writes, `videos` rows, upload endpoint, ADR. #28 decision 1 (the private RLS-scoped store) stays open and unprejudiced — nothing here persists a transcript.
- Chunking with overlap and SRT stitching. Find the ceiling; do not build past it.
- Reading iguana's memory state (cut — see above).
- Cloud transcription endpoints in the recommendation. The vanilla-vs-KB comparison uses a local vanilla model.
## Acceptance
- [ ] `caveman: me turn one new file into timestamped Swedish text, using only machines me already own`
- [ ] **The answer to this issue's question is the exit code.** Zero human keystrokes between invocation and SRT → exit 0. Any `kickstart` → exit 3, log records `LATCHED`.
- [ ] Output SRT's final cue `end` is within 2 s of the container duration, and cue count ≥ 15 per audio-minute — an SRT that silently drops the last 40 s must not pass
- [ ] 3 s fixture WAV committed; before/after control implemented; three outcome combinations unit-tested red-first against recorded responses
- [ ] Timing at 4, 16, 32 and 64 audio-minutes — or the length at which it first fails, **with the failure mode named**
- [ ] KB-Whisper vs vanilla diff on the same audio, with a count of material/measurement errors per side, and a recommendation to #28 decision 2
- [ ] Writeup in `docs/research/` alongside `whisper-stt-feasibility.md`
## What changed in this rewrite
Dropped the memory pre-flight and the error-string unit test (both false-green). Added the positive-control fixture, the deliberate ceiling test using audio already on disk, and the KB-vs-vanilla comparison that can delete most of #28 decision 2. Corrected the premise: the `/bygge` analysis and page steps were **not** interactive — only transcription was.
## Refs
- `scripts/spike-media/` (committed 2026-08-13), #32 (blocks this), #30 (independent)
- infra#274 (closed — shape A, the proof, and the four-causes-one-error-string history), infra#273 (LaunchAgent shipped; Metal OOM root cause; watchdog still open)
- #28 decisions 2, 3, 4
mathias
changed title from Spike 1/3: a new video file → Swedish SRT, through LiteLLM only (koala + iguana) to Spike B: does a new file reach a Swedish SRT unattended? (positive control, deliberate ceiling)2026-08-13 09:38:58 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Rewritten 2026-08-13 after review. Blocked on #32. Independent of #30 — run #30 first, it is cheaper and more likely to change the ADR.
Question to answer
Transcription itself is proven — infra#274 verified the LiteLLM →
iguana/kb-whisperhop end to end. What is unproven is unattendedness: does a new file get to an SRT with no human keystroke between invocation and output.That is the only leg of the
/byggepipeline that was interactive. The analysis and page steps were already scripts (scripts/spike-media/, committed).Positive control instead of a memory pre-flight
The original asked for a unit test discriminating "latched backend" from "transcription failure" via the error string, and a pre-flight reading iguana's resident model footprint. Both are wrong.
infra#274 records
{"error":"failed to process audio"}appearing for four distinct causes before the wedge was diagnosed — missing--tmp-dir, ffmpeg offPATH, wrong multipart field name, audio format. A test over that string is a green test for a discriminator that provably does not discriminate.And the memory pre-flight predicts a condition the control observes. It also drags
ollama psand iguana host knowledge into tapir's repo, duplicating infra#273's open watchdog item.Instead: ship a ~3 s Swedish fixture WAV in
scripts/spike-media/fixtures/. POST it before the real file and again after.launchctl kickstart, do not retryThree fixture-result combinations → three classifications, unit-testable against recorded responses with no model call. This is also exactly infra#273's still-open "health check that transcribes rather than pings".
Find the ceiling deliberately
#28 decision 4 (in-process vs worker) is driven by the epic's own use cases — "a recorded meeting, a lecture capture", i.e. 45–90 minutes. A rate measured on a 4-minute file will be multiplied by 15 and be wrong, because both known failure modes are length-dependent and non-linear: the SIGABRT on large files (infra#273 layer 3 — "restarting yields a healthy process that fails identically on the same file") and the Metal OOM, whose probability rises with allocation size.
IMG_1233.wavis already 16 kHz mono conformant. Concatenate it ×4, ×8, ×16 → 16 / 32 / 64 audio-minutes and POST each. Twenty minutes of wall-clock for the most decision-changing number in the epic.Settle decision 2 while the audio is already in hand
#28 decision 2 builds detect-then-route, an extra inference pass, an override field and a mixed-language policy — all resting on one inherited, locally unverified claim: "KB-Whisper is roughly half the WER of vanilla large-v3 on Swedish."
If that gap does not appear on your domain — handheld outdoor Swedish, construction jargon, two speakers, wind — decision 2 collapses to "always vanilla, no detection, no routing" and Slice 3 loses its most complex part.
Cheapest decisive form: same
IMG_1233.wavthrough KB-Whisper and through a vanilla-Whisper endpoint on iguana, diff the SRTs, count differences on domain terms (bärlina,nock,bottenplatta,9,40,1,80). No WER harness — a diff and a count of material-and-measurement errors settles it. Ground truth is the existing SRT plus your own ears on four minutes.Traps to encode, not rediscover
-H 'Expect:'mandatory (elsecurl (52)). Multipart field isfile(audio_file→ "Invalid request"). Transcode client-side with ffmpeg to strict 16 kHz mono PCM WAV; do not use whisper-server's--convert. Key from the secret store, never inline.Non-goals
videosrows, upload endpoint, ADR. #28 decision 1 (the private RLS-scoped store) stays open and unprejudiced — nothing here persists a transcript.Acceptance
caveman: me turn one new file into timestamped Swedish text, using only machines me already ownkickstart→ exit 3, log recordsLATCHED.endis within 2 s of the container duration, and cue count ≥ 15 per audio-minute — an SRT that silently drops the last 40 s must not passdocs/research/alongsidewhisper-stt-feasibility.mdWhat changed in this rewrite
Dropped the memory pre-flight and the error-string unit test (both false-green). Added the positive-control fixture, the deliberate ceiling test using audio already on disk, and the KB-vs-vanilla comparison that can delete most of #28 decision 2. Corrected the premise: the
/byggeanalysis and page steps were not interactive — only transcription was.Refs
scripts/spike-media/(committed 2026-08-13), #32 (blocks this), #30 (independent)Spike 1/3: a new video file → Swedish SRT, through LiteLLM only (koala + iguana)to Spike B: does a new file reach a Swedish SRT unattended? (positive control, deliberate ceiling)