feat: upload audio, get a private Swedish transcript, discuss it (#33, ADR-033) #39

Merged
mathias merged 299 commits from feat/33-audio-upload-transcript into main 2026-10-03 14:13:37 +00:00
Showing only changes of commit 77fe6954e5 - Show all commits
+31 -13
View File
@@ -1353,11 +1353,13 @@ would leak across tenants **and pass CI green**.
`berget/kb-whisper` returns sub-second times and a different `language` token.
- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.**
#33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does
not retrieve; it stuffs the transcript into the prompt and cuts the tail.
not retrieve; it stuffs the transcript into the prompt and cuts the tail. Measured against
the offered models: `phi4-mini` 400s at ~13k tokens; `gemma4-26b` is right to 90,000 chars and
silently drops the head at 120,000.
**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on
iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal
OOM, deliberately not provoked.
iguana; and the Metal OOM, deliberately not provoked. (Whether the offered chat models can hold
an hour of Swedish *was* an unknown here; it is now measured, see decision 8.)
**Decision.**
@@ -1436,13 +1438,21 @@ OOM, deliberately not provoked.
8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers
look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism
as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model)
carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour
meeting to its first third, so uploaded transcripts take their own cap,
`TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at
measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a
guess until the chat slice measures what the offered models actually accept; it is not
evidence that they can.
as ADR-027's summary gate, **before any model is called**. ADR-027's decisions (ephemeral,
switchable models, trust-the-model) carry over unchanged. **Two things must differ, and both
were measured** (`docs/research/chat-context-over-long-transcripts-2026-10-01.md`):
- **The transcript budget.** The 18,000-char caption default cuts a one-hour meeting to its
first third. Uploaded transcripts take their own `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`,
default **90,000** — the measured ceiling of the best offered model (`iguana/gemma4-26b`),
which answers correctly there but **silently loses the head of a 120,000-char transcript**
with no error and no truncation flag. (An earlier draft of this ADR proposed 120,000 as a
guess; the measurement refuted it.) The mechanism is a per-request `chat.Request.MaxChars`.
- **The opening model.** The offered list leads with the summarizer default `koala/phi4-mini`,
whose 8,192-token context returns HTTP 400 on an hour of speech (~13,000 tokens). Chat over
an upload opens on `TAPIR_UPLOAD_CHAT_MODEL` (default `iguana/gemma4-26b`); the switcher
still lists every model, and picking `phi4-mini` on a long recording fails loudly and
inline. A per-model budget is not built.
Past ~90,000 chars (≈100 minutes) no offered model can answer by stuffing; see triggers.
9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py`
hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection;
@@ -1465,7 +1475,13 @@ deferred. The scope cut — chat first, no rich analysis, no video — is what k
dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e.
private audio leaving the homelab, silently unless the per-row endpoint is surfaced.
- A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it.
- Chat over a long transcript is bounded by model context, not by anything in Tapir.
- Chat over a long transcript is bounded by model context, not by anything in Tapir, and the
failure at the edge is *silent* for gemma (the head is dropped, no error). The 90,000 default
sits just inside the measured limit and is only as good as that single measurement.
- **The default chat model for uploads is a large ollama model on iguana — the same host as
`iguana/kb-whisper`.** infra#273 is exactly that a large resident ollama model can Metal-OOM
whisper-server permanently. Chatting with a transcript can therefore load the model that
wedges the transcriber. Not solved here; the berget fallback limits the blast radius.
- Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`).
**Validation / resolution triggers** (each observable by someone other than the author):
@@ -1473,8 +1489,10 @@ deferred. The scope cut — chat first, no rich analysis, no video — is what k
request at the current cap fails with whisper-side (not ingress) errors.
- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback,
swap the primary/fallback order and revisit infra#273.
- *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose
answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap.
- *Retrieval chat*: build chunked retrieval when a recording's transcript exceeds the measured
90,000-char ceiling (≈100 minutes) — i.e. the moment the duration cap is raised above that, or a
real recording turns out denser than the fixture — or when the offered model list changes and
the ceiling is re-measured lower.
- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one
replica, ADR-018 is revisited.
- *The decision itself*: if the maintainer has not used the feature unprompted within the