feat: upload audio, get a private Swedish transcript, discuss it (#33, ADR-033) #39
+31
-13
@@ -1353,11 +1353,13 @@ would leak across tenants **and pass CI green**.
|
||||
`berget/kb-whisper` returns sub-second times and a different `language` token.
|
||||
- **The existing chat path truncates at 18,000 chars (~22 min of speech), keeping the head.**
|
||||
#33 assumed chat "retrieves rather than generates" and so has no length ceiling. It does
|
||||
not retrieve; it stuffs the transcript into the prompt and cuts the tail.
|
||||
not retrieve; it stuffs the transcript into the prompt and cuts the tail. Measured against
|
||||
the offered models: `phi4-mini` 400s at ~13k tokens; `gemma4-26b` is right to 90,000 chars and
|
||||
silently drops the head at 120,000.
|
||||
|
||||
**Unknowns, stated.** Real-speech accuracy; compressed-input behaviour; concurrent long jobs on
|
||||
iguana; whether the models chat offers can hold an hour of Swedish (~50 k chars); and the Metal
|
||||
OOM, deliberately not provoked.
|
||||
iguana; and the Metal OOM, deliberately not provoked. (Whether the offered chat models can hold
|
||||
an hour of Swedish *was* an unknown here; it is now measured, see decision 8.)
|
||||
|
||||
**Decision.**
|
||||
|
||||
@@ -1436,13 +1438,21 @@ OOM, deliberately not provoked.
|
||||
|
||||
8. **Chat reuses `chat.Service`; it is not forked.** The uploaded-transcript chat handlers
|
||||
look the row up through the RLS-scoped store, so a non-owner gets 404 by the same mechanism
|
||||
as ADR-027's summary gate. ADR-027's decisions (ephemeral, switchable models, trust-the-model)
|
||||
carry over unchanged. **One thing must differ:** the 18,000-char default cuts a one-hour
|
||||
meeting to its first third, so uploaded transcripts take their own cap,
|
||||
`TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`. **Provisional default 120,000 chars (~2.4 h at
|
||||
measured density)**, with the existing `Truncated` flag surfaced honestly. The default is a
|
||||
guess until the chat slice measures what the offered models actually accept; it is not
|
||||
evidence that they can.
|
||||
as ADR-027's summary gate, **before any model is called**. ADR-027's decisions (ephemeral,
|
||||
switchable models, trust-the-model) carry over unchanged. **Two things must differ, and both
|
||||
were measured** (`docs/research/chat-context-over-long-transcripts-2026-10-01.md`):
|
||||
- **The transcript budget.** The 18,000-char caption default cuts a one-hour meeting to its
|
||||
first third. Uploaded transcripts take their own `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS`,
|
||||
default **90,000** — the measured ceiling of the best offered model (`iguana/gemma4-26b`),
|
||||
which answers correctly there but **silently loses the head of a 120,000-char transcript**
|
||||
with no error and no truncation flag. (An earlier draft of this ADR proposed 120,000 as a
|
||||
guess; the measurement refuted it.) The mechanism is a per-request `chat.Request.MaxChars`.
|
||||
- **The opening model.** The offered list leads with the summarizer default `koala/phi4-mini`,
|
||||
whose 8,192-token context returns HTTP 400 on an hour of speech (~13,000 tokens). Chat over
|
||||
an upload opens on `TAPIR_UPLOAD_CHAT_MODEL` (default `iguana/gemma4-26b`); the switcher
|
||||
still lists every model, and picking `phi4-mini` on a long recording fails loudly and
|
||||
inline. A per-model budget is not built.
|
||||
Past ~90,000 chars (≈100 minutes) no offered model can answer by stuffing; see triggers.
|
||||
|
||||
9. **Not decided here.** Rich `AVSNITT`/`ALTERNATIV` analysis (gated on #30 — `analyze_srt.py`
|
||||
hardcodes `max_tokens: 6000`); video, keyframes, synced playback (#28); language detection;
|
||||
@@ -1465,7 +1475,13 @@ deferred. The scope cut — chat first, no rich analysis, no video — is what k
|
||||
dependency that matters) — a Metal OOM now means *every upload fails over to berget*, i.e.
|
||||
private audio leaving the homelab, silently unless the per-row endpoint is surfaced.
|
||||
- A 90-minute job holds the single slot for ~7.5 minutes; a second user's upload queues behind it.
|
||||
- Chat over a long transcript is bounded by model context, not by anything in Tapir.
|
||||
- Chat over a long transcript is bounded by model context, not by anything in Tapir, and the
|
||||
failure at the edge is *silent* for gemma (the head is dropped, no error). The 90,000 default
|
||||
sits just inside the measured limit and is only as good as that single measurement.
|
||||
- **The default chat model for uploads is a large ollama model on iguana — the same host as
|
||||
`iguana/kb-whisper`.** infra#273 is exactly that a large resident ollama model can Metal-OOM
|
||||
whisper-server permanently. Chatting with a transcript can therefore load the model that
|
||||
wedges the transcriber. Not solved here; the berget fallback limits the blast radius.
|
||||
- Tapir's own ingress body limit for a 200 MB upload is **unverified** (it lives in `infra`).
|
||||
|
||||
**Validation / resolution triggers** (each observable by someone other than the author):
|
||||
@@ -1473,8 +1489,10 @@ deferred. The scope cut — chat first, no rich analysis, no video — is what k
|
||||
request at the current cap fails with whisper-side (not ingress) errors.
|
||||
- *Berget-first*: if more than a quarter of uploads in any week are served by the fallback,
|
||||
swap the primary/fallback order and revisit infra#273.
|
||||
- *Retrieval chat*: if the chat slice finds an offered model cannot answer a question whose
|
||||
answer is in the last third of a 60-minute transcript, chunked retrieval replaces the larger cap.
|
||||
- *Retrieval chat*: build chunked retrieval when a recording's transcript exceeds the measured
|
||||
90,000-char ceiling (≈100 minutes) — i.e. the moment the duration cap is raised above that, or a
|
||||
real recording turns out denser than the fixture — or when the offered model list changes and
|
||||
the ceiling is re-measured lower.
|
||||
- *Worker*: if a transcription job ever degrades the reading UI, or Tapir runs more than one
|
||||
replica, ADR-018 is revisited.
|
||||
- *The decision itself*: if the maintainer has not used the feature unprompted within the
|
||||
|
||||
Reference in New Issue
Block a user