docs(research): chat over a long transcript -- what the offered models really hold

phi4-mini 400s at ~13k tokens; gemma4-26b is correct to 90,000 chars and
silently loses the HEAD at 120,000 (no error, no truncation flag). The
end-needle test alone passed at 120k -- only a head-needle exposed it.
Records the default-model trap and the iguana contention hazard.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

docs/research/chat-context-over-long-transcripts-2026-10-01.md

@@ -0,0 +1,66 @@
+# Chat over a long transcript — what the offered models really hold (tapir#33, 2026-10-01)
+
+Question: ADR-033 reuses `chat.Service`, which stuffs the transcript into the prompt and
+cuts the tail. #33 assumed chat "retrieves rather than generates" and so has no length
+ceiling. It does not retrieve, so its ceiling is the model's context. What is it, for the
+models chat actually offers?
+
+## Answer
+
+| Model (via the NodePort gateway) | Result | Limit found |
+|---|---|---|
+| `koala/phi4-mini` | correct to 30,000 chars; **HTTP 400 at 50,000** (13,332 tokens vs an 8,192-token context) | ~22,000 chars in production, where the client adds `max_tokens: 1500` |
+| `iguana/gemma4-26b` | correct at 18k, 50k and **90,000 chars** (26.5 k tokens); **at 120,000 chars it silently loses the head** | **90,000 chars** |
+| `berget/mistral-small` | answered "the transcript does not contain it" for a fact **at the end** of an 18,000-char and a 50,000-char transcript | unreliable here, one run each — see caveats |
+
+The dangerous one is gemma at 120,000 chars: **no error, no truncation flag** — the reported
+prompt tokens fell from 26,548 (90 k chars) to 16,387 (120 k chars), and the model answered
+"it does not say who the chair is" about a sentence at the very start of the transcript. The
+serving layer truncated the prompt from the front. `chat.Service` reports `Truncated: false`
+because *it* did not cut anything.
+
+## What this changes
+
+1. **The budget for an uploaded transcript is 90,000 chars, not a round 120,000.** That is the
+   best offered model's measured limit, ~100 minutes of speech at the fixture's density.
+   `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS` defaults to it.
+2. **Chat over an upload must not open on the summarizer default.** The offered list is
+   `[SummarizerModel, FallbackModel, CloudFallbackModel]`, so `koala/phi4-mini` is first — and
+   400s on an hour of speech. `TAPIR_UPLOAD_CHAT_MODEL` (default `iguana/gemma4-26b`) is what
+   the upload chat opens with; the switcher still lists every model.
+3. **A per-model budget does not exist yet.** Someone who switches the dropdown to
+   `phi4-mini` on a long recording gets a loud HTTP 400, shown inline as "couldn't answer".
+   That fails honestly but unhelpfully. Filed as a follow-up.
+4. **The "retrieval chat" trigger in ADR-033 is now concrete:** any recording past 90,000 chars
+   (≈100 min) cannot be answered by stuffing, for any offered model.
+
+## Method
+
+`POST /v1/chat/completions` through `http://koala:30401/v1`, `temperature: 0`,
+`max_tokens: 200` (production sends 1500 for chat via the summary token cap), the same system
+framing as `chat.Service`. The transcript is the 60-minute synthetic fixture from
+`audio-length-ceiling-2026-10-01.md` truncated/repeated to N chars, with one unique sentence
+planted either at the **end** ("lagret på Kvarngatan ska tömmas … av Pelle", question: who/when)
+or at the **head** ("kvällens ordförande är Ingrid", question: who chairs). Needle at the
+end alone would have missed the gemma failure — a serving layer that truncates from the front
+keeps the tail, so the end-needle passed at 120,000 chars; only the head-needle exposed it.
+Grounded = the answer contains the planted fact.
+
+## What this did not measure
+
+- **Real speech**: the fixture is a repeated 82-second script, so retrieval "difficulty" is
+  artificially easy except for the planted needle. One question per run, one run per cell.
+- **`berget/mistral-small`'s failure**: it missed an end-needle at 18,000 chars where the
+  others succeeded. One run; could be the model, the prompt framing, or position bias. Not
+  investigated; do not read it as a ranking.
+- **Latency under load**, **concurrent** chat while whisper is transcribing, and in particular
+  the hazard below.
+- **`koala/qwen36-35b-a3b` or any frontier model** — not in the offered list.
+
+## A hazard this surfaces and does not solve
+
+`iguana/gemma4-26b` is a large ollama model on the same host as `iguana/kb-whisper`. infra#273
+is precisely that a large resident ollama model can Metal-OOM whisper-server permanently. Making
+gemma the default *chat* model for uploads means chatting with a transcript can load the model
+that wedges the transcriber. The primary-fails-over-to-berget design limits the blast radius but
+does not remove it. Recorded in ADR-033 and filed.