Follow-up to #33 / ADR-033. Measured in docs/research/chat-context-over-long-transcripts-2026-10-01.md.
chat.Service stuffs the transcript into the prompt; it does not retrieve (#33 assumed it did). Measured against the models chat offers:
koala/phi4-mini — correct to 30k chars, HTTP 400 at 50k (8,192-token context). First in the offered list, i.e. the default.
iguana/gemma4-26b — correct to 90,000 chars; at 120,000 it silently drops the head: no error, Truncated stays false, answers "not in the text" about a sentence at the start.
berget/mistral-small — missed a fact at the end of an 18k and a 50k transcript (one run each; unexplained).
PR ships: upload chat opens on TAPIR_UPLOAD_CHAT_MODEL (gemma4-26b) with a 90,000-char budget. Not built:
Per-model budgets. Switching the dropdown to phi4-mini on a long recording gives a loud 400 shown inline as "couldn't answer". Honest but unhelpful; a per-model cap (or hiding models that cannot hold the transcript) would fix it.
A guard against silent truncation. The serving layer can cut the prompt from the front without telling chat.Service. Compare the response's usage.prompt_tokens with an estimate and surface "the model did not see the whole transcript".
Retrieval / chunked chat — required for anything past ~90,000 chars (≈100 min at the fixture's density). Trigger per ADR-033: the duration cap is raised above that, a real recording is denser than the fixture, or the model list changes and the ceiling is re-measured lower.
Method note worth keeping: a needle at the end passes against a front-truncating server; only a needle at the head exposed the gemma failure.
Follow-up to #33 / ADR-033. Measured in `docs/research/chat-context-over-long-transcripts-2026-10-01.md`.
`chat.Service` stuffs the transcript into the prompt; it does **not** retrieve (#33 assumed it did). Measured against the models chat offers:
- `koala/phi4-mini` — correct to 30k chars, **HTTP 400 at 50k** (8,192-token context). First in the offered list, i.e. the default.
- `iguana/gemma4-26b` — correct to **90,000 chars**; at **120,000 it silently drops the head**: no error, `Truncated` stays false, answers "not in the text" about a sentence at the start.
- `berget/mistral-small` — missed a fact at the *end* of an 18k and a 50k transcript (one run each; unexplained).
PR ships: upload chat opens on `TAPIR_UPLOAD_CHAT_MODEL` (gemma4-26b) with a 90,000-char budget. **Not built:**
1. **Per-model budgets.** Switching the dropdown to `phi4-mini` on a long recording gives a loud 400 shown inline as "couldn't answer". Honest but unhelpful; a per-model cap (or hiding models that cannot hold the transcript) would fix it.
2. **A guard against silent truncation.** The serving layer can cut the prompt from the front without telling `chat.Service`. Compare the response's `usage.prompt_tokens` with an estimate and surface "the model did not see the whole transcript".
3. **Retrieval / chunked chat** — required for anything past ~90,000 chars (≈100 min at the fixture's density). Trigger per ADR-033: the duration cap is raised above that, a real recording is denser than the fixture, or the model list changes and the ceiling is re-measured lower.
Method note worth keeping: a needle at the *end* passes against a front-truncating server; only a needle at the *head* exposed the gemma failure.
Refs: #33, ADR-033, ADR-027.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Follow-up to #33 / ADR-033. Measured in
docs/research/chat-context-over-long-transcripts-2026-10-01.md.chat.Servicestuffs the transcript into the prompt; it does not retrieve (#33 assumed it did). Measured against the models chat offers:koala/phi4-mini— correct to 30k chars, HTTP 400 at 50k (8,192-token context). First in the offered list, i.e. the default.iguana/gemma4-26b— correct to 90,000 chars; at 120,000 it silently drops the head: no error,Truncatedstays false, answers "not in the text" about a sentence at the start.berget/mistral-small— missed a fact at the end of an 18k and a 50k transcript (one run each; unexplained).PR ships: upload chat opens on
TAPIR_UPLOAD_CHAT_MODEL(gemma4-26b) with a 90,000-char budget. Not built:phi4-minion a long recording gives a loud 400 shown inline as "couldn't answer". Honest but unhelpful; a per-model cap (or hiding models that cannot hold the transcript) would fix it.chat.Service. Compare the response'susage.prompt_tokenswith an estimate and surface "the model did not see the whole transcript".Method note worth keeping: a needle at the end passes against a front-truncating server; only a needle at the head exposed the gemma failure.
Refs: #33, ADR-033, ADR-027.
Created via git-mcp on behalf of @mathias