chat: dedicated completion-token budget (TAPIR_CHAT_MAX_TOKENS) #9

Open
opened 2026-06-11 07:58:40 +00:00 by mathias · 0 comments
Owner

Context

The per-video chat (ADR-027, v0.23.0) builds its llm.Client per model with
llm.WithMaxTokens(cfg.SummaryMaxTokens) (default 1500) — reusing the
summarizer's completion budget to avoid the same small-context overflow trap
(ADR-022: prompt + max_tokens must fit phi4-mini's 8k window).

Problem

A summary needs only a few hundred tokens, so 1500 is right for it. A chat answer
(especially an explanatory multi-paragraph reply) can plausibly want more, and
1500 may clip it mid-sentence. Chat and summarization are different shapes sharing
one budget knob today.

Proposal

  • Add TAPIR_CHAT_MAX_TOKENS (own default, e.g. 1024–2048), wired only into the
    chat client factory (buildChat, cmd/tapir/processor.go).
  • Keep the overflow guard in mind: with the transcript truncated to
    TAPIR_MAX_TRANSCRIPT_CHARS (18000 ≈ ~4.5k tokens) plus the chat budget, the
    total must still fit the smallest offered model's context. Document the budget
    arithmetic next to the knob.

Scope

Pure config + wiring. No engine change. Low priority — surfaces only on long
answers from the smallest model.

Source: ADR-027, internal/adapters/chat, cmd/tapir/processor.go#buildChat.

## Context The per-video chat (ADR-027, v0.23.0) builds its `llm.Client` per model with `llm.WithMaxTokens(cfg.SummaryMaxTokens)` (default **1500**) — reusing the summarizer's completion budget to avoid the same small-context overflow trap (ADR-022: prompt + max_tokens must fit phi4-mini's 8k window). ## Problem A summary needs only a few hundred tokens, so 1500 is right for it. A chat answer (especially an explanatory multi-paragraph reply) can plausibly want more, and 1500 may clip it mid-sentence. Chat and summarization are different shapes sharing one budget knob today. ## Proposal - Add `TAPIR_CHAT_MAX_TOKENS` (own default, e.g. 1024–2048), wired only into the chat client factory (`buildChat`, `cmd/tapir/processor.go`). - Keep the overflow guard in mind: with the transcript truncated to `TAPIR_MAX_TRANSCRIPT_CHARS` (18000 ≈ ~4.5k tokens) plus the chat budget, the total must still fit the smallest offered model's context. Document the budget arithmetic next to the knob. ## Scope Pure config + wiring. No engine change. Low priority — surfaces only on long answers from the smallest model. Source: ADR-027, `internal/adapters/chat`, `cmd/tapir/processor.go#buildChat`.
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mathias/tapir#9