fix(llm): send generous max_tokens on every request

The copied OpenAI-compatible client sent no max_tokens. Thinking models
(qwen3, deepseek-r1) spend their budget on the reasoning trace and return
EMPTY content when max_tokens is unset, which the summarizer treats as an
error. ADR-004 says change Tapir's copy rather than the hyperguild upstream,
so set a generous default (8192) leaving room for both reasoning and output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-02 20:55:49 +02:00
co-authored by Claude Opus 4.8
parent 2695b5d91e
commit c40b46b661
2 changed files with 33 additions and 0 deletions
+12
View File
@@ -17,11 +17,20 @@ import (
"time"
)
// defaultMaxTokens is sent on every request. Tapir CHANGES this from the
// hyperguild copy (ADR-004 says change the copy, not the upstream): thinking
// models (qwen3, deepseek-r1) spend their budget on reasoning and return EMPTY
// content when max_tokens is unset or too low. A generous ceiling leaves room
// for both the reasoning trace and the actual summary. See
// docs/homelab-integration.md.
const defaultMaxTokens = 8192
// Client calls an OpenAI-compatible chat completions endpoint.
type Client struct {
baseURL string
apiKey string
model string
maxTokens int
httpClient *http.Client
}
@@ -31,6 +40,7 @@ func New(baseURL, apiKey, model string, timeout time.Duration) *Client {
baseURL: strings.TrimRight(baseURL, "/"),
apiKey: apiKey,
model: model,
maxTokens: defaultMaxTokens,
httpClient: &http.Client{Timeout: timeout},
}
}
@@ -39,6 +49,7 @@ type chatRequest struct {
Model string `json:"model"`
Messages []message `json:"messages"`
Temperature float64 `json:"temperature"`
MaxTokens int `json:"max_tokens,omitempty"`
}
type message struct {
@@ -62,6 +73,7 @@ func (c *Client) Complete(ctx context.Context, system, user string) (string, err
{Role: "user", Content: user},
},
Temperature: 0.2,
MaxTokens: c.maxTokens,
}
b, err := json.Marshal(body)
if err != nil {