feat(summarizer): move local fallback off koala to iguana/gemma4-26b
koala now carries other GPU loads, so the first fallback should not run there. Change the default chain to koala/phi4-mini → iguana/gemma4-26b → berget/mistral-small: the local fallback now runs on iguana (M2 Ultra headroom, different host = different egress IP for the rare fallback fetch). gemma4-26b is the brain-validated homelab general-purpose model (agentsquad H2/H3 executor) and returned valid summary JSON on the real prompt in a smoke test (~37s incl. cold-load — fine for a fallback path). Pure config default (TAPIR_FALLBACK_MODEL); chain mechanism (ADR-022) unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -33,7 +33,10 @@ type Config struct {
|
||||
SummarizerModel string
|
||||
// FallbackModel is the LOCAL fallback alias tried when the primary fails or
|
||||
// returns unparseable output (ADR-022). Kept local so content stays on the
|
||||
// homelab stack. Empty disables it. Default a bigger-context local model.
|
||||
// homelab stack. Default is an IGUANA model (not koala) so the fallback runs
|
||||
// on a different host than the koala primary — koala carries other loads, and
|
||||
// a different host also means a different egress IP for the (rare) fallback.
|
||||
// Empty disables it.
|
||||
FallbackModel string
|
||||
// CloudFallbackModel is the worst-case EXTERNAL fallback alias, tried only
|
||||
// after every local endpoint has failed (ADR-022). For client deployments set
|
||||
@@ -148,7 +151,7 @@ func (c Config) DexConfigured() bool { return strings.TrimSpace(c.OIDCIssuer) !=
|
||||
const (
|
||||
defaultGatewayURL = "http://koala:30401/v1"
|
||||
defaultSummarizerModel = "koala/phi4-mini"
|
||||
defaultFallbackModel = "koala/phi4-14b"
|
||||
defaultFallbackModel = "iguana/gemma4-26b"
|
||||
defaultCloudFallbackModel = "berget/mistral-small"
|
||||
defaultSummaryMaxTokens = 1500
|
||||
defaultMaxTranscriptChars = 18000
|
||||
|
||||
Reference in New Issue
Block a user