mathias/tapir
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(summarizer): move local fallback off koala to iguana/gemma4-26b
CI / Lint / Test / Vet (push) Successful in 10s
Details
CI / Build & Import (push) Successful in 10s
Details

Browse Source 
koala now carries other GPU loads, so the first fallback should not run there.
Change the default chain to koala/phi4-mini → iguana/gemma4-26b → berget/mistral-small:
the local fallback now runs on iguana (M2 Ultra headroom, different host = different
egress IP for the rare fallback fetch). gemma4-26b is the brain-validated homelab
general-purpose model (agentsquad H2/H3 executor) and returned valid summary JSON on
the real prompt in a smoke test (~37s incl. cold-load — fine for a fallback path).

Pure config default (TAPIR_FALLBACK_MODEL); chain mechanism (ADR-022) unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
mathias
2026-06-11 07:54:56 +02:00
co-authored by Claude Opus 4.8
parent fe56e2fe01
commit a9be5f285b
3 changed files with 20 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
DECISIONS.md
+12 -4
View File
@@ -827,10 +827,18 @@ The fix is resilience around it, not replacing it.
**Decision.**
1. **Ordered endpoint chain (`summarizer.NewChain`).** Endpoints are tried in order; the first to
return a *parseable* summary wins. Default chain:
`koala/phi4-mini` (primary, local) → `koala/phi4-14b` (fallback, local) →
`berget/mistral-small` (worst-case, external). All three are reached through the **one** LiteLLM
gateway by alias — the gateway already fronts both llama-swap and berget — so a fallback is a
different alias, not a second client config.
`koala/phi4-mini` (primary, local) → `iguana/gemma4-26b` (fallback, local on a
*different host*) → `berget/mistral-small` (worst-case, external). All three are reached through
the **one** LiteLLM gateway by alias — the gateway already fronts both llama-swap and berget — so
a fallback is a different alias, not a second client config.
**Update 2026-06-11:** the local fallback moved from `koala/phi4-14b` to `iguana/gemma4-26b`.
koala now carries other GPU loads, so keeping the fallback on koala competed with them; iguana
(M2 Ultra) has the headroom, and a different host is also a different egress IP for the rare
fallback fetch. `gemma4-26b` is the brain-validated homelab general-purpose model (agentsquad
H2/H3 executor) and returned valid summary JSON on the real prompt in a smoke test
(~37s incl. cold-load — fine for a path hit only when the fast primary fails). Pure config:
`TAPIR_FALLBACK_MODEL`.
2. **A parse failure advances the chain, same as a transport error.** "Reliably summarized" means
*parseable summary returned*, not *HTTP 200*. This is the behaviour the old Primary→Fallback
shape missed.