feat(summarizer): resilient endpoint chain with local→cloud fallback (ADR-022)
The first friendly-pilot live run produced zero summaries: koala/phi4-mini hit three silent failure modes — 8k context overflow on long transcripts (HTTP 400), intermittent malformed JSON (highlights as a bare string), and no fallback wired at all (summarizer.New(primary, nil)). Keep phi4-mini as the fast primary and add resilience around it: - Ordered endpoint chain (summarizer.NewChain): phi4-mini → koala/phi4-14b (local) → berget/mistral-small (worst-case external). All reached through the one LiteLLM gateway by alias. - A parse failure now advances the chain like a transport error — the old Primary→Fallback shape returned the parse error without trying anyone else. - Tolerant parse: highlights/takeaways coerce string→[]string, absorbing the common small-model quirk without spending a fallback round-trip. - Transcript truncation (TAPIR_MAX_TRANSCRIPT_CHARS=18000) prevents the overflow rather than recovering from it; validated to fit phi4-mini's 8k window. - Bounded completion budget (TAPIR_SUMMARY_MAX_TOKENS=1500) — the old 8192 budget itself contributed to the overflow. Local-first guarantee preserved by ordering: external endpoint is tried only after every local one fails. TAPIR_CLOUD_FALLBACK_MODEL="" disables it entirely for client/NDA deployments. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,10 +26,11 @@ import (
|
||||
// longer matches a real non-pending scenario.
|
||||
var scenarioCoverage = map[string]string{
|
||||
// ai_routing.feature
|
||||
"Local AI produces the summary": "TestSummarize_LocalSucceeds",
|
||||
"Local AI fails and the user has a BYO provider configured": "TestSummarize_FallsBackToBYO",
|
||||
"Local AI fails and the user has no BYO provider": "TestSummarize_LocalFailsNoBYO_NoExternalSend",
|
||||
"A user without BYO never has content sent externally": "TestSummarize_NoBYO_ContentOnlyLocal",
|
||||
"Local AI produces the summary": "TestSummarize_LocalSucceeds",
|
||||
"Local AI fails and the user has a BYO provider configured": "TestSummarize_FallsBackToBYO",
|
||||
"Local AI fails and the user has no BYO provider": "TestSummarize_LocalFailsNoBYO_NoExternalSend",
|
||||
"A user without BYO never has content sent externally": "TestSummarize_NoBYO_ContentOnlyLocal",
|
||||
"A model returns unparseable output and the next endpoint succeeds": "TestSummarize_FallsBackOnMalformedOutput",
|
||||
|
||||
// landing_page.feature
|
||||
"An unauthenticated visit to the root is sent to the welcome page": "TestUnauthenticatedRootRedirectsToWelcome",
|
||||
|
||||
Reference in New Issue
Block a user