0 Releases 36 Tags

RSS Feed

• v0.15.0 e9b5a3f3e7

  Compare

  feat(summarizer): resilient endpoint chain with local→cloud fallback (ADR-022)

  mathias released this 2026-06-10 06:36:16 +00:00 | 50 commits to main since this release

  The first friendly-pilot live run produced zero summaries: koala/phi4-mini hit
  three silent failure modes — 8k context overflow on long transcripts (HTTP 400),
  intermittent malformed JSON (highlights as a bare string), and no fallback wired
  at all (summarizer.New(primary, nil)).

  Keep phi4-mini as the fast primary and add resilience around it:

  • Ordered endpoint chain (summarizer.NewChain): phi4-mini → koala/phi4-14b
    (local) → berget/mistral-small (worst-case external). All reached through the
    one LiteLLM gateway by alias.
  • A parse failure now advances the chain like a transport error — the old
    Primary→Fallback shape returned the parse error without trying anyone else.
  • Tolerant parse: highlights/takeaways coerce string→[]string, absorbing the
    common small-model quirk without spending a fallback round-trip.
  • Transcript truncation (TAPIR_MAX_TRANSCRIPT_CHARS=18000) prevents the overflow
    rather than recovering from it; validated to fit phi4-mini's 8k window.
  • Bounded completion budget (TAPIR_SUMMARY_MAX_TOKENS=1500) — the old 8192 budget
    itself contributed to the overflow.

  Local-first guarantee preserved by ordering: external endpoint is tried only
  after every local one fails. TAPIR_CLOUD_FALLBACK_MODEL="" disables it entirely
  for client/NDA deployments.

  Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

  Downloads

  • Source Code (ZIP)
  • Source Code (TAR.GZ)