mathias/tapir
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(runner): 429 backoff — skip still-throttled videos, persist status

Browse Source 
After ProcessNewVideo the runner records transcript_status per outcome:
rate_limited (stamps the backoff clock), none, or fetched. Before fetching, a
video inside the TAPIR_FETCH_BACKOFF window is skipped (SkippedRateLimited) so a
just-429'd caption endpoint is not re-hit; once the window expires it retries.

Backoff/clock injected via variadic Options (WithBackoff, WithClock) so existing
New call sites and the fake-driven loop tests stay valid. Backoff 0 = always retry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
mathias
2026-06-03 22:56:07 +02:00
co-authored by Claude Opus 4.8
parent ce2fc62ef8
commit 40a614c8d4
3 changed files with 163 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
cmd/tapir/main.go
+2 -2
View File
@@ -125,10 +125,10 @@ func cmdRun(ctx context.Context, log *slog.Logger) error {
if engine == nil {
return fmt.Errorf("run: incomplete summarization config (gateway, youtube credentials, secrets file)")
}
r := runner.New(engine.Source, st, engine, cfg.UserID, log)
r := runner.New(engine.Source, st, engine, cfg.UserID, log, runner.WithBackoff(cfg.FetchBackoff))
log.Info("starting run", "user", cfg.UserID, "model", cfg.SummarizerModel,
"gateway", cfg.GatewayURL, "poll_interval", cfg.PollInterval)
"gateway", cfg.GatewayURL, "poll_interval", cfg.PollInterval, "fetch_backoff", cfg.FetchBackoff)
return r.Loop(ctx, cfg.PollInterval)
}