feat(youtube): process-wide caption-fetch rate gate (ADR-014 item 2)

ADR-014 item 2 — a single per-egress-IP rate gate shared by every caption
fetch — was specced but only per-video backoff (rate_limited_at) shipped.
Build the real gate now: it is load-bearing once ADR-018 puts auto-summarize
on an in-process schedule across multiple users (all fetches leave one pod's
egress IP, concurrently with live "Summarize" clicks — without a shared gate
that self-inflicts 429s every cycle).

globalFetchGate (golang.org/x/time/rate, default 2s/req burst 1) is consulted
in httpDo before every live outbound fetch — player, watch-page, timedtext —
so the scheduler runners and the web click-path serialise through one limiter
regardless of how many users/goroutines are upstream. The test seam
(a.transport != nil) skips the gate so fakes are not throttled.

TAPIR_FETCH_RATE (Go duration, default 2s, 0 = unlimited) wires SetFetchRate in
cmdRun; the existing per-video backoff stays as the complementary 429 handler.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 23:36:17 +02:00
co-authored by Claude Opus 4.8
parent 88294d38bc
commit f5021a8436
8 changed files with 176 additions and 1 deletions
+9
View File
@@ -235,6 +235,15 @@ func (a *Adapter) httpDo(ctx context.Context, client *http.Client, method, url s
for k, v := range headers {
req.Header.Set(k, v)
}
// Process-wide rate gate (ADR-014 item 2): every live caption fetch — player,
// watch-page, and timedtext baseUrl — passes the shared per-egress-IP limiter
// so the scheduler and the click-path cannot collectively trip 429s. Skipped
// when a.transport is set (the test seam) so fakes are not throttled.
if a.transport == nil {
if err := WaitFetchGate(ctx); err != nil {
return nil, 0, fmt.Errorf("fetch gate %s %s: %w", method, url, err)
}
}
resp, err := client.Do(req)
if err != nil {
return nil, 0, fmt.Errorf("%s %s: %w", method, url, err)