feat(youtube): process-wide caption-fetch rate gate (ADR-014 item 2)
ADR-014 item 2 — a single per-egress-IP rate gate shared by every caption fetch — was specced but only per-video backoff (rate_limited_at) shipped. Build the real gate now: it is load-bearing once ADR-018 puts auto-summarize on an in-process schedule across multiple users (all fetches leave one pod's egress IP, concurrently with live "Summarize" clicks — without a shared gate that self-inflicts 429s every cycle). globalFetchGate (golang.org/x/time/rate, default 2s/req burst 1) is consulted in httpDo before every live outbound fetch — player, watch-page, timedtext — so the scheduler runners and the web click-path serialise through one limiter regardless of how many users/goroutines are upstream. The test seam (a.transport != nil) skips the gate so fakes are not throttled. TAPIR_FETCH_RATE (Go duration, default 2s, 0 = unlimited) wires SetFetchRate in cmdRun; the existing per-video backoff stays as the complementary 429 handler. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -235,6 +235,15 @@ func (a *Adapter) httpDo(ctx context.Context, client *http.Client, method, url s
|
||||
for k, v := range headers {
|
||||
req.Header.Set(k, v)
|
||||
}
|
||||
// Process-wide rate gate (ADR-014 item 2): every live caption fetch — player,
|
||||
// watch-page, and timedtext baseUrl — passes the shared per-egress-IP limiter
|
||||
// so the scheduler and the click-path cannot collectively trip 429s. Skipped
|
||||
// when a.transport is set (the test seam) so fakes are not throttled.
|
||||
if a.transport == nil {
|
||||
if err := WaitFetchGate(ctx); err != nil {
|
||||
return nil, 0, fmt.Errorf("fetch gate %s %s: %w", method, url, err)
|
||||
}
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("%s %s: %w", method, url, err)
|
||||
|
||||
Reference in New Issue
Block a user