feat(youtube): process-wide caption-fetch rate gate (ADR-014 item 2)

ADR-014 item 2 — a single per-egress-IP rate gate shared by every caption
fetch — was specced but only per-video backoff (rate_limited_at) shipped.
Build the real gate now: it is load-bearing once ADR-018 puts auto-summarize
on an in-process schedule across multiple users (all fetches leave one pod's
egress IP, concurrently with live "Summarize" clicks — without a shared gate
that self-inflicts 429s every cycle).

globalFetchGate (golang.org/x/time/rate, default 2s/req burst 1) is consulted
in httpDo before every live outbound fetch — player, watch-page, timedtext —
so the scheduler runners and the web click-path serialise through one limiter
regardless of how many users/goroutines are upstream. The test seam
(a.transport != nil) skips the gate so fakes are not throttled.

TAPIR_FETCH_RATE (Go duration, default 2s, 0 = unlimited) wires SetFetchRate in
cmdRun; the existing per-video backoff stays as the complementary 429 handler.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 23:36:17 +02:00
co-authored by Claude Opus 4.8
parent 88294d38bc
commit f5021a8436
8 changed files with 176 additions and 1 deletions
+24
View File
@@ -65,6 +65,17 @@ type Config struct {
// expires the video is retried. Zero means "always retry" (no backoff).
FetchBackoff time.Duration
// FetchRate is the minimum interval between outbound caption fetches across the
// whole process — the shared per-egress-IP rate gate (ADR-014 item 2). It is
// the gate that makes auto-summarize-on-a-schedule safe: scheduler runners and
// the web click-path serialise through it. Zero = unlimited (dev/tests).
FetchRate time.Duration
// DiscoveryInterval, when > 0, makes `serve` run in-process scheduled discovery
// for ALL users on that cadence (ADR-018). Zero/unset = disabled, so dev and
// tests never auto-fetch. Single-replica assumption — see cmdServe.
DiscoveryInterval time.Duration
// HTTPAddr is the listen address for `tapir serve` (the Stage-0 web UI).
HTTPAddr string
@@ -97,6 +108,7 @@ const (
defaultOAuthRedirectAddr = "localhost:8080"
defaultHTTPAddr = ":8080"
defaultFetchBackoff = time.Hour
defaultFetchRate = 2 * time.Second
defaultPublicURL = "https://tapir.d-ma.be"
)
@@ -144,6 +156,18 @@ func Load() (Config, error) {
}
c.FetchBackoff = backoff
fetchRate, err := durationOr("TAPIR_FETCH_RATE", defaultFetchRate)
if err != nil {
return Config{}, err
}
c.FetchRate = fetchRate
discovery, err := durationOr("TAPIR_DISCOVERY_INTERVAL", 0)
if err != nil {
return Config{}, err
}
c.DiscoveryInterval = discovery
return c, nil
}