feat(discovery): drop Shorts and livestreams before the caption fetch (ADR-023)
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s

The scarce resource is the per-IP timedtext caption fetch (ADR-014); the pilot's
candidate set was mostly Shorts/clips/livestreams, each burning a fetch (a "none"
result is a completed fetch — it costs budget even when it yields nothing).

NewVideos now enriches candidates with one cheap Data API videos.list call
(contentDetails.duration + snippet.liveBroadcastContent — the quota API, a
DIFFERENT limit from the timedtext 429) and drops, before returning: videos
shorter than TAPIR_MIN_VIDEO_SECONDS (default 60) and any live/upcoming
broadcast. Dropped videos are never persisted, so the list declutters too.

Degrade-open: MinVideoSeconds=0 disables it (no quota call); a videos.list error
returns candidates unfiltered so discovery never breaks on a metadata hiccup. The
paste-a-URL path (VideoByID) is not filtered — an explicit request is honoured.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-10 19:34:45 +02:00
co-authored by Claude Opus 4.8
parent 1aa8a97f95
commit 9db06d8a63
7 changed files with 250 additions and 4 deletions
+36
View File
@@ -855,6 +855,42 @@ reply still parses; a transcript within budget is unchanged).
---
## ADR-023 — Drop Shorts/livestreams at discovery to protect the caption budget
**Status:** Accepted (2026-06-10). **Builds on ADR-014** (per-IP caption rate limit is the
binding constraint) and the ADR-022 live-run findings.
**Context.** The scarce resource is the unofficial timedtext caption fetch (per-egress-IP
429, ~3 successful/pass). The first multi-user run showed the candidate set was mostly noise —
Shorts, sub-minute clips, and live broadcasts — each of which still consumes a caption-fetch
attempt (and a "none" result is a *completed* fetch, so it costs budget even when it yields
nothing). Spending the rate-limited budget on content the user will not read is the waste to
cut first; it is cheaper and lower-risk than raising the ceiling (multi-IP, Whisper).
Duration and live status are NOT in the playlistItems discovery response, but they ARE in the
Data API `videos.list` (contentDetails.duration + snippet.liveBroadcastContent) — the official
**quota-based** API (1 unit/call, 50 ids/call), which is a *different* limit from the timedtext
429. So one cheap quota call buys a filter that saves many expensive throttled fetches.
**Decision.**
1. `NewVideos` enriches its candidates with a single `videos.list` call and drops, before
returning: videos shorter than `TAPIR_MIN_VIDEO_SECONDS` (default 60) and any `live`/
`upcoming` broadcast. Dropped videos are never persisted, so they also declutter the list.
2. The filter is **degrade-open**: `MinVideoSeconds=0` disables it (no quota call), and a
`videos.list` error returns the candidates unfiltered — discovery must never break because a
metadata call hiccuped (worst case = pre-ADR-023 behaviour).
3. The paste-a-URL path (`VideoByID`) is **not** filtered — an explicit user request for a
specific video (even a Short) is honoured.
**Reversibility.** Pure discovery-time filter + config. `TAPIR_MIN_VIDEO_SECONDS=0` restores
the old behaviour. No schema change, no effect on already-stored videos.
**Quota note.** Per-channel enrichment adds ~1 unit/channel/pass. At pilot scale (≤3 users)
this is well under the 10k/day cap; at larger scale, batch `videos.list` across channels
(50 ids/call) by collecting all discovered ids per pass before enriching.
---
## Rejected alternatives
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not