feat(youtube): acquire captions via player/timedtext baseUrl (ADR-010)
CI / Lint / Test / Vet (push) Successful in 6s
CI / Build & Import (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped

The Data API captions.download endpoint is owner-only: every subscription
video the user does not own returned HTTP 403, producing 0 summaries and a
~150-line error spew in the first live Stage-0 run. Captions-first (ADR-007)
is sound; only the acquisition mechanism was wrong.

FetchTranscript now resolves caption tracks from the InnerTube player
response (ANDROID client, unauthenticated) and GETs the chosen track's
timedtext baseUrl with a plain http.Client — no OAuth token, which can break
the endpoint. The srv3 XML, json3, and legacy <transcript> formats all parse;
non-asr tracks in a preferred language win. Watch-page ytInitialPlayerResponse
scrape is the fallback when InnerTube returns no tracks.

Degrade, don't error (explicit quick-fix): no captionTracks, empty baseUrl, a
non-200 fetch, or an unparseable body yield Source=none, not an error. Only
genuine transport faults error — this kills the spew. OAuth stays on
ListSubscriptions/NewVideos (Data API); only transcript fetch goes unauthed.

Validated live from koala: the ANDROID client returned working baseUrls and
real transcript text for public videos the run identity does not own.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-02 23:08:13 +02:00
co-authored by Claude Opus 4.8
parent fa16a62e6d
commit 8b14ef4add
4 changed files with 530 additions and 127 deletions
+51 -1
View File
@@ -120,7 +120,8 @@ secret surface (see VISION Stage 2); it gets the existing, vetted secret path.
## ADR-007 — Captions-first; audio-download + speech-to-text deferred
**Status:** Accepted (2026-06-02)
**Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API
`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand.
**Context.** YouTube's Data API does not expose transcripts. Options: official captions
(clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey,
@@ -153,6 +154,55 @@ governs advancement. Reversible: if demand appears, a new ADR opens the Future C
---
## ADR-010 — Third-party caption acquisition via the timedtext/player baseUrl
**Status:** Accepted (2026-06-02)
**Context.** ADR-007 settled *captions-first*. The first build used the YouTube Data API
`captions.list` + `captions.download` endpoints to acquire them. A live Stage-0 run proved that
`captions.download` is **owner-only**: it requires the OAuth identity to own the video, so every
subscription video the user does *not* own returns HTTP 403. Result: 0 summaries produced and a
~150-line error spew. The captions-first decision is sound; only the *acquisition mechanism* was
wrong.
**Decision.** Acquire captions from the **player response + timedtext baseUrl**, not the Data API
`captions` endpoints:
1. `POST https://www.youtube.com/youtubei/v1/player` with an **InnerTube `ANDROID` client
context** (no API key, no OAuth). Read
`captions.playerCaptionsTracklistRenderer.captionTracks[]`. Each track carries `baseUrl`,
`languageCode`, and `kind` (`"asr"` = auto-generated).
2. Select by `PreferredLanguages`, preferring non-`asr` when both exist.
3. **GET the track's `baseUrl` unauthenticated** (plain `http.Client`, no OAuth token attached —
the token can break the timedtext endpoint). The ANDROID `baseUrl` is pinned to `fmt=srv3`
(timedtext XML); the parser also accepts `json3` and the legacy `<transcript>` XML.
A **watch-page scrape** of `ytInitialPlayerResponse` is the documented fallback if InnerTube
returns no `captionTracks`.
**Live validation (from koala, 2026-06-02):** the `ANDROID` InnerTube client returned 6
`captionTracks` with working `baseUrl`s for a public video the run identity does not own, and the
unauthenticated `baseUrl` GET returned real transcript text. `ANDROID` is the client of record
(historically returns baseUrls without a PoToken). The `WEB` client and watch-page scrape are
fallbacks.
**Consequences.**
- Works for **any public captioned video**, not just owned ones — this is the fix for the 403 wall.
- **ToS-grey:** `youtubei`/`timedtext` are unofficial endpoints. They can break when Google shifts
InnerTube client requirements or introduces PoToken gating. Mitigation: degrade, never error — a
missing/empty/403/unparseable caption yields `domain.Transcript{Source: SourceNone}`, so a future
breakage produces "no transcript" rather than a crash or error spew. Only genuine transport
(network) faults error.
- **No OAuth** is needed for the transcript fetch. OAuth is still required for `ListSubscriptions`
and `NewVideos` (Data API) — only the transcript path goes unauthenticated.
- **Still no Whisper.** Speech-to-text stays deferred (ADR-007 unchanged).
**Supersedes:** the *acquisition mechanism* of ADR-007 (Data API `captions.download`
player/timedtext baseUrl) only. ADR-007's captions-first stance and the STT deferral stand.
---
## ADR-009 — Trunk-Based Development
**Status:** Accepted (2026-06-02)