Files
tapir/DECISIONS.md
T
mathiasandClaude Opus 4.8 8b14ef4add
CI / Lint / Test / Vet (push) Successful in 6s
CI / Build & Import (push) Failing after 1s
CI / Mirror to GitHub (push) Has been skipped
feat(youtube): acquire captions via player/timedtext baseUrl (ADR-010)
The Data API captions.download endpoint is owner-only: every subscription
video the user does not own returned HTTP 403, producing 0 summaries and a
~150-line error spew in the first live Stage-0 run. Captions-first (ADR-007)
is sound; only the acquisition mechanism was wrong.

FetchTranscript now resolves caption tracks from the InnerTube player
response (ANDROID client, unauthenticated) and GETs the chosen track's
timedtext baseUrl with a plain http.Client — no OAuth token, which can break
the endpoint. The srv3 XML, json3, and legacy <transcript> formats all parse;
non-asr tracks in a preferred language win. Watch-page ytInitialPlayerResponse
scrape is the fallback when InnerTube returns no tracks.

Degrade, don't error (explicit quick-fix): no captionTracks, empty baseUrl, a
non-200 fetch, or an unparseable body yield Source=none, not an error. Only
genuine transport faults error — this kills the spew. OAuth stays on
ListSubscriptions/NewVideos (Data API); only transcript fetch goes unauthed.

Validated live from koala: the ANDROID client returned working baseUrls and
real transcript text for public videos the run identity does not own.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 23:08:13 +02:00

13 KiB
Raw Blame History

Architecture Decision Records — Tapir

Lightweight ADRs. Each records a decision, its context, and its consequences at the time it was made. Decisions are append-only: to reverse one, add a new ADR that supersedes it (note the supersession in both). This is the reversibility record the VISION guardrails refer to.

Status values: Accepted · Superseded by ADR-NNN · Proposed.


ADR-001 — Go, not Python

Status: Accepted (2026-06-02)

Context. The first design draft assumed a Python/FastAPI/arq stack. The homelab is a Go estate (template-go-web, template-go-agent, mcp-chassis, hyperguild/ingestion). A Python service would be the only one of its kind, outside every shared convention, chassis, and CI pattern.

Decision. Tapir is written in Go. Python is used only where it makes no sense not to (e.g. if an ML component later genuinely requires it — none does today).

Consequences. Reuses homelab Go conventions and the mcp-chassis auth pattern if/when Tapir exposes an MCP surface. Closes the door on the Python async ecosystem (acceptable — Go's concurrency model fits the watcher/worker shape).


ADR-002 — No Supabase; reuse existing Dex / ESO / Postgres conventions

Status: Accepted (2026-06-02)

Context. The draft proposed self-hosted Supabase for auth + RLS + secrets (Vault). The homelab already runs Dex (OIDC), ESO + 1Password (secrets), and a postgres18 instance that the May-2026 architecture review is actively de-coupling by blast radius. Supabase would bundle a second auth system, a second secrets store, and its own Postgres — three duplications of things just consolidated.

Decision. No Supabase. Auth, secrets, and persistence reuse the homelab's existing primitives: Dex for identity (when external identity is needed), ESO + 1Password for secret custody, Postgres with per-tenant role/grant and the tenant= namespace label scheme for isolation when Future B/C arrives.

Consequences. No new infra to operate or back up. Multi-tenancy uses the architecture review's SC7/P6 primitives rather than Supabase RLS. For Stage 0 (single user) most of this is dormant; it activates at Stage 1.


ADR-003 — Standalone-first; brain is one sink behind an interface

Status: Accepted (2026-06-02)

Context. Tapir was initially framed as both a standalone service and a brain-ingestion pipeline. The maintainer clarified that standalone is the more important of the two.

Decision. Tapir is a standalone service. Delivery of summaries is via a Sink interface with multiple implementations. The user's own store is the primary sink; the brain is one optional sink among others. The engine does not know or care which sinks are attached.

Consequences. Brain ingestion can never dictate the core architecture (a named drift signal in VISION). Homelab mode is "the brain sink is enabled"; it is not a separate build.


ADR-004 — Copy the llm package from hyperguild/ingestion; do not lift to a shared lib

Status: Accepted (2026-06-02)

Context. Spike S5 (infra/docs/superpowers/handoffs/2026-06-02-video-adapter-placement.md) examined what Tapir could reuse from hyperguild/ingestion. The internal/llm package (Client + Router) is stdlib-only and its Router implements exactly the Primary→Fallback pattern Tapir needs for local-first → BYO-AI. It is ~3.5 KB and stable.

Decision. Copy the llm package into Tapir and own it. Do not lift it into a shared module. Lifting would couple Tapir's release cycle to the monolith to save ~100 lines — a bad trade against the "standalone owes nothing" bias.

Consequences. Minor duplication. Tapir owes nothing to hyperguild/ingestion at the dependency level. Primary = local stack via LiteLLM/piguard alias; Fallback = user BYO key.


ADR-005 — Brain sink is an HTTP brain-mcp client, not the filesystem brain package

Status: Accepted (2026-06-02)

Context. hyperguild/ingestion's internal/brain package manipulates the brain git checkout on the local filesystem (os.WriteFile into brain/wiki/...). That assumes co-location with the brain repo — fine for the monolith, wrong for a standalone service where brain is one network-reached sink.

Decision. Tapir's brain sink is a thin HTTP adapter calling brain-mcp's brain_ingest tool. It does not import or replicate the filesystem brain package.

Consequences. The brain sink works regardless of where Tapir runs. Adds a dependency on brain-mcp availability when that sink is enabled (acceptable; sinks fail independently).


ADR-006 — Outbound OAuth (YouTube/Vimeo) written fresh

Status: Accepted (2026-06-02)

Context. hyperguild/ingestion's internal/oauth package is the MCP server's inbound auth (client_credentials, authenticating claude.ai). Tapir needs outbound authorization-code OAuth to YouTube/Vimeo with per-user token custody — a different concern that happens to share a name.

Decision. Write the YouTube/Vimeo OAuth client fresh using golang.org/x/oauth2. Store per-user refresh tokens via the homelab's ESO/1Password convention (ADR-002), not a new secrets system.

Consequences. No reuse from the monolith here. Token custody is the highest-value secret surface (see VISION Stage 2); it gets the existing, vetted secret path.


ADR-007 — Captions-first; audio-download + speech-to-text deferred

Status: Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API captions.download → player/timedtext baseUrl); captions-first stance and STT deferral stand.

Context. YouTube's Data API does not expose transcripts. Options: official captions (clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey, GPU-contending with the JEPA PoC on koala, breakage-prone via yt-dlp).

Decision. The core path is captions-only. When a video has no usable transcript, Tapir records "no transcript" and moves on. Audio-download + speech-to-text is a deferred, clearly-bounded optional component, not part of the core path or Stage 0.

Consequences. Some videos won't be summarized at Stage 0 — accepted. The "is this useful to me?" hypothesis is testable on captioned videos alone. Avoids GPU contention and ToS risk in the validated path.


ADR-008 — Future C (public multi-tenant SaaS) deferred behind the Stage 0 gate

Status: Accepted (2026-06-02)

Context. "Real users soon" was initially asserted, which would load Google OAuth verification, billing, and hardened multi-tenant custody early. Lowered to Future B (15 trusted users), with the maintainer as the real first customer.

Decision. Build for Stage 0 then Stage 1 (Future B). Public SaaS machinery (sign-up, billing, Google OAuth app verification at scale) is not built until the Stage 0 self-use gate passes and real external demand appears.

Consequences. Smaller, faster first build. The staged Definition of Success in VISION governs advancement. Reversible: if demand appears, a new ADR opens the Future C scope.


ADR-010 — Third-party caption acquisition via the timedtext/player baseUrl

Status: Accepted (2026-06-02)

Context. ADR-007 settled captions-first. The first build used the YouTube Data API captions.list + captions.download endpoints to acquire them. A live Stage-0 run proved that captions.download is owner-only: it requires the OAuth identity to own the video, so every subscription video the user does not own returns HTTP 403. Result: 0 summaries produced and a ~150-line error spew. The captions-first decision is sound; only the acquisition mechanism was wrong.

Decision. Acquire captions from the player response + timedtext baseUrl, not the Data API captions endpoints:

  1. POST https://www.youtube.com/youtubei/v1/player with an InnerTube ANDROID client context (no API key, no OAuth). Read captions.playerCaptionsTracklistRenderer.captionTracks[]. Each track carries baseUrl, languageCode, and kind ("asr" = auto-generated).
  2. Select by PreferredLanguages, preferring non-asr when both exist.
  3. GET the track's baseUrl unauthenticated (plain http.Client, no OAuth token attached — the token can break the timedtext endpoint). The ANDROID baseUrl is pinned to fmt=srv3 (timedtext XML); the parser also accepts json3 and the legacy <transcript> XML.

A watch-page scrape of ytInitialPlayerResponse is the documented fallback if InnerTube returns no captionTracks.

Live validation (from koala, 2026-06-02): the ANDROID InnerTube client returned 6 captionTracks with working baseUrls for a public video the run identity does not own, and the unauthenticated baseUrl GET returned real transcript text. ANDROID is the client of record (historically returns baseUrls without a PoToken). The WEB client and watch-page scrape are fallbacks.

Consequences.

  • Works for any public captioned video, not just owned ones — this is the fix for the 403 wall.
  • ToS-grey: youtubei/timedtext are unofficial endpoints. They can break when Google shifts InnerTube client requirements or introduces PoToken gating. Mitigation: degrade, never error — a missing/empty/403/unparseable caption yields domain.Transcript{Source: SourceNone}, so a future breakage produces "no transcript" rather than a crash or error spew. Only genuine transport (network) faults error.
  • No OAuth is needed for the transcript fetch. OAuth is still required for ListSubscriptions and NewVideos (Data API) — only the transcript path goes unauthenticated.
  • Still no Whisper. Speech-to-text stays deferred (ADR-007 unchanged).

Supersedes: the acquisition mechanism of ADR-007 (Data API captions.download → player/timedtext baseUrl) only. ADR-007's captions-first stance and the STT deferral stand.


ADR-009 — Trunk-Based Development

Status: Accepted (2026-06-02)

Context. Platform-wide convention (homelab architecture review invariant; gitea-mcp #27): commit directly to main, one logical change per commit, every commit deployable.

Decision. Tapir follows TBD. Commit directly to main. No feature branches or PRs for solo/agent work; short-lived agent/<desc> branches only when parallel agents are active on the repo simultaneously. CI is the quality gate, not branch protection.

Consequences. Consistent with the rest of the estate. Depends on the direct-to-main write path tracked in gitea-mcp #35 (item #1).


Rejected alternatives

Approaches considered during the 2026-06-02 planning + grill session and deliberately not taken. Recorded so a later session doesn't re-propose them as if they were fresh ideas. Each maps to the ADR that settles it.

Rejected Why rejected Settled by
Python / FastAPI / arq stack Would be the only non-Go service in the estate, outside every shared chassis/convention/CI pattern ADR-001
Self-hosted Supabase (auth + RLS + Vault) Duplicates Dex, ESO+1Password, and the postgres instance being actively de-coupled; three new things to run and back up ADR-002
Living inside the hyperguild/ingestion monolith The only worthwhile reuse is ~100 lines (llm) that get copied anyway; standalone-first wants the service to owe the monolith nothing ADR-003, ADR-004
Lifting shared packages into a brain-common module Couples Tapir's release cycle to the monolith for negligible code savings ADR-004
Importing/replicating the filesystem brain package Assumes co-location with the brain git checkout; wrong for a standalone networked service ADR-005
Reusing ingestion's oauth package for YouTube/Vimeo Same name, opposite direction — it's inbound MCP-server auth, not outbound provider OAuth ADR-006
Global cross-tenant videos/transcripts table (dedup) Reintroduces the cross-domain DB coupling the homelab review is removing; at 15 users, re-summarizing is cheaper than the coupling data-model.md
Audio-download + Whisper STT in the core path ToS-grey, breakage-prone (yt-dlp), contends for koala GPU with the JEPA PoC; captions alone test the core hypothesis ADR-007
Building multi-tenant SaaS / Google OAuth verification now "Real users soon" was lowered to Future B; SaaS machinery before the Stage 0 self-use gate is the primary documented anti-goal ADR-008, VISION
Delegating the S5 reuse spike to an agent swarm A 1-hour sequential read-and-judge with a single coupled conclusion; orchestration overhead exceeds the work, and it's Diamond-1 judgment the maintainer wanted to own (process note)

If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — not a silent reversal.