tapir/DECISIONS.md

# Architecture Decision Records — Tapir

Lightweight ADRs. Each records a decision, its context, and its consequences at the time
it was made. Decisions are **append-only**: to reverse one, add a new ADR that supersedes
it (note the supersession in both). This is the reversibility record the VISION guardrails
refer to.

Status values: `Accepted` · `Superseded by ADR-NNN` · `Proposed`.

---

## ADR-001 — Go, not Python

**Status:** Accepted (2026-06-02)

**Context.** The first design draft assumed a Python/FastAPI/arq stack. The homelab is a
Go estate (`template-go-web`, `template-go-agent`, `mcp-chassis`, `hyperguild/ingestion`).
A Python service would be the only one of its kind, outside every shared convention, chassis,
and CI pattern.

**Decision.** Tapir is written in Go. Python is used only where it makes no sense not to
(e.g. if an ML component later genuinely requires it — none does today).

**Consequences.** Reuses homelab Go conventions and the `mcp-chassis` auth pattern if/when
Tapir exposes an MCP surface. Closes the door on the Python async ecosystem (acceptable —
Go's concurrency model fits the watcher/worker shape).

---

## ADR-002 — No Supabase; reuse existing Dex / ESO / Postgres conventions

**Status:** Accepted (2026-06-02)

**Context.** The draft proposed self-hosted Supabase for auth + RLS + secrets (Vault). The
homelab already runs **Dex** (OIDC), **ESO + 1Password** (secrets), and a **postgres18**
instance that the May-2026 architecture review is actively *de*-coupling by blast radius.
Supabase would bundle a second auth system, a second secrets store, and its own Postgres —
three duplications of things just consolidated.

**Decision.** No Supabase. Auth, secrets, and persistence reuse the homelab's existing
primitives: Dex for identity (when external identity is needed), ESO + 1Password for secret
custody, Postgres with per-tenant role/grant and the `tenant=` namespace label scheme for
isolation when Future B/C arrives.

**Consequences.** No new infra to operate or back up. Multi-tenancy uses the architecture
review's SC7/P6 primitives rather than Supabase RLS. For Stage 0 (single user) most of this
is dormant; it activates at Stage 1.

---

## ADR-003 — Standalone-first; brain is one sink behind an interface

**Status:** Accepted (2026-06-02)

**Context.** Tapir was initially framed as both a standalone service and a brain-ingestion
pipeline. The maintainer clarified that **standalone is the more important** of the two.

**Decision.** Tapir is a standalone service. Delivery of summaries is via a **Sink
interface** with multiple implementations. The user's own store is the primary sink; the
brain is *one optional sink among others*. The engine does not know or care which sinks are
attached.

**Consequences.** Brain ingestion can never dictate the core architecture (a named drift
signal in VISION). Homelab mode is "the brain sink is enabled"; it is not a separate build.

---

## ADR-004 — Copy the `llm` package from `hyperguild/ingestion`; do not lift to a shared lib

**Status:** Accepted (2026-06-02)

**Context.** Spike S5 (`infra/docs/superpowers/handoffs/2026-06-02-video-adapter-placement.md`)
examined what Tapir could reuse from `hyperguild/ingestion`. The `internal/llm` package
(`Client` + `Router`) is stdlib-only and its `Router` implements exactly the
Primary→Fallback pattern Tapir needs for local-first → BYO-AI. It is ~3.5 KB and stable.

**Decision.** **Copy** the `llm` package into Tapir and own it. Do not lift it into a shared
module. Lifting would couple Tapir's release cycle to the monolith to save ~100 lines —
a bad trade against the "standalone owes nothing" bias.

**Consequences.** Minor duplication. Tapir owes nothing to `hyperguild/ingestion` at the
dependency level. Primary = local stack via LiteLLM/piguard alias; Fallback = user BYO key.

---

## ADR-005 — Brain sink is an HTTP brain-mcp client, not the filesystem `brain` package

**Status:** Accepted (2026-06-02)

**Context.** `hyperguild/ingestion`'s `internal/brain` package manipulates the brain git
checkout **on the local filesystem** (`os.WriteFile` into `brain/wiki/...`). That assumes
co-location with the brain repo — fine for the monolith, wrong for a standalone service
where brain is one network-reached sink.

**Decision.** Tapir's brain sink is a thin **HTTP adapter calling brain-mcp's `brain_ingest`
tool**. It does not import or replicate the filesystem `brain` package.

**Consequences.** The brain sink works regardless of where Tapir runs. Adds a dependency on
brain-mcp availability when that sink is enabled (acceptable; sinks fail independently).

---

## ADR-006 — Outbound OAuth (YouTube/Vimeo) written fresh

**Status:** Accepted (2026-06-02)

**Context.** `hyperguild/ingestion`'s `internal/oauth` package is the MCP **server's
inbound** auth (client_credentials, authenticating claude.ai). Tapir needs **outbound**
authorization-code OAuth to YouTube/Vimeo with per-user token custody — a different concern
that happens to share a name.

**Decision.** Write the YouTube/Vimeo OAuth client fresh using `golang.org/x/oauth2`. Store
per-user refresh tokens via the homelab's ESO/1Password convention (ADR-002), not a new
secrets system.

**Consequences.** No reuse from the monolith here. Token custody is the highest-value
secret surface (see VISION Stage 2); it gets the existing, vetted secret path.

---

## ADR-007 — Captions-first; audio-download + speech-to-text deferred

**Status:** Accepted (2026-06-02). Acquisition mechanism superseded by ADR-010 (Data API
`captions.download` → player/timedtext baseUrl); captions-first stance and STT deferral stand.

**Context.** YouTube's Data API does not expose transcripts. Options: official captions
(clean, limited coverage) vs. audio download + local Whisper (broad coverage, ToS-grey,
GPU-contending with the JEPA PoC on koala, breakage-prone via yt-dlp).

**Decision.** The core path is **captions-only**. When a video has no usable transcript,
Tapir records "no transcript" and moves on. Audio-download + speech-to-text is a deferred,
clearly-bounded *optional* component, not part of the core path or Stage 0.

**Consequences.** Some videos won't be summarized at Stage 0 — accepted. The "is this
useful to me?" hypothesis is testable on captioned videos alone. Avoids GPU contention and
ToS risk in the validated path.

---

## ADR-008 — Future C (public multi-tenant SaaS) deferred behind the Stage 0 gate

**Status:** Accepted (2026-06-02)

**Context.** "Real users soon" was initially asserted, which would load Google OAuth
verification, billing, and hardened multi-tenant custody early. Lowered to Future B (1–5
trusted users), with the maintainer as the real first customer.

**Decision.** Build for Stage 0 then Stage 1 (Future B). Public SaaS machinery (sign-up,
billing, Google OAuth app verification at scale) is **not built** until the Stage 0
self-use gate passes **and** real external demand appears.

**Consequences.** Smaller, faster first build. The staged Definition of Success in VISION
governs advancement. Reversible: if demand appears, a new ADR opens the Future C scope.

---

## ADR-009 — Trunk-Based Development

**Status:** Accepted (2026-06-02)

**Context.** Platform-wide convention (homelab architecture review invariant; gitea-mcp #27):
commit directly to `main`, one logical change per commit, every commit deployable.

**Decision.** Tapir follows TBD. Commit directly to `main`. No feature branches or PRs for
solo/agent work; short-lived `agent/<desc>` branches only when parallel agents are active on
the repo simultaneously. CI is the quality gate, not branch protection.

**Consequences.** Consistent with the rest of the estate. Depends on the direct-to-main write
path tracked in gitea-mcp #35 (item #1).

---

## ADR-010 — Third-party caption acquisition via the timedtext/player baseUrl

**Status:** Accepted (2026-06-02)

**Context.** ADR-007 settled *captions-first*. The first build used the YouTube Data API
`captions.list` + `captions.download` endpoints to acquire them. A live Stage-0 run proved that
`captions.download` is **owner-only**: it requires the OAuth identity to own the video, so every
subscription video the user does *not* own returns HTTP 403. Result: 0 summaries produced and a
~150-line error spew. The captions-first decision is sound; only the *acquisition mechanism* was
wrong.

**Decision.** Acquire captions from the **player response + timedtext baseUrl**, not the Data API
`captions` endpoints:

1. `POST https://www.youtube.com/youtubei/v1/player` with an **InnerTube `ANDROID` client
   context** (no API key, no OAuth). Read
   `captions.playerCaptionsTracklistRenderer.captionTracks[]`. Each track carries `baseUrl`,
   `languageCode`, and `kind` (`"asr"` = auto-generated).
2. Select by `PreferredLanguages`, preferring non-`asr` when both exist.
3. **GET the track's `baseUrl` unauthenticated** (plain `http.Client`, no OAuth token attached —
   the token can break the timedtext endpoint). The ANDROID `baseUrl` is pinned to `fmt=srv3`
   (timedtext XML); the parser also accepts `json3` and the legacy `<transcript>` XML.

A **watch-page scrape** of `ytInitialPlayerResponse` is the documented fallback if InnerTube
returns no `captionTracks`.

**Live validation (from koala, 2026-06-02):** the `ANDROID` InnerTube client returned 6
`captionTracks` with working `baseUrl`s for a public video the run identity does not own, and the
unauthenticated `baseUrl` GET returned real transcript text. `ANDROID` is the client of record
(historically returns baseUrls without a PoToken). The `WEB` client and watch-page scrape are
fallbacks.

**Consequences.**

- Works for **any public captioned video**, not just owned ones — this is the fix for the 403 wall.
- **ToS-grey:** `youtubei`/`timedtext` are unofficial endpoints. They can break when Google shifts
  InnerTube client requirements or introduces PoToken gating. Mitigation: degrade, never error — a
  missing/empty/403/unparseable caption yields `domain.Transcript{Source: SourceNone}`, so a future
  breakage produces "no transcript" rather than a crash or error spew. Only genuine transport
  (network) faults error.
- **No OAuth** is needed for the transcript fetch. OAuth is still required for `ListSubscriptions`
  and `NewVideos` (Data API) — only the transcript path goes unauthenticated.
- **Still no Whisper.** Speech-to-text stays deferred (ADR-007 unchanged).

**Supersedes:** the *acquisition mechanism* of ADR-007 (Data API `captions.download` →
player/timedtext baseUrl) only. ADR-007's captions-first stance and the STT deferral stand.

---

## ADR-011 — Web read-surface at Stage 0: Dex authn (single-user authz), action signal, public ingress + GitOps

**Status:** Accepted (2026-06-02)

**Context.** The Stage-0 CLI (`list`/`show`) reads summaries but doesn't capture the Stage-0
*headline test* — whether the maintainer **acts** on a summary (watches/skips/saves because of
it). A browser surface is wanted, and the maintainer chose to deploy it properly:
**`tapir.d-ma.be` via the homelab ingress, full k3s/Flux GitOps**, with **Dex login from the
start** rather than a Tailscale-only no-auth dev page. The data-model note says "auth is
dormant at Stage 0", so logging in early is a deliberate deviation worth recording.

**Decision.**
1. Add a Stage-0 web reader (`tapir serve`, HTMX+Templ) over the existing `store` — a new
   transport, not a core change (ADR-003). Pages: summary list + full view + **watch/skip/save**
   actions recorded in a new `summary_actions` table. The action signal instruments the
   Stage-0 success metric directly.
2. **Authentication** via Dex OIDC (ADR-002 already names Dex for identity). **Authorization
   stays trivial**: an allowlist of one subject (the maintainer). NO user CRUD, NO per-tenant
   isolation — that authz/isolation work is the real Stage-1/2 line and stays deferred. The
   distinction (authn now, authz/tenancy later) is what keeps this honest with VISION.
3. **Deploy** at `tapir.d-ma.be` via the existing homelab pattern: gitea CI (buildah) →
   registry → **Flux** reconciling manifests in `mathias/infra` `k3s/apps/tapir/`; secrets via
   **ESO + 1Password**; `postgres18` reached in-cluster. This is application of existing
   convention, not a new infra decision.

**Consequences.** The maintainer gets a real, authenticated reading surface and the Stage-0
metric becomes queryable. A browser session-login path now exists (distinct from
`mcp-chassis`'s inbound Bearer-JWT validation — not the same code). Wiring login early adds a
Dex static-client registration + ingress/TLS as prerequisites. If multiple users ever arrive,
authorization/isolation is a new ADR (Stage 1) — this one deliberately does not build it.
Full spec: `docs/ui-spec.md`.

---

## ADR-012 — Open Stage 1: multi-user with enforced isolation (RLS) in the same slice

**Status:** Accepted (2026-06-03)

**Context.** ADR-011 shipped a single-user web reader with an allowlist of one, deferring
multi-user authz/isolation to "a new ADR (Stage 1)". The maintainer has chosen to **open
Stage 1 now** — multi-user registration, per-user YouTube connect, and account management —
ahead of the formal Stage-0 self-use gate. VISION's hard invariant ("data isolation is a
promise, not a feature flag … holds from the first user") and the drift signal ("building
Stage 1+ machinery before the gate") make one thing non-negotiable: **multi-user features
must not ship before isolation is enforced.**

**Decision.**
1. **Open Stage 1.** Build registration (explicit, not just-in-time: a Dex-authenticated
   subject with no `users` row completes a registration step that creates it), per-user
   web-initiated YouTube OAuth connect (distinct from the CLI `tapir auth`), and account
   management (view / disconnect / delete).
2. **Isolation is DB-enforced via Postgres Row-Level Security**, not application-layer
   filtering — realising ADR-002's per-tenant-role intent. RLS is `FORCE`d on every
   user-owned table (the app connects as the non-superuser table-owner `tapir` role, which
   would otherwise bypass RLS); every request scopes rows via `tapir.current_user_id`
   (`SET LOCAL` inside a transaction), routed through a single structural helper so scoping
   is not per-query opt-in.
3. **The Stage-2 isolation bar is pulled forward into THIS slice, not deferred.** A real
   isolation test (two users, disposable/embedded Postgres) — user A reads/writes zero of
   user B's rows across every table — ships green **with** the multi-user features. No
   multi-user feature merges ahead of that test passing.

**Consequences.** Stage 1 + the Stage-2 isolation guarantee land together; isolation is
structural (DB), so it cannot be forgotten per-query. Adds `video_connections` and
`subscriptions` tables, a web OAuth callback, and a registration surface. The single-user
allowlist (ADR-011) is replaced by per-subject `users` rows. Reversible only by a superseding
ADR. This deliberately advances ahead of the Stage-0 gate — recorded as the maintainer's
explicit call, with isolation as the guardrail that keeps it safe.

---

## ADR-013 — Account deletion is Tapir-side only; the Dex identity is left intact

**Status:** Accepted (2026-06-03)

**Context.** Stage 1 (ADR-012) added account deletion. A registered user is two things: a
`users` row (plus all their data, cascade-linked) in Tapir's Postgres, and a subject identity
in **Dex** (the homelab OIDC provider, shared across the estate — Tapir does not own it).
"Delete my account" could mean (a) erase all Tapir-side data and secrets, or (b) that plus
deprovision the Dex identity. The maintainer chose (a).

**Decision.** Deleting a Tapir account removes **only Tapir-side state**:
- The `users` row, cascading to all user-owned tables (`videos`, `transcripts`, `summaries`,
  `sink_deliveries`, `video_connections`, and — via an **explicit delete**, because it has no
  FK — `summary_actions`). The delete test asserts the cascade reaches every table and leaves
  other users' rows untouched.
- All of that user's secrets in the SecretStore (the per-user YouTube refresh-token refs).

The **Dex identity is deliberately left intact.** Tapir does not deprovision, disable, or
modify the shared Dex directory.

**Consequences.**
- **Clean re-registration:** a deleted user who logs in again arrives as a Dex-authenticated
  subject with no `users` row, so they hit the registration gate as a "new" user — no special
  resurrection path needed. This is a feature of the choice, not an accident.
- **Right-to-erasure is partial.** The user's *identity* still exists in Dex after deletion.
  For Future B (trusted friends) this is acceptable: Dex is the maintainer's own directory and
  the identity carries no Tapir content. **But if Tapir ever moves toward Future C (real
  external/public users), this is a GDPR-shaped gap** — a true "delete my account" there must
  also deprovision or anonymise the Dex identity, which is a new ADR and likely a Dex-admin
  integration Tapir does not currently have.
- **Blast radius stays small:** Tapir never holds write access to the shared identity provider,
  consistent with the estate's blast-radius-minimisation posture (ADR-002, architecture review).

**Reversibility.** Adding Dex deprovisioning later is a superseding ADR; nothing about the
current choice blocks it. Recorded now because "deletion is partial by design" is a deliberate
semantic that future-Tapir (and any compliance review) must know was chosen, not overlooked.

---

## ADR-014 — Timedtext 429 handling: per-host backoff + honest in-flight UX, before any Whisper reconsideration

**Status:** Accepted (2026-06-03)

**Context.** ADR-010 acquires captions from the unauthenticated `timedtext` baseUrl. Live runs
show that endpoint **rate-limits per source IP (HTTP 429) under volume** — many videos fetched
in one pass from one egress IP. Stage 1 (ADR-012) made this sharper in two ways: multiple users
now drive fetches from the *same cluster egress IP*, and the v0.4.0 "Summarize" button fires an
**immediate, synchronous-feeling** fetch on click (HTMX polls `/v/{videoId}/status`), so a 429
now surfaces as a *user-facing stall* rather than a background batch hiccup. A throttle
(`TAPIR_FETCH_DELAY`) exists but is a fixed inter-fetch delay, not 429-aware, and does not
coordinate across the concurrent click-path and the `tapir run` batch path.

This ADR is **not** a decision to build Whisper. ADR-007/010 keep STT deferred *pending
measurement of the sustainable caption rate* — and that rate cannot be measured while the
client reacts badly to the 429s it already provokes. Fix the backoff and the UX first; the
clean data then tells you whether Whisper is warranted.

**Decision.**

1. **429-aware backoff at the fetch layer.** On a 429 from the timedtext/InnerTube fetch,
   respect `Retry-After` when present; otherwise exponential backoff with jitter. This replaces
   reliance on a fixed `TAPIR_FETCH_DELAY` alone (which stays as a floor/politeness delay).
2. **A single per-egress-IP rate gate** shared by *both* the `tapir run` batch path and the
   web click path, so they cannot collectively exceed the sustainable rate. Concurrency into
   the timedtext endpoint is serialised/limited at this gate regardless of how many users or
   goroutines are upstream. (The 429 is per *IP*, not per user — so the gate is process-/
   cluster-egress-wide, not per-`withUser`.)
3. **Honest in-flight UX (the product-shaping part).** The status poll distinguishes states
   the user can understand instead of a spinner that silently stalls:
   - *summarizing* — actively processing (the existing tapir spinner).
   - *queued / waiting for rate limit* — fetch deferred behind the rate gate; show a calm
     "queued, this can take a few minutes when busy" state, not a stuck spinner.
   - *no transcript* — terminal, per ADR-010's degrade-never-error (a 429 that exhausts retries
     resolves to `SourceNone`, same as any unavailable caption — it must not present as a hard
     error to the user).
   The spinner promising imminence is the wrong signal under rate-limiting; the UX must be able
   to say "waiting" truthfully.
4. **Measurement before Whisper.** Only once (1)-(3) are in and a real sustainable
   per-IP rate is observed do we revisit whether caption coverage is good enough or whether the
   deferred Whisper fallback (ADR-007) is finally warranted. That reconsideration is a future
   ADR, gated on this data.

**Consequences.**
- Caption fetching becomes well-behaved under multi-user load instead of self-inflicting 429s;
  the endpoint is treated as the shared, rate-limited resource it is.
- The click-path UX stays honest: "waiting" reads as waiting, failure degrades to "no
  transcript", never a stuck spinner or error spew.
- A future per-IP cooldown / second egress IP / proxy becomes an option the rate gate can sit
  in front of without UX changes.
- **Still no Whisper** — and now there's a clean path to the *data* that decides whether it's
  ever needed (`docs/homelab-integration.md` and a future ADR own that measurement).

**Open (tracked, not in this ADR's scope):** the actual sustainable rate number; whether a
dedicated egress IP / outbound proxy is worth it; CronJob-driven `tapir run` interaction with
the rate gate (the batch path moves into k3s per the deferred CronJob item).

---

## Rejected alternatives

Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
taken**. Recorded so a later session doesn't re-propose them as if they were fresh ideas. Each
maps to the ADR that settles it.

| Rejected | Why rejected | Settled by |
|----------|-------------|-----------|
| Python / FastAPI / arq stack | Would be the only non-Go service in the estate, outside every shared chassis/convention/CI pattern | ADR-001 |
| Self-hosted Supabase (auth + RLS + Vault) | Duplicates Dex, ESO+1Password, and the postgres instance being actively de-coupled; three new things to run and back up | ADR-002 |
| Living inside the `hyperguild/ingestion` monolith | The only worthwhile reuse is ~100 lines (`llm`) that get copied anyway; standalone-first wants the service to owe the monolith nothing | ADR-003, ADR-004 |
| Lifting shared packages into a `brain-common` module | Couples Tapir's release cycle to the monolith for negligible code savings | ADR-004 |
| Importing/replicating the filesystem `brain` package | Assumes co-location with the brain git checkout; wrong for a standalone networked service | ADR-005 |
| Reusing `ingestion`'s `oauth` package for YouTube/Vimeo | Same name, opposite direction — it's inbound MCP-server auth, not outbound provider OAuth | ADR-006 |
| Global cross-tenant `videos`/`transcripts` table (dedup) | Reintroduces the cross-domain DB coupling the homelab review is removing; at 1–5 users, re-summarizing is cheaper than the coupling | data-model.md |
| Audio-download + Whisper STT in the core path | ToS-grey, breakage-prone (yt-dlp), contends for koala GPU with the JEPA PoC; captions alone test the core hypothesis | ADR-007 |
| Building multi-tenant SaaS / Google OAuth verification now | "Real users soon" was lowered to Future B; SaaS machinery before the Stage 0 self-use gate is the primary documented anti-goal | ADR-008, VISION |
| Delegating the S5 reuse spike to an agent swarm | A 1-hour sequential read-and-judge with a single coupled conclusion; orchestration overhead exceeds the work, and it's Diamond-1 judgment the maintainer wanted to own | (process note) |

If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
not a silent reversal.