# Tapir — Architecture (C4 + sequences)

Diagrams describe **current assumptions**, not a frozen design. They are diffable Mermaid
so they live in version control and render in Gitea. When a decision here changes, record it
in `DECISIONS.md` and update the relevant diagram in the same commit.

Altitude: C4 Level 1 (Context) and Level 2 (Container), plus sequence diagrams for the two
behaviours that carry the most design risk. Clean Architecture layering is described after
the diagrams.

---

## C4 L1 — System Context

Who and what Tapir talks to. (Brain and external AI are dashed — both optional.)

```mermaid
graph TB
    user["User<br/>(maintainer; later, trusted users)"]
    tapir["Tapir<br/>watches subscriptions,<br/>summarizes new videos"]
    yt["YouTube<br/>(Data API + captions)"]
    vimeo["Vimeo<br/>(API + text tracks)"]
    local["Local AI stack<br/>(LiteLLM / piguard alias)"]
    byo["User's BYO AI<br/>(Claude / OpenAI / Gemini)"]
    brain["brain<br/>(brain-mcp)"]

    user -->|connects accounts,<br/>reads summaries| tapir
    tapir -->|subscriptions,<br/>new-video events,<br/>captions| yt
    tapir -->|subscriptions,<br/>text tracks| vimeo
    tapir -->|summarize<br/>PRIMARY| local
    tapir -.->|summarize<br/>FALLBACK, opt-in| byo
    tapir -.->|optional sink:<br/>brain_ingest| brain

    classDef opt stroke-dasharray: 5 5;
    class byo,brain opt;
```

---

## C4 L2 — Containers

Inside Tapir. The **engine** is provider- and sink-agnostic; everything external is an
adapter behind an interface (Clean Architecture ports & adapters).

```mermaid
graph TB
    subgraph tapir["Tapir (Go)"]
        http["tapir serve<br/>(HTMX+Templ web surface:<br/>read summaries, connect,<br/>account, summarize)"]
        watcher["Watcher<br/>detects new videos<br/>(WebSub + poll)"]
        engine["Summarization engine<br/>(use-case core)"]
        resolver["Transcript resolver<br/>(captions-first)"]
        router["AI router<br/>(llm.Router:<br/>Primary -> Fallback)"]
        store[("User store<br/>(Postgres,<br/>per-user isolated)")]

        subgraph sinks["Sink adapters (Sink interface)"]
            sink_store["Store sink<br/>(primary)"]
            sink_brain["Brain sink<br/>(HTTP brain-mcp)"]
        end

        subgraph providers["Provider adapters (VideoSource interface)"]
            p_yt["YouTube adapter"]
            p_vimeo["Vimeo adapter"]
        end
    end

    http --> store
    watcher --> p_yt
    watcher --> p_vimeo
    watcher -->|NewVideo event| engine
    engine --> resolver
    resolver --> p_yt
    resolver --> p_vimeo
    engine --> router
    engine --> sink_store
    engine -.-> sink_brain
    sink_store --> store

    classDef opt stroke-dasharray: 5 5;
    class sink_brain opt;
```

**Interfaces that keep the engine pure (the ports):**

- `VideoSource` — list subscriptions, detect new videos, fetch transcript. Implemented by
  YouTube and Vimeo adapters.
- `Summarizer` — turn a transcript + context into highlights/takeaways. Backed by the
  `llm.Router` (Primary local, Fallback BYO).
- `Sink` — deliver a summary. Implemented by the store sink (primary) and brain sink
  (optional). New sinks add an implementation, nothing else.
- `SecretStore` — fetch/store per-user OAuth tokens and BYO keys. Backed by ESO/1Password.

The engine depends only on these interfaces, never on YouTube, brain, or a concrete model.
This is what makes "standalone vs homelab" a configuration of which adapters are wired, not
two codebases (ADR-003).

---

## Web surface — `tapir serve` (Stage 1, ADR-011 → ADR-012)

A later transport added over the **unchanged** engine/ports/sinks core (ADR-003): `tapir serve`
is an HTMX+Templ reader/writer (`internal/web`) over the existing `store`. It added no business
logic to the engine — it reads the store and, for one action, kicks the existing engine. ADR-011
shipped it single-user; ADR-012 opened multi-user with DB-enforced (RLS) isolation.

```mermaid
graph TB
    browser["Browser<br/>(Dex-authenticated user)"]
    subgraph web["internal/web (tapir serve)"]
        oidc["oidc<br/>Dex OIDC session<br/>(authenticate-only)"]
        gate["registration gate<br/>new subject -> /register"]
        pages["summary list + detail<br/>(read) + actions"]
        connect["/oauth/youtube/callback<br/>per-user token connect"]
        account["account<br/>(disconnect, delete)"]
        summarize["Summarize button<br/>-> background goroutine"]
    end
    store[("store<br/>(Postgres, RLS per user)")]
    engine["Summarization engine<br/>(unchanged core)"]
    secrets["SecretStore<br/>(per-user token refs)"]

    browser --> oidc
    oidc --> gate
    gate --> pages
    pages --> store
    connect --> secrets
    connect --> store
    account --> store
    account --> secrets
    summarize -->|background| engine
    summarize -->|HTMX status poll| store
    engine --> store
```

- **Dex OIDC session layer** (`internal/web/oidc`) — **authenticate-only** (ADR-012). It proves
  *who*; authorization/isolation is the DB's job (RLS), not the session's.
- **Registration gate** — a Dex subject with no `users` row is routed to `/register`, which
  creates the `users` row + the `user_identities` mapping (migration 004). Returning subjects
  pass straight through.
- **Web-initiated YouTube connect** — `/oauth/youtube/connect` → `/oauth/youtube/callback`
  persists a **per-user** refresh-token ref (`youtube/<userID>/refresh_token`) via `SecretStore`
  and a `video_connections` row (ADR-006, migration 005). Distinct from the CLI `tapir auth`.
- **Account management** — `/account` offers disconnect and **delete account**. Delete removes
  only Tapir-side state (cascade across the user's tables + secret refs); the shared Dex identity
  is left intact (ADR-013).
- **Immediate summarization** — the web "Summarize" button (`POST /v/{id}/summarize`) fires the
  engine in a **background goroutine** inside `serve`; the page HTMX-polls `/v/{id}/status`,
  showing a Charmbracelet spinner while in-flight (and an honest "queued/waiting" state under
  rate-limiting — ADR-014).
- **Summarization mode** — `users.auto_summarize` (migration 006). Default is **true** for new
  users (migration 011, ADR-018); all existing rows were back-filled via migration 012. Auto:
  every new video is summarized. Manual: new videos appear unsummarized; the button sets
  `videos.summarize_requested`, which the next `tapir run` processes and clears. Both the click
  path and the batch `tapir run` drive the same unchanged engine.

The engine, ports, and sink adapters are **untouched** by all of the above — the web surface only
reads the store and triggers the existing engine. Adding it changed wiring, not the core (ADR-003).

---

## In-process scheduler (ADR-018)

`cmdServe` launches a background goroutine when `TAPIR_DISCOVERY_INTERVAL > 0`. On each tick
it calls `store.ListAllUsers` (un-RLS'd admin query), builds a per-user `runner.Runner`, and
calls `RunOnce` for each registered user in sequence.

```mermaid
sequenceDiagram
    participant S as Scheduler goroutine
    participant DB as Postgres (RLS)
    participant YT as YouTube timedtext
    participant LLM as LiteLLM gateway

    loop every TAPIR_DISCOVERY_INTERVAL
        S->>DB: ListAllUsers() [un-RLS'd]
        loop per user
            S->>DB: GetAutoSummarize(userID)
            S->>YT: ListSubscriptions + NewVideos
            Note over S,YT: WaitFetchGate(ctx) throttles<br/>all fetches to TAPIR_FETCH_RATE
            alt transcript available
                S->>LLM: Summarize
                S->>DB: Deliver(summary)
            else 429
                S->>DB: SetTranscriptStatus(rate_limited)
            end
        end
    end
```

**Single-replica constraint (load-bearing).** The scheduler lives in the web process;
`replicas: 1` in the k3s deployment manifest is not cosmetic — running `tapir serve` at
>1 replica makes every replica run the full discovery loop, causing every registered user
to be fetched in parallel from the same egress IP (429s + duplicate work). Do not scale
`serve` past 1 replica without first moving discovery to a k8s CronJob or adding leader
election. The process logs a `Warn` at startup when scheduled discovery is enabled as a
reminder.

---

## Process-wide timedtext rate gate

**`internal/adapters/youtube/gate.go`** (ADR-014 item 2): a single `rate.Limiter`
(`golang.org/x/time/rate`) shared across **all** Adapter instances. Every `httpDo` call for
a caption fetch passes through `WaitFetchGate(ctx)` before hitting YouTube. This serialises
the scheduler loop AND the web click-path through the same per-egress-IP budget. Configured
via `TAPIR_FETCH_RATE` (Go duration, default `2s`). Setting it to `0` disables the gate
(dev/tests only).

This is the precondition that makes scheduled auto-summarize safe: without the gate, a
multi-user scheduler pass could fire many concurrent timedtext requests from the same IP
within seconds, triggering 429s for all users.

---

## Sequence — core use case: new video summarized

```mermaid
sequenceDiagram
    participant Src as VideoSource<br/>(YouTube/Vimeo)
    participant W as Watcher
    participant E as Engine
    participant R as Transcript resolver
    participant AI as AI router
    participant S as Sink(s)

    Src->>W: new upload (WebSub push / poll)
    W->>E: NewVideo{user, channel, videoID}
    E->>R: resolve transcript(videoID)
    R->>Src: fetch captions
    alt captions available
        Src-->>R: captions text
        R-->>E: Transcript{source: captions}
        E->>AI: summarize(transcript, userContext)
        AI-->>E: Summary{highlights, takeaways}
        E->>S: deliver(summary)
        S-->>E: ok
    else no transcript
        Src-->>R: none
        R-->>E: NoTranscript
        E->>S: deliver(skipped: no transcript)
    end
```

---

## Sequence — AI routing (local-first, BYO fallback)

```mermaid
sequenceDiagram
    participant E as Engine
    participant R as llm.Router
    participant P as Primary<br/>(local: LiteLLM/piguard)
    participant F as Fallback<br/>(user BYO key)

    E->>R: summarize(transcript)
    R->>P: complete(prompt)
    alt local succeeds
        P-->>R: summary
        R-->>E: summary (fallback_used = false)
    else local fails (error/timeout/unavailable)
        P-->>R: error
        alt user has BYO configured
            R->>F: complete(prompt)
            F-->>R: summary
            R-->>E: summary (fallback_used = true)
        else no BYO
            R-->>E: error (queued for retry)
        end
    end
```

> "Reliably" (VISION / the BYO trigger) is operationalized as: **Primary returned without
> error within timeout.** Richer quality scoring can layer on later without changing the
> interface. `fallback_used` is recorded per summary so "is the local stack good enough?"
> becomes a query, not a guess.

---

## Clean Architecture layering

```mermaid
graph LR
    subgraph domain["Domain (entities)"]
        d["User, Subscription, Video,<br/>Transcript, Summary"]
    end
    subgraph usecase["Use cases (engine)"]
        u["SummarizeNewVideo,<br/>ConnectAccount,<br/>RouteAI"]
    end
    subgraph ports["Ports (interfaces)"]
        po["VideoSource, Summarizer,<br/>Sink, SecretStore"]
    end
    subgraph adapters["Adapters (infra)"]
        a["YouTube, Vimeo, llm.Router,<br/>Store sink, Brain sink,<br/>ESO secret store, HTTP, Postgres"]
    end

    a --> po
    po --> u
    u --> d
```

Dependencies point inward only. Domain knows nothing of YouTube, brain, Postgres, or any
model. Adapters are swappable; tests target the use-case core through fake adapters (see the
Gherkin features in `docs/use-cases/`).

---

## Out of scope in these diagrams (deferred per ADRs)

- Audio-download + speech-to-text resolver (ADR-007) — would be an additional `VideoSource`
  fallback path, drawn when built.
- Per-user isolation is **live, not deferred**: Postgres RLS `FORCE`d on every user-owned table
  (ADR-012, migration 003), realising ADR-002's per-tenant intent at the DB layer. The coarser
  multi-tenant primitives (per-namespace NetworkPolicy, Kyverno, tenant label) remain a
  Stage-2 hardening item, not exercised yet.
- Public SaaS surface (sign-up, billing) — Future C, not built (ADR-008).