tapir/docs/data-model.md

# Tapir — Data Model

Current assumptions for the persistence model. Scoped to **Stage 0 (single user)** and
**Stage 1 (Future B, 1–5 trusted users)**. Future C concerns (sharding, cross-tenant
dedup) are explicitly excluded and noted at the end.

Persistence is Postgres (ADR-002), reusing the homelab's instance with a per-tenant role
when Stage 1 arrives. Secrets (OAuth tokens, BYO keys) are **not** stored in these tables —
only opaque references to them; the secret material lives in ESO/1Password (ADR-002, ADR-006).

## Design decisions baked into this model

- **Per-user isolation, not a shared global video table.** The earlier draft proposed a
  global `videos`/`transcripts` table deduped across tenants. Rejected for Future B: it
  reintroduces exactly the cross-domain coupling the homelab architecture review is
  removing, and at 1–5 users the cost of occasionally re-summarizing the same video is
  trivial compared to the isolation it would cost. Each user's data is self-contained.
  (Revisit only if Future C makes GPU/transcription cost dominate — a new ADR, not a default.)
- **Secrets by reference only.** Tables hold a `secret_ref` (opaque string/UUID resolved via
  the `SecretStore` port), never tokens or keys.
- **The brain sink is just a delivery target.** No brain-specific tables. Whether a summary
  was also delivered to brain is recorded as sink status on the summary.

## Entities

```mermaid
erDiagram
    USER ||--o{ VIDEO_CONNECTION : has
    USER ||--o{ AI_CREDENTIAL : has
    VIDEO_CONNECTION ||--o{ SUBSCRIPTION : exposes
    SUBSCRIPTION ||--o{ VIDEO : "produces (per user)"
    VIDEO ||--o| TRANSCRIPT : "has at most one"
    VIDEO ||--o| SUMMARY : "has at most one"
    SUMMARY ||--o{ SINK_DELIVERY : "delivered via"

    USER {
        uuid id PK
        text display_name
        timestamptz created_at
    }
    VIDEO_CONNECTION {
        uuid id PK
        uuid user_id FK
        text provider "youtube | vimeo"
        text provider_account
        text token_secret_ref "-> SecretStore, never the token"
        text status "active | revoked | error"
        timestamptz connected_at
    }
    AI_CREDENTIAL {
        uuid id PK
        uuid user_id FK
        text provider "anthropic | openai | gemini"
        text key_secret_ref "-> SecretStore, never the key"
        text status
        timestamptz created_at
    }
    SUBSCRIPTION {
        uuid id PK
        uuid user_id FK
        uuid connection_id FK
        text channel_id
        text channel_title
        timestamptz websub_expires "nullable; youtube push lease"
        bool active
    }
    VIDEO {
        uuid id PK
        uuid user_id FK
        uuid subscription_id FK
        text provider
        text provider_video_id
        text title
        int duration_s
        timestamptz published_at
        text url
        timestamptz seen_at
    }
    TRANSCRIPT {
        uuid video_id PK_FK
        uuid user_id FK
        text source "captions | none"
        text language
        text content "null when source = none"
        timestamptz resolved_at
    }
    SUMMARY {
        uuid id PK
        uuid user_id FK
        uuid video_id FK
        text summary
        jsonb highlights
        jsonb takeaways
        text ai_provider "local | anthropic | openai | gemini"
        text ai_model
        bool fallback_used
        timestamptz created_at
    }
    SINK_DELIVERY {
        uuid id PK
        uuid summary_id FK
        text sink "store | brain"
        text status "pending | delivered | error"
        text detail "nullable; error message etc"
        timestamptz updated_at
    }
```

## Notes per entity

- **USER** — at Stage 0 there is exactly one row. At Stage 1, identity comes via Dex; this
  table holds the Tapir-side profile keyed to the Dex subject.
- **VIDEO_CONNECTION** — a connected YouTube/Vimeo account. `token_secret_ref` resolves to
  the OAuth refresh token via `SecretStore`. Revocation flips `status`, doesn't delete history.
- **AI_CREDENTIAL** — optional, per provider, per user (ADR-004's Fallback). Absent for users
  who only use the local stack. One row per provider max.
- **SUBSCRIPTION** — a watched channel. `websub_expires` tracks the YouTube push lease so the
  watcher knows when to re-subscribe; null for poll-based (Vimeo).
- **VIDEO** — one row per (user, video) — note `user_id`, reflecting the per-user-isolation
  decision. The same video seen by two users is two rows. `seen_at` is when Tapir detected it.
- **TRANSCRIPT** — at most one per video. `source = none` records "checked, no usable
  transcript" so the watcher doesn't reprocess (ADR-007). `content` null in that case.
- **SUMMARY** — at most one per video. `fallback_used` + `ai_provider`/`ai_model` make the
  "is local good enough?" question queryable (the Stage 0 quality signal). `highlights`/
  `takeaways` as jsonb to stay schema-flexible while the output format settles.
- **SINK_DELIVERY** — one row per (summary, sink) attempt. This is where "also sent to brain"
  lives — no brain tables, just a delivery row with `sink = brain`. Sinks fail independently;
  a failed brain delivery doesn't fail the store delivery.

## Isolation invariant (Stage 1+) — LIVE

Every user-owned table carries `user_id`, and isolation is **enforced at the DB layer**, not
only in application code. ADR-011 shipped this surface single-user (one allowlisted subject,
enforcement dormant); **ADR-012 opened Stage 1 and turned enforcement on in the same slice.**

Enforcement is **Postgres Row-Level Security** (migration `003_rls.up.sql`):

- RLS is `ENABLE`d **and** `FORCE`d on every user-owned table — `users`, `videos`,
  `transcripts`, `summaries`, `summary_actions`, `video_connections`. `FORCE` is load-bearing:
  the app connects as the table **owner** (`tapir` role), and owners bypass RLS unless forced.
- Each policy keys off the per-request GUC `tapir.current_user_id`, set transaction-locally by
  the store's `withUser` helper via `set_config('tapir.current_user_id', $1, true)` — it
  auto-resets on commit/rollback, so it never leaks across a pooled connection.
- `current_setting('tapir.current_user_id', true)` uses `missing_ok = true`: an **unset** GUC
  yields `NULL`, the predicate matches no rows, and access **denies by default**.
- `sink_deliveries` has no `user_id`; its policy derives ownership from the parent summary via
  `EXISTS (SELECT 1 FROM summaries …)`.
- `user_identities` (the Dex-subject → user_id map) is **deliberately not RLS-enabled** — it is
  auth plumbing read *before* a user_id is known; putting RLS there would deadlock. It holds no
  user data.

The Stage-2 isolation bar is **pulled forward, not deferred**: `internal/adapters/store/rls_test.go`
runs two users against a non-superuser, non-`BYPASSRLS` role and asserts user A reads/writes zero
of user B's rows across every table. It ships green with the multi-user features (ADR-012); no
multi-user feature merges ahead of it passing.

## Job / processing state

Execution state (queued / running / retrying) for the watch→resolve→summarize→deliver
pipeline is owned by the worker runtime, not modelled as first-class domain tables here.
`SINK_DELIVERY.status` and `TRANSCRIPT.source` capture the durable, queryable outcomes; a
thin `jobs` table may be added when a "what's processing" view is needed (mirrors the worker
queue, doesn't replace it). Deferred until there's a reason.

## Explicitly out of scope (Future C)

- Global cross-tenant video/transcript dedup (rejected above).
- Sharding / per-tenant physical databases.
- Soft-delete + full audit trail on connections/credentials (a Stage 2 hardening item; add
  via ADR when Stage 2 work starts).