From 74f4fd7f2a2a1cf49ff37d5554caa9cd3766ba6e Mon Sep 17 00:00:00 2001 From: Mathias Date: Wed, 3 Jun 2026 22:17:15 +0200 Subject: [PATCH] docs(data-model): reconcile schema with migrations 002-006 The ER diagram and notes predated migrations 002-006. Brought them to code truth: - add summary_actions (002), user_identities (004), video_connections (005) with real columns/constraints; rename token_secret_ref -> token_ref to match migration 005. - add users.auto_summarize and videos.summarize_requested (006). - note FORCE RLS coverage and sink_deliveries' EXISTS-derived policy (003); user_identities NOT RLS'd. - mark AI_CREDENTIAL and SUBSCRIPTION as planned (no table exists; only videos.subscription_id, no FK). Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/data-model.md | 80 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 62 insertions(+), 18 deletions(-) diff --git a/docs/data-model.md b/docs/data-model.md index c956cb8..70bc410 100644 --- a/docs/data-model.md +++ b/docs/data-model.md @@ -23,11 +23,16 @@ only opaque references to them; the secret material lives in ESO/1Password (ADR- ## Entities +Solid entities below are **persisted today** (migrations 001–006). `AI_CREDENTIAL` and +`SUBSCRIPTION` are **planned, not yet a table** — kept in the model for intent; see the notes. + ```mermaid erDiagram + USER ||--|| USER_IDENTITY : "logs in via (Dex subject)" USER ||--o{ VIDEO_CONNECTION : has - USER ||--o{ AI_CREDENTIAL : has - VIDEO_CONNECTION ||--o{ SUBSCRIPTION : exposes + USER ||--o{ SUMMARY_ACTION : records + USER ||--o{ AI_CREDENTIAL : "has (planned)" + VIDEO_CONNECTION ||--o{ SUBSCRIPTION : "exposes (planned)" SUBSCRIPTION ||--o{ VIDEO : "produces (per user)" VIDEO ||--o| TRANSCRIPT : "has at most one" VIDEO ||--o| SUMMARY : "has at most one" @@ -36,14 +41,20 @@ erDiagram USER { uuid id PK text display_name + bool auto_summarize "default false -> manual mode out of the box (migration 006)" + timestamptz created_at + } + USER_IDENTITY { + text dex_subject PK + uuid user_id FK "UNIQUE -> USER, ON DELETE CASCADE; NOT RLS-enabled" timestamptz created_at } VIDEO_CONNECTION { uuid id PK - uuid user_id FK + uuid user_id FK "-> USER, ON DELETE CASCADE" text provider "youtube | vimeo" - text provider_account - text token_secret_ref "-> SecretStore, never the token" + text provider_account "nullable" + text token_ref "-> SecretStore, never the token" text status "active | revoked | error" timestamptz connected_at } @@ -66,14 +77,15 @@ erDiagram } VIDEO { uuid id PK - uuid user_id FK - uuid subscription_id FK + uuid user_id FK "-> USER, ON DELETE CASCADE" + uuid subscription_id "nullable; no FK at Stage 0" text provider text provider_video_id text title int duration_s timestamptz published_at text url + bool summarize_requested "default false -> manual-mode queue flag (migration 006)" timestamptz seen_at } TRANSCRIPT { @@ -87,7 +99,7 @@ erDiagram SUMMARY { uuid id PK uuid user_id FK - uuid video_id FK + uuid video_id "no FK to videos; (user_id, video_id) UNIQUE is the dedup key" text summary jsonb highlights jsonb takeaways @@ -98,26 +110,53 @@ erDiagram } SINK_DELIVERY { uuid id PK - uuid summary_id FK + uuid summary_id FK "-> SUMMARY, ON DELETE CASCADE; ownership derived via this FK" text sink "store | brain" text status "pending | delivered | error" text detail "nullable; error message etc" timestamptz updated_at } + SUMMARY_ACTION { + uuid id PK + uuid user_id FK "-> USER" + text video_id "TEXT, not FK (mirrors summaries' standalone key)" + text action "watched | skipped | saved" + timestamptz acted_at + } ``` +`SUMMARY_ACTION` has `UNIQUE (user_id, video_id, action)`; `VIDEO_CONNECTION` has +`UNIQUE (user_id, provider)` (one connection per provider — reconnect upserts in place). +RLS (`ENABLE` + `FORCE`) is on **every solid user-owned table above** — `users`, `videos`, +`transcripts`, `summaries`, `summary_actions`, `video_connections`. `sink_deliveries` is +RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally **not** RLS'd +(auth plumbing). See the *Isolation invariant* section for the mechanism. + ## Notes per entity -- **USER** — at Stage 0 there is exactly one row. At Stage 1, identity comes via Dex; this - table holds the Tapir-side profile keyed to the Dex subject. -- **VIDEO_CONNECTION** — a connected YouTube/Vimeo account. `token_secret_ref` resolves to - the OAuth refresh token via `SecretStore`. Revocation flips `status`, doesn't delete history. -- **AI_CREDENTIAL** — optional, per provider, per user (ADR-004's Fallback). Absent for users - who only use the local stack. One row per provider max. -- **SUBSCRIPTION** — a watched channel. `websub_expires` tracks the YouTube push lease so the - watcher knows when to re-subscribe; null for poll-based (Vimeo). +- **USER** — one row per registered user (Stage 1, ADR-012; no longer single-row). The Tapir-side + profile; the Dex identity is held separately in `USER_IDENTITY`, not on this row. `auto_summarize` + (migration 006) is the per-user mode flag: `FALSE` (default) = manual, `TRUE` = auto-summarize + every new video. +- **USER_IDENTITY** (migration 004) — the `dex_subject → user_id` map. `dex_subject` is the PK, + `user_id` a `UNIQUE` FK to `users` with `ON DELETE CASCADE`. This is the bridge resolved at login + *before* a `user_id` is known, so it is **deliberately not RLS-enabled** (it holds no user data; + RLS here would deadlock the lookup that yields the id used for scoping). Account deletion cascades + the mapping away (ADR-013). +- **VIDEO_CONNECTION** (migration 005) — a connected YouTube/Vimeo account. `token_ref` resolves to + the OAuth refresh token via `SecretStore` (per-user scheme `youtube//refresh_token`). + `UNIQUE (user_id, provider)`: one connection per provider, reconnect upserts. Revocation/disconnect + flips `status`, doesn't delete history. FORCE RLS'd. +- **AI_CREDENTIAL** — *planned, no table yet.* Optional, per provider, per user (ADR-004's Fallback). + BYO keys are currently resolved via `SecretStore` refs without a dedicated table; this entity is + modelled for when per-credential metadata is needed. +- **SUBSCRIPTION** — *planned, no table yet.* A watched channel; `websub_expires` would track the + YouTube push lease. At Stage 0/1 `videos.subscription_id` is a nullable column with **no FK** (the + subscriptions table is not part of the shipped store-sink slice — migration 001). - **VIDEO** — one row per (user, video) — note `user_id`, reflecting the per-user-isolation decision. The same video seen by two users is two rows. `seen_at` is when Tapir detected it. + `summarize_requested` (migration 006) is the manual-mode queue flag: the web "Summarize" button + sets it `TRUE`; the next `tapir run` picks it up, summarizes, and clears it back to `FALSE`. - **TRANSCRIPT** — at most one per video. `source = none` records "checked, no usable transcript" so the watcher doesn't reprocess (ADR-007). `content` null in that case. - **SUMMARY** — at most one per video. `fallback_used` + `ai_provider`/`ai_model` make the @@ -125,7 +164,12 @@ erDiagram `takeaways` as jsonb to stay schema-flexible while the output format settles. - **SINK_DELIVERY** — one row per (summary, sink) attempt. This is where "also sent to brain" lives — no brain tables, just a delivery row with `sink = brain`. Sinks fail independently; - a failed brain delivery doesn't fail the store delivery. + a failed brain delivery doesn't fail the store delivery. No own `user_id`; RLS ownership is + derived from the parent summary via `EXISTS` (migration 003). +- **SUMMARY_ACTION** (migration 002) — records the maintainer's act on a summary (watch / skip / + save) — the column that makes the Stage-0 headline metric ("acts on ≥1 summary") queryable + (ui-spec.md §5, ADR-011). `video_id` is `TEXT` and **not** FK-constrained, mirroring summaries' + standalone `(user_id, video_id)` key. `UNIQUE (user_id, video_id, action)`. FORCE RLS'd. ## Isolation invariant (Stage 1+) — LIVE