From 1ad1966672d79202224b32c7e3668ceb1cf47c5a Mon Sep 17 00:00:00 2001 From: Mathias Date: Sat, 6 Jun 2026 11:25:52 +0200 Subject: [PATCH] docs(data-model): add CHANNEL_ERRORS + LOGIN_EVENTS entities, transcript_status columns, auto_summarize default update --- docs/data-model.md | 40 +++++++++++++++++++++++++++++++--------- 1 file changed, 31 insertions(+), 9 deletions(-) diff --git a/docs/data-model.md b/docs/data-model.md index 70bc410..261b61e 100644 --- a/docs/data-model.md +++ b/docs/data-model.md @@ -23,7 +23,7 @@ only opaque references to them; the secret material lives in ESO/1Password (ADR- ## Entities -Solid entities below are **persisted today** (migrations 001–006). `AI_CREDENTIAL` and +Solid entities below are **persisted today** (migrations 001–013). `AI_CREDENTIAL` and `SUBSCRIPTION` are **planned, not yet a table** — kept in the model for intent; see the notes. ```mermaid @@ -37,11 +37,12 @@ erDiagram VIDEO ||--o| TRANSCRIPT : "has at most one" VIDEO ||--o| SUMMARY : "has at most one" SUMMARY ||--o{ SINK_DELIVERY : "delivered via" + USER ||--o{ CHANNEL_ERROR : "reports unavailable channels" USER { uuid id PK text display_name - bool auto_summarize "default false -> manual mode out of the box (migration 006)" + bool auto_summarize "default true for new users (migration 011, ADR-018)" timestamptz created_at } USER_IDENTITY { @@ -87,6 +88,8 @@ erDiagram text url bool summarize_requested "default false -> manual-mode queue flag (migration 006)" timestamptz seen_at + text transcript_status "none|rate_limited|fetched (migration 007)" + timestamptz rate_limited_at "backoff clock for 429 retries (migration 007)" } TRANSCRIPT { uuid video_id PK_FK @@ -123,21 +126,30 @@ erDiagram text action "watched | skipped | saved" timestamptz acted_at } + CHANNEL_ERROR { + uuid user_id FK + text channel_id + text channel_name + timestamptz first_seen + timestamptz last_seen + } ``` `SUMMARY_ACTION` has `UNIQUE (user_id, video_id, action)`; `VIDEO_CONNECTION` has `UNIQUE (user_id, provider)` (one connection per provider — reconnect upserts in place). RLS (`ENABLE` + `FORCE`) is on **every solid user-owned table above** — `users`, `videos`, -`transcripts`, `summaries`, `summary_actions`, `video_connections`. `sink_deliveries` is -RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally **not** RLS'd -(auth plumbing). See the *Isolation invariant* section for the mechanism. +`transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. +`sink_deliveries` is RLS'd via an `EXISTS` on its parent summary; `user_identities` is +intentionally **not** RLS'd (auth plumbing). See the *Isolation invariant* section for the +mechanism. ## Notes per entity - **USER** — one row per registered user (Stage 1, ADR-012; no longer single-row). The Tapir-side profile; the Dex identity is held separately in `USER_IDENTITY`, not on this row. `auto_summarize` - (migration 006) is the per-user mode flag: `FALSE` (default) = manual, `TRUE` = auto-summarize - every new video. + (migration 006) is the per-user mode flag: `TRUE` = auto-summarize every new video. Default is + **true** for new users (migration 011, ADR-018); existing rows were back-filled via migration 012 + with RLS bypass. - **USER_IDENTITY** (migration 004) — the `dex_subject → user_id` map. `dex_subject` is the PK, `user_id` a `UNIQUE` FK to `users` with `ON DELETE CASCADE`. This is the bridge resolved at login *before* a `user_id` is known, so it is **deliberately not RLS-enabled** (it holds no user data; @@ -157,6 +169,9 @@ RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally decision. The same video seen by two users is two rows. `seen_at` is when Tapir detected it. `summarize_requested` (migration 006) is the manual-mode queue flag: the web "Summarize" button sets it `TRUE`; the next `tapir run` picks it up, summarizes, and clears it back to `FALSE`. + `transcript_status` and `rate_limited_at` (migration 007) track caption-fetch outcomes for + rate-limit backoff: `NULL` = not attempted; `rate_limited` = 429 seen, skip until + `NOW() - rate_limited_at > TAPIR_FETCH_BACKOFF`; `fetched` = resolved; `none` = no transcript. - **TRANSCRIPT** — at most one per video. `source = none` records "checked, no usable transcript" so the watcher doesn't reprocess (ADR-007). `content` null in that case. - **SUMMARY** — at most one per video. `fallback_used` + `ai_provider`/`ai_model` make the @@ -170,6 +185,12 @@ RLS'd via an `EXISTS` on its parent summary; `user_identities` is intentionally save) — the column that makes the Stage-0 headline metric ("acts on ≥1 summary") queryable (ui-spec.md §5, ADR-011). `video_id` is `TEXT` and **not** FK-constrained, mirroring summaries' standalone `(user_id, video_id)` key. `UNIQUE (user_id, video_id, action)`. FORCE RLS'd. +- **CHANNEL_ERRORS** (migration 013) — channels that returned HTTP 404 (deleted or private) on + the most recent discovery pass. Upserted per scheduler pass (`last_seen` refreshed each run); + surfaced on the account page as a warning. Cascades on user deletion. Primary key is + `(user_id, channel_id)`. FORCE RLS'd. +- **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the + Stage-0 gate query (VISION §Stage 0: "returned and used in ≥2 distinct weeks"). ## Isolation invariant (Stage 1+) — LIVE @@ -180,8 +201,9 @@ enforcement dormant); **ADR-012 opened Stage 1 and turned enforcement on in the Enforcement is **Postgres Row-Level Security** (migration `003_rls.up.sql`): - RLS is `ENABLE`d **and** `FORCE`d on every user-owned table — `users`, `videos`, - `transcripts`, `summaries`, `summary_actions`, `video_connections`. `FORCE` is load-bearing: - the app connects as the table **owner** (`tapir` role), and owners bypass RLS unless forced. + `transcripts`, `summaries`, `summary_actions`, `video_connections`, `channel_errors`. + `FORCE` is load-bearing: the app connects as the table **owner** (`tapir` role), and owners + bypass RLS unless forced. - Each policy keys off the per-request GUC `tapir.current_user_id`, set transaction-locally by the store's `withUser` helper via `set_config('tapir.current_user_id', $1, true)` — it auto-resets on commit/rollback, so it never leaks across a pooled connection.