docs: reconcile requirements + architecture with ADR-020
Bring the living docs current with the recency-bounded auto-summarize + sparse
honesty + feed IA bundle (ADR-020):
- requirements (BDD): summarize_mode.feature — auto now summarizes RECENT new
videos; added a scenario for older videos (listed, on-demand), recency note.
- architecture.md: summarization-mode + new list-surface paragraph; scheduler
diagram + two-path table + three-phase pass now show the recency pre-filter;
dropped stale "Summarize now".
- data-model.md: auto_summarize is recent-only, older on-demand.
- README.md: one-line recency note on the serve scheduler.
- ui-spec.md: appended the as-built ADR-020 row (supersedes earlier copy/sort).
- specs/{video-card-states,newest-first-ordering,scheduled-discovery}.md:
superseded/extended banners pointing at ADR-020 (kept as design records).
Docs-only; task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -147,9 +147,17 @@ graph TB
|
||||
rate-limiting — ADR-014).
|
||||
- **Summarization mode** — `users.auto_summarize` (migration 006). Default is **true** for new
|
||||
users (migration 011, ADR-018); all existing rows were back-filled via migration 012. Auto:
|
||||
every new video is summarized. Manual: new videos appear unsummarized; the button sets
|
||||
`videos.summarize_requested`, which the next `tapir run` processes and clears. Both the click
|
||||
path and the batch `tapir run` drive the same unchanged engine.
|
||||
new videos **published within the recency window** (`TAPIR_AUTO_SUMMARIZE_WINDOW`, default ~7d,
|
||||
ADR-020) are summarized automatically; older videos are discovered and listed but wait for an
|
||||
explicit "Summarize". Manual: new videos appear unsummarized; the button sets
|
||||
`videos.summarize_requested`, which the next `tapir run` processes and clears. A manual request
|
||||
bypasses the recency bound. Both the click path and the batch `tapir run` drive the same
|
||||
unchanged engine.
|
||||
- **List surface (ADR-020)** — the list reads `ListVideos` ordered summarized-first, then
|
||||
`published_at DESC NULLS LAST`. The web layer collapses the noise so summaries are not buried:
|
||||
un-summarized videos older than the recency window fold into one "Show N older videos"
|
||||
disclosure, and caption-less videos collapse to a single count line. Copy surfaces scarcity
|
||||
honestly (queue counts, gradual-fill note) — it never implies the feed is fuller than it is.
|
||||
|
||||
The engine, ports, and sink adapters are **untouched** by all of the above — the web surface only
|
||||
reads the store and triggers the existing engine. Adding it changed wiring, not the core (ADR-003).
|
||||
@@ -174,6 +182,7 @@ sequenceDiagram
|
||||
loop per user
|
||||
S->>DB: GetAutoSummarize(userID)
|
||||
S->>YT: ListSubscriptions + NewVideos
|
||||
Note over S,DB: auto: skip videos published before<br/>TAPIR_AUTO_SUMMARIZE_WINDOW (ADR-020);<br/>older ones listed, await manual request
|
||||
Note over S,YT: WaitFetchGate(ctx) throttles<br/>all fetches to TAPIR_FETCH_RATE
|
||||
alt transcript available
|
||||
S->>LLM: Summarize
|
||||
@@ -214,19 +223,22 @@ Both paths share `globalFetchGate` — rate limiting is **respected in both**, n
|
||||
|
||||
| Path | Trigger | Order | Rationale |
|
||||
|------|---------|-------|-----------|
|
||||
| **Foreground** | User clicks "Summarize now" on any non-summarized card (`POST /v/{id}/retry-now` for rate-limited; `POST /v/{id}/summarize` for pending) | Single chosen video | On-demand value: user picks a specific video to read now |
|
||||
| **Background batch** | Scheduled discovery pass every `TAPIR_DISCOVERY_INTERVAL` | **Newest-first across all channels** (see below) | Onboarding prioritisation: most recent, relevant videos surface first |
|
||||
| **Foreground** | User clicks "Summarize" on any non-summarized card (`POST /v/{id}/retry-now` for rate-limited; `POST /v/{id}/summarize` for pending) | Single chosen video | On-demand value: user picks a specific video to read now — bypasses the recency bound |
|
||||
| **Background batch** | Scheduled discovery pass every `TAPIR_DISCOVERY_INTERVAL` | **Newest-first across all channels** (see below), **bounded to the recency window** (ADR-020) | Onboarding prioritisation within bounded load: recent videos auto-fill; the older back-catalogue stays on-demand |
|
||||
|
||||
The rationale for both paths is **onboarding prioritisation** — a new user should get summaries
|
||||
of their most recent, relevant videos quickly while the older back-catalogue fills in behind,
|
||||
all within the honest shared rate limit.
|
||||
The rationale for both paths is **onboarding prioritisation under an honest, bounded load** — a
|
||||
new user gets summaries of their most recent videos automatically, while the older back-catalogue
|
||||
is listed but summarised only on demand, so it never re-drives the shared rate gate every cycle.
|
||||
|
||||
### Newest-first batch ordering (ADR-018)
|
||||
|
||||
Within each scheduled pass, `RunOnce` uses a three-phase structure:
|
||||
|
||||
1. **Discover + persist**: walk all channels, `UpsertVideo` every candidate (so it appears in
|
||||
the list), apply pre-filters (seen/manual/backoff), collect surviving candidates.
|
||||
the list), apply pre-filters (seen/manual/backoff/**recency**), collect surviving candidates.
|
||||
The recency pre-filter (ADR-020) drops auto-mode videos published before
|
||||
`now - TAPIR_AUTO_SUMMARIZE_WINDOW` unless they are explicitly requested; an undated video is
|
||||
never aged out. They remain persisted/listed — only auto-summarisation is skipped.
|
||||
2. **Sort**: order candidates `published_at DESC, NULLS LAST, discovery_pos ASC`. Videos with
|
||||
no publish date (schema 001: nullable) sort after all dated content. The sort is in-memory
|
||||
(`slices.SortStableFunc`) — at current scale this is fine.
|
||||
@@ -235,7 +247,8 @@ Within each scheduled pass, `RunOnce` uses a three-phase structure:
|
||||
Before (per-channel inline): `[chanA-old, chanA-mid, chanB-new, chanB-null]`
|
||||
After (newest-first): `[chanB-new, chanA-mid, chanA-old, chanB-null]`
|
||||
|
||||
The set of processed videos is identical; only the order within a pass changes.
|
||||
The set of *processed* videos now also excludes auto-mode back-catalogue beyond the recency
|
||||
window (those stay listed, summarised on demand); within the processed set, only order changes.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user