docs: replace evasion framing with honest onboarding-prioritisation rationale
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s

'Try now' and the newest-first batch implement onboarding prioritisation:
foreground (user-clicked 'Try now') summarises a chosen video on demand;
background batch summarises newest-first; both honour the shared rate gate.

Remove any prior framing that described 'Try now' as making traffic 'look
organic to YouTube' or as rate-limit evasion — that was not the rationale
and contradicts ADR-014's explicit account-safety constraint.

Correct statement: rate limiting is respected, not evaded. TAPIR_FETCH_RATE
and TAPIR_FETCH_BACKOFF are honest rate controls; they govern how fast Tapir
fetches captions, not how the requests appear to YouTube.

Architecture: add two-path model table (foreground/background, both through
globalFetchGate) and newest-first batch ordering doc (three-phase RunOnce,
before/after example).

ui-spec: add 'Try now' row with correct rationale; add pipeline stats bar row;
update Summarized-only filter row to mention sort-to-top.
This commit is contained in:
2026-06-06 21:29:28 +02:00
parent 0c0225f9c6
commit e472015c76
2 changed files with 32 additions and 1 deletions
+29
View File
@@ -208,6 +208,35 @@ This is the precondition that makes scheduled auto-summarize safe: without the g
multi-user scheduler pass could fire many concurrent timedtext requests from the same IP
within seconds, triggering 429s for all users.
### Two-path summarisation model
Both paths share `globalFetchGate` — rate limiting is **respected in both**, not routed around.
| Path | Trigger | Order | Rationale |
|------|---------|-------|-----------|
| **Foreground** | User clicks "Try now" on a rate-limited card (`POST /v/{id}/retry-now`) | Single chosen video | On-demand value: user picks a specific video to read now |
| **Background batch** | Scheduled discovery pass every `TAPIR_DISCOVERY_INTERVAL` | **Newest-first across all channels** (see below) | Onboarding prioritisation: most recent, relevant videos surface first |
The rationale for both paths is **onboarding prioritisation** — a new user should get summaries
of their most recent, relevant videos quickly while the older back-catalogue fills in behind,
all within the honest shared rate limit.
### Newest-first batch ordering (ADR-018)
Within each scheduled pass, `RunOnce` uses a three-phase structure:
1. **Discover + persist**: walk all channels, `UpsertVideo` every candidate (so it appears in
the list), apply pre-filters (seen/manual/backoff), collect surviving candidates.
2. **Sort**: order candidates `published_at DESC, NULLS LAST, discovery_pos ASC`. Videos with
no publish date (schema 001: nullable) sort after all dated content. The sort is in-memory
(`slices.SortStableFunc`) — at current scale this is fine.
3. **Process**: feed candidates to the engine in sorted order through `globalFetchGate`.
Before (per-channel inline): `[chanA-old, chanA-mid, chanB-new, chanB-null]`
After (newest-first): `[chanB-new, chanA-mid, chanA-old, chanB-null]`
The set of processed videos is identical; only the order within a pass changes.
---
## Sequence — core use case: new video summarized
+3 -1
View File
@@ -173,5 +173,7 @@ distinguishable.
| **Auto/manual summarization mode** | Per-user `auto_summarize`; manual lists new videos unsummarized and queues via `summarize_requested`; a mode toggle at `/account/summarize-mode`. Default is **true** for new users (migration 011, ADR-018); existing rows back-filled via migration 012. | Control over compute/noise — only summarize what the user cares about. | migration 006 (`748d5eb`, `bdbdce7`, `3014ee0`, `a269d4a`); migration 011/012 |
| **Public landing page** | `/welcome` mounted **outside** the auth guard; unauthenticated `/` redirects there; logout returns there (not `/auth/login`). (The spec guarded everything except `/healthz` and `/auth/*`.) | A first-time visitor needs a public "what is this / get started" page before the login wall. | `d83943c`, `0fdf2f7`, `3a27bf1`, `d208110`, `8ca374e`, `f15f57f` |
| **Invite onboarding** | Second registration path alongside Google OIDC. `tapir invite <email>` (CLI) creates a Dex local-password CRD in the `auth` namespace and prints an invite URL valid for 7 days. `/invite/{token}` (web) is a public page where the recipient sets a password; on submit, the Dex password is activated and the user is redirected to login. Token expiry is 7 days (`inviteTTL = 7 * 24 * time.Hour` in `cmd/tapir/invite.go`). The token is single-use: `ClaimInvitation` consumes it atomically on POST. | Allows inviting users who do not have a Google account or who should not use the Google OIDC upstream. | migration 009; `cmd/tapir/invite.go`; `internal/web/invite.go` |
| **Summarized-only filter** | `?summarized=1` query param on the list view. When set, only videos with a completed summary (`SummaryRow.Summarized = true`) are shown. Rendered as a "Summarized only" checkbox in the filter form. | Lets users focus on videos that are ready to read without scrolling past unsummarized entries. | `internal/web/view.go` (`Filter.OnlySummarized`) |
| **Summarized-only filter** | `?summarized=1` query param on the list view. When set, only videos with a completed summary (`SummaryRow.Summarized = true`) are shown. Rendered as a "Summarized only" checkbox in the filter form. Summarized videos also sort to the top of the unfiltered list (`ORDER BY (s.id IS NOT NULL) DESC, seen_at DESC`). | Lets users focus on videos that are ready to read; newly landing summaries are visible at the top without filtering. | `internal/web/view.go` (`Filter.OnlySummarized`, `ListVideos` ORDER BY) |
| **"Try now" button (rate-limited cards)** | `POST /v/{id}/retry-now` — replaces the passive "Retrying later" chip on rate-limited video cards with an active button. Clicking it: (1) clears the video's `rate_limited_at` backoff in the DB, (2) triggers an immediate `ProcessVideo` call through the shared `globalFetchGate`. If the fetch succeeds a summary lands; if YouTube returns 429 again the backoff is re-stamped and the card reverts. The rate gate is **respected, not bypassed** — this is on-demand foreground summarisation for a user-chosen video, not rate-limit evasion. The rationale is onboarding prioritisation: a user can choose a specific video to get a summary of quickly while the background batch fills in the rest newest-first. | Fast onboarding value — users get a summary of a chosen video without waiting for the next scheduled pass. | `internal/web/handlers.go` (`handleRetryNow`); `internal/adapters/youtube/gate.go` |
| **Pipeline stats bar** | A one-line status bar above the video list: `N summarized · M fetching soon · K no captions`. Computed from the unfiltered row set; hidden when all videos are summarized. Gives the user a clear read on pipeline state without any interaction. | Replaces the "why is nothing happening?" confusion when most videos are pending or rate-limited. | `internal/web/view.go` (`PipelineStats`, `pipelineStats`) |
| **Unavailable channels (account page)** | The `/account` page shows a "Unavailable channels" section when any channels returned HTTP 404 on the last discovery pass. Lists channel name, an "unavailable" badge, and the first-seen date. Data sourced from the `channel_errors` table (migration 013). | Surfaces silent failures so users know why some subscribed channels produce no new videos. | migration 013; `internal/web/account.go`; `internal/adapters/youtube/youtube.go` (`domain.ErrChannelUnavailable`) |