diff --git a/DECISIONS.md b/DECISIONS.md index 2cb66e3..c65a027 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1060,6 +1060,74 @@ no migration, no stored state to unwind. Spec: `docs/specs/chat-with-transcript. --- +## ADR-028 — Onboarding burst: pick likely-good videos, summarize them with a stronger model + +**Status:** Accepted (2026-06-11). **Refines ADR-018** (the connect-time burst) and **ADR-020** +(recency-bounded auto-summarize). **Builds on ADR-022** (the endpoint chain), **ADR-023** +(the discovery-time `videos.list` enrichment), and **ADR-021** (the shared transcript cache). +Triggered by a Phase-1 investigation of the live pilot DB. + +**Context.** A new user's first session decides whether they return (the Stage-0 gate, ADR-016). +The connect-time burst (ADR-018: summarize ≤`TAPIR_ONBOARD_SUMMARIZE_COUNT` newest videos so the +feed isn't empty) *fires* in production, but a live-DB investigation of the second pilot user +("Jonte") found it delivers a weak first impression for two reasons, and ruled out a third idea: + +1. **Junk picks.** Selection was pure newest-first (`NewestUnsummarizedVideoIDs`, + `ORDER BY published_at DESC`) with **no quality signal**. Jonte's live burst-3 were a + stock-ticker **livestream** + two regional news clips — newest, not best. The cheap signals + that *could* gate this (duration, live status) are fetched by ADR-023's `videos.list` + enrichment at discovery and then **thrown away**: the `videos.duration_s` column (migration + 001) was never written. +2. **Weakest model on the first impression.** All burst summaries ran on `koala/phi4-mini` — the + documented weak link (ADR-022 was born from its failures). The stronger, brain-validated + `iguana/gemma4-26b` was never used, even though the burst is only ~3 summaries. +3. **Cached-first instant summaries — REJECTED.** The idea: skip the fetch, summarize + already-cached transcripts (ADR-021) instantly. The pilot numbers kill it — only **11 videos** + overlap between the two users (~3% of each ~350–400-video library), **0** cached-and- + unsummarized, and a new user's newest-20 unsummarized are **20/20 NOT cached**. Newest-first + and cached-first are structurally incompatible: fresh uploads are exactly what nobody has + fetched. An empty lever at pilot scale. + +**Decision.** +1. **Persist `duration_s` at discovery.** `filterLowValue` (ADR-023) already has each candidate's + duration in hand; carry it onto the kept `domain.Video` and have `UpsertVideo` write it, + COALESCE-preserving a known value (the channel-title backfill stance, migration 014). No new + migration — the column exists. The connect-triggered discovery pass runs *before* the burst, + so a fresh user's candidates are enriched in time. +2. **Junk-avoiding selection.** A new `OnboardBurstVideoIDs(userID, limit, minSeconds, maxSeconds)` + keeps the newest-first order but drops a video when its duration is *known* and outside + `[minSeconds, maxSeconds]` — `minSeconds` = `TAPIR_MIN_VIDEO_SECONDS` (60, the Shorts floor), + `maxSeconds` = new `TAPIR_ONBOARD_MAX_VIDEO_SECONDS` (default 14400 = 4h, to drop multi-hour + livestream VODs that pass the live filter once ended). A NULL duration is **unknown** — kept + (degrade-open) but ranked after known-good rows. **has-captions stays un-gateable pre-fetch** + (only knowable after a gate fetch or a ~0-probability cache hit); selection only *avoids + known-junk*, it does not *promise* captions. +3. **Stronger model for the burst only.** `TAPIR_ONBOARD_SUMMARIZER_MODEL` (default + `iguana/gemma4-26b`) leads a burst-specific summarizer chain (onboard model first, then the + standard ADR-022 chain as resilience, deduped), wrapped in a burst-specific processor over the + *same* store/cache/sink — a pure wiring choice; the engine and ports are unchanged (ADR-003). + Empty or equal-to-primary collapses the burst back onto the shared processor. + +**Not a throughput change.** The caption rate gate (ADR-014) and the foreground priority lane +(ADR-026) are untouched — same pacing, same cap. This changes *which* ≤3 videos the burst spends +its fetches on and *which model* summarizes them, never how fast or how many. The engine's +existing read-stored-first (ADR-021) is unchanged and still yields a free instant summary on the +rare cache hit — we simply do not *select* for cache hits. + +**Consequences.** Better odds of a strong first session: the burst avoids the obvious junk and +runs the better model on the one impression that decides return. The selection improvement is +forward-looking — existing rows have NULL `duration_s` until their next discovery pass backfills +it (lazy, like channel_title); a brand-new user benefits immediately because connect-discovery +runs first. `duration_s` becoming live also unblocks future length-aware features (feed sorting, +"long read" badges) for free. + +**Reversibility.** Pure config + wiring + one column write + one query, no migration. +`TAPIR_ONBOARD_MAX_VIDEO_SECONDS=0` (and `TAPIR_MIN_VIDEO_SECONDS=0`) restores pure newest-first; +`TAPIR_ONBOARD_SUMMARIZER_MODEL=""` collapses the burst back to the shared processor. +Spec: `docs/specs/onboarding-wow-burst.md`. + +--- + ## Rejected alternatives Approaches considered during the 2026-06-02 planning + grill session and **deliberately not @@ -1083,6 +1151,7 @@ maps to the ADR that settles it. | Feedback-based Stage 0 gate (friends saying it's useful) | Politeness bias makes asked-for feedback the least reliable signal; return-usage is the real test | ADR-016 | | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 | | k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 | +| Cached-transcript-first onboarding burst (instant, zero-fetch picks) | Live pilot DB: ~3% cross-user video overlap, 0 cached-and-unsummarized, a new user's newest-20 are 20/20 uncached — newest-first and cached-first are structurally incompatible. Empty lever at pilot scale | ADR-028 | If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — not a silent reversal. diff --git a/docs/specs/onboarding-wow-burst.md b/docs/specs/onboarding-wow-burst.md index ce1fec5..00a59fc 100644 --- a/docs/specs/onboarding-wow-burst.md +++ b/docs/specs/onboarding-wow-burst.md @@ -1,74 +1,128 @@ -# Spec — First-session "wow": verify & improve the onboarding summary burst +# Spec — Onboarding "wow" burst: better picks, stronger model -**Repo:** tapir · **Size:** small-medium · **Solo session, INVESTIGATE-FIRST.** Read CLAUDE.md, -DECISIONS.md (ADR-018 onboarding burst origins, ADR-020 recency, ADR-021 transcript store, -ADR-022 model chain, ADR-023 Shorts/metadata filter, ADR-024 caption memory), and the -connect/onboarding flow. TBD, conventional commits. +**Repo:** tapir · **Size:** medium · **Solo session** (not a swarm). -**The concern (maintainer):** a brand-new user needs to see *some* good summaries quickly on -connect — a first-session "wow" — or they don't come back (the Stage-0 gate is return usage). -**This is a curation/latency problem for ~3 videos, NOT a throughput/429 problem.** Fetching 3 -captions is nowhere near the rate limit; the wall only bites at hundreds. So nothing here fetches -harder or touches the rate-gate budget meaningfully — it's about picking the *right* few videos -and making sure they actually land. Do NOT turn this into a bulk-fetch / rate-pressure change. +> **Status: built (v0.25.0, ADR-028).** This supersedes the original investigate-first brief +> (committed as the prior version of this file): Phase 1 was run against the live pilot DB and its +> findings are folded into "Why this exists" below; Phase 2 was built as described here. The one +> brief lever NOT built — the honest "the rest fill in over the coming days" framing copy — is +> listed under *Explicitly NOT in this slice*. -## PHASE 1 — verify what actually happens on connect today. REPORT and STOP before building. -The maintainer does not know if the existing onboarding burst is even firing. Establish ground -truth: -1. **Does the connect-time summarize burst still fire?** There was a cap-3 onboarding burst - (summarize newest ~3 on YouTube connect, through the gate). Trace the *current* connect flow - (OAuth callback → discovery → summarize). Is the burst still wired, or did a later refactor - (recency window ADR-020, discovery changes, the ADR-023/024 filters) bypass or break it? -2. **If it fires, what does it deliver?** For a realistic new connect: how many of the burst - videos actually reach *summarized* vs. land caption-less / queued / rate-limited / weak? Use - Jonas's actual onboarding history if traceable (his first summaries — fast, or trickled?). -3. **How much transcript-cache overlap exists between users?** Query the shared transcript store - (ADR-021): for the existing users, how many videos does a *new* user's subscription set - already have cached transcripts for? This decides whether "cached-first" (instant, zero-fetch - wow) is a real lever or nearly empty at pilot scale. **This number drives the Phase-2 blend.** -4. **What cheap "likely-good" signals are already available at connect?** ADR-023 already pulls - Data API `videos.list` metadata (duration, liveBroadcastContent) — is caption-availability or - anything quality-predictive already in hand, or what would it cost to know "this video has - captions / reasonable length" *before* spending a fetch? +**Why this exists.** A new user's first session decides whether they return (the Stage-0 gate, +VISION.md). On connect, Tapir fires a capped burst (≤`TAPIR_ONBOARD_SUMMARIZE_COUNT`, default 3) +that summarizes the user's newest unsummarized videos so the feed isn't empty (the burst itself +works — wired in `cmd/tapir/discovery.go` → `cmd/tapir/main.go` `onboard`). A Phase-1 +investigation of the live pilot DB found the burst *fires* but delivers a **weak first +impression** for two concrete reasons, and ruled out a third idea: -**Report findings + a recommended Phase-2 shape, then STOP.** The build differs sharply by what -Phase 1 finds: -- burst not firing → fix is "make it fire" (and Phase 2 selection is gravy). -- fires but picks bad videos (caption-less/weak) → fix is the selection logic below. -- fires and works fine → the wow gap is *quality*, not selection → use the stronger model for the - burst (see Phase 2 option C), maybe nothing else. -- near-zero cache overlap → drop cached-first; Phase 2 is just "likely-good fresh". +1. **Picks are junk.** Selection is pure newest-first (`videos.NewestUnsummarizedVideoIDs`, + `ORDER BY published_at DESC`) with **zero quality signal**. Pilot user "Jonte"'s live burst-3 + were a stock-ticker **livestream** + two regional news clips — the newest, not the best. +2. **Weakest model on the first impression.** All of Jonte's summaries ran on + `koala/phi4-mini` (the documented weak link — ADR-022 was born from its failures). The + stronger, brain-validated `iguana/gemma4-26b` was never used for the burst. +3. **Cached-first is empty at pilot scale — REJECTED.** The idea (summarize already-cached + transcripts instantly, zero fetch) dies on the numbers: only **11 videos** overlap between the + two pilot users (~3% of each library), **0** cached-and-unsummarized, and a new user's + newest-20 unsummarized are **20/20 NOT cached** — newest-first and cached-first are + structurally incompatible (fresh uploads are exactly what nobody has fetched yet). Not built. -## PHASE 2 — improve the burst (design AFTER Phase 1 findings; maintainer picks the blend) -Candidate levers (the maintainer chose 1+2; final blend decided by Phase 1's overlap number): -- **(1) Cached-transcript-first:** fill burst slots from videos whose transcripts are ALREADY in - the shared store (ADR-021) — instant, zero fetch, guaranteed-captioned. Strength depends on the - Phase-1 overlap number. -- **(2) Likely-good selection (not just newest):** for uncached slots, pick videos predicted to - summarize well — has-captions (use/extend the ADR-023 metadata path), reasonable length (not a - 3h podcast that truncates to mush, not a <2min clip), from the user's more-engaged channels if - that signal exists. Skip newest-but-caption-less rather than burning a wow slot on a "no - transcript" card. -- **(C) Stronger model for the burst only:** run the ~3 onboarding summaries through - `gemma4-26b` (stronger) instead of `phi4-mini`, even though slower — first impressions matter - disproportionately and it's only 3 videos, so quality > speed here. Cheap to try, possibly the - highest-wow lever if Phase 1 shows the burst fires but summaries underwhelm. -- **Honest framing:** present the burst as "summaries of a few of your videos to get you started — - the rest fill in over the coming days," so "only a few" reads as intentional taster, not failure. +This is a **curation/latency problem for ~3 videos, NOT a throughput/429 problem** — fetching 3 +captions is nowhere near the rate limit. Nothing here fetches harder or pressures the rate gate; +it picks the right few videos and runs a better model on them. -**The blend (cached-instant vs. relevant-fresh) is explicitly deferred to the maintainer after -Phase 1** reports the overlap number — do not hard-code it. +Read `CLAUDE.md`, `DECISIONS.md` (esp. ADR-014, ADR-018, ADR-020, ADR-021, ADR-022, ADR-023, +and the new **ADR-028**), and `VISION.md` (the Stage-0 gate) first. TBD — commit directly to +`main`, one logical change per commit, conventional commits, `task check` green before each +commit, `templ generate` if any view changes (none expected). -## Boundaries -- NOT a throughput/429 change — it's ~3 videos; do not bulk-fetch or pressure the rate gate. -- No credentials-based caption fetch (ADR-010/026 dead end — auth doesn't work on timedtext). -- No client-side fetcher / extension (out of scope; that's a throughput idea, not a wow idea). -- Respect ADR-021 (read cached transcripts, never re-fetch), ADR-023/024 (don't undo the filters). -- Investigate-first: no selection-logic changes until Phase 1 is reported. +## Decisions already made (do not reopen) +- **Not a throughput change.** The caption rate gate (ADR-014) is untouched — same pacing, same + priority lane (ADR-026). This slice changes *which* ≤3 videos the burst spends its fetches on + and *which model* summarizes them, never how fast or how many. +- **Cached-first is dropped** (ADR-028, the 3% overlap). The engine's existing read-stored-first + (ADR-021, `resolveTranscript`) stays — it already gives a free instant summary on the rare + cache hit, transparently. We do not *select* for cache hits. +- **has-captions is not a pre-fetch signal.** It is only knowable after a gate fetch (or a cache + hit, ~0 for new videos). Selection can only *avoid known-junk* (Shorts/live/over-long) — it + cannot *guarantee* captions. The spec is honest about this: better odds, not a promise. +- **No credentialed caption fetch** (ADR-010/ADR-026 dead end). **No client extension.** -## Tests (Phase 2, once shaped) -- Burst fires on connect and summarizes the selected set through the gate. -- Cached-transcript videos in the burst summarize with NO fetch (assert zero caption calls for - those). -- Caption-less newest videos are NOT chosen for the burst (no "no transcript" wow-killer cards). -- (If model lever) burst uses the configured stronger model; rest of pipeline unchanged. +## 1. Persist `duration_s` at discovery (the enabling change) +The `videos.duration_s` column exists (migration 001) but is **never written** — ADR-023's +`filterLowValue` (`internal/adapters/youtube/youtube.go`) already fetches each candidate's +duration via the cheap quota `videos.list` call, uses it to drop Shorts/live, then **discards +it**. Stop discarding: +- Add `DurationSeconds int` to `domain.Video`. +- In `filterLowValue`, set `DurationSeconds` on each kept video from the `videos.list` `meta`. +- `UpsertVideo` writes `duration_s`, **COALESCE-preserving** a known value (never overwrite a + real duration with 0/unknown), mirroring the `channel_title` backfill stance (migration 014). +- No new migration — the column is already there. + +Consequence: a fresh user's connect-triggered discovery pass runs **before** the onboard burst +(`Enqueue`: `run()` then `onboard()`), so duration is populated for the burst's candidates at +connect. Existing rows backfill on their next discovery pass; until then their `duration_s` is +NULL and treated as "unknown" (§2). + +## 2. Junk-avoiding burst selection +New store method, RLS-scoped via `withUser`: + +``` +OnboardBurstVideoIDs(ctx, userID string, limit, minSeconds, maxSeconds int) ([]string, error) +``` + +- Same base as the old `NewestUnsummarizedVideoIDs`: the user's videos with no summary yet, + `ORDER BY published_at DESC NULLS LAST, seen_at DESC`, `LIMIT limit`. +- **Exclude known-junk**: a row is dropped only when `duration_s IS NOT NULL` **and** + (`duration_s < minSeconds` OR `duration_s > maxSeconds`). A NULL duration is **unknown** — kept + (degrade-open: never starve the burst because metadata is missing), but ordered *after* rows + with a known-good duration so a freshly-enriched good pick wins when both exist. +- `minSeconds` reuses `TAPIR_MIN_VIDEO_SECONDS` (default 60 — the Shorts floor, ADR-023). + `maxSeconds` is new: `TAPIR_ONBOARD_MAX_VIDEO_SECONDS` (default 14400 = 4h) — drops the + multi-hour livestream VODs that pass the live filter once ended. +- `minSeconds<=0` and `maxSeconds<=0` each disable that bound (so `0/0` == the old + newest-first behaviour, the reversibility lever). +- The burst switches to this method; `NewestUnsummarizedVideoIDs` is removed (fully superseded — + `OnboardBurstVideoIDs(., 0, 0)` is identical pure-newest behaviour). + +## 3. Stronger model for the burst +The burst summarizes only ≤3 videos, so a slower, stronger model is affordable exactly here. +- New config `TAPIR_ONBOARD_SUMMARIZER_MODEL` (default `iguana/gemma4-26b` — the brain-validated + homelab general-purpose model, already the ADR-022 fallback). +- Build a **burst-specific summarizer chain** that puts the onboard model **first**, then the + standard chain (primary → local fallback → cloud) as resilience, deduped. Wrap it in a + burst-specific `engineProcessor` reusing the same store/transcript-cache/sink — a pure wiring + choice, engine and ports unchanged (Clean Architecture, ADR-003). +- The `onboard` closure uses the burst processor instead of `app.Processor`. +- **Collapse cleanly**: when `OnboardSummarizerModel` is empty or equals `SummarizerModel`, the + onboard path reuses `app.Processor` (no separate chain) — the reversibility lever. +- Local-first preserved: the onboard model is a local alias; the cloud endpoint stays last in the + chain, so a client/NDA deployment with `TAPIR_CLOUD_FALLBACK_MODEL=""` keeps burst content + local too. + +## 4. Behaviour spec + docs +- Add scenarios to `docs/use-cases/connect_account.feature` (the connect → burst flow): burst + skips a too-long/live video in favour of a reasonable-length one; burst summarizes with the + stronger model first. Map them in `scenarioCoverage` so `TestScenarioCoverage` stays green. +- Update `docs/architecture/architecture.md` (the onboarding-burst section) to describe the + junk-avoiding selection + the burst model override. +- ADR-028 in `DECISIONS.md` records the rationale (incl. the rejected cached-first lever). + +## Success criteria +- `task check` green (fmt, vet, lint, `go test -p 1 ./...`). +- A unit test proves `OnboardBurstVideoIDs` drops a known too-long / sub-min video and keeps a + good one, newest-first, RLS-scoped, unsummarized-only. +- A test proves discovery persists `duration_s` and does not clobber it on re-upsert. +- A test proves the burst chain leads with the onboard model (then the standard chain). +- Config defaults + bounds tested (`OnboardMaxVideoSeconds`, `OnboardSummarizerModel`). +- No change to the rate gate, fetch pacing, or burst cap. `0/0` + empty model == prior behaviour. + +## Explicitly NOT in this slice +- Cached-first selection (rejected, ADR-028). +- Any caption-availability *guarantee* (impossible pre-fetch). +- **Honest "taster" framing copy** ("summaries of a few of your videos to get you started — the + rest fill in over the coming days"). A good lever from the original brief, but it's a UI/copy + change with no backend dependency; deferred to a UI pass, tracked as an issue. +- Backfilling `duration_s` for existing rows via a migration (it backfills lazily on discovery). +- Return-nudges / digests (ADR-020: poisons the unprompted-return signal). +- Raising fetch throughput, multi-IP, or Whisper (out of scope; the gate is deliberate).