mathias
b8fbc5a805
The concern is new-user first-contact: see some GOOD summaries fast or they don't return. Reframed as curation/latency for ~3 videos, NOT a 429/throughput problem (3 fetches is nowhere near the wall). Phase 1 (report-and-stop) verifies whether the existing cap-3 onboarding burst even fires today, what it delivers, and — critically — how much transcript-cache overlap exists between users (drives the blend). Phase 2 levers: cached-transcript-first (instant, zero-fetch), likely-good selection (has-captions/good-length, not just newest), and optionally the stronger model for the burst's few summaries. Blend deferred to the maintainer post-Phase-1. Explicitly NOT bulk-fetch, NOT credentials (ADR-010/026 dead end), NOT a client extension.
5.4 KiB
Spec — First-session "wow": verify & improve the onboarding summary burst
Repo: tapir · Size: small-medium · Solo session, INVESTIGATE-FIRST. Read CLAUDE.md, DECISIONS.md (ADR-018 onboarding burst origins, ADR-020 recency, ADR-021 transcript store, ADR-022 model chain, ADR-023 Shorts/metadata filter, ADR-024 caption memory), and the connect/onboarding flow. TBD, conventional commits.
The concern (maintainer): a brand-new user needs to see some good summaries quickly on connect — a first-session "wow" — or they don't come back (the Stage-0 gate is return usage). This is a curation/latency problem for ~3 videos, NOT a throughput/429 problem. Fetching 3 captions is nowhere near the rate limit; the wall only bites at hundreds. So nothing here fetches harder or touches the rate-gate budget meaningfully — it's about picking the right few videos and making sure they actually land. Do NOT turn this into a bulk-fetch / rate-pressure change.
PHASE 1 — verify what actually happens on connect today. REPORT and STOP before building.
The maintainer does not know if the existing onboarding burst is even firing. Establish ground truth:
- Does the connect-time summarize burst still fire? There was a cap-3 onboarding burst (summarize newest ~3 on YouTube connect, through the gate). Trace the current connect flow (OAuth callback → discovery → summarize). Is the burst still wired, or did a later refactor (recency window ADR-020, discovery changes, the ADR-023/024 filters) bypass or break it?
- If it fires, what does it deliver? For a realistic new connect: how many of the burst videos actually reach summarized vs. land caption-less / queued / rate-limited / weak? Use Jonas's actual onboarding history if traceable (his first summaries — fast, or trickled?).
- How much transcript-cache overlap exists between users? Query the shared transcript store (ADR-021): for the existing users, how many videos does a new user's subscription set already have cached transcripts for? This decides whether "cached-first" (instant, zero-fetch wow) is a real lever or nearly empty at pilot scale. This number drives the Phase-2 blend.
- What cheap "likely-good" signals are already available at connect? ADR-023 already pulls
Data API
videos.listmetadata (duration, liveBroadcastContent) — is caption-availability or anything quality-predictive already in hand, or what would it cost to know "this video has captions / reasonable length" before spending a fetch?
Report findings + a recommended Phase-2 shape, then STOP. The build differs sharply by what Phase 1 finds:
- burst not firing → fix is "make it fire" (and Phase 2 selection is gravy).
- fires but picks bad videos (caption-less/weak) → fix is the selection logic below.
- fires and works fine → the wow gap is quality, not selection → use the stronger model for the burst (see Phase 2 option C), maybe nothing else.
- near-zero cache overlap → drop cached-first; Phase 2 is just "likely-good fresh".
PHASE 2 — improve the burst (design AFTER Phase 1 findings; maintainer picks the blend)
Candidate levers (the maintainer chose 1+2; final blend decided by Phase 1's overlap number):
- (1) Cached-transcript-first: fill burst slots from videos whose transcripts are ALREADY in the shared store (ADR-021) — instant, zero fetch, guaranteed-captioned. Strength depends on the Phase-1 overlap number.
- (2) Likely-good selection (not just newest): for uncached slots, pick videos predicted to summarize well — has-captions (use/extend the ADR-023 metadata path), reasonable length (not a 3h podcast that truncates to mush, not a <2min clip), from the user's more-engaged channels if that signal exists. Skip newest-but-caption-less rather than burning a wow slot on a "no transcript" card.
- (C) Stronger model for the burst only: run the ~3 onboarding summaries through
gemma4-26b(stronger) instead ofphi4-mini, even though slower — first impressions matter disproportionately and it's only 3 videos, so quality > speed here. Cheap to try, possibly the highest-wow lever if Phase 1 shows the burst fires but summaries underwhelm. - Honest framing: present the burst as "summaries of a few of your videos to get you started — the rest fill in over the coming days," so "only a few" reads as intentional taster, not failure.
The blend (cached-instant vs. relevant-fresh) is explicitly deferred to the maintainer after Phase 1 reports the overlap number — do not hard-code it.
Boundaries
- NOT a throughput/429 change — it's ~3 videos; do not bulk-fetch or pressure the rate gate.
- No credentials-based caption fetch (ADR-010/026 dead end — auth doesn't work on timedtext).
- No client-side fetcher / extension (out of scope; that's a throughput idea, not a wow idea).
- Respect ADR-021 (read cached transcripts, never re-fetch), ADR-023/024 (don't undo the filters).
- Investigate-first: no selection-logic changes until Phase 1 is reported.
Tests (Phase 2, once shaped)
- Burst fires on connect and summarizes the selected set through the gate.
- Cached-transcript videos in the burst summarize with NO fetch (assert zero caption calls for those).
- Caption-less newest videos are NOT chosen for the burst (no "no transcript" wow-killer cards).
- (If model lever) burst uses the configured stronger model; rest of pipeline unchanged.