The first friendly-pilot live run produced zero summaries: koala/phi4-mini hit
three silent failure modes — 8k context overflow on long transcripts (HTTP 400),
intermittent malformed JSON (highlights as a bare string), and no fallback wired
at all (summarizer.New(primary, nil)).
Keep phi4-mini as the fast primary and add resilience around it:
- Ordered endpoint chain (summarizer.NewChain): phi4-mini → koala/phi4-14b
(local) → berget/mistral-small (worst-case external). All reached through the
one LiteLLM gateway by alias.
- A parse failure now advances the chain like a transport error — the old
Primary→Fallback shape returned the parse error without trying anyone else.
- Tolerant parse: highlights/takeaways coerce string→[]string, absorbing the
common small-model quirk without spending a fallback round-trip.
- Transcript truncation (TAPIR_MAX_TRANSCRIPT_CHARS=18000) prevents the overflow
rather than recovering from it; validated to fit phi4-mini's 8k window.
- Bounded completion budget (TAPIR_SUMMARY_MAX_TOKENS=1500) — the old 8192 budget
itself contributed to the overflow.
Local-first guarantee preserved by ordering: external endpoint is tried only
after every local one fails. TAPIR_CLOUD_FALLBACK_MODEL="" disables it entirely
for client/NDA deployments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update the CLAUDE.md orientation block: last tag v0.14.0, migrations
001–015, and a transcript-persistence bullet (shared non-RLS store,
engine reads stored-first). The stale "v0.9.0" reference is corrected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the ADR-021 promise as a mapped BDD scenario: re-analyzing a
stored video reads the stored transcript and does not fetch captions.
Since paste-a-URL and the onboarding burst summarize through the same
engine chokepoint (resolveTranscript, store-first), this one scenario
covers their reuse path too — there is exactly one gated caption entry
point (youtube.FetchTranscript → WaitFetchGate) and one engine caller in
front of it, so the dedup is structural, not per-feature.
Mapped to TestProcessNewVideo_SecondSummarizeDoesNotRefetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The engine now resolves transcripts store-first: a stored transcript —
including a stored SourceNone — is summarized without touching YouTube,
so re-analysis never re-fetches. On a miss it fetches through the source
(caption call still gated, ADR-014) and persists the terminal outcome for
the next analysis by any user. A transient SourceRateLimited is surfaced
to the runner for per-user backoff but never cached, so persistence can
never mask a 429 as a permanent "no transcript".
The TranscriptStore is optional (nil → fetch every time), keeping the
pure-core and scaffold wiring valid. cmd/tapir wires the store as both
summary sink and transcript cache, so `tapir run` and the web summarize
path (incl. paste + onboarding) all share the dedup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reshape the dead per-user transcripts table (PK videos.id, user_id,
RLS-FORCEd — never read or written by app code) into the shared public
caption store ADR-021 specifies: keyed by (provider, provider_video_id),
no user_id, NOT RLS-scoped. Migration 015 (reversible). Add
ports.TranscriptStore + Store.GetTranscript/SaveTranscript via the raw
pool (no withUser): public content, shared across users by construction.
SaveTranscript persists only terminal outcomes (captions/none) and
refuses SourceRateLimited so a transient 429 can never be stored as a
false permanent absence (ADR-014).
Flip the isolation proof: transcripts leaves the RLS-scoped set;
TestTranscriptsTableIsSharedNotRLS asserts it is the SINGLE non-RLS
surface (writable/readable with no user scope, no user_id column, RLS off
on it alone, still on every user-owned table) — the proof the
public-content classification was applied exactly here and leaked nowhere.
appPool made idempotent so two tests can build it. Adjust the 010/011/014
up-down migration tests for the new HEAD. account.go: user deletion no
longer strips shared transcripts. Reconcile data-model.md + CLAUDE.md.
Wiring the engine to read-stored-first is the next commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Persist transcripts in a single shared table keyed by
(provider, provider_video_id) — public caption content, NOT RLS-scoped —
so re-analysis (re-summarize, paste of an already-seen video, a second
user with overlapping subs) never re-fetches from YouTube. The avoided
cost is the rate-gated, reputation-risky caption fetch (ADR-010/014), not
LLM re-summarization, which is why this reopens the transcripts half of
the "no global cross-tenant table" rejection while videos stay per-user.
Summaries remain RLS-scoped (ADR-012 unchanged). The gate is neither
bypassed nor weakened — persistence reduces fetch frequency, not pacing.
Annotate the rejected-alternatives row to record the partial reopen.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The free-text 'channel' filter was dead: it exact-matched SummaryRow.Channel,
which is just the provider ('youtube'), because videos never stored their source
channel. Now they do.
- migration 014: videos.channel_title (nullable; existing rows backfill on the
next discovery pass, pasted videos immediately).
- discovery (NewVideos) + paste (VideoByID) populate channel_title; UpsertVideo
persists it, preserving an existing title when an update arrives empty.
- store.DistinctChannels lists a user's channels (RLS-scoped); SummaryRow carries
ChannelTitle via the shared projection.
- Filter: single Channel -> Channels []string, matching on ChannelTitle; the feed
renders a multi-select of DistinctChannels (hidden until channels exist).
- migrate tests: 014 reversibility + fixed the relative-step counts in the 010/011
up/down tests (014 shifted the topology).
TDD throughout: channel persist + distinct, adapter channel wiring, multi-channel
filter match, handler channel filter, migration up/down.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
hasConnected was computed only inside the buckets.empty() branch (it was added
for the empty-state copy), so a connected user WITH videos got hasConnected=false
and never saw the paste box (#2-regression of the v0.12.0 paste UI). Compute it
on every list render. Test: connected user with a non-empty feed sees the box.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds paste_url.feature (valid/invalid/not-found/dedup, +@pending no-captions)
and an onboarding-burst scenario on connect; all non-pending scenarios mapped in
the coverage gate to their existing Go tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5b: connected users get a 'Summarize any video' URL input on the feed; submit
posts to /paste (HTMX) and swaps the resulting card / inline error into the feed.
7: the connect flash now sets expectations for the async onboarding burst —
'finding your subscriptions, your newest videos will appear below as summarized'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5c: app.Fetcher = a per-user YouTube videoFetcher, so POST /paste mounts and
resolves arbitrary-video metadata (Feature 2 goes live).
6: the connect trigger now runs an onboarding burst after discovery — summarize
up to TAPIR_ONBOARD_SUMMARIZE_COUNT of the user's newest unsummarized videos via
the gated Processor (Feature 1). Hard cap; explicit so it bypasses recency; every
fetch still through globalFetchGate. No-op when count=0 or queue-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
POST /paste: parse the video id, fetch metadata via the VideoFetcher port (Data
API, ungated), upsert a subscription-less row scoped to the user (idempotent =
dedup), and — unless already summarized — RequestSummarize + start immediate
processing through the SAME globalFetchGate as the Summarize button. Explicit
paste overrides the recency window; a captionless video degrades to the honest
'no transcript' terminal state via the engine (ADR-010). Invalid URL -> 400,
not-found -> 404, both add nothing. Route mounts only when a Fetcher is wired.
Moves the video-not-found sentinel to domain (shared by adapter + web, no
cross-adapter coupling). Tests: valid add+queue, invalid, not-found, dedup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
videos.list (part=snippet) for a single id, including channels the user does
not follow. Data API call (1 quota unit), NOT the rate-limited caption path —
ungated metadata; only the later transcript fetch hits globalFetchGate. Returns
a subscription-less domain.Video scoped to the user, or ErrVideoNotFound for a
deleted/private/typo'd id. Foundation for Feature 2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Returns up to limit of a user's newest videos (published_at DESC, NULLS LAST)
that have no summary yet. RLS-scoped via withUser — the test proves a second
user's newer video never leaks. Drives the connect-time onboarding burst
(Feature 1); the caller routes each through the shared rate gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bounds the connect-time onboarding summary burst (Feature 1). Hard-capped at 5
and clamped (negative->0, >cap->cap) so onboarding can never bulk-fetch; 0
disables. The cap bounds COUNT only — every fetch still flows through the shared
caption rate gate (ADR-014).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pure parser for watch?v=, youtu.be/, shorts/, embed/, and bare ids; rejects
non-YouTube hosts and malformed input. Foundation for paste-a-URL summarize
(Feature 2). No fetch, no gate interaction — parsing only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A newly connected account showed no videos until the next 2h scheduled pass —
the gap that made onboarding look broken (a second user connected, saw nothing,
read as failure). The connect callback now fires an out-of-band discovery pass
for the connecting user, so videos appear promptly.
Concurrency: scheduled and connect-triggered passes share one lock (serialize),
preserving the single-fetcher invariant (ADR-018). A trigger interleaves between
the scheduler's per-user passes rather than fetching concurrently or waiting for
a whole pass. The trigger runs on the server ctx (survives the redirect) and is
non-blocking for the request goroutine.
Scope: connect-trigger only. The optional login-refresh / "Discover now" button
from #6 are intentionally not built — an unconditional login hook risks 429
storms (per the ticket's own recommendation); defer until wanted.
TDD: TestCallbackTriggersDiscovery, TestSerializeRunsOneAtATime,
TestDiscoveryTriggerEnqueueRunsUser; new BDD scenario mapped.
Refs #6
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scheduler enumerates every user_identities row (ListAllUsers) and ran a
discovery pass for each — including users who never connected a video source.
Their per-user runner then tried to resolve a YouTube refresh token that was
never minted, logging a spurious "secrets: ref not found:
youtube/<uid>/refresh_token" every tick (e.g. stale Dex-era orphan identities
left by the Authentik migration).
Skip users whose ConnectionsForUser is empty before running their pass. Removes
the recurring noise — which actively misled a debug session into thinking a
healthy onboarded user was broken — with no change to connected users.
Refs #7
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the gap where docs/use-cases/*.feature claimed to be the behavior spec
but nothing executed them — so scenarios drifted (the stale "auto summarizes
every new video" and "manual is the default" were proof).
Decision (per issue #5 BDD-runner fork): no godog — keep .feature as design
records, add a cheap name-coverage gate instead. TestScenarioCoverage parses
every scenario and asserts each non-@pending one maps to an existing Go test in
the scenarioCoverage manifest; it flags unmapped scenarios, missing/renamed
tests, and stale entries. It checks the link, not that the test exercises the
scenario (the deliberate trade for skipping godog).
Also:
- Fix the stale ADR-018 drift: "Manual is the default" -> auto is the default
for new users; added an explicit default scenario + a plain manual scenario.
- Tag 4 documented-but-unbuilt/untested scenarios @pending with reasons (Vimeo
connect, BYO config flow, logout->welcome, re-register-after-delete) so they
are tracked without a false coverage claim.
- CLAUDE.md BDD section now describes the real setup (design records + the gate
+ @pending convention) instead of claiming an executable spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the living docs current with the recency-bounded auto-summarize + sparse
honesty + feed IA bundle (ADR-020):
- requirements (BDD): summarize_mode.feature — auto now summarizes RECENT new
videos; added a scenario for older videos (listed, on-demand), recency note.
- architecture.md: summarization-mode + new list-surface paragraph; scheduler
diagram + two-path table + three-phase pass now show the recency pre-filter;
dropped stale "Summarize now".
- data-model.md: auto_summarize is recent-only, older on-demand.
- README.md: one-line recency note on the serve scheduler.
- ui-spec.md: appended the as-built ADR-020 row (supersedes earlier copy/sort).
- specs/{video-card-states,newest-first-ordering,scheduled-discovery}.md:
superseded/extended banners pointing at ADR-020 (kept as design records).
Docs-only; task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the architecture decision behind this bundle: bound auto-summarize to
a recency window (refines ADR-018; bounds load against the ADR-014 gate without
fetching harder), surface scarcity honestly, and collapse the un-summarized
back-catalogue in a single feed. Records the return-nudge as a deliberate
non-goal (it would contaminate the Stage-0 unprompted-return signal, ADR-016).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the card copy guard so "Try now", "Summarize now", "Fetching soon", and
"the next run" can't silently return to any card state (UX review honesty pass).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Watched and skipped are mutually exclusive (the store clears one when the other
is set), but rendered as three independent-looking buttons the exclusivity was
invisible. Group watched|skipped into a single segmented control and keep Saved
apart as an independent toggle (UX review C5). HTMX posting and the active/✓/
aria-pressed semantics are unchanged; extracted a shared actionButton component.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The register step asked the user to accept "the terms of use" with no terms
linked anywhere — ceremony accepting nothing on a friends-only tool (UX review
C4). Remove the checkbox and the server-side acceptance requirement; only a
display name is required now.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The summary detail page only returned to the list via the brand logo. Add an
explicit "← Summaries" link at the top (UX review C3).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
At current scale the date-range pickers are dead weight (UX review C1/C2):
- Drop the From/To date inputs from the filter bar; keep the Channel field and
the "Summarized only" toggle. (Filter still parses from/to for hand-built
URLs and apply() compatibility — only the UI is removed.)
- Hide the filter bar entirely on a genuinely empty account (no rows AND no
active filter) so the connect CTA stands alone; a filter that matches nothing
still shows the bar so it can be cleared.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop the un-summarized back-catalogue from burying the readable summaries
(UX review B3/B4). One feed, with a noise-collapse — not sections:
- Summarized + recent un-summarized videos lead inline as cards.
- Un-summarized videos older than the recency window collapse into a single
"Show N older videos — summarize on demand" disclosure (they will not
auto-fill; they are manual-only). Window comes from App.RecencyWindow
(= cfg.AutoSummarizeWindow); 0 disables the collapse (all inline).
- Caption-less videos collapse into one honest line ("N videos have no
captions and can't be summarized") instead of N dead terminal cards.
bucketRows is a pure classifier (cutoff-driven; undated rows never age out);
App gains RecencyWindow + an injectable clock for the cutoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ListVideos now sorts summarized-first, then published_at DESC with undated
videos last, then seen_at DESC as a tiebreak (was seen_at only). Aligns the
list with the recency framing — newest content surfaces first — so the
recency-bounded feed reads coherently (UX review B2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In automatic mode the scheduler now only summarizes videos published within
TAPIR_AUTO_SUMMARIZE_WINDOW (default ~7d). Older videos are still discovered
and listed — they keep the manual "Summarize" affordance — but are not
auto-processed, so a large back-catalogue (the maintainer's ~256-deep queue)
stops self-inflicting 429s against the per-IP caption gate each cycle (UX
review B1, recency design).
- runner.WithAutoWindow + Stats.SkippedTooOld; tooOld() treats a zero window
as disabled and an undated video as never-aged-out (processed, not stranded).
- An explicit manual request bypasses the bound even in auto mode (requested
videos are loaded in auto mode when a window is active).
- Wired through cmdRun, the scheduler's per-user runner, sumStats, and pass
logging. config: TAPIR_AUTO_SUMMARIZE_WINDOW (default 168h), .env.example.
- Account copy (A7) updated to match: "Automatic summarizes new videos from
about the last week; older videos stay browsable — summarize on demand."
The rate gate is untouched; the manual path still serialises through it. This
bounds auto LOAD, it does not fetch harder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The product promise is "decide what's worth your time", but the detail page
buried Takeaways — the verdict that answers that — below the full Summary.
Reorder to Takeaways → Highlights → Summary so the attention-saving payload
leads (UX review A8). Data already existed; this is a section reorder only.
Conditional sections mean a video without takeaways/highlights still leads
with the Summary naturally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The landing page promised "if you have an invite link, it will set up your
account automatically" — but invites moved to Authentik (ADR-019); Tapir no
longer handles invite links and "Get Started" goes straight to OIDC. Replace
with honest "invite-only — if you've been invited, sign in" and set the
gradual-fill expectation before the login wall (UX review A5). Test pins the
stale phrase out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the sparse reality legible instead of implying abundance or imminence
(UX review A1-A4, A6):
- Empty-connected state drops the impossible "Run `tapir run`" instruction
(no shell for web users; discovery is in-process since ADR-018) for a
passive "summaries appear gradually, check back later".
- Pipeline bar reframes counts by what the user can do: "N ready · M in queue
· K no captions" (was "summarized / fetching soon / pending").
- A one-line note explains captions are fetched slowly on purpose to respect
YouTube's limits — turning confusing emptiness into intentional design.
- Card state for throttled videos reads "In queue", not "Fetching soon…"
(256 items behind a per-IP gate are not all imminent — ADR-014).
- Quiet nudge button drops the over-promising "now": "Summarize", not
"Summarize now". On click the card still honestly becomes "Queued".
- Queued card says "summarizing shortly", not "waiting for the next run"
(no scheduler jargon).
Pure copy/label — no logic, DB, or fetch-rate change. The rate gate is
untouched; scarcity is surfaced, never engineered around.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Prioritized UX findings for the product as it actually is — sparse feed,
respected caption rate limit, recency-bounded auto-summarize (incoming),
single-user. 15 findings, NOW/LATER tagged. P0s target the first-contact
return-cliff that the Stage-0 gate depends on.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record the invite-provisioning removal; mark ADR-017 superseded; fix
ui-spec invite-onboarding + auth-delegation sections to reflect Authentik.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Authentik owns invites now (infra ADR-0001). Delete adapters/dex, the
/invite set-password UI, the tapir invite CLI, the InvitationStore/
DexPasswordCreator ports + App wiring, the invite Templ pages, and the
invite Taskfile target. New users are invited via Authentik, log in via
OIDC, and hit the existing /register gate. invitations table (mig 009)
left in place (append-only; harmless). task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tapir used human-readable substitutions ('@' -> '-at-', '.' -> '-dot-') when
deriving the Password CR name from an email. Dex's internal passwordID() maps
every non-[a-z0-9-] character to plain '-'. This caused a name mismatch:
Tapir wrote the CR as 'mathias-at-d-ma-dot-be', Dex looked it up as
'mathias-d-ma-be', got not-found, and returned 'Invalid credentials' on every
invite login — while static configmap passwords (a different code path) worked
fine. Diagnosed by adding the email to staticPasswords and confirming login
succeeded, proving the kubernetes CR lookup was the failure point.
The original NOTE claimed Dex's kubernetes storage types Hash as []byte,
requiring the bcrypt string to be base64-encoded before storage. This was
wrong: Dex v2.41 stores and compares the hash field as a plain string. The
base64-encoding caused every invite login to fail with 'Invalid credentials'
because Dex passed the base64 bytes (starting with 'J' not '$') directly to
bcrypt. Static passwords in the configmap always used raw bcrypt strings and
worked fine — confirming the dynamic CR encoding was the bug.
Five explicit footer states, status-primary:
1. Summarized — chip + actions, no button (unchanged)
2. No captions (TranscriptStatus=="none") — NEW: 'No transcript available' muted
text, no button, no POST URL. Removes the dead-end 'Summarize' button that
tried and failed when there were no captions to fetch.
3. Queued (SummarizeRequested) — chip + muted text, no button (unchanged)
4. Rate-limited — 'Fetching soon…' + quiet 'Summarize now' → /retry-now
5. Pending — 'Not summarized' + quiet 'Summarize now' → /summarize
One verb ('Summarize now'), one quiet style (.btn-quiet, renamed from .btn-retry
which was state-specific). User doesn't see the internal pipeline distinction;
both buttons post to their existing handlers unchanged. Form class renamed
card-nudge-form. Dropped engineer-facing tooltip; user-facing hint added.
'Try now' wording removed entirely.
'Try now' and the newest-first batch implement onboarding prioritisation:
foreground (user-clicked 'Try now') summarises a chosen video on demand;
background batch summarises newest-first; both honour the shared rate gate.
Remove any prior framing that described 'Try now' as making traffic 'look
organic to YouTube' or as rate-limit evasion — that was not the rationale
and contradicts ADR-014's explicit account-safety constraint.
Correct statement: rate limiting is respected, not evaded. TAPIR_FETCH_RATE
and TAPIR_FETCH_BACKOFF are honest rate controls; they govern how fast Tapir
fetches captions, not how the requests appear to YouTube.
Architecture: add two-path model table (foreground/background, both through
globalFetchGate) and newest-first batch ordering doc (three-phase RunOnce,
before/after example).
ui-spec: add 'Try now' row with correct rationale; add pipeline stats bar row;
update Summarized-only filter row to mention sort-to-top.
Restructures RunOnce from per-channel inline processing to collect-sort-process:
Phase 1 — discover, persist (UpsertVideo), apply pre-filters (seen/manual/backoff)
and collect surviving candidates with their discovery position.
Phase 2 — sort candidates by published_at DESC, NULLS LAST, pos ASC tiebreak
so videos with no publish date never jump ahead of dated content.
Phase 3 — process in sorted order through the unchanged globalFetchGate.
Before (per-channel): chanA=[v-old, v-mid], chanB=[v-new, v-null]
→ [v-old, v-mid, v-new, v-null]
After (newest-first): [v-new, v-mid, v-old, v-null]
Same set of videos processed; only the order changes within a pass. All existing
behaviour is preserved: failure isolation, backoff skip, manual mode,
channel-unavailable, stats. In-memory sort; no new table or persisted queue.
The ordering is onboarding prioritisation — new users get summaries of their most
recent, relevant videos first; the back-catalogue fills in behind across subsequent
passes. Both this background batch and the foreground 'Try now' button honour the
shared globalFetchGate: rate limiting is respected, not evaded.
Three UX improvements for the pending-transcript state:
1. Summarized videos sort to top (ORDER BY (s.id IS NOT NULL) DESC, seen_at DESC)
so completed summaries are always immediately visible without filtering.
ListVideos default limit raised from 50 to 500 to show the full backlog.
2. Pipeline stats bar above the video list: '2 summarized · 256 fetching soon · 12
no captions' — computed from the unfiltered row set, hidden when everything is
summarized.
3. 'Try now' button on rate-limited cards replaces the passive 'Retrying later'
chip. POST /v/{id}/retry-now clears rate_limited_at then calls ProcessVideo
through the shared globalFetchGate — same rate limiting as the scheduler, safe
under concurrent use.
YouTube channels that 404 on playlist discovery (deleted/private) are now:
1. Wrapped in domain.ErrChannelUnavailable by the YouTube adapter (instead of
a generic error), so the runner can identify them without string-matching.
2. Stored per-user in channel_errors (migration 013, RLS-guarded) via runner's
new UpsertChannelError path — removed from the generic Errors counter,
counted separately as ChannelUnavailable.
3. Shown on the account page under "Unavailable channels" with name, chip-warn
badge, and first-seen date, so users know why some subscribed channels
produce no videos.
Migration 011's UPDATE ran without tapir.current_user_id set, so FORCE RLS
blocked all rows and 0 users were updated (skipped_manual=607 in scheduler).
Migration 012 temporarily drops FORCE so the table owner can run the UPDATE,
then restores it.
homelab-integration.md gains a "Scheduled discovery" section documenting
TAPIR_DISCOVERY_INTERVAL and TAPIR_FETCH_RATE and the load-bearing
single-replica constraint (in-process scheduler → replicas: 1 is required;
>1 double-runs discovery). VISION Stage 0 carries a pointer to ADR-018's
gate-clock reset so nothing in docs implies the window started before
unprompted use was possible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migration 011 flips the auto_summarize column default to TRUE and brings
existing rows (maintainer + current registrations) along. Onboarded friends
now get zero-friction discovery: scheduled discovery (ADR-018) both discovers
AND summarizes new videos, so a user's list fills and summarizes itself
instead of presenting an empty list of manual Summarize buttons.
Safe only because the process-wide caption-fetch rate gate (ADR-014 item 2,
prior commit) now exists — auto + scheduled + multi-user would otherwise
self-inflict 429s every cycle. The down migration reverts the default but
intentionally leaves existing rows as-is (no surprise manual regression on
rollback). RegisterUser already lets the column default drive the value, so
no app change is needed; the account-page manual toggle still works.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage 0's "returns and reads in >=2 weeks" gate can't be met while discovery
is host-side manual (`tapir run`): a newly onboarded user sees an empty list
and never comes back. Make Tapir watch on its own.
cmdServe launches a background goroutine (when TAPIR_DISCOVERY_INTERVAL > 0)
that runs a discovery pass for ALL users on that cadence: enumerate via the
un-RLS'd ListAllUsers, then run each user's pass through the EXISTING
runner.Runner — the only new code is the per-user loop, not a new scheduler.
Run-once-on-startup then ticked; ctx-cancelled on SIGTERM; per-user failures
(including buildUserRunner errors) are logged and skipped so one bad user
never aborts the rest. interval <= 0 disables it entirely (dev/tests).
buildUserRunner binds each runner to that user's own YouTube refresh token
(web.YouTubeTokenRef) — the Stage-1 per-tenant ref — reusing buildProcessor's
engine wiring. SetFetchRate is also wired in cmdServe so the click-path shares
the gate.
SINGLE-REPLICA is now load-bearing: the loop lives in the web process, so >1
replica double-runs discovery (429s + duplicate work). Documented in cmdServe
and warned at startup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The in-process scheduler (ADR-018) needs to enumerate every user to run a
discovery pass each. user_identities is the un-RLS'd map; add ListAllUsers as
a plain pool query (no withUser) — the same enumerate-then-act pattern
UserBySubject and the login_events gate query established. Scoping it to a
single user would defeat the point; user_identities carries no RLS by design.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-014 item 2 — a single per-egress-IP rate gate shared by every caption
fetch — was specced but only per-video backoff (rate_limited_at) shipped.
Build the real gate now: it is load-bearing once ADR-018 puts auto-summarize
on an in-process schedule across multiple users (all fetches leave one pod's
egress IP, concurrently with live "Summarize" clicks — without a shared gate
that self-inflicts 429s every cycle).
globalFetchGate (golang.org/x/time/rate, default 2s/req burst 1) is consulted
in httpDo before every live outbound fetch — player, watch-page, timedtext —
so the scheduler runners and the web click-path serialise through one limiter
regardless of how many users/goroutines are upstream. The test seam
(a.transport != nil) skips the gate so fakes are not throttled.
TAPIR_FETCH_RATE (Go duration, default 2s, 0 = unlimited) wires SetFetchRate in
cmdRun; the existing per-video backoff stays as the complementary 429 handler.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ActiveWeeks computes per-user distinct active weeks (reads login_events UNION
acts summary_actions) for the gate (VISION/ADR-016: usage in >=2 distinct
weeks). Because the user-owned tables are FORCE RLS under a non-superuser owner,
a single cross-user query is deny-all; instead it enumerates users from the
un-RLS'd identity map and counts each inside withUser — no privilege escalation,
no policy change.
`tapir report` prints the per-user table and the pass/fail verdict (needs only
TAPIR_DB_DSN). Pure formatter + store query are unit-tested, including the
cross-table shared-week dedup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The registration gate, once it resolves the authenticated subject to a tapir
user_id, calls StampLogin (store-throttled to one row per user per day). Best-
effort: a stamp failure is logged and swallowed so it never breaks the request.
This is what makes the read-side Stage-0 usage signal actually accrue.
Tests cover the happy-path stamp, the same-day throttle, and that an
unregistered subject (redirected to /register) is never stamped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
StampLogin appends one login_events row per user per day via an atomic
INSERT ... SELECT ... WHERE NOT EXISTS, run through withUser so the throttle
probe is itself RLS-scoped to the caller. DeleteUser now deletes login_events
explicitly (no FK = no cascade — the summary_actions footgun, repeated).
Extends the two-user RLS isolation proof and the delete-account proof to cover
login_events, and adds throttle / new-day / user-scoping tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage-0 usage measurement (VISION/ADR-016): summary_actions captures acts
(watch/skip/save) but not reads. A reader who logs in weekly and clicks
nothing is invisible — for a reading product that return is the signal the
gate ("usage in >=2 distinct weeks") is defined on. login_events records
THAT a user was active, append-only, one row per user per active day.
Per-user isolation via the same GUC-keyed FORCE RLS policy as migration 003.
No FK to users (mirrors summary_actions) — the cascade footgun is handled by
DeleteUser in a later commit. Adds an up/down reversibility test and registers
the table in both truncate helpers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Logged-out visitors on /welcome and /invite should not see Account or
Log out. Split Layout into Layout (authenticated, full nav) and
PublicLayout (public, brand-only header). WelcomePage + InvitePage
variants now use PublicLayout.
The Stage-1 onboarding path: an invited user opens their emailed link,
sets a password, and Tapir creates their Dex local-password account so
they can log in. Mounted on root OUTSIDE Auth.Middleware — the visitor
has no Dex session yet; the token in the path is the capability.
handleInviteForm previews the token (no consume) and shows the form, or
a clear "expired / already used" page. handleInviteSubmit validates the
password BEFORE consuming the token (a typo is retryable), then claims
the invite exactly once, bcrypt-hashes (cost 12), and creates the Dex
account — mapping ErrPasswordExists -> "log in instead" and ErrForbidden
-> "contact the administrator". Off-cluster (App.Dex nil) it degrades to
a "deployed-only" message without burning the token. On success it sets
an account_created flash and redirects to /auth/login.
Welcome sub-text now states access is invite-only. Handlers depend on
narrow ports (InvitationStore, DexPasswordCreator) so tests use fakes;
cmdServe wires the store + an in-cluster dex.PasswordClient.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mints a single-use invitation and prints the absolute claim URL for the
operator to send. The URL base is TAPIR_PUBLIC_URL (default
https://tapir.d-ma.be). runInvite is factored from config/store wiring so
it's unit-tested against a fake inviter — no Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Writes passwords.dex.coreos.com CRs against the in-cluster Kubernetes
API using the pod's service-account token + cluster CA (no kubectl /
client-go dependency). NewPasswordClient returns ErrNotInCluster off
cluster so the web layer degrades gracefully in dev.
Load-bearing: Dex's kubernetes storage types Password.Hash as []byte,
which k8s JSON-marshals as base64 — so the `hash` field carries the
base64 of the bcrypt string, not the raw string. Storing the raw string
makes Dex's base64-decode-on-login produce garbage and every login fail.
409 -> ErrPasswordExists, 401/403 -> ErrForbidden (RBAC missing) so the
handler can give precise messages. Tested against an httptest TLS server.
bcrypt cost-12 hashing lives in the web handler; golang.org/x/crypto was
already a transitive dep (now promoted in go.sum).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage-1 email onboarding: Mathias mints an invite, the recipient claims
it to set a Dex password. Invitations exist before their user, so the
table carries no user_id FK and is deliberately outside RLS — the
32-byte crypto-random token is the capability (single-use, time-boxed).
ClaimInvitation consumes atomically (UPDATE ... WHERE used_at IS NULL
... RETURNING) so concurrent claims of one token can't both succeed.
PeekInvitation validates the link for the form without consuming it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A discovered-but-unsummarized video whose caption fetch was rate-limited now
shows a passive ⏳ "Retrying later" chip (dim CharmDim styling, not the accent)
instead of the Summarize button — the user cannot fix a 429, the runner retries
automatically once the backoff window expires. Regenerated views_templ.go.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After ProcessNewVideo the runner records transcript_status per outcome:
rate_limited (stamps the backoff clock), none, or fetched. Before fetching, a
video inside the TAPIR_FETCH_BACKOFF window is skipped (SkippedRateLimited) so a
just-429'd caption endpoint is not re-hit; once the window expires it retries.
Backoff/clock injected via variadic Options (WithBackoff, WithClock) so existing
New call sites and the fake-driven loop tests stay valid. Backoff 0 = always retry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds FetchBackoff (Go duration, default 1h) controlling how long the run loop
waits before re-fetching a transcript that returned HTTP 429. Zero means always
retry. Not required by ValidateForRun — a zero/unset value is a valid policy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The engine already distinguishes SourceNone from SourceRateLimited internally
but collapsed both into Skipped. Expose the source string so the runner can
persist the right transcript_status and apply rate-limit backoff, without the
engine taking on any store/retry concern (dependencies still point inward).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surfaces videos.transcript_status (migration 007) on SummaryRow and adds
SetTranscriptStatus / GetTranscriptStatus / RateLimitedVideoIDs.
SetTranscriptStatus is the single choke point for the rate-limit lifecycle:
"rate_limited" stamps rate_limited_at = NOW(), every other status clears it,
so the runner's backoff window and the UI badge read one consistent source.
RateLimitedVideoIDs is the per-pass loader (mirrors SeenVideoIDs) the runner
uses to skip still-throttled videos without re-hitting the caption endpoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Logout was only reachable from /welcome. Users who are logged in had no way
to sign out from any app page (list, detail, account). Added to the shared
nav alongside Account.
Adds videos.transcript_status (NULL|none|fetched|rate_limited) and
videos.rate_limited_at, so the runner can record a 429 and skip re-fetching a
still-throttled video until a backoff window elapses. Columns inherit the
existing videos RLS policy (migration 003); no policy change needed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The baseUrl fetch mapped every non-200 to SourceNone, recording a 429 as a
permanent "no captions". 429 is the IP being rate-limited, not an absent
transcript. Return SourceRateLimited (still a graceful degrade, no error) so
the runner can retry after a backoff window. Other non-200s (403/404/5xx)
stay SourceNone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Videos rows are created by `tapir run`, not when a YouTube account is
connected, so a freshly-connected account correctly shows an empty list —
but the old empty state ("No videos yet") gave no clue why or what to do.
Split it on whether the user has any connection:
- connected, no videos: a distinct accent callout telling them to run
`tapir run` to discover subscriptions.
- not connected: a prompt with a Connect YouTube button.
handleList fetches connections only when the list is empty. Includes
web-shot captures of all three states under docs/ux-review/fixes/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
429 from the caption endpoint means the IP is rate-limited (retry later),
not that the video has no captions. Distinguishing it from SourceNone is the
prerequisite for the runner's backoff/retry logic.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "Get Started" button drops users straight into the shared Dex flow,
which has no separate "register" option — registration completes
automatically after first login. Users new to Tapir had no signal that
signing in is also how they sign up. Reword the sub-text to say so
explicitly, keeping the single Dex CTA (sign-in and sign-up are one flow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The .btn class sets color:var(--accent-fg), but the generic a{} and
a:visited{} rules outrank it on <a> elements, so anchor buttons (the
landing "Get Started" CTA, "Connect YouTube") rendered their label
accent-on-accent — invisible. Add a.btn / a.btn:visited to restore the
button foreground.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror to github.com was failing with 'unsupported in libcrypto' on every run
(OpenSSL 3.6 / OpenSSH 10 dropped support for the existing key format). Removed
rather than leave it polluting the CI signal. Re-add when the deploy key is
rotated to ed25519.
The Stage-0 ui-spec (ADR-011) predated multi-user and several UX
features. Appended a "Deviations and additions (as-built)" table —
without rewriting the spec — recording each feature shipped beyond it
(multi-user+RLS, registration gate, per-user YouTube connect, account
management, immediate web summarization, charmbracelet spinner,
auto/manual mode, public landing page) with the why and the
commit/ADR that covers each. Preserves the intent-vs-reality split.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The .feature spec lagged shipped behaviour. Added three files, no
duplication of existing scenarios:
- registration.feature: new Dex subject -> registration gate (users +
user_identities), returning subject straight through, account delete
is tapir-side only and leaves other users intact, clean
re-registration (ADR-012, ADR-013).
- summarize_mode.feature: auto summarizes every new video; manual
(default) leaves them unsummarized until queued; queued video is
processed and the flag cleared (migration 006).
- landing_page.feature: unauthenticated / -> /welcome, Get Started for
guests, summary link + logout for authed users, logout -> /welcome.
Scoped to built features only — no Vimeo/Whisper/billing scenarios.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The C4 view predated the web transport. Added a "Web surface" section
with a component diagram and prose covering: tapir serve (HTMX+Templ
over the unchanged store), oidc authenticate-only session, registration
gate (users + user_identities), per-user YouTube web connect callback,
account disconnect/delete (ADR-013 tapir-side only), immediate
summarization via background goroutine + HTMX status poll, and
auto/manual summarize mode (migration 006). Relabeled the L2 http node
to tapir serve. Corrected the out-of-scope isolation bullet: RLS is live
(ADR-012, migration 003), not deferred. ADR-003 stance preserved — the
engine/ports/sinks core is untouched; the web is a new transport.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ER diagram and notes predated migrations 002-006. Brought them to
code truth:
- add summary_actions (002), user_identities (004), video_connections
(005) with real columns/constraints; rename token_secret_ref ->
token_ref to match migration 005.
- add users.auto_summarize and videos.summarize_requested (006).
- note FORCE RLS coverage and sink_deliveries' EXISTS-derived policy
(003); user_identities NOT RLS'd.
- mark AI_CREDENTIAL and SUBSCRIPTION as planned (no table exists; only
videos.subscription_id, no FK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "Isolation invariant" section said enforcement was dormant at
Stage 0. ADR-012 turned it on: Postgres RLS ENABLE+FORCE on every
user-owned table (migration 003_rls.up.sql), keyed off the
tapir.current_user_id GUC set by the store's withUser helper, deny-by-
default on an unset GUC. The Stage-2 two-user isolation test
(internal/adapters/store/rls_test.go) is pulled forward and passing.
Kept the ADR-011 single-user history honest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The "Current build state" section described a scaffolded, intentionally
RED repo (ErrNotImplemented, go 1.23, unverified confirm items). All
stale: build is v0.4.0 green, go 1.26.1 (go.mod), Stage 1 multi-user
with RLS shipped (ADR-012, migrations 001-006). Rewrote to describe the
actual adapters, cmd subcommands (incl. serve), and how to build/run.
Replaced the "confirm" list with the resolved homelab facts (LiteLLM
koala:30401, model-as-config) now pinned in homelab-integration.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-010 sat between ADR-008 and ADR-009. Moved ADR-009 (TBD) ahead of
ADR-010 (timedtext caption acquisition) so the file reads 001..014 in
numeric order. Pure structural move, no content change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Package comment said "Stage-0 ... (ADR-011)" and User.Subject said
"single-user allowlist (ADR-011)". Both stale: ADR-012 opened Stage 1
(multi-user, RLS-enforced isolation). Subject is now the user_identities
lookup key (migration 004) resolving to a per-user UUID; an unknown
subject hits the registration gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a configurable fakeAuth (StubAuth can't express the logged-out case)
and assert: /welcome renders the logged-out CTA with no session and the
logged-in controls with one, unauthenticated / redirects to /welcome,
unauthenticated deep links redirect to /auth/login, and an authenticated
root still renders the list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire GET /welcome on the root mux alongside /healthz, outside
Auth.Middleware. handleWelcome peeks the session via CurrentUser (no
redirect) and renders the logged-out or logged-in WelcomePage variant.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the public landing page: a static Charm-box tapir mascot (reusing the
shared tapirLine helpers and charm palette), a tagline, and a single
'Get Started' CTA into the shared Dex flow — sign-in and sign-up are the
same URL (ADR-012). Logged-in visitors get a greeting plus links back into
the app and to log out. Regenerated views_templ.go committed alongside.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Logout was bouncing the just-logged-out visitor straight back into a Dex
login. Land them on the public /welcome page instead — an intentional UX
fix. Cookie clearing and server-side session deletion are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An unauthenticated visit to / now lands on the public /welcome page
instead of bouncing straight to Dex. Deeper guarded paths still redirect
to /auth/login so the post-login round-trip returns the visitor to the
page they asked for. isPublicPath runs first, so no redirect loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The landing page must render without a session. Add /welcome to
isPublicPath so the auth middleware lets it through (alongside /healthz
and /auth/*), and assert the bypass in the public-paths test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the plain ASCII spinner with a Charmbracelet-style TUI panel rendered in
the browser: a dark terminal card holding a rounded purple box (╭─╮╰─╯│) around a
pink block-char tapir (▄▓▀█) with mint eyes (◕ ◕) and a snout that wiggles ∩→∪→~
across three cross-faded frames, plus a lipgloss-style progress bar whose mint
fill grows over a dim track via an 8s CSS clip animation.
Colours come from named palette consts (CharmPurple/Pink/Mint/Cream/Dim) applied
as inline span styles, so the box and CSS share one source of truth. Frames are
built by a small run/line helper that pads each row to a fixed interior width —
an internal test asserts every box row is the same cell width so the border stays
flush. Honours prefers-reduced-motion (static frame + partial bar).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the engine wiring (YouTube source, AI-router summarizer, store sink)
into buildProcessor, shared by cmdRun and cmdServe. It returns (nil, nil) — not
an error — on incomplete config, which is the queue-only fallback for serve.
engineProcessor adapts the engine to web.Processor: load the video row, run the
engine, clear the manual queue flag on a produced summary (mirrors the runner).
cmdServe wires it onto web.App.Processor; cmdRun reuses buildProcessor so the
wiring is no longer duplicated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mathias
