A newly connected account showed no videos until the next 2h scheduled pass —
the gap that made onboarding look broken (a second user connected, saw nothing,
read as failure). The connect callback now fires an out-of-band discovery pass
for the connecting user, so videos appear promptly.
Concurrency: scheduled and connect-triggered passes share one lock (serialize),
preserving the single-fetcher invariant (ADR-018). A trigger interleaves between
the scheduler's per-user passes rather than fetching concurrently or waiting for
a whole pass. The trigger runs on the server ctx (survives the redirect) and is
non-blocking for the request goroutine.
Scope: connect-trigger only. The optional login-refresh / "Discover now" button
from #6 are intentionally not built — an unconditional login hook risks 429
storms (per the ticket's own recommendation); defer until wanted.
TDD: TestCallbackTriggersDiscovery, TestSerializeRunsOneAtATime,
TestDiscoveryTriggerEnqueueRunsUser; new BDD scenario mapped.
Refs #6
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scheduler enumerates every user_identities row (ListAllUsers) and ran a
discovery pass for each — including users who never connected a video source.
Their per-user runner then tried to resolve a YouTube refresh token that was
never minted, logging a spurious "secrets: ref not found:
youtube/<uid>/refresh_token" every tick (e.g. stale Dex-era orphan identities
left by the Authentik migration).
Skip users whose ConnectionsForUser is empty before running their pass. Removes
the recurring noise — which actively misled a debug session into thinking a
healthy onboarded user was broken — with no change to connected users.
Refs #7
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stop the un-summarized back-catalogue from burying the readable summaries
(UX review B3/B4). One feed, with a noise-collapse — not sections:
- Summarized + recent un-summarized videos lead inline as cards.
- Un-summarized videos older than the recency window collapse into a single
"Show N older videos — summarize on demand" disclosure (they will not
auto-fill; they are manual-only). Window comes from App.RecencyWindow
(= cfg.AutoSummarizeWindow); 0 disables the collapse (all inline).
- Caption-less videos collapse into one honest line ("N videos have no
captions and can't be summarized") instead of N dead terminal cards.
bucketRows is a pure classifier (cutoff-driven; undated rows never age out);
App gains RecencyWindow + an injectable clock for the cutoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In automatic mode the scheduler now only summarizes videos published within
TAPIR_AUTO_SUMMARIZE_WINDOW (default ~7d). Older videos are still discovered
and listed — they keep the manual "Summarize" affordance — but are not
auto-processed, so a large back-catalogue (the maintainer's ~256-deep queue)
stops self-inflicting 429s against the per-IP caption gate each cycle (UX
review B1, recency design).
- runner.WithAutoWindow + Stats.SkippedTooOld; tooOld() treats a zero window
as disabled and an undated video as never-aged-out (processed, not stranded).
- An explicit manual request bypasses the bound even in auto mode (requested
videos are loaded in auto mode when a window is active).
- Wired through cmdRun, the scheduler's per-user runner, sumStats, and pass
logging. config: TAPIR_AUTO_SUMMARIZE_WINDOW (default 168h), .env.example.
- Account copy (A7) updated to match: "Automatic summarizes new videos from
about the last week; older videos stay browsable — summarize on demand."
The rate gate is untouched; the manual path still serialises through it. This
bounds auto LOAD, it does not fetch harder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Authentik owns invites now (infra ADR-0001). Delete adapters/dex, the
/invite set-password UI, the tapir invite CLI, the InvitationStore/
DexPasswordCreator ports + App wiring, the invite Templ pages, and the
invite Taskfile target. New users are invited via Authentik, log in via
OIDC, and hit the existing /register gate. invitations table (mig 009)
left in place (append-only; harmless). task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage 0's "returns and reads in >=2 weeks" gate can't be met while discovery
is host-side manual (`tapir run`): a newly onboarded user sees an empty list
and never comes back. Make Tapir watch on its own.
cmdServe launches a background goroutine (when TAPIR_DISCOVERY_INTERVAL > 0)
that runs a discovery pass for ALL users on that cadence: enumerate via the
un-RLS'd ListAllUsers, then run each user's pass through the EXISTING
runner.Runner — the only new code is the per-user loop, not a new scheduler.
Run-once-on-startup then ticked; ctx-cancelled on SIGTERM; per-user failures
(including buildUserRunner errors) are logged and skipped so one bad user
never aborts the rest. interval <= 0 disables it entirely (dev/tests).
buildUserRunner binds each runner to that user's own YouTube refresh token
(web.YouTubeTokenRef) — the Stage-1 per-tenant ref — reusing buildProcessor's
engine wiring. SetFetchRate is also wired in cmdServe so the click-path shares
the gate.
SINGLE-REPLICA is now load-bearing: the loop lives in the web process, so >1
replica double-runs discovery (429s + duplicate work). Documented in cmdServe
and warned at startup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-014 item 2 — a single per-egress-IP rate gate shared by every caption
fetch — was specced but only per-video backoff (rate_limited_at) shipped.
Build the real gate now: it is load-bearing once ADR-018 puts auto-summarize
on an in-process schedule across multiple users (all fetches leave one pod's
egress IP, concurrently with live "Summarize" clicks — without a shared gate
that self-inflicts 429s every cycle).
globalFetchGate (golang.org/x/time/rate, default 2s/req burst 1) is consulted
in httpDo before every live outbound fetch — player, watch-page, timedtext —
so the scheduler runners and the web click-path serialise through one limiter
regardless of how many users/goroutines are upstream. The test seam
(a.transport != nil) skips the gate so fakes are not throttled.
TAPIR_FETCH_RATE (Go duration, default 2s, 0 = unlimited) wires SetFetchRate in
cmdRun; the existing per-video backoff stays as the complementary 429 handler.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ActiveWeeks computes per-user distinct active weeks (reads login_events UNION
acts summary_actions) for the gate (VISION/ADR-016: usage in >=2 distinct
weeks). Because the user-owned tables are FORCE RLS under a non-superuser owner,
a single cross-user query is deny-all; instead it enumerates users from the
un-RLS'd identity map and counts each inside withUser — no privilege escalation,
no policy change.
`tapir report` prints the per-user table and the pass/fail verdict (needs only
TAPIR_DB_DSN). Pure formatter + store query are unit-tested, including the
cross-table shared-week dedup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Stage-1 onboarding path: an invited user opens their emailed link,
sets a password, and Tapir creates their Dex local-password account so
they can log in. Mounted on root OUTSIDE Auth.Middleware — the visitor
has no Dex session yet; the token in the path is the capability.
handleInviteForm previews the token (no consume) and shows the form, or
a clear "expired / already used" page. handleInviteSubmit validates the
password BEFORE consuming the token (a typo is retryable), then claims
the invite exactly once, bcrypt-hashes (cost 12), and creates the Dex
account — mapping ErrPasswordExists -> "log in instead" and ErrForbidden
-> "contact the administrator". Off-cluster (App.Dex nil) it degrades to
a "deployed-only" message without burning the token. On success it sets
an account_created flash and redirects to /auth/login.
Welcome sub-text now states access is invite-only. Handlers depend on
narrow ports (InvitationStore, DexPasswordCreator) so tests use fakes;
cmdServe wires the store + an in-cluster dex.PasswordClient.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mints a single-use invitation and prints the absolute claim URL for the
operator to send. The URL base is TAPIR_PUBLIC_URL (default
https://tapir.d-ma.be). runInvite is factored from config/store wiring so
it's unit-tested against a fake inviter — no Postgres.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After ProcessNewVideo the runner records transcript_status per outcome:
rate_limited (stamps the backoff clock), none, or fetched. Before fetching, a
video inside the TAPIR_FETCH_BACKOFF window is skipped (SkippedRateLimited) so a
just-429'd caption endpoint is not re-hit; once the window expires it retries.
Backoff/clock injected via variadic Options (WithBackoff, WithClock) so existing
New call sites and the fake-driven loop tests stay valid. Backoff 0 = always retry.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the engine wiring (YouTube source, AI-router summarizer, store sink)
into buildProcessor, shared by cmdRun and cmdServe. It returns (nil, nil) — not
an error — on incomplete config, which is the queue-only fallback for serve.
engineProcessor adapts the engine to web.Processor: load the video row, run the
engine, clear the manual queue flag on a produced summary (mirrors the runner).
cmdServe wires it onto web.App.Processor; cmdRun reuses buildProcessor so the
wiring is no longer duplicated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GET /account shows the registered display name, the signed-in email, the
user's connected video accounts (status + when), a Connect-YouTube link
when none is connected, and the disconnect / delete controls. Linked from
the header nav.
POST /account/disconnect/{provider}: deletes the OAuth token from the
SecretStore (resolved from the connection's own token_ref, provider-
agnostic) and the connection row. Does NOT delete the account.
POST /account/delete: confirm-before-destroy (a <details> disclosure gates
the destructive submit — works without JS). Captures token refs, calls
store.DeleteUser (cascades all rows), purges every secret, then routes to
/auth/logout to clear the session. Tapir-side only — Dex is left untouched
(decision 2026-06-03).
Account handlers depend on a narrow SecretRemover (Delete) and the extended
Store port; cmd/tapir serve shares one file-backed SecretStore between the
connect flow and account management.
Tests: account page renders connections + name + Connect link; disconnect
removes token (fake records Delete) + row and keeps the account; delete
wipes users/summaries/connections/identities and purges the token, then
redirects to logout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add GET /oauth/youtube/connect and /oauth/youtube/callback, mounted inside
the login + registration guard so CurrentUserID is always set and every
connection binds to the authenticated tapir user.
- connect: generate a per-user CSRF state (single-use, short TTL, bound to
the user), redirect to Google consent with access_type=offline and
prompt=consent so a refresh token comes back.
- callback: verify the state belongs to this user, exchange the code via the
existing auth.Exchange, persist the refresh token under a PER-USER ref
(web.YouTubeTokenRef = "youtube/<userID>/refresh_token") so tenants never
collide, then UpsertConnection (provider=youtube, status=active). Any
failure renders a clean error page and leaves no half-written state.
Reuses auth.Exchange and adds auth.AuthCodeURL (offline + consent) rather
than the CLI's listener/terminal flow (ADR-006: web flow, not CLI). The
ConnectHandler depends on a narrow web.Connections port, not the concrete
store. Wired in cmdServe only when YT client credentials are present;
TAPIR_YT_CONNECT_REDIRECT_URL configures the callback URL. Per-user token-ref
scheme documented in docs/homelab-integration.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multi-user web surface. Two layered middlewares: Auth.Middleware (Dex
session required) wraps registrationGate, which resolves the authenticated
subject -> tapir user_id once per request via the new web.Identity port and
stashes it. A subject with no tapir user is redirected to GET /register
(display name + accept-terms); POST /register calls RegisterUser then
redirects to /. /register is inside the auth guard but exempt from the gate
(/auth/* and /healthz too).
Current-user seam: CurrentUserID(r) (string, bool) returns the resolved id
from the request context. The list/detail/action handlers now scope by it,
replacing the single configured App.UserID (removed). App gains an Identity
field; *store.Store satisfies both Store and Identity. cmd/tapir wires
Identity: st and drops UserID.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-011's single-user authz (ID-token subject must equal AllowedSubject,
else 403) is replaced by ADR-012's model: Dex authentication is the only
gate — any Dex-authenticated subject may establish a session. Whether that
subject has a tapir user, and routing to registration if not, is decided
downstream in internal/web (next commit).
Removals (noted): oidc.Config.AllowedSubject + its required-field check + the
callback 403 branch; config.Config.AllowedSubject + TAPIR_ALLOWED_SUBJECT env
wiring; the AllowedSubject arg in cmdServe. ui-spec.md updated to reflect the
supersession. Sessions, cookie signing, login/callback/logout unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
serve now uses oidc.DexAuth (single-user allowlist authz, ADR-011) when
TAPIR_OIDC_ISSUER is set, falling back to allow-all StubAuth for local dev.
Adds the Dex config fields (TAPIR_OIDC_ISSUER/DEX_CLIENT_ID/SECRET/
OIDC_REDIRECT_URL/SESSION_SECRET/ALLOWED_SUBJECT) + Config.DexConfigured().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the lane-C reader surface: list, detail, and an action button-group
fragment over the lane-A store reads/actions, behind the web.Auth seam.
- Templ components (base layout, list+filters, detail, ActionButtons) with
committed *_templ.go so go build/task check work without the templ binary;
`task generate` regenerates. Filters and action toggles are HTMX-swapped and
degrade to plain form GET/POST (POST→303→GET) without JS.
- Handlers (internal/web): GET / (channel+date filters, in-memory),
GET /v/{videoId}, POST /v/{videoId}/action (re-click clears, else SetAction;
store enforces watched↔skipped exclusion), GET /healthz (no auth). Store ops
run as the configured UserID; Auth only gates.
- `tapir serve` wires store + StubAuth{Subject: cfg.UserID} + http.Server on
TAPIR_HTTP_ADDR (default :8080), graceful shutdown on signal. Handlers depend
only on web.Auth — Conductor swaps StubAuth → oidc.DexAuth at merge (one line
in cmdServe).
- Handler tests: real store (embedded-postgres) + StubAuth — list rows+state,
HTMX fragment vs full page, channel filter, detail highlights/takeaways,
404, action toggle+clear, no-JS redirect, bad-verb 400.
New dep: github.com/a-h/templ — the house default for typed server-rendered
HTML (CLAUDE.md stack, ui-spec.md §3). Generated code is committed so the
templ binary is build-time-optional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main.go dispatches `tapir auth` (interactive OAuth → persist refresh token via
SecretStore) and `tapir run` (wire YouTube source + local summarizer + store
sink, build engine, run the dedup-aware loop). Config-driven so live creds plug
in at demo time; SIGINT stops the loop cleanly. Block kept minimal so Worker E's
list/show cases union cleanly at merge.
Add .env.example documenting every TAPIR_* var and a README demo runbook. Pin
the summarizer alias-as-config decision and record the max_tokens fix in
docs/homelab-integration.md (clears two `confirm` items).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tapir list — table of stored summaries (date, title|id, channel, AI,
fallback), recent-first. tapir show <video-id> — full summary with
highlights and takeaways. DSN + user id from TAPIR_DB_DSN/TAPIR_USER_ID,
never hardcoded. main.go gains a minimal os.Args[1] dispatcher kept flat
so Worker F's auth/run cases union cleanly at merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Minimal main that identifies the binary (gives the CI smoke test something to
grep). HTTP server, watcher, adapter wiring, and config come with the build.
mathiasandClaude Opus 4.8