The isolation proof for ADR-012. embedded-postgres's default user is a
SUPERUSER, which bypasses RLS regardless of FORCE — a test run as it would be
fake-green. So this test creates a dedicated non-superuser role ("app", mirroring
the prod owner tapir), grants it DML, asserts rolsuper is false, and runs every
scoped query as that role.
Assertions: (1) deny-all — with no GUC set, every isolated table returns zero
rows, proving the enforcement path is live, not bypassed; (2) a connection scoped
to user A sees exactly its own one row in every table (and B likewise); (3)
cross-user UPDATE/DELETE aimed at B's rows touches zero rows; (4) B's rows survive
unchanged, verified via the superuser pool.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add Store.withUser(ctx, userID, fn) — a single choke point that BEGINs a tx,
sets the transaction-local GUC tapir.current_user_id via
set_config(..., true), runs fn, and commits. set_config is used over SET LOCAL
because it is parameterizable; the local flag means the value auto-resets on
commit/rollback so a pooled connection never leaks one request's user into the
next.
Route all 9 DB-touching methods through it (Deliver, HasSummary, SeenVideoIDs,
ListSummaries, GetSummaryByVideo, SetAction, ClearAction, ActionsFor, and
UpsertVideo; attachActions flows via ActionsFor). Scoping is now structural —
not a per-query opt-in someone can forget — and arms the migration-003 RLS
policies. Method signatures and existing WHERE clauses are unchanged (defence in
depth; superuser DSNs in existing tests bypass RLS so behaviour is preserved).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enable AND FORCE row-level security on every user-owned table (users, videos,
transcripts, summaries, summary_actions, sink_deliveries) per ADR-012. Each
policy keys off the per-request GUC tapir.current_user_id; an unset GUC yields
NULL → deny-all (the safe default).
FORCE is load-bearing: the app connects as the table owner (tapir), and owners
bypass RLS unless forced. Without FORCE the policies are dead for the prod user.
sink_deliveries has no user_id; its policy derives ownership from the summary it
belongs to via EXISTS against the GUC, so it is self-contained rather than
silently depending on summaries' own RLS being applied to a subquery.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render a muted one-line lede beneath the title/meta of each summary
card via previewText(r.Summary, 160). New .card-preview token clamps to
one line. Regenerated views_templ.go committed (CI has no templ binary).
Responsive 16:9 youtube-nocookie iframe rendered when the id is valid;
omitted (graceful) otherwise so summary/highlights/takeaways still show.
Regenerated views_templ.go.
One-line lede for list cards: collapses whitespace, prefers the first
sentence within budget, else word-boundary truncation with an ellipsis.
Pure and rune-based (multibyte-safe). Table-driven tests cover empty,
short, first-sentence, word-boundary, and multibyte cases.
Maintainer's call to open Stage 1 ahead of the Stage-0 gate. Non-negotiable:
multi-user ships WITH DB-enforced isolation (Postgres RLS, FORCE'd on the
owner role, per-request tapir.current_user_id) and a passing two-user isolation
test in the same slice — the VISION Stage-2 bar pulled forward, not deferred.
Replaces ADR-011's allowlist-of-one with per-subject users rows; adds
video_connections + subscriptions.
Diagnosis of a live run: caption tracks resolve fine (player + watch-page scrape),
but the timedtext baseUrl fetch returns 429 under back-to-back volume — YouTube
rate-limits the unauthenticated caption-download endpoint per IP. A per-video
delay spaces the fetches. (Follow-up: treat 429 distinctly from genuine
no-caption instead of silently degrading to SourceNone; consider Whisper if the
endpoint stays hostile at any sustainable rate.)
The ANDROID InnerTube player now returns 0 captionTracks (PoToken-gated), so the
watch-page scrape is the real path — but it sent an Android *app* User-Agent,
which makes YouTube serve a page WITHOUT ytInitialPlayerResponse, so captions
came back empty. Result: 87% of a live run skipped as 'no transcript' despite
the videos having captions. A desktop-browser UA returns the player JSON with
captionTracks. Confirmed live: app-UA=0 tracks, browser-UA=1.
First-login chicken-egg: TAPIR_ALLOWED_SUBJECT can't be known until the user
logs in once, but the allowlist gates login. Echo the (non-secret, opaque)
subject in the forbidden response so the maintainer can read it in the browser,
set the 1P item, and lock the allowlist.
go.mod requires >=1.26.1 but the Dockerfile pinned golang:1.25 -> 'go.mod
requires go >= 1.26.1 (running go 1.25.11)'. Also revert the ci.yml XDG hack:
the real rootless-buildah fix is a user ~/.config/containers/storage.conf (vfs +
writable runroot), which fixes plain buildah for every repo without workflow
changes.
The act_runner is a systemd service with no login session → XDG_RUNTIME_DIR
unset → rootless buildah uses root-owned /run/containers and fails 'mkdir
/run/containers: permission denied'. Set XDG_RUNTIME_DIR to a per-job mktemp
dir so its runroot is writable. (check + on:/go-version fixes already landed;
this unblocks the image build → registry.)
CI check failed in internal/adapters/store: parallel 'go test ./...' starts
multiple embedded-postgres instances against the shared ~/.embedded-postgres-go
data dir → 'another postgres running in data directory'. -p 1 runs one package
at a time, so only one embedded-postgres is live. (Locally flaky, deterministic
in CI's fresh env.)
CI check failed because go.mod declared 'go 1.25.0' — the only repo not on the
host's 1.26.x line. actions/setup-go then tried to download 1.25.0 on the
self-hosted runner (fails), while cobalt (1.26.1) and template-go-agent (1.26)
serve from the host/cache. Deps need >=1.25, so 1.26.1 satisfies them and
matches the estate.
Bare 'on:' is a YAML boolean (Norway problem) — parsed as the key True, not
the string 'on'. Gitea's workflow loader then materialised 0 jobs and every run
failed instantly. Quoting "on": fixes dispatch.
Light + dark list, mobile (375px) card reflow, and the reader detail page in
both schemes — captured via web-shot against the seeded serve. Visible evidence
that the Top 5 UX fixes landed.
Implements the Top 5 fixes from the Stage-0 UX review (docs/ux-review/UX-REVIEW.md):
1. Dark mode: full light+dark custom-property palette (bg/fg/muted/line/accent/
card) under :root + @media (prefers-color-scheme: dark), applied to body.
color-scheme: light dark is now actually honoured — summary text was invisible
on a dark canvas before.
2. Table -> responsive card list: one card per summary (title link, channel·date
meta, provider chip, fallback badge, action state). Single-column reflow at
375px, no horizontal crush.
3. Minimal design system: 4/8px spacing scale, one accent, styled accent links
(underline-on-hover), real buttons with active/pressed state, consistent
radius and dividers — applied across list + detail.
4. Detail page as a reader: prose capped at 38rem, title->meta->summary->
highlights->takeaways hierarchy with section rules, 1.7 line-height. Meta is
built from non-empty parts (detailMeta) so the no-video edge case no longer
renders a stray "· — ·".
5. Contrast + a11y: muted bumped to #595959 (~7:1, clears WCAG AA), fallback
badge gets vertical padding + aria-label/title, friendly first-run empty
state, hx-indicator on the filter form.
Tests updated for the card markup (table -> cards); missing date is now omitted
rather than em-dashed. task check green; HTMX action toggles verified working.
Stage-0 web UI needs a self-contained container image. Two changes:
- Dockerfile: multi-stage build (golang:1.25 builder, CGO off + static
link, -trimpath -s -w) into distroless static nonroot. The committed
templ output and vendored asset mean a plain `go build` suffices — no
codegen or CDN at build/run time. Existing .gitea CI already builds and
pushes localhost:5000/tapir:<sha> + mirrors to GitHub (deploy patch
intentionally omitted — cutover is held), so it only needed this file.
- Vendor htmx 1.9.12 locally (internal/web/static/, embed.FS, served at
/static/ outside the auth guard) and point Layout at /static/htmx.min.js
instead of unpkg. The deployed UI must not depend on an external CDN
being reachable from the cluster.
task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reviewer pass against running `tapir serve` UI, seeded with 11
representative summaries (5 channels, rich+sparse, local+fallback,
varied action states). Captured via Playwright on koala k3s across
desktop/mobile and light/dark color schemes.
Two blocking findings: dark mode is unreadable (summary text near-black
on a dark canvas — `color-scheme: light dark` declared but `--fg`
hardcoded and no body background), and the list is a 6-column table that
does not reflow on mobile. Plus a sub-4.5:1 muted color, an unstyled
"admin table" surface, and small correctness nits (stray `· — ·` meta
join, cramped fallback badge). HTMX action toggles verified working.
Includes UX-REVIEW.md (severity-tagged findings + Top-5 sleek list) and
16 screenshots. No code changed — drives the next UI iteration.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
serve now uses oidc.DexAuth (single-user allowlist authz, ADR-011) when
TAPIR_OIDC_ISSUER is set, falling back to allow-all StubAuth for local dev.
Adds the Dex config fields (TAPIR_OIDC_ISSUER/DEX_CLIENT_ID/SECRET/
OIDC_REDIRECT_URL/SESSION_SECRET/ALLOWED_SUBJECT) + Config.DexConfigured().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the lane-C reader surface: list, detail, and an action button-group
fragment over the lane-A store reads/actions, behind the web.Auth seam.
- Templ components (base layout, list+filters, detail, ActionButtons) with
committed *_templ.go so go build/task check work without the templ binary;
`task generate` regenerates. Filters and action toggles are HTMX-swapped and
degrade to plain form GET/POST (POST→303→GET) without JS.
- Handlers (internal/web): GET / (channel+date filters, in-memory),
GET /v/{videoId}, POST /v/{videoId}/action (re-click clears, else SetAction;
store enforces watched↔skipped exclusion), GET /healthz (no auth). Store ops
run as the configured UserID; Auth only gates.
- `tapir serve` wires store + StubAuth{Subject: cfg.UserID} + http.Server on
TAPIR_HTTP_ADDR (default :8080), graceful shutdown on signal. Handlers depend
only on web.Auth — Conductor swaps StubAuth → oidc.DexAuth at merge (one line
in cmdServe).
- Handler tests: real store (embedded-postgres) + StubAuth — list rows+state,
HTMX fragment vs full page, channel filter, detail highlights/takeaways,
404, action toggle+clear, no-JS redirect, bad-verb 400.
New dep: github.com/a-h/templ — the house default for typed server-rendered
HTML (CLAUDE.md stack, ui-spec.md §3). Generated code is committed so the
templ binary is build-time-optional.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds internal/web/oidc.DexAuth, the production web.Auth impl behind the seam
(ADR-011, docs/ui-spec.md §6). Standard Authorization Code flow against Dex:
- Routes() mounts /auth/login (state+nonce, redirect to authorize),
/auth/callback (code exchange, ID-token verify, nonce check, allowlist:
sub must equal Config.AllowedSubject else 403, set session, redirect /),
/auth/logout (clear session).
- Middleware redirects unauthenticated requests to /auth/login, slides the
session expiry on each authenticated request; /healthz and /auth/* bypass.
- CurrentUser resolves the principal from the session cookie.
- Sessions: server-side in-memory store (single Stage-0 replica) keyed by an
HMAC-SHA256 (HS256) signed, HttpOnly, Secure, SameSite=Lax cookie with a
short TTL + sliding refresh. State->nonce pending map is one-time + expiring
(replay/CSRF defense). Tokens are never logged.
Constructor New(ctx, Config, ...Option); the six-field Config (Issuer,
ClientID, ClientSecret, RedirectURL, SessionSecret, AllowedSubject) is what
cmd/tapir wires from TAPIR_OIDC_*/TAPIR_DEX_*/TAPIR_SESSION_SECRET/
TAPIR_ALLOWED_SUBJECT. Options (clock, TTL, insecure cookies) are test-only.
Tests use a fake OIDC issuer via httptest (discovery + JWKS + token endpoint
signing an RS256 ID token) — no live Dex: login 302s to authorize; callback
for the allowlisted sub sets a session and 302s to /; non-allowlisted sub 403;
middleware redirects unauthenticated and passes authenticated; logout clears;
plus expiry, tampered-cookie, and unknown-state cases.
Deps (per ADR-006 / ui-spec §6): adds github.com/coreos/go-oidc/v3 — the
homelab-standard OIDC lib, small, handles discovery + JWKS + ID-token
verification; pairs with the already-present golang.org/x/oauth2. go-jose/v4
(transitive via go-oidc) is used directly only in tests to sign the fake
issuer's tokens.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lets the Dex session layer (lane B) and page handlers (lane C) build independently:
handlers depend only on web.Auth; oidc.DexAuth (B) and StubAuth (dev) implement it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The summary_actions table records the maintainer's act on a summary, the
column that makes the Stage-0 headline test ("acts on >=1 summary") queryable
(ui-spec.md §5, ADR-011). This is the gate lanes B/C build on.
- Migration 002: summary_actions (id, user_id, video_id TEXT, action, acted_at)
with a CHECK on action IN ('watched','skipped','saved') and a UNIQUE
(user_id, video_id, action). Per-user isolation: every row carries user_id.
- New actions.go: SetAction (idempotent, atomic watched<->skipped mutual
exclusion in one tx; saved independent), ClearAction, ActionsFor for the list
view, plus Go-side action validation.
- reads.go: additive SummaryRow.Actions, populated by composing ActionsFor
(Go-side, not a SQL join — summaries.video_id is UUID, actions.video_id TEXT).
- embedded-postgres tests: set/clear, mutual exclusion, saved coexistence,
idempotency, invalid rejection, user scoping, read-view surfacing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reader + watch/skip/save (instruments the Stage-0 'acts on a summary' metric),
HTMX+Templ over the existing store, Dex OIDC login with single-user allowlist
authz (authn now, tenancy deferred), deployed at tapir.d-ma.be via Flux GitOps
with ESO secrets and in-cluster postgres18. Build decomposed into 4 gated lanes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NewVideos called search.list at 100 quota units/call. With ~143 channels one
discovery pass = 14,300 units > the 10,000/day YouTube Data API cap, exhausting
the whole day in a single loop (live quotaExceeded).
Switch to playlistItems.list (1 unit/call) against the channel's uploads
playlist. For a standard channel id UCxxxx the uploads playlist is UUxxxx,
derived at zero API cost (uploadsPlaylistID). Non-standard ids fall back to
channels.list (1 unit) to read contentDetails.relatedPlaylists.uploads. Newest-
first ordering and MaxVideosPerSubscription cap preserved.
Side effect: removing search.list also removes the accountDelegationForbidden
error that endpoint threw for one channel — no separate hardening needed.
New per-pass quota: /subscriptions (1) + ~1/channel discovery (143) + any
channels.list fallbacks ≈ 145 units/day, well under 10k. Caption fetch (ADR-010
timedtext) uses no Data API quota.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vimeo texttracks API is owner-only (worse than YouTube, no public timedtext) → defer.
Whisper viable as a no-caption fallback; berget/whisper-large-v3 already on the gateway
(cloud), local options on iguana/koala carry setup+GPU-contention cost (ADR-007). Both
investigated inline (session sub-agents are network-sandboxed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Data API captions.download endpoint is owner-only: every subscription
video the user does not own returned HTTP 403, producing 0 summaries and a
~150-line error spew in the first live Stage-0 run. Captions-first (ADR-007)
is sound; only the acquisition mechanism was wrong.
FetchTranscript now resolves caption tracks from the InnerTube player
response (ANDROID client, unauthenticated) and GETs the chosen track's
timedtext baseUrl with a plain http.Client — no OAuth token, which can break
the endpoint. The srv3 XML, json3, and legacy <transcript> formats all parse;
non-asr tracks in a preferred language win. Watch-page ytInitialPlayerResponse
scrape is the fallback when InnerTube returns no tracks.
Degrade, don't error (explicit quick-fix): no captionTracks, empty baseUrl, a
non-200 fetch, or an unparseable body yield Source=none, not an error. Only
genuine transport faults error — this kills the spew. OAuth stays on
ListSubscriptions/NewVideos (Data API); only transcript fetch goes unauthed.
Validated live from koala: the ANDROID client returned working baseUrls and
real transcript text for public videos the run identity does not own.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tapir auth binds localhost:8080 on koala and prints the consent URL (no browser
auto-open), so it works headless via 'ssh -L 8080:localhost:8080 koala'. run/list/
show are already non-interactive; document the 'op run --env-file' invocation with
a service-account token so secrets resolve without an interactive signin. Also
correct the stale 'Pre-code' status.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Worker F's README told users to 'cp .env.example .env' but a blanket .env.*
gitignore rule silently dropped it. Un-ignore .env.example (real .env stays
ignored) and generate the template from internal/config: every TAPIR_* var,
which command needs it, accurate defaults, op/port-forward notes for demo time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
main.go dispatches `tapir auth` (interactive OAuth → persist refresh token via
SecretStore) and `tapir run` (wire YouTube source + local summarizer + store
sink, build engine, run the dedup-aware loop). Config-driven so live creds plug
in at demo time; SIGINT stops the loop cleanly. Block kept minimal so Worker E's
list/show cases union cleanly at merge.
Add .env.example documenting every TAPIR_* var and a README demo runbook. Pin
the summarizer alias-as-config decision and record the max_tokens fix in
docs/homelab-integration.md (clears two `confirm` items).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RunOnce walks the user's subscriptions, upserts each candidate video (assigning
its durable store id), skips videos already summarized via the store's
SeenVideoIDs (cross-restart dedup the engine's in-memory map can't provide),
and processes the rest through the engine. Loop adds an optional poll cadence;
per-item errors are collected, not fatal. Tested with fakes — no live deps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Sink port carries only a Summary, so video title/url/published_at would
never reach the store. UpsertVideo (new file, store.go untouched) persists them
and returns the durable videos.id UUID, idempotent on
(user_id, provider, provider_video_id). The run loop uses that id as v.ID, so
it equals summaries.video_id and SeenVideoIDs dedup survives restarts.
subscription_id stays NULL: the YouTube resource id is not a UUID (Stage 0).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`tapir auth` mints a refresh token for the single Stage-0 user: bind a local
redirect listener, print the consent URL (offline access + forced consent so
Google returns a refresh token), validate the state param, exchange the code,
and persist the refresh token through the SecretStore port. Written fresh on
x/oauth2 (ADR-006). Token is never logged or returned. Tests cover exchange,
missing-refresh-token rejection, and the full listener flow with httptest.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tapir list — table of stored summaries (date, title|id, channel, AI,
fallback), recent-first. tapir show <video-id> — full summary with
highlights and takeaways. DSN + user id from TAPIR_DB_DSN/TAPIR_USER_ID,
never hardcoded. main.go gains a minimal os.Args[1] dispatcher kept flat
so Worker F's auth/run cases union cleanly at merge.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ListSummaries (recent-first, user-scoped, limit) and GetSummaryByVideo
LEFT JOIN videos for title/url/published_at, null-safe when no videos
row exists. Channel mirrors provider for now — channel_title lives on
the not-yet-migrated subscriptions table (data-model.md). New file so it
does not collide with Worker F's concurrent edits to store.go.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements ports.SecretStore over a 0600 JSON file as a stand-in for op/ESO so
the demo runs without live op. Put persists atomically (temp + rename) and
merges; Get returns ErrNotFound for unknown refs so a missing token fails loud.
Behind the port, so swapping to op/ESO later is wiring, not code (ADR-002).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Parse TAPIR_* env into a typed Config with homelab defaults (gateway URL,
summarizer model, token ref, redirect addr). Secrets (gateway key, OAuth
client secret) come from env only; the refresh token never lives here — it is
addressed by an opaque ref behind the SecretStore port. Per-command validation
(ValidateForAuth/ValidateForRun) so each command demands only what it needs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The copied OpenAI-compatible client sent no max_tokens. Thinking models
(qwen3, deepseek-r1) spend their budget on the reasoning trace and return
EMPTY content when max_tokens is unset, which the summarizer treats as an
error. ADR-004 says change Tapir's copy rather than the hyperguild upstream,
so set a generous default (8192) leaving room for both reasoning and output.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements ports.Sink over Postgres (pgx/v5 + pgxpool, DSN from env per
estate convention). This is the primary sink (ADR-003) and the source of
the engine's durable, cross-restart dedup — the in-engine processed map is
process-lifetime only.
- Migrations (golang-migrate, NNN_name.up/down.sql per estate convention,
applied from an embedded FS on New): users, videos, transcripts,
summaries, sink_deliveries. Every user-owned table carries user_id
(Stage-0 per-user isolation promise, data-model.md). summaries has
UNIQUE(user_id, video_id) — at most one summary per video; highlights /
takeaways are jsonb.
- Deliver upserts the summary idempotently on (user_id, video_id)
(ON CONFLICT DO UPDATE) inside one tx with its sink_delivery row. Re-
delivering the same summary updates in place, never duplicates or errors.
- Dedup reads (store methods, not a new port): HasSummary(ctx,userID,
videoID) and SeenVideoIDs(ctx,userID) — both user_id-scoped, so one
user never sees another's videos.
summaries.video_id is intentionally not FK-constrained to videos at Stage 0:
the sink receives only a Summary, so the dedup key stands alone; video-row
persistence is the engine/source's concern, deferred.
Tested against a real in-process Postgres via embedded-postgres (real SQL:
constraints, ON CONFLICT, jsonb, user_id scoping) — no docker, no live
cluster, no creds, fully offline.
Deps: golang-migrate/migrate/v4 and jackc/pgx/v5 (runtime),
fergusstrange/embedded-postgres + stretchr/testify (test-only). go mod tidy
raised the go directive to 1.25.0 (minimum required by the dep graph;
estate elsewhere already runs 1.26.1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mathias
