Restructures RunOnce from per-channel inline processing to collect-sort-process:
Phase 1 — discover, persist (UpsertVideo), apply pre-filters (seen/manual/backoff)
and collect surviving candidates with their discovery position.
Phase 2 — sort candidates by published_at DESC, NULLS LAST, pos ASC tiebreak
so videos with no publish date never jump ahead of dated content.
Phase 3 — process in sorted order through the unchanged globalFetchGate.
Before (per-channel): chanA=[v-old, v-mid], chanB=[v-new, v-null]
→ [v-old, v-mid, v-new, v-null]
After (newest-first): [v-new, v-mid, v-old, v-null]
Same set of videos processed; only the order changes within a pass. All existing
behaviour is preserved: failure isolation, backoff skip, manual mode,
channel-unavailable, stats. In-memory sort; no new table or persisted queue.
The ordering is onboarding prioritisation — new users get summaries of their most
recent, relevant videos first; the back-catalogue fills in behind across subsequent
passes. Both this background batch and the foreground 'Try now' button honour the
shared globalFetchGate: rate limiting is respected, not evaded.
tapir
Watches a user's YouTube/Vimeo subscriptions and, when a subscribed channel posts a new video, summarizes it into highlights and takeaways using a local-first AI stack with an optional, per-user BYO-AI fallback. Standalone-first; feeding a personal knowledge base ("brain") is one optional sink, not the reason Tapir exists. Written in Go.
Status
Stage-0 demo slice built. The engine, ports, domain, and adapters (llm, summarizer,
captions-first youtube, Postgres store) are implemented and green, with the tapir
CLI (auth / run / list / show) wiring the end-to-end loop for a single user. The
remaining work to open the Stage-0 clock is the live run on real subscriptions — see
"Running the Stage-0 demo" below. The guardrail docs (vision, decisions, architecture, data
model, behavior specs) remain the source of intent.
Read these first (the guardrails)
| Doc | What it is |
|---|---|
VISION.md |
Product vision, principles, and the staged Definition of Success. Stage 0 ("useful to me") is the gate before any multi-user work. |
DECISIONS.md |
Architecture Decision Records (append-only). Why Go, why no Supabase, standalone-first, captions-first, etc. |
docs/architecture/architecture.md |
C4 context + container diagrams, key sequence diagrams, and the Clean Architecture layering (Mermaid). |
docs/data-model.md |
Entities and the per-user isolation model (Stage 0 / Stage 1 scope). |
docs/use-cases/ |
Gherkin .feature files — the BDD behavior spec that seeds the test suite. |
Approach
- Clean Architecture / ports & adapters. A provider- and sink-agnostic engine depends only
on interfaces (
VideoSource,Summarizer,Sink,SecretStore). YouTube, Vimeo, the AI router, the user store, and the brain sink are adapters. "Standalone vs homelab" is a wiring choice, not two codebases. - TDD/BDD. The
.featurefiles are the living behavior spec; the use-case core is tested through fake adapters. Behavior is specified as executable scenarios, not prose that drifts. - Trunk-Based Development. Commit directly to
main, one logical change per commit, every commit deployable (see ADR-009). CI is the quality gate.
Running the Stage-0 demo
Tapir runs on your YouTube account(s): authorize once, then run the
watch→summarize→deliver loop. All configuration is via TAPIR_* environment
variables — copy .env.example to .env and fill it in (no
secrets are committed; at demo time source them from op, e.g. op run -- ...).
# 1. configure (UUID user id, gateway URL+key, Postgres DSN, YouTube OAuth app,
# summarizer model). See .env.example for every variable.
cp .env.example .env && $EDITOR .env
set -a && . ./.env && set +a # export them into the shell
go build -o bin/tapir ./cmd/tapir
# 2. one-time: authorize YouTube. Opens a consent URL, captures the redirect on
# TAPIR_OAUTH_REDIRECT_ADDR, and stores the refresh token via the SecretStore
# (a 0600 file at Stage 0). The token is never logged.
./bin/tapir auth
# 3. run: detect new videos across your subscriptions, summarize, deliver to the
# store. Single pass; set TAPIR_DISCOVERY_INTERVAL (e.g. 2h) for the serve loop.
./bin/tapir run
Live prerequisites at demo time: the LiteLLM gateway reachable
(TAPIR_GATEWAY_URL + a valid key — resolve from op, the documented
sk-local-123 is stale), a Postgres DSN (TAPIR_DB_DSN, migrations apply on
first connect), and a registered YouTube OAuth client whose authorized redirect
URI matches TAPIR_OAUTH_REDIRECT_ADDR. The summarizer model
(TAPIR_SUMMARIZER_MODEL, default koala/phi4-mini) is overridable; pick the
final alias when the gateway is reachable (see docs/homelab-integration.md).
Web surface (tapir serve)
tapir serve starts the HTMX+Templ web UI on :8080. Users log in via Dex OIDC (local
password or Google); a new Dex subject is routed to /register to create a Tapir account.
Stage 1 is multi-user: each user connects their own YouTube account from the browser and
manages their own summaries under DB-enforced RLS isolation. When TAPIR_DISCOVERY_INTERVAL
is set (e.g. 2h), the serve process runs a scheduled discovery pass for every registered
user automatically — no CronJob required. See docs/homelab-integration.md for the full
config reference.
Headless on koala
koala has no browser and no interactive op session, so the two interactive
edges are handled without changing any code:
Secrets via an op service account (no op signin). Create a 1Password
service account with read on the HomeLab vault, export its token, and keep one
op-style env file tapir.env — secret values as op:// refs, the rest as
literals — then prefix every command with op run:
export OP_SERVICE_ACCOUNT_TOKEN=ops_... # the service-account token
cat > tapir.env <<'EOF'
TAPIR_USER_ID=<your-uuid>
TAPIR_GATEWAY_URL=http://koala:30401/v1
TAPIR_GATEWAY_KEY=op://HomeLab/LITELLM_MASTER_KEY/password
TAPIR_SUMMARIZER_MODEL=koala/phi4-mini
TAPIR_DB_DSN=op://HomeLab/TAPIR_DB_DSN/password
TAPIR_YT_CLIENT_ID=op://HomeLab/TAPIR_YT_OAUTH/client_id
TAPIR_YT_CLIENT_SECRET=op://HomeLab/TAPIR_YT_OAUTH/client_secret
TAPIR_YT_TOKEN_REF=youtube/refresh_token
TAPIR_SECRETS_FILE=/home/mathias/.config/tapir/secrets.json
TAPIR_OAUTH_REDIRECT_ADDR=localhost:8080
EOF
chmod 600 tapir.env
(Create the TAPIR_DB_DSN and TAPIR_YT_OAUTH items in the HomeLab vault
first; LITELLM_MASTER_KEY already exists. sk-local-123 is stale.)
tapir auth over an SSH tunnel. The auth command binds a listener on
localhost:8080 on koala and prints the consent URL to stdout — it never
opens a browser. Forward that port to your laptop, run auth, and approve in your
laptop's browser; the redirect returns through the tunnel:
# from your laptop:
ssh -L 8080:localhost:8080 koala
# now on koala (one-time):
op run --env-file tapir.env -- ./bin/tapir auth
# → copy the printed URL into your laptop browser, approve; token is stored.
The Google OAuth client must be a Desktop/Web type with authorized redirect
http://localhost:8080/callback (matching TAPIR_OAUTH_REDIRECT_ADDR), the
YouTube Data API enabled, and your account added as a test user.
Everything else is already non-interactive — once the token is stored, run headless:
op run --env-file tapir.env -- ./bin/tapir run # detect → summarize → deliver
op run --env-file tapir.env -- ./bin/tapir list # read summaries back
op run --env-file tapir.env -- ./bin/tapir show <video-id>
Postgres (postgres18) is ClusterIP-only; from the koala host reach it with
kubectl port-forward -n databases svc/postgres18 5432:5432 and point
TAPIR_DB_DSN at localhost:5432 (migrations apply on first connect).
Conventions
Reuses homelab conventions: Go, Dex for identity, ESO + 1Password for secrets, Postgres for persistence. No new auth or secrets system (ADR-002).
Next
First implementation step: scaffold the Go service (engine + interfaces + the copied llm
package), captions-first, with the store and brain sink adapters — decomposable into
independent units suitable for a Claude Code swarm. Tracked as the first build issue.