mathiasandClaude Opus 4.8
27fd33c99c
Bring the living docs current with the recency-bounded auto-summarize + sparse
honesty + feed IA bundle (ADR-020):
- requirements (BDD): summarize_mode.feature — auto now summarizes RECENT new
videos; added a scenario for older videos (listed, on-demand), recency note.
- architecture.md: summarization-mode + new list-surface paragraph; scheduler
diagram + two-path table + three-phase pass now show the recency pre-filter;
dropped stale "Summarize now".
- data-model.md: auto_summarize is recent-only, older on-demand.
- README.md: one-line recency note on the serve scheduler.
- ui-spec.md: appended the as-built ADR-020 row (supersedes earlier copy/sort).
- specs/{video-card-states,newest-first-ordering,scheduled-discovery}.md:
superseded/extended banners pointing at ADR-020 (kept as design records).
Docs-only; task check green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
154 lines
7.5 KiB
Markdown
154 lines
7.5 KiB
Markdown
# tapir
|
|
|
|
Watches a user's YouTube/Vimeo subscriptions and, when a subscribed channel posts a new
|
|
video, summarizes it into highlights and takeaways using a **local-first** AI stack with an
|
|
optional, per-user **BYO-AI** fallback. **Standalone-first**; feeding a personal knowledge
|
|
base ("brain") is one optional sink, not the reason Tapir exists. Written in Go.
|
|
|
|
## Status
|
|
|
|
**Stage-0 demo slice built.** The engine, ports, domain, and adapters (`llm`, `summarizer`,
|
|
captions-first `youtube`, Postgres `store`) are implemented and green, with the `tapir`
|
|
CLI (`auth` / `run` / `list` / `show`) wiring the end-to-end loop for a single user. The
|
|
remaining work to *open* the Stage-0 clock is the live run on real subscriptions — see
|
|
"Running the Stage-0 demo" below. The guardrail docs (vision, decisions, architecture, data
|
|
model, behavior specs) remain the source of intent.
|
|
|
|
## Read these first (the guardrails)
|
|
|
|
| Doc | What it is |
|
|
|-----|-----------|
|
|
| [`VISION.md`](VISION.md) | Product vision, principles, and the **staged Definition of Success**. Stage 0 ("useful to me") is the gate before any multi-user work. |
|
|
| [`DECISIONS.md`](DECISIONS.md) | Architecture Decision Records (append-only). Why Go, why no Supabase, standalone-first, captions-first, etc. |
|
|
| [`docs/architecture/architecture.md`](docs/architecture/architecture.md) | C4 context + container diagrams, key sequence diagrams, and the Clean Architecture layering (Mermaid). |
|
|
| [`docs/data-model.md`](docs/data-model.md) | Entities and the per-user isolation model (Stage 0 / Stage 1 scope). |
|
|
| [`docs/use-cases/`](docs/use-cases/) | **Gherkin `.feature` files** — the BDD behavior spec that seeds the test suite. |
|
|
|
|
## Approach
|
|
|
|
- **Clean Architecture / ports & adapters.** A provider- and sink-agnostic engine depends only
|
|
on interfaces (`VideoSource`, `Summarizer`, `Sink`, `SecretStore`). YouTube, Vimeo, the AI
|
|
router, the user store, and the brain sink are adapters. "Standalone vs homelab" is a wiring
|
|
choice, not two codebases.
|
|
- **TDD/BDD.** The `.feature` files are the living behavior spec; the use-case core is tested
|
|
through fake adapters. Behavior is specified as executable scenarios, not prose that drifts.
|
|
- **Trunk-Based Development.** Commit directly to `main`, one logical change per commit, every
|
|
commit deployable (see ADR-009). CI is the quality gate.
|
|
|
|
## Running the Stage-0 demo
|
|
|
|
Tapir runs on your YouTube account(s): authorize once, then run the
|
|
watch→summarize→deliver loop. All configuration is via `TAPIR_*` environment
|
|
variables — copy [`.env.example`](.env.example) to `.env` and fill it in (no
|
|
secrets are committed; at demo time source them from op, e.g. `op run -- ...`).
|
|
|
|
```sh
|
|
# 1. configure (UUID user id, gateway URL+key, Postgres DSN, YouTube OAuth app,
|
|
# summarizer model). See .env.example for every variable.
|
|
cp .env.example .env && $EDITOR .env
|
|
set -a && . ./.env && set +a # export them into the shell
|
|
|
|
go build -o bin/tapir ./cmd/tapir
|
|
|
|
# 2. one-time: authorize YouTube. Opens a consent URL, captures the redirect on
|
|
# TAPIR_OAUTH_REDIRECT_ADDR, and stores the refresh token via the SecretStore
|
|
# (a 0600 file at Stage 0). The token is never logged.
|
|
./bin/tapir auth
|
|
|
|
# 3. run: detect new videos across your subscriptions, summarize, deliver to the
|
|
# store. Single pass; set TAPIR_DISCOVERY_INTERVAL (e.g. 2h) for the serve loop.
|
|
./bin/tapir run
|
|
```
|
|
|
|
Live prerequisites at demo time: the LiteLLM gateway reachable
|
|
(`TAPIR_GATEWAY_URL` + a valid key — resolve from op, the documented
|
|
`sk-local-123` is stale), a Postgres DSN (`TAPIR_DB_DSN`, migrations apply on
|
|
first connect), and a registered YouTube OAuth client whose authorized redirect
|
|
URI matches `TAPIR_OAUTH_REDIRECT_ADDR`. The summarizer model
|
|
(`TAPIR_SUMMARIZER_MODEL`, default `koala/phi4-mini`) is overridable; pick the
|
|
final alias when the gateway is reachable (see `docs/homelab-integration.md`).
|
|
|
|
### Web surface (`tapir serve`)
|
|
|
|
`tapir serve` starts the HTMX+Templ web UI on `:8080`. Users log in via Dex OIDC (local
|
|
password or Google); a new Dex subject is routed to `/register` to create a Tapir account.
|
|
Stage 1 is multi-user: each user connects their own YouTube account from the browser and
|
|
manages their own summaries under DB-enforced RLS isolation. When `TAPIR_DISCOVERY_INTERVAL`
|
|
is set (e.g. `2h`), the serve process runs a scheduled discovery pass for every registered
|
|
user automatically — no CronJob required. In auto mode only videos published within
|
|
`TAPIR_AUTO_SUMMARIZE_WINDOW` (default ~7d, ADR-020) are summarised automatically; older videos
|
|
are listed and summarised on demand, so a large back-catalogue doesn't keep re-driving the
|
|
caption rate gate. See `docs/homelab-integration.md` for the full config reference.
|
|
|
|
### Headless on koala
|
|
|
|
koala has no browser and no interactive `op` session, so the two interactive
|
|
edges are handled without changing any code:
|
|
|
|
**Secrets via an `op` service account** (no `op signin`). Create a 1Password
|
|
service account with read on the `HomeLab` vault, export its token, and keep one
|
|
op-style env file `tapir.env` — secret values as `op://` refs, the rest as
|
|
literals — then prefix every command with `op run`:
|
|
|
|
```sh
|
|
export OP_SERVICE_ACCOUNT_TOKEN=ops_... # the service-account token
|
|
|
|
cat > tapir.env <<'EOF'
|
|
TAPIR_USER_ID=<your-uuid>
|
|
TAPIR_GATEWAY_URL=http://koala:30401/v1
|
|
TAPIR_GATEWAY_KEY=op://HomeLab/LITELLM_MASTER_KEY/password
|
|
TAPIR_SUMMARIZER_MODEL=koala/phi4-mini
|
|
TAPIR_DB_DSN=op://HomeLab/TAPIR_DB_DSN/password
|
|
TAPIR_YT_CLIENT_ID=op://HomeLab/TAPIR_YT_OAUTH/client_id
|
|
TAPIR_YT_CLIENT_SECRET=op://HomeLab/TAPIR_YT_OAUTH/client_secret
|
|
TAPIR_YT_TOKEN_REF=youtube/refresh_token
|
|
TAPIR_SECRETS_FILE=/home/mathias/.config/tapir/secrets.json
|
|
TAPIR_OAUTH_REDIRECT_ADDR=localhost:8080
|
|
EOF
|
|
chmod 600 tapir.env
|
|
```
|
|
|
|
(Create the `TAPIR_DB_DSN` and `TAPIR_YT_OAUTH` items in the `HomeLab` vault
|
|
first; `LITELLM_MASTER_KEY` already exists. `sk-local-123` is stale.)
|
|
|
|
**`tapir auth` over an SSH tunnel.** The auth command binds a listener on
|
|
`localhost:8080` *on koala* and prints the consent URL to stdout — it never
|
|
opens a browser. Forward that port to your laptop, run auth, and approve in your
|
|
laptop's browser; the redirect returns through the tunnel:
|
|
|
|
```sh
|
|
# from your laptop:
|
|
ssh -L 8080:localhost:8080 koala
|
|
# now on koala (one-time):
|
|
op run --env-file tapir.env -- ./bin/tapir auth
|
|
# → copy the printed URL into your laptop browser, approve; token is stored.
|
|
```
|
|
|
|
The Google OAuth client must be a **Desktop/Web** type with authorized redirect
|
|
`http://localhost:8080/callback` (matching `TAPIR_OAUTH_REDIRECT_ADDR`), the
|
|
YouTube Data API enabled, and your account added as a test user.
|
|
|
|
**Everything else is already non-interactive** — once the token is stored, run
|
|
headless:
|
|
|
|
```sh
|
|
op run --env-file tapir.env -- ./bin/tapir run # detect → summarize → deliver
|
|
op run --env-file tapir.env -- ./bin/tapir list # read summaries back
|
|
op run --env-file tapir.env -- ./bin/tapir show <video-id>
|
|
```
|
|
|
|
Postgres (`postgres18`) is ClusterIP-only; from the koala host reach it with
|
|
`kubectl port-forward -n databases svc/postgres18 5432:5432` and point
|
|
`TAPIR_DB_DSN` at `localhost:5432` (migrations apply on first connect).
|
|
|
|
## Conventions
|
|
|
|
Reuses homelab conventions: Go, Dex for identity, ESO + 1Password for secrets, Postgres for
|
|
persistence. No new auth or secrets system (ADR-002).
|
|
|
|
## Next
|
|
|
|
First implementation step: scaffold the Go service (engine + interfaces + the copied `llm`
|
|
package), captions-first, with the store and brain sink adapters — decomposable into
|
|
independent units suitable for a Claude Code swarm. Tracked as the first build issue.
|