docs: spec in-process scheduled discovery + auto-summarize + rate-gate finish
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 13s

The Stage-0 usability fix: tapir serve runs discovery for all users on an
interval (reusing the existing Runner.Loop, enumerate-users-then-withUser),
auto-summarize defaults ON for Future-B users so the list fills itself, and the
ADR-014 process-wide per-egress-IP rate gate is confirmed/finished in the same
slice because in-process + auto + multi-user makes it load-bearing. Records the
single-replica constraint as load-bearing, and a fallback (auto-summarize OFF
until the gate exists) so the dangerous combination never ships half-built.
This commit is contained in:
mathias
2026-06-05 12:59:50 +00:00
parent 4706c508e9
commit ca9bf2657f
+87
View File
@@ -0,0 +1,87 @@
# Spec — In-process scheduled discovery + auto-summarize + rate-gate finish
**Repo:** tapir · **Size:** medium · **Solo session** (not a swarm).
**Why this exists.** The Stage-0 gate ("me or a friend returns and reads/acts in ≥2 separate
weeks") cannot be met because the system is not usable *unprompted*: discovery (`tapir run`) is
host-side manual, so a newly onboarded user sees an empty list and never comes back. This slice
makes Tapir watch on its own — the thing that makes the gate experiment actually runnable.
Read `CLAUDE.md` + `DECISIONS.md` (esp. ADR-012, ADR-014, and the new ADR-018) first. TBD —
commit directly to `main`, one logical change per commit, conventional commits, `task check`
green before each commit. `templ generate` if any view changes.
## Decisions already made (do not reopen)
- **In-process scheduler**, NOT a k8s CronJob (maintainer's call: simpler deploy, acceptable
coupling at 3 users). The known cost — discovery shares the web process's lifetime and egress
— is accepted and recorded in ADR-018.
- **Auto-summarize ON** for the maintainer + onboarded friends (zero-friction: the list fills
and summarizes itself).
- **Gate clock resets** to when this ships (ADR-018) — until unprompted use is possible, the
prior window measured nothing.
## 1. In-process scheduled discovery (core)
- In `tapir serve` startup, launch a background goroutine that runs discovery for ALL users on
an interval: env `TAPIR_DISCOVERY_INTERVAL` (Go duration, e.g. `2h`). **Unset or 0 = disabled**
(so dev/tests never auto-fetch).
- **Reuse the existing `runner.Runner` + `Loop`/`RunOnce`. Do NOT write a new scheduler.** The
per-user Runner already exists; the new work is **iterating users** and running one pass each
per tick. Enumerate users from the un-RLS'd `user_identities` (the same enumerate-then-act
pattern the login_events gate query established), then run each user's pass **inside that
user's RLS scope** (`withUser`).
- **Stateless timing:** run-once-on-startup, then every interval — exactly the existing `Loop`
shape. Do NOT persist schedule state; a pod restart just restarts the cycle. Acceptable at this
scale. Do not build cron-in-Go.
- **Graceful shutdown:** the goroutine respects `ctx` cancellation so a pod term doesn't wedge.
- **Failure isolation in the loop:** one user's pass failing (or one channel/video) must not
abort the other users or crash `serve` — log and continue. (RunOnce already collects per-item
errors; preserve that at the per-user level too.)
## 2. Auto-summarize default ON for Future-B users
- New registrations default `auto_summarize = true` (so onboarded friends get zero-friction);
keep the account-page toggle so a user can switch to manual. One-off update existing user rows
to `true` as well (maintainer + any current users).
- Consequence (intended): scheduled discovery both discovers AND summarizes new videos — which
is the point, and is why §3 is mandatory in the same slice.
## 3. Finish/confirm the ADR-014 shared per-egress-IP rate gate (NOW load-bearing)
- In-process scheduling + auto-summarize + multiple users = all caption fetches leave the **one
web pod's egress IP**, concurrently with any live "Summarize" button clicks. The timedtext
endpoint rate-limits per IP (ADR-010/014). Without a shared gate this self-inflicts 429s every
cycle.
- **Confirm in code whether ADR-014 item 2 (a single PROCESS-WIDE rate gate) exists.**
Reconciliation flagged it as possibly built only as per-*video* backoff. If it is not a
process-wide gate, **build it now**: ONE shared limiter (token-bucket / min-interval) that
every timedtext/caption fetch passes through — scheduler loop AND click-path alike. Per-process,
not per-user, not per-video.
- Keep the existing per-video 429 backoff (`rate_limited_at` + retry window) — complementary: the
gate prevents tripping 429; the backoff handles it if one still happens.
- Honest UX (ADR-014 item 3) still applies: a fetch waiting on the gate shows "queued/waiting",
never a stuck spinner.
## 4. Gate-clock reset — already recorded in ADR-018; verify VISION reflects it
- ADR-018 (committed) resets the Stage-0 34 week window to start when this ships, and revises the
check-in date. VISION Stage 0 carries a pointer to it. The build doesn't re-decide this; just
ensure nothing in docs still implies the clock started earlier.
## Tests
- **Scheduler:** fake clock + fake Runner → N users each get one pass per tick; one user's failure
doesn't stop the others; `ctx` cancel stops the loop; interval=0 disables it entirely.
- **Rate gate:** concurrent fetches (scheduler + simulated click) are serialized/limited through
the ONE gate — assert max-in-flight / min-interval honored regardless of caller.
- **Auto-summarize default:** new registration → `auto_summarize = true`; account-page toggle
still flips it.
## Out of scope / known constraints
- No CronJob / k8s objects (in-process chosen).
- **SINGLE-REPLICA ASSUMPTION (load-bearing).** In-process scheduling means if `tapir serve` ever
runs >1 replica, every replica runs the discovery loop → every user fetched in parallel (429s +
duplicate work). At 3 users this is single-replica, fine — but the build MUST note this
constraint in ADR-018 / deploy docs so a future scale-up doesn't silently double-run.
- No persisted schedules, no multi-pod coordination.
## Fallback if the session runs short
Ship the scheduler with **auto-summarize OFF** (discovery only; manual Summarize button) until the
process-wide rate gate (§3) is confirmed/built. NEVER ship auto-summarize-on-a-schedule without the
gate — that combination self-inflicts 429s for every user every cycle. Auto-summarize ON is gated
on §3 being done.