docs: add ADR-018 — in-process scheduled discovery, auto-summarize, gate-clock reset
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s

Records the Stage-0 usability decision: tapir serve runs discovery for all users
on an interval (reusing runner.Loop), auto-summarize defaults ON so the list fills
itself, and the gate clock resets to when this ships (unprompted use was
impossible before, so the prior window measured nothing — framed as starting the
clock when the experiment can run, not dodging a failing gate). Records the
in-process-vs-CronJob tradeoff, the load-bearing single-replica constraint, and
that it makes ADR-014 item 2 (process-wide rate gate) a hard requirement folded
into the build. Cross-referenced ADR-014/016; added CronJob to rejected-alts.
This commit is contained in:
mathias
2026-06-05 13:03:18 +00:00
parent ca9bf2657f
commit 88294d38bc
+59 -3
View File
@@ -346,6 +346,8 @@ semantic that future-Tapir (and any compliance review) must know was chosen, not
**Status:** Accepted (2026-06-03). **Partially implemented as of v0.6.0** — see the
implementation note at the end; the shared per-egress-IP rate gate (decision item 2) may not be
fully realised. Verify against `internal/runner` + the youtube adapter before treating as done.
**ADR-018 (in-process scheduled discovery + auto-summarize) makes decision item 2 load-bearing
and folds its confirm/finish into that build.**
**Context.** ADR-010 acquires captions from the unauthenticated `timedtext` baseUrl. Live runs
show that endpoint **rate-limits per source IP (HTTP 429) under volume** — many videos fetched
@@ -412,7 +414,9 @@ multiple videos can still each hit the endpoint and collectively trip the per-IP
remains open. (Resolved during reconciliation: the v0.6.0 report's reference to "migration 008
videos.rate_limited_at" was a **mislabel** — both columns shipped in migration **007**, not a
missing 008. No migration was lost; the sequence legitimately skips 008. `rate_limited_at`
exists and the runner's column reads are sound.)
exists and the runner's column reads are sound.) **ADR-018 makes item 2 a hard requirement:
in-process scheduled discovery + auto-summarize drives all users' fetches through one pod egress,
so the process-wide gate is confirmed/built as part of that slice.**
---
@@ -476,7 +480,8 @@ decision doc (`infra/docs/superpowers/handoffs/`). Build + reboot-validation: in
**Status:** Accepted (2026-06-03). Revises the Stage 0 definition in VISION.md (supersedes the
original "useful to me, specifically" gate and folds in the old Stage 1 "a trusted user returns"
test).
test). **The clock-start is further revised by ADR-018** (starts when scheduled discovery ships,
since unprompted use was not possible before then).
**Context.** The original Stage 0 gate was "the maintainer reads summaries weekly for four weeks
and acts on one." The maintainer chose to change it to include friendly users, reasoning that
@@ -509,7 +514,7 @@ evidence the gate passed — use-evidence remains open.
near-identical to the new test); hardening renumbered to Stage 1.
- New drift signal added: declaring the gate passed on polite feedback rather than return-usage.
- The 2026-07-01 check-in now asks "is anyone (me or a friend) coming back unprompted?", not
"am I using it weekly?".
"am I using it weekly?". (Check-in date itself revised by ADR-018.)
**Reversibility.** A superseding ADR could tighten it back to maintainer-only or raise it to
require multiple returning users. Recorded with the full rationale (including the self-scrutiny
@@ -607,6 +612,56 @@ OIDC.
---
## ADR-018 — Make Tapir usable unprompted: in-process scheduled discovery, auto-summarize default, gate-clock reset
**Status:** Accepted (2026-06-03)
**Context.** The Stage-0 gate (ADR-016) measures whether the maintainer or a friend *returns and
reads/acts* over weeks. But the system could not actually be used that way: discovery (`tapir
run`) was **host-side manual**, so a newly onboarded user saw an empty list and had no reason to
return. The gate was structurally unmeetable — not because the product failed, but because the
*workflow* depended on the maintainer SSHing in to trigger each pass. The runner already has a
`Loop(ctx, interval)` (run-on-start, then every tick) and per-user mode/dedup/backoff; what was
missing was *invocation* — nothing called it for the deployed users on a schedule.
**Decision.**
1. **In-process scheduled discovery.** `tapir serve` launches a background goroutine that runs
discovery for **all users** on an interval (`TAPIR_DISCOVERY_INTERVAL`; unset/0 = off). It
enumerates users (un-RLS'd `user_identities`) and runs each user's pass inside `withUser`,
reusing the existing `runner` — not a new scheduler. Stateless timing; ctx-cancellable;
per-user failures isolated.
2. **Auto-summarize default ON** for Future-B users (new registrations default `true`; existing
rows updated), so discovery both populates *and* summarizes — the list fills itself. The
per-user manual toggle remains.
3. **Gate-clock reset.** The Stage-0 34 week window (ADR-016) **starts when this ships**, because
unprompted use was impossible before it. This is starting the clock when the experiment can
actually run, **not** a reset to dodge a failing gate (the prior window measured nothing —
there was no way to use the system unprompted). The 2026-07-01 check-in moves accordingly to
~34 weeks after this deploys.
**Why in-process and not a k8s CronJob.** Maintainer's call: simpler deploy (no second deployable),
acceptable at Future-B scale (13 users). The CronJob's advantage — failure isolation between
discovery and the web/read path — was weighed and traded away knowingly.
**Consequences / constraints.**
- **Discovery shares the web process's lifetime and egress.** A wedged discovery pass can degrade
the reading UI (the coupling a CronJob would have avoided). Accepted at this scale.
- **SINGLE-REPLICA ASSUMPTION (load-bearing).** If `tapir serve` ever runs >1 replica, *every*
replica runs the discovery loop → every user fetched in parallel (429s + duplicate work). Tapir
must stay single-replica while in-process scheduling is enabled, or this is revisited (move to
CronJob, or add leader-election). Recorded so a future scale-up doesn't silently double-run.
- **Makes ADR-014 item 2 (shared per-egress-IP rate gate) load-bearing.** In-process + auto +
multi-user drives all caption fetches through one pod egress, concurrent with click-path
Summarize. The build confirms/finishes the process-wide rate gate in the same slice; without it
the system self-inflicts 429s every cycle. **Auto-summarize ON is gated on the rate gate
existing** (fallback: ship discovery with auto OFF until it does).
**Reversibility.** Disable via `TAPIR_DISCOVERY_INTERVAL=0` (reverts to manual `tapir run`).
Moving to a CronJob later is a superseding ADR; the per-user `runner` is unchanged either way.
Spec: `docs/specs/scheduled-discovery.md`.
---
## Rejected alternatives
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
@@ -629,6 +684,7 @@ maps to the ADR that settles it.
| Supabase for per-user credential storage | Adds a second datastore for a few encrypted strings PG18 already holds; reopens ADR-002 | ADR-015, infra#88 |
| Feedback-based Stage 0 gate (friends saying it's useful) | Politeness bias makes asked-for feedback the least reliable signal; return-usage is the real test | ADR-016 |
| Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 |
| k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 |
If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
not a silent reversal.