docs: add ADR-018 — in-process scheduled discovery, auto-summarize, gate-clock reset
CI / Lint / Test / Vet (push) Successful in 12s
CI / Build & Import (push) Successful in 10s

Records the Stage-0 usability decision: tapir serve runs discovery for all users
on an interval (reusing runner.Loop), auto-summarize defaults ON so the list fills
itself, and the gate clock resets to when this ships (unprompted use was
impossible before, so the prior window measured nothing — framed as starting the
clock when the experiment can run, not dodging a failing gate). Records the
in-process-vs-CronJob tradeoff, the load-bearing single-replica constraint, and
that it makes ADR-014 item 2 (process-wide rate gate) a hard requirement folded
into the build. Cross-referenced ADR-014/016; added CronJob to rejected-alts.
This commit is contained in:
mathias
2026-06-05 13:03:18 +00:00
parent ca9bf2657f
commit 88294d38bc
+59 -3
View File
@@ -346,6 +346,8 @@ semantic that future-Tapir (and any compliance review) must know was chosen, not
**Status:** Accepted (2026-06-03). **Partially implemented as of v0.6.0** — see the **Status:** Accepted (2026-06-03). **Partially implemented as of v0.6.0** — see the
implementation note at the end; the shared per-egress-IP rate gate (decision item 2) may not be implementation note at the end; the shared per-egress-IP rate gate (decision item 2) may not be
fully realised. Verify against `internal/runner` + the youtube adapter before treating as done. fully realised. Verify against `internal/runner` + the youtube adapter before treating as done.
**ADR-018 (in-process scheduled discovery + auto-summarize) makes decision item 2 load-bearing
and folds its confirm/finish into that build.**
**Context.** ADR-010 acquires captions from the unauthenticated `timedtext` baseUrl. Live runs **Context.** ADR-010 acquires captions from the unauthenticated `timedtext` baseUrl. Live runs
show that endpoint **rate-limits per source IP (HTTP 429) under volume** — many videos fetched show that endpoint **rate-limits per source IP (HTTP 429) under volume** — many videos fetched
@@ -412,7 +414,9 @@ multiple videos can still each hit the endpoint and collectively trip the per-IP
remains open. (Resolved during reconciliation: the v0.6.0 report's reference to "migration 008 remains open. (Resolved during reconciliation: the v0.6.0 report's reference to "migration 008
videos.rate_limited_at" was a **mislabel** — both columns shipped in migration **007**, not a videos.rate_limited_at" was a **mislabel** — both columns shipped in migration **007**, not a
missing 008. No migration was lost; the sequence legitimately skips 008. `rate_limited_at` missing 008. No migration was lost; the sequence legitimately skips 008. `rate_limited_at`
exists and the runner's column reads are sound.) exists and the runner's column reads are sound.) **ADR-018 makes item 2 a hard requirement:
in-process scheduled discovery + auto-summarize drives all users' fetches through one pod egress,
so the process-wide gate is confirmed/built as part of that slice.**
--- ---
@@ -476,7 +480,8 @@ decision doc (`infra/docs/superpowers/handoffs/`). Build + reboot-validation: in
**Status:** Accepted (2026-06-03). Revises the Stage 0 definition in VISION.md (supersedes the **Status:** Accepted (2026-06-03). Revises the Stage 0 definition in VISION.md (supersedes the
original "useful to me, specifically" gate and folds in the old Stage 1 "a trusted user returns" original "useful to me, specifically" gate and folds in the old Stage 1 "a trusted user returns"
test). test). **The clock-start is further revised by ADR-018** (starts when scheduled discovery ships,
since unprompted use was not possible before then).
**Context.** The original Stage 0 gate was "the maintainer reads summaries weekly for four weeks **Context.** The original Stage 0 gate was "the maintainer reads summaries weekly for four weeks
and acts on one." The maintainer chose to change it to include friendly users, reasoning that and acts on one." The maintainer chose to change it to include friendly users, reasoning that
@@ -509,7 +514,7 @@ evidence the gate passed — use-evidence remains open.
near-identical to the new test); hardening renumbered to Stage 1. near-identical to the new test); hardening renumbered to Stage 1.
- New drift signal added: declaring the gate passed on polite feedback rather than return-usage. - New drift signal added: declaring the gate passed on polite feedback rather than return-usage.
- The 2026-07-01 check-in now asks "is anyone (me or a friend) coming back unprompted?", not - The 2026-07-01 check-in now asks "is anyone (me or a friend) coming back unprompted?", not
"am I using it weekly?". "am I using it weekly?". (Check-in date itself revised by ADR-018.)
**Reversibility.** A superseding ADR could tighten it back to maintainer-only or raise it to **Reversibility.** A superseding ADR could tighten it back to maintainer-only or raise it to
require multiple returning users. Recorded with the full rationale (including the self-scrutiny require multiple returning users. Recorded with the full rationale (including the self-scrutiny
@@ -607,6 +612,56 @@ OIDC.
--- ---
## ADR-018 — Make Tapir usable unprompted: in-process scheduled discovery, auto-summarize default, gate-clock reset
**Status:** Accepted (2026-06-03)
**Context.** The Stage-0 gate (ADR-016) measures whether the maintainer or a friend *returns and
reads/acts* over weeks. But the system could not actually be used that way: discovery (`tapir
run`) was **host-side manual**, so a newly onboarded user saw an empty list and had no reason to
return. The gate was structurally unmeetable — not because the product failed, but because the
*workflow* depended on the maintainer SSHing in to trigger each pass. The runner already has a
`Loop(ctx, interval)` (run-on-start, then every tick) and per-user mode/dedup/backoff; what was
missing was *invocation* — nothing called it for the deployed users on a schedule.
**Decision.**
1. **In-process scheduled discovery.** `tapir serve` launches a background goroutine that runs
discovery for **all users** on an interval (`TAPIR_DISCOVERY_INTERVAL`; unset/0 = off). It
enumerates users (un-RLS'd `user_identities`) and runs each user's pass inside `withUser`,
reusing the existing `runner` — not a new scheduler. Stateless timing; ctx-cancellable;
per-user failures isolated.
2. **Auto-summarize default ON** for Future-B users (new registrations default `true`; existing
rows updated), so discovery both populates *and* summarizes — the list fills itself. The
per-user manual toggle remains.
3. **Gate-clock reset.** The Stage-0 34 week window (ADR-016) **starts when this ships**, because
unprompted use was impossible before it. This is starting the clock when the experiment can
actually run, **not** a reset to dodge a failing gate (the prior window measured nothing —
there was no way to use the system unprompted). The 2026-07-01 check-in moves accordingly to
~34 weeks after this deploys.
**Why in-process and not a k8s CronJob.** Maintainer's call: simpler deploy (no second deployable),
acceptable at Future-B scale (13 users). The CronJob's advantage — failure isolation between
discovery and the web/read path — was weighed and traded away knowingly.
**Consequences / constraints.**
- **Discovery shares the web process's lifetime and egress.** A wedged discovery pass can degrade
the reading UI (the coupling a CronJob would have avoided). Accepted at this scale.
- **SINGLE-REPLICA ASSUMPTION (load-bearing).** If `tapir serve` ever runs >1 replica, *every*
replica runs the discovery loop → every user fetched in parallel (429s + duplicate work). Tapir
must stay single-replica while in-process scheduling is enabled, or this is revisited (move to
CronJob, or add leader-election). Recorded so a future scale-up doesn't silently double-run.
- **Makes ADR-014 item 2 (shared per-egress-IP rate gate) load-bearing.** In-process + auto +
multi-user drives all caption fetches through one pod egress, concurrent with click-path
Summarize. The build confirms/finishes the process-wide rate gate in the same slice; without it
the system self-inflicts 429s every cycle. **Auto-summarize ON is gated on the rate gate
existing** (fallback: ship discovery with auto OFF until it does).
**Reversibility.** Disable via `TAPIR_DISCOVERY_INTERVAL=0` (reverts to manual `tapir run`).
Moving to a CronJob later is a superseding ADR; the per-user `runner` is unchanged either way.
Spec: `docs/specs/scheduled-discovery.md`.
---
## Rejected alternatives ## Rejected alternatives
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
@@ -629,6 +684,7 @@ maps to the ADR that settles it.
| Supabase for per-user credential storage | Adds a second datastore for a few encrypted strings PG18 already holds; reopens ADR-002 | ADR-015, infra#88 | | Supabase for per-user credential storage | Adds a second datastore for a few encrypted strings PG18 already holds; reopens ADR-002 | ADR-015, infra#88 |
| Feedback-based Stage 0 gate (friends saying it's useful) | Politeness bias makes asked-for feedback the least reliable signal; return-usage is the real test | ADR-016 | | Feedback-based Stage 0 gate (friends saying it's useful) | Politeness bias makes asked-for feedback the least reliable signal; return-usage is the real test | ADR-016 |
| Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 | | Reverse the Dex-write invite flow (Google OIDC only) | Some intended Future-B users won't use Google; OIDC-only leaves them with no onboarding path — invite flow is load-bearing | ADR-017 |
| k8s CronJob for scheduled discovery (vs in-process) | At Future-B scale the in-process scheduler is simpler to deploy; CronJob's failure-isolation benefit was weighed and traded away knowingly (revisit if >1 replica or load grows) | ADR-018 |
If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one — If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
not a silent reversal. not a silent reversal.