docs(architecture): add in-process scheduler sequence, rate gate description, auto_summarize default fix

This commit is contained in:
2026-06-06 11:27:18 +02:00
parent 1ad1966672
commit 252a4ebd9e
+57 -2
View File
@@ -145,8 +145,9 @@ graph TB
engine in a **background goroutine** inside `serve`; the page HTMX-polls `/v/{id}/status`,
showing a Charmbracelet spinner while in-flight (and an honest "queued/waiting" state under
rate-limiting — ADR-014).
- **Summarization mode** — `users.auto_summarize` (migration 006). Auto: every new video is
summarized. Manual (default): new videos appear unsummarized; the button sets
- **Summarization mode** — `users.auto_summarize` (migration 006). Default is **true** for new
users (migration 011, ADR-018); all existing rows were back-filled via migration 012. Auto:
every new video is summarized. Manual: new videos appear unsummarized; the button sets
`videos.summarize_requested`, which the next `tapir run` processes and clears. Both the click
path and the batch `tapir run` drive the same unchanged engine.
@@ -155,6 +156,60 @@ reads the store and triggers the existing engine. Adding it changed wiring, not
---
## In-process scheduler (ADR-018)
`cmdServe` launches a background goroutine when `TAPIR_DISCOVERY_INTERVAL > 0`. On each tick
it calls `store.ListAllUsers` (un-RLS'd admin query), builds a per-user `runner.Runner`, and
calls `RunOnce` for each registered user in sequence.
```mermaid
sequenceDiagram
participant S as Scheduler goroutine
participant DB as Postgres (RLS)
participant YT as YouTube timedtext
participant LLM as LiteLLM gateway
loop every TAPIR_DISCOVERY_INTERVAL
S->>DB: ListAllUsers() [un-RLS'd]
loop per user
S->>DB: GetAutoSummarize(userID)
S->>YT: ListSubscriptions + NewVideos
Note over S,YT: WaitFetchGate(ctx) throttles<br/>all fetches to TAPIR_FETCH_RATE
alt transcript available
S->>LLM: Summarize
S->>DB: Deliver(summary)
else 429
S->>DB: SetTranscriptStatus(rate_limited)
end
end
end
```
**Single-replica constraint (load-bearing).** The scheduler lives in the web process;
`replicas: 1` in the k3s deployment manifest is not cosmetic — running `tapir serve` at
>1 replica makes every replica run the full discovery loop, causing every registered user
to be fetched in parallel from the same egress IP (429s + duplicate work). Do not scale
`serve` past 1 replica without first moving discovery to a k8s CronJob or adding leader
election. The process logs a `Warn` at startup when scheduled discovery is enabled as a
reminder.
---
## Process-wide timedtext rate gate
**`internal/adapters/youtube/gate.go`** (ADR-014 item 2): a single `rate.Limiter`
(`golang.org/x/time/rate`) shared across **all** Adapter instances. Every `httpDo` call for
a caption fetch passes through `WaitFetchGate(ctx)` before hitting YouTube. This serialises
the scheduler loop AND the web click-path through the same per-egress-IP budget. Configured
via `TAPIR_FETCH_RATE` (Go duration, default `2s`). Setting it to `0` disables the gate
(dev/tests only).
This is the precondition that makes scheduled auto-summarize safe: without the gate, a
multi-user scheduler pass could fire many concurrent timedtext requests from the same IP
within seconds, triggering 429s for all users.
---
## Sequence — core use case: new video summarized
```mermaid