From 7e9f5d7ca5865f8817e22b44aee90aefcfb944e8 Mon Sep 17 00:00:00 2001 From: Mathias Date: Fri, 2 Oct 2026 08:49:06 +0200 Subject: [PATCH] docs(adr): ADR-035 -- progress as real phases, the same transitions in the log, every judged spot kept (#42 #43 #44) Co-Authored-By: Claude Sonnet 5.5 --- DECISIONS.md | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/DECISIONS.md b/DECISIONS.md index 420e550..553ceeb 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1596,7 +1596,8 @@ on the slow numbers and is reopened: 31B scored 97% against 92%. timeout at a time. Both engines fail → ADR-033's fallback path. 6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed - segments, untouched: `content` is authoritative for chat, `segments` only for time. `endpoint` + segments, untouched (**amended by ADR-035: they now carry the merged words, timing stays the + base's**): `content` is authoritative for chat, `segments` only for time. `endpoint` lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute mark and both readings**, so the user can check them against the audio. The transcript page says in @@ -1659,6 +1660,77 @@ unchanged. --- +## ADR-035 — Show an upload's progress as real phases, log the same transitions, and keep every judged spot + +**Status:** Proposed (2026-10-02). Decider: Mathias (pending). Amends ADR-034 decision 6 (segments). +Closes #42, #43 and #44. + +**Context (measured, not assumed).** The first real run (a 63.6-minute recording, 21 min 49 s) exposed +three gaps. (1) The page said only `queued`/`transcribing`/`done`; during 22 minutes nothing told the +user whether the job was working, hung or dead, and after a crash the row would say `transcribing` +until the next start. (2) The server log had no line about the job between start-up and the final +line, so an operator saw it only from outside (spool files, iguana CPU). (3) Evaluating the result +showed that spots the model *kept* as the base are counted but not stored (so ADR-034's "no worse than +the better single engine" cannot be measured after the fact, and the audio is not retained), insertions +and deletions cannot be placed without context, and the saved `segments` still carry the base engine's +wording while only `content` is merged. + +**Decision.** + +1. **Phases are data on the row.** A new `progress` JSONB column (migration 020, same RLS-scoped table) + holds an ordered list of steps `{key, label, state, started_at, finished_at, done, total}` with state + `pending|running|done|failed|skipped`. The plan is declared up front so the page shows what is coming: + received, preparing audio, then the transcriber's own steps (two-engine: engine A, engine B, + comparing, deciding n of N; single: transcribing), then saving. +2. **One source for the page and the log.** An `internal/progress` Tracker is the only writer. Every + transition is saved to the row and logged by the same call, so the UI and the log cannot disagree. + Engines and the reconciler report through a context-carried `Reporter` (no-op when absent), so the + `Transcriber` interface does not change. +3. **Alive is not the same as progressing, and the page says which it knows.** A heartbeat goroutine, + independent of any engine call, stamps `alive_at` every 10 s; `progress_at` moves only on a real + transition or advance. The page derives: *running*; *quiet* (the deciding step has not advanced for + 5 minutes, so the model is probably waiting or slow); *lost* (`alive_at` older than 45 s, so the + service stopped reporting and may have restarted or crashed). The two engine calls are single blocking + requests, so inside them the page shows elapsed time only: **no percentage is ever shown that does not + come from real counts.** Only the deciding step has a bar. +4. **The page.** A server-rendered step list in tapir's existing tokens and mono font, swapped in place + by the existing HTMX poll (every 2 s while unfinished, none once done or failed). Finished steps + collapse to one line with their duration; the running step has a CSS-only spinner that respects + `prefers-reduced-motion`; a failed step and a lost service are visibly different from running. + No new JavaScript, framework or stylesheet. +5. **Logs.** One INFO line per phase transition and engine call (upload id, endpoint, elapsed), the + reconcile counts, `adjudicated n/N` at most once a minute, a WARN when an engine soft-fails or the + breaker trips, and a "still in phase X" line every 5 minutes. **Ids, counts, durations, endpoints and + error kinds only: never transcript text, disputed words or the filename**, asserted by a test. +6. **Every judged spot is stored.** `reconciliation.contested` keeps each model-decided spot with its + verdict (changed, kept base, uncertain) and a few words of left context, bounded at 2,000. The page + still lists only changed and uncertain spots. This makes ADR-034's second condition measurable on the + next real run. +7. **Segments carry the merged words.** The reconciled words replace the base's in the saved `segments`; + timing stays the base's. If the segments cannot be aligned to the merged text (a word wrapped across + a segment edge), they stay the base's and the result says so in the log. This replaces ADR-034 + decision 6's "segments stay untouched". + +**Alternatives rejected.** Server-sent events or a websocket: a long-lived connection through the +ingress, with buffering to get wrong, for a page that changes every few seconds at best; polling is what +the page already does. A percentage from the engine calls: there is no data behind it. A separate +progress table: progress belongs to the row and inherits its RLS. Logging only through the UI: an +operator has no page. + +**Consequences.** Positive: running, quiet, lost and failed are distinguishable by the user and the +operator; the next blind check can answer ADR-034's second condition. Negative / watch: two more writes +per transition and one per 10 s per running job (one job at a time, so negligible); a heartbeat proves +the process is alive, not that an engine is making progress inside one request, which only the job +timeout (60 min) bounds. The stored spot list grows to a few hundred small objects per hour of audio. + +**Validation.** A fake engine held open past the thresholds must change the page (quiet, lost); a log +capture on a synthetic transcript must contain none of its words; a screenshot review of the running, +quiet, lost, failed and done states against the existing pages (the visual claims tests cannot assert). +Revisit if the heartbeat write ever shows in the database's load, or if a longer recording cap makes +five-minute quiet detection too coarse. + +--- + ## Rejected alternatives Approaches considered during the 2026-06-02 planning + grill session and **deliberately not