feat: upload audio, get a private Swedish transcript, discuss it (#33, ADR-033) #39

Merged
mathias merged 299 commits from feat/33-audio-upload-transcript into main 2026-10-03 14:13:37 +00:00
Showing only changes of commit 7e9f5d7ca5 - Show all commits
+73 -1
View File
@@ -1596,7 +1596,8 @@ on the slow numbers and is reopened: 31B scored 97% against 92%.
timeout at a time. Both engines fail → ADR-033's fallback path.
6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed
segments, untouched: `content` is authoritative for chat, `segments` only for time. `endpoint`
segments, untouched (**amended by ADR-035: they now carry the merged words, timing stays the
base's**): `content` is authoritative for chat, `segments` only for time. `endpoint`
lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same
RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute
mark and both readings**, so the user can check them against the audio. The transcript page says in
@@ -1659,6 +1660,77 @@ unchanged.
---
## ADR-035 — Show an upload's progress as real phases, log the same transitions, and keep every judged spot
**Status:** Proposed (2026-10-02). Decider: Mathias (pending). Amends ADR-034 decision 6 (segments).
Closes #42, #43 and #44.
**Context (measured, not assumed).** The first real run (a 63.6-minute recording, 21 min 49 s) exposed
three gaps. (1) The page said only `queued`/`transcribing`/`done`; during 22 minutes nothing told the
user whether the job was working, hung or dead, and after a crash the row would say `transcribing`
until the next start. (2) The server log had no line about the job between start-up and the final
line, so an operator saw it only from outside (spool files, iguana CPU). (3) Evaluating the result
showed that spots the model *kept* as the base are counted but not stored (so ADR-034's "no worse than
the better single engine" cannot be measured after the fact, and the audio is not retained), insertions
and deletions cannot be placed without context, and the saved `segments` still carry the base engine's
wording while only `content` is merged.
**Decision.**
1. **Phases are data on the row.** A new `progress` JSONB column (migration 020, same RLS-scoped table)
holds an ordered list of steps `{key, label, state, started_at, finished_at, done, total}` with state
`pending|running|done|failed|skipped`. The plan is declared up front so the page shows what is coming:
received, preparing audio, then the transcriber's own steps (two-engine: engine A, engine B,
comparing, deciding n of N; single: transcribing), then saving.
2. **One source for the page and the log.** An `internal/progress` Tracker is the only writer. Every
transition is saved to the row and logged by the same call, so the UI and the log cannot disagree.
Engines and the reconciler report through a context-carried `Reporter` (no-op when absent), so the
`Transcriber` interface does not change.
3. **Alive is not the same as progressing, and the page says which it knows.** A heartbeat goroutine,
independent of any engine call, stamps `alive_at` every 10 s; `progress_at` moves only on a real
transition or advance. The page derives: *running*; *quiet* (the deciding step has not advanced for
5 minutes, so the model is probably waiting or slow); *lost* (`alive_at` older than 45 s, so the
service stopped reporting and may have restarted or crashed). The two engine calls are single blocking
requests, so inside them the page shows elapsed time only: **no percentage is ever shown that does not
come from real counts.** Only the deciding step has a bar.
4. **The page.** A server-rendered step list in tapir's existing tokens and mono font, swapped in place
by the existing HTMX poll (every 2 s while unfinished, none once done or failed). Finished steps
collapse to one line with their duration; the running step has a CSS-only spinner that respects
`prefers-reduced-motion`; a failed step and a lost service are visibly different from running.
No new JavaScript, framework or stylesheet.
5. **Logs.** One INFO line per phase transition and engine call (upload id, endpoint, elapsed), the
reconcile counts, `adjudicated n/N` at most once a minute, a WARN when an engine soft-fails or the
breaker trips, and a "still in phase X" line every 5 minutes. **Ids, counts, durations, endpoints and
error kinds only: never transcript text, disputed words or the filename**, asserted by a test.
6. **Every judged spot is stored.** `reconciliation.contested` keeps each model-decided spot with its
verdict (changed, kept base, uncertain) and a few words of left context, bounded at 2,000. The page
still lists only changed and uncertain spots. This makes ADR-034's second condition measurable on the
next real run.
7. **Segments carry the merged words.** The reconciled words replace the base's in the saved `segments`;
timing stays the base's. If the segments cannot be aligned to the merged text (a word wrapped across
a segment edge), they stay the base's and the result says so in the log. This replaces ADR-034
decision 6's "segments stay untouched".
**Alternatives rejected.** Server-sent events or a websocket: a long-lived connection through the
ingress, with buffering to get wrong, for a page that changes every few seconds at best; polling is what
the page already does. A percentage from the engine calls: there is no data behind it. A separate
progress table: progress belongs to the row and inherits its RLS. Logging only through the UI: an
operator has no page.
**Consequences.** Positive: running, quiet, lost and failed are distinguishable by the user and the
operator; the next blind check can answer ADR-034's second condition. Negative / watch: two more writes
per transition and one per 10 s per running job (one job at a time, so negligible); a heartbeat proves
the process is alive, not that an engine is making progress inside one request, which only the job
timeout (60 min) bounds. The stored spot list grows to a few hundred small objects per hour of audio.
**Validation.** A fake engine held open past the thresholds must change the page (quiet, lost); a log
capture on a synthetic transcript must contain none of its words; a screenshot review of the running,
quiet, lost, failed and done states against the existing pages (the visual claims tests cannot assert).
Revisit if the heartbeat write ever shows in the database's load, or if a longer recording cap makes
five-minute quiet detection too coarse.
---
## Rejected alternatives
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not