Compare commits
43
Commits
1a89eb3868
...
7029dd1d7f
@@ -121,6 +121,14 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes (
|
|||||||
Two-engine reconciliation (ADR-034, migration 019, off unless `TAPIR_RECONCILE_BASE_MODEL`):
|
Two-engine reconciliation (ADR-034, migration 019, off unless `TAPIR_RECONCILE_BASE_MODEL`):
|
||||||
`internal/reconcile` (word diff, disputes, a local-model adjudicator) and
|
`internal/reconcile` (word diff, disputes, a local-model adjudicator) and
|
||||||
`transcribe.Ensemble`; every model must be a local alias, enforced at start-up.
|
`transcribe.Ensemble`; every model must be a local alias, enforced at start-up.
|
||||||
|
Progress and logging (ADR-035, migration 020): `internal/progress.Tracker` is the ONE writer of a
|
||||||
|
job's phases (saved to the row for the page, logged by the same call); engines and the reconciler
|
||||||
|
report through `progress.From(ctx)`, so `uploads.Transcriber` is unchanged, and a transcriber may
|
||||||
|
implement `uploads.Planner` to declare its own phases. The page tells running / quiet / lost from
|
||||||
|
two timestamps (`alive_at` heartbeat vs `progress_at`) and never shows a percentage that does not
|
||||||
|
come from real counts. **Nothing logged may carry transcript text, words from it, or the spool path.**
|
||||||
|
`TAPIR_SHOT_DIR=… go test -run TestRenderUploadProgressStatesForReview ./internal/web` renders every
|
||||||
|
state for a screenshot review.
|
||||||
**Lint caveat:** `task check` does not fail on lint (#41: the Taskfile `|| echo` swallows
|
**Lint caveat:** `task check` does not fail on lint (#41: the Taskfile `|| echo` swallows
|
||||||
findings) — run `golangci-lint run ./...` yourself until that is fixed.
|
findings) — run `golangci-lint run ./...` yourself until that is fixed.
|
||||||
- `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch
|
- `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch
|
||||||
|
|||||||
+73
-1
@@ -1596,7 +1596,8 @@ on the slow numbers and is reopened: 31B scored 97% against 92%.
|
|||||||
timeout at a time. Both engines fail → ADR-033's fallback path.
|
timeout at a time. Both engines fail → ADR-033's fallback path.
|
||||||
|
|
||||||
6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed
|
6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed
|
||||||
segments, untouched: `content` is authoritative for chat, `segments` only for time. `endpoint`
|
segments, untouched (**amended by ADR-035: they now carry the merged words, timing stays the
|
||||||
|
base's**): `content` is authoritative for chat, `segments` only for time. `endpoint`
|
||||||
lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same
|
lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same
|
||||||
RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute
|
RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute
|
||||||
mark and both readings**, so the user can check them against the audio. The transcript page says in
|
mark and both readings**, so the user can check them against the audio. The transcript page says in
|
||||||
@@ -1659,6 +1660,77 @@ unchanged.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## ADR-035 — Show an upload's progress as real phases, log the same transitions, and keep every judged spot
|
||||||
|
|
||||||
|
**Status:** Proposed (2026-10-02). Decider: Mathias (pending). Amends ADR-034 decision 6 (segments).
|
||||||
|
Closes #42, #43 and #44.
|
||||||
|
|
||||||
|
**Context (measured, not assumed).** The first real run (a 63.6-minute recording, 21 min 49 s) exposed
|
||||||
|
three gaps. (1) The page said only `queued`/`transcribing`/`done`; during 22 minutes nothing told the
|
||||||
|
user whether the job was working, hung or dead, and after a crash the row would say `transcribing`
|
||||||
|
until the next start. (2) The server log had no line about the job between start-up and the final
|
||||||
|
line, so an operator saw it only from outside (spool files, iguana CPU). (3) Evaluating the result
|
||||||
|
showed that spots the model *kept* as the base are counted but not stored (so ADR-034's "no worse than
|
||||||
|
the better single engine" cannot be measured after the fact, and the audio is not retained), insertions
|
||||||
|
and deletions cannot be placed without context, and the saved `segments` still carry the base engine's
|
||||||
|
wording while only `content` is merged.
|
||||||
|
|
||||||
|
**Decision.**
|
||||||
|
|
||||||
|
1. **Phases are data on the row.** A new `progress` JSONB column (migration 020, same RLS-scoped table)
|
||||||
|
holds an ordered list of steps `{key, label, state, started_at, finished_at, done, total}` with state
|
||||||
|
`pending|running|done|failed|skipped`. The plan is declared up front so the page shows what is coming:
|
||||||
|
received, preparing audio, then the transcriber's own steps (two-engine: engine A, engine B,
|
||||||
|
comparing, deciding n of N; single: transcribing), then saving.
|
||||||
|
2. **One source for the page and the log.** An `internal/progress` Tracker is the only writer. Every
|
||||||
|
transition is saved to the row and logged by the same call, so the UI and the log cannot disagree.
|
||||||
|
Engines and the reconciler report through a context-carried `Reporter` (no-op when absent), so the
|
||||||
|
`Transcriber` interface does not change.
|
||||||
|
3. **Alive is not the same as progressing, and the page says which it knows.** A heartbeat goroutine,
|
||||||
|
independent of any engine call, stamps `alive_at` every 10 s; `progress_at` moves only on a real
|
||||||
|
transition or advance. The page derives: *running*; *quiet* (the deciding step has not advanced for
|
||||||
|
5 minutes, so the model is probably waiting or slow); *lost* (`alive_at` older than 45 s, so the
|
||||||
|
service stopped reporting and may have restarted or crashed). The two engine calls are single blocking
|
||||||
|
requests, so inside them the page shows elapsed time only: **no percentage is ever shown that does not
|
||||||
|
come from real counts.** Only the deciding step has a bar.
|
||||||
|
4. **The page.** A server-rendered step list in tapir's existing tokens and mono font, swapped in place
|
||||||
|
by the existing HTMX poll (every 2 s while unfinished, none once done or failed). Finished steps
|
||||||
|
collapse to one line with their duration; the running step has a CSS-only spinner that respects
|
||||||
|
`prefers-reduced-motion`; a failed step and a lost service are visibly different from running.
|
||||||
|
No new JavaScript, framework or stylesheet.
|
||||||
|
5. **Logs.** One INFO line per phase transition and engine call (upload id, endpoint, elapsed), the
|
||||||
|
reconcile counts, `adjudicated n/N` at most once a minute, a WARN when an engine soft-fails or the
|
||||||
|
breaker trips, and a "still in phase X" line every 5 minutes. **Ids, counts, durations, endpoints and
|
||||||
|
error kinds only: never transcript text, disputed words or the filename**, asserted by a test.
|
||||||
|
6. **Every judged spot is stored.** `reconciliation.contested` keeps each model-decided spot with its
|
||||||
|
verdict (changed, kept base, uncertain) and a few words of left context, bounded at 2,000. The page
|
||||||
|
still lists only changed and uncertain spots. This makes ADR-034's second condition measurable on the
|
||||||
|
next real run.
|
||||||
|
7. **Segments carry the merged words.** The reconciled words replace the base's in the saved `segments`;
|
||||||
|
timing stays the base's. If the segments cannot be aligned to the merged text (a word wrapped across
|
||||||
|
a segment edge), they stay the base's and the result says so in the log. This replaces ADR-034
|
||||||
|
decision 6's "segments stay untouched".
|
||||||
|
|
||||||
|
**Alternatives rejected.** Server-sent events or a websocket: a long-lived connection through the
|
||||||
|
ingress, with buffering to get wrong, for a page that changes every few seconds at best; polling is what
|
||||||
|
the page already does. A percentage from the engine calls: there is no data behind it. A separate
|
||||||
|
progress table: progress belongs to the row and inherits its RLS. Logging only through the UI: an
|
||||||
|
operator has no page.
|
||||||
|
|
||||||
|
**Consequences.** Positive: running, quiet, lost and failed are distinguishable by the user and the
|
||||||
|
operator; the next blind check can answer ADR-034's second condition. Negative / watch: two more writes
|
||||||
|
per transition and one per 10 s per running job (one job at a time, so negligible); a heartbeat proves
|
||||||
|
the process is alive, not that an engine is making progress inside one request, which only the job
|
||||||
|
timeout (60 min) bounds. The stored spot list grows to a few hundred small objects per hour of audio.
|
||||||
|
|
||||||
|
**Validation.** A fake engine held open past the thresholds must change the page (quiet, lost); a log
|
||||||
|
capture on a synthetic transcript must contain none of its words; a screenshot review of the running,
|
||||||
|
quiet, lost, failed and done states against the existing pages (the visual claims tests cannot assert).
|
||||||
|
Revisit if the heartbeat write ever shows in the database's load, or if a longer recording cap makes
|
||||||
|
five-minute quiet detection too coarse.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Rejected alternatives
|
## Rejected alternatives
|
||||||
|
|
||||||
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
|
Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package main
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"mime/multipart"
|
"mime/multipart"
|
||||||
@@ -21,6 +22,7 @@ import (
|
|||||||
|
|
||||||
"git.d-ma.be/mathias/tapir/internal/adapters/store"
|
"git.d-ma.be/mathias/tapir/internal/adapters/store"
|
||||||
"git.d-ma.be/mathias/tapir/internal/config"
|
"git.d-ma.be/mathias/tapir/internal/config"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
"git.d-ma.be/mathias/tapir/internal/web"
|
"git.d-ma.be/mathias/tapir/internal/web"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -156,6 +158,23 @@ func TestE2E_UploadTranscribeAndChat(t *testing.T) {
|
|||||||
assert.Equal(t, "done", status)
|
assert.Equal(t, "done", status)
|
||||||
assert.NotEmpty(t, endpoint)
|
assert.NotEmpty(t, endpoint)
|
||||||
assert.NotEmpty(t, content)
|
assert.NotEmpty(t, content)
|
||||||
|
|
||||||
|
// ADR-035: a finished job's phases are all closed, and it was reporting while it ran.
|
||||||
|
var progJSON string
|
||||||
|
var progressAt, aliveAt *time.Time
|
||||||
|
require.NoError(t, raw.QueryRow(ctx,
|
||||||
|
`SELECT COALESCE(progress::text, '[]'), progress_at, alive_at FROM uploaded_transcripts WHERE user_id = $1`, uid).
|
||||||
|
Scan(&progJSON, &progressAt, &aliveAt))
|
||||||
|
var steps []domain.ProgressStep
|
||||||
|
require.NoError(t, json.Unmarshal([]byte(progJSON), &steps))
|
||||||
|
require.NotEmpty(t, steps, "the job recorded no phases")
|
||||||
|
for _, st := range steps {
|
||||||
|
assert.Contains(t, []domain.StepState{domain.StepDone, domain.StepSkipped}, st.State,
|
||||||
|
"phase %q was left %q after the job finished", st.Key, st.State)
|
||||||
|
}
|
||||||
|
assert.Equal(t, "save", steps[len(steps)-1].Key)
|
||||||
|
assert.NotNil(t, progressAt, "progress_at must have been stamped")
|
||||||
|
t.Logf("phases: %d, heartbeat stamped: %v", len(steps), aliveAt != nil)
|
||||||
entries, err := os.ReadDir(cfg.UploadTmpDir)
|
entries, err := os.ReadDir(cfg.UploadTmpDir)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
assert.Empty(t, entries, "the audio must be deleted once transcribed")
|
assert.Empty(t, entries, "the audio must be deleted once transcribed")
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
@@ -37,6 +38,12 @@ func (f *fakeUploadsStore) CompleteUpload(context.Context, string, string, domai
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (f *fakeUploadsStore) SaveProgress(context.Context, string, string, []domain.ProgressStep, time.Time) error {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeUploadsStore) Heartbeat(context.Context, string, string) error { return nil }
|
||||||
|
|
||||||
func (f *fakeUploadsStore) FailInterruptedUploads(context.Context) (int, error) {
|
func (f *fakeUploadsStore) FailInterruptedUploads(context.Context) (int, error) {
|
||||||
f.sweeps++
|
f.sweeps++
|
||||||
return 2, f.sweepErr
|
return 2, f.sweepErr
|
||||||
|
|||||||
+5
-2
@@ -153,7 +153,10 @@ erDiagram
|
|||||||
float audio_seconds
|
float audio_seconds
|
||||||
text error_kind "typed: interrupted | transcription_failed — never a provider string"
|
text error_kind "typed: interrupted | transcription_failed — never a provider string"
|
||||||
bool had_video "the file carried a video track; only its audio was used (migration 018)"
|
bool had_video "the file carried a video track; only its audio was used (migration 018)"
|
||||||
jsonb reconciliation "NULL for a single engine; else status, exact counts and a bounded list of contested spots (migration 019, ADR-034)"
|
jsonb reconciliation "NULL for a single engine; else status, exact counts and a bounded list of every judged spot (migration 019, ADR-034; all spots stored, ADR-035)"
|
||||||
|
jsonb progress "the job's ordered phases: key, label, state, times, done/total; NULL until the job starts (migration 020, ADR-035)"
|
||||||
|
timestamptz progress_at "when a phase last really changed or advanced (migration 020)"
|
||||||
|
timestamptz alive_at "heartbeat from the running job, every ~10 s; not progress (migration 020)"
|
||||||
timestamptz created_at
|
timestamptz created_at
|
||||||
timestamptz finished_at
|
timestamptz finished_at
|
||||||
}
|
}
|
||||||
@@ -225,7 +228,7 @@ mechanism.
|
|||||||
grounds that it holds only public caption content, and a recorded meeting is the opposite.
|
grounds that it holds only public caption content, and a recorded meeting is the opposite.
|
||||||
`user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing →
|
`user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing →
|
||||||
done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK
|
done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK
|
||||||
constraints). `reconciliation` (migration 019, ADR-034) is NULL when one engine produced the text, so the page can never claim a second opinion that was not there; otherwise it holds the status (`reconciled` / `single_engine` / `skipped`), exact counts (differences, settled automatically, decided by the model, changed, uncertain, model errors) and up to 200 contested spots (each a position in seconds, both readings and which was used). The spots hold fragments of the private transcript, which is why they live in this RLS-scoped row. `had_video` (migration 018) records that the file carried a video track — a Teams meeting with the camera off is still a video file — so the page can say only its audio was used and the video was discarded. `endpoint` records which backend served it — `berget/...` means the audio left the
|
constraints). `reconciliation` (migration 019, ADR-034) is NULL when one engine produced the text, so the page can never claim a second opinion that was not there; otherwise it holds the status (`reconciled` / `single_engine` / `skipped`), exact counts (differences, settled automatically, decided by the model, changed, uncertain, model errors) and up to 2,000 judged spots (ADR-035: every one the model decided, kept or changed or unsure, each a position in seconds, both readings, which was used, and the few words before it; the page lists only the changed and uncertain ones). `progress`, `progress_at` and `alive_at` (migration 020, ADR-035) let the page show the job's phases and tell a running job from a quiet one from a lost service: `alive_at` is a heartbeat from a goroutine independent of any engine call, `progress_at` moves only when work really moved; the saved `segments` carry the reconciled words with the base engine's timing. The spots hold fragments of the private transcript, which is why they live in this RLS-scoped row. `had_video` (migration 018) records that the file carried a video track — a Teams meeting with the camera off is still a video file — so the page can say only its audio was used and the video was discarded. `endpoint` records which backend served it — `berget/...` means the audio left the
|
||||||
homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90
|
homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90
|
||||||
minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion.
|
minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion.
|
||||||
- **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the
|
- **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the
|
||||||
|
|||||||
@@ -156,3 +156,61 @@ Feature: Upload an audio recording, get a private Swedish transcript, discuss it
|
|||||||
Scenario: A single-engine transcript claims no second opinion
|
Scenario: A single-engine transcript claims no second opinion
|
||||||
When only one engine was configured
|
When only one engine was configured
|
||||||
Then the page does not mention reconciliation
|
Then the page does not mention reconciliation
|
||||||
|
|
||||||
|
# ADR-035: progress, the log, and the judged spots
|
||||||
|
|
||||||
|
Scenario: I can see which phase my upload is in
|
||||||
|
Given a recording is being transcribed
|
||||||
|
When I open its page
|
||||||
|
Then it lists every phase in order, the finished ones with how long each took
|
||||||
|
And the running phase is marked, with how long it has run
|
||||||
|
And the page updates in place every two seconds until the job ends
|
||||||
|
|
||||||
|
Scenario: A bar appears only where the count is real
|
||||||
|
When the deciding phase has decided 142 of 282 differences
|
||||||
|
Then the page shows a bar with "142 of 282"
|
||||||
|
And an engine call, which is one request with nothing to count, never gets a bar or a percentage
|
||||||
|
|
||||||
|
Scenario: A slow engine call is not mistaken for a dead service
|
||||||
|
When an engine call has run for 25 minutes and the service is still reporting
|
||||||
|
Then the page shows the elapsed time and says nothing is wrong
|
||||||
|
|
||||||
|
Scenario: A service that stopped reporting is called lost
|
||||||
|
When nothing has been heard from the service for longer than the heartbeat allows
|
||||||
|
Then the page says the service stopped reporting and may have restarted or crashed
|
||||||
|
And the running phase carries a warning mark and no spinner
|
||||||
|
And the page keeps checking
|
||||||
|
|
||||||
|
Scenario: A model that stops advancing is called quiet
|
||||||
|
When the deciding phase has not advanced for five minutes and the service is still reporting
|
||||||
|
Then the page says nothing new has happened for that long and that the service is still running
|
||||||
|
And it does not say the service is lost
|
||||||
|
|
||||||
|
Scenario: A failed job shows where it stopped
|
||||||
|
When a job fails
|
||||||
|
Then the failed phase is marked and what never started stays pending
|
||||||
|
And the page stops checking
|
||||||
|
|
||||||
|
Scenario: A finished job keeps its record of what was done
|
||||||
|
When a job is done
|
||||||
|
Then the page keeps a collapsed list of the phases with how long each took
|
||||||
|
|
||||||
|
Scenario: The heartbeat runs on its own while an engine call is stuck
|
||||||
|
When an engine call does not return
|
||||||
|
Then the row is still stamped alive every few seconds
|
||||||
|
And no heartbeat outlives its job
|
||||||
|
|
||||||
|
Scenario: The log names every phase and never the transcript
|
||||||
|
When a job runs
|
||||||
|
Then the log has a line per phase in the order it ran, each naming the upload
|
||||||
|
And no line contains transcript text or the spool path
|
||||||
|
|
||||||
|
Scenario: Every judged spot is stored but only the ones to check are listed
|
||||||
|
When the model has changed, kept and been unsure about spots
|
||||||
|
Then every one is stored with its verdict and the words before it
|
||||||
|
And the page lists only the changed and uncertain ones, with the words before each
|
||||||
|
|
||||||
|
Scenario: The saved segments carry the merged words
|
||||||
|
When the model changes a word
|
||||||
|
Then the segment containing it carries the reconciled word
|
||||||
|
And the segment's timing is unchanged
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
ALTER TABLE uploaded_transcripts
|
||||||
|
DROP COLUMN alive_at,
|
||||||
|
DROP COLUMN progress_at,
|
||||||
|
DROP COLUMN progress;
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
-- Migration 020: where an upload job is, and whether the service is still reporting (ADR-035).
|
||||||
|
--
|
||||||
|
-- progress the job's ordered phases as JSON (key, label, state, times, done/total).
|
||||||
|
-- progress_at when a phase last REALLY changed or advanced.
|
||||||
|
-- alive_at a heartbeat stamped every few seconds by a goroutine independent of any
|
||||||
|
-- engine call. The two differ on purpose: alive says the service is still
|
||||||
|
-- reporting, progress says work is moving, and a page that cannot tell a
|
||||||
|
-- crashed service from a slow model is the failure this exists to end.
|
||||||
|
--
|
||||||
|
-- All three are NULL until the job starts. They sit on the already-FORCE'd, RLS-scoped
|
||||||
|
-- uploaded_transcripts, so the exact-set isolation test is unaffected.
|
||||||
|
ALTER TABLE uploaded_transcripts
|
||||||
|
ADD COLUMN progress JSONB,
|
||||||
|
ADD COLUMN progress_at TIMESTAMPTZ,
|
||||||
|
ADD COLUMN alive_at TIMESTAMPTZ;
|
||||||
@@ -68,6 +68,26 @@ func (s *Store) CompleteUpload(ctx context.Context, userID, id string, t domain.
|
|||||||
t.Text, segs, t.Endpoint, t.AudioSeconds, rec)
|
t.Text, segs, t.Endpoint, t.AudioSeconds, rec)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SaveProgress records the job's phases and when something last really changed
|
||||||
|
// (ADR-035). It replaces the previous snapshot: the Tracker holds the whole plan.
|
||||||
|
func (s *Store) SaveProgress(ctx context.Context, userID, id string, steps []domain.ProgressStep, at time.Time) error {
|
||||||
|
b, err := json.Marshal(steps)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("store: save progress: marshal: %w", err)
|
||||||
|
}
|
||||||
|
return s.updateUpload(ctx, userID, id, "save progress",
|
||||||
|
`UPDATE uploaded_transcripts SET progress = $3, progress_at = $4
|
||||||
|
WHERE id = $1 AND user_id = $2`, b, at)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Heartbeat stamps the upload as alive now. It does not touch progress_at: alive says
|
||||||
|
// the service is still reporting, not that work is moving (ADR-035).
|
||||||
|
func (s *Store) Heartbeat(ctx context.Context, userID, id string) error {
|
||||||
|
return s.updateUpload(ctx, userID, id, "heartbeat",
|
||||||
|
`UPDATE uploaded_transcripts SET alive_at = NOW()
|
||||||
|
WHERE id = $1 AND user_id = $2`)
|
||||||
|
}
|
||||||
|
|
||||||
// FailUpload moves an upload to failed with a typed reason.
|
// FailUpload moves an upload to failed with a typed reason.
|
||||||
func (s *Store) FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error {
|
func (s *Store) FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error {
|
||||||
return s.updateUpload(ctx, userID, id, "fail upload",
|
return s.updateUpload(ctx, userID, id, "fail upload",
|
||||||
@@ -165,19 +185,21 @@ func (s *Store) FailInterruptedUploads(ctx context.Context) (int, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const uploadSelect = `SELECT id, user_id, title, status, content, segments, endpoint,
|
const uploadSelect = `SELECT id, user_id, title, status, content, segments, endpoint,
|
||||||
audio_seconds, error_kind, had_video, reconciliation, created_at, finished_at FROM uploaded_transcripts`
|
audio_seconds, error_kind, had_video, reconciliation, progress, progress_at, alive_at,
|
||||||
|
created_at, finished_at FROM uploaded_transcripts`
|
||||||
|
|
||||||
// scanUpload scans one uploadSelect row. withText controls whether the transcript
|
// scanUpload scans one uploadSelect row. withText controls whether the transcript
|
||||||
// body and segments are kept (GetUpload) or dropped (ListUploads).
|
// body and segments are kept (GetUpload) or dropped (ListUploads).
|
||||||
func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error {
|
func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error {
|
||||||
var (
|
var (
|
||||||
status, errKind, endpoint, content *string
|
status, errKind, endpoint, content *string
|
||||||
segments, reconciliation []byte
|
segments, reconciliation, progress []byte
|
||||||
seconds *float64
|
seconds *float64
|
||||||
finished *time.Time
|
finished, progressAt, aliveAt *time.Time
|
||||||
)
|
)
|
||||||
if err := row.Scan(&u.ID, &u.UserID, &u.Title, &status, &content, &segments, &endpoint,
|
if err := row.Scan(&u.ID, &u.UserID, &u.Title, &status, &content, &segments, &endpoint,
|
||||||
&seconds, &errKind, &u.HadVideo, &reconciliation, &u.CreatedAt, &finished); err != nil {
|
&seconds, &errKind, &u.HadVideo, &reconciliation, &progress, &progressAt, &aliveAt,
|
||||||
|
&u.CreatedAt, &finished); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
u.Status = domain.UploadStatus(deref(status))
|
u.Status = domain.UploadStatus(deref(status))
|
||||||
@@ -192,6 +214,17 @@ func scanUpload(row pgx.Row, u *domain.UploadedTranscript, withText bool) error
|
|||||||
if !withText {
|
if !withText {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
if len(progress) > 0 {
|
||||||
|
if err := json.Unmarshal(progress, &u.Progress); err != nil {
|
||||||
|
return fmt.Errorf("unmarshal progress: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if progressAt != nil {
|
||||||
|
u.ProgressAt = *progressAt
|
||||||
|
}
|
||||||
|
if aliveAt != nil {
|
||||||
|
u.AliveAt = *aliveAt
|
||||||
|
}
|
||||||
if len(reconciliation) > 0 {
|
if len(reconciliation) > 0 {
|
||||||
var r domain.Reconciliation
|
var r domain.Reconciliation
|
||||||
if err := json.Unmarshal(reconciliation, &r); err != nil {
|
if err := json.Unmarshal(reconciliation, &r); err != nil {
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
package store_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/adapters/store"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Progress and heartbeat of an upload job (ADR-035, migration 020). Like the other
|
||||||
|
// upload store tests these run as the superuser, so the cross-user cases prove the
|
||||||
|
// statement's own user_id predicate; the database-enforced isolation is covered by the
|
||||||
|
// RLS tests, and migration 020 adds columns to the already-FORCE'd table, not a table.
|
||||||
|
|
||||||
|
func startedUpload(t *testing.T, s *store.Store) string {
|
||||||
|
t.Helper()
|
||||||
|
seedUsers(t, userA, userB)
|
||||||
|
id, err := s.CreateUpload(context.Background(), userA, "m", false)
|
||||||
|
require.NoError(t, err)
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_AFreshUploadHasNoProgressAndNoHeartbeat(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
|
||||||
|
got, err := s.GetUpload(context.Background(), userA, id)
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Empty(t, got.Progress)
|
||||||
|
assert.True(t, got.ProgressAt.IsZero())
|
||||||
|
assert.True(t, got.AliveAt.IsZero())
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_StepsAndTheirTimesRoundTripInOrder(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC)
|
||||||
|
steps := []domain.ProgressStep{
|
||||||
|
{Key: "audio", Label: "Preparing audio", State: domain.StepDone, StartedAt: at, FinishedAt: at.Add(40 * time.Second)},
|
||||||
|
{Key: "decide", Label: "Deciding", State: domain.StepRunning, StartedAt: at.Add(time.Minute), Done: 12, Total: 282},
|
||||||
|
{Key: "save", Label: "Saving", State: domain.StepPending},
|
||||||
|
}
|
||||||
|
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, steps, at.Add(2*time.Minute)))
|
||||||
|
|
||||||
|
got, err := s.GetUpload(ctx, userA, id)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, got.Progress, 3)
|
||||||
|
assert.Equal(t, []string{"audio", "decide", "save"}, []string{got.Progress[0].Key, got.Progress[1].Key, got.Progress[2].Key})
|
||||||
|
assert.Equal(t, domain.StepRunning, got.Progress[1].State)
|
||||||
|
assert.Equal(t, 12, got.Progress[1].Done)
|
||||||
|
assert.Equal(t, 282, got.Progress[1].Total)
|
||||||
|
assert.True(t, got.Progress[0].FinishedAt.Equal(at.Add(40*time.Second)), "a step's finish time must survive the round trip")
|
||||||
|
assert.True(t, got.Progress[2].StartedAt.IsZero(), "a pending step has no start time, not a zero date")
|
||||||
|
assert.True(t, got.ProgressAt.Equal(at.Add(2*time.Minute)))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_AHeartbeatMovesAliveAtAndNotProgressAt(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC)
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "audio", Label: "a", State: domain.StepRunning}}, at))
|
||||||
|
before := time.Now().Add(-time.Second)
|
||||||
|
|
||||||
|
require.NoError(t, s.Heartbeat(ctx, userA, id))
|
||||||
|
|
||||||
|
got, err := s.GetUpload(ctx, userA, id)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.True(t, got.AliveAt.After(before), "the heartbeat must stamp the row with now")
|
||||||
|
assert.True(t, got.ProgressAt.Equal(at), "alive is not progress: a heartbeat must not move progress_at")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ASecondSaveReplacesTheFirst(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
at := time.Date(2026, 10, 2, 8, 30, 0, 0, time.UTC)
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepRunning}}, at))
|
||||||
|
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepDone}, {Key: "b", Label: "B", State: domain.StepRunning}}, at.Add(time.Minute)))
|
||||||
|
|
||||||
|
got, err := s.GetUpload(ctx, userA, id)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, got.Progress, 2)
|
||||||
|
assert.Equal(t, domain.StepDone, got.Progress[0].State)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ByAnotherUserChangesNothing(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
errSave := s.SaveProgress(ctx, userB, id, []domain.ProgressStep{{Key: "x", Label: "X", State: domain.StepDone}}, time.Now())
|
||||||
|
errBeat := s.Heartbeat(ctx, userB, id)
|
||||||
|
|
||||||
|
assert.ErrorIs(t, errSave, store.ErrUploadNotFound)
|
||||||
|
assert.ErrorIs(t, errBeat, store.ErrUploadNotFound)
|
||||||
|
got, err := s.GetUpload(ctx, userA, id)
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.Empty(t, got.Progress, "B's write must not have reached A's upload")
|
||||||
|
assert.True(t, got.AliveAt.IsZero())
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ListDoesNotDragItAlong(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepRunning}}, time.Now()))
|
||||||
|
|
||||||
|
rows, err := s.ListUploads(ctx, userA)
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, rows, 1)
|
||||||
|
assert.Empty(t, rows[0].Progress, "the list view needs a status, not every row's phases")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_SurvivesCompletion(t *testing.T) {
|
||||||
|
s := newStore(t)
|
||||||
|
id := startedUpload(t, s)
|
||||||
|
ctx := context.Background()
|
||||||
|
require.NoError(t, s.StartTranscribing(ctx, userA, id))
|
||||||
|
require.NoError(t, s.SaveProgress(ctx, userA, id, []domain.ProgressStep{{Key: "a", Label: "A", State: domain.StepDone}}, time.Now()))
|
||||||
|
|
||||||
|
require.NoError(t, s.CompleteUpload(ctx, userA, id, domain.Transcription{Text: "x", Endpoint: "e"}))
|
||||||
|
|
||||||
|
got, err := s.GetUpload(ctx, userA, id)
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, got.Progress, 1, "a finished upload keeps its record of what was done, and how long each step took")
|
||||||
|
}
|
||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
|
|
||||||
"git.d-ma.be/mathias/tapir/internal/domain"
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -30,13 +31,30 @@ func NewEnsemble(base, alt, fallback Backend, adj reconcile.Adjudicator, opts re
|
|||||||
return &Ensemble{base: base, alt: alt, fallback: fallback, adj: adj, opts: opts}
|
return &Ensemble{base: base, alt: alt, fallback: fallback, adj: adj, opts: opts}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Transcribe implements uploads.Transcriber.
|
// Plan implements uploads.Planner: the phases this ensemble reports, so the page can
|
||||||
|
// show them before they run. The fallback is a phase only when one is configured.
|
||||||
|
func (e *Ensemble) Plan() []progress.Planned {
|
||||||
|
plan := []progress.Planned{
|
||||||
|
{Key: "engine_a", Label: "Transcribing with " + e.base.Name()},
|
||||||
|
{Key: "engine_b", Label: "Transcribing with " + e.alt.Name()},
|
||||||
|
{Key: "compare", Label: "Comparing the two transcripts"},
|
||||||
|
{Key: "decide", Label: "Deciding the differences"},
|
||||||
|
}
|
||||||
|
if e.fallback != nil {
|
||||||
|
plan = append(plan, progress.Planned{Key: "fallback", Label: "Cloud fallback (" + e.fallback.Name() + ")"})
|
||||||
|
}
|
||||||
|
return plan
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transcribe implements uploads.Transcriber. Each phase is reported through the
|
||||||
|
// progress.Reporter in ctx (a no-op when nobody listens).
|
||||||
func (e *Ensemble) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) {
|
func (e *Ensemble) Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error) {
|
||||||
b, berr := e.base.Transcribe(ctx, wavPath)
|
rep := progress.From(ctx)
|
||||||
|
b, berr := e.run(ctx, rep, "engine_a", e.base, wavPath)
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
return domain.Transcription{}, err
|
return domain.Transcription{}, err
|
||||||
}
|
}
|
||||||
a, aerr := e.alt.Transcribe(ctx, wavPath)
|
a, aerr := e.run(ctx, rep, "engine_b", e.alt, wavPath)
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
return domain.Transcription{}, err
|
return domain.Transcription{}, err
|
||||||
}
|
}
|
||||||
@@ -48,6 +66,10 @@ func (e *Ensemble) Transcribe(ctx context.Context, wavPath string) (domain.Trans
|
|||||||
return domain.Transcription{}, err
|
return domain.Transcription{}, err
|
||||||
}
|
}
|
||||||
out.Endpoint = fmt.Sprintf("%s + %s, reconciled by %s", e.base.Name(), e.alt.Name(), e.opts.Reconciler)
|
out.Endpoint = fmt.Sprintf("%s + %s, reconciled by %s", e.base.Name(), e.alt.Name(), e.opts.Reconciler)
|
||||||
|
if rc := out.Reconciliation; rc != nil {
|
||||||
|
rep.Note("reconciled", "status", string(rc.Status), "disputes", rc.Disputes, "auto_resolved", rc.AutoResolved,
|
||||||
|
"adjudicated", rc.Adjudicated, "changed", rc.Changed, "unsure", rc.Unsure, "errors", rc.Errors)
|
||||||
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
case berr == nil:
|
case berr == nil:
|
||||||
return e.single(b, e.base.Name(), fmt.Sprintf("%s failed: %v", e.alt.Name(), aerr)), nil
|
return e.single(b, e.base.Name(), fmt.Sprintf("%s failed: %v", e.alt.Name(), aerr)), nil
|
||||||
@@ -59,13 +81,27 @@ func (e *Ensemble) Transcribe(ctx context.Context, wavPath string) (domain.Trans
|
|||||||
if e.fallback == nil {
|
if e.fallback == nil {
|
||||||
return domain.Transcription{}, both
|
return domain.Transcription{}, both
|
||||||
}
|
}
|
||||||
f, ferr := e.fallback.Transcribe(ctx, wavPath)
|
f, ferr := e.run(ctx, rep, "fallback", e.fallback, wavPath)
|
||||||
if ferr != nil {
|
if ferr != nil {
|
||||||
return domain.Transcription{}, errors.Join(both, fmt.Errorf("%s: %w", e.fallback.Name(), ferr))
|
return domain.Transcription{}, errors.Join(both, fmt.Errorf("%s: %w", e.fallback.Name(), ferr))
|
||||||
}
|
}
|
||||||
return e.single(f, e.fallback.Name(), "both local engines failed"), nil
|
return e.single(f, e.fallback.Name(), "both local engines failed"), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// run is one engine call, reported as a phase. A job that was stopped is not a broken
|
||||||
|
// engine, so a cancelled context is never reported as a failed phase.
|
||||||
|
func (e *Ensemble) run(ctx context.Context, rep progress.Reporter, key string, be Backend, wav string) (domain.Transcription, error) {
|
||||||
|
rep.Begin(key, 0)
|
||||||
|
t, err := be.Transcribe(ctx, wav)
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
rep.Done(key)
|
||||||
|
case ctx.Err() == nil:
|
||||||
|
rep.Fail(key, err)
|
||||||
|
}
|
||||||
|
return t, err
|
||||||
|
}
|
||||||
|
|
||||||
// single is a transcript from one engine only: nothing to reconcile, and the result
|
// single is a transcript from one engine only: nothing to reconcile, and the result
|
||||||
// says why, because a reader should know the second opinion was missing.
|
// says why, because a reader should know the second opinion was missing.
|
||||||
func (e *Ensemble) single(t domain.Transcription, engine, reason string) domain.Transcription {
|
func (e *Ensemble) single(t domain.Transcription, engine, reason string) domain.Transcription {
|
||||||
|
|||||||
@@ -0,0 +1,162 @@
|
|||||||
|
package transcribe_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/adapters/transcribe"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
||||||
|
)
|
||||||
|
|
||||||
|
// What the Ensemble reports to the page and the log (ADR-035). The reporter here is a
|
||||||
|
// recording double carried in the context, exactly as the worker's Tracker is.
|
||||||
|
|
||||||
|
type recReporter struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
events []string
|
||||||
|
notes []map[string]any
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *recReporter) add(e string) {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
r.events = append(r.events, e)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *recReporter) Begin(key string, _ int) { r.add("begin:" + key) }
|
||||||
|
func (r *recReporter) Advance(key string, _ int) { r.add("advance:" + key) }
|
||||||
|
func (r *recReporter) Done(key string) { r.add("done:" + key) }
|
||||||
|
func (r *recReporter) Fail(key string, _ error) { r.add("fail:" + key) }
|
||||||
|
func (r *recReporter) Note(msg string, kv ...any) {
|
||||||
|
m := map[string]any{"msg": msg}
|
||||||
|
for i := 0; i+1 < len(kv); i += 2 {
|
||||||
|
m[kv[i].(string)] = kv[i+1]
|
||||||
|
}
|
||||||
|
r.mu.Lock()
|
||||||
|
r.notes = append(r.notes, m)
|
||||||
|
r.mu.Unlock()
|
||||||
|
r.add("note:" + msg)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *recReporter) joined() string {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
return strings.Join(r.events, ",")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsemblePlan_DeclaresBothEnginesAndTheReconcileStepsByName(t *testing.T) {
|
||||||
|
r := newEnsemble("a", "a", nil, nil, false, &verdictAdj{})
|
||||||
|
|
||||||
|
plan := r.e.Plan()
|
||||||
|
|
||||||
|
require.Len(t, plan, 4)
|
||||||
|
assert.Equal(t, []string{"engine_a", "engine_b", "compare", "decide"}, []string{plan[0].Key, plan[1].Key, plan[2].Key, plan[3].Key})
|
||||||
|
assert.Contains(t, plan[0].Label, "iguana/pianissimo", "the page says which engine is running")
|
||||||
|
assert.Contains(t, plan[1].Label, "iguana/kb-whisper")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsemblePlan_TheCloudFallbackIsAStepOnlyWhenOneIsConfigured(t *testing.T) {
|
||||||
|
with := newEnsemble("a", "a", nil, nil, true, &verdictAdj{}).e.Plan()
|
||||||
|
without := newEnsemble("a", "a", nil, nil, false, &verdictAdj{}).e.Plan()
|
||||||
|
|
||||||
|
assert.Equal(t, "fallback", with[len(with)-1].Key)
|
||||||
|
assert.Len(t, without, 4, "no fallback configured: the page must not promise one")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsembleProgress_BothEnginesAreReportedInTheOrderTheyRun(t *testing.T) {
|
||||||
|
rep := &recReporter{}
|
||||||
|
r := newEnsemble("Totalt 1015 kronor idag.", "Totalt 1050 kronor idag.", nil, nil, false, &verdictAdj{v: reconcile.VerdictAlt})
|
||||||
|
|
||||||
|
_, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav")
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
got := rep.joined()
|
||||||
|
a, aDone := strings.Index(got, "begin:engine_a"), strings.Index(got, "done:engine_a")
|
||||||
|
b, bDone := strings.Index(got, "begin:engine_b"), strings.Index(got, "done:engine_b")
|
||||||
|
require.True(t, a >= 0 && aDone > a && b > aDone && bDone > b, "events out of order: %s", got)
|
||||||
|
assert.NotContains(t, got, "fail:")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsembleProgress_AnEngineThatFailsIsShownFailedAndTheOtherStillRuns(t *testing.T) {
|
||||||
|
rep := &recReporter{}
|
||||||
|
r := newEnsemble("text", "text", errors.New("pianissimo down"), nil, false, &verdictAdj{})
|
||||||
|
|
||||||
|
got, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav")
|
||||||
|
|
||||||
|
require.NoError(t, err, "one engine down is a degraded result, not a failed job")
|
||||||
|
assert.Equal(t, domain.ReconcileSingle, got.Reconciliation.Status)
|
||||||
|
ev := rep.joined()
|
||||||
|
assert.Contains(t, ev, "fail:engine_a")
|
||||||
|
assert.Contains(t, ev, "done:engine_b")
|
||||||
|
assert.NotContains(t, ev, "begin:compare", "nothing to compare with one engine")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsembleProgress_TheFallbackIsReportedOnlyWhenBothEnginesFailed(t *testing.T) {
|
||||||
|
rep := &recReporter{}
|
||||||
|
r := newEnsemble("t", "t", errors.New("a down"), errors.New("b down"), true, &verdictAdj{})
|
||||||
|
|
||||||
|
_, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav")
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
ev := rep.joined()
|
||||||
|
assert.Contains(t, ev, "fail:engine_a")
|
||||||
|
assert.Contains(t, ev, "fail:engine_b")
|
||||||
|
assert.Contains(t, ev, "begin:fallback")
|
||||||
|
assert.Contains(t, ev, "done:fallback")
|
||||||
|
|
||||||
|
rep2 := &recReporter{}
|
||||||
|
ok := newEnsemble("x", "x", nil, nil, true, &verdictAdj{})
|
||||||
|
_, err = ok.e.Transcribe(progress.With(context.Background(), rep2), "/a.wav")
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.NotContains(t, rep2.joined(), "fallback", "a healthy run never touches the cloud step")
|
||||||
|
}
|
||||||
|
|
||||||
|
// cancellingBackend ends the job in the middle of its own call, like a shutdown.
|
||||||
|
type cancellingBackend struct {
|
||||||
|
name string
|
||||||
|
cancel context.CancelFunc
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c cancellingBackend) Name() string { return c.name }
|
||||||
|
func (c cancellingBackend) Transcribe(ctx context.Context, _ string) (domain.Transcription, error) {
|
||||||
|
c.cancel()
|
||||||
|
return domain.Transcription{}, ctx.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsembleProgress_AShutdownMidEngineIsNotReportedAsAFailedEngine(t *testing.T) {
|
||||||
|
rep := &recReporter{}
|
||||||
|
ctx, cancel := context.WithCancel(progress.With(context.Background(), rep))
|
||||||
|
calls := &[]string{}
|
||||||
|
alt := &orderedBackend{name: "iguana/kb-whisper", text: "x", calls: calls}
|
||||||
|
e := transcribe.NewEnsemble(cancellingBackend{name: "iguana/pianissimo", cancel: cancel}, alt, nil, &verdictAdj{}, ensembleOpts)
|
||||||
|
|
||||||
|
_, err := e.Transcribe(ctx, "/a.wav")
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, context.Canceled)
|
||||||
|
assert.NotContains(t, rep.joined(), "fail:", "a stopped job is not a broken engine: the page must not blame one")
|
||||||
|
assert.Empty(t, *calls, "and the second engine never starts")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsembleProgress_TheReconcileCountsAreNotedForTheOperator(t *testing.T) {
|
||||||
|
rep := &recReporter{}
|
||||||
|
r := newEnsemble("Totalt 1015 kronor idag.", "Totalt 1050 kronor idag.", nil, nil, false, &verdictAdj{v: reconcile.VerdictAlt})
|
||||||
|
|
||||||
|
_, err := r.e.Transcribe(progress.With(context.Background(), rep), "/a.wav")
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
require.Len(t, rep.notes, 1)
|
||||||
|
n := rep.notes[0]
|
||||||
|
assert.Equal(t, "reconciled", n["msg"])
|
||||||
|
assert.Equal(t, 1, n["disputes"])
|
||||||
|
assert.Equal(t, 1, n["changed"])
|
||||||
|
assert.Equal(t, 0, n["unsure"])
|
||||||
|
assert.Equal(t, 0, n["errors"])
|
||||||
|
}
|
||||||
@@ -147,14 +147,18 @@ const (
|
|||||||
ReconcileSkipped ReconcileStatus = "skipped"
|
ReconcileSkipped ReconcileStatus = "skipped"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ContestedSpot is a place the engines disagreed that a reader may want to check
|
// ContestedSpot is a place the engines disagreed that the model judged: it changed it,
|
||||||
// against the audio: one the model changed, or one it was unsure about.
|
// kept the base's reading, or could not decide. Every judged spot is stored, not only
|
||||||
|
// the ones a reader should check, because the audio is not retained and the kept ones
|
||||||
|
// are what a later evaluation needs (ADR-035). The page lists only the changed and
|
||||||
|
// uncertain ones.
|
||||||
type ContestedSpot struct {
|
type ContestedSpot struct {
|
||||||
Seconds float64 `json:"seconds"` // roughly where in the recording
|
Seconds float64 `json:"seconds"` // roughly where in the recording
|
||||||
Base string `json:"base"` // what the base engine heard
|
Base string `json:"base"` // what the base engine heard
|
||||||
Alt string `json:"alt"` // what the other engine heard
|
Alt string `json:"alt"` // what the other engine heard
|
||||||
Used string `json:"used"` // "base" or "alt": the reading now in the text
|
Used string `json:"used"` // "base" or "alt": the reading now in the text
|
||||||
Unsure bool `json:"unsure"` // the model could not decide; the base was kept
|
Unsure bool `json:"unsure"` // the model could not decide; the base was kept
|
||||||
|
Before string `json:"before,omitempty"` // the few words just before the spot, to find it by
|
||||||
}
|
}
|
||||||
|
|
||||||
// Reconciliation records how a merged transcript came about, so the page can say so
|
// Reconciliation records how a merged transcript came about, so the page can say so
|
||||||
@@ -175,6 +179,32 @@ type Reconciliation struct {
|
|||||||
Contested []ContestedSpot `json:"contested,omitempty"`
|
Contested []ContestedSpot `json:"contested,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// StepState is where one phase of an upload job stands (ADR-035).
|
||||||
|
type StepState string
|
||||||
|
|
||||||
|
const (
|
||||||
|
StepPending StepState = "pending"
|
||||||
|
StepRunning StepState = "running"
|
||||||
|
StepDone StepState = "done"
|
||||||
|
StepFailed StepState = "failed"
|
||||||
|
// StepSkipped is a step that never ran because the job finished without it, for
|
||||||
|
// example the second engine's step when the first engine's text was used alone.
|
||||||
|
StepSkipped StepState = "skipped"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ProgressStep is one named phase of an upload job. Total is zero for a phase with no
|
||||||
|
// count; only a phase whose work is really countable carries one (ADR-035: no
|
||||||
|
// percentage that does not come from real counts).
|
||||||
|
type ProgressStep struct {
|
||||||
|
Key string `json:"key"`
|
||||||
|
Label string `json:"label"`
|
||||||
|
State StepState `json:"state"`
|
||||||
|
StartedAt time.Time `json:"started_at,omitzero"`
|
||||||
|
FinishedAt time.Time `json:"finished_at,omitzero"`
|
||||||
|
Done int `json:"done,omitempty"`
|
||||||
|
Total int `json:"total,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
// UploadStatus is the state of an uploaded-audio transcription job.
|
// UploadStatus is the state of an uploaded-audio transcription job.
|
||||||
type UploadStatus string
|
type UploadStatus string
|
||||||
|
|
||||||
@@ -218,6 +248,13 @@ type UploadedTranscript struct {
|
|||||||
// Reconciliation is how a merged transcript came about, or nil for a single
|
// Reconciliation is how a merged transcript came about, or nil for a single
|
||||||
// engine (ADR-034). Set by Get, not by List.
|
// engine (ADR-034). Set by Get, not by List.
|
||||||
Reconciliation *Reconciliation
|
Reconciliation *Reconciliation
|
||||||
CreatedAt time.Time
|
// Progress is the job's phases, in order. Set by Get, not by List (ADR-035).
|
||||||
FinishedAt time.Time // zero until the job reaches done or failed
|
Progress []ProgressStep
|
||||||
|
// ProgressAt is when a phase last really changed or advanced; AliveAt is the
|
||||||
|
// heartbeat. They differ on purpose: alive says the service is still reporting,
|
||||||
|
// progress says work is moving (ADR-035). Both are zero until the job starts.
|
||||||
|
ProgressAt time.Time
|
||||||
|
AliveAt time.Time
|
||||||
|
CreatedAt time.Time
|
||||||
|
FinishedAt time.Time // zero until the job reaches done or failed
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,269 @@
|
|||||||
|
// Package progress tracks the phases of one upload job (ADR-035). A Tracker is the
|
||||||
|
// only writer of a job's progress: every transition is saved to the job's row and
|
||||||
|
// logged by the same call, so what the page shows and what the log says cannot
|
||||||
|
// disagree. The engines and the reconciler report through a Reporter carried in the
|
||||||
|
// context, so the Transcriber interface stays what it was.
|
||||||
|
package progress
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Planned is one phase of a job, declared up front so the page can show what is coming.
|
||||||
|
type Planned struct {
|
||||||
|
Key, Label string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reporter is what a long-running component reports to. Begin starts a phase (total
|
||||||
|
// is the number of countable units, 0 when there is no honest count), Advance says how
|
||||||
|
// many are done, Done closes the phase. Fail closes a phase that did not work while
|
||||||
|
// the job goes on (an engine that was down, the other one's text used instead) and
|
||||||
|
// says why in the log. Note is a log line for the operator that costs no write.
|
||||||
|
type Reporter interface {
|
||||||
|
Begin(key string, total int)
|
||||||
|
Advance(key string, done int)
|
||||||
|
Done(key string)
|
||||||
|
Fail(key string, err error)
|
||||||
|
Note(msg string, kv ...any)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sink is where a job's progress is persisted. Both calls are best effort: progress
|
||||||
|
// is for people, so a failed write is logged and never stops the job.
|
||||||
|
type Sink interface {
|
||||||
|
SaveProgress(ctx context.Context, steps []domain.ProgressStep, at time.Time) error
|
||||||
|
Beat(ctx context.Context) error
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
defaultMinSave = 2 * time.Second
|
||||||
|
logAdvanceEach = time.Minute
|
||||||
|
)
|
||||||
|
|
||||||
|
// Option tunes a Tracker.
|
||||||
|
type Option func(*Tracker)
|
||||||
|
|
||||||
|
// WithClock fixes the clock, for tests.
|
||||||
|
func WithClock(now func() time.Time) Option { return func(t *Tracker) { t.now = now } }
|
||||||
|
|
||||||
|
// WithMinSave sets the least time between two saves caused by Advance alone. Begin,
|
||||||
|
// Done, Fail and Finish always save.
|
||||||
|
func WithMinSave(d time.Duration) Option { return func(t *Tracker) { t.minSave = d } }
|
||||||
|
|
||||||
|
// Tracker implements Reporter for one job.
|
||||||
|
type Tracker struct {
|
||||||
|
ctx context.Context
|
||||||
|
sink Sink
|
||||||
|
log *slog.Logger
|
||||||
|
now func() time.Time
|
||||||
|
minSave time.Duration
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
steps []domain.ProgressStep
|
||||||
|
lastAdvanceSave time.Time
|
||||||
|
lastAdvanceLog time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewTracker plans a job's phases. ctx is used for the saves, so it should outlive the
|
||||||
|
// job's own timeout: a job that timed out still gets its failure recorded.
|
||||||
|
func NewTracker(ctx context.Context, sink Sink, log *slog.Logger, plan []Planned, opts ...Option) *Tracker {
|
||||||
|
t := &Tracker{ctx: ctx, sink: sink, log: log, now: time.Now, minSave: defaultMinSave}
|
||||||
|
for _, o := range opts {
|
||||||
|
o(t)
|
||||||
|
}
|
||||||
|
for _, p := range plan {
|
||||||
|
t.steps = append(t.steps, domain.ProgressStep{Key: p.Key, Label: p.Label, State: domain.StepPending})
|
||||||
|
}
|
||||||
|
return t
|
||||||
|
}
|
||||||
|
|
||||||
|
// Start saves the plan, every phase pending.
|
||||||
|
func (t *Tracker) Start() {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
t.log.Info("job started", "steps", len(t.steps))
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Begin implements Reporter.
|
||||||
|
func (t *Tracker) Begin(key string, total int) {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
s := t.find(key)
|
||||||
|
if s == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.State, s.StartedAt, s.FinishedAt, s.Done, s.Total = domain.StepRunning, t.now(), time.Time{}, 0, total
|
||||||
|
t.log.Info("step started", "step", key, "total", total)
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Advance implements Reporter. The count is always kept; it is saved at most once per
|
||||||
|
// minimum interval and logged at most once a minute, so a counted phase does not turn
|
||||||
|
// into a write and a log line per unit.
|
||||||
|
func (t *Tracker) Advance(key string, done int) {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
s := t.find(key)
|
||||||
|
if s == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.Done = done
|
||||||
|
now := t.now()
|
||||||
|
if t.lastAdvanceLog.IsZero() || now.Sub(t.lastAdvanceLog) >= logAdvanceEach {
|
||||||
|
t.lastAdvanceLog = now
|
||||||
|
t.log.Info("step progress", "step", key, "done", done, "total", s.Total)
|
||||||
|
}
|
||||||
|
if t.lastAdvanceSave.IsZero() || now.Sub(t.lastAdvanceSave) >= t.minSave {
|
||||||
|
t.lastAdvanceSave = now
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Done implements Reporter.
|
||||||
|
func (t *Tracker) Done(key string) {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
s := t.find(key)
|
||||||
|
if s == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
t.close(s, domain.StepDone)
|
||||||
|
t.log.Info("step finished", "step", key, "elapsed", s.FinishedAt.Sub(s.StartedAt))
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fail implements Reporter: one phase failed and the job carries on.
|
||||||
|
func (t *Tracker) Fail(key string, err error) {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
s := t.find(key)
|
||||||
|
if s == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
t.close(s, domain.StepFailed)
|
||||||
|
t.log.Warn("step failed", "step", key, "elapsed", s.FinishedAt.Sub(s.StartedAt), "err", err)
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Note implements Reporter.
|
||||||
|
func (t *Tracker) Note(msg string, kv ...any) { t.log.Info(msg, kv...) }
|
||||||
|
|
||||||
|
// FailRunning marks the phase that was running as failed, because the whole job
|
||||||
|
// failed. What never started stays pending, which is what shows the user where the
|
||||||
|
// job stopped.
|
||||||
|
func (t *Tracker) FailRunning() {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
for i := range t.steps {
|
||||||
|
if t.steps[i].State == domain.StepRunning {
|
||||||
|
t.close(&t.steps[i], domain.StepFailed)
|
||||||
|
t.log.Warn("step failed", "step", t.steps[i].Key, "elapsed", t.steps[i].FinishedAt.Sub(t.steps[i].StartedAt))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Finish closes the job: a phase still running is done, a phase that never ran is
|
||||||
|
// skipped.
|
||||||
|
func (t *Tracker) Finish() {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
for i := range t.steps {
|
||||||
|
switch t.steps[i].State {
|
||||||
|
case domain.StepRunning:
|
||||||
|
t.close(&t.steps[i], domain.StepDone)
|
||||||
|
case domain.StepPending:
|
||||||
|
t.steps[i].State = domain.StepSkipped
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.log.Info("job finished")
|
||||||
|
t.save()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Heartbeat stamps the job's row every beatEvery and logs which phase is still running
|
||||||
|
// every stillEvery, until ctx ends. It runs on its own, independent of any engine call:
|
||||||
|
// a heartbeat that stopped means the process is gone, not that a request is slow.
|
||||||
|
func (t *Tracker) Heartbeat(ctx context.Context, beatEvery, stillEvery time.Duration) {
|
||||||
|
beat, still := time.NewTicker(beatEvery), time.NewTicker(stillEvery)
|
||||||
|
defer beat.Stop()
|
||||||
|
defer still.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-beat.C:
|
||||||
|
if err := t.sink.Beat(ctx); err != nil && ctx.Err() == nil {
|
||||||
|
t.log.Warn("progress: heartbeat not saved", "err", err)
|
||||||
|
}
|
||||||
|
case <-still.C:
|
||||||
|
t.logStill()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (t *Tracker) logStill() {
|
||||||
|
t.mu.Lock()
|
||||||
|
defer t.mu.Unlock()
|
||||||
|
for _, s := range t.steps {
|
||||||
|
if s.State == domain.StepRunning {
|
||||||
|
t.log.Info("still running", "step", s.Key, "elapsed", t.now().Sub(s.StartedAt), "done", s.Done, "total", s.Total)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// close ends a phase. Callers hold t.mu.
|
||||||
|
func (t *Tracker) close(s *domain.ProgressStep, state domain.StepState) {
|
||||||
|
s.State, s.FinishedAt = state, t.now()
|
||||||
|
if s.StartedAt.IsZero() {
|
||||||
|
s.StartedAt = s.FinishedAt
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// find returns the phase with this key, or nil. Callers hold t.mu.
|
||||||
|
func (t *Tracker) find(key string) *domain.ProgressStep {
|
||||||
|
for i := range t.steps {
|
||||||
|
if t.steps[i].Key == key {
|
||||||
|
return &t.steps[i]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// save persists a copy of the phases. Callers hold t.mu. A failed write is logged and
|
||||||
|
// dropped: progress is cosmetic and must never fail the job it describes.
|
||||||
|
func (t *Tracker) save() {
|
||||||
|
snapshot := append([]domain.ProgressStep(nil), t.steps...)
|
||||||
|
if err := t.sink.SaveProgress(t.ctx, snapshot, t.now()); err != nil && t.ctx.Err() == nil {
|
||||||
|
t.log.Warn("progress: not saved", "err", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type ctxKey struct{}
|
||||||
|
|
||||||
|
// With returns a context that carries r.
|
||||||
|
func With(ctx context.Context, r Reporter) context.Context {
|
||||||
|
return context.WithValue(ctx, ctxKey{}, r)
|
||||||
|
}
|
||||||
|
|
||||||
|
// From returns the Reporter carried by ctx, or one that does nothing, so a component
|
||||||
|
// can always report without checking whether anyone is listening.
|
||||||
|
func From(ctx context.Context) Reporter {
|
||||||
|
if r, ok := ctx.Value(ctxKey{}).(Reporter); ok {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
return noop{}
|
||||||
|
}
|
||||||
|
|
||||||
|
type noop struct{}
|
||||||
|
|
||||||
|
func (noop) Begin(string, int) {}
|
||||||
|
func (noop) Advance(string, int) {}
|
||||||
|
func (noop) Done(string) {}
|
||||||
|
func (noop) Fail(string, error) {}
|
||||||
|
func (noop) Note(string, ...any) {}
|
||||||
@@ -0,0 +1,344 @@
|
|||||||
|
package progress_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeSink records every saved snapshot and every heartbeat.
|
||||||
|
type fakeSink struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
saves [][]domain.ProgressStep
|
||||||
|
at []time.Time
|
||||||
|
beats int
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSink) SaveProgress(_ context.Context, steps []domain.ProgressStep, at time.Time) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.saves = append(f.saves, append([]domain.ProgressStep(nil), steps...))
|
||||||
|
f.at = append(f.at, at)
|
||||||
|
return f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSink) Beat(context.Context) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.beats++
|
||||||
|
return f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSink) last(t *testing.T) []domain.ProgressStep {
|
||||||
|
t.Helper()
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
require.NotEmpty(t, f.saves, "nothing was saved")
|
||||||
|
return f.saves[len(f.saves)-1]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSink) saveCount() int {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return len(f.saves)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSink) beatCount() int {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return f.beats
|
||||||
|
}
|
||||||
|
|
||||||
|
// syncBuf is a log sink safe to read while the heartbeat goroutine is still writing.
|
||||||
|
type syncBuf struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
b strings.Builder
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *syncBuf) Write(p []byte) (int, error) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.b.Write(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *syncBuf) String() string {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// clock is a hand-wound clock so durations and throttling are exact.
|
||||||
|
type clock struct{ t time.Time }
|
||||||
|
|
||||||
|
func (c *clock) now() time.Time { return c.t }
|
||||||
|
func (c *clock) tick(d time.Duration) { c.t = c.t.Add(d) }
|
||||||
|
func newClock() *clock { return &clock{t: time.Date(2026, 10, 2, 8, 0, 0, 0, time.UTC)} }
|
||||||
|
|
||||||
|
func plan() []progress.Planned {
|
||||||
|
return []progress.Planned{{Key: "audio", Label: "Preparing audio"}, {Key: "decide", Label: "Deciding"}, {Key: "save", Label: "Saving"}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTracker(t *testing.T) (*progress.Tracker, *fakeSink, *clock, *syncBuf) {
|
||||||
|
t.Helper()
|
||||||
|
sink, clk, buf := &fakeSink{}, newClock(), &syncBuf{}
|
||||||
|
log := slog.New(slog.NewTextHandler(buf, &slog.HandlerOptions{Level: slog.LevelDebug}))
|
||||||
|
return progress.NewTracker(context.Background(), sink, log, plan(), progress.WithClock(clk.now), progress.WithMinSave(2*time.Second)), sink, clk, buf
|
||||||
|
}
|
||||||
|
|
||||||
|
func stateOf(steps []domain.ProgressStep, key string) domain.StepState {
|
||||||
|
for _, s := range steps {
|
||||||
|
if s.Key == key {
|
||||||
|
return s.State
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_PlanIsShownUpFrontInOrderAllPending(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
|
||||||
|
tr.Start()
|
||||||
|
|
||||||
|
steps := sink.last(t)
|
||||||
|
require.Len(t, steps, 3)
|
||||||
|
require.Equal(t, []string{"audio", "decide", "save"}, []string{steps[0].Key, steps[1].Key, steps[2].Key})
|
||||||
|
for _, s := range steps {
|
||||||
|
require.Equal(t, domain.StepPending, s.State, s.Key)
|
||||||
|
}
|
||||||
|
require.Equal(t, "Preparing audio", steps[0].Label)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_BeginRunsAStepAndDoneRecordsItsDuration(t *testing.T) {
|
||||||
|
tr, sink, clk, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
require.Equal(t, domain.StepRunning, stateOf(sink.last(t), "audio"))
|
||||||
|
clk.tick(90 * time.Second)
|
||||||
|
tr.Done("audio")
|
||||||
|
|
||||||
|
s := sink.last(t)[0]
|
||||||
|
require.Equal(t, domain.StepDone, s.State)
|
||||||
|
require.Equal(t, 90*time.Second, s.FinishedAt.Sub(s.StartedAt), "the finished step must carry how long it took")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_AdvanceCountsAreSavedAndThrottled(t *testing.T) {
|
||||||
|
tr, sink, clk, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("decide", 282)
|
||||||
|
base := sink.saveCount()
|
||||||
|
|
||||||
|
tr.Advance("decide", 1)
|
||||||
|
clk.tick(500 * time.Millisecond)
|
||||||
|
tr.Advance("decide", 2)
|
||||||
|
clk.tick(500 * time.Millisecond)
|
||||||
|
tr.Advance("decide", 3)
|
||||||
|
|
||||||
|
require.Equal(t, base+1, sink.saveCount(), "advances inside the minimum interval must not each hit the database")
|
||||||
|
clk.tick(2 * time.Second)
|
||||||
|
tr.Advance("decide", 4)
|
||||||
|
require.Equal(t, base+2, sink.saveCount())
|
||||||
|
got := sink.last(t)[1]
|
||||||
|
require.Equal(t, 4, got.Done)
|
||||||
|
require.Equal(t, 282, got.Total)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_DoneFlushesTheFinalCountEvenInsideTheThrottle(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("decide", 10)
|
||||||
|
tr.Advance("decide", 9)
|
||||||
|
|
||||||
|
tr.Advance("decide", 10)
|
||||||
|
tr.Done("decide")
|
||||||
|
|
||||||
|
got := sink.last(t)[1]
|
||||||
|
require.Equal(t, domain.StepDone, got.State)
|
||||||
|
require.Equal(t, 10, got.Done, "a finished step must show its full count, not the last throttled one")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_FinishSkipsWhatNeverRanAndClosesWhatDid(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
tr.Done("audio")
|
||||||
|
tr.Begin("decide", 5)
|
||||||
|
|
||||||
|
tr.Finish()
|
||||||
|
|
||||||
|
steps := sink.last(t)
|
||||||
|
require.Equal(t, domain.StepDone, stateOf(steps, "audio"))
|
||||||
|
require.Equal(t, domain.StepDone, stateOf(steps, "decide"), "a step still running when the job completes is closed")
|
||||||
|
require.Equal(t, domain.StepSkipped, stateOf(steps, "save"), "a step that never ran is skipped, not left pending")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_FailRunningMarksTheRunningStepFailedAndLeavesTheRestPending(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
tr.Done("audio")
|
||||||
|
tr.Begin("decide", 5)
|
||||||
|
|
||||||
|
tr.FailRunning()
|
||||||
|
|
||||||
|
steps := sink.last(t)
|
||||||
|
require.Equal(t, domain.StepDone, stateOf(steps, "audio"))
|
||||||
|
require.Equal(t, domain.StepFailed, stateOf(steps, "decide"))
|
||||||
|
require.Equal(t, domain.StepPending, stateOf(steps, "save"), "what never started stays pending: it shows where the job stopped")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_AKeyedFailMarksOneStepFailedLogsWhyAndTheJobGoesOn(t *testing.T) {
|
||||||
|
tr, sink, _, buf := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
|
||||||
|
tr.Fail("audio", errors.New("gateway said no"))
|
||||||
|
tr.Begin("decide", 5)
|
||||||
|
|
||||||
|
steps := sink.last(t)
|
||||||
|
require.Equal(t, domain.StepFailed, stateOf(steps, "audio"))
|
||||||
|
require.Equal(t, domain.StepRunning, stateOf(steps, "decide"), "a soft failure must not stop the next step from running")
|
||||||
|
require.Contains(t, buf.String(), "level=WARN")
|
||||||
|
require.Contains(t, buf.String(), "gateway said no")
|
||||||
|
require.Contains(t, buf.String(), "step=audio")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_NoteIsALogLineAndNeverASave(t *testing.T) {
|
||||||
|
tr, sink, _, buf := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
before := sink.saveCount()
|
||||||
|
|
||||||
|
tr.Note("reconciled", "disputes", 362, "auto_resolved", 80)
|
||||||
|
|
||||||
|
require.Equal(t, before, sink.saveCount(), "a note is for the operator; it must not cost a database write")
|
||||||
|
require.Contains(t, buf.String(), "reconciled")
|
||||||
|
require.Contains(t, buf.String(), "disputes=362")
|
||||||
|
require.Contains(t, buf.String(), "auto_resolved=80")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_AnUnknownKeyIsIgnored(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
before := sink.saveCount()
|
||||||
|
|
||||||
|
require.NotPanics(t, func() {
|
||||||
|
tr.Begin("no-such-step", 0)
|
||||||
|
tr.Advance("no-such-step", 1)
|
||||||
|
tr.Done("no-such-step")
|
||||||
|
tr.Fail("no-such-step", errors.New("x"))
|
||||||
|
})
|
||||||
|
require.Equal(t, before, sink.saveCount())
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_ASinkFailureIsLoggedAndNeverStopsTheJob(t *testing.T) {
|
||||||
|
tr, sink, _, buf := newTracker(t)
|
||||||
|
sink.err = errors.New("db down")
|
||||||
|
|
||||||
|
require.NotPanics(t, func() {
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
tr.Done("audio")
|
||||||
|
})
|
||||||
|
require.Contains(t, buf.String(), "level=WARN", "a lost progress write is worth a WARN")
|
||||||
|
require.Contains(t, buf.String(), "db down")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_EveryTransitionIsLoggedByTheSameCall(t *testing.T) {
|
||||||
|
tr, _, clk, buf := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
|
clk.tick(3 * time.Second)
|
||||||
|
tr.Done("audio")
|
||||||
|
|
||||||
|
out := buf.String()
|
||||||
|
require.Contains(t, out, "step=audio")
|
||||||
|
require.Contains(t, out, "started")
|
||||||
|
require.Contains(t, out, "finished")
|
||||||
|
require.Contains(t, out, "elapsed=3s")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_AdvanceIsLoggedAtMostOncePerMinute(t *testing.T) {
|
||||||
|
tr, _, clk, buf := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("decide", 100)
|
||||||
|
|
||||||
|
for i := 1; i <= 30; i++ {
|
||||||
|
clk.tick(1 * time.Second)
|
||||||
|
tr.Advance("decide", i)
|
||||||
|
}
|
||||||
|
require.LessOrEqual(t, strings.Count(buf.String(), "step progress"), 1,
|
||||||
|
"30 seconds of progress is at most one log line, not thirty")
|
||||||
|
clk.tick(61 * time.Second)
|
||||||
|
tr.Advance("decide", 31)
|
||||||
|
require.Equal(t, 2, strings.Count(buf.String(), "step progress"))
|
||||||
|
require.Contains(t, buf.String(), "done=31")
|
||||||
|
require.Contains(t, buf.String(), "total=100")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_HeartbeatBeatsOnItsOwnUntilStopped(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
|
||||||
|
go func() {
|
||||||
|
defer close(done)
|
||||||
|
tr.Heartbeat(ctx, 5*time.Millisecond, time.Hour)
|
||||||
|
}()
|
||||||
|
|
||||||
|
require.Eventually(t, func() bool { return sink.beatCount() >= 3 }, 2*time.Second, 5*time.Millisecond,
|
||||||
|
"a heartbeat that does not depend on any engine call is the point")
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("Heartbeat did not stop when its context ended")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTracker_HeartbeatSaysWhichStepIsStillRunningAndForHowLong(t *testing.T) {
|
||||||
|
tr, _, clk, buf := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("decide", 100)
|
||||||
|
clk.tick(7 * time.Minute)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
go tr.Heartbeat(ctx, time.Hour, 5*time.Millisecond)
|
||||||
|
|
||||||
|
require.Eventually(t, func() bool { return strings.Contains(buf.String(), "still running") }, 2*time.Second, 5*time.Millisecond)
|
||||||
|
require.Contains(t, buf.String(), "step=decide")
|
||||||
|
require.Contains(t, buf.String(), "elapsed=7m0s")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFrom_WithoutAReporterReturnsOneThatDoesNothing(t *testing.T) {
|
||||||
|
r := progress.From(context.Background())
|
||||||
|
|
||||||
|
require.NotNil(t, r)
|
||||||
|
require.NotPanics(t, func() {
|
||||||
|
r.Begin("x", 3)
|
||||||
|
r.Advance("x", 1)
|
||||||
|
r.Done("x")
|
||||||
|
r.Fail("x", errors.New("e"))
|
||||||
|
r.Note("n", "k", 1)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWithAndFrom_ACarriedReporterIsTheOneReturned(t *testing.T) {
|
||||||
|
tr, sink, _, _ := newTracker(t)
|
||||||
|
tr.Start()
|
||||||
|
ctx := progress.With(context.Background(), tr)
|
||||||
|
|
||||||
|
progress.From(ctx).Begin("audio", 0)
|
||||||
|
|
||||||
|
require.Equal(t, domain.StepRunning, stateOf(sink.last(t), "audio"))
|
||||||
|
}
|
||||||
+118
-29
@@ -6,11 +6,17 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"git.d-ma.be/mathias/tapir/internal/domain"
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
)
|
)
|
||||||
|
|
||||||
// MaxContested bounds the stored list of contested spots. The counts on the
|
// MaxContested is the default bound on the stored list of judged spots. A 90-minute
|
||||||
// Reconciliation are always exact; only this list is capped.
|
// recording has a few hundred, so this is a guard against a runaway, not a budget. The
|
||||||
const MaxContested = 200
|
// counts on the Reconciliation are always exact; only the list is capped.
|
||||||
|
const MaxContested = 2000
|
||||||
|
|
||||||
|
// beforeWords is how many words of left context a stored spot carries: enough to find
|
||||||
|
// the place again, small enough to stay out of the way.
|
||||||
|
const beforeWords = 8
|
||||||
|
|
||||||
// Options configure Reconcile. Zero Window and BreakAfter take their defaults.
|
// Options configure Reconcile. Zero Window and BreakAfter take their defaults.
|
||||||
type Options struct {
|
type Options struct {
|
||||||
@@ -20,6 +26,8 @@ type Options struct {
|
|||||||
// BreakAfter is how many consecutive model errors stop the reconciliation, so a
|
// BreakAfter is how many consecutive model errors stop the reconciliation, so a
|
||||||
// model that is down costs three timeouts, not one per dispute.
|
// model that is down costs three timeouts, not one per dispute.
|
||||||
BreakAfter int
|
BreakAfter int
|
||||||
|
// MaxContested caps the stored judged spots; zero takes the package default.
|
||||||
|
MaxContested int
|
||||||
// BaseEngine, AltEngine and Reconciler are recorded on the result.
|
// BaseEngine, AltEngine and Reconciler are recorded on the result.
|
||||||
BaseEngine, AltEngine, Reconciler string
|
BaseEngine, AltEngine, Reconciler string
|
||||||
}
|
}
|
||||||
@@ -31,22 +39,50 @@ type Options struct {
|
|||||||
// and the result says why. The only error it returns is a cancelled context, which
|
// and the result says why. The only error it returns is a cancelled context, which
|
||||||
// is the job stopping, not a verdict on the model.
|
// is the job stopping, not a verdict on the model.
|
||||||
func Reconcile(ctx context.Context, base, alt domain.Transcription, adj Adjudicator, o Options) (domain.Transcription, error) {
|
func Reconcile(ctx context.Context, base, alt domain.Transcription, adj Adjudicator, o Options) (domain.Transcription, error) {
|
||||||
window, breakAfter := o.Window, o.BreakAfter
|
window, breakAfter, maxSpots := o.Window, o.BreakAfter, o.MaxContested
|
||||||
if window <= 0 {
|
if window <= 0 {
|
||||||
window = 100
|
window = 100
|
||||||
}
|
}
|
||||||
if breakAfter <= 0 {
|
if breakAfter <= 0 {
|
||||||
breakAfter = 3
|
breakAfter = 3
|
||||||
}
|
}
|
||||||
|
if maxSpots <= 0 {
|
||||||
|
maxSpots = MaxContested
|
||||||
|
}
|
||||||
|
rep := progress.From(ctx)
|
||||||
|
|
||||||
|
rep.Begin("compare", 0)
|
||||||
bt, at := Tokenize(base.Text), Tokenize(alt.Text)
|
bt, at := Tokenize(base.Text), Tokenize(alt.Text)
|
||||||
ds := FindDisputes(bt, at)
|
ds := FindDisputes(bt, at)
|
||||||
|
rep.Done("compare")
|
||||||
|
|
||||||
|
// The deciding phase counts what the MODEL decides: the trivial disputes are settled
|
||||||
|
// without it, so counting them would leave the bar short of its end.
|
||||||
|
deciding := 0
|
||||||
|
for _, d := range ds {
|
||||||
|
if !AutoResolve(d) {
|
||||||
|
deciding++
|
||||||
|
}
|
||||||
|
}
|
||||||
rec := &domain.Reconciliation{
|
rec := &domain.Reconciliation{
|
||||||
Status: domain.ReconcileDone, Base: o.BaseEngine, Alt: o.AltEngine, Reconciler: o.Reconciler, Disputes: len(ds),
|
Status: domain.ReconcileDone, Base: o.BaseEngine, Alt: o.AltEngine, Reconciler: o.Reconciler, Disputes: len(ds),
|
||||||
}
|
}
|
||||||
seconds := positioner(base, len(bt))
|
seconds := positioner(base, len(bt))
|
||||||
|
spot := func(d Dispute, used string, unsure bool) {
|
||||||
|
before, _ := ContextAround(bt, d, window)
|
||||||
|
if w := strings.Fields(before); len(w) > beforeWords {
|
||||||
|
before = strings.Join(w[len(w)-beforeWords:], " ")
|
||||||
|
}
|
||||||
|
if len(rec.Contested) < maxSpots {
|
||||||
|
rec.Contested = append(rec.Contested, domain.ContestedSpot{
|
||||||
|
Seconds: seconds(d.BaseStart), Base: d.Base, Alt: d.Alt, Used: used, Unsure: unsure, Before: before,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
rep.Begin("decide", deciding)
|
||||||
verdicts := make([]Verdict, len(ds))
|
verdicts := make([]Verdict, len(ds))
|
||||||
consecutive := 0
|
consecutive, handled := 0, 0
|
||||||
for i, d := range ds {
|
for i, d := range ds {
|
||||||
if AutoResolve(d) {
|
if AutoResolve(d) {
|
||||||
rec.AutoResolved++
|
rec.AutoResolved++
|
||||||
@@ -65,7 +101,9 @@ func Reconcile(ctx context.Context, base, alt domain.Transcription, adj Adjudica
|
|||||||
rec.Errors++
|
rec.Errors++
|
||||||
consecutive++
|
consecutive++
|
||||||
if consecutive >= breakAfter {
|
if consecutive >= breakAfter {
|
||||||
return skipped(base, rec, fmt.Sprintf("the model failed %d times in a row: %v", consecutive, err)), nil
|
failure := fmt.Errorf("the model failed %d times in a row: %w", consecutive, err)
|
||||||
|
rep.Fail("decide", failure)
|
||||||
|
return skipped(base, rec, failure.Error()), nil
|
||||||
}
|
}
|
||||||
v = VerdictUnsure
|
v = VerdictUnsure
|
||||||
} else {
|
} else {
|
||||||
@@ -76,30 +114,94 @@ func Reconcile(ctx context.Context, base, alt domain.Transcription, adj Adjudica
|
|||||||
switch v {
|
switch v {
|
||||||
case VerdictAlt:
|
case VerdictAlt:
|
||||||
rec.Changed++
|
rec.Changed++
|
||||||
addContested(rec, domain.ContestedSpot{Seconds: seconds(d.BaseStart), Base: d.Base, Alt: d.Alt, Used: "alt"})
|
spot(d, "alt", false)
|
||||||
case VerdictUnsure:
|
case VerdictUnsure:
|
||||||
rec.Unsure++
|
rec.Unsure++
|
||||||
addContested(rec, domain.ContestedSpot{Seconds: seconds(d.BaseStart), Base: d.Base, Alt: d.Alt, Used: "base", Unsure: true})
|
spot(d, "base", true)
|
||||||
|
default:
|
||||||
|
spot(d, "base", false)
|
||||||
}
|
}
|
||||||
|
handled++
|
||||||
|
rep.Advance("decide", handled)
|
||||||
|
}
|
||||||
|
rep.Done("decide")
|
||||||
|
|
||||||
|
out := base
|
||||||
|
out.Text, out.Segments = merge(base, bt, at, ds, verdicts)
|
||||||
|
out.Reconciliation = rec
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// merge applies the verdicts: the base's words stand except where the alternative's
|
||||||
|
// reading was chosen. It returns the merged text and the base's segments carrying the
|
||||||
|
// merged words (timing is the base's). Segments are rewritten only when their words line
|
||||||
|
// up with the text; a word wrapped across a segment edge makes them differ, and guessing
|
||||||
|
// would put words in the wrong place, so then the base's segments are returned as they
|
||||||
|
// were (ADR-035). The base's own segments are never edited in place.
|
||||||
|
func merge(base domain.Transcription, bt, at []Token, ds []Dispute, verdicts []Verdict) (string, []domain.Segment) {
|
||||||
|
segOf, aligned := segmentIndex(base.Segments, len(bt))
|
||||||
|
var perSeg [][]string
|
||||||
|
var changed map[int]bool
|
||||||
|
if aligned {
|
||||||
|
perSeg, changed = make([][]string, len(base.Segments)), map[int]bool{}
|
||||||
}
|
}
|
||||||
|
|
||||||
var parts []string
|
var parts []string
|
||||||
|
keep := func(from, to int) {
|
||||||
|
for i := from; i < to; i++ {
|
||||||
|
parts = append(parts, bt[i].Text)
|
||||||
|
if aligned {
|
||||||
|
perSeg[segOf[i]] = append(perSeg[segOf[i]], bt[i].Text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
pos := 0
|
pos := 0
|
||||||
for i, d := range ds {
|
for i, d := range ds {
|
||||||
parts = appendTexts(parts, bt[pos:d.BaseStart])
|
keep(pos, d.BaseStart)
|
||||||
if verdicts[i] == VerdictAlt {
|
if verdicts[i] == VerdictAlt {
|
||||||
parts = appendTexts(parts, at[d.AltStart:d.AltEnd])
|
for _, t := range at[d.AltStart:d.AltEnd] {
|
||||||
|
parts = append(parts, t.Text)
|
||||||
|
}
|
||||||
|
if aligned {
|
||||||
|
anchor := min(d.BaseStart, len(bt)-1)
|
||||||
|
for _, t := range at[d.AltStart:d.AltEnd] {
|
||||||
|
perSeg[segOf[anchor]] = append(perSeg[segOf[anchor]], t.Text)
|
||||||
|
}
|
||||||
|
changed[segOf[anchor]] = true
|
||||||
|
for k := d.BaseStart; k < d.BaseEnd; k++ {
|
||||||
|
changed[segOf[k]] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
parts = appendTexts(parts, bt[d.BaseStart:d.BaseEnd])
|
keep(d.BaseStart, d.BaseEnd)
|
||||||
}
|
}
|
||||||
pos = d.BaseEnd
|
pos = d.BaseEnd
|
||||||
}
|
}
|
||||||
parts = appendTexts(parts, bt[pos:])
|
keep(pos, len(bt))
|
||||||
|
|
||||||
out := base
|
if !aligned {
|
||||||
out.Text = strings.Join(parts, " ")
|
return strings.Join(parts, " "), base.Segments
|
||||||
out.Reconciliation = rec
|
}
|
||||||
return out, nil
|
segs := append([]domain.Segment(nil), base.Segments...)
|
||||||
|
for k := range changed {
|
||||||
|
segs[k].Text = strings.Join(perSeg[k], " ")
|
||||||
|
}
|
||||||
|
return strings.Join(parts, " "), segs
|
||||||
|
}
|
||||||
|
|
||||||
|
// segmentIndex maps each base token to the segment it came from. aligned is false when
|
||||||
|
// there are no segments or their word count is not the text's.
|
||||||
|
func segmentIndex(segs []domain.Segment, tokens int) (segOf []int, aligned bool) {
|
||||||
|
if len(segs) == 0 || tokens == 0 {
|
||||||
|
return nil, false
|
||||||
|
}
|
||||||
|
segOf = make([]int, 0, tokens)
|
||||||
|
for k, s := range segs {
|
||||||
|
for range Tokenize(s.Text) {
|
||||||
|
segOf = append(segOf, k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return segOf, len(segOf) == tokens
|
||||||
}
|
}
|
||||||
|
|
||||||
// skipped is the base text, none of the partial changes, and the reason.
|
// skipped is the base text, none of the partial changes, and the reason.
|
||||||
@@ -112,13 +214,6 @@ func skipped(base domain.Transcription, rec *domain.Reconciliation, reason strin
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func appendTexts(dst []string, ts []Token) []string {
|
|
||||||
for _, t := range ts {
|
|
||||||
dst = append(dst, t.Text)
|
|
||||||
}
|
|
||||||
return dst
|
|
||||||
}
|
|
||||||
|
|
||||||
// positioner maps a token index of the base to the second it starts at: through the
|
// positioner maps a token index of the base to the second it starts at: through the
|
||||||
// segments when there are any, proportionally otherwise.
|
// segments when there are any, proportionally otherwise.
|
||||||
func positioner(base domain.Transcription, tokens int) func(int) float64 {
|
func positioner(base domain.Transcription, tokens int) func(int) float64 {
|
||||||
@@ -144,9 +239,3 @@ func positioner(base domain.Transcription, tokens int) func(int) float64 {
|
|||||||
return base.Segments[k].Start
|
return base.Segments[k].Start
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func addContested(rec *domain.Reconciliation, c domain.ContestedSpot) {
|
|
||||||
if len(rec.Contested) < MaxContested {
|
|
||||||
rec.Contested = append(rec.Contested, c)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package reconcile_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
||||||
|
)
|
||||||
|
|
||||||
|
// What Reconcile reports while it works (ADR-035): the comparison, then the deciding
|
||||||
|
// phase with the real number of model decisions as its total.
|
||||||
|
|
||||||
|
type repRec struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
events []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *repRec) add(e string) {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
r.events = append(r.events, e)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *repRec) Begin(key string, total int) { r.add(fmt.Sprintf("begin:%s:%d", key, total)) }
|
||||||
|
func (r *repRec) Advance(key string, done int) { r.add(fmt.Sprintf("advance:%s:%d", key, done)) }
|
||||||
|
func (r *repRec) Done(key string) { r.add("done:" + key) }
|
||||||
|
func (r *repRec) Fail(key string, _ error) { r.add("fail:" + key) }
|
||||||
|
func (r *repRec) Note(string, ...any) {}
|
||||||
|
func (r *repRec) joined() string {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
return strings.Join(r.events, ",")
|
||||||
|
}
|
||||||
|
func withRep(r *repRec) context.Context { return progress.With(context.Background(), r) }
|
||||||
|
func reconcileWith(t *testing.T, ctx context.Context, base, alt string, adj reconcile.Adjudicator) error {
|
||||||
|
t.Helper()
|
||||||
|
_, err := reconcile.Reconcile(ctx, tr(base), tr(alt), adj, opts)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// two compound/filler disputes the model is never asked about, then two it is.
|
||||||
|
const (
|
||||||
|
progBase = "i crossborder delen ska vi ju betala totalt 1015 kronor till SEB idag"
|
||||||
|
progAlt = "i crossborderdelen ska vi betala totalt 1050 kronor till SCB idag"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestReconcileProgress_TheDecidingPhaseTotalIsTheModelDecisionsNotTheDisputes(t *testing.T) {
|
||||||
|
rep := &repRec{}
|
||||||
|
|
||||||
|
require.NoError(t, reconcileWith(t, withRep(rep), progBase, progAlt, always(reconcile.VerdictAlt)))
|
||||||
|
|
||||||
|
assert.Equal(t,
|
||||||
|
"begin:compare:0,done:compare,begin:decide:2,advance:decide:1,advance:decide:2,done:decide",
|
||||||
|
rep.joined(),
|
||||||
|
"4 disputes, 2 settled without the model: the bar counts the 2 the model decides, so it can reach its end")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileProgress_TextsThatAgreeStillClosePhasesSoNothingIsLeftRunning(t *testing.T) {
|
||||||
|
rep := &repRec{}
|
||||||
|
|
||||||
|
require.NoError(t, reconcileWith(t, withRep(rep), "samma text hela vägen", "samma text hela vägen", always(reconcile.VerdictAlt)))
|
||||||
|
|
||||||
|
assert.Equal(t, "begin:compare:0,done:compare,begin:decide:0,done:decide", rep.joined())
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileProgress_AModelErrorStillCountsAsADecisionTaken(t *testing.T) {
|
||||||
|
rep := &repRec{}
|
||||||
|
adj := &scripted{fn: func(i int, _ reconcile.Request) (reconcile.Verdict, error) {
|
||||||
|
if i == 0 {
|
||||||
|
return reconcile.VerdictBase, errors.New("one timeout")
|
||||||
|
}
|
||||||
|
return reconcile.VerdictAlt, nil
|
||||||
|
}}
|
||||||
|
|
||||||
|
require.NoError(t, reconcileWith(t, withRep(rep), progBase, progAlt, adj))
|
||||||
|
|
||||||
|
assert.Contains(t, rep.joined(), "advance:decide:1,advance:decide:2", "a spot that errored was still handled; the bar must move past it")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileProgress_TheBreakerTrippingShowsTheDecidingPhaseFailedNotDone(t *testing.T) {
|
||||||
|
rep := &repRec{}
|
||||||
|
adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) {
|
||||||
|
return reconcile.VerdictBase, errors.New("model down")
|
||||||
|
}}
|
||||||
|
small := opts
|
||||||
|
small.BreakAfter = 2
|
||||||
|
|
||||||
|
_, err := reconcile.Reconcile(withRep(rep), tr(progBase), tr(progAlt), adj, small)
|
||||||
|
|
||||||
|
require.NoError(t, err, "a model that is down degrades the result; it does not fail the job")
|
||||||
|
assert.Contains(t, rep.joined(), "fail:decide")
|
||||||
|
assert.NotContains(t, rep.joined(), "done:decide")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileProgress_AShutdownIsNeitherDoneNorFailed(t *testing.T) {
|
||||||
|
rep := &repRec{}
|
||||||
|
ctx, cancel := context.WithCancel(withRep(rep))
|
||||||
|
adj := &scripted{fn: func(int, reconcile.Request) (reconcile.Verdict, error) {
|
||||||
|
cancel()
|
||||||
|
return reconcile.VerdictAlt, nil
|
||||||
|
}}
|
||||||
|
|
||||||
|
err := reconcileWith(t, ctx, progBase, progAlt, adj)
|
||||||
|
|
||||||
|
require.ErrorIs(t, err, context.Canceled)
|
||||||
|
assert.NotContains(t, rep.joined(), "fail:decide")
|
||||||
|
assert.NotContains(t, rep.joined(), "done:decide")
|
||||||
|
}
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
package reconcile_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Every judged spot is stored with a few words of left context, and the saved segments
|
||||||
|
// carry the merged words while keeping the base's timing (ADR-035 decisions 6 and 7).
|
||||||
|
|
||||||
|
func TestReconcileSpots_AKeptSpotAndAChangedSpotAreBothStoredWithTheirVerdict(t *testing.T) {
|
||||||
|
adj := &scripted{fn: func(i int, _ reconcile.Request) (reconcile.Verdict, error) {
|
||||||
|
if i == 0 {
|
||||||
|
return reconcile.VerdictBase, nil
|
||||||
|
}
|
||||||
|
return reconcile.VerdictAlt, nil
|
||||||
|
}}
|
||||||
|
|
||||||
|
out := merge(t, tr(progBase), tr(progAlt), adj)
|
||||||
|
|
||||||
|
require.Len(t, out.Reconciliation.Contested, 2)
|
||||||
|
assert.Equal(t, "base", out.Reconciliation.Contested[0].Used)
|
||||||
|
assert.Equal(t, "1015", out.Reconciliation.Contested[0].Base)
|
||||||
|
assert.Equal(t, "alt", out.Reconciliation.Contested[1].Used)
|
||||||
|
assert.Equal(t, 1, out.Reconciliation.Changed, "the exact counts are unchanged by storing more")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSpots_AutoResolvedDisputesAreNotStored(t *testing.T) {
|
||||||
|
out := merge(t, tr("i crossborder delen ska vi ju betala"), tr("i crossborderdelen ska vi betala"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
assert.Empty(t, out.Reconciliation.Contested, "compounds and fillers were never judged; they are counted, not listed")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSpots_ASpotCarriesTheWordsJustBeforeIt(t *testing.T) {
|
||||||
|
out := merge(t, tr("vi skickar in ett special meddelande till taxsystemet"), tr("vi skickar in ett CAMT meddelande till taxsystemet"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
require.Len(t, out.Reconciliation.Contested, 1)
|
||||||
|
assert.Equal(t, "vi skickar in ett", out.Reconciliation.Contested[0].Before)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSpots_TheContextIsAFewWordsNotTheWholeMeeting(t *testing.T) {
|
||||||
|
long := "ett två tre fyra fem sex sju åtta nio tio elva tolv"
|
||||||
|
out := merge(t, tr(long+" X slut"), tr(long+" Y slut"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
require.Len(t, out.Reconciliation.Contested, 1)
|
||||||
|
assert.Equal(t, "fem sex sju åtta nio tio elva tolv", out.Reconciliation.Contested[0].Before,
|
||||||
|
"eight words: enough to find the place, small enough to stay out of the way")
|
||||||
|
}
|
||||||
|
|
||||||
|
func segmented(text string, segs ...domain.Segment) domain.Transcription {
|
||||||
|
t := tr(text)
|
||||||
|
t.Segments = segs
|
||||||
|
return t
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_AChangedWordReplacesTheBaseWordInItsOwnSegmentOnly(t *testing.T) {
|
||||||
|
base := segmented("ett två tre. fyra fem sex. sju åtta nio.",
|
||||||
|
domain.Segment{Start: 0, End: 10, Text: "ett två tre."},
|
||||||
|
domain.Segment{Start: 10, End: 20, Text: "fyra fem sex."},
|
||||||
|
domain.Segment{Start: 20, End: 30, Text: "sju åtta nio."})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("ett två tre. fyra femm sex. sju åtta nio."), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
require.Len(t, out.Segments, 3)
|
||||||
|
assert.Equal(t, "ett två tre.", out.Segments[0].Text, "an untouched segment keeps its exact text")
|
||||||
|
assert.Equal(t, "fyra femm sex.", out.Segments[1].Text)
|
||||||
|
assert.Equal(t, "sju åtta nio.", out.Segments[2].Text)
|
||||||
|
assert.InDelta(t, 10.0, out.Segments[1].Start, 1e-9, "timing stays the base's")
|
||||||
|
assert.InDelta(t, 20.0, out.Segments[1].End, 1e-9)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_AWordOnlyTheAlternativeHeardLandsInTheSegmentItBelongsTo(t *testing.T) {
|
||||||
|
base := segmented("a b c. d e f.", domain.Segment{Start: 0, End: 5, Text: "a b c."}, domain.Segment{Start: 5, End: 10, Text: "d e f."})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("a b c. d NY e f."), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
assert.Equal(t, "a b c.", out.Segments[0].Text)
|
||||||
|
assert.Equal(t, "d NY e f.", out.Segments[1].Text)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_AWordOnlyTheBaseHeardIsRemovedFromItsSegment(t *testing.T) {
|
||||||
|
base := segmented("a b c. d X e f.", domain.Segment{Start: 0, End: 5, Text: "a b c."}, domain.Segment{Start: 5, End: 10, Text: "d X e f."})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("a b c. d e f."), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
assert.Equal(t, "d e f.", out.Segments[1].Text)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_AKeptBaseLeavesEverySegmentAsItWas(t *testing.T) {
|
||||||
|
base := segmented("a x b", domain.Segment{Start: 0, End: 4, Text: "a x b"})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("a y b"), always(reconcile.VerdictBase))
|
||||||
|
|
||||||
|
assert.Equal(t, base.Segments, out.Segments)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_TheCallersSegmentsAreNeverChangedInPlace(t *testing.T) {
|
||||||
|
segs := []domain.Segment{{Start: 0, End: 4, Text: "a x b"}}
|
||||||
|
base := segmented("a x b", segs...)
|
||||||
|
|
||||||
|
_ = merge(t, base, tr("a y b"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
assert.Equal(t, "a x b", segs[0].Text, "Reconcile builds new segments; it does not edit the base transcription's")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_SegmentsThatDoNotLineUpWithTheTextAreLeftAsTheBases(t *testing.T) {
|
||||||
|
// A word wrapped across a segment edge makes the segments' word count differ from
|
||||||
|
// the joined text's. Guessing would put words in the wrong place, so leave them.
|
||||||
|
base := segmented("a x b c", domain.Segment{Start: 0, End: 4, Text: "a x"}, domain.Segment{Start: 4, End: 8, Text: "b c d"})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("a y b c"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
assert.Equal(t, base.Segments, out.Segments)
|
||||||
|
assert.Equal(t, "a y b c", out.Text, "the merged text is unaffected")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcileSegments_ASpanThatCrossesASegmentEdgeIsRemovedFromBothSegments(t *testing.T) {
|
||||||
|
base := segmented("a b c d", domain.Segment{Start: 0, End: 5, Text: "a b"}, domain.Segment{Start: 5, End: 10, Text: "c d"})
|
||||||
|
|
||||||
|
out := merge(t, base, tr("a X d"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
|
require.Equal(t, "a X d", out.Text)
|
||||||
|
assert.Equal(t, "a X", out.Segments[0].Text)
|
||||||
|
assert.Equal(t, "d", out.Segments[1].Text, "'c' was replaced too: it must not linger in the second segment")
|
||||||
|
}
|
||||||
@@ -84,7 +84,12 @@ func TestReconcile_BaseAndUnsureVerdictsKeepTheBaseButOnlyUnsureIsFlagged(t *tes
|
|||||||
|
|
||||||
assert.Equal(t, "Totalt 1015 kronor till SEB.", keptBase.Text)
|
assert.Equal(t, "Totalt 1015 kronor till SEB.", keptBase.Text)
|
||||||
assert.Zero(t, keptBase.Reconciliation.Unsure)
|
assert.Zero(t, keptBase.Reconciliation.Unsure)
|
||||||
assert.Empty(t, keptBase.Reconciliation.Contested, "a confident 'the base is right' is not a spot worth the reader's time")
|
require.Len(t, keptBase.Reconciliation.Contested, 2,
|
||||||
|
"every judged spot is stored, kept ones too: without them the reconciler cannot be compared with either engine alone (ADR-035)")
|
||||||
|
for _, c := range keptBase.Reconciliation.Contested {
|
||||||
|
assert.Equal(t, "base", c.Used)
|
||||||
|
assert.False(t, c.Unsure, "a confident 'the base is right' is stored but is not an uncertain spot")
|
||||||
|
}
|
||||||
assert.Equal(t, "Totalt 1015 kronor till SEB.", unsure.Text, "unsure keeps the base")
|
assert.Equal(t, "Totalt 1015 kronor till SEB.", unsure.Text, "unsure keeps the base")
|
||||||
assert.Equal(t, 2, unsure.Reconciliation.Unsure)
|
assert.Equal(t, 2, unsure.Reconciliation.Unsure)
|
||||||
require.Len(t, unsure.Reconciliation.Contested, 2)
|
require.Len(t, unsure.Reconciliation.Contested, 2)
|
||||||
@@ -224,14 +229,16 @@ func TestReconcile_ACancelledJobIsAnErrorNotASkippedReconciliation(t *testing.T)
|
|||||||
assert.ErrorIs(t, err, context.Canceled, "a shutdown must not be recorded as 'the model was unavailable'")
|
assert.ErrorIs(t, err, context.Canceled, "a shutdown must not be recorded as 'the model was unavailable'")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReconcile_KeepsTheBaseSegmentsDurationAndNamesTheEngines(t *testing.T) {
|
func TestReconcile_KeepsTheBaseTimingAndDurationAndNamesTheEngines(t *testing.T) {
|
||||||
base := tr("a x b")
|
base := tr("a x b")
|
||||||
base.Segments = []domain.Segment{{Start: 0, End: 4, Text: "a x b"}}
|
base.Segments = []domain.Segment{{Start: 0, End: 4, Text: "a x b"}}
|
||||||
base.AudioSeconds = 3821.5
|
base.AudioSeconds = 3821.5
|
||||||
|
|
||||||
out := merge(t, base, tr("a y b"), always(reconcile.VerdictAlt))
|
out := merge(t, base, tr("a y b"), always(reconcile.VerdictAlt))
|
||||||
|
|
||||||
assert.Equal(t, base.Segments, out.Segments, "the timed segments are the base engine's, untouched")
|
require.Len(t, out.Segments, 1)
|
||||||
|
assert.InDelta(t, 0.0, out.Segments[0].Start, 1e-9, "timing is the base engine's")
|
||||||
|
assert.InDelta(t, 4.0, out.Segments[0].End, 1e-9)
|
||||||
assert.InDelta(t, 3821.5, out.AudioSeconds, 1e-9)
|
assert.InDelta(t, 3821.5, out.AudioSeconds, 1e-9)
|
||||||
assert.Equal(t, "iguana/pianissimo", out.Reconciliation.Base)
|
assert.Equal(t, "iguana/pianissimo", out.Reconciliation.Base)
|
||||||
assert.Equal(t, "iguana/kb-whisper", out.Reconciliation.Alt)
|
assert.Equal(t, "iguana/kb-whisper", out.Reconciliation.Alt)
|
||||||
@@ -264,18 +271,25 @@ func TestReconcile_WithNoSegmentsASpotIsPlacedProportionally(t *testing.T) {
|
|||||||
assert.InDelta(t, 90.0, out.Reconciliation.Contested[0].Seconds, 1e-9, "token 9 of 10 in 100 s")
|
assert.InDelta(t, 90.0, out.Reconciliation.Contested[0].Seconds, 1e-9, "token 9 of 10 in 100 s")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReconcile_ContestedSpotsAreCappedButTheCountsAreExact(t *testing.T) {
|
func TestReconcile_JudgedSpotsAreCappedButTheCountsAreExact(t *testing.T) {
|
||||||
a, b := make([]string, 0, 600), make([]string, 0, 600)
|
a, b := make([]string, 0, 600), make([]string, 0, 600)
|
||||||
for i := 0; i < 300; i++ {
|
for i := 0; i < 300; i++ {
|
||||||
a = append(a, fmt.Sprintf("w%d", i), "same", "same2")
|
a = append(a, fmt.Sprintf("w%d", i), "same", "same2")
|
||||||
b = append(b, fmt.Sprintf("v%d", i), "same", "same2")
|
b = append(b, fmt.Sprintf("v%d", i), "same", "same2")
|
||||||
}
|
}
|
||||||
|
capped := opts
|
||||||
|
capped.MaxContested = 50
|
||||||
|
|
||||||
out := merge(t, tr(strings.Join(a, " ")), tr(strings.Join(b, " ")), always(reconcile.VerdictUnsure))
|
out, err := reconcile.Reconcile(context.Background(), tr(strings.Join(a, " ")), tr(strings.Join(b, " ")), always(reconcile.VerdictUnsure), capped)
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
assert.Equal(t, 300, out.Reconciliation.Unsure)
|
assert.Equal(t, 300, out.Reconciliation.Unsure)
|
||||||
assert.Equal(t, 300, out.Reconciliation.Disputes)
|
assert.Equal(t, 300, out.Reconciliation.Disputes)
|
||||||
assert.Len(t, out.Reconciliation.Contested, reconcile.MaxContested, "the stored list is bounded; the counts are not")
|
assert.Len(t, out.Reconciliation.Contested, 50, "the stored list is bounded; the counts are not")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestReconcile_TheDefaultCapIsAFewThousandSoAWholeMeetingIsKept(t *testing.T) {
|
||||||
|
assert.Equal(t, 2000, reconcile.MaxContested, "a 90-minute recording has a few hundred judged spots; the cap is a guard, not a budget")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReconcile_AVerdictReturnedAlongsideAnErrorIsIgnored(t *testing.T) {
|
func TestReconcile_AVerdictReturnedAlongsideAnErrorIsIgnored(t *testing.T) {
|
||||||
|
|||||||
+109
-8
@@ -10,6 +10,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.d-ma.be/mathias/tapir/internal/domain"
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrStopped is returned by Submit once the worker has stopped.
|
// ErrStopped is returned by Submit once the worker has stopped.
|
||||||
@@ -26,13 +27,52 @@ type Transcriber interface {
|
|||||||
Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error)
|
Transcribe(ctx context.Context, wavPath string) (domain.Transcription, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Planner is an optional extra on a Transcriber: it declares its own phases (for
|
||||||
|
// example one per engine) so the page can show them. A Transcriber that does not
|
||||||
|
// implement it gets a single "transcribing" phase from the worker. Whatever a Planner
|
||||||
|
// declares it reports itself, through progress.From(ctx) (ADR-035).
|
||||||
|
type Planner interface {
|
||||||
|
Plan() []progress.Planned
|
||||||
|
}
|
||||||
|
|
||||||
// WorkerStore is the slice of the private transcript store the worker drives.
|
// WorkerStore is the slice of the private transcript store the worker drives.
|
||||||
type WorkerStore interface {
|
type WorkerStore interface {
|
||||||
StartTranscribing(ctx context.Context, userID, id string) error
|
StartTranscribing(ctx context.Context, userID, id string) error
|
||||||
CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error
|
CompleteUpload(ctx context.Context, userID, id string, t domain.Transcription) error
|
||||||
FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error
|
FailUpload(ctx context.Context, userID, id string, kind domain.UploadErrorKind) error
|
||||||
|
SaveProgress(ctx context.Context, userID, id string, steps []domain.ProgressStep, at time.Time) error
|
||||||
|
Heartbeat(ctx context.Context, userID, id string) error
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WorkerOption tunes a Worker.
|
||||||
|
type WorkerOption func(*Worker)
|
||||||
|
|
||||||
|
// WithHeartbeat sets how often a running job stamps its row alive and how often it logs
|
||||||
|
// which phase is still running. The defaults are 10 s and 5 min (ADR-035).
|
||||||
|
func WithHeartbeat(beat, still time.Duration) WorkerOption {
|
||||||
|
return func(w *Worker) { w.beatEvery, w.stillEvery = beat, still }
|
||||||
|
}
|
||||||
|
|
||||||
|
// WithLogger replaces slog.Default() as the worker's logger.
|
||||||
|
func WithLogger(l *slog.Logger) WorkerOption { return func(w *Worker) { w.log = l } }
|
||||||
|
|
||||||
|
const (
|
||||||
|
defaultBeatEvery = 10 * time.Second
|
||||||
|
defaultStillEvery = 5 * time.Minute
|
||||||
|
)
|
||||||
|
|
||||||
|
// jobSink binds the store to one job so a progress.Tracker can persist through it.
|
||||||
|
type jobSink struct {
|
||||||
|
st WorkerStore
|
||||||
|
userID, id string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s jobSink) SaveProgress(ctx context.Context, steps []domain.ProgressStep, at time.Time) error {
|
||||||
|
return s.st.SaveProgress(ctx, s.userID, s.id, steps, at)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s jobSink) Beat(ctx context.Context) error { return s.st.Heartbeat(ctx, s.userID, s.id) }
|
||||||
|
|
||||||
// Worker transcribes accepted uploads in-process, ONE AT A TIME (ADR-033). It is
|
// Worker transcribes accepted uploads in-process, ONE AT A TIME (ADR-033). It is
|
||||||
// not a second deployable: it shares the web process's lifetime, which is why it
|
// not a second deployable: it shares the web process's lifetime, which is why it
|
||||||
// inherits ADR-018's single-replica assumption — the queue is process-local. The
|
// inherits ADR-018's single-replica assumption — the queue is process-local. The
|
||||||
@@ -45,6 +85,9 @@ type Worker struct {
|
|||||||
tmpDir string
|
tmpDir string
|
||||||
timeout time.Duration
|
timeout time.Duration
|
||||||
|
|
||||||
|
log *slog.Logger
|
||||||
|
beatEvery, stillEvery time.Duration
|
||||||
|
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
queue []Job
|
queue []Job
|
||||||
stopped bool
|
stopped bool
|
||||||
@@ -53,8 +96,15 @@ type Worker struct {
|
|||||||
|
|
||||||
// NewWorker wires a Worker. timeout bounds each job (conversion + transcription);
|
// NewWorker wires a Worker. timeout bounds each job (conversion + transcription);
|
||||||
// a job that outlives it is failed rather than left spinning.
|
// a job that outlives it is failed rather than left spinning.
|
||||||
func NewWorker(s WorkerStore, c Converter, t Transcriber, tmpDir string, timeout time.Duration) *Worker {
|
func NewWorker(s WorkerStore, c Converter, t Transcriber, tmpDir string, timeout time.Duration, opts ...WorkerOption) *Worker {
|
||||||
return &Worker{store: s, conv: c, tr: t, tmpDir: tmpDir, timeout: timeout, wake: make(chan struct{}, 1)}
|
w := &Worker{
|
||||||
|
store: s, conv: c, tr: t, tmpDir: tmpDir, timeout: timeout, wake: make(chan struct{}, 1),
|
||||||
|
beatEvery: defaultBeatEvery, stillEvery: defaultStillEvery,
|
||||||
|
}
|
||||||
|
for _, o := range opts {
|
||||||
|
o(w)
|
||||||
|
}
|
||||||
|
return w
|
||||||
}
|
}
|
||||||
|
|
||||||
// Submit queues a job. It never blocks. It satisfies Submitter.
|
// Submit queues a job. It never blocks. It satisfies Submitter.
|
||||||
@@ -125,9 +175,17 @@ func (w *Worker) stop() {
|
|||||||
// uploaded file and the converted WAV are deleted on every path — the audio is
|
// uploaded file and the converted WAV are deleted on every path — the audio is
|
||||||
// never retained (ADR-033). A failure is recorded with a TYPED kind; a shutdown
|
// never retained (ADR-033). A failure is recorded with a TYPED kind; a shutdown
|
||||||
// mid-job is not a failure and leaves the row for the start-up sweep.
|
// mid-job is not a failure and leaves the row for the start-up sweep.
|
||||||
|
//
|
||||||
|
// Every phase is reported through one progress.Tracker, which saves it to the row for
|
||||||
|
// the page and logs it, and a heartbeat stamps the row on its own while the job runs
|
||||||
|
// (ADR-035). Nothing logged or saved here carries transcript text or the file's name.
|
||||||
func (w *Worker) Process(ctx context.Context, job Job) {
|
func (w *Worker) Process(ctx context.Context, job Job) {
|
||||||
defer func() { _ = os.Remove(job.Path) }()
|
defer func() { _ = os.Remove(job.Path) }()
|
||||||
log := slog.Default().With("upload", job.UploadID)
|
base := w.log
|
||||||
|
if base == nil {
|
||||||
|
base = slog.Default()
|
||||||
|
}
|
||||||
|
log := base.With("upload", job.UploadID)
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
|
|
||||||
if err := w.store.StartTranscribing(ctx, job.UserID, job.UploadID); err != nil {
|
if err := w.store.StartTranscribing(ctx, job.UserID, job.UploadID); err != nil {
|
||||||
@@ -136,32 +194,75 @@ func (w *Worker) Process(ctx context.Context, job Job) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
plan := []progress.Planned{{Key: "received", Label: "Received"}, {Key: "audio", Label: "Preparing the audio"}}
|
||||||
|
planner, planned := w.tr.(Planner)
|
||||||
|
if planned {
|
||||||
|
plan = append(plan, planner.Plan()...)
|
||||||
|
} else {
|
||||||
|
plan = append(plan, progress.Planned{Key: "transcribe", Label: "Transcribing"})
|
||||||
|
}
|
||||||
|
plan = append(plan, progress.Planned{Key: "save", Label: "Saving the transcript"})
|
||||||
|
tr := progress.NewTracker(ctx, jobSink{st: w.store, userID: job.UserID, id: job.UploadID}, log, plan)
|
||||||
|
tr.Start()
|
||||||
|
tr.Begin("received", 0)
|
||||||
|
tr.Done("received")
|
||||||
|
|
||||||
|
hbCtx, stopBeat := context.WithCancel(ctx)
|
||||||
|
hbDone := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
defer close(hbDone)
|
||||||
|
tr.Heartbeat(hbCtx, w.beatEvery, w.stillEvery)
|
||||||
|
}()
|
||||||
|
defer func() { stopBeat(); <-hbDone }() // no heartbeat outlives its job
|
||||||
|
|
||||||
|
// abort records where the job stopped. A shutdown (the parent context ended) is not
|
||||||
|
// a failure: the row is left for the start-up sweep, so the step stays as it was.
|
||||||
|
abort := func() {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
tr.FailRunning()
|
||||||
|
}
|
||||||
|
w.fail(ctx, job, log)
|
||||||
|
}
|
||||||
|
|
||||||
wav, err := w.tempWAV()
|
wav, err := w.tempWAV()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Error("upload: temp wav", "err", err)
|
log.Error("upload: temp wav", "err", err)
|
||||||
w.fail(ctx, job, log)
|
abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
defer func() { _ = os.Remove(wav) }()
|
defer func() { _ = os.Remove(wav) }()
|
||||||
|
|
||||||
jobCtx, cancel := context.WithTimeout(ctx, w.timeout)
|
jobCtx, cancel := context.WithTimeout(ctx, w.timeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
tr.Begin("audio", 0)
|
||||||
if err := w.conv.ToWAV(jobCtx, job.Path, wav); err != nil {
|
if err := w.conv.ToWAV(jobCtx, job.Path, wav); err != nil {
|
||||||
log.Error("upload: convert", "err", err)
|
log.Error("upload: convert", "err", err)
|
||||||
w.fail(ctx, job, log)
|
abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
t, err := w.tr.Transcribe(jobCtx, wav)
|
tr.Done("audio")
|
||||||
|
|
||||||
|
if !planned {
|
||||||
|
tr.Begin("transcribe", 0)
|
||||||
|
}
|
||||||
|
t, err := w.tr.Transcribe(progress.With(jobCtx, tr), wav)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Error("upload: transcribe", "err", err, "elapsed", time.Since(start).Round(time.Second))
|
log.Error("upload: transcribe", "err", err, "elapsed", time.Since(start).Round(time.Second))
|
||||||
w.fail(ctx, job, log)
|
abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if !planned {
|
||||||
|
tr.Done("transcribe")
|
||||||
|
}
|
||||||
|
|
||||||
|
tr.Begin("save", 0)
|
||||||
if err := w.store.CompleteUpload(ctx, job.UserID, job.UploadID, t); err != nil {
|
if err := w.store.CompleteUpload(ctx, job.UserID, job.UploadID, t); err != nil {
|
||||||
log.Error("upload: store transcript", "err", err)
|
log.Error("upload: store transcript", "err", err)
|
||||||
w.fail(ctx, job, log)
|
abort()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
tr.Done("save")
|
||||||
|
tr.Finish()
|
||||||
log.Info("upload transcribed", "endpoint", t.Endpoint, "audio_s", t.AudioSeconds,
|
log.Info("upload transcribed", "endpoint", t.Endpoint, "audio_s", t.AudioSeconds,
|
||||||
"segments", len(t.Segments), "elapsed", time.Since(start).Round(time.Second))
|
"segments", len(t.Segments), "elapsed", time.Since(start).Round(time.Second))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,165 @@
|
|||||||
|
package uploads_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/progress"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/uploads"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Progress, heartbeat and logging of the worker (ADR-035, #42, #43).
|
||||||
|
|
||||||
|
// plannedTr is a transcriber that declares its own phases and reports them, like the
|
||||||
|
// two-engine Ensemble does.
|
||||||
|
type plannedTr struct{ *fakeTranscriber }
|
||||||
|
|
||||||
|
func (plannedTr) Plan() []progress.Planned {
|
||||||
|
return []progress.Planned{{Key: "engine_a", Label: "Transcribing with A"}, {Key: "engine_b", Label: "Transcribing with B"}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p plannedTr) Transcribe(ctx context.Context, wav string) (domain.Transcription, error) {
|
||||||
|
r := progress.From(ctx)
|
||||||
|
r.Begin("engine_a", 0)
|
||||||
|
r.Done("engine_a")
|
||||||
|
r.Begin("engine_b", 0)
|
||||||
|
r.Done("engine_b")
|
||||||
|
return p.fakeTranscriber.Transcribe(ctx, wav)
|
||||||
|
}
|
||||||
|
|
||||||
|
func keysAndStates(steps []domain.ProgressStep) []string {
|
||||||
|
var out []string
|
||||||
|
for _, s := range steps {
|
||||||
|
out = append(out, s.Key+":"+string(s.State))
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// syncLog is a log sink safe to read while the worker's goroutines write.
|
||||||
|
type syncLog struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
b bytes.Buffer
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *syncLog) Write(p []byte) (int, error) {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
return l.b.Write(p)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *syncLog) String() string {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
return l.b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
func newProgressWorker(t *testing.T, r *wrig, tr uploads.Transcriber, opts ...uploads.WorkerOption) *uploads.Worker {
|
||||||
|
t.Helper()
|
||||||
|
return uploads.NewWorker(r.st, r.conv, tr, r.tmp, time.Minute, opts...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_ASingleEngineJobShowsEveryPhaseDone(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
r.w.Process(context.Background(), r.spool(t, "a"))
|
||||||
|
|
||||||
|
assert.Equal(t, []string{"received:done", "audio:done", "transcribe:done", "save:done"}, keysAndStates(r.st.lastSteps()))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_ATranscriberThatPlansItsOwnPhasesGetsThemBetweenAudioAndSaving(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
w := newProgressWorker(t, r, plannedTr{r.tr})
|
||||||
|
|
||||||
|
w.Process(context.Background(), r.spool(t, "a"))
|
||||||
|
|
||||||
|
assert.Equal(t, []string{"received:done", "audio:done", "engine_a:done", "engine_b:done", "save:done"}, keysAndStates(r.st.lastSteps()))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_AFailureMarksWhereItStoppedAndLeavesTheRestPending(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
setup func(*wrig)
|
||||||
|
want []string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
"conversion fails", func(r *wrig) { r.conv.err = errors.New("moov atom not found") },
|
||||||
|
[]string{"received:done", "audio:failed", "transcribe:pending", "save:pending"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"every engine fails", func(r *wrig) { r.tr.err = errors.New("iguana down") },
|
||||||
|
[]string{"received:done", "audio:done", "transcribe:failed", "save:pending"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
tt.setup(r)
|
||||||
|
|
||||||
|
r.w.Process(context.Background(), r.spool(t, "a"))
|
||||||
|
|
||||||
|
assert.Equal(t, tt.want, keysAndStates(r.st.lastSteps()))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_AShutdownIsNotRecordedAsAFailedStep(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
r.tr.block = make(chan struct{})
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); r.w.Process(ctx, r.spool(t, "a")) }()
|
||||||
|
require.Eventually(t, func() bool {
|
||||||
|
return strings.Contains(strings.Join(keysAndStates(r.st.lastSteps()), ","), "transcribe:running")
|
||||||
|
}, 2*time.Second, time.Millisecond)
|
||||||
|
|
||||||
|
cancel()
|
||||||
|
<-done
|
||||||
|
|
||||||
|
assert.Contains(t, keysAndStates(r.st.lastSteps()), "transcribe:running",
|
||||||
|
"a shutdown leaves the row for the start-up sweep; it must not pose as a failure of the step")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_TheHeartbeatBeatsWhileAnEngineCallIsStuckAndStopsWhenTheJobEnds(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
r.tr.block = make(chan struct{})
|
||||||
|
r.st.beatDelay = 20 * time.Millisecond
|
||||||
|
w := newProgressWorker(t, r, r.tr, uploads.WithHeartbeat(5*time.Millisecond, time.Hour))
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); w.Process(context.Background(), r.spool(t, "a")) }()
|
||||||
|
|
||||||
|
require.Eventually(t, func() bool { return r.st.beatCount() >= 3 }, 2*time.Second, 5*time.Millisecond,
|
||||||
|
"the page tells a slow engine from a dead service by this heartbeat, so it must not wait on the engine")
|
||||||
|
close(r.tr.block)
|
||||||
|
<-done
|
||||||
|
after := r.st.beatCount()
|
||||||
|
time.Sleep(40 * time.Millisecond)
|
||||||
|
|
||||||
|
assert.Equal(t, after, r.st.beatCount(), "no heartbeat may outlive its job")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWorkerProgress_TheLogNamesEveryPhaseInOrderAndNeverTheTranscriptOrThePath(t *testing.T) {
|
||||||
|
r := newWRig(t, time.Minute)
|
||||||
|
r.tr.result = domain.Transcription{Text: "zebrakaka secret meeting words", Endpoint: "iguana/kb-whisper", AudioSeconds: 3}
|
||||||
|
buf := &syncLog{}
|
||||||
|
w := newProgressWorker(t, r, r.tr, uploads.WithLogger(slog.New(slog.NewTextHandler(buf, nil))))
|
||||||
|
|
||||||
|
w.Process(context.Background(), r.spool(t, "a"))
|
||||||
|
|
||||||
|
out := buf.String()
|
||||||
|
audio := strings.Index(out, "step=audio")
|
||||||
|
transcribe := strings.Index(out, "step=transcribe")
|
||||||
|
save := strings.Index(out, "step=save")
|
||||||
|
require.True(t, audio >= 0 && transcribe > audio && save > transcribe, "phases must be logged in the order they ran:\n%s", out)
|
||||||
|
assert.Contains(t, out, "upload=a", "every line says which upload it is about")
|
||||||
|
assert.NotContains(t, out, "zebrakaka", "no transcript text in the log")
|
||||||
|
assert.NotContains(t, out, r.tmp, "no spool path, which carries the upload's identity, in the log")
|
||||||
|
}
|
||||||
@@ -23,6 +23,11 @@ type wStore struct {
|
|||||||
completed map[string]domain.Transcription
|
completed map[string]domain.Transcription
|
||||||
failed map[string]domain.UploadErrorKind
|
failed map[string]domain.UploadErrorKind
|
||||||
startErr error
|
startErr error
|
||||||
|
saves [][]domain.ProgressStep
|
||||||
|
beats int
|
||||||
|
// beatDelay makes each heartbeat write slow, so one is still in flight when a job
|
||||||
|
// ends -- which is what proves the worker waits for its heartbeat to finish.
|
||||||
|
beatDelay time.Duration
|
||||||
}
|
}
|
||||||
|
|
||||||
func newWStore() *wStore {
|
func newWStore() *wStore {
|
||||||
@@ -50,6 +55,36 @@ func (s *wStore) FailUpload(_ context.Context, _, id string, k domain.UploadErro
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *wStore) SaveProgress(_ context.Context, _, _ string, steps []domain.ProgressStep, _ time.Time) error {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.saves = append(s.saves, append([]domain.ProgressStep(nil), steps...))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *wStore) Heartbeat(context.Context, string, string) error {
|
||||||
|
time.Sleep(s.beatDelay)
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.beats++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *wStore) lastSteps() []domain.ProgressStep {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if len(s.saves) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s.saves[len(s.saves)-1]
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *wStore) beatCount() int {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.beats
|
||||||
|
}
|
||||||
|
|
||||||
func (s *wStore) snapshot() (map[string]domain.Transcription, map[string]domain.UploadErrorKind) {
|
func (s *wStore) snapshot() (map[string]domain.Transcription, map[string]domain.UploadErrorKind) {
|
||||||
s.mu.Lock()
|
s.mu.Lock()
|
||||||
defer s.mu.Unlock()
|
defer s.mu.Unlock()
|
||||||
|
|||||||
+115
-10
@@ -9,6 +9,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
"time"
|
||||||
"unicode/utf8"
|
"unicode/utf8"
|
||||||
|
|
||||||
"github.com/a-h/templ"
|
"github.com/a-h/templ"
|
||||||
@@ -32,7 +33,15 @@ const (
|
|||||||
uploadOverhead = 1 << 20
|
uploadOverhead = 1 << 20
|
||||||
maxTitleRunes = 200
|
maxTitleRunes = 200
|
||||||
// uploadPollEvery is how often an unfinished upload re-asks for its status.
|
// uploadPollEvery is how often an unfinished upload re-asks for its status.
|
||||||
uploadPollEvery = "5s"
|
uploadPollEvery = "2s"
|
||||||
|
// serviceSilentAfter is how long a job may go without any signal (heartbeat or
|
||||||
|
// progress) before the page calls the service lost. The heartbeat is every 10 s, so
|
||||||
|
// 45 s is four missed beats (ADR-035).
|
||||||
|
serviceSilentAfter = 45 * time.Second
|
||||||
|
// countedQuietAfter is how long a COUNTED phase (the deciding one) may go without
|
||||||
|
// advancing before the page says it is quiet. Each decision has a two-minute timeout,
|
||||||
|
// so five minutes is more than two in a row. A phase with no count is never quiet.
|
||||||
|
countedQuietAfter = 5 * time.Minute
|
||||||
)
|
)
|
||||||
|
|
||||||
// uploadsView is what the /uploads page renders.
|
// uploadsView is what the /uploads page renders.
|
||||||
@@ -137,7 +146,7 @@ func (a *App) handleUploadDetail(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
a.render(w, r, UploadPage(*u, a.Chat != nil))
|
a.render(w, r, UploadPage(*u, a.Chat != nil, a.now()))
|
||||||
}
|
}
|
||||||
|
|
||||||
// handleUploadStatus renders the status fragment HTMX polls. Unfinished states
|
// handleUploadStatus renders the status fragment HTMX polls. Unfinished states
|
||||||
@@ -148,7 +157,7 @@ func (a *App) handleUploadStatus(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
a.render(w, r, uploadStatus(*u, a.Chat != nil))
|
a.render(w, r, uploadStatus(*u, a.Chat != nil, a.now()))
|
||||||
}
|
}
|
||||||
|
|
||||||
func (a *App) loadOwnedUpload(w http.ResponseWriter, r *http.Request) (*domain.UploadedTranscript, bool) {
|
func (a *App) loadOwnedUpload(w http.ResponseWriter, r *http.Request) (*domain.UploadedTranscript, bool) {
|
||||||
@@ -274,9 +283,9 @@ func (a *App) uploadChatView(u domain.UploadedTranscript, model string, history
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// maxSpotsShown is how many contested spots the page lists. The stored list holds up to
|
// maxSpotsShown is how many spots to check the page lists. The stored list holds every
|
||||||
// 200 and the counts are exact; a page of two hundred "check this" items is not a page
|
// judged spot and the counts are exact; a page of two hundred "check this" items is not a
|
||||||
// anyone reads, so the rest are counted, not listed.
|
// page anyone reads, so the rest are counted, not listed.
|
||||||
const maxSpotsShown = 50
|
const maxSpotsShown = 50
|
||||||
|
|
||||||
// minuteMark renders where in the recording a spot is, as the nearest minute.
|
// minuteMark renders where in the recording a spot is, as the nearest minute.
|
||||||
@@ -284,15 +293,111 @@ func minuteMark(seconds float64) string {
|
|||||||
return fmt.Sprintf("~%d min", int(math.Round(seconds/60)))
|
return fmt.Sprintf("~%d min", int(math.Round(seconds/60)))
|
||||||
}
|
}
|
||||||
|
|
||||||
// shownSpots is the part of the contested list the page renders.
|
// shownSpots is the part of the stored list the page renders: the spots a reader should
|
||||||
|
// check, changed or uncertain, in order, capped. A spot the model confidently kept is
|
||||||
|
// stored (ADR-035) but is not one to check, so it is not listed.
|
||||||
func shownSpots(r *domain.Reconciliation) []domain.ContestedSpot {
|
func shownSpots(r *domain.Reconciliation) []domain.ContestedSpot {
|
||||||
if len(r.Contested) > maxSpotsShown {
|
var out []domain.ContestedSpot
|
||||||
return r.Contested[:maxSpotsShown]
|
for _, c := range r.Contested {
|
||||||
|
if c.Used == "alt" || c.Unsure {
|
||||||
|
out = append(out, c)
|
||||||
|
if len(out) == maxSpotsShown {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return r.Contested
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
// moreSpots is how many spots to check are not listed, from the exact counts.
|
// moreSpots is how many spots to check are not listed, from the exact counts.
|
||||||
func moreSpots(r *domain.Reconciliation) int {
|
func moreSpots(r *domain.Reconciliation) int {
|
||||||
return max(0, r.Changed+r.Unsure-len(shownSpots(r)))
|
return max(0, r.Changed+r.Unsure-len(shownSpots(r)))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// jobHealth is what the page can honestly say about a job it is still polling.
|
||||||
|
type jobHealth string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// healthRunning: the service is reporting and work is moving, or there is nothing yet
|
||||||
|
// to say otherwise.
|
||||||
|
healthRunning jobHealth = "running"
|
||||||
|
// healthQuiet: the service is alive but a counted phase has not advanced for a while.
|
||||||
|
healthQuiet jobHealth = "quiet"
|
||||||
|
// healthLost: nothing has been heard from the service for too long. It may have
|
||||||
|
// restarted or crashed; the start-up sweep fails the row once it is back.
|
||||||
|
healthLost jobHealth = "lost"
|
||||||
|
)
|
||||||
|
|
||||||
|
// uploadHealth tells running from quiet from lost for an upload being transcribed (the
|
||||||
|
// page calls it for no other state: a queued or finished job has no heartbeat to judge). The
|
||||||
|
// two timestamps mean different things (ADR-035): alive_at is the heartbeat, which says
|
||||||
|
// the service is still reporting; progress_at moves only when work really moved. A
|
||||||
|
// single blocking engine call has no sub-progress, so only a counted phase can be quiet.
|
||||||
|
// silent is how long since the signal that decided it.
|
||||||
|
func uploadHealth(u domain.UploadedTranscript, now time.Time) (health jobHealth, silent time.Duration) {
|
||||||
|
last := u.AliveAt
|
||||||
|
if u.ProgressAt.After(last) {
|
||||||
|
last = u.ProgressAt
|
||||||
|
}
|
||||||
|
if last.IsZero() {
|
||||||
|
return healthRunning, 0
|
||||||
|
}
|
||||||
|
if d := now.Sub(last); d > serviceSilentAfter {
|
||||||
|
return healthLost, d
|
||||||
|
}
|
||||||
|
for _, s := range u.Progress {
|
||||||
|
if s.State == domain.StepRunning && s.Total > 0 && !u.ProgressAt.IsZero() {
|
||||||
|
if d := now.Sub(u.ProgressAt); d > countedQuietAfter {
|
||||||
|
return healthQuiet, d
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return healthRunning, 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// stepDuration renders how long a phase took or has run: "<1s", "42s", "3m 12s", "1h 02m".
|
||||||
|
func stepDuration(d time.Duration) string {
|
||||||
|
switch {
|
||||||
|
case d < time.Second:
|
||||||
|
return "<1s"
|
||||||
|
case d < time.Minute:
|
||||||
|
return fmt.Sprintf("%ds", int(d.Seconds()))
|
||||||
|
case d < time.Hour:
|
||||||
|
return fmt.Sprintf("%dm %02ds", int(d.Minutes()), int(d.Seconds())%60)
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("%dh %02dm", int(d.Hours()), int(d.Minutes())%60)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// stepTime is the time shown beside a phase: how long it has run, or how long it took.
|
||||||
|
func stepTime(s domain.ProgressStep, now time.Time) string {
|
||||||
|
switch s.State {
|
||||||
|
case domain.StepRunning:
|
||||||
|
return stepDuration(now.Sub(s.StartedAt))
|
||||||
|
case domain.StepDone, domain.StepFailed:
|
||||||
|
return stepDuration(s.FinishedAt.Sub(s.StartedAt))
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// stepMark is the glyph at the head of a phase line.
|
||||||
|
func stepMark(s domain.ProgressStep) string {
|
||||||
|
switch s.State {
|
||||||
|
case domain.StepDone:
|
||||||
|
return "✓"
|
||||||
|
case domain.StepFailed:
|
||||||
|
return "✗"
|
||||||
|
case domain.StepSkipped:
|
||||||
|
return "–"
|
||||||
|
}
|
||||||
|
return "·"
|
||||||
|
}
|
||||||
|
|
||||||
|
// minutesAgo is a whole number of minutes for the health notes: "6 min", or "under a
|
||||||
|
// minute" when it is less.
|
||||||
|
func minutesAgo(d time.Duration) string {
|
||||||
|
if m := int(d.Minutes()); m >= 1 {
|
||||||
|
return fmt.Sprintf("%d min", m)
|
||||||
|
}
|
||||||
|
return "under a minute"
|
||||||
|
}
|
||||||
|
|||||||
@@ -45,11 +45,11 @@ templ UploadsPage(v uploadsView) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// UploadPage is one upload's page: its title and the polling status region.
|
// UploadPage is one upload's page: its title and the polling status region.
|
||||||
templ UploadPage(u domain.UploadedTranscript, chatEnabled bool) {
|
templ UploadPage(u domain.UploadedTranscript, chatEnabled bool, now time.Time) {
|
||||||
@Layout("Tapir — " + u.Title) {
|
@Layout("Tapir — " + u.Title) {
|
||||||
<article class="account">
|
<article class="account">
|
||||||
<h1>{ u.Title }</h1>
|
<h1>{ u.Title }</h1>
|
||||||
@uploadStatus(u, chatEnabled)
|
@uploadStatus(u, chatEnabled, now)
|
||||||
<p><a href="/uploads">All recordings</a></p>
|
<p><a href="/uploads">All recordings</a></p>
|
||||||
</article>
|
</article>
|
||||||
}
|
}
|
||||||
@@ -57,7 +57,7 @@ templ UploadPage(u domain.UploadedTranscript, chatEnabled bool) {
|
|||||||
|
|
||||||
// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the
|
// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the
|
||||||
// page re-asks; done and failed render without it, so nothing polls forever.
|
// page re-asks; done and failed render without it, so nothing polls forever.
|
||||||
templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool) {
|
templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool, now time.Time) {
|
||||||
switch u.Status {
|
switch u.Status {
|
||||||
case domain.UploadDone:
|
case domain.UploadDone:
|
||||||
<div class="upload-status upload-done" role="status">
|
<div class="upload-status upload-done" role="status">
|
||||||
@@ -67,6 +67,12 @@ templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool) {
|
|||||||
of audio by <span class="chip">{ u.Endpoint }</span>.
|
of audio by <span class="chip">{ u.Endpoint }</span>.
|
||||||
</p>
|
</p>
|
||||||
@reconciliationNote(u.Reconciliation)
|
@reconciliationNote(u.Reconciliation)
|
||||||
|
if len(u.Progress) > 0 {
|
||||||
|
<details class="what-was-done">
|
||||||
|
<summary>What was done</summary>
|
||||||
|
@progressSteps(u.Progress, now, false)
|
||||||
|
</details>
|
||||||
|
}
|
||||||
if chatEnabled {
|
if chatEnabled {
|
||||||
<p><a class="btn" href={ templ.SafeURL("/uploads/" + u.ID + "/chat") }>Ask about this recording</a></p>
|
<p><a class="btn" href={ templ.SafeURL("/uploads/" + u.ID + "/chat") }>Ask about this recording</a></p>
|
||||||
}
|
}
|
||||||
@@ -79,6 +85,12 @@ templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool) {
|
|||||||
<div class="upload-status upload-failed flash flash-error" role="alert">
|
<div class="upload-status upload-failed flash flash-error" role="alert">
|
||||||
@videoNotice(u.HadVideo)
|
@videoNotice(u.HadVideo)
|
||||||
<p>{ uploadFailureMessage(u.ErrorKind) }</p>
|
<p>{ uploadFailureMessage(u.ErrorKind) }</p>
|
||||||
|
if len(u.Progress) > 0 {
|
||||||
|
<details class="what-was-done" open>
|
||||||
|
<summary>Where it stopped</summary>
|
||||||
|
@progressSteps(u.Progress, now, false)
|
||||||
|
</details>
|
||||||
|
}
|
||||||
</div>
|
</div>
|
||||||
default:
|
default:
|
||||||
<div
|
<div
|
||||||
@@ -90,7 +102,13 @@ templ uploadStatus(u domain.UploadedTranscript, chatEnabled bool) {
|
|||||||
>
|
>
|
||||||
@videoNotice(u.HadVideo)
|
@videoNotice(u.HadVideo)
|
||||||
if u.Status == domain.UploadTranscribing {
|
if u.Status == domain.UploadTranscribing {
|
||||||
<p>Transcribing… a long recording takes a few minutes.</p>
|
{{ health, silent := uploadHealth(u, now) }}
|
||||||
|
if len(u.Progress) > 0 {
|
||||||
|
@progressSteps(u.Progress, now, health != healthLost)
|
||||||
|
@healthNote(health, silent)
|
||||||
|
} else {
|
||||||
|
<p>Transcribing… a long recording takes a few minutes.</p>
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
<p>Queued — waiting for its turn to be transcribed.</p>
|
<p>Queued — waiting for its turn to be transcribed.</p>
|
||||||
}
|
}
|
||||||
@@ -142,13 +160,17 @@ templ reconciliationNote(r *domain.Reconciliation) {
|
|||||||
{ strconv.Itoa(r.Adjudicated) } decided by the model ({ r.Reconciler }),
|
{ strconv.Itoa(r.Adjudicated) } decided by the model ({ r.Reconciler }),
|
||||||
{ strconv.Itoa(r.Unsure) } uncertain.
|
{ strconv.Itoa(r.Unsure) } uncertain.
|
||||||
</p>
|
</p>
|
||||||
if len(r.Contested) > 0 {
|
if len(shownSpots(r)) > 0 {
|
||||||
<details>
|
<details>
|
||||||
<summary>Spots to check against the audio</summary>
|
<summary>Spots to check against the audio</summary>
|
||||||
<ul>
|
<ul>
|
||||||
for _, c := range shownSpots(r) {
|
for _, c := range shownSpots(r) {
|
||||||
<li>
|
<li>
|
||||||
{ minuteMark(c.Seconds) }: { c.Base } / { c.Alt } —
|
{ minuteMark(c.Seconds) }:
|
||||||
|
if c.Before != "" {
|
||||||
|
<span class="muted">…{ c.Before }</span>
|
||||||
|
}
|
||||||
|
{ c.Base } / { c.Alt } —
|
||||||
if c.Unsure {
|
if c.Unsure {
|
||||||
<span class="chip chip-warn">uncertain</span> kept { c.Used }
|
<span class="chip chip-warn">uncertain</span> kept { c.Used }
|
||||||
} else {
|
} else {
|
||||||
@@ -170,3 +192,54 @@ templ reconciliationNote(r *domain.Reconciliation) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// progressSteps is the job's phases as a log that fills in as it runs: finished ones
|
||||||
|
// collapse to a line with how long they took, the running one has a spinner and its
|
||||||
|
// elapsed time, and what has not started is dimmed. live is false when the spinner would
|
||||||
|
// be a lie (the service is lost, or the job is over), so the running phase shows a
|
||||||
|
// warning mark instead. Only a counted phase gets a bar (ADR-035).
|
||||||
|
templ progressSteps(steps []domain.ProgressStep, now time.Time, live bool) {
|
||||||
|
<ol class="steps" aria-label="Progress">
|
||||||
|
for _, s := range steps {
|
||||||
|
<li
|
||||||
|
class={ "step step-" + string(s.State) }
|
||||||
|
if s.State == domain.StepRunning {
|
||||||
|
aria-current="step"
|
||||||
|
}
|
||||||
|
>
|
||||||
|
if s.State == domain.StepRunning && live {
|
||||||
|
<span class="step-mark" aria-hidden="true"><span class="spinner"></span></span>
|
||||||
|
} else if s.State == domain.StepRunning {
|
||||||
|
<span class="step-mark step-mark-warn" aria-hidden="true">!</span>
|
||||||
|
} else {
|
||||||
|
<span class="step-mark" aria-hidden="true">{ stepMark(s) }</span>
|
||||||
|
}
|
||||||
|
<span class="step-label">{ s.Label }</span>
|
||||||
|
if t := stepTime(s, now); t != "" {
|
||||||
|
<span class="step-time">{ t }</span>
|
||||||
|
}
|
||||||
|
if s.State == domain.StepRunning && s.Total > 0 {
|
||||||
|
<progress class="step-bar" value={ strconv.Itoa(s.Done) } max={ strconv.Itoa(s.Total) }></progress>
|
||||||
|
<span class="step-count">{ strconv.Itoa(s.Done) } of { strconv.Itoa(s.Total) }</span>
|
||||||
|
}
|
||||||
|
</li>
|
||||||
|
}
|
||||||
|
</ol>
|
||||||
|
}
|
||||||
|
|
||||||
|
// healthNote says so when the job is not simply running: the service stopped reporting,
|
||||||
|
// or a counted phase has gone quiet. Running says nothing: the spinner does.
|
||||||
|
templ healthNote(h jobHealth, silent time.Duration) {
|
||||||
|
switch h {
|
||||||
|
case healthLost:
|
||||||
|
<p class="step-health step-health-lost" role="alert">
|
||||||
|
The service stopped reporting { minutesAgo(silent) } ago. It may have restarted or crashed.
|
||||||
|
This page keeps checking; if nothing changes, upload the recording again.
|
||||||
|
</p>
|
||||||
|
case healthQuiet:
|
||||||
|
<p class="step-health step-health-quiet">
|
||||||
|
Nothing new for { minutesAgo(silent) }: the model may be slow. The service is still running,
|
||||||
|
and the job gives up on its own if it takes too long.
|
||||||
|
</p>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,244 @@
|
|||||||
|
package web_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/web"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The page's progress display (ADR-035, #42): real phases from the row, and the three
|
||||||
|
// honest states the page can tell apart -- running, quiet (counted phase not advancing)
|
||||||
|
// and lost (the service stopped reporting). Real router, real Postgres, a fixed clock.
|
||||||
|
|
||||||
|
var progressNow = time.Date(2026, 10, 2, 9, 0, 0, 0, time.UTC)
|
||||||
|
|
||||||
|
func fixedClock(app *web.App) { app.Now = func() time.Time { return progressNow } }
|
||||||
|
|
||||||
|
// seedProgress writes the job's phases and the two timestamps onto an existing row.
|
||||||
|
func seedProgress(t *testing.T, id string, steps []domain.ProgressStep, progressAt, aliveAt time.Time) {
|
||||||
|
t.Helper()
|
||||||
|
b, err := json.Marshal(steps)
|
||||||
|
require.NoError(t, err)
|
||||||
|
var alive any
|
||||||
|
if !aliveAt.IsZero() {
|
||||||
|
alive = aliveAt
|
||||||
|
}
|
||||||
|
_, err = rawPool(t).Exec(context.Background(),
|
||||||
|
`UPDATE uploaded_transcripts SET progress = $2, progress_at = $3, alive_at = $4 WHERE id = $1`,
|
||||||
|
id, b, progressAt, alive)
|
||||||
|
require.NoError(t, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func step(key, label string, state domain.StepState, startedAgo, ranFor time.Duration, done, total int) domain.ProgressStep {
|
||||||
|
s := domain.ProgressStep{Key: key, Label: label, State: state, Done: done, Total: total}
|
||||||
|
if state != domain.StepPending {
|
||||||
|
s.StartedAt = progressNow.Add(-startedAgo)
|
||||||
|
}
|
||||||
|
if state == domain.StepDone || state == domain.StepFailed {
|
||||||
|
s.FinishedAt = s.StartedAt.Add(ranFor)
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// twoEngineRun is a job in its second engine: three steps done, one running, three to come.
|
||||||
|
func twoEngineRun() []domain.ProgressStep {
|
||||||
|
return []domain.ProgressStep{
|
||||||
|
step("received", "Received", domain.StepDone, 5*time.Minute, 0, 0, 0),
|
||||||
|
step("audio", "Preparing the audio", domain.StepDone, 5*time.Minute, 40*time.Second, 0, 0),
|
||||||
|
step("engine_a", "Transcribing with iguana/pianissimo", domain.StepDone, 4*time.Minute, 38*time.Second, 0, 0),
|
||||||
|
step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepRunning, 3*time.Minute+12*time.Second, 0, 0, 0),
|
||||||
|
step("compare", "Comparing the two transcripts", domain.StepPending, 0, 0, 0, 0),
|
||||||
|
step("decide", "Deciding the differences", domain.StepPending, 0, 0, 0, 0),
|
||||||
|
step("save", "Saving the transcript", domain.StepPending, 0, 0, 0, 0),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func statusBody(t *testing.T, app *web.App, id string) string {
|
||||||
|
t.Helper()
|
||||||
|
rec := do(t, app, get("/uploads/"+id+"/status"))
|
||||||
|
require.Equal(t, 200, rec.Code)
|
||||||
|
return body(t, rec)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ARunningJobShowsEveryPhaseInOrderWithTheRunningOneMarked(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
seedProgress(t, id, twoEngineRun(), progressNow.Add(-10*time.Second), progressNow.Add(-4*time.Second))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
last := -1
|
||||||
|
for _, label := range []string{
|
||||||
|
"Received", "Preparing the audio", "Transcribing with iguana/pianissimo", "Transcribing with iguana/kb-whisper",
|
||||||
|
"Comparing the two transcripts", "Deciding the differences", "Saving the transcript",
|
||||||
|
} {
|
||||||
|
i := strings.Index(b, label)
|
||||||
|
require.Greater(t, i, last, "phase %q missing or out of order:\n%s", label, b)
|
||||||
|
last = i
|
||||||
|
}
|
||||||
|
assert.Equal(t, 3, strings.Count(b, "step-done"), "three finished phases collapse to a done line")
|
||||||
|
assert.Equal(t, 1, strings.Count(b, `aria-current="step"`), "exactly one phase is the current one")
|
||||||
|
assert.Contains(t, b, "3m 12s", "the running phase shows how long it has run")
|
||||||
|
assert.Contains(t, b, "38s", "a finished phase shows how long it took")
|
||||||
|
assert.Contains(t, b, `hx-trigger="every 2s"`, "an unfinished job re-asks every two seconds")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_OnlyACountedPhaseGetsABarAndItShowsRealCounts(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
steps := twoEngineRun()
|
||||||
|
steps[3] = step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepDone, 3*time.Minute, 2*time.Minute, 0, 0)
|
||||||
|
steps[4] = step("compare", "Comparing the two transcripts", domain.StepDone, 1*time.Minute, time.Second, 0, 0)
|
||||||
|
steps[5] = step("decide", "Deciding the differences", domain.StepRunning, 50*time.Second, 0, 142, 282)
|
||||||
|
seedProgress(t, id, steps, progressNow.Add(-5*time.Second), progressNow.Add(-3*time.Second))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, `<progress`)
|
||||||
|
assert.Contains(t, b, `value="142"`)
|
||||||
|
assert.Contains(t, b, `max="282"`)
|
||||||
|
assert.Contains(t, b, "142 of 282")
|
||||||
|
|
||||||
|
// ...and a phase with no honest count never gets one: no percentage out of thin air.
|
||||||
|
app2, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app2)
|
||||||
|
id2 := seedUpload(t, userID, "m2", "transcribing", "")
|
||||||
|
seedProgress(t, id2, twoEngineRun(), progressNow.Add(-10*time.Second), progressNow.Add(-4*time.Second))
|
||||||
|
assert.NotContains(t, statusBody(t, app2, id2), "<progress", "an engine call is one request: there is nothing to count")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ALongEngineCallWithAFreshHeartbeatIsStillJustRunning(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
steps := twoEngineRun()
|
||||||
|
steps[3] = step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepRunning, 25*time.Minute, 0, 0, 0)
|
||||||
|
seedProgress(t, id, steps, progressNow.Add(-25*time.Minute), progressNow.Add(-3*time.Second))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "25m 00s")
|
||||||
|
assert.NotContains(t, b, "step-health-lost")
|
||||||
|
assert.NotContains(t, b, "step-health-quiet",
|
||||||
|
"a single blocking request has no sub-progress to go quiet about; the heartbeat says the service is alive")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_AHeartbeatThatStoppedMeansTheServiceIsLostAndTheJobIsNotCalledRunning(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
seedProgress(t, id, twoEngineRun(), progressNow.Add(-3*time.Minute), progressNow.Add(-3*time.Minute))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "step-health-lost")
|
||||||
|
assert.Contains(t, b, "stopped reporting")
|
||||||
|
assert.Contains(t, b, "3 min", "how long ago it was last heard from")
|
||||||
|
assert.NotContains(t, b, "spinner", "a lost job must not wear the running spinner")
|
||||||
|
assert.Contains(t, b, "step-mark-warn", "the running phase of a lost job carries a warning mark, in the error colour, not the accent")
|
||||||
|
assert.Contains(t, b, `hx-trigger="every`, "it keeps asking: a restarted service comes back, or the start-up sweep fails the row")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ACountedPhaseThatStopsAdvancingGoesQuietNotLost(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
steps := twoEngineRun()
|
||||||
|
steps[5] = step("decide", "Deciding the differences", domain.StepRunning, 20*time.Minute, 0, 142, 282)
|
||||||
|
seedProgress(t, id, steps, progressNow.Add(-6*time.Minute), progressNow.Add(-3*time.Second))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "step-health-quiet")
|
||||||
|
assert.Contains(t, b, "Nothing new for 6 min")
|
||||||
|
assert.NotContains(t, b, "stopped reporting", "the service is alive; it is the model that is slow")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_BeforeTheFirstHeartbeatAJobIsRunningUntilItHasBeenSilentTooLong(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
fresh := seedUpload(t, userID, "fresh", "transcribing", "")
|
||||||
|
seedProgress(t, fresh, twoEngineRun(), progressNow.Add(-5*time.Second), time.Time{})
|
||||||
|
silent := seedUpload(t, userID, "silent", "transcribing", "")
|
||||||
|
seedProgress(t, silent, twoEngineRun(), progressNow.Add(-2*time.Minute), time.Time{})
|
||||||
|
|
||||||
|
assert.NotContains(t, statusBody(t, app, fresh), "step-health-lost", "the first beat has not been due yet")
|
||||||
|
assert.Contains(t, statusBody(t, app, silent), "step-health-lost", "no beat and no progress for two minutes is a lost service")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_AQueuedJobShowsNoPhasesAndNoLostWarning(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "queued", "")
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "Queued")
|
||||||
|
assert.NotContains(t, b, `class="steps"`)
|
||||||
|
assert.NotContains(t, b, "step-health")
|
||||||
|
assert.Contains(t, b, `hx-trigger="every 2s"`)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_AFailedJobShowsWhichPhaseFailedAndWhatNeverStarted(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "failed", "transcription_failed")
|
||||||
|
steps := twoEngineRun()
|
||||||
|
steps[3] = step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepFailed, 3*time.Minute, time.Minute, 0, 0)
|
||||||
|
seedProgress(t, id, steps, progressNow.Add(-2*time.Minute), progressNow.Add(-2*time.Minute))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "Transcription failed")
|
||||||
|
assert.Contains(t, b, "step-failed")
|
||||||
|
assert.Contains(t, b, "step-pending", "what never started says where the job stopped")
|
||||||
|
assert.NotContains(t, b, "hx-trigger", "a failed job stops polling")
|
||||||
|
assert.NotContains(t, b, "step-health", "a finished job is not 'lost'")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_ADoneJobKeepsItsRecordOfWhatWasDoneAndHowLongEachTook(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "done", "Synthetic transcript text.")
|
||||||
|
steps := []domain.ProgressStep{
|
||||||
|
step("received", "Received", domain.StepDone, 20*time.Minute, 0, 0, 0),
|
||||||
|
step("audio", "Preparing the audio", domain.StepDone, 20*time.Minute, 40*time.Second, 0, 0),
|
||||||
|
step("decide", "Deciding the differences", domain.StepDone, 15*time.Minute, 9*time.Minute+30*time.Second, 282, 282),
|
||||||
|
step("save", "Saving the transcript", domain.StepSkipped, 0, 0, 0, 0),
|
||||||
|
}
|
||||||
|
seedProgress(t, id, steps, progressNow.Add(-time.Minute), progressNow.Add(-time.Minute))
|
||||||
|
|
||||||
|
b := statusBody(t, app, id)
|
||||||
|
|
||||||
|
assert.Contains(t, b, "Synthetic transcript text.")
|
||||||
|
assert.Contains(t, b, "What was done")
|
||||||
|
assert.Contains(t, b, "9m 30s")
|
||||||
|
assert.Contains(t, b, "step-skipped")
|
||||||
|
assert.NotContains(t, b, "hx-trigger", "a finished upload must not poll forever")
|
||||||
|
assert.NotContains(t, b, "step-health", "a finished job is never 'lost' however old its heartbeat")
|
||||||
|
assert.NotContains(t, b, "spinner")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadProgress_TheFullPageRendersTheSameDisplayAsTheFragment(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||||
|
seedProgress(t, id, twoEngineRun(), progressNow.Add(-10*time.Second), progressNow.Add(-4*time.Second))
|
||||||
|
|
||||||
|
rec := do(t, app, get("/uploads/"+id))
|
||||||
|
|
||||||
|
require.Equal(t, 200, rec.Code)
|
||||||
|
b := body(t, rec)
|
||||||
|
assert.Contains(t, b, "Transcribing with iguana/kb-whisper", "the page a user opens, not only the polled fragment, shows the phases")
|
||||||
|
assert.Contains(t, b, "3m 12s")
|
||||||
|
}
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
package web_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TestRenderUploadProgressStatesForReview writes the upload page in each progress state
|
||||||
|
// to $TAPIR_SHOT_DIR as standalone HTML (light and dark), for a screenshot review of the
|
||||||
|
// visual claims tests cannot assert: that it looks like tapir and reads like a log of
|
||||||
|
// phases (ADR-035, #42). Opt-in: it does nothing unless the variable is set. The text is
|
||||||
|
// synthetic. Screenshot with any browser, for example:
|
||||||
|
//
|
||||||
|
// TAPIR_SHOT_DIR=/tmp/shots go test -run TestRenderUploadProgressStatesForReview ./internal/web
|
||||||
|
// chromium --headless --screenshot=/tmp/shots/running.png --window-size=900,900 /tmp/shots/running-light.html
|
||||||
|
func TestRenderUploadProgressStatesForReview(t *testing.T) {
|
||||||
|
dir := os.Getenv("TAPIR_SHOT_DIR")
|
||||||
|
if dir == "" {
|
||||||
|
t.Skip("set TAPIR_SHOT_DIR to render the progress states for a screenshot review")
|
||||||
|
}
|
||||||
|
require.NoError(t, os.MkdirAll(dir, 0o750))
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
fixedClock(app)
|
||||||
|
|
||||||
|
decide := twoEngineRun()
|
||||||
|
decide[3] = step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepDone, 9*time.Minute, 4*time.Minute, 0, 0)
|
||||||
|
decide[4] = step("compare", "Comparing the two transcripts", domain.StepDone, 5*time.Minute, time.Second, 0, 0)
|
||||||
|
decide[5] = step("decide", "Deciding the differences", domain.StepRunning, 4*time.Minute+30*time.Second, 0, 142, 282)
|
||||||
|
|
||||||
|
failed := twoEngineRun()
|
||||||
|
failed[3] = step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepFailed, 3*time.Minute, time.Minute, 0, 0)
|
||||||
|
|
||||||
|
done := []domain.ProgressStep{
|
||||||
|
step("received", "Received", domain.StepDone, 22*time.Minute, 0, 0, 0),
|
||||||
|
step("audio", "Preparing the audio", domain.StepDone, 22*time.Minute, 41*time.Second, 0, 0),
|
||||||
|
step("engine_a", "Transcribing with iguana/pianissimo", domain.StepDone, 21*time.Minute, 2*time.Minute+10*time.Second, 0, 0),
|
||||||
|
step("engine_b", "Transcribing with iguana/kb-whisper", domain.StepDone, 19*time.Minute, 4*time.Minute+5*time.Second, 0, 0),
|
||||||
|
step("compare", "Comparing the two transcripts", domain.StepDone, 15*time.Minute, 2*time.Second, 0, 0),
|
||||||
|
step("decide", "Deciding the differences", domain.StepDone, 15*time.Minute, 14*time.Minute, 282, 282),
|
||||||
|
step("save", "Saving the transcript", domain.StepDone, time.Minute, time.Second, 0, 0),
|
||||||
|
}
|
||||||
|
|
||||||
|
cases := []struct {
|
||||||
|
name, status, content string
|
||||||
|
steps []domain.ProgressStep
|
||||||
|
progressAgo, aliveAgo time.Duration
|
||||||
|
}{
|
||||||
|
{"running", "transcribing", "", twoEngineRun(), 10 * time.Second, 4 * time.Second},
|
||||||
|
{"deciding", "transcribing", "", decide, 3 * time.Second, 3 * time.Second},
|
||||||
|
{"quiet", "transcribing", "", decide, 6 * time.Minute, 4 * time.Second},
|
||||||
|
{"lost", "transcribing", "", twoEngineRun(), 3 * time.Minute, 3 * time.Minute},
|
||||||
|
{"failed", "failed", "transcription_failed", failed, 2 * time.Minute, 2 * time.Minute},
|
||||||
|
{"done", "done", "Synthetic transcript text for the review.", done, time.Minute, time.Minute},
|
||||||
|
{"queued", "queued", "", nil, 0, 0},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
id := seedUpload(t, userID, "Project notes (synthetic)", c.status, c.content)
|
||||||
|
if c.steps != nil {
|
||||||
|
seedProgress(t, id, c.steps, progressNow.Add(-c.progressAgo), progressNow.Add(-c.aliveAgo))
|
||||||
|
}
|
||||||
|
page := body(t, do(t, app, get("/uploads/"+id)))
|
||||||
|
for _, theme := range []string{"light", "dark"} {
|
||||||
|
html := page
|
||||||
|
if theme == "dark" {
|
||||||
|
html = strings.Replace(html, "<html", `<html data-theme="dark"`, 1)
|
||||||
|
}
|
||||||
|
require.NoError(t, os.WriteFile(filepath.Join(dir, c.name+"-"+theme+".html"), []byte(html), 0o600))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
package web_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/stretchr/testify/assert"
|
||||||
|
"github.com/stretchr/testify/require"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The stored judged spots include the ones the model kept as the base (ADR-035), for a
|
||||||
|
// later evaluation. The page lists only the ones a reader should check: changed and
|
||||||
|
// uncertain.
|
||||||
|
|
||||||
|
func TestUploadStatus_OnlyChangedAndUncertainSpotsAreListedNotTheKeptOnes(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
id := seedUpload(t, userID, "m", "done", "Synthetic transcript.")
|
||||||
|
setReconciliation(t, id, `{"status":"reconciled","base":"a","alt":"b","reconciler":"c","disputes":3,"adjudicated":3,"changed":1,"unsure":1,
|
||||||
|
"contested":[
|
||||||
|
{"seconds":60,"base":"CHANGEDBASE","alt":"CHANGEDALT","used":"alt","unsure":false},
|
||||||
|
{"seconds":120,"base":"KEPTBASE","alt":"KEPTALT","used":"base","unsure":false},
|
||||||
|
{"seconds":180,"base":"UNSUREBASE","alt":"UNSUREALT","used":"base","unsure":true}]}`)
|
||||||
|
|
||||||
|
b := body(t, do(t, app, get("/uploads/"+id+"/status")))
|
||||||
|
|
||||||
|
assert.Contains(t, b, "CHANGEDBASE")
|
||||||
|
assert.Contains(t, b, "UNSUREALT")
|
||||||
|
assert.NotContains(t, b, "KEPTBASE", "a spot the model confidently kept is not one for the reader to check")
|
||||||
|
assert.NotContains(t, b, "KEPTALT")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadStatus_TheMoreCountIgnoresKeptSpotsAndMatchesTheListedKind(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
id := seedUpload(t, userID, "m", "done", "Synthetic transcript.")
|
||||||
|
var spots []string
|
||||||
|
for i := 0; i < 60; i++ {
|
||||||
|
spots = append(spots, fmt.Sprintf(`{"seconds":%d,"base":"b%d","alt":"a%d","used":"alt","unsure":false}`, i*10, i, i))
|
||||||
|
spots = append(spots, fmt.Sprintf(`{"seconds":%d,"base":"k%d","alt":"j%d","used":"base","unsure":false}`, i*10+5, i, i))
|
||||||
|
}
|
||||||
|
setReconciliation(t, id, `{"status":"reconciled","base":"a","alt":"b","reconciler":"c","disputes":400,"changed":60,"contested":[`+strings.Join(spots, ",")+`]}`)
|
||||||
|
|
||||||
|
b := body(t, do(t, app, get("/uploads/"+id+"/status")))
|
||||||
|
|
||||||
|
assert.Contains(t, b, "10 more", "60 changed, 50 shown: the 60 kept ones do not count")
|
||||||
|
assert.NotContains(t, b, "k0 / j0", "a kept spot is not rendered even when it is among the first stored")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadStatus_WhenNothingNeedsCheckingThereIsNoSpotsSection(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
id := seedUpload(t, userID, "m", "done", "Synthetic transcript.")
|
||||||
|
setReconciliation(t, id, `{"status":"reconciled","base":"a","alt":"b","reconciler":"c","disputes":1,"adjudicated":1,
|
||||||
|
"contested":[{"seconds":60,"base":"KEPTBASE","alt":"KEPTALT","used":"base","unsure":false}]}`)
|
||||||
|
|
||||||
|
b := body(t, do(t, app, get("/uploads/"+id+"/status")))
|
||||||
|
|
||||||
|
require.Contains(t, b, "Reconciled from two transcripts")
|
||||||
|
assert.NotContains(t, b, "Spots to check", "an empty list would invite checking nothing")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadStatus_ASpotShowsTheWordsJustBeforeIt(t *testing.T) {
|
||||||
|
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||||
|
id := seedUpload(t, userID, "m", "done", "Synthetic transcript.")
|
||||||
|
setReconciliation(t, id, `{"status":"reconciled","base":"a","alt":"b","reconciler":"c","disputes":1,"changed":1,
|
||||||
|
"contested":[{"seconds":2640,"base":"special","alt":"CAMT","used":"alt","unsure":false,"before":"vi skickar in ett"}]}`)
|
||||||
|
|
||||||
|
b := body(t, do(t, app, get("/uploads/"+id+"/status")))
|
||||||
|
|
||||||
|
assert.Contains(t, b, "vi skickar in ett", "the words before the spot are how a reader finds it in the audio")
|
||||||
|
assert.Contains(t, b, "special")
|
||||||
|
assert.Contains(t, b, "CAMT")
|
||||||
|
}
|
||||||
+446
-115
@@ -176,7 +176,7 @@ func UploadsPage(v uploadsView) templ.Component {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// UploadPage is one upload's page: its title and the polling status region.
|
// UploadPage is one upload's page: its title and the polling status region.
|
||||||
func UploadPage(u domain.UploadedTranscript, chatEnabled bool) templ.Component {
|
func UploadPage(u domain.UploadedTranscript, chatEnabled bool, now time.Time) templ.Component {
|
||||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||||
@@ -226,7 +226,7 @@ func UploadPage(u domain.UploadedTranscript, chatEnabled bool) templ.Component {
|
|||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = uploadStatus(u, chatEnabled).Render(ctx, templ_7745c5c3_Buffer)
|
templ_7745c5c3_Err = uploadStatus(u, chatEnabled, now).Render(ctx, templ_7745c5c3_Buffer)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -246,7 +246,7 @@ func UploadPage(u domain.UploadedTranscript, chatEnabled bool) templ.Component {
|
|||||||
|
|
||||||
// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the
|
// uploadStatus is the region HTMX polls. Unfinished states carry hx-trigger so the
|
||||||
// page re-asks; done and failed render without it, so nothing polls forever.
|
// page re-asks; done and failed render without it, so nothing polls forever.
|
||||||
func uploadStatus(u domain.UploadedTranscript, chatEnabled bool) templ.Component {
|
func uploadStatus(u domain.UploadedTranscript, chatEnabled bool, now time.Time) templ.Component {
|
||||||
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||||
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||||
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||||
@@ -311,44 +311,58 @@ func uploadStatus(u domain.UploadedTranscript, chatEnabled bool) templ.Component
|
|||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
|
if len(u.Progress) > 0 {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "<details class=\"what-was-done\"><summary>What was done</summary>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = progressSteps(u.Progress, now, false).Render(ctx, templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "</details> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
if chatEnabled {
|
if chatEnabled {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 21, "<p><a class=\"btn\" href=\"")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "<p><a class=\"btn\" href=\"")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var15 templ.SafeURL
|
var templ_7745c5c3_Var15 templ.SafeURL
|
||||||
templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinURLErrs(templ.SafeURL("/uploads/" + u.ID + "/chat"))
|
templ_7745c5c3_Var15, templ_7745c5c3_Err = templ.JoinURLErrs(templ.SafeURL("/uploads/" + u.ID + "/chat"))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 71, Col: 73}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 77, Col: 73}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var15))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 22, "\">Ask about this recording</a></p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "\">Ask about this recording</a></p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 23, "<details open><summary>Transcript</summary><pre class=\"transcript\">")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 25, "<details open><summary>Transcript</summary><pre class=\"transcript\">")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var16 string
|
var templ_7745c5c3_Var16 string
|
||||||
templ_7745c5c3_Var16, templ_7745c5c3_Err = templ.JoinStringErrs(u.Content)
|
templ_7745c5c3_Var16, templ_7745c5c3_Err = templ.JoinStringErrs(u.Content)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 75, Col: 40}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 81, Col: 40}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var16))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var16))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 24, "</pre></details></div>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "</pre></details></div>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
case domain.UploadFailed:
|
case domain.UploadFailed:
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 25, "<div class=\"upload-status upload-failed flash flash-error\" role=\"alert\">")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "<div class=\"upload-status upload-failed flash flash-error\" role=\"alert\">")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -356,51 +370,69 @@ func uploadStatus(u domain.UploadedTranscript, chatEnabled bool) templ.Component
|
|||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 26, "<p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "<p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var17 string
|
var templ_7745c5c3_Var17 string
|
||||||
templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.JoinStringErrs(uploadFailureMessage(u.ErrorKind))
|
templ_7745c5c3_Var17, templ_7745c5c3_Err = templ.JoinStringErrs(uploadFailureMessage(u.ErrorKind))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 81, Col: 42}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 87, Col: 42}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var17))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var17))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 27, "</p></div>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 29, "</p>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
if len(u.Progress) > 0 {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 30, "<details class=\"what-was-done\" open><summary>Where it stopped</summary>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = progressSteps(u.Progress, now, false).Render(ctx, templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 31, "</details>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 32, "</div>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
default:
|
default:
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 28, "<div class=\"upload-status upload-working\" role=\"status\" hx-get=\"")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 33, "<div class=\"upload-status upload-working\" role=\"status\" hx-get=\"")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var18 string
|
var templ_7745c5c3_Var18 string
|
||||||
templ_7745c5c3_Var18, templ_7745c5c3_Err = templ.ResolveAttributeValue("/uploads/" + u.ID + "/status")
|
templ_7745c5c3_Var18, templ_7745c5c3_Err = templ.ResolveAttributeValue("/uploads/" + u.ID + "/status")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 87, Col: 43}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 99, Col: 43}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var18)
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var18)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 29, "\" hx-trigger=\"")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 34, "\" hx-trigger=\"")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var19 string
|
var templ_7745c5c3_Var19 string
|
||||||
templ_7745c5c3_Var19, templ_7745c5c3_Err = templ.ResolveAttributeValue("every " + uploadPollEvery)
|
templ_7745c5c3_Var19, templ_7745c5c3_Err = templ.ResolveAttributeValue("every " + uploadPollEvery)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 88, Col: 43}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 100, Col: 43}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var19)
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var19)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 30, "\" hx-swap=\"outerHTML\">")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 35, "\" hx-swap=\"outerHTML\">")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -409,17 +441,33 @@ func uploadStatus(u domain.UploadedTranscript, chatEnabled bool) templ.Component
|
|||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
if u.Status == domain.UploadTranscribing {
|
if u.Status == domain.UploadTranscribing {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 31, "<p>Transcribing… a long recording takes a few minutes.</p>")
|
health, silent := uploadHealth(u, now)
|
||||||
if templ_7745c5c3_Err != nil {
|
if len(u.Progress) > 0 {
|
||||||
return templ_7745c5c3_Err
|
templ_7745c5c3_Err = progressSteps(u.Progress, now, health != healthLost).Render(ctx, templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 36, " ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = healthNote(health, silent).Render(ctx, templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 37, "<p>Transcribing… a long recording takes a few minutes.</p>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 32, "<p>Queued — waiting for its turn to be transcribed.</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 38, "<p>Queued — waiting for its turn to be transcribed.</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 33, "</div>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 39, "</div>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -463,20 +511,20 @@ func UploadChatPage(u domain.UploadedTranscript, v chatView) templ.Component {
|
|||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
ctx = templ.InitializeContext(ctx)
|
ctx = templ.InitializeContext(ctx)
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 34, "<article class=\"account\"><h1>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 40, "<article class=\"account\"><h1>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var22 string
|
var templ_7745c5c3_Var22 string
|
||||||
templ_7745c5c3_Var22, templ_7745c5c3_Err = templ.JoinStringErrs(u.Title)
|
templ_7745c5c3_Var22, templ_7745c5c3_Err = templ.JoinStringErrs(u.Title)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 106, Col: 16}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 124, Col: 16}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var22))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var22))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 35, "</h1><section id=\"chat-section\" class=\"chat-dock chat-dock-open\"><h2 class=\"chat-heading\">Ask about this recording</h2><p class=\"chat-scope muted\">Answers come only from this recording's transcript, which is private to you.</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 41, "</h1><section id=\"chat-section\" class=\"chat-dock chat-dock-open\"><h2 class=\"chat-heading\">Ask about this recording</h2><p class=\"chat-scope muted\">Answers come only from this recording's transcript, which is private to you.</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -484,20 +532,20 @@ func UploadChatPage(u domain.UploadedTranscript, v chatView) templ.Component {
|
|||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 36, "</section><p><a href=\"")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 42, "</section><p><a href=\"")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var23 templ.SafeURL
|
var templ_7745c5c3_Var23 templ.SafeURL
|
||||||
templ_7745c5c3_Var23, templ_7745c5c3_Err = templ.JoinURLErrs(templ.SafeURL("/uploads/" + u.ID))
|
templ_7745c5c3_Var23, templ_7745c5c3_Err = templ.JoinURLErrs(templ.SafeURL("/uploads/" + u.ID))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 112, Col: 49}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 130, Col: 49}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var23))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var23))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 37, "\">Back to the transcript</a></p></article>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 43, "\">Back to the transcript</a></p></article>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -537,7 +585,7 @@ func videoNotice(hadVideo bool) templ.Component {
|
|||||||
}
|
}
|
||||||
ctx = templ.ClearChildren(ctx)
|
ctx = templ.ClearChildren(ctx)
|
||||||
if hadVideo {
|
if hadVideo {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 38, "<p class=\"muted video-notice\">This file contained a video track. Only its audio was used; the video was discarded and nothing from it was kept.</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 44, "<p class=\"muted video-notice\">This file contained a video track. Only its audio was used; the video was discarded and nothing from it was kept.</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -574,262 +622,281 @@ func reconciliationNote(r *domain.Reconciliation) templ.Component {
|
|||||||
if r != nil {
|
if r != nil {
|
||||||
switch r.Status {
|
switch r.Status {
|
||||||
case domain.ReconcileDone:
|
case domain.ReconcileDone:
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 39, "<div class=\"reconciliation\"><p>Reconciled from two transcripts (")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 45, "<div class=\"reconciliation\"><p>Reconciled from two transcripts (")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var26 string
|
var templ_7745c5c3_Var26 string
|
||||||
templ_7745c5c3_Var26, templ_7745c5c3_Err = templ.JoinStringErrs(r.Base)
|
templ_7745c5c3_Var26, templ_7745c5c3_Err = templ.JoinStringErrs(r.Base)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 139, Col: 47}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 157, Col: 47}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var26))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var26))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 40, " and ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 46, " and ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var27 string
|
var templ_7745c5c3_Var27 string
|
||||||
templ_7745c5c3_Var27, templ_7745c5c3_Err = templ.JoinStringErrs(r.Alt)
|
templ_7745c5c3_Var27, templ_7745c5c3_Err = templ.JoinStringErrs(r.Alt)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 139, Col: 61}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 157, Col: 61}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var27))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var27))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 41, "): ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 47, "): ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var28 string
|
var templ_7745c5c3_Var28 string
|
||||||
templ_7745c5c3_Var28, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Disputes))
|
templ_7745c5c3_Var28, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Disputes))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 140, Col: 32}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 158, Col: 32}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var28))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var28))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 42, " differences, ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 48, " differences, ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var29 string
|
var templ_7745c5c3_Var29 string
|
||||||
templ_7745c5c3_Var29, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.AutoResolved))
|
templ_7745c5c3_Var29, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.AutoResolved))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 141, Col: 36}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 159, Col: 36}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var29))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var29))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 43, " settled automatically, ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 49, " settled automatically, ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var30 string
|
var templ_7745c5c3_Var30 string
|
||||||
templ_7745c5c3_Var30, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Adjudicated))
|
templ_7745c5c3_Var30, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Adjudicated))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 142, Col: 35}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 160, Col: 35}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var30))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var30))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 44, " decided by the model (")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 50, " decided by the model (")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var31 string
|
var templ_7745c5c3_Var31 string
|
||||||
templ_7745c5c3_Var31, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reconciler)
|
templ_7745c5c3_Var31, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reconciler)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 142, Col: 74}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 160, Col: 74}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var31))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var31))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 45, "), ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 51, "), ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var32 string
|
var templ_7745c5c3_Var32 string
|
||||||
templ_7745c5c3_Var32, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Unsure))
|
templ_7745c5c3_Var32, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(r.Unsure))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 143, Col: 30}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 161, Col: 30}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var32))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var32))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 46, " uncertain.</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 52, " uncertain.</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
if len(r.Contested) > 0 {
|
if len(shownSpots(r)) > 0 {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 47, "<details><summary>Spots to check against the audio</summary><ul>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 53, "<details><summary>Spots to check against the audio</summary><ul>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
for _, c := range shownSpots(r) {
|
for _, c := range shownSpots(r) {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 48, "<li>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 54, "<li>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var33 string
|
var templ_7745c5c3_Var33 string
|
||||||
templ_7745c5c3_Var33, templ_7745c5c3_Err = templ.JoinStringErrs(minuteMark(c.Seconds))
|
templ_7745c5c3_Var33, templ_7745c5c3_Err = templ.JoinStringErrs(minuteMark(c.Seconds))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 151, Col: 33}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 169, Col: 33}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var33))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var33))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 49, ": ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 55, ": ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var34 string
|
if c.Before != "" {
|
||||||
templ_7745c5c3_Var34, templ_7745c5c3_Err = templ.JoinStringErrs(c.Base)
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 56, "<span class=\"muted\">…")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 151, Col: 45}
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var34))
|
var templ_7745c5c3_Var34 string
|
||||||
if templ_7745c5c3_Err != nil {
|
templ_7745c5c3_Var34, templ_7745c5c3_Err = templ.JoinStringErrs(c.Before)
|
||||||
return templ_7745c5c3_Err
|
if templ_7745c5c3_Err != nil {
|
||||||
}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 171, Col: 44}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 50, " / ")
|
}
|
||||||
if templ_7745c5c3_Err != nil {
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var34))
|
||||||
return templ_7745c5c3_Err
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 57, "</span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var35 string
|
var templ_7745c5c3_Var35 string
|
||||||
templ_7745c5c3_Var35, templ_7745c5c3_Err = templ.JoinStringErrs(c.Alt)
|
templ_7745c5c3_Var35, templ_7745c5c3_Err = templ.JoinStringErrs(c.Base)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 151, Col: 57}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 173, Col: 18}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var35))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var35))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 51, " — ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 58, " / ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var36 string
|
||||||
|
templ_7745c5c3_Var36, templ_7745c5c3_Err = templ.JoinStringErrs(c.Alt)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 173, Col: 30}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var36))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 59, " — ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
if c.Unsure {
|
if c.Unsure {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 52, "<span class=\"chip chip-warn\">uncertain</span> kept ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 60, "<span class=\"chip chip-warn\">uncertain</span> kept ")
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
var templ_7745c5c3_Var36 string
|
|
||||||
templ_7745c5c3_Var36, templ_7745c5c3_Err = templ.JoinStringErrs(c.Used)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 153, Col: 70}
|
|
||||||
}
|
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var36))
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 53, "<span class=\"chip\">changed</span> used ")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var37 string
|
var templ_7745c5c3_Var37 string
|
||||||
templ_7745c5c3_Var37, templ_7745c5c3_Err = templ.JoinStringErrs(c.Used)
|
templ_7745c5c3_Var37, templ_7745c5c3_Err = templ.JoinStringErrs(c.Used)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 155, Col: 58}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 175, Col: 70}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var37))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var37))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
|
} else {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 61, "<span class=\"chip\">changed</span> used ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var38 string
|
||||||
|
templ_7745c5c3_Var38, templ_7745c5c3_Err = templ.JoinStringErrs(c.Used)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 177, Col: 58}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var38))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 54, "</li>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 62, "</li>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 55, "</ul>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 63, "</ul>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
if moreSpots(r) > 0 {
|
if moreSpots(r) > 0 {
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 56, "<p class=\"muted\">and ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 64, "<p class=\"muted\">and ")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var38 string
|
var templ_7745c5c3_Var39 string
|
||||||
templ_7745c5c3_Var38, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(moreSpots(r)))
|
templ_7745c5c3_Var39, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(moreSpots(r)))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 161, Col: 57}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 183, Col: 57}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var38))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var39))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 57, " more not listed</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 65, " more not listed</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 58, "</details>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 66, "</details>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 59, "</div>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 67, "</div>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
case domain.ReconcileSingle:
|
case domain.ReconcileSingle:
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 60, "<p class=\"muted reconciliation\">Only one engine produced this transcript (")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 68, "<p class=\"muted reconciliation\">Only one engine produced this transcript (")
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
var templ_7745c5c3_Var39 string
|
|
||||||
templ_7745c5c3_Var39, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reason)
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 167, Col: 88}
|
|
||||||
}
|
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var39))
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 61, "), so there was no second opinion to reconcile it with.</p>")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
|
||||||
return templ_7745c5c3_Err
|
|
||||||
}
|
|
||||||
case domain.ReconcileSkipped:
|
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 62, "<p class=\"muted reconciliation\">Two transcripts were made but could not be reconciled (")
|
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var40 string
|
var templ_7745c5c3_Var40 string
|
||||||
templ_7745c5c3_Var40, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reason)
|
templ_7745c5c3_Var40, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reason)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 169, Col: 101}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 189, Col: 88}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var40))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var40))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 63, "); the text shown is ")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 69, "), so there was no second opinion to reconcile it with.</p>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
case domain.ReconcileSkipped:
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 70, "<p class=\"muted reconciliation\">Two transcripts were made but could not be reconciled (")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
var templ_7745c5c3_Var41 string
|
var templ_7745c5c3_Var41 string
|
||||||
templ_7745c5c3_Var41, templ_7745c5c3_Err = templ.JoinStringErrs(r.Base)
|
templ_7745c5c3_Var41, templ_7745c5c3_Err = templ.JoinStringErrs(r.Reason)
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 169, Col: 132}
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 191, Col: 101}
|
||||||
}
|
}
|
||||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var41))
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var41))
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 64, "'s, unreconciled.</p>")
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 71, "); the text shown is ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var42 string
|
||||||
|
templ_7745c5c3_Var42, templ_7745c5c3_Err = templ.JoinStringErrs(r.Base)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 191, Col: 132}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var42))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 72, "'s, unreconciled.</p>")
|
||||||
if templ_7745c5c3_Err != nil {
|
if templ_7745c5c3_Err != nil {
|
||||||
return templ_7745c5c3_Err
|
return templ_7745c5c3_Err
|
||||||
}
|
}
|
||||||
@@ -839,4 +906,268 @@ func reconciliationNote(r *domain.Reconciliation) templ.Component {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// progressSteps is the job's phases as a log that fills in as it runs: finished ones
|
||||||
|
// collapse to a line with how long they took, the running one has a spinner and its
|
||||||
|
// elapsed time, and what has not started is dimmed. live is false when the spinner would
|
||||||
|
// be a lie (the service is lost, or the job is over), so the running phase shows a
|
||||||
|
// warning mark instead. Only a counted phase gets a bar (ADR-035).
|
||||||
|
func progressSteps(steps []domain.ProgressStep, now time.Time, live bool) templ.Component {
|
||||||
|
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||||
|
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||||
|
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||||
|
return templ_7745c5c3_CtxErr
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||||
|
if !templ_7745c5c3_IsBuffer {
|
||||||
|
defer func() {
|
||||||
|
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err == nil {
|
||||||
|
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
ctx = templ.InitializeContext(ctx)
|
||||||
|
templ_7745c5c3_Var43 := templ.GetChildren(ctx)
|
||||||
|
if templ_7745c5c3_Var43 == nil {
|
||||||
|
templ_7745c5c3_Var43 = templ.NopComponent
|
||||||
|
}
|
||||||
|
ctx = templ.ClearChildren(ctx)
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 73, "<ol class=\"steps\" aria-label=\"Progress\">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
for _, s := range steps {
|
||||||
|
templ_7745c5c3_Var44 := []any{"step step-" + string(s.State)}
|
||||||
|
templ_7745c5c3_Err = templ.RenderCSSItems(ctx, templ_7745c5c3_Buffer, templ_7745c5c3_Var44...)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 74, "<li class=\"")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var45 string
|
||||||
|
templ_7745c5c3_Var45, templ_7745c5c3_Err = templ.ResolveAttributeValue(templ.CSSClasses(templ_7745c5c3_Var44).String())
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 1, Col: 0}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var45)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 75, "\"")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
if s.State == domain.StepRunning {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 76, " aria-current=\"step\"")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 77, ">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
if s.State == domain.StepRunning && live {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 78, "<span class=\"step-mark\" aria-hidden=\"true\"><span class=\"spinner\"></span></span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
} else if s.State == domain.StepRunning {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 79, "<span class=\"step-mark step-mark-warn\" aria-hidden=\"true\">!</span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 80, "<span class=\"step-mark\" aria-hidden=\"true\">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var46 string
|
||||||
|
templ_7745c5c3_Var46, templ_7745c5c3_Err = templ.JoinStringErrs(stepMark(s))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 215, Col: 61}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var46))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 81, "</span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 82, "<span class=\"step-label\">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var47 string
|
||||||
|
templ_7745c5c3_Var47, templ_7745c5c3_Err = templ.JoinStringErrs(s.Label)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 217, Col: 38}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var47))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 83, "</span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
if t := stepTime(s, now); t != "" {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 84, "<span class=\"step-time\">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var48 string
|
||||||
|
templ_7745c5c3_Var48, templ_7745c5c3_Err = templ.JoinStringErrs(t)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 219, Col: 32}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var48))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 85, "</span> ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.State == domain.StepRunning && s.Total > 0 {
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 86, "<progress class=\"step-bar\" value=\"")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var49 string
|
||||||
|
templ_7745c5c3_Var49, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Done))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 222, Col: 60}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var49)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 87, "\" max=\"")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var50 string
|
||||||
|
templ_7745c5c3_Var50, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Total))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 222, Col: 90}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var50)
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 88, "\"></progress> <span class=\"step-count\">")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var51 string
|
||||||
|
templ_7745c5c3_Var51, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Done))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 223, Col: 52}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var51))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 89, " of ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var52 string
|
||||||
|
templ_7745c5c3_Var52, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Total))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 223, Col: 81}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var52))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 90, "</span>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 91, "</li>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 92, "</ol>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// healthNote says so when the job is not simply running: the service stopped reporting,
|
||||||
|
// or a counted phase has gone quiet. Running says nothing: the spinner does.
|
||||||
|
func healthNote(h jobHealth, silent time.Duration) templ.Component {
|
||||||
|
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
|
||||||
|
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
|
||||||
|
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
|
||||||
|
return templ_7745c5c3_CtxErr
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
|
||||||
|
if !templ_7745c5c3_IsBuffer {
|
||||||
|
defer func() {
|
||||||
|
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
|
||||||
|
if templ_7745c5c3_Err == nil {
|
||||||
|
templ_7745c5c3_Err = templ_7745c5c3_BufErr
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
ctx = templ.InitializeContext(ctx)
|
||||||
|
templ_7745c5c3_Var53 := templ.GetChildren(ctx)
|
||||||
|
if templ_7745c5c3_Var53 == nil {
|
||||||
|
templ_7745c5c3_Var53 = templ.NopComponent
|
||||||
|
}
|
||||||
|
ctx = templ.ClearChildren(ctx)
|
||||||
|
switch h {
|
||||||
|
case healthLost:
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 93, "<p class=\"step-health step-health-lost\" role=\"alert\">The service stopped reporting ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var54 string
|
||||||
|
templ_7745c5c3_Var54, templ_7745c5c3_Err = templ.JoinStringErrs(minutesAgo(silent))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 236, Col: 54}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var54))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 94, " ago. It may have restarted or crashed. This page keeps checking; if nothing changes, upload the recording again.</p>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
case healthQuiet:
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 95, "<p class=\"step-health step-health-quiet\">Nothing new for ")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
var templ_7745c5c3_Var55 string
|
||||||
|
templ_7745c5c3_Var55, templ_7745c5c3_Err = templ.JoinStringErrs(minutesAgo(silent))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 241, Col: 40}
|
||||||
|
}
|
||||||
|
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var55))
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 96, ": the model may be slow. The service is still running, and the job gives up on its own if it takes too long.</p>")
|
||||||
|
if templ_7745c5c3_Err != nil {
|
||||||
|
return templ_7745c5c3_Err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
var _ = templruntime.GeneratedTemplate
|
var _ = templruntime.GeneratedTemplate
|
||||||
|
|||||||
@@ -765,6 +765,31 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
|
|||||||
.flash-success { background: var(--accent-weak); color: var(--accent); border-color: var(--accent); }
|
.flash-success { background: var(--accent-weak); color: var(--accent); border-color: var(--accent); }
|
||||||
.flash-error { background: var(--err-bg); color: var(--err-fg); border-color: var(--err-line); }
|
.flash-error { background: var(--err-bg); color: var(--err-fg); border-color: var(--err-line); }
|
||||||
|
|
||||||
|
/* upload progress (ADR-035): a log of phases that fills in as the job runs */
|
||||||
|
.steps { list-style: none; margin: var(--s3) 0; padding: 0; font-family: var(--mono); font-size: .86rem; }
|
||||||
|
.step { display: flex; align-items: baseline; gap: var(--s2); padding: .2rem 0; color: var(--muted); flex-wrap: wrap; }
|
||||||
|
.step-mark { flex: none; width: 1.1rem; text-align: center; }
|
||||||
|
.step-label { flex: 1 1 auto; }
|
||||||
|
.step-time { margin-left: auto; color: var(--muted); font-weight: 400; font-variant-numeric: tabular-nums; }
|
||||||
|
.step-done { color: var(--fg); }
|
||||||
|
.step-done .step-mark { color: var(--mint); }
|
||||||
|
.step-running { color: var(--fg); font-weight: 600; }
|
||||||
|
.step-running .step-mark { color: var(--accent); }
|
||||||
|
.step-failed { color: var(--err-fg); }
|
||||||
|
.step-skipped, .step-pending { color: var(--muted); }
|
||||||
|
.step-skipped { opacity: .7; }
|
||||||
|
.step-pending { opacity: .75; }
|
||||||
|
.step-bar { flex: none; width: calc(100% - 1.6rem); margin: 0 0 0 1.6rem; height: .5rem; accent-color: var(--accent); }
|
||||||
|
.step-count { flex: none; width: calc(100% - 1.6rem); margin-left: 1.6rem; font-weight: 400; color: var(--muted); }
|
||||||
|
.step-running .step-mark-warn { color: var(--err-fg); }
|
||||||
|
.spinner { display: inline-block; vertical-align: -.1em; width: .8rem; height: .8rem; box-sizing: border-box; border: 2px solid var(--line); border-top-color: var(--accent); border-radius: 50%; animation: tapir-spin 1s linear infinite; }
|
||||||
|
@keyframes tapir-spin { to { transform: rotate(360deg); } }
|
||||||
|
@media (prefers-reduced-motion: reduce) { .spinner { animation: none; border-color: var(--accent); } }
|
||||||
|
.step-health { margin: var(--s2) 0; padding: var(--s2) var(--s3); border-radius: var(--radius); font-size: .9rem; border: 1px solid var(--line); }
|
||||||
|
.step-health-lost { background: var(--err-bg); color: var(--err-fg); border-color: var(--err-line); }
|
||||||
|
.step-health-quiet { background: var(--accent-weak); color: var(--fg); border-color: var(--accent); }
|
||||||
|
.what-was-done summary { cursor: pointer; color: var(--muted); }
|
||||||
|
|
||||||
/* htmx loading feedback */
|
/* htmx loading feedback */
|
||||||
.htmx-indicator { opacity: 0; transition: opacity .2s; color: var(--muted); font-size: .8rem; }
|
.htmx-indicator { opacity: 0; transition: opacity .2s; color: var(--muted); font-size: .8rem; }
|
||||||
.htmx-request .htmx-indicator, .htmx-request.htmx-indicator { opacity: 1; }
|
.htmx-request .htmx-indicator, .htmx-request.htmx-indicator { opacity: 1; }
|
||||||
|
|||||||
@@ -134,6 +134,19 @@ var scenarioCoverage = map[string]string{
|
|||||||
"A cancelled job never reaches the cloud": "TestEnsemble_ACancelledJobNeverReachesTheCloud",
|
"A cancelled job never reaches the cloud": "TestEnsemble_ACancelledJobNeverReachesTheCloud",
|
||||||
"A single-engine transcript claims no second opinion": "TestUploadStatus_ASingleEngineTranscriptDoesNotClaimASecondOpinion",
|
"A single-engine transcript claims no second opinion": "TestUploadStatus_ASingleEngineTranscriptDoesNotClaimASecondOpinion",
|
||||||
|
|
||||||
|
// audio_upload.feature, ADR-035
|
||||||
|
"I can see which phase my upload is in": "TestUploadProgress_ARunningJobShowsEveryPhaseInOrderWithTheRunningOneMarked",
|
||||||
|
"A bar appears only where the count is real": "TestUploadProgress_OnlyACountedPhaseGetsABarAndItShowsRealCounts",
|
||||||
|
"A slow engine call is not mistaken for a dead service": "TestUploadProgress_ALongEngineCallWithAFreshHeartbeatIsStillJustRunning",
|
||||||
|
"A service that stopped reporting is called lost": "TestUploadProgress_AHeartbeatThatStoppedMeansTheServiceIsLostAndTheJobIsNotCalledRunning",
|
||||||
|
"A model that stops advancing is called quiet": "TestUploadProgress_ACountedPhaseThatStopsAdvancingGoesQuietNotLost",
|
||||||
|
"A failed job shows where it stopped": "TestUploadProgress_AFailedJobShowsWhichPhaseFailedAndWhatNeverStarted",
|
||||||
|
"A finished job keeps its record of what was done": "TestUploadProgress_ADoneJobKeepsItsRecordOfWhatWasDoneAndHowLongEachTook",
|
||||||
|
"The heartbeat runs on its own while an engine call is stuck": "TestWorkerProgress_TheHeartbeatBeatsWhileAnEngineCallIsStuckAndStopsWhenTheJobEnds",
|
||||||
|
"The log names every phase and never the transcript": "TestWorkerProgress_TheLogNamesEveryPhaseInOrderAndNeverTheTranscriptOrThePath",
|
||||||
|
"Every judged spot is stored but only the ones to check are listed": "TestUploadStatus_OnlyChangedAndUncertainSpotsAreListedNotTheKeptOnes",
|
||||||
|
"The saved segments carry the merged words": "TestReconcileSegments_AChangedWordReplacesTheBaseWordInItsOwnSegmentOnly",
|
||||||
|
|
||||||
// summarize_new_video.feature
|
// summarize_new_video.feature
|
||||||
"A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered",
|
"A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered",
|
||||||
"A subscribed channel posts a video with no usable transcript": "TestVideoWithNoTranscriptIsSkipped",
|
"A subscribed channel posts a video with no usable transcript": "TestVideoWithNoTranscriptIsSkipped",
|
||||||
|
|||||||
Reference in New Issue
Block a user