Compare commits
33
Commits
+21
-8
@@ -1735,17 +1735,30 @@ wording while only `content` is merged.
|
||||
`network`, `error`), never its text, which can hold a provider's response body, the spool path or
|
||||
a word from the transcript (`internal/errkind`). The stored reconcile reason and an engine's
|
||||
failure reason are kinds; the page no longer shows provider text.
|
||||
- **Every local engine call has its own deadline** (`TAPIR_TRANSCRIBE_ENGINE_TIMEOUT`, default 25
|
||||
minutes), so a wedged engine fails while the job lives and the fallback can run; the job's own
|
||||
- **Every local engine call has its own deadline** (`TAPIR_TRANSCRIBE_ENGINE_TIMEOUT`, default 20
|
||||
minutes: two wedged engines in sequence then leave 20 of the job's 60 minutes for a fallback),
|
||||
so a wedged engine fails while the job lives and the fallback can run; the job's own
|
||||
deadline or a cancel still never reaches the cloud.
|
||||
- **The duration limit is checked against what is converted**, not the file's header: ffmpeg is
|
||||
capped at the limit plus a minute, a longer result fails the job (`audio_too_long`), and a result
|
||||
under 80% of the claimed length and more than two minutes short fails it
|
||||
(`audio_incomplete`) instead of returning a partial transcript with no warning.
|
||||
- **The queue is bounded**, 20 waiting in total and 3 per user; a full queue is a 429 with a typed
|
||||
reason, never a stuck row or a server error.
|
||||
- **Bad input is a 4xx**: a non-UUID id is a 404, a malformed multipart header is a 400 whose text is
|
||||
not logged, and a title or file name with control bytes or invalid UTF-8 is cleaned, not a 500.
|
||||
under 80% of the claimed length and more than two minutes short is **transcribed and noted** on
|
||||
the audio step ("the audio is 20 min long; the file claimed 1 h 19 min"), not refused. The second
|
||||
adversarial review showed a complete variable-bitrate MP3 whose header over-estimated its length
|
||||
by 4x was wrongly refused by the first version of this check; a note keeps a truncated file
|
||||
visible without turning away real recordings.
|
||||
- **The queue is bounded**, 20 waiting in total and 3 per user, and its room is checked before the
|
||||
upload is spooled (not after 200 MB and an ffprobe run); a full queue is a 429 with a typed reason,
|
||||
never a stuck row or a server error.
|
||||
- **The configured upload chat model is callable.** It is added to the shared chat service's model
|
||||
list when uploads are on and it is local, because the service re-maps a model it does not offer
|
||||
(a valid `iguana/gemma4-31b` was being silently swapped for `koala/phi4-mini`).
|
||||
- **Bad input is a 4xx**: a non-UUID id is a 404, a malformed or cut-off multipart body is a 400
|
||||
whose text is not logged, and a title with control bytes, bidirectional controls or invalid UTF-8 is
|
||||
cleaned (emoji joiners, Swedish letters and variation selectors are kept), not a 500.
|
||||
- **The image was built and inspected** (buildah on koala, 2026-10-02): the first build FAILED
|
||||
(`tar: xz: Cannot exec`, the golang image has no xz); fixed, rebuilt at 180 MB, ffmpeg and ffprobe
|
||||
run as `nonroot` in distroless. CI does not build the image on a pull request
|
||||
(`if: github.event_name != 'pull_request'`), so the first CI build of it is the merge to main.
|
||||
Not done, filed as follow-ups: a vendored ffmpeg tarball, CSRF cookie review, ffprobe CPU and
|
||||
stream-count limits, mov external references, `alive_at` clock skew, a hung database blocking the
|
||||
tracker, an unbounded gateway response body, a user-facing notice for a recording that is partial but
|
||||
|
||||
@@ -36,7 +36,11 @@ FROM golang:1.26 AS ffmpeg
|
||||
ARG FFMPEG_VERSION=7.0.2
|
||||
ARG FFMPEG_SHA256=abda8d77ce8309141f83ab8edf0596834087c52467f6badf376a6a2a4c87cf67
|
||||
|
||||
# xz is not in the golang image (found by building it: `tar: xz: Cannot exec`).
|
||||
RUN set -eu; \
|
||||
apt-get update; \
|
||||
apt-get install -y --no-install-recommends xz-utils; \
|
||||
rm -rf /var/lib/apt/lists/*; \
|
||||
mkdir -p /out; \
|
||||
for dir in releases old-releases; do \
|
||||
curl -fsSL -o /out/ffmpeg.tar.xz "https://johnvansickle.com/ffmpeg/${dir}/ffmpeg-${FFMPEG_VERSION}-amd64-static.tar.xz" && break; \
|
||||
|
||||
@@ -134,6 +134,13 @@ func chatModels(cfg config.Config) []string {
|
||||
add(cfg.SummarizerModel)
|
||||
add(cfg.FallbackModel)
|
||||
add(cfg.CloudFallbackModel)
|
||||
// The model chat over an uploaded recording opens on must be one the shared service can
|
||||
// call: it re-maps a model it does not offer, which silently swapped a configured 31B for
|
||||
// phi4-mini, whose context cannot hold an hour of speech. Local only, and only when
|
||||
// uploads are on, so the video chat is unchanged otherwise (second adversarial review).
|
||||
if cfg.UploadsEnabled && config.IsLocalModel(cfg.UploadChatModel) {
|
||||
add(cfg.UploadChatModel)
|
||||
}
|
||||
return models
|
||||
}
|
||||
|
||||
|
||||
@@ -44,3 +44,42 @@ func TestUploadChatModels_NoLocalModelMeansNoneOfferedNotACloudDefault(t *testin
|
||||
t.Errorf("uploadChatModels = %v, want none", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A valid local TAPIR_UPLOAD_CHAT_MODEL that is not in the video chat's chain used to be
|
||||
// accepted by config and then silently ignored: the shared chat service re-maps a model it
|
||||
// does not offer, so the user got koala/phi4-mini, whose 8,192-token context cannot hold an
|
||||
// hour of speech (second adversarial review).
|
||||
|
||||
func TestChatModels_AnUploadChatModelOutsideTheChainIsOfferedWhenUploadsAreOn(t *testing.T) {
|
||||
var cfg config.Config
|
||||
cfg.SummarizerModel = "koala/phi4-mini"
|
||||
cfg.FallbackModel = "iguana/gemma4-26b"
|
||||
cfg.UploadsEnabled = true
|
||||
cfg.UploadChatModel = "iguana/gemma4-31b"
|
||||
|
||||
service := chatModels(cfg)
|
||||
upload := uploadChatModels(cfg)
|
||||
|
||||
if !reflect.DeepEqual(service, []string{"koala/phi4-mini", "iguana/gemma4-26b", "iguana/gemma4-31b"}) {
|
||||
t.Errorf("chatModels = %v: the shared service must be able to call the configured upload model", service)
|
||||
}
|
||||
if !reflect.DeepEqual(upload, service) {
|
||||
t.Errorf("uploadChatModels = %v, want %v", upload, service)
|
||||
}
|
||||
}
|
||||
|
||||
func TestChatModels_NothingIsAddedWhenUploadsAreOffOrTheModelIsNotLocal(t *testing.T) {
|
||||
var cfg config.Config
|
||||
cfg.SummarizerModel = "koala/phi4-mini"
|
||||
cfg.UploadChatModel = "iguana/gemma4-31b"
|
||||
|
||||
if got := chatModels(cfg); !reflect.DeepEqual(got, []string{"koala/phi4-mini"}) {
|
||||
t.Errorf("uploads off: chatModels = %v, want the video chain unchanged", got)
|
||||
}
|
||||
|
||||
cfg.UploadsEnabled = true
|
||||
cfg.UploadChatModel = "berget/mistral-small"
|
||||
if got := chatModels(cfg); !reflect.DeepEqual(got, []string{"koala/phi4-mini"}) {
|
||||
t.Errorf("a cloud upload chat model was added to the service: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -43,3 +43,36 @@ func TestBuildTranscriber_AWedgedPrimaryReachesTheFallbackBeforeTheJobDeadline(t
|
||||
require.NoError(t, err, "the wedged primary used the whole job and the fallback never ran")
|
||||
assert.Equal(t, "berget/kb-whisper", got.Endpoint)
|
||||
}
|
||||
|
||||
func TestBuildTranscriber_AWedgedBaseEngineIsASoftFailureInTheEnsemble(t *testing.T) {
|
||||
// The same deadline must guard the ensemble's BASE engine, not only the chain's primary
|
||||
// (second adversarial review: the wiring was tested for the chain alone).
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
_ = r.ParseMultipartForm(1 << 20)
|
||||
if r.FormValue("model") == "iguana/pianissimo" {
|
||||
<-r.Context().Done()
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"text":"hej","duration":1,"segments":[{"start":0,"end":1,"text":"hej"}]}`))
|
||||
}))
|
||||
defer srv.Close()
|
||||
cfg := uploadCfg(t, true)
|
||||
cfg.GatewayURL = srv.URL
|
||||
cfg.TranscribeModel = "iguana/kb-whisper"
|
||||
cfg.ReconcileBaseModel = "iguana/pianissimo"
|
||||
cfg.ReconcileModel = "iguana/gemma4-31b"
|
||||
cfg.TranscribeEngineTimeout = 150 * time.Millisecond
|
||||
wav := filepath.Join(t.TempDir(), "a.wav")
|
||||
require.NoError(t, os.WriteFile(wav, []byte("RIFF....WAVEfmt "), 0o600))
|
||||
job, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
|
||||
got, err := buildTranscriber(cfg).Transcribe(job, wav)
|
||||
|
||||
require.NoError(t, err, "a wedged base engine must not use up the job")
|
||||
assert.Equal(t, "hej", got.Text)
|
||||
require.NotNil(t, got.Reconciliation)
|
||||
assert.Equal(t, "single_engine", string(got.Reconciliation.Status))
|
||||
assert.Contains(t, got.Reconciliation.Reason, "timeout")
|
||||
}
|
||||
|
||||
@@ -23,7 +23,7 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-06),
|
||||
`{"error":"failed to process audio"}` string — it has had at least five causes. Tapir's knobs:
|
||||
`TAPIR_UPLOADS_ENABLED` (off by default), `TAPIR_UPLOAD_MAX_MB` (200), `TAPIR_UPLOAD_MAX_MINUTES`
|
||||
(90), `TAPIR_TRANSCRIBE_MODEL`, `TAPIR_TRANSCRIBE_FALLBACK_MODEL` (empty disables the cloud
|
||||
fallback), `TAPIR_TRANSCRIBE_TIMEOUT` (60m, the whole job), `TAPIR_TRANSCRIBE_ENGINE_TIMEOUT` (25m, one engine call: a wedged engine then fails while the job lives and the fallback can run), `TAPIR_UPLOAD_TMP_DIR`, `TAPIR_FFMPEG_BIN`,
|
||||
fallback), `TAPIR_TRANSCRIBE_TIMEOUT` (60m, the whole job), `TAPIR_TRANSCRIBE_ENGINE_TIMEOUT` (20m, one engine call: a wedged engine then fails while the job lives and the fallback can run), `TAPIR_UPLOAD_TMP_DIR`, `TAPIR_FFMPEG_BIN`,
|
||||
`TAPIR_FFPROBE_BIN`, `TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS` (90000 — the measured ceiling of
|
||||
`iguana/gemma4-26b`, which silently drops the head above it), `TAPIR_UPLOAD_CHAT_MODEL`
|
||||
(`iguana/gemma4-26b`; must be a local `koala/` or `iguana/` alias, since the whole private transcript is sent to it; `koala/phi4-mini`'s 8,192-token context 400s on an hour of speech). `ffmpeg` + `ffprobe` are in the runtime image (static, pinned by sha256; #34 decided A on 2026-10-02); Tapir's own ingress body limit for a 200 MB upload is unverified (infra).
|
||||
|
||||
@@ -235,9 +235,10 @@ Feature: Upload an audio recording, get a private Swedish transcript, discuss it
|
||||
When a file claims to be short but converts to more than the limit
|
||||
Then the job fails with a plain explanation and no engine receives the audio
|
||||
|
||||
Scenario: A truncated recording is refused, not half transcribed
|
||||
When a file converts to far less audio than it claimed
|
||||
Then the job fails and asks for the complete recording
|
||||
Scenario: A recording much shorter than its file claimed is transcribed and noted
|
||||
When a file converts to under 80 percent of the length it claimed and more than two minutes short
|
||||
Then it is transcribed anyway and its audio step says how long the audio really is
|
||||
And a complete recording whose header over-estimated its length is not refused
|
||||
|
||||
Scenario: A full queue is a retry, not an error
|
||||
When too many recordings are already waiting
|
||||
|
||||
@@ -78,7 +78,7 @@ printf data > "$last"`)
|
||||
assert.Less(t, fw, in)
|
||||
}
|
||||
|
||||
func TestFFmpegToWAV_CapsTheOutputLengthWhenAMaximumIsSet(t *testing.T) {
|
||||
func TestFFmpegToWAV_PassesAnOutputLengthCapAfterTheInputWhenAMaximumIsSet(t *testing.T) {
|
||||
// The duration limit is a CLAIM in the file's header; a file that lies about it can
|
||||
// convert to hours of audio (adversarial review F2: a 2 h file claiming 10 s became a
|
||||
// 120-minute WAV). The cap is an OUTPUT option, after -i, with a minute of slack.
|
||||
@@ -97,7 +97,7 @@ printf data > "$last"`)
|
||||
assert.Less(t, strings.Index(joined, " -i /spool/in "), strings.Index(joined, " -t 5460 "), "-t after -i is an output cap; before it, an input one")
|
||||
}
|
||||
|
||||
func TestFFmpegToWAV_WithNoMaximumThereIsNoLengthCap(t *testing.T) {
|
||||
func TestFFmpegToWAV_PassesNoLengthCapWhenNoMaximumIsSet(t *testing.T) {
|
||||
rec := filepath.Join(t.TempDir(), "argv")
|
||||
bin := stub(t, `printf '%s\n' "$@" > `+rec+`
|
||||
for a; do last="$a"; done
|
||||
|
||||
@@ -66,7 +66,7 @@ const (
|
||||
defaultTranscribeModel = "iguana/kb-whisper"
|
||||
defaultTranscribeFallbackModel = "berget/kb-whisper"
|
||||
defaultTranscribeTimeout = 60 * time.Minute
|
||||
defaultTranscribeEngineTimeout = 25 * time.Minute
|
||||
defaultTranscribeEngineTimeout = 20 * time.Minute
|
||||
defaultMaxUploadedChars = 90000
|
||||
defaultUploadChatModel = "iguana/gemma4-26b"
|
||||
)
|
||||
|
||||
@@ -76,14 +76,14 @@ func TestLoad_ALocalUploadChatModelIsAccepted(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoad_TheEngineTimeoutDefaultsToTwentyFiveMinutesAndMustBePositive(t *testing.T) {
|
||||
func TestLoad_TheEngineTimeoutDefaultsToTwentyMinutesAndMustBePositive(t *testing.T) {
|
||||
clearUploadEnv(t)
|
||||
cfg, err := Load()
|
||||
if err != nil {
|
||||
t.Fatalf("Load: %v", err)
|
||||
}
|
||||
if cfg.TranscribeEngineTimeout != 25*time.Minute {
|
||||
t.Errorf("TranscribeEngineTimeout = %v, want 25m: a 90-minute recording takes about 8 minutes on kb-whisper", cfg.TranscribeEngineTimeout)
|
||||
if cfg.TranscribeEngineTimeout != 20*time.Minute {
|
||||
t.Errorf("TranscribeEngineTimeout = %v, want 20m: a 90-minute recording takes about 8 minutes on kb-whisper, and two wedged engines in sequence must leave the job time to reach a fallback", cfg.TranscribeEngineTimeout)
|
||||
}
|
||||
|
||||
setEnv(t, map[string]string{"TAPIR_TRANSCRIBE_ENGINE_TIMEOUT": "7m"})
|
||||
|
||||
@@ -203,6 +203,10 @@ type ProgressStep struct {
|
||||
FinishedAt time.Time `json:"finished_at,omitzero"`
|
||||
Done int `json:"done,omitempty"`
|
||||
Total int `json:"total,omitempty"`
|
||||
// Note is a short plain remark about the phase that the reader should see even though
|
||||
// it did not fail, for example that the audio is much shorter than the file claimed.
|
||||
// Durations and counts only: never transcript text, a name or a path.
|
||||
Note string `json:"note,omitempty"`
|
||||
}
|
||||
|
||||
// UploadStatus is the state of an uploaded-audio transcription job.
|
||||
@@ -233,10 +237,6 @@ const (
|
||||
// UploadErrBusy: too many recordings were already waiting, so this one was not
|
||||
// queued. The user's to retry later (ADR-035 review).
|
||||
UploadErrBusy UploadErrorKind = "queue_full"
|
||||
// UploadErrIncomplete: the audio, once converted, ends far earlier than the file
|
||||
// claimed: a truncated recording that would otherwise give a partial transcript with
|
||||
// no warning.
|
||||
UploadErrIncomplete UploadErrorKind = "audio_incomplete"
|
||||
)
|
||||
|
||||
// UploadedTranscript is a user's PRIVATE transcript of audio they uploaded. It is
|
||||
|
||||
@@ -186,6 +186,20 @@ func (t *Tracker) Finish() {
|
||||
t.save()
|
||||
}
|
||||
|
||||
// Annotate attaches a short plain note to a phase and saves it at once. It is for a
|
||||
// remark the reader should see although nothing failed. Durations and counts only: a note
|
||||
// is shown on a page about a private recording.
|
||||
func (t *Tracker) Annotate(key, note string) {
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
s := t.find(key)
|
||||
if s == nil {
|
||||
return
|
||||
}
|
||||
s.Note = note
|
||||
t.save()
|
||||
}
|
||||
|
||||
// Heartbeat stamps the job's row every beatEvery and logs which phase is still running
|
||||
// every stillEvery, until ctx ends. It runs on its own, independent of any engine call:
|
||||
// a heartbeat that stopped means the process is gone, not that a request is slow.
|
||||
|
||||
@@ -387,3 +387,16 @@ func TestWithAndFrom_ACarriedReporterIsTheOneReturned(t *testing.T) {
|
||||
|
||||
require.Equal(t, domain.StepRunning, stateOf(sink.last(t), "audio"))
|
||||
}
|
||||
|
||||
func TestTracker_AnnotateAttachesANoteToAStepAndSavesItAtOnce(t *testing.T) {
|
||||
tr, sink, _, _ := newTracker(t)
|
||||
tr.Start()
|
||||
tr.Begin("audio", 0)
|
||||
before := sink.saveCount()
|
||||
|
||||
tr.Annotate("audio", "20 min of the 79 the file claimed")
|
||||
|
||||
require.Equal(t, before+1, sink.saveCount())
|
||||
require.Equal(t, "20 min of the 79 the file claimed", sink.last(t)[0].Note)
|
||||
require.NotPanics(t, func() { tr.Annotate("no-such-step", "x") })
|
||||
}
|
||||
|
||||
@@ -37,10 +37,10 @@ func TestReconcileSpots_AutoResolvedDisputesAreNotStored(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestReconcileSpots_ASpotCarriesTheWordsJustBeforeIt(t *testing.T) {
|
||||
out := merge(t, tr("vi skickar in ett vanligt meddelande till kassasystemet"), tr("vi skickar in ett KORT meddelande till kassasystemet"), always(reconcile.VerdictAlt))
|
||||
out := merge(t, tr("grannen målar ett vanligt staket mot gatan"), tr("grannen målar ett KORT staket mot gatan"), always(reconcile.VerdictAlt))
|
||||
|
||||
require.Len(t, out.Reconciliation.Contested, 1)
|
||||
assert.Equal(t, "vi skickar in ett", out.Reconciliation.Contested[0].Before)
|
||||
assert.Equal(t, "grannen målar ett", out.Reconciliation.Contested[0].Before)
|
||||
}
|
||||
|
||||
func TestReconcileSpots_TheContextIsAFewWordsNotTheWholeMeeting(t *testing.T) {
|
||||
|
||||
@@ -58,12 +58,21 @@ func NewIntake(s Store, p Prober, sub Submitter, l Limits, tmpDir string) *Intak
|
||||
return &Intake{store: s, prober: p, sub: sub, limits: l, tmpDir: tmpDir}
|
||||
}
|
||||
|
||||
// busyMessage is what a user is told when the queue is full.
|
||||
const busyMessage = "Too many recordings are waiting to be transcribed. Please try again in a few minutes."
|
||||
|
||||
// Accept spools src to disk (bounded by MaxBytes), probes it, validates it, records
|
||||
// a queued upload and submits the job. It returns the upload id. A file that fails
|
||||
// validation yields a *RejectError and creates NO row — rejection is synchronous
|
||||
// and leaves nothing behind. On success the temp file is handed to the job and the
|
||||
// worker deletes it; on every failure path Accept deletes it itself.
|
||||
func (in *Intake) Accept(ctx context.Context, userID, title string, src io.Reader) (string, error) {
|
||||
// Refuse before spooling when the queue has no room, if the submitter can say so: a full
|
||||
// queue otherwise cost up to the upload cap on disk, an ffprobe run and a failed row for
|
||||
// every refused attempt.
|
||||
if rc, ok := in.sub.(interface{ HasRoom(userID string) bool }); ok && !rc.HasRoom(userID) {
|
||||
return "", &RejectError{Reason: ReasonBusy, Message: busyMessage}
|
||||
}
|
||||
f, err := os.CreateTemp(in.tmpDir, "upload-*")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("uploads: spool: %w", err)
|
||||
@@ -114,7 +123,7 @@ func (in *Intake) Accept(ctx context.Context, userID, title string, src io.Reade
|
||||
if ferr := in.store.FailUpload(ctx, userID, id, domain.UploadErrBusy); ferr != nil {
|
||||
return "", fmt.Errorf("uploads: submit: %w (and failing the row: %v)", err, ferr)
|
||||
}
|
||||
return "", &RejectError{Reason: ReasonBusy, Message: "Too many recordings are waiting to be transcribed. Please try again in a few minutes."}
|
||||
return "", &RejectError{Reason: ReasonBusy, Message: busyMessage}
|
||||
}
|
||||
// A row that will never run must not sit queued: that is the stuck spinner.
|
||||
if ferr := in.store.FailUpload(ctx, userID, id, domain.UploadErrInterrupted); ferr != nil {
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
@@ -29,3 +30,37 @@ func TestAccept_AFullQueueIsABusyRejectionNotAServerErrorAndLeavesNothingBehind(
|
||||
assert.Equal(t, domain.UploadErrBusy, r.st.failed["up-1"], "the row must not sit queued: that is the stuck spinner")
|
||||
assert.Empty(t, r.tmpFiles(t), "the audio is deleted")
|
||||
}
|
||||
|
||||
// roomSubmitter is a Submitter that can say, before anything is spooled, that there is no room.
|
||||
type roomSubmitter struct {
|
||||
fakeSubmitter
|
||||
room bool
|
||||
}
|
||||
|
||||
func (r *roomSubmitter) HasRoom(string) bool { return r.room }
|
||||
|
||||
func TestAccept_ANoRoomQueueIsRefusedBeforeAnythingIsSpooledProbedOrStored(t *testing.T) {
|
||||
// Checking only at Submit meant 200 MB was spooled and ffprobe run for every refused
|
||||
// attempt, and a failed row was left behind each time (second adversarial review).
|
||||
r := newRig(t)
|
||||
full := &roomSubmitter{room: false}
|
||||
in := uploads.NewIntake(r.st, r.pr, full, uploads.Limits{MaxBytes: 1024, MaxDuration: 90 * time.Minute}, r.tmp)
|
||||
|
||||
_, err := in.Accept(context.Background(), "user-1", "t", strings.NewReader("data"))
|
||||
|
||||
var rej *uploads.RejectError
|
||||
require.True(t, errors.As(err, &rej))
|
||||
assert.Equal(t, uploads.ReasonBusy, rej.Reason)
|
||||
assert.Empty(t, r.st.created, "no row for an upload that was never accepted")
|
||||
assert.Empty(t, r.pr.paths, "ffprobe was never run on it")
|
||||
assert.Empty(t, r.tmpFiles(t), "nothing was spooled")
|
||||
assert.Empty(t, full.jobs)
|
||||
}
|
||||
|
||||
func TestAccept_ASubmitterThatCannotSayStillWorksAsBefore(t *testing.T) {
|
||||
r := newRig(t)
|
||||
|
||||
_, err := r.in.Accept(context.Background(), "user-1", "t", strings.NewReader("data"))
|
||||
|
||||
require.NoError(t, err, "a Submitter without HasRoom is accepted: the check at Submit remains")
|
||||
}
|
||||
|
||||
+47
-14
@@ -130,6 +130,24 @@ func NewWorker(s WorkerStore, c Converter, t Transcriber, tmpDir string, timeout
|
||||
return w
|
||||
}
|
||||
|
||||
// HasRoom reports whether a job from this user would be accepted right now. It lets the
|
||||
// intake refuse BEFORE spooling up to the upload cap and running ffprobe; Submit still
|
||||
// enforces the same limits atomically, so a race costs at worst one refused submit.
|
||||
func (w *Worker) HasRoom(userID string) bool {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
if w.stopped || len(w.queue) >= w.maxQueued {
|
||||
return false
|
||||
}
|
||||
mine := 0
|
||||
for _, q := range w.queue {
|
||||
if q.UserID == userID {
|
||||
mine++
|
||||
}
|
||||
}
|
||||
return mine < w.maxPerUser
|
||||
}
|
||||
|
||||
// Submit queues a job. It never blocks. It satisfies Submitter.
|
||||
func (w *Worker) Submit(j Job) error {
|
||||
w.mu.Lock()
|
||||
@@ -275,11 +293,18 @@ func (w *Worker) Process(ctx context.Context, job Job) {
|
||||
abort(domain.UploadErrTranscriptionFailed)
|
||||
return
|
||||
}
|
||||
if kind, bad := w.audioLengthProblem(wav, job.Duration); bad {
|
||||
abort(kind)
|
||||
log.Error("upload: converted audio length rejected", "kind", string(kind))
|
||||
got, tooLong := w.audioLengthProblem(wav)
|
||||
if tooLong {
|
||||
abort(domain.UploadErrTooLong)
|
||||
log.Error("upload: converted audio is over the limit", "audio_s", int(got.Seconds()), "limit_s", int(w.maxDuration.Seconds()))
|
||||
return
|
||||
}
|
||||
if w.maxDuration > 0 && job.Duration > 0 {
|
||||
if note := shortfallNote(got, job.Duration); note != "" {
|
||||
log.Warn("upload: audio is much shorter than the file claimed", "audio_s", int(got.Seconds()), "claimed_s", int(job.Duration.Seconds()))
|
||||
tr.Annotate("audio", note)
|
||||
}
|
||||
}
|
||||
tr.Done("audio")
|
||||
|
||||
if !planned {
|
||||
@@ -343,21 +368,29 @@ const (
|
||||
)
|
||||
|
||||
// audioLengthProblem measures the converted WAV (16 kHz mono 16-bit PCM is 32,000 bytes a
|
||||
// second after a 44-byte header) against the limit and against what the file claimed.
|
||||
func (w *Worker) audioLengthProblem(wav string, claimed time.Duration) (domain.UploadErrorKind, bool) {
|
||||
// second after a 44-byte header) against the limit. It returns the real length and whether
|
||||
// that is over the limit by more than the tolerance.
|
||||
func (w *Worker) audioLengthProblem(wav string) (got time.Duration, tooLong bool) {
|
||||
if w.maxDuration <= 0 {
|
||||
return "", false
|
||||
return 0, false
|
||||
}
|
||||
info, err := os.Stat(wav)
|
||||
if err != nil {
|
||||
return "", false
|
||||
return 0, false
|
||||
}
|
||||
got := time.Duration(float64(max(info.Size()-44, 0)) / 32000 * float64(time.Second))
|
||||
switch {
|
||||
case got > w.maxDuration+tooLongSlack:
|
||||
return domain.UploadErrTooLong, true
|
||||
case float64(got) < incompleteBelow*float64(claimed) && claimed-got > incompleteShortfall:
|
||||
return domain.UploadErrIncomplete, true
|
||||
got = time.Duration(float64(max(info.Size()-44, 0)) / 32000 * float64(time.Second))
|
||||
return got, got > w.maxDuration+tooLongSlack
|
||||
}
|
||||
|
||||
// shortfallNote is a remark for the audio step when the converted audio is much shorter
|
||||
// than the file claimed, or "" when it is not. It is a NOTE and never a failure: a
|
||||
// container header can over-estimate a complete file (a variable-bitrate MP3 without a Xing
|
||||
// header claimed 79 minutes for 20 real ones), so refusing would turn away real
|
||||
// recordings, while saying nothing about a truncated one would hide a partial transcript.
|
||||
func shortfallNote(got, claimed time.Duration) string {
|
||||
if float64(got) >= incompleteBelow*float64(claimed) || claimed-got <= incompleteShortfall {
|
||||
return ""
|
||||
}
|
||||
return "", false
|
||||
return fmt.Sprintf("The audio is %s long; the file claimed %s. If the recording was cut off, upload the complete one.",
|
||||
FormatDuration(got), FormatDuration(claimed))
|
||||
}
|
||||
|
||||
@@ -69,15 +69,84 @@ func TestWorkerDuration_JustOverTheLimitWithinTheSlackStillTranscribes(t *testin
|
||||
assert.Contains(t, completed, "a", "ffmpeg rounds: 20 s over a 90-minute limit is not a lie")
|
||||
}
|
||||
|
||||
func TestWorkerDuration_AFileThatEndsFarEarlierThanItClaimedIsRefusedNotHalfTranscribed(t *testing.T) {
|
||||
func audioStepNote(r *wrig) string {
|
||||
for _, st := range r.st.lastSteps() {
|
||||
if st.Key == "audio" {
|
||||
return st.Note
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func TestWorkerDuration_AFileThatEndsFarEarlierThanItClaimedIsStillTranscribedAndSaysSo(t *testing.T) {
|
||||
// A container header can over-estimate a complete file (VBR MP3 without a Xing header
|
||||
// claimed 79 minutes for 20 real ones). Refusing it was wrong; staying silent about a
|
||||
// truncated one is wrong too. So it is transcribed, and the audio step carries a note.
|
||||
r := newWRig(t, time.Minute)
|
||||
w := durationWorker(t, r, 20*60, 90*time.Minute)
|
||||
|
||||
w.Process(context.Background(), jobWith(r, t, 60*time.Minute))
|
||||
w.Process(context.Background(), jobWith(r, t, 79*time.Minute))
|
||||
|
||||
_, failed := r.st.snapshot()
|
||||
assert.Equal(t, domain.UploadErrIncomplete, failed["a"], "a recording truncated at a third must not return a third of a transcript with no warning")
|
||||
assert.Empty(t, r.tr.paths)
|
||||
completed, failed := r.st.snapshot()
|
||||
assert.Empty(t, failed, "a complete recording with a bad header estimate must not be refused")
|
||||
assert.Contains(t, completed, "a")
|
||||
note := audioStepNote(r)
|
||||
assert.Contains(t, note, "20 min", "what was really transcribed")
|
||||
assert.Contains(t, note, "1 h 19 min", "what the file claimed")
|
||||
assert.Contains(t, note, "cut off", "and what to do if that is not a surprise")
|
||||
}
|
||||
|
||||
func TestWorkerDuration_TheShortfallNoteIsPinnedAtEightyPercentAndTwoMinutes(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
claimed time.Duration
|
||||
wav float64
|
||||
noted bool
|
||||
}{
|
||||
{"79 percent of an hour", 60 * time.Minute, 0.79 * 3600, true},
|
||||
{"81 percent of an hour", 60 * time.Minute, 0.81 * 3600, false},
|
||||
{"exactly two minutes short and under 80 percent", 6 * time.Minute, 4 * 60, false},
|
||||
{"two minutes and a second short and under 80 percent", 6 * time.Minute, 4*60 - 1, true},
|
||||
{"unknown claimed length", 0, 20 * 60, false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
r := newWRig(t, time.Minute)
|
||||
w := durationWorker(t, r, tt.wav, 90*time.Minute)
|
||||
|
||||
w.Process(context.Background(), jobWith(r, t, tt.claimed))
|
||||
|
||||
assert.Equal(t, tt.noted, audioStepNote(r) != "")
|
||||
completed, _ := r.st.snapshot()
|
||||
assert.Contains(t, completed, "a", "a note never stops the job")
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerDuration_TheTooLongToleranceIsExactlyThirtySeconds(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
wav float64
|
||||
fail bool
|
||||
}{
|
||||
{"at the limit plus 30 s", 90*60 + 30, false},
|
||||
{"at the limit plus 31 s", 90*60 + 31, true},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
r := newWRig(t, time.Minute)
|
||||
w := durationWorker(t, r, tt.wav, 90*time.Minute)
|
||||
|
||||
w.Process(context.Background(), jobWith(r, t, 90*time.Minute))
|
||||
|
||||
_, failed := r.st.snapshot()
|
||||
if tt.fail {
|
||||
assert.Equal(t, domain.UploadErrTooLong, failed["a"])
|
||||
} else {
|
||||
assert.Empty(t, failed)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerDuration_AnEstimateThatIsMerelyALittleOffIsNotTruncation(t *testing.T) {
|
||||
|
||||
@@ -78,3 +78,15 @@ func TestWorkerSubmit_AJobThatHasLeftTheQueueFreesItsPlace(t *testing.T) {
|
||||
|
||||
assert.NoError(t, w.Submit(r.spool(t, "second")), "the cap counts jobs waiting, not jobs ever submitted")
|
||||
}
|
||||
|
||||
func TestWorkerHasRoom_SaysWhetherASubmitWouldBeAccepted(t *testing.T) {
|
||||
w := queueWorker(t, 3, 2)
|
||||
assert.True(t, w.HasRoom("alice"))
|
||||
require.NoError(t, submit(w, "alice", 1))
|
||||
require.NoError(t, submit(w, "alice", 2))
|
||||
|
||||
assert.False(t, w.HasRoom("alice"), "alice is at her share")
|
||||
assert.True(t, w.HasRoom("bob"), "bob is not")
|
||||
require.NoError(t, submit(w, "bob", 1))
|
||||
assert.False(t, w.HasRoom("carol"), "the queue as a whole is at its cap")
|
||||
}
|
||||
|
||||
+14
-3
@@ -152,6 +152,10 @@ func (a *App) uploadFailure(w http.ResponseWriter, r *http.Request, userID strin
|
||||
a.logger().Info("upload rejected", "reason", string(uploads.ReasonTooLarge))
|
||||
a.renderUploadsPage(w, r, userID, http.StatusRequestEntityTooLarge,
|
||||
"This file is larger than the upload limit.")
|
||||
case errors.Is(err, io.ErrUnexpectedEOF):
|
||||
// The sender hung up mid-file: theirs to retry, not a server fault.
|
||||
a.logger().Info("upload rejected", "reason", "truncated")
|
||||
a.renderUploadsPage(w, r, userID, http.StatusBadRequest, "The upload was cut off before it finished. Please try again.")
|
||||
default:
|
||||
// Not err: it can carry the file's name, a spool path or a database value.
|
||||
a.logger().Error("upload failed", "kind", errkind.Of(err))
|
||||
@@ -218,6 +222,12 @@ func titleFromFilename(name string) string {
|
||||
// uuidShape is the form of an upload id. Anything else cannot be one, so it is a 404.
|
||||
var uuidShape = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$`)
|
||||
|
||||
// isBidiControl reports the invisible characters that change text direction. Emoji joiners
|
||||
// (ZWJ, ZWNJ) and variation selectors are not among them and are kept.
|
||||
func isBidiControl(r rune) bool {
|
||||
return r >= 0x202A && r <= 0x202E || r >= 0x2066 && r <= 0x2069 || r == 0x200E || r == 0x200F || r == 0x061C
|
||||
}
|
||||
|
||||
// cleanTitle makes a title safe to store and show: invalid UTF-8 dropped, control
|
||||
// characters (NUL, newlines, escapes) turned into spaces, runs of space collapsed, trimmed
|
||||
// and clipped. A title is cosmetic; bad bytes in it must never fail an upload, and Postgres
|
||||
@@ -225,7 +235,10 @@ var uuidShape = regexp.MustCompile(`^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4
|
||||
func cleanTitle(s string) string {
|
||||
s = strings.ToValidUTF8(s, "")
|
||||
s = strings.Map(func(r rune) rune {
|
||||
if unicode.IsControl(r) {
|
||||
switch {
|
||||
case isBidiControl(r):
|
||||
return -1 // dropped: U+202E would reverse the rest of the title on its owner's page
|
||||
case unicode.IsControl(r):
|
||||
return ' '
|
||||
}
|
||||
return r
|
||||
@@ -250,8 +263,6 @@ func uploadFailureMessage(k domain.UploadErrorKind) string {
|
||||
return "Too many recordings were waiting to be transcribed when this one was uploaded. Please upload it again in a few minutes."
|
||||
case domain.UploadErrTooLong:
|
||||
return "This recording is longer than the limit once its audio is read (the file's header said otherwise). Please upload a shorter recording."
|
||||
case domain.UploadErrIncomplete:
|
||||
return "This recording ends much earlier than the file says it should; it may be cut off or damaged. Please upload the complete recording."
|
||||
case domain.UploadErrTranscriptionFailed:
|
||||
return "Transcription failed on every available service. Please upload it again later."
|
||||
default:
|
||||
|
||||
@@ -218,6 +218,9 @@ templ progressSteps(steps []domain.ProgressStep, now time.Time, live bool) {
|
||||
if t := stepTime(s, now); t != "" {
|
||||
<span class="step-time">{ t }</span>
|
||||
}
|
||||
if s.Note != "" {
|
||||
<span class="step-note">{ s.Note }</span>
|
||||
}
|
||||
if s.State == domain.StepRunning && s.Total > 0 {
|
||||
<progress class="step-bar" value={ strconv.Itoa(s.Done) } max={ strconv.Itoa(s.Total) }></progress>
|
||||
<span class="step-count">{ strconv.Itoa(s.Done) } of { strconv.Itoa(s.Total) }</span>
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
package web_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"testing"
|
||||
@@ -60,3 +62,48 @@ func TestUploadPost_AMalformedMultipartHeaderIsA400AndItsTextIsNotLogged(t *test
|
||||
assert.Empty(t, sub.jobs)
|
||||
assert.NotContains(t, buf.String(), "zebrakaka", "a file name is private content")
|
||||
}
|
||||
|
||||
func TestUploadPost_ABodyCutOffInsideTheFilePartIsA400NotA500(t *testing.T) {
|
||||
app, sub := newUploadApp(t, audioOK, 200<<20)
|
||||
req := uploadReq(t, "t", "a.m4a", bytes.Repeat([]byte("a"), 4096))
|
||||
raw, err := io.ReadAll(req.Body)
|
||||
require.NoError(t, err)
|
||||
req.Body = io.NopCloser(bytes.NewReader(raw[:len(raw)-2000])) // the sender hung up mid-file
|
||||
buf := &lockedBuf{}
|
||||
app.Log = slog.New(slog.NewTextHandler(buf, nil))
|
||||
|
||||
rec := do(t, app, req)
|
||||
|
||||
assert.Equal(t, http.StatusBadRequest, rec.Code, "a connection that drops mid-upload is the sender's to retry, not a server fault")
|
||||
assert.Empty(t, sub.jobs)
|
||||
assert.NotContains(t, buf.String(), "level=ERROR")
|
||||
}
|
||||
|
||||
func TestUploadPost_BidirectionalControlCharactersAreStrippedFromATitle(t *testing.T) {
|
||||
// U+202E reverses the text that follows it, so a title could render reversed on its
|
||||
// owner's own page (second adversarial review).
|
||||
app, sub := newUploadApp(t, audioOK, 200<<20)
|
||||
|
||||
rec := do(t, app, uploadReq(t, "Sprint \u202Ereview \u2066x\u2069 \u200Fok", "a.m4a", []byte("x")))
|
||||
|
||||
require.Equal(t, http.StatusSeeOther, rec.Code)
|
||||
var title string
|
||||
require.NoError(t, rawPool(t).QueryRow(context.Background(), `SELECT title FROM uploaded_transcripts WHERE id = $1`, sub.jobs[0].UploadID).Scan(&title))
|
||||
for _, r := range []string{"\u202E", "\u2066", "\u2069", "\u200F"} {
|
||||
assert.NotContains(t, title, r)
|
||||
}
|
||||
assert.Contains(t, title, "Sprint")
|
||||
assert.Contains(t, title, "review")
|
||||
}
|
||||
|
||||
func TestUploadPost_SwedishLettersAndEmojiSurviveTheTitleCleaning(t *testing.T) {
|
||||
app, sub := newUploadApp(t, audioOK, 200<<20)
|
||||
|
||||
// Includes a ZWJ sequence (woman + ZWJ + laptop) and a ZWNJ: joiners are not direction controls.
|
||||
rec := do(t, app, uploadReq(t, "Möte åäö 🎙️ 👩\u200d💻 a\u200cb", "a.m4a", []byte("x")))
|
||||
|
||||
require.Equal(t, http.StatusSeeOther, rec.Code)
|
||||
var title string
|
||||
require.NoError(t, rawPool(t).QueryRow(context.Background(), `SELECT title FROM uploaded_transcripts WHERE id = $1`, sub.jobs[0].UploadID).Scan(&title))
|
||||
assert.Equal(t, "Möte åäö 🎙️ 👩\u200d💻 a\u200cb", title, "joiners and variation selectors belong to the text")
|
||||
}
|
||||
|
||||
@@ -465,7 +465,6 @@ func TestUploadStatus_ALengthFailureIsExplainedInWordsAndNotAsAGenericError(t *t
|
||||
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||
tests := []struct{ kind, want string }{
|
||||
{"audio_too_long", "longer than the limit"},
|
||||
{"audio_incomplete", "ends much earlier than"},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.kind, func(t *testing.T) {
|
||||
|
||||
@@ -251,3 +251,18 @@ func TestUploadProgress_TheFullPageCarriesTheVerySameProgressListAsTheFragment(t
|
||||
assert.Equal(t, stepsBlock(t, fragment), stepsBlock(t, full),
|
||||
"the page a user opens and the fragment HTMX swaps in must show the same list, or the display jumps on the first poll")
|
||||
}
|
||||
|
||||
func TestUploadProgress_AStepNoteIsShownUnderItsStepAndEscaped(t *testing.T) {
|
||||
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||
fixedClock(app)
|
||||
id := seedUpload(t, userID, "m", "transcribing", "")
|
||||
steps := twoEngineRun()
|
||||
steps[1].Note = "The audio is 20 min long; the file claimed 1 h 19 min. <b>cut off</b>?"
|
||||
seedProgress(t, id, steps, progressNow.Add(-10*time.Second), progressNow.Add(-4*time.Second))
|
||||
|
||||
b := statusBody(t, app, id)
|
||||
|
||||
assert.Contains(t, b, "The audio is 20 min long")
|
||||
assert.Contains(t, b, "step-note")
|
||||
assert.NotContains(t, b, "<b>cut off</b>", "a note is words, never markup")
|
||||
}
|
||||
|
||||
@@ -47,6 +47,9 @@ func TestRenderUploadProgressStatesForReview(t *testing.T) {
|
||||
step("save", "Saving the transcript", domain.StepDone, time.Minute, time.Second, 0, 0),
|
||||
}
|
||||
|
||||
noted := twoEngineRun()
|
||||
noted[1].Note = "The audio is 20 min long; the file claimed 1 h 19 min. If the recording was cut off, upload the complete one."
|
||||
|
||||
cases := []struct {
|
||||
name, status, content string
|
||||
steps []domain.ProgressStep
|
||||
@@ -58,6 +61,7 @@ func TestRenderUploadProgressStatesForReview(t *testing.T) {
|
||||
{"lost", "transcribing", "", twoEngineRun(), 3 * time.Minute, 3 * time.Minute},
|
||||
{"failed", "failed", "transcription_failed", failed, 2 * time.Minute, 2 * time.Minute},
|
||||
{"done", "done", "Synthetic transcript text for the review.", done, time.Minute, time.Minute},
|
||||
{"noted", "transcribing", "", noted, 10 * time.Second, 4 * time.Second},
|
||||
{"queued", "queued", "", nil, 0, 0},
|
||||
}
|
||||
for _, c := range cases {
|
||||
|
||||
@@ -62,11 +62,11 @@ func TestUploadStatus_ASpotShowsTheWordsJustBeforeIt(t *testing.T) {
|
||||
app, _ := newUploadApp(t, audioOK, 200<<20)
|
||||
id := seedUpload(t, userID, "m", "done", "Synthetic transcript.")
|
||||
setReconciliation(t, id, `{"status":"reconciled","base":"a","alt":"b","reconciler":"c","disputes":1,"changed":1,
|
||||
"contested":[{"seconds":2640,"base":"vanligt","alt":"KORT","used":"alt","unsure":false,"before":"vi skickar in ett"}]}`)
|
||||
"contested":[{"seconds":2640,"base":"vanligt","alt":"KORT","used":"alt","unsure":false,"before":"grannen målar ett"}]}`)
|
||||
|
||||
b := body(t, do(t, app, get("/uploads/"+id+"/status")))
|
||||
|
||||
assert.Contains(t, b, "vi skickar in ett", "the words before the spot are how a reader finds it in the audio")
|
||||
assert.Contains(t, b, "grannen målar ett", "the words before the spot are how a reader finds it in the audio")
|
||||
assert.Contains(t, b, "vanligt")
|
||||
assert.Contains(t, b, "KORT")
|
||||
}
|
||||
|
||||
@@ -1034,70 +1034,89 @@ func progressSteps(steps []domain.ProgressStep, now time.Time, live bool) templ.
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
}
|
||||
if s.State == domain.StepRunning && s.Total > 0 {
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 86, "<progress class=\"step-bar\" value=\"")
|
||||
if s.Note != "" {
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 86, "<span class=\"step-note\">")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var49 string
|
||||
templ_7745c5c3_Var49, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Done))
|
||||
templ_7745c5c3_Var49, templ_7745c5c3_Err = templ.JoinStringErrs(s.Note)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 222, Col: 60}
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 222, Col: 37}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var49)
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var49))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 87, "\" max=\"")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 87, "</span> ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
}
|
||||
if s.State == domain.StepRunning && s.Total > 0 {
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 88, "<progress class=\"step-bar\" value=\"")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var50 string
|
||||
templ_7745c5c3_Var50, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Total))
|
||||
templ_7745c5c3_Var50, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Done))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 222, Col: 90}
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 225, Col: 60}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var50)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 88, "\"></progress> <span class=\"step-count\">")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 89, "\" max=\"")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var51 string
|
||||
templ_7745c5c3_Var51, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Done))
|
||||
templ_7745c5c3_Var51, templ_7745c5c3_Err = templ.ResolveAttributeValue(strconv.Itoa(s.Total))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 223, Col: 52}
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 225, Col: 90}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var51))
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ_7745c5c3_Var51)
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 89, " of ")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 90, "\"></progress> <span class=\"step-count\">")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var52 string
|
||||
templ_7745c5c3_Var52, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Total))
|
||||
templ_7745c5c3_Var52, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Done))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 223, Col: 81}
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 226, Col: 52}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var52))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 90, "</span>")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 91, " of ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var53 string
|
||||
templ_7745c5c3_Var53, templ_7745c5c3_Err = templ.JoinStringErrs(strconv.Itoa(s.Total))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 226, Col: 81}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var53))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 92, "</span>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 91, "</li>")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 93, "</li>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 92, "</ol>")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 94, "</ol>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
@@ -1123,45 +1142,45 @@ func healthNote(h jobHealth, silent time.Duration) templ.Component {
|
||||
}()
|
||||
}
|
||||
ctx = templ.InitializeContext(ctx)
|
||||
templ_7745c5c3_Var53 := templ.GetChildren(ctx)
|
||||
if templ_7745c5c3_Var53 == nil {
|
||||
templ_7745c5c3_Var53 = templ.NopComponent
|
||||
templ_7745c5c3_Var54 := templ.GetChildren(ctx)
|
||||
if templ_7745c5c3_Var54 == nil {
|
||||
templ_7745c5c3_Var54 = templ.NopComponent
|
||||
}
|
||||
ctx = templ.ClearChildren(ctx)
|
||||
switch h {
|
||||
case healthLost:
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 93, "<p class=\"step-health step-health-lost\" role=\"alert\">The service stopped reporting ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var54 string
|
||||
templ_7745c5c3_Var54, templ_7745c5c3_Err = templ.JoinStringErrs(minutesAgo(silent))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 236, Col: 54}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var54))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 94, " ago. It may have restarted or crashed. This page keeps checking; if nothing changes, upload the recording again.</p>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
case healthQuiet:
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 95, "<p class=\"step-health step-health-quiet\">Nothing new for ")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 95, "<p class=\"step-health step-health-lost\" role=\"alert\">The service stopped reporting ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var55 string
|
||||
templ_7745c5c3_Var55, templ_7745c5c3_Err = templ.JoinStringErrs(minutesAgo(silent))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 241, Col: 40}
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 239, Col: 54}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var55))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 96, ": the model may be slow. The service is still running, and the job gives up on its own if it takes too long.</p>")
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 96, " ago. It may have restarted or crashed. This page keeps checking; if nothing changes, upload the recording again.</p>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
case healthQuiet:
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 97, "<p class=\"step-health step-health-quiet\">Nothing new for ")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
var templ_7745c5c3_Var56 string
|
||||
templ_7745c5c3_Var56, templ_7745c5c3_Err = templ.JoinStringErrs(minutesAgo(silent))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ.Error{Err: templ_7745c5c3_Err, FileName: `internal/web/uploads.templ`, Line: 244, Col: 40}
|
||||
}
|
||||
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var56))
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 98, ": the model may be slow. The service is still running, and the job gives up on its own if it takes too long.</p>")
|
||||
if templ_7745c5c3_Err != nil {
|
||||
return templ_7745c5c3_Err
|
||||
}
|
||||
|
||||
@@ -780,6 +780,7 @@ a.btn, a.btn:visited { color: var(--accent-fg); }
|
||||
.step-skipped { opacity: .7; }
|
||||
.step-pending { opacity: .75; }
|
||||
.step-bar { flex: none; width: calc(100% - 1.6rem); margin: 0 0 0 1.6rem; height: .5rem; accent-color: var(--accent); }
|
||||
.step-note { flex: none; width: calc(100% - 1.6rem); margin-left: 1.6rem; font-weight: 400; color: var(--muted); font-family: system-ui, -apple-system, sans-serif; font-size: .82rem; }
|
||||
.step-count { flex: none; width: calc(100% - 1.6rem); margin-left: 1.6rem; font-weight: 400; color: var(--muted); }
|
||||
.step-running .step-mark-warn { color: var(--err-fg); }
|
||||
.spinner { display: inline-block; vertical-align: -.1em; width: .8rem; height: .8rem; box-sizing: border-box; border: 2px solid var(--line); border-top-color: var(--accent); border-radius: 50%; animation: tapir-spin 1s linear infinite; }
|
||||
|
||||
@@ -135,26 +135,26 @@ var scenarioCoverage = map[string]string{
|
||||
"A single-engine transcript claims no second opinion": "TestUploadStatus_ASingleEngineTranscriptDoesNotClaimASecondOpinion",
|
||||
|
||||
// audio_upload.feature, ADR-035
|
||||
"I can see which phase my upload is in": "TestUploadProgress_ARunningJobShowsEveryPhaseInOrderWithTheRunningOneMarked",
|
||||
"A bar appears only where the count is real": "TestUploadProgress_OnlyACountedPhaseGetsABarAndItShowsRealCounts",
|
||||
"A slow engine call is not mistaken for a dead service": "TestUploadProgress_ALongEngineCallWithAFreshHeartbeatIsStillJustRunning",
|
||||
"A service that stopped reporting is called lost": "TestUploadProgress_AHeartbeatThatStoppedMeansTheServiceIsLostAndTheJobIsNotCalledRunning",
|
||||
"A model that stops advancing is called quiet": "TestUploadProgress_ACountedPhaseThatStopsAdvancingGoesQuietNotLost",
|
||||
"A failed job shows where it stopped": "TestUploadProgress_AFailedJobShowsWhichPhaseFailedAndWhatNeverStarted",
|
||||
"A finished job keeps its record of what was done": "TestUploadProgress_ADoneJobKeepsItsRecordOfWhatWasDoneAndHowLongEachTook",
|
||||
"The heartbeat runs on its own while an engine call is stuck": "TestWorkerProgress_TheHeartbeatBeatsWhileAnEngineCallIsStuckAndStopsWhenTheJobEnds",
|
||||
"The log names every phase and never the transcript": "TestWorkerProgress_OnASuccessfulJobTheLogNamesEveryPhaseInOrderAndNeverTheTranscriptOrThePath",
|
||||
"Every judged spot is stored but only the ones to check are listed": "TestUploadStatus_OnlyChangedAndUncertainSpotsAreListedNotTheKeptOnes",
|
||||
"Chat about a recording is offered with local models only": "TestUploadChatPage_DoesNotOfferACloudModelEvenThoughTheServiceKnowsOne",
|
||||
"A forged cloud model is never sent the transcript": "TestUploadChatPost_AForgedCloudModelIsReplacedByALocalOneBeforeTheTranscriptIsSent",
|
||||
"A wedged engine does not use up the whole job": "TestChain_AHungPrimaryReachesTheFallbackWhenItHasItsOwnDeadline",
|
||||
"A file that lies about its length is refused": "TestWorkerDuration_AConvertedLengthOverTheLimitFailsTheJobBeforeAnyEngineSeesIt",
|
||||
"A truncated recording is refused, not half transcribed": "TestWorkerDuration_AFileThatEndsFarEarlierThanItClaimedIsRefusedNotHalfTranscribed",
|
||||
"A full queue is a retry, not an error": "TestAccept_AFullQueueIsABusyRejectionNotAServerErrorAndLeavesNothingBehind",
|
||||
"One user cannot fill the queue for everyone": "TestWorkerSubmit_OneUserCannotFillTheQueueForEveryoneElse",
|
||||
"Errors are shown as kinds, never their text": "TestReconcile_TheBreakerReasonCarriesAKindNotTheModelsText",
|
||||
"Input that cannot be right is a client error": "TestUploadRoutes_ANonUUIDIdIs404NotA500",
|
||||
"The saved segments carry the merged words": "TestReconcileSegments_AChangedWordReplacesTheBaseWordInItsOwnSegmentOnly",
|
||||
"I can see which phase my upload is in": "TestUploadProgress_ARunningJobShowsEveryPhaseInOrderWithTheRunningOneMarked",
|
||||
"A bar appears only where the count is real": "TestUploadProgress_OnlyACountedPhaseGetsABarAndItShowsRealCounts",
|
||||
"A slow engine call is not mistaken for a dead service": "TestUploadProgress_ALongEngineCallWithAFreshHeartbeatIsStillJustRunning",
|
||||
"A service that stopped reporting is called lost": "TestUploadProgress_AHeartbeatThatStoppedMeansTheServiceIsLostAndTheJobIsNotCalledRunning",
|
||||
"A model that stops advancing is called quiet": "TestUploadProgress_ACountedPhaseThatStopsAdvancingGoesQuietNotLost",
|
||||
"A failed job shows where it stopped": "TestUploadProgress_AFailedJobShowsWhichPhaseFailedAndWhatNeverStarted",
|
||||
"A finished job keeps its record of what was done": "TestUploadProgress_ADoneJobKeepsItsRecordOfWhatWasDoneAndHowLongEachTook",
|
||||
"The heartbeat runs on its own while an engine call is stuck": "TestWorkerProgress_TheHeartbeatBeatsWhileAnEngineCallIsStuckAndStopsWhenTheJobEnds",
|
||||
"The log names every phase and never the transcript": "TestWorkerProgress_OnASuccessfulJobTheLogNamesEveryPhaseInOrderAndNeverTheTranscriptOrThePath",
|
||||
"Every judged spot is stored but only the ones to check are listed": "TestUploadStatus_OnlyChangedAndUncertainSpotsAreListedNotTheKeptOnes",
|
||||
"Chat about a recording is offered with local models only": "TestUploadChatPage_DoesNotOfferACloudModelEvenThoughTheServiceKnowsOne",
|
||||
"A forged cloud model is never sent the transcript": "TestUploadChatPost_AForgedCloudModelIsReplacedByALocalOneBeforeTheTranscriptIsSent",
|
||||
"A wedged engine does not use up the whole job": "TestChain_AHungPrimaryReachesTheFallbackWhenItHasItsOwnDeadline",
|
||||
"A file that lies about its length is refused": "TestWorkerDuration_AConvertedLengthOverTheLimitFailsTheJobBeforeAnyEngineSeesIt",
|
||||
"A recording much shorter than its file claimed is transcribed and noted": "TestWorkerDuration_AFileThatEndsFarEarlierThanItClaimedIsStillTranscribedAndSaysSo",
|
||||
"A full queue is a retry, not an error": "TestAccept_AFullQueueIsABusyRejectionNotAServerErrorAndLeavesNothingBehind",
|
||||
"One user cannot fill the queue for everyone": "TestWorkerSubmit_OneUserCannotFillTheQueueForEveryoneElse",
|
||||
"Errors are shown as kinds, never their text": "TestReconcile_TheBreakerReasonCarriesAKindNotTheModelsText",
|
||||
"Input that cannot be right is a client error": "TestUploadRoutes_ANonUUIDIdIs404NotA500",
|
||||
"The saved segments carry the merged words": "TestReconcileSegments_AChangedWordReplacesTheBaseWordInItsOwnSegmentOnly",
|
||||
|
||||
// summarize_new_video.feature
|
||||
"A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered",
|
||||
|
||||
Reference in New Issue
Block a user