Compare commits
18
Commits
@@ -100,7 +100,7 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes (
|
|||||||
- Adapters present under `internal/adapters/`: `youtube` (captions-first `VideoSource`,
|
- Adapters present under `internal/adapters/`: `youtube` (captions-first `VideoSource`,
|
||||||
timedtext/InnerTube acquisition per ADR-010), `summarizer` + `llm` (the copied AI router,
|
timedtext/InnerTube acquisition per ADR-010), `summarizer` + `llm` (the copied AI router,
|
||||||
now a resilient endpoint chain — local primary → local fallback → external worst-case,
|
now a resilient endpoint chain — local primary → local fallback → external worst-case,
|
||||||
parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–017),
|
parse-failure-aware, ADR-004 + ADR-022), `store` (Postgres, golang-migrate migrations 001–019),
|
||||||
`secrets` (file-backed `SecretStore`). The brain HTTP sink (ADR-005) is the remaining
|
`secrets` (file-backed `SecretStore`). The brain HTTP sink (ADR-005) is the remaining
|
||||||
optional sink.
|
optional sink.
|
||||||
- Stage 1 is open (ADR-012): multi-user with **DB-enforced** isolation — Postgres RLS `FORCE`d
|
- Stage 1 is open (ADR-012): multi-user with **DB-enforced** isolation — Postgres RLS `FORCE`d
|
||||||
@@ -118,6 +118,11 @@ The repo is **green and shipping** — last tag `v0.15.0`. `task check` passes (
|
|||||||
in `uploaded_transcripts`; the audio is never retained; chat reuses `chat.Service`. The
|
in `uploaded_transcripts`; the audio is never retained; chat reuses `chat.Service`. The
|
||||||
isolation test pins the **exact set** of non-RLS tables (`TestRowSecurityOffTablesAreExactlyTheKnownSet`).
|
isolation test pins the **exact set** of non-RLS tables (`TestRowSecurityOffTablesAreExactlyTheKnownSet`).
|
||||||
The runtime image does not yet contain ffmpeg — that is a pending decision recorded in ADR-033.
|
The runtime image does not yet contain ffmpeg — that is a pending decision recorded in ADR-033.
|
||||||
|
Two-engine reconciliation (ADR-034, migration 019, off unless `TAPIR_RECONCILE_BASE_MODEL`):
|
||||||
|
`internal/reconcile` (word diff, disputes, a local-model adjudicator) and
|
||||||
|
`transcribe.Ensemble`; every model must be a local alias, enforced at start-up.
|
||||||
|
**Lint caveat:** `task check` does not fail on lint (#41: the Taskfile `|| echo` swallows
|
||||||
|
findings) — run `golangci-lint run ./...` yourself until that is fixed.
|
||||||
- `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch
|
- `cmd/tapir` subcommands: `list`, `show`, `auth` (interactive host-side OAuth), `run` (batch
|
||||||
watch→summarize), `serve` (the HTMX+Templ web reader/writer under `internal/web`, a new
|
watch→summarize), `serve` (the HTMX+Templ web reader/writer under `internal/web`, a new
|
||||||
transport over the unchanged engine/ports — ADR-003). `tapir env` prints config.
|
transport over the unchanged engine/ports — ADR-003). `tapir env` prints config.
|
||||||
|
|||||||
+14
-4
@@ -1601,10 +1601,13 @@ Mode on iguana is the unproven suspect).
|
|||||||
plain words what was done — "reconciled from two transcripts: N differences, K uncertain" — and
|
plain words what was done — "reconciled from two transcripts: N differences, K uncertain" — and
|
||||||
lists the uncertain spots. A reconciled transcript is never presented as simply "the" transcript.
|
lists the uncertain spots. A reconciled transcript is never presented as simply "the" transcript.
|
||||||
|
|
||||||
7. **Configuration.** `TAPIR_TRANSCRIBE_SECOND_MODEL` (empty = off), `TAPIR_RECONCILE_MODEL`
|
7. **Configuration.** `TAPIR_RECONCILE_BASE_MODEL` (empty = off; the engine whose text is the base,
|
||||||
(required when a second engine is set), `TAPIR_RECONCILE_CHECK_ORDER` (off). The default job
|
e.g. `iguana/pianissimo` — `TAPIR_TRANSCRIBE_MODEL` stays the other engine, e.g. kb-whisper),
|
||||||
timeout rises from 30 to 60 minutes: ~4 min of kb-whisper and ~40 s of pianissimo per hour of audio,
|
`TAPIR_RECONCILE_MODEL` (the adjudicator; required when a base engine is set),
|
||||||
plus the adjudication.
|
`TAPIR_RECONCILE_CHECK_ORDER` (off). All three models must be local aliases (`koala/` or `iguana/`):
|
||||||
|
a cloud alias is a start-up error, and the base engine must differ from the other one. The default
|
||||||
|
job timeout rises from 30 to 60 minutes: ~4 min of kb-whisper and ~40 s of pianissimo per hour of
|
||||||
|
audio, plus the adjudication.
|
||||||
|
|
||||||
8. **Not decided here.** A cloud adjudicator (rejected: private content); tuning the prompt; making
|
8. **Not decided here.** A cloud adjudicator (rejected: private content); tuning the prompt; making
|
||||||
`gemma4-31b` the default; a "reconciling" progress state (the page says "transcribing" for the whole
|
`gemma4-31b` the default; a "reconciling" progress state (the page says "transcribing" for the whole
|
||||||
@@ -1624,6 +1627,13 @@ Mode on iguana is the unproven suspect).
|
|||||||
the wrong one (it was right 92% on a hand-picked set, ~97% for 31b).
|
the wrong one (it was right 92% on a hand-picked set, ~97% for 31b).
|
||||||
- Reading the page takes more words: the reconciliation summary is new surface to keep honest.
|
- Reading the page takes more words: the reconciliation summary is new surface to keep honest.
|
||||||
|
|
||||||
|
**First live smoke (2026-10-01, synthetic audio, 3 disputes):** all 3 were changed to the
|
||||||
|
alternative's reading; against the known script 2 were right and **1 was wrong** (a name replaced by a
|
||||||
|
pronoun, a correct base reading lost). That is one run of three, not a measurement, but it is the failure
|
||||||
|
this design must expect: the reconciler can overwrite a correct reading, which is why changed spots, not
|
||||||
|
only uncertain ones, are listed on the page, and why the decision below is a blind check on a real
|
||||||
|
recording.
|
||||||
|
|
||||||
**Validation / resolution triggers.**
|
**Validation / resolution triggers.**
|
||||||
- **The test that decides it:** re-upload the same real recording after merge; the speaker judges, blind,
|
- **The test that decides it:** re-upload the same real recording after merge; the speaker judges, blind,
|
||||||
~30 disputes where the reconciled text differs from pianissimo alone. Keep the feature only if the
|
~30 disputes where the reconciled text differs from pianissimo alone. Keep the feature only if the
|
||||||
|
|||||||
@@ -74,6 +74,11 @@ func TestE2E_UploadTranscribeAndChat(t *testing.T) {
|
|||||||
cfg.GatewayKey = key
|
cfg.GatewayKey = key
|
||||||
cfg.UploadsEnabled = true
|
cfg.UploadsEnabled = true
|
||||||
cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool")
|
cfg.UploadTmpDir = filepath.Join(t.TempDir(), "spool")
|
||||||
|
if os.Getenv("TAPIR_E2E_RECONCILE") == "1" {
|
||||||
|
// Two engines and a local model, as ADR-034 describes.
|
||||||
|
cfg.ReconcileBaseModel = envOrDefault("TAPIR_E2E_BASE_MODEL", "iguana/pianissimo")
|
||||||
|
cfg.ReconcileModel = envOrDefault("TAPIR_E2E_RECONCILE_MODEL", "iguana/gemma4-26b")
|
||||||
|
}
|
||||||
cfg.FFmpegBin = envOrDefault("TAPIR_E2E_FFMPEG", "ffmpeg")
|
cfg.FFmpegBin = envOrDefault("TAPIR_E2E_FFMPEG", "ffmpeg")
|
||||||
cfg.FFprobeBin = envOrDefault("TAPIR_E2E_FFPROBE", "ffprobe")
|
cfg.FFprobeBin = envOrDefault("TAPIR_E2E_FFPROBE", "ffprobe")
|
||||||
|
|
||||||
@@ -92,7 +97,7 @@ func TestE2E_UploadTranscribeAndChat(t *testing.T) {
|
|||||||
// 1. Upload.
|
// 1. Upload.
|
||||||
f, err := os.Open(audioPath)
|
f, err := os.Open(audioPath)
|
||||||
require.NoError(t, err)
|
require.NoError(t, err)
|
||||||
defer f.Close()
|
defer func() { _ = f.Close() }()
|
||||||
pr, pw := io.Pipe()
|
pr, pw := io.Pipe()
|
||||||
mw := multipart.NewWriter(pw)
|
mw := multipart.NewWriter(pw)
|
||||||
go func() {
|
go func() {
|
||||||
@@ -134,6 +139,15 @@ func TestE2E_UploadTranscribeAndChat(t *testing.T) {
|
|||||||
`SELECT status, endpoint, content, audio_seconds, had_video FROM uploaded_transcripts WHERE user_id = $1`, uid).
|
`SELECT status, endpoint, content, audio_seconds, had_video FROM uploaded_transcripts WHERE user_id = $1`, uid).
|
||||||
Scan(&status, &endpoint, &content, &secs, &hadVideo))
|
Scan(&status, &endpoint, &content, &secs, &hadVideo))
|
||||||
t.Logf("done: endpoint=%s audio_s=%.1f transcript_chars=%d had_video=%v", endpoint, secs, len(content), hadVideo)
|
t.Logf("done: endpoint=%s audio_s=%.1f transcript_chars=%d had_video=%v", endpoint, secs, len(content), hadVideo)
|
||||||
|
if os.Getenv("TAPIR_E2E_RECONCILE") == "1" {
|
||||||
|
var recJSON string
|
||||||
|
require.NoError(t, raw.QueryRow(ctx,
|
||||||
|
`SELECT COALESCE(reconciliation::text, 'NULL') FROM uploaded_transcripts WHERE user_id = $1`, uid).Scan(&recJSON))
|
||||||
|
require.NotEqual(t, "NULL", recJSON, "a two-engine run must record its reconciliation")
|
||||||
|
t.Logf("reconciliation: %.600s", recJSON)
|
||||||
|
assert.Contains(t, endpoint, "reconciled by", "every contributor is named in the endpoint")
|
||||||
|
assert.Contains(t, page, "Reconciled from two transcripts")
|
||||||
|
}
|
||||||
if os.Getenv("TAPIR_E2E_HAD_VIDEO") == "1" {
|
if os.Getenv("TAPIR_E2E_HAD_VIDEO") == "1" {
|
||||||
// A Teams meeting with the camera off: a video file whose audio is what matters.
|
// A Teams meeting with the camera off: a video file whose audio is what matters.
|
||||||
assert.True(t, hadVideo, "the file had a video track and the row must say so")
|
assert.True(t, hadVideo, "the file had a video track and the row must say so")
|
||||||
|
|||||||
+40
-10
@@ -7,10 +7,13 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"time"
|
||||||
|
|
||||||
"git.d-ma.be/mathias/tapir/internal/adapters/audio"
|
"git.d-ma.be/mathias/tapir/internal/adapters/audio"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/adapters/llm"
|
||||||
"git.d-ma.be/mathias/tapir/internal/adapters/transcribe"
|
"git.d-ma.be/mathias/tapir/internal/adapters/transcribe"
|
||||||
"git.d-ma.be/mathias/tapir/internal/config"
|
"git.d-ma.be/mathias/tapir/internal/config"
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/reconcile"
|
||||||
"git.d-ma.be/mathias/tapir/internal/uploads"
|
"git.d-ma.be/mathias/tapir/internal/uploads"
|
||||||
"git.d-ma.be/mathias/tapir/internal/web"
|
"git.d-ma.be/mathias/tapir/internal/web"
|
||||||
)
|
)
|
||||||
@@ -52,16 +55,7 @@ func buildUploads(ctx context.Context, cfg config.Config, st uploadsStore, log *
|
|||||||
log.Warn("uploads: marked interrupted by a restart (audio is not retained; they must be re-uploaded)", "count", n)
|
log.Warn("uploads: marked interrupted by a restart (audio is not retained; they must be re-uploaded)", "count", n)
|
||||||
}
|
}
|
||||||
|
|
||||||
// No overall client timeout: a 90 minute recording takes minutes. The deadline
|
worker := uploads.NewWorker(st, &audio.FFmpeg{Bin: cfg.FFmpegBin}, buildTranscriber(cfg),
|
||||||
// is the worker's per-job context.
|
|
||||||
client := &http.Client{}
|
|
||||||
primary := transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeModel, client)
|
|
||||||
var fallback transcribe.Backend
|
|
||||||
if cfg.TranscribeFallbackModel != "" && cfg.TranscribeFallbackModel != cfg.TranscribeModel {
|
|
||||||
fallback = transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeFallbackModel, client)
|
|
||||||
}
|
|
||||||
|
|
||||||
worker := uploads.NewWorker(st, &audio.FFmpeg{Bin: cfg.FFmpegBin}, transcribe.NewChain(primary, fallback),
|
|
||||||
cfg.UploadTmpDir, cfg.TranscribeTimeout)
|
cfg.UploadTmpDir, cfg.TranscribeTimeout)
|
||||||
intake := uploads.NewIntake(st, &audio.FFprobe{Bin: cfg.FFprobeBin}, worker,
|
intake := uploads.NewIntake(st, &audio.FFprobe{Bin: cfg.FFprobeBin}, worker,
|
||||||
uploadLimits(cfg), cfg.UploadTmpDir)
|
uploadLimits(cfg), cfg.UploadTmpDir)
|
||||||
@@ -98,3 +92,39 @@ func wireUploads(app *web.App, cfg config.Config, in *uploads.Intake) {
|
|||||||
app.UploadChatMaxChars = cfg.MaxUploadedTranscriptChars
|
app.UploadChatMaxChars = cfg.MaxUploadedTranscriptChars
|
||||||
app.UploadChatModel = cfg.UploadChatModel
|
app.UploadChatModel = cfg.UploadChatModel
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// adjudicatorTimeout bounds one adjudication call. A call takes ~2 s on gemma4-26b and ~10 s
|
||||||
|
// on 31b; this is the wait before an unresponsive model is counted as an error.
|
||||||
|
const adjudicatorTimeout = 2 * time.Minute
|
||||||
|
|
||||||
|
// buildTranscriber builds what the worker transcribes with. By default that is the
|
||||||
|
// single-engine chain (the primary, then the optional cloud fallback). When
|
||||||
|
// TAPIR_RECONCILE_BASE_MODEL is set it is the two-engine ensemble (ADR-034): the base
|
||||||
|
// engine and the primary run on the same WAV and a LOCAL model reconciles them. Config
|
||||||
|
// has already refused any non-local model, so nothing here can send private audio or
|
||||||
|
// text to a cloud model for reconciliation; the cloud fallback is unchanged.
|
||||||
|
func buildTranscriber(cfg config.Config) uploads.Transcriber {
|
||||||
|
// No overall client timeout: a 90 minute recording takes minutes. The deadline is
|
||||||
|
// the worker's per-job context.
|
||||||
|
client := &http.Client{}
|
||||||
|
primary := transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeModel, client)
|
||||||
|
var fallback transcribe.Backend
|
||||||
|
if cfg.TranscribeFallbackModel != "" && cfg.TranscribeFallbackModel != cfg.TranscribeModel {
|
||||||
|
fallback = transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.TranscribeFallbackModel, client)
|
||||||
|
}
|
||||||
|
if cfg.ReconcileBaseModel == "" {
|
||||||
|
return transcribe.NewChain(primary, fallback)
|
||||||
|
}
|
||||||
|
|
||||||
|
base := transcribe.NewGateway(cfg.GatewayURL, cfg.GatewayKey, cfg.ReconcileBaseModel, client)
|
||||||
|
// Temperature 0 and a short reply: the adjudicator's accuracy was measured that way.
|
||||||
|
chat := llm.New(cfg.GatewayURL, cfg.GatewayKey, cfg.ReconcileModel, adjudicatorTimeout,
|
||||||
|
llm.WithTemperature(0), llm.WithMaxTokens(200))
|
||||||
|
var adj reconcile.Adjudicator = reconcile.NewLLMAdjudicator(chat)
|
||||||
|
if cfg.ReconcileCheckOrder {
|
||||||
|
adj = reconcile.NewOrderChecked(adj)
|
||||||
|
}
|
||||||
|
return transcribe.NewEnsemble(base, primary, fallback, adj, reconcile.Options{
|
||||||
|
BaseEngine: cfg.ReconcileBaseModel, AltEngine: cfg.TranscribeModel, Reconciler: cfg.ReconcileModel,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|||||||
@@ -10,6 +10,7 @@ import (
|
|||||||
"github.com/stretchr/testify/assert"
|
"github.com/stretchr/testify/assert"
|
||||||
"github.com/stretchr/testify/require"
|
"github.com/stretchr/testify/require"
|
||||||
|
|
||||||
|
"git.d-ma.be/mathias/tapir/internal/adapters/transcribe"
|
||||||
"git.d-ma.be/mathias/tapir/internal/config"
|
"git.d-ma.be/mathias/tapir/internal/config"
|
||||||
"git.d-ma.be/mathias/tapir/internal/domain"
|
"git.d-ma.be/mathias/tapir/internal/domain"
|
||||||
"git.d-ma.be/mathias/tapir/internal/web"
|
"git.d-ma.be/mathias/tapir/internal/web"
|
||||||
@@ -154,3 +155,33 @@ func TestWireUploads_NilIntakeLeavesTheAppUntouched(t *testing.T) {
|
|||||||
assert.Nil(t, app.Uploads, "a nil *Intake must not become a non-nil interface that mounts the routes")
|
assert.Nil(t, app.Uploads, "a nil *Intake must not become a non-nil interface that mounts the routes")
|
||||||
assert.Zero(t, app.UploadChatMaxChars)
|
assert.Zero(t, app.UploadChatMaxChars)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBuildTranscriber_DefaultIsTheSingleEngineChainAsBefore(t *testing.T) {
|
||||||
|
cfg := uploadCfg(t, true)
|
||||||
|
|
||||||
|
tr := buildTranscriber(cfg)
|
||||||
|
|
||||||
|
assert.IsType(t, &transcribe.Chain{}, tr, "with no base engine configured nothing changes")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildTranscriber_ABaseEngineTurnsOnTheEnsemble(t *testing.T) {
|
||||||
|
cfg := uploadCfg(t, true)
|
||||||
|
cfg.ReconcileBaseModel = "iguana/pianissimo"
|
||||||
|
cfg.ReconcileModel = "iguana/gemma4-26b"
|
||||||
|
|
||||||
|
tr := buildTranscriber(cfg)
|
||||||
|
|
||||||
|
assert.IsType(t, &transcribe.Ensemble{}, tr)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildUploads_WiresTheEnsembleWhenConfigured(t *testing.T) {
|
||||||
|
cfg := uploadCfg(t, true)
|
||||||
|
cfg.ReconcileBaseModel = "iguana/pianissimo"
|
||||||
|
cfg.ReconcileModel = "iguana/gemma4-26b"
|
||||||
|
|
||||||
|
in, w, err := buildUploads(context.Background(), cfg, &fakeUploadsStore{}, quietLog())
|
||||||
|
|
||||||
|
require.NoError(t, err)
|
||||||
|
assert.NotNil(t, in)
|
||||||
|
assert.NotNil(t, w)
|
||||||
|
}
|
||||||
|
|||||||
+2
-1
@@ -153,6 +153,7 @@ erDiagram
|
|||||||
float audio_seconds
|
float audio_seconds
|
||||||
text error_kind "typed: interrupted | transcription_failed — never a provider string"
|
text error_kind "typed: interrupted | transcription_failed — never a provider string"
|
||||||
bool had_video "the file carried a video track; only its audio was used (migration 018)"
|
bool had_video "the file carried a video track; only its audio was used (migration 018)"
|
||||||
|
jsonb reconciliation "NULL for a single engine; else status, exact counts and a bounded list of contested spots (migration 019, ADR-034)"
|
||||||
timestamptz created_at
|
timestamptz created_at
|
||||||
timestamptz finished_at
|
timestamptz finished_at
|
||||||
}
|
}
|
||||||
@@ -224,7 +225,7 @@ mechanism.
|
|||||||
grounds that it holds only public caption content, and a recorded meeting is the opposite.
|
grounds that it holds only public caption content, and a recorded meeting is the opposite.
|
||||||
`user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing →
|
`user_id NOT NULL ... ON DELETE CASCADE`, FORCE RLS'd. `status` runs `queued → transcribing →
|
||||||
done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK
|
done | failed`; `done` requires `content`, `failed` requires a typed `error_kind` (CHECK
|
||||||
constraints). `had_video` (migration 018) records that the file carried a video track — a Teams meeting with the camera off is still a video file — so the page can say only its audio was used and the video was discarded. `endpoint` records which backend served it — `berget/...` means the audio left the
|
constraints). `reconciliation` (migration 019, ADR-034) is NULL when one engine produced the text, so the page can never claim a second opinion that was not there; otherwise it holds the status (`reconciled` / `single_engine` / `skipped`), exact counts (differences, settled automatically, decided by the model, changed, uncertain, model errors) and up to 200 contested spots (each a position in seconds, both readings and which was used). The spots hold fragments of the private transcript, which is why they live in this RLS-scoped row. `had_video` (migration 018) records that the file carried a video track — a Teams meeting with the camera off is still a video file — so the page can say only its audio was used and the video was discarded. `endpoint` records which backend served it — `berget/...` means the audio left the
|
||||||
homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90
|
homelab. The audio itself is never stored. `segments` keeps the timestamps (re-transcribing 90
|
||||||
minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion.
|
minutes costs ~7). Unlike `summary_actions`/`login_events` it **does** cascade on user deletion.
|
||||||
- **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the
|
- **LOGIN_EVENTS** (migration 010) — throttled one-row-per-(user, date) login stamp. Used by the
|
||||||
|
|||||||
@@ -28,6 +28,14 @@ it** — endpoints and aliases drift, and this file is a snapshot (2026-06-06),
|
|||||||
`iguana/gemma4-26b`, which silently drops the head above it), `TAPIR_UPLOAD_CHAT_MODEL`
|
`iguana/gemma4-26b`, which silently drops the head above it), `TAPIR_UPLOAD_CHAT_MODEL`
|
||||||
(`iguana/gemma4-26b`; `koala/phi4-mini`'s 8,192-token context 400s on an hour of speech). Needs `ffmpeg` + `ffprobe`
|
(`iguana/gemma4-26b`; `koala/phi4-mini`'s 8,192-token context 400s on an hour of speech). Needs `ffmpeg` + `ffprobe`
|
||||||
in the runtime image; Tapir's own ingress body limit for a 200 MB upload is unverified (infra).
|
in the runtime image; Tapir's own ingress body limit for a 200 MB upload is unverified (infra).
|
||||||
|
**Two-engine mode (ADR-034), off unless `TAPIR_RECONCILE_BASE_MODEL` is set:** it runs
|
||||||
|
`iguana/pianissimo` (the base text) and `TAPIR_TRANSCRIBE_MODEL` (kb-whisper) on the same audio, then a
|
||||||
|
LOCAL chat model (`TAPIR_RECONCILE_MODEL`, e.g. `iguana/gemma4-26b`: 92% against a speaker's blind answers
|
||||||
|
at ~2 s a call; `iguana/gemma4-31b` 97% but ~5x slower) chooses between their disagreements.
|
||||||
|
`TAPIR_RECONCILE_CHECK_ORDER=true` asks twice per dispute with the order swapped (doubles the time).
|
||||||
|
Every model must start `koala/` or `iguana/`: a cloud alias is a start-up error. The default job timeout is
|
||||||
|
60 minutes. `iguana/pianissimo` is KlangAI's Swedish ASR (a Parakeet/FastConformer on MLX, served by a small
|
||||||
|
FastAPI shell on iguana:8102) and speaks the same `/v1/audio/transcriptions` API.
|
||||||
- **Auth key:** treat as a secret via the SecretStore port; do not hardcode in committed code.
|
- **Auth key:** treat as a secret via the SecretStore port; do not hardcode in committed code.
|
||||||
NOTE: the documented `sk-local-123` returned `401` on `30401/v1/models` (2026-06-02) — it is
|
NOTE: the documented `sk-local-123` returned `401` on `30401/v1/models` (2026-06-02) — it is
|
||||||
stale or the gateway now enforces the `LITELLM_MASTER_KEY` (1Password `op://HomeLab/LITELLM_MASTER_KEY`).
|
stale or the gateway now enforces the `LITELLM_MASTER_KEY` (1Password `op://HomeLab/LITELLM_MASTER_KEY`).
|
||||||
|
|||||||
@@ -106,3 +106,53 @@ Feature: Upload an audio recording, get a private Swedish transcript, discuss it
|
|||||||
Given my recording's transcript is longer than the caption default
|
Given my recording's transcript is longer than the caption default
|
||||||
When I ask a question about it
|
When I ask a question about it
|
||||||
Then the transcript is sent with the upload budget, not the 18,000-character caption default
|
Then the transcript is sent with the upload budget, not the 18,000-character caption default
|
||||||
|
|
||||||
|
# ADR-034, two engines and a local reconciler (off unless TAPIR_RECONCILE_BASE_MODEL is set)
|
||||||
|
|
||||||
|
Scenario: Two engines transcribe the audio and their disagreements are reconciled
|
||||||
|
Given two-engine transcription is enabled
|
||||||
|
When a recording is transcribed
|
||||||
|
Then both engines run on the same audio, the fast one first
|
||||||
|
And a local model chooses between their disagreements
|
||||||
|
And the endpoint names every engine and the model that took part
|
||||||
|
|
||||||
|
Scenario: Trivial disagreements never reach the model
|
||||||
|
When the engines differ only by a compound written as one word or two, or by a filler word
|
||||||
|
Then the base reading is kept without a model call
|
||||||
|
|
||||||
|
Scenario: Differences close together are one decision
|
||||||
|
When two differences are separated by a single shared word
|
||||||
|
Then they are asked about as one dispute
|
||||||
|
|
||||||
|
Scenario: An unsure or changed reading is flagged for me
|
||||||
|
When the model changes a reading or cannot decide
|
||||||
|
Then the page lists the spot with a minute mark, both readings and which was used
|
||||||
|
And an unsure spot keeps the base reading and is marked uncertain
|
||||||
|
|
||||||
|
Scenario: The model cannot be a cloud model
|
||||||
|
When the reconciliation model or base engine is not a koala/ or iguana/ alias
|
||||||
|
Then the service refuses to start
|
||||||
|
And a cloud alias that merely contains a local host name is refused too
|
||||||
|
|
||||||
|
Scenario: One engine failing does not fail the upload
|
||||||
|
When one of the two engines fails
|
||||||
|
Then the other engine's transcript is used
|
||||||
|
And the page says only one engine produced it and why
|
||||||
|
And the cloud fallback is not used
|
||||||
|
|
||||||
|
Scenario: Only when both local engines fail does the cloud fallback run
|
||||||
|
When both local engines fail and a fallback is configured
|
||||||
|
Then the fallback transcribes it and the row records that it did
|
||||||
|
|
||||||
|
Scenario: A model that is down gives an unreconciled transcript, not a failed job
|
||||||
|
When the reconciliation model fails three times in a row
|
||||||
|
Then the base transcript is kept unreconciled with the reason
|
||||||
|
And the page says it could not be reconciled
|
||||||
|
|
||||||
|
Scenario: A cancelled job never reaches the cloud
|
||||||
|
When the job is cancelled during either engine
|
||||||
|
Then neither the second engine nor the cloud fallback is called
|
||||||
|
|
||||||
|
Scenario: A single-engine transcript claims no second opinion
|
||||||
|
When only one engine was configured
|
||||||
|
Then the page does not mention reconciliation
|
||||||
|
|||||||
@@ -25,12 +25,16 @@ import (
|
|||||||
// docs/homelab-integration.md.
|
// docs/homelab-integration.md.
|
||||||
const defaultMaxTokens = 8192
|
const defaultMaxTokens = 8192
|
||||||
|
|
||||||
|
// defaultTemperature is what every caller got before WithTemperature existed.
|
||||||
|
const defaultTemperature = 0.2
|
||||||
|
|
||||||
// Client calls an OpenAI-compatible chat completions endpoint.
|
// Client calls an OpenAI-compatible chat completions endpoint.
|
||||||
type Client struct {
|
type Client struct {
|
||||||
baseURL string
|
baseURL string
|
||||||
apiKey string
|
apiKey string
|
||||||
model string
|
model string
|
||||||
maxTokens int
|
maxTokens int
|
||||||
|
temp float64
|
||||||
httpClient *http.Client
|
httpClient *http.Client
|
||||||
usageHook func(model string, prompt, completion int)
|
usageHook func(model string, prompt, completion int)
|
||||||
}
|
}
|
||||||
@@ -51,6 +55,12 @@ func WithMaxTokens(n int) Option {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithTemperature overrides the sampling temperature. Zero IS a valid value and is sent
|
||||||
|
// as zero: the reconciler's accuracy was measured at temperature 0 and must run there.
|
||||||
|
func WithTemperature(t float64) Option {
|
||||||
|
return func(c *Client) { c.temp = t }
|
||||||
|
}
|
||||||
|
|
||||||
// WithUsageHook registers a callback fired after a successful completion with the
|
// WithUsageHook registers a callback fired after a successful completion with the
|
||||||
// model and the prompt/completion token counts from the response usage block. It
|
// model and the prompt/completion token counts from the response usage block. It
|
||||||
// keeps this copied, stdlib-only package (ADR-004) decoupled from metrics: the
|
// keeps this copied, stdlib-only package (ADR-004) decoupled from metrics: the
|
||||||
@@ -66,6 +76,7 @@ func New(baseURL, apiKey, model string, timeout time.Duration, opts ...Option) *
|
|||||||
apiKey: apiKey,
|
apiKey: apiKey,
|
||||||
model: model,
|
model: model,
|
||||||
maxTokens: defaultMaxTokens,
|
maxTokens: defaultMaxTokens,
|
||||||
|
temp: defaultTemperature,
|
||||||
httpClient: &http.Client{Timeout: timeout},
|
httpClient: &http.Client{Timeout: timeout},
|
||||||
}
|
}
|
||||||
for _, opt := range opts {
|
for _, opt := range opts {
|
||||||
@@ -105,7 +116,7 @@ func (c *Client) Complete(ctx context.Context, system, user string) (string, err
|
|||||||
{Role: "system", Content: system},
|
{Role: "system", Content: system},
|
||||||
{Role: "user", Content: user},
|
{Role: "user", Content: user},
|
||||||
},
|
},
|
||||||
Temperature: 0.2,
|
Temperature: c.temp,
|
||||||
MaxTokens: c.maxTokens,
|
MaxTokens: c.maxTokens,
|
||||||
}
|
}
|
||||||
b, err := json.Marshal(body)
|
b, err := json.Marshal(body)
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package llm
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// capturedTemperature runs one Complete against a fake gateway and returns the
|
||||||
|
// "temperature" field the request carried, and whether it was present at all.
|
||||||
|
func capturedTemperature(t *testing.T, opts ...Option) (float64, bool) {
|
||||||
|
t.Helper()
|
||||||
|
var body map[string]any
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
raw, _ := io.ReadAll(r.Body)
|
||||||
|
_ = json.Unmarshal(raw, &body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
_ = json.NewEncoder(w).Encode(map[string]any{"choices": []map[string]any{{"message": map[string]any{"role": "assistant", "content": "ok"}}}})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
if _, err := New(srv.URL, "", "m", 5*time.Second, opts...).Complete(context.Background(), "s", "u"); err != nil {
|
||||||
|
t.Fatalf("Complete: %v", err)
|
||||||
|
}
|
||||||
|
v, ok := body["temperature"].(float64)
|
||||||
|
return v, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClient_DefaultTemperatureIsUnchanged(t *testing.T) {
|
||||||
|
got, ok := capturedTemperature(t)
|
||||||
|
|
||||||
|
if !ok || got != 0.2 {
|
||||||
|
t.Fatalf("default temperature = %v (present %v), want 0.2: existing callers must not change behaviour", got, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClient_WithTemperatureZeroSendsExactlyZero(t *testing.T) {
|
||||||
|
// The reconciler's accuracy (92%/97%) was measured at temperature 0. Zero must be
|
||||||
|
// SENT, not treated as "unset" and replaced by the default.
|
||||||
|
got, ok := capturedTemperature(t, WithTemperature(0))
|
||||||
|
|
||||||
|
if !ok || got != 0 {
|
||||||
|
t.Fatalf("temperature = %v (present %v), want an explicit 0", got, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClient_WithTemperatureSendsTheGivenValue(t *testing.T) {
|
||||||
|
got, ok := capturedTemperature(t, WithTemperature(0.7))
|
||||||
|
|
||||||
|
if !ok || got != 0.7 {
|
||||||
|
t.Fatalf("temperature = %v (present %v), want 0.7", got, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -62,7 +62,7 @@ func (g *Gateway) Transcribe(ctx context.Context, path string) (domain.Transcrip
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return domain.Transcription{}, fmt.Errorf("transcribe: open audio: %w", err)
|
return domain.Transcription{}, fmt.Errorf("transcribe: open audio: %w", err)
|
||||||
}
|
}
|
||||||
defer f.Close()
|
defer func() { _ = f.Close() }()
|
||||||
info, err := f.Stat()
|
info, err := f.Stat()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return domain.Transcription{}, fmt.Errorf("transcribe: stat audio: %w", err)
|
return domain.Transcription{}, fmt.Errorf("transcribe: stat audio: %w", err)
|
||||||
@@ -95,7 +95,7 @@ func (g *Gateway) Transcribe(ctx context.Context, path string) (domain.Transcrip
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err)
|
return domain.Transcription{}, fmt.Errorf("transcribe: %s: %w", g.model, err)
|
||||||
}
|
}
|
||||||
defer resp.Body.Close()
|
defer func() { _ = resp.Body.Close() }()
|
||||||
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
if resp.StatusCode < 200 || resp.StatusCode > 299 {
|
||||||
snip, _ := io.ReadAll(io.LimitReader(resp.Body, 200))
|
snip, _ := io.ReadAll(io.LimitReader(resp.Body, 200))
|
||||||
return domain.Transcription{}, &StatusError{Code: resp.StatusCode, Snippet: strings.TrimSpace(string(snip))}
|
return domain.Transcription{}, &StatusError{Code: resp.StatusCode, Snippet: strings.TrimSpace(string(snip))}
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -42,6 +43,16 @@ type UploadConfig struct {
|
|||||||
// NOT the summarizer default: koala/phi4-mini has an 8,192-token context and
|
// NOT the summarizer default: koala/phi4-mini has an 8,192-token context and
|
||||||
// returns HTTP 400 on an hour of speech (~13,000 tokens).
|
// returns HTTP 400 on an hour of speech (~13,000 tokens).
|
||||||
UploadChatModel string
|
UploadChatModel string
|
||||||
|
// ReconcileBaseModel turns on two-engine transcription (ADR-034): the engine whose
|
||||||
|
// TEXT is the base of the merged transcript (e.g. iguana/pianissimo). TranscribeModel
|
||||||
|
// stays the other engine (e.g. kb-whisper). Empty = off.
|
||||||
|
ReconcileBaseModel string
|
||||||
|
// ReconcileModel is the LOCAL chat model that chooses between the two engines'
|
||||||
|
// disagreements. Required when ReconcileBaseModel is set.
|
||||||
|
ReconcileModel string
|
||||||
|
// ReconcileCheckOrder asks the model twice per dispute with the reading order
|
||||||
|
// swapped and keeps only agreement. It doubles the time; off by default.
|
||||||
|
ReconcileCheckOrder bool
|
||||||
}
|
}
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -49,7 +60,7 @@ const (
|
|||||||
defaultUploadMaxMinutes = 90
|
defaultUploadMaxMinutes = 90
|
||||||
defaultTranscribeModel = "iguana/kb-whisper"
|
defaultTranscribeModel = "iguana/kb-whisper"
|
||||||
defaultTranscribeFallbackModel = "berget/kb-whisper"
|
defaultTranscribeFallbackModel = "berget/kb-whisper"
|
||||||
defaultTranscribeTimeout = 30 * time.Minute
|
defaultTranscribeTimeout = 60 * time.Minute
|
||||||
defaultMaxUploadedChars = 90000
|
defaultMaxUploadedChars = 90000
|
||||||
defaultUploadChatModel = "iguana/gemma4-26b"
|
defaultUploadChatModel = "iguana/gemma4-26b"
|
||||||
)
|
)
|
||||||
@@ -62,6 +73,8 @@ func loadUploadConfig() (UploadConfig, error) {
|
|||||||
FFmpegBin: envOr("TAPIR_FFMPEG_BIN", "ffmpeg"),
|
FFmpegBin: envOr("TAPIR_FFMPEG_BIN", "ffmpeg"),
|
||||||
FFprobeBin: envOr("TAPIR_FFPROBE_BIN", "ffprobe"),
|
FFprobeBin: envOr("TAPIR_FFPROBE_BIN", "ffprobe"),
|
||||||
UploadChatModel: envOr("TAPIR_UPLOAD_CHAT_MODEL", defaultUploadChatModel),
|
UploadChatModel: envOr("TAPIR_UPLOAD_CHAT_MODEL", defaultUploadChatModel),
|
||||||
|
ReconcileBaseModel: os.Getenv("TAPIR_RECONCILE_BASE_MODEL"),
|
||||||
|
ReconcileModel: os.Getenv("TAPIR_RECONCILE_MODEL"),
|
||||||
}
|
}
|
||||||
|
|
||||||
if v := os.Getenv("TAPIR_UPLOADS_ENABLED"); v != "" {
|
if v := os.Getenv("TAPIR_UPLOADS_ENABLED"); v != "" {
|
||||||
@@ -93,6 +106,17 @@ func loadUploadConfig() (UploadConfig, error) {
|
|||||||
}
|
}
|
||||||
u.TranscribeTimeout = timeout
|
u.TranscribeTimeout = timeout
|
||||||
|
|
||||||
|
if v := os.Getenv("TAPIR_RECONCILE_CHECK_ORDER"); v != "" {
|
||||||
|
on, err := strconv.ParseBool(v)
|
||||||
|
if err != nil {
|
||||||
|
return UploadConfig{}, fmt.Errorf("config: TAPIR_RECONCILE_CHECK_ORDER=%q: %w", v, err)
|
||||||
|
}
|
||||||
|
u.ReconcileCheckOrder = on
|
||||||
|
}
|
||||||
|
if err := u.validateReconcile(); err != nil {
|
||||||
|
return UploadConfig{}, err
|
||||||
|
}
|
||||||
|
|
||||||
chars, err := intOr("TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", defaultMaxUploadedChars)
|
chars, err := intOr("TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", defaultMaxUploadedChars)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return UploadConfig{}, err
|
return UploadConfig{}, err
|
||||||
@@ -116,3 +140,40 @@ func positiveInt(key string, fallback int) (int, error) {
|
|||||||
}
|
}
|
||||||
return n, nil
|
return n, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// localPrefixes are the host prefixes of the models that run on the homelab. Private
|
||||||
|
// meeting content may be sent to these and to nothing else for reconciliation
|
||||||
|
// (ADR-034): a cloud alias is refused at start-up rather than discovered later.
|
||||||
|
var localPrefixes = []string{"koala/", "iguana/"}
|
||||||
|
|
||||||
|
func isLocalModel(alias string) bool {
|
||||||
|
for _, p := range localPrefixes {
|
||||||
|
if strings.HasPrefix(alias, p) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// validateReconcile checks the two-engine settings. They are inert while no base
|
||||||
|
// engine is set; once one is, the setup must be complete and entirely local.
|
||||||
|
func (u UploadConfig) validateReconcile() error {
|
||||||
|
if u.ReconcileBaseModel == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if u.ReconcileModel == "" {
|
||||||
|
return fmt.Errorf("config: TAPIR_RECONCILE_BASE_MODEL is set but TAPIR_RECONCILE_MODEL (the adjudicator) is not")
|
||||||
|
}
|
||||||
|
for _, m := range []struct{ name, alias string }{
|
||||||
|
{"TAPIR_RECONCILE_BASE_MODEL", u.ReconcileBaseModel},
|
||||||
|
{"TAPIR_RECONCILE_MODEL", u.ReconcileModel},
|
||||||
|
} {
|
||||||
|
if !isLocalModel(m.alias) {
|
||||||
|
return fmt.Errorf("config: %s=%q is not a local model (koala/ or iguana/): private meeting content must not leave the homelab for reconciliation", m.name, m.alias)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if u.ReconcileBaseModel == u.TranscribeModel {
|
||||||
|
return fmt.Errorf("config: TAPIR_RECONCILE_BASE_MODEL and TAPIR_TRANSCRIBE_MODEL are both %q: two engines must differ", u.TranscribeModel)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ func clearUploadEnv(t *testing.T) {
|
|||||||
"TAPIR_TRANSCRIBE_MODEL", "TAPIR_TRANSCRIBE_FALLBACK_MODEL", "TAPIR_TRANSCRIBE_TIMEOUT",
|
"TAPIR_TRANSCRIBE_MODEL", "TAPIR_TRANSCRIBE_FALLBACK_MODEL", "TAPIR_TRANSCRIBE_TIMEOUT",
|
||||||
"TAPIR_UPLOAD_TMP_DIR", "TAPIR_FFMPEG_BIN", "TAPIR_FFPROBE_BIN",
|
"TAPIR_UPLOAD_TMP_DIR", "TAPIR_FFMPEG_BIN", "TAPIR_FFPROBE_BIN",
|
||||||
"TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", "TAPIR_UPLOAD_CHAT_MODEL",
|
"TAPIR_MAX_UPLOADED_TRANSCRIPT_CHARS", "TAPIR_UPLOAD_CHAT_MODEL",
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL", "TAPIR_RECONCILE_MODEL", "TAPIR_RECONCILE_CHECK_ORDER",
|
||||||
} {
|
} {
|
||||||
unset(t, k)
|
unset(t, k)
|
||||||
}
|
}
|
||||||
@@ -43,8 +44,11 @@ func TestLoad_UploadDefaults(t *testing.T) {
|
|||||||
if c.TranscribeFallbackModel != "berget/kb-whisper" {
|
if c.TranscribeFallbackModel != "berget/kb-whisper" {
|
||||||
t.Errorf("TranscribeFallbackModel = %q", c.TranscribeFallbackModel)
|
t.Errorf("TranscribeFallbackModel = %q", c.TranscribeFallbackModel)
|
||||||
}
|
}
|
||||||
if c.TranscribeTimeout != 30*time.Minute {
|
if c.TranscribeTimeout != 60*time.Minute {
|
||||||
t.Errorf("TranscribeTimeout = %v, want 30m (a 90 min recording took ~7.5 min)", c.TranscribeTimeout)
|
t.Errorf("TranscribeTimeout = %v, want 60m (ADR-034: two engines plus ~300 adjudications)", c.TranscribeTimeout)
|
||||||
|
}
|
||||||
|
if c.ReconcileBaseModel != "" || c.ReconcileModel != "" || c.ReconcileCheckOrder {
|
||||||
|
t.Errorf("reconciliation must be OFF by default: %q %q %v", c.ReconcileBaseModel, c.ReconcileModel, c.ReconcileCheckOrder)
|
||||||
}
|
}
|
||||||
if c.UploadTmpDir != os.TempDir() {
|
if c.UploadTmpDir != os.TempDir() {
|
||||||
t.Errorf("UploadTmpDir = %q, want os.TempDir()", c.UploadTmpDir)
|
t.Errorf("UploadTmpDir = %q, want os.TempDir()", c.UploadTmpDir)
|
||||||
@@ -139,3 +143,75 @@ func TestLoad_UploadRejectsInvalidValues(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestLoad_ReconcileSettingsAreParsed(t *testing.T) {
|
||||||
|
clearUploadEnv(t)
|
||||||
|
setEnv(t, map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo",
|
||||||
|
"TAPIR_RECONCILE_MODEL": "iguana/gemma4-31b",
|
||||||
|
"TAPIR_RECONCILE_CHECK_ORDER": "true",
|
||||||
|
})
|
||||||
|
|
||||||
|
c, err := Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.ReconcileBaseModel != "iguana/pianissimo" || c.ReconcileModel != "iguana/gemma4-31b" || !c.ReconcileCheckOrder {
|
||||||
|
t.Errorf("not applied: %+v", c.UploadConfig)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_ReconcileRefusesAnUnsafeOrIncompleteSetup(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
env map[string]string
|
||||||
|
}{
|
||||||
|
{"a base engine with no adjudicator", map[string]string{"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo"}},
|
||||||
|
{"a cloud adjudicator: private meeting content must not reach it", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "berget/mistral-small",
|
||||||
|
}},
|
||||||
|
{"a cloud adjudicator from another provider", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "gemini/gemini-2.5-pro",
|
||||||
|
}},
|
||||||
|
{"an adjudicator alias with no host prefix", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "gemma4-26b",
|
||||||
|
}},
|
||||||
|
{"a cloud alias that merely contains a local host name", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "berget/iguana/gemma4-26b",
|
||||||
|
}},
|
||||||
|
{"a cloud base engine", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "berget/kb-whisper", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b",
|
||||||
|
}},
|
||||||
|
{"the same engine twice", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/kb-whisper", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b",
|
||||||
|
}},
|
||||||
|
{"a bad order-check flag", map[string]string{
|
||||||
|
"TAPIR_RECONCILE_BASE_MODEL": "iguana/pianissimo", "TAPIR_RECONCILE_MODEL": "iguana/gemma4-26b", "TAPIR_RECONCILE_CHECK_ORDER": "perhaps",
|
||||||
|
}},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
clearUploadEnv(t)
|
||||||
|
setEnv(t, tt.env)
|
||||||
|
|
||||||
|
if _, err := Load(); err == nil {
|
||||||
|
t.Errorf("Load accepted %v", tt.env)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoad_AnAdjudicatorWithoutABaseEngineIsIgnoredNotAnError(t *testing.T) {
|
||||||
|
clearUploadEnv(t)
|
||||||
|
t.Setenv("TAPIR_RECONCILE_MODEL", "iguana/gemma4-26b")
|
||||||
|
|
||||||
|
c, err := Load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Load: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if c.ReconcileBaseModel != "" {
|
||||||
|
t.Errorf("reconciliation must stay off without a base engine")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -122,6 +122,18 @@ var scenarioCoverage = map[string]string{
|
|||||||
"I can ask about my recording and get a grounded answer": "TestUploadChatPost_SendsTheOwnersTranscriptWithTheUploadBudget",
|
"I can ask about my recording and get a grounded answer": "TestUploadChatPost_SendsTheOwnersTranscriptWithTheUploadBudget",
|
||||||
"A long recording is not cut to its first third": "TestAnswerMaxCharsOverridesTheServiceCap",
|
"A long recording is not cut to its first third": "TestAnswerMaxCharsOverridesTheServiceCap",
|
||||||
|
|
||||||
|
// audio_upload.feature, ADR-034
|
||||||
|
"Two engines transcribe the audio and their disagreements are reconciled": "TestEnsemble_BothEnginesRunInOrderAndTheirDisagreementIsReconciled",
|
||||||
|
"Trivial disagreements never reach the model": "TestReconcile_TrivialDisputesAreResolvedWithoutAskingTheModel",
|
||||||
|
"Differences close together are one decision": "TestFindDisputes_TwoDifferencesSeparatedByOneSharedWordAreOneDispute",
|
||||||
|
"An unsure or changed reading is flagged for me": "TestUploadStatus_ListsTheContestedSpotsWithAMinuteMarkBothReadingsAndWhichWasUsed",
|
||||||
|
"The model cannot be a cloud model": "TestLoad_ReconcileRefusesAnUnsafeOrIncompleteSetup",
|
||||||
|
"One engine failing does not fail the upload": "TestEnsemble_WhenOneEngineFailsTheOtherIsUsedAndItIsSaidSo",
|
||||||
|
"Only when both local engines fail does the cloud fallback run": "TestEnsemble_OnlyWhenBothLocalEnginesFailDoesTheCloudFallbackRun",
|
||||||
|
"A model that is down gives an unreconciled transcript, not a failed job": "TestEnsemble_AModelThatIsDownGivesTheBaseTextUnreconciledNotAFailedJob",
|
||||||
|
"A cancelled job never reaches the cloud": "TestEnsemble_ACancelledJobNeverReachesTheCloud",
|
||||||
|
"A single-engine transcript claims no second opinion": "TestUploadStatus_ASingleEngineTranscriptDoesNotClaimASecondOpinion",
|
||||||
|
|
||||||
// summarize_new_video.feature
|
// summarize_new_video.feature
|
||||||
"A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered",
|
"A subscribed channel posts a video that has captions": "TestSubscribedVideoWithCaptionsIsSummarizedAndDelivered",
|
||||||
"A subscribed channel posts a video with no usable transcript": "TestVideoWithNoTranscriptIsSkipped",
|
"A subscribed channel posts a video with no usable transcript": "TestVideoWithNoTranscriptIsSkipped",
|
||||||
|
|||||||
Reference in New Issue
Block a user