TAPIR_E2E_RECONCILE=1 runs pianissimo + kb-whisper + gemma4-26b on the audio and
asserts the reconciliation is recorded, every contributor is named in the endpoint, and
the page says what was done. Run 2026-10-01 on 82 s of synthetic Swedish: 3 disputes, all
3 changed; against the known script 2 were right (virket, a missing 'procent') and 1 was
WRONG (Maria -> Vi replaced a correct base reading). A smoke test, not a measurement.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Default is the single-engine chain exactly as before. With TAPIR_RECONCILE_BASE_MODEL set,
the base engine and the primary run on the same WAV and a LOCAL model (config has already
refused any non-local alias) reconciles them, at temperature 0 with a short reply -- the
conditions its accuracy was measured under. The cloud fallback is unchanged and runs
only when both local engines fail.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Found because the project's lint step cannot fail: the Taskfile runs
'golangci-lint run ./... || echo "not installed -- skipping"', so a failing lint
prints that message and exits 0. CI's log showed these findings and still passed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Zero is sent as zero (the field has no omitempty). The reconciler's measured accuracy
was at temperature 0, so it runs there; every other caller keeps 0.2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The reconciler was measured at temperature 0 but the client hardcodes 0.2. Zero must
be sent explicitly, and the default must not change for existing callers.
Red: WithTemperature is undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TAPIR_RECONCILE_BASE_MODEL turns it on and needs TAPIR_RECONCILE_MODEL; every model must
be a local alias (koala/ or iguana/, matched as a PREFIX): a cloud adjudicator is a
start-up error because private meeting content must not reach it; the two engines must
differ; default job timeout 60m. Off by default. Mutation-checked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Off by default; a base engine needs an adjudicator; every model must be a local alias
(koala/ or iguana/): a cloud adjudicator is a start-up error because private meeting
content must not reach it; the base engine must differ from the other engine; default
job timeout 60m (two engines plus adjudication).
Red: the Config fields do not exist.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TAPIR_RECONCILE_BASE_MODEL, not 'second model': the engine added to turn the
feature on is the one whose TEXT is the base, and the old name hid that. All three
models must be local aliases; the base must differ from the other engine. ADR is
still Proposed, so this is an in-place correction.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reconciled: the two engines, the deciding model and the exact counts; the contested
spots (changed or uncertain) listed with a minute mark and both readings, the first
50 shown and the rest counted from the exact totals. Single-engine and
model-unavailable results say why. A nil reconciliation renders nothing. Readings are
escaped. Mutation-checked: the cap, the minute rounding and the 'more' count.
uploads.go and the templ pair are one commit: the helpers and the template need each other.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reconciled: who the two engines and the deciding model were, and the counts
(differences, settled automatically, decided by the model, uncertain). The contested
spots are listed with a minute mark, both readings and which was used, uncertain ones
marked, readings escaped, and a very long list shortened with its true count. A
single-engine transcript makes no claim of a second opinion; one-engine and
model-unavailable cases say why.
Red: the page renders none of this yet.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
CompleteUpload writes it (SQL NULL when there was none, never the JSON text 'null':
the page claims a second opinion only if the column is non-NULL); GetUpload reads it,
List does not. Mutation-checked.
domain.go and the store are one commit: the store tests need the new field.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
JSONB, NULL for a single engine so the page cannot claim a second opinion. On the
already FORCE'd, RLS-scoped table, so the exact-set isolation test is unaffected.
Up and down are one commit: golang-migrate pairs them.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Counts and contested spots round-trip; a single-engine transcript has NO
reconciliation (nil, not empty -- the page must not claim a second opinion); a
skipped reconciliation keeps its reason.
Red: Transcription.Reconciliation is stored nowhere and UploadedTranscript has none.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Runs the fast engine then the other from one WAV and reconciles; one engine down
uses the other and says so; the model down gives the base text unreconciled; the
cloud fallback runs only when BOTH local engines fail and is recorded as such; a
cancelled job stops everything and never reaches the cloud. The endpoint lists every
contributor. Mutation-checked, including both cancellation points.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing showed each of the two cancellation checks was redundant with the
other in my first test. Now a cancel during the first engine must not start the second
or the cloud, and a cancel during the second must not be recorded as a single-engine
success ('kb-whisper failed').
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both run in order (the fast one first) and are reconciled; identical texts cost no
model call; one engine down uses the other and says so, with the cloud fallback
untouched; only when BOTH local engines fail does the fallback run, and the row says
berget served it; both failing with no fallback reports both errors; a cancelled
job never reaches the cloud; a model that is down yields the base text unreconciled,
not a failed job.
Red: transcribe.Ensemble is undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Base text stands except where the model picks the alternative's reading; trivial
disputes skip the model; the reading order alternates; the base's context goes to
the model. Fails soft: three consecutive model errors return the base text
unreconciled (none of the partial changes) with the reason; a verdict returned with
an error is ignored; a cancelled job is an error, not 'skipped'. Counts are exact,
the stored list of contested spots is capped at 200 and carries the second each
starts at. Mutation-checked on nine behaviours.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing showed the forced-unsure line was untested because my fake
always returned unsure with its errors.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Status (reconciled / single_engine / skipped), exact counts, and a bounded list of
contested spots (changed or unsure) with a position and both readings.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Disputes separated by a single shared word are merged into one decision by design,
so several of my fixtures were secretly one dispute. Corrected before any
implementation existed; the rule is unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Identical texts and trivial disputes cost no model call; an alt verdict takes the
alternative's own words, base/unsure keep the base and only unsure is flagged;
insertions and deletions land in the right place; the reading order alternates; the
model gets the base's context. Three consecutive model errors stop the whole
reconciliation (base text, none of the partial changes, reason recorded) while a
success in between resets the breaker; a cancelled job is an error, not 'skipped'.
Spots carry the second they start at; the stored list is capped, the counts are not.
Red: Reconcile, Options, MaxContested and domain.Reconcile* are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sends the exact wording that scored 92% (gemma4-26b) and 97% (gemma4-31b); maps
1/2 back to base/alt under both orders; an unreadable reply is unsure, a transport
error is an error. OrderChecked asks twice with the order swapped and keeps only
agreement. Mutation-checked: answer mapping, unparseable-as-guess, the word
boundary and the order check each turn a test red.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing the adjudicator showed dropping the word boundary went unnoticed:
'ANSWER: 12' would be read as 1.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Readings in the requested order; 1/2 mapped to base/alt under both orders;
UNSURE, an unparseable or out-of-range reply is unsure rather than a guess; an
empty reading is described, not blank; a model error is an error, distinguishable
from 'unsure'. The optional order-checked wrapper asks twice with the order swapped
and keeps only agreement.
Red: Request, Verdict, NewLLMAdjudicator and NewOrderChecked are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Disputes carry both readings with their own spelling; differences separated by
one shared word merge into one decision. Compound splits/joins and one or two
filler particles are resolved without a model; 'nej', numbers and content words
never are. Mutation-checked (merge distance, filler limit, 'nej', compound rule).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A dispute carries both readings with their own spelling; differences separated by
a single shared word are one decision, two shared words keep them apart; compound
splits/joins and a short run of filler words are resolved without a model, while
numbers, content words, and 'nej' (an answer) are not.
Red: FindDisputes, AutoResolve and ContextAround are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Prefix/suffix trimmed first; adjacent deletes and inserts read as one Replace;
beyond 3,000 edits (two texts sharing almost nothing) the middle is one Replace
instead of gigabytes of trace. Mutation-checked: a wrong tie-break, missing
replace coalescing, wrong suffix accounting and a punctuation-bearing key each
turn tests red.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A random-input property test compares the equal runs against a dynamic-programming
LCS: a diff that is valid but not minimal would turn real agreement into disputes
the model is then asked about. A 6,500-word, 380-edit case mirrors a real hour.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Keeps the word as written, keys it without case/punctuation/hyphens/plus, joins a
lone dash or full stop to its neighbour so it is not a word the engines can
disagree about. Red: package reconcile does not exist.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Proposed, decider pending. Two local engines on one WAV; word-level diff with
pianissimo as the base (readability and timing, NOT accuracy -- that was a tie);
trivial disputes resolved without a model; the rest adjudicated by a LOCAL model
(alias must start koala/ or iguana/) whose only freedom is choosing between the
two readings. Reconciliation never fails a job; the page says what was done and
lists the uncertain spots. Off by default. Validation: re-upload the real
recording and keep the feature only if the reconciled reading is right >= 85% of
the time and no worse than the better single engine.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Eleven local models, 120 calls each, scored against the speaker's blind answers
on 30 disagreements, with pass criteria fixed before the run. gemma4-31b (97%)
and gemma4-26b (92%) pass; nothing else clears accuracy and order-consistency.
Records why 31b is slow (MLX gives the same speed; Low Power Mode is the
unproven suspect) and the caveats: 30 items, one recording, thinking models
under-budgeted. Aggregate numbers only; this repo is public.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A near tie on accuracy (blind A/B by the speaker: 17 vs 13 on the 30 spans that
change meaning) and a 6x difference in speed (40 s vs 4m08s for 63.7 min).
Different error profiles: pianissimo better on terms and numbers, kb-whisper on
everyday words. Aggregate numbers only -- no transcript text or domain terms,
this repo is public. Also records the mid-word wrapping bug the comparison found
and its fix.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
whisper.cpp wraps at ~60 chars and can wrap inside a word; the response's text
field joins segments with newlines, so chat saw 'nyt' / 't godkännande'. Text
is now rebuilt from the raw segments: concatenated when any segment carries a
leading space (whisper.cpp), joined by a space when none do (berget's sentence
level), whitespace collapsed; the text field is only a fallback.
Checked against the first real one-hour transcript with pianissimo's vocabulary
as an independent yardstick: 36 spans changed, 31 now consist entirely of words
pianissimo also produced, words unknown to pianissimo fell 224 -> 177 and word
agreement rose 92.4% -> 93.1%. Five rejoined spans still contain an unknown
word (names/loanwords or an unlucky glue) -- a residual risk, small.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
