TAPIR_E2E_RECONCILE=1 runs pianissimo + kb-whisper + gemma4-26b on the audio and
asserts the reconciliation is recorded, every contributor is named in the endpoint, and
the page says what was done. Run 2026-10-01 on 82 s of synthetic Swedish: 3 disputes, all
3 changed; against the known script 2 were right (virket, a missing 'procent') and 1 was
WRONG (Maria -> Vi replaced a correct base reading). A smoke test, not a measurement.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Default is the single-engine chain exactly as before. With TAPIR_RECONCILE_BASE_MODEL set,
the base engine and the primary run on the same WAV and a LOCAL model (config has already
refused any non-local alias) reconciles them, at temperature 0 with a short reply -- the
conditions its accuracy was measured under. The cloud fallback is unchanged and runs
only when both local engines fail.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Found because the project's lint step cannot fail: the Taskfile runs
'golangci-lint run ./... || echo "not installed -- skipping"', so a failing lint
prints that message and exits 0. CI's log showed these findings and still passed.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Zero is sent as zero (the field has no omitempty). The reconciler's measured accuracy
was at temperature 0, so it runs there; every other caller keeps 0.2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The reconciler was measured at temperature 0 but the client hardcodes 0.2. Zero must
be sent explicitly, and the default must not change for existing callers.
Red: WithTemperature is undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TAPIR_RECONCILE_BASE_MODEL turns it on and needs TAPIR_RECONCILE_MODEL; every model must
be a local alias (koala/ or iguana/, matched as a PREFIX): a cloud adjudicator is a
start-up error because private meeting content must not reach it; the two engines must
differ; default job timeout 60m. Off by default. Mutation-checked.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Off by default; a base engine needs an adjudicator; every model must be a local alias
(koala/ or iguana/): a cloud adjudicator is a start-up error because private meeting
content must not reach it; the base engine must differ from the other engine; default
job timeout 60m (two engines plus adjudication).
Red: the Config fields do not exist.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TAPIR_RECONCILE_BASE_MODEL, not 'second model': the engine added to turn the
feature on is the one whose TEXT is the base, and the old name hid that. All three
models must be local aliases; the base must differ from the other engine. ADR is
still Proposed, so this is an in-place correction.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reconciled: the two engines, the deciding model and the exact counts; the contested
spots (changed or uncertain) listed with a minute mark and both readings, the first
50 shown and the rest counted from the exact totals. Single-engine and
model-unavailable results say why. A nil reconciliation renders nothing. Readings are
escaped. Mutation-checked: the cap, the minute rounding and the 'more' count.
uploads.go and the templ pair are one commit: the helpers and the template need each other.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Reconciled: who the two engines and the deciding model were, and the counts
(differences, settled automatically, decided by the model, uncertain). The contested
spots are listed with a minute mark, both readings and which was used, uncertain ones
marked, readings escaped, and a very long list shortened with its true count. A
single-engine transcript makes no claim of a second opinion; one-engine and
model-unavailable cases say why.
Red: the page renders none of this yet.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
CompleteUpload writes it (SQL NULL when there was none, never the JSON text 'null':
the page claims a second opinion only if the column is non-NULL); GetUpload reads it,
List does not. Mutation-checked.
domain.go and the store are one commit: the store tests need the new field.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
JSONB, NULL for a single engine so the page cannot claim a second opinion. On the
already FORCE'd, RLS-scoped table, so the exact-set isolation test is unaffected.
Up and down are one commit: golang-migrate pairs them.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Counts and contested spots round-trip; a single-engine transcript has NO
reconciliation (nil, not empty -- the page must not claim a second opinion); a
skipped reconciliation keeps its reason.
Red: Transcription.Reconciliation is stored nowhere and UploadedTranscript has none.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Runs the fast engine then the other from one WAV and reconciles; one engine down
uses the other and says so; the model down gives the base text unreconciled; the
cloud fallback runs only when BOTH local engines fail and is recorded as such; a
cancelled job stops everything and never reaches the cloud. The endpoint lists every
contributor. Mutation-checked, including both cancellation points.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing showed each of the two cancellation checks was redundant with the
other in my first test. Now a cancel during the first engine must not start the second
or the cloud, and a cancel during the second must not be recorded as a single-engine
success ('kb-whisper failed').
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Both run in order (the fast one first) and are reconciled; identical texts cost no
model call; one engine down uses the other and says so, with the cloud fallback
untouched; only when BOTH local engines fail does the fallback run, and the row says
berget served it; both failing with no fallback reports both errors; a cancelled
job never reaches the cloud; a model that is down yields the base text unreconciled,
not a failed job.
Red: transcribe.Ensemble is undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Base text stands except where the model picks the alternative's reading; trivial
disputes skip the model; the reading order alternates; the base's context goes to
the model. Fails soft: three consecutive model errors return the base text
unreconciled (none of the partial changes) with the reason; a verdict returned with
an error is ignored; a cancelled job is an error, not 'skipped'. Counts are exact,
the stored list of contested spots is capped at 200 and carries the second each
starts at. Mutation-checked on nine behaviours.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing showed the forced-unsure line was untested because my fake
always returned unsure with its errors.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Status (reconciled / single_engine / skipped), exact counts, and a bounded list of
contested spots (changed or unsure) with a position and both readings.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Disputes separated by a single shared word are merged into one decision by design,
so several of my fixtures were secretly one dispute. Corrected before any
implementation existed; the rule is unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Identical texts and trivial disputes cost no model call; an alt verdict takes the
alternative's own words, base/unsure keep the base and only unsure is flagged;
insertions and deletions land in the right place; the reading order alternates; the
model gets the base's context. Three consecutive model errors stop the whole
reconciliation (base text, none of the partial changes, reason recorded) while a
success in between resets the breaker; a cancelled job is an error, not 'skipped'.
Spots carry the second they start at; the stored list is capped, the counts are not.
Red: Reconcile, Options, MaxContested and domain.Reconcile* are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sends the exact wording that scored 92% (gemma4-26b) and 97% (gemma4-31b); maps
1/2 back to base/alt under both orders; an unreadable reply is unsure, a transport
error is an error. OrderChecked asks twice with the order swapped and keeps only
agreement. Mutation-checked: answer mapping, unparseable-as-guess, the word
boundary and the order check each turn a test red.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Mutation-testing the adjudicator showed dropping the word boundary went unnoticed:
'ANSWER: 12' would be read as 1.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Readings in the requested order; 1/2 mapped to base/alt under both orders;
UNSURE, an unparseable or out-of-range reply is unsure rather than a guess; an
empty reading is described, not blank; a model error is an error, distinguishable
from 'unsure'. The optional order-checked wrapper asks twice with the order swapped
and keeps only agreement.
Red: Request, Verdict, NewLLMAdjudicator and NewOrderChecked are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Disputes carry both readings with their own spelling; differences separated by
one shared word merge into one decision. Compound splits/joins and one or two
filler particles are resolved without a model; 'nej', numbers and content words
never are. Mutation-checked (merge distance, filler limit, 'nej', compound rule).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A dispute carries both readings with their own spelling; differences separated by
a single shared word are one decision, two shared words keep them apart; compound
splits/joins and a short run of filler words are resolved without a model, while
numbers, content words, and 'nej' (an answer) are not.
Red: FindDisputes, AutoResolve and ContextAround are undefined.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Prefix/suffix trimmed first; adjacent deletes and inserts read as one Replace;
beyond 3,000 edits (two texts sharing almost nothing) the middle is one Replace
instead of gigabytes of trace. Mutation-checked: a wrong tie-break, missing
replace coalescing, wrong suffix accounting and a punctuation-bearing key each
turn tests red.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A random-input property test compares the equal runs against a dynamic-programming
LCS: a diff that is valid but not minimal would turn real agreement into disputes
the model is then asked about. A 6,500-word, 380-edit case mirrors a real hour.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Keeps the word as written, keys it without case/punctuation/hyphens/plus, joins a
lone dash or full stop to its neighbour so it is not a word the engines can
disagree about. Red: package reconcile does not exist.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Proposed, decider pending. Two local engines on one WAV; word-level diff with
pianissimo as the base (readability and timing, NOT accuracy -- that was a tie);
trivial disputes resolved without a model; the rest adjudicated by a LOCAL model
(alias must start koala/ or iguana/) whose only freedom is choosing between the
two readings. Reconciliation never fails a job; the page says what was done and
lists the uncertain spots. Off by default. Validation: re-upload the real
recording and keep the feature only if the reconciled reading is right >= 85% of
the time and no worse than the better single engine.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Eleven local models, 120 calls each, scored against the speaker's blind answers
on 30 disagreements, with pass criteria fixed before the run. gemma4-31b (97%)
and gemma4-26b (92%) pass; nothing else clears accuracy and order-consistency.
Records why 31b is slow (MLX gives the same speed; Low Power Mode is the
unproven suspect) and the caveats: 30 items, one recording, thinking models
under-budgeted. Aggregate numbers only; this repo is public.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A near tie on accuracy (blind A/B by the speaker: 17 vs 13 on the 30 spans that
change meaning) and a 6x difference in speed (40 s vs 4m08s for 63.7 min).
Different error profiles: pianissimo better on terms and numbers, kb-whisper on
everyday words. Aggregate numbers only -- no transcript text or domain terms,
this repo is public. Also records the mid-word wrapping bug the comparison found
and its fix.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
whisper.cpp wraps at ~60 chars and can wrap inside a word; the response's text
field joins segments with newlines, so chat saw 'nyt' / 't godkännande'. Text
is now rebuilt from the raw segments: concatenated when any segment carries a
leading space (whisper.cpp), joined by a space when none do (berget's sentence
level), whitespace collapsed; the text field is only a fallback.
Checked against the first real one-hour transcript with pianissimo's vocabulary
as an independent yardstick: 36 spans changed, 31 now consist entirely of words
pianissimo also produced, words unknown to pianissimo fell 224 -> 177 and word
agreement rose 92.4% -> 93.1%. Five rejoined spans still contain an unknown
word (names/loanwords or an unlucky glue) -- a residual risk, small.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Found by comparing the first real one-hour transcript against pianissimo:
whisper.cpp wraps at ~60 chars and can wrap INSIDE a word. The continuation
segment has no leading space, the text field joins segments with newlines, and
a segment can be a lone '.'. Measured 2026-10-01: 7% of segments (96/1332) in a
synthetic hour, 42/1302 lines in the real one. My earlier tests used short
sentences and could not see it. berget's segments are whole sentences with no
leading space, so the two backends need different joins.
Red: Text still joins by newline.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Rejections are synchronous and leave no row, so an operator could not tell 'the
user never retried' from 'the page refused it' (it happened during the first
real-recording trial). The reason is operational; the filename and title are
the user's private content.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Run 2026-10-01 with 82 s of synthetic Swedish muxed with black video (H.264 +
AAC .mp4, built with the real ffmpeg): accepted, had_video=true, transcribed by
iguana/kb-whisper, the page said only its audio was used, and chat answered.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A Teams meeting with the camera off is still a video file. It is accepted, only
its audio is transcribed, and the page says so in queued, transcribing, done and
failed -- not just at the end.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
My assertion expected a lowercase 'only'; the sentence begins 'Only'. The user
facing wording is unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Up and down are one commit: golang-migrate pairs them. RLS is unchanged: the
column is on the already-FORCE'd table, so the exact-set isolation test still
holds.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Video with no audio is rejected for having none. A file with a video track is
accepted and marked had_video; every state of its page (queued, transcribing,
done, failed) says only its audio was used and the video was discarded, and a
plain audio file shows no such notice.
Red: the had_video column does not exist.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
