Follow-up to #33 / ADR-033. Evidence: docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md (aggregate only; the recording is private).
Finding. On a real 63.7-minute recording, iguana/pianissimo (KlangAI Swedish Parakeet, MLX) is a near tie on accuracy with iguana/kb-whisper (blind A/B by the speaker on the 30 meaning-changing disagreements: pianissimo 17, kb-whisper 13) and ~6x faster (40 s vs 4m08s). Pianissimo is better on terminology and numbers, kb-whisper on everyday words; their errors differ in kind (kb-whisper garbles, pianissimo substitutes something fluent and wrong). It works through Tapir's existing gateway adapter unchanged (verified end to end).
Options — a decision for you
A. Middle tier: kb-whisper -> pianissimo -> berget. A wedged whisper (infra#273) then no longer sends private audio to the cloud. Needs transcribe.Chain to take a list of backends instead of primary + one fallback. Cheap, and it shrinks the #35 egress risk.
B. Run both on every upload (40 s extra per hour) and give chat both texts or a diff of where they disagree. Costs a second column or a second table row per upload and a UX decision; the best use of the finding that they err in different places.
C. Switch the primary to pianissimo. Not supported by the evidence: 17 vs 13 is not significant, one speaker, one domain.
D. Leave as is.
Open question that gates A: pianissimo is a separate MLX process on the same host. Whether it survives the memory pressure that latches whisper-server (infra#273) was not tested.
Follow-up to #33 / ADR-033. Evidence: `docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md` (aggregate only; the recording is private).
**Finding.** On a real 63.7-minute recording, `iguana/pianissimo` (KlangAI Swedish Parakeet, MLX) is a **near tie on accuracy** with `iguana/kb-whisper` (blind A/B by the speaker on the 30 meaning-changing disagreements: pianissimo 17, kb-whisper 13) and **~6x faster** (40 s vs 4m08s). Pianissimo is better on terminology and numbers, kb-whisper on everyday words; their errors differ in kind (kb-whisper garbles, pianissimo substitutes something fluent and wrong). It works through Tapir's existing gateway adapter unchanged (verified end to end).
**Options — a decision for you**
- **A. Middle tier:** kb-whisper -> pianissimo -> berget. A wedged whisper (infra#273) then no longer sends private audio to the cloud. Needs `transcribe.Chain` to take a list of backends instead of primary + one fallback. Cheap, and it shrinks the #35 egress risk.
- **B. Run both on every upload** (40 s extra per hour) and give chat both texts or a diff of where they disagree. Costs a second column or a second table row per upload and a UX decision; the best use of the finding that they err in different places.
- **C. Switch the primary to pianissimo.** Not supported by the evidence: 17 vs 13 is not significant, one speaker, one domain.
- **D. Leave as is.**
**Open question that gates A:** pianissimo is a separate MLX process on the same host. Whether it survives the memory pressure that latches whisper-server (infra#273) was **not tested**.
Refs: #33, #35, infra#273, ADR-033.
---
_Created via git-mcp on behalf of @mathias_
Built: option B (run both, reconcile locally) — ADR-034, on PR #39
Chosen by the maintainer. Off by default; TAPIR_RECONCILE_BASE_MODEL=iguana/pianissimo + TAPIR_RECONCILE_MODEL=iguana/gemma4-26b turns it on.
What it does: both engines run on the same WAV (the 40 s one first), a word-level diff finds the disagreements, trivial ones (a compound split, a filler) are resolved without a model, and a local model (alias must start koala/ or iguana/, enforced at start-up) picks a reading for the rest. Unsure keeps the base and is flagged. Everything degrades softly: one engine down → the other; the model down → base text unreconciled with the reason; both engines down → the cloud fallback (#35), recorded. The transcript page says what was done and lists the changed/uncertain spots with a minute mark and both readings.
Evidence: docs/research/local-model-adjudicator-sweep-2026-10-01.md (gemma4-31b 97%, gemma4-26b 92% against the speaker's blind answers; nothing else passed). First live smoke on synthetic audio: 3 disputes, 2 right, 1 wrong (a name replaced by a pronoun) — one run of three, not a measurement.
Still open — the test that decides it (ADR-034 Validation): re-upload the real recording with reconciliation on; the speaker judges ~30 changed spots blind; keep the feature only if the reconciled reading is right >= 85% of the time and no worse than the better single engine.
Not done: pianissimo as a middle tier in the cloud-fallback chain (option A) — the ensemble already keeps audio local when either engine works, but a both-down case still goes to berget (#35). Reconcile time is estimated (~10 min per hour on 26b), not measured.
## Built: option B (run both, reconcile locally) — ADR-034, on PR #39
Chosen by the maintainer. **Off by default**; `TAPIR_RECONCILE_BASE_MODEL=iguana/pianissimo` + `TAPIR_RECONCILE_MODEL=iguana/gemma4-26b` turns it on.
What it does: both engines run on the same WAV (the 40 s one first), a word-level diff finds the disagreements, trivial ones (a compound split, a filler) are resolved without a model, and a **local** model (alias must start `koala/` or `iguana/`, enforced at start-up) picks a reading for the rest. Unsure keeps the base and is flagged. Everything degrades softly: one engine down → the other; the model down → base text unreconciled with the reason; both engines down → the cloud fallback (#35), recorded. The transcript page says what was done and lists the changed/uncertain spots with a minute mark and both readings.
Evidence: `docs/research/local-model-adjudicator-sweep-2026-10-01.md` (gemma4-31b 97%, gemma4-26b 92% against the speaker's blind answers; nothing else passed). **First live smoke on synthetic audio: 3 disputes, 2 right, 1 wrong** (a name replaced by a pronoun) — one run of three, not a measurement.
**Still open — the test that decides it** (ADR-034 Validation): re-upload the real recording with reconciliation on; the speaker judges ~30 changed spots blind; keep the feature only if the reconciled reading is right >= 85% of the time and no worse than the better single engine.
Not done: pianissimo as a *middle tier* in the cloud-fallback chain (option A) — the ensemble already keeps audio local when either engine works, but a both-down case still goes to berget (#35). Reconcile time is estimated (~10 min per hour on 26b), not measured.
---
_Created via git-mcp on behalf of @mathias_
Decision (Mathias, 2026-10-02): superseded by ADR-034. The two-engine mode (pianissimo + kb-whisper, reconciled by a local model) is this issue's option B, built. A first check on the real recording: 21 of 30 sampled changes better, 1 worse (docs/research/reconcile-blind-check-2026-10-02.md). Its option A (pianissimo as a middle tier in the cloud-fallback chain) is not built and not needed now: in two-engine mode the cloud fallback only runs when both local engines fail. If single-engine mode with a local middle tier is wanted later, open a new issue with that scope.
**Decision (Mathias, 2026-10-02): superseded by ADR-034.** The two-engine mode (pianissimo + kb-whisper, reconciled by a local model) is this issue's option B, built. A first check on the real recording: 21 of 30 sampled changes better, 1 worse (`docs/research/reconcile-blind-check-2026-10-02.md`). Its option A (pianissimo as a middle tier in the cloud-fallback chain) is not built and not needed now: in two-engine mode the cloud fallback only runs when both local engines fail. If single-engine mode with a local middle tier is wanted later, open a new issue with that scope.
Closing.
---
_Created via git-mcp on behalf of @mathias_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Follow-up to #33 / ADR-033. Evidence:
docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md(aggregate only; the recording is private).Finding. On a real 63.7-minute recording,
iguana/pianissimo(KlangAI Swedish Parakeet, MLX) is a near tie on accuracy withiguana/kb-whisper(blind A/B by the speaker on the 30 meaning-changing disagreements: pianissimo 17, kb-whisper 13) and ~6x faster (40 s vs 4m08s). Pianissimo is better on terminology and numbers, kb-whisper on everyday words; their errors differ in kind (kb-whisper garbles, pianissimo substitutes something fluent and wrong). It works through Tapir's existing gateway adapter unchanged (verified end to end).Options — a decision for you
transcribe.Chainto take a list of backends instead of primary + one fallback. Cheap, and it shrinks the #35 egress risk.Open question that gates A: pianissimo is a separate MLX process on the same host. Whether it survives the memory pressure that latches whisper-server (infra#273) was not tested.
Refs: #33, #35, infra#273, ADR-033.
Created via git-mcp on behalf of @mathias
Built: option B (run both, reconcile locally) — ADR-034, on PR #39
Chosen by the maintainer. Off by default;
TAPIR_RECONCILE_BASE_MODEL=iguana/pianissimo+TAPIR_RECONCILE_MODEL=iguana/gemma4-26bturns it on.What it does: both engines run on the same WAV (the 40 s one first), a word-level diff finds the disagreements, trivial ones (a compound split, a filler) are resolved without a model, and a local model (alias must start
koala/origuana/, enforced at start-up) picks a reading for the rest. Unsure keeps the base and is flagged. Everything degrades softly: one engine down → the other; the model down → base text unreconciled with the reason; both engines down → the cloud fallback (#35), recorded. The transcript page says what was done and lists the changed/uncertain spots with a minute mark and both readings.Evidence:
docs/research/local-model-adjudicator-sweep-2026-10-01.md(gemma4-31b 97%, gemma4-26b 92% against the speaker's blind answers; nothing else passed). First live smoke on synthetic audio: 3 disputes, 2 right, 1 wrong (a name replaced by a pronoun) — one run of three, not a measurement.Still open — the test that decides it (ADR-034 Validation): re-upload the real recording with reconciliation on; the speaker judges ~30 changed spots blind; keep the feature only if the reconciled reading is right >= 85% of the time and no worse than the better single engine.
Not done: pianissimo as a middle tier in the cloud-fallback chain (option A) — the ensemble already keeps audio local when either engine works, but a both-down case still goes to berget (#35). Reconcile time is estimated (~10 min per hour on 26b), not measured.
Created via git-mcp on behalf of @mathias
Decision (Mathias, 2026-10-02): superseded by ADR-034. The two-engine mode (pianissimo + kb-whisper, reconciled by a local model) is this issue's option B, built. A first check on the real recording: 21 of 30 sampled changes better, 1 worse (
docs/research/reconcile-blind-check-2026-10-02.md). Its option A (pianissimo as a middle tier in the cloud-fallback chain) is not built and not needed now: in two-engine mode the cloud fallback only runs when both local engines fail. If single-engine mode with a local middle tier is wanted later, open a new issue with that scope.Closing.
Created via git-mcp on behalf of @mathias