docs(adr): ADR-034 transcribe with two engines, reconcile with a local model

Proposed, decider pending. Two local engines on one WAV; word-level diff with
pianissimo as the base (readability and timing, NOT accuracy -- that was a tie);
trivial disputes resolved without a model; the rest adjudicated by a LOCAL model
(alias must start koala/ or iguana/) whose only freedom is choosing between the
two readings. Reconciliation never fails a job; the page says what was done and
lists the uncertain spots. Off by default. Validation: re-upload the real
recording and keep the feature only if the reconciled reading is right >= 85% of
the time and no worse than the better single engine.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

DECISIONS.md

@@ -1533,6 +1533,113 @@ and ADR-021 are untouched.
 
 ---
 
+## ADR-034 — Transcribe with two engines and let a local model reconcile their disagreements
+
+**Status:** Proposed (2026-10-01). **Decider: Mathias — pending.** Issue #40 (decision B) and #33.
+Builds on ADR-033 (private upload transcripts), ADR-022 (endpoint chain, a pattern reused), ADR-018
+(single replica), ADR-025 (honest status). Evidence:
+`docs/research/asr-kb-whisper-vs-pianissimo-2026-10-01.md` and
+`docs/research/local-model-adjudicator-sweep-2026-10-01.md`.
+
+**Context.** On the first real one-hour recording `iguana/kb-whisper` and `iguana/pianissimo` were
+a near tie on accuracy (the speaker's blind verdicts on the 30 meaning-changing disagreements:
+pianissimo 17, kb-whisper 13) and they **err in different places**: pianissimo is better on terms and
+numbers, kb-whisper on ordinary words; kb-whisper's errors garble, pianissimo's read as plausible.
+Pianissimo is ~6x faster (40 s vs 4m08s for 63.7 min) and runs through the same adapter. Two engines
+that disagree in ~380 places per hour, with a ~92% word agreement, is information a single-engine
+pipeline throws away. Separately, eleven local models were scored on adjudicating those disagreements
+against the speaker's answers: **`gemma4-31b` 97% and `gemma4-26b` 92%** passed the pre-set bar
+(>= 80% accuracy, >= 85% order-swap consistency, <= 10% unparseable), nothing else did.
+
+**Unknowns, stated.** The adjudicator evidence is 30 items from one speaker and one domain, chosen by
+Opus as decidable; the other ~350 disagreements (fillers, compounds, small words) have no ground
+truth. Whether a *merged* transcript is better than the better single engine has not been measured.
+Reconcile time per hour is estimated, not measured. `gemma4-31b`'s slowness is unexplained (Low Power
+Mode on iguana is the unproven suspect).
+
+**Decision.**
+
+1. **Two engines, one WAV, both local, one after the other.** Each upload is transcribed by
+   `iguana/pianissimo` and `iguana/kb-whisper` from the same converted WAV, sequentially (concurrency
+   stays 1, ADR-033). Each engine fails independently; the berget fallback (ADR-033) applies only when
+   **both** fail. Off by default: with no second engine configured nothing changes.
+
+2. **Align by word, with pianissimo as the base text.** Both texts are tokenised (lower case, letters
+   and digits only) and diffed word by word (Myers); neighbouring differences separated by a single
+   equal word are merged into one **dispute**. The base is pianissimo: sentence-level prose with
+   punctuation and millisecond timestamps, where kb-whisper's output is wrapped fragments. **This
+   choice is about readability and timing, not accuracy — accuracy was a tie — so it is the cheaper
+   mistake, not a measured winner.** A replaced span takes the alternative's own words.
+
+3. **Resolve the trivial ones without a model.** A dispute that is identical once spaces are removed
+   (a compound split or joined) keeps the base; so does a dispute that is only a short filler word
+   on one side. Everything else is adjudicated. (Opus's classification of 380 disputes found ~46
+   compound and ~26 punctuation-only residue, ~65 filler.)
+
+4. **A local model adjudicates the rest, bounded.** Each dispute goes to a local chat model (default
+   `iguana/gemma4-26b`; `gemma4-31b` is configurable and scored higher but is ~5x slower) with ~100
+   words of context either side and the two readings, in the wording that was measured. The reading
+   order alternates by dispute so position bias averages out. The verdict is *base*, *alternative* or
+   *unsure*; **unsure keeps the base and is flagged**. An optional order-check mode asks twice with the
+   order swapped and treats disagreement as unsure; it is **off by default** because it doubles the time
+   and the measured consistency of the passing models is 93%.
+   - **The model must be local.** Its alias must begin `koala/` or `iguana/`; a cloud alias is a
+     start-up configuration error. Private meeting content never goes to a cloud model for this step.
+   - **Bounded impact.** The reconciler can only choose between two readings the engines already
+     produced; it cannot write new text. Speech that says "answer 2" can at worst flip one dispute.
+
+5. **Reconciliation never fails a job.** One engine fails → its text is used, recorded as
+   single-engine. The adjudicator is down (three consecutive errors) → the base text is used
+   unreconciled, recorded as skipped with the reason, and the remaining disputes are not retried one
+   timeout at a time. Both engines fail → ADR-033's fallback path.
+
+6. **What is stored and shown.** `content` is the merged text. `segments` stay the base engine's timed
+   segments, untouched: `content` is authoritative for chat, `segments` only for time. `endpoint`
+   lists every contributor and the reconciler. A new `reconciliation` column (migration 019, same
+   RLS-scoped table) records the status, counts, and the **uncertain and changed spots with a minute
+   mark and both readings**, so the user can check them against the audio. The transcript page says in
+   plain words what was done — "reconciled from two transcripts: N differences, K uncertain" — and
+   lists the uncertain spots. A reconciled transcript is never presented as simply "the" transcript.
+
+7. **Configuration.** `TAPIR_TRANSCRIBE_SECOND_MODEL` (empty = off), `TAPIR_RECONCILE_MODEL`
+   (required when a second engine is set), `TAPIR_RECONCILE_CHECK_ORDER` (off). The default job
+   timeout rises from 30 to 60 minutes: ~4 min of kb-whisper and ~40 s of pianissimo per hour of audio,
+   plus the adjudication.
+
+8. **Not decided here.** A cloud adjudicator (rejected: private content); tuning the prompt; making
+   `gemma4-31b` the default; a "reconciling" progress state (the page says "transcribing" for the whole
+   job); feeding the uncertain spots to chat; whether pianissimo should also be a middle tier in the
+   fallback chain (#40, option A).
+
+**Consequences.**
+- **Positive.** Uses the finding that the engines err differently; stays local end to end; degrades
+  to today's behaviour at every failure; and is honest on the page about what it did.
+- **Negative / watch.**
+  - Job time grows: roughly 300 adjudicated disputes x ~2 s on `gemma4-26b` ~ 10 minutes per hour of
+    audio — an estimate, not a measurement; double with order-check on, ~5x with `gemma4-31b`.
+  - Two gemma-class calls sit on iguana beside whisper. During the sweep a 17-20 GB model was resident
+    beside whisper-server without wedging it, far below the 52 GB that did (infra#273); the hazard is
+    reduced, not removed.
+  - A merged transcript can be wrong where both engines are wrong, and where the adjudicator picks
+    the wrong one (it was right 92% on a hand-picked set, ~97% for 31b).
+  - Reading the page takes more words: the reconciliation summary is new surface to keep honest.
+
+**Validation / resolution triggers.**
+- **The test that decides it:** re-upload the same real recording after merge; the speaker judges, blind,
+  ~30 disputes where the reconciled text differs from pianissimo alone. Keep the feature only if the
+  reconciled reading is right at least 85% of the time **and** no worse than the better single engine.
+  If not, switch it off (config) and keep the single-engine path.
+- Reconcile time above 25 minutes per hour of audio: drop to a smaller adjudicator or skip more
+  disputes automatically.
+- A different recording or speaker in which the engines agree far less (or far more): re-measure before
+  trusting the 92%/97% figures.
+
+**Reversibility.** Off by default and additive: the second engine, the reconciler and the column are
+unused when the settings are empty; the down-migration drops the column; the single-engine path is
+unchanged.
+
+---
+
 ## Rejected alternatives
 
 Approaches considered during the 2026-06-02 planning + grill session and **deliberately not
@@ -1560,6 +1667,8 @@ maps to the ADR that settles it.
 | Uploaded transcripts as a `user_id` column + partial RLS policy on `transcripts` | Turns ADR-021's single exception into an exception with a private lane inside it; the isolation test asserts `transcripts` has no `user_id`. A separate RLS table keeps both boundaries independently assertable | ADR-033 |
 | whisper-server `--convert` for server-side transcode | Its `--tmp-dir` is created at process start and rots under macOS periodic cleanup | ADR-033 |
 | Calling the public `llm-api.d-ma.be` from in-cluster Tapir | Its ingress returns 413 above ~8–10 MB (~5 min of WAV); the NodePort has no such cap | ADR-033 |
+| A cloud model (e.g. Opus) as the adjudicator | It scored 30/30 but would send private meeting content to a cloud API; local gemma4-26b/31b passed the same bar | ADR-034 |
+| Pick one engine globally (pianissimo only, or kb-whisper only) | A near tie on accuracy with different error profiles; choosing one discards the information in their disagreement | ADR-034 |
 
 If a future case genuinely reopens one of these, that's a new ADR superseding the relevant one —
 not a silent reversal.

docs/research/local-model-adjudicator-sweep-2026-10-01.md

@@ -0,0 +1,75 @@
+# Can a local model adjudicate two ASR transcripts? (tapir#33, 2026-10-01)
+
+Question: when `iguana/kb-whisper` and `iguana/pianissimo` disagree about a word or phrase,
+can a **local** model say which reading is right, well enough to merge the two transcripts?
+Ground truth: the speaker's own blind answers on 30 disagreements of one real recording
+(see `asr-kb-whisper-vs-pianissimo-2026-10-01.md`; no transcript text is recorded here,
+this repo is public). Reference: Claude Opus 5.5 agreed with the speaker on all 30.
+
+## Method
+
+- **Items:** 30 disagreement spans that Opus had picked as changing meaning, the speaker's
+  verdict as truth (pianissimo right 17, kb-whisper right 13).
+- **Prompt:** one fixed English instruction (two transcriptions of Swedish speech differ at
+  ONE marked spot; say which reading the speaker most likely said; reply `ANSWER: 1|2|UNSURE`
+  plus one sentence). Each item was asked in **both reading orders** and in **two contexts**:
+  narrow (+-8 words) and wide (+-100 words). 120 calls per model, temperature 0.
+- **Models:** 11, all local (`koala/*`, `iguana/*`): nothing left the homelab. Run one iguana
+  model at a time, each unloaded afterwards, to avoid the memory pressure behind infra#273;
+  whisper was confirmed healthy afterwards.
+- **Pass criteria, fixed before the run:** wide context, strict accuracy >= 80% (an
+  unparseable or "unsure" answer counts as wrong), order-swap consistency >= 85%,
+  unparseable <= 10%.
+
+## Results (wide context, 60 calls per model)
+
+| Model | Correct | 95% CI | Same pick when order swapped | Unparseable | s/call | Pass |
+|---|---|---|---|---|---|---|
+| `iguana/gemma4-31b` | 58 = 97% | 89-99% | 93% | 0 | 10.4 | **yes** |
+| `iguana/gemma4-26b` | 55 = 92% | 82-96% | 93% | 0 | 2.0 | **yes** |
+| `iguana/ornith-35b` | 51 = 85% | 74-92% | 77% | 0 | 2.5 | no (consistency) |
+| `koala/phi4-14b` | 50 = 83% | 72-91% | 80% | 0 | 1.0 | no (consistency) |
+| `koala/qwen36-35b-a3b` | 49 = 82% | 70-89% | 83% | 0 | 3.3 | no (consistency) |
+| `iguana/granite4` | 47 = 78% | 66-87% | 83% | 0 | 4.6 | no |
+| `iguana/qwen3-14b` | 44 = 73% | 61-83% | 87% | 0 | 4.4 | no (accuracy) |
+| `iguana/gpt-oss-20b` | 40 = 67% | 54-77% | 100% | 17 | 11.2 | no (unparseable) |
+| `koala/phi4-mini` | 37 = 62% | 49-73% | 24% | 1 | 0.5 | no |
+| `iguana/gemma4-26b-think` | 23 = 38% | 27-51% | 100% | 37 | 15.3 | no (unparseable) |
+| `koala/qwen35-9b-fast` | 0 | n/a | n/a | 60 | 7.2 | no (no output) |
+
+Narrow context only (+-8 words) costs 6-12 points for the models that pass: gemma4-31b 92%,
+gemma4-26b 83%, ornith-35b 80%. Context is not decoration: Opus also used evidence from
+elsewhere in the recording (an amount stated earlier, a term that recurs).
+
+Position bias: the passing models pick "Reading 1" 47-51% of the time. `phi4-mini` picks it 15%
+and flips its answer when the order is swapped 76% of the time: it carries almost no signal.
+
+## Reading it
+
+- **Two models pass:** the gemma4 pair. Both clear the bar and neither shows position bias.
+  26b is ~5x faster; their intervals overlap, so 30 items cannot separate 92% from 97%.
+- **The thinking models were not tested fairly.** `gpt-oss-20b` and `gemma4-26b-think` spend
+  the 600-token budget reasoning and often never reach the answer line. When gemma4-26b-think
+  did answer, it was right 23 of 23; gpt-oss-20b 93% of the answered. A larger budget would
+  make them eligible; at 15 s a call it would not make them practical.
+
+## What this does not show
+
+- **Thirty items, one recording, one speaker, one domain.** The 30 were chosen by Opus as
+  decidable, so they overstate how easy the typical disagreement is; ~350 others are mostly
+  spelling, compounds and fillers, which no ground truth covers.
+- Opus agreeing 30/30 is not a measure of Opus over all 380 disagreements.
+- Accuracy was measured at temperature 0 on one prompt; no prompt tuning was tried.
+- The speaker's answers are the only truth; they are one person's recollection of what they said.
+
+## Why gemma4-31b is slow, and what was ruled out
+
+On iguana (M2 Ultra, 64 GB) the dense 31B runs at **103 tok/s prefill, 7.9 tok/s decode**; the
+26B MoE at 641 and 39.6. Ruled out as the cause: memory pressure (no swap, 45% free, model 100%
+on GPU), context length (32k vs 4k made no difference), and the **inference engine** — MLX 4-bit
+of the same model measured 76 tok/s prefill and 8.6 decode, the same within noise. (Terse
+output saves only ~3.6 s of ~9.3 s because prefill dominates.) Two unrelated engines at the same
+speed points at the machine: ~140 GB/s effective memory bandwidth on a part rated for 800, and
+**iguana has macOS Low Power Mode on** (`pmset` `lowpowermode 1`). That is the leading suspect
+and is **not proven**: turning it off needs root and was not done. A first MLX benchmark was
+wrong because `mlx_lm.server` caches prompts; the figures above are from the corrected run.