Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7f8e3677b3 | ||
|
|
e3aa661cf2 |
+4
-2
@@ -1554,8 +1554,10 @@ against the speaker's answers: **`gemma4-31b` 97% and `gemma4-26b` 92%** passed
|
|||||||
**Unknowns, stated.** The adjudicator evidence is 30 items from one speaker and one domain, chosen by
|
**Unknowns, stated.** The adjudicator evidence is 30 items from one speaker and one domain, chosen by
|
||||||
Opus as decidable; the other ~350 disagreements (fillers, compounds, small words) have no ground
|
Opus as decidable; the other ~350 disagreements (fillers, compounds, small words) have no ground
|
||||||
truth. Whether a *merged* transcript is better than the better single engine has not been measured.
|
truth. Whether a *merged* transcript is better than the better single engine has not been measured.
|
||||||
Reconcile time per hour is estimated, not measured. `gemma4-31b`'s slowness is unexplained (Low Power
|
Reconcile time per hour is estimated, not measured. `gemma4-31b`'s slowness was Low Power Mode on
|
||||||
Mode on iguana is the unproven suspect).
|
iguana: with it off the 31B decodes at 22.3 tok/s, not 7.9 (2026-10-02, same-machine before/after on
|
||||||
|
a different synthetic prompt; see the research note). The reconciler default `gemma4-26b` was chosen
|
||||||
|
on the slow numbers and is reopened: 31B scored 97% against 92%.
|
||||||
|
|
||||||
**Decision.**
|
**Decision.**
|
||||||
|
|
||||||
|
|||||||
@@ -70,6 +70,16 @@ on GPU), context length (32k vs 4k made no difference), and the **inference engi
|
|||||||
of the same model measured 76 tok/s prefill and 8.6 decode, the same within noise. (Terse
|
of the same model measured 76 tok/s prefill and 8.6 decode, the same within noise. (Terse
|
||||||
output saves only ~3.6 s of ~9.3 s because prefill dominates.) Two unrelated engines at the same
|
output saves only ~3.6 s of ~9.3 s because prefill dominates.) Two unrelated engines at the same
|
||||||
speed points at the machine: ~140 GB/s effective memory bandwidth on a part rated for 800, and
|
speed points at the machine: ~140 GB/s effective memory bandwidth on a part rated for 800, and
|
||||||
**iguana has macOS Low Power Mode on** (`pmset` `lowpowermode 1`). That is the leading suspect
|
**iguana had macOS Low Power Mode on** (`pmset` `lowpowermode 1`). A first MLX benchmark was
|
||||||
and is **not proven**: turning it off needs root and was not done. A first MLX benchmark was
|
|
||||||
wrong because `mlx_lm.server` caches prompts; the figures above are from the corrected run.
|
wrong because `mlx_lm.server` caches prompts; the figures above are from the corrected run.
|
||||||
|
|
||||||
|
**Confirmed 2026-10-02.** The owner turned Low Power Mode off (`lowpowermode 0`, read back over
|
||||||
|
ssh) and `gemma4:31b` was re-measured with nothing else resident: **217 tok/s prefill, 22.3
|
||||||
|
tok/s decode** (4 runs, spread 22.2-22.6), against 103 and 7.9 with it on, so 2.1x and 2.8x. The
|
||||||
|
slowness was the power mode, not the engine or the model. Caveats on the comparison: the new run
|
||||||
|
used a different, synthetic Swedish prompt (~700 words, the original items held real transcript
|
||||||
|
text and were deleted) and the model answered in 26-43 tokens, not the ~200 of the first run, so
|
||||||
|
it is a same-machine before/after, not a controlled A/B. `gemma4:26b` and the MLX figures were
|
||||||
|
not re-measured. The 31B's 97% judge accuracy is therefore usable at roughly a third of the
|
||||||
|
latency the sweep implied; the 26B-vs-31B choice in ADR-034 was made on the slow numbers and is
|
||||||
|
reopened by this.
|
||||||
|
|||||||
Reference in New Issue
Block a user