From e3aa661cf27561609da3c563ac27b5a38c2fee7f Mon Sep 17 00:00:00 2001 From: Mathias Date: Fri, 2 Oct 2026 07:13:10 +0200 Subject: [PATCH] docs(research): Low Power Mode confirmed as the cause of slow gemma4-31b (22.3 vs 7.9 tok/s decode) Co-Authored-By: Claude Sonnet 5.5 --- .../local-model-adjudicator-sweep-2026-10-01.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/docs/research/local-model-adjudicator-sweep-2026-10-01.md b/docs/research/local-model-adjudicator-sweep-2026-10-01.md index d98abd6..d541171 100644 --- a/docs/research/local-model-adjudicator-sweep-2026-10-01.md +++ b/docs/research/local-model-adjudicator-sweep-2026-10-01.md @@ -70,6 +70,16 @@ on GPU), context length (32k vs 4k made no difference), and the **inference engi of the same model measured 76 tok/s prefill and 8.6 decode, the same within noise. (Terse output saves only ~3.6 s of ~9.3 s because prefill dominates.) Two unrelated engines at the same speed points at the machine: ~140 GB/s effective memory bandwidth on a part rated for 800, and -**iguana has macOS Low Power Mode on** (`pmset` `lowpowermode 1`). That is the leading suspect -and is **not proven**: turning it off needs root and was not done. A first MLX benchmark was +**iguana had macOS Low Power Mode on** (`pmset` `lowpowermode 1`). A first MLX benchmark was wrong because `mlx_lm.server` caches prompts; the figures above are from the corrected run. + +**Confirmed 2026-10-02.** The owner turned Low Power Mode off (`lowpowermode 0`, read back over +ssh) and `gemma4:31b` was re-measured with nothing else resident: **217 tok/s prefill, 22.3 +tok/s decode** (4 runs, spread 22.2-22.6), against 103 and 7.9 with it on, so 2.1x and 2.8x. The +slowness was the power mode, not the engine or the model. Caveats on the comparison: the new run +used a different, synthetic Swedish prompt (~700 words, the original items held real transcript +text and were deleted) and the model answered in 26-43 tokens, not the ~200 of the first run, so +it is a same-machine before/after, not a controlled A/B. `gemma4:26b` and the MLX figures were +not re-measured. The 31B's 97% judge accuracy is therefore usable at roughly a third of the +latency the sweep implied; the 26B-vs-31B choice in ADR-034 was made on the slow numbers and is +reopened by this.