model stringclasses 9
values | weights stringlengths 18 53 | revision stringlengths 40 40 ⌀ | file stringclasses 9
values | quant stringclasses 10
values | runtime stringclasses 2
values | runtime_version stringclasses 5
values | prompt_tokens int64 512 4.1k | prefill_tps float64 224 11.2k | decode_tps float64 20 202 | peak_memory_gb float64 12.6 234 ⌀ | generated_tokens int64 128 128 | trials int64 3 3 | date timestamp[s]date 2026-10-23 00:00:00 2026-10-30 00:00:00 | chip stringclasses 1
value | gpu_cores int64 80 80 | memory_gb int64 256 256 | macos stringclasses 2
values | size_gb float64 12.1 233 ⌀ | size_x_decode_gbps float64 723 5k ⌀ | size_source stringclasses 1
value | pass_b unknown | pass_a unknown | setting stringclasses 2
values | schema int64 2 2 ⌀ | depth int64 0 4.1k ⌀ | kv_type stringclasses 1
value | prefill_tps_median float64 1.25k 9.07k ⌀ | prefill_tps_std float64 0.39 101 ⌀ | decode_tps_median float64 49 141 ⌀ | decode_tps_std float64 0 9.1 ⌀ | ttft_s float64 0.08 18.3 ⌀ | ttft_method stringclasses 2
values | samples_prefill listlengths 3 3 ⌀ | samples_decode listlengths 3 3 ⌀ | thermal_before stringclasses 1
value | swap_before_mb float64 0 0 ⌀ | peak_rss_gb float64 19 22.9 ⌀ | peak_wired_gb float64 31.6 35.8 ⌀ | wired_before_gb float64 10.4 11.3 ⌀ | wall_s float64 8.1 23.3 ⌀ | thermal_after stringclasses 1
value | swap_after_mb float64 0 0 ⌀ | macos_build stringclasses 2
values | model_id stringclasses 1
value | wired_limit_mb stringclasses 2
values | low_power_mode stringclasses 1
value | bench_commit stringclasses 2
values | decode_depth int64 0 4.1k ⌀ | flash_attn int64 1 1 ⌀ | n_ubatch int64 512 2.05k ⌀ | n_batch int64 2.05k 2.05k ⌀ | decode_tps_depth0 float64 21.8 201 ⌀ | decode_tps_depth0_std float64 0.04 2.6 ⌀ | prefill_at_depth_tps float64 1.1k 4.57k ⌀ | samples_decode_depth0 listlengths 3 3 ⌀ | model_size_gb float64 12.1 233 ⌀ | extra_args listlengths 0 0 ⌀ | peak_memory_method stringclasses 2
values | raw_source listlengths 2 2 ⌀ | measured_with stringclasses 1
value |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Humanizer 12B | jialinyyzz/humanizer | null | null | 8-bit (own conversion of jialinyyzz/humanizer) | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 2,678.669 | 67.407 | 13.363 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.652 | 852.8 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Humanizer 12B | jialinyyzz/humanizer | null | null | 8-bit (own conversion of jialinyyzz/humanizer) | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 3,431.218 | 64.376 | 14.391 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.652 | 814.5 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Humanizer 12B | jialinyyzz/humanizer | null | humanizer-12b-Q8_0.gguf | Q8_0 | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 2,817.342834 | 64.033496 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Humanizer 12B | jialinyyzz/humanizer | null | humanizer-12b-Q8_0.gguf | Q8_0 | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 2,677.075854 | 64.033496 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.8 27B | mlx-community/Qwen3.8-27B-4bit | 10c35caafbb80f7dc6a7a432cdd11af10a6d4818 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 1,499.021 | 54.688 | 16.396 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 16.055 | 878 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.8 27B | mlx-community/Qwen3.8-27B-4bit | 10c35caafbb80f7dc6a7a432cdd11af10a6d4818 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 1,865.992 | 53.565 | 18.264 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 16.055 | 860 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.8 27B | unsloth/Qwen3.8-27B-GGUF | 4ca720788d1e01f1bff70c033e0d0028fd02e502 | Qwen3.8-27B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 1,118.664287 | 48.178545 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.8 27B | unsloth/Qwen3.8-27B-GGUF | 4ca720788d1e01f1bff70c033e0d0028fd02e502 | Qwen3.8-27B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 1,100.705395 | 48.178545 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
gpt-oss 20B | mlx-community/gpt-oss-20b-MXFP4-Q8 | 773a7da77e569019bb0fd17a554b263738d669a3 | null | MXFP4-Q8 | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 7,008.633 | 172.723 | 12.614 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.076 | 2,085.8 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
gpt-oss 20B | mlx-community/gpt-oss-20b-MXFP4-Q8 | 773a7da77e569019bb0fd17a554b263738d669a3 | null | MXFP4-Q8 | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 11,190.208 | 168.503 | 13.021 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.076 | 2,034.8 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
gpt-oss 20B | ggml-org/gpt-oss-20b-GGUF | 15dcfc7e7c608c6161507211ed43c5b590801b4b | gpt-oss-20b-MXFP4.gguf | MXFP4 | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 6,492.885657 | 200.043543 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
gpt-oss 20B | ggml-org/gpt-oss-20b-GGUF | 15dcfc7e7c608c6161507211ed43c5b590801b4b | gpt-oss-20b-MXFP4.gguf | MXFP4 | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 6,174.045041 | 200.043543 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.6 35B-A3B | mlx-community/Qwen3.6-35B-A3B-4bit | 38740b847e4cb78f352aba30aa41c76e08e6eb46 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 5,583.738 | 145.8 | 20.191 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.402 | 2,974.6 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.6 35B-A3B | mlx-community/Qwen3.6-35B-A3B-4bit | 38740b847e4cb78f352aba30aa41c76e08e6eb46 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 9,725.467 | 144.35 | 21.445 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.402 | 2,945 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.6 35B-A3B | unsloth/Qwen3.6-35B-A3B-GGUF | a483e9e6cbd595906af30beda3187c2663a1118c | Qwen3.6-35B-A3B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 4,888.624789 | 134.485957 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Qwen3.6 35B-A3B | unsloth/Qwen3.6-35B-A3B-GGUF | a483e9e6cbd595906af30beda3187c2663a1118c | Qwen3.6-35B-A3B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 4,646.248236 | 134.485957 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Gemma 4 31B | mlx-community/gemma-4-31b-it-4bit | 696d436c404745a59f30e4939a658162b0a9e57f | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 1,224.441 | 43.828 | 18.295 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.412 | 807 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Gemma 4 31B | mlx-community/gemma-4-31b-it-4bit | 696d436c404745a59f30e4939a658162b0a9e57f | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 1,406.015 | 39.273 | 20.936 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.412 | 723.1 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Gemma 4 31B | unsloth/gemma-4-31B-it-GGUF | c1ac76e99d5513b141e8adde7288b85c3f9c32ec | gemma-4-31B-it-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 1,068.719747 | 43.225143 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Gemma 4 31B | unsloth/gemma-4-31B-it-GGUF | c1ac76e99d5513b141e8adde7288b85c3f9c32ec | gemma-4-31B-it-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 1,015.074927 | 43.225143 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Devstral Small 2 24B | mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit | 97228558fcbdd1adefd39d515155db3a894b37f7 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 1,745.659 | 62.913 | 14.727 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 15.103 | 950.2 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Devstral Small 2 24B | mlx-community/Devstral-Small-2-24B-Instruct-2512-4bit | 97228558fcbdd1adefd39d515155db3a894b37f7 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 2,085.667 | 60.125 | 15.907 | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 15.103 | 908.1 | sum of safetensors sizes, measured 2026-10-26 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Devstral Small 2 24B | unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF | 6e458b8add42681bfd023de5eab93637694aaf82 | Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 512 | 1,468.839071 | 59.704368 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Devstral Small 2 24B | unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF | 6e458b8add42681bfd023de5eab93637694aaf82 | Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 620d2225c (b11669) | 4,096 | 1,411.076108 | 59.704368 | null | 128 | 3 | 2026-10-23T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | null | null | null | null | null | llama-bench defaults (-ub 512), decode depth 0 | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null | null |
Flint 40B-A2.5B | RecursiveSI/flint-int4 | 3b93f9053b530488405ae9fca51d4900d6ce21f0 | null | int4 (official) | mlx | mlx-lm 0.32.3.dev12+g4cfc64784, mlx 0.32.4 | 512 | 5,367.82 | 137.91 | 23.286 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 22.569 | 3,112.5 | null | null | null | null | 2 | 512 | f16 | 5,395.47 | 57.36 | 136.31 | 2.87 | 0.1 | estimate p/prefill_tps | [
5406.116,
5301.867,
5395.473
] | [
136.207,
136.313,
141.224
] | no thermal warning recorded | 0 | 22.9 | 33.8 | 11.3 | 8.3 | no thermal warning recorded | 0 | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | null | null | null | null | null | null | null | null | null | null | null | null |
Flint 40B-A2.5B | RecursiveSI/flint-int4 | 3b93f9053b530488405ae9fca51d4900d6ce21f0 | null | int4 (official) | mlx | mlx-lm 0.32.3.dev12+g4cfc64784, mlx 0.32.4 | 4,096 | 9,095.07 | 140.95 | 23.775 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 22.569 | 3,181.1 | null | null | null | null | 2 | 4,096 | f16 | 9,066.25 | 100.68 | 141.47 | 2.6 | 0.45 | estimate p/prefill_tps | [
9011.94,
9066.254,
9207.024
] | [
138.134,
141.468,
143.263
] | no thermal warning recorded | 0 | 22.9 | 35.8 | 10.8 | 8.1 | no thermal warning recorded | 0 | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | null | null | null | null | null | null | null | null | null | null | null | null |
Nanbeige4.3 33B | fcmeyer/Nanbeige4.3-33B-MLX-4bit | e134e560ad92608dc3ce45474b31ec32236b9954 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 512 | 1,248.87 | 51.43 | 19.32 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.631 | 958.2 | null | null | null | null | 2 | 512 | f16 | 1,248.55 | 1.5 | 51.39 | 0.1 | 0.41 | estimate p/prefill_tps | [
1250.507,
1248.55,
1247.564
] | [
51.394,
51.347,
51.535
] | no thermal warning recorded | 0 | 19.2 | 31.6 | 10.4 | 14.4 | no thermal warning recorded | 0 | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | null | null | null | null | null | null | null | null | null | null | null | null |
Nanbeige4.3 33B | fcmeyer/Nanbeige4.3-33B-MLX-4bit | e134e560ad92608dc3ce45474b31ec32236b9954 | null | 4-bit | mlx | mlx-lm 0.32.2, mlx 0.32.4 | 4,096 | 1,501.53 | 49.01 | 20.208 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.631 | 913.1 | null | null | null | null | 2 | 4,096 | f16 | 1,503.75 | 13.23 | 48.98 | 0.15 | 2.73 | estimate p/prefill_tps | [
1503.752,
1513.515,
1487.336
] | [
48.985,
49.171,
48.879
] | no thermal warning recorded | 0 | 19 | 32.9 | 10.5 | 23.3 | no thermal warning recorded | 0 | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | null | null | null | null | null | null | null | null | null | null | null | null |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 4,716.607707 | 118.946402 | null | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,980.2 | null | {
"raw_source": "runs/matched-20261026/flint_llama_decode_depth.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 2.4,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | {
"raw_source": "runs/matched-20261026/flint_llama_prefill_ub2048.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 0,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | superseded by rerun 2026-10-26 15:31 CET (same settings) | 2 | 512 | f16 | null | 5.571537 | null | 0.877303 | 0.11 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
4711.04,
4716.61,
4722.18
] | [
118.15,
118.803,
119.886
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | null | 512 | 1 | 2,048 | 2,048 | 116.823387 | 0.359713 | null | [
116.463,
116.824,
117.183
] | 25 | [] | not sampled (run by run_matched.sh, not bench.py) | [
"runs/matched-20261026/flint_llama_prefill_ub2048.out",
"runs/matched-20261026/flint_llama_decode_depth.out"
] | runs/matched-20261026/run_matched.sh |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 6,705.356551 | 113.258212 | null | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,837.7 | null | {
"raw_source": "runs/matched-20261026/flint_llama_decode_depth.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 2.4,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | {
"raw_source": "runs/matched-20261026/flint_llama_prefill_ub2048.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 0,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | superseded by rerun 2026-10-26 15:31 CET (same settings) | 2 | 4,096 | f16 | null | 20.708197 | null | 4.784268 | 0.61 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
6690.44,
6696.63,
6729
] | [
107.739,
115.819,
116.217
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | null | 4,096 | 1 | 2,048 | 2,048 | 116.823387 | 0.359713 | null | [
116.463,
116.824,
117.183
] | 25 | [] | not sampled (run by run_matched.sh, not bench.py) | [
"runs/matched-20261026/flint_llama_prefill_ub2048.out",
"runs/matched-20261026/flint_llama_decode_depth.out"
] | runs/matched-20261026/run_matched.sh |
Nanbeige4.3 33B | bartowski/Nanbeige_Nanbeige4.3-33B-GGUF | 9beb68421fdf7be095093a6888325fa2c83ddd38 | Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 1,143.477391 | 44.765878 | null | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.789 | 930.6 | null | {
"raw_source": "runs/matched-20261026/nanbeige_llama_decode_depth.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 0.9,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | {
"raw_source": "runs/matched-20261026/nanbeige_llama_prefill_ub2048.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 1.4,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | null | 2 | 512 | f16 | null | 2.394663 | null | 0.312394 | 0.45 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1145.1,
1144.6,
1140.73
] | [
44.4061,
44.9684,
44.9231
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | null | 512 | 1 | 2,048 | 2,048 | 44.814071 | 0.215771 | null | [
45.0241,
44.593,
44.8251
] | 20.8 | [] | not sampled (run by run_matched.sh, not bench.py) | [
"runs/matched-20261026/nanbeige_llama_prefill_ub2048.out",
"runs/matched-20261026/nanbeige_llama_decode_depth.out"
] | runs/matched-20261026/run_matched.sh |
Nanbeige4.3 33B | bartowski/Nanbeige_Nanbeige4.3-33B-GGUF | 9beb68421fdf7be095093a6888325fa2c83ddd38 | Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 1,119.03451 | 42.026394 | null | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.789 | 873.7 | null | {
"raw_source": "runs/matched-20261026/nanbeige_llama_decode_depth.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 0.9,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | {
"raw_source": "runs/matched-20261026/nanbeige_llama_prefill_ub2048.out",
"swap_before_mb": 0,
"swap_after_mb": 0,
"aikido_cpu_pct_before": 1.4,
"thermal_before": "Note: No thermal warning level has been recorded; Note: No performance warning level has been recorded"
} | null | 2 | 4,096 | f16 | null | 4.71661 | null | 0.434926 | 3.66 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1123.92,
1114.51,
1118.67
] | [
41.7734,
42.5286,
41.7772
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | null | 4,096 | 1 | 2,048 | 2,048 | 44.814071 | 0.215771 | null | [
45.0241,
44.593,
44.8251
] | 20.8 | [] | not sampled (run by run_matched.sh, not bench.py) | [
"runs/matched-20261026/nanbeige_llama_prefill_ub2048.out",
"runs/matched-20261026/nanbeige_llama_decode_depth.out"
] | runs/matched-20261026/run_matched.sh |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 4,682.385826 | 118.906304 | 25.4 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,979.2 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.4,
"peak_wired_gb": 36.4,
"wired_before_gb": 23.7,
"wall_s": 6.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 24.5,
"peak_wired_gb": 10.9,
"wired_before_gb": 10.1,
"wall_s": 4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | llama-bench defaults (-ub 512), decode depth 0 | 2 | 0 | f16 | null | 13.91924 | null | 1.129049 | 0.11 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
4689.72,
4691.11,
4666.33
] | [
119.741,
119.356,
117.622
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 0 | 1 | 512 | 2,048 | null | null | 4,572.749952 | null | 25 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 4,452.253004 | 118.906304 | 25.4 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,979.2 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.4,
"peak_wired_gb": 36.4,
"wired_before_gb": 23.7,
"wall_s": 6.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.4,
"peak_wired_gb": 36.7,
"wired_before_gb": 26.2,
"wall_s": 4.8,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | llama-bench defaults (-ub 512), decode depth 0 | 2 | 0 | f16 | null | 19.493189 | null | 1.129049 | 0.92 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
4430.96,
4456.57,
4469.23
] | [
119.741,
119.356,
117.622
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 0 | 1 | 512 | 2,048 | null | null | 4,572.749952 | null | 25 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Nanbeige4.3 33B | bartowski/Nanbeige_Nanbeige4.3-33B-GGUF | 9beb68421fdf7be095093a6888325fa2c83ddd38 | Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 1,132.010886 | 44.6769 | 21.4 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.789 | 928.8 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 21.4,
"peak_wired_gb": 33.9,
"wired_before_gb": 11.9,
"wall_s": 16.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 20.8,
"peak_wired_gb": 32,
"wired_before_gb": 10.2,
"wall_s": 4.5,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | llama-bench defaults (-ub 512), decode depth 0 | 2 | 0 | f16 | null | 11.046821 | null | 0.140707 | 0.45 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1139.97,
1136.67,
1119.4
] | [
44.7586,
44.5144,
44.7577
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 0 | 1 | 512 | 2,048 | null | null | 1,099.076865 | null | 20.8 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Nanbeige4.3 33B | bartowski/Nanbeige_Nanbeige4.3-33B-GGUF | 9beb68421fdf7be095093a6888325fa2c83ddd38 | Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 1,048.67263 | 44.6769 | 21.9 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 20.789 | 928.8 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 21.4,
"peak_wired_gb": 33.9,
"wired_before_gb": 11.9,
"wall_s": 16.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 21.9,
"peak_wired_gb": 33.5,
"wired_before_gb": 17.8,
"wall_s": 16.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | llama-bench defaults (-ub 512), decode depth 0 | 2 | 0 | f16 | null | 1.191478 | null | 0.140707 | 3.91 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1048.24,
1047.76,
1050.02
] | [
44.7586,
44.5144,
44.7577
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 0 | 1 | 512 | 2,048 | null | null | 1,099.076865 | null | 20.8 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Humanizer 12B | jialinyyzz/humanizer | null | humanizer-12b-Q8_0.gguf | Q8_0 | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 2,835.073972 | 63.466577 | 14.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.67 | 804.1 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 14.8,
"peak_wired_gb": 27.3,
"wired_before_gb": 10.3,
"wall_s": 21,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 14,
"peak_wired_gb": 26.9,
"wired_before_gb": 10.9,
"wall_s": 8.1,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 3.42784 | null | 0.295046 | 0.18 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
2839.02,
2832.81,
2833.4
] | [
63.3319,
63.2629,
63.8049
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 63.924016 | 0.132606 | null | null | 12.7 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Humanizer 12B | jialinyyzz/humanizer | null | humanizer-12b-Q8_0.gguf | Q8_0 | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 2,984.093245 | 59.720471 | 14.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.67 | 756.7 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 14.8,
"peak_wired_gb": 27.3,
"wired_before_gb": 10.3,
"wall_s": 21,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 14,
"peak_wired_gb": 26.9,
"wired_before_gb": 10.9,
"wall_s": 8.1,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 16.011389 | null | 1.70799 | 1.37 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
3002.58,
2974.51,
2975.19
] | [
57.8612,
60.0804,
61.2198
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 63.924016 | 0.132606 | null | null | 12.7 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Qwen3.8 27B | unsloth/Qwen3.8-27B-GGUF | 4ca720788d1e01f1bff70c033e0d0028fd02e502 | Qwen3.8-27B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 1,120.494094 | 47.748041 | 17.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 16.464 | 786.1 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 17.8,
"peak_wired_gb": 29.7,
"wired_before_gb": 13.9,
"wall_s": 29.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 16.8,
"peak_wired_gb": 28.3,
"wired_before_gb": 10.1,
"wall_s": 16.7,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 2.061526 | null | 0.116933 | 0.46 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1118.67,
1122.73,
1120.09
] | [
47.7035,
47.6599,
47.8807
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 47.707768 | 0.114953 | null | null | 16.5 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Qwen3.8 27B | unsloth/Qwen3.8-27B-GGUF | 4ca720788d1e01f1bff70c033e0d0028fd02e502 | Qwen3.8-27B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 1,300.051976 | 46.912132 | 17.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 16.464 | 772.4 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 17.8,
"peak_wired_gb": 29.7,
"wired_before_gb": 13.9,
"wall_s": 29.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 16.8,
"peak_wired_gb": 28.3,
"wired_before_gb": 10.1,
"wall_s": 16.7,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 3.016313 | null | 0.390684 | 3.15 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1302.24,
1301.3,
1296.61
] | [
46.4622,
47.1084,
47.1658
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 47.707768 | 0.114953 | null | null | 16.5 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
gpt-oss 20B | ggml-org/gpt-oss-20b-GGUF | 15dcfc7e7c608c6161507211ed43c5b590801b4b | gpt-oss-20b-MXFP4.gguf | MXFP4 | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 6,597.13058 | 201.842685 | 12.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.11 | 2,444.3 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 12.8,
"peak_wired_gb": 25,
"wired_before_gb": 10.2,
"wall_s": 7.1,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 12.5,
"peak_wired_gb": 24.8,
"wired_before_gb": 11.6,
"wall_s": 4.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 26.904383 | null | 3.365005 | 0.08 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
6604.54,
6619.56,
6567.3
] | [
200.036,
205.725,
199.767
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 201.136561 | 1.123219 | null | null | 12.1 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
gpt-oss 20B | ggml-org/gpt-oss-20b-GGUF | 15dcfc7e7c608c6161507211ed43c5b590801b4b | gpt-oss-20b-MXFP4.gguf | MXFP4 | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 7,203.281073 | 193.284282 | 12.8 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 12.11 | 2,340.7 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 12.8,
"peak_wired_gb": 25,
"wired_before_gb": 10.2,
"wall_s": 7.1,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 12.5,
"peak_wired_gb": 24.8,
"wired_before_gb": 11.6,
"wall_s": 4.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 25.332206 | null | 9.096564 | 0.57 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
7231.06,
7181.45,
7197.33
] | [
182.784,
198.764,
198.305
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 201.136561 | 1.123219 | null | null | 12.1 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Qwen3.6 35B-A3B | unsloth/Qwen3.6-35B-A3B-GGUF | a483e9e6cbd595906af30beda3187c2663a1118c | Qwen3.6-35B-A3B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 4,876.749768 | 136.07115 | 22.9 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 22.135 | 3,011.9 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 22.9,
"peak_wired_gb": 35.5,
"wired_before_gb": 10.1,
"wall_s": 10.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 22.4,
"peak_wired_gb": 34.1,
"wired_before_gb": 10.2,
"wall_s": 6.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 12.300266 | null | 2.30153 | 0.1 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
4865.71,
4890.01,
4874.53
] | [
133.43,
137.649,
137.135
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 138.73714 | 2.604697 | null | null | 22.1 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Qwen3.6 35B-A3B | unsloth/Qwen3.6-35B-A3B-GGUF | a483e9e6cbd595906af30beda3187c2663a1118c | Qwen3.6-35B-A3B-UD-Q4_K_M.gguf | UD-Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 6,073.331582 | 131.667489 | 22.9 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 22.135 | 2,914.5 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 22.9,
"peak_wired_gb": 35.5,
"wired_before_gb": 10.1,
"wall_s": 10.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 22.4,
"peak_wired_gb": 34.1,
"wired_before_gb": 10.2,
"wall_s": 6.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 17.901799 | null | 5.315701 | 0.67 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
6093.7,
6060.07,
6066.23
] | [
126.214,
131.954,
136.834
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 138.73714 | 2.604697 | null | null | 22.1 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Gemma 4 31B | unsloth/gemma-4-31B-it-GGUF | c1ac76e99d5513b141e8adde7288b85c3f9c32ec | gemma-4-31B-it-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 1,083.822925 | 43.036496 | 23.4 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.324 | 788.6 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 23.4,
"peak_wired_gb": 37.3,
"wired_before_gb": 10.3,
"wall_s": 32.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 21.4,
"peak_wired_gb": 34.5,
"wired_before_gb": 10.4,
"wall_s": 19.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 1.832493 | null | 0.071317 | 0.47 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1085.79,
1083.52,
1082.16
] | [
43.047,
42.9605,
43.102
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 43.897899 | 0.207981 | null | null | 18.3 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Gemma 4 31B | unsloth/gemma-4-31B-it-GGUF | c1ac76e99d5513b141e8adde7288b85c3f9c32ec | gemma-4-31B-it-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 1,170.997043 | 39.558883 | 23.4 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 18.324 | 724.9 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 23.4,
"peak_wired_gb": 37.3,
"wired_before_gb": 10.3,
"wall_s": 32.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 21.4,
"peak_wired_gb": 34.5,
"wired_before_gb": 10.4,
"wall_s": 19.4,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 1.690732 | null | 0.540604 | 3.5 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1169.63,
1172.89,
1170.48
] | [
38.9586,
39.7108,
40.0073
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 43.897899 | 0.207981 | null | null | 18.3 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Devstral Small 2 24B | unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF | 6e458b8add42681bfd023de5eab93637694aaf82 | Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 1,491.318852 | 59.823371 | 16.2 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 14.334 | 857.5 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 16.2,
"peak_wired_gb": 28.2,
"wired_before_gb": 12.3,
"wall_s": 23.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 15.2,
"peak_wired_gb": 27.3,
"wired_before_gb": 11,
"wall_s": 13.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 4.512609 | null | 0.44255 | 0.34 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1493.12,
1486.18,
1494.65
] | [
60.0891,
60.0685,
59.3125
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 60.227897 | 0.145082 | null | null | 14.3 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Devstral Small 2 24B | unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF | 6e458b8add42681bfd023de5eab93637694aaf82 | Devstral-Small-2-24B-Instruct-2512-Q4_K_M.gguf | Q4_K_M | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 1,613.081221 | 56.82501 | 16.2 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 14.334 | 814.5 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 16.2,
"peak_wired_gb": 28.2,
"wired_before_gb": 12.3,
"wall_s": 23.3,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 15.2,
"peak_wired_gb": 27.3,
"wired_before_gb": 11,
"wall_s": 13.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 3.529279 | null | 0.777309 | 2.54 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
1615.7,
1609.07,
1614.47
] | [
55.9277,
57.2912,
57.2562
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 60.227897 | 0.145082 | null | null | 14.3 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 512 | 4,678.070445 | 117.706009 | 25.9 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,949.1 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.9,
"peak_wired_gb": 37.8,
"wired_before_gb": 11.6,
"wall_s": 11.8,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.3,
"peak_wired_gb": 37.8,
"wired_before_gb": 10.8,
"wall_s": 6.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 512 | f16 | null | 35.854545 | null | 1.056946 | 0.11 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
4636.79,
4701.43,
4695.99
] | [
117.957,
118.615,
116.546
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 512 | 1 | 2,048 | 2,048 | 118.388104 | 1.218374 | null | null | 25 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Flint 40B-A2.5B | unsloth/flint-GGUF | 20468edfc38ab43d6914cc88b08cbecb05d0fc62 | flint-UD-Q4_K_XL.gguf | UD-Q4_K_XL | llama.cpp | llama.cpp 9125a2ebe (b11896) | 4,096 | 6,692.118074 | 111.185899 | 25.9 | 128 | 3 | 2026-10-26T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.0.1 | 25.055 | 2,785.8 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.9,
"peak_wired_gb": 37.8,
"wired_before_gb": 11.6,
"wall_s": 11.8,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 0,
"peak_rss_gb": 25.3,
"peak_wired_gb": 37.8,
"wired_before_gb": 10.8,
"wall_s": 6.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 0
} | null | 2 | 4,096 | f16 | null | 33.122473 | null | 6.14859 | 0.61 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
6693.93,
6724.3,
6658.13
] | [
104.139,
115.46,
113.958
] | null | null | null | null | null | null | null | null | 26A434 | Mac17,15 | 237568 | 0 | 1025283-dirty | 4,096 | 1 | 2,048 | 2,048 | 118.388104 | 1.218374 | null | null | 25 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Mistral-Large-4-1T-A52B | unsloth/Mistral-Large-4-1T-A52B-GGUF | 1eb4373d37e76f6e7b4b8efa20e1b1ca40b3b964 | UD-IQ1_S/Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.gguf | UD-IQ1_S | llama.cpp | llama.cpp a34de8618 (b12015) | 512 | 355.1797 | 21.479393 | 234.1 | 128 | 3 | 2026-10-30T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.2 | 233.01 | 5,004.9 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 679.56,
"peak_rss_gb": 234.1,
"peak_wired_gb": 244.5,
"wired_before_gb": 235.1,
"wall_s": 86.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 631.56,
"raw_output": "runs/2026-10-30/Mistral-Large-4-1T-A52B-UD-IQ1_S-... | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 679.56,
"peak_rss_gb": 233.7,
"peak_wired_gb": 243.7,
"wired_before_gb": 8.5,
"wall_s": 89.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 679.56,
"raw_output": "runs/2026-10-30/Mistral-Large-4-1T-A52B-UD-IQ1_S-00... | null | 2 | 512 | f16 | null | 0.385759 | null | 0.019533 | 1.44 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
354.754,
355.278,
355.507
] | [
21.4939,
21.487,
21.4572
] | null | null | null | null | null | null | null | null | 26B123 | Mac17,15 | 245760 | 0 | e47aef7-dirty | 512 | 1 | 2,048 | 2,048 | 21.774015 | 0.039822 | null | null | 233 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
Mistral-Large-4-1T-A52B | unsloth/Mistral-Large-4-1T-A52B-GGUF | 1eb4373d37e76f6e7b4b8efa20e1b1ca40b3b964 | UD-IQ1_S/Mistral-Large-4-1T-A52B-UD-IQ1_S-00001-of-00006.gguf | UD-IQ1_S | llama.cpp | llama.cpp a34de8618 (b12015) | 4,096 | 223.530838 | 19.986769 | 234.1 | 128 | 3 | 2026-10-30T00:00:00 | Apple M5 Ultra | 80 | 256 | 27.2 | 233.01 | 4,657.1 | null | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 679.56,
"peak_rss_gb": 234.1,
"peak_wired_gb": 244.5,
"wired_before_gb": 235.1,
"wall_s": 86.6,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 631.56,
"raw_output": "runs/2026-10-30/Mistral-Large-4-1T-A52B-UD-IQ1_S-... | {
"thermal_before": "no thermal warning recorded",
"swap_before_mb": 679.56,
"peak_rss_gb": 233.7,
"peak_wired_gb": 243.7,
"wired_before_gb": 8.5,
"wall_s": 89.9,
"thermal_after": "no thermal warning recorded",
"swap_after_mb": 679.56,
"raw_output": "runs/2026-10-30/Mistral-Large-4-1T-A52B-UD-IQ1_S-00... | null | 2 | 4,096 | f16 | null | 1.20285 | null | 0.000682 | 18.32 | estimate n_prompt/prefill_tps (excludes load, tokenisation, sampling) | [
224.917,
222.756,
222.92
] | [
19.987,
19.986,
19.9873
] | null | null | null | null | null | null | null | null | 26B123 | Mac17,15 | 245760 | 0 | e47aef7-dirty | 4,096 | 1 | 2,048 | 2,048 | 21.774015 | 0.039822 | null | null | 233 | [] | sampled RSS of llama-bench, 1 s (mmap pages included) | null | null |
M5 Ultra LLM speed: MLX vs llama.cpp
Measured token speeds for popular open models on a Mac Studio M5 Ultra, running each model in both MLX and llama.cpp. Every speed here was measured (TTFT and GB × decode are derived from them), and every model file is pinned to a commit so you can rerun it.
Hardware
- Apple M5 Ultra, 36-core CPU: 12 super cores and 24 performance cores (
sysctl:hw.perflevel0"Super", 12 logical CPUs;hw.perflevel1"Performance", 24). - 80-core GPU.
- 256 GB unified memory.
- Memory bandwidth 1.2 TB/s: Apple's stated peak from its tech specs, not a figure measured here.
Results
Tokens per second, mean ± standard deviation of 3 trials. Prefill is reading a 512- or 4096-token prompt; decode is generating 128 tokens with that prompt already in the KV cache (the "Decode depth" column gives the depth actually used). Decode 4096 also gives the median of the 3 trials ("med"). GB is the weight size on disk.
| Model | Runtime | Quant | GB | Prefill 512 | Prefill 4096 | Decode 512 | Decode 4096 | Decode depth | GB × decode ¹ |
|---|---|---|---|---|---|---|---|---|---|
| Humanizer 12B | MLX ³ | 8-bit (own conversion of jialinyyzz/humanizer) | 12.7 | 2,679 | 3,431 | 67.4 | 64.4 | 512 / 4096 | 853 GB/s |
| Humanizer 12B | llama.cpp -ub 2048 | Q8_0 | 12.7 | 2,835 ± 3 | 2,984 ± 16 | 63.5 ± 0.3 | 59.7 ± 1.7 (med 60.1) | 512 / 4096 | 804 GB/s |
| Qwen3.8 27B | MLX ³ | 4-bit | 16.1 | 1,499 | 1,866 | 54.7 | 53.6 | 512 / 4096 | 878 GB/s |
| Qwen3.8 27B | llama.cpp -ub 2048 | UD-Q4_K_M | 16.5 | 1,120 ± 2 | 1,300 ± 3 | 47.7 ± 0.1 | 46.9 ± 0.4 (med 47.1) | 512 / 4096 | 786 GB/s |
| gpt-oss 20B | MLX ³ | MXFP4-Q8 | 12.1 | 7,009 | 11,190 | 172.7 | 168.5 | 512 / 4096 | 2,086 GB/s |
| gpt-oss 20B | llama.cpp -ub 2048 | MXFP4 | 12.1 | 6,597 ± 27 | 7,203 ± 25 | 201.8 ± 3.4 | 193.3 ± 9.1 (med 198.3) | 512 / 4096 | 2,444 GB/s |
| Qwen3.6 35B-A3B | MLX ³ | 4-bit | 20.4 | 5,584 | 9,725 | 145.8 | 144.3 | 512 / 4096 | 2,975 GB/s |
| Qwen3.6 35B-A3B | llama.cpp -ub 2048 | UD-Q4_K_M | 22.1 | 4,877 ± 12 | 6,073 ± 18 | 136.1 ± 2.3 | 131.7 ± 5.3 (med 132.0) | 512 / 4096 | 3,012 GB/s |
| Gemma 4 31B | MLX ³ | 4-bit | 18.4 | 1,224 | 1,406 | 43.8 | 39.3 | 512 / 4096 | 807 GB/s |
| Gemma 4 31B | llama.cpp -ub 2048 | Q4_K_M | 18.3 | 1,084 ± 2 | 1,171 ± 2 | 43.0 ± 0.1 | 39.6 ± 0.5 (med 39.7) | 512 / 4096 | 789 GB/s |
| Devstral Small 2 24B | MLX ³ | 4-bit | 15.1 | 1,746 | 2,086 | 62.9 | 60.1 | 512 / 4096 | 950 GB/s |
| Devstral Small 2 24B | llama.cpp -ub 2048 | Q4_K_M | 14.3 | 1,491 ± 5 | 1,613 ± 4 | 59.8 ± 0.4 | 56.8 ± 0.8 (med 57.3) | 512 / 4096 | 858 GB/s |
| Flint 40B-A2.5B | MLX | int4 (official) | 22.6 | 5,368 ± 57 | 9,095 ± 101 | 137.9 ± 2.9 | 140.9 ± 2.6 (med 141.5) | 512 / 4096 | 3,112 GB/s |
| Flint 40B-A2.5B | llama.cpp -ub 2048 | UD-Q4_K_XL | 25.1 | 4,678 ± 36 | 6,692 ± 33 | 117.7 ± 1.1 | 111.2 ± 6.1 (med 114.0) | 512 / 4096 | 2,949 GB/s |
| Nanbeige4.3 33B | MLX | 4-bit | 18.6 | 1,249 ± 2 | 1,502 ± 13 | 51.4 ± 0.1 | 49.0 ± 0.1 (med 49.0) | 512 / 4096 | 958 GB/s |
| Nanbeige4.3 33B | llama.cpp -ub 2048 | Q4_K_M | 20.8 | 1,143 ± 2 | 1,119 ± 5 | 44.8 ± 0.3 | 42.0 ± 0.4 (med 41.8) | 512 / 4096 | 931 GB/s |
| Mistral-Large-4-1T-A52B | llama.cpp -ub 2048 | UD-IQ1_S | 233.0 | 355 ± 0 | 224 ± 1 | 21.5 ± 0.0 | 20.0 ± 0.0 (med 20.0) | 512 / 4096 | 5,005 GB/s |
¹ Weight size on disk × decode tok/s at the 512-token prompt. For a dense model it approximates the weight bandwidth used (M5 Ultra: 1.2 TB/s, Apple's stated peak). For MoE models (Flint 40B-A2.5B, Qwen3.6 35B-A3B, gpt-oss 20B, Mistral-Large-4-1T-A52B) each token reads only the active experts, so it is not a physical bandwidth, only a size-normalised figure.
³ v1 MLX rows (2026-10-23, mlx-lm 0.32.2): mean of 3 trials, no standard deviation or per-trial values stored. Decode is measured right after the prompt (depth = prompt length), like the other MLX rows. GB was measured on 2026-10-26 from the same files.
Other settings, left out of the table (same models, tok/s, mean of 3 trials): llama-bench defaults (-ub 512), decode depth 0: Humanizer 12B p512 prefill 2,817, decode 64.0, Humanizer 12B p4096 prefill 2,677, decode 64.0, Qwen3.8 27B p512 prefill 1,119, decode 48.2, Qwen3.8 27B p4096 prefill 1,101, decode 48.2, gpt-oss 20B p512 prefill 6,493, decode 200.0, gpt-oss 20B p4096 prefill 6,174, decode 200.0, Qwen3.6 35B-A3B p512 prefill 4,889, decode 134.5, Qwen3.6 35B-A3B p4096 prefill 4,646, decode 134.5, Gemma 4 31B p512 prefill 1,069, decode 43.2, Gemma 4 31B p4096 prefill 1,015, decode 43.2, Devstral Small 2 24B p512 prefill 1,469, decode 59.7, Devstral Small 2 24B p4096 prefill 1,411, decode 59.7, Flint 40B-A2.5B p512 prefill 4,682, decode 118.9, Flint 40B-A2.5B p4096 prefill 4,452, decode 118.9, Nanbeige4.3 33B p512 prefill 1,132, decode 44.7, Nanbeige4.3 33B p4096 prefill 1,049, decode 44.7; superseded by rerun 2026-10-26 15:31 CET (same settings): Flint 40B-A2.5B p512 prefill 4,717, decode 118.9, Flint 40B-A2.5B p4096 prefill 6,705, decode 113.3; mlx_lm.benchmark --prefill-step-size 512 (default 2048): Flint 40B-A2.5B p4096 prefill 5,956, decode 139.3, Nanbeige4.3 33B p4096 prefill 1,356, decode 49.1.
MLX generates faster on 6 of 8 models, but llama.cpp wins gpt-oss 20B
Prompt processing: MLX is 1.15–1.6× faster on all 8 models
Adjusted for file size, MLX's decode lead shrinks to 11% or less
M5 Ultra: MLX vs llama.cpp on 8 local models
MLX decodes faster on 6 of 8 models; llama.cpp wins gpt-oss 20B
All llama.cpp rows use the matched settings (2026-10-26). The MLX rows of Flint 40B-A2.5B and Nanbeige4.3 33B are from the same day. The MLX rows of the six other models are v1 runs from 2026-10-23 (mlx-lm 0.32.2, same mlx_lm.benchmark settings, no standard deviation stored). The earlier llama.cpp runs with llama-bench's defaults are kept in the footnote.
Mistral Large 4 ("Le Chonk"), 1T parameters
Measured on 2026-10-30, the day after the weights were released. Mistral Large 4 is a 1-trillion-parameter MoE (128 experts, 4 active + 1 shared, ~52B active parameters per token, MLA attention).
- Quant: unsloth UD-IQ1_S (233 GB, ~1.8 bits per weight on average), revision
1eb4373. It is the only published quant that fits in 256 GB: the smallest MLX build (mlx-community 2-bit) is 338 GB, so there is no MLX row. Expect a quality loss compared with the full model; this section measures speed only. - Runtime: llama.cpp b12015 (
a34de8618); Mistral Large 4 support landed in #31376 (b11999). An earlier run on b11999 gave the same decode speed (21.7 / 20.2 tok/s) and 4% lower prefill at 512 tokens (339 tok/s); its rows are inruns/2026-10-30/b11999-mistral-large-4-rows.jsonl. - Memory: the GPU wired limit was raised to 240 GiB (
iogpu.wired_limit_mb=245760). Wired memory peaked at 245.8 GB; swap stayed at 0.7 GB; no thermal warning.
Its llama-bench row is in the results table above. Speculative decoding with the DSpark draft shipped by unsloth (dspark-Mistral-Large-4-1T-A52B-Q8_0.gguf, 2.2 GB), measured on llama-server with tools/spec_ab.py: decode tok/s from the server's own timings, 512 generated tokens (ignore_eos), 3 trials per line after a warm-up, --spec-type draft-dspark --spec-draft-n-max 3, reasoning effort "low", same server settings (-c 8192 -np 1 -fa on) with and without the draft:
| Prompt | Temperature | Baseline | DSpark (3 drafted tokens) | Speed-up | Draft acceptance |
|---|---|---|---|---|---|
| code | 0 | 21.6 ± 0.0 | 29.4 ± 0.3 | ×1.36 | 58% |
| prose | 0 | 21.6 ± 0.0 | 27.0 ± 0.1 | ×1.25 | 50% |
| json | 0 | 21.6 ± 0.1 | 35.9 ± 0.1 | ×1.66 | 77% |
| code | 1 | 21.6 ± 0.0 | 31.4 ± 2.4 | ×1.45 | 63% |
| prose | 1 | 21.6 ± 0.0 | 26.3 ± 1.2 | ×1.22 | 48% |
| json | 1 | 21.6 ± 0.0 | 34.9 ± 1.8 | ×1.61 | 74% |
The speed-up depends on how predictable the text is: ×1.6 on JSON, ×1.25 on prose. Unsloth's "about 2×" was not reached here. Rows: results_spec.jsonl.
Quants differ (1-bit for the 1T model, 4-bit / 8-bit / MXFP4 for the others), so this shows what runs, not a like-for-like comparison.
Quantizations
Matched as closely as the published files allow. They are not identical formats:
| Model | MLX | GGUF (llama.cpp) |
|---|---|---|
| Humanizer 12B | 8-bit affine, group 64 (own conversion) | Q8_0 |
| Qwen3.8 27B | 4-bit affine, group 64 | UD-Q4_K_M (Unsloth dynamic) |
| gpt-oss 20B | MXFP4-Q8 (MXFP4 experts, 8-bit attention) | MXFP4 |
| Qwen3.6 35B-A3B | 4-bit affine, group 64 (8-bit gates) | UD-Q4_K_M (Unsloth dynamic) |
| Gemma 4 31B | 4-bit affine, group 64 | Q4_K_M |
| Devstral Small 2 24B | 4-bit affine, group 64 | Q4_K_M |
| Flint 40B-A2.5B | int4, group 64 (official) | UD-Q4_K_XL (Unsloth dynamic) |
| Nanbeige4.3 33B | 4-bit affine, group 64 | Q4_K_M |
File sizes differ by model, and decode reads the weights once per token, so a larger file decodes slower. Decode at the 512-token prompt (MLX vs llama.cpp, tok/s), and the gap left after adjusting for size ("GB × decode" column):
- Humanizer 12B (dense): same size; MLX +6% (67.4 vs 63.5).
- Qwen3.8 27B (dense): GGUF 2.5% larger; MLX +15% (54.7 vs 47.7), +12% after size adjustment.
- gpt-oss 20B (MoE): same size; llama.cpp +17% (201.8 vs 172.7).
- Qwen3.6 35B-A3B (MoE): GGUF 8.5% larger; MLX +7% (145.8 vs 136.1), a tie after size adjustment: size explains it.
- Gemma 4 31B (dense): same size; near tie, MLX +2% (43.8 vs 43.0), and a tie at depth 4096 (39.3 vs 39.6).
- Devstral Small 2 24B (dense): GGUF 5% smaller, yet MLX +5% (62.9 vs 59.8), +11% after size adjustment.
- Flint 40B-A2.5B (MoE): GGUF 11% larger; MLX +17% (137.9 vs 117.7), +6% after size adjustment.
- Nanbeige4.3 33B (dense): GGUF 12% larger; MLX +15% (51.4 vs 44.8), +3% after size adjustment: size explains most of it.
For the MoE models the size adjustment is only a normalisation, not a bandwidth: each token reads only the active experts (footnote ¹).
Long context
Same 8 models (tier 0) at contexts from 512 to 128k tokens, with an f16 and an 8-bit KV cache. Tokens per second, mean of 3 trials. The rows, with standard deviations and every trial, are in results_long.jsonl.
Decode (tok/s, 128 tokens generated with the context already in the KV cache):
| Model | Runtime | KV cache | GB | 512 | 4k | 16k | 32k | 64k | 128k |
|---|---|---|---|---|---|---|---|---|---|
| Humanizer 12B | MLX | f16 | 12.7 | 67.9 | 64.3 | 64.4 | 63.1 | 61.7 | 56.8 |
| Humanizer 12B | MLX | 8-bit, g64 | 12.7 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Humanizer 12B | llama.cpp | f16 | 12.7 | 64.4 | 60.6 | 58.6 | 55.5 | 49.6 | 40.8 |
| Humanizer 12B | llama.cpp | q8_0 | 12.7 | 56.8 | 52.0 | 50.9 | 48.7 | 41.7 | 32.4 |
| Qwen3.8 27B | MLX | f16 | 16.1 | 54.8 | 54.0 | 51.1 | 48.6 | 43.5 | 37.0 |
| Qwen3.8 27B | MLX | 8-bit, g64 | 16.1 | 54.5 | 53.2 | 49.0 | 44.9 | 37.8 | 28.4 |
| Qwen3.8 27B | llama.cpp | f16 | 16.5 | 48.6 | 47.9 | 45.4 | 42.4 | 37.1 | 30.2 |
| Qwen3.8 27B | llama.cpp | q8_0 | 16.5 | 47.3 | 45.9 | 43.9 | 40.9 | 36.0 | 29.0 |
| gpt-oss 20B | MLX | f16 | 12.1 | 185.4 | 179.9 | 161.3 | 146.4 | 121.4 | 94.7 |
| gpt-oss 20B | MLX | 8-bit, g64 | 12.1 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| gpt-oss 20B | llama.cpp | f16 | 12.1 | 209.7 | 200.3 | 172.2 | 154.3 | 121.7 | 84.9 |
| gpt-oss 20B | llama.cpp | q8_0 | 12.1 | 192.5 | 179.3 | 169.2 | 150.5 | 123.7 | 89.2 |
| Qwen3.6 35B-A3B | MLX | f16 | 20.4 | 160.1 | 151.6 | 146.4 | 140.9 | 125.0 | 107.4 |
| Qwen3.6 35B-A3B | MLX | 8-bit, g64 | 20.4 | 151.8 | 147.8 | 139.9 | 125.4 | 105.0 | 77.7 |
| Qwen3.6 35B-A3B | llama.cpp | f16 | 22.1 | 143.5 | 141.2 | 133.0 | 120.4 | 105.4 | 84.6 |
| Qwen3.6 35B-A3B | llama.cpp | q8_0 | 22.1 | 135.3 | 134.7 | 125.0 | 115.9 | 99.7 | 79.2 |
| Gemma 4 31B | MLX | f16 | 18.4 | 45.2 | 40.4 | 33.6 | 27.1 | 19.8 | 12.6 |
| Gemma 4 31B | MLX | 8-bit, g64 | 18.4 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Gemma 4 31B | llama.cpp | f16 | 18.3 | 43.8 | 41.1 | 38.0 | 34.6 | 29.1 | 21.9 |
| Gemma 4 31B | llama.cpp | q8_0 | 18.3 | 39.7 | 38.0 | 35.0 | 31.0 | 25.2 | 18.1 |
| Devstral Small 2 24B | MLX | f16 | 15.1 | 64.8 | 61.9 | 53.5 | 47.4 | 37.4 | 26.2 |
| Devstral Small 2 24B | MLX | 8-bit, g64 | 15.1 | 60.9 | 58.8 | 52.6 | 46.8 | 37.4 | 27.0 |
| Devstral Small 2 24B | llama.cpp | f16 | 14.3 | 60.4 | 58.1 | 49.7 | 43.3 | 34.1 | 23.0 |
| Devstral Small 2 24B | llama.cpp | q8_0 | 14.3 | 56.4 | 54.9 | 50.5 | 45.1 | 36.9 | 27.1 |
| Flint 40B-A2.5B | MLX | f16 | 22.6 | 146.4 | 132.4 | 135.0 | 120.6 | 114.3 | 92.0 |
| Flint 40B-A2.5B | MLX | 8-bit, g64 | 22.6 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Flint 40B-A2.5B | llama.cpp | f16 | 25.1 | 120.4 | 118.1 | 106.2 | 100.5 | 87.6 | 67.4 |
| Flint 40B-A2.5B | llama.cpp | q8_0 | 25.1 | 104.4 | 100.7 | 98.8 | 92.7 | 82.6 | 66.2 |
| Nanbeige4.3 33B | MLX | f16 | 18.6 | 52.2 | 49.9 | 43.5 | 37.7 | 29.5 | 21.5 |
| Nanbeige4.3 33B | MLX | 8-bit, g64 | 18.6 | 48.9 | 46.7 | 40.0 | 33.4 | 25.7 | 17.0 |
| Nanbeige4.3 33B | llama.cpp | f16 | 20.8 | 45.6 | 43.1 | 35.8 | 30.4 | 22.9 | 15.1 |
| Nanbeige4.3 33B | llama.cpp | q8_0 | 20.8 | 42.4 | 41.0 | 35.9 | 30.8 | 24.0 | 16.2 |
Prefill (tok/s, reading a prompt of the context length from an empty cache):
| Model | Runtime | KV cache | GB | 512 | 4k | 16k | 32k | 64k | 128k |
|---|---|---|---|---|---|---|---|---|---|
| Humanizer 12B | MLX | f16 | 12.7 | 2,710 | 3,430 | 3,349 | 3,064 | 2,673 | 2,071 |
| Humanizer 12B | MLX | 8-bit, g64 | 12.7 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Humanizer 12B | llama.cpp | f16 | 12.7 | 2,861 | 3,051 | 2,741 | 2,405 | 1,894 | 1,327 |
| Humanizer 12B | llama.cpp | q8_0 | 12.7 | 2,825 | 3,017 | 2,710 | 2,375 | 1,874 | 1,312 |
| Qwen3.8 27B | MLX | f16 | 16.1 | 1,508 | 1,908 | 1,838 | 1,716 | 1,510 | 1,220 |
| Qwen3.8 27B | MLX | 8-bit, g64 | 16.1 | 1,507 | 1,849 | 1,706 | 1,504 | 1,137 | 733 |
| Qwen3.8 27B | llama.cpp | f16 | 16.5 | 1,127 | 1,323 | 1,234 | 1,125 | 931 | 718 |
| Qwen3.8 27B | llama.cpp | q8_0 | 16.5 | 1,126 | 1,319 | 1,225 | 1,115 | 943 | 719 |
| gpt-oss 20B | MLX | f16 | 12.1 | 7,315 | 11,768 | 10,639 | 9,109 | 7,270 | 5,092 |
| gpt-oss 20B | MLX | 8-bit, g64 | 12.1 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| gpt-oss 20B | llama.cpp | f16 | 12.1 | 6,569 | 7,242 | 6,005 | 4,681 | 3,346 | 2,105 |
| gpt-oss 20B | llama.cpp | q8_0 | 12.1 | 6,527 | 7,224 | 5,961 | 4,769 | 3,394 | 2,150 |
| Qwen3.6 35B-A3B | MLX | f16 | 20.4 | 5,742 | 9,732 | 9,077 | 7,893 | 6,096 | 4,137 |
| Qwen3.6 35B-A3B | MLX | 8-bit, g64 | 20.4 | 5,768 | 9,449 | 8,049 | 6,379 | 4,138 | 2,361 |
| Qwen3.6 35B-A3B | llama.cpp | f16 | 22.1 | 4,902 | 6,170 | 5,423 | 4,604 | 3,425 | 2,300 |
| Qwen3.6 35B-A3B | llama.cpp | q8_0 | 22.1 | 4,877 | 6,095 | 5,363 | 4,550 | 3,491 | 2,373 |
| Gemma 4 31B | MLX | f16 | 18.4 | 1,276 | 1,470 | 1,373 | 1,257 | 1,068 | 828 |
| Gemma 4 31B | MLX | 8-bit, g64 | 18.4 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Gemma 4 31B | llama.cpp | f16 | 18.3 | 1,094 | 1,192 | 1,065 | 928 | 729 | 511 |
| Gemma 4 31B | llama.cpp | q8_0 | 18.3 | 1,083 | 1,173 | 1,054 | 928 | 737 | 519 |
| Devstral Small 2 24B | MLX | f16 | 15.1 | 1,804 | 2,188 | 1,985 | 1,816 | 1,503 | 1,109 |
| Devstral Small 2 24B | MLX | 8-bit, g64 | 15.1 | 1,784 | 1,953 | 1,539 | 1,178 | 732 | 413 |
| Devstral Small 2 24B | llama.cpp | f16 | 14.3 | 1,491 | 1,639 | 1,413 | 1,189 | 905 | 615 |
| Devstral Small 2 24B | llama.cpp | q8_0 | 14.3 | 1,488 | 1,636 | 1,416 | 1,194 | 908 | 613 |
| Flint 40B-A2.5B | MLX | f16 | 22.6 | 5,586 | 9,144 | 8,499 | 7,366 | 5,786 | 4,015 |
| Flint 40B-A2.5B | MLX | 8-bit, g64 | 22.6 | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ | not supported ⁴ |
| Flint 40B-A2.5B | llama.cpp | f16 | 25.1 | 4,745 | 6,846 | 5,606 | 4,460 | 3,158 | 1,984 |
| Flint 40B-A2.5B | llama.cpp | q8_0 | 25.1 | 4,680 | 6,714 | 5,492 | 4,413 | 3,132 | 1,984 |
| Nanbeige4.3 33B | MLX | f16 | 18.6 | 1,275 | 1,545 | 1,392 | 1,179 | 916 | 630 |
| Nanbeige4.3 33B | MLX | 8-bit, g64 | 18.6 | 1,262 | 1,336 | 935 | 655 | 381 | 203 |
| Nanbeige4.3 33B | llama.cpp | f16 | 20.8 | 1,157 | 1,155 | 929 | 736 | 519 | 321 |
| Nanbeige4.3 33B | llama.cpp | q8_0 | 20.8 | 1,112 | 1,145 | 923 | 734 | 519 | 325 |
⁴ not supported (mlx-lm 0.32.2: RotatingKVCache quantization not implemented). These models use sliding-window attention, whose rotating KV cache mlx-lm cannot quantize; the run stops with NotImplementedError: RotatingKVCache Quantization NYI. The failed rows stay in results_long.jsonl (error field) and count in no figure.
GB is the weight size on disk. k = 1024 tokens (128k = 131072).
At 128K, MLX decodes faster on 7 of 8 models; Gemma 4 is the exception
Prefill: MLX's lead grows with context, 1.6–2.4× faster at 128K
Long context on M5 Ultra: speed at 4K, 32K and 128K tokens
llama.cpp q8_0 KV cache at 128K: 9–34% less memory, same prefill
Method (long context)
- llama.cpp (llama-bench, same build as the main table): per context N two passes — cold prefill
-p N -n 0, and work at depth-p 2048 -n 128 -d N(decode tok/s with N tokens already in the KV cache, plus a 2048-token prefill on top of the context);-ngl 99 -fa on -b 2048 -ub 2048 -r 3; KV cache f16 and q8_0 (-ctk/-ctv). The prefill table shows the cold prefill; the 2048-token prefill at depth is inresults_long.jsonl(prefill_at_depth_tps). The raw llama-bench output of every pass is inruns/2026-10-26/andruns/2026-10-27/(home directory paths replaced by~). - MLX:
mlx_lm.benchmark -p N -g 128 -n 3(decode follows the N-token prompt = decode at depth N), plustools/mlx_depth.pywith an 8-bit KV cache (--kv-bits 8 --kv-group-size 64). - MLX 8-bit KV rows: measured with
tools/mlx_depth.py(mlx_lm'sstream_generatewithkv_bits=8,kv_group_size=64), not withmlx_lm.benchmark, so MLX f16 vs MLX 8-bit is not a strict like-for-like comparison. mlx-lm 0.32.2 cannot quantize sliding-window (rotating) KV caches, so Humanizer 12B, gpt-oss 20B, Gemma 4 31B and Flint 40B-A2.5B have no MLX 8-bit rows (footnote ⁴). - 60 s idle cooldown between passes; contexts 512, 4096, 16384, 32768, 65536, 131072; 8 models, tier 0.
- Run started 2026-10-26 18:30 CET, finished 2026-10-27 08:57 CET.
- TTFT (
ttft_s) in the llama.cpp rows is an estimate (N / prefill tok/s), not a measurement;ttft_methodsays how each row got it.
Method
- MLX:
mlx_lm.benchmark -p {512,4096} -g 128 -n 3(default--prefill-step-size 2048). Decode runs right after the prompt, so it is measured at depth 512 or 4096. Each row also stores the median, the standard deviation and every trial. - llama.cpp, matched settings:
llama-bench -ngl 99 -fa on -ctk f16 -ctv f16 -b 2048 -ub 2048 -r 3, Metal, in two passes:- prefill:
-p 512,4096 -n 0.-ub 2048matches MLX's prefill step of 2048 tokens. - decode:
-p 0 -n 128 -d 0,512,4096. Each prompt length gets its decode at depth = prompt length, like MLX. Depth 0 is stored asdecode_tps_depth0.
- prefill:
- llama-bench's defaults (
-ub 512, decode at depth 0) against MLX's defaults are in the footnote under the table, along with MLX at--prefill-step-size 512. - Flint and Nanbeige were first run by
runs/matched-20261026/run_matched.sh; its commands, log and raw outputs are published there (run_matched.sh,runs.log, the raw llama-bench / mlx_lm*.outfiles). The six other models and a rerun of Flint were measured bybench.pywith the same llama-bench settings (15:21–15:33 CET); the Flint rerun replaces the first matched Flint rows, which stay in the footnote. That rerun's console log (exact llama-bench commands, cooldowns, mean per point) isruns/matched-20261026/rerun-1521.log. llama-bench's raw JSONL was not saved for it; the per-trial values of its 14 rows are inresults.jsonl(samples_prefill,samples_decode). Home directory paths in the published logs are replaced by~. - Tool versions: llama.cpp 9125a2ebe (b11896) built from source for every llama.cpp row in the table; mlx-lm git 4cfc647 for Flint (support not yet in a release), mlx-lm 0.32.2 for the other MLX rows; mlx 0.32.4.
- A run that failed (
error) or during which the Mac started to swap (warning) stays inresults.jsonl/results_long.jsonlfor the record, but counts in no table figure. Rows with asettingfield were run with non-matched settings; they stay in the file and appear only in the footnote. - Long-context runs (
bench.py --profile long) are inresults_long.jsonland in the Long context section, with their own method.
Sanity check
llama-bench on Llama-2-7B Q4_0 (ggml-org/Llama-2-7B-GGUF, rev 8bf9812c) on this Mac at b11896: pp512 5057.44 ± 10.01, tg128 178.92 ± 0.70 tok/s. The M5 Ultra 80-core entry in llama.cpp discussion #4167 (commit c1d0e7a) gives 4944.66 / 179.10. The machine performs as expected.
Caveats
mlx_lm.benchmark'sprompt_tpsis timed up to the first generated token, so it includes that token's forward pass and sampling. Itsgeneration_tpscounts 128 tokens over the time of 127, about +0.8%.- Both tools use random-token prompts. On MoE models this may route experts differently from real text.
- TTFT (
ttft_s) is estimated as prompt tokens / prefill tok/s unlessttft_methodsays it was measured. - 3 trials per point (llama-bench's default is 5).
- llama-bench decode at depth 4096: the first of the 3 repetitions is the slowest on every model of the 15:21 rerun (e.g. Flint 104.1 vs 115.5 / 114.0; gpt-oss 182.8 vs 198.8 / 198.3), which widens the standard deviation. It is not background load: the machine was otherwise idle, swap 0. The median is shown next to the mean for that column.
- Short runs with a fixed 128-token output. They show raw engine speed, not what a long chat feels like, which also depends on the prompt cache and the app.
Environment
Mac Studio M5 Ultra on AC power. Swap stayed at 0 throughout and macOS recorded no thermal or performance warning. During the first matched pass (Flint and Nanbeige, run_matched.sh) the machine was not idle: other user processes were present at times (browser, a Go build and lint, desktop apps), and the Aikido endpoint network-filter proxy was running at 0 to 2.4% CPU, sampled before each run. The 15:21–15:33 CET rerun (the six other models and Flint) ran with the machine otherwise idle: swap 0, no thermal warning, only desktop apps at ≤63% of one core, sampled every 30 s. Model, runtime versions, macOS version, the bench.py commit and the date are in every row of results.jsonl.
Reproduce
Build llama.cpp from source at b11803 or later (Flint needs it; b11896 was used), rather than brew install llama.cpp.
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout b11896 && cmake -B build && cmake --build build --target llama-bench -j
export LLAMA_BENCH=$PWD/build/bin/llama-bench
brew install uv
uv run bench.py # downloads the pinned files into ~/hf-bench, appends to results.jsonl
uv run bench.py --only gemma
uv run bench.py --profile long --tier 3 --only gpt-oss-120b # long context, appends to results_long.jsonl
Add a model by appending to models.json (MLX repo and GGUF file, each with its commit).
Columns of results.jsonl
model, weights (repo), revision, file, quant, runtime, runtime_version, prompt_tokens, prefill_tps, decode_tps, peak_memory_gb, generated_tokens, trials, date, chip, gpu_cores, memory_gb, macos.
Rows written by bench.py v2 (schema: 2) also have kv_type, n_ubatch (llama.cpp), depth and decode_depth (both = tokens in the KV cache when the measured decode starts), decode_tps_depth0 (llama.cpp), size_gb (weight bytes on disk / 1e9), size_x_decode_gbps (= size_gb × decode_tps), *_median, *_std, samples_*, ttft_s with ttft_method, thermal, swap and wired-memory state (in pass_a / pass_b for llama.cpp), bench_commit, and error or warning when a run went wrong. Rows ingested from raw files have raw_source and measured_with. The v1 MLX rows also have size_gb and size_x_decode_gbps, added later (size_source). A setting field marks a run with non-matched settings. results_custom.jsonl holds runs with other settings (each labelled in setting).
peak_memory_gb means two things. For MLX it is the peak reported by mlx-lm (v1 rows: mean over the trials; v2 rows: max). For llama.cpp (v2 rows only) it is the sampled RSS of llama-bench, mmap'd pages included (peak_memory_method).
Corrections and requests for models are welcome in the Community tab.
- Downloads last month
- 803
