Repository navigation
Misc. bug: When the “--fit on” parameter is used, one layer is not offloaded to GPU (it remains on the CPU), which causes performance to drop by half. #31531
Description
Name and Version
version: 0.8.0-dev (build 0, commit unknown)
built with GNU 16.2.1 for Linux x86_64
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
llama-server
Command line
stdbuf -oL ~/src/llama.cpp/llama.cpp-b11960/build/bin/llama-server --models-preset models.ini --host 0.0.0.0 --port 1235 --models-max 1 --api-key "***" --temp 0 --mirostat 0 --device ROCm0,ROCm1 -lv 4 --no-log-timestamps 2>&1 | awk '{print "[" strftime("%Y-%m-%d %H:%M:%S") "]", $0; fflush()}' >> llama-server-lv4.logProblem description & steps to reproduce
This occurs across various versions of llama.cpp. Each time I installed a new version, I had to fix the bug manually. I’ve finally decided to submit a bug report :) I’m loading the Qwen3.8-27B in Q6_K quantization; I’m not sure if this bug also occurs with other models.
[2026-11-02 12:55:39] [55649] I load_tensors: offloading output layer to GPU [2026-11-02 12:55:39] [55649] I load_tensors: offloading 64 repeating layers to GPU [2026-11-02 12:55:39] [55649] I load_tensors: offloaded 65/66 layers to GPU
65/66
But if you fix this, the operating speed doubles!
First Bad Commit
It happens in every version.
Relevant log output
Logs
...
[2026-11-02 12:53:11] I srv server_on_li: listening on http://0.0.0.0:1235
[2026-11-02 12:55:34] I srv load: spawning server instance with name=Qwen3.8-27B--UD-Q6_K--q8_0--Text on port 55649
[2026-11-02 12:55:34] I srv load: spawning server instance with args:
[2026-11-02 12:55:34] I srv load: /root/src/llama.cpp/llama.cpp-b11960/build/bin/llama-server
[2026-11-02 12:55:34] I srv load: --host
[2026-11-02 12:55:34] I srv load: 127.0.0.1
[2026-11-02 12:55:34] I srv load: --log-colors
[2026-11-02 12:55:34] I srv load: off
[2026-11-02 12:55:34] I srv load: --no-log-timestamps
[2026-11-02 12:55:34] I srv load: --mirostat
[2026-11-02 12:55:34] I srv load: 0
[2026-11-02 12:55:34] I srv load: --port
[2026-11-02 12:55:34] I srv load: 55649
[2026-11-02 12:55:34] I srv load: --temperature
[2026-11-02 12:55:34] I srv load: 0
[2026-11-02 12:55:34] I srv load: --alias
[2026-11-02 12:55:34] I srv load: Qwen3.8-27B--UD-Q6_K--q8_0--Text
[2026-11-02 12:55:34] I srv load: --cache-type-k
[2026-11-02 12:55:34] I srv load: q8_0
[2026-11-02 12:55:34] I srv load: --cache-type-v
[2026-11-02 12:55:34] I srv load: q8_0
[2026-11-02 12:55:34] I srv load: --device
[2026-11-02 12:55:34] I srv load: ROCm0,ROCm1
[2026-11-02 12:55:34] I srv load: --flash-attn
[2026-11-02 12:55:34] I srv load: on
[2026-11-02 12:55:34] I srv load: --fit
[2026-11-02 12:55:34] I srv load: on
[2026-11-02 12:55:34] I srv load: --fit-target
[2026-11-02 12:55:34] I srv load: 512
[2026-11-02 12:55:34] I srv load: --load-mode
[2026-11-02 12:55:34] I srv load: mmap
[2026-11-02 12:55:34] I srv load: --log-verbosity
[2026-11-02 12:55:34] I srv load: 4
[2026-11-02 12:55:34] I srv load: --model
[2026-11-02 12:55:34] I srv load: Qwen3.8-27B--UD-Q6_K.gguf
[2026-11-02 12:55:34] I srv load: --n-predict
[2026-11-02 12:55:34] I srv load: -1
[2026-11-02 12:55:34] I srv load: --parallel
[2026-11-02 12:55:34] I srv load: 1
[2026-11-02 12:55:34] I srv load: --split-mode
[2026-11-02 12:55:34] I srv load: layer
[2026-11-02 12:55:34] I srv load: --threads
[2026-11-02 12:55:34] I srv load: 10
[2026-11-02 12:55:34] [55649] I srv llama_server: initializing ...
[2026-11-02 12:55:34] [55649] I cmn common_param: common_params_print_info: build 0 (unknown) with GNU 16.2.1 for Linux x86_64
[2026-11-02 12:55:34] [55649] I cmn common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
[2026-11-02 12:55:34] [55649] I cmn common_param: device_info:
[2026-11-02 12:55:34] [55649] I cmn common_param: - ROCm0 : AMD Radeon VII (16368 MiB, 16348 MiB free)
[2026-11-02 12:55:34] [55649] I cmn common_param: - ROCm1 : AMD Radeon VII (16368 MiB, 16348 MiB free)
...
[2026-11-02 12:55:38] [55649] I print_info: arch = qwen35
[2026-11-02 12:55:38] [55649] I print_info: vocab_only = 0
[2026-11-02 12:55:38] [55649] I print_info: no_alloc = 0
[2026-11-02 12:55:38] [55649] I print_info: n_ctx_train = 262144
[2026-11-02 12:55:38] [55649] I print_info: n_embd_inp = 5120
[2026-11-02 12:55:38] [55649] I print_info: n_embd = 5120
[2026-11-02 12:55:38] [55649] I print_info: n_embd_out = 5120
[2026-11-02 12:55:38] [55649] I print_info: n_layer = 64
[2026-11-02 12:55:38] [55649] I print_info: n_layer_all = 65
[2026-11-02 12:55:38] [55649] I print_info: n_head = 24
[2026-11-02 12:55:38] [55649] I print_info: n_head_kv = 4
[2026-11-02 12:55:38] [55649] I print_info: n_rot = 64
[2026-11-02 12:55:38] [55649] I print_info: n_swa = 0
[2026-11-02 12:55:38] [55649] I print_info: is_swa_any = 0
[2026-11-02 12:55:38] [55649] I print_info: non_causal_type = 0
[2026-11-02 12:55:38] [55649] I print_info: n_embd_head_k = 256
[2026-11-02 12:55:38] [55649] I print_info: n_embd_head_v = 256
[2026-11-02 12:55:38] [55649] I print_info: n_gqa = 6
[2026-11-02 12:55:38] [55649] I print_info: n_embd_k_gqa = 1024
[2026-11-02 12:55:38] [55649] I print_info: n_embd_v_gqa = 1024
[2026-11-02 12:55:38] [55649] I print_info: f_norm_eps = 0.0e+00
[2026-11-02 12:55:38] [55649] I print_info: f_norm_rms_eps = 1.0e-06
[2026-11-02 12:55:38] [55649] I print_info: f_clamp_kqv = 0.0e+00
[2026-11-02 12:55:38] [55649] I print_info: f_max_alibi_bias = 0.0e+00
[2026-11-02 12:55:38] [55649] I print_info: f_logit_scale = 0.0e+00
[2026-11-02 12:55:38] [55649] I print_info: f_attn_scale = 0.0e+00
[2026-11-02 12:55:38] [55649] I print_info: f_attn_value_scale = 0.0000
[2026-11-02 12:55:38] [55649] I print_info: n_ff = 17408
[2026-11-02 12:55:38] [55649] I print_info: n_expert = 0
[2026-11-02 12:55:38] [55649] I print_info: n_expert_used = 0
[2026-11-02 12:55:38] [55649] I print_info: n_expert_groups = 0
[2026-11-02 12:55:38] [55649] I print_info: n_group_used = 0
[2026-11-02 12:55:38] [55649] I print_info: causal attn = 1
[2026-11-02 12:55:38] [55649] I print_info: pooling type = -1
[2026-11-02 12:55:38] [55649] I print_info: rope type = 40
[2026-11-02 12:55:38] [55649] I print_info: rope scaling = linear
[2026-11-02 12:55:38] [55649] I print_info: freq_base_train = 10000000.0
[2026-11-02 12:55:38] [55649] I print_info: freq_scale_train = 1
[2026-11-02 12:55:38] [55649] I print_info: n_ctx_orig_yarn = 262144
[2026-11-02 12:55:38] [55649] I print_info: rope_yarn_log_mul = 0.0000
[2026-11-02 12:55:38] [55649] I print_info: rope_finetuned = unknown
[2026-11-02 12:55:38] [55649] I print_info: mrope sections = [11, 11, 10, 0]
[2026-11-02 12:55:38] [55649] I print_info: ssm_d_conv = 4
[2026-11-02 12:55:38] [55649] I print_info: ssm_d_inner = 6144
[2026-11-02 12:55:38] [55649] I print_info: ssm_d_state = 128
[2026-11-02 12:55:38] [55649] I print_info: ssm_dt_rank = 48
[2026-11-02 12:55:38] [55649] I print_info: ssm_n_group = 16
[2026-11-02 12:55:38] [55649] I print_info: ssm_dt_b_c_rms = 0
[2026-11-02 12:55:38] [55649] I print_info: model type = 27B
[2026-11-02 12:55:38] [55649] I print_info: model params = 27.32 B
[2026-11-02 12:55:38] [55649] I print_info: general.name = Qwen3.8-27B
...
[2026-11-02 12:55:39] [55649] I load_tensors: offloading output layer to GPU
[2026-11-02 12:55:39] [55649] I load_tensors: offloading 64 repeating layers to GPU
[2026-11-02 12:55:39] [55649] I load_tensors: offloaded 65/66 layers to GPU
[2026-11-02 12:55:39] [55649] I load_tensors: CPU_Mapped model buffer size = 1285.52 MiB
[2026-11-02 12:55:39] [55649] I load_tensors: ROCm0 model buffer size = 8604.03 MiB
[2026-11-02 12:55:39] [55649] I load_tensors: ROCm1 model buffer size = 10730.49 MiB
...
[2026-11-02 12:55:43] [55649] I llama_context: constructing llama_context
[2026-11-02 12:55:43] [55649] I llama_context: n_seq_max = 1
[2026-11-02 12:55:43] [55649] I llama_context: n_ctx = 220416
[2026-11-02 12:55:43] [55649] I llama_context: n_ctx_seq = 220416
[2026-11-02 12:55:43] [55649] I llama_context: n_batch = 2048
[2026-11-02 12:55:43] [55649] I llama_context: n_ubatch = 512
[2026-11-02 12:55:43] [55649] I llama_context: causal_attn = 1
[2026-11-02 12:55:43] [55649] I llama_context: flash_attn = enabled
[2026-11-02 12:55:43] [55649] I llama_context: kv_unified = false
[2026-11-02 12:55:43] [55649] I llama_context: freq_base = 10000000.0
[2026-11-02 12:55:43] [55649] I llama_context: freq_scale = 1
[2026-11-02 12:55:43] [55649] I llama_context: n_rs_seq = 0
[2026-11-02 12:55:43] [55649] I llama_context: n_outputs_max = 1
[2026-11-02 12:55:43] [55649] I llama_context: n_outputs_max_per_seq = 1
[2026-11-02 12:55:43] [55649] I llama_context: n_ctx_seq (220416) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
[2026-11-02 12:55:43] [55649] I llama_context: ROCm_Host output buffer size = 0.95 MiB
[2026-11-02 12:55:43] [55649] I llama_kv_cache: ROCm0 KV buffer size = 3659.25 MiB
[2026-11-02 12:55:43] [55649] I llama_kv_cache: ROCm1 KV buffer size = 3659.25 MiB
[2026-11-02 12:55:43] [55649] I llama_kv_cache: size = 7318.50 MiB (220416 cells, 16 layers, 1/1 seqs), K (q8_0): 3659.25 MiB, V (q8_0): 3659.25 MiB
[2026-11-02 12:55:43] [55649] I llama_kv_cache: n_rot_k = 256, n_embd_head_k_all = 256
[2026-11-02 12:55:43] [55649] I llama_kv_cache: n_rot_v = 64, n_embd_head_k_all = 256
[2026-11-02 12:55:43] [55649] I llama_memory_recurrent: CPU RS buffer size = 3.12 MiB
[2026-11-02 12:55:43] [55649] I llama_memory_recurrent: ROCm0 RS buffer size = 71.70 MiB
[2026-11-02 12:55:43] [55649] I llama_memory_recurrent: ROCm1 RS buffer size = 74.81 MiB
[2026-11-02 12:55:43] [55649] I llama_memory_recurrent: size = 149.62 MiB ( 1 cells, 64 layers, 1 seqs 0 rs_seq), R (f32): 5.62 MiB, S (f32): 144.00 MiB, P (f32): 0.00 MiB
...
Activity
fit : account for skipped NextN layers when setting n_gpu_layers (ggm…