Vulkan: FA quantized-KV dequant scratch can exceed maxBufferSize at long context #30048

Name and Version

llama-cli.exe --version
version: 0.5.0-dev (build 11372, commit 94b877457)
built with Clang 20.1.8 for Windows x86_64

Built from upstream llama.cpp commit 94b877457(b11352). My local repository has additional personal documentation/benchmark tooling commits only; no llama.cpp or ggml inference source changes were applied to this baseline binary.

Operating systems

Windows

GGML backends

Vulkan

Hardware

AMD Radeon 8060S / Ryzen AI Max+ 395 (Strix Halo)

Models

unsloth/Qwen3.5-2B-GGUF:UD-Q4_K_XL

Problem description & steps to reproduce

With Vulkan Flash Attention and quantized KV cache, the use_dequant_kv fast path can try to allocate a combined F16 K+V scratch buffer larger than the device's maxBufferSize / maxMemoryAllocationSize.
The failure occurs just above 1,048,576 KV entries and the requested allocation size matches the expected combined F16 K+V scratch size exactly.
Related optimization: #25494

llama-cli.exe \
-m Qwen3.5-2B-UD-Q4_K_XL.gguf \
-f prompt.txt \
-c 2097152 \
-ngl 999 \
-b 2048 \
-ub 512 \
-fa 1 \
-ctk q8_0 \
-ctv q8_0 \
--override-kv qwen35.context_length=int:2097152 \
--rope-scaling yarn \
--rope-scale 8 \
--yarn-orig-ctx 262144 \
-n 1

The prompt.txt file contains approximately 1.89M tokens. Any prompt that grows the KV cache beyond 1,048,576 entries should reproduce the issue with this model/configuration.
The failure occurs during prompt evaluation, so generation settings are not relevant.

With -ub 512, prompt processing succeeds through 1,048,576 tokens. The next microbatch grows the KV cache to 1,049,088 entries, at which point Vulkan requests a 2,148,532,224-byte (2049 MiB) buffer and fails.

With -ub1024, there is a different but related long-context behavior around the same 1,048,576-token boundary: the Flash-Attention mask itself reaches approximately 2 GiB and the graph falls back/splits, reducing local prefill throughput to roughly 1.1 t/s instead of failing immediately.
Reducing to -ub512 avoids that mask-size threshold for this prompt, but exposes the combined F16 K+V scratch allocation issue described above.

Root cause analysis

use_dequant_kv checks the F16 K and V sizes individually against maxStorageBufferRange, but later allocates k_f16_sz + v_f16_sz as one Vulkan scratch buffer.
The combined allocation is not checked against maxBufferSize / maxMemoryAllocationSize. On this device both limits are exactly 2 GiB, which explains the 2049 MiB allocation failure above.

Expected behavior

The use_dequant_kv optimization should not request a scratch buffer larger than the device's maxBufferSize or maxMemoryAllocationSize.
If the combined F16 K+V scratch does not fit, the optimization should be disabled and Flash Attention should fall back to the existing quantized-KV path instead of failing the decode.

I tested a minimal guard for the combined scratch size, and the same 2M workload completed successfully.
I can provide the patch if useful.

First Bad Commit

No response

Relevant log output

81.12.844.142 I slot   operator(): id  0 | task 0 | cached n_tokens = 1044480, memory_seq_rm [1044480, end)
81.31.538.696 I slot print_timing: id  0 | task 0 | prompt processing, n_tokens = 1046528, progress = 0.55, t = 4888.29 s / 214.09 tokens per second
81.31.538.711 I slot   operator(): id  0 | task 0 | cached n_tokens = 1046528, memory_seq_rm [1046528, end)
81.50.151.925 I slot print_timing: id  0 | task 0 | prompt processing, n_tokens = 1048576, progress = 0.56, t = 4906.91 s / 213.69 tokens per second
81.50.151.943 I slot   operator(): id  0 | task 0 | cached n_tokens = 1048576, memory_seq_rm [1048576, end)
ggml_vulkan: Device memory allocation of size 2148532224 failed.
ggml_vulkan: Requested buffer size exceeds device buffer size limit: ErrorOutOfDeviceMemory
81.54.761.331 E srv  update_slots: decode() failed: Requested buffer size exceeds device buffer size limit: ErrorOutOfDeviceMemory
81.54.761.340 E srv    send_error: task id = 0, error: decode() failed: Requested buffer size exceeds device buffer size limit: ErrorOutOfDeviceMemory
81.54.761.347 I slot      release: id  0 | task 0 | stop processing: n_tokens = 1050624, truncated = 0

maxStorageBufferRange       = 4294967295
maxMemoryAllocationSize     = 0x80000000
maxBufferSize               = 0x80000000

stderr.log