Skip to content

webgpu : add MEAN support - #31591

Merged
ggerganov merged 1 commit into
ggml-org:masterfrom
LeON-Nie-code:preview/webgpu-mean
Nov 5, 2026
Merged

ggerganov merged 1 commit into
ggml-org:masterfrom
LeON-Nie-code:preview/webgpu-mean

Conversation

@LeON-Nie-code

Copy link
Copy Markdown
Contributor

Overview

Add WebGPU support for F32 MEAN with contiguous rows, including offset views and strided higher dimensions. Reuse the row-sum shader and correct its output indexing for 2D dispatch.

Includes correctness and performance tests, plus updated operation support documentation. Related to #14909.

Additional information

Tested on NVIDIA L4 with native WebGPU.

test-backend-ops test -b WebGPU -o MEAN,SUM_ROWS,SUM
test-backend-ops perf -b WebGPU -o MEAN
  • All 17 MEAN cases passed against the CPU reference; all 38 supported reduction cases passed together.
  • Covers sliced/permuted inputs, reduction boundaries and 65,537-row dispatch. The large-row SUM_ROWS regression fails on upstream and passes with this change.
  • Release build, local regression tests, clang-format 22 and EditorConfig checks passed.
  • WikiText perplexity, 32 chunks at context 512: E2B 29.6885 -> 29.6811; E4B 27.7562 -> 27.7657.

Full-model llama-bench results with Gemma3n E2B/E4B Q4_K_M. Five runs per version, alternating execution order, with in-process warmup. Both versions used the same weights, two CPU threads, batch/microbatch 512 and flash attention off. Model loading and warmup are excluded. Times are mean +/- SD for separate prompt-processing and decoding workloads.

Model Test Upstream (s) PR (s) Time reduction
Gemma3n E2B pp512 0.744 +/- 0.004 0.674 +/- 0.003 9.4%
Gemma3n E2B tg128 4.201 +/- 0.112 3.669 +/- 0.248 12.7%
Gemma3n E4B pp512 1.137 +/- 0.005 0.975 +/- 0.005 14.3%
Gemma3n E4B tg128 4.769 +/- 0.168 3.992 +/- 0.250 16.3%

Both models execute MEAN in the first 10 FFN layers. Traces confirm all 1,280 calls in 128 decode steps move from CPU to WebGPU.

Requirements

Assisted-by: Codex
Copilot AI balanced review requested due to automatic review settings November 3, 2026 10:18
@LeON-Nie-code
LeON-Nie-code requested review from a team and ggerganov as code owners November 3, 2026 10:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions github-actions Bot added documentation Improvements or additions to documentation testing Everything test related ggml changes relating to the ggml tensor library for machine learning WebGPU labels Nov 3, 2026

@yomaytk yomaytk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebGPU changes look fine, thanks! Also, fixing the existing bug in sum_rows shader is nice.

@yomaytk yomaytk added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Nov 5, 2026
@ggerganov
ggerganov merged commit 553d2c2 into ggml-org:master Nov 5, 2026
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. testing Everything test related WebGPU

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants