This is a patch release on top of v1.30.0, containing targeted model-validation and memory-safety fixes, improved Plugin EP data transfers, and build and packaging fixes.

Highlights

Bug Fixes

• Hardened contrib-op shape inference with additional rank, attribute, initializer, and overflow validation for CausalConvWithState, GatedDeltaNet, GroupQueryAttention, and SparseAttention (#32607).
• Prevented out-of-bounds reads in EmbedLayerNormalization shape inference with scalar beta, ConvTransposeWithDynamicPads, and shape-inference input-data lookup (#32610, #32679, #32681).
• Rejected malformed graphs containing inputs to zero-input operator schemas or subgraphs on non-control-flow nodes (#32633, #32641).
• Prevented an out-of-bounds read during session initialization when a Loop or Scan body input is also an initializer, rejecting the invalid graph instead of crashing (#32609).
• Fixed a heap buffer overflow caused by reusing packed sub-byte tensor storage for a larger full-byte tensor (#32611).
• Preserved the caller's execution stream in Plugin EP single-tensor data transfers, allowing asynchronous copies to receive the supplied stream while retaining synchronous-copy behavior (#32666).

Build and Packaging

• Fixed Windows build-tools setup to use system-trusted certificates and improved proxy handling (#33142).
• Selected Python 3.12 explicitly before package validation in the combined packaging stage (#33561).
• Restricted FP8 XQA test cases to supported GPU compute capabilities (#32559).

Contributors

Thanks to our 8 contributors for this release!

@adrastogi, @bbernhar, @eserscor, @miaobin, @shiyi9801, @tianleiwu, @wangw-1991, @xiaofeihan1

Full Changelog: v1.30.0...v1.30.1

These release notes were drafted with AI assistance.

ONNX Runtime WebGPU Plugin EP 0.5.0 expands attention, mixture-of-experts, and quantized operator support, improves GPU kernel performance, and strengthens runtime integration and correctness.

Highlights

Performance

• Enabled subgroup-matrix kernels on AMD GPUs (#32667).
• Added an FP16 GEMV specialization for single-row decode projections (#33070).
• Improved data movement in Concat, Slice, Transpose, and pooling, and vectorized NHWC depthwise convolution across channels (#32497, #32498).
• Enabled local-window flash attention for GroupQueryAttention (#32462).
• Optimized recurrent and chunked GatedDeltaNet execution (#32645, #33143).
• Optimized sparse attention indexers and SparsePagedAttention softmax, and cached sparse-indexer QSA visibility and scores (#32613, #32656, #33017).

Operator and Model Support

• Added native GatedDeltaNet and VarlenCausalConvWithState kernels, including subsequent state-update-capacity support for GatedDeltaNet (#32510, #32477, #32767).
• Added SparseAttentionIndexer, DynamicSparseAttention, and SparsePagedAttention, plus a packed sparse attention indexer for continuous batching (#32528, #32529, #32533, #32618).
• Extended PagedAttention with full-context and local non-causal attention, and per-head Q/K normalization (#32763, #32792, #32808).
• Added unquantized MoE and expanded QMoE activation, gating, routing, and packed-input support (#32447).
• Added block-scaled FP8 QMoE, INT2 expert weights, and mixed-width integer QMoE execution (#32794, #32878).
• Added Qwen4-Exp Engram support and VarlenNGramHashMapping for packed and variable-length batching (#32285, #32358, #32467).
• Extended GatherBlockQuantized to FP8 and FP4 data. Out-of-range indices now produce zero output slices for both floating-point and existing integer formats (#32480).
• Added GroupNorm, SkipGroupNorm, and 3D ConvTranspose support (#32463, #32688).
• Added weightless hyper-connection primitives: BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix (#32687).
• Added SiLU/Swish/Sigmoid activation modes to GatedRMSNorm and QuickGelu fusion in the im2col-matmul convolution path (#32512, #32560).

Runtime Integration and Configuration

• Added adapterIndex to select a physical Dawn adapter independently of the ORT WebGPU context identifier, deviceId (#32319).
• Added the opt-in enableMatmulFp32Accumulation provider option for MatMulNBits and its fused variants; the default remains false (#29599).
• Added concurrent independent-Session execution on eligible ORT hosts, with Session-owned command recordings and separate buffer-reuse tracking; retained serialized compatibility for older hosts (#32587, #32874, #33082).
• Improved external-device integration with context-aware memory identity, host-owned cache coverage, and stateless shared data transfers (#33110, #33208).

Reliability and Correctness

• Fixed out-of-bounds stores in subgroup-matrix MatMulNBits and added bounds checks for writes based on seqlens_k (#32582, #32623).
• Fixed grouped-convolution underflow and vectorization across group boundaries (#32478, #33129).
• Fixed mixed-type LayerNormalization shaders and retained FP32 arithmetic in LayerNormalization (#32629, #32896).
• Fixed DequantizeLinear scalar/vector shader-cache collisions and 1D block_size handling (#33122, #32734).
• Fixed initializer uploads on GPUs without Resizable BAR (#32816).
• Fixed OrtMemoryInfo ABI mismatches across the plugin boundary and bundled Dawn platform cleanup (#33571, #33526).

Build and Native Hosting

• Added a statically linked WebGPU plugin EP build configuration (#32395).
• Isolated the bundled Dawn implementation and added standalone Dawn API package support for native hosts, with external-Dawn plugin CI coverage (#32964, #33006, #33077).
• Updated Dawn and fixed WebGPU builds with LTO enabled and subgroup-configuration header dependencies (#32686, #32813, #32672).

Runtime Compatibility

The minimum ORT version is 1.24.4. Concurrent independent Sessions are enabled on ORT 1.28.3 and later in the 1.28 series, 1.30.1 and later in the 1.30 series, and 1.31 or later. Other supported versions, including 1.29.x, use serialized compatibility. In serialized mode, callers must serialize all WebGPU operations on the same device, including Session creation/destruction, allocation, I/O binding, transfers, and Run; the plugin does not enforce this serialization. Concurrent mode does not permit overlapping I/O binding and Run on the same Session (#33082).

Known compatibility caveat: ORT 1.26.0 was reported to reject the plugin during kernel creation because of its host-side plugin-version check. Forcing serialized execution does not resolve that restriction (#33082).

Contributors

Thanks to everyone who contributed to this release:

@aciddelgado, @adrastogi, @AkZGhost, @apsonawane, @edgchen1, @feich-ms, @gramalingam, @gyagp, @jchen10, @jiafatom, @justinchuby, @kunal-vaishnavi, @kylo5aby, @linzuojian, @mei1127, @mustjab, @prathikr, @qjia7, @sayanshaw24, @SharmaRithik, @skottmckay, @tianleiwu, @xadupre, @xhcao, @xiaofeihan1.

This release covers changes affecting ONNX Runtime WebGPU Plugin EP code and packaging since 0.4.0.

These release notes were prepared with AI assistance from commit history and pull request metadata.

ONNX Runtime 1.31.1 is a patch release that fixes an OpenVINO-enabled Python build regression and corrects WASM Debug external-data tests. These notes cover changes since ONNX Runtime 1.31.0.

Highlights

• Fixed a provider-info namespace mismatch that caused compilation failures when building with both OpenVINO and Python enabled (#33327).
• Fixed WASM Debug test failures by skipping file-backed external-data cases unsupported by the WASM test module, while retaining compile-output checks, buffer-destination coverage, and native test coverage. This is a test-only change and does not change runtime behavior (#33226).

Contributors

Thanks to our 3 human contributors for this release!

@eserscor, @mustjab, @tianleiwu

Full Changelog: rel-1.31.0...rel-1.31.1

Release highlights were prepared with AI assistance.

ONNX Runtime 1.31.0 stabilizes model-package and EPContext data APIs, improves CPU model loading and quantized MoE inference, expands device-based execution-provider selection, and strengthens model-loading and runtime reliability. These notes cover changes since ONNX Runtime 1.30.0. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP releases.

Highlights

• Promoted the model-package API and EPContext data callbacks to stable C and C++ APIs, with EPContext callback support added across language bindings (#33166, #32265).
• Reduced CPU session-initialization overhead by parallelizing eligible weight prepacking, and accelerated block-wise INT4/INT8 QMoE experts with MLAS QNBit kernels and grouped expert dispatch (#31691, #32644, #32668).
• Added x86 FP16 LayerNorm/RMSNorm acceleration, Arm KleidiAI SVE2.1 FP16 GEMM support, and additional RISC-V vector kernels (#32715, #32670, #32710).
• Enabled CoreML participation in device-based EP selection, including Apple Neural Engine selection through PREFER_NPU (#31975).
• Added workspace-memory accounting and verification for constrained-memory graph partitioning, alongside broader model, graph, and tensor validation (#31962, #32189).

Announcements & Compatibility

• CUDA Plugin EP v0.3.0 and WebGPU Plugin EP v0.5.0 are matched with ONNX Runtime 1.31.0. For contrib-op compatibility, core and the plugin must use the same contributed-operator schemas; building both from the same ONNX Runtime revision is recommended. Schema mismatches can cause incorrect execution or crashes and may not be detected during plugin registration.
• GatherBlockQuantized behavior change: out-of-range indices now produce zero output slices across CPU, CUDA, and WebGPU for both integer and floating-point quantized formats. This deliberately differs from ONNX Gather error semantics (#32480).
• Experimental API migration: model-package consumers should use OrtApi::GetModelPackageApi and the stable OrtModelPackageApi table. EPContext callback registration now uses stable APIs; the corresponding experimental lookup entries were removed. The stable native callback contract carries callback/state pointers, with payload limits enforced by application callbacks, bindings, or EPs rather than a native read-options policy (#33166, #32265).
• ACL EP is deprecated and will be removed in a future release. Source builds using --use_acl now receive a deprecation warning (#32564).
• The ONNX dependency remains at 1.22.0. The initial ONNX 1.23 integration was reverted pending upstream fixes; this release does not ship that integration's new opset or FLOAT6 support (#33073).
• Supported native builds now default to 1DS telemetry. Windows TraceLogging remains selectable with --use_windows_telemetry or onnxruntime_USE_WINDOWS_TELEMETRY=ON; telemetry can be disabled with --no_telemetry (#32384).

New Features

Core APIs & Runtime

• Stabilized the model-package C and C++ APIs for loading packages, selecting compatible model variants, and creating sessions (#33166).
• Stabilized named EPContext data read/write callbacks for application-managed storage. Added EP support negotiation so registered callbacks do not silently fall back to filesystem I/O (#32265).
• Extended the Compile API to write external initializers and EP context data to buffers, enabling in-memory compilation workflows for models larger than 2 GB (#31347).
• Added workspace-memory accounting, source reporting, and post-partition reservation verification. Set session.strict_workspace_verification=1 to fail initialization on workspace declaration overruns or nonzero orphaned reservations; strict verification is not supported for ORT-format model loads (#31962, #32189).
• Added the opt-in source-build option onnxruntime_DISABLE_DEVICE_DISCOVERY=ON to skip platform accelerator probing and report only the CPU device (#32592).

Shared Contrib Operators

These updates affect core operator schemas and cross-provider behavior, not only the separately released GPU plugins.

• Extended EngramGate and NGramHashMapping for Qwen4-Exp, and added VarlenNGramHashMapping for packed DeepSeek Engram batching with request-local history. Packed hashing also supports Qwen4-Exp head offsets, EOS resets, and segment boundaries (#32285, #32358, #32467).
• Added the activation attribute to GatedRMSNorm for SiLU, its Swish alias, and Sigmoid gating across CPU, CUDA, and WebGPU. SiLU remains the default (#32512).
• Added weightless hyper-connection operators BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix, with CPU, CUDA, and WebGPU implementations (#32687).
• Extended GatherBlockQuantized to FP8/FP4 data, full-axis blocks, and scale broadcasting, and improved CPU UINT8 dequantization. See the out-of-range-index behavior change above (#32480, #32621).

Plugin Execution Provider Infrastructure

• Added support for statically linked plugin EPs, with a static WebGPU plugin build configuration (#32395).
• Preserved caller streams in single-tensor plugin data transfers and handled null plugin allocator returns (#32666, #32683).
• Deduplicated AutoEP custom-op domains and corrected EP assignment reporting after NHWC layout transformation (#32711, #33184).
• Added reference guidance for compiled-model compatibility validation, including multi-device scenarios (#29168).

Execution Provider Updates

CUDA EP

CUDA updates include sparse attention and packed continuous batching, expanded paged speculative decoding, INT2 and mixed-width QMoE execution, opt-in MatMul auto-tuning, improved MXFP4 support on Blackwell consumer GPUs, and reliability fixes.

See the CUDA Plugin EP v0.3.0 release notes for detailed features, configuration options, limitations, and fixes. Most CUDA-specific PRs are covered there rather than repeated here. Use the matched core/plugin combination and observe the contrib-op schema compatibility requirements above.

WebGPU EP

• Expanded generative AI support with sparse attention, packed continuous-batching indexers, GatedDeltaNet, and broader quantized MoE support, including block-scaled FP8 and INT2/mixed-width weights (#32528, #32529, #32533, #32618, #32510, #32794, #32878).
• Improved native provider integration and performance, including explicit session-stream concurrency, subgroup-matrix kernels on AMD GPUs, and standalone Dawn API package support (#32587, #32667, #33006).

Detailed WebGPU kernel additions, optimizations, and correctness fixes will be covered in the separate WebGPU Plugin EP v0.5.0 release notes.

CoreML EP

• Exposed CoreML through OrtEpFactory, device enumeration, SessionOptionsAppendExecutionProvider_V2, and EP selection policies. NPU selection requires iOS 16/macOS 13 or later; GPU selection requires iOS 15/macOS 12 or later. In builds advertising both CoreML and WebGPU for the same Apple GPU, PREFER_GPU/MAX_PERFORMANCE select CoreML through the existing EP-name tie-breaker; WebGPU remains available through explicit device selection (#31975).

WebNN EP

• Added `...

This release brings sparse attention and packed continuous-batching support, INT2 and mixed-width QMoE execution,
decode-kernel tuning, and reliability fixes to the ONNX Runtime CUDA Plugin EP. Highlights include new QSA/CSA
attention operators, expanded paged speculative decoding, opt-in MatMul auto-tuning, and improved MXFP4 support
on Blackwell consumer GPUs.

Compatibility

The minimum ONNX Runtime core version remains 1.24.4, but meeting that minimum alone does not guarantee
compatibility for contributed operators. Core and the plugin must use the same contributed-operator schemas;
building both from the same ONNX Runtime revision is recommended. Schema mismatches can cause incorrect execution
or crashes and may not be detected when registering the plugin (#32855).

The matched ONNX Runtime version is 1.31.0 for this release.

Highlights

Plugin Runtime and CUDA Graphs

• Added checked GroupQueryAttention workspace recipes for KV-cache preparation and XQA, FlashAttention, memory-efficient, and unfused routes, plus conservative workspace estimation for resource-constrained graph partitioning. cuDNN workspace is reported as unavailable rather than incorrectly estimated as zero (#32446, #32453, #32454, #32617).
• Bounded sliding-window GQA workspace by the resident or staged KV extent rather than the unbounded absolute sequence length, and consolidated runtime scratch sizing and sequence-length-independent route eligibility into shared helpers (#32602, #32945, #33001).
• Replaced eligible decode-path memset operations with small clearing kernels for fpA_intB split-K semaphores, paged XQA semaphores, and compact convolution state updates, reducing CUDA Graph dependency overhead (#32884).
• Replaced shared ones-buffer bias broadcasts in Gemm and DecoderAttention with direct broadcasts on the consuming CUDA stream, removed the constant-ones cache, and corrected zero-K Gemm bias scaling (#32852).

Attention and KV Caching

• Added a cuDNN paged SDPA decode tier to PagedAttention for eligible FP16/BF16 causal workloads with unquantized KV caches. It is selected ahead of FlashAttention when XQA is not the winner, automatically considered on SM90+, and can be disabled with ORT_ENABLE_CUDNN_FLASH_ATTENTION=0. Follow-up fixes stabilize default-scale cache keys and backend selection during CUDA Graph execution (#32493, #32624).
• Expanded non-causal PagedAttention beyond FlashAttention, including portable paths for small cache pages and supported XQA speculative paths; added coverage for head size 128 with 16/32/64-token pages and ragged batches (#32619).
• Added causal paged speculative XQA for FP16 query/output, INT8 KV caches, head size 128, GQA group size 6, and query widths 2-8. Unsupported widths and non-causal H128 workloads retain existing fallbacks (#32740).
• Documented the block-eviction layout contract for sliding-window GQA caches, including how to index resident tokens after multi-token speculative or chunked-prefill steps (#32419).
• Allowed GQA to borrow unquantized past K/V without allocating or copying present caches when new K/V is empty and both present-cache outputs are omitted, and corrected the unfused sliding-window bound to include the current token (#33175).
• Skipped redundant present-cache copies in ONNX Attention when the outputs alias the external KV-cache inputs (#31150).

Sparse Attention and Continuous Batching

• Added CUDA SparsePagedAttention and DynamicSparseAttention for externally selected attention indices over paged and contiguous KV caches. These support Qwen4-Exp QSA selected-main-cache attention and DeepSeek V4 CSA joint local/selected-auxiliary attention, with one softmax across participating entries and optional attention sinks (#32524, #32525).
• Added SparseAttentionIndexer for QSA token selection and CSA compressed-block selection, including rotary preparation, scoring, deterministic TopK, and cache updates (#32526).
• Added PackedSparseAttentionIndexer for packed, variable-length continuous batching, with per-request metadata, fixed-capacity state, and selection outputs compatible with SparsePagedAttention (#32618).
• Optimized dense and packed QSA indexing, added packed QK inputs and broader mask/rotary-cache support, and added optional transactional QSA state-update capture to the packed indexer so speculative decoding can replay an accepted prefix (#32656).
• Parallelized DynamicSparseAttention across selected-token tiles, bounded split workspace, replaced quadratic duplicate validation with a GPU bitmap, and added single-split fast paths for large-context inference (#32671).

New Operator and Model Support

• Extended EngramGate and NGramHashMapping for Qwen4-Exp, including normalized gated-value output, autoregressive history, per-head offsets, and EOS/segment resets (#32285).
• Added VarlenNGramHashMapping for packed DeepSeek Engram batching, preserving request-local history and preventing n-gram mixing across request boundaries, then extended it with Qwen4-Exp hashing semantics (#32358, #32467).
• Added an activation attribute to GatedRMSNorm for SiLU, its Swish alias, and Sigmoid gating. SiLU remains the default (#32512).
• Added the weightless hyper-connection fusion operators BranchwiseRMSNorm, ScaledSiLU, HyperConnectionPreMix, and HyperConnectionPostMix (#32687).
• Extended GatherBlockQuantized to FP8 and FP4 data, including full-axis blocks and scale broadcasting. Behavior change: out-of-range indices now produce zero output slices for both integer and floating-point quantized formats, rather than following ONNX Gather error semantics (#32480).

Quantized MatMul and MoE

• Added per-projection QMoE weight-width attributes and CUDA execution for uniform INT2 and mixed INT2/INT4 FC1/FC2 combinations (2,2), (2,4), and (4,2), with packed decode paths and a scratch-bounded dequantization fallback for supported configurations (#32697, #32743, #32761).
• Enabled eligible packed INT2/mixed-width QMoE prefill by default on SM80+ with FP16/BF16 activations, symmetric quantization, and interleaved fused SwiGLU. Support covers block sizes 32/64/128 and the three FC1/FC2 combinations above, subject to alignment, expert-count, and workspace restrictions (#32963, #33045, #33005, #33092).
• Added block-scaled FP8 QMoE support with routed-expert compaction, alongside the existing per-expert global-scale mode (#32887).
• Made NVFP4 QMoE decode consume schema-native packed weights and scales directly, avoiding decode-only repacking while retaining grouped-GEMM paths for larger workloads (#32325).
• Fixed MXFP4 W4A16 QMoE session creation and dispatch on SM120/SM121 and other supported SM80+ configurations, corrected the plugin adapter's FLOAT8E8M0 type mapping, and accelerated FP4 conversion in decode and grouped GEMM (#33067).
• Added BF16 routing support to CUDA MoE and documented/tested packed token inputs for MoE and QMoE (#32659, #32522).
• Added opt-in FP16/BF16 MatMul auto-tuning via ep.cuda.enable_gemm_auto_tune=1 or ORT_CUDA_GEMM_AUTO_TUNE=1, choosing among cuBLAS, small-N GEMV, and eligible tinygemm2 kernels on SM90+. cuBLAS remains the default. Graph-enabled sessions validate candidates with CUDA Graph replay before caching the winner; warm-up inference is required before capture (#32876, #32901, #33085).
• Improved MatMulNBits small-M GEMV tiling, biased near-tie tactic selection toward GEMV for small decode batches, and enabl...

This is a patch release on top of v1.28.2, containing memory-safety and model-validation fixes, a plugin execution fix, and packaging infrastructure updates.

Highlights

Bug Fixes

• Fixed an out-of-bounds read in EmbedLayerNormalization shape inference when beta is a scalar (#32610).
• Fixed a heap buffer overflow when reusing a packed sub-byte buffer for a full-byte tensor (#32611).
• Fixed an out-of-bounds read when a Loop or Scan body input is also an initializer (#32609).
• Hardened shape inference for applicable contrib operators, including GroupQueryAttention, SparseAttention, and CausalConvWithState (#32607).
• Fixed out-of-bounds reads in ConvTransposeWithDynamicPads and InferenceContextImpl::getInputData (#32679, #32681).
• Rejected invalid graph structures: inputs to a zero-input operator schema and subgraphs on non-control-flow nodes (#32633, #32641).
• Preserved streams during single-tensor data transfers in the plugin EP (#32666).

Build and Test Reliability

• Unblocked packaging with Linux image prerequisites and an authenticated ARM64 pip index, mirrored Dawn checkouts, authenticated Android pip and Gradle feeds, and authenticated NuGet feeds for mobile workloads (#31724, #29743, #32400, #32005).
• Installed uuid-dev for the Linux minimal CoreML build and removed the Hugging Face download dependency from a quantization test (#32491, #32443).

Contributors

Thanks to our 10 contributors for this release!

@wangw-1991, @bbernhar, @shiyi9801, @tianleiwu, @adrastogi, @eserscor, @jiafatom, @miaobin, @sanaa-hamel-microsoft, @xiaofeihan1

Full Changelog: v1.28.2...v1.28.3

These release notes were drafted with assistance from GitHub Copilot.

This release brings expanded quantized inference and attention support, kernel performance improvements, and
reliability fixes to the ONNX Runtime CUDA Plugin EP. Highlights include 2-bit MatMulNBits, INT4 paged KV caches,
speculative decoding improvements, compact recurrent-state updates, and safer CUDA Graph execution across sessions.

Please refer to the CUDA Plugin EP Quick Start for usage and installation guidance. The matched ONNX Runtime version is 1.30.0.

Highlights

Plugin Runtime and CUDA Graphs

• Gave each plugin EP session its own device arena, preventing cross-session reuse of CUDA Graph buffers and honoring per-session arena options (#32807).
• Added workspace-memory accounting and reporting for resource-constrained graph partitioning, including packed attention workspace recipes and estimates, and optional-input-aware shape handling (#31962, #32283, #32312, #32321).
• Made GatherND capture-safe and kept CudaAsyncBuffer staging buffers alive across graph replay (#32120, #32121).
• Fixed shared-cache scratch lifetime in CUDA MHA, reset plugin EP stream chunks before release, and corrected the user-stream test lifetime (#31968, #31983, #32449).
• Fixed CUDA plugin device discovery on WSL, with improved hardware-identity matching and handling of devices missing from platform discovery (#32517).

Attention and KV Caching

• Added bidirectional GroupQueryAttention support and an is_causal attribute to PagedAttention (#31704, #32225).
• Added INT4 paged KV caches with per-channel scales, including FP16 XQA decode and speculative-decode paths. Packed INT4 cache storage uses half the bytes of INT8 cache storage; availability is controlled by the onnxruntime_USE_INT4_KV_CACHE build option, which defaults to enabled (#32515).
• Enabled split-KV for paged FlashAttention decode and improved PagedAttention dispatch diagnostics (#32102, #32099).
• Expanded paged XQA with group-size-6 support, head size 256, FP16-cache decode at head size 256, and native block tables for 128-token pages (#32108, #32229, #32263, #32127).
• Added paged XQA speculative verification for metadata-bounded query groups of 2-8 tokens at head size 256 and group size 6, including ragged batches and INT8, FP8, and native FP16/BF16 KV caches (#32340).
• Added the session.gqa_value_layout session option for applications using a BNHS GroupQueryAttention Value cache, with graph-inserted layout conversions; BNSH remains the default (#32139).

New Operator and Model Support

• Added compact GatedDeltaNet with transition capture for replaying accepted speculative prefixes, plus BF16 support (#32282, #32307).
• Added VarlenCausalConvWithState for packed, variable-length continuous batching, and compact state updates that avoid duplicating full convolution-state checkpoints for speculative prefixes (#32168, #32290).
• Added CUDA kernels for the DeepSeek Engram contrib operators EngramGate and NGramHashMapping (#32268).
• Registered BF16 ReduceMean kernels (#32326).

Quantized MatMul and MoE

• Added 2-bit MatMulNBits support, including dequantization and specialized GEMV/batched paths, and fused floating-point-activation/integer-weight (fpA_intB) GEMM/GEMV kernels for eligible FP16/BF16 workloads (#32693, #32699).
• Made compact fpA_intB kernels the default build configuration, extended compact MatMulNBits to BF16, and refined GEMV eligibility checks. The complete kernel matrix remains available through onnxruntime_USE_FPA_INTB_GEMM_FULL (#32324, #32721, #32338).
• Enabled FP4 QMoE kernels by default in CUDA builds and added Windows support for Blackwell SM120 (#32096, #32163).
• Added an opt-in FP8 DeepGEMM decode path for eligible SM90 QMoE workloads, controlled by ORT_QMOE_FP4_DEEPGEMM and disabled by default. This path is disabled on Windows because of upstream header requirements (#32122, #32485).
• Bounded QMoE workspace with configurable row tiling and bounded FP8 weight-dequantization scratch by tiling over N (#32097, #32129).
• Vectorized NVFP4 weight dequantization for prefill, tuned NVFP4 GEMV tiling for Qwen multi-token prediction, and extended speculative-decode GEMVs to 64 rows (#32128, #32140, #32289).
• Tuned FP4/FP8 GEMV scheduling for 48-SM SM121 GPUs and improved FP8 GEMV residency for grids just past two blocks per SM (#32408, #32409, #32433).
• Reduced expensive fpA_intB tactic-profiling work and added optional, size-gated M-row chunking for large MatMulNBits workloads. Chunking is disabled by default and can be configured with ep.cuda.matmul_nbits_m_chunk_size or ORT_MATMULNBITS_M_CHUNK_SIZE (#32758, #32810).

General Kernel Performance

• Parallelized ArgMax/ArgMin over wide last axes, optimized wide-last-axis TopK, and accelerated low-lane INT64 CumSum (#32092, #32404, #32238).
• Added a single-memcpy Slice fast path for contiguous subregions (#28902).
• Passed Concat per-input metadata by value and avoided pinned buffers in CUDA Split and Concat fast paths (#32119, #32410).

Reliability and Correctness

• Fixed ScatterElements reduction dispatch by element type and Abs signed-zero handling (#29879, #31477).
• Handled zero-element inputs/bias in BiasGelu and FastGelu as no-ops and zero-sized outputs in CUDA random generator kernels (#31698, #31997).
• Added bounds and shape validation for per-element Split sizes, 8-bit MatMulNBits g_idx, GatherElements counts, and ScatterND index depth (#29461, #31643, #32030, #32034).
• Validated CUDA NMS mask sizes, QDQ element counts, per-channel ImageScaler bias, and Crop inputs (#32014, #32029, #32002, #32157).
• Bounded RemovePadding sequence-token counts and Whisper beam-search cross-QK layer/head indices (#31994, #31998).
• Hardened integer arithmetic in RotaryEmbedding, SparseAttention, CUDA reduction scans, and softmax offsets (#31995, #31996, #32137, #32330).

...

ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1.

Highlights

• Expanded CUDA inference support with variable-length causal convolution for continuous batching, speculative decoding in paged XQA, and INT4 paged KV caches with per-channel scales (#32168, #32340, #32515).
• Improved WebGPU PagedAttention, added GPT-OSS support and INT8 KV-cache block quantization, and extended convolution optimizations (#31727, #32277, #32284, #32420).
• Added fused CPU LinearAttention kernels for AVX-512, Arm64 NEON, and SVE, plus AVX2 LayerNorm/RMSNorm acceleration (#31674, #31973, #32178, #32356).
• Added Go bindings for the ONNX Runtime C API and DeepSeek Engram contrib operators (#29615, #32268).

Announcements & Compatibility

• FP4 QMoE kernels are now enabled by default in CUDA builds, with Windows build support added in this release. Source builds can opt out with -Donnxruntime_USE_FP4_QMOE=OFF (#32096, #32163).
• CUDA fpA-intB builds now default to a compact kernel set for FP16 activations, INT4/INT8 weights, scale-only quantization, and block_size=32. Set -Donnxruntime_USE_FPA_INTB_GEMM_FULL=ON when building from source to retain the full kernel set, including BF16, zero-point, bias, larger-block-size, and native Hopper variants (#32324).
• CPU FP16 Gemm and MatMul execution is gated on hardware acceleration. CPU-assigned FP16 nodes without a matching kernel now fall back to FP32 (#32301, #32197).
• WebGPU plugin EP packaging now supports Linux AArch64. Plugin versions were advanced to WebGPU 0.4.0 and CUDA 0.2 (#32287, #31960, #31970).

Security & Reliability

Model Loading, Memory, and Input Validation

• Limited nested model-graph depth and canonicalized external-data locations to harden model loading (#32344, #32135).
• Added checked rounding for BFC arena allocations and fixed prepacked-weight reference lifetimes (#32010, #32040).
• Strengthened shape, rank, and parameter validation for Split, Scan, GatherND, ScatterND, SpaceToDepth/DepthToSpace, Crop, Conv, Normalizer, and pooling (#29461, #31668, #32034, #32039, #32076, #32157, #32160, #32161, #32345, #32349).
• Hardened generation and attention input handling, including attention-attribute narrowing, BifurcationDetector inputs, generation subgraph shapes, and QEmbed segment inputs. BeamSearch buffer expansion now uses dynamic shape storage (#31648, #31701, #32009, #32078, #32144).
• Validated TreeEnsemble node references and bounded subtree comparison, rejected non-finite CPU RoiAlign coordinates, and required ImageScaler bias to match the channel count (#32031, #32043, #32011, #32002).
• Added an allowlist of safe LoRA adapter parameter data types, validated MatMulFpQ4 shape inputs, and checked MLAS blockwise quantization/dequantization index ranges (#31682, #32032, #32007).

GPU Bounds and Resource Lifetimes

• Hardened CUDA indexing and buffer-size arithmetic in MatMulNBits, RemovePadding, RotaryEmbedding, SparseAttention, Whisper beam search, NMS, QDQ, and GatherElements (#31643, #31994, #31995, #31996, #31998, #32014, #32029, #32030).
• Fixed overflow in CUDA reduction scans and Softmax offset arithmetic, and handled zero-sized outputs in CUDA random-generator kernels (#32137, #32330, #31997).
• Fixed CUDA MultiHeadAttention shared-cache scratch lifetimes and kept CudaAsyncBuffer staging storage alive across CUDA graph replay (#31968, #32121).
• Fixed WebGPU out-of-bounds subgroup-matrix loads for partial tiles, zero-initialized writable device-allocator buffers, and rejected foreign GPU handles in built-in data transfers (#32364, #32063, #32317).

Dependencies and Tooling

• Upgraded Protobuf to 33.6 and refreshed Python documentation dependencies, including an ONNX security-related update (#29906, #32190, #32424).
• Updated JavaScript dependencies including js-yaml, joi, fast-uri, and the Next.js end-to-end fixture (#32397, #32486, #32488, #32505, #32508).
• Pinned GitHub Actions to full-length commit SHAs and strengthened packaging infrastructure with authenticated package feeds and NPM network isolation (#32176, #32005, #32440).

New Features

Core APIs & Runtime

• Added Go bindings for the ONNX Runtime C API (#29615).
• Extended memory importing with host-pointer support and added access to preallocated outputs through KernelContext::GetPreallocatedOutput (#29726, #32089).
• Added packed-attention workspace recipes and estimates, and made workspace input-shape handling aware of optional inputs (#32283, #32321, #32312).
• Added DeepSeek Engram contrib operators, EngramGate and NGramHashMapping, and expanded kernel coverage for Qwen-3.5 operators (#32268, #32106).

Plugin Execution Providers

• Added LoRA adapter support with plugin EP allocators, preserved custom allocators during EP registration, and reset plugin stream chunks before release (#32221, #32272, #31983).
• Fixed CUDA plugin device discovery on WSL and improved Windows ARM64 packaging (#32517, #32355).

...

This is a patch release on top of v1.29.0, containing GroupQueryAttention capability and KV-cache layout improvements, plugin Execution Provider performance tooling updates, and targeted graph and optimizer fixes.

GroupQueryAttention

• Added bidirectional GroupQueryAttention support on CPU and CUDA through a backward-compatible causal attribute, with explicit handling for unsupported execution paths (#31704)
• Added a session option and Execution Provider metadata contract for using the BNHS Value KV-cache layout, with graph transformations that preserve compatibility with the existing BNSH operator schema (#32139)
• Added CPU support for attention_bias with a sliding-window KV cache, including explicit position IDs and post-eviction bias indexing (#32302)

Runtime and Performance Tools

• Fixed Compile API model serialization when output-model and custom initializer-location callbacks are used together, preventing duplicate graph fields in emitted models (#32303)
• Updated onnxruntime_perf_test to use plugin Execution Provider device allocators for generated inputs, loaded test data, and pre-allocated outputs, avoiding unnecessary per-run host/device copies (#32244)

Bug Fixes and Documentation

• Hardened FastGelu fusion to skip malformed Mul and Pow patterns (#32016)
• Added validation for in-memory external initializer references, rejecting unregistered or mismatched data before graph transformation (#32042)
• Restored the C API documentation workflow by switching the pinned Doxygen download to the official GitHub release asset (#32210)

Contributors

Thanks to our 7 contributors for this release!

@adrastogi, @apsonawane, @edgchen1, @javier-intel, @jnagi-intel, @tianleiwu, @Wayne-Ch

_{Release highlights were drafted with AI assistance and are subject to release-team review.}

Full Changelog: v1.29.0...v1.29.1

This release brings kernel performance improvements, expanded operator support, and reliability fixes to the ONNX Runtime WebGPU Plugin EP.

Highlights

Performance

• Optimized MatMulNBits wide-tile execution using subgroup shuffle (#31703)
• Supplied fused activation parameters to Conv/MatMul as uniforms, and fused eight more activations into Conv (#32116, #32117)
• Added fused activation support to the im2col Conv path (#32185)
• Vectorized Split when every output segment is vec4-aligned (#32251)
• Shared subgroup matrix MatMul with pointwise Conv, and pinned subgroup size to 32 for subgroup-matrix MatMul/Gemm (#32304, #32306)
• Selected the pooling path by occupancy rather than output size (#32313)
• Prepacked Conv weights for the im2col-matmul path (#32420)
• Scaled Dawn pipeline compilation workers with CPU count (#29820)
• Added PagedAttention metadata and GPT-OSS support, and continued optimizing PagedAttention (2/n) (#32277, #31727)
• Added int8 KV cache block quantization (#32284)

New Operator and Model Support

• Implemented WebGPU subgroup-size-control infrastructure and enabled the subgroup matrix path for WASM builds (#32056, #32269)
• Added more kernels for Qwen-3.5 ops and DeepSeek Engram contrib ops (EngramGate, NGramHashMapping) (#32106, #32268)

Reliability and Correctness

• Fixed the 1D-dispatch shader fast path (#32343)
• Fixed out-of-bounds subgroup-matrix loads for trailing partial tiles (#32364)
• Made MatMulNaiveProgram's pipeline cache key cover everything it bakes into WGSL (#32048)
• Reported copy_tensors misuse instead of terminating the process, and rejected foreign GPU handles in the built-in data transfer (#32315, #32317)

Build, Packaging, and Tooling

• Enabled Linux AArch64 WebGPU plugin EP packages (#32287)
• Separated onnxruntime_providers compilation from its archive (#32162)
• Ran wgsl_template Python tests in CI (#32214)

Contributors

Thanks to everyone who contributed to this release:

@4n4ny4, @daijh, @edgchen1, @hariharans29, @jchen10, @kunal-vaishnavi, @qjia7, @rvandermeulen, @sushraja-msft, @tianleiwu, @xhcao, @xiaofeihan1

This release covers commits affecting WebGPU Plugin EP code and packaging.

This summary was drafted with AI assistance from commit history and PR metadata, and reviewed before publishing.