Skip to content

Releases

b12142

b12142 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 06 Nov 02:22
b12142
3b83d12
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

vulkan : check combined K+V f16 scratch size in FA dequant path (#30487)

Fixes #30048

Assisted-by: Claude

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12141

b12141 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 06 Nov 01:49
b12141
e152332
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

llama-server: Use reference-counted checkpoints instead of deep-copies (#30137)

  • checkpoints: reduce overhead

Assisted-by: Codex

  • rebase

Adapt the slot save/restore checkpoint appendix from #26004 to the
shared_ptr checkpoint list and the common_state_data buffers.

Assisted-by: Claude Opus 5.5

Co-authored-by: cwriter cwriter@localhost

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12140

b12140 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 06 Nov 01:11
b12140
1fa7f1c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

ggml : support truncate=false for YaRN (#30206)

  • ggml : support truncate=false for YaRN

YaRN calculates correction intervals to switch frequency interpolation
methods, but the setting to not round the boundaries of these intervals
to integers (truncate=false) was not supported.

Models that use truncate=false include gpt-oss and some PLaMo-3
models; to support them correctly, the GGML operations and CPU backend
are modified to handle calculations without rounding. The default
behavior is truncate=true, and existing calls not explicitly
configured will maintain their previous behavior.

  • support CUDA backend

  • add tests

  • move RoPE helpers to internal header

  • format

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12138

b12138 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 22:07
b12138
f7b41cc
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

metal : use mul_mm for 32 <= K < 64 (#30982)

  • metal : use mul_mm for 32 <= K < 64

  • metal : add bf16 x bf16 mul mat kernel

Co-authored-by: Georgi Gerganov ggerganov@gmail.com

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12137

b12137 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 21:49
b12137
c55b094
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

server : reject non-object session updates (#31714)

Prevent malformed realtime session updates from throwing an uncaught
exception and terminating the server

Signed-off-by: Adrien Gallouët angt@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12136

b12136 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 21:32
b12136
2c19244
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

fit : account for skipped NextN layers when setting n_gpu_layers (#31569)

  • fit : account for skipped NextN layers when setting n_gpu_layers

Without MTP, fit counts n_layer + 1 layers while the model loader counts
n_layer_all + 1 for n_gpu_layers, including the NextN layers which hold no
weights. A fit result that offloads all layers then leaves layer 0 on the CPU
("offloaded 65/66 layers" for Qwen3.8-27B on two GPUs).

Add the skipped NextN layers to n_gpu_layers and to the tensor split of the
device that holds the output layer so the loader assigns the layers that
fit measured.

Fixes #31531

  • fit : pass hp_ngl_skip by reference like the other hparams

  • fit : add hp_ngl_skip to common_get_device_memory_data

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12135

b12135 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 21:13
b12135
553d2c2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

webgpu : add MEAN support (#31591)

Assisted-by: Codex

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12132

b12132 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 20:33
b12132
2ada830
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

common : drop u8path() and use generic_string() (#31681)

Signed-off-by: Adrien Gallouët angt@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12131

b12131 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 20:06
b12131
5a081c0
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

server : drop u8path() and use generic_string() (#31682)

Signed-off-by: Adrien Gallouët angt@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading

b12130

b12130 Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 05 Nov 19:36
b12130
7e42b95
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

rpc : align code with common.cpp (#31684)

Signed-off-by: Adrien Gallouët angt@huggingface.co

Website:

Attestations:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI:

Loading