Skip to content

Releases

Patch release v5.22.2

Patch release v5.22.2 Latest
Latest

Choose a tag to compare

View all tags
@vasqu vasqu released this 06 Nov 15:43
v5.22.2
6d195c6

Patch release v5.22.2

Some in between patch for security and distributed fixes. See the included commits 🤗

Full Changelog: v5.22.1...v5.22.2

Contributors

Rocketknight1, michaelbenayoun, and 3outeille
Assets 2
Loading

Patch release v5.22.1

Patch release v5.22.1

Choose a tag to compare

View all tags
@ArthurZucker ArthurZucker released this 30 Oct 09:36
v5.22.1
446a570

Patch release v5.22.1

We just had to fix the documentation!

** [Fix] Remove WIP doc for Ml4 (#50124) from @remi-or

Contributors

remi-or
Loading

v5.22.0: Mistral Large 4 (Le Chonk)

v5.22.0: Mistral Large 4 (Le Chonk)

Choose a tag to compare

View all tags
@LysandreJik LysandreJik released this 29 Oct 14:15
v5.22.0
0f01f27
This commit was signed with the committer’s verified signature.
Cyrilvallez Cyril Vallez
SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA
Verified
Learn about vigilant mode.

New Models

Mistral Large 4 (Le Chonk)

image

Mistral Large 4 is Mistral's largest and most capable model to date: a 1 trillion-parameter natively multimodal Mixture-of-Experts model with 49 billion active parameters per token (52 billion including embeddings and output layers), trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs in Mistral's own datacenters in Europe.

Mistral Large 4 unifies instruction following, reasoning, coding, and agentic workflows in a single set of weights. While a generalist model, it shows especially good performance on workloads such as cybersecurity, finance, and law, and it surpasses frontier closed models on visual grounding. It is natively fluent in 160+ languages, including every official language of the European Union.

Key Features

Mistral Large 4 includes the following architectural choices:

  • ~1T total / ~52B active parameters: granular Mixture-of-Experts with 128 routed experts (4 active per token) and 1 shared expert.
  • 1M context length, for long-context workloads.
  • Multimodal input: accepts text and image input, with text output. Pixtral-style vision encoder trained to handle variable image sizes and aspect ratios.
  • Instruct and Reasoning functionalities with function calls, reasoning effort configurable per request.

Mistral Large 4 offers the following capabilities:

  • Reasoning Mode: three reasoning efforts — low, medium, high (default) — so the same model can answer a quick chat reply or work through a complex agentic run.
  • Vision: analyzes images, charts, technical drawings, and documents, with state-of-the-art visual grounding among open weights.
  • Multilingual: natively fluent in 160+ languages, including all official languages of the European Union.
  • System Prompt: strong adherence and support for system prompts.
  • Agentic: native function calling, JSON output, and strict JSON schema enforcement.
  • Cybersecurity: state of the art among open-weight models for security work, while refusing malicious requests.
  • Large Context Window: supports a 1M context window.

The model is released under the Mistral Ecosystem Open-Weight License: a modified version of the MIT license, for both commercial and non-commercial use — operating a Model-as-a-Service business requires a separate commercial license with Mistral.

Recommended Settings

  • Reasoning Effort:
    • 'low' / 'medium' / 'high' — default is 'high'.
    • Use reasoning_effort="high" for complex prompts and agentic usage, 'low' for fast chat replies and 'medium' for day to day usage.
  • Sampling: for all reasoning efforts, we recommend the following sampling parameters:
    • temperature=1.0
    • top_p=0.95
  • System Prompt: define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.
  • Tools: keep the set of tools well-defined and limit their number to the minimum required for the use case.
  • Context: the checkpoint supports a 1M context window.

Breaking changes

Quantization

FP8 quantized models can now be used for adapter training (such as LoRA with PEFT). Backward passes were added to the dense and grouped FP8 matmuls, and is_trainable is now True. Backward also dequantizes experts in chunks and skips autograd at inference, which cuts peak memory in the MoE layer backward from about 28.9 GiB to 2.8 GiB at 8k tokens. Separately, the AMD CI image now install

Bugfixes and improvements

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @tarekziade
    • Fix _init_weights gaps in 7 audio models and enable test_can_init_all_missing_weights for them (#49386)
    • Fix image backend equivalence tests for Pillow 13 (#50105)
    • [Janus] Fix processor iterating a string prompt character by character (#50044)
    • [timesfm] Fix fp16 overflow on pad_val in _forward_transform (#50021)
    • Pin the shared Hub MIC fleet dataset (#49802)
    • Remove legacy benchmark/ and disable automatic benchmark runs (#50040)
    • [minimax_m3_vl] Gate the integration tests on total accelerator memory (#50025)
    • [moshi] Skip Moshiko integration tests whose checkpoint is gone (#50017)
  • @eustlb
    • Add Nemotron 3.5 Transcribe (#50084)
    • [Nemotron3Diarization] add left context support (#49283)

Contributors

tarekziade, ydshieh, and 21 other contributors
Loading

Patch release v5.21.1

Patch release v5.21.1

Choose a tag to compare

View all tags
@ArthurZucker ArthurZucker released this 29 Oct 08:35
v5.21.1
b58f4b3

Patch release v5.21.1

** Fix rope_scaling setter for configs that assign it during init (#50080 )

We broke remote code loading in vllm, this fixes it sorry everyone!

Loading

Release: v5.21.0

Release: v5.21.0

Choose a tag to compare

View all tags
@Cyrilvallez Cyrilvallez released this 26 Oct 18:17
v5.21
dded62b
This commit was signed with the committer’s verified signature.
Cyrilvallez Cyril Vallez
SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA
Verified
Learn about vigilant mode.

Release v5.21.0

New Model additions

MiniCPM-OCR

MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family. It keeps the MiniCPM-V 4.6 architecture, which pairs a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts and tables.

Links: Documentation | Paper

Flint

flint_benchmarks_nvfp4

Flint is Recursive's mixture-of-experts language model. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, it uses hybrid attention (sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers). It also has learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, and the first two layers are dense.

Links: Documentation

UltraVBert

UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view. The connector merges each 4 x 4 block of vision patches into one token, so a tile becomes 64 image tokens, and RMS-normalizes them to the scale of UltraBERT's token embeddings. The max_image_tiles setting (27 by default) caps the number of tiles per page.

Links: Documentation

Gradium Phonon TTS

Gradium Phonon TTS is Gradium's family of on-device text-to-speech models, trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and the text, then emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends. The latents are decoded to 24 kHz audio by a Pocket TTS audio tokenizer. Unlike Pocket TTS, voice conditioning uses 512-dimensional latents from a separate, unreleased speaker codec, so voices come as precomputed latents and cannot be cloned from a recording.

Links: Documentation

Pocket TTS

Pocket TTS is Kyutai's 100M-parameter text-to-speech model, built as a Continuous Audio Language Model that runs faster than real time on CPU. A causal transformer emits one continuous audio latent per frame (12.5 Hz) through a one-step flow head, and an end-of-speech head decides when to stop. A Mimi-based audio tokenizer encodes voice prompts and decodes the generated latents to 24 kHz audio. Two flow heads are released, Lagrangian self-distillation (default) and drifting, and voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states.

Links: Documentation | [Paper](https://huggingface.co/papers/2509.06926 (the Continuous Audio Language Models paper, arXiv 2509.06926). It is the only paper the documentation links, and it describes the architecture family Pocket TTS belongs to. The docs do not cite a separate Pocket TTS technical report.) | [Blog Post](NONE. The documentation links no official blog post. It points only to the original code repository at https://github.com/kyutai-labs/pocket-tts.)

Pocket TTS Audio Tokenizer

The Pocket TTS audio tokenizer is the codec used by Pocket TTS. It has Mimi's architecture (SEANet convolutions plus a small causal transformer on each side), but its residual vector quantizer is replaced by a continuous 32-dimensional bottleneck, the VAE-GAN from Continuous Audio Language Models. It converts 24 kHz audio into latents at 12.5 Hz and decodes them back to audio. It serves as the audio_tokenizer of PocketTTSProcessor, encoding voice prompts and decoding generated latents, and its encoder and decoder can also be loaded on their own.

Links: Documentation | Paper

Breaking changes

Fixed Gemma4 processor imports (the Gemma4, Gemma4 unified and EmbeddingGemma2 processors) so from transformers import * no longer fails in environments that have PIL but not torch; no user action is needed.

  • 🚨 Fix gemma4 cross imports resulting in import failing without torch but with PIL (#49748) by @Sai-Suraj-27

Added an "fp8" backend to QuantizedCache, alongside quanto and hqq, that stores KV states in torch.float8_e4m3fn with a single scale per tensor; enable it with cache_implementation="quantized" and cache_config={"backend": "fp8"}.

Tensor parallelism now has a dedicated embedding style, shards every model's input embeddings by default, and supports widths that aren't divisible by the TP size, so ColwiseParallel and RowwiseParallel now apply only to linear layers; custom TP plans that applied them to embeddings must switch to the new embedding style.

  • 🚨 TP: give embeddings their own style, shard input embeddings by default, and shard widths that don't divide by the TP size (#49570) by @IlyasMoutawwakil

Every MoE TP and EP plan now shards its experts, so models no longer keep all experts on every rank, and a new test_moe_parallel_plans_shard_experts test enforces this; custom or downstream MoE parallel plans must add expert sharding to pass it and to avoid wasted memory.

Cache

This release improves cache handling across the library. Flash Attention now works with static caches, BLOOM no longer crashes with static cache or compiled generate, chunked prefill supports cache continuation, and the Qwen4-Exp indexer's position history stays aligned with its cached keys. Linear-attention caches also gain new getters (is_recording_past, get_recurrent_state, conv-state access) that stop exposing cache internals and prepare for paged-c

Vision

Vision image processors now composite PNG transparency onto white instead of exposing black pixels, and background_color is preserved correctly across the DeepSeek-VL and Janus processors. Model fixes cover SmolVLM's inputs_merger when image tokens aren't a multiple of the patch size, HunYuan-VL image assignment during batched beam search, and Ernie4.5-VL-MoE expert sharding under TP/EP. The DETR

Generation

This release fixes a range of generation bugs: BLOOM crashes with static cache or compiled generate, generate dropping the type of dict-like inputs such as ModelOutput, and incorrect image assignment or placeholder handling ...

Read more

Contributors

tarekziade, RyanMullins, and 65 other contributors
Loading

Patch release v5.20.2

Patch release v5.20.2

Choose a tag to compare

View all tags
@Cyrilvallez Cyrilvallez released this 19 Oct 17:45
v5.20.2
18329f1
This commit was signed with the committer’s verified signature.
Cyrilvallez Cyril Vallez
SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA
Verified
Learn about vigilant mode.

Patch release: v5.20.2

This patch notably solves an issue where some inputs could loose their original type inside prepare_inputs_for_generation in generate, leading to problems with multimodal models. It also fixes some other issues listed on the below commits.

It contains the following commits:

Contributors

hmellor, qgallouedec, and Cyrilvallez
Loading

Release v5.20.1

Release v5.20.1

Choose a tag to compare

View all tags
@vasqu vasqu released this 16 Oct 13:02
v5.20.1
0ea8779

Release v5.20.1

Special surprise including the new model Beam from Reflection AI 🥳

Gemini_Generated_Image_ox4hjlox4hjlox4h

Beam is a token-efficient, frontier-scale model focused on coding and agentic use cases. It is built for teams that want to run a powerful model on their own hardware, infrastructure, or cloud, and customize it to their specific needs.

Links: Documentation

Some real patch fixes

We broke loading models on older torch versions 😢 should be fixed now. But here is the full list of fixes

Full Changelog: v5.20.0...v5.20.1

Contributors

IlyasMoutawwakil, yonigozlan, and 3 other contributors
Loading

Release v5.20.0

Release v5.20.0

Choose a tag to compare

View all tags
@vasqu vasqu released this 15 Oct 18:05
v5.20.0
48e4bcd

Release v5.20.0

New Model additions

Muse Spark Open

image

Muse Spark Open is a multimodal mixture-of-experts model from Meta that takes text, images, audio, and video as input and generates text. It has 816B total parameters (42B active per token) and a context window of up to 1,048,576 tokens shared across modalities. It is trained for agentic coding tasks such as multi-file editing, debugging, and tool use. Checkpoints are available in BF16 and NVFP4 formats.

Links: Documentation | Blog Post

Ling 3.0

Ling 3.0 Flash is a hybrid Mixture-of-Experts language model from InclusionAI. Its decoder groups five Kimi Delta Attention (KDA) layers with one Multi-head Latent Attention (MLA) layer. The MLA blocks use interleaved rotary embeddings and a head-wise output gate, and the feed-forward blocks combine routed and shared experts. It is natively supported in Transformers as BailingHybridForCausalLM, so it loads without trust_remote_code=True.

Links: Documentation

Kolibri1

Kolibri 1 is Aleph Alpha's mixture-of-experts reasoning model, focused on German and English. Each of its 50 layers routes every token to 6 of 384 experts using sigmoid routing, and adds one shared expert. Four sliding-window attention layers alternate with one full-attention layer, and RoPE is applied only in the sliding-window layers.

Links: Documentation

MiniCPM-V 4.7

MiniCPM-V is a series of efficient multimodal large language models developed by OpenBMB. Like MiniCPM-V 4.6, the MiniCPM-V 4.7 architecture pairs a SigLIP vision encoder that has a window-attention merger with a Qwen3.5 language model backbone, and supports both 4x and 16x visual downsampling modes.

The main addition over 4.6 is canvas M-RoPE: instead of numbering visual tokens along a single 1-D sequence, the model lays every image out on a 2-D canvas and assigns each visual token a (temporal, height, width) position, so slices of the same image keep their spatial relationship and video frames keep their temporal order.

UnlimitedOcr

UnlimitedOcr is a single 3B-parameter OCR model from Baidu with a standard 32,768-token context length. It builds on DeepSeek OCR and replaces all decoder attention layers with Reference Sliding Window Attention (R-SWA). This keeps the KV cache constant during decoding, so the model can transcribe dozens of pages in a single forward pass. It supports "gundam" mode (640x640 tiles with dynamic cropping) and "base" mode (a single 1024x1024 view), and it can output region detections with bounding boxes.

Links: Documentation | Paper

UltraBERT

UltraBERT is a family of bidirectional text encoders trained with a combination of masking ratios, pruning, and distillation for classification and retrieval tasks. The released encoders support sequences of up to 32,768 tokens. They use rotary position embeddings, RMSNorm, SwiGLU feed-forward layers, and alternating local and global attention (every third layer is global). Checkpoints include UltraBERT-150M and UltraBERT-380M for masked language modeling and fine-tuning, and UltraBERT-Embed-150M for text embeddings.

Links: Documentation | Paper

MOSS-Transcribe-Diarize

MOSS-Transcribe-Diarize 0.9B is an end-to-end audio understanding model for long-form multi-speaker transcription, speaker diarization, and timestamps. It combines a Whisper-style encoder, a 4× frame merge step, a multi-modal projector, and a Qwen3 language model. It produces joint transcription and diarization in a single pass, with segments formatted as [start][S01]text[end]. It supports long-form audio through Whisper-window chunking in the processor, promptable transcription with hotword biasing, and 50+ languages.

Links: Documentation

Breaking changes

🚨 General breaking notice that inference with EP has regressions in speed; we are working on fixing this ASAP! 🚨

Deprecated checks and methods have been removed from cache_utils.py (affecting models such as minimax_m3_vl and pi0), so any code still calling those deprecated cache methods must switch to the current cache API.

Accidental imports from other models were removed from several model files (e.g. embedding_gemma2, ijepa, lightglue), and the modular converter now avoids adding such cross-model imports. If you imported these names through those model modules, import them from their original modules instead.

The minimum kernels version is now 0.17.2, which allows use_fallback=KernelizeFallback.ALL so that a failed kernel load falls back to the original implementation instead of raising an error. Upgrade kernels to 0.17.2 or later.

  • 🚨 Bump min kernels to 0.17.2 for better fallback (#49235) by @SunMarc

A new SubConfigSpec centralizes the initialization of sub-configs, so default sub-configs are collected from dataclass fields instead of by instantiating the class, and this touches the configuration utilities and many model configs. Custom or existing nested configs may need to be updated to declare their sub-configs through this mechanism.

The LUKE and mLUKE tokenizers now treat entity special tokens such as <ent> as special tokens (not split, honored by skip_special_tokens, normalized) while keeping the same spacing as regular vocabulary tokens when decoding. Tokenization and decoding output involving these tokens may therefore differ from before, so check any code or saved outputs that depend on the old behavior.

Exporters

Transformers exporters now support quantization directly during export across Dynamo, ONNX, OpenVINO, and ExecuTorch. The new unified quantization API integrates PT2E, ONNX Runtime, and NNCF quantizers, enabling static and dynamic quantization as well as weight-only compression (INT8/INT4), with configurable calibration workflows. This makes it easier to export optimized models for efficient inference across different backends.

For more information see the docs: https://huggingface.co/docs/transformers/exporters_quantization

Parallelization

This release improves distributed training and inference. FSDP × TP model and optimizer checkpoints can now be saved and loaded across different parallel configurations, and expert-parallel dispatch, tensor-parallel plans, static-cache generate(), and the caching allocator's byte count were fixed. Tied LM heads in composite models now get their own TP plan entry, enable_expert_parallel applies the EP plan alone again, TPU loading uses the `tor

Generation

This release fixes static-cache generate() bugs with host-side inputs and under tensor parallelism. It also speeds up Whisper generation on TPU by cutting device syncs and recompilation, and fixes Whisper batched timestamp decoding for tensor and NumPy inputs. Pipelines get a cleanup of generation-argument handling that removes spurious warnings.

...

Read more

Contributors

francip, shadeMe, and 44 other contributors
Loading

Release v5.19.0

Release v5.19.0

Choose a tag to compare

View all tags
@vasqu vasqu released this 06 Oct 16:39
v5.19.0
c4c4605

Release v5.19.0

New Model additions

EmbeddingGemma2

image

EmbeddingGemma 2 is a multimodal embedding model from Google built on the Gemma 4 architecture. It encodes text, images, audio, and video, individually or combined in one input, into a shared 768-dimensional vector space for cross-modal retrieval, semantic similarity, clustering, and classification. It uses Matryoshka Representation Learning, so embeddings can be truncated to 512, 256, or 128 dimensions. It also offers configurable visual and video token budgets, and unused vision or audio towers can be disabled at load time to save memory.

Links: Documentation

Breaking changes

All MoE models whose routers compute logits now return them when output_router_logits=True, following the Qwen3-MoE pattern (a router_logits recorder on the base model, MoeModelOutputWithPast from the backbone, and a MoE causal LM output from the head), so code that relied on the previous outputs or their absence should read the router logits from these output classes.

  • 🚨 Return router logits from every MoE model that computes them (#48920) by @qgallouedec

Owlv2ForObjectDetection.embed_image_query now selects the query box with the highest objectness score, as in the original OWLv2 notebook, instead of the OWL-ViT heuristic, so image-guided query embeddings and detections may differ from earlier releases.

The "paged|" prefix for SDPA and flash attention implementations is deprecated, so users should set the regular attention implementation (e.g. sdpa or flash_attention_2) for continuous batching instead of paged|sdpa or paged|flash_attention_2.

  • 🚨 Attention 🚨 Deprecate "paged|" prefix for SDPA and flash (#49112) by @remi-or

The regular flash and SDPA attention functions (flash_attention.py, sdpa_attention.py) now support continuous batching directly, and "paged|..." implementations for these are redirected to them, while eager still requires the "paged|eager" prefix.

  • 🚨 Attention 🚨 Make regular attention support CB (#49101) by @remi-or

In continuous batching, the cache update for the index-based and block-table paths is now fused into a single call, which slightly changes the cache update function's behavior and affects any custom code that calls the separate update paths.

  • 🚨 [CB] 🚨 Fuse update for index and block table path (#49088) by @remi-or

Continuous batching internals changed in preparation for removing "paged": `max

  • 🚨 [CB] 🚨 Little fixes before removing "paged" (#49069) by @remi-or

Parallelization

Expert parallelism gains a token-dispatch implementation, selected via the new ep_dispatch_experts plan rule and now the default for Qwen3 MoE and Mellum, which removes the requirement that EP size equal TP size. The Trainer was also adapted to work with expert parallelism, and the docs now note that PEFT adapters support tensor parallelism. A CI-related fix for pipeline-parallel chart2table inference was also included.

Cache

This release fixes quantized cache handling: generate no longer mutates the user's cache_config, and QuantizedLayer.reorder_cache is repaired. It also adds per-layer cache configuration, so DynamicCache and StaticCache initialize each layer from its own config (sliding window, attention chunk size, conv states, and attention head counts) to better support heterogeneous models. Separately, the deprecation cycle on mask and cache

Bugfixes and improvements

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @vasqu
  • @remi-or
    • [Fix] Remove old test file with two ancient tests (#49280)
    • [CB] Make CB more device agnostic and add support for XPU (#49156)
    • 🚨 Attention 🚨 Deprecate "paged|" prefix for SDPA and flash (#49112)
    • 🚨 Attention 🚨 Make regular attention support CB (#49101)
    • 🚨 [CB] 🚨 Fuse update for index and block table path (#49088)
    • [Refactor] Make some flash-attention utils more readable (#49071)
    • 🚨 [CB] 🚨 Little fixes before removing "paged" (#49069)

Contributors

tarekziade, dacorvo, and 21 other contributors
Loading

Release 5.18.0

Release 5.18.0

Choose a tag to compare

View all tags
@vasqu vasqu released this 30 Sep 16:46
v5.18.0
a906d3c

New Model additions

Nemotron 3 Diarization

image

Nemotron 3 Diarization is an open-weight streaming speaker diarization model designed to determine "who spoke when" in real-world audio. It supports both streaming and offline inference, handles up to eight speakers, and orders speaker outputs by each speaker's first arrival in the input audio.

The model uses the Arrival-Order Speaker Cache (AOSC) 1 and FIFO queue introduced for Streaming Sortformer 1, 2. A single checkpoint supports configurable latency profiles, from an 80 ms input buffer to a 30.4 s offline-style buffer, and configurable output frame resolution in multiples of 10 ms. With chunked inference, the maximum audio duration is not limited.

Links: Documentation

NemotronH Omni

NemotronH Omni is a multimodal reasoning model from NVIDIA that pairs the NemotronH hybrid
Mamba-Transformer language model with a RADIO vision encoder and an optional Parakeet-based sound encoder.
Image (and video) patches are projected through a RADIO tower and a pixel-shuffle MLP into the language model's
embedding space at the <image> / <video> context-token positions; audio clips are projected in the same way at
<audio> positions. The result is a single autoregressive model that reasons jointly over text, images, video and
sound.

Links: Documentation

HyperCLOVAX Vision V2

HyperCLOVAX Vision V2 is a multimodal vision-language model developed by NAVER. It combines the HyperClovaX language model backbone with a Qwen2.5-VL vision encoder. The model supports text, image, and video inputs and is capable of chain-of-thought reasoning via built-in thinking tokens (<think>...</think>).

Links: Documentation

GTE

GTE was proposed in mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval by Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li and Min Zhang.

GTE is a BERT-style bidirectional encoder that replaces absolute position embeddings with RoPE, uses a gated MLP, and applies layer normalization after each residual connection. The same architecture backs Alibaba's gte-*-v1.5, gte-multilingual-* and gte-en-mlm-* checkpoints as well as Snowflake's snowflake-arctic-embed-m-v2.0.

Links: Documentation

Breaking changes

Bugfixes and improvements

  • fix incorrect hub tokenizer class (#48641) by @itazap
  • [CI] Deduplicate Nvidia/AMD CI reply comments and add headers (#48655) by @ydshieh
  • Update dev (#48654) by @vasqu
  • [tests] Fix NougatModelIntegrationTest: pin artifact revision and update golden values (#48638) by @ydshieh
  • [fix] Fix GlmOcr integration tests: wrong token IDs and image token decode bug (#48650) by @ydshieh
  • [GLM 5.3 Flash] Preserve original names when saving text checkpoints (#48676) by @Dovis01
  • Register activation kernel layers on XPU (#47858) by @jiqing-feng
  • Mirror the registered pytree flatten when building automatic dynamic shapes (#48578) by @IlyasMoutawwakil
  • Fix AutoImageProcessor requiring torchvision when only Pillow is installed (#48616) by @blipbyte
  • Assert cached decode matches recomputing without a cache (#48289) by @IlyasMoutawwakil
  • [higgs_audio_v2] Fix: use config.num_codebooks in audio labels tensor (Simple Fix!) (#48560) by @b-re-w
  • [MoE] Fix eager EP (#48653) by @vasqu
  • [MusicgenMelody] Fix conditioning silently dropped at generation step 0 (#48679) by @ydshieh
  • Fix deepspeed ci (#48640) by @SunMarc
  • Synthetic test assets (#48589) by @tarekziade
  • Skip the expert-parallel sentinel masking when expert parallelism is off (#48201) by @qgallouedec
  • [Fix] yolos offload issue (#48688) by @molbap
  • [serge] Fix 2 integration tests for model minimax failing with output_mismatch (tensor values differ (2)) (#48515) by @sergereview[bot]
  • [serge] Fix 2 integration tests for model mistral failing with other (other (2)) (#48429) by @sergereview[bot]
  • Fix Qwen2.5-VL temporal RoPE for fractional video intervals (#48669) by @yeyeyeping
  • Fix slow integration tests on XPU (#48611) by @jiqing-feng
  • QA: Added a MemoryCleanupMixin class for tests (#48681) by @tarekziade
  • [serge] Fix 1 integration test for model flex_olmo failing with other (other (1)) (#48668) by @sergereview[bot]
  • Fix D-FINE / RT-DETR main loss being computed over the denoising queries (#48528) by @stefan-it
  • [CI] Replace hardcoded username allowlists with author_association check in workflow triggers (#48712) by @ydshieh
  • Add flash_attention_4 in attn_implementation AutoModel docstring (#48684) by @3manifold
  • Keep the expert-parallel sentinel slots out of the router gradient (#48689) by @qgallouedec
  • [DeepseekV3] OOM cascade root-cause investigation (generator ref leak in conversion_mapping) (#48720) by @ydshieh
  • Use huggingface_hub httpx export (#48685) by @Wauplin
  • unifying device_mesh init to enable PP + TP inference (#48155) by @3outeille
  • Enable FSDP2 + expert parallelism via a 2-D (fsdp, tp) device mesh (#48516) by @qgallouedec
  • Switch daily CI to torch 2.14 — update expected outputs (#48750) by @ydshieh
  • fix videomae load error (#48675) by @sywangyi
  • Fix silently random-initializing RTDetrModel/SEWDForCTC loads (wrong base_model_prefix) (#48744) by @
  • Fix Glm4vMoeIntegrationTest: offload_folder + MemoryCleanupMixin (#48776) by @ydshieh
  • [InternVL] Normalize num_patches before np.cumsum (#48469) by @lorenzozanee
  • Kernel api doc, part 2 (#46889) by @michaelbenayoun
  • Fix hidden state selection on Gemma4 assistant's first prefill step (#48704) by @glistening
  • Fix inkling embedding norm (#48786) by @Cyrilvallez
  • Fix TextToAudioPipeline crash for tokenizer-only models (#48505) by @jiqing-feng
  • Cast pixel values to the patch embedding dtype in DeepSeek-OCR-2 (#48632) by @jiqing-feng
  • add kernel mapping entry for RMSNormGated, KDA, Conv1D on XPU (#48702) by @kaixuanliu
  • Bump peft version requirement (#48716) by @shniubobo
  • [serge] Fix 12 integration tests for model edgetam failing with import_or_config (other (12)) (#48322) by @sergereview[bot]
  • Restore legacy tensor-parallel initialization for compatibility (#48797) by @3outeille
  • Fix kosmos flaky test (#48800) by @IlyasMoutawwakil
  • Add integration tests for MuseGlimmerAssistantModel (#48796) by @ydshieh
  • Fix AutoModel.from_pretrained not restoring modules_to_save weights (#48595) by @shniubobo
  • [docs] mrope and axial rope (#48717) by @stevhliu
  • Fix decoder only path for older bert variants (#48785) by @Cyrilvallez
  • [Fix] Clean-up ternaries in the DeepSeek family (#48447) by @remi-or
  • [utils] Add MUSA support for Flash Attention 2 (#48612) by @XiaomingFun233
  • [generate] Drop attention mask early without padding (#48814) by @Cyrilvallez
  • Pin utf-8 in test_can_init_all_missing_weights source read (Windows non-UTF-8 locale fix) (#48819) by @dltsum
  • Relax static-cache tolerance in test_generate_with_static_cache (1e-5 → 5e-5) (#48815) by @ydshieh
  • Detect nested rope_parameters without relying on layer_types (#48798) by @hmellor
  • Support for MoE in the GGUF integration (#48529) by @SunMarc
  • [docs] gguf (#46357) by @stevhliu
  • [apply_chat_template] pass sampling_rate to call (#48794) by @eustlb
  • [CI] Add link checker (#48160) by @stevhliu
  • Qwen3.8 GGUF (#48660) by @SunMarc
  • docs: fix broken #combining-with-fsdp2 anchor in expert_parallelism.md (#48854) by @ydshieh
  • [GPTNeoXJapanese] Fix RoPE ignoring partial_rotary_factor (#48652) by @blipbyte
  • QA: deactivate rule 41 (#48852) by @tarekziade
  • Fix reset on the dynamic cache layers (#48809) by @jiqing-feng
  • [generate] Make all methods and logits processors agnostic to lm_head output size (#48846) by @Cyrilvallez
  • Fix kosmos (#48853) by @Cyrilvallez
  • Add test for causal only variant of some encoder-decoder models (#48760) by @nandan2003
  • Fix ESMFold2 ligand iPLDDT weighting: mol_type non-polymer code is 3, not 4 (#48831) by @faustomilletari
  • Fix flaky RfDetr test_sav...
Read more

Contributors

studioego, salkan0, and 88 other contributors
Loading