v4.57.0: Qwen3-Next, Vault Gemma, Qwen3 VL, LongCat Flash, Flex OLMO, LFM2 VL, BLT, Qwen3 OMNI MoE, Parakeet, EdgeTAM, OLMO3

v4.57.0: Qwen3-Next, Vault Gemma, Qwen3 VL, LongCat Flash, Flex OLMO, LFM2 VL, BLT, Qwen3 OMNI MoE, Parakeet, EdgeTAM, OLMO3 huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k v4.57.0: Qwen3-Next, Vault Gemma, Qwen3 VL, LongCat Flash, Flex OLMO, LFM2 VL, BLT, Qwen3 OMNI MoE, Parakeet, EdgeTAM, OLMO3 LysandreJik released this 03 Oct 17:04 · 3629 commits to main since this release v4.57.0 8ac2b91 New model additions Qwen3 Next The Qwen3-Next series represents the Qwen team's next-generation foundation models, optimized for extreme context length and large-scale parameter efficiency. The series introduces a suite of architectural innovations designed to maximize performance while minimizing computational cost: Hybrid Attention : Replaces standard attention with the combination of Gated DeltaNet and Gated Attention , enabling efficient context modeling.

This Funding is relevant to the technology intelligence record because it involves GitHub, Google, NVIDIA, OpenAI. The source article should remain the factual reference for follow-up coverage.

• huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k v4.57.0: Qwen3-Next, Vault Gemma, Qwen3 VL, LongCat Flash, Flex OLMO, LFM2 VL, BLT, Qwen3 OMNI MoE, Parakeet, EdgeTAM, OLMO3 LysandreJik released this 03 Oct 17:04 · 3629 commits to main since this release v4.57.0 8ac2b91 New model additions Qwen3 Next The Qwen3-Next series represents the Qwen team's next-generation foundation models, optimized for extreme context length and large-scale parameter efficiency.
• The series introduces a suite of architectural innovations designed to maximize performance while minimizing computational cost: Hybrid Attention : Replaces standard attention with the combination of Gated DeltaNet and Gated Attention , enabling efficient context modeling.
• High-Sparsity MoE : Achieves an extreme low activation ratio as 1:50 in MoE layers — drastically reducing FLOPs per token while preserving model capacity.
• Multi-Token Prediction(MTP) : Boosts pretraining model performance, and accelerates inference.
• Other Optimizations : Includes techniques such as zero-centered and weight-decayed layernorm , Gated Attention , and other stabilizing enhancements for robust training.
• Built on this architecture, they trained and open-sourced Qwen3-Next-80B-A3B — 80B total parameters, only 3B active — achieving extreme sparsity and efficiency.