Release v5.7.0

Release v5.7.0 huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release v5.7.0 vasqu released this 28 Apr 18:32 · 1378 commits to main since this release v5.7.0 6ffbb07 Release v5.7.0 New Model additions Laguna Laguna is Poolside's mixture-of-experts language model family that extends standard SwiGLU MoE transformers with two key innovations. It features per-layer head counts allowing different decoder layers to have different query-head counts while sharing the same KV cache shape, and implements a sigmoid MoE router with auxiliary-loss-free load balancing that uses element-wise sigmoid of gate logits plus learned per-expert bias for router scoring.

This ModelRelease is relevant to the technology intelligence record because it involves GitHub, DeepSeek, Mistral AI, AMD. The source article should remain the factual reference for follow-up coverage.

• huggingface / transformers Public Notifications You must be signed in to change notification settings Fork 34.7k Star 167k Release v5.7.0 vasqu released this 28 Apr 18:32 · 1378 commits to main since this release v5.7.0 6ffbb07 Release v5.7.0 New Model additions Laguna Laguna is Poolside's mixture-of-experts language model family that extends standard SwiGLU MoE transformers with two key innovations.
• It features per-layer head counts allowing different decoder layers to have different query-head counts while sharing the same KV cache shape, and implements a sigmoid MoE router with auxiliary-loss-free load balancing that uses element-wise sigmoid of gate logits plus learned per-expert bias for router scoring.
• Links: Documentation Laguna XS.2 implementation ( #45673 ) by @joerowell in #45673 DEIMv2 DEIMv2 (DETR with Improved Matching v2) is a real-time object detection model that extends DEIM with DINOv3 features and spans eight model sizes from X to Atto for diverse deployment scenarios.
• It uses a Spatial Tuning Adapter (STA) for larger variants to convert DINOv3's single-scale output into multi-scale features, while ultra-lightweight models employ pruned HGNetv2 backbones.
• The unified design achieves superior performance-cost trade-offs, with DEIMv2-X reaching 57.8 AP with only 50.3M parameters and DEIMv2-S being the first sub-10M model to exceed 50 AP on COCO.
• Links: Documentation | Paper model: Add DEIMv2 to Transformers ( #44339 ) by @harshaljanjani in #44339 Attention Several attention-related bugs were fixed across multiple models, including a cross-attention cache type error in T5Gemma2 for long inputs, incorrect cached forward behavior in Qwen3.5's gated-delta-net linear attention, and a crash in GraniteMoeHybrid when no Mamba layers are present.