2026-11-03

We have released a Coder build of DeepSeek-V4.1-Flash-1017, compressed with GSQ and RCO, in which half of the model's routed experts have been removed. DeepSeek-V4.1-Flash-1017 is a sparse mixture-of-experts model: 384 routed experts per layer, of which 6 are active per token, 763.2B parameters, 551.9B of them in the transformer. What's inside One expert-pruned GGUF: 192 of 384 experts kept in every layer, 72.1 GB of transformer weights, against 138.2 GB for our unpruned IQ2_XXS build. 1 bit per parameter, counted against the original model: 72.12 GB divided by the original 551.9B transformer parameters gives 1.05 bits, which we round to 1. The removed experts count as zero bits. No individual weight is stored below 1 bit: the 280.1B parameters the Coder keeps average 2.06 bits. The shared Engram file (110.6 GB) and vision projector are the same files as in the unpruned release, so they need to be downloaded only once. GSQ (Gumbel-Softmax Quantization): post-training scalar quantization that jointly learns grid assignments and group scales, closing most of the gap between scalar and vector quantization at low bit-widths while remaining deployable in standard GGUF types. RCO (Riemannian Constrained Optimization): enforces exact budgets by gradient descent on the task loss, without per-constraint tuning. Here it selects which experts to keep, optimising the KL divergence to the unpruned model, under one exact budget per layer. Why one budget per layer GGUF stores a single expert count for the whole model, so every layer must keep exactly the same number of experts; uneven layers would need padding that gives back the memory pruning saved. RCO treats each layer's budget as a hard constraint and holds all of them exactly at every optimisation step, while the layers are still optimised jointly through one loss on the full model. A penalty formulation would only meet the budgets approximately and would need one tuned coefficient per layer. Results LiveCodeBench v6: 88.57 against 89.71 for the base model, retaining 98.7% SWE-bench Verified: 77.00 against 81.20, retaining 94.8% SWE-bench Verified runs long agentic trajectories over real repositories, and tolerates the capacity reduction less well than single-problem code generation, as expected when errors accumulate across turns. Real-world examples, each a single HTML file written by the Coder: an animated SVG of a pelican riding a bicycle, a three.js voxel pagoda garden, and a playable first-person voxel sandbox. For full post incl. videos see: https://x.com/mhelcig/status/2117257900863156518 Links Coder (expert-pruned): https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF Quantized models (unpruned): https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF GSQ: paper https://arxiv.org/abs/2604.18556 | code https://github.com/IST-DASLab/GSQ RCO: paper https://arxiv.org/abs/2605.00649 | code https://github.com/IST-DASLab/RCO The repository ships the complete per-tensor RCO allocation. The GGUF runs in llama.cpp b11870 or newer and in Strata. The Coder build is an experimental release and feedback is welcome, particularly on capabilities that were not represented in the calibration mixture. Requests for models to quantize or prune are also welcome. From the ISTA Deep Algorithms and Systems Lab. submitted by /u/Loginhe to r/LocalLLaMA [link] [comments]

There's a lot of talk here about not chasing tok/s by lobotomizing the model, and most of what I post is still speed numbers. So this is the other half: what the models I run at home actually get done on my own work, with Claude models added as fixed reference points. Everything local runs on my own engine (capy-flash) on 2x RTX 5070 Ti 16 GB, Ryzen 9900X, 96 GB DDR5. No expert skipping for flash next. The tasks I took 9 real regressions from the private history of that engine (C++/CUDA). For each one I checked out the code right before the fix and wrote the bug report the way I'd file it to myself: symptom, numbers, logs, no hint about where to look. Performance cliffs, numeric drift, a bug that only shows up on WSL2 with two GPUs. None of it has ever been public. The agent has to find the cause and write a patch. It runs sandboxed on a Mac with no network and no CUDA toolchain, so it can't build or run anything and has to reason from the code. 80 steps max, 3 runs per task per model. Claude models run in Claude Code, local models in OpenCode. I tried the local models in Claude Code through a bridge first and they did much worse there, so each side gets the harness it works best in. Scoring Every patch is scored twice: automatically, against a hidden key built from the real fix: right files and functions, patch precision, a few checks on the code; by Claude Opus 5 as a blind judge. It sees the bug report, the code, the real fix labelled as "one valid solution, not the only one" and the compiler output, and gives 1 / 0.75 / 0.5 / 0.25 / 0. A patch that doesn't compile is capped at 0.25 in both. The two scores correlate at r = 0.79 per patch and put the models in the same order. The judge is a bit stricter than the key, and it isn't kinder to Claude: judge minus auto score is negative for every Claude model and around zero for the Qwen ones. Turning a real bug into a task with a fair key is slow, and one model over 9 tasks x 3 runs takes 4 to 8 hours on my box. So 9 tasks is what I have, and the confidence intervals are wide, about ±0.2. The tiers hold. Two models less than ~0.1 apart are a coin flip. Results Judge score 0-1, tokens include reasoning. CIs and the auto score are in the image. Model Where Score Tokens/task Min/task Claude Opus 5.5 API 0.74 16k 3 Claude Sonnet 5.5 API 0.58 8k 1 Qwen3.8-Flash-Next NVFP4 local 0.50 53k 10 Qwen3.8-Flash-Next 3-bit local 0.48 53k 9 Claude Haiku 5.5 API 0.47 45k 4 Qwen3.8-27B 3-bit local 0.39 111k 18 Qwen3.8-27B NVFP4 local 0.34 81k 13 Claude Sonnet 5 API 0.28 43k 8 Claude Sonnet 4.5 API 0.19 15k 6 Claude Haiku 4.5 API 0.13 23k 4 Gemma 4 26B-A4B (thinking) local 0.07 33k 5 What 3 bits did This is the part that matters for the tok/s argument. Same models, NVFP4 vs 3-bit, paired on the same tasks. Flash-Next (3 bits on the experts only): score -0.02, tokens x1.03. As far as 9 tasks can tell, it's free. 27B: score +0.05, inside the noise. But it generates 40 % more tokens per task (x1.40, CI 1.25-1.59). The 3-bit 27B decodes faster on my engine, 153 vs 132 tok/s, and still needed 18 minutes per task instead of 13. Part of that gap was a slow 3-bit prefill path I've fixed since; a rerun on the newer build still came out at x1.30 tokens. So the damage from quantization didn't show up as a lower score. It showed up as the model thinking longer to get to the same place. Looking only at tok/s, the 3-bit 27B is the better deal. Per bug fixed, it isn't. Non-compiling patches don't tell a clear story: Flash-Next 5/27 at 3-bit vs 1/27, the 27B 2/27 vs 4/27. The rest Flash-Next at home sits with Haiku 5.5 and above the older Sonnets. Sonnet 5.5 is a step up, Opus 5.5 is ahead of everything. Tokens per task say as much as the score. Sonnet 5.5 gets further with 8k tokens than Flash-Next with 53k, which is 10 minutes per bug on my box against a minute and a half. The dense 27B is behind the MoE (Flash-Next +0.16 paired, P = 0.92) and burns more tokens getting there. Gemma 4 26B didn't work as an agent here. With thinking off it never edited a file in 9 tasks. With thinking on it edits, mostly wrong. Limits 9 tasks from one codebase in one domain (GPU inference), bug reports written in Italian. This is how models do on my work, not a general ranking. Different harnesses for Claude and for the local models. The judge is a Claude model. The automatic score is there to check it and they agree, but it's still a limit. No builds or tests, so this measures diagnosis and patching from reading code, not the full edit-build-test loop. I'm not publishing the tasks, they'd stop being useful. If you've checked your own quants on real work instead of perplexity or KL, I'd like to know whether you see the same pattern: same score, longer thinking. submitted by /u/FeydRowan to r/LocalLLaMA [link] [comments]

Measured, refusals 85%→0% even in thinking, 3 sizes 12 GB GPU VRAM, 64GB RAM + SSD via Strata Engine We took ISTA-DASLab's GSQ-RCO quants of DeepSeek-V4.1-Flash-1017 (the ones people run in Strata with a 12 GB GPU and the Engram table on SSD) and removed the refusals. We only touched 82 dense tensors per file: attention output, shared expert down proj and the Engram value proj. The 384 routed experts are byte for byte the same as ISTA's, so their GSQ rounding is kept. Headers and file sizes match too, so you just swap the shards and keep using ISTA's Engram file. Refusals are on JailbreakBench, stock rung -> ours, same Strata setup: Rung Size (+110 GB Engram) Refusals Refusals in thinking (of 30) MMLU-500 KL vs stock IQ1_M 121 GB 88.9% -> 0% 27 -> 1 80.6% 0.021 IQ2_XXS 138 GB 85.2% -> 0% 27 -> 0 82.8% 0.023 IQ2_XS 155 GB 86.4% -> 0% 27 -> 2 81.6% 0.027 We also ran GPQA-Diamond and AIME25 on IQ2_XXS against stock to make sure it didn't get dumber. GPQA came out 83.3% vs 83.3%, AIME 80.0% vs 76.7% (only 30 problems, so call it the same). Things to know: IQ2_XS still refuses inside its thinking on 2 of 30 prompts. The other two are at 0 and 1, so I'd grab IQ2_XXS. We tested at 8K context on a rented GPU. Haven't checked long context or speed on a 12 GB card ourselves, so if you try it please post your tok/s and RAM. Strata only for now. If you're on llama.cpp, our older release with the same edit works there: https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF Model: https://huggingface.co/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GSQ-RCO-GGUF All the measurements: https://huggingface.co/datasets/AtomicChat/DeepSeek-V4.1-Flash-1017-Uncensored-GGUF-metrics Strata Engine: https://github.com/niko1221/strata Model page: https://atomic.chat/models/deepseek-v4-flash-0731 Thanks to DeepSeek for the base model (MIT), ISTA-DASLab for GSQ-RCO and the Engram file, and the Strata dev for the engine. I'm on the Atomic.Chat Local Ai team that made this. submitted by /u/gladkos to r/LocalLLaMA [link] [comments]

We ran DeepSWE, Terminal Bench, Automation Bench, HLE, SciCode at 512K context on our DeepSeek UD version 3.1 GGUFs and show we're >10-20% better than GSQ-RCO We ran all other benchmarks (14 in total) to 128K. AIME2025 64x, DeepSWE/TB 3x. We addressed some issues people mentioned with these new benchmarks: 32K context was too short - we did 512K context for agentic + 128K for general tasks Saturated benchmarks - new unsaturated agentic style ones including Terminal Bench, DeepSWE and Automation Bench Benchmarks are noisy - we did 35.1B tokens of benchmarking vs 1.3B before so 27x more data . See this section where we did 64x AIME 2025 runs, 3x DeepSWE, 3x TB runs and more. Our graders are wrong - we used Math Verify , improved graders, used gpt-6-luna xhigh for HLE, and all official graders for Terminal Bench (Harbor), DeepSWE and the rest. See graders and infra fixes For DeepSWE v1.1, we used mini-swe-agent and chose 500 max steps (334 steps was the max runs we saw, so none went over). Terminal Bench 2.1 used mini-swe-agent as well, and we set max steps to 150 with no max limit. We had to exclude 7/89 tasks since they couldn't pass on our infra - see https://unsloth.ai/docs/models/deepseek-v4.1-flash#excluded-terminal-bench-tasks-due-to-issues We also re-checked ISTA's numbers as we have done many more runs (27x more data). GSQ-RCO IQ1_M claims 98.7% recovery, but we measured 97.24 [96.23, 98.21] with a p-value of 0.0013. GSQ-RCO IQ2_XS retaining 100% claimed recovery is possible with a p-value of 0.21. We measured 98.38% [97.52, 99.22]. So this is possible. GSQ-RCO IQ2_XXS claims 99.80% recovery but we measured 98.38 [97.52, 99.22] with a p-value of 0.0001. So this is nearly impossible. Paired difference to the official base (our runs, same questions, sign-flip permutation test, two-sided): AIME25 GPQA-D LCB v6 GSQ-RCO IQ1_M -1.61 ( p=0.02 ) -2.89 ( p=0.004 ) -3.24 ( p=0.00002 ) GSQ-RCO IQ2_XXS -0.47 (p=0.25) -2.07 ( p=0.02 ) -2.00 ( p=0.01 ) GSQ-RCO IQ2_XS -0.42 (p=0.48) -0.51 (p=0.42) -0.38 (p=0.52) The claimed recovery tested directly against our measurement (the card's definition: sum of the 3 scores / the base's sum, vs official FP8/FP4): p (no loss) = paired sign-flip permutation test of recovery = 100%; p (recovery >= claim) = one-sided paired bootstrap over questions. Claimed recovery Measured recovery [95% CI] p (no loss) p (recovery >= claim) GSQ-RCO IQ1_M 98.70 97.24 [96.23, 98.21] 0.00002 0.0013 GSQ-RCO IQ2_XXS 100.00 98.38 [97.52, 99.22] 0.0003 0.0001 GSQ-RCO IQ2_XS 99.80 99.54 [98.88, 100.18] 0.16 0.21 If we can get more details on how the benchmarks were run from GSQ-RCO, we are more than happy to run more. We provided all our setups and methods in our blog at https://unsloth.ai/docs/models/deepseek-v4.1-flash#dynamic-v3.1-update-november-2nd-2026 All details and data and methods at https://unsloth.ai/docs/models/deepseek-v4.1-flash#dynamic-v3.1-update-november-2nd-2026 . Our new Unsloth Dynamic 3.1 GGUFs for DeepSeek are at https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF submitted by /u/danielhanchen to r/LocalLLaMA [link] [comments]

In 2024 i purchased an RTX 8000 48gb for $2400. As the market has changed I’ve kept an eye on prices. Two years later, they are available around $2100. Its a little slower than 3090 at inference (672gb/s vs 936gb/s) but it has double the memory so it can run larger models / contexts / higher quants. Rtx 8000 also wins on power, 265w vs 700w (2x 3090 to match 48gb) In 2024 two 3090s would cost around $1600-1800 now a single 3090 is in the $1500-$1600 range, making rtx 8000 cheaper per gb. Prices for strix halo and dgx spark have also gone way up. Rtx pro 6000 has double the memory but 8x the cost. In the last two years, I haven’t had any issues with drivers, compatibility software or anything. I can run all “local” tier models entirely in vram, usually at 256k tokens and sometimes at q6 (Qwen 3.8 27b, llama 3 70b, QWQ,). Larger moe models run great too with layers offloaded to system ram like gpt oss 120b and now with strata, QFN runs q4_xs @ 390k context at 850t/s prompt processing and 60-70t/s token generation. (64gb ddr4 system memory, i9 9900k cpu) $2100 for a gpu is not cheap but might be the best “do everything” option right now for home-lab setups. Cheaper than dual 3090, Strix halo, dgx spart, m5 max too. Fascinating to see prices decline as the market as a whole has more than doubled. submitted by /u/triynizzles1 to r/LocalLLaMA [link] [comments]

v0.32.0 Highlights This release features 814 commits from 363 contributors (121 new)! Structured decisions and generic parsers : the new POST /v1/systemone endpoint answers choice , noul and score questions over a structured state ( #59299 ), is served by default ( #60677 , #60696 ), and also runs on Cloudflare/clef-omni ( #61064 ), LiquidAI/d1-3B ( #60676 ) and DiffusionGemma ( #60466 ). The new hf tool and reasoning parser ( --tool-call-parser hf , --reasoning-parser hf ) reads the checkpoint's response_template , so models that ship it need no model-specific parser ( #58604 , #58602 ), in both the Python and Rust frontends ( #59005 ). Low-precision KV cache everywhere : NVFP4 KV cache now runs on SM8x and SM12x ( #46963 ), SM90 ( #60620 ) and with XQA decode on SM12x ( #60452 ), and on MiniMax-M3 sparse attention on SM100 ( #59300 ); GLM-5.x defaults to an FP8 KV cache on SM100, halving KV memory per rank with 2.3-5.5% more throughput at a 7-11% TTFT cost in the PR's benchmark ( #60140 , #60281 ); fp8_ds_mla on SM90 gives GLM-5.3-Flash up to 31% more output throughput ( #59246 ); ROCm gains the 4-bit ultraquant_4bit KV cache with about 2x the density of FP8 ( #57057 ). Model Runner V2 : a GPU n-gram speculator with variable-length drafts ( #40704 ), Elastic EP ( #53934 ), pipeline parallelism with prefill context parallelism for MLA models ( #59139 ), a stock torch.compile mode ( #59079 ), and MRV2 is now the default on ROCm with the model-specific MRV1 exceptions removed ( #56360 ). Faster startup and serving : vllm download-kernels installs FlashInfer's precompiled kernels ( #58765 ); vllm preload --preload-autotune keeps the FlashInfer autotune table warm across restarts ( #60085 ); a shared-memory tensor arena broadcasts large multimodal tensors to workers with one copy ( #51207 ); tokenizers and processors load from the local snapshot ( #60825 ); and with --api-server-count above 1 each connection goes to the least-loaded API worker, for 43-85% more req/s on short requests (Qwen3-0.6B, 32 input / 1 output tokens) in the PR's benchmark ( #60723 ). Model support : new EmbeddingGemma 2 omni-modal embeddings ( #60254 ), MiniCPM-V 4.7 ( #58674 ) and Nemotron 3.5 ASR ( #59827 ); video input in the Transformers modeling backend ( #57441 ); GLM-5.3 sparse MLA index conversions reused across layers (3.5-3.9x on that kernel, #59464 ) and fused multi-step draft decode (13.3% at concurrency 1, #57443 ); HiSparse per-request residency caching for up to 58% more throughput on GLM-5.3-NVFP4 on B200 with long, fully GPU-resident contexts at high concurrency ( #60083 ). Security : fixes for GHSA-4hhp-h66f-j5j7 (chat template resource exhaustion, #50300 ), GHSA-q97c-p368-75c4 (tokenizer max_token_id , #59491 ) and GHSA-frvg-qw88-hww9 (P2P ZMQ socket exhaustion, #51504 ); CORS no longer allows every origin by default ( #61009 ); untrusted images are limited to raster formats ( #60022 ); shared-memory multimodal cache handles are signed ( #59357 ); and JSON schemas nested deeper than 64 levels are rejected ( #60036 ). Breaking changes : Python 3.10 dropped ( #60402 ); default CORS origins restricted ( #61009 ); transformers>=5.16.1,<5.20.0 and tokenizers>=0.23.2 ( #57387 , #59796 ); torch.compile sequence parallelism and AsyncTP passes removed ( #60517 ); cascade attention removed ( #61126 ); mamba_cache_mode="all" removed ( #58997 ); max_tokens no longer reserved from the context window ( #57035 ); Responses API reasoning stream events renamed ( #59652 ). Release Artifacts Python Wheels Platform Install PyPI (CUDA 13.0) pip install vllm PyPI (CUDA 13.0, uv) uv pip install vllm --torch-backend=auto ROCm pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.32.0/rocm723 XPU uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.32.0/xpu --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match Docker Images Platform Docker Image CUDA 13.0 (Default) docker pull vllm/vllm-openai:v0.32.0 CUDA 12.9 docker pull vllm/vllm-openai:v0.32.0-cu129 CUDA 13.0 + Ubuntu 24.04 docker pull vllm/vllm-openai:v0.32.0-ubuntu2404 CUDA 12.9 + Ubuntu 24.04 docker pull vllm/vllm-openai:v0.32.0-cu129-ubuntu2404 ROCm docker pull vllm/vllm-openai-rocm:v0.32.0 CPU docker pull vllm/vllm-openai-cpu:v0.32.0 XPU docker pull vllm/vllm-openai-xpu:v0.32.0 Other Artifacts Pre-built release artifacts are available in the Assets section at the bottom of this page, including: Source distribution tarball CUDA 12.9 Python wheels for x86_64 and arm64 CUDA 13.0 Python wheels for x86_64 and arm64 CPU Python wheels for x86_64, arm64, and macOS XPU Python wheel for x86_64 Model Support New models : EmbeddingGemma 2 ( google/embeddinggemma-2 ), an omni-modal text/image/audio/video embedding model, with unused modality encoders skipped through --limit-mm-per-prompt ( #60254 , #60289 ); MiniCPM-V 4.7 ( #58674 ); Nemotron 3.5 ASR ( nvidia/nemotron-3.5-asr-streaming-0.6b ) complete-audio transcription; native realtime streaming is not yet supported ( #59827 ); Cloudflare/clef-omni ( #61064 ) and LiquidAI/d1-3B ( #60676 ) on /v1/systemone ; PLaMo 3 reasoning and tool parsers ( #56575 ); and the zerank-2 reranker score template ( #60415 ). Transformers modeling backend : video inputs ( #57441 ), and Glm, Arcee, CWM, Mellum ( #59679 ), GPT-NeoX, Phi, Seed-OSS and Jais2 ( #59701 ) now served through it. New model capabilities : LoRA for BERT, RoBERTa and XLM-RoBERTa embedding models ( #61033 ); Granite as an EAGLE3/DFlash target ( #60855 ); Qwen3-ASR EAGLE3 ( #52824 ); Sarvam MLA EAGLE3/DSpark with PP ( #55902 ); GLM-5.3-Flash with the DFlash2 drafter ( #56983 ); and MiniMax-M3 EAGLE3 at PP > 1 on MRV2 ( #57197 ) with MTP sharing target embeddings under PP ( #58648 ). GLM-5.3 and GLM-5.3-Flash : sparse MLA index conversion reused across layers ( #59464 ), fused multi-step draft decode ( #57443 ), FP8 KV cache default on SM100 ( #60140 , #60281 ), fp8_ds_mla on SM90 ( #59246 ), qlnorm skipped for MHA (4.4-7.7% TTFT, #58845 ), long-context indexer prefill sharded across TP (up to 25% lower TTFT at 1M context, #54951 ), DCP for the kpool indexer ( #59211 , #60032 ), KDA prefill checkpoints with FlashKDA ( #56960 ), FP8 KV on SM90 sparse MLA ( #55222 ), --mamba-ssm-cache-dtype honored for KDA state ( #59147 ), a generic packed KV layout ( #57169 ), MTP index sharing restored ( #61079 ), and fixes for vision-tower crashes ( #59126 ), non-square image encoder budgets ( #59565 ), DP startup ( #56974 ), video timestamps ( #55647 ), DFlash2/EAGLE3 aux hidden states ( #60763 ) and null-block writes ( #59528 ). DeepSeek-V4 and V4.1 : decoder-side SWA bounded replay ( #58132 ) with replay layers in CUDA graphs and trimmed in PIECEWISE graphs ( compilation_config.decoder_replay_trim_threshold , #59532 ) plus last KV-source layer trimming (+3.6%, #59894 ), packed FP8 Engram exchange across DP ( #59923 ), faster CPU-offloaded Engram lookups ( #59327 ), fused DeepSeek V4 + MTP drafting ( #58463 ), encoder CUDA graph packing for V4.1 and Kimi-K3 (11% lower mean TTFT, #58925 ), EPLB for DSpark drafts ( #57549 ), and fixes for NVFP4 checkpoints with 0% draft acceptance ( #59655 ), MegaMoE shared-expert finalize ordering ( #59927 ), worst-case vision dummy inputs ( #57898 , #59271 ), compressor ring null-block writes ( #58560 ), Triton 3.9 indexer quantization ( #60716 ) and decoder-replay CUDA graphs with FlashInfer EP MoE backends ( #61041 ). Qwen : Qwen3.8-Flash-Next (Qwen4Exp) merged QSA projections in one GEMM (1.45-1.68x, #59533 ), fused QSA pre-indexer ( #57097 ), sequence parallelism through --enable-hc-sp ( #56322 ), SM100 low-latency decode GEMM plans (up to 2.7x vs cuBLAS on small-M, #59214 ), --kv-cache-dtype-skip-layers honored ( #60023 ), and PLE loading for INC ( #59990 ), Quark ( #59443 ) and pre-SM89 GPUs ( #60142 ); Qwen3-VL/Qwen3.5/Qwen3.8 M-RoPE cache 4x smaller and Qwen3.5 YaRN matching Transformers ( #59853 ); Qwen3-Omni M-RoPE fixes ( #58890 , #59842 ); Qwen3-ASR on the upstream processor ( #57146 ); Qwen3-VL unknown video fps ( #60501 ) and per-video options with the multimodal cache ( #55203 ). Kimi-K3 and MiniMax-M3 : Kimi-K3 FlashInfer fused speculative KDA on SM100 ( #54255 ), fused vision processor (10-20x preprocessing at 1080p-8K, #59278 ), fused K/V pack on the DCP prefill path ( #56881 ), and DSpark KV cache back to 4 groups ( #59733 ); MiniMax-M3 MSA prefill with an E4M3 query (26-37% faster with FP8 KV, #60684 ), FP8 indexer cache on non-SM100 GPUs ( #59081 ), Triton indexer tuning ( #56151 , #59481 ), and indexer score fixes ( #60194 , #54597 ). Multimodal : device-side input normalization for Qwen3-VL/Qwen3.5/Qwen4Exp ( #51289 ), GLM-V and GLM-OCR ( #55389 ), Llama Nemotron VL ( #57928 ) and Kimi K2.5/K3 ( #59278 ), kept fused under compile_mm_encoder ( #59195 ); encoder inputs sorted by modality (6.5-8.2% more req/s, #60196 ); Mistral 3 preprocessing 8% faster ( #57531 ); Pixtral compile_mm_encoder ( #57168 ); and fixes for Llama 4 / Mistral Large 3 EAGLE drafters (regression since v0.28.0, #57633 ), Phi-4 MM audio OOM ( #60614 ), DeepSeek-OCR zero-sum pixels ( #59417 ), MiMo-V2.5/V2.6 with fastsafetensors ( #58142 ), MiMo-V2.6 FP8 KV cache ( #58128 ), Voxtral realtime ( #45022 ), Nemotron Super vision LayerNorm ( #61401 ), K-EXAONE config and MTP RoPE ( #60187 ), Nemotron Parse tied lm_head ( #60272 ), LongCat-Flash MTP loading ( #60080 ) and Ling-3.0-flash-VL ( #59026 ). Engine Core Model Runner V2 : GPU n-gram speculator ( #40704 ), Elastic EP with drain-then-commit scaling ( #53934 ), PP with PCP for MLA models ( #59139 ), STOCK_TORCH_COMPILE mode ( #59079 ), dynamic K in autoregressive speculators ( #57053 ), MTP sampling-mask replay ( #59359 ) and per-module LM heads ( #58921 ), non-blocking H2D copies on WSL2 ( #60215 ), and fixes for int64 index offsets at long context ( #60736 ), warmup ( #61013 ), GDN prefill checkpoints ( #59536 ) and MRV2+PP+MTP align state ( #60210 ). Speculative decoding : fused multi-step draft decode on FlashInfer trtllm-gen ( #58371 ), FlashInfer ReplaySSM for Mamba2 MTP ( #52928 ), in-graph metadata for DFlash/DSpark drafts ( #60692 ), and correctness fixes for ngram on hybrid recurrent models ( #56531 ), Mamba prefill checkpoints with MTP and prefix caching ( #59759 , #60797 ), stop-string trimming ( #47616 ), draft model config ( #61240 ), adaptive block verification ( #60945 ), DSpark bonus slots ( #59105 ) and draft models on the Transformers backend ( #59293 ). Hybrid models and prefix caching : Mamba2 internal prefill checkpoints for NemotronH, Granite-4.0-H, Falcon-H1 and Zamba2 ( #57329 ), stale block hashes dropped when streaming sessions truncate ( #59103 ), and fixes for quantized Mamba-2 with TP ( #58083 ), spec decode page sizes ( #59975 ), prefix-cache bounds ( #55601 , #60533 ) and sm90 Mamba2 NaNs ( #61718 ). Sleep and RL : sleep releases workspace scratch (up to 4.5 GiB/GPU, #59156 ) and KV-init state (about 4.6 GiB/GPU, #59158 ), opt-in sleep_mode_offload_cudagraph ( #59160 , #59523 ); token-level weight-version spans on outputs ( #59582 , #61070 ); weight-transfer communicators released without stopping inference ( #61060 ); --sleep-preserve-parameter-names 'draft.*' ( #61046 ); modelexpress weight-transfer backend ( #58399 ); POST /weight_checker ( #51350 ); weight-transfer metrics ( #55781 ); --enable-nccl-comm-suspend releasing the MNNVL workspace ( #59360 ); and sleep(level=2) keeping FP8 KV scales ( #57163 ). HiSparse : per-request residency cache (up to 58% at high concurrency with GPU-resident contexts, #60083 ), lower scheduler overhead ( #57930 ), GPU residency owned by HiSparse ( #60647 ), less staging memory ( #60429 ), host cache gauges ( #58949 , #59485 ), and fixes for decode token limits ( #60682 ) and TP hangs ( #60941 ). Startup and memory : shared-memory tensor arena ( --enable-shm-tensor-arena , on by default, #51207 ), Fast Start with PP ( #55477 ), daemon-held weights counted against gpu_memory_utilization ( #57298 ), autotune preload ( #60085 ), local tokenizer snapshots ( #60825 ), sparse_mla_prefill_workspace_factor saving several GiB at 1M context ( #61023 ), cached memory reclaimed during weight loading ( #57516 ), fewer Triton sampler warmup compiles ( #58605 ), sharded state loader with PP ( #52110 ), and reload_weights with Run:ai streamer ( #58124 ). Kernels and runtime : MoE weighted-sum tuning (1.17-2.37x, #59731 ), flash-maxsim kernels for ColBERT/ColPali scoring ( #40337 ), fallback detokenization with 69% less CPU at 32K tokens ( #60497 ), faster KV block free queue ( #60269 ), Triton kernels off tl.make_block_ptr ( #60502 ), fusion passes restored under PyTorch 2.15 ( #57299 ) and static FP8 quant fusion on recent torch ( #61108 ). Batch invariance : custom all-reduce under VLLM_BATCH_INVARIANT=1 ( #58623 ), deterministic split-K LoRA shrink ( #59377 ), every attention backend checked ( #60271 ), SM120 TP configs ( #58495 ), and ROCm support ( #52231 ). Observability : vllm:prompt_tokens_cached_by_source_total ( #56318 ), vllm:num_requests_kv_fetch_by_stage ( #58874 ), vllm:admission_rejections_total ( #55812 ), vllm:watermark_requests ( #58715 ), --custom-histogram-buckets ( #48867 ), a JSON log formatter via LoggingConfig.formatter ( #58739 ), and per-session profiling controls ( #57875 ). Large Scale Serving MoE communication : FlashInfer CuteDSL MegaMoE for more models via --moe-backend flashinfer_cutedsl ( #54049 ), FP8 combine for the FlashInfer one-sided all2all (+4% at DEP8, #57995 ), NCCL symmetric-memory MoE reduce-scatter (9.6-16% lower latency, #49194 ), DeepEPv2 hybrid NVLink+RDMA mode on by default ( #57991 ), Kimi-K3 shared experts with DeepEP v2 ( #56875 ), and opt-in EPLB migration batching ( #52641 ). Context parallelism : decode requests sharded across PCP ranks ( #52162 ) and TokenSpeed MLA with block-interleaved DCP for heterogeneous NIXL disaggregation ( #59462 ). KV connectors : NIXL host-buffer copies coalesced ( #54483 ), mixed block sizes ( #58860 ), peer metadata recovery ( #55471 ) and Qwen3.8-Flash-Next MTP acceptance in P/D ( #59997 ); Mooncake coalesced hybrid/MLA RDMA regions ( #57952 ) and bootstrap retries ( #55923 ); MoRIIO Kimi-K3 DSpark READ ( #57700 ); DeepSeek-V4.1 P/D skipping bounded replay for loaded KV ( #59197 ); and KVCR pool-and-index API ( #59899 ). Data parallelism : streaming requests pinned to their engine ( #59017 ), per-engine RNG streams ( #59788 ), and placement groups without ray[default] ( #57648 ). Hardware & Performance NVIDIA : native 64-head FlashMLA sparse prefill on SM100 (1.16-1.18x TTFT, #60029 ), block-FP8 DeepGEMM experts skipping padding ( #59128 ), SM120 split-K attention ( #58482 ), DGX Spark Qwen4Exp GEMM plans ( #59753 , #59632 , #60027 ), OAI Triton MXFP4 MoE on SM12x ( #58877 ), and FlashInfer autotune cache per rank ( #57635 ). AMD ROCm : DeepSeek-V4 AITER MegaMoEV2 via --moe-backend aiter_mega_moe ( #59685 ) with opt-in shared-expert fusion (3-8%, #61004 ), opt-in mono decode for DeepSeek-V4.1-Flash on MI355X ( VLLM_ROCM_MONO_DECODE=1 , #60397 ), opt-in Gluon sparse MLA (up to +16% on GLM-5.2, #53492 ), paged MXFP4 indexer ( #58671 ), opt-in a4w4 MoE for DSv4 ( #58819 ), ultraquant_4bit KV cache ( #57057 ), aiter_flydsl GDN prefill ( #57560 ), Kimi-K3 prefill checkpoints (about 20% better TTFT, #58344 ), fused MLA decode ( #57640 ) and per-rank latent MoE shards ( #59591 ), GLM-5.3-Flash split-K sparse MLA decode (5-15% lower ITL, #58584 ), native merge_attn_states (2.4x, #60159 ), ROCM_SEGMENTED_ATTN for RDNA ( #59132 ), gfx11 W4A16 tuning (up to 23% TTFT, #56301 ), MRV2 default with model-specific MRV1 exceptions removed ( #56360 ), and fixes for sleep-mode segfaults on ROCm 7.2+ ( #60367 ) and MLA weight reloads ( #60389 ). Intel XPU : MLA in FLASH_ATTN_MLA ( #58297 ), Arc Pro B70 FP8 GEMM (2.0-2.73x, #56063 ) and fused MoE configs ( #53065 ), fused SYCL LayerNorm ( #57172 ), and fixes for Gemma-2 softcap ( #58936 ) and DP+EP MoE ( #57787 ). CPU : vectorized sampler (up to 58x on s390x, #53913 ), aarch64 causal conv1d (about 9%, #54093 ), POWER10 AWQ/GPTQ ( #59149 ), and PowerPC dtype=auto choosing bfloat16 ( #58528 ). Other : tpu-inference v0.31.0 ( #60190 ) and out-of-tree platforms registering multiple attention backends ( #41091 ). Quantization New capabilities : online re-quantization of linear layers ( --quantization-config.linear fp8_per_channel , #55684 ), per-token NVFP4 CuTe-DSL MoE ( #50030 ) with ReLU2 ( #56740 ), MXFP4 MoE oracle with CUTLASS W4A4 and BF16 fallback ( #60079 ), compressed-tensors MXFP4 MoE with FP8 activations ( #59815 ), ModelOpt IQ2_XS/IQ2_XXS/Q8_0 on SM120/SM121 ( #59604 ), int4_per_token_head KV with non-power-of-two heads ( #56198 ), and Humming preferred over Marlin on SM90 ( #56997 ). Fixes : silent CUTLASS NVFP4/MXFP4 MoE corruption under EP ( #59042 ), GPTQ with non-default safetensors names ( #60411 ), moe_wna16 8-bit asymmetric zero points ( #58950 ), and ModelOpt mixed-precision NVFP4 detection ( #56050 ). API & Frontend New endpoints and options : /v1/systemone ( #59299 , #60677 , #60696 ), hf parser ( #58604 ), vllm download-kernels ( #58765 ), output_mode: "text" and stop_reason in /inference/v1/generate ( #58588 , #60251 ), mm_payload for render requests ( #61427 , #60728 ), Responses min_p and stop_token_ids ( #48084 , #56592 ), LLM.chat(priority=...) ( #57243 ), and /generative_scoring cache_salt ( #60841 ). Behavior : max_tokens is an upper bound instead of a reservation, so long prompts are no longer rejected ( #57035 ); least-loaded API worker selection ( #60723 ); tool parser compatibility checked at startup ( #59749 ). Tool calling and structured output : parallel_tool_calls=false enforced ( #50502 , #59298 ), tool_choice="required" streaming ( #60354 ) and native XML parsers ( #47512 ), Step-3.5 and minimax_m3 on the parser engine ( #59321 , #59743 ), xgrammar propertyNames ( #57594 ), and LoRA PEFT rank_pattern / alpha_pattern applied with unsupported features rejected ( #59801 , #60225 ). Rust frontend : --grpc-port ( #59659 ), --stream-interval ( #60589 ), logprob_token_ids ( #59549 ), Control.Shutdown with vllm-proto 0.5.0 ( #59316 ), qwen3_json parser ( #60136 ), and Kimi K3 grammar fixes ( #59190 , #59395 ). Security GHSA-4hhp-h66f-j5j7 : chat templates are validated statically and rendered under a timeout ( VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT ), user-supplied chat_template in Responses chat_template_kwargs is rejected, and the Rust frontend caps template execution ( #50300 ). GHSA-q97c-p368-75c4 : tokenizer max_token_id off-by-one fixed ( #59491 ). GHSA-frvg-qw88-hww9 : P2P ZMQ peer sessions are bounded by VLLM_P2P_MAX_PEERS (default 128), VLLM_P2P_HANDSHAKE_TIMEOUT_S (default 30) and VLLM_P2P_IDLE_TIMEOUT_S (default 300), so unique peer addresses can no longer exhaust ZMQ sockets ( #51504 ). Default CORS settings no longer allow every origin in the Python and Rust frontends; set --allowed-origins to permit browser origins ( #61009 ). Untrusted images are opened with a raster-format allowlist ( ALLOWED_IMAGE_FORMATS ), denying EPS/PostScript and WMF ( #60022 ); shared-memory multimodal cache handles are HMAC-signed and forged handles rejected ( #59357 ). Structured-output JSON schemas nested more than 64 levels return 400 ( #60036 ); chunked /v1/embeddings requests keep cache_salt ( #47696 ); Phi-4 MM audio no longer holds multi-GiB attention matrices ( #60614 ); DeepSeek-OCR zero-sum pixel tensors no longer kill EngineCore ( #59417 ). Security dependency bumps: aiohttp>=3.14.3 , pillow>=12.3.0 , cryptography, urllib3, pyjwt and others ( #59249 , #59315 , #59427 ). Dependencies transformers>=5.16.1,<5.20.0 (minimum raised from 5.10.4, #57387 , #59762 ) and tokenizers>=0.23.2 ( #59796 ). NIXL 1.5.0 ( #56907 ), CuTe DSL 4.8.0 block-scale APIs ( #59480 ), humming-kernels==0.1.17 with initial Rubin support ( #61556 , #61858 ), InstantTensor 0.2.2 ( #61167 ), and flashinfer-jit-cache excluded from wheel install_requires ( #60052 ). ROCm: AITER 0.1.24.post1 ( #59794 ) and MoRI v1.2.3.post1 ( #56073 ). XPU: auto-round-lib 0.16.0 ( #60976 ). TPU: tpu-inference v0.31.0 ( #60190 ). Build Python 3.11-3.14 supported ( #60402 ); macOS wheels built with the stable ABI ( #60312 ); CUDA source builds require GCC 13 and fail early otherwise ( #60259 ); Docker images compile Python bytecode at build time for faster container startup ( #55422 ); VLLM_USE_PRECOMPILED finds a compatible published wheel within the last 20 commits ( #58871 ); the vendored DeepGEMM uses TORCH_LIBRARY (abi3, #48962 ). Breaking Changes & Deprecations Python 3.10 is no longer supported; the minimum is Python 3.11 ( #60402 ). Default CORS origins changed from * to none; set --allowed-origins for browser clients ( #61009 ). transformers>=5.16.1,<5.20.0 and tokenizers>=0.23.2 are required ( #57387 , #59796 ). The torch.compile sequence parallelism and AsyncTP passes were removed, along with PassConfig.enable_sp , fuse_gemm_comms and sp_min_token_num ; model-level sequence parallelism is unaffected ( #60517 ). Cascade attention was removed; --disable-cascade-attn still parses but only warns and will be removed in v0.33.0 ( #61126 ). mamba_cache_mode="all" was removed; --mamba-cache-mode all now fails validation ( #58997 ). Previously deprecated APIs were removed: wait_for_inflight_requests (use mode="wait" ), AsyncLLM.pause_generation() , IOProcessor.output_to_response , vllm.scripts.main() and vllm bench throughput --dataset ( #60923 ). The sonnet benchmark dataset was removed as scheduled ( #60129 ). Tensorizer support was removed from LoRA ( #60024 ), and the PyO3 vllm._rust_tool_parser bridge was removed ( #59744 ). API changes: max_tokens is treated as an upper bound on output rather than reserved from the context window ( #57035 ); Responses API streaming uses response.content_part.added/done with part.type="reasoning_text" instead of response.reasoning_part.added/done ( #59652 ); SamplingParams.prompt_logprob_token_ids is per row ( #56984 ); /inference/v1/generate logprobs use integer token IDs ( #58181 ); sleep API responses are standardized and invalid levels return 400 ( #52864 ); return_mm_kwargs is deprecated in favor of mm_payload ( #61427 ). Watermarking: when the engine has watermarking configured, an omitted request field means enabled and only false opts out; unsupported combinations warn and fall back ( #56801 ). Rust frontend: qwen3_xml now parses Qwen3-Coder XML (alias of qwen3_coder ); use the new qwen3_json for the JSON format ( #60136 ). New defaults: shared-memory tensor arena on ( #51207 ); GLM-5.x FP8 KV cache on SM100 with --kv-cache-dtype auto ( #60140 , #60281 ); VLLM_DEEPEP_V2_ALLOW_HYBRID_MODE=1 ( #57991 ); Humming preferred over Marlin on SM90 ( #56997 ); sparse_mla_prefill_workspace_factor 2 above 256K max_model_len ( #61023 ); MRV2 on ROCm with model-specific MRV1 exceptions removed ( #56360 ); VLLM_ROCM_AITER_MLA_DCP_VERIFY=auto ( #59965 ); dtype=auto picks bfloat16 on PowerPC ( #58528 ); MXFP4 W4A4 checkpoints fall back to W4A16 with a warning when no W4A4 kernel exists ( #60079 ). Stricter validation: incompatible --prefix-match-unit ( #58021 ), max_num_scheduled_tokens=0 ( #60714 ), HiSparse with cudagraph_mode=FULL ( #59688 ), and negative --device-ids ( #49053 ) are rejected at startup. New Contributors @0xViking made their first contribution in #60053 @1fanwang made their first contribution in #40986 @a-sidorova made their first contribution in #55335 @AbhayShuklaIIT made their first contribution in #59763 @ahmed-bsod made their first contribution in #58655 @ai-yang made their first contribution in #58088 @AIwork4me made their first contribution in #54706 @AlanFokCo made their first contribution in #57163 @AMD-yanfeiwang made their first contribution in #61004 @aniskumar-nv made their first contribution in #53020 @Anurag-M1 made their first contribution in #57693 @Aravind-11 made their first contribution in #60098 @awesome-pro made their first contribution in #58895 @Ayu190505 made their first contribution in #57995 @beenpow made their first contribution in #57648 @big-yellow-duck made their first contribution in #59132 @blueberry808 made their first contribution in #58932 @bluemelov1 made their first contribution in #58560 @bnkaiteam made their first contribution in #60507 @brandonfzhang made their first contribution in #56881 @Btwestyo made their first contribution in #59924 @bvolpato made their first contribution in #55203 @C1ves made their first contribution in #58890 @charlesli640 made their first contribution in #59977 @cheese-cakee made their first contribution in #59079 @CorgiBoyG made their first contribution in #55596 @damienlaine made their first contribution in #45022 @danichan-mkm made their first contribution in #59285 @danigarciaoca made their first contribution in #56372 @debanshd made their first contribution in #57476 @devpramod made their first contribution in #60855 @devshah-cohere made their first contribution in #60286 @DGlan made their first contribution in #59875 @dhairyashilRG made their first contribution in #58911 @DOCCA0 made their first contribution in #52641 @Dovis01 made their first contribution in #56994 @ebfio made their first contribution in #58165 @ejrtks1020 made their first contribution in #59180 @eky-amd made their first contribution in #54857 @gaby made their first contribution in #59126 @haotiaa made their first contribution in #60766 @harshaladhav-amd made their first contribution in #58268 @hdq66666 made their first contribution in #59695 @hoaivoke made their first contribution in #61033 @hpdkhoa made their first contribution in #60747 @hubert-marek made their first contribution in #61000 @hubunt made their first contribution in #47512 @huishi-hs made their first contribution in #58053 @hyukjlee made their first contribution in #59965 @Id545 made their first contribution in #59247 @ieaves made their first contribution in #52824 @ischencheng made their first contribution in #60763 @Jeff-Tseng-dev made their first contribution in #59396 @jiangyunfan1 made their first contribution in #57780 @JimmyWang0417 made their first contribution in #61081 @jin-amd made their first contribution in #58904 @JRJRJPRO made their first contribution in #59879 @Kevinli24 made their first contribution in #58878 @Kylinny made their first contribution in #60354 @Lcy0801 made their first contribution in #58975 @Legend-2727 made their first contribution in #59293 @lianakoleva made their first contribution in #60136 @LiRunGuo made their first contribution in #56198 @lobo235 made their first contribution in #60364 @LostFox11 made their first contribution in #58542 @MadhavJivrajani made their first contribution in #57635 @MatthewCYM made their first contribution in #57146 @Mazukiri made their first contribution in #60036 @MohammadHijjawi97 made their first contribution in #59072 @MohitAMD made their first contribution in #55053 @nishantrevur made their first contribution in #59541 @njriasan made their first contribution in #60502 @NNNtrance made their first contribution in #55647 @NolenLiang made their first contribution in #56974 @notimesea made their first contribution in #59105 @ntheanh201 made their first contribution in #57734 @nv-hwoo made their first contribution in #58399 @ojeda-e made their first contribution in #59107 @Ostring24 made their first contribution in #50502 @peperunas made their first contribution in #58938 @pondzikk made their first contribution in #55601 @praveingk made their first contribution in #60161 @puneetsharma21 made their first contribution in #48074 @rdlh made their first contribution in #60415 @rmhaskarnvidia made their first contribution in #59081 @roG0d made their first contribution in #55937 @roipony made their first contribution in #40337 @shijie-lyu made their first contribution in #55771 @shiyuan680 made their first contribution in #51350 @shreyanshj10 made their first contribution in #60359 @siliangchen-amd made their first contribution in #57108 @Sohaib-Ahmed21 made their first contribution in #59827 @spa5k made their first contribution in #60497 @sricursion made their first contribution in #60307 @SunnyR made their first contribution in #59811 @superdiaodiao made their first contribution in #55717 @surajsharan made their first contribution in #60225 @surawut-jirasaktavee made their first contribution in #60876 @Thiago4532 made their first contribution in #58014 @TommyTranX made their first contribution in #48039 @TylerMa-debugg made their first contribution in #59917 @ultramancode made their first contribution in #60786 @usmanmateen made their first contribution in #59336 @v-shobhit made their first contribution in #59655 @vaclavcadek made their first contribution in #49602 @viktorkou made their first contribution in #58124 @vovani made their first contribution in #60194 @WarrierRajeev made their first contribution in https://github.com/vllm-project/vllm/pull/59135 @www6v made their first contribution in https://github.com/vllm-project/vllm/pull/59749 @xiaofei-zheng made their first contribution in https://github.com/vllm-project/vllm/pull/55104 @xiaoyu-xyz made their first contribution in https://github.com/vllm-project/vllm/pull/55161 @xuantengh made their first contribution in https://github.com/vllm-project/vllm/pull/56740 @yantinglai made their first contribution in https://github.com/vllm-project/vllm/pull/59530 @YashasviAsthana made their first contribution in https://github.com/vllm-project/vllm/pull/59020 @YidaWeng made their first contribution in https://github.com/vllm-project/vllm/pull/59373 @yongqiw-i made their first contribution in https://github.com/vllm-project/vllm/pull/59159 @YueshenZ made their first contribution in https://github.com/vllm-project/vllm/pull/59257 @YuvrajSinghBhadoria2 made their first contribution in https://github.com/vllm-project/vllm/pull/57035 @zhangchengzhucufe-dev made their first contribution in https://github.com/vllm-project/vllm/pull/59886 @ZhenchengLin made their first contribution in https://github.com/vllm-project/vllm/pull/59251 @ztysth made their first contribution in https://github.com/vllm-project/vllm/pull/59721 Contributors @BugenZhao , @aoshen02 , @stefankoncarevic , @mgoin , @khluu , @hmellor , @yewentao256 , @lesj0610 , @MatthewBonanni , @jperezdealgaba , @ZJY0516 , @AndreasKaratzas , @aarushjain29 , @Sunt-ing , @NickLucche , @vadiklyutiy , @hickeyma , @divakar-amd , @yma11 , @jzakrzew , @djramic , @simondanielsson , @microslaw , @Thangnguyenvn98 , @mjkvaak-amd , @njhill , @chaojun-zhang , @sfeng33 , @zhenwei-intel , @andyxning , @matteso1 , @vMaroon , @alec-flowers , @simon-veitner-redhat , @TheEpicDolphin , @cagrikymk , @atalman , @shijie-lyu , @cheese-cakee , @Juntian777 , @ivanium , @chaunceyjiang , @louie-tsai , @Dovis01 , @apex-mochen , @wzhao18 , @venkywonka , @pmanczak , @stecasta , @jinzhen-lin , @gty111 , @noooop , @Rohan138 , @rasmith , @laithsakka , @Fangzhou-Ai , @tlrmchlsmth , @Isotr0py , @cjackal , @vllmellm , @liusy58 , @reidliu41 , @peperunas , @shimib , @ojeda-e , @yonigozlan , @ShuoleiWang , @lucamotz , @ai-jz , @he-yufeng , @DimensionSTP , @taneem-ibrahim , @LucasWilkinson , @wangshangsam , @whx-sjtu , @micah-wil , @bzsuni , @robertgshaw2-redhat , @arpera , @sricursion , @mawong-amd , @mmastrac , @rmhaskarnvidia , @nvbfalk , @debanshd , @abmfy , @shaohuaxi , @jin-amd , @xijiaat, @ashraf-bhuiyan, @beenpow , @LioEinaudi, @Ronald1995, @R3hankhan123, @ZhengGong-amd, @Akashcodes732, @S1ro1, @yuvalluria, @mganczarenko, @DarkLight1337, @pisceskkk, @YidaWeng , @TQCB, @gau-nernst, @askliar, @benchislett, @jackLei0901, @vx120, @amd-mghanimi, @YueshenZ , @aniskumar-nv , @fululi12, @liuzijing2014, @zyongye, @Prudhvivuda, @kliuae, @wjabbour, @AbhayShuklaIIT , @WarrierRajeev , @Jeff-Tseng-dev , @yifanFengg, @elvircrn, @zupengwang, @tangzzycc, @jeejeelee, @roikoren755, @JohnQinAMD, @LeizhenZhang, @Chinmay-Kulkarni-AMD, @sungbin1015, @zhangchengzhucufe-dev, @fxmarty-amd, @lucianommartins, @nishantrevur , @hyukjlee , @andylolu2, @wangxiyuan, @stu-cao, @vineethsaivs, @zzaebok, @WoosukKwon, @franciscojavierarceo, @hpdkhoa , @gcanlin, @zigzagcai, @ultramancode , @warrendeng, @surajsharan , @subhashpolisetti, @ilmarkov, @awesome-pro , @faaany, @spandantiwari, @almogtavor, @ahmed-bsod , @danigarciaoca , @Alnusjaponica, @NNNtrance , @vaclavcadek , @YashasviAsthana , @QwertyJack, @Ubospica, @AlanFokCo , @jungjiyu, @lcskrishna, @YukioZzz, @hubunt , @sashko-zakharchuk, @staryxchen, @yzong-rh, @Ostring24 , @taking-lying-flat, @drakosha, @CorgiBoyG , @QHarshil, @zhou9402, @shiyuan680 , @fjosw, @a-sidorova , @GOavi101, @mrodden, @fuscof-ibm, @Taimys, @Rukhaiya2004, @KernelClint, @almayne, @JulienDarve, @sshlyapn, @eldarkurtic, @rbrugaro-amd, @GuyStone, @ZhenchengLin , @ArcheyChen, @djmmoss, @hjjq, @namgyu-youn, @jiangyunfan1 , @ganeshr10, @markmc, @yantinglai , @Id545 , @roipony , @JadenMathias, @hclsys, @Anurag-M1 , @ukannika, @blueberry808 , @nv-hwoo , @guanxingithub, @stmatengss, @mohit-sarvam, @Legend-2727 , @xiaoyu-xyz , @hdq66666 , @Lcy0801 , @gaby , @erictang000, @jdebache, @Ayu190505 , @DOCCA0 , @C1ves , @maeehart, @wangyicong52, @AIwork4me , @Woolgathererer, @yongqiw-i , @divyvasal, @1fanwang , @TyroneNel, @ShengleiFu, @bluemelov1 , @kamb-code, @LiRunGuo , @Thiago4532 , @harshaljanjani, @ieaves , @NolenLiang , @sudhanshu112233shukla, @abhayjoshi201, @charlesli640 , @Sohaib-Ahmed21 , @akii96, @notimesea , @JRJRJPRO , @usberkeley, @Mazukiri , @brandonfzhang , @amd-xavierwang, @sawsa307, @czhu-cohere, @wtdcode, @tarukumar, @LostFox11 , @big-yellow-duck , @SherifWaly, @roG0d , @esmeetu, @mgehre-amd, @samnordmann, @SunnyR , @ovidiusm, @cleonard530, @eky-amd , @yinli-systems, @MadhavJivrajani , @cquil11, @ywang96, @PatrykSaffer, @PatchouliTIS, @aditi-amd, @git-jxj, @vovani , @MohammadHijjawi97 , @TommyTranX , @viktorkou , @xaguilar-amd, @MohitAMD , @danichan-mkm , @pondzikk , @ebfio , @mkhazraee, @ai-yang , @xuantengh , @dhairyashilRG , @www6v , @v-shobhit , @ejrtks1020 , @Kylinny , @bnkaiteam , @huishi-hs , @jiahanc, @shen-shanshan, @zxd1997066, @Aravind-11 , @usmanmateen , @devshah-cohere , @praveingk , @JartX, @TylerMa-debugg , @lobo235 , @dougbtv, @Tialo, @kyleliang-nv, @YeonwooSung, @Kevinli24 , @aijanai, @spa5k , @DGlan , @Btwestyo , @xianbaoqian, @rdlh , @hao-aaron, @errmakov, @amd-sriram, @ntheanh201 , @LCAIZJ, @damienlaine , @shreyanshj10 , @ischencheng , @0xViking , @yisustc, @bigPYJ1151, @Roy214, @Gregory-Pereira, @YuvrajSinghBhadoria2 , @surawut-jirasaktavee , @Oxygen56, @andakai, @BolinSNLHM, @Dao007forever, @siliangchen-amd , @xiaofei-zheng , @gc-fu, @puneetsharma21 , @linitra24, @devpramod , @yankay, @hlin99, @Godmook, @bvolpato , @UNIDY2002, @tc-mb, @JaredforReal, @hubert-marek , @Zhenzhong1, @AMD-yanfeiwang , @thuongvu, @JimmyWang0417 , @MatthewCYM , @i-m-aditya, @lianakoleva , @peakcrosser7, @hoaivoke , @njriasan , @tjtanaa, @haotiaa , @superdiaodiao , @josh-schell-amd, @ztysth , @sungsooha, @Etelis, @natsala13, @ZhouXiaoya25

Explore central Warsaw in voxels - slightly inspired by Minecraft, and MagicaVoxel. You can change the date, time of day and weather, explore the city, or click on a pedestrian to control them (here a bit of inspiration from Dungeon Keeper). Works best on desktop, but I tried to make it usable on mobile too. The initial load can take a while. It's based on several open datasets, including: - OpenStreetMap ( https://www.openstreetmap.org/#map=6/52.02/19.14 ) - buildings, streets, sidewalks, tram tracks, etc - GUGiK / Geoportal ( https://www.geoportal.gov.pl/ ) - LiDAR for building heights and trees, aerial imagery for colours - Warsaw's city tree database ( https://mapa.um.warszawa.pl/ ) - individual tree species, used for seasonal foliage Open-source ( https://github.com/stared/warsaw-vibe-voxel ), so feel invited to (re)use it as you wish! Comments URL: https://news.ycombinator.com/item?id=50339221 Points: 264 # Comments: 42

...** potential step toward self-improving** AI . Again: folks outside our bubble have no clue what’s happening — Chubby Source: https://x.com/kimmonismus/status/2117343192245064004 We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: https://t.co/DwTwzXzSbE — Rikiya Takehi Source: https://x.com/rikiyatakehi/status/2117313742891327994 submitted by /u/stealthispost to r/accelerate [link] [comments]

The top configuration of the Raspberry Pi 5 now costs three times more than its launch price. www.theverge.com/tech/1024342... Linked page: The Pi 5 16GB now costs triple its original price.

OpenAI : OpenAI partners with Cathedral to test its security defenses and revamps its Security Bug Bounty program, including quadrupling rewards for critical findings — We're planning our next internal security sprint at OpenAI, bringing new researchers and security partners together with our own teams …

What's Changed ollama launch claude now applies the model's output token limit for Claude Code, so cloud models no longer run into the 128k default cap Cloud web search responses on /api/chat are now decoded correctly, including compressed responses OpenAI-compatible endpoints no longer override a model's packaged sampler defaults — clients that send no temperature / top_p get the model's own values instead of 1.0, matching /api/chat /v1/responses now preserves readable input content, including prior assistant refusals, instead of flattening it Ollama's app can now retry ChatGPT disconnect failures instead of leaving the connection in a broken state ollama launch codex verifies ChatGPT routing before configuring, and reports a clear error when routing is unavailable Faster top-k sampling on MLX for large vocabularies Full Changelog : v0.40.13...v0.40.14

A few notes on Anthropic's new Claude Fable 5.1 - with Max thinking level I got the best SVG pelican I've had from any Anthropic model (at a hefty cost of $3.30!), which I then had it animate simonwillison.net/2026/Sep/1/c...

submitted by /u/1-800-methdyke to r/LocalLLaMA [link] [comments]

Here's what you probably missed in October 2026: 🧠 LLMs Ornith-1.5-122B - Handles million-token agent tasks on a single GPU. Raon-1-mini-preview - Runs Korean agentic AI on a single RTX 5090. Timestral-209M - Forecasts time series in one pass with 21 quantiles. Carbon-X - Reads DNA in chat and predicts variant effects on one GPU. Sage-A1.5 - Orchestrates scientific agents with self-review. Spark-X2.6-4B - Advances on-device agents with controllable thinking. Nanbeige4.3-33B - Teacher model leads agentic coding and cowork. Flint - MoE coding agent investigates repos and fixes bugs on one GPU. Hy-Memory-30B-A3B - Writes durable agent memory at 3B active parameters. Underdog-Saluki-27B-1.0 - Packs Qwen3.8 tool calling into under 8 GB. D3 - Answers choice, yes-no, and score questions in one call. LFM2.5-24B-A2B - Packs fast on-device agent reasoning into 2B active. Qwen3.5-2B-RCOL - Low-bit quants hold quality under 1 GB. GLM-5.3-UNCENSORED-EXL3-3.0bpw - Fits a 753B MoE in 273 GB. Humanizer - Rewrites AI drafts to sound human in English and Chinese. Kolibri 1 - Brings 78B MoE reasoning to German and English. GEV-26B-Decide - Thinks only when needed on Gemma-4. Victoria - Trims Qwen3.8-Flash-Next experts for a single GPU. Hearim Gateway - Routes Jev System One decisions over local LLMs. Sharp-Spark-X2.5-4B-GGUF - Fits a long-context coder in 6GB. LingBot-World 2.0 Realtime - Runs a world model at 16 FPS. deepseek-v4.1-flash-gguf - Runs DeepSeek Flash on Apple Silicon. ThreadShelf - Archives and searches your AI chat history locally. Aurora1.0-150M - Packs an efficient tiny model you can run anywhere. Intern-S2-397B - Scales up scientific reasoning and agents. NeoHorse-1-9B - Agentic model steps toward self-improvement. Atria-Dawn-Preview - Agentic model aims for verifiable research results. Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF - Speeds local reasoning. 🔀 Multimodal MiniMax-M3.1-Flash - Runs multimodal agentic AI with a 1M token context. Indra-OCR - Reads PDFs and handwriting on CPU at 44 pages per second. Qwen3.8-Flash-Next-FP4-Local - Runs a quantized 125B model on Blackwell GPUs. MiniCPM-V-4.7-1B - Packs efficient multimodal AI for phones and tablets. Muse-Spark-Open-816B-A42B - Packs 816B parameters into a multimodal agent. MiniCPM-V-4.7-35B-A3B - MoE runs multimodal AI smoothly on phones. DeepSeek-V4.1-Flash-1017 - Sharpens agents and vision. FrogNano-4B-2609-GGUF - Adds image input and MTP speed to quants. Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF - Drops refusals without retraining. D1-3B - Runs calibrated decisions on edge devices in one pass. Clef-Flash - Brings low-latency decisions to a 9B Qwen model. GLM5.3-Flash-E224-DGX-Spark - Packs NVFP4 MoE for two DGX Sparks. Clef Model - Turns state and questions into calibrated decisions. EmbeddingGemma 2 - Maps text, image, video, and audio into one space. JEV-27B-VL - Brings calibrated vision decisions to Qwen3.8. HuatuoGPT-3-9B - Brings one-stage medical reasoning to local models. Realtime-Venus - Holds full-duplex conversations with voice and video. Agnes-3.0-Flash Preview - Packs long reasoning into 33B weights. ZDTaichu5.0-9B - Brings spatial reasoning and agent skills to VLMs. 🖼️ Image Kromoss - Fine-tunes MOSS-Image for eight-step turbo generation. Qwen-Image-2.1-Turbo-Uncensored-GGUF - Drops encoder refusals locally. MOSS-Image-Base - Brings layout control to local image generation. MOSS-Image-SFT - Sharpens text and layout control in image generation. Qwen-Image-2.1-Turbo - Renders images in eight fast steps. Qwen-Image-2.1-Uncensored-Quantized - Ships with filters off. FLUX 3 Image Klein - Blends up to 10 references into one image. Iris-3B - Paints pixels directly for images, depth, and restoration. Qwen-Image-2.1 Fun ControlNet Union - Drives eight controls at once. Agate Preview 003 - Renders 512px images from only 261M parameters. Wulver v0.5 - Fine-tunes Krea 2 for anime and furry art. PrunaSuperPoint - Cuts SuperPoint keypoints up to 2.1x faster on edge. VisionHOPE - Backbone self-modifies to see and segment images. Anima-Light-Lavender - Boosts natural-language anime prompts. Krea 2 Turbo SDA LoRA - Restores sampling diversity. SMACK LoRA - Adds punchy impact effects to MiniMax H3 video. Krea 2 Lineart Edit - Turns sketches into rendered images. Krea-2-merge-tool - Merges and analyzes Krea 2 models on Windows. Krea-2-Pixel-Art - Adds retro pixels at three sizes to Krea 2. MageTrail - Brings Danbooru-style tuning to Microsoft's MageFlow base. 🎬 Video DMAD - Distills MiniMax H3 into a four-step audio-video generator. LTX-2.5-22b-IC-LoRA-Layout-To-Render - VFX tools add native 4K and 8K video post production. DuoMatching - Distills Wan2.1 into a two-step video generator. MiniMax-H3-X2-Detail-VAE - Sharpens and upscales H3 video. LTX-2.5 IC-LoRA Ingredients - Keeps characters and props consistent in video. FastH3 V2 - Runs fast video models on consumer GPUs. Meridian - Re-creates camera moves around existing video. 🎧 Audio Fish Audio Drama 3 - Speaks for 16 characters in one prompt. Drifting-TTS - Makes one-step Turkish speech on low latency. Whistle - Runs tiny on-device speech recognition without a GPU. EMA Lightning - Makes a tiny, fast Turkish text-to-speech model. GRVL LoRAs - Give YuE2 raspy rock-soul female vocals. YuE2 BLGR LoRAs - Add Bulgarian choir and Tuvan throat singing. YuE2 Qawwali LoRAs - Add Sufi singing to music generation. Audio Slop Server - Runs many audio models on one GPU. audio.cpp YuE2 - Runs Suno-class music in 8-9 GB of VRAM. YuE2 Real-Audio Tokenizer - Lets you tune music on your own tracks. YuE2 Instrumental LoRA - Writes score-led instrumental tracks. YuE2-hum-to-song - Turns a hummed melody into a full song. Rumik-oss-1 - Speaks 22 Indian languages with expressive control. Confucius4-R2T2 - Delivers low-latency streaming speech recognition. 🤖 Agents Tencent Octop - Runs a self-hosted multi-agent assistant at home. 🎲 3D AssetFurnace - Turns a prompt into a rigged, animated 3D character. ⚡ LoRA MiniMax-H3-Person-Remover-LoRA - Deletes people from video. Pruna LoRAs - Cut Qwen-Image-2.1 down to six fast steps. Krea 2 Inpaint Edit LoRA - Fills masked areas in ComfyUI. YuE2 Deathmetal LoRA - Adds melodic death metal to AI songs. TRBDR Folk Troubadour LoRAs - Give YuE2 a 1960s balladeer. AnyAngle LoRA - Retains style while changing camera angles. QTSTRM Quiet Storm LoRAs - Bring 80s R&B vocals to YuE2. LTX-2.5 Restore IC-LoRA - Revives old footage in color. MiniMax-H3 Motion Continuity Repair V2 - Fixes broken action frames. MiniMax-H3 CrossView LoRA - Shoots a scene from new angles. Krea2-Turbo-Distill-2step-LoRA - Cuts image steps from eight to two. TaoMate-H3-3Step-ComfyUI - Speeds up H3 runs in ComfyUI. Minimax-H3-ComfyUI LoRAs - Add style and VFX editing to H3 video. BFS-Best-Face-Swap LoRAs - Swap faces across Qwen Image and Flux. 🏋️ Training LoRAlab TrainerStudio - Trains latest models on low-VRAM GPUs. ModelFiche - Organizes image-model training for agents and reviewers. 📊 Datasets Danbooru 2026 Tag Cleaning Corrections Dataset - Released with cleaned-up tag corrections for training. ☰ UI TinyLuma - Is an 8MB desktop editor for finishing AI images. Goated Prompter - Refines image and video prompts locally. Fooocus-Qwen-Image-2.1 - Brings Qwen-Image editing to small GPUs. Slopus - Turns MiniMax H3 video generation into a desktop app. Image Studio - Brings local Qwen-Image 2.1 editing to Apple Silicon. Pinhole - Turns any GPU into a simple one-button local image generator. H3 Prompt Studio - Directs MiniMax H3 films from a local app. Qamba Studio - Edits AI video end-to-end on your own machine. PotionUI - Puts self-hosted diffusion studios in a browser. H3 Studio - Gives MiniMax-H3 video a local Mac web UI. Livebound - Brings a local publishing workspace to CivitAI creators. 💻 Hardware AMD EPYC 9006 Venice CPUs - Listed from $700 to $14,904. Inno3D AWS-511Z-R1 Tower Workstation - Handles up to 4 GPUs. GMKtec EVO-X5 Pro - Runs up to 320B LLMs locally. AceMagic F9A - Packs 131 TOPS AI into a 2L mini workstation. Innodisk DDR5-8000 RDIMM - Launched for edge AI and servers. Panasonic Toughbook 34 And G3 - Bring AI-ready rugged laptops. Minisforum AtomMan G1 Pro - Adds a 16 GB RX 9060 XT to its mini PCs. Cervoz T449 And T459 - Add rugged MIL-STD SSDs for edge AI. 🛠️ Other Tools Open-omnivoice-tts - Serves a 0.3 second voice clone API. VRAM Manager - Flags idle AI servers holding GPU memory. qwen-image-local - Runs Qwen-Image-2.1 on a 12 GB GPU. Lumibelle Video Studio - Turns a screenplay into a local film. AutoRef - Coaches FLUX.2 into composing multiple reference images. MiniMax Code - Turns terminal prompts into working projects. Tool-prune - Prunes tool schemas to cut agent prompt costs. ZCode - Open sources an AI coding workbench in three forms. YuE2-Radio - Creates an endless local AI music radio. Inkvec - Traces images into clean vector SVGs in the browser. wan2gp-h3-latent-continue - Lets Wan2GP continue H3 video from latents. Jeff - Puts a fast 0.8B decision model in front of your 27B. ComfyUI Releases Speed & Performance Boosts ComfyUI-VELA-H3 - Speeds up VDN H3 on MiniMax H3 in ComfyUI. Veda-on-ComfyUI - Speeds up MiniMax H3 in ComfyUI. ComfyUI H3 W4A4 VSA - Cuts Ref2VA video time down to 3 minutes. ComfyUI-Ref2VA-VSA - Speeds character video generation to 72 seconds. ComfyUI-Spectrum-Qwen2.1 - Speeds up image generation by about 2x. ComfyUI-Qwen-Image-2.1-PromptEnhancer-MTP - Runs faster in ComfyUI with MTP support. ComfyUI-HunyuanImage3 - Runs an 80B image model on a single GPU. ComfyUI-AppleSilicon-FP8 - Runs quantized models on Mac. LynnReal-Omni - Packs many video tasks into one fast model. Video & Image Creation H3 Long Shot Studio - Makes multi-shot films inside ComfyUI. ComfyUI-Olm-SoL-Refiner - Brings video refinement to nodes. Fizgig H3 Still - Renders true MiniMax H3 frames in ComfyUI. Fizgig H3 Tweaks - Gives ComfyUI fine control over MiniMax H3. Scumble - A free layer-based AI inpainting editor. ComfyUI-BubbleText - Puts real text inside generated speech bubbles. ComfyUI-OutpaintMask - Builds interactive outpainting masks. Director's Console - Designs cinematic shots and runs ComfyUI. ComfyUI-Prompt-Studio - Turns reference images into production prompts. Ming Image Prompt Builder - Draws structured prompts in ComfyUI. ComfyUI-BingImageSelector - Picks search images before you run. Workflow & Productivity ComfyUI-QueueWorkbench - Manages and edits queued jobs in place. ComfyUI-NodeSnapshots - Smoothes navigation in large workflows. ComfyUI-TypedDecision - Brings Jev-style decisions to local workflows. comfyui-obvpm - Tidies up workflows with wire bundles and presets. ComfyUI_Assistant - Adds a chat robot that edits your workflows. Comfy-compare-copy-buttons - Adds a copy button to the ComfyUI Compare node. Comfyui-workflow-background - Sets custom wallpapers for ComfyUI. Prompt-vault - Stores and sorts your AI prompts locally. Comfyui-prompt-library - Brings weighted prompts and templates to ComfyUI. Genkai-ComfyUI-Nodes - Syncs video playback with timed prompts. Music & Audio ComfyUI-FL-YuE2 - Brings music generation and LoRA training to nodes. ComfyUI-QwenASR - Adds speech nodes and subtitles to ComfyUI. Mobile & Access ComfyPort - Brings ComfyUI workflows to a native Android client. Need to go further back? Check out September's post or the full archive at LocalAI News . If there's anything wrong, let me know in the comments and I'll see you in the next one! submitted by /u/vramkickedin to r/StableDiffusion [link] [comments]

New Model additions Nemotron 3 Diarization Nemotron 3 Diarization is an open-weight streaming speaker diarization model designed to determine "who spoke when" in real-world audio. It supports both streaming and offline inference, handles up to eight speakers, and orders speaker outputs by each speaker's first arrival in the input audio. The model uses the Arrival-Order Speaker Cache (AOSC) 1 and FIFO queue introduced for Streaming Sortformer 1 , 2 . A single checkpoint supports configurable latency profiles, from an 80 ms input buffer to a 30.4 s offline-style buffer, and configurable output frame resolution in multiples of 10 ms. With chunked inference, the maximum audio duration is not limited. Links: Documentation Add Nemotron3Diarization ( #49056 ) by @eustlb in #49056 NemotronH Omni NemotronH Omni is a multimodal reasoning model from NVIDIA that pairs the NemotronH hybrid Mamba-Transformer language model with a RADIO vision encoder and an optional Parakeet-based sound encoder. Image (and video) patches are projected through a RADIO tower and a pixel-shuffle MLP into the language model's embedding space at the <image> / <video> context-token positions; audio clips are projected in the same way at <audio> positions. The result is a single autoregressive model that reasons jointly over text, images, video and sound. Links: Documentation Add support for Nemotron Omni ( #46509 ) by @meatybobby in #46509 HyperCLOVAX Vision V2 HyperCLOVAX Vision V2 is a multimodal vision-language model developed by NAVER. It combines the HyperClovaX language model backbone with a Qwen2.5-VL vision encoder. The model supports text, image, and video inputs and is capable of chain-of-thought reasoning via built-in thinking tokens ( <think>...</think> ). Links: Documentation add HyperClovaX Vision ( #44314 ) by @jp1924 in #44314 GTE GTE was proposed in mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval by Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li and Min Zhang. GTE is a BERT-style bidirectional encoder that replaces absolute position embeddings with RoPE, uses a gated MLP, and applies layer normalization after each residual connection. The same architecture backs Alibaba's gte-*-v1.5 , gte-multilingual-* and gte-en-mlm-* checkpoints as well as Snowflake's snowflake-arctic-embed-m-v2.0 . Links: Documentation model: Add GTE to Transformers ( #48416 ) by @harshaljanjani in #48416 Breaking changes 🚨 [ROCm] gpt-oss: route FA3 to aiter-flash-attn, generate ROCm fixtures ( #46837 ) by @Abdennacer-Badaoui 🚨 Speed up detr image processing ( #48066 ) by @guarin 🚨 Remap indexers layer_type ( #48974 ) by @Cyrilvallez 🚨 [vLLM] Fix video token counting for Transformers backend video inputs (Part 1) ( #48894 ) by @harshaljanjani 🚨 🚨Bring some dinos to modern standards ( #46266 ) by @molbap 🚨 [ Kernels ] Bump version ( #48714 ) by @vasqu Bugfixes and improvements fix incorrect hub tokenizer class ( #48641 ) by @itazap [CI] Deduplicate Nvidia/AMD CI reply comments and add headers ( #48655 ) by @ydshieh Update dev ( #48654 ) by @vasqu [tests] Fix NougatModelIntegrationTest: pin artifact revision and update golden values ( #48638 ) by @ydshieh [fix] Fix GlmOcr integration tests: wrong token IDs and image token decode bug ( #48650 ) by @ydshieh [GLM 5.3 Flash] Preserve original names when saving text checkpoints ( #48676 ) by @Dovis01 Register activation kernel layers on XPU ( #47858 ) by @jiqing-feng Mirror the registered pytree flatten when building automatic dynamic shapes ( #48578 ) by @IlyasMoutawwakil Fix AutoImageProcessor requiring torchvision when only Pillow is installed ( #48616 ) by @blipbyte Assert cached decode matches recomputing without a cache ( #48289 ) by @IlyasMoutawwakil [higgs_audio_v2] Fix: use config.num_codebooks in audio labels tensor (Simple Fix!) ( #48560 ) by @b-re-w [ MoE ] Fix eager EP ( #48653 ) by @vasqu [MusicgenMelody] Fix conditioning silently dropped at generation step 0 ( #48679 ) by @ydshieh Fix deepspeed ci ( #48640 ) by @SunMarc Synthetic test assets ( #48589 ) by @tarekziade Skip the expert-parallel sentinel masking when expert parallelism is off ( #48201 ) by @qgallouedec [Fix] yolos offload issue ( #48688 ) by @molbap [serge] Fix 2 integration tests for model minimax failing with output_mismatch (tensor values differ (2)) ( #48515 ) by @sergereview[bot] [serge] Fix 2 integration tests for model mistral failing with other (other (2)) ( #48429 ) by @sergereview[bot] Fix Qwen2.5-VL temporal RoPE for fractional video intervals ( #48669 ) by @yeyeyeping Fix slow integration tests on XPU ( #48611 ) by @jiqing-feng QA: Added a MemoryCleanupMixin class for tests ( #48681 ) by @tarekziade [serge] Fix 1 integration test for model flex_olmo failing with other (other (1)) ( #48668 ) by @sergereview[bot] Fix D-FINE / RT-DETR main loss being computed over the denoising queries ( #48528 ) by @stefan-it [CI] Replace hardcoded username allowlists with author_association check in workflow triggers ( #48712 ) by @ydshieh Add flash_attention_4 in attn_implementation AutoModel docstring ( #48684 ) by @3manifold Keep the expert-parallel sentinel slots out of the router gradient ( #48689 ) by @qgallouedec [DeepseekV3] OOM cascade root-cause investigation (generator ref leak in conversion_mapping) ( #48720 ) by @ydshieh Use huggingface_hub httpx export ( #48685 ) by @Wauplin unifying device_mesh init to enable PP + TP inference ( #48155 ) by @3outeille Enable FSDP2 + expert parallelism via a 2-D (fsdp, tp) device mesh ( #48516 ) by @qgallouedec Switch daily CI to torch 2.14 — update expected outputs ( #48750 ) by @ydshieh fix videomae load error ( #48675 ) by @sywangyi Fix silently random-initializing RTDetrModel / SEWDForCTC loads (wrong base_model_prefix ) ( #48744 ) by @ Fix Glm4vMoeIntegrationTest: offload_folder + MemoryCleanupMixin ( #48776 ) by @ydshieh [InternVL] Normalize num_patches before np.cumsum ( #48469 ) by @lorenzozanee Kernel api doc, part 2 ( #46889 ) by @michaelbenayoun Fix hidden state selection on Gemma4 assistant's first prefill step ( #48704 ) by @glistening Fix inkling embedding norm ( #48786 ) by @Cyrilvallez Fix TextToAudioPipeline crash for tokenizer-only models ( #48505 ) by @jiqing-feng Cast pixel values to the patch embedding dtype in DeepSeek-OCR-2 ( #48632 ) by @jiqing-feng add kernel mapping entry for RMSNormGated, KDA, Conv1D on XPU ( #48702 ) by @kaixuanliu Bump peft version requirement ( #48716 ) by @shniubobo [serge] Fix 12 integration tests for model edgetam failing with import_or_config (other (12)) ( #48322 ) by @sergereview[bot] Restore legacy tensor-parallel initialization for compatibility ( #48797 ) by @3outeille Fix kosmos flaky test ( #48800 ) by @IlyasMoutawwakil Add integration tests for MuseGlimmerAssistantModel ( #48796 ) by @ydshieh Fix AutoModel.from_pretrained not restoring modules_to_save weights ( #48595 ) by @shniubobo [docs] mrope and axial rope ( #48717 ) by @stevhliu Fix decoder only path for older bert variants ( #48785 ) by @Cyrilvallez [Fix] Clean-up ternaries in the DeepSeek family ( #48447 ) by @remi-or [utils] Add MUSA support for Flash Attention 2 ( #48612 ) by @XiaomingFun233 [generate] Drop attention mask early without padding ( #48814 ) by @Cyrilvallez Pin utf-8 in test_can_init_all_missing_weights source read (Windows non-UTF-8 locale fix) ( #48819 ) by @dltsum Relax static-cache tolerance in test_generate_with_static_cache (1e-5 → 5e-5) ( #48815 ) by @ydshieh Detect nested rope_parameters without relying on layer_types ( #48798 ) by @hmellor Support for MoE in the GGUF integration ( #48529 ) by @SunMarc [docs] gguf ( #46357 ) by @stevhliu [apply_chat_template] pass sampling_rate to call ( #48794 ) by @eustlb [CI] Add link checker ( #48160 ) by @stevhliu Qwen3.8 GGUF ( #48660 ) by @SunMarc docs: fix broken #combining-with-fsdp2 anchor in expert_parallelism.md ( #48854 ) by @ydshieh [GPTNeoXJapanese] Fix RoPE ignoring partial_rotary_factor ( #48652 ) by @blipbyte QA: deactivate rule 41 ( #48852 ) by @tarekziade Fix reset on the dynamic cache layers ( #48809 ) by @jiqing-feng [generate] Make all methods and logits processors agnostic to lm_head output size ( #48846 ) by @Cyrilvallez Fix kosmos ( #48853 ) by @Cyrilvallez Add test for causal only variant of some encoder-decoder models ( #48760 ) by @nandan2003 Fix ESMFold2 ligand iPLDDT weighting: mol_type non-polymer code is 3, not 4 ( #48831 ) by @faustomilletari Fix flaky RfDetr test_save_load ( #48869 ) by @ydshieh fix(ci): harden GitHub Actions workflows ( #48160 ) ( #48834 ) by @hf-security-analysis[bot] Fix Glm4MoeIntegrationTest: split into 3 tests, switch to GLM-4.5-Air ( #48820 ) by @ydshieh Fix StaticCache for Mllama and enable torch.compile ( #48141 ) by @jiqing-feng Fix Minimax M2 partial_rotary_factor by remapping legacy rotary_dim ( #48486 ) by @dajiaohuang Fix Pixtral image processor do_pad option ( #48872 ) by @jzakrzew [CB] Add pause mechanism ( #48462 ) by @remi-or Fix DFlash sampled candidates losing the batch dimension ( #48281 ) by @VaggelisGian Fix sliding window cache when assistant model calls generate ( #48280 ) by @VaggelisGian Fix GLM rope_parameters dict shared mutation across test instances ( #48895 ) by @ydshieh [ feat ] Allow untying hidden_states[-1] from last_hidden_state via the model config ( #48087 ) by @tomaarsen Fix Idefics2 padding when the first example has no image ( #48753 ) by @Arnavsharma2 Don't fail model loading when the accelerator can't report free memory ( #48136 ) by @studioego [Improvement] Rework the docstring and comments of mHC ( #48888 ) by @remi-or [serge] Fix 3 integration tests for model deepseek_vl failing with other (other (3)) ( #48536 ) by @sergereview[bot] Add auto_docstring task model overrides ( #47815 ) by @guarin Make LongcatFlashConfig self-consistent without building the model ( #48899 ) by @hmellor [ DSA ] Only save latents on dsa with indexer as well ( #48876 ) by @vasqu [Chat Parsing] Coerce oneOf tool arguments ( #48719 ) by @yonigozlan Always materialize the causal mask in Doge so sdpa stays causal ( #48821 ) by @blipbyte Fix docstring argument names that don't match signatures ( #48474 ) by @IshaanPotle Mark test_generate_with_static_cache as flaky for olmo and bigbird_pegasus ( #48856 ) by @ydshieh Clean up some MoE models' integration tests ( #48833 ) by @ydshieh Fix T5 tied weights order for lm_head ( #48238 ) by @vaibhavmashal Size the device_map buffer from the largest leaf module ( #47211 ) by @dhruv7477 Fix compile cache tests ( #48923 ) by @Cyrilvallez Update arXiv citation in NeoMME model doc ( #48926 ) by @tonywu71 Standardise Aria's MoE onto the experts interface ( #48907 ) by @hmellor Fix possessive typo in Whisper long-form warning ( #48877 ) by @erikdw Modular conversion small fix ( #48934 ) by @zucchini-nlp fix(vibevoice-asr): use integer ceiling division for audio token count ( #48864 ) by @ege-arhan Improve lazy import error messages ( #48602 ) by @karatarassul4-max Fix tests due to dropping attn mask ( #48903 ) by @SunMarc docs: fix docstring parameters that do not match signatures ( #48904 ) by @simpleqt DOC: Improve documentation for remap_legacy_layer_types function ( #48651 ) by @mathewOracle Align special tokens on the text config ( #48847 ) by @qgallouedec docs: fix dead doc links in i18n READMEs and the xlnet docstring ( #48905 ) by @simpleqt Fix AXK2 integration test: update CUDA expected text and rename class ( #48941 ) by @ydshieh Resolve the Hub revision once per load instead of passing a private _commit_hash around ( #47611 ) by @Wauplin Keep image processor backends in sync on keys and dtypes ( #48739 ) by @yupengtang Fix infeasible cost matrix errors in hugarian matcher losses ( #47730 ) by @guarin Fix more stuff ( #48979 ) by @zucchini-nlp Skip an unnecessary image copy in the torchvision image normalization path ( #48897 ) by @jzakrzew Better attn default for gguf ( #48935 ) by @SunMarc Add pose estimation keypoint preprocessing to Sapiens2ImageProcessor ( #47199 ) by @Sainava Fix image processor class-level size mutation and min_pixels handling ( #48916 ) by @Gracy769 Contract the mamba2 chunk scan with einsum instead of broadcast-then-sum ( #48978 ) by @tarekziade Fix typos in DeepseekV4 comments ( #48953 ) by @Janiarafath Reduce peak memory in examples_torch CI job (OOM fix) ( #48983 ) by @ydshieh Fix vibevoice TTS batched audio index ( #48902 ) by @ebezzam [CB] [Major] Upgrade the cache to support different attention types ( #47809 ) by @remi-or Fix SwitchTransformers Top1 router: raw logits, expert capacity accounting, and router losses ( #48421 ) by @yurekami Fix Qwen3OmniMoeIntegrationTest OOM ( #48987 ) by @ydshieh Let prefix_allowed_tokens_fn override model -inf and raise an exception on unsatisfiable generation constraints. ( #48927 ) by @ksh108405 [generate] Simplify candidate generators by removing required update_candidate_strategy ( #48982 ) by @Cyrilvallez [generate] Always correctly restrict assisted decoding with max length/eos token ( #48981 ) by @Cyrilvallez QA: applied ruff rule PLW1514 ( #48990 ) by @tarekziade Update tokenizer gguf support ( #48656 ) by @SunMarc [vLLM] Fix video token counting for Transformers backend video inputs (Part 2) ( #48900 ) by @harshaljanjani Update ggml kernels path ( #48991 ) by @SunMarc Enable compressed-tensors FP8 kernels on MPS (torch >= 2.15) ( #48985 ) by @Isalia20 Fix mps autocast handling in rotary embeddings ( #49006 ) by @Isalia20 Fix static cache per layer head shapes ( #48619 ) by @dacorvo Summarization examples: download NLTK punkt_tab, not punkt ( #49014 ) by @davanstrien [docs] Fix legacy hf CLI references (transformers) ( #48988 ) by @Wauplin Fix RecurrentGemma compiled generation with StaticCache ( #48961 ) by @sywangyi updated GraniteMoeHybrid expectations (CPU) ( #49016 ) by @tarekziade OpenVINO HF Exporter ( #47003 ) by @IlyasMoutawwakil fix noisy comments ( #49013 ) by @tarekziade Fix assisted decoding for VLM due to dropping attn mask ( #49019 ) by @SunMarc Deprecate min-max pixels ( #49021 ) by @zucchini-nlp Keep special token ids the tokenizer does not define ( #48708 ) by @albertvillanova Added more usage of MemoryCleanupMixin ( #49011 ) by @tarekziade Shorten noisy OpenVINO SDPA comment ( #49033 ) by @tarekziade PEFT x Dtensor-based TP integration ( #48485 ) by @michaelbenayoun fix(zamba): add use_associative_scan config flag to avoid torch.compile slowdown ( #48331 ) by @msnliu Scope GITHUB_TOKEN permissions per job ( #49046 ) by @hf-security-analysis[bot] Fix mask creation not being skipped under torch.compile ( #48975 ) by @jiqing-feng [Chat] Loading GGUF models served with the Chat CLI ( #49031 ) by @ariG23498 Pin GitHub Actions to commit SHAs ( #49049 ) by @hf-security-analysis[bot] Only seed numpy in BigBird block-sparse attention during training ( #49037 ) by @jiqing-feng QA: Fix llama4 leak ( #49042 ) by @tarekziade Fix startup failures: drop permissions reusable-workflow callers cannot grant ( #49055 ) by @paulinebm Fix startup failures: drop pull-requests: read from check_failed_tests.yml ( #49057 ) by @paulinebm Register the remaining mamba-ssm kernel layers on XPU ( #49035 ) by @jiqing-feng Add workflow for building XPU CI Docker images ( #49039 ) by @regisss Fix assistant masks for processors ( #48793 ) by @Rocketknight1 Add MPS maintainer ( #49052 ) by @Rocketknight1 [docs] Loading behavior ( #49023 ) by @stevhliu [Nemotron3Diarization] nit: hub pr merged to main ( #49065 ) by @eustlb fix(ci): harden GitHub Actions workflows ( #49057 ) ( #49059 ) by @hf-security-analysis[bot] Honor config.output_router_logits in the MoE VLM wrappers ( #48885 ) by @qgallouedec Fix XPU Docker image build ( #49073 ) by @regisss Fix assisted eos token condition ( #49075 ) by @Cyrilvallez [serge] Fix OOM in Moshi integration tests with MemoryCleanupMixin ( #48839 ) by @sergereview[bot] Fix odd head_dim validation for RoPE configurations ( #48524 ) by @somuai Keep already-decoded array arguments unchanged ( #49062 ) by @yonigozlan Fix NaN in Parakeet eager attention with padded batches ( #49070 ) by @ArthurZucker Bump huggingface_hub upper bound to <3.0 (transformers) ( #49083 ) by @Wauplin Keep attention_mask as None in OPT's causal mask creation ( #49002 ) by @jiqing-feng Add Trainer.end ( #48875 ) by @qgallouedec Add image processing tester init ( #48829 ) by @guarin [Executorch] Add MLX recipe ( #48910 ) by @metascroy [Zamba] Fix associative scan breaking ONNX export and OOM in integration test ( #49092 ) by @ydshieh Fix RGB early-return skipping PNG tRNS compositing ( #49005 ) by @cs-fisha Use namespaced dataset ids in docs and PyTorch examples ( #49017 ) by @davanstrien Fix MaskFormerSwin attention mask dtype to follow hidden states ( #49032 ) by @kaixuanliu [NemotronH-Omni] Fix device mismatch in test tensor creation ( #49102 ) by @ydshieh [serge] Fix 2 integration tests for model cvt failing with output_mismatch (tensor values differ (2)) ( #49076 ) by @sergereview[bot] [serge] Fix 2 integration tests for model hy_v3 failing with output_mismatch (tensor values differ (2)) ( #49068 ) by @sergereview[bot] [serge] Fix 2 integration tests for model pvt_v2 failing with other ( #49067 ) by @sergereview[bot] Keep the eos ids the config declares ( #49082 ) by @albertvillanova Use grouped_mm on TPU devices under torch.compile ( #49097 ) by @salkan0 [serge] Fix 2 integration tests for model jamba failing with other (other (2)) ( #49044 ) by @sergereview[bot] Remap the legacy Gemma 1 hidden_act in the config post-init ( #49084 ) by @PCfVW Fix exporters import on torch < 2.9 (is_contiguous_or_false) ( #49124 ) by @ydshieh [gemma3n] fix audio test fixture: use hf_hub_download instead of load_dataset ( #49128 ) by @ydshieh qwen3 models map to wrong tokenizer class on the hub ( #49116 ) by @itazap Restore the Unicode whitespace set in the GPT-SW3 tokenizer ( #48912 ) by @David-Wu1119 [AMD] Fix some integration tests ( #49153 ) by @Abdennacer-Badaoui [Gemma] Fix test_model_7b_fp16_static_cache expected value for cuda 8 after #49084 ( #49133 ) by @ydshieh Fix UMT5 decoder self-attention not being causal ( #49135 ) by @the-cross-art Skip flash tests that fall back to a hub kernel when kernels is missing ( #49129 ) by @Abdennacer-Badaoui Auto generate model inits ( #47829 ) by @guarin Fix get_json_schema dropping items/enum for unions of list/dict/Literal types ( #49136 ) by @JoeyTan21 Pick the default flash implementation based on the current hardware ( #49109 ) by @Abdennacer-Badaoui Deprecate the use_mamba_kernels config flag that no longer has any effect ( #49155 ) by @albertvillanova [CI] ssh-runner: add optional cache_type input to switch between bucket and EFS runners ( #49159 ) by @ydshieh [generation] Encode multimodal data only once ( #45783 ) by @zucchini-nlp No more -hf repo names for ESMC ( #49158 ) by @Rocketknight1 [MPS] Remove cu_seqlens_k clone workaround for metal-flash-sdpa ( #49091 ) by @Isalia20 [PerceptionLM] Restore test_inputs_embeds overrides to fix flaky test ( #49165 ) by @ydshieh [CB] Fix failing tests discovered when using the B200 ( #49171 ) by @remi-or [imagegpt/vilt/trocr] fix cache fixture tests: use hf_hub_download instead of load_dataset ( #49178 ) by @ydshieh Fix lost call ( #49163 ) by @molbap Document image_like_kwargs ( #49180 ) by @guarin [CacheHardIntegrationTest] use dedicated safetensors repo to fix Xet bucket cache corruption ( #49182 ) by @ydshieh Document running the example scripts on Hugging Face Jobs ( #49050 ) by @davanstrien Fix backslash handling in generate flag values in transformers chat ( #48709 ) by @JHC56 Finish removing the MPS autocast workaround ( #49157 ) by @rubenG1009 Fix Trainer checkpoint resume crashing on CPU with multiple processes ( #49123 ) by @neevmodh Fix command syntax for optimum-cli export ( #46451 ) by @Ahwar QA: fix noisy comment checker ( #49184 ) by @tarekziade update to torch 2.14 ( #49199 ) by @sywangyi Video processors - general maintenance ( #48251 ) by @zucchini-nlp [Parakeet] Convert NeMo's stochastic depth to layerdrop ( #49191 ) by @Deep-unlearning [MPS] Let mps sdpa handle grouped query attention directly ( #49187 ) by @Isalia20 Replace datasets that no longer load with maintained uploads ( #49018 ) by @davanstrien Fail fast on eval OOM under auto_find_batch_size ( #49198 ) by @qgallouedec Fix SequenceBiasLogitsProcessor edge cases: token id 0 and prefix equal to context ( #49117 ) by @lucaluo925 Map bare list, tuple and dict annotations to the right JSON schema type in get_json_schema ( #49145 ) by @825pranav [ Kernels ] Sync mamba version ( #49205 ) by @vasqu gguf user defined tokens ( #49008 ) by @SunMarc QA: restore masking_utils export comment with a noqa ( #49209 ) by @tarekziade Fix deepstack features for mixed-input ( #49177 ) by @zucchini-nlp Fix stale _added_tokens_encoder entries in cpmant and wav2vec2 ( #47440 ) by @ishan-1010 Fix additional_special_tokens data loss with extra_special_tokens ( #47848 ) by @erichanwang Fix MPS GQA version gating ( #49210 ) by @Isalia20 Add Strix Halo (gfx1151) Atlas Inference Hub-kernel path for Qwen3.5/3.6/3.8 Gated DeltaNet ( #49127 ) by @AzeezIsh Fix MiniMax M3 partial 3D vision rotary embeddings ( #49164 ) by @cuichenx Fix missing router_logits in Qwen3.5-MoE and other MoE models ( #49179 ) by @zucchini-nlp [Nemotron3Diarization] fix streaming last stft frame dropped ( #49167 ) by @eustlb Significant community contributions The following contributors have made significant changes to the library over the last release: @harshaljanjani model: Add GTE to Transformers ( #48416 ) [vLLM] Fix video token counting for Transformers backend video inputs (Part 2) ( #48900 ) 🚨 [vLLM] Fix video token counting for Transformers backend video inputs (Part 1) ( #48894 ) @eustlb [Nemotron3Diarization] fix streaming last stft frame dropped ( #49167 ) [Nemotron3Diarization] nit: hub pr merged to main ( #49065 ) Add Nemotron3Diarization ( #49056 ) [apply_chat_template] pass sampling_rate to call ( #48794 ) @tarekziade QA: restore masking_utils export comment with a noqa ( #49209 ) QA: fix noisy comment checker ( #49184 ) QA: Fix llama4 leak ( #49042 ) Shorten noisy OpenVINO SDPA comment ( #49033 ) Added more usage of MemoryCleanupMixin ( #49011 ) fix noisy comments ( #49013 ) updated GraniteMoeHybrid expectations (CPU) ( #49016 ) QA: applied ruff rule PLW1514 ( #48990 ) Contract the mamba2 chunk scan with einsum instead of broadcast-then-sum ( #48978 ) QA: deactivate rule 41 ( #48852 ) QA: Added a MemoryCleanupMixin class for tests ( #48681 ) Synthetic test assets ( #48589 ) @ydshieh [CacheHardIntegrationTest] use dedicated safetensors repo to fix Xet bucket cache corruption ( #49182 ) [imagegpt/vilt/trocr] fix cache fixture tests: use hf_hub_download instead of load_dataset ( #49178 ) [PerceptionLM] Restore test_inputs_embeds overrides to fix flaky test ( #49165 ) [CI] ssh-runner: add optional cache_type input to switch between bucket and EFS runners ( #49159 ) [Gemma] Fix test_model_7b_fp16_static_cache expected value for cuda 8 after #49084 ( #49133 ) [gemma3n] fix audio test fixture: use hf_hub_download instead of load_dataset ( #49128 ) Fix exporters import on torch < 2.9 (is_contiguous_or_false) ( #49124 ) [NemotronH-Omni] Fix device mismatch in test tensor creation ( #49102 ) [Zamba] Fix associative scan breaking ONNX export and OOM in integration test ( #49092 ) Fix Qwen3OmniMoeIntegrationTest OOM ( #48987 ) Reduce peak memory in examples_torch CI job (OOM fix) ( #48983 ) Fix AXK2 integration test: update CUDA expected text and rename class ( #48941 ) Clean up some MoE models' integration tests ( #48833 ) Mark test_generate_with_static_cache as flaky for olmo and bigbird_pegasus ( #48856 ) Fix GLM rope_parameters dict shared mutation across test instances ( #48895 ) Fix Glm4MoeIntegrationTest: split into 3 tests, switch to GLM-4.5-Air ( #48820 ) Fix flaky RfDetr test_save_load ( #48869 ) docs: fix broken #combining-with-fsdp2 anchor in expert_parallelism.md ( #48854 ) Relax static-cache tolerance in test_generate_with_static_cache (1e-5 → 5e-5) ( #48815 ) Add integration tests for MuseGlimmerAssistantModel ( #48796 ) Fix Glm4vMoeIntegrationTest: offload_folder + MemoryCleanupMixin ( #48776 ) Switch daily CI to torch 2.14 — update expected outputs ( #48750 ) [DeepseekV3] OOM cascade root-cause investigation (generator ref leak in conversion_mapping) ( #48720 ) [CI] Replace hardcoded username allowlists with author_association check in workflow triggers ( #48712 ) [MusicgenMelody] Fix conditioning silently dropped at generation step 0 ( #48679 ) [fix] Fix GlmOcr integration tests: wrong token IDs and image token decode bug ( #48650 ) [tests] Fix NougatModelIntegrationTest: pin artifact revision and update golden values ( #48638 ) [CI] Deduplicate Nvidia/AMD CI reply comments and add headers ( #48655 ) @molbap Fix lost call ( #49163 ) 🚨 🚨Bring some dinos to modern standards ( #46266 ) [Fix] yolos offload issue ( #48688 ) @remi-or [CB] Fix failing tests discovered when using the B200 ( #49171 ) [CB] [Major] Upgrade the cache to support different attention types ( #47809 ) [Improvement] Rework the docstring and comments of mHC ( #48888 ) [CB] Add pause mechanism ( #48462 ) [Fix] Clean-up ternaries in the DeepSeek family ( #48447 ) @jiqing-feng Keep attention_mask as None in OPT's causal mask creation ( #49002 ) Register the remaining mamba-ssm kernel layers on XPU ( #49035 ) Only seed numpy in BigBird block-sparse attention during training ( #49037 ) Fix mask creation not being skipped under torch.compile ( #48975 ) Fix StaticCache for Mllama and enable torch.compile ( #48141 ) Fix reset on the dynamic cache layers ( #48809 ) Cast pixel values to the patch embedding dtype in DeepSeek-OCR-2 ( #48632 ) Fix TextToAudioPipeline crash for tokenizer-only models ( #48505 ) Fix slow integration tests on XPU ( #48611 ) Register activation kernel layers on XPU ( #47858 ) @Wauplin Bump huggingface_hub upper bound to <3.0 (transformers) ( #49083 ) [docs] Fix legacy hf CLI references (transformers) ( #48988 ) Resolve the Hub revision once per load instead of passing a private _commit_hash around ( #47611 ) Use huggingface_hub httpx export ( #48685 ) @meatybobby Add support for Nemotron Omni ( #46509 ) @Sainava Add pose estimation keypoint preprocessing to Sapiens2ImageProcessor ( #47199 ) @jp1924 add HyperClovaX Vision ( #44314 )