Abstract
gguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf " MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in the MiniCPM-V series . It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include: 📱 The First Open-Source MoE Model with Lossless, Smooth On-Device Inference. With our Lookahead Expert Scheduling (LES) technique introduced alongside MiniCPM-V 4.7, combined with fine-grained layer-wise quantization and DSpark speculative sampling integration, MiniCPM-V 4.7 35B-A3B becomes the first open-source MoE model to run smoothly on-device without compromising performance , achieving a decoding speed of 13+ tokens/s on iPhone 17 Pro Max . 🔥 A New Pareto Frontier in Performance and Efficiency. As model capabilities continue to advance, inference efficiency has become an important measure of a model's value. On resource-constrained devices, delivering the strongest possible capabilities within a limited resource budget calls for higher capability density. The 35B-A3B variant of MiniCPM-V 4.7 advances the performance–efficiency Pareto frontier for MLLMs. In Instruct mode, 35B-A3B achieves a higher overall score in less inference time than Qwen3.6 35B-A3B . 🚀 Efficient Architecture. Based on the intra-ViT early compression technique from LLaVA-UHD v4 , MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports mixed 4x/16x visual token compression , allowing developers to balance visual detail and inference cost across tasks. 💻 Broad Platform and Chip Support. MiniCPM-V 4.7 supports iOS, Android, and HarmonyOS , together with open-source on-device deployment support. It also provides day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm , extending deployment options across mobile and local computing devices. 🛠️ Developer Friendly. MiniCPM-V 4.7 supports inference frameworks including SGLang, vLLM, llama.cpp, and Ollama, along with fine-tuning frameworks including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds FP8 quantization support . We also release Self-Replay Fine-Tuning , a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier." submitted by /u/Skyline34rGt to r/LocalLLaMA [link] [comments]