Xiaomi enters AI World with MiMo 7B: Monster that beats OpenAI

Xiaomi has made a giant leap in the field of artificial intelligence with the open-sourcing of its initial specialist reasoning model, Xiaomi MiMo. This historic 7B parameter model has already stirred the waters in the field of AI by outcompeting substantially larger rivals such as OpenAI’s closed-source o1-mini model and Alibaba’s larger QwQ-Preview model with 32B parameters. The model is a big win for Xiaomi’s newly created Big Model Core Team and is proof of the company’s increasing focus on pushing AI capabilities forward from hardware to innovative software development.

Breaking Benchmarks with Innovative Reasoning

What is so remarkable about MiMo is its high performance on difficult reasoning tests compared to its relatively small size. On publicly available mathematical reasoning (AIME 24-25) and code competition (LiveCodeBench v5) evaluation sets, the 7B parameter model beat out rivals with substantially higher parameter sizes.

The development group attributes their accomplishment to their innovative two-pronged strategy:

Pre-training Innovations

• Rich Reasoning Corpus: Emphasis is on extracting rich reasoning information
• Synthetic Enhancement: Generation of around 200B tokens of expert-level reasoning data
• Progressive Difficulty Training: Three separate phases of increasing difficulty of implementation
• Extensive Training: Total training over a staggering 25T tokens

Post-training Breakthroughs

The evolution of MiMo did not end with pre-training. Post-training innovations from the research team further improved the model’s capabilities:

1. Test Difficulty Driven Reward: An innovative method to tackle the sparsity of rewards in algorithmic tasks of complexity
2. Simple Data Re-sampling Approach: Application of methods to stabilize the training of reinforcement learning
3. Seamless Rollout System: An efficiency-oriented system that speeded up RL training by 2.29 times and verification by 1.96 times

What This Portends for Xiaomi’s Future in AI

The launch of MiMo marks Xiaomi’s serious play in the field of AI. While Xiaomi has established its credibility through hardware innovation, the move indicates a strategic shift towards cutting-edge research and development in AI. By open-sourcing the model itself, Xiaomi is also embracing the open and collective nature of AI innovation, potentially accelerating innovation in the industry.

Developers and enthusiasts of AI looking to experiment with or create extensions to MiMo can acquire the model from Xiaomi’s Hugging Face repository together with detailed technical documentation.

Source: Hugging Face, GitHub

HyperOS Downloader Easily check if your phone is eligible for HyperOS 2.0 update!