Xiaomi Claims 1,000+ Tokens/Sec on 1T MoE Model with 8-GPU Server

Core summary

One sentence to understand this update

Xiaomi announced achieving over 1,000 tokens per second output on its 1 trillion parameter MoE model, MiMo-V2.5-Pro UltraSpeed, using a single standard 8-GPU server.

Impact & opportunity

What this could mean

This breakthrough in inference speed on large models using standard hardware presents a significant opportunity for developers to deploy high-performance LLMs more cost-effectively.