Gemma 4 12B QAT MTP Achieves 120 tok/s on 12GB VRAM, Benchmarked by User

Core summary

One sentence to understand this update

A Reddit user benchmarked Google's new Gemma 4 12B QAT (Quantization-Aware Training) model, achieving an impressive 120 tokens/second on a 12GB VRAM GPU, fitting entirely in VRAM.

Impact & opportunity

What this could mean

This demonstrates significant efficiency gains for local LLM inference on consumer-grade GPUs, enabling builders to deploy powerful models with limited hardware resources.