vLLM v0.31.1rc0 Enhances Metrics with Cached Prompt Token Exposure

Core summary

One sentence to understand this update

vLLM released v0.31.1rc0, introducing a new metric that exposes cached prompt tokens broken down by cache tier.

Impact & opportunity

What this could mean

This update provides developers with deeper insights into cache utilization, allowing for better optimization of LLM inference performance and resource management.