🛡️ SMG: Engine-Agnostic LLM Gateway in Rust

The direction here is clear: building a high-performance LLM gateway that is completely decoupled from the underlying inference engines. Written in Rust, SMG standardizes various backends—like vLLM, TRT-LLM, and SGLang—into fully compatible OpenAI and Anthropic API endpoints. The hard part is not simple API routing, but implementing deep performance optimizations at the gateway level. SMG tackles this by introducing a gRPC pipeline, KV cache-aware routing, and tokenization caching. It also packs in advanced features like WASM plugin support, MCP integration, and multi-tenant authentication, making it a very comprehensive piece of infrastructure. It is essentially exploring the next generation of AI deployment architecture. For engineering teams managing large-scale, self-hosted open-source models where throughput, latency, and resource efficiency are critical, this highly optimized gateway offers a compelling technical solution.