Back to list
High-Potential
C++

🛠️ Runanywhere SDKs: Local AI Inference Toolkit

10,297 stars380 forksC++
androidapple-intelligencecppdiffusion-modelsedgeflutterinferenceioskotlinllamacppllmmultimodal
This is a C++ toolkit focused on cross-platform local AI inference. Its goal is straightforward: to help developers deploy LLMs and diffusion models directly onto edge devices, with support for iOS, Android, Flutter, and Kotlin. While it clearly builds upon established inference engines like llama.cpp, its main value lies in providing production-ready SDK wrappers. This means mobile developers can integrate on-device AI capabilities without having to wrestle with low-level C++ compilation and memory management. As edge devices become more capable, running AI models locally is becoming a standard requirement. This project provides practical infrastructure for developers looking to reduce reliance on cloud APIs, lower latency, and improve user privacy.