ncnn

Tencent/ncnn

Neural network inference for mobile

PersonalEnterpriseLocal inferenceLibrary / SDKOther / custom
github.com
Preview of ncnn
Preview of ncnn

About

Tencent's fast, dependency-free inference framework.

From the repository: “ncnn is a high-performance neural network inference framework optimized for the mobile platform”

llama.cppLLM inference in C/C++ on CPUs and GPUsCLILibrary / SDK86
vLLMHigh-throughput LLM serving engineDocker / self-hostLibrary / SDK86
UnslothFast local fine-tuning and running of LLMsWebLibrary / SDK81
SGLangFast serving framework for LLMsDocker / self-hostLibrary / SDK81

Topics