Best open-source local LLM command-line tools

Open-source runtimes for running LLMs on your own hardware.

8 projects

Ollamaollama/ollama

Run open-weight LLMs locally with one commandPersonal Enterprise Local inference Docker / self-host Desktop CLI★ 182kMITcommit 22 hours ago

90worth

llama.cppggml-org/llama.cpp

LLM inference in C/C++ on CPUs and GPUsPersonal Local inference CLI Library / SDK★ 130kMITcommit 16 hours ago

86worth

whisper.cppggml-org/whisper.cpp

Whisper speech recognition in C/C++Personal Enterprise Local inference Media & photos CLI Library / SDK★ 54kMITcommit yesterday

79worth

ColibriJustVugg/colibri

Run large MoE models on everyday hardwarePersonal Local inference CLI★ 39kApache-2.0commit last week

79worth

llmfitAlexsJones/llmfit

Find which LLMs run on your hardwarePersonal Local inference CLI★ 37kMITcommit 19 hours ago

79worth

llamafilemozilla-ai/llamafile

Run an LLM from a single filePersonal Local inference CLI★ 26kcommit 3 days ago

71worth

exoexo-explore/exo

Run frontier AI models across your own devicesPersonal Local inference CLI★ 48kApache-2.0commit last month

67worth

ds4antirez/ds4

Local DeepSeek 4 inference enginePersonal Local inference CLI★ 23kMITcommit 2 weeks ago

63worth

Narrow down

For