Best open-source local LLM command-line tools for personal use

Open-source runtimes for running LLMs on your own hardware.

Filters · 3 Personal, Local inference, CLI

For

Personal8Enterprise2

Purpose

AI & LLM tooling85Agents & frameworks29Local inference8RAG & vector DBs6Coding assistants41Evals & observability2Agent skills10MCP servers13Developer tools98Web frameworks5Databases & backends4Editors & IDEs6API tools2Self-hosted apps15Notes & docs8Automation10Data & analytics7Infra & DevOps21Security & identity23Communication1Media & photos8Design & creative2

Runs on

Web3Docker / self-host2Desktop6Mobile1CLI8Library / SDK7Extension / plugin0

License

Permissive7Copyleft0Source-available0Other / custom1

More

Has live demo0Commit in last 90 days8

8 projects

SortWorth scoreTrendingMost starsRecently updatedNewest in catalog

O

Ollamaollama/ollama

Run open-weight LLMs locally with one commandPersonalEnterpriseLocal inferenceDocker / self-hostDesktopCLI★ 182kMITcommit 21 hours ago

90worth

☆ Save

l

llama.cppggml-org/llama.cpp

LLM inference in C/C++ on CPUs and GPUsPersonalLocal inferenceCLILibrary / SDK★ 130kMITcommit 15 hours ago

86worth

☆ Save

w

whisper.cppggml-org/whisper.cpp

Whisper speech recognition in C/C++PersonalEnterpriseLocal inferenceMedia & photosCLILibrary / SDK★ 54kMITcommit yesterday

79worth

☆ Save

C

ColibriJustVugg/colibri

Run large MoE models on everyday hardwarePersonalLocal inferenceCLI★ 39kApache-2.0commit last week

79worth

☆ Save

l

llmfitAlexsJones/llmfit

Find which LLMs run on your hardwarePersonalLocal inferenceCLI★ 37kMITcommit 18 hours ago

79worth

☆ Save

l

llamafilemozilla-ai/llamafile

Run an LLM from a single filePersonalLocal inferenceCLI★ 26kcommit 3 days ago

71worth

☆ Save

e

exoexo-explore/exo

Run frontier AI models across your own devicesPersonalLocal inferenceCLI★ 48kApache-2.0commit last month

67worth

☆ Save

d

ds4antirez/ds4

Local DeepSeek 4 inference enginePersonalLocal inferenceCLI★ 23kMITcommit 2 weeks ago

63worth

☆ Save