Assessing
Discovered Ollama — probably the simplest way to run an LLM on your own hardware. ollama run llama3 — and the model is already answering, the REST API comes up automatically, zero fiddling with dependencies and configs.
What wins me over is exactly this "Docker-like" approach to models: pull, run, list — familiar verbs, just with models instead of images. Still assessing for now: running lightweight models and seeing how far consumer hardware can go.