Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> so llama-swap can load

Just in case you haven't seen it yet, llama.cpp now has a router mode that lets you hot-swap models. I've switched over from llama-swap and have been happy with it.



I still use llama-swap as its configuration allows me to tailor llama.cpp settings per-model, set autoload timeouts, web ui log viewing, lots of great metrics, and load/unload model on a click. Llama-swap also technically lets you port-forward to some other app/service, like a remote service.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: