Show HN: Open-Source Load Balancer for Llama.cpp
11 by mcharytoniuk | 0 comments on Hacker News.
Stateful load balancer customized for llama.cpp (with a reverse proxy).