Chi Kim on Nostr: Llama.cpp now supports the distributed inference, meaning you can use multiple ...

Llama.cpp now supports the distributed inference, meaning you can use multiple computers to speed up the response time! Network is the main bottleneck, so all machines need to be hard wired, not connected through wifi. ##LLm #AI #ML https://github.com/ggerganov/llama.cpp/tree/master/examples/rpc