vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

readme : trim trailing whitespace

Browse source
This commit is contained in:
Radoslav Gerganov 2024-05-10 14:35:24 +03:00
parent 7975f43eb1
commit df54adabea
1 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

6
examples/rpc/README.md
View file

@ -1,6 +1,6 @@
## Overview ## Overview
The `rpc-server` allows running a `ggml` backend on a remote host. The `rpc-server` allows running a `ggml` backend on a remote host.
The RPC backend communicates with one or several instances of `rpc-server` and offloads computations to them. The RPC backend communicates with one or several instances of `rpc-server` and offloads computations to them.
This can be used for distributed LLM inference with `llama.cpp` in the following way: This can be used for distributed LLM inference with `llama.cpp` in the following way:
@ -37,9 +37,9 @@ cd build-rpc-cuda
cmake .. -DLLAMA_CUDA=ON -DLLAMA_RPC=ON cmake .. -DLLAMA_CUDA=ON -DLLAMA_RPC=ON
make -j make -j
``` ```
Then, start the `rpc-server` with the backend: Then, start the `rpc-server` with the backend:
```bash ```bash
$ bin/rpc-server 0.0.0.0 50052 $ bin/rpc-server 0.0.0.0 50052
create_backend: using CUDA backend create_backend: using CUDA backend