vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

server: bench: fix doc

Browse source
This commit is contained in:
Pierrick HYMBERT 2024-03-09 01:23:52 +01:00
parent ab0a59d6d3
commit f425240e1d
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
examples/server/bench/README.md
View file

@ -53,7 +53,8 @@ The benchmark values can be overridden with:
- `SERVER_BENCH_URL` server url prefix for chat completions, default `http://localhost:8080/v1` - `SERVER_BENCH_URL` server url prefix for chat completions, default `http://localhost:8080/v1`
- `SERVER_BENCH_N_PROMPTS` total prompts to randomly select in the benchmark, default `480` - `SERVER_BENCH_N_PROMPTS` total prompts to randomly select in the benchmark, default `480`
- `SERVER_BENCH_MODEL_ALIAS` model alias to pass in the completion request, default `my-model` - `SERVER_BENCH_MODEL_ALIAS` model alias to pass in the completion request, default `my-model`
- `SERVER_BENCH_MAX_TOKENS` max tokens to predict, default: `1024` - `SERVER_BENCH_MAX_TOKENS` max tokens to predict, default: `512`
- `SERVER_BENCH_DATASET` path to the benchmark dataset file
Or with [k6 options](https://k6.io/docs/using-k6/k6-options/reference/): Or with [k6 options](https://k6.io/docs/using-k6/k6-options/reference/):