vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

misc

Browse source
This commit is contained in:
Xuan Son Nguyen 2024-11-20 11:09:16 +01:00
parent 49cdfd3fc2
commit d7de41302b
2 changed files with 2 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

2
.devops/nix/python-scripts.nix
View file

@ -34,7 +34,7 @@ let
# server tests # server tests
openai openai
behave pytest
prometheus-client prometheus-client
]; ];
in in

12
examples/server/tests/README.md
View file

@ -1,19 +1,9 @@
# Server tests # Server tests
Python based server tests scenario using [BDD](https://en.wikipedia.org/wiki/Behavior-driven_development) Python based server tests scenario using [pytest](https://docs.pytest.org/en/stable/).
and [behave](https://behave.readthedocs.io/en/latest/):
* [issues.feature](./features/issues.feature) Pending issues scenario
* [parallel.feature](./features/parallel.feature) Scenario involving multi slots and concurrent requests
* [security.feature](./features/security.feature) Security, CORS and API Key
* [server.feature](./features/server.feature) Server base scenario: completion, embedding, tokenization, etc...
Tests target GitHub workflows job runners with 4 vCPU. Tests target GitHub workflows job runners with 4 vCPU.
Requests are
using [aiohttp](https://docs.aiohttp.org/en/stable/client_reference.html), [asyncio](https://docs.python.org/fr/3/library/asyncio.html)
based http client.
Note: If the host architecture inference speed is faster than GitHub runners one, parallel scenario may randomly fail. Note: If the host architecture inference speed is faster than GitHub runners one, parallel scenario may randomly fail.
To mitigate it, you can increase values in `n_predict`, `kv_size`. To mitigate it, you can increase values in `n_predict`, `kv_size`.