vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

server: tests: add infinite loop scenario

Browse source
This commit is contained in:
Pierrick HYMBERT 2024-02-20 23:11:59 +01:00
parent 6b9dc4f291
commit 68574c6f98
2 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

4
examples/server/tests/features/server.feature
View file

@ -42,7 +42,7 @@ Feature: llama.cpp server
""" """
Write another very long music lyrics. Write another very long music lyrics.
""" """
And 512 max tokens to predict And 256 max tokens to predict
Given concurrent completion requests Given concurrent completion requests
Then the server is busy Then the server is busy
And all slots are busy And all slots are busy
@ -62,7 +62,7 @@ Feature: llama.cpp server
""" """
Write another very long music lyrics. Write another very long music lyrics.
""" """
And 512 max tokens to predict And 256 max tokens to predict
And streaming is enabled And streaming is enabled
Given concurrent OAI completions requests Given concurrent OAI completions requests
Then the server is busy Then the server is busy

2
examples/server/tests/features/steps/steps.py
View file

@ -176,7 +176,7 @@ def oai_chat_completions(context, user_prompt):
model=context.model, model=context.model,
max_tokens=context.n_predict, max_tokens=context.n_predict,
stream=context.enable_streaming, stream=context.enable_streaming,
seed = context.seed seed=context.seed
) )
if context.enable_streaming: if context.enable_streaming:
completion_response = { completion_response = {