vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

SimpleChat: model request field for openai/equivalent compat

Browse source 
May help testing with openai/equivalent web services, if they
require this field.
This commit is contained in:
HanishKVC 2024-05-29 23:23:50 +05:30
parent 85fd2d0d84
commit 0e7880a694
2 changed files with 8 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
examples/server/public_simplechat/readme.md
View file

@ -30,7 +30,7 @@ NOTE: Wrt options sent with the request, it mainly sets temperature, max_tokens
However if someone wants they can update the js file or equivalent member in gMe as needed.
NOTE: One may be able to use this to chat with openai api web-service /chat/completions endpoint, in a very
limited / minimal way. One will need to set openai url and authorization bearer key in settings ui.
limited / minimal way. One will need to set model, openai url and authorization bearer key in settings ui.
## usage
@ -232,7 +232,7 @@ for a minimal chatting experimentation by setting the below.
* https://api.openai.com/v1 or similar
* Wrt request body - gMe.chatRequestOptions
* model
* model (settings ui)
* any additional fields if required in future
* Wrt request headers - gMe.headers

6
examples/server/public_simplechat/simplechat.js
View file

@ -699,6 +699,7 @@ class Me {
}
// Add needed fields wrt json object to be sent wrt LLM web services completions endpoint.
this.chatRequestOptions = {
"model": "gpt-3.5-turbo",
"temperature": 0.7,
"max_tokens": 1024,
"n_predict": 1024,
@ -763,6 +764,11 @@ class Me {
inp.el.placeholder = "Bearer OPENAI_API_KEY";
elDiv.appendChild(inp.div);
inp = ui.el_creatediv_input("SetModel", "Model", "text", this.chatRequestOptions["model"], (val)=>{
this.chatRequestOptions["model"] = val;
});
elDiv.appendChild(inp.div);
let bb = ui.el_creatediv_boolbutton("SetStream", "Stream", {true: "[+] yes stream", false: "[-] do oneshot"}, this.bStream, (val)=>{
this.bStream = val;
});