vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

SimpleChat:Readme: Add quickstart block, title to image, cleanup

Browse source
This commit is contained in:
HanishKVC 2024-06-16 00:11:48 +05:30
parent daafaefaf1
commit 030c09d56a
1 changed files with 13 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

18
examples/server/public_simplechat/readme.md
View file

@ -3,6 +3,13 @@
by Humans for All. by Humans for All.
## quickstart
To run from the build dir
bin/llama-server -m path/model.gguf --path ../examples/server/public_simplechat
Continue reading for the details.
## overview ## overview
@ -14,7 +21,7 @@ own system prompts.
This allows seeing the generated text / ai-model response in oneshot at the end, after it is fully generated, This allows seeing the generated text / ai-model response in oneshot at the end, after it is fully generated,
or potentially as it is being generated, in a streamed manner from the server/ai-model. or potentially as it is being generated, in a streamed manner from the server/ai-model.
![Chat and Settings screens](./simplechat_screens.png) ![Chat and Settings screens](./simplechat_screens.png "Chat and Settings screens")
Auto saves the chat session locally as and when the chat is progressing and inturn at a later time when you Auto saves the chat session locally as and when the chat is progressing and inturn at a later time when you
open SimpleChat, option is provided to restore the old chat session, if a matching one exists. open SimpleChat, option is provided to restore the old chat session, if a matching one exists.
@ -183,10 +190,11 @@ It is attached to the document object. Some of these can also be updated using t
user at runtime by directly modifying gMe.chatRequestOptions, setting ui entries will be auto user at runtime by directly modifying gMe.chatRequestOptions, setting ui entries will be auto
created. created.
cache_prompt option supported by example/server is allowed to be controlled by user. So that cache_prompt option supported by example/server is allowed to be controlled by user, so that
any caching supported with system-prompt and chat history if usable can get used. If one has any caching supported wrt system-prompt and chat history, if usable can get used. When chat
enabled chat history sliding window, then the chat history caching may or maynot kick in at history sliding window is enabled, cache_prompt logic may or may not kick in at the backend
the backend, based on aspects related to positional encoding, attention mechanism etal. wrt same, based on aspects related to model, positional encoding, attention mechanism etal.
However system prompt should ideally get the benefit of caching.
headers - maintains the list of http headers sent when request is made to the server. By default headers - maintains the list of http headers sent when request is made to the server. By default
Content-Type is set to application/json. Additionally Authorization entry is provided, which can Content-Type is set to application/json. Additionally Authorization entry is provided, which can