vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

SimpleChat: Save message internally in handle_response itself

Browse source 
This ensures that throwing the caught exception again for higher
up logic, doesnt lose the response collated till that time.

Go through theResp.assistant in catch block, just to keep simple
consistency wrt backtracing just in case.

Update the readme file.
This commit is contained in:
HanishKVC 2024-05-30 04:46:23 +05:30
parent cdb4f6d243
commit 872ee2c73d
2 changed files with 13 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

10
examples/server/public_simplechat/readme.md
View file

@ -108,6 +108,9 @@ Once inside
* Using NewChat one can start independent chat sessions.
* two independent chat sessions are setup by default.
* When you want to print, switching ChatHistoryInCtxt to Full and clicking on the chat session button of
interest, will display the full chat history till then wrt same, if you want full history for printing.
## Devel note
@ -135,6 +138,13 @@ It is attached to the document object. Some of these can also be updated using t
the logic assumes that the text sent from the server follows utf-8 encoding.
in streaming mode - if there is any exception, the logic traps the same and tries to ensure
that text generated till then is not lost.
if a very long text is being generated, which leads to no user interaction for sometime and
inturn the machine goes into power saving mode or so, the platform may stop network connection,
leading to exception.
apiEP - select between /completions and /chat/completions endpoint provided by the server/ai-model.
bCompletionFreshChatAlways - whether Completion mode collates complete/sliding-window history when

4
examples/server/public_simplechat/simplechat.js
View file

@ -428,6 +428,8 @@ class SimpleChat {
this.latestResponse = "";
} catch (error) {
theResp.assistant = this.latestResponse;
this.add(Roles.Assistant, theResp.assistant);
this.latestResponse = "";
throw error;
}
} else {
@ -438,6 +440,7 @@ class SimpleChat {
theResp.assistant = du.trim_garbage_at_end(origMsg);
theResp.trimmed = origMsg.substring(theResp.assistant.length);
}
this.add(Roles.Assistant, theResp.assistant);
return theResp;
}
@ -601,7 +604,6 @@ class MultiChatUI {
});
let theResp = await chat.handle_response(resp, apiEP, this.elDivChat);
chat.add(Roles.Assistant, theResp.assistant);
if (chatId == this.curChatId) {
chat.show(this.elDivChat);
if (theResp.trimmed.length > 0) {