SimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548)

* SimpleChat:DU:BringIn local helper js modules using importmap Use it to bring in a simple trim garbage at end logic, which is used to trim received response. Also given that importmap assumes esm / standard js modules, so also global variables arent implicitly available outside the modules. So add it has a member of document for now * SimpleChat:DU: Add trim garbage at end in loop helper * SimpleChat:DU:TrimGarbage if unable try skip char and retry * SimpleChat:DU: Try trim using histogram based info TODO: May have to add max number of uniq chars in histogram at end of learning phase. * SimpleChat:DU: Switch trim garbage hist based to maxUniq simple Instead of blindly building histogram for specified substring length, and then checking if any new char within specified min garbage length limit, NOW exit learn state when specified maxUniq chars are found. Inturn there should be no new chars with in the specified min garbage length required limit. TODO: Need to track char classes like alphabets, numerals and special/other chars. * SimpleChat:DU: Bring in maxType to the mix along with maxUniq Allow for more uniq chars, but then ensure that a given type of char ie numerals or alphabets or other types dont cross the specified maxType limit. This allows intermixed text garbage to be identified and trimmed. * SimpleChat:DU: Cleanup debug log messages * SimpleChat:UI: Move html ui base helpers into its own module * SimpleChat:DU:Avoid setting frequence/Presence penalty Some models like llama3 found to try to be over intelligent by repeating garbage still, but by tweaking the garbage a bit so that it is not exactly same. So avoid setting these penalties and let the model's default behaviour work out, as is. Also the simple minded histogram based garbage trimming from end, works to an extent, when the garbage is more predictable and repeatative. * SimpleChat:UI: Add and use a para-create-append helper Also update the config params dump to indicate that now one needs to use document to get hold of gMe global object, this is bcas of moving to module type js. Also add ui.mjs to importmap * SimpleChat:UI: Helper to create bool button and use it wrt settings * SimpleChat:UI: Add Select helper and use it wrt ChatHistoryInCtxt * SimpleChat:UI:Select: dict-name-value, value wrt default, change Take a dict/object of name-value pairs instead of just names. Inturn specify the actual value wrt default, rather than the string representing that value. Trap the needed change event rather than click wrt select. * SimpleChat:UI: Add Div wrapped label+element helpers Move settings related elements to use the new div wrapped ones. * SimpleChat:UI:Add settings button and bring in settings ui * SimpleChat:UI:Settings make boolean button text show meaning * SimpleChat: Update a bit wrt readme and notes in du * SimpleChat: GarbageTrim enable/disable, show trimmed part ifany * SimpleChat: highlight trim, garbage trimming bitmore aggressive Make it easy for end user to identified the trimmed text. Make garbage trimming logic, consider a longer repeat garbage substring. * SimpleChat: Cleanup a bit wrt Api end point related flow Consolidate many of the Api end point related basic meta data into ApiEP class. Remove the hardcoded ApiEP/Mode settings from html+js, instead use the generic select helper logic, inturn in the settings block. Move helper to generate the appropriate request json string based on ApiEP into SimpleChat class itself. * SimpleChat:Move extracting assistant response to SimpleChat class so also the trimming of garbage. * SimpleChat:DU: Bring in both trim garbage logics to try trim * SimpleChat: Cleanup readme a bit, add one more chathistory length * SimpleChat:Stream:Initial handshake skeleton Parse the got stream responses and try extract the data from it. It allows for a part read to get a single data line or multiple data line. Inturn extract the json body and inturn the delta content/message in it. * SimpleChat: Move handling oneshot mode server response Move handling of the oneshot mode server response into SimpleChat. Also add plumbing for moving multipart server response into same. * SimpleChat: Move multi part server response handling in * SimpleChat: Add MultiPart Response handling, common trimming Add logic to call into multipart/stream server response handling. Move trimming of garbage at the end into the common handle_response helper. Add new global flag to control between oneshot and multipart/stream mode of fetching response. Allow same to be controlled by user. If in multipart/stream mode, send the stream flag to the server. * SimpleChat: show streamed generative text as it becomes available Now that the extracting of streamed generated text is implemented, add logic to show the same on the screen. * SimpleChat:DU: Add NewLines helper class To work with an array of new lines. Allow adding, appending, shifting, ... * SimpleChat:DU: Make NewLines shift more robust and flexible * SimpleChat:HandleResponseMultiPart using NewLines helper Make handle_response_multipart logic better and cleaner. Now it allows for working with the situation, where the delta data line got from server in stream mode, could be split up when recving, but still the logic will handle it appropriately. ALERT: Rather except (for now) for last data line wrt a request's response. * SimpleChat: Disable console debug by default by making it dummy Parallely save a reference to the original func. * SimpleChat:MultiPart/Stream flow cleanup Dont try utf8-decode and newlines-add_append if no data to work on. If there is no more data to get (ie done is set), then let NewLines instance return line without newline at end, So that we dont miss out on any last-data-line without newline kind of scenario. Pass stream flag wrt utf-8 decode, so that if any multi-byte char is only partly present in the passed buffer, it can be accounted for along with subsequent buffer. At sametime, bcas of utf-8's characteristics there shouldnt be any unaccounted bytes at end, for valid block of utf8 data split across chunks, so not bothering calling with stream set to false at end. LATER: Look at TextDecoder's implementation, for any over intelligence, it may be doing.. If needed, one can use done flag to account wrt both cases. * SimpleChat: Move baseUrl to Me and inturn gMe This should allow easy updating of the base url at runtime by the end user. * SimpleChat:UI: Add input element helper * SimpleChat: Add support for changing the base url This ensures that if the user is running the server with a different port or wants to try connect to server on a different machine, then this can be used. * SimpleChat: Move request headers into Me and gMe Inturn allow Authorization to be sent, if not empty. * SimpleChat: Rather need to use append to insert headers * SimpleChat: Allow Authorization header to be set by end user * SimpleChat:UI+: Return div and element wrt creatediv helpers use it to set placeholder wrt Authorization header. Also fix copy-paste oversight. * SimpleChat: readme wrt authorization, maybe minimal openai testing * SimpleChat: model request field for openai/equivalent compat May help testing with openai/equivalent web services, if they require this field. * SimpleChat: readme stream-utf-8 trim-english deps, exception2error * Readme: Add a entry for simplechat in the http server section * SimpleChat:WIP:Collate internally, Stream mode Trap exceptions This can help ensure that data fetched till that point, can be made use of, rather than losing it. On some platforms, the time taken wrt generating a long response, may lead to the network connection being broken when it enters some user-no-interaction related power saving mode. * SimpleChat:theResp-origMsg: Undo a prev change to fix non trim When the response handling was moved into SimpleChat, I had changed a flow bit unnecessarily and carelessly, which resulted in the non trim flow, missing out on retaining the ai assistant response. This has been fixed now. * SimpleChat: Save message internally in handle_response itself This ensures that throwing the caught exception again for higher up logic, doesnt lose the response collated till that time. Go through theResp.assistant in catch block, just to keep simple consistency wrt backtracing just in case. Update the readme file. * SimpleChat:Cleanup: Add spacing wrt shown req-options * SimpleChat:UI: CreateDiv Divs map to GridX2 class This allows the settings ui to be cleaner structured. * SimpleChat: Show Non SettingsUI config field by default * SimpleChat: Allow for multiline system prompt Convert SystemPrompt into a textarea with 2 rows. Reduce user-input-textarea to 2 rows from 3, so that overall vertical space usage remains same. Shorten usage messages a bit, cleanup to sync with settings ui. * SimpleChat: Add basic skeleton for saving and loading chat Inturn when ever a chat message (system/user/model) is added, the chat will be saved into browser's localStorage. * SimpleChat:ODS: Add a prefix to chatid wrt ondiskstorage key * SimpleChat:ODS:WIP:TMP: Add UI to load previously saved chat This is a temporary flow * SimpleChat:ODS:Move restore/load saved chat btn setup to Me This also allows being able to set the common system prompt ui element to loaded chat's system prompt. * SimpleChat:Readme updated wrt save and restore chat session info * SimpleChat:Show chat session restore button, only if saved session * SimpleChat: AutoCreate ChatRequestOptions settings to an extent * SimpleChat: Update main README wrt usage with server
2024-06-01 21:50:18 +05:30 · 2024-06-01 21:50:18 +05:30 · 2ac95c9d56
commit 2ac95c9d56
parent 750f60c03e
7 changed files with 1029 additions and 175 deletions
--- a/examples/server/public_simplechat/simplechat.js
+++ b/examples/server/public_simplechat/simplechat.js
@ -2,6 +2,9 @@
 // A simple completions and chat/completions test related web front end logic
 // by Humans for All

+import * as du from "./datautils.mjs";
+import * as ui from "./ui.mjs"
+
 class Roles {
    static System = "system";
    static User = "user";
@ -9,40 +12,65 @@ class Roles {
 }

 class ApiEP {
-    static Chat = "chat";
-    static Completion = "completion";
+    static Type = {
+        Chat: "chat",
+        Completion: "completion",
+    }
+    static UrlSuffix = {
+        'chat': `/chat/completions`,
+        'completion': `/completions`,
+    }
+
+    /**
+     * Build the url from given baseUrl and apiEp id.
+     * @param {string} baseUrl
+     * @param {string} apiEP
+     */
+    static Url(baseUrl, apiEP) {
+        if (baseUrl.endsWith("/")) {
+            baseUrl = baseUrl.substring(0, baseUrl.length-1);
+        }
+        return `${baseUrl}${this.UrlSuffix[apiEP]}`;
+    }
+
 }

+
 let gUsageMsg = `
    <p class="role-system">Usage</p>
    <ul class="ul1">
-    <li> Set system prompt above, to try control ai response charactersitic, if model supports same.</li>
+    <li> System prompt above, to try control ai response characteristics.</li>
        <ul class="ul2">
-        <li> Completion mode normally wont have a system prompt.</li>
+        <li> Completion mode - no system prompt normally.</li>
        </ul>
+    <li> Use shift+enter for inserting enter/newline.</li>
    <li> Enter your query to ai assistant below.</li>
-        <ul class="ul2">
-        <li> Completion mode doesnt insert user/role: prefix implicitly.</li>
-        <li> Use shift+enter for inserting enter/newline.</li>
-        </ul>
    <li> Default ContextWindow = [System, Last Query+Resp, Cur Query].</li>
        <ul class="ul2">
-        <li> experiment iRecentUserMsgCnt, max_tokens, model ctxt window to expand</li>
+        <li> ChatHistInCtxt, MaxTokens, ModelCtxt window to expand</li>
        </ul>
    </ul>
 `;

+
 /** @typedef {{role: string, content: string}[]} ChatMessages */

+/** @typedef {{iLastSys: number, xchat: ChatMessages}} SimpleChatODS */
+
 class SimpleChat {

-    constructor() {
+    /**
+     * @param {string} chatId
+     */
+    constructor(chatId) {
+        this.chatId = chatId;
        /**
         * Maintain in a form suitable for common LLM web service chat/completions' messages entry
         * @type {ChatMessages}
         */
        this.xchat = [];
        this.iLastSys = -1;
+        this.latestResponse = "";
    }

    clear() {
@ -50,6 +78,27 @@ class SimpleChat {
        this.iLastSys = -1;
    }

+    ods_key() {
+        return `SimpleChat-${this.chatId}`
+    }
+
+    save() {
+        /** @type {SimpleChatODS} */
+        let ods = {iLastSys: this.iLastSys, xchat: this.xchat};
+        localStorage.setItem(this.ods_key(), JSON.stringify(ods));
+    }
+
+    load() {
+        let sods = localStorage.getItem(this.ods_key());
+        if (sods == null) {
+            return;
+        }
+        /** @type {SimpleChatODS} */
+        let ods = JSON.parse(sods);
+        this.iLastSys = ods.iLastSys;
+        this.xchat = ods.xchat;
+    }
+
    /**
     * Recent chat messages.
     * If iRecentUserMsgCnt < 0
@ -94,6 +143,15 @@ class SimpleChat {
        return rchat;
    }

+    /**
+     * Collate the latest response from the server/ai-model, as it is becoming available.
+     * This is mainly useful for the stream mode.
+     * @param {string} content
+     */
+    append_response(content) {
+        this.latestResponse += content;
+    }
+
    /**
     * Add an entry into xchat
     * @param {string} role
@ -107,6 +165,7 @@ class SimpleChat {
        if (role == Roles.System) {
            this.iLastSys = this.xchat.length - 1;
        }
+        this.save();
        return true;
    }

@ -121,10 +180,8 @@ class SimpleChat {
        }
        let last = undefined;
        for(const x of this.recent_chat(gMe.iRecentUserMsgCnt)) {
-            let entry = document.createElement("p");
+            let entry = ui.el_create_append_p(`${x.role}: ${x.content}`, div);
            entry.className = `role-${x.role}`;
-            entry.innerText = `${x.role}: ${x.content}`;
-            div.appendChild(entry);
            last = entry;
        }
        if (last !== undefined) {
@ -132,21 +189,45 @@ class SimpleChat {
        } else {
            if (bClear) {
                div.innerHTML = gUsageMsg;
+                gMe.setup_load(div, this);
                gMe.show_info(div);
            }
        }
+        return last;
+    }
+
+    /**
+     * Setup the fetch headers.
+     * It picks the headers from gMe.headers.
+     * It inserts Authorization only if its non-empty.
+     * @param {string} apiEP
+     */
+    fetch_headers(apiEP) {
+        let headers = new Headers();
+        for(let k in gMe.headers) {
+            let v = gMe.headers[k];
+            if ((k == "Authorization") && (v.trim() == "")) {
+                continue;
+            }
+            headers.append(k, v);
+        }
+        return headers;
    }

    /**
     * Add needed fields wrt json object to be sent wrt LLM web services completions endpoint.
     * The needed fields/options are picked from a global object.
+     * Add optional stream flag, if required.
     * Convert the json into string.
     * @param {Object} obj
     */
-    request_jsonstr(obj) {
+    request_jsonstr_extend(obj) {
        for(let k in gMe.chatRequestOptions) {
            obj[k] = gMe.chatRequestOptions[k];
        }
+        if (gMe.bStream) {
+            obj["stream"] = true;
+        }
        return JSON.stringify(obj);
    }

@ -157,7 +238,7 @@ class SimpleChat {
        let req = {
            messages: this.recent_chat(gMe.iRecentUserMsgCnt),
        }
-        return this.request_jsonstr(req);
+        return this.request_jsonstr_extend(req);
    }

    /**
@ -180,7 +261,60 @@ class SimpleChat {
        let req = {
            prompt: prompt,
        }
-        return this.request_jsonstr(req);
+        return this.request_jsonstr_extend(req);
+    }
+
+    /**
+     * Return a string form of json object suitable for specified api endpoint.
+     * @param {string} apiEP
+     */
+    request_jsonstr(apiEP) {
+        if (apiEP == ApiEP.Type.Chat) {
+            return this.request_messages_jsonstr();
+        } else {
+            return this.request_prompt_jsonstr(gMe.bCompletionInsertStandardRolePrefix);
+        }
+    }
+
+    /**
+     * Extract the ai-model/assistant's response from the http response got.
+     * Optionally trim the message wrt any garbage at the end.
+     * @param {any} respBody
+     * @param {string} apiEP
+     */
+    response_extract(respBody, apiEP) {
+        let assistant = "";
+        if (apiEP == ApiEP.Type.Chat) {
+            assistant = respBody["choices"][0]["message"]["content"];
+        } else {
+            try {
+                assistant = respBody["choices"][0]["text"];
+            } catch {
+                assistant = respBody["content"];
+            }
+        }
+        return assistant;
+    }
+
+    /**
+     * Extract the ai-model/assistant's response from the http response got in streaming mode.
+     * @param {any} respBody
+     * @param {string} apiEP
+     */
+    response_extract_stream(respBody, apiEP) {
+        let assistant = "";
+        if (apiEP == ApiEP.Type.Chat) {
+            if (respBody["choices"][0]["finish_reason"] !== "stop") {
+                assistant = respBody["choices"][0]["delta"]["content"];
+            }
+        } else {
+            try {
+                assistant = respBody["choices"][0]["text"];
+            } catch {
+                assistant = respBody["content"];
+            }
+        }
+        return assistant;
    }

    /**
@ -239,53 +373,99 @@ class SimpleChat {
        return sysPrompt;
    }

-}

-
-let gBaseURL = "http://127.0.0.1:8080";
-let gChatURL = {
-    'chat': `${gBaseURL}/chat/completions`,
-    'completion': `${gBaseURL}/completions`,
-}
-
-
-/**
- * Set the class of the children, based on whether it is the idSelected or not.
- * @param {HTMLDivElement} elBase
- * @param {string} idSelected
- * @param {string} classSelected
- * @param {string} classUnSelected
- */
-function el_children_config_class(elBase, idSelected, classSelected, classUnSelected="") {
-    for(let child of elBase.children) {
-        if (child.id == idSelected) {
-            child.className = classSelected;
-        } else {
-            child.className = classUnSelected;
+    /**
+     * Handle the multipart response from server/ai-model
+     * @param {Response} resp
+     * @param {string} apiEP
+     * @param {HTMLDivElement} elDiv
+     */
+    async handle_response_multipart(resp, apiEP, elDiv) {
+        let elP = ui.el_create_append_p("", elDiv);
+        if (!resp.body) {
+            throw Error("ERRR:SimpleChat:SC:HandleResponseMultiPart:No body...");
        }
+        let tdUtf8 = new TextDecoder("utf-8");
+        let rr = resp.body.getReader();
+        this.latestResponse = "";
+        let xLines = new du.NewLines();
+        while(true) {
+            let { value: cur,  done: done } = await rr.read();
+            if (cur) {
+                let curBody = tdUtf8.decode(cur, {stream: true});
+                console.debug("DBUG:SC:PART:Str:", curBody);
+                xLines.add_append(curBody);
+            }
+            while(true) {
+                let curLine = xLines.shift(!done);
+                if (curLine == undefined) {
+                    break;
+                }
+                if (curLine.trim() == "") {
+                    continue;
+                }
+                if (curLine.startsWith("data:")) {
+                    curLine = curLine.substring(5);
+                }
+                let curJson = JSON.parse(curLine);
+                console.debug("DBUG:SC:PART:Json:", curJson);
+                this.append_response(this.response_extract_stream(curJson, apiEP));
+            }
+            elP.innerText = this.latestResponse;
+            elP.scrollIntoView(false);
+            if (done) {
+                break;
+            }
+        }
+        console.debug("DBUG:SC:PART:Full:", this.latestResponse);
+        return this.latestResponse;
    }
-}

-/**
- * Create button and set it up.
- * @param {string} id
- * @param {(this: HTMLButtonElement, ev: MouseEvent) => any} callback
- * @param {string | undefined} name
- * @param {string | undefined} innerText
- */
-function el_create_button(id, callback, name=undefined, innerText=undefined) {
-    if (!name) {
-        name = id;
+    /**
+     * Handle the oneshot response from server/ai-model
+     * @param {Response} resp
+     * @param {string} apiEP
+     */
+    async handle_response_oneshot(resp, apiEP) {
+        let respBody = await resp.json();
+        console.debug(`DBUG:SimpleChat:SC:${this.chatId}:HandleUserSubmit:RespBody:${JSON.stringify(respBody)}`);
+        return this.response_extract(respBody, apiEP);
    }
-    if (!innerText) {
-        innerText = id;
+
+    /**
+     * Handle the response from the server be it in oneshot or multipart/stream mode.
+     * Also take care of the optional garbage trimming.
+     * @param {Response} resp
+     * @param {string} apiEP
+     * @param {HTMLDivElement} elDiv
+     */
+    async handle_response(resp, apiEP, elDiv) {
+        let theResp = {
+            assistant: "",
+            trimmed: "",
+        }
+        if (gMe.bStream) {
+            try {
+                theResp.assistant = await this.handle_response_multipart(resp, apiEP, elDiv);
+                this.latestResponse = "";
+            } catch (error) {
+                theResp.assistant = this.latestResponse;
+                this.add(Roles.Assistant, theResp.assistant);
+                this.latestResponse = "";
+                throw error;
+            }
+        } else {
+            theResp.assistant = await this.handle_response_oneshot(resp, apiEP);
+        }
+        if (gMe.bTrimGarbage) {
+            let origMsg = theResp.assistant;
+            theResp.assistant = du.trim_garbage_at_end(origMsg);
+            theResp.trimmed = origMsg.substring(theResp.assistant.length);
+        }
+        this.add(Roles.Assistant, theResp.assistant);
+        return theResp;
    }
-    let btn = document.createElement("button");
-    btn.id = id;
-    btn.name = name;
-    btn.innerText = innerText;
-    btn.addEventListener("click", callback);
-    return btn;
+
 }


@ -302,14 +482,16 @@ class MultiChatUI {
        this.elDivChat = /** @type{HTMLDivElement} */(document.getElementById("chat-div"));
        this.elBtnUser = /** @type{HTMLButtonElement} */(document.getElementById("user-btn"));
        this.elInUser = /** @type{HTMLInputElement} */(document.getElementById("user-in"));
-        this.elSelectApiEP = /** @type{HTMLSelectElement} */(document.getElementById("api-ep"));
+        this.elDivHeading = /** @type{HTMLSelectElement} */(document.getElementById("heading"));
        this.elDivSessions = /** @type{HTMLDivElement} */(document.getElementById("sessions-div"));
+        this.elBtnSettings = /** @type{HTMLButtonElement} */(document.getElementById("settings"));

        this.validate_element(this.elInSystem, "system-in");
        this.validate_element(this.elDivChat, "chat-div");
        this.validate_element(this.elInUser, "user-in");
-        this.validate_element(this.elSelectApiEP, "api-ep");
+        this.validate_element(this.elDivHeading, "heading");
        this.validate_element(this.elDivChat, "sessions-div");
+        this.validate_element(this.elBtnSettings, "settings");
    }

    /**
@ -350,13 +532,18 @@ class MultiChatUI {
            this.handle_session_switch(this.curChatId);
        }

+        this.elBtnSettings.addEventListener("click", (ev)=>{
+            this.elDivChat.replaceChildren();
+            gMe.show_settings(this.elDivChat);
+        });
+
        this.elBtnUser.addEventListener("click", (ev)=>{
            if (this.elInUser.disabled) {
                return;
            }
-            this.handle_user_submit(this.curChatId, this.elSelectApiEP.value).catch((/** @type{Error} */reason)=>{
+            this.handle_user_submit(this.curChatId, gMe.apiEP).catch((/** @type{Error} */reason)=>{
                let msg = `ERRR:SimpleChat\nMCUI:HandleUserSubmit:${this.curChatId}\n${reason.name}:${reason.message}`;
-                console.debug(msg.replace("\n", ":"));
+                console.error(msg.replace("\n", ":"));
                alert(msg);
                this.ui_reset_userinput();
            });
@ -377,6 +564,8 @@ class MultiChatUI {
            // allow user to insert enter into the system prompt using shift+enter.
            // while just pressing enter key will lead to setting the system prompt.
            if ((ev.key === "Enter") && (!ev.shiftKey)) {
+                let value = this.elInSystem.value;
+                this.elInSystem.value = value.substring(0,value.length-1);
                let chat = this.simpleChats[this.curChatId];
                chat.add_system_anytime(this.elInSystem.value, this.curChatId);
                chat.show(this.elDivChat);
@ -392,34 +581,12 @@ class MultiChatUI {
     * @param {boolean} bSwitchSession
     */
    new_chat_session(chatId, bSwitchSession=false) {
-        this.simpleChats[chatId] = new SimpleChat();
+        this.simpleChats[chatId] = new SimpleChat(chatId);
        if (bSwitchSession) {
            this.handle_session_switch(chatId);
        }
    }

-    /**
-     * Try read json response early, if available.
-     * @param {Response} resp
-     */
-    async read_json_early(resp) {
-        if (!resp.body) {
-            throw Error("ERRR:SimpleChat:MCUI:ReadJsonEarly:No body...");
-        }
-        let tdUtf8 = new TextDecoder("utf-8");
-        let rr = resp.body.getReader();
-        let gotBody = "";
-        while(true) {
-            let { value: cur,  done: done} = await rr.read();
-            let curBody = tdUtf8.decode(cur);
-            console.debug("DBUG:SC:PART:", curBody);
-            gotBody += curBody;
-            if (done) {
-                break;
-            }
-        }
-        return JSON.parse(gotBody);
-    }

    /**
     * Handle user query submit request, wrt specified chat session.
@ -434,7 +601,7 @@ class MultiChatUI {
        // So if user wants to simulate a multi-chat based completion query,
        // they will have to enter the full thing, as a suitable multiline
        // user input/query.
-        if ((apiEP == ApiEP.Completion) && (gMe.bCompletionFreshChatAlways)) {
+        if ((apiEP == ApiEP.Type.Completion) && (gMe.bCompletionFreshChatAlways)) {
            chat.clear();
        }

@ -447,41 +614,26 @@ class MultiChatUI {
        }
        chat.show(this.elDivChat);

-        let theBody;
-        let theUrl = gChatURL[apiEP]
-        if (apiEP == ApiEP.Chat) {
-            theBody = chat.request_messages_jsonstr();
-        } else {
-            theBody = chat.request_prompt_jsonstr(gMe.bCompletionInsertStandardRolePrefix);
-        }
+        let theUrl = ApiEP.Url(gMe.baseURL, apiEP);
+        let theBody = chat.request_jsonstr(apiEP);

        this.elInUser.value = "working...";
        this.elInUser.disabled = true;
        console.debug(`DBUG:SimpleChat:MCUI:${chatId}:HandleUserSubmit:${theUrl}:ReqBody:${theBody}`);
+        let theHeaders = chat.fetch_headers(apiEP);
        let resp = await fetch(theUrl, {
            method: "POST",
-            headers: {
-                "Content-Type": "application/json",
-            },
+            headers: theHeaders,
            body: theBody,
        });

-        let respBody = await resp.json();
-        //let respBody = await this.read_json_early(resp);
-        console.debug(`DBUG:SimpleChat:MCUI:${chatId}:HandleUserSubmit:RespBody:${JSON.stringify(respBody)}`);
-        let assistantMsg;
-        if (apiEP == ApiEP.Chat) {
-            assistantMsg = respBody["choices"][0]["message"]["content"];
-        } else {
-            try {
-                assistantMsg = respBody["choices"][0]["text"];
-            } catch {
-                assistantMsg = respBody["content"];
-            }
-        }
-        chat.add(Roles.Assistant, assistantMsg);
+        let theResp = await chat.handle_response(resp, apiEP, this.elDivChat);
        if (chatId == this.curChatId) {
            chat.show(this.elDivChat);
+            if (theResp.trimmed.length > 0) {
+                let p = ui.el_create_append_p(`TRIMMED:${theResp.trimmed}`, this.elDivChat);
+                p.className="role-trim";
+            }
        } else {
            console.debug(`DBUG:SimpleChat:MCUI:HandleUserSubmit:ChatId has changed:[${chatId}] [${this.curChatId}]`);
        }
@ -500,7 +652,7 @@ class MultiChatUI {
        }
        elDiv.replaceChildren();
        // Btn for creating new chat session
-        let btnNew = el_create_button("New CHAT", (ev)=> {
+        let btnNew = ui.el_create_button("New CHAT", (ev)=> {
            if (this.elInUser.disabled) {
                console.error(`ERRR:SimpleChat:MCUI:NewChat:Current session [${this.curChatId}] awaiting response, ignoring request...`);
                alert("ERRR:SimpleChat\nMCUI:NewChat\nWait for response to pending query, before starting new chat session");
@ -514,7 +666,7 @@ class MultiChatUI {
            }
            this.new_chat_session(chatIdGot, true);
            this.create_session_btn(elDiv, chatIdGot);
-            el_children_config_class(elDiv, chatIdGot, "session-selected", "");
+            ui.el_children_config_class(elDiv, chatIdGot, "session-selected", "");
        });
        elDiv.appendChild(btnNew);
        // Btns for existing chat sessions
@ -528,7 +680,7 @@ class MultiChatUI {
    }

    create_session_btn(elDiv, cid) {
-        let btn = el_create_button(cid, (ev)=>{
+        let btn = ui.el_create_button(cid, (ev)=>{
            let target = /** @type{HTMLButtonElement} */(ev.target);
            console.debug(`DBUG:SimpleChat:MCUI:SessionClick:${target.id}`);
            if (this.elInUser.disabled) {
@ -537,7 +689,7 @@ class MultiChatUI {
                return;
            }
            this.handle_session_switch(target.id);
-            el_children_config_class(elDiv, target.id, "session-selected", "");
+            ui.el_children_config_class(elDiv, target.id, "session-selected", "");
        });
        elDiv.appendChild(btn);
        return btn;
@ -567,46 +719,183 @@ class MultiChatUI {
 class Me {

    constructor() {
+        this.baseURL = "http://127.0.0.1:8080";
        this.defaultChatIds = [ "Default", "Other" ];
        this.multiChat = new MultiChatUI();
+        this.bStream = true;
        this.bCompletionFreshChatAlways = true;
        this.bCompletionInsertStandardRolePrefix = false;
+        this.bTrimGarbage = true;
        this.iRecentUserMsgCnt = 2;
+        this.sRecentUserMsgCnt = {
+            "Full": -1,
+            "Last0": 1,
+            "Last1": 2,
+            "Last2": 3,
+            "Last4": 5,
+        };
+        this.apiEP = ApiEP.Type.Chat;
+        this.headers = {
+            "Content-Type": "application/json",
+            "Authorization": "", // Authorization: Bearer OPENAI_API_KEY
+        }
        // Add needed fields wrt json object to be sent wrt LLM web services completions endpoint.
        this.chatRequestOptions = {
+            "model": "gpt-3.5-turbo",
            "temperature": 0.7,
            "max_tokens": 1024,
-            "frequency_penalty": 1.2,
-            "presence_penalty": 1.2,
-            "n_predict": 1024
+            "n_predict": 1024,
+            //"frequency_penalty": 1.2,
+            //"presence_penalty": 1.2,
        };
    }

    /**
+     * Disable console.debug by mapping it to a empty function.
+     */
+    debug_disable() {
+        this.console_debug = console.debug;
+        console.debug = () => {
+
+        };
+    }
+
+    /**
+     * Setup the load saved chat ui.
+     * @param {HTMLDivElement} div
+     * @param {SimpleChat} chat
+     */
+    setup_load(div, chat) {
+        if (!(chat.ods_key() in localStorage)) {
+            return;
+        }
+        div.innerHTML += `<p class="role-system">Restore</p>
+        <p>Load previously saved chat session, if available</p>`;
+        let btn = ui.el_create_button(chat.ods_key(), (ev)=>{
+            console.log("DBUG:SimpleChat:SC:Load", chat);
+            chat.load();
+            queueMicrotask(()=>{
+                chat.show(div);
+                this.multiChat.elInSystem.value = chat.get_system_latest();
+            });
+        });
+        div.appendChild(btn);
+    }
+
+    /**
+     * Show the configurable parameters info in the passed Div element.
+     * @param {HTMLDivElement} elDiv
+     * @param {boolean} bAll
+     */
+    show_info(elDiv, bAll=false) {
+
+        let p = ui.el_create_append_p("Settings (devel-tools-console document[gMe])", elDiv);
+        p.className = "role-system";
+
+        if (bAll) {
+
+            ui.el_create_append_p(`baseURL:${this.baseURL}`, elDiv);
+
+            ui.el_create_append_p(`Authorization:${this.headers["Authorization"]}`, elDiv);
+
+            ui.el_create_append_p(`bStream:${this.bStream}`, elDiv);
+
+            ui.el_create_append_p(`bCompletionFreshChatAlways:${this.bCompletionFreshChatAlways}`, elDiv);
+
+            ui.el_create_append_p(`bCompletionInsertStandardRolePrefix:${this.bCompletionInsertStandardRolePrefix}`, elDiv);
+
+            ui.el_create_append_p(`bTrimGarbage:${this.bTrimGarbage}`, elDiv);
+
+            ui.el_create_append_p(`iRecentUserMsgCnt:${this.iRecentUserMsgCnt}`, elDiv);
+
+            ui.el_create_append_p(`ApiEndPoint:${this.apiEP}`, elDiv);
+
+        }
+
+        ui.el_create_append_p(`chatRequestOptions:${JSON.stringify(this.chatRequestOptions, null, " - ")}`, elDiv);
+        ui.el_create_append_p(`headers:${JSON.stringify(this.headers, null, " - ")}`, elDiv);
+
+    }
+
+    /**
+     * Auto create ui input elements for fields in ChatRequestOptions
+     * Currently supports text and number field types.
     * @param {HTMLDivElement} elDiv
     */
-    show_info(elDiv) {
+    show_settings_chatrequestoptions(elDiv) {
+        let typeDict = {
+            "string": "text",
+            "number": "number",
+        };
+        let fs = document.createElement("fieldset");
+        let legend = document.createElement("legend");
+        legend.innerText = "ChatRequestOptions";
+        fs.appendChild(legend);
+        elDiv.appendChild(fs);
+        for(const k in this.chatRequestOptions) {
+            let val = this.chatRequestOptions[k];
+            let type = typeof(val);
+            if (!((type == "string") || (type == "number"))) {
+                continue;
+            }
+            let inp = ui.el_creatediv_input(`Set${k}`, k, typeDict[type], this.chatRequestOptions[k], (val)=>{
+                if (type == "number") {
+                    val = Number(val);
+                }
+                this.chatRequestOptions[k] = val;
+            });
+            fs.appendChild(inp.div);
+        }
+    }

-        var p = document.createElement("p");
-        p.innerText = "Settings (devel-tools-console gMe)";
-        p.className = "role-system";
-        elDiv.appendChild(p);
+    /**
+     * Show settings ui for configurable parameters, in the passed Div element.
+     * @param {HTMLDivElement} elDiv
+     */
+    show_settings(elDiv) {

-        var p = document.createElement("p");
-        p.innerText = `bCompletionFreshChatAlways:${this.bCompletionFreshChatAlways}`;
-        elDiv.appendChild(p);
+        let inp = ui.el_creatediv_input("SetBaseURL", "BaseURL", "text", this.baseURL, (val)=>{
+            this.baseURL = val;
+        });
+        elDiv.appendChild(inp.div);

-        p = document.createElement("p");
-        p.innerText = `bCompletionInsertStandardRolePrefix:${this.bCompletionInsertStandardRolePrefix}`;
-        elDiv.appendChild(p);
+        inp = ui.el_creatediv_input("SetAuthorization", "Authorization", "text", this.headers["Authorization"], (val)=>{
+            this.headers["Authorization"] = val;
+        });
+        inp.el.placeholder = "Bearer OPENAI_API_KEY";
+        elDiv.appendChild(inp.div);

-        p = document.createElement("p");
-        p.innerText = `iRecentUserMsgCnt:${this.iRecentUserMsgCnt}`;
-        elDiv.appendChild(p);
+        let bb = ui.el_creatediv_boolbutton("SetStream", "Stream", {true: "[+] yes stream", false: "[-] do oneshot"}, this.bStream, (val)=>{
+            this.bStream = val;
+        });
+        elDiv.appendChild(bb.div);

-        p = document.createElement("p");
-        p.innerText = `chatRequestOptions:${JSON.stringify(this.chatRequestOptions)}`;
-        elDiv.appendChild(p);
+        bb = ui.el_creatediv_boolbutton("SetCompletionFreshChatAlways", "CompletionFreshChatAlways", {true: "[+] yes fresh", false: "[-] no, with history"}, this.bCompletionFreshChatAlways, (val)=>{
+            this.bCompletionFreshChatAlways = val;
+        });
+        elDiv.appendChild(bb.div);
+
+        bb = ui.el_creatediv_boolbutton("SetCompletionInsertStandardRolePrefix", "CompletionInsertStandardRolePrefix", {true: "[+] yes insert", false: "[-] dont insert"}, this.bCompletionInsertStandardRolePrefix, (val)=>{
+            this.bCompletionInsertStandardRolePrefix = val;
+        });
+        elDiv.appendChild(bb.div);
+
+        bb = ui.el_creatediv_boolbutton("SetTrimGarbage", "TrimGarbage", {true: "[+] yes trim", false: "[-] dont trim"}, this.bTrimGarbage, (val)=>{
+            this.bTrimGarbage = val;
+        });
+        elDiv.appendChild(bb.div);
+
+        let sel = ui.el_creatediv_select("SetChatHistoryInCtxt", "ChatHistoryInCtxt", this.sRecentUserMsgCnt, this.iRecentUserMsgCnt, (val)=>{
+            this.iRecentUserMsgCnt = this.sRecentUserMsgCnt[val];
+        });
+        elDiv.appendChild(sel.div);
+
+        sel = ui.el_creatediv_select("SetApiEP", "ApiEndPoint", ApiEP.Type, this.apiEP, (val)=>{
+            this.apiEP = ApiEP.Type[val];
+        });
+        elDiv.appendChild(sel.div);
+
+        this.show_settings_chatrequestoptions(elDiv);

    }

@ -619,6 +908,9 @@ let gMe;
 function startme() {
    console.log("INFO:SimpleChat:StartMe:Starting...");
    gMe = new Me();
+    gMe.debug_disable();
+    document["gMe"] = gMe;
+    document["du"] = du;
    for (let cid of gMe.defaultChatIds) {
        gMe.multiChat.new_chat_session(cid);
    }