Commit graph

2995 commits

Author SHA1 Message Date
HanishKVC
fee887fe31 ChatON:Common:Update the cmdline argument name used
Had forgotten to update it before
2024-05-06 11:27:56 +05:30
HanishKVC
58e1ff16bc ChatON: switch to ordered_json from json library
to be in sync with the json namespace in server.
2024-05-06 11:27:56 +05:30
HanishKVC
a630564c48 ChatON:ChatTemplateApplyCAPI remaining base logic
As c doesnt have the concept of pass by reference, and inturn the
existing c api uses pointers wrt llama chat message structure, so
switching to same wrt chat_tmpl_apply logics.

Also fix a oversight in previous commit and add the remaining logic.
2024-05-06 11:27:56 +05:30
HanishKVC
308d3bf3ff ChatON:WIP:Add c api wrapper for chat_template_apply
Initial skeletons

Update existing logics to help with same. Also the inbetween helper
was having a bad signature wrt returning status and data, thats also
fixed.
2024-05-06 11:27:56 +05:30
HanishKVC
e62699f923 ChatON: Add alertAssistantAtEnd flag & logic wrt MultiMsgs Apply
While sending the current chat session along with new user query
to the model, many models expect that a tag be added at the end
to indicate that user is expecting the model to respond, this
flags allows for the same.
2024-05-06 11:27:56 +05:30
HanishKVC
ea3a0f19cc ChatON: Rather check for tmpl existance in single_ex 2024-05-06 11:27:56 +05:30
HanishKVC
01c8db70f7 ChatON+Main: Add C_API wrapper for single
Add a c api wrapper for a single message tagging scenario.

Inturn to match convention followed by existing chat_apply_template
code, make it return the size expected of the tagged message string
buffer. Update internal single logic to help with same.

Explicitly check if tmpl specified is available in the loaded json
or not and then return a error if not found.
2024-05-06 11:27:56 +05:30
HanishKVC
13857f29d6 ChatON+Main: Updates wrt detailed meta json
Fix a oversight wrt key name.

Add a alert in case if passed meta json file contains begin(BoS)
wrt assistant role, similar to check for end (EoS) wrt user role.
Bcas normally both (ie EoS wrt User and BoS wrt Assistant) shouldnt
be needed.

Update main wrt begin & prefix and suffix & end addition.
2024-05-06 11:27:56 +05:30
HanishKVC
b9e31304a5 ChatON: Update to new detailed format wrt llama2 and llama3
Wrt llama2
* add bos wrt llama2 system and user begins, but not assistant
* split system suffix into suffix and end, and add systemuser-system
  flags so that end can be avoided wrt system+user message combo
* add eos wrt assistant end
* With these potentially this should work with main and server flows

Wrt llama3
* add empty begin, end fields and systemuser-system flags
* This should potentially work with main and server flows
2024-05-06 11:27:56 +05:30
HanishKVC
bf1167bfdb ChatON: Backup the current simple meta json file 2024-05-06 11:27:56 +05:30
HanishKVC
0cd7c62706 ChatON: Keep compiler happy
Move helpers to the begining, so can avoid adding prototype
declerations/function signatures to the begining

Get the char * wrt string data in the c++ string.
2024-05-06 11:27:56 +05:30
HanishKVC
6a0214c067 ChatON:MetaOK->MetaDump: Alert if user->end is needed or not
Because user messages dont normally need a EoS token.
2024-05-06 11:27:56 +05:30
HanishKVC
344857b6cb ChatOn:ChatOnTemplateApply: suffix,end flag based control
Also fix a oversight wrt begin, when flag based begin adding control
was introduced.

NOTE: Currently system role suffix/end conditional adding always
triggered, if 1st system prompt seen or additional system prompt
is seen.
2024-05-06 11:27:56 +05:30
HanishKVC
f8ae21cec7 ChatON:ChatTemplateApplySingle: update begin+prefix, suffix+end 2024-05-06 11:27:56 +05:30
HanishKVC
5d76f08d37 ChatON: Need to explicitly specify string to use c_str 2024-05-06 11:27:56 +05:30
HanishKVC
7ba0144e42 ChatOn:chaton_tmpl_role_kv: try except to ignore missing ifany
Cas of above reason, switch to directly accessing the keys in
dump helper, which is inturn used by meta_ok check
2024-05-06 11:27:56 +05:30
HanishKVC
adab5775bf ChatON: more detailed/spreadout json fields 2024-05-06 11:27:56 +05:30
HanishKVC
3f09eb5dea ChatOn: ChatTemplateApply[Ex] return tagged msgs parts detail
Now there is a simple and extended version of returning tagged
messages.

The extended version returns the tagged string, as well as the
details of the parts that make up that tagged message interms of
the type of parts and the lengths of the parts.
2024-05-06 11:27:56 +05:30
HanishKVC
825a78abaa ChatOn: ChatTemplateApplySingle[Ex] return parts detail
Now there is a simple and extended version of returning tagged
message wrt a single role and its content.

The extended version returns the tagged string, as well as the
details of the parts that make up that tagged message interms of
the type of parts and the lengths of the parts.
2024-05-06 11:27:56 +05:30
HanishKVC
92e780fb1a ChatON:ChatParts: Allow flexibility for more refined tokenization 2024-05-06 11:27:56 +05:30
HanishKVC
6b23f15ffe ChatON:ChatOnMetaJSon: Add suffix wrt assistant messages 2024-05-06 11:27:56 +05:30
HanishKVC
d1899728aa ChatON: Test ChatParts in chat-template-apply 2024-05-06 11:27:56 +05:30
HanishKVC
9de1d6017f ChatON:ChatParts class initial go
Helps keep user prompt and chat-hs-template tag parts seperate,
but in sequence
2024-05-06 11:27:56 +05:30
HanishKVC
3064a36e74 ChatON+:Update tmpl_role_kv to retrieve wrt multiple keys
Use the same for user role's begin and prefix entries.
2024-05-06 11:27:56 +05:30
HanishKVC
f1f39c5256 ChatON:Add Monarch model template, which uses Begin + Prefix
Inturn Begin/BoS is added only for non 1st user messages in a
system+user prompts chain.
2024-05-06 11:27:56 +05:30
HanishKVC
724ff38345 ChatOn: Wrap getting begin in try-catch,
so that even if a role doesnt contain begin, the logic will work
fine.
2024-05-06 11:27:56 +05:30
HanishKVC
d70fca7a45 ChatOn: Add begin to the mix along with prefix
Dump shows user->begin.

chat-template-apply[-single] updated to work with begin and prefix

TODO: need to wrap begin in a try-catch, so that irrespective of
role, begin+prefix will work, irrespoective of whether that role
has a begin entry or not.
2024-05-06 11:27:56 +05:30
HanishKVC
0f713d4c4f ChatOn: meta json update wrt the new begin related fields 2024-05-06 11:27:56 +05:30
HanishKVC
bdd279c0c9 ChatOn:User Begin+Prefix note update, keep things simple consistent 2024-05-06 11:27:56 +05:30
HanishKVC
84367b9fd1 ChatON: Add template for DeepSeek
Was looking at the tokenized vector, and noticed that the EOS
mentioned by existing chat_apply_template of llama.cpp, is different
from what I noticed in tokenizer_config.json of deepseek llm, so
I have added two entries

* "deepseek-alt" which matches llama.cpp's chat_apply_template and
* "deepseek" which matches that in tokenizer_config.json.

This impacts the assistant suffix and reverse prompt entries.

CasOfThis: Need to look into other entries which I added previously
at a later time. However as the default logic should be picking the
EOS from model file, so I assume reverse-prompt being outofsync,
may not matter beyond a limit, potentially.
2024-05-06 11:27:56 +05:30
HanishKVC
f4b54069f6 ChatON: Add template for Gemma 2024-05-06 11:27:56 +05:30
HanishKVC
2a8028fba8 ChatON: Add Zephyr template to meta-json file 2024-05-06 11:27:56 +05:30
HanishKVC
57bd772bfd ChatON: Cleanup logging
Avoid showing on screen the debug messages.

meta-dump can either show on screen or not, based on how LOGXLN
is defined.
2024-05-06 11:27:56 +05:30
HanishKVC
217544e5ff ChatON: Keep compiler happy
Order the functions so that no need for seperate prototypes

Also use kv_bool wrt boolean entries.

Convert string to c char *
2024-05-06 11:27:56 +05:30
HanishKVC
3f9dfc240c ChatON: Check for the boolean entries in meta-json 2024-05-06 11:27:56 +05:30
HanishKVC
42f6b45547 ChatON: Use the constants defined for the keys 2024-05-06 11:27:56 +05:30
HanishKVC
efb758ba7d ChatON: Rename helpers to kv suffix, updated wrt metaok
rename because they return value of specified key.

[main] update metaok to take template-id, so that one can cross
check that all needed entries are there wrt that template-id in
the chaton-meta-json file
2024-05-06 11:27:56 +05:30
HanishKVC
e8c24c0767 ChatOn:MetaOk: Allows template-id based cross check
For a given template-id, cross check, all needed entries are there
in the json.
2024-05-06 11:27:56 +05:30
HanishKVC
b1055641e9 ChatON: Update the notes a bit 2024-05-06 11:27:56 +05:30
HanishKVC
11b47fbcfc ChatON:MetaJson: Add key constants, check metaJson loaded ifNeeded 2024-05-06 11:27:56 +05:30
HanishKVC
221ccd6462 ChatOn: Add SystemUser-1st-User-Has-Prefix flag support
Llama2 seems to need it, so chaton-meta-json sample file updated
to use same.
2024-05-06 11:27:56 +05:30
HanishKVC
f03dd2439f ChatOn:No global-begin/end in ChatApplyTmplSingle, ChatApplyTmpl
Avoid adding global begin/end markers wrt ChatApplyTmplSingle.

Add ChatApplyTmpl which goes through a vector of messages.
2024-05-06 11:27:56 +05:30
HanishKVC
c4cf0e9075 ChatON:Cleanup: BeginEnd, Debug log
Update the note

Rename global-prefix|suffix to global-begin|end.

Rename chat-apply-template to chat-apply-template-single, cas it
handles only a single message.

Add some debug log messages to the helper functions
2024-05-06 11:27:56 +05:30
HanishKVC
d87d27512e ChatOn: update sample meta json a bit
Move [inst] [/inst] wrt llama2 from global to individual role
specific parts.

Avoid an extra \n wrt prefixes of llama3
2024-05-06 11:27:55 +05:30
HanishKVC
cdbe4f06ce Chaton:Sample Meta JSON cleanup 2024-05-06 11:27:55 +05:30
HanishKVC
050d329e7e ChatOn+Main: Initial go at chaton in main interactive flow 2024-05-06 11:27:55 +05:30
HanishKVC
1374a64200 Chaton:Meta: Add chatml meta data to sample meta json file 2024-05-06 11:27:55 +05:30
HanishKVC
093abc29a2 ChatOn: Update sample meta json to be a valid json 2024-05-06 11:27:55 +05:30
HanishKVC
dc56be951d ChatOn:Main: Load and dump any specified chaton meta file 2024-05-06 11:27:55 +05:30
HanishKVC
35f25196a0 ChatOn:Common: Add the needed cmdline arg params and its parsing 2024-05-06 11:27:55 +05:30