ochafik
cc2c712cf9
Merge remote-tracking branch 'origin/master' into r1-toolcall
2025-02-08 14:35:10 +00:00
Nikolaos Pothitos
3ab410f55f
readme : update front-end framework ( #11753 )
...
After the migration to React with #11688
2025-02-08 10:43:04 +01:00
Xuan-Son Nguyen
0cf867160c
server : (webui) fix numeric settings being saved as string ( #11739 )
...
* server : (webui) fix numeric settings being saved as string
* add some more comments
2025-02-08 10:42:34 +01:00
Xuan-Son Nguyen
2fb3c32a16
server : (webui) migrate project to ReactJS with typescript ( #11688 )
...
* init version
* fix auto scroll
* bring back copy btn
* bring back thought process
* add lint and format check on CI
* remove lang from html tag
* allow multiple generations at the same time
* lint and format combined
* fix unused var
* improve MarkdownDisplay
* fix more latex
* fix code block cannot be selected while generating
2025-02-06 17:32:29 +01:00
Olivier Chafik
33efcb3c59
Update README.md
2025-02-05 16:20:11 +00:00
Olivier Chafik
098629df15
disable some failing chatml tests
2025-02-05 16:15:19 +00:00
Olivier Chafik
39b50c37dc
Update README.md
2025-02-05 15:53:48 +00:00
Olivier Chafik
e6d9b52480
align Command R7B w/ --think / reasoning_content behaviour
2025-02-05 15:47:37 +00:00
ochafik
f3e9f8b62a
fix test_thoughts
2025-02-05 12:34:27 +00:00
ochafik
d20c2ce4e7
Merge branch 'r1-toolcall' of github.com:ochafik/llama.cpp into r1-toolcall
2025-02-05 12:16:42 +00:00
ochafik
9d7c3cc51b
--think to force any model to return reasoning_content (or just parse <think> for deepseek r1)
2025-02-05 12:16:37 +00:00
Olivier Chafik
1f1f06aa26
Merge branch 'master' into r1-toolcall
2025-02-05 01:10:45 +00:00
Olivier Chafik
5d60cebbcc
Update test_tool_call.py
2025-02-04 17:48:29 +00:00
Xuan-Son Nguyen
3962fc1a79
server : add try..catch to places not covered by set_exception_handler ( #11620 )
...
* server : add try..catch to places not covered by set_exception_handler
* log_server_request: rm try catch, add reminder
2025-02-04 18:25:42 +01:00
Olivier Chafik
933f7a186e
Merge branch 'master' into r1-toolcall
2025-02-04 15:56:25 +00:00
Olivier Chafik
db288b60cb
tool-call: command r7b fix for normal responses (#11608 )
...
* fix command r7b normal response regex + add to server test
* test multiline non-tool-call responses in test-chat
2025-02-04 15:48:53 +00:00
Olivier Chafik
b2d17287aa
update readme section about common model tool call formats
...
./build/bin/test-chat ../minja/build/tests/*.jinja 2>/dev/null
2025-02-04 14:27:38 +00:00
Olivier Chafik
39c1d8163b
return thoughts in reasoning_content field
2025-02-04 11:37:09 +00:00
ochafik
1f5ec59809
ensure deepseek r1 thoughts parsed even w/o tool calls
2025-02-04 04:48:08 +00:00
ochafik
b6e14a4101
fix mistral expectation
2025-02-04 04:26:49 +00:00
ochafik
812544ab8b
server: check that content is null when we get tool_calls
2025-02-04 04:14:15 +00:00
ochafik
86994db697
fix spaces
2025-02-04 03:47:52 +00:00
ochafik
78b47bb0e9
fix test_calc_result
2025-02-04 03:46:26 +00:00
ochafik
326e7002b3
update test_calc_result
2025-02-04 03:13:13 +00:00
Olivier Chafik
30ea3591c9
update to minja's new api
2025-02-03 23:53:27 +00:00
Olivier Chafik
bc6d910f6d
Merge branch 'master' into r1-toolcall
2025-02-03 23:51:31 +00:00
Olivier Chafik
cde3833239
tool-call: allow --chat-template chatml w/ --jinja, default to chatml upon parsing issue, avoid double bos (#11616 )
...
* tool-call: allow `--jinja --chat-template chatml`
* fix double bos issue (drop bos/eos tokens from jinja template)
* add missing try catch around jinja parsing to default to chatml
* Simplify default chatml logic
2025-02-03 23:49:27 +00:00
Xuan-Son Nguyen
b3451785ac
server : (webui) revert hacky solution from #11626 ( #11634 )
2025-02-04 00:10:52 +01:00
Woof Dog
1d1e6a90bc
server : (webui) allow typing and submitting during llm response ( #11626 )
2025-02-03 23:16:27 +01:00
Olivier Chafik
c6214ee9d6
rm unneeded vocab
2025-02-03 19:59:50 +00:00
Olivier Chafik
7dc271fb37
tool-calls: add deepseek r1 template + accommodate broken official template slightly better
2025-02-03 19:59:33 +00:00
Olivier Chafik
569610ee77
tool-calls: accommodate variety of wrong tool call opening tags both Qwen 32B and 7B distills like to spit out
2025-02-03 18:57:55 +00:00
Olivier Chafik
4cb0e1d873
Merge branch 'jinja-chatml' into r1-toolcall
2025-02-03 17:15:14 +00:00
Daniel Bevenius
5598f475be
server : remove CPPHTTPLIB_NO_EXCEPTIONS define ( #11622 )
...
This commit removes the CPPHTTPLIB_NO_EXCEPTIONS define from the server
code.
The motivation for this is that when using a debug build the server
would crash when an exception was throws and terminate the server
process, as it was unhandled. When CPPHTTPLIB_NO_EXCEPTIONS is set
cpp_httplib will not call the exception handler, which would normally
return a 500 error to the client. This caused tests to fail when using
a debug build.
Fixes: https://github.com/ggerganov/llama.cpp/issues/11613
2025-02-03 16:45:38 +01:00
Olivier Chafik
5d18d76b69
fix double bos issue (drop bos/eos tokens from jinja template)
2025-02-03 13:59:16 +00:00
ochafik
a76073cf88
minimize diffs
2025-02-03 10:58:52 +00:00
ochafik
77ae97e7d6
Update test_tool_call.py
2025-02-03 10:28:30 +00:00
mashdragon
d92cb67e37
server : (webui) Fix Shift+Enter handling ( #11609 )
...
* Fix Shift+Enter handling
`exact` on the Enter handler means the message is not sent when Shift+Enter is pressed anyway
* build index.html.gz
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
2025-02-03 10:42:55 +01:00
ochafik
1e9acd2d31
tool-call: allow --jinja --chat-template chatml
2025-02-03 04:07:11 +00:00
ochafik
5e6f2a21ae
add deepseek models to server tool call section in readme
2025-02-03 02:44:42 +00:00
ochafik
19bea4ecc3
tell DS R1 not to overthink (weather test)
2025-02-03 02:24:30 +00:00
ochafik
ae9d5812a7
tool-calls: add DeepSeek R1 Qwen 7B to server test_hello_world
2025-02-03 02:24:30 +00:00
ochafik
04be723b33
tool-call: fix command-r7b parsing when response is multiline
2025-02-03 02:24:30 +00:00
ochafik
08716281f2
rename tests
2025-02-03 02:24:30 +00:00
ochafik
c80cb30938
update logs
2025-02-03 02:24:30 +00:00
ochafik
28345877e4
server/oai: ensure content is null when there are tool calls
2025-02-03 02:24:30 +00:00
ochafik
130ca222c9
DeepSeek R1: parse thoughts / return in separate field in API (non streamed mode)
2025-02-03 02:24:30 +00:00
ochafik
87de852b7f
pass vocab to common_chat_params_init
2025-02-03 02:24:30 +00:00
Olivier Chafik
bfcce4d693
tool-call: support Command R7B (+ return tool_plan "thoughts" in API) (#11585 )
...
* `tool-call`: support Command R7B (w/ tool_plan return)
* `tool-call`: cleaner preservation of tokens + warn when likely bad chat template override
* `tool-call`: test cleanup / handle lazy grammar triggers
2025-02-02 09:25:38 +00:00
Olivier Chafik
a83f528688
tool-call: fix llama 3.x and functionary 3.2, play nice w/ pydantic_ai package, update readme (#11539 )
...
* An empty tool_call_id is better than none!
* sync: minja (tool call name optional https://github.com/google/minja/pull/36 )
* Force-disable parallel_tool_calls if template doesn't support it
* More debug logs
* Llama 3.x tools: accept / trigger on more varied spaced outputs
* Fix empty content for functionary v3.2 tool call
* Add proper tool call docs to server README
* readme: function calling *is* supported now
* Apply suggestions from code review
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2025-01-31 14:15:25 +00:00
