vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3951 commits 380 branches 3056 tags 365 MiB
Commit graph

3951 commits

This branch
This branch
All branches
Author SHA1 Message Date
MaggotHATE
44bbd6337a Quick fixes by comments 2024-10-14 11:43:45 +05:00
MaggotHATE
cca842fbd3 Fixed arg after update 2024-10-12 18:46:13 +05:00
MaggotHATE
ea85a51af1
Merge branch 'ggerganov:master' into master 2024-10-12 18:38:06 +05:00
MaggotHATE
68557eb7a0 Merge branch 'master' of https://github.com/MaggotHATE/llama.cpp-xtc 2024-10-12 18:36:14 +05:00
MaggotHATE
9c43a01c5d Removed xtc_threshold_max 2024-10-12 18:35:56 +05:00
Georgi Gerganov
edc265661c
server : add option to time limit the generation phase (#9865) 
ggml-ci
 2024-10-12 16:14:27 +03:00
Georgi Gerganov
1bde94dd02
server : remove self-extend features (#9860) 
* server : remove self-extend

ggml-ci

* server : fix context limit check to use slot.n_past

ggml-ci
 2024-10-12 16:06:31 +03:00
Georgi Gerganov
95c76e8e92
server : remove legacy system_prompt feature (#9857) 
* server : remove legacy system_prompt feature

ggml-ci

* readme : update [no ci]

* server : fix non-transformer logic + remove response from /props
 2024-10-12 14:51:54 +03:00
Georgi Gerganov
11ac9800af
llama : improve infill support and special token detection (#9798) 
* llama : improve infill support

ggml-ci

* llama : add more FIM token strings

ggml-ci

* server : update prompt on slot restore (#9800)

* gguf : deprecate old FIM token KVs
 2024-10-12 08:21:51 +03:00
R0CKSTAR
943d20b411
musa : update doc (#9856) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2024-10-12 08:09:53 +03:00
MaggotHATE
dfe587a5f3
Merge branch 'ggerganov:master' into master 2024-10-12 00:41:34 +05:00
Diego Devesa
96776405a1
ggml : move more prints to the ggml log system (#9839) 
* ggml : move more prints to the ggml log system

* show BLAS OpenMP warnings in all builds using debug print
 2024-10-11 15:34:45 +02:00
MaggotHATE
acada1a5e7 Made algorithm safer and more readable 2024-10-11 15:36:25 +05:00
MaggotHATE
3968369071 Fixed labels in old server UI 2024-10-11 11:53:19 +05:00
MaggotHATE
882a603bda
Merge branch 'master' into master 2024-10-11 11:26:05 +05:00
Diego Devesa
7eee341bee
common : use common_ prefix for common library functions (#9805) 
* common : use common_ prefix for common library functions

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-10-10 22:57:42 +02:00
Diego Devesa
0e9f760eb1
rpc : add backend registry / device interfaces (#9812) 
* rpc : add backend registry / device interfaces

* llama : add llama_supports_rpc API

* ggml_backend_rpc_start_rpc_server -> ggml_backend_rpc_start_server
 2024-10-10 20:14:55 +02:00
R0CKSTAR
cf8e0a3bb9
musa: add docker image support (#9685) 
* mtgpu: add docker image support

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

* mtgpu: enable docker workflow

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>

---------

Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2024-10-10 20:10:37 +02:00
MaggotHATE
72db625bd4 Added XTC to server UIs 2024-10-10 22:59:23 +05:00
Diego Devesa
c7499c557c
examples : do not use common library in simple example (#9803) 
* examples : do not use common library in simple example

* add command line parser, simplify code
 2024-10-10 19:50:49 +02:00
MaggotHATE
f7a383ffb3 Initial server support 2024-10-10 21:48:49 +05:00
MaggotHATE
2107882cf5 Renamed parameters, fixed info and defaults 
* probability is at 0 by default, but XTC is included in sampling queue
* threshold higher than 0.5 switches XTC off
 2024-10-10 19:35:28 +05:00
MaggotHATE
ba29d31fb7
Merge branch 'ggerganov:master' into master 2024-10-10 11:42:50 +05:00
Diego Devesa
c81f3bbb05
cmake : do not build common library by default when standalone (#9804) 2024-10-09 18:49:52 +02:00
Georgi Gerganov
e7022064ab
perplexity : fix integer overflow (#9783) 
* perplexity : fix integer overflow

ggml-ci

* perplexity : keep n_vocab as int and make appropriate casts

ggml-ci
 2024-10-09 17:00:18 +03:00
MaggotHATE
37e02e34a1
Added XTC to README 2024-10-09 14:08:02 +05:00
MaggotHATE
ed535bb2ae
Merge branch 'ggerganov:master' into master 2024-10-09 14:00:55 +05:00
Georgi Gerganov
3dc48fe75a
examples : remove llama.vim 
An updated version will be added in #9787
 2024-10-09 10:55:42 +03:00
MaggotHATE
d0b1053897
Fixed incorrect min_keep check 2024-10-09 00:59:46 +05:00
MaggotHATE
6feb6b399c
Update dump info in common 2024-10-08 21:15:37 +05:00
MaggotHATE
c19fb26042
Merged back lost commits in common and arg 2024-10-08 21:11:35 +05:00
MaggotHATE
09bc6d507c
Updated info in common and args 2024-10-08 20:57:36 +05:00
MaggotHATE
81a0c2603c
Simplified algorithm and more tests 2024-10-08 18:38:43 +05:00
MaggotHATE
8110f783d1
Merge branch 'ggerganov:master' into master 2024-10-08 18:36:38 +05:00
Diego Devesa
dca1d4b58a
ggml : fix BLAS with unsupported types (#9775) 
* ggml : do not use BLAS with types without to_float

* ggml : return pointer from ggml_internal_get_type_traits to avoid unnecessary copies

* ggml : rename ggml_internal_get_type_traits -> ggml_get_type_traits

it's not really internal if everybody uses it
 2024-10-08 14:21:43 +02:00
Xuan Son Nguyen
458367a906
server : better security control for public deployments (#9776) 
* server : more explicit endpoint access settings

* protect /props endpoint

* fix tests

* update server docs

* fix typo

* fix tests
 2024-10-08 13:27:04 +02:00
standby24x7
fa42aa6d89
scripts : fix spelling typo in messages and comments (#9782) 
Signed-off-by: Masanari Iida <standby24x7@gmail.com>
 2024-10-08 09:19:53 +03:00
MaggotHATE
98b204c918
Merge branch 'ggerganov:master' into master 2024-10-08 01:20:14 +05:00
MaggotHATE
dbe9ef7783
Added XTC to test-sampling 2024-10-08 01:19:39 +05:00
Diego Devesa
6374743747
ggml : add backend registry / device interfaces to BLAS backend (#9752) 
* ggml : add backend registry / device interfaces to BLAS backend

* fix mmap usage when using host buffers
 2024-10-07 21:55:08 +02:00
Andrew Minh Nguyen
f1af42fa8c
Update building for Android (#9672) 
* docs : clarify building Android on Termux

* docs : update building Android on Termux

* docs : add cross-compiling for Android

* cmake : link dl explicitly for Android
 2024-10-07 09:37:31 -07:00
Georgi Gerganov
6279dac039
flake.lock: Update (#9753) 
Flake lock file updates:

• Updated input 'flake-parts':
    'github:hercules-ci/flake-parts/bcef6817a8b2aa20a5a6dbb19b43e63c5bf8619a?narHash=sha256-HO4zgY0ekfwO5bX0QH/3kJ/h4KvUDFZg8YpkNwIbg1U%3D' (2024-09-12)
  → 'github:hercules-ci/flake-parts/3d04084d54bedc3d6b8b736c70ef449225c361b1?narHash=sha256-K5ZLCyfO/Zj9mPFldf3iwS6oZStJcU4tSpiXTMYaaL0%3D' (2024-10-01)
• Updated input 'flake-parts/nixpkgs-lib':
    'https://github.com/NixOS/nixpkgs/archive/356624c12086a18f2ea2825fed34523d60ccc4e3.tar.gz?narHash=sha256-Ss8QWLXdr2JCBPcYChJhz4xJm%2Bh/xjl4G0c0XlP6a74%3D' (2024-09-01)
  → 'https://github.com/NixOS/nixpkgs/archive/fb192fec7cc7a4c26d51779e9bab07ce6fa5597a.tar.gz?narHash=sha256-0xHYkMkeLVQAMa7gvkddbPqpxph%2BhDzdu1XdGPJR%2BOs%3D' (2024-10-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/1925c603f17fc89f4c8f6bf6f631a802ad85d784?narHash=sha256-J%2BPeFKSDV%2BpHL7ukkfpVzCOO7mBSrrpJ3svwBFABbhI%3D' (2024-09-26)
  → 'github:NixOS/nixpkgs/bc947f541ae55e999ffdb4013441347d83b00feb?narHash=sha256-NOiTvBbRLIOe5F6RbHaAh6%2B%2BBNjsb149fGZd1T4%2BKBg%3D' (2024-10-04)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2024-10-07 09:35:42 -07:00
MaggotHATE
4c44e3da5a
Merge branch 'ggerganov:master' into master 2024-10-07 21:28:09 +05:00
Georgi Gerganov
d5ac8cf2f2
ggml : add metal backend registry / device (#9713) 
* ggml : add metal backend registry / device

ggml-ci

* metal : fix names [no ci]

* metal : global registry and device instances

ggml-ci

* cont : alternative initialization of global objects

ggml-ci

* llama : adapt to backend changes

ggml-ci

* fixes

* metal : fix indent

* metal : fix build when MTLGPUFamilyApple3 is not available

ggml-ci

* fix merge

* metal : avoid unnecessary singleton accesses

ggml-ci

* metal : minor fix [no ci]

* metal : g_state -> g_ggml_ctx_dev_main [no ci]

* metal : avoid reference of device context in the backend context

ggml-ci

* metal : minor [no ci]

* metal : fix maxTransferRate check

* metal : remove transfer rate stuff

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2024-10-07 18:27:51 +03:00
Paul Tsochantaris
96b6912103
metal : single allocation of encode_async block (#9747) 
* Single allocation of encode_async block with non-ARC capture in ggml-metal.m

* Moving Block_release to the deallocation code

* Release encode block when re-setting encoding buffer count if needed

* Update ggml/src/ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-10-07 15:26:31 +03:00
Georgi Gerganov
d5cb86844f
contrib : simplify + minor edits [no ci] 2024-10-06 14:15:27 +03:00
MaggotHATE
39940e5fa3
Algorithm rework 
1. Scan token from top till the first non-penalizable
2. Remove the last captured token (the least probable above threshold)
3. Shift all tokens to override the remaining penalizable
4. Penalize and put them at the the bottom.
 2024-10-06 16:15:12 +05:00
MaggotHATE
094caea359
Merge branch 'ggerganov:master' into master 2024-10-06 16:06:32 +05:00
Georgi Gerganov
f4b2dcdf49
readme : fix typo [no ci] 2024-10-06 13:49:41 +03:00
Georgi Gerganov
b6d6c5289f
sync : llama.cpp 2024-10-06 12:53:28 +03:00