vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
2536 commits 380 branches 3056 tags 365 MiB
Commit graph

2536 commits

This branch
This branch
All branches
Author SHA1 Message Date
Concedo
9bc2e35b2e Merge branch 'master' into concedo_experimental 2023-11-03 23:51:32 +08:00
Concedo
373c20ad51 print error log if tunnel fails 2023-11-03 23:48:21 +08:00
slaren
abb77e7319
ggml-cuda : move row numbers to x grid dim in mmv kernels (#3921) 2023-11-03 12:13:09 +01:00
Concedo
c794fd5ceb sampler seed added (+1 squashed commits) 
Squashed commits:

[8a1b0d3d] sampler seed added
 2023-11-03 17:30:16 +08:00
Concedo
d7729ac3eb Merge branch 'master' into concedo_experimental 2023-11-03 16:00:05 +08:00
Georgi Gerganov
8f961abdc4
speculative : change default p_accept to 0.5 + CLI args (#3919) 
ggml-ci
 2023-11-03 09:41:56 +02:00
Georgi Gerganov
05816027d6
common : YAYF (yet another YARN fix) (#3925) 
ggml-ci
 2023-11-03 09:24:00 +02:00
cebtenzzre
3fdbe6b66b
llama : change yarn_ext_factor placeholder to -1 (#3922) 2023-11-03 08:31:58 +02:00
Concedo
8c14c81b33 hopefully this fixes the dotnet nonsense 2023-11-03 11:23:56 +08:00
Concedo
bc2027b008 Merge remote-tracking branch 'ceb/fix-fast-ext-factor' into concedo_experimental 2023-11-03 11:21:14 +08:00
Concedo
c07c9b857d Merge branch 'master' into concedo_experimental 
# Conflicts:
#	README.md
 2023-11-03 11:17:07 +08:00
cebtenzzre
25fef506cf llama : change yarn_ext_factor placeholder to -1 2023-11-02 21:53:59 -04:00
Kerfuffle
629f917cd6
cuda : add ROCM aliases for CUDA pool stuff (#3918) 2023-11-02 21:58:22 +02:00
Andrei
51b2fc11f7
cmake : fix relative path to git submodule index (#3915) 2023-11-02 21:40:31 +02:00
Georgi Gerganov
224e7d5b14
readme : add notice about #3912 2023-11-02 20:44:12 +02:00
Georgi Gerganov
c7743fe1c1
cuda : fix const ptrs warning causing ROCm build issues (#3913) 2023-11-02 20:32:11 +02:00
Oleksii Maryshchenko
d6069051de
cuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903) 
* Using cuda memory pools for async alloc/dealloc.

* If cuda device doesnt support memory pool than use old implementation.

* Removed redundant cublasSetStream

---------

Co-authored-by: Oleksii Maryshchenko <omaryshchenko@dtis.com>
 2023-11-02 19:10:39 +02:00
Concedo
879061c5d5 noavx2 clblast selector 2023-11-02 23:13:16 +08:00
Concedo
c7c3f3d9ab updated lite 2023-11-02 22:46:54 +08:00
Concedo
b0c7b88eac try fix clouflare tunnel (+2 squashed commit) 
Squashed commit:

[87d96bf2] update remote option

[c30bc909] updated fixed colab (+1 squashed commits)

Squashed commits:

[97b77563] updated fixed colab (+2 squashed commit)

Squashed commit:

[d851b04c] replaced cloudflare manual dl with remotetunnel in colab

[90ff1790] updated lite
 2023-11-02 22:27:35 +08:00
Georgi Gerganov
4ff1046d75
gguf : print error for GGUFv1 files (#3908) 2023-11-02 16:22:30 +02:00
Concedo
6dbb8d82b0 Merge branch 'master' into concedo_experimental 
# Conflicts:
#	CMakeLists.txt
#	models/ggml-vocab-llama.gguf
 2023-11-02 20:51:45 +08:00
Concedo
42eabf2f2f rope fixes 2023-11-02 20:41:16 +08:00
slaren
21958bb393
cmake : disable LLAMA_NATIVE by default (#3906) 2023-11-02 14:10:33 +02:00
Concedo
bc4ff72317 not working merge 2023-11-02 17:52:40 +08:00
Georgi Gerganov
2756c4fbff
gguf : remove special-case code for GGUFv1 (#3901) 
ggml-ci
 2023-11-02 11:20:21 +02:00
Georgi Gerganov
1efae9b7dc
llm : prevent from 1-D tensors being GPU split (#3697) 2023-11-02 09:54:44 +02:00
Concedo
fca7a4c054 added noavx2 model for clblast (+1 squashed commits) 
Squashed commits:

[291ecae6] added noavx2 mode for clblast (+1 squashed commits)

Squashed commits:

[562bc872] wip adding noavx2 cl
 2023-11-02 15:22:34 +08:00
cebtenzzre
b12fa0d1c1
build : link against build info instead of compiling against it (#3879) 
* cmake : fix build when .git does not exist

* cmake : simplify BUILD_INFO target

* cmake : add missing dependencies on BUILD_INFO

* build : link against build info instead of compiling against it

* zig : make build info a .cpp source instead of a header

Co-authored-by: Matheus C. França <matheus-catarino@hotmail.com>

* cmake : revert change to CMP0115

---------

Co-authored-by: Matheus C. França <matheus-catarino@hotmail.com>
 2023-11-02 08:50:16 +02:00
Georgi Gerganov
4d719a6d4e
cuda : check if this fixes Pascal card regression (#3882) 2023-11-02 08:35:10 +02:00
Georgi Gerganov
183b3fac6c
metal : fix build errors and kernel sig after #2268 (#3898) 2023-11-02 08:33:37 +02:00
Concedo
82267e5e69 switched back to clinfo since it's possibly more cross platform and can get memory vals easily 2023-11-02 14:12:05 +08:00
cebtenzzre
2fffa0d61f
cuda : fix RoPE after #2268 (#3897) 2023-11-02 07:49:44 +02:00
slaren
d480d2c204 ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel (#3891) 
* ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel

* fix warnings

(cherry picked from commit d02e98cde0)
 2023-11-02 11:19:53 +08:00
Concedo
1ab18ecb53 Merge commit 'c43c2da8af' into concedo_experimental 
# Conflicts:
#	llama.cpp
 2023-11-02 11:17:59 +08:00
cebtenzzre
0eb332a10f
llama : fix llama_context_default_params after #2268 (#3893) 2023-11-01 19:29:14 -04:00
slaren
d02e98cde0
ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel (#3891) 
* ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel

* fix warnings
 2023-11-01 23:10:09 +01:00
cebtenzzre
898aeca90a
llama : implement YaRN RoPE scaling (#2268) 
Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>
Co-authored-by: Jeffrey Quesnelle <jquesnelle@gmail.com>
 2023-11-01 18:04:33 -04:00
Georgi Gerganov
c43c2da8af
llm : fix llm_build_kqv taking unused tensor (benign, #3837) 2023-11-01 23:08:30 +02:00
Georgi Gerganov
523e49b111
llm : fix falcon norm after refactoring (#3837) 2023-11-01 23:00:50 +02:00
Georgi Gerganov
e16b9fa4ba
metal : multi-simd softmax (#3710) 
ggml-ci
 2023-11-01 21:25:00 +02:00
Georgi Gerganov
ff8f9a88da
common : minor (#3715) 2023-11-01 21:15:55 +02:00
Georgi Gerganov
50337961a6
llm : add llm_build_context (#3881) 
* llm : add llm_build_context

* llm : deduce norm eps based on type + explict max_alibi_bias, clamp_kqv

* llm : restore the non-graph llm_build_ functional API

ggml-ci

* llm : cleanup + comments
 2023-11-01 20:11:02 +02:00
bandoti
0e40806c1c
common : allow caller to handle help/argument exceptions (#3715) 
* Allow caller to handle help/argument exceptions

* Prepend newline to usage output

* Add new gpt_params_parse_ex function to hide arg-parse impl

* Fix issue blocking success case

* exit instead of returning false

* Update common/common.h

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Update common/common.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-11-01 19:42:01 +02:00
Concedo
21588cefd4 tunnel code done (+1 squashed commits) 
Squashed commits:

[b4bc7d20] wip integration of trycloudflare
 2023-11-01 23:28:23 +08:00
staviq
a2758d08e4
log : make generating separate log files optional (#3787) 
* impl --log-new, --log-append

* Update common/log.h

Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>

* Update common/log.h

Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>

* Apply suggestions from code review

Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>

---------

Co-authored-by: cebtenzzre <cebtenzzre@gmail.com>
 2023-11-01 16:18:27 +02:00
l3utterfly
e75dfdd31b
sampling : null grammar field after reset (#3885) 2023-11-01 15:40:43 +02:00
Concedo
3b227fc704 automatic gpu layer detection 2023-11-01 20:55:26 +08:00
Concedo
b395dbf6f5 wip layer calculator 2023-11-01 20:04:10 +08:00
Georgi Gerganov
9a3b4f6c86
ggml : fix UNUSED macro (#3762) 2023-11-01 13:50:45 +02:00