vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1197 commits 380 branches 3056 tags 365 MiB
Commit graph

1197 commits

This branch
This branch
All branches
Author SHA1 Message Date
Cebtenzzre
3d0d55f295 train : use std::lround instead of add + cast (clang-tidy) 
train-text-from-scratch.cpp:1991:61: warning: casting (double + 0.5) to integer leads to incorrect rounding; consider using lround (#include <cmath>) instead [bugprone-incorrect-roundings]
    int impr_plot = std::isnan(opt->loss_after) ? 0 : -(int)(1 + (opt->loss_before - opt->loss_after) * 10.0f + 0.5f);
                                                            ^
 2023-09-05 19:49:05 -04:00
Cebtenzzre
4c703231cf save-load-state : remove a redundant c_str() call (clang-tidy) 
This is recommended by the 'readability-redundant-string-cstr' check.
 2023-09-05 19:49:05 -04:00
Cebtenzzre
6a9d1bd023 quantize : do not use 'else' after 'return' (clang-tidy) 
This is recommended by the 'readability-else-after-return' check.
 2023-09-05 19:49:05 -04:00
Cebtenzzre
b3d79bd346 grammar-parser : reserve space in vector (clang-tidy) 
This is recommended by the 'performance-inefficient-vector-operation'
check.
 2023-09-05 19:49:05 -04:00
Cebtenzzre
22ff1408b5 do not throw unhandled exceptions (clang-tidy) 
This is recommended by the 'bugprone-exception-escape' check.
 2023-09-05 19:49:05 -04:00
Cebtenzzre
87b90f505d train : remove inaccurate my_llama_hparams::operator== (clang-tidy) 
This operator overload is not used anyway - explicitly deleting it seems
to have no effect on compilation.

train-text-from-scratch.cpp:174:16: warning: comparing object representation of type 'my_llama_hparams' which does not have a unique object representation; consider comparing the members of the object manually [bugprone-suspicious-memory-comparison]
        return memcmp(this, &other, sizeof(my_llama_hparams));
               ^
 2023-09-05 19:49:02 -04:00
Cebtenzzre
f75865610d ggml-alloc : remove useless assignment (clang-tidy) 
This is recommended by the 'clang-analyzer-deadcode.DeadStores' check.
 2023-09-05 18:49:07 -04:00
Cebtenzzre
49a700ccff server : make token_with_probs non-const (clang-tidy) 
According to the 'performance-no-automatic-move' check, this allows an
automatic move of the return value (std::move) instead of a copy.
 2023-09-05 18:46:19 -04:00
Cebtenzzre
0b0fe663f8 prefer const reference parameters (clang-tidy) 
This is recommended by the 'performance-unnecessary-value-param' check.
 2023-09-05 18:45:19 -04:00
Cebtenzzre
aae2be0f08 fix some unidiomatic conditions (clang-tidy) 
These are recommended by the 'readability-container-size-empty' and
'readability-simplify-boolean-expr' checks.
 2023-09-05 18:45:08 -04:00
Cebtenzzre
688aae46e5 .clang-tidy : update checks 
bugprone-misplaced-widening-cast doesn't get along with casts needed for
-Wsign-compare, and the 'misc' category has a few useful checks.
 2023-09-05 18:36:58 -04:00
Cebtenzzre
1dd3fd9323 llama : fix some -Wunused warnings 2023-09-05 18:36:03 -04:00
Cebtenzzre
ceee3a6c22 build : silence misleading -Warray-bounds warning 
There is a -Warray-bounds warning from g++ 13.2.1 in
test-llama-grammar.cpp that is a false-positive because there is a
ternary that special-cases zero in the std::vector code.

/usr/include/c++/13.2.1/bits/stl_algobase.h:398:17: warning: array subscript 0 is outside array bounds of ‘const llama_grammar_element* [0]’ [-Warray-bounds=]
  398 |         { *__to = *__from; }
      |           ~~~~~~^~~~~~~~~
 2023-09-05 18:31:24 -04:00
Cebtenzzre
9e2023156e
make : use new flag variables for recent changes (#3019) 2023-09-05 15:12:00 -04:00
Cebtenzzre
de2fe892af
examples : replace fprintf to stdout with printf (#3017) 2023-09-05 15:10:27 -04:00
Erik Scholz
c9c3220c48
convert: fix convert.py not working with int filename_stem (#3028) 
* fix implicit int to string conversion
* convert : remove an obsolete pyright comment

---------

Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>
 2023-09-05 19:41:00 +02:00
Kawrakow
d59bd97065
Guard against all weights in a super-block being zero (#3010) 
* Guard against all weights in a super-block being zero

* Also guard against extremely small weights

Closes #2982 

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
 2023-09-05 09:55:33 +02:00
Georgi Gerganov
35938ee3b0
llama : update logic for number of threads when using BLAS 2023-09-05 10:46:39 +03:00
Georgi Gerganov
921772104b
speculative : add grammar support (#2991) 
* speculative : add grammar support

* grammars : add json_arr.gbnf

* grammar : add comments to new grammar file

* grammar : remove one nested level

* common : warm-up with 2 tokens - seems to work better

* speculative : print draft token pieces

* speculative : reuse grammar parser + better logs and comments

* speculative : avoid grammar_mem

* make : fix speculative build
 2023-09-05 08:46:17 +03:00
Georgi Gerganov
2ba85c8609
py : minor 2023-09-04 22:50:50 +03:00
Georgi Gerganov
e36ecdccc8
build : on Mac OS enable Metal by default (#2901) 
* build : on Mac OS enable Metal by default

* make : try to fix build on Linux

* make : move targets back to the top

* make : fix target clean

* llama : enable GPU inference by default with Metal

* llama : fix vocab_only logic when GPU is enabled

* common : better `n_gpu_layers` assignment

* readme : update Metal instructions

* make : fix merge conflict remnants

* gitignore : metal
 2023-09-04 22:26:24 +03:00
slaren
bd33e5ab92
ggml-opencl : store GPU buffer in ggml_tensor::extra (#2994) 2023-09-04 14:59:52 +02:00
Cebtenzzre
3103568144
llama-bench : make cpp file non-executable (#2999) 2023-09-04 13:40:18 +03:00
Leng Yue
5b8530d88c
make : add speculative example (#3003) 2023-09-04 13:39:57 +03:00
Aarni Koskela
e4386f417f
server : add a subtle loading animation to the edit box (#2466) 
* editorconfig: add override for the server HTML (which already is 2-space indented)

* server: add a subtle loading animation to the edit box
 2023-09-04 16:28:55 +08:00
Jiahao Li
35195689cd
2x faster (rms) norm cuda kernels (3.7% e2e improvement) (#2985) 
* 2x faster (rms) norm cuda kernels

* Fix code style
 2023-09-04 08:53:30 +02:00
slaren
cf9b08485c
ggml-alloc : use virtual memory for measurement (#2973) 
* ggml-alloc : use virtual memory for measurement

* compatibility fixes for MAP_ANONYMOUS

* fallback to fixed address for systems without virtual memory
 2023-09-03 20:34:09 +02:00
Georgi Gerganov
47068e5170
speculative : PoC for speeding-up inference via speculative sampling (#2926) 
* speculative : initial example

* speculative : print encoding speed

* speculative : add --draft CLI arg
 2023-09-03 15:12:08 +03:00
Georgi Gerganov
8f429fa511
perplexity : fix ETA by warming up the model with an empty run 2023-09-03 13:43:17 +03:00
Kerfuffle
6519e9c99c
gguf(python): Fix special vocab handling when id < 0 (#2984) 2023-09-03 04:38:43 -06:00
Georgi Gerganov
b7f2aa9e51
metal : restore 363f0bf and fix reduce in F16_F32 kernels (#2986) 2023-09-03 13:23:33 +03:00
Alon
73a12a6344
cov : disable comment in PRs (#2989) 2023-09-03 13:19:01 +03:00
opparco
3730134776
llama : fix bpe tokenize from byte (#2889) 2023-09-03 13:18:09 +03:00
Georgi Gerganov
d9151e6f57
metal : revert 6af0bab until we fix it 
This restores the generated text to be the same as before #2959
 2023-09-03 12:40:56 +03:00
Alon
afc43d5f82
cov : add Code Coverage and codecov.io integration (#2928) 
* update .gitignore

* makefile: add coverage support (lcov, gcovr)

* add code-coverage workflow

* update code coverage workflow

* wun on ubuntu 20.04

* use gcc-8

* check why the job hang

* add env vars

* add LLAMA_CODE_COVERAGE=1 again

* - add CODECOV_TOKEN
- add missing make lcov-report

* install lcov

* update make file -pb flag

* remove unused  GGML_NITER from workflows

* wrap coverage output files in COV_TARGETS
 2023-09-03 11:48:49 +03:00
Wentai Zhang
6460f758db
opencl : fix a bug in ggml_cl_pool_malloc() for ggml_cl_mul_mat_f32() (#2955) 
Co-authored-by: Wentai Zhang <wentaizhang@tencent.com>
 2023-09-03 11:46:44 +03:00
Kawrakow
ca82cf7bac
metal : more optimizations (#2959) 
* Very minor speedup via simd-group synchronization in f16 x f32

* Another very minor speedup on metal

* Quite significant PP speedup on metal

* Another attempt

* Minor

* Massive improvement for TG for fp16

* ~4-5% improvement for Q8_0 TG on metal

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-09-03 11:06:22 +03:00
kchro3
6a31a3bd98
swift : add support for k-quants (#2983) 2023-09-03 09:21:05 +03:00
Kerfuffle
cff7b0bf07
convert.py : BPE fixes (#2938) 
* convert.py: BPE fixes?

* Remove unnecessary conditional in addl token error handling
 2023-09-03 08:52:13 +03:00
Ido S
340af42f09
docs : add catai to README.md (#2967) 2023-09-03 08:50:51 +03:00
momonga
c42f0ec6b3
examples : fix gpt-neox (#2943) 
Co-authored-by: mmnga <mmnga1mmnga@gmail.com>
 2023-09-03 08:36:28 +03:00
kchro3
2753415afd
swift : add missing c file to Package.swift (#2978) 2023-09-03 08:27:25 +03:00
Cebtenzzre
bc054af97a
make : support overriding CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS (#2886) 
* make : remove unused -DGGML_BIG_ENDIAN

* make : put preprocessor stuff in CPPFLAGS

* make : pass Raspberry Pi arch flags to g++ as well

* make : support overriding CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS

* make : fix inverted conditional
 2023-09-03 08:26:59 +03:00
Kerfuffle
3358c381f6
logging: Fix creating empty file even when disabled (#2966) 
* logging: Fix creating empty file even when disabled

* Minor formatting fix

Co-authored-by: staviq <staviq@gmail.com>

---------

Co-authored-by: staviq <staviq@gmail.com>
 2023-09-02 11:53:55 -06:00
bandoti
52315a4216
readme : update clblast instructions (#2903) 
* Update Windows CLBlast instructions

* Update Windows CLBlast instructions

* Remove trailing whitespace
 2023-09-02 15:53:18 +03:00
Karsten Weiss
8b56b4f2c3
metal : show all Metal device instances in the system (#2952) 
* ggml_metal_init: Show all Metal device instances in the system

Also show the default Metal device that was picked.

* Update ggml-metal.m

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-09-02 15:29:09 +03:00
Jhen-Jie Hong
21f3d1be86
k-quants : fix build on armv7 (android only) (#2920) 
* k-quants : fix build on armv7

* ggml : cleanup unused arm32 specific impl

* k-quants : avoid some unused vzero / mzero define

* ggml-alloc : use 4g for MEASURE_MAX_SIZE in 32-bit arm
 2023-09-02 15:23:45 +03:00
Jhen-Jie Hong
571083f508
server : avoid aniprompt in probabilities of final response (#2849) 2023-09-02 08:31:46 +08:00
Engininja2
f04d002844
cuda : vsubss4 for older versions of ROCm/clang (#2942) 2023-09-01 23:33:19 +02:00
ZHAOKAI WANG
69fdbb9abc
readme : quick start command fix (#2908) 
* quick start command fix

* quick start win command fix
 2023-09-01 17:06:44 +03:00