Johannes Gäßler
6b73ef1201
YAML result logging + preset script ( #2657 )
2023-08-28 17:59:39 +02:00
alonfaraj
75fafcbccc
make : fix tests build ( #2855 )
...
* makefile:
- fix test name
- add missing tests build
* editorconfig : fixes
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-28 18:38:35 +03:00
grahameth
be475f60af
llama.cpp : fix wrong vsnprintf call in MS compiler ( #2856 )
...
Co-authored-by: grahameth <->
2023-08-28 18:38:12 +03:00
Ronny Brendel
3af6b86301
ggml : tiny ggml_vec_dot_q4_K_q8_K AVX2 improvement ( #2819 )
2023-08-28 15:51:08 +03:00
Georgi Gerganov
35feac6560
ggml : sync (mem align to header + conv_transpose_2d fixes + ggml_alloc) ( #2852 )
...
* ggml : sync (mem align to header + conv_transpose_2d fixes)
ggml-ci
* ggml-alloc : minor fix
* ggml-alloc : sync more fixes
2023-08-28 14:24:53 +03:00
Johannes Gäßler
92b1bbd2ec
CUDA: fix RoPE asserts, block sizes ( #2833 )
2023-08-28 14:23:55 +03:00
YellowRoseCx
cf5d918073
Koboldcpp-ROCm Port ( #399 )
...
* koboldcpp-ROCm Port
commit 3416c986d9d9a31c3cdefd7e7bd4d9438d72ba35
Merge: 5eb17f0 4c4e435#5 )
* add hipBLAS to README
* new build arg LLAMA_CUDA_MMQ_Y
* fix half2 decomposition
* Add intrinsics polyfills for AMD
* AMD assembly optimized __dp4a
* Allow overriding CC_TURING
* use "ROCm" instead of "CUDA"
* ignore all build dirs
* Add Dockerfiles
* fix llama-bench
* fix -nommq help for non CUDA/HIP
---------
Co-Authored-By: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Co-Authored-By: ardfork <134447697+ardfork@users.noreply.github.com>
Co-Authored-By: funnbot <22226942+funnbot@users.noreply.github.com>
Co-Authored-By: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Co-Authored-By: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
Co-Authored-By: jammm <2500920+jammm@users.noreply.github.com>
Co-Authored-By: jdecourval <7315817+jdecourval@users.noreply.github.com>
commit b34f4bd2724733e188ec4f6074042f66a5ed28c9
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Aug 19 17:12:52 2023 -0500
Update README.md
commit 7d1196108ad330b32845546fb3472c2172a0b6b8
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Mon Aug 14 23:03:12 2023 -0500
remove force DMMV
commit cd61aa0d9e16627935c7978adf488a679ddfa745
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Aug 12 17:24:31 2023 -0500
restore main_gpu parameter
commit 4a042f326830271a4c31104051b7b08e08ac234e
Author: Henri Vasserman <henv@hot.ee>
Date: Sat Aug 12 10:51:46 2023 +0300
gfx1100 support
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
Co-authored-by: jammm <2500920+jammm@users.noreply.github.com>
Co-authored-by: jdecourval <7315817+jdecourval@users.noreply.github.com>
commit 8913bc6fea97d3cb860937b0461f455c6abe3ea1
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 10:16:02 2023 +0300
Allow overriding CC_TURING
commit e77a4c37a756c002e97173f4122e088fb304e18a
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 10:00:07 2023 +0300
Merge 'origin/master' into hipblas
commit cc4c4e355cd553b1557d5fba2562e824db93f9b4
Author: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Date: Fri Aug 11 09:43:14 2023 +0300
New __dp4a assembly
Now compatible with gfx900 and faster as well.
commit 1a03b709848ce68d5bf5966237756167e2cac540
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 09:30:28 2023 +0300
Undo mess
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
commit 4366ff9ba1b1f12e494118ef9b5198479022fcc5
Author: DannyDaemonic <DannyDaemonic@gmail.com>
Date: Thu Aug 10 13:11:36 2023 -0700
Handle `ENABLE_VIRTUAL_TERMINAL_PROCESSING` more gracefully on earlier versions of Windows.
commit 811ff855a24323cafddc95c1b8aca711fef05f76
Author: Christian Demsar <crasm@git.vczf.us>
Date: Thu Aug 10 10:28:27 2023 -0400
Add --n-predict -2 for stopping generation on full context (#2565 )
commit 37c9717aaa6815b6a5be21aaab970212f20fe6bf
Author: Martin Krasser <krasserm@googlemail.com>
Date: Thu Aug 10 12:16:38 2023 +0200
Fix grammar-based sampling issue in server (#2566 )
commit d18ecd5b9e5dde58ae08a3eef1637406159ddaca
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Thu Aug 10 13:19:41 2023 -0500
make mmq gen faster for amd
commit 243894a952147a4fac5b6aee748861a0df6cc2c6
Author: Henri Vasserman <henv@hot.ee>
Date: Thu Aug 10 12:14:40 2023 +0300
ws fix
commit ac2f14da445ea87d73539adbd29d19ff2c9eba58
Author: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Date: Thu Aug 10 12:11:27 2023 +0300
AMD assembly optimized __dp4a
Doesn't seem to work for gfx900, so commented out.
commit 9dba0c985f140ddded8cbb671f139e81fff82eed
Author: Henri Vasserman <henv@hot.ee>
Date: Thu Aug 10 12:09:28 2023 +0300
Fix merge
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
Co-authored-by: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
commit f570b5cb1070591527a82d94bba408927b37778d
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 22:11:20 2023 -0500
Revert "revert cuda changes as they are bugggy"
This reverts commit 1541bf879772aeeed8ff646bfc52185c2a88b79b.
commit 1541bf879772aeeed8ff646bfc52185c2a88b79b
Author: Concedo <39025047+LostRuins@users.noreply.github.com>
Date: Wed Aug 9 22:36:41 2023 +0800
revert cuda changes as they are bugggy
commit bacc20203efb1839aa313858a04d75255bb4b7f4
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 20:37:17 2023 -0500
Merge remote-tracking branch 'upstream/concedo'
commit b7cb4cfd109986bd66e8fd382d1e2516eaddfebb
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 20:00:52 2023 -0500
additional fixes
commit fadae727baa3735ad3e0667384d6e05ca056b3ef
Merge: 518eb2a 8f8ab6ccae6a848f8ab6c4c0610ba4cfc44024f9125d43e04024f91a66ab6212864cd91456af5bfea0d91456aaf1c1cb70d64dc1664a00ae43362310728c5ab40550c31486eb4336231a32f8e3fc6c74d2ade639f4cde52d68a88e5894e0a069731682cde52d6a638e8054a84e09a78e8054ad831f6294dc44#5 )
* initialize rocblas
commit 5b838d47874536ebffc2f6cb25877e0476a9402d
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Mon Jul 24 03:10:35 2023 -0500
amd multigpu full layer offload w/o vram scratch
commit 9bfb2fdd68000670bda85c4e9748d72f5af09764
Merge: b379f9d 66328fc56995caf0361093db70b5f0a2ec44667568d1a2ec4466db5cd36b185ffafcb8fe1cbf5615941514afcb8fe0c48c2c4978a3e6104662347463e61046630780e4e548bf7735c5a1d16309#2140 )
commit 4539bc2761a7a23b588b5420b9d3fd1962ff63e5
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Jul 8 01:36:14 2023 -0500
update makefile for changes
commit 912e31ec523eac9ef308f0d28bc2d93aab7c3ecb
Merge: 74e2703 ddaa4f2f9108ba7735c5a9afc3e37337ee76e43d2907dc3e3733c6115db19ae7b04419f146088f7eda663f1347d3a04419f1894bb16effd3494bbbb16effc75#1 )
* kquants_iter for hipblas and add gfx803
* Update CMakeLists.txt with hipblas kquants_iter and DMMV_F16
* remove dmmv_f16 for now
commit 096f0b055e11b7d930842f86146d0e5013c5dce6
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Jun 28 15:27:02 2023 -0500
revert unnecessary hipblas conditionals
commit d81e81adffd6eb59e280ae1885864bb5fbd9bba6
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Jun 28 14:48:23 2023 -0500
Update Makefile hipblas nvcc correction
commit c8ae94524ac1e5c830be54f7d2034cec1e5c8345e35a6031447ccbe35a603161adf7346c66a25558342fe8df7346ccd55dd2fbe7487137e6ddb15b1f00fa266d47a5dd2fbe6ea67e229b20568fe67e229b7ca6f7c156b2416496f7c15637a61df8e9fc45a8161df8e9217a83652999685f902d254a7a785f902d5c44362e80b50b5704362e805a4fa5b3d717366dffa5b3d73651ba4ce4ad75d6eb721645d6eb7216433091a9bd39fdaa1d2d433879fdaa1d250#1607
commit a4648c1e7c4c8b3fb0ecb1bb4c8b3fb10730d921af3ea593a4f6c2174bf6a86df80ce7a1fcdcc2f80ce7a4e0600ace3ac7876a600ace39c8b19fefef94c66115b833a0b2d5fb8ee340a0b2d5f2918bab4562a5ee028bab45611e2956630b5c92952956630a3d0fe6384f048af00fe6384755605560d9ec127f68e089b1c9127f68eb5a070cbccb608b55070cbcc1bda3296d50aa0aefa6ae1295510aefa6ab71baeb4821b0fd45baeb482a94289073a5321107194173d0e61107194e6b04c0d48a3b85b204c0d480d7d83cfbad0cb67cc50799fdc1b67cc50dadfcbc262e216aa0fcbc262eb9c73def1f4cef87c73def129ad8ea75ef0d70f1d8ea75e952d194586334637ed194586f652ab9d117f15c5c2ab9d11f373b4a53104aaae13b4a53138fa1caa480b2da20a1caa48611ecc056519fef51e9ecacd571d164afcc37d571d1629f608aa33dd0eabc608aa33d9f3a004b2a01db7a01297e3677235284685f367723544cd3e1984ce00e005f779354a63c10e80fd8363adc
2023-08-28 17:05:06 +08:00
igarnier
dd0dc366da
llama.h : add missing struct keyword for C compat in callback type ( #2847 )
2023-08-28 11:19:59 +03:00
Georgi Gerganov
f55538c3cc
metal : fix memory leak ( #2762 )
...
* metal : fix memory leak
* metal : fix encoders memory leak
* metal : clean up more memory resources
* metal : fix more leaks
* metal : reuse dispatch queue + autoreleasepool
* metal : reuse array for command buffers and encoders
* ggml : assert for odd number of blocks on ARM
15M tinyllama is an example
2023-08-28 10:59:08 +03:00
Cebtenzzre
ebcee207b6
quantize : make output filename optional again ( #2823 )
...
* quantize : make output filename optional again
* quantize : fix path parsing on Windows
suggested by @slaren
2023-08-28 09:32:25 +03:00
JohnnyB
3e8ff47af6
devops : added systemd units and set versioning to use date. ( #2835 )
...
* Corrections and systemd units
* Missing dependency clblast
2023-08-28 09:31:24 +03:00
Concedo
4b00916ac7
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .dockerignore
# .github/workflows/build.yml
# CMakeLists.txt
# Makefile
# README.md
# flake.lock
# flake.nix
# tests/CMakeLists.txt
2023-08-28 14:19:05 +08:00
Georgi Gerganov
103cfafc77
gguf : fix strings to not be null-terminated ( #2839 )
...
* gguf : fix strings to not be null-terminated
ggml-ci
* gguf : fix gguf_add_tensor name
2023-08-27 21:50:22 +03:00
Georgi Gerganov
c10704d01e
llama : fix MPI threads ( close #2827 )
2023-08-27 18:55:41 +03:00
Olivier Chafik
230d46c723
examples : update llama2.c converter to read vocab and write models in GGUF format ( #2751 )
...
* llama2.c: direct gguf output (WIP)
* Simplify vector building logic
* llama2.c gguf conversion: fix token types in converter
* llama2.c: support copying vocab from a llama gguf model file
* llama2.c: update default path for vocab model + readme
* llama2.c: use defines for gguf keys
* llama2.c: escape whitespaces w/ U+2581 in vocab converter the llama.cpp way
* llama2.c converter: cleanups + take n_ff from config
2023-08-27 17:13:31 +03:00
Kawrakow
463173a6c0
llama : speedup tokenization ( #2831 )
...
* Speedup tokenization
On current master it takes ~3.2 seconds to tokenize
Wikitext. With this change it becomes ~525 ms.
* Fixit: it was missing the piece after the last found occurence
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-27 16:50:33 +03:00
Georgi Gerganov
eaa13a48ff
falcon : fix CUDA inference by making K and Q contiguous ( #2830 )
...
* falcon : fix CUDA inference by making K and Q contiguous
ggml-ci
* cuda : add assert to guard from non-cont ropes
2023-08-27 16:40:48 +03:00
Georgi Gerganov
da7455d046
readme : fix headings
2023-08-27 15:52:34 +03:00
Georgi Gerganov
25423e9185
scripts : helper convert script
2023-08-27 15:24:58 +03:00
Kawrakow
a6d1189fdd
k_quants tuning for Falcon-7b ( #2816 )
...
* Make ggml-cuda.cu build with QK_K = 64
Using LLAMA_CUDA_FORCE_DMMV = ON and -nommq it runs and produces
a meaningful result.
* k_quants tuning for Falcon-7b
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-27 15:19:59 +03:00
Georgi Gerganov
c48c5bb0b0
readme : update hot topics
2023-08-27 14:44:35 +03:00
Georgi Gerganov
d0cee0d36d
gguf : add 64-bit support (GGUF v2) ( #2821 )
...
* gguf : bump version to 2
* gguf : add support for 64-bit (no backwards comp yet)
* gguf : v1 backwards comp
* gguf.py : bump GGUF version
* gguf.py : uint64_t on all lengths, sizes and counts, enums still uint32_t
* gguf.py : string lengths uint32_t
* gguf : update all counts to 64-bit
* gguf.py : string len uint64_t and n_dims uint32_t
* gguf : fix typo
* llama.cpp : print gguf version
---------
Co-authored-by: klosax <131523366+klosax@users.noreply.github.com>
2023-08-27 14:19:54 +03:00
Georgi Gerganov
edd4c14817
llama : more tokenizer fixes ( #2810 )
...
* tests : write a Python tokenizer test (wip)
* llama : prefix input text for tokenization with whitespace
* llama : distinguish pieces from decoded text + fix detokenization
* common : add comments
* examples : no longer manually add leading space when tokenizing
* tests : use Python to generate tokenizer tests for C++
* tests : add option to tokenize text files
ggml-ci
* tests : add test-tokenizer-1.py
* llama.cpp : fix LF token
* hellaswag : move the concat space for clarity
* tests : add falcon tests (py + cpp, currently do not pass Unicode)
ggml-ci
* common : temporary separate llama_detokenize calls for SPM and BPE
---------
Co-authored-by: klosax <131523366+klosax@users.noreply.github.com>
2023-08-27 14:19:19 +03:00
Przemysław Pawełczyk
1591e2e590
ggml : detect SSSE3 ( #2825 )
...
* ggml : add ggml_cpu_has_ssse3
* llama : show SSSE3 in system info
2023-08-27 11:10:25 +03:00
slaren
789c8c945a
ci : add LoRA test to CI ( #2650 )
...
* ci : add lora test
ggml-ci
* move lora summary to the top, add lora logs
ggml-ci
* ci : decrease CPU ppl runs to 2 to avoide 20 min timeout
ggml-ci
* add 7b lora test
use 1 thread for CUDA generation tests
ggml-ci
* add test with q8_0 (cpu only)
ggml-ci
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-27 10:03:27 +03:00
Concedo
9d5b4238e8
added config to class.py
2023-08-27 10:32:01 +08:00
Bruce MacDonald
c1ac54b77a
server : add /detokenize endpoint ( #2802 )
...
* Add a /detokenize endpoint to the example server
* remove trailing white-space
2023-08-27 07:11:45 +08:00
Kerfuffle
730d9c681e
convert.py : advanced option ( #2753 )
...
* Allow convert.py to convert to q8_0
Fix issue with bounded_parallel_map and greedy consuming iterator
Display elapsed time during conversion
* Add --concurrency option
Minor improvements to help text
Clean up bounded_parallel_map function a bit
* Massive speed improvement thanks to Cebtenzzre
* Refactor types
2023-08-26 23:13:36 +03:00
Tim Miller
c7d92e6dfe
llama : use Unicode Escape Sequence to replace encoded characters ( #2814 )
...
The use of special characters within source files can break compiling on some computers with different region and language settings. Using Unicode escape sequences should allow for the code to be compiled on all setups without needing to change your computers settings or switch regions.
2023-08-26 21:27:07 +03:00
Tungsten842
61d1a2895e
flake.nix : add rocm support and cleanup ( #2808 )
2023-08-26 21:19:44 +03:00
Cebtenzzre
741ca7dd1c
llama : move #includes out of _GNU_SOURCE conditional ( #2817 )
2023-08-26 21:17:51 +03:00
Dr. Tom Murphy VII Ph.D
72f895c923
main : fix bug (penalize_nl=false doesn't work) + suppress warning on mingw ( #1528 )
...
* Fix bug in main.cpp where penalize_nl=false has no effect. It modifies the underlying logits array, but at this point we are already working on the candidates copy.
* Suppress redefinition warning for NOMINMAX on mingw. In my installation, this macro is already defined by /usr/lib/gcc/x86_64-w64-mingw32/11/include/c++/x86_64-w64-mingw32/bits/os_defines.h:45.
* main : fix indentation
* main : pass ctx to llama_token_nl()
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-26 21:12:56 +03:00
Cebtenzzre
50526f37eb
llama : use std::abs in llama_sample_tail_free ( #2800 )
...
Plain 'abs' casts the input to int.
2023-08-26 19:53:52 +03:00
Georgi Gerganov
04f4b1eb10
k-quants : remove unnecessary tensor shape restrictions ( #2811 )
2023-08-26 17:37:35 +03:00
Kawrakow
7592375403
Better perplexity for 2- and 3-bit quantization for LLaMA-v2-70B ( #2807 )
...
* Better perplexity for 2- and 3-bit quantization for the 70B model
* PR comment
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-26 17:27:49 +03:00
Kawrakow
771551a793
Fix HellaSwag ( #2805 )
...
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-26 16:48:53 +03:00
Volodymyr Vitvitskyi
f305bad11e
flake : build llama.cpp on Intel with nix ( #2795 )
...
Problem
-------
`nix build` fails with missing `Accelerate.h`.
Changes
-------
- Fix build of the llama.cpp with nix for Intel: add the same SDK frameworks as
for ARM
- Add `quantize` app to the output of nix flake
- Extend nix devShell with llama-python so we can use convertScript
Testing
-------
Testing the steps with nix:
1. `nix build`
Get the model and then
2. `nix develop` and then `python convert.py models/llama-2-7b.ggmlv3.q4_0.bin`
3. `nix run llama.cpp#quantize -- open_llama_7b/ggml-model-f16.gguf ./models/ggml-model-q4_0.bin 2`
4. `nix run llama.cpp#llama -- -m models/ggml-model-q4_0.bin -p "What is nix?" -n 400 --temp 0.8 -e -t 8`
Co-authored-by: Volodymyr Vitvitskyi <volodymyrvitvitskyi@SamsungPro.local>
2023-08-26 16:25:39 +03:00
Nigel Bosch
a2ca4e9de9
Handle null rope scaling value ( #2793 )
2023-08-26 14:11:17 +02:00
klosax
2ba83c8685
Fix spm whitespaces ( #2806 )
...
* llama.cpp : fix spm whitespace escaping + clean up
* main.cpp : spm - add whitespace in front of prompt
* test-tokenizer-0.cpp : spm - add whitespace in front of prompt
2023-08-26 13:45:53 +02:00
lon
bae5c5f679
examples : skip unnecessary external lib in server README.md how-to ( #2804 )
2023-08-26 16:07:43 +08:00
Marcus Dunn
232caf3c15
llama : fix struct decl ( #2790 )
2023-08-25 19:17:15 +03:00
Kawrakow
d046dcee08
Faster perplexity computation ( #2786 )
...
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-25 19:05:02 +03:00
Matt Pulver
c82742ac9c
llama : add llama_beam_search() ( #2267 )
...
* Add llama_beam_search().
* Add '// Beam search' heading to llama.{h,cpp} after llama_grammar_accept_token().
* Add space around * pointers and & references.
* Add spaces around comparison and assignment operators.
* Prefer west const.
* Use llama_ prefix for structs in global namespace.
* Delete obsolete comment from an earlier revision.
* Change eos to eob in llama_beam and llama_beam_view structs.
2023-08-25 18:18:48 +03:00
Nigel Bosch
28b2c996ca
convert.py : Get rope scale from HuggingFace models ( #2772 )
...
* Get rope scale from HF models
* Save rope scale only for linear scaling
* Rewrite for clarity
2023-08-25 16:41:52 +02:00
slaren
154725c543
llama-bench : add model sizes ( #2771 )
...
* llama-bench : add model sizes
* more compact markdown output
* back to GiB
* adjust column sizes
2023-08-25 15:16:19 +02:00
slaren
12e2e33a97
convert.py : export rope freq_base when converting CodeLlama from an HF model ( #2773 )
2023-08-25 14:08:53 +02:00
Jhen-Jie Hong
29674ab4e8
server : display token probabilities in the UI ( #2489 )
...
* server : add n_probs param in chat UI
* server : keep message data array & show in probabilites component
* server : add simple popover component
* server : fix completion_probabilities undefined if not set n_probs
* server : implement Probabilites
* server : handle bytes
* server : make n_probs max to 10 for easy scroll
* server : adjust for dark/light mode
* server : Fix regenerated prompt
* server : update index.html.hpp
* server : convert prob to percentage + show original value as div title
* server : fix Probabilites not used if included empty str
* server : skip byte pair in display probabilites
* server : remove array check of completion_probabilities in messages
* skip empty array or byte pair (> 1) in Probabilites
* generate index.html.hpp
* fix incorrect prob convert if the str is already a known token
* use final response to show probabilities on stop
* revert unnecessary change
* correct probabilites usage
* remove unused function
* always send partial response for get correct probs of last to_send
* fix typo
* fix content of format_final_response
* refactor probs render & make pColor transparent if not found
* send empty string when got stop_pos in partial
* avoid unnecessary empty data event & send rest of partial tokens on stop
* use <br /> for new line
* skip -1 tok in loop to avoid send '' on end
* trim last new lines on stop
* revert unnecessary change
2023-08-25 18:32:45 +08:00
Georgi Gerganov
5439a0ab57
ci : pip install gguf in editable mode ( #2782 )
...
ggml-ci
2023-08-25 13:03:25 +03:00
M. Yusuf Sarıgöz
8194cd8772
gguf : export objects to user code ( #2780 )
...
* gguf export more objects to user code
* gguf export all objects to user code for now
* gguf : bump version
2023-08-25 12:43:41 +03:00
Henri Vasserman
6bbc598a63
ROCm Port ( #1087 )
...
* use hipblas based on cublas
* Update Makefile for the Cuda kernels
* Expand arch list and make it overrideable
* Fix multi GPU on multiple amd architectures with rocblas_initialize() (#5 )
* add hipBLAS to README
* new build arg LLAMA_CUDA_MMQ_Y
* fix half2 decomposition
* Add intrinsics polyfills for AMD
* AMD assembly optimized __dp4a
* Allow overriding CC_TURING
* use "ROCm" instead of "CUDA"
* ignore all build dirs
* Add Dockerfiles
* fix llama-bench
* fix -nommq help for non CUDA/HIP
---------
Co-authored-by: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
Co-authored-by: funnbot <22226942+funnbot@users.noreply.github.com>
Co-authored-by: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Co-authored-by: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
Co-authored-by: jammm <2500920+jammm@users.noreply.github.com>
Co-authored-by: jdecourval <7315817+jdecourval@users.noreply.github.com>
2023-08-25 12:09:42 +03:00
