0cc4m
78b1d8351f
Add OpenCL compile options
2023-05-19 21:18:57 +02:00
Georgi Gerganov
2d5db48371
ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 ( #1508 )
...
* ggml : use F16 instead of F32 in Q4_0, Q4_1 and Q8_0
* llama : bump LLAMA_FILE_VERSION to 3
* cuda : update Q4 and Q8 dequantize kernels
* ggml : fix AVX dot products
* readme : update performance table + hot topics
2023-05-19 22:17:18 +03:00
Georgi Gerganov
6986c7835a
tests : add missing header
2023-05-19 21:17:28 +03:00
Evan Jones
943e6081cc
examples : add persistent chat ( #1495 )
...
* examples : add persistent chat
* examples : fix whitespace
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-05-19 20:39:51 +03:00
Jason McCartney
7694b52b9a
main : make reverse prompt option act as a stop token in non-interactive mode ( #1032 )
...
* Make reverse prompt option act as a stop token in non-interactive scenarios
* Making requested review changes
* Update gpt_params_parse and fix a merge error
* Revert "Update gpt_params_parse and fix a merge error"
This reverts commit 2bb2ff1748
2023-05-19 20:24:59 +03:00
David Kennedy
79e3efb0e9
readme : adds WizardLM to the list of supported models ( #1485 )
2023-05-19 20:16:30 +03:00
Georgi Gerganov
4b7e245adf
minor : fix compile warnings
2023-05-19 20:14:51 +03:00
Erik Scholz
5ea4339273
make kv_f16 the default for api users ( #1517 )
2023-05-18 19:31:01 +02:00
DannyDaemonic
ee9654138a
Fixes #1511 lambda issue for w64devkit (mingw) ( #1513 )
...
* Fix for w64devkit and mingw
2023-05-18 19:30:40 +02:00
Concedo
f5e1fe46e1
up ver
2023-05-18 17:15:03 +08:00
Concedo
f65bae760a
Merge remote-tracking branch 'occam/opencl-dev' into concedo_experimental
...
# Conflicts:
# ggml-opencl.cpp
2023-05-18 15:52:35 +08:00
Concedo
dae9a14b50
disable CL f16 for now until it's sorted out
2023-05-18 14:23:56 +08:00
0cc4m
b73c437e83
Fix convert_row_f16 kernel issue
2023-05-18 08:05:19 +02:00
0cc4m
0df55da4ca
Deduplicate dequant kernels
2023-05-18 07:35:40 +02:00
Stephan Walter
dc271c52ed
Remove unused n_parts parameter ( #1509 )
2023-05-17 22:12:01 +00:00
0cc4m
67dbd356b6
Remove redundant constant values
2023-05-17 19:20:46 +02:00
rankaiyx
c238b5873a
benchmark-matmul: Print the average of the test results ( #1490 )
2023-05-17 16:47:58 +02:00
Concedo
b8fe4bd37d
fixed wrong function call
2023-05-17 21:15:01 +08:00
Concedo
3530d4b753
added more debugging
2023-05-17 19:17:09 +08:00
Concedo
411687f300
remove more type decls from cl kernel.
2023-05-17 18:37:39 +08:00
Concedo
32bdc6b603
fix for older opencl devices, swap numbers
2023-05-17 18:25:46 +08:00
Concedo
76dc539172
change help text
2023-05-17 18:06:47 +08:00
Concedo
487ac226b4
need to set the unshuffle before loading the model
2023-05-17 17:58:21 +08:00
Concedo
31c105c746
fix typo
2023-05-17 17:18:12 +08:00
Concedo
2c6ac06936
gpu offload not working for other arch. debug in future.
2023-05-17 17:13:01 +08:00
Concedo
57230b5196
upgrade all other formats
2023-05-17 16:28:20 +08:00
Concedo
00da2a5f4e
neox is updated
2023-05-17 14:56:54 +08:00
Concedo
90fe9096b4
clean and refactoring pass before supporting newer models for different arch
2023-05-17 11:23:29 +08:00
Concedo
60ee00428b
updated lite
2023-05-17 10:26:36 +08:00
Concedo
d8d39f1ba8
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# Makefile
2023-05-17 10:07:43 +08:00
Concedo
f561fe5a4a
switch back to ofast for c
2023-05-17 10:04:54 +08:00
Concedo
504a2aa874
Merge remote-tracking branch 'fixmake/concedo' into concedo_experimental
2023-05-17 10:01:57 +08:00
Concedo
327763c21b
Merge remote-tracking branch 'occam/opencl-dev' into concedo_experimental
2023-05-17 10:01:22 +08:00
Tom Jobbins
2b2646931b
convert.py: Support models which are stored in a single pytorch_model.bin ( #1469 )
...
* Support models in a single pytorch_model.bin
* Remove spurious line with typo
2023-05-17 00:04:35 +02:00
Ilya Kurdyukov
42627421ec
~7% faster Q5_1 AVX2 code ( #1477 )
2023-05-16 18:36:47 +00:00
0cc4m
de10afa80f
Fix tensor load to device
...
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
2023-05-16 18:49:49 +02:00
horenbergerb
f29c25e7a1
hacky fix for linux cublas build
2023-05-16 12:29:04 -04:00
Concedo
417711be46
add more QOL
2023-05-17 00:11:28 +08:00
András Salamon
9560655409
define default model path once, sync path with readme ( #1366 )
2023-05-16 17:46:34 +02:00
Concedo
94ef3e81cf
inc allocation
2023-05-16 23:32:35 +08:00
Concedo
954b87eb05
working checkpoint
2023-05-16 22:33:21 +08:00
Concedo
84c1bc7822
Merge remote-tracking branch 'occam/opencl-dev' into concedo_experimental
2023-05-16 22:00:15 +08:00
0cc4m
b3ff66d87f
Fix error in convert f16 to f32 kernel call
2023-05-16 13:05:33 +02:00
Concedo
8e394f9913
progress
2023-05-16 17:29:55 +08:00
Concedo
196fbba527
Merge branch 'opencl-dev2' into concedo_experimental
...
# Conflicts:
# CMakeLists.txt
2023-05-16 17:04:33 +08:00
sandyiscool
2a5ee023ad
Add alternate include path for openblas ( #1476 )
...
In some linux distributions (fedora, for example), the include path for openblas is located at '/usr/local/include'
2023-05-16 10:30:15 +02:00
Concedo
554340f565
revert library back first
2023-05-16 15:45:05 +08:00
0cc4m
342d346c13
Generate dequant_mul_mat kernels from simple templates
2023-05-16 07:42:01 +02:00
Concedo
e4e6994353
Not working, don't use. testing a merge
2023-05-16 12:33:24 +08:00
0cc4m
1747c598fa
Fix CMakeLists.txt
2023-05-15 19:51:23 +02:00
