Concedo
0fe3c9cf96
stronger banning bias
2023-09-01 01:25:23 +08:00
Concedo
fe4a233d79
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .devops/tools.sh
# llama.cpp
2023-09-01 00:47:06 +08:00
DannyDaemonic
e8422de39e
@vxiiduu's fix for PrefetchVirtualMemory ( #2930 )
...
Reimplement fix for `PrefetchVirtualMemory`.
Co-authored-by: vxiiduu <73044267+vxiiduu@users.noreply.github.com>
2023-08-31 04:21:45 -07:00
Concedo
bc02f7663f
allow sse3 in failsafe
2023-08-31 18:07:17 +08:00
Concedo
07b02af8bc
fixed tab ordering , update lite for panel alignment
2023-08-31 16:33:00 +08:00
Concedo
e2fd30b5d1
reverted the failsafe removal, since they dropped support for dll check
2023-08-31 15:39:32 +08:00
Cebtenzzre
92d0b751a7
convert : fix python 3.8 support, modernize type annotations ( #2916 )
...
* convert : fix python 3.8 support
* convert : sort imports
* convert : fix required parameters in convert-llama-ggmlv3-to-gguf
* convert : fix mypy errors in convert-llama-ggmlv3-to-gguf
* convert : use PEP 585 generics and PEP 604 unions
Now that we have `from __future__ import annotations`, we can use this
modern syntax in Python 3.7 instead of restricting support to Python 3.9
or 3.10 respectively.
* gguf.py : a tuple is already a tuple
* add mypy.ini
* convert : add necessary `type: ignore` comments
* gguf-py: bump version
2023-08-31 08:02:23 +03:00
Johannes Gäßler
8afe228000
CUDA: mul_mat_q=true llama_context_params default ( #2912 )
2023-08-30 21:46:19 +02:00
Concedo
b6914ebd04
hotfix to revert the auto ctx scaling first, i didnt do it properly
2023-08-31 00:58:52 +08:00
Henri Vasserman
71d6975559
[Docker] fix tools.sh argument passing. ( #2884 )
...
* [Docker] fix tools.sh argument passing.
This should allow passing multiple arguments to containers with
the full image that are using the tools.sh frontend.
Fix from https://github.com/ggerganov/llama.cpp/issues/2535#issuecomment-1697091734
2023-08-30 19:14:53 +03:00
Concedo
5cd0309610
renamed incorrect identifier
2023-08-30 23:06:39 +08:00
Concedo
0ee394ae1b
falcon disable offload only for clblast
2023-08-30 22:35:24 +08:00
Concedo
29757de61f
cmake disable buggy logs
2023-08-30 22:15:33 +08:00
Concedo
aa4ad830e2
log.h is broken so disable it first
...
Merge branch 'master' into concedo_experimental
# Conflicts:
# .github/workflows/build.yml
# .gitignore
# Makefile
# README.md
# tests/CMakeLists.txt
2023-08-30 21:58:54 +08:00
Concedo
a2a4eefa07
slight change to logits
2023-08-30 21:27:51 +08:00
Georgi Gerganov
b532a69b2f
convert.py : use dir name to name the llama
2023-08-30 13:29:40 +03:00
Concedo
1301bd7e29
Fix to skip GPU offloading so falcon models work correctly
2023-08-30 18:26:41 +08:00
Georgi Gerganov
c90d135eb4
examples : fix underscore in beam-search + .gitignore ( close #2900 )
2023-08-30 12:53:24 +03:00
M. Yusuf Sarıgöz
0d1c706181
gguf : add workflow for Pypi publishing ( #2896 )
...
* gguf : add workflow for Pypi publishing
* gguf : add workflow for Pypi publishing
* fix trailing whitespace
2023-08-30 12:47:40 +03:00
alonfaraj
9509294420
make : add test and update CI ( #2897 )
...
* build ci: run make test
* makefile:
- add all
- add test
* enable tests/test-tokenizer-0-llama
* fix path to model
* remove gcc-8 from macos build test
* Update Makefile
* Update Makefile
2023-08-30 12:42:51 +03:00
Concedo
d4c22a8b02
updated lite, added autorope config based on trained ctxlen, hotfix for falcon gpu broken
2023-08-30 16:50:55 +08:00
Gilad S
35092fb547
docs : add node-llama-cpp to README.md ( #2885 )
2023-08-30 11:40:12 +03:00
Kerfuffle
dc07dc492e
convert : various script cleanups/fixes + merges and special token handling ( #2842 )
...
* convert: Fix permute calls and method/func definitions
* Cleanups for gguf-py
* Minor types cleanups.
* Initial implementation of handling merges and special tokens
* convert: Handle special tokens and merges in vocab only mode
convert: Vocab only mode no longer requires loading model tensors
* gguf: Refactor tensor name mapping
* convert: Fix type hint for special_token_types in SpecialVocab
* Use common special vocab handling in various conversion scripts
* First pass at implementing suggested changes
* Second pass
* gguf: SpecialVocab: Fix issue with special token content not in a dict
gguf: SpecialVocab: Allow skipping handling of merges
* convert-falcon-hf-to-gguf: Support --vocab-only option, bail out if no tokenizer.json
* convert-gptneox-hf-to-gguf and convert: Only handle merges for BPE tokenizer
* gguf: SpecialVocab: Actually set load_merges in object
* Uniform args parsing and vocab only mode for convert examples
* convert.py: Set gpt2 as tokenizer model when using BPE
* Squish last type warning in gguf.py - yay!
2023-08-30 11:25:50 +03:00
chaihahaha
ad9ddcff6e
llm.vim : stop generation at multiple linebreaks, bind to <F2> ( #2879 )
2023-08-30 09:50:55 +03:00
staviq
8341a25957
main : log file ( #2748 )
...
* initial, base LOG macro
* add *.log to .gitignore
* added basic log file handler
* reverted log auto endline to better mimic printf
* remove atomics and add dynamic log target
* log_enable/disable, LOG_TEE, basic usage doc
* update .gitignore
* mv include to common, params, help msg
* log tostring helpers, token vectors pretty prints
* main: replaced fprintf/LOG_TEE, some trace logging
* LOG_DISABLE_LOGS compile flag, wrapped f in macros
* fix LOG_TEELN and configchecker
* stub LOG_DUMP_CMDLINE for WIN32 for now
* fix msvc
* cleanup main.cpp:273
* fix stray whitespace after master sync
* log : fix compile warnings
- do not use C++20 stuff
- use PRIu64 to print uint64_t
- avoid string copies by using const ref
- fix ", ##__VA_ARGS__" warnings
- compare strings with == and !=
* log : do not append to existing log + disable file line func by default
* log : try to fix Windows build
* main : wip logs
* main : add trace log
* review: macro f lowercase, str append to sstream
* review: simplify ifs and str comparisons
* fix MSVC, formatting, FMT/VAL placeholders
* review: if/else cleanup
* review: if/else cleanup (2)
* replace _ prefix with _impl suffix
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-30 09:29:32 +03:00
Cebtenzzre
849408957c
tests : add a C compliance test ( #2848 )
...
* tests : add a C compliance test
* make : build C compliance test by default
* make : fix clean and make sure C test fails on clang
* make : move -Werror=implicit-int to CFLAGS
2023-08-30 09:20:26 +03:00
Concedo
89495c0716
handle token unbanning over api
2023-08-30 10:51:49 +08:00
Concedo
f2c02dd06d
Merge branch 'master' into concedo_experimental
...
# Conflicts:
# .gitignore
# CMakeLists.txt
# Makefile
# README.md
# tests/test-grad0.cpp
2023-08-30 10:51:28 +08:00
YellowRoseCx
d7bdfbdd78
Update Makefile for misc amd gpu targetting ( #407 )
...
adds the hipBlas gpu_target $(shell $(ROCM_PATH)/llvm/bin/amdgpu-arch)
back to the gpu_target line, possibly allowing misc gpu arch's like gfx1031 or gfx1032 etc to be built
2023-08-30 09:54:15 +08:00
slaren
06abf8eeba
ggml : add view_src and view_offs to ggml_tensor for views ( #2874 )
...
* ggml : add view_src and view_offs
* update ggml-alloc to use view_src
* update ggml_diag_mask to work correctly with automatic inplace
* exclude other ops that set an inplace flag from automatic inplace
2023-08-29 23:24:42 +02:00
slaren
c03a243abf
remove outdated references to -eps and -gqa from README ( #2881 )
2023-08-29 23:17:34 +02:00
Kawrakow
fa3582f509
Tell users attmepting to run perplexity with too few tokens to use more ( #2882 )
...
Closes #2858
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-29 23:55:45 +03:00
Kawrakow
e37e69dcc3
10X faster BPE tokenizer ( #2876 )
...
* 10X faster BPE tokenizer
* Remove comment that no longer applies
---------
Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
2023-08-29 23:55:03 +03:00
Concedo
380fa0f0ca
fixed broken typical sampler issues
2023-08-29 23:50:59 +08:00
maddes8cht
53885d7256
py : fix "usage" messages ( #2873 )
...
convert-to-gguf python scripts
2023-08-29 16:51:02 +03:00
jameswu2014
bcce96ba4d
convert.py : fix baichuan7B support ( #2870 )
...
* [Fix]: convert.py support baichuan7B
* convert.py : fix trailing whitespaces
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-29 12:48:41 +03:00
Jhen-Jie Hong
74e0caeb82
readme : add react-native binding ( #2869 )
2023-08-29 12:30:10 +03:00
Cebtenzzre
d4b5e16c32
make : fix clang tests build, add missing examples ( #2859 )
...
* make : do not pass headers to the compiler
This fixes building tests with clang.
* make : add missing examples
* make : fix build-info.h dependencies
2023-08-29 11:42:41 +03:00
Georgi Gerganov
3a007648f2
metal : add option to disable debug logs ( close #2764 )
2023-08-29 11:33:46 +03:00
Georgi Gerganov
611363ac79
scripts : add pipefail
2023-08-29 10:50:30 +03:00
Marcus Dunn
95b6e5212f
added struct to llama_dump_timing_info_yaml's llama_context ( #2857 )
...
fixes C compat.
2023-08-29 09:33:27 +03:00
xaedes
44c117f41e
train : mem usage and other improvements ( #2439 )
...
* fix track_max_mem in forward_batch_wo_cache_flash_attn_train
* remove unnecessary Adam(W) optimizer tensors.
reduces optimizer memory overhead from 7*modelsize to 2*modelsize.
additionally allows to optimize models with more than 2^31 parameters by replacing int with int64_t.
bumps training checkpoint file version, but old checkpoints can still be read.
new version with less tensors is saved.
* add gradient clipping to AdamW
* Fix reset of unused g->nodes and g->grads to NULL
* implement gradient checkpointing for training
reduces memory overhead from O(n_layer) to O(sqrt(n_layer))
as explained in readme of https://github.com/cybertronai/gradient-checkpointing
* remove unused compute buffer 3
* add and use function ggml_build_backward_expand to avoid stack overflows with large maximum number of nodes
GGML_API void ggml_build_backward_expand(struct ggml_context * ctx, struct ggml_cgraph * gf, struct ggml_cgraph * gb, bool keep);
* change AdamW decay parameter to work like the torch AdamW decay parameter
It is now relative to Adam learning rate `alpha*sched`.
Before that it was relative to `sched` only.
`alpha` being the maximum learning rate and `sched` being a scaling parameter in [0..1]
* change default AdamW weight decay parameter used in training to 0.1 as used in nanoGPT
* change default AdamW weight decay parameter defined in ggml to 0.0, making Adam default instead of AdamW
btw: the default weight decay parameter for torch.optim.AdamW is 0.01
* bug fixes for cross entropy loss
ggml_cross_entropy_loss: sums where not correctly added in workload of each thread
ggml_cross_entropy_loss_back: simplify backward process, reducing numerical issues
guard usage of exp f16 lookup in cross entropy by #define GGML_CROSS_ENTROPY_EXP_FP16
cross entropy loss is only used once during training, but it is quite sensitive to numerical errors introduced by exp-f16-lookup.
so exp-f16-lookup for cross entropy loss is disabled by default, trading better gradients for very slightly worse runtime performance.
* fix test-grad0 for cross_entropy_loss
the second argument to cross_entropy_loss must sum up to 1 for each row
* fix test-grad0 for soft_max
dont use only sum as aggregation, because sum of softmax is always 1 -> finite differences should not work
instead use sum(log(soft_max()*(1-eps)+eps)); use eps to avoid log(0)
* improve finite differences of test-grad0 by using double instead of float
* change cross_entropy_loss to output average over all rows
this helps keeping the loss and gradients in a sane range
* improve gradient checkpointing
sqrt(n_layers) is only the best checkpoint step when mem size of checkpoints and mem size of layers are equal.
since layers require more memory than the single-tensor-checkpoint we use, the optimal values are compute different:
```
given: n, u, v
objective: minimize(a*u+b*v) where a*b=n, a>0, b>0
b=n/a
minimize(a*u+v*n/a)
diff(a*u+v*n/a, a) = u - (v*n/a)/a
diff(a*u+v*n/a, a) == 0
u - (v*n/a)/a == 0
u == v*n/(a*a)
u*a*a = v*n
a*a = v*n/u
a = sqrt(n*v/u)
```
this change results in more checkpoints, requiring less layers to store between checkpoints, overall improving memory usage.
* disable gradient checkpointing debug output
* llama : fix rope usage in train-text-from-scratch after ChatGLM change
* add more training parameters:
--enable-restart N Only for Adam optimizer. Enable restarts of cos-decay
--disable-restart N Only for Adam optimizer. Disable restarts of cos-decay
--opt-past N Number of optimization iterations to track for delta convergence test. Disabled when zero.
--opt-delta N Maximum delta for delta convergence test. Disabled when <= zero.
--opt-max-no-improvement N Maximum number of optimization iterations with no improvement. Disabled when <= zero.
--adam-epsf N AdamW epsilon for convergence test. Disabled when <= zero.
--adam-min-alpha N Adam minimum learning rate alpha, usually 0.1 * alpha
* replace memcpy with reshape operation so that the graph is not cut at the input
this makes it possible to store other values into the input tensor and then simply recompute the graph without rebuilding it
* remove unused function argument from get_example_targets_batch
* measure and print total training time
* add optimization callback to ggml_opt_resume_g
this callback is called before each iteration with custom data and pointer to learning schedule parameter (only used in Adam(W)).
can be used for dynamic learning schedule and setting input data for batches before each iteration
* use optimization callback in training
allows dynamic learning schedule and different batch data for each iteration without relying on low n_iter and high n_examples parameters
reduces runtime by avoiding restart of optimization function and improves training convergence by providing a different batch for each iteration
* add minimum number of tensor dimensions to apply weight decay (default 2)
this allows to not apply weight decay to bias parameters
* rename training parameter cos-decay-alpha to cos-decay-min and clarify that adam-min-alpha also applies to warmup
* fix increase of model.train_samples and model.train_tokens
now that each optimizer iteration gets its own batch we need to multiply by number of opt iterations
* change sampling parameters for prediction after training to defaults of common.h
and clarify what is context for prediction and what are generated tokens
* tighten abs error bounds for cross_entropy_loss in test-grad0
* add conditional compilation of using F16 exp in flash attention
uncomment `// #define GGML_FLASH_ATTN_EXP_FP16` to enable usage of f16 exp in flash attention
* tighten abs error bounds for flash_attn in test-grad0
* tighten abs error bounds for sqrt in test-grad0
* remove out-commented vectorized code of opt_adam
the vectorized code might be bit faster for low number of parameters, but it had a big memory usage overhead
* ggml : update ggml_rms_norm_back with configurable eps
* llama training : fix ggml_rms_norm_back calls to pass configurable eps
* remove trailing whitespace
* add train function using automatic gradient checkpointing backward pass and allocator
* in train function replace add_inplace by regular add
because using add_inplace seems to result in different gradients
* don't use allocate hash_map on context
because the context has no_alloc=True when using memory allocator resulting in NULL data pointers
* correctly clone reshape and permute operations by also cloning tensor->nb values
* fix variable name and add missing type cast
* terminate recursive tensor cloning when reaching tensor without src tensors
* correctly clone view tensors by setting data pointers
without this the checkpointing would only work when being used together with memory allocator
* fix variable names
* swap arguments to commutative ops to be the same as in `forward_batch_wo_cache_flash_attn`
* add input tensors as checkpoints
so that recursive tensor cloning of gradient checkpointing terminates on input tensors
* fix variable name and add missing boolean negation
* make sure some tensors are not reallocated by inserting new temporary nodes depending on them:
output and parameter gradient tensors need to be available at the end of the graph execution
parameter gradient tensors also need to be available before the graph execution because they are set to zero before each optimizer iteration
checkpoint tensors are allocated all together to reduce memory allocator fragmentation
afterwards, in addition to the temporary nodes, we also need to reset the temporary leafs
* fix ASSERT to work with zero layers
* add training options whether to use allocator and/or unified training function
* integrate unified training function which may use memory allocator
the unified training function also supports arguments whether to use flash attention and/or gradient checkpointing
* format name of cloned tensors with " (clone)" suffix
* set names for tensors in unified train function for easier debugging
* allocate graph on context using ggml_new_graph
* remove handwritten training functions
* remove unused training parameters "use_scratch" and "use_unified"
* remove trailing whitespace
* remove unused train params: mem_compute1_gb & mem_compute2_gb
mem_compute_gb is used for compute when automatic memory allocator is not enabled, otherwise it can be very small to only hold the tensor definitions
mem_compute0_gb is used for automatic memory allocator (as long as measurement of max required size is not implemented)
* remove unused forward_batch function
* add debug asserts in ggml_allocr_alloc to some common pitfalls when using this function directly
* only use ggml_allocr_alloc when tensor has NULL data and is no view
* fix test when to create temporary backward graph
temporary backward graph is only necessary when using checkpointing
* fix memory "leak" in optimizers
each iteration a new cplan with new memory for work data was allocated.
now cplan creation only happens at the start of optimization, with each iteration reusing the cplan and its work data.
* reverse order of for loop in ggml_build_backward_expand to save memory when using gradient checkpointing and allocator
with this loop order gradient checkpointing with allocator on 16 layer model saves 13% memory; 2 layer memory it saves 2% memory.
the computation results are the same
* add missing lctx argument to get_example_targets_batch
* implement llama model file saving using gguf
checkpoint loading and saving disabled, to be replaced by loading and saving via gguf
* implement loading/saving of checkpointing files using GGUF
* bug fixes
* add checkpoint file version for future compatibility
* update readme with gguf filenames
* save & load opt->just_initialized value
* add first draft for checkpoint conversion script
* add gguf arch and ftype
* save opt parameter counter as uint64
* add gguf key and tensor names for optimizer and training
* add layer_norm_rms_eps to checkpoint convert script
* use same GGUF_GET_KEY macro as in llama.cpp
* use norm_rms_eps, and rope parameters and command line options to set them
* fix memory corruption bug in gguf
ctx->kv and ctx->infos was reallocated using not-aligned realloc, but freed with aligned free.
to fix this a GGML_ALIGNED_REALLOC was added, but there is no posix_memalign_realloc function.
so on non-windows and non-mingw32 platforms we fall back to aligned malloc, followed by copying
and freeing the old data.
* add gguf example cmake file
* bug fixes in tokenize_file
* bug fixes in load_llama_model_gguf
* bug fix: init model when no checkpoint was loaded
* bug fix in read_tensor_by_name
* bug fix in load_opt_context_gguf
* avoid printing lots of spaced on the unusual case that loss gets nan
* set name of tensors with empty name from what was read from gguf
* remove trailing whitespace
* print data checksums before saving and after loading to verify correctness
* bug fixes for convert-train-checkpoint-to-gguf
* temporarily add code to write old checkpoint files
used to verify that old checkpoint files are correctly converted to gguf
* bug fixes for convert-train-checkpoint-to-gguf.py loading checkpoints with opt_version=0
* remove code used to verify correctness of checkpoint file conversion
* remove trailing whitespace
* remove prediction related code
use main for prediction, it is better optimized
* update train-text-from-scratch README.md
* fix non-windows GGML_ALIGNED_REALLOC
* add missing blank line at end of file
* remove GGML_ALIGNED_REALLOC and use normal malloc/realloc/free for gguf ctx->kv & ctx->infos
* train : fix compile warnings
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-28 22:51:47 +03:00
slaren
43033b7bb4
llama-bench : set locale to utf8 ( #2832 )
2023-08-28 19:19:18 +02:00
Johannes Gäßler
6b73ef1201
YAML result logging + preset script ( #2657 )
2023-08-28 17:59:39 +02:00
alonfaraj
75fafcbccc
make : fix tests build ( #2855 )
...
* makefile:
- fix test name
- add missing tests build
* editorconfig : fixes
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2023-08-28 18:38:35 +03:00
grahameth
be475f60af
llama.cpp : fix wrong vsnprintf call in MS compiler ( #2856 )
...
Co-authored-by: grahameth <->
2023-08-28 18:38:12 +03:00
Ronny Brendel
3af6b86301
ggml : tiny ggml_vec_dot_q4_K_q8_K AVX2 improvement ( #2819 )
2023-08-28 15:51:08 +03:00
Georgi Gerganov
35feac6560
ggml : sync (mem align to header + conv_transpose_2d fixes + ggml_alloc) ( #2852 )
...
* ggml : sync (mem align to header + conv_transpose_2d fixes)
ggml-ci
* ggml-alloc : minor fix
* ggml-alloc : sync more fixes
2023-08-28 14:24:53 +03:00
Johannes Gäßler
92b1bbd2ec
CUDA: fix RoPE asserts, block sizes ( #2833 )
2023-08-28 14:23:55 +03:00
YellowRoseCx
cf5d918073
Koboldcpp-ROCm Port ( #399 )
...
* koboldcpp-ROCm Port
commit 3416c986d9d9a31c3cdefd7e7bd4d9438d72ba35
Merge: 5eb17f0 4c4e435#5 )
* add hipBLAS to README
* new build arg LLAMA_CUDA_MMQ_Y
* fix half2 decomposition
* Add intrinsics polyfills for AMD
* AMD assembly optimized __dp4a
* Allow overriding CC_TURING
* use "ROCm" instead of "CUDA"
* ignore all build dirs
* Add Dockerfiles
* fix llama-bench
* fix -nommq help for non CUDA/HIP
---------
Co-Authored-By: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Co-Authored-By: ardfork <134447697+ardfork@users.noreply.github.com>
Co-Authored-By: funnbot <22226942+funnbot@users.noreply.github.com>
Co-Authored-By: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Co-Authored-By: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
Co-Authored-By: jammm <2500920+jammm@users.noreply.github.com>
Co-Authored-By: jdecourval <7315817+jdecourval@users.noreply.github.com>
commit b34f4bd2724733e188ec4f6074042f66a5ed28c9
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Aug 19 17:12:52 2023 -0500
Update README.md
commit 7d1196108ad330b32845546fb3472c2172a0b6b8
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Mon Aug 14 23:03:12 2023 -0500
remove force DMMV
commit cd61aa0d9e16627935c7978adf488a679ddfa745
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Aug 12 17:24:31 2023 -0500
restore main_gpu parameter
commit 4a042f326830271a4c31104051b7b08e08ac234e
Author: Henri Vasserman <henv@hot.ee>
Date: Sat Aug 12 10:51:46 2023 +0300
gfx1100 support
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
Co-authored-by: jammm <2500920+jammm@users.noreply.github.com>
Co-authored-by: jdecourval <7315817+jdecourval@users.noreply.github.com>
commit 8913bc6fea97d3cb860937b0461f455c6abe3ea1
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 10:16:02 2023 +0300
Allow overriding CC_TURING
commit e77a4c37a756c002e97173f4122e088fb304e18a
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 10:00:07 2023 +0300
Merge 'origin/master' into hipblas
commit cc4c4e355cd553b1557d5fba2562e824db93f9b4
Author: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Date: Fri Aug 11 09:43:14 2023 +0300
New __dp4a assembly
Now compatible with gfx900 and faster as well.
commit 1a03b709848ce68d5bf5966237756167e2cac540
Author: Henri Vasserman <henv@hot.ee>
Date: Fri Aug 11 09:30:28 2023 +0300
Undo mess
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
commit 4366ff9ba1b1f12e494118ef9b5198479022fcc5
Author: DannyDaemonic <DannyDaemonic@gmail.com>
Date: Thu Aug 10 13:11:36 2023 -0700
Handle `ENABLE_VIRTUAL_TERMINAL_PROCESSING` more gracefully on earlier versions of Windows.
commit 811ff855a24323cafddc95c1b8aca711fef05f76
Author: Christian Demsar <crasm@git.vczf.us>
Date: Thu Aug 10 10:28:27 2023 -0400
Add --n-predict -2 for stopping generation on full context (#2565 )
commit 37c9717aaa6815b6a5be21aaab970212f20fe6bf
Author: Martin Krasser <krasserm@googlemail.com>
Date: Thu Aug 10 12:16:38 2023 +0200
Fix grammar-based sampling issue in server (#2566 )
commit d18ecd5b9e5dde58ae08a3eef1637406159ddaca
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Thu Aug 10 13:19:41 2023 -0500
make mmq gen faster for amd
commit 243894a952147a4fac5b6aee748861a0df6cc2c6
Author: Henri Vasserman <henv@hot.ee>
Date: Thu Aug 10 12:14:40 2023 +0300
ws fix
commit ac2f14da445ea87d73539adbd29d19ff2c9eba58
Author: Engininja2 <139037756+Engininja2@users.noreply.github.com>
Date: Thu Aug 10 12:11:27 2023 +0300
AMD assembly optimized __dp4a
Doesn't seem to work for gfx900, so commented out.
commit 9dba0c985f140ddded8cbb671f139e81fff82eed
Author: Henri Vasserman <henv@hot.ee>
Date: Thu Aug 10 12:09:28 2023 +0300
Fix merge
---------
Co-authored-by: ardfork <134447697+ardfork@users.noreply.github.com>
Co-authored-by: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
commit f570b5cb1070591527a82d94bba408927b37778d
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 22:11:20 2023 -0500
Revert "revert cuda changes as they are bugggy"
This reverts commit 1541bf879772aeeed8ff646bfc52185c2a88b79b.
commit 1541bf879772aeeed8ff646bfc52185c2a88b79b
Author: Concedo <39025047+LostRuins@users.noreply.github.com>
Date: Wed Aug 9 22:36:41 2023 +0800
revert cuda changes as they are bugggy
commit bacc20203efb1839aa313858a04d75255bb4b7f4
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 20:37:17 2023 -0500
Merge remote-tracking branch 'upstream/concedo'
commit b7cb4cfd109986bd66e8fd382d1e2516eaddfebb
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Aug 9 20:00:52 2023 -0500
additional fixes
commit fadae727baa3735ad3e0667384d6e05ca056b3ef
Merge: 518eb2a 8f8ab6ccae6a848f8ab6c4c0610ba4cfc44024f9125d43e04024f91a66ab6212864cd91456af5bfea0d91456aaf1c1cb70d64dc1664a00ae43362310728c5ab40550c31486eb4336231a32f8e3fc6c74d2ade639f4cde52d68a88e5894e0a069731682cde52d6a638e8054a84e09a78e8054ad831f6294dc44#5 )
* initialize rocblas
commit 5b838d47874536ebffc2f6cb25877e0476a9402d
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Mon Jul 24 03:10:35 2023 -0500
amd multigpu full layer offload w/o vram scratch
commit 9bfb2fdd68000670bda85c4e9748d72f5af09764
Merge: b379f9d 66328fc56995caf0361093db70b5f0a2ec44667568d1a2ec4466db5cd36b185ffafcb8fe1cbf5615941514afcb8fe0c48c2c4978a3e6104662347463e61046630780e4e548bf7735c5a1d16309#2140 )
commit 4539bc2761a7a23b588b5420b9d3fd1962ff63e5
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Sat Jul 8 01:36:14 2023 -0500
update makefile for changes
commit 912e31ec523eac9ef308f0d28bc2d93aab7c3ecb
Merge: 74e2703 ddaa4f2f9108ba7735c5a9afc3e37337ee76e43d2907dc3e3733c6115db19ae7b04419f146088f7eda663f1347d3a04419f1894bb16effd3494bbbb16effc75#1 )
* kquants_iter for hipblas and add gfx803
* Update CMakeLists.txt with hipblas kquants_iter and DMMV_F16
* remove dmmv_f16 for now
commit 096f0b055e11b7d930842f86146d0e5013c5dce6
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Jun 28 15:27:02 2023 -0500
revert unnecessary hipblas conditionals
commit d81e81adffd6eb59e280ae1885864bb5fbd9bba6
Author: YellowRoseCx <80486540+YellowRoseCx@users.noreply.github.com>
Date: Wed Jun 28 14:48:23 2023 -0500
Update Makefile hipblas nvcc correction
commit c8ae94524ac1e5c830be54f7d2034cec1e5c8345e35a6031447ccbe35a603161adf7346c66a25558342fe8df7346ccd55dd2fbe7487137e6ddb15b1f00fa266d47a5dd2fbe6ea67e229b20568fe67e229b7ca6f7c156b2416496f7c15637a61df8e9fc45a8161df8e9217a83652999685f902d254a7a785f902d5c44362e80b50b5704362e805a4fa5b3d717366dffa5b3d73651ba4ce4ad75d6eb721645d6eb7216433091a9bd39fdaa1d2d433879fdaa1d250#1607
commit a4648c1e7c4c8b3fb0ecb1bb4c8b3fb10730d921af3ea593a4f6c2174bf6a86df80ce7a1fcdcc2f80ce7a4e0600ace3ac7876a600ace39c8b19fefef94c66115b833a0b2d5fb8ee340a0b2d5f2918bab4562a5ee028bab45611e2956630b5c92952956630a3d0fe6384f048af00fe6384755605560d9ec127f68e089b1c9127f68eb5a070cbccb608b55070cbcc1bda3296d50aa0aefa6ae1295510aefa6ab71baeb4821b0fd45baeb482a94289073a5321107194173d0e61107194e6b04c0d48a3b85b204c0d480d7d83cfbad0cb67cc50799fdc1b67cc50dadfcbc262e216aa0fcbc262eb9c73def1f4cef87c73def129ad8ea75ef0d70f1d8ea75e952d194586334637ed194586f652ab9d117f15c5c2ab9d11f373b4a53104aaae13b4a53138fa1caa480b2da20a1caa48611ecc056519fef51e9ecacd571d164afcc37d571d1629f608aa33dd0eabc608aa33d9f3a004b2a01db7a01297e3677235284685f367723544cd3e1984ce00e005f779354a63c10e80fd8363adc
2023-08-28 17:05:06 +08:00
