vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

gguf-split: split and merge gguf per batch of tensors (#6135)

Browse source 
* gguf-split: split and merge gguf files per tensor

* gguf-split: build with make toolchain

* gguf-split: rename `--split-tensors-size` to `--split-max-tensors`. Set general.split_count KV to all split

* split : minor style + fix compile warnings

* gguf-split: remove --upload not implemented

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
This commit is contained in:
Pierrick Hymbert 2024-03-19 12:05:44 +01:00 • committed by GitHub
parent b80cf3b2d1
commit d0d5de42e5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 510 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
examples/gguf-split/CMakeLists.txt Normal file
View file

@ -0,0 +1,5 @@
set(TARGET gguf-split)
add_executable(${TARGET} gguf-split.cpp)
install(TARGETS ${TARGET} RUNTIME)
target_link_libraries(${TARGET} PRIVATE common llama ${CMAKE_THREAD_LIBS_INIT})
target_compile_features(${TARGET} PRIVATE cxx_std_11)