vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
649 commits 380 branches 3056 tags 365 MiB
Commit graph

649 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andrei Betlen
66ad132575 Update llama.cpp 2023-05-31 15:56:55 -07:00
Andrei Betlen
656190750d Update llama.cpp 2023-05-31 15:56:55 -07:00
Andrei Betlen
80c18cb665 Update llama.cpp (remove llama_get_kv_cache) 2023-05-31 15:56:55 -07:00
Andrei Betlen
bf9f02d8ee Update llama.cpp 2023-05-31 15:56:55 -07:00
Andrei Betlen
5bbf40aa47 Update llama.cpp 2023-05-31 15:56:55 -07:00
Mug
fd64310276 Fix decode errors permanently 2023-05-31 15:56:55 -07:00
Mug
bdbaf5dc76 Fixed end of text wrong type, and fix n_predict behaviour 2023-05-31 15:56:55 -07:00
Andrei Betlen
81c4c10389 Update type signature to allow for null pointer to be passed. 2023-05-31 15:56:55 -07:00
Mug
8229410a4e More reasonable defaults 2023-05-31 15:56:55 -07:00
Andrei Betlen
b6ce5133d9 Add bindings for LoRA adapters. Closes #88 2023-05-31 15:56:55 -07:00
Andrei Betlen
3693449c07 Update llama.cpp 2023-05-31 15:56:55 -07:00
Andrei Betlen
d595f330e2 Update llama.cpp 2023-05-31 15:56:55 -07:00
Andrei Betlen
ce0ca60b56 Update llama.cpp (llama_mmap_supported) 2023-05-31 15:56:49 -07:00
Mug
d0a7ce9abf Make windows users happy (hopefully) 2023-05-31 15:25:57 -07:00
Mug
848b4021a3 Better custom library debugging 2023-05-31 15:25:57 -07:00
Mug
c8b5d0b963 Use environment variable for library override 2023-05-31 15:25:57 -07:00
Mug
d1b3517477 Allow local llama library usage 2023-05-31 15:25:57 -07:00
Mug
b36c04c99e Added iterative search to prevent instructions from being echoed, add ignore eos, add no-mmap, fixed 1 character echo too much bug 2023-05-31 15:25:57 -07:00
Andrei Betlen
f25a81309e Update model paths to be more clear they should point to file 2023-05-31 15:25:57 -07:00
Mug
e19909249d More interoperability to the original llama.cpp, and arguments now work 2023-05-31 15:25:57 -07:00
Andrei Betlen
d5680144c5 Bugfix: Wrong size of embeddings. Closes #47 2023-05-31 15:25:57 -07:00
Mug
29e9fb66a3 Better llama.cpp interoperability 
Has some too many newline issues so WIP

(Update) Fixed too many newlines, now onto args.

Still needs shipping work so you could do "python -m llama_cpp.examples." etc.
 2023-05-31 15:25:57 -07:00
Andrei Betlen
ce66405da1 Add quantize example 2023-05-31 15:25:57 -07:00
Mug
739e8d4c9b Fix bug in init_break not being set when exited via antiprompt and others. 2023-05-31 15:25:57 -07:00
Mug
ae1f37f505 Fix repeating instructions and an antiprompt bug 2023-05-31 15:25:57 -07:00
Mug
3c1020b866 Fix stripping instruction prompt 2023-05-31 15:25:57 -07:00
Mug
0bfad75406 Added instruction mode, fixed infinite generation, and various other fixes 2023-05-31 15:25:57 -07:00
Mug
9e872410da Add instruction mode 2023-05-31 15:25:57 -07:00
Mug
15bea0946b Chat llama.cpp example implementation 2023-05-31 15:25:57 -07:00
MillionthOdin16
2b8147e7a8 Update llama_cpp.py 2023-05-31 15:25:57 -07:00
Andrei Betlen
62ce167b22 Update low level api example 2023-05-31 15:25:57 -07:00
Andrei Betlen
a71cda6546 Update llama.cpp 2023-05-31 15:25:57 -07:00
Andrei Betlen
a279acd680 Update llama.cpp (llama_n_embd) 2023-05-31 15:25:57 -07:00
Andrei Betlen
ef3c152257 Update llama.cpp (llama_progress_callback) 2023-05-31 15:25:57 -07:00
Andrei Betlen
def46dd9a6 Add example based on stripped down version of main.cpp from llama.cpp 2023-05-31 15:25:57 -07:00
Andrei Betlen
5bb1bc74d1 Fix type signature of token_to_str 2023-05-31 15:25:57 -07:00
Andrei Betlen
a7a6d88793 Fix ctypes typing issue for Arrays 2023-05-31 15:25:57 -07:00
Andrei Betlen
019650f416 Fix array type signatures 2023-05-31 15:25:57 -07:00
Andrei Betlen
a3da39af79 Bugfix: cross-platform method to find shared lib 2023-05-31 15:24:39 -07:00
Andrei Betlen
bd1c657f80 Bugfix: wrong signature for quantize function 2023-05-31 15:24:10 -07:00
Andrei Betlen
ef5a9a6160 Update llama.cpp and re-organize low-level api 2023-05-31 15:16:27 -07:00
Andrei Betlen
d9dfdec2bd Initial commit (llama_cpp.py, llama-cpp-python) 2023-05-31 15:16:11 -07:00
Henri Vasserman
ffb06a345e
OpenLLaMA 3B support (#1588) 
This adds support to llama.cpp to load the model.

Currently missing are changes that are required from convert.py to convert the model correctly. It needs some changes to start reading the JSON configuration for HF models instead of deriving the values by guessing.

Co-authored-by: FNsi <125447286+FNsi@users.noreply.github.com>
 2023-05-30 21:24:22 +03:00
Georgi Gerganov
7552ac5863
ggml : sync cgraph import / export API 2023-05-29 19:31:44 +03:00
Georgi Gerganov
5d1830b99d
ggml : fix bug in ggml_alibi 2023-05-29 19:30:49 +03:00
DannyDaemonic
248367605e
Work around for recalculating logits in cached prompts (Fixes #1585) (#1609) 
* Work around for recalculating logits in cached prompts
 2023-05-29 05:13:40 -07:00
Jiří Podivín
0e730dd23b
Adding git in container package dependencies (#1621) 
Git added to build packages for version information in docker image

Signed-off-by: Jiri Podivin <jpodivin@gmail.com>
 2023-05-28 21:45:50 -07:00
Johannes Gäßler
3b126f654f
LLAMA_DEBUG adds debug symbols (#1617) 2023-05-28 21:01:02 +02:00
Kerfuffle
1b78ed2081
Only show -ngl option when relevant + other doc/arg handling updates (#1625) 
1. Add a `LLAMA_SUPPORTS_GPU_OFFLOAD` define to `llama.h` (defined when compiled with CLBlast or cuBLAS)
2. Update the argument handling in the common example code to only show the `-ngl`, `--n-gpu-layers` option when GPU offload is possible.
3. Add an entry for the `-ngl`, `--n-gpu-layers` option to the `main` and `server` examples documentation
4. Update `main` and `server` examples documentation to use the new style dash separator argument format
5. Update the `server` example to use dash separators for its arguments and adds `-ngl` to `--help` (only shown when compiled with appropriate support). It will still support `--memory_f32` and `--ctx_size` for compatibility.
6. Add a warning discouraging use of `--memory-f32` for the `main` and `server` examples `--help` text as well as documentation. Rationale: https://github.com/ggerganov/llama.cpp/discussions/1593#discussioncomment-6004356
 2023-05-28 11:48:57 -06:00
Vladimir Zorin
337aea1139
examples : add --alias option to gpt_params to set use friendly model name (#1614) 2023-05-28 20:14:24 +03:00