Commit graph

  • 20a55f014b doc: fix thw wrong address of BLIS.md Aisuko 2023-06-09 04:28:05 +00:00
  • 45d34f4041 Add gelu implementation to metal. Adam Treat 2023-06-08 23:30:49 -04:00
  • fd0eb663ce llama, main : constrain sampling to grammar Evan Jones 2023-05-31 00:20:51 -04:00
  • db8781dfd0 Rename function name edp1096 2023-06-09 09:23:52 +09:00
  • a3a4087643
    Create msbuild top 2023-06-09 05:56:13 +07:00
  • 72ff5282bf
    metal : add Q2_K implementation (#1762) Kawrakow 2023-06-08 22:28:21 +03:00
  • ccd85e0a6b
    Apply suggestions from code review Henri Vasserman 2023-06-08 22:17:46 +03:00
  • 61befcba7b
    Apply suggestions from code review Henri Vasserman 2023-06-08 22:14:43 +03:00
  • 5a46032f31 Fixing merge conflicts Iwan Kawrakow 2023-06-08 20:17:45 +03:00
  • 1c97f2919a metal : add Q2_K implementation Iwan Kawrakow 2023-06-08 18:07:47 +03:00
  • 0bf7cf1b29
    Revert "ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738)" master-0bf7cf1 Georgi Gerganov 2023-06-08 20:48:14 +03:00
  • 8432d4d9f7
    ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738) master-8432d4d le.chang 2023-06-09 00:47:56 +08:00
  • 6fa1613f15
    Metal inference enhancement - put hard-wired relative path of ggml-model.model file using a patch file due to lack of NSBundle environment Hyun-joo KIM 2023-06-09 01:47:36 +09:00
  • 0f291e1f65
    metal : Q6_K implementation (#1752) Kawrakow 2023-06-08 19:46:22 +03:00
  • dee692a63e
    compability with basic_api, change api path to /extra SammCheese 2023-06-08 15:56:25 +02:00
  • b4e9e185d3
    fix legacy streaming SammCheese 2023-06-08 15:21:00 +02:00
  • 9a8da35ec4
    working streaming. TODO: fix lite SammCheese 2023-06-08 06:18:23 +02:00
  • 97971291e9
    draft: token streaming SammCheese 2023-06-07 00:48:00 +02:00
  • 7f181600c7
    Metal inference enhancement - put hard-wired relative path of ggml-model.model file due to lack of NSBundle environment Hyun-joo KIM 2023-06-09 01:24:22 +09:00
  • c1be42404a Much better Q6_K implementation for metal Iwan Kawrakow 2023-06-08 18:59:49 +03:00
  • 0ae7b6ea11
    Merge branch 'master' into tensor-layer-num Daniel Kuntz 2023-06-08 10:59:56 -04:00
  • d18245e57a Fixed indent Daniel Kuntz 2023-06-08 10:54:11 -04:00
  • a6a0fa338a cleanup indentation, fixing cublas build Concedo 2023-06-08 22:40:53 +08:00
  • b9f7cf7024 added -funsafe-math-optimizations flag back to armv7 l3utterfly 2023-06-08 20:41:00 +08:00
  • 2b496429c3 Fix crash in server example caused by oob due to no show words scanning - Replaced scanning code by lookahead based strategy Berthold, Alexander 2023-06-08 12:41:34 +02:00
  • d84ceb8be0 Added support for latest Android NDK which uses Clang instead of GCC l3utterfly 2023-06-08 18:47:05 +08:00
  • a50d003302 fixup! Windows nvcc bug workaround JohannesGaessler 2023-06-08 11:21:51 +02:00
  • ffd76ab684 !fixup Johannes 2023-06-08 11:00:34 +02:00
  • d811633652 !fixup Johannes 2023-06-08 10:49:50 +02:00
  • fb9d16f20f Merge branch 'master' of https://github.com/ggerganov/llama.cpp l3utterfly 2023-06-08 16:40:31 +08:00
  • a979e71ddc add obj flags to all output make targets Concedo 2023-06-08 16:28:26 +08:00
  • a5cd1955d6 !fixup Johannes 2023-06-08 10:25:09 +02:00
  • 4a82c8d45c clang-tidy : add config back Iwan Kawrakow 2023-06-08 11:08:19 +03:00
  • 85a9714316 Windows nvcc bug workaround Johannes 2023-06-08 10:02:37 +02:00
  • 8fc8179919
    Add llama.cpp docker support for non-latin languages (#1673) qingfengfenga 2023-06-08 15:58:53 +08:00
  • 85f902d5c4
    Merge 'origin/master' into hipblas Henri Vasserman 2023-06-08 10:50:28 +03:00
  • 9eb9d14216 Merge remote-tracking branch 'origin/master' into ik/q6_k_metal Iwan Kawrakow 2023-06-08 10:41:29 +03:00
  • 22d03847f5 Metal implementation for Q6_K Iwan Kawrakow 2023-06-08 10:09:52 +03:00
  • b50b570ed9
    ggml : fix fprintf warnings (#1720) master-b50b570 Steven Roussey 2023-06-08 00:12:28 -07:00
  • 53aba3f393
    clang-tidy : restore dot file from accidental deletion Georgi Gerganov 2023-06-08 10:09:08 +03:00
  • 4161bdc04d
    metal : add Q4_K implementation (#1733) Kawrakow 2023-06-08 10:08:23 +03:00
  • 0035858273
    k-quants : add missing compile definition to CMakeLists (#1748) master-0035858 johnson442 2023-06-08 08:02:48 +01:00
  • a498987551 Add missing compile definition to CMakeLists for k_quants johnson442 2023-06-08 03:51:55 +01:00
  • 6b7487d104
    store view offset and permute axes in opt[0] instead of storing it in padding xaedes 2023-06-08 02:33:57 +02:00
  • 8cea3ab9e5 add examples for llava.py ningshanwutuobang 2023-06-08 04:50:31 +08:00
  • 64a06536cb Merge remote-tracking branch 'upstream/master' Randall Fitzgerald 2023-06-07 12:23:49 -04:00
  • 6635f7efce updated lite Concedo 2023-06-08 00:20:32 +08:00
  • e2b059d335 fix fprintf warnings Steven Roussey 2023-06-07 08:44:50 -07:00
  • 6ed4893391 fixed add end condition for generating ningshanwutuobang 2023-06-07 23:44:54 +08:00
  • 49a6be3d87 add llama metal compile flags as an option Concedo 2023-06-07 22:29:38 +08:00
  • b13fa0b9c6 Add opencl release memory edp1096 2023-06-07 22:50:43 +09:00
  • 7b0707ff26 Merge branch 'master' into concedo_experimental Concedo 2023-06-07 17:06:56 +08:00
  • e3c26e815d ggml : Load data into int8x16x4_t using vld4q_s8 on arm64 lindeer 2023-06-07 16:26:29 +08:00
  • 5c64a0952e
    k-quants : allow to optionally disable at compile time (#1734) master-5c64a09 Georgi Gerganov 2023-06-07 10:59:52 +03:00
  • 63f163f0dd
    build : enable k-quants by default Georgi Gerganov 2023-06-07 10:53:59 +03:00
  • edaafec6cc
    k-quants : put behind optional compile flag LLAMA_K_QUANTS Georgi Gerganov 2023-06-07 10:14:15 +03:00
  • e78c675a6e Merge branch 'master' into concedo_experimental Concedo 2023-06-07 15:23:29 +08:00
  • 75ad0c0614 Fix after merge with master Iwan Kawrakow 2023-06-07 09:53:50 +03:00
  • a5becf764b Optimizing q4_K metal dot some more Iwan Kawrakow 2023-06-07 09:16:01 +03:00
  • 1e903f6b82 Optimizing Q4_K on metal Iwan Kawrakow 2023-06-06 20:07:37 +03:00
  • 6f8f39fbaf Metal implementation for Q4_K Iwan Kawrakow 2023-06-06 17:42:28 +03:00
  • 5b57a5b726
    flake : update to support metal on m1/m2 (#1724) jacobi petrucciani 2023-06-07 00:15:31 -04:00
  • 4dc62c545d
    readme : add June roadmap Georgi Gerganov 2023-06-07 07:15:08 +03:00
  • 92ee6a940b
    update flake to support metal on m1/m2 jacobi petrucciani 2023-06-06 23:13:23 -04:00
  • 35a84916fb
    main: add the possibility to open the prompt cache read-only (#1640) master-35a8491 Willy Tarreau 2023-06-07 04:10:17 +02:00
  • 40ed39d7a7 Modify Dockerfile default character set to improve compatibility (#1673) qingfengfenga 2023-06-07 09:59:18 +08:00
  • 994ebe1218
    Merge branch 'ggerganov:master' into master qingfengfenga 2023-06-07 09:51:20 +08:00
  • 4362e805a4
    Merge 'origin/master' into hipblas Henri Vasserman 2023-06-06 23:14:40 +03:00
  • 2d7bf110ed
    llama : fix vram_scratch var master-2d7bf11 Georgi Gerganov 2023-06-06 22:54:39 +03:00
  • 2a4e41a086
    llama : fix compile warnings Georgi Gerganov 2023-06-06 22:41:53 +03:00
  • 17366df842
    Multi GPU support, CUDA refactor, CUDA scratch buffer (#1703) master-17366df Johannes Gäßler 2023-06-06 21:33:23 +02:00
  • fa221a27cc fixup! CUDA multi GPU + scratch JohannesGaessler 2023-06-06 21:05:54 +02:00
  • 8c6c3347cc docker: add support for CUDA in docker canardleteer 2023-05-14 17:36:14 -07:00
  • 0f7f3f174d CUDA multi GPU + scratch JohannesGaessler 2023-05-23 09:17:31 +02:00
  • 44f906e853
    metal : add f16 support master-44f906e Georgi Gerganov 2023-06-06 20:16:57 +03:00
  • d5b111f53d
    Clblast fixes + enhancements to save VRAM and offload more layers (#1675) master-d5b111f LostRuins 2023-06-07 01:00:01 +08:00
  • b6dd367b45
    Update ggml-opencl.cpp 0cc4m 2023-06-06 17:55:41 +02:00
  • fa5b3d7365
    fix makefile. Henri Vasserman 2023-06-06 18:47:00 +03:00
  • 1ba4ce4ad7
    Revert "warp size fixes" Henri Vasserman 2023-06-06 18:41:08 +03:00
  • 5d6eb72164
    warp size fixes Henri Vasserman 2023-06-06 18:32:41 +03:00
  • ed603dcafc Merge branch 'master' into concedo_experimental Concedo 2023-06-06 23:12:01 +08:00
  • ba1f617d7d add free for context ningshanwutuobang 2023-06-06 22:57:04 +08:00
  • c046db5197 lite bugfixes, buffer size changes, fixed a topk bug. Concedo 2023-06-06 22:38:25 +08:00
  • 9c6117cd8d fixed sampling ningshanwutuobang 2023-06-06 22:29:34 +08:00
  • a91487093b add test example for embd input ningshanwutuobang 2023-06-06 22:06:51 +08:00
  • 33091a9bd3
    Merge 'origin/master' into hipblas Henri Vasserman 2023-06-06 16:19:23 +03:00
  • 2d43387daf
    ggml : fix builds, add ggml-quants-k.o (close #1712, close #1710) master-2d43387 Georgi Gerganov 2023-06-06 10:18:03 +03:00
  • 7ad7750c5c
    gitignore : add .clang-tidy Georgi Gerganov 2023-06-06 09:55:10 +03:00
  • 7a74dee6b4
    llama : temporary disable Q6_K output quantization (#1711) master-7a74dee Georgi Gerganov 2023-06-06 09:39:38 +03:00
  • d5b60af230 Makefile: add missing ggml-quants-k.o to server target Efruit 2023-06-06 01:27:15 -04:00
  • 590250f7a9
    metal : add checks for buffer size (#1706) master-590250f Spencer Sutton 2023-06-05 23:28:17 -04:00
  • 8dd5b0854f disable tty Spencer Kim 2023-06-05 18:00:26 -07:00
  • 827fd74b02 Add checks for buffer size on metal Spencer Sutton 2023-06-05 17:31:51 -04:00
  • b3d605dc75 Use std::runtime_error exceptions. KerfuffleV2 2023-06-05 14:33:53 -06:00
  • 7ed5aca9ca Clean up thread blocks with spares calculation a bit KerfuffleV2 2023-06-05 13:36:35 -06:00
  • fe9ed7d36f Threaded dequantizing and f16 to f32 conversion KerfuffleV2 2023-06-05 13:18:55 -06:00
  • 5c7b0e79a7 Add support for quantizing already quantized models KerfuffleV2 2023-06-04 11:33:18 -06:00
  • f4c55d3bd7
    docs : add performance troubleshoot + example benchmark documentation (#1674) Yuval Peled 2023-06-05 23:32:36 +03:00
  • 3970f417b5
    Merge branch 'master' into master Georgi Gerganov 2023-06-05 23:32:06 +03:00
  • f1465624c2
    readme : fix typo (#1700) Foul-Tarnished 2023-06-05 22:28:37 +02:00