vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Introduce C-style API (#370)

Browse source 
* Major refactoring - introduce C-style API

* Clean up

* Add <cassert>

* Add <iterator>

* Add <algorithm> ....

* Fix timing reporting and accumulation

* Measure eval time only for single-token calls

* Change llama_tokenize return meaning
This commit is contained in:
Georgi Gerganov 2023-03-22 07:32:36 +02:00 • committed by GitHub
parent da0e9fe90c
commit f5a77a629b
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
14 changed files with 1954 additions and 1752 deletions
Download patch file Download diff file Expand all files Collapse all files

7
ggml.h
View file

@ -741,6 +741,13 @@ enum ggml_opt_result ggml_opt(
struct ggml_opt_params params,
struct ggml_tensor * f);
//
// quantization
//
size_t ggml_quantize_q4_0(float * src, void * dst, int n, int k, int qk, int64_t * hist);
size_t ggml_quantize_q4_1(float * src, void * dst, int n, int k, int qk, int64_t * hist);
//
// system info
//