vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Adding a simple program to measure speed of dot products (#1041)

Browse source 
On my Mac, the direct Q4_1 product is marginally slower
(~69 vs ~55 us for Q4_0). The SIMD-ified ggml version
is now almost 2X slower (~121 us).

On a Ryzen 7950X CPU, the direct product for Q4_1 quantization
is faster than the AVX2 implementation (~60 vs ~62 us).

---------

Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com>
This commit is contained in:
Kawrakow 2023-04-18 21:00:14 +02:00 • committed by GitHub
parent 7faa7460f0
commit 5ecff35151
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 326 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
pocs/CMakeLists.txt Normal file
View file

@ -0,0 +1,12 @@
# dependencies
find_package(Threads REQUIRED)
# third-party
include_directories(${CMAKE_CURRENT_SOURCE_DIR})
if (EMSCRIPTEN)
else()
add_subdirectory(vdot)
endif()