Multi GPU support, CUDA refactor, CUDA scratch buffer (#1703)

Browse source

* CUDA multi GPU + scratch

ggml_cuda_compute_forward

Tensor parallelism

ggml_cuda_add

ggml_cuda_rms_norm

ggml_cuda_silu

CUDA scratch buffer

--main-gpu CLI option

...

This commit is contained in:

Johannes Gäßler 2023-06-06 21:33:23 +02:00 • committed by GitHub

parent 44f906e853

commit 17366df842

No known key found for this signature in database

GPG key ID: 4AEE18F83AFDEB23

12 changed files with 1221 additions and 544 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1319

ggml-cuda.cu

View file

File diff suppressed because it is too large Load diff