vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Disabled Auto-Format

Browse source
This commit is contained in:
dhruvanand24 2025-01-29 17:55:41 +05:30
parent 971f2f0d04
commit ad622ca97e
1 changed files with 243 additions and 240 deletions
Download patch file Download diff file Expand all files Collapse all files

39
README.md
View file

@ -23,7 +23,7 @@ Inference of Meta's [LLaMA](https://arxiv.org/abs/2302.13971) model (and others)
- Hugging Face Inference Endpoints now support GGUF out of the box! https://github.com/ggerganov/llama.cpp/discussions/9669
- Hugging Face GGUF editor: [discussion](https://github.com/ggerganov/llama.cpp/discussions/9268) | [tool](https://huggingface.co/spaces/CISCai/gguf-editor)
---
----
## Description
@ -49,25 +49,25 @@ Instructions for adding support for new models: [HOWTO-add-model.md](docs/develo
#### Text-only
- [x] LLaMA 🦙
- [X] LLaMA 🦙
- [x] LLaMA 2 🦙🦙
- [x] LLaMA 3 🦙🦙🦙
- [x] [Mistral 7B](https://huggingface.co/mistralai/Mistral-7B-v0.1)
- [X] [Mistral 7B](https://huggingface.co/mistralai/Mistral-7B-v0.1)
- [x] [Mixtral MoE](https://huggingface.co/models?search=mistral-ai/Mixtral)
- [x] [DBRX](https://huggingface.co/databricks/dbrx-instruct)
- [x] [Falcon](https://huggingface.co/models?search=tiiuae/falcon)
- [x] [Chinese LLaMA / Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca) and [Chinese LLaMA-2 / Alpaca-2](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2)
- [x] [Vigogne (French)](https://github.com/bofenghuang/vigogne)
- [x] [BERT](https://github.com/ggerganov/llama.cpp/pull/5423)
- [x] [Koala](https://bair.berkeley.edu/blog/2023/04/03/koala/)
- [x] [Baichuan 1 & 2](https://huggingface.co/models?search=baichuan-inc/Baichuan) + [derivations](https://huggingface.co/hiyouga/baichuan-7b-sft)
- [x] [Aquila 1 & 2](https://huggingface.co/models?search=BAAI/Aquila)
- [x] [Starcoder models](https://github.com/ggerganov/llama.cpp/pull/3187)
- [x] [Refact](https://huggingface.co/smallcloudai/Refact-1_6B-fim)
- [x] [MPT](https://github.com/ggerganov/llama.cpp/pull/3417)
- [x] [Bloom](https://github.com/ggerganov/llama.cpp/pull/3553)
- [X] [Falcon](https://huggingface.co/models?search=tiiuae/falcon)
- [X] [Chinese LLaMA / Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca) and [Chinese LLaMA-2 / Alpaca-2](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2)
- [X] [Vigogne (French)](https://github.com/bofenghuang/vigogne)
- [X] [BERT](https://github.com/ggerganov/llama.cpp/pull/5423)
- [X] [Koala](https://bair.berkeley.edu/blog/2023/04/03/koala/)
- [X] [Baichuan 1 & 2](https://huggingface.co/models?search=baichuan-inc/Baichuan) + [derivations](https://huggingface.co/hiyouga/baichuan-7b-sft)
- [X] [Aquila 1 & 2](https://huggingface.co/models?search=BAAI/Aquila)
- [X] [Starcoder models](https://github.com/ggerganov/llama.cpp/pull/3187)
- [X] [Refact](https://huggingface.co/smallcloudai/Refact-1_6B-fim)
- [X] [MPT](https://github.com/ggerganov/llama.cpp/pull/3417)
- [X] [Bloom](https://github.com/ggerganov/llama.cpp/pull/3553)
- [x] [Yi models](https://huggingface.co/models?search=01-ai/Yi)
- [x] [StableLM models](https://huggingface.co/stabilityai)
- [X] [StableLM models](https://huggingface.co/stabilityai)
- [x] [Deepseek models](https://huggingface.co/models?search=deepseek-ai/deepseek)
- [x] [Qwen models](https://huggingface.co/models?search=Qwen/Qwen)
- [x] [PLaMo-13B](https://github.com/ggerganov/llama.cpp/pull/3557)
@ -153,7 +153,7 @@ Instructions for adding support for new models: [HOWTO-add-model.md](docs/develo
<details>
<summary>UIs</summary>
_(to have a project listed here, it should clearly state that it depends on `llama.cpp`)_
*(to have a project listed here, it should clearly state that it depends on `llama.cpp`)*
- [AI Sublime Text plugin](https://github.com/yaroslavyaroslav/OpenAI-sublime-text) (MIT)
- [cztomsik/ava](https://github.com/cztomsik/ava) (MIT)
@ -222,7 +222,7 @@ _(to have a project listed here, it should clearly state that it depends on `lla
## Supported backends
| Backend | Target devices |
| ---------------------------------- | --------------------- |
| --- | --- |
| [Metal](docs/build.md#metal-build) | Apple Silicon |
| [BLAS](docs/build.md#blas-build) | All |
| [BLIS](docs/backend/BLIS.md) | All |
@ -327,6 +327,7 @@ To learn more about model quantization, [read this documentation](examples/quant
</details>
## [`llama-server`](examples/server)
#### A lightweight, [OpenAI API](https://github.com/openai/openai-openapi) compatible, HTTP server for serving LLMs.
@ -396,6 +397,7 @@ To learn more about model quantization, [read this documentation](examples/quant
</details>
## [`llama-perplexity`](examples/perplexity)
#### A tool for measuring the perplexity [^1][^2] (and other quality metrics) of a model over a given text.
@ -475,6 +477,7 @@ To learn more about model quantization, [read this documentation](examples/quant
</details>
## Contributing
- Contributors can open PRs
@ -503,7 +506,6 @@ To learn more about model quantization, [read this documentation](examples/quant
#### Seminal papers and background on the models
If your issue is with model generation quality, then please at least scan the following links and papers to understand the limitations of LLaMA models. This is especially important when choosing an appropriate model size and appreciating both the significant and subtle differences between LLaMA models and ChatGPT:
- LLaMA:
- [Introducing LLaMA: A foundational, 65-billion-parameter large language model](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/)
- [LLaMA: Open and Efficient Foundation Language Models](https://arxiv.org/abs/2302.13971)
@ -514,3 +516,4 @@ If your issue is with model generation quality, then please at least scan the fo
- [Training language models to follow instructions with human feedback](https://arxiv.org/abs/2203.02155)
#### References