From 5b64d6f4d34d2640733bd03b836f29cd7f0beb5d Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Fri, 12 Apr 2024 07:45:23 +0200 Subject: [PATCH] infill : add download instructions for model This commit adds instructions on how to download a CodeLlama model using the `hf.sh` script. This will download the model and place it in the `models` directory which is the same model use later by the infill example. Signed-off-by: Daniel Bevenius --- examples/infill/README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/examples/infill/README.md b/examples/infill/README.md index 8c97f719b..c17ced94e 100644 --- a/examples/infill/README.md +++ b/examples/infill/README.md @@ -36,6 +36,11 @@ The `infill` program offers a seamless way to interact with LLaMA models, allowi ### Example +Download a CodeLlama model: +```console +scripts/hf.sh --repo TheBloke/CodeLlama-13B-GGUF --file codellama-13b.Q5_K_S.gguf --outdir models +``` + ```bash ./infill -t 10 -ngl 0 -m models/codellama-13b.Q5_K_S.gguf -c 4096 --temp 0.7 --repeat_penalty 1.1 -n 20 --in-prefix "def helloworld():\n print(\"hell" --in-suffix "\n print(\"goodbye world\")\n " ```