From cf025dfbdb6432d2e8c717fb46cadfde397891e8 Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 16:24:19 +0330 Subject: [PATCH 1/6] added numa dependencies as to use numa in numactl --- README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/README.md b/README.md index 086618315..9ac48bbbe 100644 --- a/README.md +++ b/README.md @@ -48,6 +48,7 @@ Inference of Meta's [LLaMA](https://arxiv.org/abs/2302.13971) model (and others)
  • Interactive mode
  • Constrained output with grammars
  • Instruct mode
  • +
  • Cli commands
  • Obtaining and using the Facebook LLaMA 2 model
  • Seminal papers and background on the models
  • Perplexity (measuring model quality)
  • @@ -893,6 +894,21 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. > ``` +### Cli commands + +- #### Numa (attempt optimizations that help on some NUMA systems): +To use it in numactl mode you will need the package + +## Unix +- Fedora linux package: +```bash +sudo dnf install numactl +``` +- Debian or Ubuntu linux package: +```bash +sudo apt install numactl +``` + ### Obtaining and using the Facebook LLaMA 2 model - Refer to [Facebook's LLaMA download page](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) if you want to access the model data. From 4adabc6640e22674f36fe0c12765de1867bfb24a Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 16:58:05 +0330 Subject: [PATCH 2/6] renamed 'cli commands' to 'numa' as to be coherent with rest of readme --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9ac48bbbe..7f81da885 100644 --- a/README.md +++ b/README.md @@ -48,7 +48,7 @@ Inference of Meta's [LLaMA](https://arxiv.org/abs/2302.13971) model (and others)
  • Interactive mode
  • Constrained output with grammars
  • Instruct mode
  • -
  • Cli commands
  • +
  • Numa
  • Obtaining and using the Facebook LLaMA 2 model
  • Seminal papers and background on the models
  • Perplexity (measuring model quality)
  • @@ -896,7 +896,8 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. ### Cli commands -- #### Numa (attempt optimizations that help on some NUMA systems): +- #### Numa +attempt optimizations that help on some NUMA systems. To use it in numactl mode you will need the package ## Unix From f83cf85c8557ae865d551a790745d7d5c744d96c Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 17:02:02 +0330 Subject: [PATCH 3/6] removed rest of 'cli commands' --- README.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/README.md b/README.md index 7f81da885..7d360f22d 100644 --- a/README.md +++ b/README.md @@ -894,9 +894,7 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. > ``` -### Cli commands - -- #### Numa +### Numa attempt optimizations that help on some NUMA systems. To use it in numactl mode you will need the package From cf74dd7fbfda25cc0b4214fc9fdacb98b5374cee Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 17:06:42 +0330 Subject: [PATCH 4/6] made instructions more clean --- README.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/README.md b/README.md index 7d360f22d..c46e63616 100644 --- a/README.md +++ b/README.md @@ -907,6 +907,18 @@ sudo dnf install numactl ```bash sudo apt install numactl ``` +-
    + Unix: + + - Fedora linux package: + ```bash + sudo dnf install numactl + ``` + - Debian or Ubuntu linux package: + ```bash + sudo apt install numactl + ``` +
    ### Obtaining and using the Facebook LLaMA 2 model From 92b6b946028e459df8209d0ecb37c841a6a3b7fb Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 17:08:03 +0330 Subject: [PATCH 5/6] removed old instructions --- README.md | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/README.md b/README.md index c46e63616..2d3706925 100644 --- a/README.md +++ b/README.md @@ -894,19 +894,6 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. > ``` -### Numa -attempt optimizations that help on some NUMA systems. -To use it in numactl mode you will need the package - -## Unix -- Fedora linux package: -```bash -sudo dnf install numactl -``` -- Debian or Ubuntu linux package: -```bash -sudo apt install numactl -``` -
    Unix: From 2b2fd541c2d8a6fc1255793c8f8a1d24d8fbabe2 Mon Sep 17 00:00:00 2001 From: Mohammadreza Hendiani Date: Sat, 20 Apr 2024 17:11:33 +0330 Subject: [PATCH 6/6] fix title as to work with table of content --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2d3706925..fd3ef477e 100644 --- a/README.md +++ b/README.md @@ -893,7 +893,7 @@ The majority (54%) are using public transit. This includes buses, trams and metr cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. > ``` - +#### Numa -
    Unix: