From 722716161188ff7e7e1d2302de70a2da4b5ccd27 Mon Sep 17 00:00:00 2001
From: brian khuu <mofosyne@gmail.com>
Date: Sun, 12 May 2024 01:04:41 +1000
Subject: [PATCH] debug-test.sh: documentation update

---
 docs/debugging-tests.md | 98 ++++++++++++++++++++++++++++++-----------
 1 file changed, 72 insertions(+), 26 deletions(-)

diff --git a/docs/debugging-tests.md b/docs/debugging-tests.md
index 5134815ca..620f65560 100644
--- a/docs/debugging-tests.md
+++ b/docs/debugging-tests.md
@@ -1,42 +1,88 @@
-# How to run & debug a specific test without anything else to keep the feedback loop short?
-There is a script called debug-test.sh in the scripts folder that takes a REGEX as its only parameter. For example, running the following command will output an interactive list from which you can select a test. It will then build & run in the debugger for you.
+# Debugging Tests Tips
+
+## How to run & debug a specific test without anything else to keep the feedback loop short?
+
+There is a script called debug-test.sh in the scripts folder whose parameter takes a REGEX and an optional test number.
+
+For example, running the following command will output an interactive list from which you can select a test. It takes this form:
+
+`debug-test.sh [OPTION]... <test_regex> <test_number>`
+
+It will then build & run in the debugger for you.
+
 ```bash
 ./scripts/debug-test.sh test-tokenizer
 
 # Once in the debugger, i.e. at the chevrons prompt, setting a breakpoint could be as follows:
 >>> b main
 ```
-Another idea is to see all the tests in general, at the least the ones prefixed with "test". This is accomplished via the following: `scripts/debug-test.sh test`.
+
+For further reference use `debug-test.sh -h` to print help.
 
 &nbsp;
 
-# How does the script work?
+### How does the script work?
 If you want to be able to use the concepts contained in the script separately, the important ones are briefly outlined below.
+
+#### Step 1: Reset and Setup folder context
+
+From base of this repository, let's create `build-ci-debug` as our build context.
+
 ```bash
-# Step 1: Prepare the Build Environment
 rm -rf build-ci-debug && mkdir build-ci-debug && cd build-ci-debug
-cmake -DCMAKE_BUILD_TYPE=Debug -DLLAMA_CUDA=1 -DLLAMA_FATAL_WARNINGS=ON .. && cd ..
-
-# Step 2: Build the Project
-make -j
-
-# Step 3: Navigate to Test Directory
-# Tip: If you see a warning about missing "cache" during installation. Then install it to save immense amounts of time!
-cd tests
-
-# Step 4: Identify Test Command for Debugging
-# The output of this command will give you the command & arguments needed to run GDB.
-# -R test-tokenizer : looks for all the test files named test-tokenizer* (R=Regex)
-# -N : "show-only" disables test execution & shows test commands that you can feed to GDB.
-# -V : Verbose Mode
-ctest -R ${test_suite} -V -N
-
-# Step 5: Debug the Test
-gdb --args ${test_path} ${test_args}
-gdb --args /home/ubuntu/workspace/llama.cpp/bin/test-tokenizer-0 "/home/ubuntu/workspace/llama.cpp/tests/../models/ggml-vocab-llama-spm.gguf"
-
-
 ```
 
+#### Step 2: Setup Build Environment and Compile Test Binaries
 
+Setup and trigger a build under debug mode. You may adapt the arguments as needed, but in this case these are sane defaults.
 
+```bash
+cmake -DCMAKE_BUILD_TYPE=Debug -DLLAMA_CUDA=1 -DLLAMA_FATAL_WARNINGS=ON ..
+make -j
+```
+
+#### Step 3.1: Identify Test Command for Debugging
+
+The output of this command will give you the command & arguments needed to run GDB.
+
+* `-R test-tokenizer` : looks for all the test files named test-tokenizer* (R=Regex)
+* `-N` : "show-only" disables test execution & shows test commands that you can feed to GDB.
+* `-V` : Verbose Mode
+
+```bash
+ctest -R "test-tokenizer" -V -N
+```
+
+This may return output similar to below (focusing on key lines to pay attention to):
+
+```bash
+...
+1: Test command: ~/llama.cpp/build-ci-debug/bin/test-tokenizer-0 "~/llama.cpp/tests/../models/ggml-vocab-llama-spm.gguf"
+1: Working Directory: .
+Labels: main
+  Test  #1: test-tokenizer-0-llama-spm
+...
+4: Test command: ~/llama.cpp/build-ci-debug/bin/test-tokenizer-0 "~/llama.cpp/tests/../models/ggml-vocab-falcon.gguf"
+4: Working Directory: .
+Labels: main
+  Test  #4: test-tokenizer-0-falcon
+...
+```
+
+So for test #1 we can tell these two pieces of relevant information:
+* Test Binary: `~/llama.cpp/build-ci-debug/bin/test-tokenizer-0`
+* Test GGUF Model: `~/llama.cpp/tests/../models/ggml-vocab-llama-spm.gguf`
+
+#### Step 3.2: Run GDB on test command
+
+Based on the ctest 'test command' report above we can then run a gdb session via this command below:
+
+```bash
+gdb --args ${Test Binary} ${Test GGUF Model}
+```
+
+Example:
+
+```bash
+gdb --args ~/llama.cpp/build-ci-debug/bin/test-tokenizer-0 "~/llama.cpp/tests/../models/ggml-vocab-llama-spm.gguf"
+```