vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

agent: ensure DATA_DIR exists

Browse source 
skip-checks:true
This commit is contained in:
Olivier Chafik 2024-04-10 11:39:35 +01:00 • committed by ochafik
parent a98f48315c
commit ea0c31b10b
3 changed files with 11 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

9
examples/agent/README.md
View file

@ -108,7 +108,10 @@ The agent can use tools written in Python, or (soon) exposed under OpenAPI endpo
so we provide a script to run them in a Docker-sandboxed environment, exposed as an OpenAPI server: so we provide a script to run them in a Docker-sandboxed environment, exposed as an OpenAPI server:
```bash ```bash
PORT=9999 examples/openai/run_sandboxed_tools.sh \ # With limactl, the default sandbox location ~/.llama.cpp/sandbox won't be writable
# (see https://github.com/lima-vm/lima/discussions/393)
# export DATA_DIR=/tmp/lima/llama.cpp/sandbox
PORT=9999 examples/agent/run_sandboxed_tools.sh \
examples/agent/tools/unsafe_python_tools.py & examples/agent/tools/unsafe_python_tools.py &
python -m examples.agent \ python -m examples.agent \
@ -127,11 +130,11 @@ so we provide a script to run them in a Docker-sandboxed environment, exposed as
</details> </details>
- [fastify.py](./fastify.py) turns a python module into an OpenAPI endpoint using FastAPI - [fastify.py](./fastify.py) turns a python module into an [OpenAPI](https://www.openapis.org/) endpoint using [FastAPI](https://fastapi.tiangolo.com/)
- [run_sandboxed_tools.sh](./run_sandboxed_tools.sh) builds and runs a Docker environment with fastify inside it, and exposes its port locally - [run_sandboxed_tools.sh](./run_sandboxed_tools.sh) builds and runs a Docker environment with fastify inside it, and exposes its port locally
- Beyond just "tools", output format can be constrained using JSON schemas or Pydantic types - Beyond just "tools", output format can be constrained using [JSON schemas](https://json-schema.org/) or [Pydantic](https://docs.pydantic.dev/latest/) types
```bash ```bash
python -m examples.agent \ python -m examples.agent \

2
examples/agent/fastify.py
View file

@ -3,6 +3,7 @@
This is useful in combination w/ the examples/agent/run_sandboxed_tools.sh This is useful in combination w/ the examples/agent/run_sandboxed_tools.sh
''' '''
import os
import fastapi, uvicorn import fastapi, uvicorn
import typer import typer
from typing import Type, List from typing import Type, List
@ -37,6 +38,7 @@ def main(files: List[str], host: str = '0.0.0.0', port: int = 8000):
for f in files: for f in files:
bind_functions(app, load_module(f)) bind_functions(app, load_module(f))
print(f'INFO: CWD = {os.getcwd()}')
uvicorn.run(app, host=host, port=port) uvicorn.run(app, host=host, port=port)
if __name__ == '__main__': if __name__ == '__main__':

4
examples/agent/run_sandboxed_tools.sh
View file

@ -20,6 +20,8 @@ BUILD_DIR=$(mktemp -d)
DATA_DIR="${DATA_DIR:-$HOME/.llama.cpp/sandbox}" DATA_DIR="${DATA_DIR:-$HOME/.llama.cpp/sandbox}"
SCRIPT_DIR=$( cd "$(dirname "$0")" ; pwd ) SCRIPT_DIR=$( cd "$(dirname "$0")" ; pwd )
mkdir -p "$DATA_DIR"
REQUIREMENTS_FILE="${REQUIREMENTS_FILE:-}" REQUIREMENTS_FILE="${REQUIREMENTS_FILE:-}"
if [[ -z "$REQUIREMENTS_FILE" && -f "$script_folder/requirements.txt" ]]; then if [[ -z "$REQUIREMENTS_FILE" && -f "$script_folder/requirements.txt" ]]; then
REQUIREMENTS_FILE="$script_folder/requirements.txt" REQUIREMENTS_FILE="$script_folder/requirements.txt"
@ -69,6 +71,6 @@ set -x
docker run \ docker run \
"$@" \ "$@" \
--mount "type=bind,source=$( realpath "$script_folder" ),target=/src,readonly" \ --mount "type=bind,source=$( realpath "$script_folder" ),target=/src,readonly" \
--mount "type=bind,source=$( realpath "$DATA_DIR" ),target=/data" \ --mount "type=bind,source=$DATA_DIR,target=/data" \
-p "$PORT:$PORT" \ -p "$PORT:$PORT" \
-it "$LLAMA_IMAGE_NAME" -it "$LLAMA_IMAGE_NAME"