agent: ensure DATA_DIR exists

skip-checks:true
This commit is contained in:
Olivier Chafik 2024-04-10 11:39:35 +01:00 committed by ochafik
parent a98f48315c
commit ea0c31b10b
3 changed files with 11 additions and 4 deletions

View file

@ -108,7 +108,10 @@ The agent can use tools written in Python, or (soon) exposed under OpenAPI endpo
so we provide a script to run them in a Docker-sandboxed environment, exposed as an OpenAPI server:
```bash
PORT=9999 examples/openai/run_sandboxed_tools.sh \
# With limactl, the default sandbox location ~/.llama.cpp/sandbox won't be writable
# (see https://github.com/lima-vm/lima/discussions/393)
# export DATA_DIR=/tmp/lima/llama.cpp/sandbox
PORT=9999 examples/agent/run_sandboxed_tools.sh \
examples/agent/tools/unsafe_python_tools.py &
python -m examples.agent \
@ -127,11 +130,11 @@ so we provide a script to run them in a Docker-sandboxed environment, exposed as
</details>
- [fastify.py](./fastify.py) turns a python module into an OpenAPI endpoint using FastAPI
- [fastify.py](./fastify.py) turns a python module into an [OpenAPI](https://www.openapis.org/) endpoint using [FastAPI](https://fastapi.tiangolo.com/)
- [run_sandboxed_tools.sh](./run_sandboxed_tools.sh) builds and runs a Docker environment with fastify inside it, and exposes its port locally
- Beyond just "tools", output format can be constrained using JSON schemas or Pydantic types
- Beyond just "tools", output format can be constrained using [JSON schemas](https://json-schema.org/) or [Pydantic](https://docs.pydantic.dev/latest/) types
```bash
python -m examples.agent \

View file

@ -3,6 +3,7 @@
This is useful in combination w/ the examples/agent/run_sandboxed_tools.sh
'''
import os
import fastapi, uvicorn
import typer
from typing import Type, List
@ -37,6 +38,7 @@ def main(files: List[str], host: str = '0.0.0.0', port: int = 8000):
for f in files:
bind_functions(app, load_module(f))
print(f'INFO: CWD = {os.getcwd()}')
uvicorn.run(app, host=host, port=port)
if __name__ == '__main__':

View file

@ -20,6 +20,8 @@ BUILD_DIR=$(mktemp -d)
DATA_DIR="${DATA_DIR:-$HOME/.llama.cpp/sandbox}"
SCRIPT_DIR=$( cd "$(dirname "$0")" ; pwd )
mkdir -p "$DATA_DIR"
REQUIREMENTS_FILE="${REQUIREMENTS_FILE:-}"
if [[ -z "$REQUIREMENTS_FILE" && -f "$script_folder/requirements.txt" ]]; then
REQUIREMENTS_FILE="$script_folder/requirements.txt"
@ -69,6 +71,6 @@ set -x
docker run \
"$@" \
--mount "type=bind,source=$( realpath "$script_folder" ),target=/src,readonly" \
--mount "type=bind,source=$( realpath "$DATA_DIR" ),target=/data" \
--mount "type=bind,source=$DATA_DIR,target=/data" \
-p "$PORT:$PORT" \
-it "$LLAMA_IMAGE_NAME"