vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #27 from one-lithe-rune/allow-sdk-dll-loading - Allow use of hip SDK (if installed) dlls on windows (#470)

Browse source 
* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.

Co-authored-by: one-lithe-rune <skapusniak@lithe-runes.com>
This commit is contained in:
YellowRoseCx 2023-10-10 04:16:33 -05:00 • committed by GitHub
parent f288c6b5e3
commit 1b25b21655
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
koboldcpp.py
View file

@ -184,6 +184,10 @@ def init_library():
os.add_dll_directory(dir_path) os.add_dll_directory(dir_path)
os.add_dll_directory(abs_path) os.add_dll_directory(abs_path)
os.add_dll_directory(os.getcwd()) os.add_dll_directory(os.getcwd())
if libname == lib_hipblas and "HIP_PATH" in os.environ:
os.add_dll_directory(os.path.join(os.environ["HIP_PATH"], "bin"))
if args.debugmode == 1:
print(f"HIP/ROCm SDK at {os.environ['HIP_PATH']} included in .DLL load path")
handle = ctypes.CDLL(os.path.join(dir_path, libname)) handle = ctypes.CDLL(os.path.join(dir_path, libname))
handle.load_model.argtypes = [load_model_inputs] handle.load_model.argtypes = [load_model_inputs]