vbatts/llama.cpp
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update convert-hf-to-gguf.py

Browse source 
This change is needed for Gemma models to be able to be quantized that have been merged with mergekit. Normal gemma models do not need this, but when a model is merged with mergekit it cannot be quantized without this change.
This commit is contained in:
rombodawg 2024-02-28 12:51:08 -05:00 • committed by GitHub
parent 08c5ee87e4
commit 645bb3d5a2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
convert-hf-to-gguf.py
View file

@ -8,10 +8,11 @@ import json
import os import os
import re import re
import sys import sys
import transformers
from enum import IntEnum from enum import IntEnum
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any, ContextManager, Iterator, Sequence, cast from typing import TYPE_CHECKING, Any, ContextManager, Iterator, Sequence, cast
from transformers import GemmaModel
import numpy as np import numpy as np
import torch import torch