Francis Couture-Harpin
4c9932c1e1
gguf-py : fix flake8 lint
2024-07-17 23:26:45 -04:00
Francis Couture-Harpin
2c18a9a4d4
gguf-py : extract metadata from model name more resiliently
...
Using more than one regex to annotate the parts of the name,
this way, the order doesn't have to be fixed
and this should work correctly for more edge cases.
Also, the total parameter count of the model is used to figure out
if a size label is not actually a size label, but a context size.
* convert_lora : fix duplicate model type key
2024-07-17 23:17:39 -04:00
brian khuu
7e9271cabf
convert_lora_to_gguf.py: remove model_name parameter. Doesn't exist in LoraModel()
2024-07-17 01:11:27 +10:00
brian khuu
eb0bf6b92f
convert-*.py: Add naming_convention_vocab_only()
2024-07-17 01:04:14 +10:00
brian khuu
5da16bb1d7
Merge branch 'master' into refactor-convert-py
2024-07-16 23:58:56 +10:00
Brian
1666f92dcd
gguf-hash : update clib.json to point to original xxhash repo ( #8491 )
...
* Update clib.json to point to Cyan4973 original xxhash
Convinced Cyan4973 to add clib.json directly to his repo, so can now point the clib package directly to him now. Previously pointed to my fork with the clib.json package metadata
https://github.com/Cyan4973/xxHash/pull/954
* gguf-hash: readme update to point to Cyan4973 xxHash repo [no ci]
2024-07-16 10:14:16 +03:00
Steve Bonds
37b12f92ab
export-lora : handle help argument ( #8497 )
...
The --help option on export-lora isn't accepted as valid. The help still gets displayed by default, but the script exits with an error message and nonzero status.
2024-07-16 10:04:45 +03:00
Georgi Gerganov
0efec57787
llama : valign + remove unused ftype ( #8502 )
2024-07-16 10:00:30 +03:00
compilade
7acfd4e8d5
convert_hf : faster lazy safetensors ( #8482 )
...
* convert_hf : faster lazy safetensors
This makes '--dry-run' much, much faster.
* convert_hf : fix memory leak in lazy MoE conversion
The '_lazy' queue was sometimes self-referential,
which caused reference cycles of objects old enough
to avoid garbage collection until potential memory exhaustion.
2024-07-15 23:13:10 -04:00
Brian
c7b3616449
Update convert_hf_to_gguf.py
...
It might help with the convert_lora_to_gguf.py script if default values were added here
Co-authored-by: compilade <git@compilade.net>
2024-07-16 07:02:07 +10:00
brian khuu
9a925b56a0
metadata.py: account for decimal point in size label within model id components
2024-07-16 06:42:38 +10:00
Francis Couture-Harpin
417d7a7c62
convert_hf : use GGUFWriter to count model parameters
2024-07-16 06:42:38 +10:00
Francis Couture-Harpin
78a42fbee5
gguf-py : use pyyaml instead of python-frontmatter
...
HF transformers already depends on pyyaml for model cards,
so it should already be in the environment
of the users of the convert scripts, unlike python-frontmatter.
This should be completely equivalent since the model cards
seem to use only YAML and never TOML.
2024-07-16 06:42:38 +10:00
brian khuu
3b1766a992
convert-*.py: flake8 remove blank line
2024-07-16 06:42:38 +10:00
brian khuu
f98f1098f9
convert-*.py: more rigorous regexp for get_model_id_components()
2024-07-16 06:42:38 +10:00
brian khuu
4e3761109d
covert-*.py: flake8 newline missing
2024-07-16 06:42:38 +10:00
brian khuu
8629b7bdc2
covert-*.py: per_model_weight_count_estimation() tensor arg type is Iterable[tuple[str, LazyTensor]]
2024-07-16 06:42:38 +10:00
brian khuu
144a7ec3a4
convert-*.py: pathlib.Path exist() --> is_file() or is_dir()
2024-07-16 06:42:38 +10:00
brian khuu
abc351c270
convert-*.py: quantized_by in model card is not relevant for converted gguf
2024-07-16 06:42:38 +10:00
brian khuu
9954b64862
convert-*.py: add logger and refactor load_model_card()
2024-07-16 06:42:38 +10:00
brian khuu
5cdb03b2fc
convert-*.py: update nix package to add python frontmatter
2024-07-16 06:42:38 +10:00
brian khuu
5ab1a84085
convert-*.py: dict_item --> Iterable
2024-07-16 06:42:38 +10:00
Brian
455c0e53ac
Apply suggestions from code review
...
Co-authored-by: compilade <git@compilade.net>
2024-07-16 06:42:38 +10:00
brian khuu
ccff6c7fb2
convert-*.py: remove reference to uuid generation
2024-07-16 06:42:38 +10:00
Brian
8156835d4a
constants.py : Revert removal of backward compatibility KEY_GENERAL_SOURCE_URL
2024-07-16 06:42:38 +10:00
Brian
2c060303a6
Update constants.py : spacing correction
2024-07-16 06:42:38 +10:00
Brian
aa4e5892a0
Update convert_hf_to_gguf.py
...
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
2024-07-16 06:42:38 +10:00
Brian
60278e4f4d
Update convert_hf_to_gguf.py
...
Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>
2024-07-16 06:42:38 +10:00
brian khuu
ad217d7249
convert-*.py: remove autogenerated uuid
2024-07-16 06:42:38 +10:00
brian khuu
f2b425c59c
convert-*.py: import cast from typing and other refactor
2024-07-16 06:42:38 +10:00
brian khuu
04c4fffdcc
convert-*.py: prepare_tensors_for_writing() --> prepare_tensors()
...
> Especially since it can be used for other purposes than "for writing", like preparing the tensors to then count and sum all their sizes.
Co-authored-by: compilade <git@compilade.net>
2024-07-16 06:42:38 +10:00
brian khuu
64707b625c
convert-*.py: remove redundant gguf_writer.add_name() calls
2024-07-16 06:42:38 +10:00
brian khuu
f8b5931180
convert-*.py: parameter_class_attribute --> size_label
2024-07-16 06:42:38 +10:00
brian khuu
6eb08ac868
convert-*.py: Removing the redundant metadata is not None from all conditions, and indenting them.
...
Co-authored-by: compilade <git@compilade.net>
2024-07-16 06:42:38 +10:00
brian khuu
4c91d077d2
convert-*.py: cast not required if Metadata.load_metadata_override returned a dict[str, Any] instead of a dict[str, object]
...
Co-authored-by: compilade <git@compilade.net>
2024-07-16 06:42:38 +10:00
Brian
74383ba6d2
Apply suggestions from code review
...
Co-authored-by: compilade <git@compilade.net>
2024-07-16 06:42:38 +10:00
brian khuu
dd14b8fdb1
convert-*.py: pyright type fixes
2024-07-16 06:42:38 +10:00
brian khuu
59a01df784
convert-*.py: refactor per model weight count estimation
2024-07-16 06:42:38 +10:00
brian khuu
2a976e1211
convert-*.py: write_tensors() --> prepare_tensors_for_writing()
2024-07-16 06:42:38 +10:00
brian khuu
fdc5a3fc80
convert-*.py: autogenerate general.uuid if missing
2024-07-16 06:42:35 +10:00
brian khuu
7ecb8f00a0
test: remove test_gguf.py and remove test_generate_any_missing_uuid()
2024-07-16 06:38:40 +10:00
brian khuu
007708e32d
gguf_writer.py: generate tensor uuid if missing
2024-07-16 06:38:40 +10:00
brian khuu
4dc8ddd35a
convert_hf_to_gguf.py: Remove code that is already in fill_templated_filename() and GGUFWriter()
2024-07-16 06:38:40 +10:00
brian khuu
2f23927d37
convert_hf_to_gguf.py: rebase error correction
2024-07-16 06:38:40 +10:00
brian khuu
5011eefeaf
convert_hf_to_gguf.py: optional, dataclass removed from type as it was unused
2024-07-16 06:38:40 +10:00
brian khuu
e9734434bd
convert-*.py: Remove self.model_name that was left in since last rebase
2024-07-16 06:38:40 +10:00
brian khuu
eaa47f5546
convert-*.py: separated unit test, hf_repo to repo_url
2024-07-16 06:38:40 +10:00
brian khuu
d060fcdbe2
convert-*.py: adjusted authorship KV store
2024-07-16 06:38:40 +10:00
brian khuu
91e65d9485
convert-*.py: add unittest to metadata class
2024-07-16 06:38:38 +10:00
brian khuu
3625a42061
convert-*.py: add heuristic to directory name fallback
...
Also add source_url for huggingface url
2024-07-16 06:37:42 +10:00
