Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Give GGUF the same filename as project name #1533

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sebaxakerhtc
Copy link
Contributor

Why?
Default filename is "unsloth" - and that's nice, but when you have multiple models on HF and try to download them to OpenWebUI - they all have the same name like "unsloth.8.0.gguf".

Tested in colab with local and HF saving of gguf.

Local save

INFO:hf-to-gguf:Set model quantization version
INFO:gguf.gguf_writer:Writing the following files:
INFO:gguf.gguf_writer:/content/test-llama-3.2-1B/test-llama-3.2-1B.Q8_0.gguf: n_tensors = 147, total_size = 1.3G
Writing: 100%|██████████| 1.31G/1.31G [00:25<00:00, 51.9Mbyte/s]
INFO:hf-to-gguf:Model successfully exported to /content/test-llama-3.2-1B/test-llama-3.2-1B.Q8_0.gguf

HF save

Unsloth: Conversion completed! Output location: /content/sebaxakerhtc/test-llama-3.2-1B-GGUF/test-llama-3.2-1B-GGUF.Q2_K.gguf
Unsloth: Saved Ollama Modelfile to sebaxakerhtc/test-llama-3.2-1B-GGUF/Modelfile
Unsloth: Uploading GGUF to Huggingface Hub...
100%
 1/1 [00:05<00:00,  5.84s/it]
test-llama-3.2-1B-GGUF.Q2_K.gguf: 
 592M/? [00:05<00:00, 371MB/s]
Saved GGUF to https://huggingface.co/sebaxakerhtc/test-llama-3.2-1B-GGUF

Why?
Default filename is "unsloth" - and that's nice, but when you have multiple models on HF and try to download them to OpenWebUI - they all have the same name like "unsloth.8.0.gguf".
@danielhanchen
Copy link
Contributor

Oh agreed attaching the original name is better

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants