As of 2023-07-02, the following GPTQ models on HuggingFace all appear to be working:
- iambestfeed/open_llama_3b_4bit_128g
- Neko-Institute-of-Science/LLaMA-7B-4bit-128g
- Neko-Institute-of-Science/LLaMA-13B-4bit-128g
- Neko-Institute-of-Science/LLaMA-30B-4bit-32g
- Neko-Institute-of-Science/LLaMA-30B-4bit-128g
- Neko-Institute-of-Science/LLaMA-65B-4bit-32g
- Neko-Institute-of-Science/LLaMA-65B-4bit-128g
- reeducator/bluemoonrp-13b
- reeducator/bluemoonrp-30b
- TehVenom/Metharme-13b-4bit-GPTQ
- TheBloke/airoboros-13B-GPTQ
- TheBloke/gpt4-x-vicuna-13B-GPTQ
- TheBloke/GPT4All-13B-snoozy-GPTQ
- TheBloke/guanaco-33B-GPTQ
- TheBloke/guanaco-65B-GPTQ
- TheBloke/h2ogpt-oasst1-512-30B-GPTQ 1
- TheBloke/koala-13B-GPTQ-4bit-128g
- TheBloke/Manticore-13B-GPTQ
- TheBloke/medalpaca-13B-GPTQ-4bit
- TheBloke/medalpaca-13B-GPTQ-4bit (compat version)
- TheBloke/Nous-Hermes-13B-GPTQ
- TheBloke/robin-65B-v2-GPTQ
- TheBloke/tulu-7B-GPTQ
- TheBloke/Tulu-13B-SuperHOT-8K-GPTQ
- TheBloke/tulu-30B-GPTQ
- TheBloke/vicuna-13B-1.1-GPTQ-4bit-128g
- TheBloke/VicUnlocked-30B-LoRA-GPTQ
- TheBloke/wizard-mega-13B-GPTQ
- TheBloke/Wizard-Vicuna-7B-Uncensored-GPTQ
- TheBloke/Wizard-Vicuna-13B-Uncensored-GPTQ
- TheBloke/WizardLM-7B-uncensored-GPTQ
- TheBloke/WizardLM-30B-Uncensored-GPTQ
- TheBloke/WizardLM-33B-V1.0-Uncensored-SuperHOT-8K-GPTQ
- tmpupload/superhot-30b-8k-no-rlhf-test-128g-GPTQ
- Yhyu13/chimera-inst-chat-13b-gptq-4bit
- Yhyu13/oasst-rlhf-2-llama-30b-7k-steps-gptq-4bit
1 This particular model, uniquely, shows somewhat worse perplexity when matmul is done by the custom CUDA kernel rather than cuBLAS. Maybe it's extra sensitive to rounding errors for some reason? Either way, it does work.
As of 2023-07-02, I have found no models that don't work.
v1 models are still unsupported, as are pickle files.