Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗
-
hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4
Text Generation • Updated • 38.9k • 33 -
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4
Text Generation • Updated • 1.45k • 5 -
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4
Text Generation • Updated • 1.61k • 15 -
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4
Text Generation • Updated • 84.6k • 73