Fuck it! You can now run *any* GGUF on the Hugging Face Hub directly with
@ollama 🔥
This has been a constant ask from the community, starting today you can point to any of the 45,000 GGUF repos on the Hub*
*Without any changes whatsoever! ⚡
All you need to do is:
ollama run hf. co/{username}/{reponame}:latest
For example, to run the Llama 3.2 1B, you can run:
ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:latest
If you want to run a specific quant, all you need to do is specify the Quant type:
ollama run hf. co/bartowski/Llama-3.2-1B-Instruct-GGUF:Q8_0
That's it! We'll work closely with Ollama to continue developing this further! ⚡