jimbocoin on Nostr: Ah but it does! Once you download the gguf file from Hugging Face, you can use ...

Ah but it does! Once you download the gguf file from Hugging Face, you can use ollama’s create command, passing in a Modelfile that specifies the path the the gguf. Then you can use ollama run to start up the model.

It’s kinda annoying but there are instructions online: https://www.markhneedham.com/blog/2023/10/18/ollama-hugging-face-gguf-models/

I used this technique to run mradermacher/dolphin-2.9.2-mixtral-8x22b-GGUF: https://huggingface.co/mradermacher/dolphin-2.9.2-mixtral-8x22b-GGUF