Hugging Face Transformers now loads GGUF: the catch
Hugging Face says its Transformers library can now load the GGUF files that llama.cpp made popular for running models at home. For now it only keeps the file compressed for some Qwen models on Apple silicon, and I think it is most useful to people who work in Python.
Plays here, from YouTube
Transformers is the Hugging Face library that most model research is built on. Hugging Face says it can now load GGUF files, the same compressed files that llama.cpp made popular for running models at home. It is a small story, but it touches most of the model files we download.
On a Mac it gets close to llama.cpp's speed, going by Hugging Face's own timings of three checkpoints on an M2 Max, because it borrows the same Metal code underneath.
The catch
For now, the post says, it only keeps the file compressed for the Qwen 3.5 family and compatible Qwen 3.8 models, and only on Apple silicon. With anything else, it quietly decompresses the file again and spends the memory you were trying to save.
The llama.cpp team is part of Hugging Face now, and even so, the post still recommends llama.cpp for everyday use.
Who it is for
I think it is most useful if you work in Python. Going by the post, you can test a model, look inside it or fine-tune it from the same file, without converting it first. The post's example loads a Qwen 3.5 4B file with from_pretrained and a gguf_file argument.
Read more
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| Close, on an M2 Max | Transformers against llama.cpp on a Mac, by Hugging Face's own timings | huggingface.co | the maker's claim |
| Qwen 3.5 and 3.8, on Apple silicon only | Where the file stays compressed, and what happens elsewhere | huggingface.co | the maker's claim |
| llama.cpp | The engine the post recommends for everyday use | huggingface.co | the maker's claim |
| Evaluating, hooks and fine-tuning | What can be done from the same file in Python | huggingface.co | the maker's claim |
| from_pretrained with gguf_file | The loading call in the post's example | huggingface.co | the maker's claim |
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.