Ternary Bonsai 2: a 27B model in under 6GB
Prism ML released a 27 billion parameter model built from Qwen 3.8 with every weight set to -1, 0 or +1, and says that brings it under 6GB. The maker says it keeps 98.2% of the original's scores, and on 19 September that was still the maker's own number.
Plays here, from YouTube
A team called Prism ML released Ternary Bonsai 2, a quant that breaks the usual pattern. A Reddit post on 17 September said it had just gone up on Hugging Face.
It is a 27 billion parameter model built from Qwen 3.8, with every weight set to -1, 0 or +1. Prism ML says that packs the whole model into 5.95GB, about a ninth of the roughly 54GB the original needs.
What the maker claims
Prism ML benchmarked it on 14 tests and says it averaged 84.78 against 86.32 for the original, which is 98.2% of the original's score. On the same table the usual 2-bit build (IQ2_XXS) averaged 72.59, or 84.1%, and a 17.6GB build (UD-Q4_K_XL) averaged 85.18, or 98.7%.
On the maker's own table, a laptop with an M5 Max reaches 47.0 tokens a second, and an M4 Pro 18.0. The card says the full 54GB original fits on neither.
What one owner found
One owner reported about 101 tokens a second on an RTX 5090 graphics card (106,396 tokens in about 17.5 minutes), and reckons it looked the best of the three models he compared. It was a single run, and he says he runs a service built on Prism ML's backend, so I would treat it as an anecdote.
A few caveats
- It needs Prism ML's own fork of llama.cpp. The model card says stock llama.cpp refuses two of the file types, and loads a third with no warning and gives garbage.
- The owner's run used a 7.2GB build, not the 5.95GB one, and the maker's M5 Max row is marked 7.2GB as well.
- When I looked on 19 September, there was no entry for it among the 80 open models on the Artificial Analysis leaderboard. So the 98.2% was still the maker's own number.
Read more
- Prism ML's model card for Ternary Bonsai 2, with its scores and speedshuggingface.co
- The Reddit post of 17 September announcing the releasereddit.com
- An owner's test of it on an RTX 5090, on Redditreddit.com
- Artificial Analysis, open-weights leaderboardartificialanalysis.ai
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| 5.95GB against about 54GB | The file's size, against the original's | huggingface.co | Prism ML's claim |
| 84.78, against 86.32 and 72.59 | Average score over 14 tests, against the original and a 2-bit build | huggingface.co | the maker's claim |
| about 47, 28 and 18 tok/s | The maker's speeds on an M5 Max, an M5 Pro and an M4 Pro | huggingface.co | Prism ML's claim |
| about 101 tok/s | One owner's run on an RTX 5090 | reddit.com | an owner's report |
| Not listed on 19 September | Ternary Bonsai 2 on Artificial Analysis | artificialanalysis.ai | my reading |
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.