measuredlocally.comLocal LLM model leaderboard, round one: eight tests, first results
Speed is only half of the question about a local model, and the other half is how good its work is. So I have started our own ranking of the models you can run at home, tested on the machines you would run them on.
Plays here, from YouTube
Two of this week's reviews of the RTX Spark laptop timed only small, older models, like Microsoft's Phi 3.5, and speed is only half of the question anyway. The other half is how good a model's work actually is.
I watch a channel called World of AI. When a new model comes out, they give it the same build tasks, a game, a 3D scene, a website, a drawing, and then they show you what it made. They now publish their own ranking from those tasks, World of AI Bench.
So I have started our own version of this, for the models you can run at home, on the machines you would actually run them on.
Round one: eight tasks
Round one has eight tasks: a Breakout game, a 3D lighthouse, a fox in the snow, a landing page, balls in a spinning heptagon, a sales dashboard, a bug to fix, and six questions with exact answers.
Every model gets the same prompts, with its thinking switched off and one attempt at each. Each build runs in a browser with the internet off, and a script checks that it draws, has no errors, fits on a phone screen, moves, and answers the keyboard or mouse. The dashboard's totals, the bug fix and the answers are checked against the right ones.
What the models built
The first model through it was GLM 5.3 Flash, which is what I run on my four DGX Sparks. It finished all eight tasks in 7.46 minutes, and every one of its six builds passed every check, as did its bug fix. Its overall score from the checks was 97.9.
It got one of the six questions wrong, and the way it did was worth a look. It wrote the right arrival time for the train, 12:25, then it checked its own work and changed it to 13:25.
This is where judges come in. The checks gave its heptagon top marks, but a few of the balls slipped straight through the walls, which the prompt told it never to allow.
Then Qwen 3.8 27B ran on my RTX 4090 and scored 74.4, in 16.7 minutes. Its game and its lighthouse stopped on mistakes in its own code before drawing anything, and its dashboard kept every panel dark and never faded them in.
The results table will build up week by week here, with the models I test locally, so that we can build up a knowledge base.
Read more
- World of AI Bench, the ranking ours followswoaibench.ai
- Windows Central's review, with its small-model timingswindowscentral.com
- Tom's Guide's live testing, with its small-model timingstomsguide.com
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| 97.9 overall | GLM 5.3 Flash on my four DGX Sparks, its overall score from the checks | my own test | measured by me |
| 7.46 minutes | GLM 5.3 Flash's time for all eight tasks | my own maths | my own maths |
| 12:25, then 13:25 | GLM's answer to the train question, before and after it checked its own work | my own test | measured by me |
| 74.4 overall, 16.7 minutes | Qwen 3.8 27B on my RTX 4090, its overall score and its time | my own test | measured by me |
| Phi-3.5-mini (Microsoft's model), Mistral 7B, Llama 3.1 8B, Llama 2 13B | The small, older models two reviews timed on the RTX Spark laptop | windowscentral.com | a reviewer's test |
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.