Four DGX Sparks twice as fast at ordinary writing: the recipe, and a real session
I moved my four DGX Sparks to a recipe that another owner published on GitHub, and ordinary writing roughly doubled on the same hardware and the same model. The recipe is only a few weeks old, and these are one owner's figures from one cluster.
Plays here, from YouTube
On Sunday 4 October I moved my four DGX Sparks to a recipe that another Spark owner published on GitHub. Both the hardware and the model, GLM 5.3 Flash, are the same as before, and what the recipe changes is the software around them.
On Monday 5 October I ran exactly the same test again. Ordinary writing roughly doubled, and code went from 58.4 tokens a second to 94.2.
Before and after
| What I timed | Before | After |
|---|---|---|
| Code | 58.4 tok/s | 94.2 tok/s |
| A short answer, about 480 tokens | 28.2 tok/s | 60.4 tok/s |
| A long answer, over 1,000 tokens | 45.6 tok/s | 89.8 tok/s |
| Reading a 15,500-token prompt, nothing cached | 2,257 tok/s | 3,072 tok/s |
| Four requests at once, in total | 65.2 tok/s | 110.7 tok/s |
Measured by me on my four DGX Sparks with GLM 5.3 Flash, before and after the change of recipe.
Most days the cluster works as an agent, with several requests running at once. When I send it four together, it now writes at 110.7 tokens a second in total, where the week before it wrote at 65.2, and that is where four boxes earn their place.
Against a Mac Studio
In episode 7 I said that one Mac Studio writes about as fast as my whole cluster, because MacStories had measured this model on a 256GB M5 Ultra at 41 tokens a second. On these numbers, for one user, the four Spark cluster is now well ahead, at 1.5 to 2.3 times the Mac's figure by my own maths (60.4 divided by 41 and 94.2 divided by 41).
A real session
One viewer asked under episode 7 to see what a chat with the cluster is really like. On Monday night, with nothing else running, I asked it what I had asked viewers the week before: what to buy for local AI with $5,000 to spend. I gave it that week's prices and the routes viewers had suggested.
Its first word came back in 0.26 seconds. It then thought for 49.0 seconds before it answered.
I asked the same question five times and the verdict moved. All four answers it finished said to skip the 64GB DGX Spark, which I agree with, but three chose second-hand graphics cards and one chose the Mac Studio.
The chat had no web search, so the model knew nothing newer than its training, and every price in its answer is its own guess. The Mac Studio it has in mind is an M3 Ultra, named 12 times in the session I timed, and the M5 and the RTX Spark never come up.
A few caveats
- This recipe is only a few weeks old, and I am only one owner reporting on one cluster.
- The recipe's author reports 90.3 tokens a second for prose on his four Sparks. Another owner reported 56.5 to 56.9 for the same model on three DGX Sparks, on NVIDIA's developer forum on 7 October.
- MacStories' prompt and mine are different, and their figure includes the model's thinking where mine had thinking switched off.
- Connected to web search, I think the model would look up those prices, find the newer machines and conclude differently. I have not tested that.
Read more
- Another Spark owner's recipe for GLM 5.3 Flash on four DGX Sparks, on GitHubgithub.com
- Another owner's speeds for GLM 5.3 Flash on three DGX Sparks, on NVIDIA's developer forumforums.developer.nvidia.com
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| 58.4 to 94.2 tok/s | Tokens a second on code, before the recipe and after | my own test | measured by me |
| 28.2 to 60.4 tok/s | Tokens a second on a short answer, before and after | my own test | measured by me |
| 110.7 against 65.2 tok/s | Tokens a second in total with four requests at once, against the week before | my own test | measured by me |
| 41 tok/s | MacStories' test of the same model on a Mac Studio | MacStories, as quoted in episode 7 | a reviewer's test |
| 1.5 to 2.3 times | My cluster against that Mac, for one user | 60.4 divided by 41 and 94.2 divided by 41 | my reading |
| 90.3 tok/s on prose, 3,426 tok/s reading | The recipe author's own figures on his four DGX Sparks | the recipe's page on GitHub | an owner's report |
My Sparks would run a little faster than these figures. I cap their graphics chips at 2,200MHz, against about 2,400 uncapped, to cut their power use. When I tested the cap on one box in August, writing speed stayed within 1 per cent, reading was 2 to 4 per cent slower, and the chip's peak power fell from 55.5W to 43.3W.
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.