measuredlocally.comDeepSeek V4.1 Flash 1017 on four DGX Sparks: SGLang against vLLM
DeepSeek's new V4.1 Flash 1017 runs at full precision on my four DGX Sparks, and the engine mattered as much as the model. On SGLang it reads and writes far faster than it did on vLLM.
Plays here, from YouTube
Naming the set-up matters on my own testing too, and DeepSeek showed me why. I run four of NVIDIA's DGX Sparks linked together, with 512GB of memory between them, for the medium to big open models.
DeepSeek's own page on Hugging Face dates V4.1 Flash 1017 to Saturday 17 October. I put it on the Sparks at full precision: DeepSeek's own weights exactly as released, the experts at 4 bits (MXFP4) and the rest at 8 bits (FP8), with nothing compressed further.
Two engines, the same weights
I started on vLLM, the engine I had always used on the Sparks. It read a 93,000-token document in 78.2 to 87.4 seconds, and one user got 49.3 tokens a second of writing.
On Monday 19 October I swapped the engine for a community recipe for SGLang, with the same weights on the same four boxes. The same document now takes 16.7 seconds to read, and one user gets 67.6 tokens a second. Six users at once went from 157.8 tokens a second in total to 252.7, and sixteen users reach 438.8, which the old set-up could not take at all.
It also read a document of 1,018,328 tokens in 260.6 seconds and found every fact I had hidden in it, a reading speed of 3,908 tokens a second by my own maths.
Against a compressed copy
On Sunday, Tech2WiLD's video called a 3.5-bit copy of the same model almost twice as fast, on four Sparks with an engine called TensorFold. For one user his description gives 93.6 tokens a second, which beats my 67.6. On the same test script, though, mine is ahead from six users up (252.7 against 213.8 tokens a second in total), and at sixteen users his figure is 271 against my 438.8. His document read took 42.1 seconds.
An owner on NVIDIA's developer forum, running DeepSeek's own weights on SGLang, reports 90 tokens a second on code for one user and 296 in total for eight. So for one user the same model gives 49.3, 67.6 or 93.6 tokens a second, depending on the engine and the copy.
What I would do
A smaller, compressed copy writes faster for one person, but at full precision my Sparks pull ahead as soon as a few people or agents share them, which is how I use them.
Read more
- DeepSeek's release on Hugging Facehuggingface.co
- Tech2WiLD's video of the 3.5-bit copyyoutube.com
- The owner's post on NVIDIA's developer forumforums.developer.nvidia.com
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| 49.3 tok/s | vLLM, one user, writing | my own test | measured by me |
| 67.6 tok/s | SGLang, one user, writing | my own test | measured by me |
| 157.8 to 252.7 tok/s | Six users at once, in total, on vLLM then on SGLang | my own test | measured by me |
| 438.8 tok/s | Sixteen users at once on SGLang, in total | my own test | measured by me |
| 3,908 tok/s | Reading a document of 1,018,328 tokens in 260.6 seconds | 1,018,328 divided by 260.6 | measured by me |
| 93.6 tok/s | A 3.5-bit copy on TensorFold, one user, from Tech2WiLD's description | see the story | a reviewer's test |
My Sparks would run a little faster than these figures. I cap their graphics chips at 2,200MHz, against about 2,400 uncapped, to cut their power use. When I tested the cap on one box in August, writing speed stayed within 1 per cent, reading was 2 to 4 per cent slower, and the chip's peak power fell from 55.5W to 43.3W.
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.