NVIDIA's NVFP4 builds of GLM 5.3 Flash: 18 checked for the fault
NVIDIA published its own NVFP4 build of GLM 5.3 Flash on 9 September. I checked 18 builds of that model and of Qwen 3.8 Flash Next for the fault I had found on my own cluster, and found it in three GLM builds, the most downloaded one among them, and in neither of NVIDIA's own.
Plays here, from YouTube
On 9 September NVIDIA published its own NVFP4 build of GLM 5.3 Flash, nine days after one for Qwen 3.8 Flash Next (the dates are from its model cards). NVFP4 is the four bit format that NVIDIA says its Blackwell chips support in hardware, and the DGX Spark has one of them. It matters which build you pick, because not every one is sound.
The fault
At the end of August I found that the build of GLM 5.3 Flash on my cluster had the fault described in an open vLLM bug report. In 69% of its expert networks, the two halves carried different scaling values, and vLLM only reads one of them. I read all 12,384 pairs from the build's files on 31 August, and only 30.9% of them were identical.
English came out fine, which is why I had not noticed, but emoji and Korean came out garbled, and some answers never stopped. In my runs on 31 August, an English answer with emoji came back with 48 broken characters and Korean answers with 6 and 5, against 0 for plain English prose.
The 18 builds
So on 11 September I checked 18 builds of these two models the same way, by reading those values straight out of the files on Hugging Face, three layers from each build.
- Both of NVIDIA's builds matched: 864 of 864 pairs in its GLM build and 1,536 of 1,536 in its Qwen build.
- So did every Qwen build I could read.
- Three GLM builds still carried the mismatch. One was the most downloaded GLM build, with 43,152 downloads in 30 days, and 685 of the 864 pairs I read from it did not match. The other two had exactly the same values.
- There were 6 builds I could not compare.
Four bit activations
NVIDIA's GLM build matched, but its model card says it squeezes the activations to four bits as well as the weights. When I tried a build like that on my boxes on 1 September, it was 20% slower with one user (13.57 engine steps a second down to 10.90, with the tokens in each step unchanged). The weights-only build I moved to that day gave 0 broken characters in 6 runs, and ran at 13.44 steps a second against 13.57.
What I would do
I am staying with the weights-only build I moved to on 1 September.
Read more
- NVIDIA's NVFP4 build of GLM 5.3 Flash on Hugging Facehuggingface.co
- NVIDIA's NVFP4 build of Qwen 3.8 Flash Next on Hugging Facehuggingface.co
- NVIDIA's post introducing the NVFP4 formatdeveloper.nvidia.com
- The open vLLM bug report that describes the faultgithub.com
The numbers in this story
| Figure | What it is | From | Kind |
|---|---|---|---|
| 9 September and 31 August | When NVIDIA published its GLM 5.3 Flash build, and its Qwen 3.8 Flash Next build | huggingface.co | the maker's claim |
| Issue 54150, still open | The vLLM bug report that describes the fault | github.com | reported |
| 69% of 12,384 pairs | Expert networks in my old build whose two halves had different scaling values | my own test | measured by me |
| 18 builds, 43,152 downloads | Builds I checked, and downloads in 30 days of the most downloaded one with the mismatch | my own test | measured by me |
| about 20% | Speed lost with a build that also squeezes the activations to four bits | my own test | measured by me |
| Fault gone, 99% of the speed | The weights-only build I moved to on 1 September | my own test | measured by me |
If a number on this page is wrong, I would rather know. Tell me in the comments under the video and I will correct it here and pin the correction there. What has been corrected so far is on the corrections page.