- *for certain definition of self hosted.
Developed by Chinese open weights model dev Z.ai, GLM 5.1 isn’t exactly a small model. At roughly 754 billion total parameters (40 billion activated for each token generated), the model requires roughly 380 GB of GPU memory just to hold the weights and another 6-26 GB per user to maintain context depending on precision.
To serve GLM-5.3 at scale, Revolut deployed 56 Nvidia B200 GPUs, equivalent in GPU count to seven eight-GPU DGX B200 systems. But as Abitbol noted, the greatest performance gains didn’t come from adding GPUs, but rather tuning the model’s hyperparameters along with switching to NVFP4 quantization, implementing disaggregated prefill and decode, and tuning context lengths.
What I don’t unterstand about stories like this: Why aren’t the US companies making a fuckton of money then? If you can self-host something at 10% of the costs, shouldn’t that mean that a datacenter can make around 90% margin with interference?
Ah…that’s just the one-off costs. Remember…they still have to pay monstrous electric bills. Maybe in the order of $100K or more a month.
Doesn’t Revolut pay for electricity either?