a lightweight hybrid reasoning MoE model with 7.9B total parameters and only 1.3B activated parameters per token. It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment.
Would this run on a cpu only pc? Would it be crazy slow if so?
Looking for some small model to cut my teeth on and I only have my laptop at the moment.
I’ve run Qwen 3.5 4b and Gemma 4 e2b on CPU only, this should be faster than those I think (fewer active parameters). If you have AVX512 or AVX10 then it should help a bit. Still slow compared to a GPU lol.
How slow 😁😅?
my laptop is crappy, so like 5 tokens per second lol, prompt processing of like 20 tokens per second
I think a decent laptop nowadays, even running CPU only, could probably do like 5x faster
Thanks for the info!