a lightweight hybrid reasoning MoE model with 7.9B total parameters and only 1.3B activated parameters per token. It is designed to deliver strong reasoning and agentic capabilities under a small inference compute footprint, making advanced model capabilities more accessible for local and resource-constrained deployment.
You must log in or register to comment.
I haven’t tried it yet, but I tested one of their previous ones with CPU interference on an Intel n97, and it was one of the best in terms of t/s performance and also prompt response quality on the metrics I tested against. When this run on llama.cpp I will try and give it a test.
anyone try this? this might be good for my crappy laptop lol
is it good enough to use with Zoo Code? is it better than Qwen 3.5 4b?
EDIT: woa
But not yet supported in llama.cpp https://github.com/ggml-org/llama.cpp/pull/26608
Would this run on a cpu only pc? Would it be crazy slow if so?
Looking for some small model to cut my teeth on and I only have my laptop at the moment.
I’ve run Qwen 3.5 4b and Gemma 4 e2b on CPU only, this should be faster than those I think (fewer active parameters). If you have AVX512 or AVX10 then it should help a bit. Still slow compared to a GPU lol.
How slow 😁😅?
my laptop is crappy, so like 5 tokens per second lol, prompt processing of like 20 tokens per second
I think a decent laptop nowadays, even running CPU only, could probably do like 5x faster
Thanks for the info!
