Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say — data center demand is expected to rival Japan's electricity usage by 2030

bat1000@sh.itjust.works · 9 days ago

Avid Amoeba@lemmy.ca · 9 days ago

Thats kinda how the whole Chinese competition started - optimization to make due with limited hw.

Also I just changed the inference engine I use for local models and Qwen 3.8 27B went from 50tps to 150tps. The new engine is optimized for my hw.

Lydia_K@lemmy.world · 8 days ago

What did you switch to?

Avid Amoeba@lemmy.ca · 8 days ago

Radiance. Was using the latest llama.cpp. This thread caught my eye as simple enough to try.

Agent641@lemmy.world · 8 days ago

What are the specs of the machine that runs it?