MadeInDex 📰🌎@lemmy.world to Technology@lemmy.worldEnglish · 2 days agoMark Zuckerberg Posts Deranged 6,500-Word Essay About Giving Everyone AI Superintelligencearchive.isexternal-linkmessage-square48linkfedilinkarrow-up11arrow-down10cross-posted to: [email protected]
arrow-up11arrow-down1external-linkMark Zuckerberg Posts Deranged 6,500-Word Essay About Giving Everyone AI Superintelligencearchive.isMadeInDex 📰🌎@lemmy.world to Technology@lemmy.worldEnglish · 2 days agomessage-square48linkfedilinkcross-posted to: [email protected]
minus-squareSeptimaeus@infosec.publinkfedilinkEnglisharrow-up0·17 hours agoAtm the meta for local inference is unified memory and “routed” local agents (multiple smaller role-specific agents in a trench coat) The former is standout for cost efficiency (e.g., 4x RDMA 48gb Minis for a 192gb cluster @ $43/gb vs a $45k b200 alone) The latter is standout for many things, including resource efficiency on smaller machines
Atm the meta for local inference is unified memory and “routed” local agents (multiple smaller role-specific agents in a trench coat)
The former is standout for cost efficiency (e.g., 4x RDMA 48gb Minis for a 192gb cluster @ $43/gb vs a $45k b200 alone)
The latter is standout for many things, including resource efficiency on smaller machines