Except the people I hear running “local models” report a lot of hardware, like a half dozen graphics cards. The common person is screwed, they have a smartphone.
Two or three years ago sure. Now…not even close. Tons of models run on basically shit now. Especially with MOEs. I run new lightweight models on an i9 and 64gb of ddr4 ram with a 4gb quadro card…in a laptop from 19’.
I think this is why a lot of these tech bros are saying silly shit about suggesting people need to rent from them. That’s their entire shtick. Otherwise they got nothing else.
If you’ve got at least 8 gigs of VRAM and at least 32 gigs of RAM you can run some pretty beefy Qwen models using Strata, far more than you normally could. Upgrade that to 12 or 16 and 64 gigs respectively and you can run a pretty decent quant of Qwen 3.8 flash next. It’s a LOT bigger than the usual 27 and 35 billion models people run locally that themselves require pretty beefy hardware. Strata is doing some magic by deciding which experts stay in VRAM vs which ones are in RAM vs which ones are so rarely used that they can just be read from the SSD as needed (though it’s best if you don’t have to result to the SSD as a backup option at all)
The federal debt and the US treasuries part of the fed balance sheet is how much taxes should increase. They’re different things, the fed can just buy bonds and put them on its balance sheet.
You used to be able to BIOS flash workstation GPUs to make them somewhat usable for gaming… not sure how far these AI GPUs have diverged from gaming cards these days and if a BIOS flash could make them playable.
AI GPUs have specialized very heavily for neural networks, including things like going down to 8- or 4-bit floats in favor of more parallelization. Most of that hardware is just useless for graphics tasks at this point.
Government bales out AI companies and the bill they write to do it will make local LLMs illegal.
I think you are right, but that seems VERY unenforceable.
Except the people I hear running “local models” report a lot of hardware, like a half dozen graphics cards. The common person is screwed, they have a smartphone.
Two or three years ago sure. Now…not even close. Tons of models run on basically shit now. Especially with MOEs. I run new lightweight models on an i9 and 64gb of ddr4 ram with a 4gb quadro card…in a laptop from 19’.
I think this is why a lot of these tech bros are saying silly shit about suggesting people need to rent from them. That’s their entire shtick. Otherwise they got nothing else.
Got some references and keywords for research?
If you’ve got at least 8 gigs of VRAM and at least 32 gigs of RAM you can run some pretty beefy Qwen models using Strata, far more than you normally could. Upgrade that to 12 or 16 and 64 gigs respectively and you can run a pretty decent quant of Qwen 3.8 flash next. It’s a LOT bigger than the usual 27 and 35 billion models people run locally that themselves require pretty beefy hardware. Strata is doing some magic by deciding which experts stay in VRAM vs which ones are in RAM vs which ones are so rarely used that they can just be read from the SSD as needed (though it’s best if you don’t have to result to the SSD as a backup option at all)
https://youtu.be/8F_5pdcD3HY
This guy does a great brain dump on how little is needed to run LLMs on basically old junk.
Successful enforcement doesn’t look like catching every pivate LLM owner.
With what money? No one is buying bonds. The US would have to print so hard it’d cause hyperinflation, which still means the bubble pops
The fed balance sheet gets bigger like it did with AIG.
Again, how? No one is buying American bonds right now. You can’t raise the debt if no one will loan you money
The federal debt and the US treasuries part of the fed balance sheet is how much taxes should increase. They’re different things, the fed can just buy bonds and put them on its balance sheet.
How much taxes should increase?
Wait, is this just printing money with extra steps? Cause it sounds a lot like printing money with extra steps
hardware that people cant USE, they are solely used for AI in those datacenters, its just going to be e-waste.
You used to be able to BIOS flash workstation GPUs to make them somewhat usable for gaming… not sure how far these AI GPUs have diverged from gaming cards these days and if a BIOS flash could make them playable.
AI GPUs have specialized very heavily for neural networks, including things like going down to 8- or 4-bit floats in favor of more parallelization. Most of that hardware is just useless for graphics tasks at this point.
They don’t even have headers for soldering a display output to them. They’re purely a compute card.
It’d work nicely as a pure accelerator for an actual GPU though, with the right firmware.
Almost back to the 3DFX pass-through days again… 😁