I am looking at resintalling my OS (currently Mint) which I have been running for a few years.
Mint is great and I may end up going back; but was thinking of trying something new.
I want to be able to play with local models and agents; I have a pretty decent system.
- AMD 3700X
- 64 GB DDR4
- AMD RX9070 16GB GPU
- 2TB NVME SSD
- A few TB of HDD’s
What are some good options for easily running models and agents; I have been using ollama to run models; but I’m sure things have moved on since I begun playing with things a year ago.
Give me a few tips; I am going to have a bit of time on my hands over the next few months.
Ah, I was worried that there was some local model distro meta that I wasn’t aware of. I have no knowledge of AMD but according to DeepSeek we seem to have somewhat comparable hardware for local LLM. I just noticed the 64GB RAM and figured you’d be on a whole different level, but apparently that setup is in the mid-high tier, spanning from 7-32B? What kind of models have you tried so far, and did any stand out? I’ve only been able to get a 13B Qwen3.5 model as most myself (which was way too slow to be worth it). From what I have picked up >24B is where you get a generally good model that goes beyond a niche purpose.
It has been a while, but last model I ran was gemma4:e4b 9.6GB. It ran very quickly; also have run deepseek-coder-v2:16b 8.9GB, again very fast…
I am looking to get back into it once I have a bit of time on my hands; I’ll look into the state of the art models again and see what I can run.