If you ahve a mid range or above GPU you can run local models. You can even run them in a VM or firejail so they do not get any network access.

But please use llama.cpp, not ollama. Ollama has a storied history of poor behavior toward the OSS community. It used llama.cpp as its core tech, while trying to conceal that it did. It refused to disclose that, despite being required by the MIT open source license. It took venture capital money as it concealing its reliance on llama.cpp. And layering closed source components on top.

Many users argue that Ollama is merely a wrapper around llama.cpp and that its developers make no contributions to the core library, effectively exploiting Georgi Gerganov’s code while failing to acknowledge his actual contribution to the success of their product.

Lots more info here, and many other sources. It made bad technical choices and acted sleazy toward OSS projects. Pls don’t reward this kind of behavior. It’s better to directly use llama.cpp. Also there is less obscuration that way, since ollama insists on misleading renamings, etc.

Friends Don’t Let Friends Use Ollama

Under the hood, it was llama.cpp doing all the work. For over a year, Ollama’s README contained no mention of llama.cpp. Not in the README, not on the website, not in their marketing materials. The project’s binary distributions didn’t include the required MIT license notice for the llama.cpp code they were shipping. This isn’t a matter of open-source etiquette, the MIT license has exactly one major requirement: include the copyright notice. Ollama didn’t.