Accenture sells AI transformation to thousands of clients while quietly telling its own employees they get a fraction of what some were already spending. The internal memo reveals a tension at the heart of every major consulting firm racing to profit from AI.
If you’re spending $1K per month on LLMs – putting aside whether this is productive or ethical for a moment – wouldn’t it make more sense to just pony up the $5K-$10K or so that it would cost to provision a private, local model? Might that not actually be superior in terms of corporate security?
Not at the moment. It’s definitely cheaper to buy LLMs as a service than running an in-house server hall. Anthropic and OpenAI are running at a loss, you know.
Now, risking to lose your data to the service provider is another thing.
Look at it this way, if I pay Open AI $1k a month, their people make sure it works they month. If I let Joe from finance spend $10k on computer parts, Joe needs to ensure that it runs or I need to hire someone who is knowledgeable on local models. If I’m a simple firm, $12k a year might be cheaper because someone else makes sure it works and I can fire Joe when the fancy autocorrect loses a bunch of clients by falsifying work. Then I don’t have to spend even bigger bucks hiring a new Joe because no one knows how to work the damn server he replaced the break room with.
It depends, sometimes the private API models are better at a specific task to the point it wouldn’t be worth using one that is worse. (Really it’s the harnesses around them tbh)
Overall sovereign AI is where it’s at. If you have good enough tooling in place
Definitely. We do this at my job. But a server big enough to provide a comparable service would cost at least $30k. And you likely need more than one. And you still need someone to maintain it.
I’ve heard that GPU’s for this work are exceedingly hard to get ahold of as well. Glm 5, which i think is near frontier model capabilities, takes around 1.5 Tb of vram un-quantized.
Not hard. Just really fucking expensive and might have a long lead time.
So hard, then.