Of course, that's a very large question, and one without a universal answer. It depends on a complex matrix of who you are, who the sovereign stack will serve, your use cases, and more. In other words, it's the same capital-versus-operating-expense question every business faces when it invests in an internal project: What's the return on my investment?
Home and business AI now run on the same hardware
Historically, consumers and business IT invested in different things, and we could clearly tell which hardware each was buying. At the start of the desktop boom, home users weren't buying Sun SPARC pizza boxes or SGI workstations. There was a business buyer and a home buyer, and never the twain needed to buy from the same pool.
For the core hardware needed to build a fully on-prem sovereign AI stack, that model has largely evaporated. Yes, we still have "consumer-grade" GeForce GPUs and "enterprise-grade" Nvidia DGX GPU servers. Unlike the usual gap in the hardware world, though, the gap between home and business AI isn't use cases or product readiness. It's simply cost.
The sovereign AI stack I run for Lansby on an Nvidia GeForce RTX 5060 Ti hosts the same AI models (within reason) and handles the same agentic workflows a 5,000-employee enterprise software company would. There are scale differences, of course. I build software for my clients on that stack, but those clients certainly aren't running unicorns on my code...at least I hope not. :)
So the gap between Lansby and an enterprise software vendor isn't how or why we want and need sovereign AI. It's how much we're willing to spend to make it happen.
Revolut's napkin math
British fintech Revolut made news earlier this month when it announced it had traded Anthropic for a completely self-hosted agentic setup, largely because of cost. Let's dig into the hypothetical costs of the switch.
According to tools such as claude-cost-calculator.com, a company of about 1,500 people, roughly a third of them technical, can expect to pay between $250K and $1.5M per year for Claude tools. That's a very broad-stroke estimate. If we're conservative, stick with the lower number, and scale it up to the average enterprise software vendor of 5,000 people, 2,000 of them engineers, we can tiptoe into Fermi estimation and say that vendor spends somewhere shy of $1M a year with Anthropic.
Revolut said it's running 56 Nvidia B200 GPUs, which have a street price of about $2.8M. It also equates that to seven Nvidia DGX B200 servers, which run about $3.6M. So, again, let's estimate a $3M capital expense for the hardware.
That napkin math has Revolut future-proofing its planned AI dependencies for the next three years, for roughly what it would otherwise pay Anthropic over the same stretch.
CapEx vs. OpEx
But it's also a CapEx-versus-OpEx decision, something rarely discussed in blanket statements or questions about going sovereign. OpEx always runs the risk of being unpredictable. I know this all too well from pricing SaaS and hosted management products as a product manager: customers don't like surprises.
Revolut has also hedged its bets against Anthropic price spikes, future internal agentic adoption, platform feature changes, and more. As we always used to say when pricing: CapEx is expensive now; OpEx is expensive tomorrow.
The human cost
Most important of all is the human cost. That includes the learning curve, the cost of supporting the hardware, software stack, and end users of the sovereign platform in-house, and the time it takes to research and test fixes.
In Revolut's case, Z.ai does offer enterprise support for its GLM models. But any time an enterprise moves away from a hosted or managed service, such as the frontier model platforms from Anthropic or OpenAI, new kinds of support friction enter the cost equation. We're back to the cost of testing, implementing, and managing local fixes in one customer's environment, versus relying on the upstream managed provider to fix something for everyone.
Neither option is better or worse, but there's a true cost difference between the two models that goes far beyond the hefty hardware investment.
So what's the true cost of sovereign AI?
It depends, but it's not cheap any way you slice it. Then again, nothing comes easy, and Revolut itself best sums up the benefits of going all-sovereign:
"...as it turned out, when tokens are cheap, you stop worrying about how many they use or what they use them for."
That's empowering.
Same hardware, smaller scale
I know Lansby's sovereign stack isn't anywhere near as powerful as it could be, but I'm not yet ready to drop $7K on doubling our GPU's VRAM or $12K on a Mac Studio to power our internal inference. I mean, I dream about it (both day- and night-dreaming...it's obsessive), but the value just isn't there yet at our scale. Still, it's the same core hardware for the same core use cases.
It's also an excellent stack when you're just starting to dabble in truly agentic, on-prem sovereign AI. And it's a great place to start if you're a mission-driven org worried about the cost of handing your community's data to platform providers.
The question for mission-driven orgs
Where is the line for those orgs? Is it simply a finance question, or is there a point where the reward of mitigating the risk is worth far more than the cost of the hardware?
Did we use AI for this post? Written by Alan Murphy, copyedited for grammar, flow, and story by Claude, Anthropic's AI assistant.