The GPU cost decision matrix every AI team needs. On-prem vs. cloud vs. hybrid analysis.
On-prem GPUs: $30K-$200K upfront, low marginal cost, 18-24 month payback. Cloud GPUs: $0 upfront, $2-4/hr for A100, instant scaling. Spot instances: 60-70% cheaper but unreliable.
Rules of thumb: If utilization <40%, cloud wins. If >70%, on-prem wins. 40-70% is the gray zone where hybrid makes sense. Always model 18-month TCO, not monthly cost.
Like this analysis?
Get the weekly engineering economics briefing — one email, every Monday.
Subscribe Free →Most AI features are margin-negative. Here's the exact framework to calculate whether your AI feature makes money.
10 minYour AI feature impresses users but destroys margin. Here's the decision framework.
9 minEveryone talks about RAG accuracy. Nobody talks about RAG economics.
11 minPublished Work
This article expands on ideas from my published work in CIO.com, Built In, Mind the Product, and HackerNoon. View published articles →
The Product Economist — Quantifying engineering economics for technology leaders, PE firms, and boards.