Training is 20% of ML costs. Serving, monitoring, and retraining are the other 80%.
Training gets all the attention, but it's 20% of total cost. The other 80%: model serving (30%), monitoring and evaluation (15%), data pipelines (20%), retraining (15%).
Optimization: serve models on CPU where latency allows (10x cheaper than GPU serving), batch predictions for non-real-time use cases, implement model compression (2-5x inference cost reduction), and set retraining triggers based on drift (not schedules).
Like this analysis?
Get the weekly engineering economics briefing — one email, every Monday.
Subscribe Free →Data pipelines cost 2-5x what most organizations budget. Here's why.
10 minWarehouse costs vary 3-5x depending on query patterns, not headline pricing.
11 minReal-time data processing costs 3-5x more than batch for most use cases.
8 minPublished Work
This article expands on ideas from my published work in CIO.com, Built In, Mind the Product, and HackerNoon. View published articles →
The Product Economist — Quantifying engineering economics for technology leaders, PE firms, and boards.