Eval-Driven Development (EDD)
An engineering practice that embeds comprehensive, multi-dimensional AI evaluation suites into the CI/CD pipeline.
“If you are not evaluating your AI application against a golden dataset in your CI/CD pipeline, you are flying blind in a probabilistic storm.”
When a foundational model provider updates their weights, your application's behavior can change overnight without a single line of your code being altered. Without an eval-driven approach, these regressions go unnoticed until they reach the end user, causing trust erosion and financial loss. EDD provides the safety net required to deploy non-deterministic systems, allowing engineering teams to confidently ship updates, switch underlying models, and optimize prompts while quantitatively proving that system quality has improved or remained stable.
Richard Ewing’s Research Thesis
Evaluation suites must be continuous, multi-dimensional, and treated as first-class citizens in the Agent Development Lifecycle.
Why This Specification Exists
Silent model updates and semantic drift break AI applications unpredictably.
Traditional unit testing and manual QA review.
Deterministic tests cannot evaluate probabilistic text or reasoning.
Continuous, automated evaluation against golden datasets using LLM-as-a-judge.
What Changes If You Believe This?
Testing moves to statistical confidence intervals and LLM-as-a-judge frameworks.
Evals consume API credits, requiring dedicated testing budgets.
Ensures tone and brand safety remain consistent across model updates.
Catches jailbreaks and toxic outputs before production.
Recommended Action by Role
Do not merge prompt changes without running them against your golden dataset.
Latest Publications & Research Activity
Salesforce and SAP are putting AI agents inside your workflows. Who tells them no?
How to Prevent Memory Loss in AI Applications
Giving an AI a bigger memory window is like giving a confused worker a bigger inbox.
Frequently Asked Questions
Q:What is a golden dataset?
A curated collection of diverse inputs paired with their verified, ideal outputs.
Inspectable Evidence Ledger
Classified evidence items supporting, extending, or refining this canonical research specification.
Recommended Citation
Ewing, R. (2026). "Eval-Driven Development (EDD)." Richard Ewing Research Canon. Available at: https://www.richardewing.io/concepts/eval-driven-development
@article{ewing_eval_driven_development,
author = {Ewing, Richard},
title = {Eval-Driven Development (EDD)},
journal = {Richard Ewing Research Canon},
year = {2026},
url = {https://www.richardewing.io/concepts/eval-driven-development}
}