Eval-Driven Development (EDD)
An engineering practice that embeds comprehensive, multi-dimensional AI evaluation suites into the CI/CD pipeline.
“If you are not evaluating your AI application against a golden dataset in your CI/CD pipeline, you are flying blind in a probabilistic storm.”
When a foundational model provider updates their weights, your application's behavior can change overnight without a single line of your code being altered. Without an eval-driven approach, these regressions go unnoticed until they reach the end user, causing trust erosion and financial loss. EDD provides the safety net required to deploy non-deterministic systems, allowing engineering teams to confidently ship updates, switch underlying models, and optimize prompts while quantitatively proving that system quality has improved or remained stable.
Multi-Hop Causal Traversal Engine
Explore how concepts dynamically feed into each other across 1-hop, 2-hop, and 3-hop transitive relationships. Click any node to navigate the causal highway.
Eval-Driven Development (EDD)
An engineering practice that embeds comprehensive, multi-dimensional AI evaluation suites into the CI/CD pipeline.
Direct Relationships (6)
Transitive Neighbors (Connected via Hop 1)
Extended Causal Ripple Effects
Richard Ewing’s Research Thesis
Evaluation suites must be continuous, multi-dimensional, and treated as first-class citizens in the Agent Development Lifecycle.
Why This Specification Exists
Silent model updates and semantic drift break AI applications unpredictably.
Traditional unit testing and manual QA review.
Deterministic tests cannot evaluate probabilistic text or reasoning.
Continuous, automated evaluation against golden datasets using LLM-as-a-judge.
What Changes If You Believe This?
Testing moves to statistical confidence intervals and LLM-as-a-judge frameworks.
Evals consume API credits, requiring dedicated testing budgets.
Ensures tone and brand safety remain consistent across model updates.
Catches jailbreaks and toxic outputs before production.
Recommended Action by Role
Do not merge prompt changes without running them against your golden dataset.
Latest Publications & Research Activity
Who’s Actually Responsible for Your AI Agents?
Salesforce and SAP are putting AI agents inside your workflows. Who tells them no?
How to Prevent Memory Loss in AI Applications
Frequently Asked Questions
Q:What is a golden dataset?
A curated collection of diverse inputs paired with their verified, ideal outputs.
Canonical Specification Origin
Evaluation suites must be continuous, multi-dimensional, and treated as first-class citizens in the Agent Development Lifecycle.
Corpus Interconnections
Richard Ewing artifacts developed around this canonical framework, including publications, execution tools, and diagnostic models.
External Adoption & Peer Citations
Documented instances where independent researchers, engineering teams, and publications have cited, implemented, or referenced this concept outside Richard Ewing’s ecosystem.
External Evidence: No independently verified references recorded yet.
This concept is part of Richard Ewing’s original baseline canon. External citations and implementations are added only upon rigorous empirical verification.
Inspectable Evidence Ledger
Classified evidence items supporting, extending, or refining this canonical research specification.
Recommended Citation
Ewing, R. (2026). "Eval-Driven Development (EDD)." Richard Ewing Research Canon. Available at: https://www.richardewing.io/concepts/eval-driven-development
@article{ewing_eval_driven_development,
author = {Ewing, Richard},
title = {Eval-Driven Development (EDD)},
journal = {Richard Ewing Research Canon},
year = {2026},
url = {https://www.richardewing.io/concepts/eval-driven-development}
}