Delivery & Reliability

AI-Assisted Development encompasses the integration of advanced Large Language Models, coding agents, and generative copilots directly into the software development lifecycle (SDLC).

By 2025/2026, tools like Cursor, GitHub Copilot, Devin, and SWE-Agent evolved from simple autocomplete engines to autonomous architectural reasoning systems. The paradigm shifted developers away from "writing code" and towards "prompt supervision, structural review, and security verification."

While AI Dev tools radically boost individual throughput, they create significant systemic risks around codebase vastness (software entropy), undocumented context fragmentation, and the unprecedented generation of undetectable AI Technical Debt.

Contract testing verifies that the interactions between service providers and consumers conform to a shared contract (API specification). Instead of testing the full integration, contract tests verify the interface agreement — what data shapes, status codes, and behaviors each side expects.

Tools: Pact (consumer-driven contract testing), Spring Cloud Contract (provider-side contracts), and OpenAPI-based contract testing. Pact workflow: 1) Consumer writes a test defining expected API behavior, 2) Pact generates a contract file, 3) Provider verifies it can satisfy the contract, 4) Both sides run independently.

Contract testing bridges the gap between unit tests (too isolated) and integration tests (too coupled). It enables independent service deployment — teams can deploy without coordinating with every consumer.

End-to-end testing verifies complete user flows through the entire application — from UI interaction to backend processing to database persistence and back. E2E tests simulate real user behavior in a real (or staging) environment.

Tools: Playwright (Microsoft, cross-browser, fastest), Cypress (developer-friendly, single browser), Selenium (legacy, broadest browser support), and Puppeteer (Chrome/Chromium only).

E2E testing best practices: Test critical paths only (login, checkout, core workflows), keep E2E tests stable (avoid flaky selectors, use data-testid attributes), run in CI/CD (not blocking development), and limit quantity (10-50 E2E tests, not 500). Visual regression testing (Percy, Chromatic) catches UI changes that functional tests miss.

Integration testing verifies that multiple components work correctly together — testing the interfaces and interactions between modules, services, databases, and external APIs. Unlike unit tests (isolated) or E2E tests (full system), integration tests focus on the boundaries between components.

Types: Component integration (testing a service with its database), API integration (testing client-server contract), Third-party integration (testing with external APIs using mocks or sandboxes), and Data integration (testing data flows between systems).

Tools: Testcontainers (spin up real databases/services in Docker for tests), Pact (contract testing for APIs), WireMock (HTTP mock server), and database-specific test utilities. Integration tests typically run in CI/CD, not on developer machines.

Kanban is a workflow management method that visualizes work, limits work in progress (WIP), and optimizes flow. Originating from Toyota's manufacturing system, it was adapted for software development by David J. Anderson.

Kanban principles: visualize the workflow (board with columns), limit WIP (set maximum items per column), manage flow (optimize throughput), make policies explicit, and improve collaboratively.

Kanban vs. Scrum: Kanban is flow-based (continuous), Scrum is iteration-based (sprints). Kanban has no prescribed roles, Scrum has Scrum Master and Product Owner. Kanban limits WIP per column, Scrum limits work per sprint. Many teams use "Scrumban" — combining elements of both.

The key insight: limiting WIP reduces context-switching, which dramatically improves productivity. A developer working on 5 things completes all 5 slower than a developer working on 2 things sequentially.

Kanban is a workflow management method that visualizes work, limits work-in-progress (WIP), and optimizes flow. Unlike Scrum's fixed sprints, Kanban uses continuous flow — items move through stages as capacity allows.

**Core principles:** - **Visualize work:** Board with columns (To Do, In Progress, Review, Done) - **Limit WIP:** Maximum items per column (e.g., 3 items in "In Progress") - **Manage flow:** Optimize cycle time (time from start to done) - **Make policies explicit:** Clear definitions of "done" for each stage - **Feedback loops:** Regular review of metrics and process

**Kanban metrics:** Cycle time (how long items take), throughput (items completed per week), WIP count, and cumulative flow diagrams.

Kanban is ideal for teams with unpredictable work (support, ops, maintenance) or teams that find Scrum sprints too rigid.

Load testing measures how a system performs under expected and peak traffic conditions. It identifies performance bottlenecks, memory leaks, and scalability limits before they affect real users.

Types: Load testing (expected traffic volume), Stress testing (beyond expected capacity — find the breaking point), Spike testing (sudden traffic surge), Soak testing (sustained load over hours — find memory leaks), and Chaos testing (failure injection under load).

Tools: k6 (Grafana, modern, JavaScript-based), Locust (Python-based, distributed), JMeter (Java-based, GUI-heavy), and Gatling (Scala-based, CI/CD friendly). Key metrics: response time (p50, p95, p99), throughput (requests/second), error rate, and resource utilization (CPU, memory, connections).

Regression testing verifies that previously working functionality still works after code changes. It catches "regressions" — bugs introduced by new code that break existing behavior. Regression testing is the safety net that enables continuous deployment.

Approaches: Automated regression suite (run the full test suite on every deployment), Selective regression (run only tests affected by changed code — tools like Jest --changedSince), Visual regression (screenshot comparison to catch UI changes — Percy, Chromatic), and Smoke testing (quick subset of critical tests run immediately after deployment).

Regression testing ROI: Without regression tests, every deployment requires manual verification of everything that could break. With regression tests, verification is automated, allowing daily (or hourly) deployments with confidence.

A crude but effective method for testing whether a presumed Zombie Asset is actually being used. The feature or service is temporarily turned off (first in staging, then production). If no one "screams" (complains about it being missing), the asset is permanently removed.

Shift-left testing is the practice of moving testing activities earlier in the software development lifecycle — from post-development QA to during and before development. The principle: the earlier you find a bug, the cheaper it is to fix.

Shift-left techniques: TDD (write tests before code), Static analysis (catch bugs without running code — TypeScript, ESLint, SonarQube), Code review (human review before merge), Pre-commit hooks (run linting and unit tests before commit), CI pipeline (run tests on every push), and Security scanning (SAST/DAST in CI/CD).

Bug cost multiplier by phase: requirements (1x), design (5x), development (10x), testing (20x), production (100x). A bug caught in development costs 10% of what it costs in production.

Shift-Left Testing is the practice of moving testing activities earlier in the software development lifecycle. Instead of testing only after code is written, shift-left integrates testing into design, development, and CI/CD stages.

**Types of shift-left:** - **Design-level testing:** Architecture reviews, threat modeling before coding - **Unit testing in development:** TDD and property-based testing during coding - **CI pipeline testing:** Automated tests run on every commit - **Security shift-left:** SAST/DAST tools integrated into PR workflows - **Compliance shift-left:** Governance checks embedded in build pipelines

For AI systems, shift-left means testing model quality, fairness, and safety during development — not after production deployment.

Sprint planning is the Scrum ceremony where the team decides what work to commit to for the upcoming sprint (typically 2 weeks). The team selects items from the prioritized product backlog based on their capacity and velocity.

Effective sprint planning: review sprint goal (what outcome are we pursuing?), select backlog items that contribute to the goal, break stories into tasks, estimate effort, and identify dependencies and risks.

Common sprint planning mistakes: overcommitting (teams consistently plan more than they can deliver), not accounting for meetings and maintenance, planning without a clear sprint goal, and spending too long in planning (timebox to 2-4 hours).

Sprint length: 2 weeks is the most common. 1 week for fast-moving teams. 3-4 weeks for teams with longer release cycles. Consistency matters more than length — pick a cadence and stick with it.

A sprint retrospective is a team ceremony held at the end of each sprint to reflect on what went well, what didn't, and what to improve. It's the primary mechanism for continuous improvement in agile teams.

Classic retro format: What went well? What didn't go well? What should we change? Teams vote on the most important items and commit to 1-3 specific action items for the next sprint.

Alternative formats: Start/Stop/Continue, 4Ls (Liked/Learned/Lacked/Longed for), Sailboat (wind = helps, anchors = hinders), and Mad/Sad/Glad.

Retro anti-patterns: not following up on action items (the #1 killer of retro effectiveness), managers dominating the conversation, team members not feeling safe to speak up, repetitive discussions without resolution, and running retros as status meetings instead of improvement sessions.

A sprint retrospective is a meeting held at the end of each sprint where the team reflects on what went well, what didn't, and what to improve. It's the core continuous improvement mechanism in agile development.

**Standard format (Start/Stop/Continue):** - **Start:** What should we begin doing? - **Stop:** What should we stop doing? - **Continue:** What's working that we should keep?

**Alternative formats:** 4Ls (Liked, Learned, Lacked, Longed For), Sailboat (wind = what propels us, anchor = what holds us back), Mad/Sad/Glad.

Effective retros lead to concrete action items (max 2-3 per sprint). Ineffective retros are therapy sessions with no follow-through. The key difference: action items are tracked and reviewed at the next retro.

Story points are a unit of estimation used in agile development to measure the relative effort, complexity, and uncertainty of a user story. They use a modified Fibonacci sequence (1, 2, 3, 5, 8, 13, 21) where higher numbers represent more effort and uncertainty.

Story points are relative, not absolute. A 5-point story is roughly 2.5x the effort of a 2-point story. The absolute time varies by team — one team's 5 might take 2 days while another team's 5 takes 4 days. This is fine because points measure relative effort within a team.

Story point criticisms: they're often misused as productivity metrics (comparing team velocities), they don't measure value (a 13-point story might deliver zero customer value), and they can be gamed (teams inflate points to look more productive).

Modern alternatives: cycle time (actual time from start to done), no estimation (just break stories into similarly-sized chunks), and outcome-based tracking (did we achieve the sprint goal?).

Story points are a relative estimation unit used in agile development to measure the effort, complexity, and uncertainty of user stories. They use the Fibonacci sequence (1, 2, 3, 5, 8, 13, 21) to estimate relative size.

**Key principle:** Story points measure relative effort, not time. A 5-point story is roughly twice as complex as a 3-point story. This relative approach accounts for different developers having different speeds.

**Estimation techniques:** Planning Poker (team members independently estimate, then discuss outliers), T-shirt sizing (S, M, L, XL as a first pass), and reference stories (compare new work to previously completed stories).

**The controversy:** Many engineering leaders argue story points are "agile theater" — energy spent estimating instead of building. Richard Ewing's perspective: story points measure activity, not value. Revenue per engineer (APER) measures what actually matters.

Technical Program Management is the discipline of coordinating complex, cross-team technical initiatives from planning through delivery. TPMs combine project management skills with technical understanding to drive programs that span multiple engineering teams.

TPM responsibilities: defining program scope and milestones, managing cross-team dependencies, risk identification and mitigation, stakeholder communication, and driving decisions when teams can't resolve conflicts.

TPMs are essential for: platform migrations, infrastructure modernization, multi-team feature launches, compliance programs (SOC 2, GDPR implementation), and technical debt reduction initiatives.

TPM vs. Engineering Manager: EMs manage people on a single team. TPMs manage programs across multiple teams. EMs focus on team health and execution. TPMs focus on cross-team coordination and delivery.

The test pyramid is a testing strategy that prescribes many fast, cheap unit tests at the base, fewer integration tests in the middle, and a small number of slow, expensive end-to-end tests at the top. Coined by Mike Cohn, the pyramid shape reflects the ideal ratio: many small tests, few large tests.

Layers: Unit tests (test individual functions/classes in isolation, milliseconds, thousands of them), Integration tests (test component interactions, seconds, hundreds), End-to-end tests (test full user flows through the real system, minutes, dozens), and Manual/exploratory tests (human verification, rare, for subjective quality).

The anti-pattern is the "ice cream cone" — many E2E tests, few unit tests. This creates slow, flaky, expensive test suites that developers avoid running. The test pyramid keeps the feedback loop fast.

Test-Driven Development (TDD) is a development methodology where you write a failing test before writing the code to make it pass. The cycle is Red → Green → Refactor: 1) Write a test that fails (Red), 2) Write the minimum code to make it pass (Green), 3) Refactor the code while keeping tests passing.

TDD benefits: Forces you to think about the interface before implementation, produces high test coverage naturally, catches regressions immediately, and creates living documentation (tests describe expected behavior). TDD works best for business logic and algorithms but may be overkill for UI code or rapid prototyping.

TDD skepticism is common but misguided: the initial velocity decrease (writing tests first feels slower) is offset by dramatically reduced debugging time, fewer production bugs, and fearless refactoring ability.

Unit testing is the practice of testing individual functions, methods, or classes in isolation from the rest of the system. Unit tests are the foundation of the test pyramid — fast (milliseconds), focused (one assertion per test), and numerous (thousands in a mature codebase).

Unit test characteristics (FIRST): Fast (run in milliseconds), Independent (no shared state between tests), Repeatable (same result every time), Self-validating (pass/fail without human inspection), and Timely (written alongside or before the code).

Tools by language: JavaScript (Jest, Vitest), Python (pytest), Java (JUnit), Go (built-in testing package), Rust (built-in #[test]). Test coverage targets: 80%+ line coverage for business logic, 90%+ for critical financial/security code.

Visual regression testing captures screenshots of UI components or pages and compares them pixel-by-pixel against baseline images to detect unintended visual changes. It catches CSS regressions, layout shifts, font changes, and responsive design issues that functional tests miss.

Tools: Percy (BrowserStack), Chromatic (Storybook), BackstopJS (open-source), and Playwright's built-in screenshot comparison. Workflow: 1) Capture baseline screenshots, 2) Run changed code, 3) Capture new screenshots, 4) Compare pixel-by-pixel, 5) Flag differences for human review.

Visual regression testing is especially valuable for: design systems (ensuring consistency across components), responsive design (testing across breakpoints), and theme changes (verifying dark mode, accessibility modes don't break).