Table of Contents
1. The Ten Agent Fault Lines
An agent doesn't fail in one generic way — it fails in specific, recognizable patterns, and naming them is what makes them catchable. The ten fault lines below group into three clusters: failures in the agent's own reasoning (Cognitive Breakdowns), failures in how it acts on and integrates with its environment (Execution Gaps), and failures in the human relationship to the agent (Trust Erosion). Each is illustrated with a worked case study drawn from banking, insurance, and financial services (BFSI), where these failures carry real regulatory and financial stakes.
Cluster 1 — Cognitive Breakdowns
1Context Collapse
Overloading the agent with unfiltered context leads to confusion, not clarity.
Case study: A wealth-management commentary agent fed a client's entire transaction history, every market headline, and the full compliance policy text in one prompt produces contradictory, unfocused portfolio advice.
How evals help: Treat "how much context, and which parts" as a testable variable, not a guess. Build golden cases that vary the context window — trimmed vs. bloated, curated vs. raw — for the same underlying task, and score each variant against ground truth or a rubric (Chapter 2). Trace inspection shows exactly which context elements the agent actually used versus silently ignored.
2Plan Fragility
Linear, brittle workflows fail when one step breaks and no recovery is possible.
Case study: A loan-origination document pipeline (extract → validate → route) silently stalls when it hits a document type it wasn't built for, with no fallback step.
How evals help: This is exactly the boundary-and-edge-case discipline from Chapter 3's Golden Dataset design — GD-007's boundary case and GD-006's cascading-failure case, generalized. Deliberately construct golden cases representing malformed or unusual inputs, and score whether the agent recovers gracefully or fails silently.
3Hallucination
Fluent, confident answers mask ungrounded reasoning and fabricated results.
Case study: An AML/KYC agent drafts a Suspicious Activity Report narrative that includes transaction details never present in the source data — a serious regulatory exposure.
How evals help: This is Chapter 1's "guessing vs. grounding" problem in its most consequential form. Score outputs against execution-based ground truth wherever it exists — did the claimed detail actually appear in the source record — rather than trusting fluent prose at face value. Scoring "groundedness" as its own rubric dimension, separate from "correctness," catches confident fabrication a surface read would sail right past.
Cluster 2 — Execution Gaps
4Integration Trap
Tool or API changes silently corrupt the agent's behavior without alerts.
Case study: A credit-bureau API used by an underwriting agent changes its response schema; the agent keeps running but silently misreads risk fields.
How evals help: This is exactly Chapter 4's two-layer point: the API's schema is deterministic and belongs to classic QA — a contract test that fails loudly the instant the schema changes. What evals add on top is catching the softer failure classic QA misses: the agent kept running and silently misinterpreted the new shape. Rerunning the Golden Dataset after any dependency change turns a silent behavioral drift into a visible score movement.
5Leaky Memory
Context leaks across sessions or users, eroding reliability and trust.
Case study: A multi-tenant collections chatbot carries context from one delinquent customer's account into the next customer's session.
How evals help: This is a Golden Dataset design pattern in its own right — Chapter 3's "isolate one thing per case" principle, applied to session boundaries instead of business rules. Construct multi-session test cases specifically built to check for cross-contamination, and score pass/fail on whether one user's session ever reflects another user's data.
6Blind Autonomy
Agents act without awareness, escalate nothing, and amplify mistakes.
Case study: A claims-adjudication agent auto-approves claims above its intended authority threshold, amplifying a pricing error across thousands of claims before anyone notices.
How evals help: Define explicit rubric thresholds for when a decision requires human review — Chapter 4's hard-gate concept, applied to the agent's own behavior — and score whether it actually escalates when it should. "Should have escalated but didn't" is scored exactly like Chapter 3's rule-level recall metric, just applied to an escalation decision.
7Black Box Behavior
Lack of reasoning visibility turns debugging into guesswork.
Case study: A fraud-triage agent misses a fraud pattern, and compliance reviewers can't reconstruct why the agent didn't flag it.
How evals help: An eval that can only see the final answer, not the reasoning trace, can't tell a lucky guess from genuine comprehension — exactly why Chapter 2's think → act → observe loop matters for evaluation, not just architecture. Logging the full trace alongside every eval run (Chapter 4's bookkeeping pattern) turns a later failure into something investigable instead of a permanent mystery.
Cluster 3 — Trust Erosion
8Locked Box Interface
Users can't see or steer agent decisions, creating opacity and disengagement.
Case study: Advisors using a robo-advisory rebalancing agent have no visibility into why it recommends a given trade, and stop trusting or using it.
How evals help: Rubric-based scoring already requires a judge — human or LLM — to see the agent's rationale, not just its final answer. Exposing that same structured rationale to the end user, not just the eval system, is what turns an opaque decision into something a person can actually steer — eval transparency and user transparency draw on the same underlying data.
9Agent Drift
Agents expand beyond their intended scope with no ownership or oversight.
Case study: A collections outreach agent, originally scoped to draft messages for human review, starts autonomously scheduling outreach and altering settlement terms.
How evals help: This is precisely why regression testing against a fixed Golden Dataset matters: rerun the same scope-defining test cases on every release, and any expansion beyond originally-approved behavior shows up as an unexpected result the dashboard flags immediately — instead of silently accumulating until someone notices by accident.
10Misaligned Metrics
Agents optimize for the wrong goals, delivering speed or volume over quality.
Case study: A trading-support agent optimized purely for execution speed skips suitability checks to maximize throughput.
How evals help: This is Chapter 5's Goodhart's Law, by name. An agent optimized for one narrow proxy — throughput, speed, a single recall number — will find the cheapest way to satisfy that proxy, exactly like Chapter 5's "flag everything" agent maximizing recall while destroying precision. The fix: track multiple, orthogonal metrics (quality alongside throughput, not instead of it), and treat any one metric moving in isolation as a reason to investigate, not celebrate.
The pattern underneath all ten
Notice that none of these fixes required a new eval concept — every one is a direct application of something already built in Chapters 1 through 5: grounding vs. guessing, execution-based scoring, rubric dimensions, regression testing against a fixed dataset, dashboards with per-category breakdowns, and Goodhart's Law awareness. The ten fault lines aren't ten new problems — they're ten concrete situations where the general discipline this series has been building actually gets used.
2. How Many Times Do You Actually Need to Run an Eval?
Chapter 1 established that one run tells you almost nothing. Chapters 3 and 4 then quietly leaned on "run it 5 times" as a convenient illustrative number without ever justifying it. It's time to.
Framing it as a statistics problem
When an eval scores something as pass/fail — did the agent find rule R10, did it correctly escalate — each run is a sample from an unknown, true underlying probability that the agent gets that specific thing right. That's a classic binomial proportion estimation problem, identical in structure to "if I flip a weighted coin some number of times, how confident can I be about its true bias?" The relevant formula, using the normal approximation for a 95% confidence interval:
margin of error ≈ 1.96 × √( p(1-p) / n )where p is the observed pass rate and n is the number of runs. Two things fall directly out of this formula, and both matter a great deal in practice.
It shrinks with the square root of n — not with n itself
This is the single most important, least intuitive fact in this section: to cut your margin of error in half, you don't need twice the runs — you need four times the runs. Concretely, for an agent with a true pass rate around 90% (p = 0.9):
| Runs (n) | Margin of error (95% confidence) |
|---|---|
| 5 | ± 26.3% |
| 10 | ± 18.6% |
| 30 | ± 10.7% |
| 50 | ± 8.3% |
| 100 | ± 5.9% |
| 200 | ± 4.2% |
| 400 | ± 2.9% |
At 5 runs — the number Chapters 3 and 4 used for illustration — a measured "90%" could honestly be anywhere from roughly 64% to 100%. That's not a precise number; it's barely a direction. It's genuinely useful for "did this obviously break" — but nowhere near precise enough to defend a ship-gate decision.
The hardest agents to measure precisely are the mediocre ones
p(1-p) is maximized at p = 0.5 and shrinks toward zero as p approaches either 0% or 100%. In plain terms: an agent that's genuinely excellent (99% reliable) or genuinely bad (5% reliable) can be measured precisely with relatively few runs, because there's little variance left to pin down. An agent sitting near a coin flip — 50/50 on some judgment call — is the statistically hardest case to nail down, and needs the most runs to say anything confident about at all.
Two different levers, two different questions
Don't confuse repeating the same case with growing the dataset — they answer different questions:
- Repeating one golden case multiple times tells you how reliable the agent is on that specific input — the variance across identical, repeated attempts.
- Adding more, different golden cases tells you how well the agent generalizes across the space of inputs — and gives you a much larger effective sample size for an overall aggregate accuracy metric.
The Rule of Three — for compliance-critical hard gates
Chapter 4's ship-gate table demanded ≥ 99% recall on compliance-sensitive rules like R4, R10, and R13. A useful shortcut for the "zero failures observed" case: if you run n trials and see zero failures, you can be roughly 95% confident the true failure rate is no higher than 3/n.
- 0 failures in 50 runs → 95% confident the true failure rate is at or below
3/50 = 6%. Not good enough to defend a 99% bar. - To be 95% confident the true failure rate is at or below 1% (i.e., ≥ 99% reliability), you need
3/n ≤ 0.01— roughly 300 clean runs with zero observed failures.
The real cost/statistical-power tradeoff
Every additional run costs real money (LLM API tokens) and real wall-clock time — and that cost grows linearly with n, while your precision only improves with the square root of n. Past a certain point, quadrupling your spend to shave a margin of error from ±8% to ±4% is rarely the best use of an eval budget. Very often, that same budget buys more real insight spent on widening the Golden Dataset than on squeezing one more decimal point of confidence out of a case you've already tested plenty.
| Purpose | Runs per case | Why |
|---|---|---|
| Quick sanity check while iterating on a prompt | 1 | High uncertainty accepted on purpose — looking for an obvious break, not a precise number |
| Day-to-day development / regression sniff-test | 3–5 | Enough to catch a clearly broken change without burning a full statistical sample |
| Consistency scoring for a specific golden case | 5–10 | Enough to tell "genuinely inconsistent" apart from "got unlucky once" |
| Pre-ship hard-gate decision on a compliance-critical metric | 30–50+, or Rule of Three toward hundreds if bar is ≥ 99% | Needs a confidence interval tight enough that even its lower bound clears the threshold |
| Ongoing production monitoring | Every real call counts as a sample | Volume accumulates naturally over time instead of needing expensive upfront repetition |
Why this section had to come last
This is the arithmetic sitting underneath two ideas this series already introduced without fully justifying them. Chapter 4's noise floor — the normal amount a metric wobbles run-to-run with no real change — is exactly the margin of error computed above. Chapter 5's advice to investigate a discontinuous jump rather than celebrate it makes sense precisely because a jump larger than your margin of error is a real signal, and a jump smaller than it is very possibly nothing at all.
3. Key Takeaways
- The Ten Agent Fault Lines — Context Collapse, Plan Fragility, Hallucination (Cognitive Breakdowns); Integration Trap, Leaky Memory, Blind Autonomy, Black Box Behavior (Execution Gaps); Locked Box Interface, Agent Drift, Misaligned Metrics (Trust Erosion) — name the specific, recurring ways agents fail, rather than leaving "the agent might be wrong somehow" as a vague, uncatchable risk.
- Every one of the ten is addressed by a general eval principle already built earlier in this series — none required a new concept, which is itself evidence the framework holds together.
- One eval run tells you almost nothing; confidence grows only with the square root of the number of runs, so quadrupling runs is needed to halve your margin of error.
- Agents near a 50/50 judgment call are statistically the hardest to measure precisely — excellent and terrible agents both need far fewer runs to characterize confidently.
- Repeating a single case (consistency) and widening the dataset (generalization) answer different questions and both matter.
- The Rule of Three gives a fast way to size how many clean runs a near-100% compliance hard gate actually requires — often hundreds, not the handful used for everyday iteration.
- Because cost grows linearly with runs but precision only grows with their square root, eval budget is often better spent widening the dataset than over-repeating a single case.
4. Glossary
| Term | Meaning |
|---|---|
| Binomial proportion | A pass/fail rate estimated from a fixed number of independent trials — the statistical model behind eval pass rates. |
| Confidence interval | A range that's likely (e.g., 95% likely) to contain the true underlying rate, given a limited number of observed samples. |
| Margin of error | Half the width of a confidence interval — how far the true value could plausibly be from the observed one. |
| Standard error | The typical amount a sample estimate is expected to vary from the true value, shrinking with the square root of sample size. |
| Rule of Three | A quick approximation: with zero observed failures in n trials, the true failure rate is roughly bounded above by 3/n at 95% confidence. |
| Statistical power | The ability of a given sample size to reliably detect a real effect (here, a real regression) rather than mistake noise for signal. |
This chapter closes out the core Eval Basics toolkit: why agents are inconsistent, what an agent is and how to judge one without a single right answer, how to build a Golden Dataset, how evals differ from — and still borrow from — classic QA, how a healthy-looking dashboard can still be lying to you, and now, the ten concrete failure patterns this whole discipline exists to catch, plus the statistics needed to trust the numbers it produces. From here, the natural next step is applying all of it to a real coding-agent product. — continue to Chapter 7: Applying the Framework with LangSmith →