Evaluation turns an impressive demonstration into a system whose strengths, weaknesses, and trade-offs can be measured. Harbour Resolution is a fictional case-resolution service that we will use to make the ideas concrete without suggesting that it is a real deployment. Before its team changes a prompt, model, or server, it needs to define what a successful resolution actually means. An operational definition is a plain, testable description of a quality, including what will be observed and how a result will be judged. For Harbour Resolution, relevant dimensions could include task accuracy, faithfulness to evidence, safety, security, response time, cost, consistency, escalation quality, and the amount of human effort required.

Human effort can be measured through reviewer minutes, handoffs, corrections, and escalations per completed case, with results separated by each relevant group of cases or users. These dimensions describe different kinds of success, so squeezing all of them into one average can conceal a serious weakness. Suppose the fictional service extracts a requested remedy, deadline, account identifier, and a flag showing whether the customer may need additional support from a case file. Field-level accuracy checks each extracted field separately, while weighting lets the team treat a harmful error in a deadline as more consequential than a harmless formatting difference. Faithfulness asks whether every material claim is supported by a source that actually applies to the case.

Safety can include attempts to take prohibited actions, delivery of unsafe advice, and failures to escalate cases that policy says require human attention. Security can include whether information and tools remain limited to the people and workflows allowed to use them. Consistency asks whether materially similar cases receive compatible treatment, while escalation quality asks whether referrals contain the right evidence and reach the appropriate reviewer. Latency means delay, but an interactive system should distinguish the time until the first useful output from the time until the complete, approved workflow finishes. Cost should include model processing, retrieval, ranking, tool calls, review, rework, and operational support rather than just the price of one model call.

A tempting dashboard might show that average accuracy rose, yet the same change could make outcomes worse for vulnerable-customer cases. Stratified reporting means breaking results into meaningful groups, such as case type, document quality, language, complexity, risk, or user cohort, so a broad average cannot hide a concentrated failure. A cohort is simply a group of cases or users that share a relevant characteristic. The evaluation dataset should resemble the work the system will actually receive, including common cases, difficult edge cases, ambiguous inputs, and low-quality documents. An edge case is an unusual but plausible situation that tests behavior near the limits of the normal workflow.

Negative cases are equally important because the correct result may be no finding, no action, or escalation rather than a confident answer. Sensitivity is the share of actual positive cases that a system detects. A hypothetical university support desk that tests only obvious policy violations might make detection look strong, but that test cannot establish representative sensitivity or reveal how many ordinary requests become false positives. Those incorrect alerts are false positives, and their review cost can make an apparently capable system operationally poor. Trusted labels record the expected result for each example, while provenance records where that expectation came from and why its source is authoritative.

A label should therefore say more than correct or incorrect when the judgment depends on a particular policy, permission, or escalation rule. Development examples support day-to-day changes, validation examples help select among candidate designs, and a final test set estimates performance after those choices have been made. A held-out set is a collection kept away from routine tuning so it can reveal whether improvements generalize beyond familiar examples. Tuning leakage occurs when information from validation or test examples influences development, making the reported result look more independent than it really is. The team should version the dataset, source material, prompt, model configuration, tools, and scoring logic so a result can be reproduced and compared fairly.

Versioning means assigning identifiable revisions, because the same model answer may be correct under one policy edition and wrong under another. When a production failure reveals a recurring pattern, a representative example belongs in a regression suite, which is a collection rerun after changes to detect the return of known problems. The suite should represent the broader failure pattern rather than accumulate every anecdote, because overfitting means tailoring a system so closely to familiar examples that it handles new ones poorly. System-level scores become much more useful when they are paired with component-level evaluation. Retrieval is the step that locates source material for the model, and retrieval recall measures how much of the relevant material the search process successfully finds.

Ranking quality asks whether the most useful results appear near the top, while freshness asks whether the selected sources are current enough for the task. Metadata is descriptive information such as document type, owner, language, or policy version, and metadata filtering uses those fields to exclude unsuitable material. Permission correctness asks whether retrieval returns only material the current actor and workflow are allowed to access. Authentication verifies an asserted identity, such as by checking credentials, while authorization determines what that identity is permitted to read or do. A prompt can tell a model not to reveal restricted records, but that instruction is not enforcement because the retrieval and tool layers must technically prevent unauthorized access.

Chunking divides a large document into retrievable sections, and poor chunking can separate a rule from its exception or return a fragment without enough context. Generation is the step that turns the supplied context into an answer, and it can be tested for correctness, groundedness, completeness, style, and appropriate handling of uncertainty. Groundedness means that claims stay connected to the provided evidence rather than being presented as unsupported facts. Evidence is information supplied by an applicable source, while an inference is a conclusion drawn from that information, so a good answer labels meaningful inference instead of disguising it as quoted policy. A generated citation can exist and still support the wrong claim, which is why citation presence and evidence applicability require separate checks.

Likewise, an answer can follow the required structure of JSON, which stands for JavaScript Object Notation, and still contain an incorrect deadline, fabricated explanation, or inapplicable policy. Syntax describes whether an output follows its required structure, whereas truth concerns whether its content is accurate and supported. Tool-use evaluation checks whether the system selected the right tool, supplied valid arguments, avoided unnecessary calls, recovered from errors, and refrained from prohibited calls. Validation checks a proposed output or action before it is used by a downstream component. Teams can evaluate validation with known valid and invalid cases, measuring how often it accepts invalid cases and how often it rejects valid ones.

An output validator may reject malformed arguments, but it cannot by itself prove that a well-formed action is appropriate for the case. End-to-end evaluation then measures the complete workflow through resolution quality, human acceptance, handling time, customer outcome, and operational failures. This separation helps the team locate whether a poor result began in retrieval, prompting, model fit, tool design, validation, or workflow policy. Replacing a model will not repair an out-of-date collection of searchable source material, a broken authorization filter, or a tool that silently maps the wrong field. Trace replay means rerunning a recorded sequence of inputs and component observations under controlled conditions so candidate components can be compared on the same case.

Replay is valuable for diagnosis, although it cannot reproduce every live condition, such as changing source data or the behavior of a real reviewer. The next choice is how each criterion will be graded. A code-based grader applies programmed checks and is a strong fit for exact structures, deterministic rules, executable tests, and known reference values. Deterministic means that the same input and rule produce the same result, such as checking that every citation identifier exists in the retrieved set. A code check can confirm that a cited identifier exists, but it usually cannot decide whether the associated policy truly governs a nuanced case. Human graders can make that domain judgment, yet they take time, cost more, and may reasonably disagree.

The fastest grader demonstrated to be reliable for a particular criterion should handle that criterion. Code checks are well suited to criteria that can be tested reliably with fixed rules. A language-model grader uses a model to assess another output against a rubric, meaning an explicit set of criteria describing what different judgments mean. Calibration means comparing that grader's judgments with trusted expert judgments to understand how reliably it applies those criteria. After that checking, a model grader may help screen larger volumes and route uncertain or high-risk judgments to qualified human graders. The same model configuration may grade its own outputs during early iteration, but independent grading and periodic human calibration reduce the chance of shared blind spots.

The grader should return a constrained output that downstream software can validate against the required format. Constrained output improves consistency of form, but it does not enforce permissions or guarantee that the judgment is true. Calibration compares a grader with trusted expert decisions so the team can discover where its scores are too harsh, too lenient, or systematically skewed. Inter-rater agreement measures how often different graders reach compatible judgments under the same rubric. Systematic disagreement should be investigated rather than averaged away, because it may reveal an ambiguous rule, missing context, or a grader that treats one cohort differently. Experiments are clearest when one major variable changes at a time, since simultaneous changes make the cause of a result difficult to identify.

A paired comparison runs both candidates on the same examples, reducing noise caused by one candidate receiving an easier collection of cases. Some model behavior is stochastic, meaning repeated runs can vary even when the input appears unchanged. Repeated runs show that variation, and a confidence interval expresses a range of values compatible with the observed sample rather than pretending that one measured score is exact. A/B testing normally uses random assignment to put eligible live cases or users into groups receiving alternative versions, then compares their outcomes. That controlled comparison can reveal user or business effects that a fixed pre-release dataset misses, while random assignment helps reduce differences between the groups unrelated to the change being tested.

Because live experiments can affect real outcomes, exposure should be staged and paired with rollback criteria that define when the change will be withdrawn. A canary release exposes a new version to a small, controlled cohort before considering broader use. Shadow mode lets a new version process live inputs without controlling the actual outcome, which is especially useful when a mistake would be consequential. Shadow results still require care because the system may not experience every interaction or timing constraint it would face while in control. Once its actions affect outcomes, those actions may also change what users do next and the information arriving later, which a shadow run cannot fully reproduce.

Human override and correction rates can reveal friction, but they are not pure measures of correctness because reviewers may accept polished errors or resist unfamiliar correct answers. Qualitative review means closely examining examples in words rather than only scores, and it often exposes failure modes the original metrics did not anticipate. When an answer is wrong, diagnosis should begin by confirming the expected answer and the authority of the source used to establish it. At the fictional Harbour Resolution service, an evaluator might discover that the supposed trusted expected answer came from an older policy, so changing the model would merely tune it toward the wrong target.

The next check is input quality and task routing, where routing means sending a case to the component or configuration suited to its needs. A hypothetical equipment-maintenance assistant might give a weak answer because a scanned warning label was unreadable or the request was sent to a text-only path, not because its reasoning failed. Retrieved evidence should then be inspected for relevance, freshness, completeness, chunk boundaries, metadata filters, and authorization. The evaluator should also check tool calls and observations, including which tool ran, what arguments it received, what it returned, and how errors were represented. Only after those upstream steps should the assembled prompt and context be examined for ambiguous criteria, conflicting instructions, weak examples, or a vague output contract, which is the specification of the required content, fields, format, and allowed values for a model response.

A prompt is guidance presented to the model, while enforcement is a technical or procedural control that remains effective even if the model misunderstands or ignores that guidance. Next inspect the model output and the checks applied to it, how the result is displayed, who authorized any action, and whether the workflow changed a record or another system. Rendering is the presentation step, and it can introduce a defect by hiding uncertainty, mismatching citations, or displaying the wrong field even when the internal output was sound. A person's confirmation that an action looks correct is not the same as authorized approval to perform it. In a hypothetical payroll workflow, an employee might confirm a corrected bank account number, but only a designated approver may be authorized to release the payment change.

The better design verifies identity, checks authorization, shows the consequential details, records the authorized approval, and limits the action, although those controls add time and can still fail if roles or source records are wrong. Following the data in this order prevents the convenient assumption that every bad outcome began with the model. Model mismatch is a supported diagnosis when sound inputs, retrieval, tools, and prompts still demand capabilities beyond the selected configuration. Those capabilities may include stronger reasoning, greater context capacity, meaning the amount of supplied information the configuration can accept and use effectively, or support for the required modality, meaning an input kind such as text, images, or audio.

A mismatch can also arise when the configuration cannot use tools with the reliability the workflow requires. That conclusion should come from comparative evaluation on representative cases rather than intuition about which model seems stronger. Prompt failure is more likely when the model has adequate evidence and capability but receives ambiguous criteria, contradictory directions, poor examples, or an unclear output contract. Retrieval failure is more likely when the supplied evidence is missing, irrelevant, stale, unauthorized, or divided into misleading chunks. Once quality is measurable and failures are localized, cost and latency can be optimized without treating them as isolated goals. Removing irrelevant context and shortening verbose tool results can reduce work, but the evaluation must show that necessary evidence was not lost.

Prompt caching can reuse earlier processing for an unchanged beginning of a later request, according to the platform's cache rules. Stable material goes before changing content where the platform supports that arrangement, but changing the instructions or source material means the old computation may no longer be reusable. Application-level caching is different: the application stores retrieved data or a computed result to reuse later. It needs rules for stopping reuse when the source, policy or freshness changes, and for limiting access to users and workflows that are still authorized. Stored material also needs privacy protections and retention limits, meaning a defined period for keeping it and removal when that period ends.

Batch processing groups asynchronous work for efficient handling when users do not need an immediate answer. The smallest model configuration that passes the workload's defined thresholds is often a sensible candidate, provided the calculation includes added review and rework. Routing can reserve stronger configurations for genuinely complex cases, although a misclassified difficult case may receive an inadequate path. In a hypothetical multilingual benefits desk, routing based only on message length could send a short but legally nuanced request to a weaker configuration, so complexity signals and cohort results would need evaluation. Tools can run in parallel only when their inputs are independent and concurrent execution is safe and authorized.

For example, two permitted read-only lookups may be independent, while changes to the same account may need an explicit order to avoid conflicting updates. Parallel work can reduce elapsed time, but it also increases simultaneous load and can leave some calls successful while others fail, so the workflow must track what actually completed before deciding what to retry. Streaming sends an interactive response progressively, improving perceived responsiveness without necessarily reducing the time until the complete approved workflow is ready. Sensible output limits and concise machine-readable intermediate results can reduce delay and processing, but limits that are too tight may truncate evidence or explanations. Retrieved information or computed results can be reused only while their freshness and authorization remain valid for the current request.

These trade-offs explain why the useful objective is cost per successful outcome rather than cost per model call. A cheaper call that causes more escalation, correction, or customer harm may be more expensive at the workflow level. Evaluation continues after release through production monitoring, which tracks behavior over time rather than assuming that a test result remains permanent. Useful signals include user feedback, override rates, source freshness, tool errors, prohibited-action attempts, and spend. Token use measures the units of text or other input and output that a model processes, making it relevant to both cost and workload monitoring. A quality proxy is an observable signal related to quality, such as correction frequency, but it is not a substitute for direct outcome measurement.

Retrieval drift is a change in the sources or search behavior reaching the model, and it can degrade answers even when the prompt and model configuration stay fixed. A latency percentile reports a threshold met by a stated share of requests, helping reveal slow-tail experiences that an average can conceal. Monitoring should compare cohorts and versions because a stable overall average may hide a failure limited to one product, language, risk group, or policy revision. For instance, the fictional Harbour Resolution dashboard could remain flat while cases using a newly revised policy increasingly retrieve its superseded predecessor. Alert thresholds define when investigation begins, but the operating design must also name who investigates, who may disable the feature, and what fallback users receive.

A fallback might route affected cases to human handling, though that option has capacity and delay limits of its own. Every proposed change should therefore name the metric it is expected to improve, the cohort it affects, and the regression it could introduce. The central habit is simple: define success, preserve representative evidence, isolate the failing component, test the proposed change, and optimize the complete successful outcome rather than the most flattering number.