Prompt engineering and structured output

Episode Five builds from project context into prompts and outputs that can be tested. By the end, you should be able to define success criteria, use examples without overfitting, distinguish requested J S O N from guaranteed structure, and place semantic validation around structured output. Now move from development context to task-specific prompting. A good prompt defines the task, relevant context, constraints, decision criteria, and output contract. It does not merely tell Claude to be accurate, careful, and professional. Why are explicit criteria important? They turn a vague preference into observable behaviour that can be evaluated and improved. This is especially important in classification and review tasks where undefined judgement creates false positives.

Text description

• Weak: identify security issues: Leaves the judgement boundary undefined, increasing ambiguity and false positives.
• Strong: require claim elements and evidence: Requires an affected system, alleged weakness, plausible impact, and supporting text; excludes generic dissatisfaction, marketing, and unsupported speculation.

Suppose Harbour Resolution wants to flag correspondence containing a credible vulnerability claim. A weak prompt says identify security issues. A stronger prompt defines the required elements, exclusions, severity rules, uncertainty handling, and evidence fields. It might require an affected system, alleged weakness, plausible impact, and supporting text before returning a positive classification. It can state that generic dissatisfaction, marketing language, and unsupported speculation are not sufficient. These criteria reduce ambiguity without requiring a larger model. Few-shot examples make the boundary more concrete. Choose examples that cover ordinary positives, ordinary negatives, and difficult near misses. Why can too many examples become harmful? They consume context, may overfit the apparent pattern, and can bury the actual instructions.

Text description

• Prompted JSON: Can improve consistency but does not guarantee valid structure.
• Schema-enforced structure: Structured Outputs or a forced schema can enforce properties such as numeric fields and formatted dates.
• Semantic validation: The application must still verify that each value came from the correct source and represents the intended fact.

Therefore, use a small representative set and evaluate whether each example changes behaviour. For extraction, examples should demonstrate missing values, conflicting evidence, multiple records, and formatting edge cases. Now distinguish requested J S O N from guaranteed schema conformance. Telling Claude to output J S O N can improve consistency, but plain prompting alone does not guarantee valid structure. Structured Outputs or a forced tool with a J S O N schema provides a stronger syntactic contract on supported models and surfaces. The application must still perform semantic validation. A schema can ensure amount is numeric, but it cannot prove the amount came from the right invoice.

A schema can ensure a date is formatted, but not that it is the effective date rather than the upload date. How should missing values be represented? Define the policy explicitly, using nullable fields, omitted optional fields, or a typed unknown state consistently. Inventing a value to satisfy a required field is a serious extraction failure. Use null only when the schema and downstream consumer understand what null means. Distinguish not present, illegible, contradictory, and not applicable when those states lead to different actions. Harbour Resolution extracts claimant name, complaint date, requested remedy, and transaction identifiers. Each field should carry source references or spans where auditability matters.

Text description

• Validate output: Check both structural rules and whether extracted values are supported by the cited source.
• Return specific errors: Describe a correctable failure, such as the wrong field type or a citation that does not support the amount.
• Request bounded repair: Ask Claude to fix the identified errors without restarting vaguely or retrying forever.
• Escalate unresolved ambiguity: If the source is genuinely ambiguous, return needs human review.

The model can also return confidence categories, but confidence must be calibrated against observed error rates before automation relies on it. What is a validation-and-retry loop? Validate the output, return specific correctable errors to Claude, and request a bounded repair rather than restarting vaguely. Do not retry forever. A structural error might say transaction identifiers must be an array of strings. A semantic error might say the cited page does not contain the extracted amount. If the source is genuinely ambiguous, the correct outcome may be needs human review rather than another attempt to force certainty. Prompt chaining can separate complex work into inspect, extract, verify, and render stages.

This often improves observability and lets each stage use the most appropriate schema and model. It also creates more calls and handoff points, so evaluate the trade-off rather than assuming more stages are always better. The Professional blueprint names zero-shot, few-shot, and chain-of-thought techniques. In production, give the model enough room to analyse difficult tasks, but constrain the final answer and do not treat exposed reasoning as an audit record. The evidence, tool traces, tests, and decisions are the auditable artefacts. A self-critique can catch some defects, but it remains correlated with the original model path. For higher assurance, use independent passes with distinct roles and preserve their disagreements.

Text description

• Extract fields and citations: The first instance proposes values and identifies their supporting source locations.
• Independently verify each field: A second instance receives the source and proposed extraction, then challenges the evidence and scope.
• Application acceptance decision: Accept verified fields; preserve disagreements or route higher-impact uncertainty for review.

What is a useful dual-pass extraction pattern? One instance extracts fields and citations, then a second independently checks each field against the source before the application accepts it. The verifier should receive the source and proposed extraction without being told that the first answer is presumed correct. For code review, one pass can generate candidate findings and another can challenge their evidence and scope. Multi-pass review costs more, so apply it where error impact justifies it. Batch processing is another exam topic. The Message Batches A P I suits large asynchronous workloads that do not require immediate responses. Current platform documentation states that batch processing reduces message costs by fifty percent.

Use a stable custom identifier to reconcile each result with its source record. Do not choose batch for an interactive support conversation with a strict response-time objective. What operational controls surround a batch? Track submission, processing state, individual result status, retries, expiry, reconciliation, and downstream idempotency. Lower unit cost does not remove workflow responsibility. Here is the prompting hierarchy to remember. First define success and failure. Then provide necessary context and discriminating examples. Then constrain the output through the strongest supported mechanism. Then validate meaning and evidence outside the model. Then evaluate on a representative dataset and iterate from observed failures. Prompt engineering begins after success criteria exist, not before.

Episode Five is complete when you can move from a vague task to measurable criteria, constrained structure, semantic validation, and representative evaluation. For source navigation, use the prompt-engineering overview, prompting best practices, Structured Outputs, output-consistency guidance, and batch-processing documentation. In the next episode we will connect long-context handling, uncertainty, escalation, and provenance into the complete Foundations reliability layer.