Get In Touch
0% complete

Data, models, and prompt engineering · Lesson 3 of 3

Structured output and validation

Constrain machine-readable responses and validate them before downstream use.

Lesson

Applications often need fields rather than free-form prose. Ask for a defined schema and use supported structured-output features when available, but still parse and validate the result. Check required keys, value types, allowed categories, lengths, and cross-field rules. Handle malformed, incomplete, or refused responses as normal failure cases rather than assuming the model always follows instructions. Validation should be deterministic and independent of the model. A syntactically valid JSON object can still contain invented identifiers or unsafe values. Use allow-lists for actions and route consequential decisions through application authorization. Log validation failures with enough context to diagnose patterns while excluding secrets and unnecessary personal data. Define the response contract before writing the prompt: required fields, types, allowed values, maximum lengths, and what represents “unknown.” If an endpoint supports constrained structured output, use it to reduce formatting errors, but keep deterministic parsing and validation in the application. Validate relationships between fields too; for example, a cited document ID must belong to the retrieved set. Handle refusal, truncation, invalid JSON, and unexpected enum values explicitly. Never execute a string or grant permission because the model returned a syntactically valid action object. Implementation drill: trace one representative request through the stages in the diagram. Write down what enters each stage, what output or evidence it produces, and which identity is allowed to perform it. Then test an empty input, an unauthorized request, a dependency timeout, and an unexpected result. For each case, state whether the application should retry, reject, abstain, or ask a person for help. Keep the observed failure as a regression example so changes to prompts, models, data, or code can be checked against the same behavior. Practice task: Does valid JSON guarantee that a model response is safe and correct?

Visual guide

Validate a model response before use

  1. Step 01 Generate

    Request the defined response schema.

  2. Step 02 Parse

    Reject malformed or incomplete output.

  3. Step 03 Validate

    Check types, values, and allowed actions.

  4. Step 04 Authorize

    Apply trusted permission checks.

Code examples

Validate an output contract
PYTHON 
ALLOWED = {'billing', 'technical', 'other'}

def validate_result(result):
    if not isinstance(result, dict):
        raise ValueError('Expected an object')
    if result.get('category') not in ALLOWED:
        raise ValueError('Unknown category')
    if not isinstance(result.get('needs_human_review'), bool):
        raise ValueError('Expected a boolean review flag')
    return result
Expected output
Invalid shapes and unknown category values fail before downstream use.

Key points

  • Use a schema where supported, then validate the parsed response.
  • Treat refusals, malformed output, and missing fields as explicit cases.
  • Authorize actions in application code, never from model output alone.

Check your understanding

Practice

Does valid JSON guarantee that a model response is safe and correct?

Your learning progress

Finish this lesson