Evaluation datasets and quality rubrics

A useful evaluation set reflects real user questions, expected evidence, acceptable answers, and known failure modes. Include common cases, edge cases, unanswerable questions, long inputs, and safety-sensitive requests. Keep a held-out set for final comparisons so repeated prompt tuning does not overfit every known example. Remove or protect personally identifiable and confidential data. Use a rubric that separates dimensions such as correctness, relevance, completeness, citation support, and safety. Human review provides valuable ground truth for subjective tasks; automated judges can help scale review but also need calibration against human judgments. Version the dataset, rubric, prompt, model, and retrieval configuration together. A high-quality dataset records the question, relevant evidence, expected behavior, and the reason a response passes or fails. Include ordinary traffic, rare but important cases, unanswerable inputs, adversarial content, and examples from different user groups. Remove or synthesize sensitive values before storing fixtures. Score dimensions separately: relevance, correctness, groundedness, completeness, safety, and refusal quality. Human reviewers should calibrate any automated judge against a shared rubric. Keep a held-out set so repeated prompt adjustments do not merely memorize the cases used during development. Implementation drill: trace one representative request through the stages in the diagram. Write down what enters each stage, what output or evidence it produces, and which identity is allowed to perform it. Then test an empty input, an unauthorized request, a dependency timeout, and an unexpected result. For each case, state whether the application should retry, reject, abstain, or ask a person for help. Keep the observed failure as a regression example so changes to prompts, models, data, or code can be checked against the same behavior. Practice task: Why retain a held-out evaluation set?