Capstone: design a grounded assistant

Design a support assistant over a small, approved collection of documents. Specify its users, supported questions, answer rubric, escalation cases, latency target, and data-retention rules. Sketch ingestion with source IDs and access labels, chunking and index refresh, retrieval, prompt construction with citations, output validation, and the serving interface. Identify which steps are deterministic and which depend on a model. Create a representative test set before tuning. Evaluate retrieval and answer quality separately, include unanswerable and adversarial questions, and trace each request without collecting unnecessary secrets. Describe least-privilege access, production monitoring, and rollback. A sound capstone explains tradeoffs and failure behavior, not just a happy-path demo. Begin the capstone with an explicit threat model: who can ask questions, which documents each user may access, and what happens when evidence is absent. Draw a data path from source ingestion through chunking and indexing to filtered retrieval, grounded prompting, answer validation, and serving. Annotate each boundary with its identity, permissions, and failure behavior. Include a citation contract and a human escalation path. The diagram is complete only when it also shows refresh and deletion, evaluation, trace governance, and the owner who responds to a failed quality or availability check. Implementation drill: trace one representative request through the stages in the diagram. Write down what enters each stage, what output or evidence it produces, and which identity is allowed to perform it. Then test an empty input, an unauthorized request, a dependency timeout, and an unexpected result. For each case, state whether the application should retry, reject, abstain, or ask a person for help. Keep the observed failure as a regression example so changes to prompts, models, data, or code can be checked against the same behavior. Practice task: Which evaluation should be performed separately in the capstone?