Get In Touch
0% complete

Generative AI and LLM applications with Python · Lesson 3 of 3

Evaluate, secure, and operate AI features

Test AI quality and failure modes before exposing generated output to users.

Lesson

1. Concept and mental model

AI features need evaluation beyond whether a request returns HTTP 200. Create a representative dataset of expected questions and useful evidence, then measure task success, answer relevance, groundedness, citation validity, latency, and cost. Human review remains important for high-impact decisions. Compare model or prompt changes against a fixed baseline so an apparent improvement in one case does not hide regressions elsewhere.

2. How it works

Treat user prompts and retrieved documents as untrusted content. They may contain attempts to override instructions or request secret data. Keep system instructions separate from user data, restrict tools to the minimum permissions, validate tool arguments, and require explicit approval for consequential actions. A prompt is not an authorization boundary. Apply normal identity and access checks in application code.

3. Use it safely and effectively

Define timeouts, bounded retries, quota handling, and a user-friendly fallback. Monitor failures, latency, token use, and quality signals while minimizing retained sensitive data. Provide a way to report incorrect output and investigate it with a safe trace identifier. Before release, test malformed model output, prompt injection, unauthorized retrieval, provider downtime, and unexpected costs. Practice by writing an evaluation case that checks whether a cited answer is actually supported by its source.

4. Syntax reference

evaluation_case = {
    'question': 'What is the reset window?',
    'source': 'Tokens expire after 24 hours.',
    'expected_fact': '24 hours',
}
assert evaluation_case['expected_fact'] in evaluation_case['source']

5. Guided walkthrough

Create fixed evaluation cases with expected evidence and acceptance criteria, then compare changes against the same set. Test unsafe and unavailable conditions deliberately, enforce permissions in trusted code, and monitor quality and cost without retaining more private data than necessary.

6. Common mistakes to avoid

  • Using a successful HTTP response as the only measure of AI feature quality.
  • Relying on prompt instructions instead of enforcing authorization and tool permissions in application code.

Visual guide

Operate an AI feature responsibly

  1. Step 01 Evaluate

    Measure quality on representative cases.

  2. Step 02 Protect

    Enforce authorization and validate tool inputs.

  3. Step 03 Monitor

    Track safe quality, latency, and cost signals.

Code examples

Worked example
PYTHON 
evaluation_case = {
    'question': 'What is the reset window?',
    'source': 'Tokens expire after 24 hours.',
    'expected_fact': '24 hours',
}
assert evaluation_case['expected_fact'] in evaluation_case['source']
Expected output
A small deterministic check confirms the expected fact exists in the supplied source.
Reject an unsupported answer claim
PYTHON 
source = 'Tokens expire after 24 hours.'
claim = 'Tokens expire after 24 hours.'
assert claim in source
print('grounded')
Expected output
grounded
Bound retries and fail clearly
PYTHON 
attempts = 0
max_attempts = 3
while attempts < max_attempts:
    attempts += 1
    if attempts == max_attempts:
        print('Stop and report provider failure')
Expected output
Stop and report provider failure

Key points

  • Test AI quality and failure modes before exposing generated output to users.
  • Prefer explicit, testable behavior over hidden side effects.
  • Validate assumptions at system boundaries and handle failure deliberately.

Check your understanding

Practice

What is the role of authorization checks in an AI application?

Your learning progress

Finish this lesson