Get In Touch
0% complete

Generative AI and LLM applications with Python · Lesson 2 of 3

Embeddings, semantic search, and RAG

Ground model answers in retrieved documents using embeddings and source-aware context.

Lesson

1. Concept and mental model

Retrieval-augmented generation (RAG) combines a search step with a language-model response. A document pipeline extracts text, splits it into meaningful chunks, attaches source metadata, and creates vector embeddings. At query time, the system embeds the question, retrieves candidate chunks by semantic similarity, optionally combines lexical search or reranking, and supplies the selected evidence to a model.

2. How it works

Chunk size and overlap affect retrieval quality: chunks that are too large dilute relevance, while chunks that are too small can lose context. Preserve document identifiers, access-control metadata, timestamps, and section titles so results can be cited and filtered. Retrieval must enforce the caller’s permissions before text is added to a prompt; otherwise search can leak documents the user cannot access.

3. Use it safely and effectively

A fluent answer is not proof that retrieval worked. Evaluate retrieval recall and ranking separately from answer correctness and groundedness. Ask the model to state when evidence is insufficient, but verify this behavior with tests rather than assuming the prompt guarantees it. Store source references and return citations that point to the retrieved material. Practice by tracing one question through chunk retrieval and checking whether every answer claim is supported by a cited passage.

4. Syntax reference

query = 'How do I rotate an API token?'
results = vector_store.search(query, top_k=5, filters={'tenant_id': current_tenant})
context = '\n'.join(item.text for item in results)
answer = model.generate(question=query, context=context)

5. Guided walkthrough

Preserve each chunk’s source and permission metadata while indexing, then apply the requesting user’s access filter during retrieval. Inspect the actual retrieved passages before generating an answer and return citations so a reader can verify the evidence.

6. Common mistakes to avoid

  • Adding retrieved documents to the prompt before applying access-control filters.
  • Treating a fluent answer or a high similarity score as proof that the cited evidence supports every claim.

Visual guide

A grounded RAG request

  1. Step 01 Index

    Chunk authorized documents and store embeddings.

  2. Step 02 Retrieve

    Search with permission filters and rank evidence.

  3. Step 03 Ground

    Generate an answer tied to selected sources.

Code examples

Worked example
PYTHON 
query = 'How do I rotate an API token?'
results = vector_store.search(query, top_k=5, filters={'tenant_id': current_tenant})
context = '\n'.join(item.text for item in results)
answer = model.generate(question=query, context=context)
Expected output
The model receives only the selected, access-filtered evidence and the question.
Filter retrieved records before building context
PYTHON 
documents = [
    {'text': 'Public guide', 'team': 'sales'},
    {'text': 'Private plan', 'team': 'finance'},
]
current_team = 'sales'
allowed = [doc for doc in documents if doc['team'] == current_team]
context = '\n'.join(doc['text'] for doc in allowed)
print(context)
Expected output
Public guide
Keep citations alongside answer text
PYTHON 
evidence = [{'text': 'Tokens expire after 24 hours.', 'source': 'policy.md'}]
answer = {'text': evidence[0]['text'], 'citations': [evidence[0]['source']]}
print(answer['citations'])
Expected output
['policy.md']

Key points

  • Ground model answers in retrieved documents using embeddings and source-aware context.
  • Prefer explicit, testable behavior over hidden side effects.
  • Validate assumptions at system boundaries and handle failure deliberately.

Check your understanding

Practice

When should document permissions be enforced in a RAG pipeline?

Your learning progress

Finish this lesson