Advanced Python · Lesson 3 of 3
Advanced Python
Build reusable analysis code with robust functions, modules, iterators, testing, and maintainable project structure.
Lesson
1. Introduction
As an analysis grows, a notebook full of copied cells becomes difficult to verify. Advanced Python techniques help separate data preparation, calculation, and reporting into pieces that are easier to test, reuse, and review.
2. Core concepts
A function should have a clear input and output contract. Modules provide namespaces, exceptions make failure explicit, and iterators let code process data incrementally. Type hints document expected values for developers and tools, but they do not validate untrusted runtime input.
3. Topics covered in this module
- Functions, parameters, return values, comprehensions, and lambda expressions
- Classes, dataclasses, iterators, generators, and context managers
- Modules, packages, environments, logging, and exception handling
- Unit testing, type hints, APIs, and responsible performance profiling
4. Syntax and how to read it
from collections.abc import Iterable
def positive_total(values: Iterable[float]) -> float:
return sum(value for value in values if value > 0)Iterable describes values that can be traversed without requiring a specific container. The generator expression avoids building an extra intermediate list.
5. Practical workflow
Move one repeated notebook operation into a function, add a normal and boundary-case test, then import that function from a separate analysis script. Profile before optimizing and record any assumptions about input schemas.
6. Common mistakes
- Sharing mutable default values between function calls.
- Catching broad exceptions and returning a believable but incorrect result.
7. Best practices
- Use small pure functions for calculations and isolate I/O at the edges.
- Add tests for empty data, nulls, duplicates, and unexpected types.
8. Practice task
Create a function that returns a new list containing non-empty, trimmed strings.
9. One possible solution
def clean_labels(values: list[str]) -> list[str]:
return [value.strip() for value in values if value.strip()]10. Summary
Build reusable analysis code with robust functions, modules, iterators, testing, and maintainable project structure. Use small pure functions for calculations and isolate I/O at the edges. Add tests for empty data, nulls, duplicates, and unexpected types.
11. What comes next
The next module, AI Tools for Data Analysts - Make Coders Jealous, builds on this foundation and applies it to a new part of the analytics workflow.
Visual guide
A reliable Advanced Python workflow
-
Step 01
Frame
Define the question, input scope, and intended output.
-
Step 02
Work
Apply the method and validate assumptions as you go.
-
Step 03
Check
Compare results with known cases and communicate limitations.
Code examples
def normalize_region(value: str) -> str:
return value.strip().casefold()
print(normalize_region(' North '))north
def safe_ratio(numerator: float, denominator: float) -> float | None:
return None if denominator == 0 else numerator / denominator
assert safe_ratio(5, 0) is NoneThe assertion passes; a zero denominator is represented explicitly.
Key points
- Build reusable analysis code with robust functions, modules, iterators, testing, and maintainable project structure.
- Use small pure functions for calculations and isolate I/O at the edges.
- Add tests for empty data, nulls, duplicates, and unexpected types.
Check your understanding
Practice
Your learning progress