Data Analysis using Pandas · Lesson 1 of 1
Data Analysis using Pandas
Inspect tabular data, select and filter records, clean values, reshape, group, join, and export results.
Lesson
1. Introduction
pandas offers labeled Series and DataFrame structures for everyday table analysis. Labels make many operations readable, but they also mean indexes and column names are part of the data contract and should be inspected.
2. Core concepts
A DataFrame has labeled rows and columns. Selection, filtering, grouping, and joins produce new views or tables; missing-value operations and type conversions need explicit business rules. A merge can multiply rows if the join keys are duplicated, so validate expected cardinality.
3. Topics covered in this module
- Series, DataFrame creation, reading CSV, schema inspection, and dtypes
- Selection with loc/iloc, boolean filters, sorting, and missing-value handling
- String/date parsing, transformations, melt, pivot, and categorical values
- Groupby aggregation, merge/join, duplicate detection, and CSV output
4. Syntax and how to read it
import pandas as pd
orders = pd.read_csv('orders.csv', parse_dates=['order_date'])
summary = (orders.groupby('region', as_index=False)
.agg(revenue=('amount', 'sum'), orders=('order_id', 'nunique')))The grouped result has one row per region. Counting unique order IDs avoids counting multiple line items as separate orders.
5. Practical workflow
Load with explicit date and type handling, inspect shape and missingness, define cleaning rules, and preserve a raw copy. After grouping or joining, verify row counts, uniqueness, and reconciliation totals.
6. Common mistakes
- Replacing missing values with zero without checking what missing means.
- Joining on non-unique dimension keys and inflating downstream totals.
7. Best practices
- Use explicit column selection and parse dates/types at ingestion.
- Keep a cleaning log and validate merge cardinality and output grain.
8. Practice task
A sales table has one row per item, but the report needs order count by region. What should you count?
9. One possible solution
Group by region and count distinct order_id values (nunique), because counting table rows would count items rather than orders.10. Summary
Inspect tabular data, select and filter records, clean values, reshape, group, join, and export results. Use explicit column selection and parse dates/types at ingestion. Keep a cleaning log and validate merge cardinality and output grain.
11. What comes next
The next module, Data Visualization using Matplotlib and Seaborn, builds on this foundation and applies it to a new part of the analytics workflow.
Visual guide
A reliable Data Analysis using Pandas workflow
-
Step 01
Frame
Define the question, input scope, and intended output.
-
Step 02
Work
Apply the method and validate assumptions as you go.
-
Step 03
Check
Compare results with known cases and communicate limitations.
Code examples
import pandas as pd
orders = pd.DataFrame({'order_id': [1, 2], 'amount': [150, 80]})
recent = orders.loc[orders['amount'] > 100, ['order_id', 'amount']]
print(recent.to_dict('records'))[{'order_id': 1, 'amount': 150}]import pandas as pd
orders = pd.DataFrame({'order_id': [1, 2], 'customer_id': [7, 7]})
customers = pd.DataFrame({'customer_id': [7], 'name': ['Mina']})
joined = orders.merge(customers, on='customer_id', validate='many_to_one')
print(len(joined) == len(orders))True when every order matches at most one customer and no order rows are lost.
Key points
- Inspect tabular data, select and filter records, clean values, reshape, group, join, and export results.
- Use explicit column selection and parse dates/types at ingestion.
- Keep a cleaning log and validate merge cardinality and output grain.
Check your understanding
Practice
Your learning progress