Project 2 - Coders of Bangalore · Lesson 1 of 1

Project 2 - Coders of Bangalore

Collect permitted public records, parse structured text, and analyze engagement with data-quality safeguards.

Lesson

1. Introduction

Collection is part of analysis, not a license to gather anything accessible on a web page. A sound project begins with a permitted source and a minimal set of fields, then checks whether observations are complete and comparable.

2. Core concepts

Parsing turns source text into structured fields; validation checks the expected types, ranges, and relationships. Engagement counts are measures with different scales and possible biases. A comparison should state when and how records were collected because rankings can change over time.

3. Topics covered in this module

Collection scope, terms, rate limits, provenance, and timestamping
HTML or JSON parsing and field normalization
Post, follower, category, and engagement summaries
Ranking rules, missingness, sampling bias, and responsible interpretation

4. Syntax and how to read it

record = {'creator': 'sample', 'posts': 12, 'followers': 800}
if record['posts'] < 0 or record['followers'] < 0:
    raise ValueError('counts cannot be negative')

A small validation rule catches impossible counts; real collection pipelines should also validate field presence and source timestamps.

5. Practical workflow

Use an approved data source or a provided sample, preserve provenance, parse into a stable schema, and reject or quarantine malformed rows with a reason. Calculate summary metrics only after checking duplicates and collection windows.

6. Common mistakes

Ignoring source terms, access controls, privacy, or rate limits during collection.
Comparing engagement counts without considering audience size, time period, and sampling bias.

7. Best practices

Prefer official APIs or permitted datasets and record collection time.
Keep raw source snapshots only when authorized and document every transformation.

8. Practice task

Two creators have 100 interactions each but audiences of 500 and 10,000. Which comparison is more informative than the raw count?

9. One possible solution

Compare interactions per audience member (with clear denominator and time window): 20% versus 1%, while noting that the sample and interaction definition still matter.

10. Summary

Collect permitted public records, parse structured text, and analyze engagement with data-quality safeguards. Prefer official APIs or permitted datasets and record collection time. Keep raw source snapshots only when authorized and document every transformation.

11. What comes next

The next module, Data Analysis using Numpy, builds on this foundation and applies it to a new part of the analytics workflow.

Visual guide

A reliable Project 2 - Coders of Bangalore workflow

Step 01 Frame
Define the question, input scope, and intended output.
Step 02 Work
Apply the method and validate assumptions as you go.
Step 03 Check
Compare results with known cases and communicate limitations.

Code examples

Rank records by followers

PYTHON

creators = [{'name': 'A', 'followers': 90}, {'name': 'B', 'followers': 120}]
ranked = sorted(creators, key=lambda row: row['followers'], reverse=True)
print(ranked[0]['name'])

Expected output

Calculate engagement per follower