Writing data code that others can trust

Why trust matters

A wrong number in a report can cost money or hurt people: a bank may approve bad loans, a hospital may order too few medicines. Data code also tends to fail silently. It does not crash; it just gives a slightly wrong answer, and nobody notices. So data people build habits that make mistakes visible early and make results repeatable. Reproducible means that someone else, or you next month, can run the same code on the same data and get the same answer.

Small functions with clear names

Put each step in its own function that does one job, such as clean_price, total_per_city or safe_mean. A function can be tested on its own and reused on next month's file. A clear name says what comes out: average_bill_rupees is better than x2 or temp. Add a one-line docstring that says what goes in, what comes out, and what happens with odd input.

Edge cases

An edge case is an unusual input at the boundary: an empty list, a single value, a blank cell, a negative number, a tie. Most bugs live there. Before writing a function, list the edge cases and decide what should happen. For example, the average of an empty list has no sensible number, so returning None (and saying so in the docstring) is safer than crashing or quietly returning 0.

def safe_mean(values):
    """Average of the numbers in values, ignoring None. Returns None if there are no numbers."""
    numbers = [v for v in values if v is not None]
    if not numbers:
        return None
    return sum(numbers) / len(numbers)

print(safe_mean([10, 20, None, 30]))   # 20.0
print(safe_mean([]))                   # None

Asserts as tiny tests

assert checks that something is true. If it is false, the program stops with an AssertionError and your message. A few assert lines under a function act as tests: they prove it works on normal cases and edge cases, and they fail loudly if a later change breaks it. This is exactly how the tasks in this course are checked.

assert safe_mean([10, 20, 30]) == 20, "normal case"
assert safe_mean([]) is None, "empty list gives None"
assert safe_mean([None, None]) is None, "only missing values gives None"
assert safe_mean([5]) == 5, "one value is its own mean"
print("all tests passed")   # all tests passed

Notebooks and scripts

A notebook (Jupyter, or Google Colab in the browser) is great for exploring: you run small cells and see tables and charts at once. Its danger is hidden state: cells can be run out of order, so the notebook on screen may not match what a fresh run gives. A script is a plain .py file that runs from top to bottom every time, which suits jobs that run daily. A good habit: explore in a notebook, then move the finished steps into functions in a .py file, and before sharing a notebook use "Restart and run all" to prove it works from the top.

Virtual environments and pip

Libraries such as pandas are installed with pip, Python's package installer. Different projects may need different versions, so each project gets its own virtual environment: a private folder of installed packages. requirements.txt records the exact versions so others can rebuild the same setup.

python -m venv .venv
# Windows:      .venv\Scripts\activate
# macOS/Linux:  source .venv/bin/activate
pip install pandas
pip freeze > requirements.txt
# Later, on another computer:
pip install -r requirements.txt

Reproducibility habits

• Never edit the raw data file. Keep it read-only and write cleaned data to a new file, so you can always start again.
• Do every cleaning step in code, not by hand in Excel, so the steps are recorded and can be re-run.
• Fix the random seed whenever randomness is used (sampling, shuffling, splitting data for machine learning), so the result is the same every run.
• Write down where the data came from and on what date, because sources change.

import random

random.seed(42)
first = random.sample(range(1, 101), 3)
random.seed(42)
second = random.sample(range(1, 101), 3)
print(first == second)   # True

Common mistakes

• Testing only the happy case. Fix: always add asserts for empty input, one item, blanks and ties.
• Overwriting the raw file with cleaned data. Fix: save to a new file such as sales_clean.csv.
• Sharing a notebook that only works when cells run in a special order. Fix: Restart and run all before sharing.
• Installing everything into the main Python. Fix: one virtual environment per project, plus requirements.txt.
• Returning 0 when a value is missing. Fix: return None and document it, so missing is never mistaken for zero.