Chapter 2 — Essential Data Libraries
Part of Python · page 2 of 4
| Level | 0+: you've worked through Chapter 1 |
| Time | 6–8 hours, best spread over three sessions |
| You need | Your python-data-analysis project folder with its .venv from Chapter 1, and an internet connection for installing libraries |
| Project milestone | The data toolkit installed inside your project, recorded in requirements.txt, and a first Harbour Pharmacy notebook that analyses a month of sales and saves two charts |
Where we are
In Chapter 1 you set up an isolated project, then wrote a sales summary using only core Python: a list of dictionaries, a few loops that tallied values into dictionaries, and a lot of careful f-string formatting. It worked, and it took a few dozen lines of careful bookkeeping.
This chapter introduces the libraries that professional analysts use every day. By the end you'll produce that same branch summary in a single line, work with a whole month of data at once, and draw charts. More importantly, you'll understand what each library is for and how they fit together, so later chapters can go deep on each one without you feeling lost.
The four libraries at the heart of this chapter, and how they stack:
┌──────────────────────────────┐
│ Seaborn: statistical charts │
├──────────────────────────────┤
│ Matplotlib: all charts │
├──────────────────────────────┤
│ pandas: tables (DataFrames) │
├──────────────────────────────┤
│ NumPy: fast arrays of numbers
└──────────────────────────────┘
Python
Each layer is built on the ones beneath it. pandas stores its columns using NumPy; Seaborn draws its charts using Matplotlib. You'll spend most of your working life in the pandas layer, and knowing what sits underneath explains a great deal of pandas' behaviour.
2.1 Libraries, packages and modules: borrowing other people's work
The idea
Nobody writes everything from scratch. A library is a collection of ready-made code, written and tested by other people, that you can pull into your own programs. Python's vocabulary has three overlapping words:
- A module is a single
.pyfile of Python code that you canimport. - A package is a folder of related modules bundled together.
- Library is the everyday, informal word for a package (or group of packages) that gives you a toolkit, such as "the pandas library".
Libraries come from two places:
- The standard library ships with Python itself, so it's already on your computer. Useful members include
math,statistics,csv,datetime,pathlibandrandom. - Third-party libraries are published by the community on the Python Package Index (PyPI), at pypi.org, which hosts hundreds of thousands of projects. The installer
pipdownloads them from PyPI.
Worked example 2.1: using the standard library
The statistics module can describe the week of sales from Chapter 1 without any installing:
import statistics
daily_sales = [412.50, 389.90, 455.10, 501.75, 610.20, 720.40, 298.00]
print(statistics.mean(daily_sales))
print(statistics.median(daily_sales))
print(round(statistics.stdev(daily_sales), 2))
Output:
483.97857142857146
455.1
142.32
import statisticsmakes the module available. You then reach its tools with a dot:statistics.mean(...), just as you reached a string's methods with a dot in Chapter 1.- The median is the middle value once the list is sorted. It's often more representative than the mean (average) when a few values are unusually large or small.
- The standard deviation (
stdev) measures how spread out the values are: roughly, how far a typical day sits from the average. Chapter 10 covers it properly.
Three ways to import
import statistics # use as statistics.mean(...)
import statistics as st # a nickname: st.mean(...)
from statistics import mean # bring one tool in directly: mean(...)
The data world has nicknames everyone uses. You'll see them in every tutorial, book and workplace, so use them too; they make your code instantly readable to other analysts:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
Under the hood: how import finds a library
When you write import pandas, Python searches a list of folders for something called pandas, in order, and uses the first match. Sound familiar? It's the same "first match wins" idea as PATH from Chapter 1, applied to libraries. The list lives in a variable called sys.path.
While your virtual environment is active, that list includes your project's private shelf, .venv/…/site-packages, and leaves out the global one. That's how isolation works for libraries. It also explains the single most common beginner error in this chapter:
ModuleNotFoundError: No module named 'pandas'
Nine times out of ten this means one of two things: the library isn't installed in this environment, or the environment isn't active (or your editor is pointing at a different Python). Check for (.venv) in your terminal prompt, or the selected interpreter in VS Code, before anything else.
2.2 Installing the data toolkit into your project
Step 1: activate your environment
Open a terminal, move into your project folder and activate .venv (section 1.2.4). Confirm you see (.venv) in the prompt. Everything in this section assumes the environment is active; otherwise you'd be installing into global Python, which is exactly what we set out to avoid.
Step 2: update pip, then install
python -m pip install --upgrade pip
python -m pip install numpy pandas pyarrow matplotlib seaborn jupyterlab openpyxl
The first line updates pip itself; newer versions handle newer package formats and give clearer error messages. The second line installs seven libraries at once. It may take a minute or two, and you'll see a long list of other packages being installed too; the next section explains why.
What each one is for:
| Library | Job | First used |
|---|---|---|
numpy |
Fast arrays of numbers: the engine underneath pandas | This chapter |
pandas |
Tables (DataFrames): the analyst's main workbench | This chapter |
pyarrow |
A high-speed data engine that pandas uses for text columns and file formats. The pandas team strongly recommends installing it alongside pandas 3 | Behind the scenes |
matplotlib |
The foundational charting library | This chapter |
seaborn |
Statistical charts with good defaults, built on Matplotlib | This chapter |
jupyterlab |
The notebook application (section 2.3) | This chapter |
openpyxl |
Lets pandas read and write Excel .xlsx files |
Chapter 3 |
Why python -m pip instead of plain pip?
python -m pip means "run the pip that belongs to this python". Because your active environment puts .venv first on PATH, python is the environment's interpreter, so its pip installs onto the environment's shelf. Plain pip usually lands in the same place, but on machines with several Pythons installed it can quietly pick a different one. python -m pip removes the guesswork, and the official Python packaging guide uses it for that reason.
A safety net: make pip refuse to install globally
You can tell pip to refuse to install anything unless a virtual environment is active. Run this once (from any terminal):
python -m pip config set global.require-virtualenv true
This writes a setting into your personal pip configuration file (the command prints the file's location as it writes), so it applies to every Python on your account. From then on, an install attempted with no environment active stops immediately with:
ERROR: Could not find an activated virtualenv (required).
That one line guarantees your libraries only ever land inside a project's .venv, which is exactly the isolation Chapter 1 set up. (In the setting's name, global refers to pip's settings section, and has nothing to do with global Python.)
Under the hood: what pip just did
- Resolved dependencies. Libraries rely on other libraries. pandas needs NumPy and a date-handling package called
python-dateutil; JupyterLab needs dozens of helpers. pip worked out the full list and picked versions that are compatible with each other. - Downloaded wheels. A wheel (a
.whlfile) is a pre-built, ready-to-install package. NumPy and pandas contain compiled code (written in C for speed), and wheels mean you don't need a compiler on your machine. - Unpacked everything into
.venv'ssite-packagesfolder. Your global Python is untouched.
Step 3: check the installation
Create ch02/check_versions.py:
import sys
import matplotlib
import numpy as np
import pandas as pd
import seaborn as sns
print("Python ", sys.version.split()[0])
print("NumPy ", np.__version__)
print("pandas ", pd.__version__)
print("Matplotlib ", matplotlib.__version__)
print("Seaborn ", sns.__version__)
Run it with python ch02/check_versions.py. At the time of writing, the output was:
Output:
Python 3.14.7
NumPy 2.5.3
pandas 3.0.6
Matplotlib 3.11.2
Seaborn 0.13.2
Your version numbers may be newer, and that's fine. What matters for this course is pandas 3.x and NumPy 2.x. If you see pandas 2.something, run python -m pip install --upgrade pandas.
Every library exposes its version as __version__ (two underscores each side). You'll use this when a tutorial behaves differently from what you see: very often, the tutorial was written for an older version.
Step 4: record what you installed in requirements.txt
Your environment is disposable (Chapter 1's house rule 2), so you need a record of what's in it. pip can write that record for you:
python -m pip freeze > requirements.txt
pip freeze lists every installed package with its exact version, and > sends that list into a file instead of the screen. Open requirements.txt and you'll see around a hundred lines like:
numpy==2.5.3
openpyxl==3.1.5
pandas==3.0.6
...
The == pins each package to an exact version. Anyone (including future you on a new laptop) can rebuild an identical environment with:
python -m pip install -r requirements.txt
This small file is what makes your analysis reproducible: the same code, run against the same library versions, produces the same results. Update it whenever you install something new.
The uv way: if you chose uv in Chapter 1,
uv add numpy pandas pyarrow matplotlib seaborn jupyterlab openpyxlinstalls everything and records it inpyproject.tomland a lock file (uv.lock) automatically, with no separate freeze step.
2.3 Jupyter notebooks: your analysis lab bench
The idea
A notebook is a document that mixes three things on one page:
- code cells, which you run one at a time,
- the output of each cell (numbers, tables, charts) displayed directly underneath it, and
- Markdown cells: formatted text for headings, explanations and findings.
Notebooks are saved as .ipynb files. Behind every open notebook runs a kernel: a live Python process that executes your cells and remembers every variable you create, for as long as it keeps running. That memory is what lets cell 5 use a table you loaded in cell 2.
Analysts love notebooks because they show each step and its result side by side, which suits the ask-a-question, look-at-the-answer, ask-another rhythm of analysis. They also make excellent reports: code, charts and written conclusions in one place.
Opening a notebook: two routes
Route A: JupyterLab in your browser
jupyter lab
Run this from your project folder with the environment active. A browser tab opens showing your project's files. Double-click the ch02 folder, then click Notebook → Python 3 (ipykernel) in the Launcher. Rename the new file (right-click → Rename) to ch02_libraries.ipynb. Because you started JupyterLab from inside the environment, the notebook's kernel uses your environment's Python and libraries. To stop JupyterLab, save your work, close the tab and press Ctrl+C in the terminal.
Route B: inside VS Code
- In VS Code's Explorer, create a new file
ch02/ch02_libraries.ipynb. VS Code opens it as a notebook. - Click Select Kernel (top right) → Python Environments → choose the one marked
.venv. - If VS Code offers to install
ipykernel, accept. (It came with JupyterLab, so usually it won't need to.)
Either route works. Many people use VS Code day to day and JupyterLab when they want a full-screen analysis space.
Confirm the notebook is using your environment
In the first cell, type this and press Shift+Enter:
import sys
sys.prefix
The result should end in python-data-analysis/.venv (with backslashes on Windows). If it points somewhere else, select the .venv kernel again.
The keys you'll use constantly
| Action | Keys |
|---|---|
| Run the cell and move to the next | Shift+Enter |
| Run the cell and stay put | Ctrl+Enter |
| Leave typing mode ("command mode") | Esc |
| Enter typing mode ("edit mode") | Enter |
| Insert a cell above / below (command mode) | A / B |
| Turn a cell into Markdown / code (command mode) | M / Y |
| Delete a cell (command mode) | D then D |
Notebooks show the last value automatically
Like the interactive shell from Chapter 1, a notebook cell displays the value of its last line without needing print(). For pandas tables this display is especially nice: a neatly formatted table. From here on, examples are written as notebook cells, so you'll often see a bare variable name on the last line; that's what triggers the display.
In this chapter, tables are shown as plain text so they fit in these notes. In a notebook they appear as tidy formatted tables with the same content.
The big gotcha: hidden state
Each cell has a number in square brackets, like [3], showing the order cells actually ran in. You can run cells in any order, re-run them, and delete them, and the kernel still remembers whatever those cells created. So a notebook can look fine on screen while depending on a variable from a cell you deleted an hour ago. Think of it like a lab bench still carrying residue from the last experiment.
The cure is a habit: before trusting or sharing results, use Restart Kernel and Run All Cells (in JupyterLab's Kernel menu; in VS Code, the Restart button then Run All). If everything runs top to bottom without errors, your notebook is honest.
2.4 NumPy: fast numbers in bulk
The idea
NumPy (short for Numerical Python) gives Python one new, extremely important type: the array, officially called an ndarray ("n-dimensional array"). An array is a grid of values that are all the same type, stored side by side in one tightly packed block of memory.
Why does that matter? Watch what a regular list does with maths:
prices_list = [4.50, 6.90, 18.99]
prices_list * 2
Output:
[4.5, 6.9, 18.99, 4.5, 6.9, 18.99]
A list multiplied by 2 repeats itself, just like text did in Chapter 1. To double each price you'd need a loop. Now the NumPy version:
import numpy as np
prices = np.array([4.50, 6.90, 18.99])
prices * 2
Output:
array([ 9. , 13.8 , 37.98])
The array applied * 2 to every element. This element-by-element behaviour is called vectorisation, and it's the single most important idea in this chapter. pandas columns behave the same way.
Under the hood: why arrays are fast
A Python list is a row of pointers to separate objects scattered around memory, each carrying its own type information. To double every item, Python must visit each object, check its type, do the maths and build a new object, one at a time.
An array stores raw numbers side by side, all the same type, so NumPy can hand the whole block to a tight loop written in the C language, which runs at the full speed of your processor. On a typical laptop, doubling a million numbers looks something like this:
big_list = list(range(1_000_000))
big_array = np.arange(1_000_000)
%timeit [x * 2 for x in big_list] # roughly 40 ms
%timeit big_array * 2 # roughly 0.7 ms: about 60 times faster
%timeit is a notebook-only helper (a "magic command") that runs a line many times and reports the typical duration. Your numbers will differ, and the gap is what matters. With the data sizes analysts handle, that gap is the difference between waiting a second and waiting a minute.
Creating arrays
week = np.array([412.50, 389.90, 455.10, 501.75, 610.20, 720.40, 298.00])
day_numbers = np.arange(1, 8) # like range(): 1 to 7, stop excluded
evenly_spaced = np.linspace(0, 1, 5) # 5 evenly spaced values from 0 to 1
empty_tally = np.zeros(4) # four zeros
print(day_numbers)
print(evenly_spaced)
print(empty_tally)
Output:
[1 2 3 4 5 6 7]
[0. 0.25 0.5 0.75 1. ]
[0. 0. 0. 0.]
Every array knows a few facts about itself:
print(week.shape) # size along each dimension
print(week.ndim) # number of dimensions
print(week.size) # total number of elements
print(week.dtype) # the single data type shared by every element
Output:
(7,)
1
7
float64
(7,) reads as "7 elements along one dimension". The trailing comma is how Python writes a tuple with one item. float64 means "64-bit floating-point number", the standard decimal type; int64 is its whole-number cousin. You'll see these dtype names constantly in pandas.
Worked example 2.2: a week of sales, vectorised
costs = np.array([250.00, 240.10, 270.30, 300.00, 350.60, 410.00, 190.50])
margin = week - costs # element by element
margin_pct = margin / week * 100 # still element by element
print(margin)
print(margin_pct.round(1))
Output:
[162.5 149.8 184.8 201.75 259.6 310.4 107.5 ]
[39.4 38.4 40.6 40.2 42.5 43.1 36.1]
Two arrays of the same length combine position by position: Monday's sales minus Monday's costs, Tuesday's minus Tuesday's, and so on. No loop required.
Summarising an array
print(week.sum())
print(week.mean())
print(week.max())
print(week.argmax()) # the POSITION of the largest value
print(week.std().round(2))
Output:
3387.85
483.9785714285714
720.4
5
131.76
argmax() answers a different question from max(): where is the biggest value? Position 5 is the sixth day, Saturday, since positions start at 0.
The standard deviation here differs slightly from the statistics.stdev result in section 2.1 (131.76 vs 142.32). NumPy's std() treats the numbers as a whole population by default, while statistics.stdev() treats them as a sample from a larger population. Chapter 10 explains when to use which; for now, just know the two exist.
A display quirk you'll meet in notebooks
Type a summary as the last line of a cell and the notebook shows something slightly surprising:
week.mean()
Output:
np.float64(483.9785714285714)
Since NumPy 2, displaying a single NumPy number also shows its type: np.float64(...) means "a NumPy 64-bit float with this value". It behaves exactly like a normal Python float. print() shows the bare number, as you saw above, and float(week.mean()) converts it to a plain Python float. Older tutorials, written before NumPy 2, show only the bare number.
Boolean masks: the most useful pattern in data analysis
Comparing an array to a value produces an array of True/False, one per element:
target = 450
week > target
Output:
array([False, False, True, True, True, True, False])
That True/False array is called a boolean mask. It has two superpowers.
1. Use it to select only the matching elements:
week[week > target]
Output:
array([455.1 , 501.75, 610.2 , 720.4 ])
Read week[week > target] as "the values of week, where week is above the target". Only positions marked True survive.
2. Count or measure with it. In arithmetic, True counts as 1 and False as 0:
print((week > target).sum()) # how many days beat the target
print((week > target).mean()) # what fraction of days beat the target
Output:
4
0.5714285714285714
Four days out of seven, or about 57%. The same "sum a mask to count, average it to get a proportion" trick works in pandas, and you'll use it weekly.
Combining conditions
Use & for "and", | for "or", and ~ for "not". Wrap each condition in brackets:
weekend = np.array([False, False, False, False, False, True, True])
week[(week > target) & ~weekend] # strong weekdays
Output:
array([455.1 , 501.75, 610.2 ])
Why the brackets? In Python, & is worked out before >, so without brackets Python would try to calculate target & ~weekend first and fail. Brackets make the order explicit. (The words and/or from Chapter 1 work only on single True/False values; for arrays you need the symbols.)
Choosing between two values: np.where
np.where(condition, value_if_true, value_if_false) builds a new array element by element:
np.where(week > target, "Above", "Below")
Output:
array(['Below', 'Below', 'Above', 'Above', 'Above', 'Above', 'Below'],
dtype='<U5')
This is Python's version of Excel's IF() applied to a whole column at once. You'll use it for labelling in Chapter 5. (The dtype='<U5' at the end is NumPy's own text type: Unicode text up to 5 characters long. In pandas you'll see text as str instead.)
Two dimensions: rows and columns
Arrays can have more than one dimension. A two-dimensional array is a table of numbers. Here are four branches (rows) across seven days (columns):
branch_week = np.array([
[520, 480, 610, 590, 700, 820, 400], # Lambton Quay
[410, 395, 450, 470, 600, 760, 380], # Cuba Street
[300, 310, 290, 350, 420, 510, 330], # Petone
[220, 230, 260, 240, 300, 380, 250], # Karori
])
print(branch_week.shape)
print(branch_week[0, 5]) # row 0 (Lambton Quay), column 5 (Saturday)
print(branch_week[2]) # the whole of row 2 (Petone)
print(branch_week[:, 0]) # every row, column 0 (Monday)
Output:
(4, 7)
820
[300 310 290 350 420 510 330]
[520 410 300 220]
Indexing takes [row, column]. A lone : means "everything along this dimension", so [:, 0] is "all rows, first column".
Understanding axis
Summaries on a 2-D array take an axis setting, and it trips everyone up at first. The trick: axis names the direction that gets collapsed.
axis=1 → collapse ACROSS each row
┌─────────────────────────────────┐
axis=0 │ 520 480 610 590 700 820 400 │ → one total per row (per branch)
↓ │ 410 395 450 470 600 760 380 │
collapse │ 300 310 290 350 420 510 330 │
DOWN each │ 220 230 260 240 300 380 250 │
column └─────────────────────────────────┘
↓ one total per column (per day)
print(branch_week.sum(axis=1)) # per branch: add across each row
print(branch_week.sum(axis=0)) # per day: add down each column
Output:
[4120 3465 2510 1880]
[1450 1415 1610 1650 2020 2470 1360]
pandas uses the same axis idea, so this picture will keep paying off.
Missing values: np.nan
Real data has gaps. NumPy marks a missing number with np.nan ("Not a Number"), a special floating-point value. It's contagious: any arithmetic involving it produces nan.
with_gap = np.array([412.50, np.nan, 455.10])
print(with_gap.sum())
print(np.nansum(with_gap))
print(np.nanmean(with_gap))
Output:
nan
867.6
433.8
np.nansum and np.nanmean skip the gaps. Because nan is a float, a column of whole numbers with even one gap becomes a float column. You'll see exactly this happen in Chapter 3, and pandas handles missing values more gracefully than raw NumPy (its summaries skip them by default).
Random numbers, reproducibly
Analysts often need realistic practice data, or random samples. NumPy's random generator does this:
rng = np.random.default_rng(seed=42)
print(rng.integers(1, 5, size=10)) # 10 whole numbers from 1 to 4
print(rng.normal(loc=100, scale=15, size=3).round(1)) # 3 bell-curve values around 100
Output:
[1 4 3 2 2 4 1 3 1 1]
[ 80.5 101.9 95.3]
The seed is a starting point for the generator. The same seed always produces the same "random" numbers, so your results are reproducible: run the cell again, or send it to a colleague, and you'll both get identical values. normal draws from the familiar bell-shaped curve: loc is its centre and scale controls the spread.
2.5 pandas: tables with superpowers
The idea
pandas is the library you'll spend most of your analyst life in. Its name comes from "panel data", an economics term for tables of observations over time. It gives you two structures:
- a Series: one column of values, each with a label; and
- a DataFrame: a table made of several Series side by side, all sharing the same row labels.
The row labels are called the index. Think of a DataFrame as a spreadsheet where every column knows its own data type, every row has a label, and every operation can be applied to a whole column at once.
Under the hood, each column is essentially a NumPy array (or, for text, a pyarrow array). So everything you just learnt, including vectorisation, boolean masks, nan and axis, carries straight across.
Series: one labelled column
import pandas as pd
sales = pd.Series(
[412.50, 389.90, 455.10, 501.75, 610.20, 720.40, 298.00],
index=["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"],
name="daily_sales",
)
sales
Output:
Mon 412.50
Tue 389.90
Wed 455.10
Thu 501.75
Fri 610.20
Sat 720.40
Sun 298.00
Name: daily_sales, dtype: float64
The left column is the index (labels), the right column holds the values, and the footer shows the Series' name and dtype. Selection works by label, and everything is vectorised:
print(sales["Sat"])
print(sales[sales > 500])
Output:
720.4
Thu 501.75
Fri 610.20
Sat 720.40
Name: daily_sales, dtype: float64
DataFrames: whole tables
You can build a DataFrame from the exact structure you used in Chapter 1: a list of dictionaries, where each dictionary is one row.
Worked example 2.3: Chapter 1's sales, now as a DataFrame
sales_records = [
{"order_id": 1001, "branch": "Petone", "product": "Vapour Rub 50g",
"unit_price": 9.99, "quantity": 1, "loyalty_member": True},
{"order_id": 1002, "branch": "Lambton Quay", "product": "Throat Lozenges 16pk",
"unit_price": 8.99, "quantity": 2, "loyalty_member": True},
{"order_id": 1003, "branch": "Karori", "product": "Throat Lozenges 16pk",
"unit_price": 8.99, "quantity": 4, "loyalty_member": False},
{"order_id": 1004, "branch": "Karori", "product": "Throat Lozenges 16pk",
"unit_price": 8.99, "quantity": 1, "loyalty_member": True},
{"order_id": 1005, "branch": "Petone", "product": "Plasters 40pk",
"unit_price": 5.99, "quantity": 1, "loyalty_member": True},
{"order_id": 1006, "branch": "Lambton Quay", "product": "Multivitamin 60tab",
"unit_price": 24.99, "quantity": 1, "loyalty_member": False},
{"order_id": 1007, "branch": "Cuba Street", "product": "Ibuprofen 200mg 24pk",
"unit_price": 6.90, "quantity": 1, "loyalty_member": True},
{"order_id": 1008, "branch": "Cuba Street", "product": "Cold & Flu Capsules 24pk",
"unit_price": 14.50, "quantity": 3, "loyalty_member": False},
]
df = pd.DataFrame(sales_records)
df
Output:
order_id branch product unit_price quantity loyalty_member
0 1001 Petone Vapour Rub 50g 9.99 1 True
1 1002 Lambton Quay Throat Lozenges 16pk 8.99 2 True
2 1003 Karori Throat Lozenges 16pk 8.99 4 False
3 1004 Karori Throat Lozenges 16pk 8.99 1 True
4 1005 Petone Plasters 40pk 5.99 1 True
5 1006 Lambton Quay Multivitamin 60tab 24.99 1 False
6 1007 Cuba Street Ibuprofen 200mg 24pk 6.90 1 True
7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False
The dictionary keys became column names, and pandas added a default index of 0 to 7 down the left. (You can also build a DataFrame from a dictionary of lists, one list per column: pd.DataFrame({"branch": [...], "quantity": [...]}). Both are common.)
The first-look toolkit
Whenever you meet a new DataFrame, these four tell you what you're dealing with:
print(df.shape) # (rows, columns)
print(df.columns.tolist())
Output:
(8, 6)
['order_id', 'branch', 'product', 'unit_price', 'quantity', 'loyalty_member']
df.dtypes
Output:
order_id int64
branch str
product str
unit_price float64
quantity int64
loyalty_member bool
dtype: object
Each column has one dtype. int64 and float64 are NumPy's number types; bool is True/False; and str marks text.
pandas 3 note: the
strdtype for text is new in pandas 3.0, released January 2026. Tutorials and books written for earlier versions show text columns asobject. Both mean "this column holds text"; the newstrtype is stricter (it only ever holds text or missing values) and faster.
df.info()
Output:
<class 'pandas.DataFrame'>
RangeIndex: 8 entries, 0 to 7
Data columns (total 6 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 order_id 8 non-null int64
1 branch 8 non-null str
2 product 8 non-null str
3 unit_price 8 non-null float64
4 quantity 8 non-null int64
5 loyalty_member 8 non-null bool
dtypes: bool(1), float64(1), int64(2), str(2)
memory usage: 679.0 bytes
info() combines shape, column names, dtypes and, crucially, the non-null count: how many values in each column are present (not missing). When a column shows fewer non-null values than there are rows, you've found missing data. All 8 are present here; Chapter 3's real dataset won't be so tidy.
df.describe()
Output:
order_id unit_price quantity
count 8.00000 8.000000 8.000000
mean 1004.50000 11.167500 1.750000
std 2.44949 6.127178 1.164965
min 1001.00000 5.990000 1.000000
25% 1002.75000 8.467500 1.000000
50% 1004.50000 8.990000 1.000000
75% 1006.25000 11.117500 2.250000
max 1008.00000 24.990000 4.000000
describe() summarises every numeric column: count, mean, standard deviation, minimum, the 25th/50th/75th percentiles (the 50% row is the median) and maximum. It's a thirty-second health check. Here it says, for example, that the median sale is 1 unit (the 50% row), the average is 1.75 units, and the largest is 4 units. (The order_id summary is meaningless, since IDs are labels rather than quantities. Ignoring columns like that is part of reading describe() sensibly.)
Selecting columns
df["product"]
Output:
0 Vapour Rub 50g
1 Throat Lozenges 16pk
2 Throat Lozenges 16pk
3 Throat Lozenges 16pk
4 Plasters 40pk
5 Multivitamin 60tab
6 Ibuprofen 200mg 24pk
7 Cold & Flu Capsules 24pk
Name: product, dtype: str
Single brackets with one column name give you a Series. Double brackets give you a DataFrame with just those columns:
df[["branch", "product", "quantity"]]
Output:
branch product quantity
0 Petone Vapour Rub 50g 1
1 Lambton Quay Throat Lozenges 16pk 2
2 Karori Throat Lozenges 16pk 4
3 Karori Throat Lozenges 16pk 1
4 Petone Plasters 40pk 1
5 Lambton Quay Multivitamin 60tab 1
6 Cuba Street Ibuprofen 200mg 24pk 1
7 Cuba Street Cold & Flu Capsules 24pk 3
Why double? The outer brackets mean "select from df", and the inner brackets are a Python list of the columns you want. Chapter 3 covers selection in much more depth.
Creating a column: vectorised, no loop
In Chapter 1 you wrote a line_total() function and called it inside a loop. In pandas:
df["revenue"] = df["unit_price"] * df["quantity"]
df[["product", "unit_price", "quantity", "revenue"]]
Output:
product unit_price quantity revenue
0 Vapour Rub 50g 9.99 1 9.99
1 Throat Lozenges 16pk 8.99 2 17.98
2 Throat Lozenges 16pk 8.99 4 35.96
3 Throat Lozenges 16pk 8.99 1 8.99
4 Plasters 40pk 5.99 1 5.99
5 Multivitamin 60tab 24.99 1 24.99
6 Ibuprofen 200mg 24pk 6.90 1 6.90
7 Cold & Flu Capsules 24pk 14.50 3 43.50
One line. pandas multiplied the two columns row by row (vectorisation again) and stored the result as a new column called revenue.
Filtering rows with a boolean mask
Exactly the same pattern as NumPy:
bulk = df["quantity"] >= 3
df[bulk]
Output:
order_id branch product unit_price quantity loyalty_member revenue
2 1003 Karori Throat Lozenges 16pk 8.99 4 False 35.96
7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False 43.50
And combined conditions, with &, |, ~ and brackets around each condition:
df[(df["branch"] == "Karori") & df["loyalty_member"]]
Output:
order_id branch product unit_price quantity loyalty_member revenue
3 1004 Karori Throat Lozenges 16pk 8.99 1 True 8.99
df["loyalty_member"] is already a column of True/False values, so it can be used as a condition directly.
Sorting and counting
df.sort_values("revenue", ascending=False).head(3)
Output:
order_id branch product unit_price quantity loyalty_member revenue
7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False 43.50
2 1003 Karori Throat Lozenges 16pk 8.99 4 False 35.96
5 1006 Lambton Quay Multivitamin 60tab 24.99 1 False 24.99
sort_values() returns a new, sorted DataFrame, and .head(3) keeps the first three rows. Chaining methods like this (do one thing, then the next) is very common in pandas code.
df["branch"].value_counts()
Output:
branch
Petone 2
Lambton Quay 2
Karori 2
Cuba Street 2
Name: count, dtype: int64
value_counts() counts how often each distinct value appears, most frequent first. It's the pandas version of the counting loop you wrote in Chapter 1. In this small sample every branch appears twice, so the tie is listed in order of first appearance; with real data the differences are usually more interesting.
Worked example 2.4: Chapter 1's summary, in pandas
Here's the payoff. The revenue-by-branch table from your Chapter 1 milestone script took a loop, a dictionary and careful bookkeeping:
df.groupby("branch")["revenue"].sum()
Output:
branch
Cuba Street 50.40
Karori 44.95
Lambton Quay 42.97
Petone 15.98
Name: revenue, dtype: float64
Same numbers as Chapter 1 (Cuba Street 50.40, Karori 44.95, Lambton Quay 42.97, Petone 15.98), in one line. Read it as: group the rows by branch, take the revenue column within each group, and add it up. Chapter 6 is devoted to groupby, because it answers a huge share of business questions.
The rest of the Chapter 1 report takes a line each:
total = df["revenue"].sum()
loyalty_share = df.loc[df["loyalty_member"], "revenue"].sum() / total
best_seller = df.groupby("product")["quantity"].sum().idxmax()
print(f"Total revenue: ${total:,.2f}")
print(f"Loyalty share: {loyalty_share:.1%}")
print(f"Best seller: {best_seller}")
Output:
Total revenue: $154.30
Loyalty share: 32.3%
Best seller: Throat Lozenges 16pk
Two new tools here:
df.loc[rows, column]selects by row condition and column name at the same time. Here: the rows whereloyalty_memberis True, and only therevenuecolumn. Chapter 3 explains.locfully.idxmax()returns the label of the largest value, the pandas cousin of NumPy'sargmax().
Changing values safely with .loc
To change some values in a column, based on a condition, use .loc with the condition and the column name together:
df["discount"] = 0.0
df.loc[df["loyalty_member"], "discount"] = 0.05
df[["order_id", "loyalty_member", "discount"]]
Output:
order_id loyalty_member discount
0 1001 True 0.05
1 1002 True 0.05
2 1003 False 0.00
3 1004 True 0.05
4 1005 True 0.05
5 1006 False 0.00
6 1007 True 0.05
7 1008 False 0.00
Read it as: in the rows where the customer is a loyalty member, set discount to 0.05.
pandas 3 note: always assign with a single
.loc. Older tutorials sometimes writedf[df["loyalty_member"]]["discount"] = 0.05, two sets of brackets one after another (called chained assignment). In pandas 3 that line never changesdf. It changes a temporary copy that's immediately thrown away, and pandas prints aChainedAssignmentErrorwarning to tell you so. This comes from a pandas 3 design called Copy-on-Write: every selection behaves like an independent copy, which also means the oldSettingWithCopyWarningyou'll see complained about online has been removed. The rule to remember is simple: to change values in place, use one.loc[rows, column] = value.
Missing values in pandas
pandas shows missing values as NaN (or None/<NA> in some dtypes), and, helpfully, its summaries skip them automatically:
quantities = pd.Series([1, 2, None, 4])
print(quantities)
print("Sum:", quantities.sum())
print("Missing:", quantities.isna().sum())
Output:
0 1.0
1 2.0
2 NaN
3 4.0
dtype: float64
Sum: 7.0
Missing: 1
Two things to notice. First, the column became float64 because of the gap, just as you saw with NumPy. Second, .isna() makes a boolean mask of "is this missing?", and .sum() counts the Trues: the same mask-and-count trick from earlier. Chapter 4 covers deciding what to do about missing values.
2.6 Matplotlib: drawing charts
The idea
Matplotlib is Python's foundational charting library. Nearly every other Python charting tool, including Seaborn and pandas' own .plot(), draws through it.
Two words form the mental model:
- A Figure is the whole canvas: the image you'd save or paste into a report.
- An Axes is one chart inside that figure: the plotting area together with its x-axis, y-axis, title and labels. A figure can hold one Axes or several.
"Axes" (the chart) and "axis" (one of the lines with tick marks) are different things, confusingly close in spelling. You'll mostly talk to the Axes.
The recommended style
The Matplotlib documentation recommends this pattern, and it's the one we'll use throughout the course:
fig, ax = plt.subplots() # 1. make a Figure with one Axes
ax.plot(...) # 2. draw on the Axes
ax.set_title(...) # 3. label it
fig.savefig(...) # 4. save it (optional)
plt.show() # 5. display it
You'll also see shorter code in tutorials that calls plt.plot(...) directly, relying on Matplotlib to keep track of "the current chart" behind the scenes. That shortcut works for quick one-off charts. The fig, ax style is more explicit, and it scales cleanly when you need two or more charts side by side, so it's worth learning first.
Worked example 2.5: a line chart of the week
import matplotlib.pyplot as plt
days = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
fig, ax = plt.subplots(figsize=(8, 4))
ax.plot(days, week, marker="o", label="Daily sales")
ax.axhline(target, color="grey", linestyle="--", label="Target ($450)")
ax.set_title("Harbour Pharmacy: sales by day")
ax.set_xlabel("Day")
ax.set_ylabel("Sales (NZD)")
ax.legend()
fig.savefig("../output/ch02_week_line.png", dpi=150, bbox_inches="tight")
plt.show()
What each line does:
| Code | Effect |
|---|---|
plt.subplots(figsize=(8, 4)) |
a figure 8 inches wide and 4 tall, containing one Axes |
ax.plot(days, week, marker="o") |
a line through the seven points, with a dot on each |
label="..." |
the name to show in the legend |
ax.axhline(target, ...) |
a horizontal reference line across the chart at the target |
set_title, set_xlabel, set_ylabel |
the chart title and axis labels |
ax.legend() |
draw the legend, using the labels given above |
fig.savefig(...) |
save as a PNG image in your output/ folder; dpi=150 sets the sharpness and bbox_inches="tight" trims empty margins |
plt.show() |
display the chart |
You'll see a line rising from Monday to a Saturday peak, then dropping sharply on Sunday, with a dashed grey target line that four of the dots sit above.
The ../output/ path means "go up one folder from where this notebook lives (ch02/), then into output/". Chapter 3 explains file paths properly.
Why end chart cells with
plt.show()? In a notebook the chart appears anyway, and withoutplt.show()the cell also prints the value of its last line: something like<matplotlib.legend.Legend at 0x7f…>. It's harmless, just noisy. In a.pyscript,plt.show()is what actually opens the chart window.
Worked example 2.6: a bar chart from a pandas result
Charts usually come straight from pandas results. Let's chart the four branches' weekly totals from the 2-D array:
branch_names = ["Lambton Quay", "Cuba Street", "Petone", "Karori"]
branch_totals = pd.Series(branch_week.sum(axis=1), index=branch_names)
fig, ax = plt.subplots(figsize=(7, 4))
ax.bar(branch_totals.index, branch_totals.values, color="steelblue")
ax.set_title("Weekly sales by branch")
ax.set_ylabel("Sales (NZD)")
plt.show()
ax.bar(x_positions, heights) draws one bar per branch. .index gives the Series' labels and .values gives its numbers.
pandas also has a .plot() shortcut that does the same job and fills in some labels for you:
fig, ax = plt.subplots(figsize=(7, 4))
branch_totals.sort_values().plot(kind="barh", ax=ax, color="steelblue")
ax.set_title("Weekly sales by branch")
ax.set_xlabel("Sales (NZD)")
plt.show()
kind="barh" makes a horizontal bar chart, which is often easier to read when the category names are long, and sorting first puts the bars in order. Passing ax=ax tells pandas to draw on your Axes, so you keep full control over the labels.
Worked example 2.7: a histogram, and two charts side by side
A histogram shows how values are distributed: it splits the range into "bins" and counts how many values fall in each. Let's simulate 500 transaction values and look at their shape, next to a second chart:
rng = np.random.default_rng(seed=1)
transaction_values = rng.gamma(shape=2.0, scale=12.0, size=500).round(2)
fig, axes = plt.subplots(1, 2, figsize=(11, 4))
axes[0].hist(transaction_values, bins=30, color="teal", edgecolor="white")
axes[0].set_title("Transaction values (500 simulated sales)")
axes[0].set_xlabel("Value (NZD)")
axes[0].set_ylabel("Number of transactions")
axes[1].plot(days, week.cumsum(), marker="o")
axes[1].set_title("Running total across the week")
axes[1].set_ylabel("Cumulative sales (NZD)")
fig.tight_layout()
plt.show()
plt.subplots(1, 2)makes 1 row and 2 columns of Axes, returned as an array calledaxes.axes[0]is the left chart andaxes[1]the right.rng.gamma(...)produces skewed, all-positive values, a realistic shape for purchase amounts: most transactions are small, with a long tail of bigger ones. The histogram shows a hump near $10–$20 and a tail stretching to the right..cumsum()is a cumulative sum, a running total.fig.tight_layout()nudges the charts apart so their labels don't overlap.
Chapter 9 is dedicated to choosing and polishing charts. For now, the pattern fig, ax = plt.subplots() → draw → label → show is the one to have in your fingers.
2.7 Seaborn: statistical charts with sensible defaults
The idea
Seaborn is built on Matplotlib and designed to work directly with pandas DataFrames. You tell it which columns to use for x, y and colour, and it handles the grouping, summarising and legends for you. Its defaults are also more attractive than Matplotlib's.
import seaborn as sns
sns.set_theme(style="whitegrid") # a clean look for every chart from now on
Worked example 2.8: revenue by branch, split by loyalty
fig, ax = plt.subplots(figsize=(8, 4))
sns.barplot(
data=df,
x="branch",
y="revenue",
hue="loyalty_member",
estimator="sum",
errorbar=None,
ax=ax,
)
ax.set_title("Revenue by branch and loyalty status")
ax.set_ylabel("Revenue (NZD)")
plt.show()
| Setting | Meaning |
|---|---|
data=df |
the DataFrame to draw from |
x="branch", y="revenue" |
the columns for each axis |
hue="loyalty_member" |
split each bar into colours by this column, with a legend added automatically |
estimator="sum" |
add up the revenue within each group |
errorbar=None |
don't draw error bars |
ax=ax |
draw on our Axes |
An important Seaborn default to know:
barplotshows the mean of each group, with a black error bar showing a 95% confidence interval (a range that expresses uncertainty about that mean; Chapter 12 explains them). That's a sensible default for scientific data and a frequent trap for business reporting, where people usually expect bar heights to be totals. Settingestimator="sum"anderrorbar=Nonegives you the plain totals most stakeholders expect.
Worked example 2.9: a distribution in one line
fig, ax = plt.subplots(figsize=(7, 4))
sns.histplot(x=transaction_values, bins=30, kde=True, ax=ax)
ax.set_title("Transaction values")
ax.set_xlabel("Value (NZD)")
plt.show()
kde=True adds a smooth curve over the bars (a kernel density estimate: a smoothed outline of the histogram's shape), which makes the overall pattern easier to see.
As a rule of thumb: reach for Seaborn for quick, good-looking charts built straight from a DataFrame, especially when you want to split by category. Reach for Matplotlib directly when you need fine control over every detail. Because Seaborn draws on a Matplotlib Axes, you can always mix the two, as you just did with ax.set_title().
2.8 The wider toolbox
You won't install these yet. They're here so the names feel familiar when they appear:
| Library | What it does | Where it appears |
|---|---|---|
openpyxl |
Read and write Excel files (already installed) | Chapter 3, Chapter 13 |
sqlite3 (standard library), SQLAlchemy |
Talk to databases using SQL | Chapter 11 |
SciPy |
Scientific functions, including statistical tests | Chapter 12 |
statsmodels |
Statistical models and tests with detailed reports | Chapter 12 |
Plotly, Streamlit |
Interactive charts and simple web dashboards | Chapter 14 |
requests |
Fetch data from websites and APIs | Later chapters, as needed |
scikit-learn |
Machine learning: prediction and classification | A natural next step after this course |
Polars |
A newer, very fast DataFrame library; an alternative to pandas | Worth a look once pandas feels comfortable |
2.9 Project milestone: Harbour Pharmacy's first notebook
Time to put the toolkit together. The operations manager wants to see a whole month at a glance: how each branch performed in total, their best days, and how often each branch cleared a $1,000-a-day target.
You don't have the real month's data yet (that arrives in Chapter 3), so you'll simulate a realistic month with NumPy. Creating plausible fake data is a genuine analyst skill: it lets you build and test an analysis before the real data turns up.
Create a new notebook, ch02/ch02_milestone.ipynb. Put each block below in its own cell, and add a Markdown cell above each one with a short heading describing what it does (for example, ## 1. Simulate the month).
Cell 1: imports and settings
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
sns.set_theme(style="whitegrid")
Cell 2: simulate 28 days of sales for four branches
rng = np.random.default_rng(seed=7)
dates = pd.date_range("2026-03-02", periods=28, freq="D")
typical_day = {"Lambton Quay": 1450, "Cuba Street": 1150, "Petone": 900, "Karori": 700}
daily = pd.DataFrame(index=dates)
for branch, typical in typical_day.items():
daily[branch] = rng.normal(loc=typical, scale=typical * 0.15, size=len(dates)).round(2)
daily.index.name = "date"
daily.head()
Output:
Lambton Quay Cuba Street Petone Karori
date
2026-03-02 1450.27 1141.63 891.02 679.29
2026-03-03 1514.98 1169.55 990.08 583.02
2026-03-04 1390.38 886.05 1094.20 698.79
2026-03-05 1256.30 1067.59 808.79 653.42
2026-03-06 1351.11 981.21 927.42 822.44
What's happening:
pd.date_range("2026-03-02", periods=28, freq="D")creates 28 consecutive dates starting on Monday 2 March. (Dates get a whole chapter, Chapter 8.)pd.DataFrame(index=dates)makes an empty table whose row labels are those dates.- The loop adds one column per branch. Each column is 28 bell-curve values centred on that branch's typical day, with a spread of 15%. The fixed seed means you'll get exactly the numbers shown here.
Cell 3: headline numbers
monthly_totals = daily.sum().sort_values(ascending=False)
monthly_totals.round(2)
Output:
Lambton Quay 37884.11
Cuba Street 31952.05
Petone 25107.09
Karori 19453.71
dtype: float64
daily.sum() adds down each column (pandas' default is axis=0, the "collapse down" direction you met with NumPy), giving one total per branch.
Cell 4: each branch's best day, and days on target
target = 1000
summary = pd.DataFrame({
"total": daily.sum().round(2),
"daily_average": daily.mean().round(2),
"best_day": daily.idxmax().dt.strftime("%a %d %b"),
"days_on_target": (daily >= target).sum(),
})
summary.sort_values("total", ascending=False)
Output:
total daily_average best_day days_on_target
Lambton Quay 37884.11 1353.00 Mon 09 Mar 27
Cuba Street 31952.05 1141.14 Mon 23 Mar 23
Petone 25107.09 896.68 Wed 25 Mar 5
Karori 19453.71 694.78 Sat 21 Mar 0
Take this one slowly; it combines several ideas:
- Building a DataFrame from a dictionary of Series: each key becomes a column, and pandas lines the rows up by their shared labels (the branch names).
daily.idxmax()finds the date label of each column's largest value..dt.strftime("%a %d %b")formats those dates as text such asSat 21 Mar. (%a= short weekday,%d= day number,%b= short month; Chapter 8 lists the rest.)(daily >= target)compares the whole table to 1,000 at once, producing a table of True/False..sum()then counts the Trues in each column: the NumPy mask-counting trick, applied to every branch at once.
Notice Karori shows zero days on target. With a typical day of $700, a single $1,000 target is unfair on smaller branches, which is exactly the kind of observation an analyst raises. (Exercise 2.8 asks you to fix it.)
Cell 5: charts, saved for the report
fig, axes = plt.subplots(1, 2, figsize=(13, 4.5))
for branch in daily.columns:
axes[0].plot(daily.index, daily[branch], label=branch, linewidth=1.5)
axes[0].axhline(target, color="grey", linestyle="--", linewidth=1)
axes[0].set_title("Daily sales by branch, March 2026 (simulated)")
axes[0].set_ylabel("Sales (NZD)")
axes[0].tick_params(axis="x", labelrotation=45)
axes[0].legend(ncols=2, fontsize=9)
monthly_totals.sort_values().plot(kind="barh", ax=axes[1], color="steelblue")
axes[1].set_title("Total sales over 28 days")
axes[1].set_xlabel("Sales (NZD)")
axes[1].set_ylabel("")
fig.tight_layout()
fig.savefig("../output/ch02_milestone.png", dpi=150, bbox_inches="tight")
plt.show()
The left chart loops over the branch columns and draws one line for each, labelled with the branch name so the legend knows what to call it. Matplotlib recognises that the x values are dates and spaces the date labels sensibly; tick_params(axis="x", labelrotation=45) tilts them so they don't collide. (pandas can draw all four lines in one go with daily.plot(ax=axes[0]), and it's handy for a quick look; on daily dates its automatic labels can get crowded, which is why we're drawing the lines ourselves here.)
What you'll see: on the left, four wiggly lines in clear bands, with Lambton Quay on top and Karori at the bottom, and the dashed target line running between Cuba Street and Petone. On the right, four horizontal bars ranked by total.
Cell 6 (Markdown): findings
End the notebook with a Markdown cell, written in plain language for the operations manager. For example:
## Findings (simulated data)
- Lambton Quay generated the most revenue over the 28 days, about $37.9k,
nearly double Karori's $19.5k.
- Lambton Quay beat the $1,000 daily target on 27 of 28 days; Karori never did.
- A single company-wide target doesn't suit branches of such different sizes.
Next step: set targets relative to each branch's typical day.
Finally, Restart Kernel and Run All to prove the notebook runs cleanly from top to bottom. Check that output/ now contains ch02_week_line.png and ch02_milestone.png.
Writing findings in plain words at the end of an analysis is a habit worth starting now. Stakeholders read the conclusions; the code is how you earn the right to state them.
2.10 Exercises
Work in a notebook, ch02/ch02_exercises.ipynb, with a Markdown heading for each exercise. Attempt each before opening the solution.
Available this chapter: 250 XP.
Warm-up (10 XP each)
Exercise 2.1: Environment health check. In a notebook cell, print sys.prefix and the pandas and NumPy versions. Then, in the terminal, regenerate requirements.txt and confirm pandas== appears in it.
Exercise 2.2: Stock check with NumPy. Create an array of stock counts, stock = np.array([120, 9, 0, 31, 18, 64, 3]), and a reorder point of 25. Using boolean masks, print (a) how many products need reordering (below 25, including zero), and (b) the stock counts of those products.
Exercise 2.3: A price list Series. Create a Series of these prices, indexed by product name: Paracetamol 500mg 20pk 4.50, Vitamin C 1000mg 60tab 18.99, Sunscreen SPF50 200ml 22.99, Plasters 40pk 5.99. Add 15% GST (rounded to 2 decimals) to make a new Series, then show only the products whose GST-inclusive price is over $10.
Show a solution ```python import pandas as pd prices = pd.Series( [4.50, 18.99, 22.99, 5.99], index=["Paracetamol 500mg 20pk", "Vitamin C 1000mg 60tab", "Sunscreen SPF50 200ml", "Plasters 40pk"], ) prices_incl = (prices * 1.15).round(2) prices_incl[prices_incl > 10] ``` **Output:** ```text Vitamin C 1000mg 60tab 21.84 Sunscreen SPF50 200ml 26.44 dtype: float64 ```Core (20 XP each)
Exercise 2.4: Your own DataFrame. Build a DataFrame from this dictionary of lists, add a revenue column, keep only the Cold & Flu rows, and sort them by revenue from highest to lowest.
data = {
"product": ["Throat Lozenges 16pk", "Paracetamol 500mg 20pk", "Vapour Rub 50g",
"Cold & Flu Capsules 24pk", "Vitamin C 1000mg 60tab", "Throat Lozenges 16pk"],
"category": ["Cold & Flu", "Pain Relief", "Cold & Flu", "Cold & Flu", "Vitamins", "Cold & Flu"],
"unit_price": [8.99, 4.50, 9.99, 14.50, 18.99, 8.99],
"quantity": [2, 3, 4, 1, 2, 5],
}
Show a solution
```python
products = pd.DataFrame(data)
products["revenue"] = products["unit_price"] * products["quantity"]
cold_flu = products[products["category"] == "Cold & Flu"]
cold_flu.sort_values("revenue", ascending=False)
```
**Output:**
```text
product category unit_price quantity revenue
5 Throat Lozenges 16pk Cold & Flu 8.99 5 44.95
2 Vapour Rub 50g Cold & Flu 9.99 4 39.96
0 Throat Lozenges 16pk Cold & Flu 8.99 2 17.98
3 Cold & Flu Capsules 24pk Cold & Flu 14.50 1 14.50
```
Exercise 2.5: Group and total. Using products from Exercise 2.4, show the total quantity sold per category, largest first. Then show the total revenue per category.
Exercise 2.6: Chart it. Draw a horizontal bar chart of revenue per category (from Exercise 2.5), with a title and axis label, and save it as output/ch02_ex6.png.
Stretch (40 XP each)
Exercise 2.7: Two dimensions. Using branch_week from section 2.4 (4 branches × 7 days), and without any loops, find: (a) each branch's weekly total, (b) each day's total across all branches, (c) the position of each branch's best day, and (d) that best day's name. Hint: argmax also accepts axis=, and a NumPy array of day names can be indexed with an array of positions.
Exercise 2.8: Fairer targets, and a rebuild drill.
- In your milestone notebook, replace the single $1,000 target with a target for each branch equal to 110% of its own daily average, and count how many days each branch hit its own target. (Hint:
daily.mean() * 1.1gives one target per branch, and comparing a DataFrame to a Series lines them up by column name.) - Then prove your environment is reproducible: close VS Code/JupyterLab, delete the
.venvfolder, recreate it (section 1.2.4), reinstall withpython -m pip install -r requirements.txt, reopen the milestone notebook, select the new.venvkernel and Restart and Run All. Your numbers should be identical.
Quick self-check (20 XP for all five)
- What's the difference between
[1, 2, 3] * 2andnp.array([1, 2, 3]) * 2? - Why is
python -m pip installsafer thanpip install? - In
df[df["quantity"] >= 3], what does the inner part produce, and what does the outer part do with it? - On a 2-D array, which
axisgives you one result per row? - Why does
df[df["loyalty_member"]]["discount"] = 0.05fail to changedfin pandas 3, and what should you write instead?
2.11 Cheat sheet
| Task | Code |
|---|---|
| Install into the active environment | python -m pip install pandas |
| Record the environment | python -m pip freeze > requirements.txt |
| Rebuild the environment | python -m pip install -r requirements.txt |
| Start JupyterLab | jupyter lab |
| Standard imports | import numpy as np, import pandas as pd, import matplotlib.pyplot as plt, import seaborn as sns |
| Make an array | np.array([...]), np.arange(a, b), np.zeros(n) |
| Array facts | .shape, .dtype, .ndim, .size |
| Summaries | .sum(), .mean(), .max(), .argmax(), .std() |
| Boolean mask | arr[arr > 10], (arr > 10).sum(), (a > 1) & (b < 5) |
| If/else on a whole array | np.where(cond, "yes", "no") |
| Along an axis | axis=0 collapses down columns; axis=1 collapses across rows |
| Reproducible randomness | rng = np.random.default_rng(seed=42) |
| Make a Series / DataFrame | pd.Series(values, index=labels), pd.DataFrame(list_of_dicts) |
| First look | df.shape, df.head(), df.dtypes, df.info(), df.describe() |
| Select columns | df["col"] (Series), df[["a", "b"]] (DataFrame) |
| New column | df["revenue"] = df["price"] * df["qty"] |
| Filter rows | df[df["qty"] >= 3] |
| Sort / count | df.sort_values("col", ascending=False), df["col"].value_counts() |
| Group and total | df.groupby("branch")["revenue"].sum() |
| Change values safely | df.loc[condition, "col"] = value |
| Missing values | s.isna().sum() |
| Chart skeleton | fig, ax = plt.subplots() → ax.plot/bar/hist(...) → ax.set_title(...) → plt.show() |
| Save a chart | fig.savefig("../output/name.png", dpi=150, bbox_inches="tight") |
| Seaborn totals bar chart | sns.barplot(data=df, x=..., y=..., estimator="sum", errorbar=None, ax=ax) |
2.12 XP tally and what's next
| Section | XP available | XP earned |
|---|---|---|
| Warm-up (2.1–2.3) | 30 | |
| Core (2.4–2.6) | 60 | |
| Stretch (2.7–2.8) | 80 | |
| Milestone notebook runs cleanly with "Restart and Run All" | 60 | |
| Self-check (all five correct) | 20 | |
| Chapter 2 total | 250 | |
| Running total (Chapters 1–2) | 440 |
Coming up in Chapter 3: Loading, Inspecting and Selecting Data. The real Harbour Pharmacy sales file arrives. You'll learn how file paths work, load CSV, Excel and JSON files with pandas, run a systematic first-look checklist that uncovers the problems hiding in real data, and master .loc, .iloc and boolean filtering to pull out exactly the rows and columns a question needs.
References and further reading
- NumPy developers. NumPy: the absolute basics for beginners. https://numpy.org/doc/stable/user/absolute_beginners.html
- NumPy developers. NumPy 2.0 migration guide (including the new scalar display). https://numpy.org/doc/stable/numpy_2_0_migration_guide.html
- pandas development team. 10 minutes to pandas. https://pandas.pydata.org/docs/user_guide/10min.html
- pandas development team. Getting started tutorials. https://pandas.pydata.org/docs/getting_started/intro_tutorials/index.html
- pandas development team. What's new in 3.0.0 (January 21, 2026). https://pandas.pydata.org/docs/whatsnew/v3.0.0.html
- pandas development team. Copy-on-Write (CoW). https://pandas.pydata.org/docs/user_guide/copy_on_write.html
- pandas development team. Migration guide for the new string data type (pandas 3.0). https://pandas.pydata.org/docs/user_guide/migration-3-strings.html
- Matplotlib development team. Quick start guide (Figures, Axes, and the recommended coding style). https://matplotlib.org/stable/users/explain/quick_start.html
- Waskom, M. Seaborn user guide and tutorial. https://seaborn.pydata.org/tutorial.html
- Waskom, M. seaborn.barplot (estimator and errorbar options). https://seaborn.pydata.org/generated/seaborn.barplot.html
- Project Jupyter. JupyterLab documentation. https://jupyterlab.readthedocs.io/en/stable/
- Microsoft. Jupyter Notebooks in VS Code. https://code.visualstudio.com/docs/datascience/jupyter-notebooks
- pip developers. pip user guide: requirements files. https://pip.pypa.io/en/stable/user_guide/#requirements-files
- McKinney, W. Python for Data Analysis, 3rd edition (free to read online; written for pandas 1.x–2.x, so expect
objectinstead ofstrand older assignment patterns). https://wesmckinney.com/book/ - VanderPlas, J. Python Data Science Handbook (free online; older library versions, with excellent explanations of NumPy and Matplotlib). https://jakevdp.github.io/PythonDataScienceHandbook/