| | | |---|---| | **Level** | 0+: you've worked through Chapter 1 | | **Time** | 6–8 hours, best spread over three sessions | | **You need** | Your `python-data-analysis` project folder with its `.venv` from Chapter 1, and an internet connection for installing libraries | | **Project milestone** | The data toolkit installed inside your project, recorded in `requirements.txt`, and a first Harbour Pharmacy notebook that analyses a month of sales and saves two charts | --- ## Where we are In Chapter 1 you set up an isolated project, then wrote a sales summary using only core Python: a list of dictionaries, a few loops that tallied values into dictionaries, and a lot of careful f-string formatting. It worked, and it took a few dozen lines of careful bookkeeping. This chapter introduces the **libraries** that professional analysts use every day. By the end you'll produce that same branch summary in a single line, work with a whole month of data at once, and draw charts. More importantly, you'll understand what each library is *for* and how they fit together, so later chapters can go deep on each one without you feeling lost. The four libraries at the heart of this chapter, and how they stack: ```text ┌──────────────────────────────┐ │ Seaborn: statistical charts │ ├──────────────────────────────┤ │ Matplotlib: all charts │ ├──────────────────────────────┤ │ pandas: tables (DataFrames) │ ├──────────────────────────────┤ │ NumPy: fast arrays of numbers └──────────────────────────────┘ Python ``` Each layer is built on the ones beneath it. pandas stores its columns using NumPy; Seaborn draws its charts using Matplotlib. You'll spend most of your working life in the pandas layer, and knowing what sits underneath explains a great deal of pandas' behaviour. --- ## 2.1 Libraries, packages and modules: borrowing other people's work ### The idea Nobody writes everything from scratch. A **library** is a collection of ready-made code, written and tested by other people, that you can pull into your own programs. Python's vocabulary has three overlapping words: - A **module** is a single `.py` file of Python code that you can `import`. - A **package** is a folder of related modules bundled together. - **Library** is the everyday, informal word for a package (or group of packages) that gives you a toolkit, such as "the pandas library". Libraries come from two places: 1. **The standard library** ships with Python itself, so it's already on your computer. Useful members include `math`, `statistics`, `csv`, `datetime`, `pathlib` and `random`. 2. **Third-party libraries** are published by the community on the **Python Package Index (PyPI)**, at [pypi.org](https://pypi.org), which hosts hundreds of thousands of projects. The installer `pip` downloads them from PyPI. ### Worked example 2.1: using the standard library The `statistics` module can describe the week of sales from Chapter 1 without any installing: ```python import statistics daily_sales = [412.50, 389.90, 455.10, 501.75, 610.20, 720.40, 298.00] print(statistics.mean(daily_sales)) print(statistics.median(daily_sales)) print(round(statistics.stdev(daily_sales), 2)) ``` **Output:** ```text 483.97857142857146 455.1 142.32 ``` - `import statistics` makes the module available. You then reach its tools with a dot: `statistics.mean(...)`, just as you reached a string's methods with a dot in Chapter 1. - The **median** is the middle value once the list is sorted. It's often more representative than the mean (average) when a few values are unusually large or small. - The **standard deviation** (`stdev`) measures how spread out the values are: roughly, how far a typical day sits from the average. Chapter 10 covers it properly. ### Three ways to import ```text import statistics # use as statistics.mean(...) import statistics as st # a nickname: st.mean(...) from statistics import mean # bring one tool in directly: mean(...) ``` The data world has **nicknames everyone uses**. You'll see them in every tutorial, book and workplace, so use them too; they make your code instantly readable to other analysts: ```text import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns ``` ### Under the hood: how `import` finds a library When you write `import pandas`, Python searches a list of folders for something called `pandas`, in order, and uses the first match. Sound familiar? It's the same "first match wins" idea as PATH from Chapter 1, applied to libraries. The list lives in a variable called `sys.path`. While your virtual environment is active, that list includes your project's private shelf, `.venv/…/site-packages`, and leaves out the global one. That's how isolation works for libraries. It also explains the single most common beginner error in this chapter: ```text ModuleNotFoundError: No module named 'pandas' ``` Nine times out of ten this means one of two things: the library isn't installed *in this environment*, or the environment isn't active (or your editor is pointing at a different Python). Check for `(.venv)` in your terminal prompt, or the selected interpreter in VS Code, before anything else. --- ## 2.2 Installing the data toolkit into your project ### Step 1: activate your environment Open a terminal, move into your project folder and activate `.venv` (section 1.2.4). Confirm you see `(.venv)` in the prompt. **Everything in this section assumes the environment is active**; otherwise you'd be installing into global Python, which is exactly what we set out to avoid. ### Step 2: update pip, then install ```bash python -m pip install --upgrade pip python -m pip install numpy pandas pyarrow matplotlib seaborn jupyterlab openpyxl ``` The first line updates `pip` itself; newer versions handle newer package formats and give clearer error messages. The second line installs seven libraries at once. It may take a minute or two, and you'll see a long list of other packages being installed too; the next section explains why. What each one is for: | Library | Job | First used | |---|---|---| | `numpy` | Fast arrays of numbers: the engine underneath pandas | This chapter | | `pandas` | Tables (DataFrames): the analyst's main workbench | This chapter | | `pyarrow` | A high-speed data engine that pandas uses for text columns and file formats. The pandas team strongly recommends installing it alongside pandas 3 | Behind the scenes | | `matplotlib` | The foundational charting library | This chapter | | `seaborn` | Statistical charts with good defaults, built on Matplotlib | This chapter | | `jupyterlab` | The notebook application (section 2.3) | This chapter | | `openpyxl` | Lets pandas read and write Excel `.xlsx` files | Chapter 3 | #### Why `python -m pip` instead of plain `pip`? `python -m pip` means "run the pip that belongs to *this* `python`". Because your active environment puts `.venv` first on PATH, `python` is the environment's interpreter, so its pip installs onto the environment's shelf. Plain `pip` usually lands in the same place, but on machines with several Pythons installed it can quietly pick a different one. `python -m pip` removes the guesswork, and the official Python packaging guide uses it for that reason. #### A safety net: make pip refuse to install globally You can tell pip to refuse to install anything unless a virtual environment is active. Run this once (from any terminal): ```bash python -m pip config set global.require-virtualenv true ``` This writes a setting into your personal pip configuration file (the command prints the file's location as it writes), so it applies to every Python on your account. From then on, an install attempted with no environment active stops immediately with: ```text ERROR: Could not find an activated virtualenv (required). ``` That one line guarantees your libraries only ever land inside a project's `.venv`, which is exactly the isolation Chapter 1 set up. (In the setting's name, `global` refers to pip's settings section, and has nothing to do with global Python.) #### Under the hood: what pip just did 1. **Resolved dependencies.** Libraries rely on other libraries. pandas needs NumPy and a date-handling package called `python-dateutil`; JupyterLab needs dozens of helpers. pip worked out the full list and picked versions that are compatible with each other. 2. **Downloaded wheels.** A **wheel** (a `.whl` file) is a pre-built, ready-to-install package. NumPy and pandas contain compiled code (written in C for speed), and wheels mean you don't need a compiler on your machine. 3. **Unpacked everything into `.venv`'s `site-packages` folder.** Your global Python is untouched. ### Step 3: check the installation Create `ch02/check_versions.py`: ```python import sys import matplotlib import numpy as np import pandas as pd import seaborn as sns print("Python ", sys.version.split()[0]) print("NumPy ", np.__version__) print("pandas ", pd.__version__) print("Matplotlib ", matplotlib.__version__) print("Seaborn ", sns.__version__) ``` Run it with `python ch02/check_versions.py`. At the time of writing, the output was: **Output:** ```text Python 3.14.7 NumPy 2.5.3 pandas 3.0.6 Matplotlib 3.11.2 Seaborn 0.13.2 ``` Your version numbers may be newer, and that's fine. What matters for this course is **pandas 3.x** and **NumPy 2.x**. If you see pandas 2.something, run `python -m pip install --upgrade pandas`. Every library exposes its version as `__version__` (two underscores each side). You'll use this when a tutorial behaves differently from what you see: very often, the tutorial was written for an older version. ### Step 4: record what you installed in `requirements.txt` Your environment is disposable (Chapter 1's house rule 2), so you need a record of what's in it. pip can write that record for you: ```bash python -m pip freeze > requirements.txt ``` `pip freeze` lists every installed package with its exact version, and `>` sends that list into a file instead of the screen. Open `requirements.txt` and you'll see around a hundred lines like: ```text numpy==2.5.3 openpyxl==3.1.5 pandas==3.0.6 ... ``` The `==` **pins** each package to an exact version. Anyone (including future you on a new laptop) can rebuild an identical environment with: ```bash python -m pip install -r requirements.txt ``` This small file is what makes your analysis **reproducible**: the same code, run against the same library versions, produces the same results. Update it whenever you install something new. > **The uv way:** if you chose uv in Chapter 1, `uv add numpy pandas pyarrow matplotlib seaborn jupyterlab openpyxl` installs everything and records it in `pyproject.toml` and a lock file (`uv.lock`) automatically, with no separate freeze step. --- ## 2.3 Jupyter notebooks: your analysis lab bench ### The idea A **notebook** is a document that mixes three things on one page: - **code cells**, which you run one at a time, - the **output** of each cell (numbers, tables, charts) displayed directly underneath it, and - **Markdown cells**: formatted text for headings, explanations and findings. Notebooks are saved as `.ipynb` files. Behind every open notebook runs a **kernel**: a live Python process that executes your cells and *remembers* every variable you create, for as long as it keeps running. That memory is what lets cell 5 use a table you loaded in cell 2. Analysts love notebooks because they show each step and its result side by side, which suits the ask-a-question, look-at-the-answer, ask-another rhythm of analysis. They also make excellent reports: code, charts and written conclusions in one place. ### Opening a notebook: two routes **Route A: JupyterLab in your browser** ```bash jupyter lab ``` Run this from your project folder with the environment active. A browser tab opens showing your project's files. Double-click the `ch02` folder, then click **Notebook → Python 3 (ipykernel)** in the Launcher. Rename the new file (right-click → Rename) to `ch02_libraries.ipynb`. Because you started JupyterLab from inside the environment, the notebook's kernel uses your environment's Python and libraries. To stop JupyterLab, save your work, close the tab and press `Ctrl+C` in the terminal. **Route B: inside VS Code** 1. In VS Code's Explorer, create a new file `ch02/ch02_libraries.ipynb`. VS Code opens it as a notebook. 2. Click **Select Kernel** (top right) → **Python Environments** → choose the one marked `.venv`. 3. If VS Code offers to install `ipykernel`, accept. (It came with JupyterLab, so usually it won't need to.) Either route works. Many people use VS Code day to day and JupyterLab when they want a full-screen analysis space. ### Confirm the notebook is using your environment In the first cell, type this and press **Shift+Enter**: ```python import sys sys.prefix ``` The result should end in `python-data-analysis/.venv` (with backslashes on Windows). If it points somewhere else, select the `.venv` kernel again. ### The keys you'll use constantly | Action | Keys | |---|---| | Run the cell and move to the next | `Shift+Enter` | | Run the cell and stay put | `Ctrl+Enter` | | Leave typing mode ("command mode") | `Esc` | | Enter typing mode ("edit mode") | `Enter` | | Insert a cell above / below (command mode) | `A` / `B` | | Turn a cell into Markdown / code (command mode) | `M` / `Y` | | Delete a cell (command mode) | `D` then `D` | ### Notebooks show the last value automatically Like the interactive shell from Chapter 1, a notebook cell **displays the value of its last line** without needing `print()`. For pandas tables this display is especially nice: a neatly formatted table. From here on, examples are written as notebook cells, so you'll often see a bare variable name on the last line; that's what triggers the display. > In this chapter, tables are shown as plain text so they fit in these notes. In a notebook they appear as tidy formatted tables with the same content. ### The big gotcha: hidden state Each cell has a number in square brackets, like `[3]`, showing the order cells actually ran in. You can run cells in any order, re-run them, and delete them, and the kernel **still remembers** whatever those cells created. So a notebook can look fine on screen while depending on a variable from a cell you deleted an hour ago. Think of it like a lab bench still carrying residue from the last experiment. The cure is a habit: before trusting or sharing results, use **Restart Kernel and Run All Cells** (in JupyterLab's **Kernel** menu; in VS Code, the **Restart** button then **Run All**). If everything runs top to bottom without errors, your notebook is honest. --- ## 2.4 NumPy: fast numbers in bulk ### The idea **NumPy** (short for *Numerical Python*) gives Python one new, extremely important type: the **array**, officially called an `ndarray` ("n-dimensional array"). An array is a grid of values that are **all the same type**, stored side by side in one tightly packed block of memory. Why does that matter? Watch what a regular list does with maths: ```python prices_list = [4.50, 6.90, 18.99] prices_list * 2 ``` **Output:** ```text [4.5, 6.9, 18.99, 4.5, 6.9, 18.99] ``` A list multiplied by 2 **repeats itself**, just like text did in Chapter 1. To double each price you'd need a loop. Now the NumPy version: ```python import numpy as np prices = np.array([4.50, 6.90, 18.99]) prices * 2 ``` **Output:** ```text array([ 9. , 13.8 , 37.98]) ``` The array applied `* 2` to **every element**. This element-by-element behaviour is called **vectorisation**, and it's the single most important idea in this chapter. pandas columns behave the same way. ### Under the hood: why arrays are fast A Python list is a row of pointers to separate objects scattered around memory, each carrying its own type information. To double every item, Python must visit each object, check its type, do the maths and build a new object, one at a time. An array stores raw numbers side by side, all the same type, so NumPy can hand the whole block to a tight loop written in the C language, which runs at the full speed of your processor. On a typical laptop, doubling a million numbers looks something like this: ```python big_list = list(range(1_000_000)) big_array = np.arange(1_000_000) %timeit [x * 2 for x in big_list] # roughly 40 ms %timeit big_array * 2 # roughly 0.7 ms: about 60 times faster ``` `%timeit` is a notebook-only helper (a "magic command") that runs a line many times and reports the typical duration. Your numbers will differ, and the gap is what matters. With the data sizes analysts handle, that gap is the difference between waiting a second and waiting a minute. ### Creating arrays ```python week = np.array([412.50, 389.90, 455.10, 501.75, 610.20, 720.40, 298.00]) day_numbers = np.arange(1, 8) # like range(): 1 to 7, stop excluded evenly_spaced = np.linspace(0, 1, 5) # 5 evenly spaced values from 0 to 1 empty_tally = np.zeros(4) # four zeros print(day_numbers) print(evenly_spaced) print(empty_tally) ``` **Output:** ```text [1 2 3 4 5 6 7] [0. 0.25 0.5 0.75 1. ] [0. 0. 0. 0.] ``` Every array knows a few facts about itself: ```python print(week.shape) # size along each dimension print(week.ndim) # number of dimensions print(week.size) # total number of elements print(week.dtype) # the single data type shared by every element ``` **Output:** ```text (7,) 1 7 float64 ``` `(7,)` reads as "7 elements along one dimension". The trailing comma is how Python writes a tuple with one item. `float64` means "64-bit floating-point number", the standard decimal type; `int64` is its whole-number cousin. You'll see these dtype names constantly in pandas. ### Worked example 2.2: a week of sales, vectorised ```python costs = np.array([250.00, 240.10, 270.30, 300.00, 350.60, 410.00, 190.50]) margin = week - costs # element by element margin_pct = margin / week * 100 # still element by element print(margin) print(margin_pct.round(1)) ``` **Output:** ```text [162.5 149.8 184.8 201.75 259.6 310.4 107.5 ] [39.4 38.4 40.6 40.2 42.5 43.1 36.1] ``` Two arrays of the same length combine position by position: Monday's sales minus Monday's costs, Tuesday's minus Tuesday's, and so on. No loop required. ### Summarising an array ```python print(week.sum()) print(week.mean()) print(week.max()) print(week.argmax()) # the POSITION of the largest value print(week.std().round(2)) ``` **Output:** ```text 3387.85 483.9785714285714 720.4 5 131.76 ``` `argmax()` answers a different question from `max()`: *where* is the biggest value? Position 5 is the sixth day, Saturday, since positions start at 0. The standard deviation here differs slightly from the `statistics.stdev` result in section 2.1 (131.76 vs 142.32). NumPy's `std()` treats the numbers as a *whole population* by default, while `statistics.stdev()` treats them as a *sample* from a larger population. Chapter 10 explains when to use which; for now, just know the two exist. #### A display quirk you'll meet in notebooks Type a summary as the last line of a cell and the notebook shows something slightly surprising: ```python week.mean() ``` **Output:** ```text np.float64(483.9785714285714) ``` Since NumPy 2, displaying a single NumPy number also shows its type: `np.float64(...)` means "a NumPy 64-bit float with this value". It behaves exactly like a normal Python float. `print()` shows the bare number, as you saw above, and `float(week.mean())` converts it to a plain Python float. Older tutorials, written before NumPy 2, show only the bare number. ### Boolean masks: the most useful pattern in data analysis Comparing an array to a value produces an array of `True`/`False`, one per element: ```python target = 450 week > target ``` **Output:** ```text array([False, False, True, True, True, True, False]) ``` That True/False array is called a **boolean mask**. It has two superpowers. **1. Use it to select only the matching elements:** ```python week[week > target] ``` **Output:** ```text array([455.1 , 501.75, 610.2 , 720.4 ]) ``` Read `week[week > target]` as "the values of `week`, *where* `week` is above the target". Only positions marked True survive. **2. Count or measure with it.** In arithmetic, True counts as 1 and False as 0: ```python print((week > target).sum()) # how many days beat the target print((week > target).mean()) # what fraction of days beat the target ``` **Output:** ```text 4 0.5714285714285714 ``` Four days out of seven, or about 57%. The same "sum a mask to count, average it to get a proportion" trick works in pandas, and you'll use it weekly. #### Combining conditions Use `&` for "and", `|` for "or", and `~` for "not". **Wrap each condition in brackets**: ```python weekend = np.array([False, False, False, False, False, True, True]) week[(week > target) & ~weekend] # strong weekdays ``` **Output:** ```text array([455.1 , 501.75, 610.2 ]) ``` Why the brackets? In Python, `&` is worked out *before* `>`, so without brackets Python would try to calculate `target & ~weekend` first and fail. Brackets make the order explicit. (The words `and`/`or` from Chapter 1 work only on single True/False values; for arrays you need the symbols.) ### Choosing between two values: `np.where` `np.where(condition, value_if_true, value_if_false)` builds a new array element by element: ```python np.where(week > target, "Above", "Below") ``` **Output:** ```text array(['Below', 'Below', 'Above', 'Above', 'Above', 'Above', 'Below'], dtype=' 500]) ``` **Output:** ```text 720.4 Thu 501.75 Fri 610.20 Sat 720.40 Name: daily_sales, dtype: float64 ``` ### DataFrames: whole tables You can build a DataFrame from the exact structure you used in Chapter 1: a **list of dictionaries**, where each dictionary is one row. ### Worked example 2.3: Chapter 1's sales, now as a DataFrame ```python sales_records = [ {"order_id": 1001, "branch": "Petone", "product": "Vapour Rub 50g", "unit_price": 9.99, "quantity": 1, "loyalty_member": True}, {"order_id": 1002, "branch": "Lambton Quay", "product": "Throat Lozenges 16pk", "unit_price": 8.99, "quantity": 2, "loyalty_member": True}, {"order_id": 1003, "branch": "Karori", "product": "Throat Lozenges 16pk", "unit_price": 8.99, "quantity": 4, "loyalty_member": False}, {"order_id": 1004, "branch": "Karori", "product": "Throat Lozenges 16pk", "unit_price": 8.99, "quantity": 1, "loyalty_member": True}, {"order_id": 1005, "branch": "Petone", "product": "Plasters 40pk", "unit_price": 5.99, "quantity": 1, "loyalty_member": True}, {"order_id": 1006, "branch": "Lambton Quay", "product": "Multivitamin 60tab", "unit_price": 24.99, "quantity": 1, "loyalty_member": False}, {"order_id": 1007, "branch": "Cuba Street", "product": "Ibuprofen 200mg 24pk", "unit_price": 6.90, "quantity": 1, "loyalty_member": True}, {"order_id": 1008, "branch": "Cuba Street", "product": "Cold & Flu Capsules 24pk", "unit_price": 14.50, "quantity": 3, "loyalty_member": False}, ] df = pd.DataFrame(sales_records) df ``` **Output:** ```text order_id branch product unit_price quantity loyalty_member 0 1001 Petone Vapour Rub 50g 9.99 1 True 1 1002 Lambton Quay Throat Lozenges 16pk 8.99 2 True 2 1003 Karori Throat Lozenges 16pk 8.99 4 False 3 1004 Karori Throat Lozenges 16pk 8.99 1 True 4 1005 Petone Plasters 40pk 5.99 1 True 5 1006 Lambton Quay Multivitamin 60tab 24.99 1 False 6 1007 Cuba Street Ibuprofen 200mg 24pk 6.90 1 True 7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False ``` The dictionary keys became column names, and pandas added a default index of 0 to 7 down the left. (You can also build a DataFrame from a **dictionary of lists**, one list per column: `pd.DataFrame({"branch": [...], "quantity": [...]})`. Both are common.) ### The first-look toolkit Whenever you meet a new DataFrame, these four tell you what you're dealing with: ```python print(df.shape) # (rows, columns) print(df.columns.tolist()) ``` **Output:** ```text (8, 6) ['order_id', 'branch', 'product', 'unit_price', 'quantity', 'loyalty_member'] ``` ```python df.dtypes ``` **Output:** ```text order_id int64 branch str product str unit_price float64 quantity int64 loyalty_member bool dtype: object ``` Each column has one dtype. `int64` and `float64` are NumPy's number types; `bool` is True/False; and **`str`** marks text. > **pandas 3 note:** the `str` dtype for text is new in pandas 3.0, released January 2026. Tutorials and books written for earlier versions show text columns as `object`. Both mean "this column holds text"; the new `str` type is stricter (it only ever holds text or missing values) and faster. ```python df.info() ``` **Output:** ```text RangeIndex: 8 entries, 0 to 7 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 order_id 8 non-null int64 1 branch 8 non-null str 2 product 8 non-null str 3 unit_price 8 non-null float64 4 quantity 8 non-null int64 5 loyalty_member 8 non-null bool dtypes: bool(1), float64(1), int64(2), str(2) memory usage: 679.0 bytes ``` `info()` combines shape, column names, dtypes and, crucially, the **non-null count**: how many values in each column are present (not missing). When a column shows fewer non-null values than there are rows, you've found missing data. All 8 are present here; Chapter 3's real dataset won't be so tidy. ```python df.describe() ``` **Output:** ```text order_id unit_price quantity count 8.00000 8.000000 8.000000 mean 1004.50000 11.167500 1.750000 std 2.44949 6.127178 1.164965 min 1001.00000 5.990000 1.000000 25% 1002.75000 8.467500 1.000000 50% 1004.50000 8.990000 1.000000 75% 1006.25000 11.117500 2.250000 max 1008.00000 24.990000 4.000000 ``` `describe()` summarises every numeric column: count, mean, standard deviation, minimum, the 25th/50th/75th percentiles (the 50% row is the median) and maximum. It's a thirty-second health check. Here it says, for example, that the median sale is 1 unit (the `50%` row), the average is 1.75 units, and the largest is 4 units. (The `order_id` summary is meaningless, since IDs are labels rather than quantities. Ignoring columns like that is part of reading `describe()` sensibly.) ### Selecting columns ```python df["product"] ``` **Output:** ```text 0 Vapour Rub 50g 1 Throat Lozenges 16pk 2 Throat Lozenges 16pk 3 Throat Lozenges 16pk 4 Plasters 40pk 5 Multivitamin 60tab 6 Ibuprofen 200mg 24pk 7 Cold & Flu Capsules 24pk Name: product, dtype: str ``` Single brackets with one column name give you a **Series**. Double brackets give you a **DataFrame** with just those columns: ```python df[["branch", "product", "quantity"]] ``` **Output:** ```text branch product quantity 0 Petone Vapour Rub 50g 1 1 Lambton Quay Throat Lozenges 16pk 2 2 Karori Throat Lozenges 16pk 4 3 Karori Throat Lozenges 16pk 1 4 Petone Plasters 40pk 1 5 Lambton Quay Multivitamin 60tab 1 6 Cuba Street Ibuprofen 200mg 24pk 1 7 Cuba Street Cold & Flu Capsules 24pk 3 ``` Why double? The outer brackets mean "select from `df`", and the inner brackets are a Python **list** of the columns you want. Chapter 3 covers selection in much more depth. ### Creating a column: vectorised, no loop In Chapter 1 you wrote a `line_total()` function and called it inside a loop. In pandas: ```python df["revenue"] = df["unit_price"] * df["quantity"] df[["product", "unit_price", "quantity", "revenue"]] ``` **Output:** ```text product unit_price quantity revenue 0 Vapour Rub 50g 9.99 1 9.99 1 Throat Lozenges 16pk 8.99 2 17.98 2 Throat Lozenges 16pk 8.99 4 35.96 3 Throat Lozenges 16pk 8.99 1 8.99 4 Plasters 40pk 5.99 1 5.99 5 Multivitamin 60tab 24.99 1 24.99 6 Ibuprofen 200mg 24pk 6.90 1 6.90 7 Cold & Flu Capsules 24pk 14.50 3 43.50 ``` One line. pandas multiplied the two columns row by row (vectorisation again) and stored the result as a new column called `revenue`. ### Filtering rows with a boolean mask Exactly the same pattern as NumPy: ```python bulk = df["quantity"] >= 3 df[bulk] ``` **Output:** ```text order_id branch product unit_price quantity loyalty_member revenue 2 1003 Karori Throat Lozenges 16pk 8.99 4 False 35.96 7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False 43.50 ``` And combined conditions, with `&`, `|`, `~` and brackets around each condition: ```python df[(df["branch"] == "Karori") & df["loyalty_member"]] ``` **Output:** ```text order_id branch product unit_price quantity loyalty_member revenue 3 1004 Karori Throat Lozenges 16pk 8.99 1 True 8.99 ``` `df["loyalty_member"]` is already a column of True/False values, so it can be used as a condition directly. ### Sorting and counting ```python df.sort_values("revenue", ascending=False).head(3) ``` **Output:** ```text order_id branch product unit_price quantity loyalty_member revenue 7 1008 Cuba Street Cold & Flu Capsules 24pk 14.50 3 False 43.50 2 1003 Karori Throat Lozenges 16pk 8.99 4 False 35.96 5 1006 Lambton Quay Multivitamin 60tab 24.99 1 False 24.99 ``` `sort_values()` returns a new, sorted DataFrame, and `.head(3)` keeps the first three rows. Chaining methods like this (do one thing, then the next) is very common in pandas code. ```python df["branch"].value_counts() ``` **Output:** ```text branch Petone 2 Lambton Quay 2 Karori 2 Cuba Street 2 Name: count, dtype: int64 ``` `value_counts()` counts how often each distinct value appears, most frequent first. It's the pandas version of the counting loop you wrote in Chapter 1. In this small sample every branch appears twice, so the tie is listed in order of first appearance; with real data the differences are usually more interesting. ### Worked example 2.4: Chapter 1's summary, in pandas Here's the payoff. The revenue-by-branch table from your Chapter 1 milestone script took a loop, a dictionary and careful bookkeeping: ```python df.groupby("branch")["revenue"].sum() ``` **Output:** ```text branch Cuba Street 50.40 Karori 44.95 Lambton Quay 42.97 Petone 15.98 Name: revenue, dtype: float64 ``` Same numbers as Chapter 1 (Cuba Street 50.40, Karori 44.95, Lambton Quay 42.97, Petone 15.98), in one line. Read it as: *group the rows by `branch`, take the `revenue` column within each group, and add it up*. Chapter 6 is devoted to `groupby`, because it answers a huge share of business questions. The rest of the Chapter 1 report takes a line each: ```python total = df["revenue"].sum() loyalty_share = df.loc[df["loyalty_member"], "revenue"].sum() / total best_seller = df.groupby("product")["quantity"].sum().idxmax() print(f"Total revenue: ${total:,.2f}") print(f"Loyalty share: {loyalty_share:.1%}") print(f"Best seller: {best_seller}") ``` **Output:** ```text Total revenue: $154.30 Loyalty share: 32.3% Best seller: Throat Lozenges 16pk ``` Two new tools here: - **`df.loc[rows, column]`** selects by *row condition and column name at the same time*. Here: the rows where `loyalty_member` is True, and only the `revenue` column. Chapter 3 explains `.loc` fully. - **`idxmax()`** returns the *label* of the largest value, the pandas cousin of NumPy's `argmax()`. ### Changing values safely with `.loc` To change some values in a column, based on a condition, use `.loc` with the condition and the column name together: ```python df["discount"] = 0.0 df.loc[df["loyalty_member"], "discount"] = 0.05 df[["order_id", "loyalty_member", "discount"]] ``` **Output:** ```text order_id loyalty_member discount 0 1001 True 0.05 1 1002 True 0.05 2 1003 False 0.00 3 1004 True 0.05 4 1005 True 0.05 5 1006 False 0.00 6 1007 True 0.05 7 1008 False 0.00 ``` Read it as: *in the rows where the customer is a loyalty member, set `discount` to 0.05*. > **pandas 3 note: always assign with a single `.loc`.** Older tutorials sometimes write `df[df["loyalty_member"]]["discount"] = 0.05`, two sets of brackets one after another (called *chained assignment*). In pandas 3 that line **never** changes `df`. It changes a temporary copy that's immediately thrown away, and pandas prints a `ChainedAssignmentError` warning to tell you so. This comes from a pandas 3 design called **Copy-on-Write**: every selection behaves like an independent copy, which also means the old `SettingWithCopyWarning` you'll see complained about online has been removed. The rule to remember is simple: **to change values in place, use one `.loc[rows, column] = value`.** ### Missing values in pandas pandas shows missing values as `NaN` (or `None`/`` in some dtypes), and, helpfully, its summaries **skip them automatically**: ```python quantities = pd.Series([1, 2, None, 4]) print(quantities) print("Sum:", quantities.sum()) print("Missing:", quantities.isna().sum()) ``` **Output:** ```text 0 1.0 1 2.0 2 NaN 3 4.0 dtype: float64 Sum: 7.0 Missing: 1 ``` Two things to notice. First, the column became `float64` because of the gap, just as you saw with NumPy. Second, `.isna()` makes a boolean mask of "is this missing?", and `.sum()` counts the Trues: the same mask-and-count trick from earlier. Chapter 4 covers deciding what to *do* about missing values. --- ## 2.6 Matplotlib: drawing charts ### The idea **Matplotlib** is Python's foundational charting library. Nearly every other Python charting tool, including Seaborn and pandas' own `.plot()`, draws through it. Two words form the mental model: - A **Figure** is the whole canvas: the image you'd save or paste into a report. - An **Axes** is one chart inside that figure: the plotting area together with its x-axis, y-axis, title and labels. A figure can hold one Axes or several. "Axes" (the chart) and "axis" (one of the lines with tick marks) are different things, confusingly close in spelling. You'll mostly talk to the Axes. ### The recommended style The Matplotlib documentation recommends this pattern, and it's the one we'll use throughout the course: ```text fig, ax = plt.subplots() # 1. make a Figure with one Axes ax.plot(...) # 2. draw on the Axes ax.set_title(...) # 3. label it fig.savefig(...) # 4. save it (optional) plt.show() # 5. display it ``` You'll also see shorter code in tutorials that calls `plt.plot(...)` directly, relying on Matplotlib to keep track of "the current chart" behind the scenes. That shortcut works for quick one-off charts. The `fig, ax` style is more explicit, and it scales cleanly when you need two or more charts side by side, so it's worth learning first. ### Worked example 2.5: a line chart of the week ```python import matplotlib.pyplot as plt days = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] fig, ax = plt.subplots(figsize=(8, 4)) ax.plot(days, week, marker="o", label="Daily sales") ax.axhline(target, color="grey", linestyle="--", label="Target ($450)") ax.set_title("Harbour Pharmacy: sales by day") ax.set_xlabel("Day") ax.set_ylabel("Sales (NZD)") ax.legend() fig.savefig("../output/ch02_week_line.png", dpi=150, bbox_inches="tight") plt.show() ``` What each line does: | Code | Effect | |---|---| | `plt.subplots(figsize=(8, 4))` | a figure 8 inches wide and 4 tall, containing one Axes | | `ax.plot(days, week, marker="o")` | a line through the seven points, with a dot on each | | `label="..."` | the name to show in the legend | | `ax.axhline(target, ...)` | a **h**orizontal reference **line** across the chart at the target | | `set_title`, `set_xlabel`, `set_ylabel` | the chart title and axis labels | | `ax.legend()` | draw the legend, using the labels given above | | `fig.savefig(...)` | save as a PNG image in your `output/` folder; `dpi=150` sets the sharpness and `bbox_inches="tight"` trims empty margins | | `plt.show()` | display the chart | You'll see a line rising from Monday to a Saturday peak, then dropping sharply on Sunday, with a dashed grey target line that four of the dots sit above. The `../output/` path means "go up one folder from where this notebook lives (`ch02/`), then into `output/`". Chapter 3 explains file paths properly. > **Why end chart cells with `plt.show()`?** In a notebook the chart appears anyway, and without `plt.show()` the cell also prints the value of its last line: something like ``. It's harmless, just noisy. In a `.py` script, `plt.show()` is what actually opens the chart window. ### Worked example 2.6: a bar chart from a pandas result Charts usually come straight from pandas results. Let's chart the four branches' weekly totals from the 2-D array: ```python branch_names = ["Lambton Quay", "Cuba Street", "Petone", "Karori"] branch_totals = pd.Series(branch_week.sum(axis=1), index=branch_names) fig, ax = plt.subplots(figsize=(7, 4)) ax.bar(branch_totals.index, branch_totals.values, color="steelblue") ax.set_title("Weekly sales by branch") ax.set_ylabel("Sales (NZD)") plt.show() ``` `ax.bar(x_positions, heights)` draws one bar per branch. `.index` gives the Series' labels and `.values` gives its numbers. pandas also has a `.plot()` shortcut that does the same job and fills in some labels for you: ```python fig, ax = plt.subplots(figsize=(7, 4)) branch_totals.sort_values().plot(kind="barh", ax=ax, color="steelblue") ax.set_title("Weekly sales by branch") ax.set_xlabel("Sales (NZD)") plt.show() ``` `kind="barh"` makes a **h**orizontal bar chart, which is often easier to read when the category names are long, and sorting first puts the bars in order. Passing `ax=ax` tells pandas to draw on *your* Axes, so you keep full control over the labels. ### Worked example 2.7: a histogram, and two charts side by side A **histogram** shows how values are *distributed*: it splits the range into "bins" and counts how many values fall in each. Let's simulate 500 transaction values and look at their shape, next to a second chart: ```python rng = np.random.default_rng(seed=1) transaction_values = rng.gamma(shape=2.0, scale=12.0, size=500).round(2) fig, axes = plt.subplots(1, 2, figsize=(11, 4)) axes[0].hist(transaction_values, bins=30, color="teal", edgecolor="white") axes[0].set_title("Transaction values (500 simulated sales)") axes[0].set_xlabel("Value (NZD)") axes[0].set_ylabel("Number of transactions") axes[1].plot(days, week.cumsum(), marker="o") axes[1].set_title("Running total across the week") axes[1].set_ylabel("Cumulative sales (NZD)") fig.tight_layout() plt.show() ``` - `plt.subplots(1, 2)` makes **1 row and 2 columns** of Axes, returned as an array called `axes`. `axes[0]` is the left chart and `axes[1]` the right. - `rng.gamma(...)` produces skewed, all-positive values, a realistic shape for purchase amounts: most transactions are small, with a long tail of bigger ones. The histogram shows a hump near $10–$20 and a tail stretching to the right. - `.cumsum()` is a **cumulative sum**, a running total. - `fig.tight_layout()` nudges the charts apart so their labels don't overlap. Chapter 9 is dedicated to choosing and polishing charts. For now, the pattern `fig, ax = plt.subplots()` → draw → label → show is the one to have in your fingers. --- ## 2.7 Seaborn: statistical charts with sensible defaults ### The idea **Seaborn** is built on Matplotlib and designed to work directly with pandas DataFrames. You tell it *which columns* to use for x, y and colour, and it handles the grouping, summarising and legends for you. Its defaults are also more attractive than Matplotlib's. ```python import seaborn as sns sns.set_theme(style="whitegrid") # a clean look for every chart from now on ``` ### Worked example 2.8: revenue by branch, split by loyalty ```python fig, ax = plt.subplots(figsize=(8, 4)) sns.barplot( data=df, x="branch", y="revenue", hue="loyalty_member", estimator="sum", errorbar=None, ax=ax, ) ax.set_title("Revenue by branch and loyalty status") ax.set_ylabel("Revenue (NZD)") plt.show() ``` | Setting | Meaning | |---|---| | `data=df` | the DataFrame to draw from | | `x="branch"`, `y="revenue"` | the columns for each axis | | `hue="loyalty_member"` | split each bar into colours by this column, with a legend added automatically | | `estimator="sum"` | add up the revenue within each group | | `errorbar=None` | don't draw error bars | | `ax=ax` | draw on our Axes | > **An important Seaborn default to know:** `barplot` shows the **mean** of each group, with a black **error bar** showing a 95% confidence interval (a range that expresses uncertainty about that mean; Chapter 12 explains them). That's a sensible default for scientific data and a frequent trap for business reporting, where people usually expect bar heights to be **totals**. Setting `estimator="sum"` and `errorbar=None` gives you the plain totals most stakeholders expect. ### Worked example 2.9: a distribution in one line ```python fig, ax = plt.subplots(figsize=(7, 4)) sns.histplot(x=transaction_values, bins=30, kde=True, ax=ax) ax.set_title("Transaction values") ax.set_xlabel("Value (NZD)") plt.show() ``` `kde=True` adds a smooth curve over the bars (a *kernel density estimate*: a smoothed outline of the histogram's shape), which makes the overall pattern easier to see. As a rule of thumb: reach for **Seaborn** for quick, good-looking charts built straight from a DataFrame, especially when you want to split by category. Reach for **Matplotlib** directly when you need fine control over every detail. Because Seaborn draws on a Matplotlib Axes, you can always mix the two, as you just did with `ax.set_title()`. --- ## 2.8 The wider toolbox You won't install these yet. They're here so the names feel familiar when they appear: | Library | What it does | Where it appears | |---|---|---| | `openpyxl` | Read and write Excel files (already installed) | Chapter 3, Chapter 13 | | `sqlite3` (standard library), `SQLAlchemy` | Talk to databases using SQL | Chapter 11 | | `SciPy` | Scientific functions, including statistical tests | Chapter 12 | | `statsmodels` | Statistical models and tests with detailed reports | Chapter 12 | | `Plotly`, `Streamlit` | Interactive charts and simple web dashboards | Chapter 14 | | `requests` | Fetch data from websites and APIs | Later chapters, as needed | | `scikit-learn` | Machine learning: prediction and classification | A natural next step after this course | | `Polars` | A newer, very fast DataFrame library; an alternative to pandas | Worth a look once pandas feels comfortable | --- ## 2.9 Project milestone: Harbour Pharmacy's first notebook Time to put the toolkit together. The operations manager wants to see a whole month at a glance: how each branch performed in total, their best days, and how often each branch cleared a $1,000-a-day target. You don't have the real month's data yet (that arrives in Chapter 3), so you'll **simulate** a realistic month with NumPy. Creating plausible fake data is a genuine analyst skill: it lets you build and test an analysis before the real data turns up. Create a new notebook, `ch02/ch02_milestone.ipynb`. Put each block below in its own cell, and add a Markdown cell above each one with a short heading describing what it does (for example, `## 1. Simulate the month`). **Cell 1: imports and settings** ```python import matplotlib.pyplot as plt import numpy as np import pandas as pd import seaborn as sns sns.set_theme(style="whitegrid") ``` **Cell 2: simulate 28 days of sales for four branches** ```python rng = np.random.default_rng(seed=7) dates = pd.date_range("2026-03-02", periods=28, freq="D") typical_day = {"Lambton Quay": 1450, "Cuba Street": 1150, "Petone": 900, "Karori": 700} daily = pd.DataFrame(index=dates) for branch, typical in typical_day.items(): daily[branch] = rng.normal(loc=typical, scale=typical * 0.15, size=len(dates)).round(2) daily.index.name = "date" daily.head() ``` **Output:** ```text Lambton Quay Cuba Street Petone Karori date 2026-03-02 1450.27 1141.63 891.02 679.29 2026-03-03 1514.98 1169.55 990.08 583.02 2026-03-04 1390.38 886.05 1094.20 698.79 2026-03-05 1256.30 1067.59 808.79 653.42 2026-03-06 1351.11 981.21 927.42 822.44 ``` What's happening: - `pd.date_range("2026-03-02", periods=28, freq="D")` creates 28 consecutive **d**ates starting on Monday 2 March. (Dates get a whole chapter, Chapter 8.) - `pd.DataFrame(index=dates)` makes an empty table whose row labels are those dates. - The loop adds one column per branch. Each column is 28 bell-curve values centred on that branch's typical day, with a spread of 15%. The fixed seed means you'll get exactly the numbers shown here. **Cell 3: headline numbers** ```python monthly_totals = daily.sum().sort_values(ascending=False) monthly_totals.round(2) ``` **Output:** ```text Lambton Quay 37884.11 Cuba Street 31952.05 Petone 25107.09 Karori 19453.71 dtype: float64 ``` `daily.sum()` adds down each column (pandas' default is `axis=0`, the "collapse down" direction you met with NumPy), giving one total per branch. **Cell 4: each branch's best day, and days on target** ```python target = 1000 summary = pd.DataFrame({ "total": daily.sum().round(2), "daily_average": daily.mean().round(2), "best_day": daily.idxmax().dt.strftime("%a %d %b"), "days_on_target": (daily >= target).sum(), }) summary.sort_values("total", ascending=False) ``` **Output:** ```text total daily_average best_day days_on_target Lambton Quay 37884.11 1353.00 Mon 09 Mar 27 Cuba Street 31952.05 1141.14 Mon 23 Mar 23 Petone 25107.09 896.68 Wed 25 Mar 5 Karori 19453.71 694.78 Sat 21 Mar 0 ``` Take this one slowly; it combines several ideas: - Building a DataFrame from a **dictionary of Series**: each key becomes a column, and pandas lines the rows up by their shared labels (the branch names). - `daily.idxmax()` finds the date label of each column's largest value. `.dt.strftime("%a %d %b")` formats those dates as text such as `Sat 21 Mar`. (`%a` = short weekday, `%d` = day number, `%b` = short month; Chapter 8 lists the rest.) - `(daily >= target)` compares the **whole table** to 1,000 at once, producing a table of True/False. `.sum()` then counts the Trues in each column: the NumPy mask-counting trick, applied to every branch at once. Notice Karori shows zero days on target. With a typical day of $700, a single $1,000 target is unfair on smaller branches, which is exactly the kind of observation an analyst raises. (Exercise 2.8 asks you to fix it.) **Cell 5: charts, saved for the report** ```python fig, axes = plt.subplots(1, 2, figsize=(13, 4.5)) for branch in daily.columns: axes[0].plot(daily.index, daily[branch], label=branch, linewidth=1.5) axes[0].axhline(target, color="grey", linestyle="--", linewidth=1) axes[0].set_title("Daily sales by branch, March 2026 (simulated)") axes[0].set_ylabel("Sales (NZD)") axes[0].tick_params(axis="x", labelrotation=45) axes[0].legend(ncols=2, fontsize=9) monthly_totals.sort_values().plot(kind="barh", ax=axes[1], color="steelblue") axes[1].set_title("Total sales over 28 days") axes[1].set_xlabel("Sales (NZD)") axes[1].set_ylabel("") fig.tight_layout() fig.savefig("../output/ch02_milestone.png", dpi=150, bbox_inches="tight") plt.show() ``` The left chart loops over the branch columns and draws one line for each, labelled with the branch name so the legend knows what to call it. Matplotlib recognises that the x values are dates and spaces the date labels sensibly; `tick_params(axis="x", labelrotation=45)` tilts them so they don't collide. (pandas can draw all four lines in one go with `daily.plot(ax=axes[0])`, and it's handy for a quick look; on daily dates its automatic labels can get crowded, which is why we're drawing the lines ourselves here.) What you'll see: on the left, four wiggly lines in clear bands, with Lambton Quay on top and Karori at the bottom, and the dashed target line running between Cuba Street and Petone. On the right, four horizontal bars ranked by total. **Cell 6 (Markdown): findings** End the notebook with a Markdown cell, written in plain language for the operations manager. For example: ```text ## Findings (simulated data) - Lambton Quay generated the most revenue over the 28 days, about $37.9k, nearly double Karori's $19.5k. - Lambton Quay beat the $1,000 daily target on 27 of 28 days; Karori never did. - A single company-wide target doesn't suit branches of such different sizes. Next step: set targets relative to each branch's typical day. ``` Finally, **Restart Kernel and Run All** to prove the notebook runs cleanly from top to bottom. Check that `output/` now contains `ch02_week_line.png` and `ch02_milestone.png`. Writing findings in plain words at the end of an analysis is a habit worth starting now. Stakeholders read the conclusions; the code is how you earn the right to state them. --- ## 2.10 Exercises Work in a notebook, `ch02/ch02_exercises.ipynb`, with a Markdown heading for each exercise. Attempt each before opening the solution. **Available this chapter: 250 XP.** ### Warm-up (10 XP each) **Exercise 2.1: Environment health check.** In a notebook cell, print `sys.prefix` and the pandas and NumPy versions. Then, in the terminal, regenerate `requirements.txt` and confirm `pandas==` appears in it.

Show a solution

```python import sys import numpy as np import pandas as pd print(sys.prefix) print("pandas", pd.__version__) print("NumPy", np.__version__) ``` The path should end in `.venv`. Then, in the terminal with the environment active: ```bash python -m pip freeze > requirements.txt ``` Open `requirements.txt` in VS Code and search for `pandas==` (`Ctrl+F`).

**Exercise 2.2: Stock check with NumPy.** Create an array of stock counts, `stock = np.array([120, 9, 0, 31, 18, 64, 3])`, and a reorder point of 25. Using boolean masks, print (a) how many products need reordering (below 25, including zero), and (b) the stock counts of those products.

Show a solution

```python import numpy as np stock = np.array([120, 9, 0, 31, 18, 64, 3]) reorder_point = 25 needs_reorder = stock < reorder_point print(needs_reorder.sum()) print(stock[needs_reorder]) ``` **Output:** ```text 4 [ 9 0 18 3] ```

**Exercise 2.3: A price list Series.** Create a Series of these prices, indexed by product name: Paracetamol 500mg 20pk 4.50, Vitamin C 1000mg 60tab 18.99, Sunscreen SPF50 200ml 22.99, Plasters 40pk 5.99. Add 15% GST (rounded to 2 decimals) to make a new Series, then show only the products whose GST-inclusive price is over $10.

Show a solution

```python import pandas as pd prices = pd.Series( [4.50, 18.99, 22.99, 5.99], index=["Paracetamol 500mg 20pk", "Vitamin C 1000mg 60tab", "Sunscreen SPF50 200ml", "Plasters 40pk"], ) prices_incl = (prices * 1.15).round(2) prices_incl[prices_incl > 10] ``` **Output:** ```text Vitamin C 1000mg 60tab 21.84 Sunscreen SPF50 200ml 26.44 dtype: float64 ```

### Core (20 XP each) **Exercise 2.4: Your own DataFrame.** Build a DataFrame from this dictionary of lists, add a `revenue` column, keep only the Cold & Flu rows, and sort them by revenue from highest to lowest. ```python data = { "product": ["Throat Lozenges 16pk", "Paracetamol 500mg 20pk", "Vapour Rub 50g", "Cold & Flu Capsules 24pk", "Vitamin C 1000mg 60tab", "Throat Lozenges 16pk"], "category": ["Cold & Flu", "Pain Relief", "Cold & Flu", "Cold & Flu", "Vitamins", "Cold & Flu"], "unit_price": [8.99, 4.50, 9.99, 14.50, 18.99, 8.99], "quantity": [2, 3, 4, 1, 2, 5], } ```

Show a solution

```python products = pd.DataFrame(data) products["revenue"] = products["unit_price"] * products["quantity"] cold_flu = products[products["category"] == "Cold & Flu"] cold_flu.sort_values("revenue", ascending=False) ``` **Output:** ```text product category unit_price quantity revenue 5 Throat Lozenges 16pk Cold & Flu 8.99 5 44.95 2 Vapour Rub 50g Cold & Flu 9.99 4 39.96 0 Throat Lozenges 16pk Cold & Flu 8.99 2 17.98 3 Cold & Flu Capsules 24pk Cold & Flu 14.50 1 14.50 ```

**Exercise 2.5: Group and total.** Using `products` from Exercise 2.4, show the total quantity sold per category, largest first. Then show the total revenue per category.

Show a solution

```python products.groupby("category")["quantity"].sum().sort_values(ascending=False) ``` **Output:** ```text category Cold & Flu 12 Pain Relief 3 Vitamins 2 Name: quantity, dtype: int64 ``` ```python products.groupby("category")["revenue"].sum().round(2) ``` **Output:** ```text category Cold & Flu 117.39 Pain Relief 13.50 Vitamins 37.98 Name: revenue, dtype: float64 ```

**Exercise 2.6: Chart it.** Draw a horizontal bar chart of revenue per category (from Exercise 2.5), with a title and axis label, and save it as `output/ch02_ex6.png`.

Show a solution

```python import matplotlib.pyplot as plt revenue_by_category = products.groupby("category")["revenue"].sum().sort_values() fig, ax = plt.subplots(figsize=(7, 3.5)) revenue_by_category.plot(kind="barh", ax=ax, color="steelblue") ax.set_title("Revenue by category") ax.set_xlabel("Revenue (NZD)") ax.set_ylabel("") fig.savefig("../output/ch02_ex6.png", dpi=150, bbox_inches="tight") plt.show() ``` `ax.set_ylabel("")` removes the automatic "category" label, which adds nothing when the bar names are already there.

### Stretch (40 XP each) **Exercise 2.7: Two dimensions.** Using `branch_week` from section 2.4 (4 branches × 7 days), and without any loops, find: (a) each branch's weekly total, (b) each day's total across all branches, (c) the position of each branch's best day, and (d) that best day's name. Hint: `argmax` also accepts `axis=`, and a NumPy array of day names can be indexed with an array of positions.

Show a solution

```python branch_week = np.array([ [520, 480, 610, 590, 700, 820, 400], [410, 395, 450, 470, 600, 760, 380], [300, 310, 290, 350, 420, 510, 330], [220, 230, 260, 240, 300, 380, 250], ]) day_names = np.array(["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]) print(branch_week.sum(axis=1)) # (a) per branch print(branch_week.sum(axis=0)) # (b) per day best_positions = branch_week.argmax(axis=1) print(best_positions) # (c) print(day_names[best_positions]) # (d) ``` **Output:** ```text [4120 3465 2510 1880] [1450 1415 1610 1650 2020 2470 1360] [5 5 5 5] ['Sat' 'Sat' 'Sat' 'Sat'] ``` Every branch peaks on Saturday. Indexing `day_names` with an *array* of positions (`day_names[[5, 5, 5, 5]]`) returns the name at each position; this is called **fancy indexing**.

**Exercise 2.8: Fairer targets, and a rebuild drill.** 1. In your milestone notebook, replace the single $1,000 target with a target for each branch equal to **110% of its own daily average**, and count how many days each branch hit its own target. (Hint: `daily.mean() * 1.1` gives one target per branch, and comparing a DataFrame to a Series lines them up by column name.) 2. Then prove your environment is reproducible: close VS Code/JupyterLab, delete the `.venv` folder, recreate it (section 1.2.4), reinstall with `python -m pip install -r requirements.txt`, reopen the milestone notebook, select the new `.venv` kernel and **Restart and Run All**. Your numbers should be identical.

Show a solution

```python branch_targets = (daily.mean() * 1.1).round(2) print(branch_targets) days_on_own_target = (daily >= branch_targets).sum() days_on_own_target ``` **Output:** ```text Lambton Quay 1488.30 Cuba Street 1255.26 Petone 986.35 Karori 764.25 dtype: float64 Lambton Quay 6 Cuba Street 7 Petone 8 Karori 6 dtype: int64 ``` When you compare a DataFrame with a Series, pandas matches the Series' labels to the DataFrame's **column names**, so each branch is compared with its own target. This automatic lining-up by label is called **alignment**, and it's one of pandas' quiet superpowers. Now every branch has a comparable measure of "a strong day". For part 2, identical numbers after a full rebuild are the proof: your seed fixed the random data, and `requirements.txt` fixed the library versions.

### Quick self-check (20 XP for all five) 1. What's the difference between `[1, 2, 3] * 2` and `np.array([1, 2, 3]) * 2`? 2. Why is `python -m pip install` safer than `pip install`? 3. In `df[df["quantity"] >= 3]`, what does the inner part produce, and what does the outer part do with it? 4. On a 2-D array, which `axis` gives you one result per row? 5. Why does `df[df["loyalty_member"]]["discount"] = 0.05` fail to change `df` in pandas 3, and what should you write instead?

Show answers

1. The list repeats itself: `[1, 2, 3, 1, 2, 3]`. The array doubles each element: `[2, 4, 6]` (vectorisation). 2. It guarantees you use the pip belonging to the `python` your terminal finds first, which is your active environment's. Plain `pip` can belong to a different Python on machines with several installed. 3. The inner part produces a boolean mask (one True/False per row). The outer brackets keep only the rows marked True. 4. `axis=1` collapses across each row, giving one result per row. 5. It's chained assignment: the first brackets produce a temporary selection, and in pandas 3 (Copy-on-Write) changing that temporary result never flows back to `df`. Write `df.loc[df["loyalty_member"], "discount"] = 0.05`.

--- ## 2.11 Cheat sheet | Task | Code | |---|---| | Install into the active environment | `python -m pip install pandas` | | Record the environment | `python -m pip freeze > requirements.txt` | | Rebuild the environment | `python -m pip install -r requirements.txt` | | Start JupyterLab | `jupyter lab` | | Standard imports | `import numpy as np`, `import pandas as pd`, `import matplotlib.pyplot as plt`, `import seaborn as sns` | | Make an array | `np.array([...])`, `np.arange(a, b)`, `np.zeros(n)` | | Array facts | `.shape`, `.dtype`, `.ndim`, `.size` | | Summaries | `.sum()`, `.mean()`, `.max()`, `.argmax()`, `.std()` | | Boolean mask | `arr[arr > 10]`, `(arr > 10).sum()`, `(a > 1) & (b < 5)` | | If/else on a whole array | `np.where(cond, "yes", "no")` | | Along an axis | `axis=0` collapses down columns; `axis=1` collapses across rows | | Reproducible randomness | `rng = np.random.default_rng(seed=42)` | | Make a Series / DataFrame | `pd.Series(values, index=labels)`, `pd.DataFrame(list_of_dicts)` | | First look | `df.shape`, `df.head()`, `df.dtypes`, `df.info()`, `df.describe()` | | Select columns | `df["col"]` (Series), `df[["a", "b"]]` (DataFrame) | | New column | `df["revenue"] = df["price"] * df["qty"]` | | Filter rows | `df[df["qty"] >= 3]` | | Sort / count | `df.sort_values("col", ascending=False)`, `df["col"].value_counts()` | | Group and total | `df.groupby("branch")["revenue"].sum()` | | Change values safely | `df.loc[condition, "col"] = value` | | Missing values | `s.isna().sum()` | | Chart skeleton | `fig, ax = plt.subplots()` → `ax.plot/bar/hist(...)` → `ax.set_title(...)` → `plt.show()` | | Save a chart | `fig.savefig("../output/name.png", dpi=150, bbox_inches="tight")` | | Seaborn totals bar chart | `sns.barplot(data=df, x=..., y=..., estimator="sum", errorbar=None, ax=ax)` | --- ## 2.12 XP tally and what's next | Section | XP available | XP earned | |---|---|---| | Warm-up (2.1–2.3) | 30 | | | Core (2.4–2.6) | 60 | | | Stretch (2.7–2.8) | 80 | | | Milestone notebook runs cleanly with "Restart and Run All" | 60 | | | Self-check (all five correct) | 20 | | | **Chapter 2 total** | **250** | | | **Running total (Chapters 1–2)** | **440** | | **Coming up in Chapter 3: Loading, Inspecting and Selecting Data.** The real Harbour Pharmacy sales file arrives. You'll learn how file paths work, load CSV, Excel and JSON files with pandas, run a systematic first-look checklist that uncovers the problems hiding in real data, and master `.loc`, `.iloc` and boolean filtering to pull out exactly the rows and columns a question needs. --- ## References and further reading - NumPy developers. *NumPy: the absolute basics for beginners*. - NumPy developers. *NumPy 2.0 migration guide* (including the new scalar display). - pandas development team. *10 minutes to pandas*. - pandas development team. *Getting started tutorials*. - pandas development team. *What's new in 3.0.0 (January 21, 2026)*. - pandas development team. *Copy-on-Write (CoW)*. - pandas development team. *Migration guide for the new string data type (pandas 3.0)*. - Matplotlib development team. *Quick start guide* (Figures, Axes, and the recommended coding style). - Waskom, M. *Seaborn user guide and tutorial*. - Waskom, M. *seaborn.barplot* (estimator and errorbar options). - Project Jupyter. *JupyterLab documentation*. - Microsoft. *Jupyter Notebooks in VS Code*. - pip developers. *pip user guide: requirements files*. - McKinney, W. *Python for Data Analysis*, 3rd edition (free to read online; written for pandas 1.x–2.x, so expect `object` instead of `str` and older assignment patterns). - VanderPlas, J. *Python Data Science Handbook* (free online; older library versions, with excellent explanations of NumPy and Matplotlib).