Week 12: Nested Data Structures

🎯 Objectives

Last week, a dictionary's values were simple things: a count, a score, a letter. This week the values get bigger. A dictionary's value can be a list, a list's items can be dictionaries, and those dictionaries can hold more lists. That's nested data, and it's how almost all real data arrives, whether from a JSON file, a spreadsheet saved as CSV, or (soon, in your project) a website.

The new skill isn't a new piece of Python. data[2]["scores"][0] is just three lookups you already know, read left to right. The skill is keeping track of where you are: after each bracket, knowing whether you're holding a list, a dictionary, a string, or a number, and so what the next bracket is allowed to be.

By the end of the week you should be able to:

• Store rows and columns of data in a list of lists, pull one value out by its position, and walk the whole thing with a loop inside a loop.
• Navigate lists of dictionaries, dictionaries of lists, and similar combinations, saying the type at every step of an expression like data[2]["scores"][0].
• Read data from JSON and CSV files into memory, and say how the two shapes differ.
• Given a description of how some data is shaped, write code that filters it and pulls out the fields a question asks for.

That last one is the heart of the week, and it's a skill for the hand and the eye before the keyboard. So do the activities below with a pencil first, and use Thonny to check yourself afterward.

✅ Must do

These three are the core of the week. Do them first.

1. The in-class activities. If you missed a class, ask a classmate or your instructor what you missed and work through it on your own.
2. The reading: one of the options in "Read about it," below.
3. PA4: SoDuckYou Parts C, D, and E. Part C is a loop inside a loop, Part D reads levels.json, and Part E is your finished test_soduckyou.py. Submit both files to Gradescope as soon as Part C works, and again when the whole PA is done.

📚 Study menu (should do)

Class covered lists of lists and nested loops, then lists of dictionaries, then reading data files and answering questions about them. Everything below is optional, but each item trains something PA4 or the rest of the semester needs, and the project starts next week with data of your choosing. Pick what you need.

If you only have an hour

1. Fifteen minutes. "Follow the brackets," below, on paper.
2. Fifteen minutes. "Shape in words, shape in code," in both directions.
3. Thirty minutes. "Filter, then pull out," written on paper first, then typed and checked.

Read about it

Pick one option.

W3Schools: the short version

• Nested Dictionaries, the page you skipped last week, then Python JSON, which shows how JSON text turns into Python dictionaries and lists.

Python for Everybody: 15. Using Web Services

• 15.4 JavaScript Object Notation - JSON and 15.5 What is JSON?, then 9.19 Group Work: Lists for practice with lists inside lists.

Python for Everybody: 8. Files, if CSV is the part you want

• 8.16 Group Work: Reading from CSV Files, which splits each line on commas the way you did in Week 10, and 8.17 Group Work: Using a CSV reader/writer, which lets the csv module do the splitting.

Grid by hand

A list of lists stores rows and columns: the first index picks the row, and the second picks the item inside it. On paper, with this grid, write down what each expression gives, or the error if it crashes.

grid = [[3, 0, 2],
        [1, 4, 0],
        [0, 5, 6]]

Then trace this program, keeping a table with columns for r, c, total, and col_totals:

row_totals = []
col_totals = [0, 0, 0]
for r in range(len(grid)):
    total = 0
    for c in range(len(grid[r])):
        total += grid[r][c]
        col_totals[c] += grid[r][c]
    row_totals.append(total)
print(row_totals, col_totals)

SoDuckYou's pond works the same way, with one difference: each row is a string instead of a list, so grid[row][col] picks a letter instead of a number.

Follow the brackets

A community garden keeps track of its plots:

garden = [
    {"plot": "A1", "gardener": "Rosa", "crops": ["tomato", "basil"],
     "size": {"rows": 4, "cols": 6}},
    {"plot": "A2", "gardener": "Theo", "crops": [],
     "size": {"rows": 2, "cols": 3}},
    {"plot": "B1", "gardener": "Mina", "crops": ["kale", "beans", "squash"],
     "size": {"rows": 5, "cols": 5}},
]

For each expression, read it one bracket at a time, and write the type you're holding after each step before you write the final value. For garden[0]["crops"], that's "list, then dict, then list: ['tomato', 'basil']." If it crashes, name the error and the bracket that caused it.

The rules you should be able to state afterward:

• Read left to right, and say the type after every bracket.
• A list takes a position (a number), and a dictionary takes a key. Using the wrong kind is the most common nested-data bug.
• The error tells you which kind of mistake it was: IndexError is a position past the end, KeyError is a key that isn't there, and TypeError usually means you're one level off from where you think you are.

Shape in words, shape in code

The week's last objective goes both ways: from a description to code, and from code to a description.

Words to code. For each description, write a small example on paper with two or three entries, then write one expression that gets the value asked for.

1. A dictionary from each team's name to a list of its players' names. Get the second player on one of your teams.
2. A list of songs, where each song is a dictionary with a title, an artist, and a length in seconds. Get the title of the last song.
3. A dictionary from each course to a list of students, where each student is a dictionary with a name and a year. Get the name of the first student in one course.

Code to words. Describe each of these in one sentence, the way the descriptions above are written:

{"ducks": [(0, 1), (1, 3)], "grid": ["ABBC", "ABBC", "ABBD", "ADDD"]}

[{"name": "Ana", "scores": [90, 85]}, {"name": "Ben", "scores": [70]}]

teams = {"Otters": ["Ana", "Ben", "Cal"], "Herons": ["Dee", "Eli"]}
teams["Otters"][1]          # 'Ben'

songs = [{"title": "Rain", "artist": "Mo", "seconds": 201},
         {"title": "Glow", "artist": "Kit", "seconds": 185}]
songs[-1]["title"]          # 'Glow'

courses = {"CS 149": [{"name": "Ana", "year": 1}, {"name": "Ben", "year": 2}]}
courses["CS 149"][0]["name"]   # 'Ana'

Notice that the expression's brackets follow the description's words in order: "a dictionary from course to a list of student dictionaries" becomes [course][0]["name"].

Filter, then pull out

Most questions about nested data have the same shape: go through a list, keep the items that pass a test, and pull out one field from each. A park service lists its trails:

trails = [
    {"name": "Ridge Loop", "park": {"name": "Shenandoah", "state": "VA"},
     "miles": 4.2, "tags": ["views", "steep"]},
    {"name": "Creek Walk", "park": {"name": "Shenandoah", "state": "VA"},
     "miles": 1.5, "tags": ["water", "easy"]},
    {"name": "Cliff Path", "park": {"name": "Grayson", "state": "VA"},
     "miles": 6.0, "tags": ["views"]},
    {"name": "Bog Boardwalk", "park": {"name": "Cranberry", "state": "WV"},
     "miles": 0.8, "tags": ["easy", "water"]},
    {"name": "Summit Trail", "park": {"name": "Shenandoah", "state": "VA"},
     "miles": 7.1, "tags": ["views", "steep"]},
]

On paper, with no computer, write these three functions:

1. short_trails(trails, park, max_miles) returns a list of the names of the trails in that park that are no longer than max_miles.
2. tagged(trails, tag) returns a list of the names of the trails that have that tag. Try it as a comprehension.
3. trails_by_state(trails) returns a dictionary from each state to how many trails it has.

Before you write each one, say out loud which level of the data each line works at: the list, one trail, or the dictionary inside it. Then type them into Thonny and check them with these asserts:

assert short_trails(trails, "Shenandoah", 5) == ["Ridge Loop", "Creek Walk"]
assert short_trails(trails, "Grayson", 5) == []
assert tagged(trails, "water") == ["Creek Walk", "Bog Boardwalk"]
assert tagged(trails, "flat") == []
assert trails_by_state(trails) == {"VA": 4, "WV": 1}

def short_trails(trails, park, max_miles):
    names = []
    for trail in trails:
        if trail["park"]["name"] == park and trail["miles"] <= max_miles:
            names.append(trail["name"])
    return names


def tagged(trails, tag):
    return [trail["name"] for trail in trails if tag in trail["tags"]]


def trails_by_state(trails):
    counts = {}
    for trail in trails:
        state = trail["park"]["state"]
        counts[state] = counts.get(state, 0) + 1
    return counts

Same data, two files

Here are three of those trails saved as a CSV file, trails.csv:

name,park,state,miles
Ridge Loop,Shenandoah,VA,4.2
Creek Walk,Shenandoah,VA,1.5
Bog Boardwalk,Cranberry,WV,0.8

This reads it into a list of lists with the tools from Week 10:

rows = []
with open("trails.csv") as file:
    for line in file:
        rows.append(line.strip().split(","))

The same three trails, saved as JSON in trails.json, take one line to read:

import json

with open("trails.json") as file:
    trails = json.load(file)

Predict each of these on paper, then make both files and check yourself.

What to take away:

• CSV is a grid of strings: you find a column by its position, skip or use the header row, and convert numbers yourself.
• JSON comes back as the lists and dictionaries it was saved as, with numbers already numbers, and it can nest.
• CSV can't nest, so the park had to be flattened into two columns. That's why PA4's levels are in JSON.

Try an AI tutor

AI is a patient drill partner for nested data, because it can make up fresh structures faster than any textbook. Keep the pencil yourself.

"I'm a CS1 student learning nested lists and dictionaries. Make up a list of three or four dictionaries, two levels deep, on any topic. Then give me eight expressions like data[1]["tags"][0], one at a time. For each one, I'll say the type after every bracket and the final value. Don't tell me if I'm right until I've answered, and if I'm wrong, ask me which bracket I got wrong instead of explaining it."

"Here is a description of some data and a function I wrote to search it. Don't rewrite my code. For each line of my loop, tell me which level of the data it's working at, and point to the first line where I'm at a different level than I think I am."

Two things to watch for this week. First, AI reaches for tools this course doesn't use, like pandas, csv.DictReader, and comprehensions nested inside comprehensions; ask for the plain loop. Second, it's surprisingly bad at counting brackets in a long literal and will confidently tell you data[2] is something it isn't. When the AI and your trace disagree, run it in the Shell, and see who was right.

🔭 Explore more

Finished early, or just curious? None of these are required.

• Add from pprint import pprint to a program and pprint a big nested structure, like the whole of levels.json. It lays the levels out so you can see the nesting.
• Open this link in your browser: https://api.open-meteo.com/v1/forecast?latitude=38.45&longitude=-78.87&current=temperature_2m&temperature_unit=fahrenheit&timezone=America/New_York. It's the current temperature in Harrisonburg, as JSON. Write the bracket path that gets the temperature out. Careful: two different keys are called temperature_2m, and only one of them is a number. That's exactly what working with a web API looks like, and some projects will.
• Browse the CORGIS JSON datasets, pick one that interests you, and write down three questions you could answer with it. It might be a start on a project idea.

✍️ Reflection

1. What did you do this week? (Check all that apply.)

• Worked through the in-class activities
• Read the sections listed for one of the textbooks
• Finished PA4 Parts C, D, and E
• Filled in and traced "Grid by hand" on paper
• Followed the brackets in the garden table
• Went from words to code and back in "Shape in words, shape in code"
• Wrote the trail functions on paper, then checked them
• Compared the CSV and JSON files
• Asked an AI tutor to drill me
• Tried something from "Explore more"
• Something else (tell us in the short answers)

2. Roughly how much time did you spend on CS 149 outside of class this week?

• Under 1 hour
• 1–2 hours
• 3–4 hours
• 5–6 hours
• 7+ hours

3. How comfortable are you reading an expression like data[2]["scores"][0] without running it?

• Comfortable: I can say the type after every bracket
• Mostly: two levels are fine, but three makes me lose my place
• Not yet: I still need to run it to find out
• I haven't tried one yet

4. How did you test your program? Which asserts did you write in test_touching and test_load_levels before you submitted, and what bug could each one have caught? Did you try two ducks that touch only at a corner, ducks that are all far apart, and a level marked as a draft?

5. Give one expression or loop over nested data this week that did not do what you expected. It could be a KeyError where you used a position on a dictionary, a TypeError from being one level off, a nested loop that compared a duck with itself, or a CSV number that turned out to be a string. Say what you expected, what happened, and the rule you were using that turned out to be wrong.

6. Which of this week's objectives feels solid, and which feels shakiest? Name them specifically, for example "reading the brackets" or "writing a filter from a description." For the shaky one, what will you do about it?

7. Did you use an AI tool this week, and did it build your understanding or replace it? The honest test: given a description of some data you haven't seen before, could you write a function that filters it and pulls out one field, on a blank page? If a video, article, or tutorial made something click, drop the link and one line on why it helped.

8. Paste your PA4 code for Parts C and D. Point out the line in touching that keeps a duck from being compared with itself.