Python - Nested Dictionaries

A dictionary can contain other dictionaries as its values, forming a hierarchical or nested structure.

Think of a nested dictionary as a magical filing cabinet where each drawer can contain another smaller filing cabinet inside. In Python, this concept allows us to create complex data structures that mirror real-world hierarchical relationships.

Let's transform our simple country capitals dictionary into something more interesting by adding layers of information about each country:

Now we have a rich structure where each country contains multiple pieces of information. Each country is like a folder containing various details about that nation.

Accessing information in nested dictionaries works like following a path through our filing cabinet. Want to know the capital of Greece? Simply use:

greek_capital = capitals["Greece"]["capital"]

Need to know about a specific landmark in Italy? Just drill down one level deeper:

colosseum_info = capitals["Italy"]["landmarks"]["Colosseum"]

The real power of nested dictionaries becomes apparent when you need to store and access related data in a structured way. Now we will add some information:

capitals["Greece"]["economy"] = {
    "currency": "Euro",
    "gdp": 189.4,
    "main_sectors": ["Tourism", "Shipping", "Agriculture"]
}

# Accessing deep nested data
greek_currency = capitals["Greece"]["economy"]["currency"]

Remember that while nested dictionaries are powerful, they should be designed thoughtfully. Too many levels of nesting can make your code harder to read and maintain. A good rule of thumb is to keep nesting to a maximum of three or four levels deep.

Nested dictionaries are commonly used in:

• Configuration settings with multiple levels of options
• Game state management where objects have multiple properties
• Data processing applications handling hierarchical information
• API responses containing nested information

Creating Nested Dictionaries Dynamically

When working with real-world data, you'll often need to transform simple dictionaries into more complex nested structures.

Imagine you're building a travel application and starting with a simple dictionary of capitals. Using loops to create nested dictionaries is like automatically upgrading your simple address book into a detailed travel guide.

The process happens in several steps. First, we start with our basic capitals dictionary:

capitals = {
    "Greece": "Athens",
    "Italy": "Rome",
    "Spain": "Madrid"
}

When we run our first loop, we're essentially taking each country and expanding its entry from a simple string (just the capital city) into a detailed information card. Each iteration of the loop creates a new nested structure for each country, complete with tracking information like whether someone has visited the country and what rating they gave it.

The second loop demonstrates how to update specific nested values systematically. Think of it as going through your travel diary and filling in ratings for each country you've visited.

The final example shows how to create even more complex nested structures by adding tourist attractions. For each country, we're creating a detailed catalog of places to visit, complete with additional information about each location.

Adding Elements to a Nested Dictionary

You can add elements at any level using standard dictionary assignments:

📌 Example:

# Add a new student
student_data['Michael'] = {'age': 22, 'major': 'Physics'}

# Add grades to Michael's data
student_data['Michael']['grades'] = {'physics': 88, 'math': 90}

# Add a specific grade
student_data['John']['grades']['history'] = 82

Loop Through Nested Dictionaries

With nested dictionaries, you iterate through each top-level key, then dive into its inner keys and values.

📌 For example:

# Loop through the main dictionary
for student, info in student_data.items():
    print(f"Student: {student}")
    print(f"Age: {info['age']}")
    
    # Loop through the nested grades dictionary if it exists
    if 'grades' in info:
        print("Grades:")
        for subject, score in info['grades'].items():
            print(f"  {subject}: {score}")
    print()

Deleting Dictionaries from a Nested Dictionary

You can delete elements using the del keyword or dictionary methods:

📌 Example:

# Remove a student completely
del student_data['Emma']

# Remove specific information
del student_data['John']['grades']['math']

# Alternative: using pop()
removed_student = student_data.pop('Michael')
removed_grade = student_data['John']['grades'].pop('programming')

Flattening Nested Dictionaries

Flattening a nested dictionary means converting a dictionary with nested dictionaries into a single-level dictionary with compound keys.

📌 Example:

👣 Step-by-step explanation:

• The function accepts three parameters:
  • nested_dict: The nested dictionary that needs to be flattened.
  • parent_key: A string that prefixes each key. It’s initially empty.
  • sep: The separator between parent and child keys, defaulting to a period (.).
• A helper function _flatten is a function that performs the actual flattening. It’s defined inside flatten_dictionary to keep everything encapsulated.
• _flatten takes two arguments:
  • obj: The current dictionary or sub-dictionary being processed.
  • parent: The string representing the current prefix (built up from parent keys).
• For each key-value pair in obj:
  • It constructs new_key by appending the current key to parent, separated by sep.
  • If the value is a dictionary, _flatten calls itself recursively to process the nested dictionary.
  • If the value is not a dictionary, it adds the new_key and value to flat_dict.
• The initial call to _flatten is made with the original nested_dict and an empty parent_key.
• Once the recursion completes, flat_dict contains all the flattened key-value pairs, and it’s returned as the function's result.

This flattened dictionary merges all nested levels into a single level, using compound keys that represent the hierarchy.

Deep Merging Nested Dictionaries

📌 Here’s a function deep_merge that merges two dictionaries, including nested dictionaries.

👣 Step-by-step explanation:

• The deep_merge function takes two dictionaries, dict1 and dict2, and merges then. It handles merging nested dictionaries recursively.
• The function iterates through each key-value pair in dict2.
• If the key exists in dict1:
  • If both dict1[key] and dict2[key] are dictionaries, it calls itself recursively to merge them.
  • Otherwise, it directly sets result[key] to dict2[key] (overwriting the value in dict1).
• Calling deep_merge(dict1, dict2) merges dict2 and dict1, updating quantities for existing entries, adding new entries, and including the new category Furniture.

Here are some situations where it might be particularly beneficial:

• Merging Configuration Files. When dealing with multiple configuration files (e.g., YAML, JSON) that need to be combined into a single configuration dictionary. This is common in applications with different environment settings (development, staging, production).
• Applying incremental updates to the configuration without completely overwriting existing settings.
• Combining Data from Multiple Sources. When you have data coming from different APIs or datasets that need to be combined into a single, coherent structure.
• Combining inventory data from different warehouses or sources, ensuring that quantities and details are accurately updated.
• Managing hierarchical data structures (e.g., categories, sub-categories, tags) and combining updates without losing information.

Note: The dictionary union operator (|) creates a new dictionary by merging dict1 and dict2, but it performs a shallow merge.

• For top-level keys, if a key exists in both dictionaries, the value from dict2 will overwrite the value from dict1.
• If the value associated with a key is itself a dictionary, this nested dictionary won't be merged; instead, it will be directly overwritten by the value from dict2.

The deep_merge function performs a recursive merge, ensuring that nested dictionaries are merged at all levels, not merely overwritten. This is useful when you need to combine dictionaries while preserving nested structures.

Feature	dict1 | dict2	deep_merge(dict1, dict2)
Dictionary Handling	overwrites nested dictionaries completely	merges nested dictionaries recursively, preserving and combining their contents
Use Case	for simple, shallow merges where you don't care about preserving nested structures	when you need to merge dictionaries deeply, maintaining and merging nested structures at all levels.
Behavior with Existing Keys	The value from dict2 replaces the value from dict1 if the key exists in both.	If the key exists in both and their values are dictionaries, it merges them deeply; otherwise, the value from dict2 replaces the one from dict1.

Conclusion. If you're dealing with simple, non-nested dictionaries or you only need a shallow merge, | is convenient and straightforward. However, if you're working with complex, nested dictionaries and need to preserve and merge their structure comprehensively, deep_merge is the way to go.

addict Library

addict is a small library that lets you work with dictionaries in a more convenient way. Instead of having to check if certain keys exist or worry about nested dictionaries, addict allows you to do something like

a.b.c.d = 1 

and it will create any missing keys on the fly.

First, install it with pip:

pip install addict 

The main thing that makes addict special is its Dict class, which lets you access dictionary keys like they're object attributes.

📌 Example:

What's cool is that you don't have to pre-create the nested structure - addict handles that for you. With a regular Python dict, you'd need to do something like:

You can also convert existing dictionaries to addict's Dict:

One really handy feature is that it won't raise KeyError exceptions for missing keys - it just returns an empty Dict object:

And when you need a regular dictionary back, just use the to_dict() method:

regular_dict = config.to_dict()

Why use addict?

• It can save you from writing endless checks for whether a key exists.
• It keeps your code readable and straightforward, especially if you’re dealing with a lot of nested data structures.

In short, addict is extremely handy if you frequently interact with complex JSON or nested dictionaries and want a more natural way to create and access nested data.

Counting

Sometimes, you need to group or count items by several attributes.

For example, you might want to count the number of people in a dataset based on their birth year, gender, and eye color. Typically, you’d have to check if each key (year, gender, eye color) exists in a dictionary. If not, you’d create it, and so on. With addict, any missing keys are automatically created on the fly when you assign a value to them.

Addict's ability to easily access and modify deeply-nested attributes makes it ideal for counting. This offers a distinct advantage over collections.Counter, as it will easily allow for counting by multiple levels.

Below is a list of dictionaries, each representing a person. Each person’s dictionary includes:

• born: the year the person was born (1980 or 1981)
• gender: M or F
• eyes: possible color values (blue or green)

We want to count how many people exist for each combination of born year, gender, and eye color.

Output:

{1980: {'F': {'blue': 1, 'green': 1}, 
        'M': {'blue': 1, 'green': 3}},
 1981: {'F': {'blue': 2, 'green': 1}, 
        'M': {'blue': 2, 'green': 1}}}

Key Points

1. No Need for Key Checks.
2. The nested structure is built on the fly.
3. You can adapt the same approach to different data sets or additional attributes.

In short, addict gives you a straightforward approach to working with nested dictionaries, making tasks like multi-criteria counting less prone to errors and more readable.

FAQs on Nested Dictionaries

What is a nested dictionary in Python?

A nested dictionary is a dictionary that contains other dictionaries as values, creating a hierarchical or multi-level data structure. You would use nested dictionaries when you need to organize complex, related data in a structured way.

How do I access values in a nested dictionary?

You can access values in a nested dictionary by chaining square bracket notation with the appropriate keys. For example:

data = {"user": {"name": "Alice", "age": 30}}
print(data["user"]["name"])  # Output: Alice

Is there a limit to how deeply I can nest dictionaries?

While there's no technical limit to dictionary nesting in Python, it's recommended to keep nesting to a maximum of three or four levels deep for code readability and maintainability.