Working with JSON Files in Python, with Examples

In this tutorial, we’ll learn how to read, write and parse JSON in Python with relevant examples. We’ll also explore popular modules in Python for working with JSON.

JSON is a lightweight data interchange format. It’s a popular format used for transmitting and receiving data between a client and a server. However, its application and use go beyond transmitting data. Machines can easily generate and parse JSON data. The acronym JSON means JavaScript Object Notation, and, as the name rightly suggests, it’s a subset of the programming language JavaScript.

JSON is a standardized data-interchange format, and it’s language independent. Virtually all programming languages support it in one way or another. It has the following structure:

It has an opening brace to the left and a closing brace to the right.
It has a collection of name/value pairs.
Each name is separated by a colon “:” from its value.
Each name/value pair is separated by a comma (,).

Below is a snippet of a JSON object:

{
    "name": "Chinedu Obi",
    "age": 24,
    "country": "Nigeria",
    "languages": [
        "Igbo",
        "English",
        "Yoruba"
    ],
    "marital status": "single",
    "employee": true,
    "experience": [
        {
            "title": "Frontend Engineering Intern",
            "company": "Andela"
        },
        {
            "title": "Frontend Engineer",
            "company": "Paystack"
        }
    ]
}

We’ll assume that the above JSON is stored in a file named employee.json for the purposes of the later code examples.

Types of JSON Data

When working with JSON objects, Python converts JSON data types to their equivalents, and vice versa. The table below shows Python data types and their JSON equivalents.

Python	JSON Equivalent
`dict`	object
`list`, `tuple`	array
`str`	string
`int`, `float`, `long`	number
`True`	true
`False`	false
`None`	null

The Difference Between the json and simplejson Modules in Python

There are several modules for encoding and decoding JSON in Python. The two most popular modules are json and simplejson. The json module is a built-in package in the Python standard library, which means we can use it out of the box without having to install it.

The simplejson module is an external Python module for encoding and decoding JSON. It’s an open-source package with backward compatibility for Python 2.5+ and Python 3.3+. It’s also fast, simple, correct and extensible.

simplejson is updated more frequently, with more up-to-date optimization than json, making it faster. If you’re working with an older version of Python below 2.6 in a legacy project, then simplejson is your best bet.

For the purposes of this tutorial, we’ll stick with the json module.

How to Read and Write JSON to a File in Python

While programming in Python, we’ll often come across the JSON data format, and it’s important to know how to read or write JSON data and files. Prior knowledge of file handling in Python will come in handy here for reading and writing JSON to files.

How to read JSON Files in Python

Much like every other read operation in Python, the with statement can be used together with the json.load() method to read JSON files.

See the code example below:

import json

with open('employee.json', 'r', encoding='utf-8') as file_object:
    employee_dict = json.load(file_object)
    print(employee_dict)

Here’s the output of the code above:

{'name': 'Chinedu Obi', 'age': 24, 'country': 'Nigeria', 'languages': ['Igbo', 'English', 'Yoruba'], 'marital status': 'single', 'employee': True, 'experience': [{'title': 'Frontend Engineering Intern', 'company': 'Andela'}, {'title': 'Frontend Engineer', 'company': 'Paystack'}]}

In the code above, we open the employee.json file in read mode. The json.load() method decodes the JSON data into a Python dictionary stored in the variable employee_dict.

Writing JSON to a file in Python

We can also perform a write operation with JSON data on files in Python. Just like in the read operation, we use the with statement alongside the json.dump() method to write JSON data to a file.

Consider the code snippet below:

import json

mother = {
    "name": "Asake Babatunde",
    "age": 28,
    "marital status": "Married",
    "children": ["Ayo", "Tolu", "Simi"],
    "staff": False,
    "next of kin": {"name": "Babatune Lanre", "relationship": "husband"},
}

with open("mother.json", "w", encoding="utf-8") as file_handle:
    json.dump(mother, file_handle, indent=4)

Here, we create a Python dictionary, mother, which contains data about a fictional mother. We open mother.json in write mode. Since there’s no such file, one is created for us. The json.dump() method encodes the Python dictionary assigned to the mother variable to a JSON equivalent, which is written into the specified file. When the above code is executed, a mother.json file containing the JSON data will appear in the root of our folder.

How to Convert a Python Dictionary to JSON (Serialization)

Serialization is the process of converting Python objects — in most cases, a dictionary — to JSON formatted data or string. When serializing, Python types are encoded to a JSON equivalent. The json module provides two methods — json.dump() and json.dumps() — for serializing Python objects to JSON format.

json.dump() is used when writing JSON equivalent of Python objects to a file.
json.dumps() (with an “s”) is used when converting a Python object to a JSON-formatted string.

Notice the syntax below:

json.dump(obj, fp, indent)

json.dumps(obj, indent)

The json.dump() method has a parameter fp, while json.dumps() doesn’t have it.

Some parameters explained:

obj: a Python object to be serialized to JSON format.
fp: a file pointer (object) with methods such as read() or write().
indent: a non-negative integer or string, to indicate indent level for pretty printing JSON data.

Example: Converting a Python object to JSON format with json.dump()

Let’s encode a Python object to equivalent JSON formatted data and write it to a file.

First, we create a Python dictionary:

import json

subject = {
    "name": "Biology",
    "teacher": {"name": "Nana Ama", "sex": "female"},
    "students_size": 24,
    "elective": True,
    "lesson days": ["Tuesday", "Friday"],
}

Let’s encode our dictionary to JSON data and write to a file:

with open('subject.json', 'w', encoding='utf-8') as file_handle:
    json.dump(subject, file_handle, indent=4)

In the example above, we pass the dictionary, file pointer and indent arguments to the json.dump method. Below is the output of our code. When the code is executed, a subject.json file containing the expected JSON data is found in our project’s root folder:

{
    "name": "Biology",
    "teacher": {
        "name": "Nana Ama",
        "sex": "female"
    },
    "students_size": 24,
    "elective": true,
    "lesson days": [
        "Tuesday",
        "Friday"
    ]
}

Our output has a pretty printing output because we added an indent argument value of 4.

Example: Converting a Python object to JSON format with json.dumps()

In this example, we’ll encode a Python object to a JSON string. We created a subject dictionary before, so we can reuse it here.

Let’s take an example with the json.dumps() method:

json_data = json.dumps(subject, indent=4)
print(json_data)
print(type(json_data))

Here’s the output of the code above:

{
    "name": "Biology",
    "teacher": {
        "name": "Nana Ama",
        "sex": "female"
    },
    "students_size": 24,
    "elective": true,
    "lesson days": [
        "Tuesday",
        "Friday"
    ]
}
<class 'str'>

As stated earlier, the json.dumps() method is used to convert Python objects to a JSON formatted string. We can see from the console that our JSON data is of type str.

How to Convert JSON to a Python Dictionary (Deserialization)

Deserialization of JSON is the process of decoding JSON objects to equivalent Python objects or Python types. We can convert JSON formatted data to Python objects using two methods — json.load() and json.loads() — which are provided by the json module.

json.load() is used when reading JSON formatted data from a file.
json.loads() (with an “s”) is used when parsing a JSON string to a Python dictionary.

Notice the syntax below:

json.load(fp)

json.loads(s)

json.dump() has a parameter fp, while json.dumps() has a parameter s. The other parameters remain the same.

Some parameters explained:

fp: a file pointer (object) with methods such as read() or write().
s: a str, bytes or bytearray instance containing a JSON document.

Example: Converting a JSON object to a Python object with json.load()

Here are the contents of a new JSON file named students.json:

[
    {
        "name": "Edidiong Ime",
        "subject": "Chemistry",
        "score": 91,
        "class": "SS 3"
    },
    {
        "name": "Jesse Umoh",
        "subject": "Chemistry",
        "score": 85,
        "class": "SS 3"
    },
    {
        "name": "Kingsley Chuks",
        "subject": "Chemistry",
        "score": 68,
        "class": "SHS 3"
    },
    {
        "name": "Martha Sam",
        "subject": "Chemistry",
        "score": 79,
        "class": "SHS 3"
    }
]

In this example, we’ll decode JSON data from the students.json file to Python objects:

import json

with open('students.json', 'r', encoding='utf-8') as file_handle:
    students_list = json.load(file_handle)
    for student in students_list:
        print(student)

Here’s the output of the code above:

{'name': 'Edidiong Ime', 'subject': 'Chemistry', 'score': 91, 'class': 'SS 3'}
{'name': 'Jesse Umoh', 'subject': 'Chemistry', 'score': 85, 'class': 'SS 3'}
{'name': 'Kingsley Chuks', 'subject': 'Chemistry', 'score': 68, 'class': 'SHS 3'}
{'name': 'Martha Sam', 'subject': 'Chemistry', 'score': 79, 'class': 'SHS 3'}

In the code snippet above, a JSON file containing a list of students is being parsed. The JSON data from the file file_handle is passed to the json.load() method, which decodes it into a list of Python dictionaries. The list items are then printed to the console.

Example: Converting a JSON object to a Python dictionary with json.loads()

In this example, let’s decode JSON data from an API endpoint from JSONPlaceholder. The requests module should be installed before you proceed with this example:

import json
import requests

response = requests.get("https://jsonplaceholder.typicode.com/comments/2")
comment = json.loads(response.text)

print(comment)

Here’s the output of the code above:

{'postId': 1, 'id': 2, 'name': 'quo vero reiciendis velit similique earum', 'email': 'Jayne_Kuhic@sydney.com', 'body': 'est natus enim nihil est dolore omnis voluptatem numquam\net omnis occaecati quod ullam at\nvoluptatem error expedita pariatur\nnihil sint nostrum voluptatem reiciendis et'}

In the Python code above, we get a response from an endpoint that returns a JSON formatted string. We pass the response as an argument to the json.loads() method to decode it into a Python dictionary.

Conclusion

In modern web development, JSON is the de facto format for exchanging data between a server and web applications. These days, REST API endpoints return JSON-formatted data, so understanding how to work with JSON is important.

Python has modules like json and simplejson for reading, writing and parsing JSON. The json module comes with the Python standard library, while simplejson is an external package and must be installed before being used.

When building RESTful APIs in Python, or using external APIs in our projects, we’ll often need to serialize Python objects to JSON and deserialize them back to Python. The approaches demonstrated in this article are used by many popular projects. The steps are generally the same.

The code from this tutorial can be found on GitHub.

Frequently Asked Questions (FAQs) about JSON in Python

How Can I Parse JSON in Python?

Parsing JSON in Python is a straightforward process thanks to the json module. You can use the json.loads() function to parse a JSON string. Here’s an example:

import json
json_string = '{"name": "John", "age": 30, "city": "New York"}'
parsed_json = json.loads(json_string)
print(parsed_json["age"])
In this code, json.loads() converts the JSON string into a Python dictionary, which you can then interact with like any other dictionary.

How Can I Convert Python Objects into JSON?

The json module provides the json.dumps() function, which can convert Python objects into JSON strings. Here’s an example:

import json
person = {"name": "John", "age": 30, "city": "New York"}
json_string = json.dumps(person)
print(json_string)
In this code, json.dumps() converts the Python dictionary into a JSON string.

How Can I Read JSON from a File in Python?

You can use the json.load() function to read JSON data from a file. Here’s an example:

import json
with open('file.json', 'r') as f:
data = json.load(f)
print(data)
In this code, json.load() reads the file and converts the JSON data into a Python object.

How Can I Write JSON to a File in Python?

You can use the json.dump() function to write JSON data to a file. Here’s an example:

import json
person = {"name": "John", "age": 30, "city": "New York"}
with open('file.json', 'w') as f:
json.dump(person, f)
In this code, json.dump() writes the Python object as JSON data into the file.

How Can I Pretty Print JSON in Python?

The json.dumps() function provides options for pretty-printing JSON. Here’s an example:

import json
person = {"name": "John", "age": 30, "city": "New York"}
json_string = json.dumps(person, indent=4)
print(json_string)
In this code, the indent parameter specifies how many spaces to use as indentation, which makes the output easier to read.

How Can I Handle Complex Python Objects with JSON?

The json module can only handle simple Python objects by default. For complex objects like custom classes, you need to provide a function that tells it how to serialize the object. Here’s an example:

import json
class Person:
def __init__(self, name, age):
self.name = name
self.age = age
def encode_person(obj):
if isinstance(obj, Person):
return obj.__dict__
return obj
person = Person("John", 30)
json_string = json.dumps(person, default=encode_person)
print(json_string)
In this code, the encode_person function is used to convert the Person object into a serializable format.

How Can I Parse JSON with Unknown Structure?

If you don’t know the structure of the JSON data in advance, you can still parse it into a Python object and explore it. Here’s an example:

import json
json_string = '{"name": "John", "age": 30, "city": "New York", "friends": ["Jane", "Doe"]}'
data = json.loads(json_string)
for key, value in data.items():
print(f"{key}: {value}")
In this code, json.loads() converts the JSON string into a Python dictionary, which you can then iterate over to explore its contents.

How Can I Handle Large JSON Files in Python?

For large JSON files, you can use the ijson package, which allows you to stream JSON data instead of loading it all into memory at once. Here’s an example:

import ijson
with open('large_file.json', 'r') as f:
objects = ijson.items(f, 'item')
for obj in objects:
print(obj)
In this code, ijson.items() generates a stream of objects from the JSON data, which you can then iterate over.

How Can I Handle JSON Errors in Python?

The json module raises a JSONDecodeError when it encounters invalid JSON. You can catch this error and handle it appropriately. Here’s an example:

import json
try:
json_string = '{"name": "John", "age": 30, "city": "New York", "friends": ["Jane", "Doe"]'
data = json.loads(json_string)
except json.JSONDecodeError as e:
print(f"Invalid JSON: {e}")
In this code, the try/except block catches the JSONDecodeError and prints an error message.

How Can I Use JSON with HTTP Requests in Python?

The requests library makes it easy to send HTTP requests with JSON data and handle the JSON responses. Here’s an example:

import requests
import json
data = {"name": "John", "age": 30, "city": "New York"}
response = requests.post('https://httpbin.org/post', json=data)
print(response.json())
In this code, requests.post() sends a POST request with JSON data, and response.json() parses the JSON response.