JSON serialization and deserialization with python

json serialization and deserialization

JSON (JavaScript Object Notation) serialization and deserialization refer to the processes of converting data between JSON format and a programming language's data structures.

Serialization is the process of converting a data object or data structure into a JSON string representation. In other words, it involves encoding the data in a format that can be easily transmitted or stored. This is typically done when you want to send data over a network or save it to a file.

To serialize a Python object and save it as a JSON file, you can use the json module provided by Python's standard library. Here's an example of how you can accomplish this:

import json

# Python object to be serialized
person = {
    "name": "John Doe",
    "age": 30,
    "city": "New York"
}

# Serialize the object to a JSON string
json_data = json.dumps(person)

# Write the JSON string to a file
with open('person.json', 'w') as json_file:
    json_file.write(json_data)

In this example, the person dictionary represents the Python object that you want to serialize. Using the json.dumps() function, you can convert the Python object into a JSON string.

Next, you can open a file in write mode (specified by 'w') using the open() function and write the JSON string to the file using the write() method. Here, the file name is "person.json", but you can choose any desired name.

After executing the code, a JSON file named "person.json" will be created, containing the serialized JSON representation of the Python object.

Note that the json.dumps() function accepts additional parameters to control the formatting and customization of the JSON output. For example, you can use indent to specify the indentation level for pretty-printing the JSON data.

Deserialization, on the other hand, is the process of converting a JSON string back into a data object or data structure in a specific programming language. It involves decoding the JSON string and reconstructing the original data.

Here's an example of a JSON file representing a collection of books:

{
  "books": [
    {
      "title": "The Great Gatsby",
      "author": "F. Scott Fitzgerald",
      "year": 1925,
      "genre": "Fiction"
    },
    {
      "title": "To Kill a Mockingbird",
      "author": "Harper Lee",
      "year": 1960,
      "genre": "Fiction"
    },
    {
      "title": "1984",
      "author": "George Orwell",
      "year": 1949,
      "genre": "Science Fiction"
    }
  ]
}

In this example, the JSON file contains a key "books" whose associated value is an array of book objects. Each book object has properties like "title", "author", "year", and "genre", representing the title, author, publication year, and genre of a book, respectively.

To deserialize this JSON file, you would read its contents and use a JSON deserialization function provided by your programming language to convert it into a corresponding data structure, such as an array of book objects in this case.

To deserialize the JSON file and convert it into a Python object, you can make use of the json module, which is part of the Python standard library. Here's an example of how you can accomplish this:

import json

# Read the JSON file
with open('books.json') as json_file:
    json_data = json.load(json_file)

# Access the deserialized data
books = json_data['books']

# Iterate over the books
for book in books:
    title = book['title']
    author = book['author']
    year = book['year']
    genre = book['genre']

    # Perform desired operations with the book data
    print(f"Title: {title}, Author: {author}, Year: {year}, Genre: {genre}")

In this example, assuming the JSON file is named "books.json", we open the file using open() and then use json.load() to deserialize the JSON data into a Python object. We can then access the deserialized data as a Python dictionary. In this case, we access the list of books using the key 'books'.

We can iterate over the list of books and access the individual book properties such as title, author, year, and genre. You can perform any desired operations with the book data within the loop.

Make sure to replace 'books.json' with the actual file path if it's different, and ensure that the JSON file is in the correct format.

Note: It's important to handle any potential exceptions that may occur during file reading or JSON deserialization, such as FileNotFoundError or json.JSONDecodeError, to ensure your code handles potential errors gracefully.

Output:

Title: The Great Gatsby, Author: F. Scott Fitzgerald, Year: 1925, Genre: Fiction
Title: To Kill a Mockingbird, Author: Harper Lee, Year: 1960, Genre: Fiction
Title: 1984, Author: George Orwell, Year: 1949, Genre: Science Fiction

Difference between json.load() and json.loads():

The json.load() function is used to parse JSON data from a file-like object, such as a file object or a StringIO object, and convert it into a Python object. Unlike json.loads(), which takes a JSON string as input, json.load() directly reads the JSON data from a file-like object.

The json.load() function has the following parameter:

fp: This parameter specifies the file-like object containing the JSON data to be parsed. It can be a file object opened in read mode ('r'), a StringIO object, or any other object that supports the file protocol (read() method).

Here's an example demonstrating the usage of json.load():

import json

# Open a file containing JSON data
with open('data.json', 'r') as file:
    # Parse the JSON data from the file
    data = json.load(file)

# Access the parsed JSON data as a Python object
print(data)

In this example, the open() function is used to open a file named 'data.json' in read mode ('r'). The file contains JSON data. The file object is then passed as the fp parameter to json.load() to parse the JSON data. The resulting Python object is stored in the data variable.

You can then work with the data object as a regular Python object, which represents the parsed JSON data structure.

The json.loads() function is used to parse a JSON-formatted string and convert it into a Python object. It takes a JSON string as input and returns the corresponding Python object.

The `json.loads()` function has the following parameters:

s: This parameter specifies the JSON string to be parsed. It should be a valid JSON-formatted string.
parse_float: This optional parameter specifies a function that will be used to parse floating-point values in the JSON string. By default, floating-point values are parsed as float objects. However, you can provide a custom function to parse these values differently.
parse_int: This optional parameter specifies a function that will be used to parse integer values in the JSON string. By default, integer values are parsed as int objects. You can provide a custom function to parse integer values differently.
parse_constant: This optional parameter specifies a function that will be used to parse the JSON constant values null, true, and false. By default, these constant values are parsed as None, True, and False, respectively. You can provide a custom function to parse these constants differently.
object_hook: This optional parameter specifies a function that will be called for each object decoded from the JSON string. This function can be used to customize the decoding process and convert JSON objects into specific Python objects.
object_pairs_hook: This optional parameter specifies a function that will be called with the result of decoding a JSON object. It can be used to modify the result before returning.

json.load() has one more additional parameter fp(file pointer), because it is used with files.

Difference between object_pairs_hook and object_hook in the json.loads() function.

Example JSON data:

{
  "person1": {"name": "John", "age": 25},
  "person2": {"name": "Alice", "age": 30}
}

Using object_pairs_hook:

import json

def custom_object_pairs_hook(pairs):
    result = {}
    for key, value in pairs:
        result[key.upper()] = value
    return result

json_data = '{"person1": {"name": "John", "age": 25}, "person2": {"name": "Alice", "age": 30}}'
result = json.loads(json_data, object_pairs_hook=custom_object_pairs_hook)
print(result)

Output:

{'PERSON1': {'name': 'John', 'age': 25}, 'PERSON2': {'name': 'Alice', 'age': 30}}

In this example, the custom_object_pairs_hook function is defined to convert the keys of each object to uppercase while keeping the values intact. The function is passed as the object_pairs_hook parameter to json.loads(). The resulting dictionary has the keys converted to uppercase.

Using object_hook:

import json

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

def custom_object_hook(obj):
    if "name" in obj and "age" in obj:
        return Person(obj["name"], obj["age"])
    return obj

json_data = '{"person1": {"name": "John", "age": 25}, "person2": {"name": "Alice", "age": 30}}'
result = json.loads(json_data, object_hook=custom_object_hook)
print(result)

Output:

{'person1': <__main__.Person object at 0x...>, 'person2': <__main__.Person object at 0x...>}

In this example, the custom_object_hook function is defined to convert each dictionary object into a Person object if it contains the keys "name" and "age". The function is passed as the object_hook parameter to json.loads(). The resulting dictionary contains Person objects instead of the original dictionaries.

In both examples, the provided custom functions (custom_object_pairs_hook and custom_object_hook) are called during the JSON deserialization process, allowing for customization of the resulting objects or dictionaries. The choice between object_pairs_hook and object_hook depends on the specific requirements and structure of the JSON data being processed.

Here's an example demonstrating the usage of json.loads() with some optional parameters:

import json

# JSON string
json_str = '{"name": "John", "age": 30, "salary": 50000.0}'

# Parse the JSON string
data = json.loads(json_str)

print(data)  # Output: {'name': 'John', 'age': 30, 'salary': 50000.0}

In this example, the json_str variable contains a JSON-formatted string. The json.loads() function is called with the json_str parameter to parse the JSON string and convert it into a Python object. The resulting Python object is stored in the data variable.

By default, the function uses the default parsing behavior for floating-point values, integer values, and JSON constants. However, you can provide custom functions for these behaviors using the optional parameters mentioned above.

json.dumps() and few of its optional parameters:

The json.dumps() function is used to serialize Python objects into a JSON-formatted string. It takes a Python object as input and returns a JSON string representation of that object.

The json.dumps() function has several optional parameters that allow you to customize the serialization process. Here are the parameters commonly used with json.dumps():

obj: This parameter specifies the Python object to be serialized into JSON. It can be any object that is JSON serializable, such as dictionaries, lists, strings, numbers, booleans, or None.
skipkeys: This optional parameter is a boolean value that specifies whether to skip serializing dictionary keys that are not of basic types (str, int, float, bool, None). If skipkeys is set to True, dictionary keys of unsupported types will be skipped during serialization. If skipkeys is False (default), a TypeError will be raised if such keys are encountered.
ensure_ascii: This optional parameter is a boolean value that specifies whether to escape non-ASCII characters in the output with Unicode escape sequences. If ensure_ascii is True (default), non-ASCII characters will be escaped. If ensure_ascii is False, non-ASCII characters will be output as is.
check_circular: This optional parameter is a boolean value that specifies whether to check for circular references during serialization. If check_circular is True (default), a ValueError will be raised if a circular reference is encountered. If check_circular is False, circular references will be handled, but the resulting JSON may not be valid.
allow_nan: This optional parameter is a boolean value that specifies whether to allow NaN, Infinity, and -Infinity as numeric values in the output. If allow_nan is True (default), these special float values will be allowed. If allow_nan is False, a ValueError will be raised if any of these values are encountered.
indent: This optional parameter specifies the indentation level to be used in the formatted JSON output. It can be an integer representing the number of spaces per indentation level, or a string containing the characters used for indentation (e.g., "\t"). If indent is None (default), no indentation will be applied, and the output will be compact.
separators: This optional parameter specifies the separators used in the output JSON string. It should be a tuple of two strings: (item_separator, key_separator). The item_separator is the string used to separate items in a JSON object or array, and the key_separator is the string used to separate keys and values in a JSON object. The default separators are (", ", ": ").
sort_keys: This optional parameter is a boolean value that specifies whether to sort the keys of dictionaries alphabetically in the output. If sort_keys is True, the dictionary keys will be sorted. If sort_keys is False (default), the keys will be serialized in the order they are encountered.
cls: More on this below

These are the most commonly used parameters of json.dumps(). You can refer to the Python documentation for the json module for a complete list of available parameters and their descriptions.

cls

In the context of the json.dumps() function, the cls parameter is used to specify a custom JSON encoder class to be used for serialization. The cls parameter stands for "class" and expects a class object that is a subclass of json.JSONEncoder.

When you pass a custom encoder class as the cls parameter to json.dumps(), it instructs the dumps() function to use that custom encoder class for serializing the Python objects into JSON.

example:

import json

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, complex):
            return [obj.real, obj.imag]
        # Let the base class default method raise the TypeError
        return json.JSONEncoder.default(self, obj)
json.dumps(2 + 1j, cls=ComplexEncoder)

The cls parameter is used to specify the ComplexEncoder class as the custom JSON encoder class. This means that when serializing the complex number 2 + 1j, the ComplexEncoder class will be responsible for converting it into the desired JSON representation.

By specifying cls=ComplexEncoder, you are telling json.dumps() to use the default() method implemented in the ComplexEncoder class to handle the serialization of the complex number object.

Note that the cls parameter is optional, and if not provided, the default behavior of json.dumps() will be used, which uses the standard json.JSONEncoder class for serialization. However, by specifying a custom encoder class using cls, you have the flexibility to define your own serialization logic and handle special cases or custom data types as needed.

Using json from the shell

echo '{"json":"obj"}' | python -m json.tool

The command echo '{"json":"obj"}' | python -m json.tool is a command-line instruction that makes use of the json.tool module in Python to format and validate JSON data

tool is a submodule in the json module.
The -m option is used to run a Python module directly as a script.

When you execute python from the command line, it searches for the Python interpreter in the directories listed in the system's PATH environment variable. Once it finds the Python interpreter, it runs the specified code or module.

The -m option is used to specify a module to be executed. In this case, json.tool is the module that is being executed.

The json.tool module is a built-in module in Python's standard library. It provides a command-line interface for formatting and validating JSON data.

How to access classes and functions inside a module from the shell without explicitly opening the python interpreter

echo '{"json":"obj"}' | python -c "import json; print(json.dumps({'json':'obj'}))"

In the example above, the python -c command is used to execute a Python command. The Python code specified within the double quotes after -c imports the json module and then calls the dumps() function to serialize the JSON data.

The echo command is used to pass the JSON data {"json":"obj"} as input to the python -c command. The JSON data is piped to the python command, which processes it and outputs the result.

The -c option in the python command stands for "command." It allows you to specify a command or a short script as a command-line argument to be executed by the Python interpreter.

Triple quotes with json(multi-line JSON-like structure)

Triple quotes (''' or """) in Python are used to define multi-line strings. They allow you to create strings that span multiple lines without the need for explicit line breaks or escape characters.

In the code snippet below, the triple quotes are used to define a multi-line string that represents the JSON-formatted data. This makes it easier to write and read the JSON data, especially when it spans multiple lines.

For example:

s_my_dict = '''
{
    "is_active": true,
    "number": 12
}
'''

In this case, the JSON-formatted string is written with proper indentation and line breaks for clarity. The triple quotes preserve the newlines and indentation within the string, so it will be parsed correctly when passed to json.loads().

Note that triple quotes can also be used to define multi-line strings that are not necessarily JSON-formatted. They are useful whenever you need to work with strings that span multiple lines in your code.

Using single or double quotes

You can achieve the same multi-line string functionality using single or double quotes as well.

In Python, single quotes (') and double quotes (") are used to define string literals. You can use them to create single-line strings or multi-line strings by using explicit line breaks and escape characters.

Here's an example of defining a multi-line string using single quotes:

# Using single quotes, same rules aplies to double quotes
s_my_dict = '{'\
'"is_active": true,'\
'"number": 12'\
'}'

# Parse the string into a dictionary
my_dict = json.loads(s_my_dict)

# Access the values
is_active = my_dict['is_active']
number = my_dict['number']

# Print the values
print("is_active:", is_active)
print("number:", number)

In both cases, the string is split into multiple lines using explicit line breaks and the backslash character (\). This allows you to visually separate the lines and make the string more readable.

However, triple quotes have the advantage of automatically preserving the newlines and indentation within the string without the need for explicit line breaks or escape characters. This makes them more convenient when working with multi-line strings that require complex formatting, such as JSON-formatted data.

JSON serialization and deserialization with python