# JSON serialization and deserialization with python ### json serialization and deserialization JSON (JavaScript Object Notation) serialization and deserialization refer to the processes of converting data between JSON format and a programming language's data structures. **Serialization** is the process of converting a data object or data structure into a JSON string representation. In other words, it involves encoding the data in a format that can be easily transmitted or stored. This is typically done when you want to send data over a network or save it to a file. To serialize a Python object and save it as a JSON file, you can use the `json` module provided by Python's standard library. Here's an example of how you can accomplish this: ```python import json # Python object to be serialized person = { "name": "John Doe", "age": 30, "city": "New York" } # Serialize the object to a JSON string json_data = json.dumps(person) # Write the JSON string to a file with open('person.json', 'w') as json_file: json_file.write(json_data) ``` In this example, the `person` dictionary represents the Python object that you want to serialize. Using the `json.dumps()` function, you can convert the Python object into a JSON string. Next, you can open a file in write mode (specified by `'w'`) using the `open()` function and write the JSON string to the file using the `write()` method. Here, the file name is "person.json", but you can choose any desired name. After executing the code, a JSON file named "person.json" will be created, containing the serialized JSON representation of the Python object. Note that the `json.dumps()` function accepts additional parameters to control the formatting and customization of the JSON output. For example, you can use `indent` to specify the indentation level for pretty-printing the JSON data. **Deserialization**, on the other hand, is the process of converting a JSON string back into a data object or data structure in a specific programming language. It involves decoding the JSON string and reconstructing the original data. Here's an example of a JSON file representing a collection of books: ```json { "books": [ { "title": "The Great Gatsby", "author": "F. Scott Fitzgerald", "year": 1925, "genre": "Fiction" }, { "title": "To Kill a Mockingbird", "author": "Harper Lee", "year": 1960, "genre": "Fiction" }, { "title": "1984", "author": "George Orwell", "year": 1949, "genre": "Science Fiction" } ] } ``` In this example, the JSON file contains a key `"books"` whose associated value is an array of book objects. Each book object has properties like `"title"`, `"author"`, `"year"`, and `"genre"`, representing the title, author, publication year, and genre of a book, respectively. To deserialize this JSON file, you would read its contents and use a JSON deserialization function provided by your programming language to convert it into a corresponding data structure, such as an array of book objects in this case. To deserialize the JSON file and convert it into a Python object, you can make use of the `json` module, which is part of the Python standard library. Here's an example of how you can accomplish this: ```python import json # Read the JSON file with open('books.json') as json_file: json_data = json.load(json_file) # Access the deserialized data books = json_data['books'] # Iterate over the books for book in books: title = book['title'] author = book['author'] year = book['year'] genre = book['genre'] # Perform desired operations with the book data print(f"Title: {title}, Author: {author}, Year: {year}, Genre: {genre}") ``` In this example, assuming the JSON file is named "books.json", we open the file using `open()` and then use `json.load()` to deserialize the JSON data into a Python object. We can then access the deserialized data as a Python dictionary. In this case, we access the list of books using the key `'books'`. We can iterate over the list of books and access the individual book properties such as title, author, year, and genre. You can perform any desired operations with the book data within the loop. Make sure to replace `'books.json'` with the actual file path if it's different, and ensure that the JSON file is in the correct format. Note: It's important to handle any potential exceptions that may occur during file reading or JSON deserialization, such as `FileNotFoundError` or `json.JSONDecodeError`, to ensure your code handles potential errors gracefully. **Output:** ```plaintext Title: The Great Gatsby, Author: F. Scott Fitzgerald, Year: 1925, Genre: Fiction Title: To Kill a Mockingbird, Author: Harper Lee, Year: 1960, Genre: Fiction Title: 1984, Author: George Orwell, Year: 1949, Genre: Science Fiction ``` ### **Difference between json.load() and json.loads():** The `json.load()` function is used to parse JSON data from a file-like object, such as a file object or a `StringIO` object, and convert it into a Python object. Unlike `json.loads()`, which takes a JSON string as input, `json.load()` directly reads the JSON data from a file-like object. The `json.load()` function has the following parameter: * `fp`: This parameter specifies the file-like object containing the JSON data to be parsed. It can be a file object opened in read mode (`'r'`), a `StringIO` object, or any other object that supports the file protocol (`read()` method). Here's an example demonstrating the usage of `json.load()`: ```python import json # Open a file containing JSON data with open('data.json', 'r') as file: # Parse the JSON data from the file data = json.load(file) # Access the parsed JSON data as a Python object print(data) ``` In this example, the `open()` function is used to open a file named `'data.json'` in read mode (`'r'`). The file contains JSON data. The file object is then passed as the `fp` parameter to `json.load()` to parse the JSON data. The resulting Python object is stored in the `data` variable. You can then work with the `data` object as a regular Python object, which represents the parsed JSON data structure. The `json.loads()` function is used to parse a JSON-formatted string and convert it into a Python object. It takes a JSON string as input and returns the corresponding Python object. ### The `json.loads()` function has the following parameters: * `s`: This parameter specifies the JSON string to be parsed. It should be a valid JSON-formatted string. * `parse_float`: This optional parameter specifies a function that will be used to parse floating-point values in the JSON string. By default, floating-point values are parsed as `float` objects. However, you can provide a custom function to parse these values differently. * `parse_int`: This optional parameter specifies a function that will be used to parse integer values in the JSON string. By default, integer values are parsed as `int` objects. You can provide a custom function to parse integer values differently. * `parse_constant`: This optional parameter specifies a function that will be used to parse the JSON constant values `null`, `true`, and `false`. By default, these constant values are parsed as `None`, `True`, and `False`, respectively. You can provide a custom function to parse these constants differently. * `object_hook`: This optional parameter specifies a function that will be called for each object decoded from the JSON string. This function can be used to customize the decoding process and convert JSON objects into specific Python objects. * `object_pairs_hook`: This optional parameter specifies a function that will be called with the result of decoding a JSON object. It can be used to modify the result before returning. json.load() has one more additional parameter fp(file pointer), because it is used with files. **Difference between** `object_pairs_hook` **and** `object_hook` **in the** `json.loads()` **function.** Example JSON data: ```python { "person1": {"name": "John", "age": 25}, "person2": {"name": "Alice", "age": 30} } ``` Using `object_pairs_hook`: ```python import json def custom_object_pairs_hook(pairs): result = {} for key, value in pairs: result[key.upper()] = value return result json_data = '{"person1": {"name": "John", "age": 25}, "person2": {"name": "Alice", "age": 30}}' result = json.loads(json_data, object_pairs_hook=custom_object_pairs_hook) print(result) ``` Output: ```python {'PERSON1': {'name': 'John', 'age': 25}, 'PERSON2': {'name': 'Alice', 'age': 30}} ``` In this example, the `custom_object_pairs_hook` function is defined to convert the keys of each object to uppercase while keeping the values intact. The function is passed as the `object_pairs_hook` parameter to `json.loads()`. The resulting dictionary has the keys converted to uppercase. Using `object_hook`: ```python import json class Person: def __init__(self, name, age): self.name = name self.age = age def custom_object_hook(obj): if "name" in obj and "age" in obj: return Person(obj["name"], obj["age"]) return obj json_data = '{"person1": {"name": "John", "age": 25}, "person2": {"name": "Alice", "age": 30}}' result = json.loads(json_data, object_hook=custom_object_hook) print(result) ``` Output: ```python {'person1': <__main__.Person object at 0x...>, 'person2': <__main__.Person object at 0x...>} ``` In this example, the `custom_object_hook` function is defined to convert each dictionary object into a `Person` object if it contains the keys "name" and "age". The function is passed as the `object_hook` parameter to `json.loads()`. The resulting dictionary contains `Person` objects instead of the original dictionaries. In both examples, the provided custom functions (`custom_object_pairs_hook` and `custom_object_hook`) are called during the JSON deserialization process, allowing for customization of the resulting objects or dictionaries. The choice between `object_pairs_hook` and `object_hook` depends on the specific requirements and structure of the JSON data being processed. Here's an example demonstrating the usage of `json.loads()` with some optional parameters: ```python import json # JSON string json_str = '{"name": "John", "age": 30, "salary": 50000.0}' # Parse the JSON string data = json.loads(json_str) print(data) # Output: {'name': 'John', 'age': 30, 'salary': 50000.0} ``` In this example, the `json_str` variable contains a JSON-formatted string. The `json.loads()` function is called with the `json_str` parameter to parse the JSON string and convert it into a Python object. The resulting Python object is stored in the `data` variable. By default, the function uses the default parsing behavior for floating-point values, integer values, and JSON constants. However, you can provide custom functions for these behaviors using the optional parameters mentioned above. ### json.dumps() and few of its optional parameters: The `json.dumps()` function is used to serialize Python objects into a JSON-formatted string. It takes a Python object as input and returns a JSON string representation of that object. The `json.dumps()` function has several optional parameters that allow you to customize the serialization process. Here are the parameters commonly used with `json.dumps()`: * `obj`: This parameter specifies the Python object to be serialized into JSON. It can be any object that is JSON serializable, such as dictionaries, lists, strings, numbers, booleans, or `None`. * `skipkeys`: This optional parameter is a boolean value that specifies whether to skip serializing dictionary keys that are not of basic types (`str`, `int`, `float`, `bool`, `None`). If `skipkeys` is set to `True`, dictionary keys of unsupported types will be skipped during serialization. If `skipkeys` is `False` (default), a `TypeError` will be raised if such keys are encountered. * `ensure_ascii`: This optional parameter is a boolean value that specifies whether to escape non-ASCII characters in the output with Unicode escape sequences. If `ensure_ascii` is `True` (default), non-ASCII characters will be escaped. If `ensure_ascii` is `False`, non-ASCII characters will be output as is. * `check_circular`: This optional parameter is a boolean value that specifies whether to check for circular references during serialization. If `check_circular` is `True` (default), a `ValueError` will be raised if a circular reference is encountered. If `check_circular` is `False`, circular references will be handled, but the resulting JSON may not be valid. * `allow_nan`: This optional parameter is a boolean value that specifies whether to allow NaN, Infinity, and -Infinity as numeric values in the output. If `allow_nan` is `True` (default), these special float values will be allowed. If `allow_nan` is `False`, a `ValueError` will be raised if any of these values are encountered. * `indent`: This optional parameter specifies the indentation level to be used in the formatted JSON output. It can be an integer representing the number of spaces per indentation level, or a string containing the characters used for indentation (e.g., `"\t"`). If `indent` is `None` (default), no indentation will be applied, and the output will be compact. * `separators`: This optional parameter specifies the separators used in the output JSON string. It should be a tuple of two strings: `(item_separator, key_separator)`. The `item_separator` is the string used to separate items in a JSON object or array, and the `key_separator` is the string used to separate keys and values in a JSON object. The default separators are `(", ", ": ")`. * `sort_keys`: This optional parameter is a boolean value that specifies whether to sort the keys of dictionaries alphabetically in the output. If `sort_keys` is `True`, the dictionary keys will be sorted. If `sort_keys` is `False` (default), the keys will be serialized in the order they are encountered. * `cls`: More on this below These are the most commonly used parameters of `json.dumps()`. You can refer to the Python documentation for the `json` module for a complete list of available parameters and their descriptions. ### cls In the context of the `json.dumps()` function, the `cls` parameter is used to specify a custom JSON encoder class to be used for serialization. The `cls` parameter stands for "class" and expects a class object that is a subclass of `json.JSONEncoder`. When you pass a custom encoder class as the `cls` parameter to `json.dumps()`, it instructs the `dumps()` function to use that custom encoder class for serializing the Python objects into JSON. example: ```python import json class ComplexEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, complex): return [obj.real, obj.imag] # Let the base class default method raise the TypeError return json.JSONEncoder.default(self, obj) json.dumps(2 + 1j, cls=ComplexEncoder) ``` The `cls` parameter is used to specify the `ComplexEncoder` class as the custom JSON encoder class. This means that when serializing the complex number `2 + 1j`, the `ComplexEncoder` class will be responsible for converting it into the desired JSON representation. By specifying `cls=ComplexEncoder`, you are telling `json.dumps()` to use the `default()` method implemented in the `ComplexEncoder` class to handle the serialization of the complex number object. Note that the `cls` parameter is optional, and if not provided, the default behavior of `json.dumps()` will be used, which uses the standard `json.JSONEncoder` class for serialization. However, by specifying a custom encoder class using `cls`, you have the flexibility to define your own serialization logic and handle special cases or custom data types as needed. ### Using json from the shell ```python echo '{"json":"obj"}' | python -m json.tool ``` The command `echo '{"json":"obj"}' | python -m json.tool` is a command-line instruction that makes use of the `json.tool` module in Python to format and validate JSON data `too`l is a submodule in the json module. The `-m` option is used to run a Python module directly as a script. When you execute `python` from the command line, it searches for the Python interpreter in the directories listed in the system's `PATH` environment variable. Once it finds the Python interpreter, it runs the specified code or module. The `-m` option is used to specify a module to be executed. In this case, `json.tool` is the module that is being executed. The `json.tool` module is a built-in module in Python's standard library. It provides a command-line interface for formatting and validating JSON data. ### How to access classes and functions inside a module from the shell without explicitly opening the python interpreter ```python echo '{"json":"obj"}' | python -c "import json; print(json.dumps({'json':'obj'}))" ``` In the example above, the `python -c` command is used to execute a Python command. The Python code specified within the double quotes after `-c` imports the `json` module and then calls the `dumps()` function to serialize the JSON data. The `echo` command is used to pass the JSON data `{"json":"obj"}` as input to the `python -c` command. The JSON data is piped to the `python` command, which processes it and outputs the result. The `-c` option in the `python` command stands for "command." It allows you to specify a command or a short script as a command-line argument to be executed by the Python interpreter. ### Triple quotes with json(multi-line JSON-like structure) Triple quotes (''' or """) in Python are used to define multi-line strings. They allow you to create strings that span multiple lines without the need for explicit line breaks or escape characters. In the code snippet below, the triple quotes are used to define a multi-line string that represents the JSON-formatted data. This makes it easier to write and read the JSON data, especially when it spans multiple lines. For example: ```python s_my_dict = ''' { "is_active": true, "number": 12 } ''' ``` In this case, the JSON-formatted string is written with proper indentation and line breaks for clarity. The triple quotes preserve the newlines and indentation within the string, so it will be parsed correctly when passed to json.loads(). Note that triple quotes can also be used to define multi-line strings that are not necessarily JSON-formatted. They are useful whenever you need to work with strings that span multiple lines in your code. ### Using single or double quotes You can achieve the same multi-line string functionality using single or double quotes as well. In Python, single quotes (') and double quotes (") are used to define string literals. You can use them to create single-line strings or multi-line strings by using explicit line breaks and escape characters. Here's an example of defining a multi-line string using single quotes: ```python # Using single quotes, same rules aplies to double quotes s_my_dict = '{'\ '"is_active": true,'\ '"number": 12'\ '}' # Parse the string into a dictionary my_dict = json.loads(s_my_dict) # Access the values is_active = my_dict['is_active'] number = my_dict['number'] # Print the values print("is_active:", is_active) print("number:", number) ``` In both cases, the string is split into multiple lines using explicit line breaks and the backslash character (\\). This allows you to visually separate the lines and make the string more readable. However, triple quotes have the advantage of automatically preserving the newlines and indentation within the string without the need for explicit line breaks or escape characters. This makes them more convenient when working with multi-line strings that require complex formatting, such as JSON-formatted data.