PHP-Style JSON Parsing in Java with Jsoniter

Share this article

JSON parsing in Java like in PHP with Jsoniter

Key Takeaways

  • Jsoniter addresses the JSON impedance mismatch in Java by offering flexible parsing strategies, including data binding with fuzzy typing, the dynamic Any data type, and a lazy parsing technique that processes only requested fields.
  • The Any data type in Jsoniter mimics PHP’s array functionality, allowing for easy handling of JSON with uncertain types and structures, making JSON parsing in Java as straightforward as in PHP.
  • Jsoniter enhances performance by avoiding unnecessary parsing and deserialization, thanks to its lazy parsing approach which leaves unaccessed parts of the JSON input in their original byte array form.
  • The library provides multiple APIs, including data binding, an iterator API, and the use of the Any type, which can be mixed and matched depending on the specific needs of the application, offering a tailored approach to JSON parsing.
  • Jsoniter supports schema-less and partial JSON processing, enabling modifications to JSON data without full deserialization and re-serialization, thus optimizing both performance and flexibility in handling JSON in Java applications.

Table of Contents

JSON originated from a weakly-typed and dynamic language, Javascript. There is an impedance mismatch between JSON’s dynamic nature and Java’s rigid typing. I found existing solutions too focused on the concept of data binding, which is too heavy-weight in some circumstance. Contrast that with PHP, where PHP we have the all-in-one data type Array, and by just one line of json_decode we can parse a complex JSON document. Jsoniter is a new library written in Java, determined to make JSON parsing in Java as easy as in PHP through a similar data type: Any. The most remarkable feature is the underlying lazy-parsing technique, which makes the parsing not only easy, but very fast.

Why JSON Is Hard to Process in Java

There are three reasons why JSON documents can be hard to process using existing parsers. I call this the “JSON impedance mismatch”.

Reason 1: Type Mismatch

When JSON is used as a data exchange format between Java and dynamic languages like PHP, the object field types might become a problem. For example, have a look at this JSON:

{
"order_id": 100098,
"order_details": {"pay_type": "cash"}
}

99% of the times, the PHP code might return the exact structure we expect. But it might also return slightly different JSON for different input conditions, due to the fact most PHP developers do not care if a variable is string or int.

{
"order_id": "100098",
"order_details": []
}

Why is order_details an empty array instead of an empty object? It is a common problem when working with PHP, where everything is an array. An array used as a non-empty map will be encoded like {"key":"value"} but an empty map is just an empty array, which will be encoded as [] instead of {}.

It is not a big problem, definitely fixable, but for historical data like logs, we have to deal with it anyway.

Reason 2: Heterogeneous Data

In Jave, we are used to homogeneous data. For example, [1, 2, 3] is an int array, ["1", "2", "3"] a String array. But how do you represent [1, 2, "3"] in Java? An object array Object[] is awkward to work with. How about [1, ["2", 3]]? Java does not have a convenient container to hold this kind of data.

Moreover, it is very common in JSON to have slightly different structures representing the same thing. For example, a success response:

{
"code": 0,
"data": "Success"
}

But for an error response:

{
"code": -1,
"error": {"msg": "Wrong Parameter", "stacktrace": "…"}
}

If we want to get the data or error message, we have make a number of null checks. Assuming the response is represented as Map<String, Object>, the code to extract the error message will look as follows:

Object errorObj = response.get("error");
if (errorObj == null)
    return "N/A";

Map<String, Object> error = (Map<String, Object>)errorObj;
Object msgObj = errorObj.get("msg");
if (msgObj == null)
    return "N/A";

return (String)msgObj;

The type casting and null checking is not fun at all. Unfortunately, it is common to extract value from a JSON five levels deep!

Reason 3: Performance and Flexibility Balance

By going with JSON, we have already chosen flexibility instead of raw performance. However, it still feels bad to parse a JSON document as Map<String, Object>, knowing that it will be very costly. I am not arguing we should choose the performance over expressiveness. But the guilt of deliberately compromising performance constantly troubles me. It is a dilemma I find myself in frequently:

  • Parse the JSON as Map<String, Object> and read values from it. Saves the trouble of defining a schema class but we have to unmarshall all the bytes, regardless if we need them or not.
  • Define a class and use data binding. It can skip unneeded parsing work, and accessing an object is faster than a hash map. But is it worth the trouble every time?
  • Some JSON parser come with a streaming API, but it is considered too low level.

There is a long way between totally type-less parsing and rigid data binding. It would be better if we have more options to choose between performance and flexibility, or both.

Parsing JSON in Java like in PHP with Jsoniter

How Jsoniter Solves the JSON Impedance Mismatch

Jsoniter is a new JSON library for Java, designed with the above problems in mind. (Disclaimer: I am its author.) Jsoniter responds to the JSON impedance mismatch with the following techniques:

  • Data binding supports “fuzzy” typing by pre-defined decoders like MaybeStringLongDecoder.
  • The Any data type represents the JSON object, similar to the way a PHP array does.
  • Lazy parsing only processes the requested fields and leaves other bytes untouched.

To demonstrate how to use Jsoniter, let’s install it first. Add the following dependency into your pom.xml (assuming you are using Maven):

<dependency>
    <groupId>com.jsoniter</groupId>
    <artifactId>jsoniter</artifactId>
    <version>0.9.8</version>
</dependency>

Or you can download the jar directly.

Jsoniter is a flexible parser, with 3 APIs you can choose from

And you will not be forced to stick with one API all the time. Use the right API for the right job, and combine them for complex cases. Now I am going to show you how to easily deal with JSON using these three APIs.

Data Binding

Jsoniter does not force you to use the Any type. For many cases, data binding is still the most comfortable API.

Simple Classes

Let’s bind this simple example:

{
"order_id": 100098,
"order_details": {"pay_type": "cash"}
}

For that, we will design a class like this:

public class Order {
    public long order_id;
    public OrderDetails order_details;
}

public class OrderDetails {
    public String pay_type;
}

To deserialize the JSON input, we will use JsonIterator:

Order order = JsonIterator.deserialize(input, Order.class);

The input can be String or byte[]. If you need to use an InputStream as input, it will be a little bit more verbose:

JsonIterator iter = JsonIterator.parse(input);
Order order = Iter.read(Order.class);
// you can close the underlying InputStream via Iter
// or directly (it does not have its own resource to dispose)
Iter.close();

A Case for Annotations

Everybody knows how simple binding works. But what about messy input?

{
"order_id": "100098",
"order_details": []
}

We will need annotation support in this case. First, we enable this optional feature through:

JsoniterAnnotationSupport.enable();

This only needs to be done once, you can put it in the main function or static initializer. Now add annotations to the Order class

public class Order {
  @JsonProperty(decoder = MaybeStringLongDecoder.class)
  public long order_id;
  @JsonProperty(decoder = MaybeEmptyArrayDecoder.class)
  public OrderDetails order_details;
}

By using the Maybe decoder, we can make the binding fuzzy about data types in some cases. If the structure itself is “dynamic”, we’d better use Any instead.

Any Data Type

Instead of defining a class describing the data schema, we can use the Any data type. It is pretty much a replacement for Map<String, Object> or List<Object>. Let’s read the same JSON as before:

{
"order_id": 100098,
"order_details": {"pay_type": "cash"}
}

This is the code to do that:

Any order = JsonIterator.read(input);
String payType = order.toString("order_details", "pay_type");

The toString method might look weird, so let me explain:

  • Get "order_details"
  • Then get "pay_type" from the "order_details"
  • Then convert the value of "pay_type" from whatever type to a string

Even in the following case the code still works, because it converts 5 to "5":

{
"order_details": {"pay_type": 5}
}

What if the input is not what we expect, for example:

{
"order_details": []
}

The code toString("order_tails", "pay_type") will not throw a NullPointerException, instead it will return the empty string. Most of the time, the empty string is what we expect.

It is worth mentioning that the parsing is done lazily. For those parts you do not read from, they will be kept in byte array form, saving the cost of full deserialization. Any is very powerful, we will cover it in details after looking at the third way to access JSON.

Iterator API

The iterator API expose the JSON data stream as an iterator. You can use the following methods to drive the iteration process:

  • whatIsNext: Look ahead at the type of the next value. It returns an instance of the ValueType enum, which I’ll come back to later. Using this method is optional – if you know the next value must be, for example, a string, you can directly call readString without checking whatIsNext first.
  • readObject: Read the next object field, returning the field name.
  • readArray: Read the next array element, return false if the end of the array was reached.
  • readString: Read an individual value as a string.

Let’s use this example input:

{"numbers": ["1", "2", ["3", "4"]]}

I have written a JUnit test to demonstrate the iterator API:

JsonIterator iter = JsonIterator.parse(
        "{'numbers': ['1', '2', ['3', '4']]}"
            .replace('\'', '"'));
// start reading the first object ("number")
assertEquals("numbers", iter.readObject());
// start reading the array
assertTrue(iter.readArray());
assertEquals("1", iter.readString());
assertTrue(iter.readArray());
assertEquals("2", iter.readString());
// start reading the inner array
assertTrue(iter.readArray());
// you can know the type of next value before reading it
assertEquals(ValueType.ARRAY, iter.whatIsNext());
assertTrue(iter.readArray());
assertEquals(ValueType.STRING, iter.whatIsNext());
assertEquals("3", iter.readString());
assertTrue(iter.readArray());
assertEquals("4", iter.readString());
// end inner array
assertFalse(iter.readArray());
// end outer array
assertFalse(iter.readArray());
// end object "number"
assertNull(iter.readObject());

It is actually just what its name suggests, an iterator: You call a method and it moves forward.

Fun with Any

Any is fun, let’s have more.

Any Container

Any is a container, that can hold all kinds of values:

  • lazy object
  • lazy array
  • lazy string
  • lazy double
  • lazy long
  • non lazy value (array, object, string, float, double, long, int, true, false, null)

If the contained value is an object or an array, we can extract elements without converting to List or Map.

For example_

[{"score":100}, {"score":102}]

We can extract value using just the path:

// will be 100
JsonIterator.deserialize(input).toInt(0, "score")

The first argument 0 get the first element out of the array. The second argument "score" get the score out of the object.

Or we can iterate the value like a collection:

Any records = JsonIterator.deserialize(input);
for (Any record : records) {
    Any.EntryIterator entryIterator = record.entries();
    while (entryIterator.next()) {
        System.out.println(
                entryIterator.key() + " / " + entryIterator.value());
    }
}
// output is:
// score / 100
// score / 102

The iterator is doing the parsing along the way. If you stop the loop in the middle, parsing will be partially done. This avoids unnecessary parsing once we have extracted the value we need.

We can even use wildcards in the extraction path:

Any records = JsonIterator.deserialize(input);
// [100, 102]
records.get('*', "score")

This will extract an Any with a list value, containing the score of each record.

Missing Value

Let’s re-visit a previous example

{
"order_details": []
}

As I explained before, toString("order_tails", "pay_type") will return the empty string. This is how toString handles the missing value. If we change to get("order_details", "pay_type"), it can tell us the value is actually missing:

Any payType = order.get("order_details", "pay_type");
if (payType.valueType() == ValueType.INVALID) {
    // not found
}

If you try to use the invalid Any instance, an exception will be thrown. In this case, Any is very similar to Java 8’s Optional. Possible value types are:

  • INVALID
  • STRING
  • NUMBER
  • NULL
  • BOOLEAN
  • ARRAY
  • OBJECT

We can see, even the “null” JSON is not actually null in Java’s sense. It will be represented as an Any instance with valueType() == ValueType.NULL. Removing null from possible return values makes extracting values from deeply nested structure much more convenient as checking null all the way through is no longer needed.

Wildcard path extraction supports missing value as well:

// input is [{"score":100}, {"value":102}]
Any records = JsonIterator.deserialize(input);
// [100]
records.get('*', "score")

Because “score” is not found in the second record, it will be excluded from the result.

Type Conversion

The toString method is just one of the conversion supported, others are:

  • toInt
  • toLong
  • toDouble
  • toFloat
  • toBoolean

Every conversion will make its best effort to convert original value to the type you want.

Besides simple types, you can convert the value into complex type by data binding. For example we can extract the value using Any, then bind into a object.

// {"numbers": ["1", "2", ["3", "4"]]}
String[] numbers = JsonIterator
        .deserialize(input)
        .get("numbers", 2)
        .as(String[].class);

The as API uses data binding as explained earlier to bind ["3", "4"] to a String[] object.

Schema-less Partial JSON Processing

Any is also mutable, and can be serialized back to JSON. If you only want to change a little bit of original input and then write it back, Any will be really handy. It will capture the input as raw bytes, and write back into JSON as it is, saving not only the cost of deserialization but also the cost of serialization. The underlying optimization happens automatically without your involvement – you write the code just as if you were working with Map<String, Object> or List<Object>:

List numbers = JsonIterator.deserialize("[1,[2, 3],4]").asList();
numbers.add(5);
// will be [1,[2,  3],4,5]
JsonStream.serialize(numbers);

This is partial processing – hard to notice where the magic is.

  • When asList is called, the list will contain 3 Any elements representing 1, [2, 3], and 4 in the original byte array, not parsed.
  • When 5 is added, the first 3 list elements remain of type Any but the 4th one is of type java.lang.Integer.
  • When we serialize the list back into JSON form, the first 3 elements will not have serialization cost, as it is still in byte array form and will be byte copied directly. Only the 4th element will be converted from integer to string.

This technique enables a whole new way to process JSON. Traditionally, we write our logic in this form:

  • JSON => Object Graph => Modified Object Graph => JSON

With Any, we can save a lot of objects:

  • JSON => Lazy Object Graph => Partially Parsed & Modified Object Graph => JSON

Summary

JSON is a flexible format and the output produced by code written in languages like PHP is often hard to process in Java. Unlike most existing parsers, Jsoniter chooses to embrace the dynamic nature instead of burying it. The innovative Any data type makes parsing JSON with uncertain types and uncertain structure easy in Java. With lazy parsing, this schema-less style parsing is even more attractive. While being extremely flexible, the performance is not compromised.

Frequently Asked Questions (FAQs) about JSON Parsing in Java with JSONiter

What is JSONiter and why is it used for JSON parsing in Java?

JSONiter, also known as json-iterator, is a high-performance JSON library in Java. It is used for parsing JSON data because it is faster and more efficient than traditional methods. JSONiter provides a simple and straightforward way to convert JSON data into Java objects and vice versa. It is highly customizable and can handle complex JSON structures, making it a preferred choice for developers working with JSON data in Java.

How does JSONiter compare to other JSON parsing libraries in Java?

JSONiter stands out from other JSON parsing libraries in Java due to its high performance and ease of use. It is designed to be fast, efficient, and flexible. Unlike other libraries, JSONiter does not require you to create a class for each JSON structure. Instead, it allows you to parse JSON data directly into Java objects. This makes it a more convenient and efficient choice for developers.

How can I install and use JSONiter in my Java project?

To use JSONiter in your Java project, you need to add it as a dependency in your project’s build file. If you’re using Maven, you can add the following dependency to your pom.xml file:

<dependency>
<groupId>com.jsoniter</groupId>
<artifactId>jsoniter</artifactId>
<version>0.9.23</version>
</dependency>

After adding the dependency, you can import the JSONiter library in your Java code and use it to parse JSON data.

Can JSONiter handle complex JSON structures?

Yes, JSONiter can handle complex JSON structures. It provides a flexible and powerful API that allows you to parse JSON data into any Java type. This includes complex types like lists, maps, and custom objects. JSONiter also supports nested JSON structures, making it a versatile choice for parsing JSON data in Java.

How can I parse a JSON array with JSONiter?

Parsing a JSON array with JSONiter is straightforward. You can use the readArray method of the JsonIterator class to parse a JSON array. This method returns an Iterator that you can use to iterate over the elements of the array. Here’s an example:

String jsonArray = "[1, 2, 3]";
JsonIterator iter = JsonIterator.parse(jsonArray);
for (int num : iter.readArray()) {
System.out.println(num);
}

How can I parse a JSON object with JSONiter?

To parse a JSON object with JSONiter, you can use the read method of the JsonIterator class. This method returns a Any object that you can use to access the properties of the JSON object. Here’s an example:

String jsonObject = "{\"name\": \"John\", \"age\": 30}";
JsonIterator iter = JsonIterator.parse(jsonObject);
Any any = iter.read();
System.out.println(any.get("name").toString());
System.out.println(any.get("age").toInt());

How can I convert a Java object to JSON with JSONiter?

JSONiter provides a simple and efficient way to convert Java objects to JSON. You can use the toJson method of the JsonStream class to convert a Java object to a JSON string. Here’s an example:

Person person = new Person("John", 30);
String json = JsonStream.serialize(person);
System.out.println(json);

Does JSONiter support streaming of JSON data?

Yes, JSONiter supports streaming of JSON data. This is particularly useful when dealing with large JSON data that cannot fit into memory. JSONiter provides a JsonStream class that you can use to stream JSON data in a memory-efficient manner.

Can I customize the behavior of JSONiter?

Yes, JSONiter is highly customizable. You can customize its behavior by configuring the JsonIterator and JsonStream classes. For example, you can configure JSONiter to use a specific naming strategy for JSON properties, or to ignore certain properties during serialization.

Is JSONiter compatible with Android?

Yes, JSONiter is compatible with Android. It is a lightweight library with no external dependencies, making it a suitable choice for Android development. You can use JSONiter to parse JSON data in your Android apps in the same way as in a standard Java application.

Tao Wen has over 10 years experience working with all sorts of platform ranging from Delphi 4 to PHP 7. He is interested in marrying different technologies to solve real world problem. Currently, he is working for Didichuxing as platform architect.

backendjsonLazinessnicolaip
Share this article
Read Next
Get the freshest news and resources for developers, designers and digital creators in your inbox each week