Parsing Large Json Files Using Jackson Streaming Api Example

In concluding pair of JSON tutorials for Java programmers, nosotros accept learned how to parse JSON using JSON-Simple library, parsing JSON array to Java array using GSon, together with inward this tutorial nosotros volition larn how to parse a large JSON file inward Java using Jackson's Streaming API. Jackson is i of the most pop JSON processing framework together with provides iii principal model to parse together with procedure JSON information including Streaming API, information binding together with tree model. Out of these three, Streaming plant at lowest flat together with tin live used to parse huge JSON answer upto fifty-fifty giga bytes of size. If y'all are familiar alongside XML parsing, hence y'all know that how hard it is to parse huge XML files alongside DOM parser because it fully loads the file inward retention earlier y'all tin procedure it. In instance y'all accept depression retention e.g. Android devices y'all can't role that to parse XML. Thankfully, XML provides SAX together with StAX parsers which are streaming based together with tin live used to procedure huge files without loading them completely inward memory. Out of these two, StAX is fifty-fifty amend because it allows force based processing where customer pulls information from parser instead of parser pushing data, which is the instance alongside SAX parser. Jackson's Streaming API is similar to StAX parser. You tin force the information y'all desire together with ignore what y'all don't want. Though functioning doesn't come upwards without cost, using Streaming API is piffling hard hence using other Jackson model which provides conduct mapping betwixt Java together with Jackson objects. You accept to handgrip all JSON information past times yourself piece using Streaming API.

Benefits of using Jackson Streaming API

There are several advantages of using Jackson's Streaming API to parse JSON String or convert Java object to JSON, but the most of import i is that its real efficient. It has to the lowest degree retention together with processing overhead together with extremely useful to parse large JSON responses, for instance a JSON answer containing thousands of lodge or listing of books or listing of electronic items downloaded from e-commerce sites similar eBay or Amazon. Talking nearly other ii model of Jackson API, information binding model converts JSON to together with from Java object based either notation or Java edible bean convention, piece Tree Model provides a mutable in-memory tree representation of a JSON document, similar to DOM parser. In short, Streaming API is most powerful, has less retention together with CPU overhead but tricky to use, piece information binding is ofttimes most convenient, on the other mitt Tree Model is most flexible. BTW, both of this model internally uses streaming API to parse JSON strings earlier converting it into respective models.

Library JARs together with Dependency

In lodge to endeavor next example, y'all ask to download together with add together Jackson streaming API inward your program's classpath. If y'all are using Maven hence y'all tin add together next dependency inward your pom.xml file :

<dependency>    <groupId>org.codehaus.jackson</groupId>    <artifactId>jackson-xc</artifactId>    <version>1.9.12</version> </dependency>

or only download together with  add next JAR inward CLASSPATH of your Java application.

C:\.m2\repository\org\codehaus\jackson\jackson-xc\1.9.12\jackson-xc-1.9.12.jar C:\.m2\repository\org\codehaus\jackson\jackson-core-asl\1.9.12\jackson-core-asl-1.9.12.jar C:\.m2\repository\org\codehaus\jackson\jackson-mapper-asl\1.9.12\jackson-mapper-asl-1.9.12.jar 

It's ofttimes easier to care dependency using Maven together with that's why I strongly advise to switch to Maven if y'all are non using it yet. You tin afterwards upgrade to newer version of Jackson library past times only changing i business inward Maven pom.xml file.

Parsing JSON inward Java using Jackson Streaming API

This API has ii principal module, i fore reading JSON together with other for writing JSON together with inward this tutorial nosotros volition larn both of them. JsonGenerator is used to write JSON piece JsonParser is used to parse a JSON file. To demonstrate both reading together with writing of JSON information inward i program, I accept created ii static methods, createJSON() together with parseJSON(). As advert suggests kickoff method creates a JSON file, which is hence read past times parseJSON() method. You tin run into inward the code that nosotros are dealing alongside quite depression level, nosotros accept non created whatever Java object to stand upwards for content of JSON, instead nosotros are writing together with reading String, numbers together with arrays.

You tin acquire an instance of JsonGenerator from JsonFactory course of report past times calling createJsonGenerator() method. You tin too render the encoding y'all are intended to use, inward our instance I accept used "UTF-8" which is a convenient default inward most cases. You tin role diverse write() methods to write contents.  Similarly, for parsing JSON, nosotros ask to exercise an instance of JsonParser, which tin too live obtained from JsonFactory.  We parse JSON past times calling nextToken() method of JsonParser inward a piece loop until nosotros accomplish JsonToken.END_OBJECT. Jackson API provides method to acquire advert together with value of token which y'all tin role to position data. Similarly piece parsing JSON array, y'all hold off until y'all acquire JsonToken.END_ARRAY identifier. Since nosotros never charge the whole file inward memory, this method tin live used to read large JSON files alongside sizes from Mega bytes to Giga bytes fifty-fifty alongside minimal retention environs e.g. inward Android smartphones or Java ME enabled devices.

Here is the sample code instance to read together with write JSON using Jackson Streaming API :

import java.io.File; import java.io.IOException;  import org.codehaus.jackson.JsonEncoding; import org.codehaus.jackson.JsonFactory; import org.codehaus.jackson.JsonGenerationException; import org.codehaus.jackson.JsonGenerator; import org.codehaus.jackson.JsonParser; import org.codehaus.jackson.JsonToken; import org.codehaus.jackson.map.JsonMappingException;  /** * Java programme to demonstrate how to role Jackson Streaming API to read together with * write JSON Strings efficiently together with fast. * * @author Javin Paul */ public class JsonJacksonStreamingAPIDemo{      public static void main(String args[]) {          System.out.println("Creating JSON file past times using Jackson Streaming API inward Java");         createJSON("jacksondemo.json");         System.out.println("done");          System.out.println("Parsing JSON file past times using Jackson Streaming API");         parseJSON("jacksondemo.json");         System.out.println("done");     }      /*      * This method exercise JSON String past times using Jackson Streaming API.      */     public static void createJSON(String path) {         try {             JsonFactory jsonfactory = new JsonFactory();             File jsonDoc = new File(path);             JsonGenerator generator = jsonfactory.createJsonGenerator(jsonDoc, JsonEncoding.UTF8);              generator.writeStartObject();             generator.writeStringField("firstname", "Garrison");             generator.writeStringField("lastname", "Paul");             generator.writeNumberField("phone", 847332223);              generator.writeFieldName("address");              generator.writeStartArray();             generator.writeString("Unit - 232");             generator.writeString("Sofia Streat");             generator.writeString("Mumbai");             generator.writeEndArray();              generator.writeEndObject();              generator.close();              System.out.println("JSON file created successfully");          } catch (JsonGenerationException jge) {             jge.printStackTrace();         } catch (JsonMappingException jme) {             jme.printStackTrace();         } catch (IOException ioex) {             ioex.printStackTrace();         }     }      /*      * This method parse JSON String past times using Jackson Streaming API example.      */     public static void parseJSON(String filename) {         try {             JsonFactory jsonfactory = new JsonFactory();             File origin = new File(filename);              JsonParser parser = jsonfactory.createJsonParser(source);              // starting parsing of JSON String             while (parser.nextToken() != JsonToken.END_OBJECT) {                 String token = parser.getCurrentName();                  if ("firstname".equals(token)) {                     parser.nextToken();  //next token contains value                     String fname = parser.getText();  //getting text field                     System.out.println("firstname : " + fname);                  }                  if ("lastname".equals(token)) {                     parser.nextToken();                     String lname = parser.getText();                     System.out.println("lastname : " + lname);                  }                  if ("phone".equals(token)) {                     parser.nextToken();                     int telephone = parser.getIntValue();  // getting numeric field                     System.out.println("phone : " + phone);                  }                  if ("address".equals(token)) {                     System.out.println("address :");                     parser.nextToken(); // adjacent token volition live '[' which agency JSON array                      // parse tokens until y'all notice ']'                     while (parser.nextToken() != JsonToken.END_ARRAY) {                         System.out.println(parser.getText());                     }                 }             }             parser.close();          } catch (JsonGenerationException jge) {             jge.printStackTrace();         } catch (JsonMappingException jme) {             jme.printStackTrace();         } catch (IOException ioex) {             ioex.printStackTrace();         }     } 

together with hither is the output of our program, when y'all run it from Eclipse or conduct from ascendency business :

Creating JSON file past times using Jackson Streaming API inward Java JSON file created successfully done Parsing JSON file past times using Jackson Streaming API firstname : Garrison lastname : Paul telephone : 847332223 address : Unit - 232 Sofia Streat Bombay done

You volition too run into file jacksondemo.json inward your projection directory alongside next JSON String :

{   "firstname":"Garrison",   "lastname":"Paul",   "phone":847332223,    "address":["Unit - 232","Sofia Streat","Mumbai"] }

That's all nearly how to role Jackson Stream API to parse JSON String and to exercise JSON from Java object. It's a powerful library alongside lots of characteristic but Streaming is best. I know its piffling chip hard together with y'all ask to write lot of code alongside hard coded filed names, it is the fastest way to read a large JSON file inward Java alongside less retention overhead. If y'all are dealing alongside normal size JSON output together with y'all don't accept a retention constraints hence y'all tin e'er role Jackson Data binding model to parse JSON to Java Object.

Further Learning
Master Java Web Services together with REST API alongside Spring Boot
REST API Design, Development & Management
tutorial)

3 Ways to parse JSON String inward Java? (tutorial)

How to convert JSON array to String array inward Java? (example)

How to convert a Map to JSON inward Java? (tutorial)

How to role Google Protocol Buffer inward Java? (tutorial)

How to role Gson to convert JSON to Java Object? (example)

5 Books to Learn REST together with RESTful Web Services (books)

P.S. - If y'all are looking for online preparation to larn how to develop RESTful Web Services inward Java using Spring framework, I advise y'all joining Eugen Paraschiv's REST alongside Spring course. The course of report has diverse options depending upon your sense flat together with how much y'all desire to larn e.g. beginner's class, intermediate class, together with master copy class. You tin bring together the i which suits y'all better, though I advise joining the master class if y'all are serious nearly becoming an goodness Java REST developer.