Dynamic Webpages with JSON

Making asynchronous HTTP requests from Webpages is an effective technique in bringing seemingly static pages to life. Asynchronous JavaScript and XML (AJAX) has become a popular technology in creating richer and more dynamic Web clients, and is often used to incorporate desktop features in the browser. However, the usual XMLHttpRequest-based AJAX clients suffer from the limitation of only being able to communicate to the server from where they are downloaded. This becomes problematic for deployment environments that span multiple domains. In addition, developers end up writing browser-specific code since each of the main browsers implements this XML request object differently.

In this article, we describe an approach based on JavaScript Object Notation (JSON) that, in the spirit of Web 2.0, makes it easy to build mashup applications without the cross-domain and cross-browser limitations of AJAX. We discuss why the JSON-based approach is elegant in adding asynchronous features to Webpages and also mention some server-side utilities that can be used to generate JSON data.

Web 2.0

The way people interact on the Web has changed drastically. Many Web applications currently being developed are open, collaborative, and in perpetual beta. Characteristics like these define “Web 2.0,” with one of its more important characteristics being a rich browser client—one that resembles traditional desktop applications. Client-side technologies like JavaScript, which were relegated to doing menial client-side tricks/validations, have come to prominence and are playing an important role in delivering a richer experience to Web users. It seems the industry has found a new respect for JavaScript, and the AJAX technology has played a significant role in making this happen.

AJAX

There are several good resources on the Web about AJAX, but for the sake of completeness and setting the stage, we describe the technology briefly here: AJAX is a Web development technique for creating interactive Web applications. The intent is to make Webpages more responsive by exchanging small amounts of data with the server behind the scenes, so the entire Webpage does not have to reload each time the user makes a change. Thus, the Webpage’s interactivity, speed, and usability are increased. This technology has been effectively used in numerous showcase applications on the Web, with Google Maps being a prime example.

While using AJAX to develop an application that required the delivery of asynchronous features to Webpages across multiple domains, we ran into a limitation that led us to research alternative methods for delivering the promise of AJAX. We found an approach that addresses the cross-domain issues, helps with the cross-browser compatibility issues, and delivers some additional benefits as well.

Cross-domain issues with AJAX

Our project involved adding dynamic features to Webpages that were delivered by servers in different domains. An example of such a service that may need to work across domains is an employee directory service that can asynchronously be invoked from Webpages on a company’s intranet and Internet domains. Our idea was to use a consolidated services model, which would be unintrusive for the owners of the Webpages. Additional complexity arose because the Webpages were delivered by different technological platforms. Thus, our approach had to work with Webpages generated by a combination of dynamic and static technologies.

JavaScript-based AJAX seemed a good approach until we hit the limitation of AJAX clients only being able to communicate to the server from where they are downloaded, thus adhering to the “same origin policy.” Since we wanted to have the AJAX service running on a server in one domain and invoked using JavaScript from Webpages served in different domains, this limitation prompted us to look elsewhere. We ultimately found that a JSON-based approach employs a clever technique to asynchronously bring dynamic features to Webpages that span multiple domains.

In fact, there are two considerable drawbacks with using the standard AJAX solution. First, it creates a cross-domain issue that would limit how content is disseminated using your AJAX solution. In Figure 1, the pages are served from different domains, and the arrow in blue depicts how the AJAX-generated content can be downloaded only from the domain that the page was originally downloaded from. Second, since the browsers implement the XMLHttpRequest object differently, the JavaScript code must be coded to work in different browsers.

Introducing JavaScript Object Notation

JavaScript Object Notation (JSON) is a lightweight data-interchange format. It is easy for humans to read and write, and easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition. JSON is a text format that is completely language-independent, but uses conventions familiar to programmers of the popular programming languages. These properties make JSON an ideal data-interchange language.

JSON is built on two structures:

• A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hashtable, keyed list, or associative array.
• An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

These are universal datastructures. Virtually all modern programming languages support them in one form or another. It makes sense that a data format interchangeable with programming languages also be based on these structures.

Below are some examples of data in the JSON format:

•  displayStocks([{'symbol':'IBM','price':'93.39'}, {'symbol':'GE','price':'35.58'},{'symbol':'CTXS', 'price':'28.59'}]);
  
  function displayStocks(stocks){
        for(var i=0; i < stocks.length; i++){
             alert(stocks[i].symbol+":"+stocks[i].price);
        } 
  }
  

  Notice the JSON code in this example; it is the code passed as an argument to the displayStocks function. This example of an associative array represents stocks, and each array element has a name/value entry (symbol and value). The elements in the array are separated by a comma (,) and appear within curly braces; the name/value pairs have a colon in between. Notice how we iterate over the stock object using the index and calling the named element to retrieve its value.

•  displayCoworkers({'coworkers': {
      'NY' : ['Jeff', 'John', 'Scott', 'Joe', 'Ajay', 'Brian'],
      'FL' : ['Tom', 'Gabe', 'Andreas', 'Jason', 'Vishal', 'Saul']
       }});
  
  function displayCoworkers(obj){
      alert(obj.coworkers.NY[0]);
      alert(obj.coworkers.FL[5]);
  }
  

  This example is the inverse of the previous example. In this case, we have an associative array where each key is associated with an array of coworkers. Notice how we need to reference the coworkers object then the state object before we can access the array of coworkers (coworkers.NY[0]). Note that in order to have the JavaScript interpreter turn your JSON notation into datastructures, you must not use quotes around the JSON object. Doing so would cause the interpreter to think the JSON object was a simple string

JSON and the cross-domain issue

Figure 2 illustrates how JSON content can be disseminated across domains using cross-browser safe JavaScript:

1. Browser requests HTML document from HTTP server from www.domain-1.com.
2. The HTML downloaded by the browser in Step 1 contains a JavaScript include that directs the browser to request a JavaScript file from a second HTTP server from a different domain, www.domain-2.com.
3. The JavaScript downloaded in the previous step is executed in the browser; it modifies the page header element to dynamically include another JavaScript include.
4. This new JavaScript include is a call to a dynamic service. The service processes the request and returns a JavaScript callback method with a JSON object as its parameter. Note that the service’s MIME type is text/JavaScript. The callback method provides loose coupling between the JSON client and the actual service being called. It is desirable to implement a service so the callback function name can be specified as part of the invocation.
5. The callback method is executed, and the page is modified to reflect the application update.
6. The above points are diagrammatically captured in the interaction model below.

The approach described above for simulating a typical asynchronous communication between the client and server consists of some little-known JavaScript techniques. Together with JSON, they overcome the most painful limitations of the traditional AJAX approach: cross-domain and cross-browser issues:

• To communicate with the server using JavaScript dynamically, we can append a JavaScript include call to the head element of the DOM (Document Object Model) of the page. This allows the page to make asynchronous calls to the server whenever the application logic dictates. Example:

  function callServer(){ 
  headElement = document.getElementsByTagName("head").item(0); 
  var script = document.createElement("script"); 
  script.setAttribute("type", "text/javascript"); 
  script.setAttribute("src","http://server.com/app/?with=params"); 
  headElement.appendChild(scriptTag); 
  }
  

• The JavaScript calls are in actuality calls to a server application (like a Java EE application), which return a response with the MIME type HTML/JavaScript. The response should simply include a callback method with JSON-formatted data. The callback method is a JavaScript method already loaded in the page and should contain a JSON-formatted string as the parameter it passes. JavaScript allows you to pass a JSON-formatted object to a method and automatically deserializes it. For example, the server could return something like this: updateStock( { stocks: [ {'ibm': '91.52}, {'ford': '5.40'}, {'ctsx': '29.56'} ] } ); And the JavaScript callback method, which is JSON-aware, would look something like this:

   function updateStock(obj){ 
     displayStock( "ibm", obj.stocks.ibm ); // A function that displays the stock info
  }
  

They key thing to note in this hypothetical example is how we access the data inside the JSON object. JavaScript automatically transforms the JSON-formatted object into a corresponding JavaScript object.

How does this approach differ from traditional AJAX techniques?

The most important thing to note about the JSON approach is that it does not use the XMLHttpRequest object, which has significant implications: First, you are not constrained to making AJAX requests to only the calling server. Second, there are no significant cross-browser considerations because the JSON technique works exactly the same way in all modern browsers, thereby avoiding the need to add browser-specific code.

Sample application

The sample application uses a service running at http://www.theinnovationvalue.com/scripts/jsonexample?. The service takes a number as an input and returns its factorial and square values in two formats: XML and JSON, depending on the return_type parameter specified in the URL. The service is implemented purely for this article to demonstrate how the call from an AJAX client in a different domain would fail, whereas the call from a JSON client would work across domains.

This URL hosts a ‘jsonexample’ service that can be invoked using your browser in the format below and computes the factorial and square value for a given number less than 10: http://www.theinnovationvalue.com/scripts/jsonexample?return_type=&number=&callback=

• return_type = JSON or XML—the format in which the return data is sought
• number = the input number whose factorial and square values are needed
• callback = an optional parameter used in conjunction with the JSON format (the callback function is described above)

If you invoke this URL: https://www.theinnovationvalue.com/scripts/jsonexample?return_type=json&number=3&callback=factorial, you will see the following JSON output: factorial({‘results’: [{‘inputvalue’: ‘3’, ‘factorialvalue’: ‘6’, ‘squarevalue’: ‘9’, ‘servertime’: ‘Wed Nov 8 15:21:34 2006’}]});

Changing the return_type to XML in the above URL results in the following output:

<response>
   <results>
      <input>3</input>
      <factorial>6</factorial>
      <square>9</square>
   </results>
   <servertime>Fri Nov  3 21:14:14 2006</servertime>
</response>

The sample application is made up of factorial.html that includes a reference to factorial.js, and both can be downloaded from Resources.

To try the sample application, the HTML and the JS should be run on a local server that has a different domain than the domain hosting the service (www.theinnovationlue.com). Once the HTML is loaded, you will see two sections.

The first section makes an asynchronous call using the AJAX XMLHttpRequest API to the service mentioned above when you enter a number and click on the button. This will not work because of the cross-domain restriction. If you look into the JavaScript console in the browser, you will notice an error as shown below:

The second section makes an asynchronous call using the JSON approach described in this article. Once you enter a number less than 10 and click the button, you will notice that it is able to invoke the service in a different domain and show the results returned as depicted below:

Regarding the cross-domain invocation, the sample application shows the superiority of the JSON approach over AJAX.

Possible server-side extensions

The approach described in this article can evolve into a framework that can be used to deliver more and more dynamic features to Webpages asynchronously. The JavaScript file or code included by the clients and housing the callback function can be dynamically generated using server-side technologies like Google Web Toolkit (GWT). Generating JavaScript using a server-side technology helps because of the well-established development practices, tooling support, and commonly available skills for a server-side technology like Java Platform, Enterprise Edition. Additionally, by generating JavaScript dynamically, you can generate context-specific parts only. GWT makes it easy to create an Eclipse project and allows Java EE developers to generate cross-browser-compliant JavaScript using a server-side programming model. It provides a fully integrated (hosted) environment that can be used for testing the generated JavaScript and has tools to generate all the assets needed to run the JavaScript in the usual Web server mode.

Additionally, there are APIs like json.jar that can be used to easily convert the output of existing services or applications into the JSON format—for example, an existing application or service that can look up employees and generates a single employee or a list of employees. By using the json.jar API, the data in the form of a POJO (plain-old Java object) can easily be converted into the JSON format, thereby allowing the service to be invoked asynchronously from a Webpage. The complete source code can be downloaded from Resources but some key snippets are shown below:

Converting a POJO into JSON format:

 JSONObject obj = new JSONObject();

obj.put("firstName", emp.firstName);   // emp is an instance of Employee
   obj.put("lastName", emp.lastName);
   obj.put("email", emp.email);
   obj.put("location", emp.location);
   obj.put("phone", emp.phone);
   String jsonStr = obj.toString();

The code above converts a simple Employee object into a JSON object using the JSONObject class.

Converting an array list of POJOs into JSON format:

JSONArray jsonArray = new JSONArray();
   for (int i = 0; i < employees.size(); i++) {
      Employee emp = (Employee) employees.get(i);

      JSONObject jsonObj = new JSONObject();
      jsonObj.put("firstName", emp.firstName);
      jsonObj.put("lastName", emp.lastName);
      jsonObj.put("email", emp.email);
      jsonObj.put("location", emp.location);
      jsonObj.put("phone", emp.phone);

      // Add to the array
      jsonArray.put(jsonObj);
   }
   String jsonStr = jsonArray.toString();

Each element in the JSONArray above is a JSON object.

Conclusion

JSON-based JavaScript browser applications are a recent development and are quickly becoming popular (as indicated by Google trends). In this article, we described a JSON-based approach for performing asynchronous calls from clients to servers across different domains. We described the key elements of the approach that help make the solution work across domains and browsers. The simplicity of the JSON data format makes this approach elegant. Additionally, it is possible to easily modify existing Java EE services or applications to generate JSON data.