Tuesday, May 27, 2014

Determining web service health by inspecting JSON response data in an Uptime Plugin.

Abstract


This article demonstrates how to write a Plugin for Uptime which:
- analyzes the contents of a JSON response payload response from a target web service, and
- uses the JSON information to determine and report health of the service to Uptime.

A sample plugin is provided.  The sample queries the health of the well-known Google Geocoding REST service by inspecting the contents of its JSON response.


Background


Uptime is an application which continuously evaluates the health of web services.  https://github.com/fzaninotto/uptime

To determine the health of a service, Uptime periodically issues web requests to the target service.

In its simplest form, Uptime declares the service is up if a successful response is received from the service.  Otherwise, it declares the service is down.

Uptime also provides a plugin interface which allows Uptime to be extended to perform custom operations.  This article exploits the plugin interface to inspect the contents of the JSON response from the target web service.



Motivation


I wanted to use Uptime to report the health of a custom web service based upon information received in a JSON response provided by the service.

Instead of relying exclusively on receipt of a 200 response, I also wanted Uptime to analyze the contents of the JSON response payload for my custom web services.  Basically, I wanted to read the contents of the JSON response and look for specific key words and associated values "pass" or "fail".

The sample plugin described below shows how to inspect the response JSON.





Uptime setup


Ubuntu:  I set up a 32-bit machine with Ubuntu Linux 12.04.

http://releases.ubuntu.com/12.04/

As root, I installed and set up the following:

Mongo DB:  http://docs.mongodb.org/manual/tutorial/install-mongodb-on-debian/

Mongo User/Password: Set a database user name and password into Mongo:

/root> mongo
> use uptime
> db.addUser('myUser','myPassword');
> exit

Git:  apt-get -y install git

G++:  apg-get -y install g++  (note: g++ was required for 'npm install')

As user, 

Node.js:  I fetched and unzipped the latest Node JS, and added it to user and root paths.

http://nodejs.org/download/

Uptime:

docs:  https://github.com/fzaninotto/uptime   Scroll down to "Installing Uptime"

You will get to the important command:  git clone git://github.com/fzaninotto/uptime.git

In my case, uptime was installed to directory /home/user/uptime/...

Set the mongo database user name and password into Uptime:

/home/user> vi uptime/config/default.yaml

user:     myUser
  password: myPassword

Start uptime

/home/user> cd uptime
/home/user/uptime> node app.js


Verification


Browse http://<your_hostname>:8082/  Verify "welcome to uptime"

Click to create your first check.  Verify Uptime correctly monitors the target.

Uptime is now set up properly.  It is ready to install the sample plugin.

Note:  Henceforth, all commands are issued as user.


Hello World Plugin



I created a hello world plugin to get started.

As a sample, the plugin is designed to evaluate the health of a well known REST service, the Google Geocoding Service.

Uptime sends a web request to the service.  You specify the Google URL in the Uptime Check configuration screen (see below).

When a response comes back from Google, Uptime passes each JSON response to the plugin.  The plugin reads and evaluates the contents of the response, and reports results back to Uptime.  In this sample, it passes if it sees "status":"OK".


Install the plugin


Fetch the sample hello world plugin here:

https://sites.google.com/site/sagitt001/uptime/uptime.plugin.helloworld.zip?attredirects=0&d=1

Unzip it to create

/home/user/uptime/plugins/helloworld/...

Edit the Uptime config file, and add the plugin name:

/home/user/uptime> vi config/default.yaml

plugins:
 - ./plugins/helloworld   <--- add this line
 - ./plugins/console
 - ./plugins/patternMatcher

Inspect the plugin


There are three files in directory plugins/helloworld/...   That's all it takes (though you can add more if you like).

jquery.autoresize.js 

This was copied unchanged from the httpOptions plugin.

It assists with presentation of the user configuration screen in Uptime.

_detailsEdit.ejs

This file is copied from httpOptions and slightly modified.

It also assists with presentation of the user configuration screen.

index.js

This file was adapted from a sample plugin developed by Alexander O'Donovan-Jones on github (see acknowledgements)

It contains two interesting sections.

exports.initWebApp.  This section assists with presentation of the user config screen.

exports.initMonitor.  This section parses the JSON response object according to user specified options.  (This is most likely the section you will change heavily to evaluate your own JSON response)


How to run it


Restart Uptime if it was still running.

Browse to your uptime.

Click tab Checks-> Create Check.  Type the following values.  Accept defaults for the rest.  Click 'Save' when done.

URL:  http://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=false
Type:  http
Name:  Google Geocoding
Poling interval: 15 s

Edit the Check again.  Click tab Checks-> Click Google Geocoding-> Click button Edit.  A new text box will appear: Hello World Options.  Type the following values on two lines, with no quotes (YAML format).  Click 'Save' when done.

Hello World Options:

geocode: google
trace: true

Restart Uptime again.

I like to pipe results to a log file, so that I can review it easily.

/home/user/uptime> node app.js > /tmp/uptime.log

I also like to monitor live progress in another command-prompt terminal

/home/user/uptime> tail -f /tmp/uptime.log

Results:


Every 15 seconds or so, the log should display messages from the JSON results analysis code in index.js.  Examples:

Evidence that user options are properly set in the config, and properly presented to the plugin:

on.PollerPolled: Entry.
on.PollerPolled: options: { trace: true, geocode: 'google' }
on.PollerPolled: t=true
on.PollerPolled: geocode=google
on.PollerPolled: url=http://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=false

Evidence that the JSON response from Google Geocoding has been properly received and parsed.

checkGoogleGeocode: Entry. body: { results:
  [ { address_components: [Object],
      formatted_address: '1600 Amphitheatre Parkway, Mountain View, CA 94043, USA',
      geometry: [Object],
      types: [Object] } ],
 status: 'OK' }
checkGoogleGeocode: status: 'OK'
checkGoogleGeocode: Exit. Success.

And, drumroll please, Uptime should display a green indication that the service is up.

Variations


Add more javascript to index.js checkGoogleGeocoding().  Inspect other values in the JSON.  Restart Uptime.  Verify.

Edit the plugin configuration.  Disable verbose tracing  trace: false  Save.  Restart Uptime. Verify fewer log messages.

Play with it.  Learn how it works.

Conclusion


The sample hello world plugin demonstrates how to query a REST service and evaluate the contents of the JSON response, using a well-known service provided by Google, Inc.

Once this works, you have all the secrets you need to query your own REST services and evaluate their JSON responses.  Change the URL in the Uptime Check configuration.  Then modify index.js to evaluate your own JSON responses.


Be nice


Don't bash the Google Geocoding service continuously.  Stop uptime or delete the plugin when you are not studying it.


Acknowledgements


Many thanks to Francois Zaninotto for creating, publishing, and supporting Uptime.

https://github.com/fzaninotto/uptime

Many thanks to Alexander O'Donovan-Jones for creating and sharing a plugin named jsonValidator.

https://github.com/aodj/uptime/blob/master/plugins/jsonValidator

And thanks to Google, Inc for providing the Google Geocoding service used in this sample.

https://developers.google.com/maps/documentation/geocoding/

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.