AJAX Interface for JSOC

Plan:

• Implement basic access to keywords and segment information via CGI-BIN GET and POST server programs to allow Javascript access to basic JSOC functions.
• Provide basic browser side html to demonstrate capability and serve as a starting point for simple use and development.
• Publish the spec for the interface to allow others to build better Client-side tools.

Overview of Export Management

Exports will be managed in the series jsoc.exports Each request will be given a RequestID which can be used to track status. Data to be exported can be requested via several methods. The data requested will be collected (e.g. tar) and placed in a data segment in a record in jsoc.exports. It will be exported (i.e. picked up by the requestor) directly from the SUMS directory.

Series: jsoc.exports will not be archived and retention time of 7 days. Thus the Requestor will have one week to fetch the data. The export record will be retained for a history log of exports.

The jsoc.exports series looks like:

P% show_info jsoc.exports -l
Prime Keys are:
        RequestID
DB Index Keys are:
        RequestID
All Keywords for series jsoc.exports:
        DataSet         (string)        Dataset requested
        ExpTime         (time)  Time of export
        FilenameFmt     (string)        File basname format
        Format          (int)   Data format code
        Notify          (string)        Notification address
        ReqTime         (time)  Time of request
        RequestID       (int)   Export request identifier
        Requestor       (string)        Name of requestor
        Size            (int)   Volume of data requested
        Status          (int)   Status of request
Segments for series jsoc.exports:
        Data                 NA generic VAR     Exported data

{ - need to add keywords for export method (URL, tape, etc) and export function requested (i.e. convert to fits, rebin, extract, etc.)}

Status of prototype implementation of the draft plan

Many of the functions described below are now implemented in series_info and in a new module called jsoc_info which has heritage in show_info but emits JSON responses. Several open-source tools have been used in developing jsoc_info as well as http://jsoc.stanford.edu/ajax/lookdata.html which is a test client-side javascript code to use jsoc_info. These include:

• Prototype Javascript Framework at http://www.prototypejs.org/ which provides AJAX style javascript functions enabling much easier jsvascript coding.

• The [http://www.json.org/ JSON home] which describes the XML-like JSON data protocols with links to software.

• [http://mjson.sourceforge.net/index.html M'sJSON] JSON parsing and generating code which is used in jsoc_info.

• And hints for compatibility found in [http://www.w3schools.com/ w3schools] DOM information pages.

The operations now supported at http://jsoc.stanford.edu/cgi-bin/ajax are:

• Implemented via show_series

  • series_list -- Implemented within "show_series". Expects a single parameter "filter" which is used in regex to limit the number of seriesnames returned. See the man page for show_series. If e.g. all series with the word "mdi" are desired then the URL will be:  http://jsoc.stanford.edu/cgi-bin/ajax/show_series?filter=mdi%5C. 

• Implemented via jsoc_info

  • series_struct -- Like "show_info -l -s" Example URL for e.g. the series mdi.vw_V_lev18 is:   http://jsoc.stanford.edu/cgi-bin/ajax/jsoc_info?op=series_struct&ds=mdi.vw_V_lev18  which returns the JSON version of the information in the shell commands "show_info -l mdi.vw_V_lev18" and "show_info -s mdi.vw_V_lev18" combined.

  • rs_summary -- Implemented in jsoc_info. Example URL is  http://jsoc.stanford.edu/cgi-bin/ajax/jsoc_info?op=rs_summary&ds=mdi.vw_V_lev18  At present returns first and last record information.

  • rs_list -- Implemented in jsoc_info. Functions similar to the shell command "show_info" to get dataseries record attributes.
• Implemented via jsoc_fetch
  • exp_request -- Primary data export request tool. Expects a recordset specification and desired export method. Exported data will be either delivered immediately or kept online for 7 days depending on calling parameters and online status of the data. If a the data is not immediately available a "RequestID" is returned to the user for later access to the exported data.
  • exp_repeat -- Repeats the request for an expired (export data package is gone after 7 days). Expects RequestID.
  • exp_status -- Checks on status of prior request, expects RequestID.
  • exp_su -- Special tool for export of SUMS "Storage Units" to remote DRMS sites.

The details are below, but some examples here may help. The lookdata call limits the recordset to 10,000 records. The present implementation can take a minute or more for 10,000 records. Use care. This and following functions expect an explicit recordset query which will resolve to a subset of the records in the given series.

The example URL here returns a few keywords for 5 minutes (5 records) from the mdi.vw_V_lev18 data.

http://jsoc.stanford.edu/cgi-bin/ajax/jsoc_info_test?ds=mdi.vw_V_lev18%5B1996.05.01%2F5m%5D&op=rs_list&key=DATAMEAN%2CT_OBS

Note the excape characters for URL encoding: '[' to '%5B', ']' to '%5D', '/' to %2F, ',' to '%2C' etc. The return json text from this query, with white space added, is: