HMI Data Products
AIA Data Products
MDI Data Products
SHA Data Products
IRIS Data Products
SID Data Products


Instrument Websites
AIA
HMI
MDI
IRIS



** Useful Links **



How do I access JSOC data?

  1. How can I browse the data in JSOC ?

    2. Data archived in the JSOC can be browsed in a variety of ways:

    • Lookdata (@ http://jsoc.stanford.edu/ajax/lookdata.html), a Web-based metadata browsing tool. It is the front end to the DRMS at Stanford and can be used to browse names of data series, to get data series keyword lists, to examine metadata for sets of records, and to generate export requests.
      (Note: See Q3 for more information on how to use the Lookdata tool)

    • Visual Catalog, an image gallery to browse the various data products available from JSOC.

  2. How can I download the data from JSOC ?

    3. Data archived in the JSOC can be obtained in a variety of ways:

    • Web Access: Data can be directly "exported" from http://jsoc2.stanford.edu/ajax/exportdata.html, a Web-based data export tool. (See tutorial document tour.pdf) or it can be downloaded as the last step in lookdata . (See next question)

      Exportdata is intended for users who have a pretty good idea of what data is available, and which data subsets they are interested in locating and exporting from the JSOC. It is NOT useful for browsing the data archive.

    • SolarSoft: You can use IDL SolarSoft to access JSOC data. This interface is described at http://www.lmsal.com/solarsoft/jsoc/ssw_jsoc_routines.html. JSOC data can also be accessed from the remote DRMS/SUMS sites that are also VSO sites by using the keyword 'site' in IDL vso_get command. For example, site= 'cfa' will get them from CfA, Harvard, site='rob' will get data from ROB. European users should see also http://wissdom.oma.be.

    • VSO sites: Some commonly requested JSOC data can also be accessed from the Virtual Solar Observatory site.

    • Script Driven: http://jsoc2.stanford.edu/ajax/exportfile.csh - Use this sample script to export recordsets when you know the query to use and you want to use a local script to automate the request. You will want to tune the script to meet your needs.

    • NetDRMS: Becoming a NetDRMS service host, you could have your own Data Records Management System development environment with automated, regular data delivery. This is a somewhat involved process. See netdrms

  3. How do I use Lookdata?

    4. Lookdata is a web based tool to browse through all the data archived in JSOC. In order to use the Lookdata webtool, one needs to take the following steps:

    • Step 1: Determine the name of the data series you wish to examine (and possibly export):
      • To search for all available data series, click the 'Fetch seriesname List' button field (Step 1) under item 1 on the lookdata webpage (see Fig1). You could also enter a search string like 'hmi' or 'aia' in the 'Seriesname filter' field and this will yield all data series containing the string (See Fig2). This filter follows the usual rules of searching for regular expressions. For example: '^hmi' would search for all data series beginning with 'hmi' etc.

        • Dataseries corresponding to HMI begin with the prefix 'hmi'. AIA data begins with 'aia', MDI data begins with 'mdi', data from the Stanford Heliosiemology Archive begins with 'sha', and data from the IRIS mission begins with 'iris'.

      • If you are not sure what the data series is called, or whether the data product you are looking for has been published, you can check the Data products to Data series Map. This page contains all the published data products.
        Note: There may be more data products available on jsoc2 than jsoc.

      Clicking on the yellow highlighted '?' symbol next to each field on the lookdata page will open up a pop-up help text box for that field.

      Fig 1:

      Fig 2:

    • Step 2: Pick Series to Use:

      • As a result of Step 1, the scroll tab on the right hand side, under '2. Pick Series to use' should get populated. Scroll through the list and click on the series that you are interested in. This will populate the tab under '3. Select Records and Get Record Count' with the series you selected. It will also open a section listing 'All Keywords, Links, and Segments'

    • Step 3: Refine the query to a manageable number:

      • The series query in the tab under '3. Select Records and Get Record Count' needs to be further refined. This can be done by selecting a particular range of dates and times, and/or by filtering based on values of particular keywords.

      Filtering by date and time is achieved by adding to the data series name, a date or dates enclosed in square brackets. For informational purposes, near the top of the 'RecordSet Select' tab, the dates of the earliest and most recent records in the data series are displayed. The following examples demonstrate this syntax (here the hmi.M_45s data series is used, but the syntax is generally applicable to most other data series):

      • hmi.M_45s[2012.4.01_01:30_TAI-2012.04.01_02:00_TAI] -This query requests all HMI magnetograms for January 1, 2011 between 01:30 and 02:00 TAI. It will return 41 records (endpoints are inclusive).
      • hmi.M_45s[2012.04.01_01:30_TAI/30m] -This query is almost identical to the above query, except here the forward-slash modifier is used to indicate the time range (the /30m means 'over the next 30 minutes'). It will return 40 records (exclusive of the final endpoint).
      • hmi.M_45s[2012.04.01_01:30_TAI/30m@3m] - This query is similar to the above queries, except that now the cadence is reduced from 45 seconds to 3 minutes (the @3m means “once per 3 minutes”). It will return 10 records.

      Date and Time syntax:
      -Several formats for expressing time can be used. In particular it understands a subset of ISO-8601 formats such as 2010-07-25T13:30:00 and the SDO/HMI (and SOHO/MDI) format of 2010.07.25_13:30_00. Trailing time or date parts that are omitted default to the start of the respective interval. E.g. 2010.07.25_13 corresponds to 13:00 UTC on 25 July 2010 as one would expect.

      -HMI data is time tagged in TAI time, 'International Atomic Time', since they are observed on SDO using TAI time. However, for the convenience of people the times can also be expressed in UTC. But HMI data has 'nicer' time stamps when expressed in TAI.

      -Note that by choice, AIA uses UTC for most user-level time stamps.

      -The offset between TAI and UTC is now 34 seconds, more than half the basic HMI observing cadence of 45 seconds. Hence, you will often not get the record you expect if you use UTC to ask for HMI data.

      The default for an omitted time zone specifier is UTC. Thus if you ask for hmi_test.M_45s[2010.07.25] you will get the magnetogram labeled hmi_test.M_45s[2010.07.25_00:00:45_TAI].

    • Further refinement using keyword logic:
      • A JSOC data series may have multiple Prime Keys. A record can be fully specified by the complete set of keyword values for the Prime Keys. The Prime Keys can be thought of as a filter acting on the full set of records in the given data series. i.e. only records matching all of the given Prime Key values will be selected. So, what does that mean?
        • Some Examples:
        • aia.lev1[2011.01.01_01:30/30m] This query requests all AIA image records for January 1, 2011 for the half hour starting at 01:30. It will return 1200 records (one image in each of 8 wavelength channels every 12 seconds).
        • aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 ?] : This query now sub-selects the data from the above query to include only those image records for which the WAVELNTH keyword equals 171 (i.e., images from the 171&Ang channel of AIA). It will return 150 records (one image every 12 seconds).
        • aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 or WAVELNTH=304 ?] : This query returns only those image records for which the WAVELNTH keyword equals either 171 or 304. It returns 300 records.

        The full set of Prime Keys for each series are listed in lookdata both on the 'Series Content' tab and at the top of the 'RecordSet Select' tab.

      For more examples see JSOC Help and the SDO Analysis Guide.

    • Step 4: Get Record Count:

      • After entering a query, press the 'GetRecordCount' button, and the tool will return the record count.
      It is necessary to press the 'GetRecordCount' button again after modifications to the query are made, to update the record count. Entering a number 'n' in the (optional) 'Record Limit' field will limit the number of records displayed, with positive numbers limiting the display to the first 'n' records and negative numbers limiting the display to the last 'n' records.

    • Step 5: Select desired keyword and segments :

      • Selecting **ALL** in these boxes will return all of each of these, but to speed things up users may want to select only a subset of keywords, segments, and/or links. Users can select individual elements in these lists by holding down the COMMAND or ALT key while clicking the left mouse button, and ranges of keywords can be selected by holding down the SHIFT key while clicking.
      Note that the 'Select Links' box will often be empty.

    • Step 6: Fetch keyword values for RecordSet :

      • Clicking the 'Fetch Keyword Values for RecordSet' tab will yield in a display of keywords titled ' Get Keyword and Segment Values Here' which is below the 'List of all Keywords, Links and Segments'. One needs to scroll down to see this list of keyword values in the filtered set of records see(Fig 3). Further refinement can be done by going back to the 'RecordSet Select' tab (by clicking on the tab itself), adjusting the query, and repeating Steps 3 through 6.

      In addition to keyword values, the right most column of the 'Values table' typically points to the DRMS archive file/section where each of the fitsio compressed 'as-is' FITS records of your selected series currently resides. You can click on individual links preview the data.

      Fig 3:

    • Step 7/8: Exporting Data:

      • If you wish to obtain the selected records, clicking on the Export' tab will bring up the 'Exportdata' page. See next question on 'How to use Exportdata' for more details.

    • Step 9: Graphing Data:

      • Below Section 8 (i.e export data tab), is a utility that can be used to graph keyword values of filtered data versus date observed, time observed and a few other quantities of the data. To create a plot, click on the pull down menu for 'Select X axis' and pick your independant variable. Click on the dropdown option for 'Select Keywords' and select the y axis component of the graph. Clicking on the 'Create Graph' tab will create a gaprh. See Fig 4: a graph of T_REC vs OBS_VR.

      Fig 4:

  4. How do I use Exportdata?
    5. You could get to exportdata either as the last step in lookdata or choose to come to the export page directly. If you come from lookdata, the 'Record set', 'Record Limit' and 'Record Count' fields will be pre-populated.

    If you come to exportdata on your own, and you know which series you are looking for, you can fill in very the 1st field of the form which is 'Record Set' and click 'Recount'. If you don't know what you are looking for, then we suggest you go to lookdata and browse through the published data products.

    • The (export) Method:

    • The 'Method' drop-down menu gives the user the option of choosing various export methods. For most remote users, we recommend the 'url-tar' or 'url' method. After selecting 'url', several new fields will appear.

    • Filename Format:
    • Processing:
    • Protocol:

      • 'Protocol' should be set to 'FITS' if we desire the fits files or 'jpg' or 'mpeg' if , and 'Compression' set to 'compress Rice'. Also of note is the “Filename Format” field allowing users to specify the format of the filenames. For example, if it is desired that the wavelength of the image be included in the filename, then one can add {WAVELNTH} to this field.

  5. How do I find which series I want?
    6. You can check the Data products to Data series Map page.
    More details are explained as part of Step 1 in question 3.
  6. How do I use the Processing tab in exportdata?
    7. This is explained under 'Processing' in question 4.




Contacts | JSOC Home | Exportdata | Lookdata | SDO-NASA | Stanford Solar Centre

SDO Privacy Notice