Data Receipt and Processing

Data Processing

Most spacecraft mission data is typically divided into "levels" of processing and refinement.

• "Raw" data is the basic telemetry transferred from the spacecraft to the ground station, packaged as a file, with only basic r/f integrity checks run, and no processing done to it.
• Level Zero refers to the cleanest, time-ordered set of data for a given instrument or spacecraft. The only processing done is typically the removal of duplicate data points (due to multiple file sends or a solid state recorder overlap), the removal or minimal repair of bad r/f data, and the matching of the end of one packet to the beginning of the next. The images taken by an instrument are left "as is" - not manipulated, not flat-fielded nor otherwise calibrated.
• Level One data is usually "unpacked" Level Zero data, generally in the cleanest state possible (barring significant, irreversible data alteration). This data is time-sequenced, quality-filtered, [need more information here]
• Level Two data has been irrevocably filtered, flat-fielded, time-sequence-merged, Fourier-transformed or otherwise changed or cleaned up from Level 0ne in a way that is irreversible. Level Two data can be re-created from Level One data, but not one cannot create Level One data from Level Two data.

BR JSOC will also include sub-levels (e.g. Level 0.3, Level 1.5) to indicate various stages of processing. For example, Level 0.3 will represent less processing (and perhaps less information) than Level 0.5. BR Important subsections of data processing include: BR

• DDS (aka High Speed Bus) Data Capture
• Offsite Data Archive (raw)
• MOC Data Product Server Data Retrieval
• Level 0 HSB Telemetry (Image) Processing
• Includes Time and Data Quality computations
• Level 0 Housekeeping Telemetry Processing
• "Merged" Housekeeping DRMS record creation
• "Averaged" Housekeeping DRMS record creation
• Production Data Achive (raw and level0)
• Data Accounting
• Level 0.5 Processing and DRMS record creation
• Orbit information
• Pointing information
• Level 1.0 Filtergram Calibration

• Removal of overscan/underscan rows & columns

• Identification of corrupted pixels
• Application of offset and gain correction
• Calculation of image center and limb position

The JSOC, for the AIA & HMI instruments, defines the processes of Level Zero data to be:

1. Telemetry Checking
2. APID Lookup
3. Image Reconstruction
4. Housekeeping Processing

All of the Level Zero data is recorded in the JSOC DRMS (database). More information on Level Zero is contained in the JSOC Level Zero Software Functional Specification. View this document at this link for more details: http://hmi.stanford.edu/development/JSOC_Documents/Level-0-Documents/JSOC-LEVEL0-PROCESSING-FS.pdf

Who uses it?

Typically, mission operations personnel and instrument operators want to see Level Zero data. It is available usually within minutes of receipt and persons very familiar with instrument/spacecraft engineering specs can make the best use of it. All kinds of raw image data and housekeeping data are present, including many useful temperatures, voltages and positions. Those requiring the most immediate spacecraft images are sometimes also Level Zero data users, such as CISM (Center for Integrated Space Weather Modeling).

What is Level 0.X?

The JSOC team has further segregated Level Zero data by making intermediate processing steps between Level Zero and Level One. These include:

• [:Lev0.1Doc:Level 0.1]

• [:Lev0.3Doc:Level 0.3/Level 0.5]

MOC Product Server

• [:MOCServer:Accessing the server] -- Information about accessing the MOC Product Server