Housekeeping Telemetry Time Averaging

The Housekeeping Telemetry Time Averaging is calculated from level 0 housekeeping day file data. The mean, maximum, minimum, and standard deviation are calculated and loaded into a output DRMS data series. An instruction file is created to specify range of time to use to calculate averages, keywords, output DRMS series name, author and owner of DRMS series.

The basic steps for creating and loading a Housekeeping Telemetry Time Averaging DRMS series is discussed below. The overview of the basic step is as follows. 1. Create Instruction File 2. Create JSD file for Mean, Maximum ,Minimum and Standard Deviation(M3SD) using the instruction file. 3. Create M3SD DRMS series based on JSD file. 4. Load the M3SD DRMS series using a dayfile and instruction file.

Create Instruction file

The basic format of the instruction file is shown below. 

##########  Template Instruction File Format  ##################
###comments can be added after # at beginning of line
TemplateName:<name of template file. Use for filename seriesname.txt>
SeriesName:<name of dataseries to create or use>
Description:<description of series in quotes>
Author:<name in quotes>
Owner:<name in quotes>
Interval:<in seconds only>
keyword:<apid in decimal format>,<long keyword name>,<user created character name-not restricted size>
.
.
.
keyword:<apid in decimal format>,<long keyword name>,<user created character name-not restricted size>

An example instruction file is shown below. 

###### Example 1 using apid 18 temperature values using users created names for short keyword name
TemplateName:hmi.temp_summary.txt
Seriesname:hmi.temp_summary
Description:"Temperature Statistics using usernames from HMI Housekeeping - Thermal/PS voltage packet"
Author:"rock"
Owner:"rock"
Interval:600
keyword:18, HMI_TS19_OPT_INSIDE_FRONT,  HTS19_INSIDE_FRONT
keyword:18, HMI_TS21_OPT_NEAR_CEB_INT,  HTS21_NEAR_CEB
keyword:18, HMI_TS15_OVN_PREAMP,        HTS15_OVEN_PREAMP
###### End Example 1

Create Maxium, Minimum, Mean and Standard Deviation JSD file

The create mean, maximum, minimum, and standard deviation jsd(cm3sd_jsd.pl) script is use to create the output drms series in DRMS using each keywords in the instruction file to create a keyword with a MAX, MIN, MEAN, and SD suffix that is a float. There will be T_START and T_START_step ( Interval value), T_START_epoch keywords auto created to represent the start packet time and interval covered in seconds. The T_START keyword will be the index. The T_START and T_START_epoch values will use UTC in units field. T_START_step will use the interval value in the template instructions file where the interval value is in seconds. The time interval or range is use to create a step in the slotted drms series. The author and owner are used to create the corresponding values in output drms series. This script does attempt to locate errors in instruction file and displays error messages.

An example jsd created based on the example instruction is shown below. 

#====Global Series Information ==============
SeriesName:  hmi.temp_summary
Description: "Temperature Statistics from HMI Housekeeping - Thermal/PS
voltage packet"
Author:      "rock"
Owner:       "rock"
Unitsize:    1
Archive:     0
Retention:   0
Tapegroup:   0
Index:       T_START
#== Keywords Series Information ==============
Keyword:T_START,time,ts_eq,record,TSEQ_EPOCH,2,UTC,"T_START"
Keyword:T_START_epoch,time,constant,record,TSEQ_EPOCH,2,UTC,"T_START_epoch"
Keyword:T_START_step,float,constant,record,600,%f,"600 sec","T_START_step"
Keyword:HTS21_NEAR_CEB_MIN,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MIN"
Keyword:HTS21_NEAR_CEB_MAX,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MAX"
Keyword:HTS21_NEAR_CEB_MEAN,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MEAN"
Keyword:HTS21_NEAR_CEB_SD,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_SD" 
.
.
.

An example of running script is shown below. The isf= is followed by path to the instruction file. Some example instruction files are in software repository at $HOME/cvs/TBL_JSOC/lev1/instruction_file/su_carl. 

 cm3sd_jsd_file.pl isf=/home/carl/cvs/TBL_JSOC/lev1/instruction_file/su_carl/hmitest1200_thermal_template.txt

The environment variable in script, HK_CM3SD_JSD_DIRECTORY, is defined at top of script. The jsd is created in the location defined in script. The jsd currently is created in the prod directory as shown below. 

#PRODUCTION SETTINGS to send jsd files to production directory
$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/prod";
#USER SETTING to send jsd files to user directory 
#turn off production by commenting out, turn on what is need below
#su_carl setting for test creating instruction file.
#$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/su_carl";
#su_rock setting for test creating instruction file.
#$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/su_rock";

Load Maxium, Minimum, Mean and Standard Deviation in DRMS Data Series

The load mean, maximum, minimum, and standard deviation executable(load_m3sd) is used to get keyword data from housekeeping day file, calculate minimum, maximum, mean and standard deviations keyword values based on time interval and write keywords to DRMS data series.

This executable takes as input the housekeeping day file and the instruction file. The dayfile contains packets which contain the keyword values outlined in the instructions files. The housekeeping day file also contains the T_START value by getting TIMECODE keyword data or packet time values per packet. The instructions file contains information for executable on which keywords to decode(see long keyword names), which keywords to calculate minimum, maximum, stand deviation, and mean values, and which keywords to load(see user created or short keyword names) in DRMS data series. The arguments passed to program are instruction filename and the housekeeping day file. The format and example of command to execute load_m3sd is shown below. 

load_m3sd  [-h] in=<day filename> isf=<instruction file> 

 details are:
  -h: help - show this message then exit(optional field)
  in=<day file name> -use full path to day file(required field)
  isf=<instruction file name> -use full path to instruction file(required field)
  Need data series already created for this to program to load keywords in data series.
  Setup interval time in this file where range of intervals tested are 60 seconds to 7200 seconds.
  Setup keywords to calculate mean, max, min and standard deviation values in this instruction file.
  Setup seriesname in this instruction file for writing keyword min,max,mean and standard deviation values.

  Example of running:
  load_m3sd  in=/home/carl/cvs/myprod/JSOC/proj/lev1_hmi/apps/20080918.0x0013 
             isf=/home/carl/cvs/TBL_JSOC/lev1/instruction_file/hmitest1200_thermal_template.txt