Housekeeping Telemetry Time Averaging
The Housekeeping Telemetry Time Averaging is calculated from level 0 housekeeping day file data. The mean, maximum, minimum, and standard deviation are calculated and loaded into a output DRMS data series. An instruction file is created to specify range of time to use to calculate averages, keywords, output DRMS series name, author and owner of DRMS series.
Time Interval Value
An important setting is the interval parameter in the instruction file. This is the range of time to calculate the averaging of keyword values. The tested and verified interval ranges are 60 seconds to 7200 seconds (2 hours) for interval ranges. For example if a user selects 600. The mean, maximum, minimum and standard deviation for a keyword value will be calculate for all points within 600 second(or 10 minute) blocks. The T_START time parameter is automatically created to represents these blocks of time. For example if 75 data points are available in dayfile data for October 26, 2009 for every 600 seconds. The first block or first T_START value will be 2009.10.26_00:00:00_UTC. The following blocks will be increment every 600 seconds or 10 minutes(i.e.,2009.10.26_00:10:00_UTC, 2009.10.26_00:20:00_UTC,2009.10.26_00:30:00_UTC, etc.) until the end of the day.
Keyword Names
The long and short keyword names are setup by using the keyword names in the housekeeping configuration files. To view housekeeping configuration files, view CVS repository at http://jsoc.stanford.edu/cvs/TBL_JSOC/lev0/hk_config_file/1.163. The user must use the long keyword name for the third field in the instruction file. The short keyword name can be a user created name or the user can select the short keyword names in the housekeeping configuration files. The short keyname names used in the instruction file should be restricted to 15 characters. For more details, view below the information on setting up the instruction file. The user created keywords or short keyword names set up in the instruction file are created in jsd file with a _MEAN,_MAX, _MIN and _SD suffixes(i.e., <user-name>_MEAN,<user-name>_MAX, <user-name>_MIN and ,<user-name-1>_SD). For more details, view below information on creating JSD file.
Basic Steps To Create Housekeeping Telemetry Time Average DRMS Series With Data
The basic steps for creating and loading a Housekeeping Telemetry Time Averaging DRMS series is discussed below. The overview of the basic step is as follows.
- Create Instruction File
- Create JSD file for Mean, Maximum ,Minimum and Standard Deviation(M3SD) using the instruction file.
- Create M3SD DRMS series based on JSD file.
- Load the M3SD DRMS series using a dayfile and instruction file.
Create Instruction file
The basic format of the instruction file is shown below.
########## Template Instruction File Format ################## ###comments can be added after # at beginning of line TemplateName:<name of template file. Use for filename seriesname.txt> SeriesName:<name of dataseries to create or use (without .jsd suffix)> Description:<description of series in quotes> Author:<name in quotes> Owner:<name in quotes> Interval:<in seconds only> keyword:<apid in decimal format>,<long keyword name>,<user created character name-restricted size to 15 characters> . . . keyword:<apid in decimal format>,<long keyword name>,<user created character name-restricted size to 15 characters>
An example instruction file is shown below.
###### Example 1 using apid 18 temperature values using users created names for short keyword name TemplateName:hmi.temp_summary.txt Seriesname:hmi.temp_summary Description:"Temperature Statistics using usernames from HMI Housekeeping - Thermal/PS voltage packet" Author:"rock" Owner:"rock" Interval:600 keyword:19, HMI_TS19_OPT_INSIDE_FRONT, HTS19_INSD_FRT keyword:19, HMI_TS21_OPT_NEAR_CEB_INT, HTS21_NEAR_CEB keyword:19, HMI_TS15_OVN_PREAMP, HTS15_OVEN_PAMP ###### End Example 1
Create Maxium, Minimum, Mean and Standard Deviation JSD file
The create mean, maximum, minimum, and standard deviation jsd(cm3sd_jsd.pl) script is use to create the output drms series in DRMS using each keywords in the instruction file to create a keyword with a MAX, MIN, MEAN, and SD suffix that is a float. There will be T_START and T_START_step ( Interval value), T_START_epoch keywords auto created to represent the start packet time and interval covered in seconds. The T_START keyword will be the index. The T_START and T_START_epoch values will use UTC in units field. T_START_step will use the interval value in the template instructions file where the interval value is in seconds. The time interval or range is use to create a step in the slotted drms series. The author and owner are used to create the corresponding values in output drms series. This script does attempt to locate errors in instruction file and displays error messages.
An example jsd created based on the example instruction is shown below.
#====Global Series Information ============== SeriesName: hmi.temp_summary Description: "Temperature Statistics from HMI Housekeeping - Thermal/PS voltage packet" Author: "rock" Owner: "rock" Unitsize: 1 Archive: 0 Retention: 0 Tapegroup: 0 Index: T_START #== Keywords Series Information ============== Keyword:T_START,time,ts_eq,record,TSEQ_EPOCH,2,UTC,"T_START" Keyword:T_START_epoch,time,constant,record,TSEQ_EPOCH,2,UTC,"T_START_epoch" Keyword:T_START_step,float,constant,record,600,%f,"600 sec","T_START_step" Keyword:HTS21_NEAR_CEB_MIN,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MIN" Keyword:HTS21_NEAR_CEB_MAX,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MAX" Keyword:HTS21_NEAR_CEB_MEAN,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_MEAN" Keyword:HTS21_NEAR_CEB_SD,int,variable,record,DRMS_MISSING_VALUE,%d,none,"HTS21_NEAR_CEB_SD" . . .
An example of running script is shown below. The isf= is followed by path to the instruction file. Some example instruction files are in software repository at $HOME/cvs/TBL_JSOC/lev1/instruction_file/su_carl.
Command Format:
cm3sd_jsd_file.pl isf=<instruction-file>
Example of Running:
cm3sd_jsd_file.pl isf=/home/carl/cvs/TBL_JSOC/lev1/instruction_file/su_carl/hmitest1200_thermal_template.txtThe environment variable in script, HK_CM3SD_JSD_DIRECTORY, is defined at top of script. The jsd is created in the location defined in script. The jsd currently is created in the prod directory as shown below.
#PRODUCTION SETTINGS to send jsd files to production directory
$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/prod";
#USER SETTING to send jsd files to user directory
#turn off production by commenting out, turn on what is need below
#su_carl setting for test creating instruction file.
#$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/su_carl";
#su_rock setting for test creating instruction file.
#$ENV{'HK_CM3SD_JSD_DIRECTORY'}="$hm/cvs/TBL_JSOC/lev1/hk_jsd_file/su_rock";
Create Maxium, Minimum, Mean and Standard Deviation DRMS Data Series
Utilize the create_series utility to create series. The format and example are shown below.
Command Format: create_series <directory-path>/<jsd file> Example: create_series /home/production/cvs/TBL_JSOC/lev1/hk_jsd_file/prod/hmi.temp_summary.jsd
Load Maxium, Minimum, Mean and Standard Deviation in DRMS Data Series
The load mean, maximum, minimum, and standard deviation executable(load_m3sd) is used to get keyword data from housekeeping day file, calculate minimum, maximum, mean and standard deviations keyword values based on time interval and write keywords to DRMS data series.
This executable takes as input the housekeeping day file and the instruction file. The dayfile contains packets which contain the keyword values outlined in the instructions files. The housekeeping day file also contains the T_START value by getting TIMECODE keyword data or packet time values per packet. The instructions file contains information for executable on which keywords to decode(see long keyword names), which keywords to calculate minimum, maximum, stand deviation, and mean values, and which keywords to load(see user created or short keyword names) in DRMS data series. The arguments passed to program are instruction filename and the housekeeping day file. The format and example of command to execute load_m3sd is shown below.
Command Format:
load_m3sd [-h] in=<day filename> isf=<instruction file>
details are:
-h: help - show this message then exit(optional field)
in=<day file name> -use full path to day file(required field)
isf=<instruction file name> -use full path to instruction file(required field)
Need data series already created for this to program to load keywords in data series.
Setup interval time in this file where range of intervals tested are 60 seconds to 7200 seconds.
Setup keywords to calculate mean, max, min and standard deviation values in this instruction file.
Setup seriesname in this instruction file for writing keyword min,max,mean and standard deviation values.
Example of running:
load_m3sd in=/home/carl/cvs/myprod/JSOC/proj/lev1_hmi/apps/20080918.0x0013
isf=/home/carl/cvs/TBL_JSOC/lev1/instruction_file/hmitest1200_thermal_template.txt- Back to ["HMIModuleTable"]
Back to FrontPage