Keywords for incorporation of GONG+ data in the JSOC DRMS/SUMS There are currently (2006.08.18) a total of 13,265 datasets of daily Doppler and Intensity data from each of the six gong sites, with a data volume of 8.9 TB (7314 Doppler, 4.9 TB; 5951 Intensity, 4.1 TB), distributed among 12 data series, one for each site and observable (prog:gong+;level:lev1, series:{SC}_{OBS}_01d). (There are also a few datasets of magnetic and modulation data, data from the Tucson site, GONG classic data, and GONG-merged data, but I will ignore those for the time being.) I plan to ingest these data into the DRMS/SUMS, where their analysis, particularly that involving site selection for merging, will be greatly facilitated. I plan to ingest them into two series, based on the observable; thus there will be two primary record indices, observation time (ObsrvTime) and site (GONG:SiteCode). Keyword selection is based on two fundamental criteria. It must be possible to export any data record from the series as a FITS file with its original FITS header; and the keywords used must support JSOC level 2 analysis modules. I also believe that to encourage use of analysis modules outside the JSOC and to avoid confusion arising from conflicting keyword definitions and uses, the use of FITS-compatible keywords in the DRMS should be avoided: keyword names should mix case, and/or include characters forbidden by the FITS standard. (The fact that the GONG keyword set includes 14 that violate the FITS standard is beside the point.) In general, a one-to-one mapping from DRMS keywords to GONG+ keywords would be required, but there are several exceptions: FITS-reserved keywords which can be supplied from the FITS exporter (SIMPLE, BITPIX, END, ...); commentary-like keywords (HISTORY, COMMENT, blank, and GONG's GHISTnnn set); and redundant keywords. Also, there is a special GONG keyword, FILLED, whose presence (usually) means that the data are not valid, merely being a repeat of the previous image. There is no reason to ingest the data segments of those records; there may not be any reason to ingest them at all, unless they provide useful information on the instrument state at the time and the reason for the gap. There are a total of 237 unique keywords found in the sampled files. The headers each contain up to 4 COMMENT records (typically 0) and up to 4 HISTORY records (typically 4). I do not believe there are any blank keyword records, but have not checked carefully. The following keywords (in addition to COMMENT and HISTORY) are occasionally repeated (never more than once): DATE, ORIGIN, SITE, SITENAME, and TIMEZONE. Strictly speaking this does not seem to violate the FITS standard except in spirit, since the standard explictly states that any number of Commentary keywords may appear in a header, but is silent on limitations on other keywords (and on the interpretation of multiple ones with conflicting values as well). In any case, it looks like these multiple instances are merely the result of erroneous processing; except for DATE, their values are probably always the same when repeated, and it would probably do no harm to suppress the second instances in FITS export. (DATE should, strictly speaking be replaced, and its previous values saved in a commentary keyword; similarly ORIGIN.) The ingestion process should watch and trap for them. The GONG FITS keywords currently used by my SOI analysis modules (I don't believe that anyone else has any that work on GONG data), are: B0 C_MA C_MI DATE-OBS } combined TIME-OBS } FNDLMBAN FNDLMBXC FNDLMBYC GONGPLUS only required for keyword classification HA L0 LAT LON SITE only required for keyword classification TIMEZONE PIXLENX PIXLENY P_ANGLE RADIUS The real (as opposed to nominal) position angle information for each image is not included in the GONG FITS header. It must be calculated from a separate table; if the requisite information were supplied at the time of ingestion, then equivalents for the keywords HA and TIMEZONE would not be required, although it might still be useful to have an HourAngle keyword for purposes of data selection. GONG FITS headers make liberal use of comments in the header card images, typically about 50%. For each commented key, the standard comment, assuming one exists, must also be scanned, and the record comment verified to assure that it is the standard. Per-record comments, if there are any, must be ingested separately. In drawing up the proposed list of DRMS keywords to be used, my general principle is for keywords that are clearly specific to the GONG instruments (or likely to be highly specific to other instruments but with different meanings, e.g. MOTOR0), or whose meaning is obscure, to prefix the FITS keyword with "GONG:". For keywords that are likely to be of general interest for data analysis and to have equivalents or near-equivalents in meaning among other datasets, I use mixed case descriptive strings; these could become the core of a set of JSOC keywords analogous to the FITS World Coordinate System (which should of course form a correspondence subset). Table 1. Proposed DRMS keywords, and how determined (Type determined by format; reals double unless noted *) Keywords in parentheses may not be necessary if Tucson site data are excluded from ingestion. DRMS name GONG key format Carr_PosAngle from (PA, HR, TIMEZONE, tables) %.6f * Img_Scale_X from (RADIUS, C_MA, PIXLENX) %.6f * Img_Scale_Y from (RADIUS, C_MA, PIXLENY) %.6f * Obsrv_Time from (DATE-0BS, TIME-OBS) %t (TAI) GONG:A(1) A(1) %.7f * GONG:A(1)_ERR A(1)_ERR %.9f * GONG:A(2) A(2) %.8f * GONG:A(2)_ERR A(2)_ERR %.9f * GONG:A(3) A(3) %.8f * GONG:A(3)_ERR A(3)_ERR %.9f * GONG:A(4) A(4) %.9f * GONG:A(4)_ERR A(4)_ERR %.9f * GONG:A(5) A(5) %.9f * GONG:A(5)_ERR A(5)_ERR %.9f * (GONG:ACCUM ACCUM %d) GONG:AN_ERR AN_ERR %.6f * GONG:AUTOCAL0 AUTOCAL0 %d GONG:AUTOCAL1 AUTOCAL1 %d Obsrvr_Lat_HG B0 %.7f * Atmos_Pressure BAROMET %.1f * GONG:BETA BETA %.9f GONG:CAL_DONE CAL_DONE T/F (GONG:CAL_LENS CAL_LENS %d) GONG:CENTER CENTER T/F -> GONG:CENTROXC CENTROXC %.4f * -> GONG:CENTROYC CENTROYC %.4f * GONG:CHI2 CHI2 %.8f * GONG:CMA_ERR CMA_ERR %.8f * GONG:CMI_ERR CMI_ERR %.8f * Commentary COMMENT... %s (\n delimit records) (GONG:COUNT COUNT %d) ImgEllipse_SemiMajor C_MA %.4f * ImgEllipse_SemiMinor C_MI %.4f * GONG:DATE-OBS DATE-OBS %s Sun_Declination DEC * 180 / Pi %.7f ? GONG:DELTA_L0 DELTA_L0 %.14f GONG:DIODE1 DIODE1 %d GONG:DIODE2 DIODE2 %d GONG:DIODE3 DIODE3 %d Obsrvr_Dist_HC DISTANCE %.9f GONG:DONUT (DONUT)?T:F T/F Data_Type from DTYPE %s / I -> Intensity, V -> Doppler, B -> Magnetogram, M -> Modulation GONG:EAST EAST %d Obsrvr_Elevation_MSL ELEV %.1f * GONG:EPITCH EPITCH %.9f GONG:EROLL EROLL %.9f GONG:EROTATOR EROTATOR %.9f GONG:EXT_QUAL EXT_QUAL %.6f * GONG:EXT_SLOP EXT_SLOP %.9f * GRASP:fndlmb_angle FNDLMBAN %.5f * GRASP:fndlmb_ma FNDLMBMA %.4f * GRASP:fndlmb_mi FNDLMBMI %.4f * GRASP:fndlmb_xc FNDLMBXC %.4f * GRASP:fndlmb_yc FNDLMBYC %.4f * GRASP_History GHIST000... GHIST037 %s GONG:GPSSTAT GPSSTAT %d GRASP:Version GRASPVER %s (GONG:GUIDE GUIDE T/F) Sun_HourAngle HA %.6f * GONG:HDLVL HDLVL %d FITS_History HISTORY..., ORIGIN... DATE... %s (\n delimit records) GONG:HUM_INS HUM_INS %.1f * Atmos_Humidity HUM_OUT %.1f * (GONG:INSTMODE INSTMODE %d) GONG:INSTTIME INSTTIME %s IRAF:DataMax IRAF-MAX %.6E * IRAF:DataMin IRAF-MIN %.6E * IRAF:FileName IRAFNAME %s GONG:ITIME ITIME %.9f -> JulianDate2000 J2000 %.9f Obsrvr_Lon_HG L0 %.5f GONG:LAST_PM LAST_PM %s Obsrvr_Lat_Geo LAT * 180 / Pi %.6f GONG:LHOFS0 LHOFS0 %d GONG:LHOFS1 LHOFS1 %d GONG:LHOFS2 LHOFS2 %d GONG:LMB_GRAD LMB_GRAD %.7f * Obsrvr_Lon_Geo LON * 180 / Pi %.6f GONG:LONG LONG %.5f * / with some exceptions, LONG and LON are identical except for precision GONG:LST LST %.9f GONG:LTV3 LTV3 %.0f * GONG:MA_ERR MA_ERR %.7f * GONG:MI_ERR MI_ERR %.7f * GONG:MOTOR0 MOTOR0 %d GONG:MOTOR1 MOTOR1 %d GONG:MOTOR2 MOTOR2 %d GONG:MOTOR3 MOTOR3 %d GONG:MOTOR4 MOTOR4 %d GONG:MTF_FILE MTF_FILE %s GONG:NZF_MA NZF_MA %d GONG:OSC_DAC OSC_DAC %d GONG:PA PA %.9f GONG:PHASE PHASE %d GONG:PITCH PITCH %.9f Sun_RightAscension RA * 180 / Pi %.7f Sun_Semidiam_App RADIUS * 6.48e5 / Pi %.3f * / cf. SEMIDIAM GONG:RAD_NET RAD_NET %.1f GONG:RIN RIN %.0f * / constant if present GONG:ROLL ROLL %.9f GONG:ROTATOR ROTATOR %.9f GONG:ROUT ROUT %.2f * / constant if present GONG:SEMIDIAM SEMIDIAM %.17f / cf. RADIUS GONG:SERIAL SERIAL %s GONG:SiteCode SITE %s GONG:SiteName SITENAME %s GONG:SKYVALUE SKYVALUE %.0f * / constant if present GONG:SLR SLR %.8f * GONG:SN_CAM SN_CAM %s GONG:SN_DAS SN_DAS %s GONG:SN_MOD SN_MOD %s GONG:SN_ROT SN_ROT %s GONG:SPT2DSK SPT2DSK %.7f * GONG:TAPELVL TAPELVL %d GONG:TEMP_INS TEMP_INS %.1f * Atmos_Temperature TEMP_OUT %.1f * GONG:TEMP_RAC TEMP_RAC %.1f * GONG:TEMP_TBL TEMP_TBL %.1f * GONG:TIME-OBS TIME-OBS %s GONG:TIMEZONE TIMEZONE %d (GONG:TYPE TYPE %d) GONG:VCOR1 VCOR1 %.13f GONG:VCOR2 VCOR2 %.12f GONG:VCOR3 VCOR3 %.13f GONG:VCOR4 VCOR4 %.16f Obsrvr_Vel_R_HC VELOCITY %.9f GONG:VELSCALE VELSCALE %.5f * GONG:VEL_BIAS VEL_BIAS %.4f * GONG:VERSION VERSION %s GONG:WAXMAP01 WAXMAP01 %s GONG:WEST WEST %d (GONG:WHEEL_1 WHEEL_1 %d) Atmos_WindDir WIND_DIR %.2f * Atmos_WindSpeed WIND_SPD %.6f * GONG:XC_ERR XC_ERR %.8f * GONG:X_CENTER X_CENTER %d GONG:YC_ERR YC_ERR %.8f * GONG:Y_CENTER Y_CENTER %d Sun_ZenithAngle ZENITH * 180 / Pi %.7f The above list does not include the GONG keywords that are, as far as I can tell, fixed in value. It is possible to define keywords with static values for these, following the same regular rule KEYWORD -> GONG:KEYWORD, or they could be simply be added by a GONG FITS file exporter. A few of course are FITS reserved. Table 2. GONG keywords to check on ingest The following keywords should always be present and have the specified values. except as indicated, corresponding DRMS keywords need not be ingested. A(6) 0. A(6)_ERR 0. A(7) 0. A(7)_ERR 0. ANN_CTYP 2 AUTOCAL T BITPIX 16 CAMX_MAX 960 CAMX_MIN 60 CAMY_MAX 1024 CAMY_MIN 0 CLKMODE 2 COMPRESS 'none_1p_64_003' CONNECT T DAS_STAT 0 DRMIN 0.2 EXTYPE 2 FCOL 3 FLATTED T FROW 3 GONGPLUS 'T ' HDBANK 0 HORIZON 80 IRAF-BPX 32 IRAFTYPE 'REAL ' LCOL 858 LROW 858 LTM1_1 1. LTM2_2 1. LTM3_3 1. MAGNETO T MAX_ITR 1 MAX_ZERO 125 MOD_TYPE 2 NAXIS 2 NAXIS1 860 NAXIS2 860 NTERMS 5 NZF_MI 0 ORDER 3 P_ANGLE 0. PIXLENX 1. PIXLENY 1. RADLIM 0.95 REJECT 1 REPLACE 0 RSCALE 1 SEARCH 2 SIMPLE T SUNUP T TAPEBANK 0 THRESH 2.5 VALID T WAT0_001 'system=physical' WAT1_001 'wtype=linear' WAT2_001 'wtype=linear' WAT3_001 'wtype=linear' WCSDIM 3 WHEEL_2 3 The following keywords may be present; if so, they should have the specified values: DONUT T RIN 0. ROUT 1.02 SKYVALUE 0. If the keyword FILLED is present, the data record, or at least its data segment, should not be ingested. In addition to the above, the following keywords are constant for all the regular sites (excluding Tucson) and ought to be included in Table 2 if only records from the regular sites are ingested: ACCUM 600 CAL_LENS 0 COUNT 300 GUIDE T INSTMODE 0 TYPE 0 WHEEL_1 0 In addition to constant-value keywords, there are a few that are constant within a restricted subset of records, particularly all records for a given site or a given observable. These could be ingested as links rather than per-record keywords. Site-dependent and constant keywords are: SITE, SITENAME, AUTOCAL0, LON, MOTOR4, and TIMEZONE. Additionally, the keywords AUTOCAL1, CENTER, ELEV, LAT, and LONG are usually, but not always fixed for a given site. In the case of CENTER this is probably just a sampling accident. Variations in the others, when they exist, need to be understood. The observable-dependent and constant keywords are DTYPE and WAXMAP01. LTV3 is evidently observable-dependent, but missing from Dopplergrams. For keywords that are *nearly* constant, but only rarely differ from their standard values, it may be useful instead of including them on a per-record basis to record non-standard values in the Commentary string, or another specially-defined text keyword for non-standard values. This would save space in the DRMS, but at the cost of overhead in processing, as the commentary would always have to be checked for deviant key values. However, since the affected keywords are unlikely to be examined except by a FITS exporter, that may not be such a problem. On export, the original GONG header structure should be reproduced to the maximum extent possible, consistent with the FITS standard, but it should also be clear that the data was exported from DRMS rather than directly from the GONG pipeline; certainly DATE and ORIGIN need to be changed according to the standard. Given that the FITS HDU's will be recognizable as coming from JSOC, it should be permissible to use the fixed point and string formats of DRMS rather than those of the original headers. Note that the original value(s) of ORIGIN and DATE will be appended to FITS_History on ingestion. Table 3 lists all of the known GONG keywords and describes how they are to be reconstructed on export as FITS files. Note that GONG uses fixed format character strings, blank-padded to a minimum length of 8 or 18 characters; we may export using free format character strings. Also, the real keyvalues in exponential format (IRAF-MAX, IRAF-MIN) are written in a format not readily duplicated by standard printf() , with a single digit in the exponent. For GONG keywords with known or suspected standard comments, the comments are included in Table 2. As noted above, the program ingesting the data should check the comments and assure that they are consistent. They should be reproduced on export, with the exception of those for FITS reserved keywords BITPIX, BSCALE, (BZERO), DATE, NAXIS, NAXIS1, NAXIS2, ORIGIN, and SIMPLE, for which standard JSOC FITS writer comments may be substitued. Table 3. GONG keywords, and how to reconstruct them on export Values marked * appear to be constant for all non-Tucson site data. A_1 GONG:A(1) A_1_ERR GONG:A(1)_ERR A_2 GONG:A(2) A_2_ERR GONG:A(2)_ERR A_3 GONG:A(3) A_3_ERR GONG:A(3)_ERR A_4 GONG:A(4) A_4_ERR GONG:A(4)_ERR A_5 GONG:A(5) A_5_ERR GONG:A(5)_ERR A_6 0. A_6_ERR 0. A_7 0. A_7_ERR 0. ACCUM 600* / Per minute accumulations ANN_CTYP 2 AN_ERR GONG:AN_ERR AUTOCAL T / automatic daily cals enabled/disabled T/F AUTOCAL0 GONG:AUTOCAL0 / Site autocal start time (minutes past hour) AUTOCAL1 GONG:AUTOCAL1 / Site autocal stop time (minutes past hour) B0 Obsrvr_Lat_HG BAROMET Atmos_Pressure / Barometric Pressure in millibars BETA GONG:BETA / Parallactic angle (radians) BITPIX replaced / FITS BITS/PIXEL BSCALE replaced / REAL = TAPE*BSCALE + BZERO BZERO replaced CAL_DONE GONG:CAL_DONE / autocal has been initiated today T/F CAL_LENS 0* / Calibration lens position CAMX_MAX 960 / image x-axis stop column CAMX_MIN 60 / image x-axis start column CAMY_MAX 1024 / image y-axis stop row CAMY_MIN 0 / image y-axis start row CENTER GONG:CENTER / auto image centering enabled/disabled T/F CENTROXC GONG:CENTROXC (?) CENTROYC GONG:CENTROYC (?) CHI2 GONG:CHI2 CLKMODE 2 / VME-SG mode: 0=gen 1=trans 2=sync-gen CMA_ERR GONG:CMA_ERR CMI_ERR GONG:CMI_ERR COMMENT successive (<=70 char) newline delimited segments of Commentary "[n] ... COMPRESS 'none_1p_64_003' CONNECT T / connected to instrument T/F COUNT 300* / number accumulations per Magnetogram state C_MA ImgEllipse_SemiMajor C_MI ImgEllipse_SemiMinor DAS_STAT 0 / DAS status DATE replaced / Date FITS file was generated DATE [2] not included DATE-OBS GONG:DATE-OBS / GPS date of observation DEC Sun_Declination / Declination (radians) * Pi / 180 (%.10f) DELTA_L0 GONG:DELTA_L0 (?) DIODE1 GONG:DIODE1 / sum of diode values from phase plane 1 DIODE2 GONG:DIODE2 / sum of diode values from phase plane 2 DIODE3 GONG:DIODE3 / sum of diode values from phase plane 3 DISTANCE Obsrvr_Dist_HC / Solar distance(AU) DONUT T iff GONG:DONUT : T; else not included DRMIN 0.2 DTYPE from Data_Type / Data type EAST GONG:EAST / Site east limb offset (arc-seconds) ELEV Obsrvr_Elevation_MSL END replaced EPITCH GONG:EPITCH / Ephemeris pitch angle (radians) EROLL GONG:EROLL / Ephemeris roll angle (radians) EROTATOR GONG:EROTATOR / Ephemeris rotator angle (radians) EXTYPE 2 EXT_QUAL GONG:EXT_QUAL EXT_SLOP GONG:EXT_SLOP FCOL 3 FILLED 'no' iff no data segment; else not included FLATTED T FNDLMBAN GRASP:fndlmb_angle FNDLMBMA GRASP:fndlmb_ma FNDLMBMI GRASP:fndlmb_mi FNDLMBXC GRASP:fndlmb_xc FNDLMBYC GRASP:fndlmb_yc FROW 3 GHIST000 successive 65-char segments of GRASP_History as available; else not included ... GHIST037 GHISTSEQ strlen (GONG:GRASP_History) / 65; GONGPLUS 'T' (or T ?) GPSSTAT GONG:GPSSTAT / GPS receiver time quality error exponent GRASPVER GRASP:Version GUIDE T* / Turret is guiding HA Sun_HourAngle / Hour angle (radians) HDBANK 0 / current hard-drive bank 0=Primary,1=Secondary HDLVL GONG:HDLVL / #minutes since last write to tape/reboot HISTORY successive (<=70 char) newline delimited segments of FITS_History "[n] ... HORIZON 80 / Site horizon limit (degrees from zenith) HUM_INS GONG:HUM_INS / Inside Relative Humidity in percent HUM_OUT Atmos_Humidity / Outside Relative Humidity in percent INSTMODE 0* / Current calibration state of instrument INSTTIME GONG:INSTTIME / INST COMPUTER TIME IRAF-BPX 32 / DATA BITS/PIXEL IRAF-MAX IRAF:DataMax / DATA MAX IRAF-MIN IRAF:DataMin / DATA MIN IRAFNAME IRAF:FileName / NAME OF IRAF IMAGE FILE IRAFTYPE 'REAL' / PIXEL TYPE ITIME GONG:ITIME / time between EOI interrupts for OBS type J2000 JulianDate2000 (?) / Julian 2000 date L0 Obsrvr_Lon_HG LAST_PM GONG:LAST_PM / date of last Preventative Maintenance LAT Obsrvr_Lat_Geo / Site latitude (radians north) * Pi / 180 (%.9f) LCOL 858 LHOFS0 GONG:LHOFS0 / Roll low/high offset LHOFS1 GONG:LHOFS1 / Pitch low/high offset LHOFS2 GONG:LHOFS2 / Rotator low/high offset LMB_GRAD GONG:LMB_GRAD LON Obsrvr_Lon_Geo / Site longitude (radians east) * Pi / 180 (%.9f) LONG GONG:LONG LROW 858 LST GONG:LST / Local apparent sidereal time (radians) LTM1_1 1. LTM2_2 1. LTM3_3 1. LTV3 GONG:LTV3 MAGNETO T / magnetogram mode on/off T/F MAX_ITR 1 MAX_ZERO 125 MA_ERR GONG:MA_ERR MI_ERR GONG:MI_ERR MOD_TYPE 2 MOTOR0 GONG:MOTOR0 / Roll encoder offset MOTOR1 GONG:MOTOR1 / Pitch encoder offset MOTOR2 GONG:MOTOR2 / Rotator encoder offset MOTOR3 GONG:MOTOR3 MOTOR4 GONG:MOTOR4 MTF_FILE GONG:MTF_FILE NAXIS replaced / NUMBER OF AXES NAXIS1 replaced / NAXIS2 replaced / NTERMS 5 NZF_MA GONG:NZF_MA NZF_MI 0 ORDER 3 ORIGIN replaced / ORIGIN [2] not included / FITS file originator OSC_DAC GONG:OSC_DAC / crystal oscillator DAC value PA GONG:PA / Solar position angle (radians) PHASE GONG:PHASE / GPS Phase Offset PITCH GONG:PITCH / Pitch position (radians) PIXLENX 1. PIXLENY 1. P_ANGLE 0. RA Sun_RightAscension / Right ascension (radians) * Pi / 180 (%.9f) RADIUS Sun_Semidiam_App / Apparent solar radius (radians) * Pi / 180 (%.9f) RADLIM 0.95 RAD_NET GONG:RAD_NET / Net Radiation in watts/square meter REJECT 1 REPLACE 0 RIN (0.) ? 0. : not included ROLL GONG:ROLL / Roll position (radians) ROTATOR GONG:ROTATOR / Rotator position (radians) ROUT (1.02) ? 1.02 : not included RSCALE 1 SEARCH 2 SEMIDIAM GONG:SEMIDIAM (%.17f) SERIAL GONG:SERIAL SIMPLE replaced / FITS STANDARD SITE GONG:SiteCode / Site abbreviation SITE [2] not included SITENAME GONG:SiteName / Site name SITENAME [2] not included SKYVALUE (0.) ? 0. : not included SLR GONG:SLR SN_CAM GONG:SN_CAM / serial number camera SN_DAS GONG:SN_DAS / serial number DAS SN_MOD GONG:SN_MOD / serial number modulator SN_ROT GONG:SN_ROT / serial number rotator SPT2DSK GONG:SPT2DSK SUNUP T / Sun is up TAPEBANK 0 / current DLT Tape bank status TAPELVL GONG:TAPELVL / % remaining on tape in current tape bank TEMP_INS GONG:TEMP_INS / Inside Temperature in degrees C TEMP_OUT Atmos_Temperature / Outside Temperature in degrees C TEMP_RAC GONG:TEMP_RAC / Rack Temperature in degrees C TEMP_TBL GONG:TEMP_TBL / Table Temperature in degrees C THRESH 2.5 TIME-OBS GONG:TIME-OBS / GPS time of observation TIMEZONE GONG:TIMEZONE / Site timezone (minutes) TIMEZONE [2] not included TYPE 0* / Observation type VALID T / Image is valid VCOR1 GONG:VCOR1 VCOR2 GONG:VCOR2 VCOR3 GONG:VCOR3 VCOR4 GONG:VCOR4 VELOCITY Obsrvr_Vel_R_HC / velocity of the solar disk (meters/sec) VELSCALE GONG:VELSCALE VEL_BIAS GONG:VEL_BIAS VERSION GONG:VERSION / Software release version WAT0_001 'system=physical' WAT1_001 'wtype=linear' WAT2_001 'wtype=linear' WAT3_001 'wtype=linear' WAXMAP01 GONG:WAXMAP01 WCSDIM 3 WEST GONG:WEST / Site west limb offset (arc-seconds) WHEEL_1 0* / Calibration wheel 1 position WHEEL_2 3 / Calibration wheel 2 position WIND_DIR Atmos_WindDir / Wind direction in degrees azimuth WIND_SPD Atmos_WindSpeed / Wind speed in meters/sec XC_ERR GONG:XC_ERR X_CENTER GONG:X_CENTER / current X position for center of solar image YC_ERR GONG:YC_ERR Y_CENTER GONG:Y_CENTER / current Y position for center of solar image ZENITH Sun_ZenithAngle / Zenith distance (radians) Limiting the size of the ingested data The total number of proposed keywords to be put into the relevant tables is about 140, 125 if certain near constant or site- or obervable- dependent keywords can be handled from ancillary (tiny) tables. The distribution of types is approximately as follows: Double: 24 Float: 56 Int: 24 Long String: 3 String: 17 Logical: 3 (Long strings are multi-card commentaries, > 60 chars; most strings are 1-20 chars, and I think most are fixed-length.) Each record would thus require about 515 bytes for the numeric data (some of the int's may in fact be char's), perhaps 200 bytes for the short strings, and around 3 kB for the long strings. The long commentary strings (COMMENT, HISTORY, GHIST*) clearly make a big difference; they should perhaps be kept as data segments rather than in DRMS. It is certainly doubtful that they would be used for many queries. Of course if the GONG data segments are kept in the form of the original FITS files, there is no reason to ingest the commentary as separate data segments at all, except for possible ease of use and to gain experience with that kind of metadata. I estimate the total number of records at about 6e6, based on an average of 450 records per daily data set. That works out to an average of about 1.25 images per minute based on roughly 9 years of data, which seems reasonable; it might be a bit more. The total size of the tables in the database would thus be (initially) 4 GB if the long commentary strings are not included, about 22 GB if they are. Since the JSOC database is currently kept on a disk with 800 GB of space, the impact of ingesting the full GONG data headers is minor; if the commentary keys are ingested as data segments, or not at all, it is negligible. Since the size of each record returned would be either 700 B or 4 kB, the virtual memory required to hold a return set of, for example, 1500 records, a day's worth, would be 1 MB or 6 MB. Even with the larger records, a year's worth of records would require about 2 GB of memory.