Calibration Version keywords

A keyword for calibration version keywords are included in level-1 data (CALVER32) through higher level products (CALVER64) starting in August 2012. This keyword is, at the start, considered to be a set of fields consisting of single hexidecimal digits.

The presently assigned values are from least significant nibble:

The default value of CALVER64 is 0 and should never be seen so long as QUALITY >= 0. The concept of a missing value for a CALVERxx field does not make sense since CALVERxx refers to the version of code or source of values vs the decisions made by that code. If for instance the limb fit fails in lev1 processing, HFCORRVR still should contain the code version but the appropriate QUALITY bit should show that that code could not produce a useful value. Higher level products may propagate CALVER64. If a new product record is made from multiple input records then that code must decide the range of input CALVER that are acceptable and if different versions are OK for the output product, which value to forward. A good choice would be to send forward the lowest level from the input records to indicate that older versions were used in the output product. This could be used later to decide which products should be updated when the input series itself is updated.

NOTE: Data products above hmi.lev1 do not have CALVER64 set for records with no usable data. So if QUALITY < 0 you should expect CALVER64 to be 0. Also due to a bug my program that updated existing records, for data made prior to 5 Sept 2012 the HFCORRVR values was not set, so values of 0 and 2 in that field should be taken to mean 2. I.e. since to date, only one version of the height of formation code was in use, HFCORRVR of 0 and 2 both mean the same thing. After a lot of discussion we agreed to not update the older ~10M records at this time. They will be corrected when the next calibration update is applied. (PHS 1 Oct 2012).

JSD entries for CALVER32 and CALVER64:

Keyword:CALVER32, int, variable, record, 0, "0x%08X", "none", "Calibration Version lev1"

Keyword:CALVER64, longlong, variable, record, 0, "0x%016llX", "none", "Calibration Version"

Prior to Aug 2012 hmi.lev1 had a keyword named HFCORRVR which can be taken as correct for fields 0 and 1 except that a 0 in the first nibble (field 0) should be changed to 2 to match the other records made with same code.

In the process of adding CALVER64 and CALVER32 for existing records these keywords should be set to 0x12 when updated with the post-Transit-of-Venus CROTA2 values. The plan for doing the August 2012 update is described at Crota2UpdatePlan.

Starting 1 Oct 2012 fields 2 and 3 are populated. The older data have not been reprocessed (as of 15 Jan 2013).

Note 2: Mod C and Mod L Processing

Regular use of the HMI mod-L frame list began at 2016.04.13_19:12:55 (fns=104683793). Mod-L requires use of filtergrams from both HMI cameras to compute the Stokes vector products. Errors in the processing were discovered in July and September 2017 that were fixed in September and October 2017. Reprocessing of the data are underway. Details are provided below.

• Check the value of CALVER64 to make sure that you use data with the appropriate processing!

The first bug discovered in Mod-L processing was the more serious because filtergrams were not properly aligned; the difference in p-angle rotation of about 0.08 degrees between the two cameras was not corrected. This resulted in misalignment of about 3 pixels at the limbs and most severely contaminated the polar field and other limb measurements that combine data from the two cameras. Data with this error should only be used with extreme caution.

The second bug resulted in the use of only half of the available 'I+V' and 'I-V' filtergrams. In mod-L two complete sets of 'I +/- V' filtergrams are collected each 90s, but the filtergram selection was not correct; only half of the available I+V and I-V filtergrams were used and they were used twice. As a consequence the expected noise in the computed values was lower than the actual noise; the 90-s mod-L values that depend on V had the same noise as the original 135-s mod-C. Data processed after 10 October 2017 has 720s values that depend on 'V' have been computed with a filtergram set obtained every 45s rather than 90s for mod-L and 135s for mod C.

How can you tell which data have been processed properly?

Bits 16-19 of CALVER64 in vector products indicate the observing sequence and basic processing applied to create the vector field products hmi.S and subsequent dependent products. Thus the 4-bit nibble to check is 0xF0000

If (CALVER64 & 0xF0000) >> 16 equals %% {comparison is bitwise, so for IDL use 'AND'}

• 0x0 -> data were collected using mod-C [no known processing issues]

• 0x1 -> not defined

• 0x2 -> mod-L data processed with incorrect alignment of filtergrams from the two cameras and duplicated 'V' filtergrams

• 0x3 -> partially corrected mod-L processing with proper alignment but still using only half the available 'I +/- V' filtergrams

• 0x4 -> mod-L with proper alignment using all appropriate filtergrams

Comments:

• The observed filtergrams actually collected with mod-L have been correct from the start.
• All mod-L data were processed incorrectly until 2017.10.10_XX-CONFIRM PRECISE TIME
• The misalignment bug was corrected in data processed after 1 September 2017 for data with T_REC 2017.08.27_00:00:00 and after
• The filtergram-selection bug was corrected in data processed after 10 October 2017 for data beginning with T_REC 2017:10:XX-CONFIRM PRECISE TIME
• Earlier mod-L data are being reprocessed to make both corrections.