HMI Flatfield Processing

(FF Processing Sequence)

Data Series Frequency Created By Used By Applied To
su_production.flatfield_fid Per FID module_flatfield module_flatfield_combine -
hmi.flatfield Daily module_flatfield_combine lev1 processing hmi.lev1
hmi.flatfield_update Daily module_flatfield_combine not currently used future observables reprocessing
hmi.offpoint_flatfield Quarterly/
Weekly
flat_inorbit.pro/
lev0a2deflev1_FULL_PZT_FSN_cron.pl
module_flatfield_combine -

Description:
su_production.flatfield_fid:
hmi.flatfield:
hmi.flatfield_update:
hmi.offpoint_flatfield:

Source Code:
~/cvs/JSOC/proj/flatfield/apps/module_flatfield
~/cvs/JSOC/proj/flatfield/apps/module_flatfield_combine
~/cvs/JSOC/proj/flatfield/off_flat_IDL/flat_inorbit.pro
~/cvs/JSOC/proj/lev0/scripts/lev0a2deflev1_FULL_PZT_FSN_cron.pl
~/cvs/JSOC/proj/lev0/scripts/pzt_flat_cron.pl
~/cvs/JSOC/proj/lev0/apps/pztflat/pzt_flatfield.pro

To run quarterly hmi.offpoint_flatfield

README

Troubleshooting

hmi.offpoint_flatfield is not being updated weekly

 Processing logs can be found at /scr21/production/qsub/flat/pzt2.*.log, 
 though they are scrubbed weekly and failures may not be caught in time.  You'll
 likely need to rerun by hand:
  1.  make sure the lev0a data are online.  You are looking for hftsacid=3021 for
      2 runs taken 24 hours apart, the last one being on the day you expect the
      update.  You need both of those runs.  This is how I find and retrieve them:

      @ fsn = `/home/jeneen/campaigns/scripts/hmi/tai2fsn 2011.10.01
      @ fsn1 = $fsn - 46100
      @ fsn2 = $fsn + 46100
      show_info -q "hmi.lev0a[$fsn1-$fsn][? hftsacid = 3021 ?]" key=fsn > /tmp/pzt.find
      show_info -q "hmi.lev0a[$fsn-$fsn2][? hftsacid = 3021 ?]" key=fsn > /tmp/pzt.find2
      @ start = `head -1 /tmp/pzt.find`
      @ end = `tail -1 /tmp/pzt.find`
      @ start2 = `head -1 /tmp/pzt.find2`
      @ end2 = `tail -1 /tmp/pzt.find2`
      echo "$i  $start'-'$end  $start2'-'$end2"
      echo "$start'-'$end," >> get_lev0a
      echo "$start2'-'$end2," >> get_lev0a
      ex -s get_lev0a'<<'FOO
      g/,$/s//]/
      FOO
 
      /home0/phil/cvs/JSOC/proj/workflow/scripts/stage_tapes_in_order.csh 5 `cat get_lev0a`

  2.  Run the PZT processing in IDL (it takes awhile to run - 1/2 hour?).
      cd /home/production/cvs/JSOC/proj/lev0/apps/pztflat
      IDL> .run pzt_flatfield
            pzt_flatfield, 2011,10,01
  
  3.  This creates binary files and ingest scripts.  Use IDL to load the binary files and
      look at them.  If they look OK, run the ingest scripts.