Differences between revisions 17 and 52 (spanning 35 versions)
⇤ ← Revision 17 as of 2011-10-21 03:45:27 →
Size: 9062
Editor: l4-m0
Comment:
← Revision 52 as of 2011-11-15 03:22:11 → ⇥
Size: 11466
Editor: l4-m0
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
The first disambiguated vector field data released by the HMI project for science analysis is a 6-day interval of 12-minute active region patches encompassing AR 11158. The region developed rapidly from February 12-17, 2011 and produced the first X-class flare of Solar Cycle 24 at 01:56 UT on February 15. The first disambiguated vector field data released by the HMI project for science analysis is a 5-day interval of 12-minute active region patches encompassing AR 11158. The region developed rapidly from February 12-16, 2011 and produced the first X-class flare of Solar Cycle 24 at ~01:56 UT on February 15.
Line 5: Line 5:
The vector magnetic field is derived from the standard HMI IQUV 720-second data series. Patches on the Sun that include AR 11158 were identified using the HMI Active Region Patch (HARP) code and then enlarged to provide a surrounding buffer of weak field. The VFISV inversion code was run with very strict convergence criteria to determine the optimal values, this required additional computing resources. The disambiguation was computed with spherical geometry in a limited region with fairly generous thresholds on field strength to optimize the annealing process. The vector magnetic field is derived from the standard HMI IQUV 720-second data series. Patches on the Sun that include AR 11158 were identified using the HMI Active Region Patch (HARP) code and then enlarged to provide a surrounding buffer of weak field. The VFISV inversion code was run with very strict convergence criteria to determine the optimal values, this required additional computing resources compared to the expected standard pipeline processing. The disambiguation was computed with spherical geometry in a limited region with fairly generous thresholds on field strength and very gradual annealing; this also requires significant additional computing resources.
Line 7: Line 7:
Data Files are provided in two formats.
 1. Cut outs of the data are provided in a tracked 650 * 600 CCD-pixel window. The inclination of the field is relative to the local line of sight and the azimuth is measured counter-clockwise from up on the CCD. Keywords provide information about the solar geometry. Note that solar north is down in the movie.
 1. Remapped images computed with a Lambert cylindrical equal area projection centered on the tracked region. The vector field components are heliographic Br, Btheta, and Bphi.		 For this specific release, data have exported into two special formats, direct cut outs and remapped images. Details are provided at '''ReleaseNotes'''.
Line 11: Line 9:
Vector field data must be used with care. There are several issues with vector magnetic field data that make inversions fail in some pixels and definitive disambiguation problemmatic. Additionally several systematic errors must be considered when using the data. These include daily, 12-hour, and other long-period variations associated with SDO's orbital velocity that cause large-scale spatial and temporal variations in sensitivity and cross-talk due to limitations in the knowledge of instrument characteristics. Various confidence and error estimates are provided, but they do not always indicate which pixels are suspect.  * For each time in the "cut out" series there are 10 ~-FITS-~ files that include three vector magnetic field quantities, computed error estimates for each quantity, the three pair-wise correlations between field strength, azimuth and inclination (the square root of the covariances), and a quality map, '''[http://hmi.stanford.edu/doc/magnetic/qual_map.pdf qual_map]''', that gives information about the quality of the processing at each pixel. ~-FITS-~ keywords provide other important information. Maps of additional quantities and covariances can be provided on request. Details can be found in some of the links at the bottom of the page.
Line 13: Line 11:
AR 11158 emerged quickly in a complex way with three major centers of flux emergence. During the build-up to the flare, mixed-polarity regions intensified and horizontal flows were apparent. At the time of the flare significant long-lasting changes in the horizontal magnetic field occurred.  . Cut outs of the original CCD data are provided in a tracked 650 * 600 CCD-pixel window. The field components are the total flux, the inclination of the field relative to the local line of sight, and the azimuth measured counter-clockwise from 'up' on the CCD. Keywords provide information about the solar geometry. Note that solar north is down in the cut-outs.
Line 15: Line 13:
The following paper describing the analysis of the vector field for this region will be submitted to Solar Physics shortly. Sections will be filled in as they become available. More detailed treatments of some aspects of the processing follow.  * The format perhaps more useful for modeling is remapped images computed with a Lambert cylindrical equal area projection centered on the center of the tracked region. The vector field components are heliographic Br, Btheta, and Bphi. Errors for each component and a quality estimate for the nearest original pixel are provided, for a total of 6 files per time step.
Line 17: Line 15:
Vector field data must be used with care. There are several issues with vector magnetic field data that make inversions fail in some pixels and make definitive disambiguation problematic. Additionally, various systematic errors must be considered when using the data. These include daily, 12-hour, and other long-period variations associated with SDO's orbital velocity that cause large-scale spatial and temporal variations in sensitivity and in cross-talk in the Q and U components of the Stokes vector; these arise in part due to limitations in the knowledge of instrument characteristics. Various confidence and error estimates are provided, but they do not always indicate which pixels are suspect.

AR 11158 emerged quickly in a complex way with three major centers of flux emergence. During the build-up to the flare, mixed-polarity regions intensified and strong horizontal flows were apparent. At the time of the flare significant long-lasting changes in the horizontal magnetic field occurred.

The following is an outline of a paper describing the analysis of the vector field for this region will be submitted to Solar Physics shortly. Sections will be filled in as they become available. Pointers to the data and more detailed treatments of some aspects of the processing follow.
Line 20: Line 23:

. 1 Intro – Vector magnetic fields (Todd et al.)		  . 1 Intro – Vector magnetic fields
Line 24: Line 26:
 . 2 Instrument and Data Processing (Sebastien & Jesper)  . 2 Instrument and Data Processing
Line 31: Line 33:
 . 3 IQUV calculation (Sebastien) ''' [http://hmi.stanford.edu/doc/magnetic/observables_code.pdf IQUV Computation] '''
  . 3.1 Computation of IQUV 		  . 3 IQUV calculation ''' [http://hmi.stanford.edu/doc/magnetic/observables_code.pdf Section 3] '''
  . 3.1 Computation of IQUV
Line 35: Line 37:
 . 4 VFISV inversion - ''' [http://hmi.stanford.edu/doc/magnetic/section4.pdf Section 4 {10/10/2011} ]'''  . 4 VFISV inversion ''' [http://hmi.stanford.edu/doc/magnetic/section4.pdf Section 4 ]'''
Line 39: Line 41:
   . 4.2.2 Filter Profile Approximation
   . 4.2.3 Convergence Criteria, multiple minima		    . 4.2.2 Filter Profile Approximation - Wavelength Hack
   . 4.2.3 Model Initialization
   . 4.2.4 Convergence Criteria, multiple minima
Line 42: Line 45:
 . 5 Disambiguation (Graham+KD) - ''' [http://hmi.stanford.edu/doc/magnetic/disambig_overview.pdf Brief Overview]'''
  . 5.1 General method, issues, and uncertainties - planar and spherical ''' [http://hmi.stanford.edu/doc/magnetic/disambig_longer.pdf Disambiguation Details] '''
  . 5.2 HMI specifics, thresholds, mapping, annealing
  . 5.3 Bitmap and confidence index
 . 6 HMI.B Data series and access (Todd)		  . 5 Disambiguation '''[http://hmi.stanford.edu/doc/magnetic/disambig_longer.pdf Section 5] '''
  . 5.1 General method, issues, and uncertainties - planar and spherical
   . 5.1.1 Selection of Pixels to Anneal
   . 5.1.2 Potential Field Calculation
   . 5.1.3 Minimum Energy Methods
   . 5.1.4 Treatment of Weak Field Areas
  . 5.2 Computation Parameters
 . 6 HMI.B Data series and access '''VectorPaper6'''
Line 48: Line 54:
  . 6.2 Details of HMI.B contents - ''' [http://hmi.stanford.edu/doc/magnetic/hmi.ME.Segments.doc hmi.ME data series] '''   . 6.2 Details of HMI.B contents
Line 54: Line 60:
  . 7.2 Coordinate Transformation and Vector Components on Export    . 7.2 Coordinate Transformation and Vector Components on Export
Line 58: Line 64:
 . 8 Known and unresolved issues with HMI data and processing
  . ''' [http://hmi.stanford.edu/doc/magnetic/hmivectorissues.pdf Vector Field Issues] '''		  . 8 Known and unresolved issues with HMI data and processing '''[http://hmi.stanford.edu/doc/magnetic/hmivectorissues.pdf Section 8 Topics]''' {10.26.2011}
Line 61: Line 66:
  . 8.2 Residual Cross-talk    . 8.3 Disambiguation issues
  . 8.4 Other systematic and instrumental issues that affect data
 . 9 Discussion of HMI measurements of AR 11158 - Validation (KD et al.)		   . 8.2 Large-scale Variations
  . 8.3 Bad Pixels
  . 8.4 Comparison of Vector and Line-of-Sight Magnetic Field
  . 8.X Inversion Dependence on Initial Guess
  . 8.X Residual Cross-talk
  . 8.X Disambiguation issues
  . 8.X Other systematic and instrumental issues that affect data
 . 9 Discussion of HMI measurements of AR 11158 - Validation
Line 69: Line 78:
 . 10 Summary (Todd et al.)  . 10 Summary
Line 71: Line 80:
  . 10.2 Uses of data – NLFFF, DAVE4VM, other    . 10.2 Uses of data – NLFFF, DAVE4VM, other
Line 74: Line 83:
 . [ftp:pail.stanford.edu/pub/HMIvector/movie Movies] of the Vector Field
   || [ftp:pail.stanford.edu/pub/HMIvector/movie/Cutout_Br.mov Br Cutout] || [ftp:pail.stanford.edu/pub/HMIvector/movie/bvec_600g.mov Vector Field] || [ftp:pail.stanford.edu/pub/HMIvector/movie/Hedgehog.mov Vector Hedgehog] ||
   || [ftp:pail.stanford.edu/pub/HMIvector/movie/Ic_Bl.mov Intensity + Blos] || [ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_1.mp4 Full Disk Regions 1-16 Feb] || [ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_2.mp4 Full Disk Regions 16-28 Feb] ||
Line 78: Line 84:
 . Vector Field Data - One-Day Tar Files of 12-minute Cadence
   || [ftp:pail.stanford.edu/pub/HMIvector/Cutout CCD Cutout ] || [ftp:pail.stanford.edu/pub/HMIvector/CEA Cylindrical Equal Area Projection ] ||		 '''ReleaseNotes for AR 11158 Vector Field Data'''
Line 81: Line 86:
 . Sample Images and fits Files
   || [ftp:pail.stanford.edu/pub/HMIvector/sample One Frame Per Day Sample Data] ||		 ===== Movies of the Vector Field =====
## [ftp:pail.stanford.edu/pub/HMIvector/movie Movies]
 .
|| [ftp:pail.stanford.edu/pub/HMIvector/movie/Cutout_Br.mov Br Cutout] ||||<style="text-align: center;"> [ftp:pail.stanford.edu/pub/HMIvector/movie/bvec_600g.mov Vector Field] ||
 ||||||<style="text-align: center;"> ||
 || [ftp:pail.stanford.edu/pub/HMIvector/movie/surface_vectorline.mpg Vector Hedgehog] || [ftp:pail.stanford.edu/pub/HMIvector/movie/surface_vectorline_west.mpg WEST] || [ftp:pail.stanford.edu/pub/HMIvector/movie/surface_vectorline_east.mpg EAST] || [ftp:pail.stanford.edu/pub/HMIvector/movie/surface_vectorline_large.mpg Vector Hedgehog (large)] ||

|| [ftp:pail.stanford.edu/pub/HMIvector/movie/Ic_Bl.mov Intensity + Blos] || [ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_1.mp4 Full Disk Regions 1-16 Feb (HARP 0883)] || [ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_2.mp4 Full Disk Regions 16-28 Feb] ||

===== Vector Field Data - One-Day Tar Files of 12-minute Cadence =====
 . Data are currently only available for testing and evaluation.
  . ClickHereForDataRelease
## || [ftp:pail.stanford.edu/pub/HMIvector/Cutout CCD Cutout ] || [ftp:pail.stanford.edu/pub/HMIvector/CEA Cylindrical Equal Area Projection ] ||

===== One-Per-Day Sample Images and FITS Files =====

 . Data are currently only available for testing and evaluation.
  . ClickHereForDataRelease
## || [ftp:pail.stanford.edu/pub/HMIvector/sample One Frame Per Day Sample Data] ||
Line 86: Line 108:
Details of some aspects of the processing can be found below. Watch for updates: Details of some aspects of the processing and some older documentaiton can be found below. Watch for updates:
Line 93: Line 115:
 . ''' [http://hmi.stanford.edu/doc/magnetic/hmi.ME.Segments.doc hmi.ME inverted data detailed description] ''' {updated 7/2011}   . ''' [http://hmi.stanford.edu/doc/magnetic/hmi.ME.Segments.doc hmi.ME inverted data detailed description] ''' {updated 7/2011}
  . ''' [http://hmi.stanford.edu/doc/magnetic/qualmap_description.pdf hmi.B QUALMAP - detailed description] ''' {updated 7/2011}
Line 101: Line 124:
  . Note on ''' [http://hmi.stanford.edu/doc/magnetic/VectorCoordinates.pdf Transformation of Vector Components when Projected ]

 . ''' [http://hmi.stanford.edu/doc/magnetic/ Vector Field Documentation Directory]''' {updated 10/2011}
   . Note on ''' [http://hmi.stanford.edu/doc/magnetic/VectorCoordinates.pdf Transformation of Vector Components when Projected ] '''
 . [http://hmi.stanford.edu/doc/magnetic/ Vector Field Documentation Directory]''' {updated 10/2011} '''
 . HMI Active Region Patches
  . HARP Data Series in JSOC/DRMS ["HARPDataSeries"]
  . HARP Quick-look Movies ''' [http://sun.stanford.edu/~turmon/data/hmi-tracking-test4/ Tracked Movie] '''
  . HARP Movie Format, Overlays, Annotations ''' [http://sun.stanford.edu/~turmon/data/tracker-movies-notes/ Movie Annotations] '''
Line 107: Line 131:
    * Back to VectorMagneticField
    * Back to ["HMIModuleTable"]
    * Back to FrontPage		  * Back to VectorMagneticField
 * Back to ["HMIModuleTable"]
 * Back to FrontPage

Measuring the Vector Magnetic Field of AR 11158 with HMI

The first disambiguated vector field data released by the HMI project for science analysis is a 5-day interval of 12-minute active region patches encompassing AR 11158. The region developed rapidly from February 12-16, 2011 and produced the first X-class flare of Solar Cycle 24 at ~01:56 UT on February 15.

The vector magnetic field is derived from the standard HMI IQUV 720-second data series. Patches on the Sun that include AR 11158 were identified using the HMI Active Region Patch (HARP) code and then enlarged to provide a surrounding buffer of weak field. The VFISV inversion code was run with very strict convergence criteria to determine the optimal values, this required additional computing resources compared to the expected standard pipeline processing. The disambiguation was computed with spherical geometry in a limited region with fairly generous thresholds on field strength and very gradual annealing; this also requires significant additional computing resources.

For this specific release, data have exported into two special formats, direct cut outs and remapped images. Details are provided at ReleaseNotes.

  • For each time in the "cut out" series there are 10 FITS files that include three vector magnetic field quantities, computed error estimates for each quantity, the three pair-wise correlations between field strength, azimuth and inclination (the square root of the covariances), and a quality map, [http://hmi.stanford.edu/doc/magnetic/qual_map.pdf qual_map], that gives information about the quality of the processing at each pixel. FITS keywords provide other important information. Maps of additional quantities and covariances can be provided on request. Details can be found in some of the links at the bottom of the page.

  • Cut outs of the original CCD data are provided in a tracked 650 * 600 CCD-pixel window. The field components are the total flux, the inclination of the field relative to the local line of sight, and the azimuth measured counter-clockwise from 'up' on the CCD. Keywords provide information about the solar geometry. Note that solar north is down in the cut-outs.
  • The format perhaps more useful for modeling is remapped images computed with a Lambert cylindrical equal area projection centered on the center of the tracked region. The vector field components are heliographic Br, Btheta, and Bphi. Errors for each component and a quality estimate for the nearest original pixel are provided, for a total of 6 files per time step.

Vector field data must be used with care. There are several issues with vector magnetic field data that make inversions fail in some pixels and make definitive disambiguation problematic. Additionally, various systematic errors must be considered when using the data. These include daily, 12-hour, and other long-period variations associated with SDO's orbital velocity that cause large-scale spatial and temporal variations in sensitivity and in cross-talk in the Q and U components of the Stokes vector; these arise in part due to limitations in the knowledge of instrument characteristics. Various confidence and error estimates are provided, but they do not always indicate which pixels are suspect.

AR 11158 emerged quickly in a complex way with three major centers of flux emergence. During the build-up to the flare, mixed-polarity regions intensified and strong horizontal flows were apparent. At the time of the flare significant long-lasting changes in the horizontal magnetic field occurred.

The following is an outline of a paper describing the analysis of the vector field for this region will be submitted to Solar Physics shortly. Sections will be filled in as they become available. Pointers to the data and more detailed treatments of some aspects of the processing follow.

Measuring the Vector Magnetic Field of AR 11158 with HMI

  • by the VectorFieldTeam

  • 1 Intro – Vector magnetic fields
    • 1.1 HMI Vector Field Overview
    • 1.2 AR 11158 – X-class Flare
  • 2 Instrument and Data Processing
    • 2.1 HMI instrument
    • 2.2 Framelist for vector field
    • 2.3 Relevant calibration of HMI instrument
    • 2.4 Specifics of vector data camera, filtergrams
    • 2.5 Processing of filtergrams – dark, flat, cosmic rays, gap filling, distortion
    • 2.6 Pipeline and special HMI processing

  • 3 IQUV calculation [http://hmi.stanford.edu/doc/magnetic/observables_code.pdf Section 3]

    • 3.1 Computation of IQUV
    • 3.2 720 calculation – interpolation in time and space, averaging
    • 3.3 Other 720s quantities (very briefly)

  • 4 VFISV inversion [http://hmi.stanford.edu/doc/magnetic/section4.pdf Section 4 ]

    • 4.1 ME method and VFISV summary
    • 4.2 Specifics of implemented VFISV for this data series
      • 4.2.1 Weighting
      • 4.2.2 Filter Profile Approximation - Wavelength Hack
      • 4.2.3 Model Initialization
      • 4.2.4 Convergence Criteria, multiple minima
    • 4.3 Formal Error estimation

  • 5 Disambiguation [http://hmi.stanford.edu/doc/magnetic/disambig_longer.pdf Section 5]

    • 5.1 General method, issues, and uncertainties - planar and spherical
      • 5.1.1 Selection of Pixels to Anneal
      • 5.1.2 Potential Field Calculation
      • 5.1.3 Minimum Energy Methods
      • 5.1.4 Treatment of Weak Field Areas
    • 5.2 Computation Parameters

  • 6 HMI.B Data series and access VectorPaper6

    • 6.1 Organization of data series
    • 6.2 Details of HMI.B contents
    • 6.3 Description of HARPs
    • 6.4 Export – general

    • 6.5 HMI Vector Data Versions and Referencing Data in publications - VectorDataReference

  • 7 Projections - [http://hmi.stanford.edu/doc/magnetic/VectorTransformation.pdf Remapping and Vector Transformation]

  • 8 Known and unresolved issues with HMI data and processing [http://hmi.stanford.edu/doc/magnetic/hmivectorissues.pdf Section 8 Topics] {10.26.2011}

    • 8.1 Periodic Errors
    • 8.2 Large-scale Variations
    • 8.3 Bad Pixels
    • 8.4 Comparison of Vector and Line-of-Sight Magnetic Field
    • 8.X Inversion Dependence on Initial Guess
    • 8.X Residual Cross-talk
    • 8.X Disambiguation issues
    • 8.X Other systematic and instrumental issues that affect data
  • 9 Discussion of HMI measurements of AR 11158 - Validation
    • 9.1 General comments on determination of vector magnetic field
    • 9.2 Time Evolution of AR 11158 – features both real and otherwise
    • 9.3 Validation and limitations of HMI processing and data
    • 9.4 Qualitative comparison with B_LoS, Hinode, and other field measures
  • 10 Summary
    • 10.1 Remaining work for pipeline modules
    • 10.2 Uses of data – NLFFF, DAVE4VM, other

Access to AR 11158 Vector Field Data

ReleaseNotes for AR 11158 Vector Field Data

Movies of the Vector Field

[ftp:pail.stanford.edu/pub/HMIvector/movie/Cutout_Br.mov Br Cutout]

[ftp:pail.stanford.edu/pub/HMIvector/movie/bvec_600g.mov Vector Field]

[ftp:pail.stanford.edu/pub/HMIvector/movie/Ic_Bl.mov Intensity + Blos]

[ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_1.mp4 Full Disk Regions 1-16 Feb (HARP 0883)]

[ftp:pail.stanford.edu/pub/HMIvector/movie/track-movie.2011-feb-NEW_2.mp4 Full Disk Regions 16-28 Feb]

Vector Field Data - One-Day Tar Files of 12-minute Cadence

One-Per-Day Sample Images and FITS Files

A collection of topical discussions of the processing

Details of some aspects of the processing and some older documentaiton can be found below. Watch for updates:

JsocWiki: VectorPaper (last edited 2013-05-01 04:35:25 by localhost)