|
Size: 31860
Comment:
|
Size: 31771
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from BarbaraThompson | |
| Line 3: | Line 4: |
| . [[ImageLink(aia_logo.png,http://aia.lmsal.com/,width=150)]] | |
| Line 6: | Line 7: |
| There are a wide range of tools available to access AIA data and images. They were designed to suit users with a wide variety of needs. This page guides a user through the AIA data system based on their needs and interests. First and foremost, a user should understand why a versatile data system is necessary. An AIA full-resolution image has 4096x4096 pixels, and AIA has [[http://aia.lmsal.com/public/instrument.htm ten different AIA wave bands, or channels]]. A single full-resolution JPEG can be up to 10 MB in size (a single uncompressed science data file can be dozens of MB) and the AIA performs these observations continuously at a rate of approximately 1 image per second. With previous solar missions, it was not unusual for a user to request a few hours of data. In the case of the AIA, this request would amount to over half a terabyte of data. Even one hour of images at a single wavelength is on the order of 10 GB in size. It is clear that a simple data access system is not suitable for the AIA. We have devised a number of tools so that the user can access specific data sets as quickly as possible. ''Please note: This guide does not contain the full range of tools and resources available to AIA data users. The user has a great deal of flexibility, and there is no single "correct" way to identify, obtain, and analyze AIA data. It is hoped that this guide will help the user get started with AIA data, and that users will familiarize themselves with the additional resources over time.'' |
. There are a wide range of tools available to access AIA data and images. They were designed to suit users with a wide variety of needs. This page guides a user through the AIA data system based on their needs and interests. First and foremost, a user should understand why a versatile data system is necessary. An AIA full-resolution image has 4096x4096 pixels, and AIA has [[http://aia.lmsal.com/public/instrument.htm ten different AIA wave bands, or channels]]. A single full-resolution JPEG can be up to 10 MB in size (a single uncompressed science data file can be dozens of MB) and the AIA performs these observations continuously at a rate of approximately 1 image per second. With previous solar missions, it was not unusual for a user to request a few hours of data. In the case of the AIA, this request would amount to over half a terabyte of data. Even one hour of images at a single wavelength is on the order of 10 GB in size. It is clear that a simple data access system is not suitable for the AIA. We have devised a number of tools so that the user can access specific data sets as quickly as possible. ''Please note: This guide does not contain the full range of tools and resources available to AIA data users. The user has a great deal of flexibility, and there is no single "correct" way to identify, obtain, and analyze AIA data. It is hoped that this guide will help the user get started with AIA data, and that users will familiarize themselves with the additional resources over time.'' |
| Line 16: | Line 12: |
You can browse full-disk movies of the Sun in all of the AIA wavelengths on the [[http://sdo.gsfc.nasa.gov/data/aiahmi/browse.php SDO Browse Data page]]. These movies are lower resolution (512x512 or 1024x1024) and are much lower cadence (15 minutes). There are also [[http://sdo.gsfc.nasa.gov/assets/img/dailymov/ daily 1024x1024 movies]] for each AIA wavelength. |
. You can browse full-disk movies of the Sun in all of the AIA wavelengths on the [[http://sdo.gsfc.nasa.gov/data/aiahmi/browse.php SDO Browse Data page]]. These movies are lower resolution (512x512 or 1024x1024) and are much lower cadence (15 minutes). There are also [[http://sdo.gsfc.nasa.gov/assets/img/dailymov/ daily 1024x1024 movies]] for each AIA wavelength. |
| Line 21: | Line 14: |
[[http://helioviewer.org/ Helioviewer]] is a versatile tool that allows the user to view images and movies from multiple data sources, zoom in on specific regions and view zoomed-in movies, and overlay the images in multiple layers to compare features and structure. Helioviewer also has the capability to track a region, meaning that the viewing box moves with the Sun's rotation. The result can be saved as a link and accessed at a later date. A full-resolution AIA image is 4096x4096 pixels, and standard displays are not able to show anything near that size. Helioviewer allows the user to choose between viewing the whole Sun in (much) lower resolution, or viewing a subfield at better or even full resolution. A desktop version, called [[http://www.jhelioviewer.org/ JHelioviewer]], can be downloaded onto your computer. The user is able to create movies and export them in MPEG or Quicktime format. |
. [[http://helioviewer.org/ Helioviewer]] is a versatile tool that allows the user to view images and movies from multiple data sources, zoom in on specific regions and view zoomed-in movies, and overlay the images in multiple layers to compare features and structure. Helioviewer also has the capability to track a region, meaning that the viewing box moves with the Sun's rotation. The result can be saved as a link and accessed at a later date. A full-resolution AIA image is 4096x4096 pixels, and standard displays are not able to show anything near that size. Helioviewer allows the user to choose between viewing the whole Sun in (much) lower resolution, or viewing a subfield at better or even full resolution. A desktop version, called [[http://www.jhelioviewer.org/ JHelioviewer]], can be downloaded onto your computer. The user is able to create movies and export them in MPEG or Quicktime format. [[ImageLink(helioviewer_snapshot.jpg,http://helioviewer.org,align=right,width=150)]] |
| Line 29: | Line 16: |
Most of the features or events a user would need have already been assembled in the [[http://lmsal.com/isolsearch Heliophysics Events Knowledgebase]]. The HEK is a comprehensive catalog of many items, including Active Regions, CMEs, Coronal Dimmings, Coronal Holes, Coronal Jets, Coronal Waves, Emerging Fluxs, Filaments, Filament Eruptions, Filament Activations, Flares, Loops, Oscillations, Sigmoids, SpraySurges, Sunspots, and Plages ([[http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html complete list here]]). It also contains data, images and movies that have already been assembled for a given event, as well as comments and annotations entered by other users. Prior to using the HEK, it is highly recommended that you view the [[http://lmsal.com/hek/index.html brief HEK tutorial]] to familiarize yourself with all of the features and options. |
. Most of the features or events a user would need have already been assembled in the [[http://lmsal.com/isolsearch Heliophysics Events Knowledgebase]]. The HEK is a comprehensive catalog of many items, including Active Regions, CMEs, Coronal Dimmings, Coronal Holes, Coronal Jets, Coronal Waves, Emerging Fluxs, Filaments, Filament Eruptions, Filament Activations, Flares, Loops, Oscillations, Sigmoids, SpraySurges, Sunspots, and Plages ([[http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html complete list here]]). It also contains data, images and movies that have already been assembled for a given event, as well as comments and annotations entered by other users. Prior to using the HEK, it is highly recommended that you view the [[http://lmsal.com/hek/index.html brief HEK tutorial]] to familiarize yourself with all of the [[ImageLink(http://lmsal.com/hek/images/isolsearch_preview.jpg,http://lmsal.com/hek/index.html,width=300,alt= "features and options", align=left)]]. |
| Line 35: | Line 18: |
The AIA has seven [[http://en.wikipedia.org/wiki/EUV EUV]] channels and three UV-Visible channels. These channels correspond to different characteristic temperatures and different regions of the solar atmosphere, ranging from 5000 degrees to over 10 million degrees. To see what each of the wavelengths look like and view a recent movie, go to helioviewer or the [[http://sdo.gsfc.nasa.gov/data/ SDO Browse Data page]]. However, browsing the [[http://lmsal.com/isolsearch Heliophysics Events Knowledgebase]] is probably the best way to determine which wavelength(s) are most appropriate for your topic of interest. |
. The AIA has seven [[http://en.wikipedia.org/wiki/EUV EUV]] channels and three UV-Visible channels. These channels correspond to different characteristic temperatures and different regions of the solar atmosphere, ranging from 5000 degrees to over 10 million degrees. To see what each of the wavelengths look like and view a recent movie, go to helioviewer or the [[http://sdo.gsfc.nasa.gov/data/ SDO Browse Data page]]. However, browsing the [[http://lmsal.com/isolsearch Heliophysics Events Knowledgebase]] is probably the best way to determine which wavelength(s) are most appropriate for your topic of interest. |
| Line 39: | Line 20: |
| Line 41: | Line 21: |
Science data files are in the [[http://en.wikipedia.org/wiki/FITS FITS]] format. Each uncompressed full-res FITS file is dozens of MB in size, whereas a full-res JPEG on the order of 10 MB or less. Therefore, it is to your advantage if jpegs or movies are sufficient. The tools described above in Section I should be able to provide you with any images or movies you need. |
. Science data files are in the [[http://en.wikipedia.org/wiki/FITS FITS]] format. Each uncompressed full-res FITS file is dozens of MB in size, whereas a full-res JPEG on the order of 10 MB or less. Therefore, it is to your advantage if jpegs or movies are sufficient. The tools described above in Section I should be able to provide you with any images or movies you need. |
| Line 45: | Line 23: |
First and foremost, you must make certain that the data you've identified is manageable in volume. Full-resolution uncompressed FITS files are dozens of MB in size, and the AIA makes approximately one observation per second. A request for all of the 304 Å files for the duration of a typical eruption, for example, can result in a file tens of gigabytes large, which can take upwards of a day to download with a standard DSL or cable internet connection. You may want to use a tool such as the [[http://us.mcafee.com/root/speedometer/default.asp Internet Connection Speedometer]] to determine your download speed before you decide how large a reasonable data request would be. Another consideration is how much data your computer will be able to store and analyze once the data has been downloaded. |
. First and foremost, you must make certain that the data you've identified is manageable in volume. Full-resolution uncompressed FITS files are dozens of MB in size, and the AIA makes approximately one observation per second. A request for all of the 304 Å files for the duration of a typical eruption, for example, can result in a file tens of gigabytes large, which can take upwards of a day to download with a standard DSL or cable internet connection. You may want to use a tool such as the [[http://us.mcafee.com/root/speedometer/default.asp Internet Connection Speedometer]] to determine your download speed before you decide how large a reasonable data request would be. Another consideration is how much data your computer will be able to store and analyze once the data has been downloaded. |
| Line 51: | Line 26: |
More often than not, this is the situation of an AIA data user. There are four options: reduce cadence, reduce the number of wavelengths, reduce resolution, or request only a subfield of the images rather than the full solar disk. For the vast majority of analysis efforts, the full 4096x4096 image is not needed. Subfield data utilities allow the user to zooming in on only the portion of the image they need, thereby drastically reducing the volume of data. Each [[http://lmsal.com/hek/index.html Heliophysics Events Knowledgebase]] event listing links to the [[http://www.lmsal.com/get_aia_data/" Cutout Service]], which allows the user to choose a subfield of data and request only that data. This can drastically reduce the volume of data the user will download. The figure at left shows a window of the cutout service. Users identify the region of the Sun they'd like to examine, shown as a pink outline box in the upper left portion of the figure. For this example, the subfield is less than 10% of the size of the total image making the data volume much more tractable. The Cutout Service also has a "tracking" option that automatically moves the observation boundaries to compensate for solar rotation. This is very useful when the user wants to studying a feature (such as an active region) for an extended period. |
. More often than not, this is the situation of an AIA data user. There are four options: reduce cadence, reduce the number of wavelengths, reduce resolution, or request only a subfield of the images rather than the full solar disk. For the vast majority of analysis efforts, the full 4096x4096 image is not needed. Subfield data utilities allow the user to zooming in on only the portion of the image they need, thereby drastically reducing the volume of data. . Each [[http://lmsal.com/hek/index.html Heliophysics Events Knowledgebase]] event listing links to the [[http://www.lmsal.com/get_aia_data/" Cutout Service]], which allows the user to choose a subfield of data and request only that data. This can drastically reduce the volume of data the user will download. The figure at left shows a window of the cutout service. Users identify the region of the Sun they'd like to examine, shown as a pink outline box in the upper left portion of the figure. For this example, the subfield is less than 10% of the size of the total image making the data volume much more tractable. The Cutout Service also has a "tracking" option that automatically moves the observation boundaries to compensate for solar rotation. This is very useful when the user wants to studying a feature (such as an active region) for an extended period. |
| Line 59: | Line 29: |
The [[http://virtualsolar.org Virtual Solar Observatory's]] search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the [[http://virtualsolar.org VSO website]], (Joe & Joe, which link do you prefer?) but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. The two key SolarSoft functions are vso_search.pro and vso_get.pro. vso_search.pro allows the user to query a vast database of solar data and set requirements regarding timing, data source, data type, resolution, as well as other options. This is an example of a vso_search.pro query: |
. The [[http://virtualsolar.org Virtual Solar Observatory's]] search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the [[http://virtualsolar.org VSO website]], (Joe & Joe, which link do you prefer?) but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. The two key SolarSoft functions are vso_search.pro and vso_get.pro. vso_search.pro allows the user to query a vast database of solar data and set requirements regarding timing, data source, data type, resolution, as well as other options. This is an example of a vso_search.pro query: |
| Line 68: | Line 34: |
| }}} The search query returned records for 57 AIA 304 Å images that are available. (The [[http://jsoc.stanford.edu/ SDO JSOC]], or Joint Science Operations Center, provides AIA and [[http://hmi.stanford.edu/ HMI]] data.) 50 of the records are full resolution (4096x4096 pixels), and 7 of the records are quarter resolution (1024x1024 pixels). To choose only one or the other, the query can be further refined: |
}}} . The search query returned records for 57 AIA 304 Å images that are available. (The [[http://jsoc.stanford.edu/ SDO JSOC]], or Joint Science Operations Center, provides AIA and [[http://hmi.stanford.edu/ HMI]] data.) 50 of the records are full resolution (4096x4096 pixels), and 7 of the records are quarter resolution (1024x1024 pixels). To choose only one or the other, the query can be further refined: |
| Line 78: | Line 41: |
}}} The search returned fewer quarter-resolution images than there are full-res images because the sampling rate of the lower resolution images is approximately 1/6 slower than the nominal AIA image cadence of 12 seconds per image. ''''' Note: ''''' All of the AIA images are available at the full resolution. Quarter-resolution images are created by binning down the 4096x4096 images. The quarter-res images are being made available in order to serve users who would like to work with AIA data but do not require full resolution. If you would like to obtain data at a specific time cadence, you can tell vso_search.pro to filter the query at a lower sampling rate (in this case, 3 minutes): {{{ IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304', pixel=4096, sample=180) |
}}} . The search returned fewer quarter-resolution images than there are full-res images because the sampling rate of the lower resolution images is approximately 1/6 slower than the nominal AIA image cadence of 12 seconds per image. ''''' Note: ''''' All of the AIA images are available at the full resolution. Quarter-resolution images are created by binning down the 4096x4096 images. The quarter-res images are being made available in order to serve users who would like to work with AIA data but do not require full resolution. If you would like to obtain data at a specific time cadence, you can tell vso_search.pro to filter the query at a lower sampling rate (in this case, 3 minutes): {{{ IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304', pixel=4096, sample=180) |
| Line 91: | Line 48: |
| The query returns full-resolution 304 Å image spaced 3 minutes apart, instead of the nominal one image per twelve seconds. There are a great deal more keywords and options available. Please see Appendix II of this document for a more complete (yet still partial) set of options and examples. Once you've identified which files in the query you'd like to obtain, vso_get.pro transfers the data to your local directory: {{{ IDL> getfile = vso_get(searchfile(6),filelist=filelist) |
. The query returns full-resolution 304 Å image spaced 3 minutes apart, instead of the nominal one image per twelve seconds. There are a great deal more keywords and options available. Please see Appendix II of this document for a more complete (yet still partial) set of options and examples. Once you've identified which files in the query you'd like to obtain, vso_get.pro transfers the data to your local directory: {{{ IDL> getfile = vso_get(searchfile[6],filelist=filelist) |
| Line 110: | Line 62: |
The keyword "filelist" provides the local name of the file(s): {{{ IDL> print, filelist(0) |
. The keyword "filelist" provides the local name of the file(s): {{{ IDL> print, filelist[0] |
| Line 117: | Line 67: |
| Line 119: | Line 68: |
| Line 122: | Line 69: |
| Level 1.5 - repointed rotated but not exposure renormalized May be compressed, if so you readfits doesn't work, use read_sdo.pro level 1.0 & 0 definitions, /uncomp_delete |
. Level 1.5 - repointed rotated but not exposure renormalized May be compressed, if so you readfits doesn't work, use read_sdo.pro level 1.0 & 0 definitions, /uncomp_delete |
| Line 131: | Line 72: |
| dem stuff, mapping, pfss overlays color tables |
. dem stuff, mapping, pfss overlays . color tables |
| Line 135: | Line 75: |
| error sources psf instrument information for writing papers | . error sources psf instrument information for writing papers |
| Line 138: | Line 77: |
| Nothing is required, but especially for early papers it would be nice to acknowledge data provider(s) and any AIA team members who helped. |
. Nothing is required, but especially for early papers it would be nice to acknowledge data provider(s) and any AIA team members who helped. |
| Line 141: | Line 79: |
| Line 143: | Line 80: |
| In this guide, we assume that all manipulation of [[http://en.wikipedia.org/wiki/FITS FITS files]] will be done using [[ttp://en.wikipedia.org/wiki/IDL_(programming_language) IDL software]] and [[http://www.lmsal.com/solarsoft/ SolarSoft]] software packages. IDL stands for Interactive Data Language, a data analysis programming language with a syntax descended from Fortran. The software is sold by [[http://www.ittvis.com ITT Visual Information Solutions]]. | . In this guide, we assume that all manipulation of [[http://en.wikipedia.org/wiki/FITS FITS files]] will be done using [[ttp://en.wikipedia.org/wiki/IDL_(programming_language) IDL software]] and [[http://www.lmsal.com/solarsoft/ SolarSoft]] software packages. IDL stands for Interactive Data Language, a data analysis programming language with a syntax descended from Fortran. The software is sold by [[http://www.ittvis.com ITT Visual Information Solutions]]. |
| Line 146: | Line 82: |
| The [[http://www.lmsal.com/solarsoft/ SolarSoft system]] is a set of integrated software libraries, data bases, and system utilities which provide a common programming and data analysis environment for solar physics. The software in the SolarSoft library has been contributed by people and projects throughout the solar physics community. The entire SolarSoft suite is free to users. See the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] for an overview, installation instructions, user guides, tutorials and documentation. |
. The [[http://www.lmsal.com/solarsoft/ SolarSoft system]] is a set of integrated software libraries, data bases, and system utilities which provide a common programming and data analysis environment for solar physics. The software in the SolarSoft library has been contributed by people and projects throughout the solar physics community. The entire SolarSoft suite is free to users. See the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] for an overview, installation instructions, user guides, tutorials and documentation. |
| Line 151: | Line 84: |
On the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] go to "Installing and Configuring Solarsoft" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to install and configure SolarSoft on your system. When installing SolarSoft, there are a variety of software sub-libraries availability. This guide requires the "AIA", "VSO", and "ONTOLOGY" software, so please select those options in addition to any other packages you may need. |
. On the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] go to "Installing and Configuring Solarsoft" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to install and configure SolarSoft on your system. When installing SolarSoft, there are a variety of software sub-libraries availability. This guide requires the "AIA", "VSO", and "ONTOLOGY" software, so please select those options in addition to any other packages you may need. |
| Line 157: | Line 86: |
| From the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] go to "Upgrading a SolarSoft Installation" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to upgrade your SolarSoft library and add new packages. This guide assumes you have the "AIA", "VSO", and "ONTOLOGY" packages installed. |
. From the [[http://www.lmsal.com/solarsoft/ SolarSoft website]] go to "Upgrading a SolarSoft Installation" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to upgrade your SolarSoft library and add new packages. This guide assumes you have the "AIA", "VSO", and "ONTOLOGY" packages installed. |
| Line 163: | Line 92: |
| Line 165: | Line 93: |
AIA data can be accessed in a variety of ways. One point of access is provided by the [[http://virtualsolar.org Virtual Solar Observatory]], or VSO. The VSO's search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the [[http://virtualsolar.org VSO website]], but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. Please note that VSO's IDL interface programs, such as vso_search.pro and vso_get.pro, are updated on a regular basis. The examples in this guide do not reflect the full range of capabilities of the program, and to be sure that you're aware of all that has been changed or added you should check the program documentation directly. Also it is important to note that these examples do not encompass the full range of variables and keywords in the vso interface software. We give simple examples to get you started, but we do not give a comprehensive summary of the VSO Solarsoft capability. We'll proceed with a series of examples that the user can try on their own computer. We'll start by searching the catalog with vso_search.pro, and the query's result can be passed to vso_get.pro to retrieve the data to your local directory. I DL input code is shown next to the IDL> prompt, and the text response to the input (if any) is shown below the command line. We'll start by defining a time range to work with: |
. AIA data can be accessed in a variety of ways. One point of access is provided by the [[http://virtualsolar.org Virtual Solar Observatory]], or VSO. The VSO's search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the [[http://virtualsolar.org VSO website]], but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. Please note that VSO's IDL interface programs, such as vso_search.pro and vso_get.pro, are updated on a regular basis. The examples in this guide do not reflect the full range of capabilities of the program, and to be sure that you're aware of all that has been changed or added you should check the program documentation directly. Also it is important to note that these examples do not encompass the full range of variables and keywords in the vso interface software. We give simple examples to get you started, but we do not give a comprehensive summary of the VSO Solarsoft capability. We'll proceed with a series of examples that the user can try on their own computer. We'll start by searching the catalog with vso_search.pro, and the query's result can be passed to vso_get.pro to retrieve the data to your local directory. I DL input code is shown next to the IDL> prompt, and the text response to the input (if any) is shown below the command line. We'll start by defining a time range to work with: |
| Line 180: | Line 100: |
| Line 188: | Line 106: |
| IDL> print_struct, searchfile(0) DETECTOR INSTRUMENT SOURCE PROVIDER INFO PHYSOBS FILEID SIZE URL GETINFO AIA SDO JSOC AIA level 14096x4096 intensity aia_lev1:304:1058400038 6.620E+04 IDL> print_struct, searchfile(0).tim |
IDL> print_struct, searchfile[0] DETECTOR INSTRUMENT SOURCE PROVIDER INFO PHYSOBS FILEID SIZE URL GETINFO AIA SDO JSOC AIA level 14096x4096 intensity aia_lev1:304:1058400038 6.620E+04 IDL> print_struct, searchfile[0].time |
| Line 195: | Line 113: |
Note: The search returned 2606 images for a time range spanning one hour. Ordinarily, one could enter just a single date into vso_search and get the listing for the full day, but SDO's data volume prohibits this. '''Example 2.''' Restrict the search to only full-resolution 304 Å images: |
. Note: The search returned 2606 images for a time range spanning one hour. Ordinarily, one could enter just a single date into vso_search and get the listing for the full day, but SDO's data volume prohibits this. . '''Example 2.''' Restrict the search to only full-resolution 304 Å images: |
| Line 201: | Line 116: |
| IDL> searchfile = vso_search(time1, time2, instr='aia', wave='304', res=1) | IDL> searchfile = vso_search(time1, time2, instr='aia', wave='304', res=1) |
| Line 207: | Line 122: |
| Set res=1 for full-resolution, res=.5 for half-resolution, res=.25 for quarter-res, etc. | . Set res=1 for full-resolution, res=.5 for half-resolution, res=.25 for quarter-res, etc. |
| Line 211: | Line 124: |
{{{ IDL> searchfile = vso_search(instr='aia', wave='193', pixels = 4096,/latest) |
{{{ IDL> searchfile = vso_search(instr='aia', wave='193', pixels = 4096,/latest) |
| Line 216: | Line 128: |
There are (at least) three ways to tell vso_search.pro to search for lower resolution data. The ''resolution'' keyword can specify the amount of reduction, the ''pixels'' keyword sets the size of the image, and the ''pscale'' keyword searches for images of a particular resolution (given in units of arcseconds), ''e.g.'': |
. There are (at least) three ways to tell vso_search.pro to search for lower resolution data. The ''resolution'' keyword can specify the amount of reduction, the ''pixels'' keyword sets the size of the image, and the ''pscale'' keyword searches for images of a particular resolution (given in units of arcseconds), ''e.g.'': |
| Line 222: | Line 132: |
| pscale=2.4 }}} Note that setting "pixels = 4096" is the same as defining "res=1" because 4096x4096 corresponds to a full-resolution image. However, "res=1" refers to the highest resolution available, so if there is data product that has lower resolution, like 2048x2048, then "res=1" will return the 2048x2048 product, but "pixels=4096" will not. '''Example 4.''' Of the quarter-resolution images available, find the 171 Å image closest to a given time: |
pscale=2.4 }}} . Note that setting "pixels = 4096" is the same as defining "res=1" because 4096x4096 corresponds to a full-resolution image. However, "res=1" refers to the highest resolution available, so if there is data product that has lower resolution, like 2048x2048, then "res=1" will return the 2048x2048 product, but "pixels=4096" will not. . '''Example 4.''' Of the quarter-resolution images available, find the 171 Å image closest to a given time: |
| Line 236: | Line 143: |
| Note: Not all of the AIA full-resolution images are also available as 1024x1024 images. The quarter-res images are currently sampled at a rate of one per 72 seconds, so the time of the nearest available quarter-res image may not be as close as the time of the nearest full-res image. If image timing is important the user may want to omit the resolution specification keyword(s), and get the complete list of images available. If the desired image is not available in the low-res format, the user can download the full-res file and bin it down to 1024x1024. '''Example 5.''' You want to examine a prominence eruption that lasts several hours, so instead of every image you'd like one every 10 minutes or so: |
. Note: Not all of the AIA full-resolution images are also available as 1024x1024 images. The quarter-res images are currently sampled at a rate of one per 72 seconds, so the time of the nearest available quarter-res image may not be as close as the time of the nearest full-res image. If image timing is important the user may want to omit the resolution specification keyword(s), and get the complete list of images available. If the desired image is not available in the low-res format, the user can download the full-res file and bin it down to 1024x1024. . '''Example 5.''' You want to examine a prominence eruption that lasts several hours, so instead of every image you'd like one every 10 minutes or so: |
| Line 245: | Line 150: |
| By setting "sample=600", we instruct vso_search to return one image per 600 seconds. The first "records returned" line corresponds to the full-resolution images, while the second line corresponds to quarter-resolution images. '''Example 6.''' You're making a query on a range of wavelengths instead of a given wavelength. |
. By setting "sample=600", we instruct vso_search to return one image per 600 seconds. The first "records returned" line corresponds to the full-resolution images, while the second line corresponds to quarter-resolution images. . '''Example 6.''' You're making a query on a range of wavelengths instead of a given wavelength. |
| Line 254: | Line 157: |
| This is also useful is you are searching on multiple data sources. AIA has images that are labeled "193 Å," while SOHO EIT and STEREO EUVI's images are labeled "195 Å", so getting wave='193-195' allows you to search on both observatories at the same time. |
. This is also useful is you are searching on multiple data sources. AIA has images that are labeled "193 Å," while SOHO EIT and STEREO EUVI's images are labeled "195 Å", so getting wave='193-195' allows you to search on both observatories at the same time. |
| Line 258: | Line 159: |
query1 = vso_get(searchfile(0:3)) (puts fits files in your local |
. query1 = vso_get(searchfile[0:3]) (puts fits files in your local |
| Line 262: | Line 162: |
| from Joe Hourcle: | . from Joe Hourclé: |
| Line 265: | Line 164: |
Here are a few IDL/SolarSoft tips, and I can write up a few sample |
Here are a few IDL/SolarSoft tips, and I can write up a few sample |
| Line 271: | Line 169: |
If within any program readfits gives the following error: |
If within any program readfits gives the following error: |
| Line 275: | Line 172: |
query1 = vso_get(searchfile(0:3),/rice) (puts fits files in your local |
query1 = vso_get(searchfile[0:3],/rice) (puts fits files in your local |
| Line 281: | Line 176: |
| /uncomp_delete | /uncomp_delete |
| Line 284: | Line 179: |
| Line 287: | Line 181: |
| this is because the FITS file was compressed. Use mreadfits_tilecomp, or | this is because the FITS file was compressed. Use mreadfits_tilecomp, or |
| Line 289: | Line 183: |
| Line 292: | Line 184: |
/rice (or /norice) |
/rice (or /norice) |
| Line 296: | Line 186: |
site=string |
site=string |
| Line 300: | Line 188: |
site='sdac' site='sao' site='mps' site='uclan' |
site='sdac' site='sao' site='mps' site='uclan' |
| Line 307: | Line 193: |
| (I'm also working on something to spread the load out better, even if they | (I'm also working on something to spread the load out better, even if they |
| Line 309: | Line 195: |
}}} |
}}} |
| Line 317: | Line 197: |
{{{ |
. {{{ |
| Line 321: | Line 199: |
| Note: The following lists the documentation for vso_search.pro as of July 20, 2010. | Note: The following lists the documentation for vso_search.pro as of July 20, 2010. |
| Line 323: | Line 201: |
| Line 326: | Line 203: |
| Line 328: | Line 204: |
| Line 330: | Line 205: |
| Line 333: | Line 207: |
| Line 335: | Line 208: |
| Line 337: | Line 209: |
| Line 347: | Line 218: |
| Line 352: | Line 222: |
| Line 377: | Line 245: |
| Line 385: | Line 252: |
| Line 393: | Line 259: |
| Line 400: | Line 265: |
| Line 405: | Line 269: |
| Line 487: | Line 350: |
| Line 498: | Line 355: |
| CategoryHomepage | . CategoryHomepage |
Preamble (please read)
There are a wide range of tools available to access AIA data and images. They were designed to suit users with a wide variety of needs. This page guides a user through the AIA data system based on their needs and interests. First and foremost, a user should understand why a versatile data system is necessary. An AIA full-resolution image has 4096x4096 pixels, and AIA has http://aia.lmsal.com/public/instrument.htm ten different AIA wave bands, or channels. A single full-resolution JPEG can be up to 10 MB in size (a single uncompressed science data file can be dozens of MB) and the AIA performs these observations continuously at a rate of approximately 1 image per second. With previous solar missions, it was not unusual for a user to request a few hours of data. In the case of the AIA, this request would amount to over half a terabyte of data. Even one hour of images at a single wavelength is on the order of 10 GB in size. It is clear that a simple data access system is not suitable for the AIA. We have devised a number of tools so that the user can access specific data sets as quickly as possible. Please note: This guide does not contain the full range of tools and resources available to AIA data users. The user has a great deal of flexibility, and there is no single "correct" way to identify, obtain, and analyze AIA data. It is hoped that this guide will help the user get started with AIA data, and that users will familiarize themselves with the additional resources over time.
Determining which data you want
I need to browse through some data before I can decide what data I'd like to request.
You can browse full-disk movies of the Sun in all of the AIA wavelengths on the http://sdo.gsfc.nasa.gov/data/aiahmi/browse.php SDO Browse Data page. These movies are lower resolution (512x512 or 1024x1024) and are much lower cadence (15 minutes). There are also http://sdo.gsfc.nasa.gov/assets/img/dailymov/ daily 1024x1024 movies for each AIA wavelength.
I'd like a more detailed look instead of just a "browse."
http://helioviewer.org/ Helioviewer is a versatile tool that allows the user to view images and movies from multiple data sources, zoom in on specific regions and view zoomed-in movies, and overlay the images in multiple layers to compare features and structure. Helioviewer also has the capability to track a region, meaning that the viewing box moves with the Sun's rotation. The result can be saved as a link and accessed at a later date. A full-resolution AIA image is 4096x4096 pixels, and standard displays are not able to show anything near that size. Helioviewer allows the user to choose between viewing the whole Sun in (much) lower resolution, or viewing a subfield at better or even full resolution. A desktop version, called http://www.jhelioviewer.org/ JHelioviewer, can be downloaded onto your computer. The user is able to create movies and export them in MPEG or Quicktime format. ImageLink(helioviewer_snapshot.jpg,http://helioviewer.org,align=right,width=150)
I'm looking for a specific feature, event or phenomenon but I haven't determined the date/times of the images I need.
Most of the features or events a user would need have already been assembled in the http://lmsal.com/isolsearch Heliophysics Events Knowledgebase. The HEK is a comprehensive catalog of many items, including Active Regions, CMEs, Coronal Dimmings, Coronal Holes, Coronal Jets, Coronal Waves, Emerging Fluxs, Filaments, Filament Eruptions, Filament Activations, Flares, Loops, Oscillations, Sigmoids, SpraySurges, Sunspots, and Plages (http://www.lmsal.com/helio-informatics/hpkb/VOEvent_Spec.html complete list here). It also contains data, images and movies that have already been assembled for a given event, as well as comments and annotations entered by other users. Prior to using the HEK, it is highly recommended that you view the http://lmsal.com/hek/index.html brief HEK tutorial to familiarize yourself with all of the ImageLink(http://lmsal.com/hek/images/isolsearch_preview.jpg,http://lmsal.com/hek/index.html,width=300,alt= "features and options", align=left).
I know the start time and end time of the event or phenomenon I'd like to study, but I don't know which wavelength(s).
The AIA has seven http://en.wikipedia.org/wiki/EUV EUV channels and three UV-Visible channels. These channels correspond to different characteristic temperatures and different regions of the solar atmosphere, ranging from 5000 degrees to over 10 million degrees. To see what each of the wavelengths look like and view a recent movie, go to helioviewer or the http://sdo.gsfc.nasa.gov/data/ SDO Browse Data page. However, browsing the http://lmsal.com/isolsearch Heliophysics Events Knowledgebase is probably the best way to determine which wavelength(s) are most appropriate for your topic of interest.
Accessing the data
I want images and/or a movie, and I don't need the full science data files.
Science data files are in the http://en.wikipedia.org/wiki/FITS FITS format. Each uncompressed full-res FITS file is dozens of MB in size, whereas a full-res JPEG on the order of 10 MB or less. Therefore, it is to your advantage if jpegs or movies are sufficient. The tools described above in Section I should be able to provide you with any images or movies you need.
I need FITS data files, and I think I know the time range and wavelengths of the images I want.
- First and foremost, you must make certain that the data you've identified is manageable in volume. Full-resolution uncompressed FITS files are dozens of MB in size, and the AIA makes approximately one observation per second. A request for all of the 304 Å files for the duration of a typical eruption, for example, can result in a file tens of gigabytes large, which can take upwards of a day to download with a standard DSL or cable internet connection.
You may want to use a tool such as the http://us.mcafee.com/root/speedometer/default.asp Internet Connection Speedometer to determine your download speed before you decide how large a reasonable data request would be. Another consideration is how much data your computer will be able to store and analyze once the data has been downloaded.
I need FITS data files, I know the time range and wavelengths of the images I want, but the available data volume is overwhelming. Case 1: Using subfields/cutouts
- More often than not, this is the situation of an AIA data user. There are four options: reduce cadence, reduce the number of wavelengths, reduce resolution, or request only a subfield of the images rather than the full solar disk. For the vast majority of analysis efforts, the full 4096x4096 image is not needed. Subfield data utilities allow the user to zooming in on only the portion of the image they need, thereby drastically reducing the volume of data.
Each http://lmsal.com/hek/index.html Heliophysics Events Knowledgebase event listing links to the http://www.lmsal.com/get_aia_data/" Cutout Service, which allows the user to choose a subfield of data and request only that data. This can drastically reduce the volume of data the user will download. The figure at left shows a window of the cutout service. Users identify the region of the Sun they'd like to examine, shown as a pink outline box in the upper left portion of the figure. For this example, the subfield is less than 10% of the size of the total image making the data volume much more tractable. The Cutout Service also has a "tracking" option that automatically moves the observation boundaries to compensate for solar rotation. This is very useful when the user wants to studying a feature (such as an active region) for an extended period.
I need FITS data files, I know the time range and wavelengths of the images I want, but the available volume is overwhelming. Case 2: Decreasing the temporal or spatial resolution
The http://virtualsolar.org Virtual Solar Observatory's search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the http://virtualsolar.org VSO website, (Joe & Joe, which link do you prefer?) but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. The two key SolarSoft functions are vso_search.pro and vso_get.pro. vso_search.pro allows the user to query a vast database of solar data and set requirements regarding timing, data source, data type, resolution, as well as other options. This is an example of a vso_search.pro query:
IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304')
Records Returned : JSOC : 50/50
Records Returned : JSOC : 7/7The search query returned records for 57 AIA 304 Å images that are available. (The http://jsoc.stanford.edu/ SDO JSOC, or Joint Science Operations Center, provides AIA and http://hmi.stanford.edu/ HMI data.) 50 of the records are full resolution (4096x4096 pixels), and 7 of the records are quarter resolution (1024x1024 pixels). To choose only one or the other, the query can be further refined:
IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304', pixel=4096)
Records Returned : JSOC : 50/50
IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304', pixel=1024)
Records Returned : JSOC : 7/7- The search returned fewer quarter-resolution images than there are full-res images because the sampling rate of the lower resolution images is approximately 1/6 slower than the nominal AIA image cadence of 12 seconds per image.
Note: All of the AIA images are available at the full resolution. Quarter-resolution images are created by binning down the 4096x4096 images. The quarter-res images are being made available in order to serve users who would like to work with AIA data but do not require full resolution. If you would like to obtain data at a specific time cadence, you can tell vso_search.pro to filter the query at a lower sampling rate (in this case, 3 minutes):
IDL> searchfile = vso_search('2010-jul-20 14:00','2010-jul-20 14:10', instr='aia', wave='304', pixel=4096, sample=180)
Records Returned : JSOC : 4/4- The query returns full-resolution 304 Å image spaced 3 minutes apart, instead of the nominal one image per twelve seconds. There are a great deal more keywords and options available. Please see Appendix II of this document for a more complete (yet still partial) set of options and examples. Once you've identified which files in the query you'd like to obtain, vso_get.pro transfers the data to your local directory:
IDL> getfile = vso_get(searchfile[6],filelist=filelist) VSO_GET: This will download 1 file(s) 1 : http://vso.tuc.noao.edu/cgi-bin/drms_test/drms_export.cgi?series=aia_synoptic2;record=304_11763447-11763447 % RDWRT_BUFF: Please wait. Downloading... % File: /cgi-bin/drms_test/drms_export.cgi?series=aia_synoptic2;record=304_11763447-11763447 % Size: 4210560 bytes % From: vso.tuc.noao.edu % To: /Users/barbara/data % HTTP::COPY: 4210560 bytes of 4210560 total bytes copied in 2.73 seconds % HTTP::COPY: Wrote 4210560 bytes to file /Users/barbara/data/aia_test.synoptic2.304A_2010-07-20T14_10_02.12Z.image_lev1p5.fits Downloading completed
- The keyword "filelist" provides the local name of the file(s):
IDL> print, filelist[0] /Users/barbara/data/aia_test.synoptic2.304A_2010-07-20T14_10_02.12Z.image_lev1p5.fits
Analyzing the data
Well, well, well. So what have we here?
- Level 1.5 - repointed rotated but not exposure renormalized May be compressed, if so you readfits doesn't work, use read_sdo.pro
level 1.0 & 0 definitions, /uncomp_delete
What analysis tools are available?
- dem stuff, mapping, pfss overlays
- color tables
Section III.c: What are the limitations of the data?
- error sources psf instrument information for writing papers
Section III.d: I'm ready to publish. Who do I acknowledge?
- Nothing is required, but especially for early papers it would be nice to acknowledge data provider(s) and any AIA team members who helped.
Appendix I: IDL and the SolarSoft software tree
Appendix I.a: What is IDL?
In this guide, we assume that all manipulation of http://en.wikipedia.org/wiki/FITS FITS files will be done using ttp://en.wikipedia.org/wiki/IDL_(programming_language) IDL software and http://www.lmsal.com/solarsoft/ SolarSoft software packages. IDL stands for Interactive Data Language, a data analysis programming language with a syntax descended from Fortran. The software is sold by http://www.ittvis.com ITT Visual Information Solutions.
Appendix I.b: What is SolarSoft?
The http://www.lmsal.com/solarsoft/ SolarSoft system is a set of integrated software libraries, data bases, and system utilities which provide a common programming and data analysis environment for solar physics. The software in the SolarSoft library has been contributed by people and projects throughout the solar physics community. The entire SolarSoft suite is free to users. See the http://www.lmsal.com/solarsoft/ SolarSoft website for an overview, installation instructions, user guides, tutorials and documentation.
Appendix I.c: I have IDL, but I don't have the SolarSoft software tree on my computer.
On the http://www.lmsal.com/solarsoft/ SolarSoft website go to "Installing and Configuring Solarsoft" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to install and configure SolarSoft on your system. When installing SolarSoft, there are a variety of software sub-libraries availability. This guide requires the "AIA", "VSO", and "ONTOLOGY" software, so please select those options in addition to any other packages you may need.
Appendix I.d: I have the SolarSoft software tree on my computer, but I'm not sure if I have the appropriate software packages installed.
From the http://www.lmsal.com/solarsoft/ SolarSoft website go to "Upgrading a SolarSoft Installation" in the menu. If the menu doesn't appear, click the "Open Menu Window" button at the top. The guide will take you through the steps to upgrade your
SolarSoft library and add new packages. This guide assumes you have the "AIA", "VSO", and "ONTOLOGY" packages installed.
Important Note: It is very important to do a regular update of the Solarsoft tree. Modifications to the tree's software are happening continually, and unless you do an update you're not going to have the most recent versions. If possible, set your computer to automatically do updates on a regular (e.g. weekly) basis.
Appendix II: Identifying and retrieving data using the Virtual Solar Observatory's IDL interface
Appendix II.a: Searching the VSO catalog using vso_search.pro, with several examples
AIA data can be accessed in a variety of ways. One point of access is provided by the http://virtualsolar.org Virtual Solar Observatory, or VSO. The VSO's search tools provide the user with a great deal of flexibility in accessing solar data. There is a search interface on the http://virtualsolar.org VSO website, but there is also an IDL interface provided within the SolarSoft library that allows the user to integrate their data search, retrieval, and analysis efforts. Please note that VSO's IDL interface programs, such as vso_search.pro and vso_get.pro, are updated on a regular basis. The examples in this guide do not reflect the full range of capabilities of the program, and to be sure that you're aware of all that has been changed or added you should check the program documentation directly. Also it is important to note that these examples do not encompass the full range of variables and keywords in the vso interface software. We give simple examples to get you started, but we do not give a comprehensive summary of the VSO Solarsoft capability. We'll proceed with a series of examples that the user can try on their own computer. We'll start by searching the catalog with vso_search.pro, and the query's result can be passed to vso_get.pro to retrieve the data to your local directory.
I DL input code is shown next to the IDL> prompt, and the text response to the input (if any) is shown below the command line. We'll start by defining a time range to work with:
IDL> time1 = '2010-jul-17 00:00' IDL> time2 = '2010-jul-17 01:00'
Example 1. Find all of the AIA images available for a given date range:
IDL> searchfile = vso_search(time1, time2, instr='aia')
Records Returned : JSOC : 2306/2306
Records Returned : JSOC : 300/300
IDL> print_struct, searchfile[0]
DETECTOR INSTRUMENT SOURCE PROVIDER INFO PHYSOBS FILEID SIZE URL GETINFO
AIA SDO JSOC AIA level 14096x4096 intensity aia_lev1:304:1058400038 6.620E+04
IDL> print_struct, searchfile[0].time
START _END
2010-07-17T00:00:02 2010-07-17T00:00:03- Note: The search returned 2606 images for a time range spanning one hour. Ordinarily, one could enter just a single date into vso_search and get the listing for the full day, but SDO's data volume prohibits this.
Example 2. Restrict the search to only full-resolution 304 Å images:
IDL> searchfile = vso_search(time1, time2, instr='aia', wave='304', res=1) Records Returned : JSOC : 286/286 IDL> print_struct, searchfile(0).time START _END 2010-07-17T00:00:02 2010-07-17T00:00:03- Set res=1 for full-resolution, res=.5 for half-resolution, res=.25 for quarter-res, etc.
Example 3. Find the most recent full-resolution 193 Å image:
IDL> searchfile = vso_search(instr='aia', wave='193', pixels = 4096,/latest) Records Returned : JSOC : 1/1
There are (at least) three ways to tell vso_search.pro to search for lower resolution data. The resolution keyword can specify the amount of reduction, the pixels keyword sets the size of the image, and the pscale keyword searches for images of a particular resolution (given in units of arcseconds), e.g.:
resolution=0.25 pixels=1024 pscale=2.4
- Note that setting "pixels = 4096" is the same as defining "res=1" because 4096x4096 corresponds to a full-resolution image. However, "res=1" refers to the highest resolution available, so if there is data product that has lower resolution, like 2048x2048, then "res=1" will return the 2048x2048 product, but "pixels=4096" will not.
Example 4. Of the quarter-resolution images available, find the 171 Å image closest to a given time:
IDL> searchfile = vso_search(near=time1, instr='aia', pixel=1024, wave='171')
Records Returned : JSOC : 1/1
IDL> print_struct, searchfile.time
START _END
2010-07-16T23:59:59 2010-07-16T24:00:00- Note: Not all of the AIA full-resolution images are also available as 1024x1024 images. The quarter-res images are currently sampled at a rate of one per 72 seconds, so the time of the nearest available quarter-res image may not be as close as the time of the nearest full-res image. If image timing is important the user may want to omit the resolution specification keyword(s), and get the complete list of images available. If the desired image is not available in the low-res format, the user can download the full-res file and bin it down to 1024x1024.
Example 5. You want to examine a prominence eruption that lasts several hours, so instead of every image you'd like one every 10 minutes or so:
IDL> searchfile = vso_search('2010-jul-16 03:00','2010-jul-16 07:00', instr='aia', wave='193', sample=600)
Records Returned : JSOC : 24/24
Records Returned : JSOC : 24/24- By setting "sample=600", we instruct vso_search to return one image per 600 seconds. The first "records returned" line corresponds to the full-resolution images, while the second line corresponds to quarter-resolution images.
Example 6. You're making a query on a range of wavelengths instead of a given wavelength.
IDL> searchfile = vso_search(near=time1, instr='aia', wave='171 - 193', sample=600) Records Returned : JSOC : 24/24 Records Returned : JSOC : 24/24
- This is also useful is you are searching on multiple data sources. AIA has images that are labeled "193 Å," while SOHO EIT and STEREO EUVI's images are labeled "195 Å", so getting wave='193-195' allows you to search on both observatories at the same time.
Appendix II.b: Retrieving data with vso_get.pro
- query1 = vso_get(searchfile[0:3]) (puts fits files in your local
directory)
- from Joe Hourclé:
Here are a few IDL/SolarSoft tips, and I can write up a few sample commands next week: mreadfits_shm.pro reads in uncompressed FITS files. mreadfits_tilecomp.pro reads compressed FITS files. SET uncomp = 0 or you'll be sorry - saves all uncompressed files in /tmp and fills hard drive read_sdo.pro takes any FITS file and sends it to _tilecomp or _shm If within any program readfits gives the following error: % READFITS: FITS header has NAXIS or NAXISi = 0, no data array read this is because the FITS file was compressed. Use mreadfits_tilecomp, or read_sdo instead. query1 = vso_get(searchfile[0:3],/rice) (puts fits files in your local directory) Here are a few IDL/SolarSoft tips, and I can write up a few sample commands next week: mreadfits_shm.pro reads in uncompressed FITS files. /uncomp_delete mreadfits_tilecomp.pro reads compressed FITS files. read_sdo.pro takes any FITS file and sends it to _tilecomp or _shm If within any program readfits gives the following error % READFITS: FITS header has NAXIS or NAXISi = 0, no data array read this is because the FITS file was compressed. Use mreadfits_tilecomp, or read_sdo instead. for vso_get: /rice (or /norice) We also hope to soon have: site=string so you can do: site='sdac' site='sao' site='mps' site='uclan' etc, rather than getting everything from NSO. (I'm also working on something to spread the load out better, even if they don't specify a site).
Appendix III: vso_search.pro documentation
-------------------------------------------------------------------- Note: The following lists the documentation for vso_search.pro as of July 20, 2010. Up-to-date documentation can be accessed using programs like doc_menu.pro or xdoc.pro ----- Documentation for /usr/local/ssw/gen/idl/interfaces/vso/vso_search.pro ----- Project : VSO Name : VSO_SEARCH() Purpose : Send a search request to VSO Explanation : Sends a query to VSO, to obtain metadata records corresponding to the records that match the query parameters. Category : Utility, Class2, VSO Syntax : IDL> a = vso_search('2005-JAN-01', instrument='eit') Examples : IDL> a = vso_search(date='2004-01-01', provider='sdac') IDL> a = vso_search(date='2002-1-4 - 2002-1-4T07:05', inst='mdi') IDL> a = vso_search(date='2004/1/4T07:40-2004/1/4T07:45', inst='trace') IDL> a = vso_search(date='2004-1-1', extent='CORONA', /FLAT) IDL> a = vso_search(date='2001-1-1', physobs='los_magnetic_field') IDL> a = vso_search(date='2004/1/1', inst='eit', /DEBUG) IDL> a = vso_search('2004/1/1','2004/12/31', wave='171 Angstrom', inst='eit') IDL> a = vso_search('2004/6/1','2004/6/15', wave='284-305 Angstrom', inst='eit') IDL> a = vso_search('2005-JAN-1', inst='eit', /FLAT, /URL) IDL> print_struct, a IDL> print_struct, a.time ; if not called w/ /FLATTEN IDL> sock_copy, a.url ; if called w/ /URLS IDL> vso_get, a ; attempt to download products Input: (note -- you must either specify DATE, START_DATE or TSTART) Optional Input: (positional) TSTART : date ; the start date TEND : date ; the end date Keywords: DATE : string ; (start date) - (end date) START_DATE : date ; the start date END_DATE : date ; the end date WAVE : string ; (min) - (max) (unit) MIN_WAVE : string ; minimum spectral range MAX_WAVE ; string ; maximum spectral range UNIT_WAVE ; string ; spectral range units (Angstrom, GHz, keV) EXTENT ; string ; VSO 'extent type' ... (FULLDISK, CORONA, LIMB, etc) PHYSOBS ; string ; VSO 'physical observable' PROVIDER ; string ; VSO ID for the data provider (SDAC, NSO, SHA, MSU, etc) SOURCE ; string ; spacecraft or observatory (SOHO, YOHKOH, BBSO, etc) synonyms : SPACECRAFT, OBSERVATORY INSTRUMENT ; string ; instrument ID (EIT, SXI-0, SXT, etc) synonym : TELESCOPE DETECTOR ; string ; detector ID (C3, EUVI, COR2, etc.) LAYOUT ; string ; layout of the data (image, spectrum, time_series, etc.) Keywords with limited support; (may not work with all archives) LEVEL ; range* ; level of the data product (see below) PIXELS ; range* ; number of pixels (see below) RESOLUTION ; range* ; effective resolution (1 = full, 0.5 = 2x2 binned, etc) PSCALE ; range* ; pixel scale, in arcseconds NEAR_TIME ; date ; return record closest to the time. See below. SAMPLE ; number ; attempt to return only one record per SAMPLE seconds. See below. (flag keywords) QUICKLOOK ; boolean ; retrieve 'quicklook' data if available (see below) LATEST ; boolean ; sets ( near=now, end=now, start=(now - 7 days) ) (see below) URLS ; boolean ; attempt to get URLs, also QUIET ; boolean ; don't print informational messages DEBUG ; boolean ; print xml soap messages FLATTEN ; boolean ; return vsoFlat Record (no sub-structures) (placeholders for the future) FILTER ; string ; filter name (same problems as detector) WAVETYPE ; string ; type of spectral range (LINE, NARROW, BROAD) (WARNING : causing errors to be thrown when using 'wave') CARTID ; string ; load a VSO Cart by its identifier EVENT ; string ; search for data using an event from HEK Outputs: a null pointer -> no matches were found (or) struct[n] : (vsoRecord or vsoFlatRecord) the metadata from the results ; ; VSO Searching: ; * By default, if a VSO data provider (ie, archive) does not support ; a given search parameter, it will default to returning everything ; that *might* match. This will occassionally result in many more ; fields being returned than expected. ; * Some data providers will limit the number of results that they ; return, and you'll see a message such as: ; Records Returned : SDAC : 1000/31772 ; You will need to break the query into smaller time ranges to get all ; of the records. Some data providers have implemented a 'summary rows' ; feature where excessive returns will cause the archive to return records ; corresponding to more than one record: ; IDL> sot = vso_search( '2006-01-01', '2007-01-01', inst='sot' ) ; Records Returned : SDAC_SOT : 488118/488118 ; IDL> help, sot ; SOT STRUCT = -> VSORECORD Array[5321] ; Examine the 'info' field for a clue as to how many observations were ; rolled up into each record. We hope to provide improved support ; for expanding these 'summary' records into individual observations ; in the future. ; * If you have a query that's not returning what you're expecting, ; please contact : joseph.a.hourcle AT nasa.gov ; (and make sure to send the query!) ; ; Source / Instrument / Detector / Phys. Obs. / Layout / Extent Tye / etc: ; * These items are enumerations. For a list of valid values, see: ; http://sdac.virtualsolar.org/cgi/show_details ; * We hope to soon have a 'vso_info' program to get the list from within IDL ; ; Dates within VSO_SEARCH: ; * All dates are in UTC. ; * Dates may be entered in any format accepted by 'anytim2utc', EXCEPT for the ; 'DATE' keyword, which expects a string containing a range ; (eg, '2004/01/01 - 2004/02/01') ; * The 'NEAR_TIME' keyword is not yet supported by all data providers. As a ; precaution, using NEAR without specifying a start or end time will use a ; window of +/- 1hr to prevent extremely large data returns from data providers ; that do not support this keyword, but it may cause the data provider to return ; no data if there is no data in that 2 hr window. You can override the window. ; * The /LATEST flag will set the following, where 'NOW' is the current time ; START_DATE=(NOW - 7 days), END_DATE=NOW, NEAR_TIME=NOW ; It will not override existing start dates, so if the data you're interested in ; lags by more than a week, you can set START_DATE further into the past, and ; it will still set END_DATE and NEAR_TIME for you. ; * Order of precidence for start and end dates: ; * DATE keyword (a string containing 2 dates) ; * TSTART/TEND (positional parameters) ; * START_DATE/END_DATE keywords ; * If no end date is specified, it will use the start of the next day ; ; Numeric Ranges : (marked as 'range*') ; * May be entered as a string or any numeric type for equality matching ; * May be a string of the format '(min) - (max)' for range matching ; * May be a string of the form '(operator) (number)' where operator is one of: ; lt gt le ge < > <= >= ; ; Level : ; * Whatever numeric value the PI assigns. ; * If the PI's designation is '1.5q', search for : LEVEL=1.5,/QUICKLOOK ; ; Quicklook : ; * Quicklook items are assumed to be generated with a focus on speed rather ; than scientific accuracy. They are useful for instrument planning and ; space weather but should not be used for science publication. ; * This concept is sometimes called 'browse' or 'near real time' (nrt) ; * Quicklook products are *not* searched by default ; ; Resolution / Pixels / Pixel Scale : (currently SDO/AIA and HMI only) ; * The "resolution" is a function of the highest level of data available. ; If the CCD is 2048x2048, but it's binned to 512x512 before downlink, ; the 512x512 product is designated as '1'. If a 2048x2048 and 512x512 ; product are both available, the 512x512 product is designated '0.25'. ; * Pixels are (currently) limited to a single dimension. (and only implemented ; for SDO data) We hope to change this in the future to support TRACE, ; Hinode and other investigations where this changed between observations ; * Pixel Scale (PSCALE) is in arc seconds. It's currently only implemented ; for SDO, which is 0.6 arcsec per pixel at full resolution for AIA. ; ;-
Email: MailTo(you AT SPAMFREE example DOT com)
...