|
Size: 42168
Comment:
|
Size: 42187
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 48: | Line 48: |
| [All examples assume that the name spaces su_bugsbunny, su_elmer and warner exist.][[BR]] | [All examples assume that the name spaces su_bugsbunny, su_elmer and warner exist, which is not true.][[BR]] |
JSOC Series Definition Files
NEWS!
There have been a number of bug fixes regarding "show_info -j" and "create_series" since the last JSOC release (version 4.7). If those commands have been troubling you, please try upgrading to the head CVS file revisions.
Introduction
The preferred way to create and manage JSOC data series is via a ".jsd" file. These files are read by the "create_series" program to make the required database tables.
At present the create_series program provides only terse messages when it can not parse a .jsd file so the format here should be followed closely. In the format rules here items in angle brackets "<>" are words, strings, or numbers to be provided by the series creator. Words are user provided single word parameters, then can not contain blanks. They may be given with or without quotation marks. Strings may also be given with or without quotation marks but must be enclosed in quote marks if they contain blanks. A string must contain at least one character so an empty comment, for ecample, must be at least one blank character. In order to differentiate between words and strings, strings begin with "str". Choices in curly brackets and delimited by vertical bars are exact words that must be supplied. Choices are shown in bold (where I can figure out how). Any line beginning with a "#" is a comment and is not examined by create_series. The tokens ( words, strings, numbers, must be comma "," separated.
Names for series, keywords, links, and segments are implemented as PostgreSQL column or table names and are not allowed to collide with Postgres reserved names.
The .jsd file consists of four sections containing definitions of global information, keywords, links, and data segments. Only the global information section is required although a series with only global information has limited use.
Global series information
This section contains global information about the JSOC data series, that applies to all records and storage units belonging to the series. It is the information kept in the "drms_series" table. The .jsd syntax takes the form:
Seriesname: <str series name> Description: <str description> Author: <str author name> Unitsize: <storage unit size> Retention: <default retention time> Archive: <archive flag> Tapegroup: <tape group> PrimeKeys: <primary keyword list> DBIndex: <index keyword list> Version: <version number>
where:
<str series name> is a string containing the name of the JSOC dataseries defined in this file. The series name has to be unique, i.e. there can be no two data series in the JSOC database with the same name. The series name will be used as a database table name so it is constrained to letters, digits, and underscores. The first character must be a letter. Case is preserved but ignored. The seriesname must include both the project and data name parts with the "." separator. See also PostgreSqlReservedNames.
Some good series names include: BR
Some series names that will not work:BR
su_bugsbunny.4myself [Starts with a number, 4]BR warner.carrot.r*bbit [More than one dot (.) and contains an asterisk (*)]BR su_elmer.sat-sun-activity [Uses a hyphen]BR
[All examples assume that the name spaces su_bugsbunny, su_elmer and warner exist, which is not true.]BR
<str description> is a string containing a description of the data series.
<str author name> is a string containing the name of the author who created the data series.
<storage unit size> is a non-zero integer indicating the number of data records that go into forming a storage unit for this series. Unless the data segments are to be stored in the Tiled Array Storage or equivalent FITS based format there is no requirement to include multiple records in a single storage unit. However there may be an efficiency aspect since records created at the same time may be used at the same time and if needed records that share a storage unit will be retrieved from tape as a whole reducing retrieval latency for nearby records. There is no disadvantage to making unitsize larger than 1.
<default retention time> is an integer denoting the default online data storage unit retention time in days, i.e. the time for which the files holding the data of the data segments will be kept online after ingestion into the DRMS. The default retention time can be overridden on a per storage unit basis. A short retention time with archive flag of 0 will result in short-lived data segments. Note that the retention time applies both to newly ingested data and to data segments restored from permanent media. Some suggested retention times may be found in DrmsSeriesNames.
<archive flag> is an integer either 0 or 1, describing whether the data segments associated with the records of this series are to be archived on permanent media. A value of 1 means that they are to be archived. If the archive flag is 0 then the <tape group> field is ignored.
<tape group> is an integer identifying the tape group to which this data series belongs. This is used to group together data storage units belonging to related data series when storing them on tape. The tape group convention is described in DrmsSeriesNames.
<primary keyword list> A list of zero or more keyword names constituting the primary index of the series. If a primary keyword list is present, those keywords should provide a complete identifier of each record. Multiple records with the same set of primary keyword values will be considered to be different versions of the same record and only the record added last will be visible. Keywords that are redundant, i.e. do not serve to differentiate records, should not be included as primary keywords. These keywords must be declared in the keyword section of the JSD for it to be valid. If no primary keywords are given, the record number is implicitly used as primary index. NOTE - For jsds which do not contain a "PrimeKeys" entry, but have an "Index" entry, the database index will consist of all keywords specified in the "Index" entry. jsds which contain a "PrimeKeys" entry, but no "DBIndex" entry will lead to the creation of a series that has no database indices. The "Index" entry was widely used before March 2008. Note 2-caps in "PrimeKeys".
<index keyword list> A list of zero or more keyword names for which data base indexes will be constructed. Indexed keywords allow much faster data retrieval. Generally keywords used for prime keys should also be index keywords. Note 3-caps in "DBIndex".
<version number> is the version number of the .jsd file.
Keywords
The keyword section contains declarations of all keywords present in records belonging to the series. Each keyword description consist of "Keyword:" followed by a comma-separated list of attributes describing the keyword. Space and tab characters are ignored. The attributes list must be contained in a single line. [To remove this restriction, must modify the function getnextline() in src/base/libdrms/drms_parser.c] The lists of keywords allowed for each series are maintained in the drms_keyword table. A keyword can be simple or a link. A simple keyword can be fixed or variable and will contain an explicit value for the entire series (fixed) or for each record (variable). A link keyword describes a pointer to a keyword in some other series.
Simple Keywords: If the keyword is a simple value this takes the following form:
Keyword: <name>, <datatype>, <recordscope>, <segmentscope>, <default value>, <format>, <unit>, <str description>
where:
<name> is a word specifying the name of the keyword. Keyword names cannot exceed 24 characters in length. They must begin with an alphabetic character and they may contain only alphanumeric characters or the underscore "_". Although keyword names are case-insensitive in SQL queries, they are case-sensitive in the DRMS API; therefore they should be considered case-sensitive, but with the caveat that a series cannot contain multiple keyword names that would not strcasecmp. See also PostgreSqlReservedNames.
<datatype> specifies the data type of the keyword value. The following types are recognized: char, short, int, longlong, float, double, time, and string.
<recordscope> specifies the variability of the keyword. Keytypes are constant, variable or slotted.
constant means that the keyword has a fixed value set by the default value.
variable means that the keyword can take on a different value for each record (or segment). If not specified it will contain the default value.
Slotted recordscopes include index, ts_eq, slot, enum, and carr. Slotted keywords used as Primekeys provide a clean mapping between floating point or other non-definite value sets to well defined 'slots' in a virtual axis. Slotted keywords involve several associated keywords whose names are formed by adding suffixes to a base name. An 'index' suffix keyword must exist if one of the other slotted indicators is present to define a base keyword. The 'index' keyword will always be an integer type and will be created automatically if not specified. When a 'base' slotted keyword is specified as a Primekey, the associated 'index' key will be the actual indexed keyword. Additional associated keywords needed depend on the slotted type. See [http://hmi.stanford.edu/doc/JSOC/DRMS_dataset_names.pdf DRMS Dataset Names] for a more detailed discussion.
<segmentscope> specifies the association of the keyword as per record or per segment within the record. Values for 'scope' are:
record means that there is an instance of the keyword for each record in the series.
segment means that there is a value for the keyword for each segment for each record. The actual keyword names will have a three digit segment number appended to the given name. Thus to reference say, "bzero" for the third segment of each record in a series one would use the name "bzero002". (what happens if the .jsd has explicit entries for each segment using this convention??) The way to access the keyword value its to treat it as a name array, i.e. "bzero" for the third segment would be "bzero![2]".
<default value> is the default value to be assigned to this keyword. A value must be supplied in the appropriate format. Sample default values are 0 for fixed-point numerical types, 0.0 or NaN for floating-point types, a one blank character string for string types and 1970.01.01 for the time type. In order to specify a null string for values of type char or string it is necessary to use numeric 0; "" will not work. [I'm not sure how these will be handled in the database.] A constant keyword has the default value for all records. When a new record is first created its keywords are populated with the default values. The default value specification can be up to 4096 characters long. [Does this imply that a string keyword value is restricted to 4096 chars?]
<format> is a format word as used by the printf family of functions for formatting an object of type <datatype>. The length of the format string is restricted to 24 characters. It must be quoted in either single or double quotes if it contains embedded white space. Times are stored internally as doubles but may be shown as familiar yyyy.mm.dd strings. Note: For time keywords, the format field should resolve to an integer, and specifies the precision of the seconds field in the printed time string. It will be used as the precision parameter to the sprint_time() function (see http://jsoc.stanford.edu/doxygen_html/timeio_8h.html#a8a2f29376aea692304cc9f323561a98). Also, see [wiki:JsocTimes JSOC Time Format Notes].
<unit> is a word specifying the physical unit of the keyword value. If no "unit" is to be specified, the word none should be used. The length of the unit string is restricted to 24 characters. It must be quoted if it contains embedded white space. Note: For time keywords, the unit field is a time-zone string, such as "PDT". It will be used as the zone parameter to the sprint_time() function.
<str description> is a string describing the meaning of the keyword. It can be up to 4096 characters in length, and must be quoted if it contains embedded white space.
For example, to declare a keyword containing a floating point value describing velocity associated with a data record, the keyword section could contain a line like
Keyword: v, float, variable, record, 0.0f, %12g, m/s, "velocity" - or - Keyword: "v", float, variable, record, "0.0f", "%12g", "m/s", "velocity" - or - Keyword: Velocity,float,variable,record,NaN,"%12g m/s","meters per second" ,"velocity [m/s]"
An example Time type keyword might be like
Keyword: t_obs, time, variable, record, 1993.01.01, TAI, "TAI", "Time of observation"
An example slotted time type keyword and associated keywords might be like
Keyword: t_rec, time, ts_eq, record, 1993.01.01, TAI, "TAI", "Time index of observation" Keyword: t_rec_index, int, index, record, 0, %d, " ", "Time index" Keyword: t_rec_epoch, time, constant, record, 1993.01.01, TAI, "TAI", "Reference epoch for time index" Keyword: t_rec_step, time, constant, record, 60.0, %lf, "s", "Index increment time span"
Link Keywords A keyword can also be a link pointing to a keyword in a record in a different series, making it possible for data records to inherit keyword values from each other.
A link keyword has <datatype> equal to link and takes the form the form:
Keyword: <name>, link, <linkname>, <target keyword>, <str description>
- where:
<name> is a word specifying the name of the keyword.
<linkname> is a word that specifies the name of the link, which must be defined elsewhere in the same JSD. The link specification (see below) will identify a record in another series from which to take the value.
<target keyword> is a word that specifies the name of the target keyword from which to get the value for this keyword from the record pointed to by the link.
<str description> is a string describing the meaning of the keyword.
Keyword: Tilt, link, HMI_ORBIT, P_Angle, "Link to current best p_angle"
- This definition means that to get the value of the keyword Tilt, follow the link named HMI_ORBIT to the data record it points to and get the value of keyword named P_Angle from that data record.
Note that while the create_series program does not complain about a link keyword without the matching record link it is probably wrong and will fail later in use...
Links
Links are pointers to records in (usually) other series. Links make it possible to effectively merge elements from other data records eliminating the necessity to make redundant copies. Links can also capture dependencies between data records such as processing history. For example a data record can contain links to the data records that were used in creating it, such as a dopplergram data record pointing to the filtergrams from which is was created. Links link records. A keyword link points to a particular keyword in a link record. A segment link points to a particular target segment in a link record. Links come in two varieties, static links and dynamic links.
Link: <name>, <target series>, <link_type>, <str description>
where:
<name> is word giving the symbolic name of the link. See also PostgreSqlReservedNames.
<target series> is a word containing the name of the series from which come the record pointed to by the link.
<link_type> can either take the value static which indicates that the link is static, i.e. based on record number, or the value dynamic which indicates that the link is dynamic, i.e. based on primary index.
<str description> is a string documenting the link for the user.
- A static link points to a single data record from the target series. The link must be established/bound by specifying the record number of the record pointed to. This is done by calling the JSOC function
- int drms_setlink_static(DRMS_Record_t *rec, const char *linkname, long long recnum)
from a process running within the JSOC environment. record is the record containing the link linkname and recnum is the target record number in the target series. A dynamic link points to the data record from the target series with the highest record number for a given primary index. This is done by calling the JSOC function:
- int drms_setlink_dynamic(DRMS_Record_t *rec, const char *linkname, DRMS_Type_t *types, DRMS_Type_Value_t *values)
Where again rec and linkname define the link and types and values specify the target record in the target series that will be linked. A linked record is used by e.g.
- DRMS_Record_t *drms_link_follow(DRMS_Record_t *rec, const char *linkname, int *status)
where rec is the record containing link linkname. drms_link_follow returns the target record in the target series. See the drms_link(3) man page for details.
Data Segments
The data segment section describes the data which is associated with records in the series being defined in the JSD but is stored in a file in SUMS rather than in the DRMS database tables. A segment may be direct or linked. A direct segment descriptor specifies the storage scope, protocol, and associated information such as data types, array sizes, etc., depending on the protocol used. A linked segment descriptor specifies the name of a record link descriptor and the name of a target segment held in the target record.
direct segments: Each direct data segment describes a chunk of data stored in a file in SUMS. Various external storage formats of the data segments are available, and are specified by the <protocol> field. Raw level data is often stored simply as files of unspecified generic type. Processed data is usually stored in an n-dimensional array of a simple type where the external storage protocol is not of interest to the user. The general form of a data segment description is
Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, <protocol>, {<protocol special>,} <str comment>The number of fields required depends on the value of <naxis> and <protocol>, so the following forms are valid
Data: <name>, <scope>, <datatype>, 0, <unit>, <protocol>, <comment> Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, fits, <tile-compression arguments>, <bzero>, <bscale>, <comment> Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, fitz, <bzero>, <bscale>, <comment> Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, bin, <bzero>, <bscale>, <comment> Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, bin.gz, <bzero>, <bscale>, <comment> Data: <name>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, tas, <tile-compression arguments>, <bzero>, <bscale>, <comment>
where:
<name> is a word containing the name of the data segment. See also PostgreSqlReservedNames.
<scope> can take the value constant, variable, or vardim. The names of these scopes can be confusing. A constant segment is one which points to the same file in SUMS for all records. It is essentially a link to the first segment (with the same name) written to the series. The constant segment will be set to point to the recnum of the first, and only, record which has had a drms_segment_write or drms_segment_write_from_file call for the constant segment. Variable and vardims segments are in many senses a normal segments which have different files in SUMS for each record. If the scope is variable then while the record contents may vary from one record to the next the dimensions are fixed by the constants in the JSD. If the scope is vardim then the dimensions of the data segment arrays may vary from record to record. Naxis does not change but axis dims are allowed to vary. For vardims the default are those from the JSD. At present there is no code to change them in a record segment, but they are stored and retrieved from the DRMS database table for each record in the series. The drms_write_segment function needs to be modified to copy the naxis and dims from the DRMS_Array_T that is passed! Vardims is used for hmi_ground.lev0 where the segment is populated with a not-yet published routine to write a FITS segment from a file while setting the dimensions to match the file. It would seem to be up to the brave user to set the dims when storing data.
<datatype> is the data type of the data in the data segment. It can be char, short, int, longlong, float, double, time, or string. Not all file formats support all these data types, see discussion for <protocol> below. [SHOULD ADD FIXED LENGTH STRINGS]
<naxis> is an integer and is the number of dimensions of an array data segment array, e.g. 1 is is a vector, 2 is a matrix, 3 is a cube etc. naxis may be 0 in which case no <axis dims> are required.
<axis dims> is a comma-space separated list of naxis positive integers giving the dimensions of the data segment array.
<unit> is a word describing the physical unit of the data segment values. If no units are to be specified use the word none.
<protocol> is the protocol used to store the data segments on disk. Supported protocol are:
bin - Binary array data in a DRMS specific internal format. The format consists of the 8-byte string "DRMS RAW", a 4-byte integer code for the data type, a 4-byte integer giving the number of axes (dimensions), a 4-byte integer for each of the dimensions giving the corresponding length of its axis, an 8-byte integer containg the total length in bytes of the following data, and the raw data in native format. All data are little-endian, and floating point numbers are IEEE, so the format is very similar to a minimal FITS file, with the stripped header information in binary format.
bin.gz - "gzipped" file containing the binary array data in the same format as above.
fits - Standard FITS format. Supports all data types.
fitz - FITS format compressed losslessly with first differencing plus Rice/Golomb entropy coding. Currently supports 8, 16 and 32 bit integer 1d and 2d arrays.
msi - Multi Scale Image format. note: this format is presently not implemented although "msi" is allowed by the JSD parser. Uses a lossless compression algorithm based on wavelet transformation followed by Rice/Golomb entropy coding, storing the bitstream in resolution progressive order. Allows lower resolution (downsampled by powers of two) versions of the image to be extracted rapidly by simply truncating the file. Currently supports 16 bit integer 1d and 2d images with dimensions that are powers of 2.
tas - Special DRMS format allows segments from multiple records to be stored in the same file and for efficient access to sub-arrays within a given segment. If the protocol is tas the AND the TAS file is to be tile-compressed, TAS tile sizes must also be specified in the <tile-compression arguments> field. TAS files may also be uncompressed, in which case, there is no tiling.
generic - Generic data. For this type of segment no processing is done by the DRMS library. The user is expected to use the Storage Unit directory provided for the record for her/his own purposes. The only DRMS API functions likely to be of use for generic data are drms_segment_write_from_file or drms_record_directory. [probably more formats to come...]
<tile-compression arguments> are the CFITSIO tiled-compression parameters that specify a compression algorithm and, optionally, tile sizes. In general, the string looks like "compress <algorithm> <tilesize1>,<tilesize2>...<tilesizeN>" when the data type is NOT a floating-point type. Tile-compression of floating point data is not supported (since it would most likely be a lossy compression). When the data are floating point, the only acceptable string is "" (empty string), which means NO COMPRESSION. <algorithm> refers to the compression algorithm desired. Permissible values are "GZIP", "Rice", or "PLIO". <tilesize1>...<tilesizeN> refer to the size of the tiles (<tilesizeN> refers to the same dimension that NAXISN refers to). If the string is not empty, then the string takes the form "compress" or "compress <algorithm> {<tilesize1>,<tilesize2>...<tilesizeN>}", where {} implies the tile sizes are optional. If the string is "compress" then by default the algorithm is Rice and the data are row-compressed. If the string contains tile sizes, then the size specified overrides the default row-compression. For the TAS protocol, in most cases the user should specify tile sizes and not rely upon row-compression.
<bzero> is the value of the offset that must be applied to the raw TAS file data to retrieve the data in physical units (the "real" values).
<bscale> is the scaling that must be applied to the raw TAS file data to retrieve the data in physical units (the "real" values).
- Data segments are numbered in the order the appear in the JSD. [THIS DESCRIPTION DOES NOT YET DESCRIBE THE BLOCKED SEGMENTS WITHIN A STORAGE UNIT]
linked segments: A linked segment is a pointer to a segment belonging to another record in another series.
Similar to keyword links, the linked segment descriptor contains only the information necessary to identify the target segment. At present the data type, and units will be taken from the target segment. While the naxis and dimensions are specified in the jsd they are also ignored at present. The protocol of course must be specified only in the target. The form is as follows:
The number of fields required depends on the value of <naxis> so the following forms are valid
Data: <name>, link, <linkname>, <target segment>, 0, <comment> Data: <name>, link, <linkname>, <target segment>, <naxis>, <axis dims>, <str comment>
- where:
<name> is a word specifying the name of the keyword.
<linkname> is a word that specifies the name of the link, which must be defined elsewhere in the same JSD. The link specification (see above) will identify a record in another series from which to find the target segment.
<target segment> is a word that specifies the name of the target segment from which to get the data for this segment from the record pointed to by the link.
<naxis> is an integer and is the number of dimensions of an array data segment array, e.g. 1 is is a vector, 2 is a matrix, 3 is a cube etc. naxis may be 0 in which case no <axis dims> are required or allowed.
<axis dims> is a comma-space separated list of naxis positive integers giving the dimensions of the data segment array.
<str description> is a string describing the content of the segment.
JSD Grammar
The complete grammar that describes the set of acceptable strings follows. Keep in mind that the only required section is Series Information. The other sections are optional. This grammar is applicable to the most recent jsd version, not older ones.
Each symbol of the grammar is enclosed in angled brackets ("<" and ">") and its definition lies to the right of the '='. Literal strings are enclosed in single quotes ("'"). Text enclosed in square brackets ("[" and "]") represents any single character from the set of characters specified in the brackets. A '-' within square brackets denotes a range of characters. For example, [A-Za-z] denotes any one of the 26 upper-case or 26 lower-case letters. S1 | S2 denotes alternation between the symbols S1 and S2 (S1 and S2 may be literals or symbols defined in a different rule). For example, 'Sanjaya' | 'Melinda' means either the string 'Sanjaya' or the string 'Melinda' satisfies the rule containing the alternation. Finally, curly brackets ("{" and "}") denote optional elements.
### Series Information ###
<SeriesName>
= 'Seriesname:' <ascii string>
<SeriesDescription>
= 'Description:' <ascii string>
<Owner>
= 'Owner:' <ascii string>
<Author>
= 'Author:' <ascii string>
<ArchiveFlag>
= 'Archive:' '0' | '1'
<UnitSize>
= 'Unitsize:' <string representing unsigned 32-bit integer [1, INT_MAX]>
<TapeGroup>
= 'Tapegroup:' <string representing unsigned 32-bit integer [0, INT_MAX]>
<Retention>
= 'Retention:' <string representing unsigned 32-bit integer [0, INT_MAX]>
<PrimaryIndex>
= 'Index:' <KeywordList>
<KeywordList>
= { <KeywordName> ',' } <KeywordName>
<KeywordName>
= <Name>
<Name>
= [A-Za-z] { <NameEnd> }
<NameEnd>
= [A-Za-z0-9_] { <NameEnd> }
*** Keywords ***
<Keyword>
= 'Keyword:' <KeyName> ',' <TypeAndFields>
<KeyName>
= <Name>
<Name>
= [A-Za-z] { <NameEnd> }
<NameEnd>
= [A-Za-z0-9_] { <NameEnd> }
<TypeAndFields>
= 'link,' <LinkName> ',' <TargetKey> ',' <Description>
| 'char,' <RecordScope> ',' <SegmentScope> ',' <DefChar> ','
<Format> ',' <Unit> ',' <Description>
| 'short,' <RecordScope> ',' <SegmentScope> ',' <DefShort> ','
< Format> ','< Unit> ',' <Description>
| 'int,' <RecordScope> ',' <SegmentScope> ',' <DefInt> ','
< Format> ',' <Unit> ',' <Description>
| 'longlong,' <RecordScope> ',' <SegmentScope> ',' <DefLongLong> ','
<Format> ',' <Unit> ',' <Description>
| 'float,' <RecordScope> ',' <SegmentScope> ',' <DefFloat> ','
<Format> ',' <Unit> ',' <Description>
| 'double,' <RecordScope> ',' <SegmentScope> ',' <DefDouble> ',
<Format> ',' <Unit> ',' <Description>
| 'time,' <RecordScope> ',' <SegmentScope> ',' <DefTime> ','
<Format> ',' <Unit> ',' <Description>
| 'string,' <RecordScope> ',' <SegmentScope> ',' <DefString> ','
<Format> ',' <Unit> ',' <Description>
<LinkName>
= <Name>
<TargetKey>
= <KeyName>
<RecordScope>
= 'constant' |'variable'
<SegmentScope>
= 'segment' | 'record'
<DefChar>
= <string representing signed 8-bit integer [SCHAR_MIN, SCHAR_MAX]>
<DefShort>
= <string representing signed 16-bit integer [SHRT_MIN, SHRT_MAX]>
<DefInt>
= <string representing signed 32-bit integer [INT_MIN, INT_MAX]>
<DefLongLong>
= <string representing signed 64-bit integer [LLONG_MIN, LLONG_MAX]>
<DefFloat>
= <string representing single-precision, floating-point number>
NOTE: numbers larger in magnitude than what a float can represent
will become infinite.
<DefDouble>
= <string representing double-precision, floating-point number>
NOTE: numbers larger in magnitude than what a double can represent
will become infinite.
<DefTime>
= <string representing time - see JSOC Time Format Notes>
<DefString>
= <ascii string no longer than DRMS_MAXPATHLEN - 1 characters, MUST HAVE LENGTH >= 1>
<Format>
= <printf conversion specification, e.g., %d, %c>
<Unit>
= <ascii string>
<Description>
= <ascii string>
### Data Segments ###
<DataSegment>
= 'Data:' <SegmentName> ',' <TypeAndFields>
<SegmentName>
= <ascii string>
<TypeAndFields>
= 'link,' <LinkName> ',' <TargetSeg> ',' <Naxis> ',' <Axes> ',' <Description>
| <Scope> ',' <Type> ',' <Naxis> ',' <Axes> ','
<Unit> ',' <Protocol> ',' <Description>
<LinkName>
= <Name>
<Name>
= [A-Za-z] { <NameEnd> }
<NameEnd>
= [A-Za-z0-9_] { <NameEnd> }
<TargetSeg>
= <SegmentName>
<Scope>
= 'constant' | 'variable' | 'vardim'
<Type>
= 'char' | 'short' | 'int' | 'longlong' | 'float' | 'double' |
'time' | 'string'
<Naxis>
= <string representing an unsigned 32-bit integer [1, INT_MAX]>
<Axes>
= <AxisDim1> ',' <AxisDim2> ',' ... ',' <AxisDimN>
where N = the integer represented by <Naxis>
<AxisDim>
= <string representing an unsigned 32-bit integer [1, INT_MAX]>
<Unit>
= <ascii string>
<Protocol>
= 'bin' | 'zip' | 'fits', <FitsProtocol> | 'fitz', <FitzProtocol> | 'msi' | 'tas' , <TasProtocol> | 'generic'
<FitsProtocol>
= <tile-compression arguments> ',' <bzero> ', ' <bscale> (see above for discussion on tile-compression arguments)
<FitzProtocol>
= <bzero> ', ' <bscale>
<TasProtocol>
= <tile-compression arguments> ',' <bzero> ', ' <bscale> (see above for discussion on tile-compression arguments)
<Description>
= <ascii string>
### Links ###
<Link>
= 'Link:' <LinkName> ',' <TargetSeries> ',' <Type> ',' <Description>
<LinkName>
= <Name>
<Name>
= [A-Za-z] { <NameEnd> }
<NameEnd>
= [A-Za-z0-9_] { <NameEnd> }
<TargetSeries>
= <ascii string>
<Type>
= 'static' | 'dynamic'
<Description>
= <ascii string>
Example .jsd File
#=====General Series Information===== Seriesname: su_arta.testfits_int Author: "arta" Owner: arta Unitsize: 1440 Archive: 0 Retention: 7 Tapegroup: 1 PrimeKeys: T_REC DBIndex: T_REC Description: MDI #=====Keywords===== Keyword:DATE, string, variable, record, " ", %s, none, " " Keyword:DATE__OBS, string, variable, record, " ", %s, none, "[DATE-OBS] " Keyword:EXPTIME, double, variable, record, nan, %f, s, " " Keyword:MJD, double, variable, record, nan, %f, none, " " Keyword:TIME, double, variable, record, nan, %f, none, " " Keyword:DSUN_OBS, double, variable, record, nan, %f, m, " " Keyword:R_SUN, double, variable, record, nan, %f, pixels, " " Keyword:R_SUN_REF, double, constant, record, 696000000.000000, %f, m, " " Keyword:CRLN_OBS, double, variable, record, nan, %f, m, " " Keyword:CRLT_OBS, double, variable, record, nan, %f, m, " " Keyword:CAR_ROT, int, variable, record, -2147483648, %d, none, " " Keyword:SAT_ROT, double, variable, record, nan, %f, none, " " Keyword:INST_ROT, double, variable, record, nan, %f, none, " " Keyword:X0, double, variable, record, nan, %f, none, " " Keyword:Y0, double, variable, record, nan, %f, none, " " Keyword:IM_SCALE, double, variable, record, nan, %f, none, " " Keyword:DATASIGN, int, variable, record, -2147483648, %d, none, " " Keyword:OBS_VR, double, variable, record, nan, %f, none, " " Keyword:OBS_VW, double, variable, record, nan, %f, none, " " Keyword:OBS_VN, double, variable, record, nan, %f, none, " " Keyword:series_num, int, variable, record, 0, %d, none, "Identifies dataset." Keyword:ds, int, variable, record, 0, %d, none, "Identifies virtual dataset." Keyword:rn, int, variable, record, 0, %d, none, "Identifies record within dataset." Keyword:DSNAME, string, variable, record, " ", %s, none, "" Keyword:RUNTIME, time, variable, record, -4712.01.01_12:00:00_UTC, 0, UTC, " " Keyword:XY_QUAL, int, variable, record, 0, %d, none, " " Keyword:EPH_QUAL, int, variable, record, 0, %d, none, " " Keyword:T_FIRST, time, variable, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " Keyword:T_LAST, time, variable, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " Keyword:VDS_VERS, string, variable, record, " ", %s, none, " " Keyword:DNAME0, string, variable, record, " ", %s, none, " " Keyword:SOURCE, string, variable, record, " ", %s, none, " " Keyword:SRC_ADJ, string, variable, record, " ", %s, none, " " Keyword:BLDVER00, int, variable, record, 0, %d, none, " " Keyword:BLDVER10, int, variable, record, 0, %d, none, " " Keyword:CALTBLS, string, variable, record, " ", %s, none, " " Keyword:QUALITY, string, variable, record, " ", %s, none, " " Keyword:DPC, string, variable, record, " ", %s, none, " " Keyword:T_REF, time, variable, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " Keyword:T_EARTH, time, variable, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " Keyword:BUNIT, string, variable, record, " ", %s, none, " " Keyword:S_MAJOR, double, variable, record, nan, %f, none, " " Keyword:S_MINOR, double, variable, record, nan, %f, none, " " Keyword:S_ANGLE, double, variable, record, nan, %f, none, " " Keyword:MAGNIFY, double, variable, record, nan, %f, none, " " Keyword:ORIENT, string, variable, record, " ", %s, none, " " Keyword:FD_SCALE, double, variable, record, nan, %f, none, " " Keyword:DATAMIN, double, variable, record, nan, %f, none, " " Keyword:DATAMAX, double, variable, record, nan, %f, none, " " Keyword:DATA_RMS, double, variable, record, nan, %f, none, " " Keyword:DATAMEAN, double, variable, record, nan, %f, none, " " Keyword:DATAMEDN, double, variable, record, nan, %f, none, " " Keyword:DATASKEW, double, variable, record, nan, %f, none, " " Keyword:DATAKURT, double, variable, record, nan, %f, none, " " Keyword:DATAVALS, int, variable, record, -2147483648, %d, none, " " Keyword:MISSVALS, int, variable, record, -2147483648, %d, none, " " Keyword:MISSPKTS, int, variable, record, -2147483648, %d, none, "" Keyword:CMPERRS, int, variable, record, -2147483648, %d, none, " " Keyword:SIZERRS, string, variable, record, " ", %s, none, "" Keyword:EARTH_B0, double, variable, record, nan, %f, none, " " Keyword:EARTH_L0, double, variable, record, nan, %f, none, " " Keyword:EARTH_CR, int, variable, record, -2147483648, %d, none, " " Keyword:EARTH_D, double, variable, record, nan, %f, none, " " Keyword:EARTH_VR, double, variable, record, nan, %f, none, " " Keyword:EARTH_VW, double, variable, record, nan, %f, none, " " Keyword:EARTH_VN, double, variable, record, nan, %f, none, " " Keyword:EARTH_R0, double, variable, record, nan, %f, none, " " Keyword:EARTH_DT, double, variable, record, nan, %f, none, " " Keyword:MDI_Y0, double, variable, record, nan, %f, none, " " Keyword:MDI_X0, double, variable, record, nan, %f, none, " " Keyword:TBLXYUPD, time, variable, record, -4712.01.01_12:00:00_UTC, 0, UTC, " " Keyword:P_ANGLE, double, variable, record, nan, %f, none, " " Keyword:FNDLMBXC, double, variable, record, nan, %f, none, " " Keyword:FNDLMBYC, double, variable, record, nan, %f, none, " " Keyword:FNDLMBMA, double, variable, record, nan, %f, none, " " Keyword:FNDLMBMI, double, variable, record, nan, %f, none, " " Keyword:FNDLMBAN, double, variable, record, nan, %f, none, " " Keyword:BLDVER15, int, variable, record, 0, %d, none, " " Keyword:DATE_OBS, string, variable, record, " ", %s, DRMS_MISSING_VALUE, " " Keyword:T_REC, time, ts_eq, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " Keyword:T_REC_step, double, constant, record, 60.000000, %lf, minute, "T_REC step " Keyword:T_REC_epoch, time, constant, record, 1993.01.01_00:04:56_TAI, 0, TAI, "" Keyword:TELESCOP, string, constant, record, "SOHO", %s, none, " " Keyword:INSTRUME, string, constant, record, "MDI", %s, none, " " Keyword:ORIGIN, string, constant, record, "SOI Science Support Center", %s, none, " " Keyword:CTYPE1, string, constant, record, "SOLARX", %s, none, " " Keyword:CRPIX1, double, variable, record, nan, %f, none, " " Keyword:CRVAL1, double, variable, record, nan, %f, none, " " Keyword:CROTA1, double, variable, record, nan, %f, none, " " Keyword:CDELT1, double, variable, record, nan, %f, none, " " Keyword:CUNIT1, string, constant, record, "arcsecond", %s, none, " " Keyword:XCEN, double, variable, record, nan, %f, none, " " Keyword:CTYPE2, string, constant, record, "SOLARY", %s, none, " " Keyword:CRPIX2, double, variable, record, nan, %f, none, " " Keyword:CRVAL2, double, variable, record, nan, %f, none, " " Keyword:CROTA2, double, variable, record, nan, %f, none, " " Keyword:CDELT2, double, variable, record, nan, %f, none, " " Keyword:CUNIT2, string, constant, record, "arcsecond", %s, none, " " Keyword:YCEN, double, variable, record, nan, %f, none, " " Keyword:T_OBS, time, variable, record, -4712.01.01_12:00:00_TAI, 0, TAI, " " #=====Segments===== Data: rice_img, variable, int, 2, 192, 192, m/s, fits, "compress Rice", 2.0, 0.35, "Fits data segment with Rice row compression" Data: rice_tile_img, variable, int, 2, 192, 192, m/s, fits, "compress Rice 96,96,1", 0.0, 1.0, "Fits data segment with Rice tile compression" Data: fitz_img, variable, int, 2, 192, 192, m/s, fitz, 0.0, 1.0, "Fitz data segment (Rice row compression)" Data: tas_img, variable, int, 2, 192, 192, m/s, tas, "compress Rice 96,96,1", 0.5, 0.015, "Tas data segment with Rice tile compression"