DRMS_Segment_struct Struct Reference

DRMS segment struct. More...

#include <drms_types.h>

Collaboration diagram for DRMS_Segment_struct:

Data Fields

int axis [DRMS_MAXRANK]
 Size of each dimension.
int blocksize [DRMS_MAXRANK]
 Block sizes for TAS storage.
double bscale
 Data scaling factor.
double bzero
 Data scaling offset.
char cparms [DRMS_MAXCPARMS]
 Storage file name.
 Contains attributes common to all records in a series.
struct DRMS_Record_structrecord
 The record this segment belongs to.

Detailed Description

DRMS segment struct.

A DRMS data segment corresponds to a named file, typically containing an n-dimensional scalar array. (It can also be a "generic" segment, which is just an unstructured file as far as DRMS is concerned.) One or more segments constitute the external data part(s) of the DRMS record pointed to by the record field. The info field points to a structure containing attributes common to all records in a series, while the segment structure itself contains the fields axis and blocksize that can vary from record to record if scope=DRMS_VARDIM.

The protocol field determines the external storage format used for storing segment data. Only protocols DRMS_BINARY, DRMS_BINZIP, DRMS_FITS, DRMS_FITZ, DRMS_GENERIC, and DRMS_TAS are fully supported in the base DRMS system (NetDRMS). Protocol DRMS_DSDS is a special protocol for dealing with the format of the Stanford SOI-MDI Data Sorage and Distribution System (DSDS) and requires support outside the DRMS library. Protocol DRMS_LOCAL likewise supports the DSDS file-format and requires a non-NetDRMS library. It differs from DRMS_DSDS in that it does not depend on the presence of DSDS - it merely allows the user to operate on files external to DSDS (eg., files that may reside on a LOCAL hard disk) that happen to have the file format that DSDS uses. DRMS_GENERIC and DRMS_MSI are also reserved for unsupported data formats. In particular, the DRMS_GENERIC protocol is used to refer to any unstructured data format or data formats of unknown structure.

Data storage for DRMS_FITS is in minimal simple FITS files, without extensions and with only the compliance- and structure-defining keywords (SIMPLE, BITPIX, NAXIS, NAXISn, and END, and optionally BLANK, BSCALE and BZERO) in the headers. All other ancillary data are to be found in the DRMS record. For the DRMS_FITZ protocol, the representation is similar except that the entire FITS file is compressed with Rice compression. (Note that because the memory representation for the data supported through the API functions drms_segment_read is the DRMS_Array_t struct, which has a maximum rank of 16, FITS hypercubes of dimension > 16 are not supported.)

For the DRMS_BINARY protocol, the data are written in a binary format, in which the first 8 bytes are the characters "DRMS RAW", the next 8(n+1) are little-endian integer representations of the data type, rank, and dimensions of the n axes, and the remainder the binary data in little-endian format. For the DRMS_BINZIP protocol the represntation is the same, except that the file is gzip compressed. The DRMS_TAS protocol (for "Tiled Array Storage") is described elsewhere, if at all. It is designed for use with data segments that are small compared with the size of the full data records, in order to minimize file access without keeping all of the segment data in the relational database, by concatenating multiple segments in the external format. The segment blocksize member is for use with the DRMS_TAS protocol.

Segment data types refer to the scalar data type for the segment, and should be mostly self-explanatory. DRMS_TYPE_TIME is a special case of double-precision floating point values representing elapsed time from a fixed epoch. Arithmetic is the same as for DRMS_TYPE_DOUBLE, only the format for string representations differs from that for normal floating-point data; see sprint_time. Data of type DRMS_TYPE_STRING are null-terminated byte sequences of any length. Data type DRMS_TYPE_STRING is not supported by the protocols DRMS_BINARY, DRMS_BINZIP, DRMS_FITS, nor DRMS_FITZ. Whether it is properly supported by the DRMS_TAS protocol is doubtful. The data type DRMS_TYPE_RAW is used to describe data that are not to be converted on read from the type of their external representation, which must then be established for type-specific operations. It should be used for DRMS_Array_t structures only, not for DRMS segments.

The scope of a segment can take on three values. The normal scope is expected to be DRMS_VARIABLE, for which the particular segment for every record has exactly the same structure (rank and dimensions), only the actual data values vary from one record to another. (Note that different segments of a record, however, need not have the same structure as one another.) If the scope is DRMS_VARDIM, then the dimensions and even rank of the particular segment may vary from one record to another, although other features of the segment, in particular the data type, must still be the same. Scope DRMS_CONSTANT is used to describe a data segment that is constant for all records. It can be used for example to describe a location index array, or a constant calibration array that applies to all records in the series, so that it can be made available to any record without having to store multiple instances externally.

Definition at line 1293 of file drms_types.h.

The documentation for this struct was generated from the following file:

Generated on Fri Mar 24 07:00:35 2017 for JSOC_Documentation by  doxygen