Data segments

The data segment section describes the binary data associated with records from the series being defined in the JSD. Each data segment is an $n$-dimensional array of a simple type. Various external storage formats of the data segment arrays are available, and are specified by the <protocol> field. The general form of a data segment description is

Data: <name>, <form>, <scope>, <datatype>, <naxis>, <axis dims>, <unit>, <protocol>

The number of fields required depends on the value of <form> and <scope>, so the following five forms are valid

Data: <name>, generic, fixed
Data: <name>, generic, variable
Data: <name>, array, fixed, <datatype>, <naxis>, <axis dims>, <unit>, <protocol>
Data: <name>, array, variable, <datatype>, <naxis>, <axis dims>, <unit>, <protocol>
Data: <name>, array, vardim, <datatype>, <naxis>, <unit>, <protocol>

where

<name> is a string containing the name of the data segment.

<form> is either generic which means that the data segment is an unstructured file or array which means that the data segment is a multi-dimensional array.

<scope> can take the value fixed which specifies that the contents of the data segment is the same for all data records in the series. If <form> is generic then <scope> can take the value variable which means that a different data segment file is associated with each record. If <form> is array then <scope> can take the value variable which means that a different data segment array of fixed size is associated with each record, or <scope> can take the value vardim which means that a different data segment array of varying size but fixed dimensionality is associated with each record.

<datatype> is the data type of the data in the data segment. It can be char, short, int, long, long long, float, double, datetime, timestamp or string. Not all file formats support all these data types, see discussion for <protocol> below. [SHOULD ADD FIXED LENGTH STRINGS]

<naxis> is the number of dimensions of an array data segment array, e.g. 1 is is a vector, 2 is a matrix, 3 is a cube etc.

<axis dims> is a comma separated list of <naxis> positive integers giving the dimensions of the data segment array.

<unit> is the physical unit of the data segment.

<protocol> is the protocol used to store the data segments on disk. Supported protocol are:

FITS Standard FITS format. Supports all data types.

FITZ FITS format compressed losslessly with first differencing plus Rice/Golomb entropy coding. Currently supports 8, 16 and 32 bit integer 1d and 2d arrays.

MSI Multi Scale Image format. Uses a lossless compression algorithm based on wavelet transformation followed by Rice/Golomb entropy coding, storing the bitstream in resolution progressive order. Allows lower resolution (downsampled by powers of two) versions of the image to be extracted rapidly by simply truncating the file. Currently supports 16 bit integer 1d and 2d images with dimensions that are powers of 2.

PNG ?

JPEG ?

TIFF ?

MPEG ?

[probably more formats to come...]

<data segment number> is an integer between 000 and 999. Data segments are numbered according to the order in which they occur in the JSD.