Time Data Type and Representations of Dates and Times in the SDO JSOC

JSOC-TN 07-001

R S Bogart
2007.05.18

Background

This document supplants the SOI Technical Note on the same subject, SOI-TN 94-116. It describes:
  1. An internal representation for physical times used in the JSOC database and software analysis libraries
  2. A set of representations of times and dates recognized by and used in the JSOC
  3. Functions in the JSOC utilities library for translation of times between their internal format and the recognized external string representations
The internal type and the basic behaviour of the time representation functions has not been altered from that described in SOI-TN 07-001; however, there are a number of features described here and extensions provided that were not covered in that document.

Description of the TIME Data Types

The JSOC utility libraries, including the DRMS API, recognize and use a data type TIME. This is also available to JSOC modules Internally, data of type TIME are equivalent to double-precision floating-point numbers in the internal machine representation, and obey all the rules of floating-point arithmetic. In the database, data of type DRMS_TYPE_TIME are stored as ???.

Physically, data of type TIME are to be taken as representing in double-precision floating point format, the elapsed time measured in Système International (SI) seconds from a standard epoch. The epoch is chosen as the instant at which the year 1977 as recorded under Temps Atomique International (TAI) began: 00:00:00.0 on Jan 1, 1977 (TAI). This epoch is selected because it is the epoch used by the IAU for the definition of dynamical time (used by the Astronomical Almanac and by NASA).

Representation of times as IEEE double-precision numbers provides a relative numeric precision of 2 * 10-16, which is roughly two orders of magnitude better than the estimated accuracy of atomic clocks; it should thus be adequate for representation of all absolute times. For times within about 36 years of the epoch (1941 < year < 2013), that translates to an absolute accuracy of 0.1 usec. It also allows for an absolute accuracy of 27 sec for all times in the age of the solar system (± 4.5 Gyr). If greater accuracy is required, for example in the measurements of elapsed time, then we suggest using double precision to represent elapsed times referred to an arbitrary epoch; if the epoch is within one day of the time specified, then time intervals can be thus expressed to an accuracy of 26 psec (0.6 cm).

Representation of Dates and Times

The expression of dates and times externally is complicated by the dual aims of supporting a broad variety of "natural" representations and of being able to use multiple time scales. The representations should include customary designations of year. month, day of month, hour, minute, second as well as the less common but useful designations of day of year, Julian day number, and Carrington rotation. The time scales supported must include, besides TAI, Terrestrial Time (or Terrestrial Dynamical Time) (TT, TDT), (Coordinated) Universal Time (UT, UTC), Ephemeris Time (ET) and Civil (Zone) Time. It is important also to distinguish the representations accepted in the environment of JSOC software, i.e. key values and parameter values, from the representations required or allowed in various formats for the external representation of data, such as FITS. Although there are information interchange standards for string representation of dates and times, such as ISO 8601, the standards are defective in that they are not syntactic; they specify forms, but not meanings. In particular, although there are standards for civil zone-time representation, the standards do not address the issue of distinction between designations of times represented by a free-running clock, such as TAI, TDT, and ET, and those for a clock with occasional discontinuities, such as UTC and civil zone times.

Free-running times can of course be represented as simple counts in seconds, or convenient multiples of seconds, from an arbitrary epoch. For that purpose the Julian Day (JD) and Modified Julian Day (MJD) notations are used, the multiple being 86400.0 and the epochs we use (in customary representation) 11:59:27.816 (UT) on 1 Jan. 4613 BCE, and 23:59:27.816 (UT) on 16 Nov. 1858, respectively. The epochs are 32.184 sec before UT minute clock ticks because JD and MJD times are based on Terrestrial Time. TDT differs from TAI by a variable amount, but for times of interest the difference is within 10 usec of its exact value at the 1977.0 TAI epoch of 32.184 sec = 0.0003725 day. For our purposes we define TDT = TAI + 32.184 sec.

The correspondence between Coordinated Universal Time (UTC), upon which all civil times are based, and TAI is even more complicated. Since 1972, it has been based on intercalation of whole seconds immediately prior to the commencement of Jan. 1 or Jul. 1 at intervals selected so that the accumulated difference between observed Universal Time (UT1) and UTC does not exceed 0.9 second at the time of intercalation. In principle seconds can be intercalated (or dropped) at the quarters of the year as well. Since the adjustment is based on observations and since the rotation rate of the Earth is not constant, it is not possible to predict with absolute accuracy when future adjustments will have to be made. There is consequently no fixed formula for deciding when to intercalate. Furthermore, prior to 1972, the adjustments were made more frequently and the allowed accumulated differences were much smaller. For our purposes, however, it should be sufficient to use a fictitious UTC based on past projection of the present formula for intercalation. Adjustment dates are to be made according to Table 1. Prior to 1960.0 there should be no difference between TAI and UTC. Neglecting secular deceleration of the Earth the present rate is consistent with an average correction rate of about one second per year. However, the decisions of when and even if to intercalate UT leap seconds are social, and there is a strong possibility that future intercalations may be suspended. It is best when referring to both future times and times in the distant past to use dynamical or atomic time.

Two alternative forms are available for full specification of a time. One uses the Julian Day number or modified Julian Day number and fraction to represent the time as a virtual floating-point number (although not the internal number); the other specifies the time in customary calendar date / time of day format. The former is really only suitable for expression of physical time units in the TDT system. Therefore, it is assumed that any time so represented is the count in days (SI seconds / 86400.0) from the appropriate epoch.

A fully specified time in the Julian Day notation is of the form:

{MJD|JD}_X[.Y]
The notation is that square brackets [] enclose optional fields, braces {} enclose sets of alternative symbols which are separated by vertical bars |, and characters in bold-face (including the underscore and period symbols) are to be reproduced literally. The characters X and Y are to be replaced by numeric values representing respectively the day number and the fraction of a day, where a day is 86400 SI seconds. (Note that this represents a change from the usage described in SOI-TN 94-116, in which offsets from a TAI epoch 32.184 sec after the JD epoch could be specified with an optional _TAI suffix.) The following forms are both valid as Julian Day representations of the same time:
JD_2450000.25
MJD_49999.75

A Calendar/Clock notation is more problematical, but essential for comparison with times expressed in common notations. We have adopted the following form for a fully specified time in the Calendar/Clock notation:

[stddate[{_time | zone}] | isodate[Ttime]]
stddate := Y[.{L|R|nnn}[.D[.X]]]
isodate := Y[-L[-D[.X]]]
time := HH[:MM[:SS[.X]]][zone]
zone := {zltr|offest|_{TAI|TT|TDT|UT|UTC|GPS|znam}
zltr := {A-I,K-Z}
offset := {+|-}HHMM
znam : a 3- or 4-letter timezone designation
In this notation, Y, L, and D are optionally signed integers, representing the year, month, and day numbers respectively. HH, MM, and SS are optionally signed integers representing the whole number of hours, minutes and seconds elapsed from the start of day in the relevant time system, and X the decimal fraction of a second. Although ISO 8601 standard date/time representations, using '-' separators in the date field and the character 'T' to separate the date from the time field , are supported, they are not recommended, as the hyphen separator can be confused with a leading minus sign in the integer fields for month and day. Also, the 'T' separator could be potentially confused with the time zone letter designation.

The initial integer represents the year number of the Common Era (ce or AD). Negative integers are used for dates before the Common Era (BC or BCE) despite the fact that there is no year 0. It is recommended, but not required, that plus or minus signs not be used unless required for the year. (This means that the function sscan_time, described below, can accept them, but the function sprint_time will not produce them, except as required.)

The second field represents the month number or name. It is either an optionally signed integer, the Roman numeral for an integer in the range [1, 12], or a three-letter designation for the month consisting of the the first three letters of its English name. (The Roman numeral and the three-letter designation can be in any combination of upper and lower case). The third field is the day number of the month, an integer. The use of day numbers less than 1 or greater than the length of the month should be discouraged, as in certain cases it can lead to erroneous results. Both the month and day fields are optional; if m issing they are assumed be equal to 1.

The fourth field of the date-time can be either an integer representing the fraction of the 86,400 second TAI day or a composite field for the clock time of day. The use of fractional day designations with time scales other than TAI or TT should be avoided. The clock time is expressed in the customary colon-separated fields, with an optional decimal point and fractional part of a second. Note that the second (minute) and third (second) fields of the clock time itself are optional; if absent they are assumed to be 0. Leading zeroes to make each integer field two-digits long are recommended but not required. Times must be expressed in 24-hour time, not 12-hour meridian time. On days when leap seconds occur in time scales based on earth rotation (UT and civil time), the extra second is expressed as time 23:59:60. For this reason, the use of hour, minute, and second numbers beyond the normal range ([0,23], [0,59], and [0,59] or [0,60]) should be avoided. If the fourth field is absent its value is assumed to be 0 (or 00:00:00.0).

The final (optional) field, is a string designating the time scale and epoch (zone). It can be a single letter, a string of three or four letters, or a signed four-digit integer representing the offset from UT in hours and minutes, the first two digits being hours and the latter minutes. Upper-case characters are recommended but not required. If the time scale designation is not provided, the default is to interpret the date time as in Universal Time (UT, UTC). If zone time is used, no check for agreement with civil time is implied; the date-time 1995.07.27_12:00:00_MST is valid and refers to the same instant regardless of whether the event referred to (if any) occurs in Colorado, Arizona, or Sonora. Valid values for the zone time string are listed in Table 2. The following strings should all be valid representations of the same time:

1995.10.09_18:00:00
1995.X.09_10:00:00.0_PST
1995.10.09_18:00:29.0_TAI
1995.10.09_18:01:01.184_TT
95.Oct.9.7500Z
1995-10-09T18:00:00

It is recommended, but not required, that the values L, D, HH, MM, and SS be expressed as unsigned 2-digit integers with leading 0's as appropriate, and that they be limited to their normal ranges: 01-12 for L, 00-59 for HH and MM, 01-31 (or less as appropriate for the month) for D, and 00-59 for SS, except for UTC-based systems in which the value 60 is used to designate leap seconds.

Since there is no standard for the physical meaning of the various time representations, we adopt a convention that for free-running clock systems, 1 minute = 60 seconds (always), 1 hour = 60 minutes, 1 day = 24 hours, and that 1 month is the conventional number of days according to the Gregorian calendar system or the proleptic Julian calendar system as appropriate. (1 year is of course 12 months). We also adopt the convention that in the TAI system, dates and times correspond exactly to those in the UT (= UTC) system for all times up to 1959.12.31_23:59:59.

It should be noted that these formats are not consistent with all of the various ANSI and ISO standards for representation of date and time of day. In the event that dates and times in other standard formats are encountered, we should be prepared to translate them. In particular all-numeric and all-hyphen separated strings, commas used for the decimal point, and civil time differentials with colons, of the forms:

19951009-180000
1995-10-09-18-00-00,0
19951009200000+02:00
could all be employed consistent with standards. These should be handled as needed. The same remark applies to a large number of civil time designations, for which there may be conventions or even national standards, but evidently no international standards; their use should be discouraged. There is also no provision at this time for representation of times by Carrington times or longitudes, e.g. CR_1336.59, CR1337:184 or CL_1337.184. In any case, such representations are more grossly dependent on the spatial location of the observer than the calendar-clock notations.

In calendric calculations, the Gregorian reckoning (400 years = 146097 days) is to be used for all dates on or after 1582.10.15.0 = JD_2299160.5. For all dates prior to this, the proleptic Julian reckoning (4 years = 1461 days) is to be used. The second immediately preceeding JD_2299160.5 is designated 1582.10.04_23:59:59.

It should be remarked that these notes apply exclusively to the reprentations of dates and times within the JSOC analysis environment, as for example in database keyword values and in module argument lists. FITS does not recognize a data type of type "Time" for either data or data attributes. The FITS standard does reserve certain keywords (any starting with "DATE") for the representation of dates and optionally times as character strings, and of reference times (e.g. "EQUINOX") as floating-point numbers. The former are required to be in ISO format. CDF does define an "EPOCH" data type similar to the present definition, but loosely defined in terms of the reference calendar and time scale. The standard representation for EPOCH data types in CDF, .e.g. "9-Oct-1995 10:00:00.000", is unsuitable for JSOC environment use because of the embedded space and because of the lack of flexibility in specifying the time scale. When presented in external data representations, TIME type data must be encoded as character strings, with information on their appropriate parsing as dates and times based on registration of the relevant keywords. This only applies to data attributes (keyword values). Data values themselves, for example in data segments, are naturally represented as double-precision numbers.

For the sake of preserving time representation information, it might be useful to set up a pair of reserved keywords for the project:

keyword TIMETYPE:
ET, TAI, TDT, TT, GMT*, UT, UTC, UT1*, GMAT*, LCT*, UT2*

keyword TIMEZONE:
A-I,K-Z, PST etc., +{1-12}, 0, -{1-12}

Functions for External Time Representations

Two functions are provided for conversion between the internal time values and the character representations as described in this note: sscan_time and sprint_time. sscan_time merely converts its single string argument to the equivalent internal TIME type value, assuming that the string can be unambiguously parsed according to one of the representation rules. If the string cannot be parsed as a recognized date and time, the function returns a fixed value equal to the time corresponding to JD_0.0 (defined as JULIAN_DAY_ZERO in the required include file).

sprint_time takes four arguments: a time t, a pointer to a string str (which must be allocated sufficient space to hold the time representation), a string type representing the time zone or other format representation to be used, and an integer precision. The precision argument, if non-negative, represents the number of decimal digits to be included in either the seconds field of standard clock time representations, or simply the number of decimal digits to be printed in the Julian Day notation. Negative values of the precision argument are interpreted to mean successive dropping of fields from the right (with time truncation rather than rounding) in standard date/clock representations.

The type argument can be any of the strings used for the zone field in the Calendar/Clock notation defined above. Valid and recognized values of znam are those listed in Table 2. Any of these values will result in printing of a string using the stddate format. An empty (or NULL) string in the type argument will result in the time being written as a UT time in the stddate format, but with no zone identifier. If the type argument is JD or MJD, the time will be written in the appropriate Julian Day representation. Finally, the special type argument value of ISO will cause the string to be written in isodate format, with the zone designation Z. Times written in this format with positive precision use the decimal point, rather than the comma preferred by the ISO 8601 standard.


Table 1: Dates and times of adjustments in UTC

(Except for the first two entries, the extra second occurs one second prior to the time of each entry.)

(a) fictitious

1959.12.31_23:59:59_UTC = 1959.12.31_23:59:59_TAI
1959.12.31_23:59:60_UTC = 1960.01.01_00:00:00_TAI
1960.01.01_00:00:00_UTC = 1960.01.01_00:00:01_TAI
1962.07.01_00:00:00_UTC = 1962.07.01_00:00:02_TAI
1964.07.01_00:00:00_UTC = 1964.07.01_00:00:03_TAI
1965.07.01_00:00:00_UTC = 1965.07.01_00:00:04_TAI
1966.07.01_00:00:00_UTC = 1966.07.01_00:00:05_TAI
1968.01.01_00:00:00_UTC = 1968.01.01_00:00:06_TAI
1969.01.01_00:00:00_UTC = 1969.01.01_00:00:07_TAI
1970.01.01_00:00:00_UTC = 1970.01.01_00:00:08_TAI
1971.01.01_00:00:00_UTC = 1971.01.01_00:00:09_TAI

(b) actual

1972.01.01_00:00:00_UTC = 1972.01.01_00:00:10_TAI
1972.07.01_00:00:00_UTC = 1972.07.01_00:00:11_TAI
1973.01.01_00:00:00_UTC = 1973.01.01_00:00:12_TAI
1974.01.01_00:00:00_UTC = 1974.01.01_00:00:13_TAI
1975.01.01_00:00:00_UTC = 1975.01.01_00:00:14_TAI
1976.01.01_00:00:00_UTC = 1976.01.01_00:00:15_TAI
1977.01.01_00:00:00_UTC = 1977.01.01_00:00:16_TAI
1978.01.01_00:00:00_UTC = 1978.01.01_00:00:17_TAI
1979.01.01_00:00:00_UTC = 1979.01.01_00:00:18_TAI
1980.01.01_00:00:00_UTC = 1980.01.01_00:00:19_TAI
1981.07.01_00:00:00_UTC = 1981.07.01_00:00:20_TAI
1982.07.01_00:00:00_UTC = 1982.07.01_00:00:21_TAI
1983.07.01_00:00:00_UTC = 1983.07.01_00:00:22_TAI
1985.07.01_00:00:00_UTC = 1985.07.01_00:00:23_TAI
1988.01.01_00:00:00_UTC = 1988.01.01_00:00:24_TAI
1990.01.01_00:00:00_UTC = 1990.01.01_00:00:25_TAI
1991.01.01_00:00:00_UTC = 1991.01.01_00:00:26_TAI
1992.07.01_00:00:00_UTC = 1992.07.01_00:00:27_TAI
1993.07.01_00:00:00_UTC = 1993.07.01_00:00:28_TAI
1994.07.01_00:00:00_UTC = 1994.07.01_00:00:29_TAI
1996.01.01_00:00:00_UTC = 1996.01.01_00:00:30_TAI
1997.07.01_00:00:00_UTC = 1997.01.01_00:00:31_TAI
1999.01.01_00:00:00_UTC = 1999.01.01_00:00:32_TAI
2006.01.01_00:00:00_UTC = 2006.01.01_00:00:33_TAI
2009.01.01_00:00:00_UTC = 2009.01.01_00:00:34_TAI
2012.01.01_00:00:00_UTC = 2012.01.01_00:00:35_TAI
2015.01.01_00:00:00_UTC = 2015.01.01_00:00:36_TAI
2017.01.01_00:00:00_UTC = 2017.01.01_00:00:37_TAI

(c) projected

none


Table 2: Supported Civil Time Zone Designations

Abbrev.   Zone (UTC)      Abbrev.   Zone (UTC)      Abbrev.      Zone (UTC)

U, PST, YDT     -8        Z, GMT, WET      0        H, AWST, SST, WST   8
T, MST, PDT     -7        A, CET           1        I, JST, AWDT        9
                                                       ACST             9:30
S, CST, MDT     -6        B, EET           2        K, JDT, AEST       10
                                                       ACDT            10:30
R, EST, CDT     -5        C                3        L  AEDT            11
Q, AST, EDT     -4        D                4        M, NZST            12
P, ADT          -3        E                5        X                 -11
   NST          -3:30        IST           5:30
O               -2        F                6        W, HST, BDT       -10
   NDT          -2:30
N               -1        G                7        V, YST, HDT, AKST  -9
                                                       NZDT            13
                                                    Y                 -12


References:

ANSI standard X3.30-1985: Representation for calendar date and ordinal date for information interchange. 1985-07-30
ANSI standard X3.43-1986: Representations of local time of day for information interchange. 1985-06-23
ANSI standard X3.51-1986: Representations of universal time, local time differentials, and United States time zone references for information interchange. 1986-06-23
Explanatory Supplement to the Astronomical Almanac, ed. K. Seidelmann, 1992.
ISO standard 2014: Writing of calendar dates in all-numeric form. 1976-04-01
ISO standard 3307: Information interchange - Representation of time of the day. 1975-03-15
ISO standard 4031: Information interchange - Representation of local time differentials. 1978-12-15
ISO standard 8601: Data elements and interchange formats - Information interchange - Representation of dates and times. 2004-12-01
McCarthy, D.M., US Naval Observatory, private communication.
NOST Standard 100-2.0: Definition of the Flexible Image Transport System (FITS)
NSSDC: CDF Users Guide, Version 2.6, 1998.