NetDRMS Release Notes

Version 1.1

New Features since previous release (V 1.0):

• FITS implementation of TAS files - TAS (Tiled-Array Storage) is a DRMS segment protocol whereby data from more than one DRMS record are stored to the same file. In previous JSOC releases, the file was a homegrown file format (that didn't work 100% and it was not cross-platorm compatible). In this release, TAS uses a FITS file format. The TAS file has an extra dimension (one more than the number of dimensions of the data) - each value along this dimension corresponds to one DRMS record. The TAS file can be compressed for non-floating point data. For floating-point data, it cannot be compressed (compression would be lossy if this were allowed). Open TAS files are cached so that multiple reads/writes to the same file do not incur unneccessary file open/close overhead. In order to use this feature, the .jsd version that creates the series that uses a TAS file must be at least version 2.1
• Implementation of data slicing of FITS and TAS segment protocols
• Overload the DRMS_Segment_t structure's per_segment field, and rename to DRMS_Keyword_t::kwflags. This was needed so that we can differentiate between keywords that existed in the original .jsd, and ones that were implicitly created. We don't always want to show the implicitly created keywords in a .jsd generated by the "show_info -j" command.
• tdsignal.c - Implemented a threaded version of alarms. You can now set an alarm, and receive it, despite being in a multi-threaded environment. Added a sample module, proj/example/apps/threadsigs.c, to demonstrate how this works.
• Added functions drms_series_cancreaterecord(), drms_series_candeleterecord(), and drms_series_canupdaterecord() that say whether the caller is able to modify records in a series.
• SUM_alloc2() - This API allows you to specify a SUNUM to allocate space for. This differs from SUM_alloc(), which picks the SUNUM for you (the next in an increasing series).
• Record-set chunking. You can use new API functions, drms_open_recordset() and drms_recordset_fetchnext(), that allow you to work with 'chunks' of records in memory. The old drms_open_records() places ALL records in memory. Working with chunks will minimize the memory footprint of your module. Also, sufficiently large queries (queries that result in a large number of records) are truncated with the drms_open_records() call, but not with the drms_open_recordset() call. Currently, you must traverse the records from the first to the last, in the order returned by the drms_open_recordset() call.
• libcmdparams.a's cmdparams structure now contains the original cmd-line arguments (use cmdparams_get_argv() to fetch them).
• show_info now lists the archive, retention, and unitsize in the series info, and the sunum for each record for which information is printed.
• Make support for building intel mac architectures. This does not mean that mac binaries will compile and run. It means that the make files will now allow an attempt to compile.

Bug Fixes:

• cfitsio - By default, the cfitsio library will be named "libcfitsio.a". However, for 64-bit machines that compile with gcc, this can be overridden in the configure script by setting the CFITSIOFNAME_GCC_X86_64 make variable.
• show_info -j - Prevent segment-specific keywords from being expanded into multiple keywords when creating the jsd entries. The jsd should have the original keyword (eg., bzero), not a proliferation of segment-specific keywords (eg., bzero_000, bzero_001, bzero_002, etc.). The latter causes recursion (if you create a series from a .jsd that contains bzero_000, bzero_001, bzero_002, etc., then you end up with a series that has bzero_000_000, bzero_000_001, bzero_000_002, etc., bzero_001_000, bzero_001_0001, etc.).
• drms_export - Fix mapping from DRMS to FITS keywords that have missing values. For certain data types, conversion between the type used by DRMS and the corresponding type used by FITS was done improperly.
• drms_segment_read() - Fix for reading of binary-protocol data segments. The faulty code assumed that the binary reader was filling in bzero/bscale values, but our binary files have no bzero/bscale values. So the faulty code got garbage bzero/bscale values and used those when converting data, which led to garbage data.
• drms_stage_records() - Loop over record-set subsets (which are specific to a series) because drms_getunits() is series-specific. Also, if, within a subset, more than the maximum allowed number of storage units are requested, loop.
• create_tables.sql - removed inoperable script with name conflicting with a valid one, commented out comments and replaced suggested actions with real actions.
• Control-C will now cleanly shut down drms modules. Previously, the server was terminated, but the client did not get the memo. So the client would keep trying to contact the server. This could generate lots of debug messages, and at worst, crashes could occur.
• Overhaul of parsing of record-set queries containing string values. Also, some problems with parsing time strings were fixed (like the failure when parsing negative time strings). Fix drms_sscanf() which is largely reponsible for parsing these values (it is now named drms_sscanf2()).
• Fix many issues with "show_info -j" not producing valid .jsd files.
• .jsd Fix: .jsd files with 'index' keywords were resulting in series without any dbindex.
• Fix a crash in drms_query_bin(). The function was calculating a buffer length the wrong way (it should have used PQgetlength()).

Performance:

• Significant improvement in TAS-file and FITS-file slice access.
• When writing records to a TAS file, no longer are the superfluous and empty slot directories created.

NetDRMS 1.1 is a subset of JSOC Version 4.7, q.v. for Full release notes.

17 Oct 2008, 14:36-0700