Release Notes JSOC V5.0 30Jan2009 ----------------------- --------- A release is a set of files, each having a specific version. And a release typcially has a version number because over time you have newer and newer releases of the same product. For example, a hypothetical 1.3 release may contain fileA#1.8, fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (eg., n00, n02). The resulting binaries have been minimally tested. At the time of the creation of the release, the release versions of each file will be the most recent. But as time passes, newer versions of some files will be made, and there is no guarantee that these changes will not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can simply use the production binaries, which are located at /home/production/cvs/JSOC. Every time a release is created, the binaries in this location get updated. Only the production user can update these binaries. So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example. If instead you want to work with stable source files, then you must have a sandbox, which is a local copy (in your home directory) of the files in the cvs depot. You would probably want to work with a sandbox if you plan on making eventual changes to the depot files. Changes you make to your sandbox files are not visible to other users until you "commit" those changes back to the cvs depot. Please see "If You Don't Have a Sandbox" below for more information on how to create a sandbox. There is also a "working" release which resides in in /home/jsoc/cvs/JSOC. New files may be placed here and existing files may be edited for common use before the next official release. Each time a release gets created, the source and binaries of the working release get updated. WARNING: the files you see here may not be stable since by the time you see them, another user may have edited them. Only the production release is guaranteed to be stable and unchanged between releases. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. If updating, you can supply the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the latest release. This will ensure that your working directory has the exact, latest release versions of the files (eg., jsoc_sync.csh -R). If checking-out, you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command to achieve the analogous result, but for a fresh checkout. WARNING: if you use the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update your sources thereafter. Use of "-R" or "-r Ver_LATEST" will result in a cvs "sticky flag" being set. jsoc_update.pl and jsoc_sync.pl clear this sticky flag. Additional Info --------------- If you are unfamiliar with the use of cvs see the file: JSOC/CM/working_with_sandbox.txt. There's a linux4 cvs gui at xim:/usr/bin/lincvs Also on our jsoc web page: http://jsoc.stanford.edu/cvs/JSOC/ Use the Apache cvs gui to see the diffs. For example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V4.7 - October 16, 2008) -------------------------------------------------------- NEW FEATURES: * RemoteSUMS implemented. If an attempt it made to access a storage unit not known to the local SUMS, but owned by a remote SUMS, a request is sent to the remote SUMS to transfer the storage unit to the local SUMS. When Stanford is the source of the storage unit, the transfer uses scp (or high-speed scp) via Stanford's j0.stanford.edu scp server. This requires the use of ssh-agent which entails some initialization, like placing a public key on an export account on j0 (details at http://jsoc.stanford.edu/jsocwiki/MOCServer#ClientSetup). The scp of files can happen either synchronously or asynchornously, depending on configuration (if the payload is above a certain size, the copy happens asynchronously). * Implementation of SUM_delete_series(). This provides support for removal of DRMS records when a series is deleted with delete_series. To enable this feature, the .jsd must specify an archive value of -1. * New localization support for non-Stanford DRMS sites. Users can now edit a text file (config.local.template) to specify various setting appropriate to their sites. The configure script will then apply those settings to the DRMS build environment. * Implemented [!...!] query construct. ... represents a PSQL where clause. If any part of a record-set query contains such a construct, then the "prime-key" logic is disabled. In other words, if two PSQL rows contain the exact same values for the prime key, then two DRMS records are created, not one as the prime-key log would dictate. * It is now possible to specify an archive value (via the jsd, the cmd-line, or the environment) of -1. Any storage unit thus labeled will cause all records whose data are contained in that storage unit to be deleted when the storage unit is deleted. * It is no longer possible to change series data retention via the environment. It was decidedly too dangerous to allow - a user could unknowingly run in a shell where the retention was set to 0. Also, don't allow users who don't own a series to reduce the retention time (they can increase it however). Retention time is specified in the jsd, and overridden with the DRMS_RETENTION=XXX cmd-line argument. Also, change the archive flag to an archive cmd-line argument (DRMS_ARCHIVE=XXX). * New show_info functionality: can provide an SUNUM and show_info will provide information about that storage unit, and the records whose data it contains. Also, new flags were added (eg, -z) that will provide storage unit information. * Added the ability to export storage units (directories) to lookdata.html. * Added SUM_info() - this provides information about a storage unit. * Added module-wide flags (--version, --ver, --vers, --vn, --about) that when used will cause the module to print out general module information, like version number. After printing this information, the module will terminate. * Added support for allowing a socket-module connection to a designated PSQL port. * It is no longer required for plan-file record-set queries that point to VDS datasets to be a path that has a trailing slash. If no such slash is present, DRMS will recognize that scenario and append one before calling soi code. * Added drms_open_nrecords(), an API function that efficiently limits the number of records requested from the PSQL database. * Documented a few API functions, like drms_open_recordset(). * Added new cmdparams API function that allows jsoc_main to reserve a set of cmd-line arguments. This ensure that module writers don't accidentally try to use an argument that jsoc_main reserves for use (like DRMS_ARCHIVE). * Added the ability to specify integer cmd-line argument ranges in the module's module_args structure (previously this was limited to floats and doubles). * For the Lev0 code a new library exists that will return per-time orbit information when provided an array of times. BUG FIXES: * Fixes that allow DRMS to build under gcc compilation. * When creating an "empty" TAS file, instead of attempting to store the entire desired array of missing values in memory so that the entire array is written, store a smaller block in memory, and then write that block several times. * When printing a jsd with show_info -j (or describe_series -j), if the series for which the jsd is being printed has a dbindex that contains an index keyword, then print the corresponding slotted keyword name, not the index keyword name. * show_info -j fixes: make cparms and bzero/bscale keywords implicit, so they aren't printed in the jsd; the output jsd is always a current-version jsd that has no version field (jsd's that have no version field refer to current-version series - a version field exists only if the series to be created is to be compatible with old code). * It is no longer required for there to be a space in otherwise empty strings in jsds. * Removed restriction on the number of characters in a record-set query, and on the number of lines in an @file record-set query. Fixed problem recognizing of not finding the terminating ']' of a record-set query under atypical situations. * Tests of time values equivalent to JD_0 were failing - they were using comparisons of doubles. Changed to do a range check (+- epsilon). * Automatically retry the scp from the MOC Server to Stanford when the original scp fails. * In the MOC LZP download script, ensure that when DOY is smaller than a 3-digit number, pre-pend with zeros. Also, ensure the script properly checks for the existence of a valid ssh-agent process.