Return to release.notes CVS log

Up to [Development] / JSOC / CM / V5.0

Add release notes for JSOC version 5.0

                       Release Notes JSOC V5.0         30Jan2009
                       -----------------------         ---------


A release is a set of files, each having a specific version.  And a release typcially
has a version number because over time you have newer and newer releases of the 
same product.  For example, a hypothetical 1.3 release may contain fileA#1.8, 
fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. 
JSOC releases are similarly versioned and contain a set of such files.  JSOC release
code is guaranteed to compile on cluster nodes (eg., n00, n02).  The resulting binaries
have been minimally tested.  At the time of the creation of the release, the
release versions of each file will be the most recent.  But as time passes, newer versions 
of some files will be made, and there is no guarantee that these changes will
not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute
properly).  

There are several ways to use this release.  If you wish to simply use pre-built
binaries, you can simply use the production binaries, which are located at 
/home/production/cvs/JSOC.  Every time a release is created, the binaries in
this location get updated.  Only the production user can update these binaries.
So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example.
If instead you want to work with stable source files, then you must have a sandbox,
which is a local copy (in your home directory) of the files in the cvs depot.  
You would probably want to work with a sandbox if you plan on making eventual 
changes to the depot files.  Changes you make to your sandbox files are not visible 
to other users until you "commit" those changes back to the cvs depot.  Please see
"If You Don't Have a Sandbox" below for more information on how to create a sandbox.  
There is also a "working" release which resides in in /home/jsoc/cvs/JSOC.  New 
files may be placed here and existing files may be edited for common use before the 
next official release.  Each time a release gets created, the source and binaries of 
the working release get updated.  WARNING: the files you see here may not be stable 
since by the time you see them, another user may have edited them. Only the production 
release is guaranteed to be stable and unchanged between releases.

Obtaining the Release
---------------------
To update your working directory to this release, or to check-out this release anew, 
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that
users may have modified files since the release was created, so use of the 
scripts documented in the web page may result in a working directory whose
content is not identical to the release.  If updating, you can supply 
the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the
latest release.  This will ensure that your working directory has the exact, latest
release versions of the files (eg., jsoc_sync.csh -R). If checking-out, 
you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command
to achieve the analogous result, but for a fresh checkout.  WARNING: if you use 
the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update 
your sources thereafter.  Use of "-R" or "-r Ver_LATEST" will result in a cvs
"sticky flag" being set.  jsoc_update.pl and jsoc_sync.pl clear this sticky flag.

Additional Info
---------------
If you are unfamiliar with the use of cvs see the file:
JSOC/CM/working_with_sandbox.txt.

There's a linux4 cvs gui at xim:/usr/bin/lincvs
Also on our jsoc web page:

http://jsoc.stanford.edu/cvs/JSOC/

Use the Apache cvs gui to see the diffs. For example, go to
http://jsoc.stanford.edu/cvs/JSOC/base/drms/
and click on the name in the File column and then click on
"diffs to previous #" to see the diffs.

Changes since previous release (V4.7 - October 16, 2008)
--------------------------------------------------------

NEW FEATURES: * RemoteSUMS implemented. If an attempt it made to
access a storage unit not known to the local SUMS, but owned by a
remote SUMS, a request is sent to the remote SUMS to transfer the
storage unit to the local SUMS. When Stanford is the source of the
storage unit, the transfer uses scp (or high-speed scp) via Stanford's
j0.stanford.edu scp server. This requires the use of ssh-agent which
entails some initialization, like placing a public key on an export
account on j0 (details at
http://jsoc.stanford.edu/jsocwiki/MOCServer#ClientSetup). The scp of
files can happen either synchronously or asynchornously, depending on
configuration (if the payload is above a certain size, the copy
happens asynchronously).

* Implementation of SUM_delete_series(). This provides support for
removal of DRMS records when a series is deleted with
delete_series. To enable this feature, the .jsd must specify an
archive value of -1.

* New localization support for non-Stanford DRMS sites. Users can now
edit a text file (config.local.template) to specify various setting
appropriate to their sites. The configure script will then apply those
settings to the DRMS build environment.

* Implemented [!...!] query construct. ... represents a PSQL where
clause. If any part of a record-set query contains such a construct,
then the "prime-key" logic is disabled. In other words, if two PSQL
rows contain the exact same values for the prime key, then two DRMS
records are created, not one as the prime-key log would dictate.

* It is now possible to specify an archive value (via the jsd, the
cmd-line, or the environment) of -1. Any storage unit thus labeled
will cause all records whose data are contained in that storage unit
to be deleted when the storage unit is deleted.

* It is no longer possible to change series data retention via the
environment. It was decidedly too dangerous to allow - a user could
unknowingly run in a shell where the retention was set to 0. Also,
don't allow users who don't own a series to reduce the retention time
(they can increase it however). Retention time is specified in the
jsd, and overridden with the DRMS_RETENTION=XXX cmd-line
argument. Also, change the archive flag to an archive cmd-line
argument (DRMS_ARCHIVE=XXX).

* New show_info functionality: can provide an SUNUM and show_info will
provide information about that storage unit, and the records whose
data it contains. Also, new flags were added (eg, -z) that will
provide storage unit information.

* Added the ability to export storage units (directories) to
lookdata.html.

* Added SUM_info() - this provides information about a storage unit.

* Added module-wide flags (--version, --ver, --vers, --vn, --about)
that when used will cause the module to print out general module
information, like version number. After printing this information, the
module will terminate.

* Added support for allowing a socket-module connection to a
designated PSQL port.

* It is no longer required for plan-file record-set queries that point
to VDS datasets to be a path that has a trailing slash. If no such
slash is present, DRMS will recognize that scenario and append one
before calling soi code.

* Added drms_open_nrecords(), an API function that efficiently limits
the number of records requested from the PSQL database.

* Documented a few API functions, like drms_open_recordset().

* Added new cmdparams API function that allows jsoc_main to reserve a
set of cmd-line arguments. This ensure that module writers don't
accidentally try to use an argument that jsoc_main reserves for use
(like DRMS_ARCHIVE).

* Added the ability to specify integer cmd-line argument ranges in the
module's module_args structure (previously this was limited to floats
and doubles).

* For the Lev0 code a new library exists that will return per-time
orbit information when provided an array of times.

BUG FIXES: 
* Fixes that allow DRMS to build under gcc compilation.

* When creating an "empty" TAS file, instead of attempting to store
the entire desired array of missing values in memory so that the
entire array is written, store a smaller block in memory, and then
write that block several times.

* When printing a jsd with show_info -j (or describe_series -j), if
the series for which the jsd is being printed has a dbindex that
contains an index keyword, then print the corresponding slotted
keyword name, not the index keyword name.

* show_info -j fixes: make cparms and bzero/bscale keywords implicit,
so they aren't printed in the jsd; the output jsd is always a
current-version jsd that has no version field (jsd's that have no
version field refer to current-version series - a version field exists
only if the series to be created is to be compatible with old code).

* It is no longer required for there to be a space in otherwise empty
strings in jsds.

* Removed restriction on the number of characters in a record-set
query, and on the number of lines in an @file record-set query. Fixed
problem recognizing of not finding the terminating ']' of a record-set
query under atypical situations.

* Tests of time values equivalent to JD_0 were failing - they were
using comparisons of doubles. Changed to do a range check (+-
epsilon).

* Automatically retry the scp from the MOC Server to Stanford when the
original scp fails.

* In the MOC LZP download script, ensure that when DOY is smaller than
a 3-digit number, pre-pend with zeros. Also, ensure the script
properly checks for the existence of a valid ssh-agent process.