Release Notes JSOC V8.5 23JUN014 ----------------------- -------- A release is a set of binary and text files, each having a specific version. The release itself is also versioned to facilitate the identification of the release. For example, release 1.3 may contain fileA#1.8, fileB#1.2, and fileC#2.2 and release 1.4 may contain fileA#2.5, fileB#2.1, and fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (e.g., n04 and solar3). At the time of the creation of the release, the versions of each file will be very recent - most will be the "head" versions. The set of files is stable. They have been tested and verified to work properly on the supported systems. As time passes, newer versions of files will be committed to the code repository and there is no guarantee that these changes will not destabilize JSOC (i.e., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can use the production binaries, which are located at /home/jsoc/cvs/Development/JSOC (The release binaries are actually in /home/jsoc/cvs/JSOC, but as our production code changes more quickly than releases are generated, we put critical fixes in the "Development" tree. To be sure you use code that has been built with these critical fixes, you'll need to use the "Development" tree. As time passes our production code will stabilize. When that happens, you should use /home/jsoc/cvs/JSOC. But for now, you should use the "Development" tree.). Every time a release is created, the binaries in this location get updated by the jsoc user. To use these binaries, you can run them directly, "/home/jsoc/cvs/JSOC/Development/bin/linux_x86_64/show_info -j hmi.M_45s", for example. Or you can put /home/jsoc/cvs/JSOC/Development/bin/linux_x86_64 in your path environment variable. If instead you want to work with stable source files, then you must have a local copy (e.g., in your home directory) of the source code in the CVS code repository. You must "build" or "make" the code this local code before it can be used. You will want to work with a local copy if you plan on making changes to the repository files (i.e., you plan on doing development). Changes you make to your local copy are not visible to other users until you commit those changes back to the CVS repository. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit (please see the section entitled Update Your Existing Working Directory ("Sandbox")). Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. Additional Info --------------- Use the Apache CVS gui to see diffs between file revisions. For example, go to http://jsoc2.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V8.4 - April 1, 2014) ---------------------------------------------------------- NEW FEATURES: + Added support for a JMD installation. The JMD (Java Mirroring Daemon) is software that automatically downloads storage units as their containing DRMS records are added to a DRMS. To enable and use the JMD, the config.local file must be modified. The parameter JMD_IS_INSTALLED must be set to non-zero, and JMD_URL must contain the URL to the JMD web server. + Added the inv2d globalhs library. + Added the mhd_32daily project. + Added a new module, drms_parsekeys, that parses keyword JSD-type descriptions, returning a JSON representation. It can be used by scripts that need the database representation of DRMS keywords. + Modified the way the DRMS_RETENTION DRMS module parameter works. It now overrides all other ways of specifying the "staging" retention. The value of this parameter is used by SUMS when setting the retention of existing SUs that are accessed. If the current retention value of an accessed SU is greater or equal to this parameter's value, then this parameter is a no-op. But if the current retention value is less than this parameter's value, then SUMS will modify the SU's retention value to be AT LEAST as large as this parameter's value. This parameter can no longer be used to reduce the retention of any storage unit (see the notes on drms_setsuretention if you need to reduce retention values). + Added a new DRMS-module general parameter, DRMS_NEWSURETENTION. The value of this parameter is what SUMS will set the retention of all newly created SUs to. It overrides the "jsd" value - the value of the Retention field in the JSD. + Added drms_setsuretention. This module is now the only way to reduce the retention of storage units. All other modules can only increase an SU's retention (by using the DRMS_RETENTION general DRMS module argument). + Modified the drms_parser.c code in lib DRMS to accept JSDs that contain a new series-info field: StagingRetention. In this field the JSD-writer specifies the staging retention (the retention applied to all SUs retrieved from SUMS). The drms_parser.c code also now displays this field when a user runs show_info -j. + Modified the SUMS' SUM_get() call to increase the retention to X + 3 days when the caller requests a retention of at least X days. + The libdrms code that initiates remote-sums requests was modified to not launch a new shell after the child process (the one that actually interacts with the export system) forks. The forked child simply interacts with the export system directly. + Modified show_info and jsoc_info to print shadow-table status. + Moved a command-line-parsing Python class from a script to a true module. Modified a couple of scripts to use this new module. + Added the publist.py script. It display a list of published series, and can optionally display a list of subscribed-to series, and the institutions subscribed to those series. Or it can display a list of subscribing institutions, and the series to which they are subscribed. This script is accessible as a cgi program too. + Added the sums_procck.py script to start and stop the various SUMS services at remote sites. Modified sum_start.NetDRMS and sum_stop.NetDRMS to use this script. Together, these scripts maintain a pid file that identifies the pids of all SUMS processes. When SUMS is running, this pid file exists, and it contains the pids of the various SUMS components. To stop SUMS, this pid file is read, and all contained processes are killed, and then this pid file is removed. + Added mrmlossynoptic, a module to generate both Mr and Mlos synoptic charts. + Added maprojbrfromblos, a module to map image to heliographic coordinates with Lat/Lon in degrees. DEFECTS FIXED: + The drms_cmdparams cmdparams wrapper was fixed. It no longer mixes status codes returned by lib cmdparams and status codes returned by DRMS. + libdsds was modified to more efficiently handle RDB.FITS_MERGE DSDS datasets. + A seg fault in the auto-retrieve-linked-records code was fixed. A check for no-linked records was missing. + Code in the socket-connect DRMS library existed to generate a GUID. But since this code lived in the client, if multiple clients connected to one drms_server, then the GUIDs could overlap - the clients do not coordinate with each other. This was fixed by moving the GUID-generation code into the server DRMS library (used by drms_server). + The auto-record-link-following code was not properly handling the case where the only type of link involved was a static one. Fixed. + Fixed some leaks in libdsds. + Fixed a bug in unpublish.pl. The order of two arguments to the SubTableMgr constructor were accidentally swapped. + Modified export code to convert all user-specified whitespace (in the out-file name specifier) to underscores. It now also removes whitespaces from the output file names that result from number-to-string conversions. + Copied the 1.5 version of json_format_string() from the json-generation library to our copy of this library (version 0.9). The original version of this function was very inefficient, causing unwarranted stress on our web server. + Fixed a crash in show_info. A record-set specification was being freed to soon, resulting in the use of freed memory. + Copied the NOAA_ARS keyword values from hmi.Mharp_720s to the definitive sharp series, hmi.sharp_720s.