Return to release.notes CVS log

Up to [Development] / JSOC / CM / V8.10

CM/V8.10/release.notes

                       Release Notes JSOC V8.10        2SEP2015
                       -----------------------        ---------


A release is a set of binary and text files, each having a specific version. The release itself is
also versioned to facilitate the identification of the release. For example, release 1.3 may 
contain fileA#1.8, fileB#1.2, and fileC#2.2 and release 1.4 may contain fileA#2.5, fileB#2.1, 
and fileC#2.9. JSOC releases are similarly versioned and contain a set of such files.  JSOC release
code is guaranteed to compile on cluster nodes (e.g., n04 and solar3).  At the time of the 
creation of the release, the versions of each file will be very recent - most will be the
"head" versions. The set of files is stable. They have been tested and verified to work properly
on the supported systems. As time passes, newer versions of files will be committed to the 
code repository and there is no guarantee that these changes will not destabilize JSOC 
(i.e., they may cause JSOC to no longer compile or execute properly).  

There are several ways to use this release.  If you wish to simply use pre-built
binaries, you can use the production binaries, which are located at 
/home/jsoc/cvs/Development/JSOC (The release binaries are actually in
/home/jsoc/cvs/JSOC, but as our production code changes more quickly
than releases are generated, we put critical fixes in the
"Development" tree. To be sure you use code that has been built with
these critical fixes, you'll need to use the "Development" tree. As
time passes our production code will stabilize. When that happens, you should use
/home/jsoc/cvs/JSOC. But for now, you should use the "Development"
tree.). Every time a release is created, the binaries in this location get updated
by the jsoc user. To use these binaries, you can run them directly, 
"/home/jsoc/cvs/JSOC/Development/bin/linux_x86_64/show_info -j hmi.M_45s", 
for example. Or you can put /home/jsoc/cvs/JSOC/Development/bin/linux_x86_64 in 
your path environment variable.

If instead you want to work with stable source files, then you must have a local copy (e.g., 
in your home directory) of the source code in the CVS code repository. You must "build" or "make" 
the code this local code before it can be used. You will want to work with a local copy if you 
plan on making changes to the repository files (i.e., you plan on doing development). Changes you 
make to your local copy are not visible to other users until you commit those changes back to the 
CVS repository.  

Obtaining the Release
---------------------
To update your working directory to this release, or to check-out this release anew, 
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit (please see the section entitled
Update Your Existing Working Directory ("Sandbox")). Please keep in mind that
users may have modified files since the release was created, so use of the 
scripts documented in the web page may result in a working directory whose
content is not identical to the release.  

Additional Info
---------------
Use the Apache CVS gui to see diffs between file revisions. For example, go to http://jsoc2.stanford.edu/cvs/JSOC/base/drms/
and click on the name in the File column and then click on "diffs to previous #" to see the diffs.

Changes since previous release (V8.8 -  April 13, 2015)
----------------------------------------------------------

NEW FEATURES:
- DRMS now has a multi-threaded SUMS daemon (MT SUMS). This daemon serves SUM_infoArray() requests and is meant to be a replacement for the original SUMS implementation. It spawns a new thread for each SUM_infoArray() request, communicating with the client via a single socket connection. The communication does not involve RPC (it utilizes JSON messages). To use this feature, you must set the config.local parameter SUMS_USEMTSUMS to 1, and you must have a Python 3.3, or greater, environment available to the server code. You must also set other config.local parameters (see below). The daemon, sumsd.py, must be manually started before clients can use it. The SUMS start and stop scripts have not yet been modified to support sumsd.py (TBD in a future release). Needs Py 3.3 or greater.
- A Python class in the localization script was converted to a "new-style" class so that it could be sub-classed.
- Implemented a new DRMS API function - drms_open_recordswithkeys(). It takes an argument that is a list of keywords. It returns a record-set structure that contains records that contain ONLY the keywords specified. Because many series have lots of keywords, use of this function can result in much smaller memory footprint compared to drms_open_records().
- jsoc_export_SU_as_is was modified to use SUM_infoArray() instead of SUM_info() so that multiple SUs could be processed at once.
- timeio, the time-support library, was updated for the June 30, 2015 leap second.
- show_info now handles invalid SUs provided in the sunum=XXX parameter.
- Since all series in the dsds namespace were removed, the dsds-exclusion regular expression used to pre-fill the Seriesname filter was removed too.
- Implemented a new script used in conjunction with findsessionrecs.pl, the program that finds all SUs created during a specified time interval, so that the path for each SU located is now printed.
- Added JSOC-specific projects to the code base: meanpf, polarfield, export_as_maproj.

LOCALIZATION:
- There are a few new config.local parameters for MT SUMS: 
  SUMS_USEMTSUMS - set to 1 if you want to use the MT SUMS service. If you do so, then you will need to manually start/stop the MT SUMS daemon (sumsd.py) as needed.
  SUMSD_MAX_THREADS - the maximum number of simultaneous SUM_infoArray() requests allowed.
  SUMSD_LISTENPORT - the MT SUMS server port through which clients connect to the MT SUMS daemon.

DEFECTS FIXED:
- vso_sum_alloc and vso_sum_put were modified so that they close the SUMS connection upon failure (changes from Niles Oien).
- Some code in lib DRMS was writing error messages to stdout. This was breaking CGI programs.
- unpublish.pl had been accidentally checked-in with a debug modification in it. This modification was reverted.
- The export manager, jsoc_export_manage, was modified so that the export system once again supports uploads of files containing multiple record-set specifications that identify records of a single series.
- The algorithm that determines the next SUMS partition to be used for new SUs was modified so that SUs are more evenly distributed among partitions.
- A crasher in show_info was fixed having to do with an uninitialized variable in an error path.
- A variable that should have been dereferenced before use was not being dereferenced. Now it is dereferenced.
- Updated the publist.py program (used by a CGI, but also used on the command line), which prints series-publication information, to use a Python Arguments class to process arguments. The previous argument-processing code implemented position-dependent arguments. It was confusing argument positions, so we jettisoned position-dependency altogether.
- The export-manager code was modified to properly handle record-set specifications containing single quotes.
- The export-manager code was modified to properly handle processing arguments that are missing values (like "grid="). In such cases, a default is used, if it exists, or the argument is ignored if there is no default.
- A check for NULL in the export-manager code is now made before using the processing-list structure.
- Before freeing the supplemental DRMS environment, a check is now made in the export-manager code to ensure that the supplemental environment was indeed initialized.
- jsoc_fetch now properly handles record-set specifications that contain single quotes.
- Added some linker flags to the make files for the SUMS applications so that SUMS builds at all remote DRMS sites.
- A bug on the export page was fixed. Before the fix, it was possible for the "Check Params for Export" to get stuck in the "Red State" when in fact submitting the export request would have been otherwise possible (because all parameters were suitable for an export).
- Fixed a bug in code that locates the stop file for IRIS processing.
- For render_image, use the T_REC series keyword if T_OBS is not defined.
- Modified the source-code interface script (dlsource.pl) to skip updating obsolete files if they do not exist in the working directory.