(file) Return to release.notes CVS log (file) (dir) Up to [Development] / JSOC / CM / V5.13

File: [Development] / JSOC / CM / V5.13 / release.notes (download)
Revision: 1.2, Wed Jun 8 18:54:07 2011 UTC (11 years, 11 months ago) by arta
Branch: MAIN
CVS Tags: Ver_LATEST, Ver_9-5, Ver_9-41, Ver_9-4, Ver_9-3, Ver_9-2, Ver_9-1, Ver_9-0, Ver_8-8, Ver_8-7, Ver_8-6, Ver_8-5, Ver_8-4, Ver_8-3, Ver_8-2, Ver_8-12, Ver_8-11, Ver_8-10, Ver_8-1, Ver_8-0, Ver_7-1, Ver_7-0, Ver_6-4, Ver_6-3, Ver_6-2, Ver_6-1, Ver_6-0, Ver_5-14, NetDRMS_Ver_LATEST, NetDRMS_Ver_9-5, NetDRMS_Ver_9-41, NetDRMS_Ver_9-4, NetDRMS_Ver_9-3, NetDRMS_Ver_9-2, NetDRMS_Ver_9-1, NetDRMS_Ver_9-0, NetDRMS_Ver_8-8, NetDRMS_Ver_8-7, NetDRMS_Ver_8-6, NetDRMS_Ver_8-5, NetDRMS_Ver_8-4, NetDRMS_Ver_8-12, NetDRMS_Ver_8-11, NetDRMS_Ver_8-10, HEAD
Changes since 1.1: +19 -32 lines
remove obsolete information for the release notes. things have changed, especially the method for obtaining the latest release.

                       Release Notes JSOC V5.13        24MAY2011
                       ------------------------        ---------

A release is a set of files, each having a specific version.  And a release typcially
has a version number because over time you have newer and newer releases of the 
same product.  For example, a hypothetical 1.3 release may contain fileA#1.8, 
fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. 
JSOC releases are similarly versioned and contain a set of such files.  JSOC release
code is guaranteed to compile on cluster nodes (eg., n00, n02).  At the
time of the creation of the release, the release versions of each file
will be the most recent.  But as time passes, newer versions of some
files will be made, and there is no guarantee that these changes will
not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute

There are several ways to use this release.  If you wish to simply use pre-built
binaries, you can simply use the production binaries, which are located at 
/home/jsoc/cvs/Development/JSOC (The release binaries are actually in
/home/jsoc/cvs/JSOC, but as our production code changes emore quickly
than releases are generated, we put critical fixes in the
"Development" tree. To be sure you use code that has been built with
these critical fixes, you'll need to use the "Development" tree. As
time passes our production code will stabilize. When that happens, you should use
/home/jsoc/cvs/JSOC. But for now, you should use the "Development"
tree.). Every time a release is created, the binaries in
this location get updated.  Only the jsoc user can update these binaries.
You could run /home/production/cvs/JSOC/Development/bin/linux_x86_64/show_keys, 
for example.

If instead you want to work with stable source files, then you must have a sandbox,
which is a local copy (in your home directory) of the files in the cvs depot.  
You would probably want to work with a sandbox if you plan on making eventual 
changes to the depot files.  Changes you make to your sandbox files are not visible 
to other users until you "commit" those changes back to the cvs depot.  Please see
"If You Don't Have a Sandbox" below for more information on how to create a sandbox.  

Obtaining the Release
To update your working directory to this release, or to check-out this release anew, 
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that
users may have modified files since the release was created, so use of the 
scripts documented in the web page may result in a working directory whose
content is not identical to the release.  

Additional Info
Use the Apache cvs gui to see the diffs between file revisions. For
example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/
and click on the name in the File column and then click on
"diffs to previous #" to see the diffs.

Changes since previous release (V5.12 - Dec 15, 2010)

- The NetDRMS checkout is now configurable. Users can specify which
Stanford-project directories they'd like to check-out, in addition to
the base files previously associated with the NetDRMS set of files. A
set of new check-out scripts exists that allows users to check-out or
update custom NetDRMS file sets.
- The configure script used to automatically configure the make system
to choose icc over gcc as the compiler. Now there exists a parameter
in the config.local file that allows users to override this behavior -
users can force the make system to use the desired compiler.
- The aia-lev-1.5-generation code now exists in the CVS tree.
- remapmags, d4vm, and nlfff now exist in the CVS tree.
- Support for localization of DRMSPGPORT and SUMPGPORT is now in
- Added the NDEBUG compile flag to release builds.
- New export scripts are accessible via the export web page:
jsoc_export_as_images and jsoc_export_as_movie.
- drms_sortandstage_records() has been added - it will sort all SUs by
tape number, then by file number on each tape. This is a performance
improvement that affects a single module run. SUMS will eventually do
this sorting so that it will maximize efficiency across multiple
module runs.
- The export programs now use drms_open_recordset() instead of
drms_open_records(). The latter imposes a limit on the number of
records that may be opened at one time, but the former has no limit.
- SUM_delete_series() is now called on chunks of 10000 SUs at a time
instead of being called on all SUs. The latter scheme can result in
SUMS being tied up for a long time if there is an attempt to delete a
large series.
- The mapping of tape group to partition set is now maintained in a
sums table. DRMS now passes the tape group during the SUM_alloc() and
SUM_alloc2() calls so that sums can perform this mapping.
- jsoc_export_manage was reorganized so that the export-processing
code and the protocol-specific code operate orthogonally.
- Added a test-mode argument to jsoc_fetch to simplify the testing of
jsoc_fetch via the export web page.
- localhost is now an allowed database host in DRMS modules.
- SUMS now writes a full tape's worth of data at a time (instead of
writing smaller amounts which would allow tape-swapping in between the
writing of these smaller chunks).
- New SUMS API function SUMLIB_SumsetGet() - maps the group number
input to the partition-set number.
- The SUMS database host, port and name are now configurable in the
config.local file.
- Added a new utility module - set_info. It works like set_keys,
except that it can ingest multiple files into a generic segment, and
it can ingest FITS files into a FITS-protocol segment.
- The EGSE compression library has been ported into the JSOC cvs
tree. All code that previous linked against an old, saved binary
library now link against the ported version of this library.
- Move ephemeris functions into a library, libdsdsmigr.a.
- The mag team has added new modules.
- A new CVS/build scheme was implemented - it facilitates better
control over the set of files that compose each "check-out" (e.g., the
JSOC check-out versus the NetDRMS check-out). The previous method,
using CVS "modules", has been problematic since inception. Now, the
user specifies a set of files (a "file-spec") and this list is used to
download only those files. This allows a user to check-out the NetDRMS
base set of files plus any desired sub-set of Stanford project files.
- The NetDRMS check-out scripts were modified to use the new CVS/build
- Modified exportmanage.pl (a script that runs jsoc_export_manage) to
facilitate live testing of the export system. Also, checked this
script into CVS (it was not previously under revision control).
- Derived an efficient SQL query used to generate reports on SUMS
usage. The scripts that evaluate this query do so by directly
accessing SUMS tables.
- Modified a few common queries to improve database performance.
- Add first provider script called by monitoring code. This script
ensures that essential production files are present. If a file is
missing, an email gets sent out.
- Workflow code/scripts added. These implement a system to manage
pipelines. The scripts automatically check for missing dependencies
and, much like 'make', they trigger generation of dependencies if they
are missing. Several pipelines (e.g., lev1) now use this system.

- Undefine localization defines (macros) before defining them - gcc
was complaining about redefinition.
- Fix typo in the make system -nofor-main flag used by ifort.
- Changes to db query strings to take advantage of better-performing
- drms_log was trying to obtain the sunum from the wrong column in a
db query - fixed.
- delete_series no longer assumes that the schema _jsoc exists (it
performs a check to look for slony-replication of the series table
being deleted).
- Fixed crash in ingest_from_fits -j. The code assumed the existence
of a keyword that was not guaranteed to exist.
- Fixed buffer overflow in drms_open_recordset() - a static buffer was
too small to hold what can be a very large query (if a series has lots
of keywords).
- Fixed bug in subscribe_series (slony-replication-service client
script). The "resume" feature was not resuming in the correct location
if there was a failure during the application of the downloaded sql
- In jsoc_export_manage, the code that creates the drms_run script was
modified to first disable history substitution. Before the fix, the
export-data web page was not properly handing [! !] queries.
- modify_series crash fixed - was re-using a freed buffer.
- A hard-coded -g compile flag was removed from the flatfield code.
- Make all flatfield modules build by default (when you type 'make',
all flatfield modules will now get built).
- module_flatfield will now call drms_stage_records() in a way that
will cause a retrieve from tape.
- XASSERT() was modified so that no code is generated in release
builds (which is the default build). Improper uses of assert() and
XASSERT() were fixed - the arguments to these macros no longer perform
any vital functions (since no code is generated in release code, this
must be the case).
- HMI_IQUV_averaging and HMI_observables are now built as part of the
default make.
- The broken dependency between the observables modules and the
interpolation library has been fixed.
- A memory leak was fixed in some mag modules.
- There was some hg_patch work done.
- A memory leak was removed from hmi_libdark.
- The render_image crop flag was fixed, sqrt and log scaling added.

- Please see http://jsoc.stanford.edu/trac/report/1 for a list and
description of most known bugs.
- The DRMS code that recovers from SUMS failures when the tape system
goes down is not yet working properly. SUMS needs to be modified to
provide the correct error code to DRMS - until then, if the tape
services crashes or is terminated, then DRMS modules can hang

Karen Tian
Powered by
ViewCVS 0.9.4