Release Notes JSOC V5.7 11MAR2010 ----------------------- --------- A release is a set of files, each having a specific version. And a release typcially has a version number because over time you have newer and newer releases of the same product. For example, a hypothetical 1.3 release may contain fileA#1.8, fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (eg., n00, n02). The resulting binaries have been minimally tested. At the time of the creation of the release, the release versions of each file will be the most recent. But as time passes, newer versions of some files will be made, and there is no guarantee that these changes will not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can simply use the production binaries, which are located at /home/production/cvs/JSOC. Every time a release is created, the binaries in this location get updated. Only the production user can update these binaries. So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example. If instead you want to work with stable source files, then you must have a sandbox, which is a local copy (in your home directory) of the files in the cvs depot. You would probably want to work with a sandbox if you plan on making eventual changes to the depot files. Changes you make to your sandbox files are not visible to other users until you "commit" those changes back to the cvs depot. Please see "If You Don't Have a Sandbox" below for more information on how to create a sandbox. There is also a "working" release which resides in in /home/jsoc/cvs/JSOC. New files may be placed here and existing files may be edited for common use before the next official release. Each time a release gets created, the source and binaries of the working release get updated. WARNING: the files you see here may not be stable since by the time you see them, another user may have edited them. Only the production release is guaranteed to be stable and unchanged between releases. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. If updating, you can supply the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the latest release. This will ensure that your working directory has the exact, latest release versions of the files (eg., jsoc_sync.csh -R). If checking-out, you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command to achieve the analogous result, but for a fresh checkout. WARNING: if you use the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update your sources thereafter. Use of "-R" or "-r Ver_LATEST" will result in a cvs "sticky flag" being set. jsoc_update.pl and jsoc_sync.pl clear this sticky flag. Additional Info --------------- If you are unfamiliar with the use of cvs see the file: JSOC/CM/working_with_sandbox.txt. There's a linux4 cvs gui at xim:/usr/bin/lincvs Also on our jsoc web page: http://jsoc.stanford.edu/cvs/JSOC/ Use the Apache cvs gui to see the diffs. For example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V5.6 - Jan 28, 2010) -------------------------------------------------------- NEW FEATURES: + NetDRMS users can request subdirectories from the Stanford proj directories (e.g., JSOC/proj/util). These users will need to contact Stanford and obtain the source subdirectories desired. They will then place these subdirectories in the JSOC/proj directory of their NetDRMS release. To properly build targets in these subdirectories, using the JSOC make system, the user needs to add entries to this configuration file - one entry for each subdirectory that contains source code to be compiled. Each entry is a space-separated pair of strings: the string "PROJDIR" followed by a subdirectory (of the proj directory). + Modified the MOC-Product-Server download code to make it easy to redo the download of products previously downloaded. It also retries failed file downloads 4 times before giving up. + Moved the output of some of the build localization code to JSOC/localization (by default). This directory is overridable. + Integrated interpolation code into the Stanford CVS tree, at JSOC/proj/interpolation. + Moved the export source code from JSOC/proj/export to JSOC/base/export. Moved the json library as well (which is sued by the export code). + Added a new module that allows access to the slony logs (for distribution) that are archived in SUMS. + Allow FITS files containing slices of images to "grow" in the number of slices over time. Such a file opened for writing is initialized to have 1 slice. As slices are written to the file, the total number of slices increases as needed. + New API function: drms_link_set(). This function facilitates the setting of links and does not require the user to provide unnecessary information needed for the previous API functions (drms_setlink_dynamic and drms_setlink_static). + Added a new script that archives the slony replication logs. + Implemented client side of subscribe_series' unsubscribe feature. This will delete the series being removed from subscription (IT IS SUPPOSED TO WARN THE USER, BUT THAT HASN'T BEEN IMPLEMENTED YET). + Added lots of logging information to the subscribe_series (slony) code. + Added checks to the slony code for successful file transfer, copy, etc. The code no longer deletes data files if a file IO error happens. + Slony subscribe_series now recovers from partial file download, resuming on the next try from where it left off. + subscribe_series will also resume from where left off if a problem occurs that prevents it from completing. + Improve security of subscribe_series by quoting strings that end up being executed in shell scripts. + Add a flag to jsoc_fetch, requestid=NOASYNCREQUEST, that causes the program to not start an export request if data are not online (won't start asynchronous processing). + jsoc_fetch will not provide information in the returned json that says if SUMS is offline (if it is offline). + The maximum number of FITS files used by lib DRMS that will remain open was increased to 256. + FITS files now contain data and file hashes that can be used for verification. Lib DRMS verifies, upon reading them, that files haven't become corrupted. + The JSOC build version was added to the level-0 series. + Use the sdo namespace, instead of sdo_ground, now that the bird is flying. + Add a DATE keyword (indicates time of processing) to all FDS-related series. + Initial submission of farside project. BUG FIXES: + Fix a bug that was causing partial file writes of a small number of FITS files open for writing during a DRMS session. + Fixed a typo in iorbit.c + Modified slony code to use a case-insensitive lookup when searching for series names + drms_copykeys() will no longer attempt to "set" keywords that are actualy linked keywords. + Fix a bug in the DRMS record-set parsing code where a double was being treated as a string. This was causing a crash. + Fix a bug in drms_stage_records() - this function was not staging if the number of records to stage was 1. + Checks for segment and file dimension matching were removed for segments of VARDIM protocol. + Introduced file locking as a method of synchronizing the slony log parser and the log archiver. + Moved slony.env to Stanford-specific directory (it shouldn't be in the NetDRMS release). + Modify many Stanford-specific slony code bits so that the code is portable to NetDRMS sites (many changes remain to be made). + In subscribe_series and subscription_manager (slony), divide the createns.sql files into schema-specific files (the schema refers to the namespace of the series being subscribed to).