Release Notes JSOC V8.1 16SEP013 ----------------------- -------- A release is a set of files, each having a specific version. And a release typcially has a version number because over time you have newer and newer releases of the same product. For example, a hypothetical 1.3 release may contain fileA#1.8, fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (eg., n00, n02). At the time of the creation of the release, the release versions of each file will be the most recent. But as time passes, newer versions of some files will be made, and there is no guarantee that these changes will not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can simply use the production binaries, which are located at /home/jsoc/cvs/Development/JSOC (The release binaries are actually in /home/jsoc/cvs/JSOC, but as our production code changes emore quickly than releases are generated, we put critical fixes in the "Development" tree. To be sure you use code that has been built with these critical fixes, you'll need to use the "Development" tree. As time passes our production code will stabilize. When that happens, you should use /home/jsoc/cvs/JSOC. But for now, you should use the "Development" tree.). Every time a release is created, the binaries in this location get updated. Only the jsoc user can update these binaries. You could run /home/production/cvs/JSOC/Development/bin/linux_x86_64/show_keys, for example. If instead you want to work with stable source files, then you must have a sandbox, which is a local copy (in your home directory) of the files in the cvs depot. You would probably want to work with a sandbox if you plan on making eventual changes to the depot files. Changes you make to your sandbox files are not visible to other users until you "commit" those changes back to the cvs depot. Please see "If You Don't Have a Sandbox" below for more information on how to create a sandbox. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit (please see the section entitled Update Your Existing Working Directory ("Sandbox")). Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. Additional Info --------------- Use the Apache cvs gui to see the diffs between file revisions. For example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V8.0 - April 22, 2012) ---------------------------------------------------------- NEW FEATURES: + Changed all JSOC scripts that use recent Perl binaries to #! the activeperl link, which then points to the most recent version of ActiveState Perl. + Added drms_dropskeys, a DRMS module that allows one to delete keywords from series. + set_info now allows the user to specify the keyword names in a case-insensitive manner. + Writes to the HISTORY and COMMENT keyowrds are now appended to the end of the current string by default and no longer overwrite the existing values. + Records automatically opened because they are linked to are now automatically closed when the parent records are closed. + Modified the @file-notation code to support ~user names in file paths. + Optimized the first-last query function. If there is no where clause on a prime-key keyword, then do not create a temporary table for that keyword - skip it. + DRMS now tracks the number of SUMS connections, and rejects new connections if the number exceeds the maximum allowed by SUMS. + DRMS now tracks each and every SUNUM that it sends to SUMS via SUM_get(). It will not allow the module to continue if there is a pending SUM_get() on an SUNUM, and that SUNUM is part of a new SUM_get() call. + Rewrote part of sdo_slony1_dump.sh, the code that creates the sql file containing a dump of the series table of the series being subscribed to, to facilitate synchronization between db and non-db commands. There was no way to do this from the original bash script, so that script now calls a new Perl script that contains the critical section. The Perl script maintains an open db connection and alternates between db and non-db commands while the db connection is held open. + Series-replication clients can now select which Perl binary to use in the client subscription code. + The Slony-log parser (which generates NetDRMS-site-specific log files) supports record deletions. If a record is deleted from a published series the deletion will be propagated to remote sites subscribed to that series. + Added the -e flag to show_info. When provided, this flag will cause the record-set specifications to be parsed into constituent parts (series name, filter, segment list, etc.). + jsoc_export_as_images now supports record limits. + render_image now has a -u flag. + cmdparams now supports the ARG_STRINGS type of module argument. This allows the user to pass arrays of strings to a program. + Created scripts to ingest IRIS FDS data-product files into iris.fds, to ingest orbit-data-file content into iris.orbit_vectors, and to ingest saahlz-data-file content into iris.saa_hlz. + Added ingestdata which ingests keyword values for multiple records. Unlike set_info, the new keywords values can differ between records. + Move the infoarray.ans-clean-up code from sum_rm to ??. + set_info now supports piped input. + Added timeslot, a module to convert a time value to a slot number (for time-slotted keywords only). + The watch-lev0 system no longer assumes a default notification domain of solarmail for external users. + The globalhs make logic ensures that all files in the project: 1. have the same CVS version, and 2. have the latest CVS version. + The redundant code in obs2helio.c and mag2helio.c has been consolidated into one source file. + Improved the error handling in the HARP-generation code. + Added mdidailysynframe, a module that produces daily synoptic maps from MDI line-of-sight magnetograms. + Added mdisynop, a module that produces synoptic charts from MDI line-of-sight magnetograms. + Added a new mtrack option that allows the caller to specify that the doppler signal due to solar differential rotation should be removed from the input prior to processing. This will facilitate the analysis of e.g. GONG site or Mt Wilson data where average subtraction is not appropriate. + Added some new rdday features that allow the user to choose rdccoverversion and rejection-list file when testing, and added some code to clean up files. + Fixed bad records in the various sharp and harp series. Modified the script fixsharp1.pl and added the script fixharp3.pl to do this work. + Added a linux_avx machine to the list of machines on which jsoc_update.pl builds. + Modified the pipeline manager code so that it runs from the production source/binary tree. + Created a gate for the HMI HARP web images so that the code that generates them runs in the production pipeline. DEFECTS FIXED: + DRMS no longer requires the existence of the drms.minvers table. If the table does not exist, then there is no requirement that the DRMS module's version have a minimum value. + Replaced the slow query used by the series-existence-check code with a faster query. + Fixed some places in the DRMS code that were not recognizing time intervals. + drms_recordset_extractfilter() was sometimes returning an uninitialized status value. It always intializes this value now. + Make sure the drms_recordset_fetchnext() returns kRecChunking_NoMoreRecs if a previous call exhausted all records. + The server-side subscription code now checks for errors when it calls the createtabstructure module. + jsoc_export_make_index now sets links to tar files in the index.html file. + jsoc_export_manage's use of tar's --remove-files was not working on solar3. Removed --remove-files and added code to delete the files with rm. + Changed the queue that the export system uses. The old queue contained nodes that had mount issues. + Fixed a problem with the export-notify code (code that sends an email message to the export initiator notifying of completion). + The export code now checks for a missing export request id when performing an exp_status request. + Reduced the limit on export payload size if is the case that at least one segment will be uncompressed upon export. + Remove a redundant function definition from cmdparams. This was tripping up the gcc build. + Fixed a memory smasher in libmisc.a. + base_strcatalloc() was not always allocating a sufficiently large buffer - fixed a memory smasher. + Initialized the _req member in the drmsArgs class. + Documented the -k flag of delete_series. + The "autobang" string, "[! 1=1 !]", was previously omitted when set_info was called with the -b flag. + Restored original behavior of suppressing output when the -q flag is set and no records are returned. + Fixed a crasher in show_info. There was a call to the show_info_return() function from a function other than DoIt(). This resulted in show_info_return() being called twice, which double-freed. Instead, return from the function, then call show_info_return() only once. + Fixed the -r flag of rebin. + Plugged memory leaks in iorbit code. + Fixed a bug in mtrack in the code that determines the Carrington elements. The bug occurred in certain cases where the image at the midpoint of the tracking interval was missing. This bug had affected the time-distance pipeline, but not the ring-diagram pipeline. + Fixed some leaks in im_patch. + Fixed an uninitialized variable bug in dlsource.pl. + Added some code to jsoc_update.pl to handle an unexpected mtab entry. + Fix the code in jsoc_update.pl that obtains the platform on remote machines. + Fix a bug in jsoc_update.pl that checks for connectivity to the build machine. + Fix a bug in the passing of array arguments to a function in jsoc_update.pl. + Remove the perl switch statement in sumstapestat.pl. When we installed the latest ActiveState Perl (our Perl installation got corrupted by CPAN), we lost support for switches (ActiveState doesn't support them). + Modifed the boundaries between the values that control the green/yellow/red indicator colors in the processing-status web page.