Release Notes JSOC V8.00 17APR013 ------------------------ -------- A release is a set of files, each having a specific version. And a release typcially has a version number because over time you have newer and newer releases of the same product. For example, a hypothetical 1.3 release may contain fileA#1.8, fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (eg., n00, n02). At the time of the creation of the release, the release versions of each file will be the most recent. But as time passes, newer versions of some files will be made, and there is no guarantee that these changes will not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can simply use the production binaries, which are located at /home/jsoc/cvs/Development/JSOC (The release binaries are actually in /home/jsoc/cvs/JSOC, but as our production code changes emore quickly than releases are generated, we put critical fixes in the "Development" tree. To be sure you use code that has been built with these critical fixes, you'll need to use the "Development" tree. As time passes our production code will stabilize. When that happens, you should use /home/jsoc/cvs/JSOC. But for now, you should use the "Development" tree.). Every time a release is created, the binaries in this location get updated. Only the jsoc user can update these binaries. You could run /home/production/cvs/JSOC/Development/bin/linux_x86_64/show_keys, for example. If instead you want to work with stable source files, then you must have a sandbox, which is a local copy (in your home directory) of the files in the cvs depot. You would probably want to work with a sandbox if you plan on making eventual changes to the depot files. Changes you make to your sandbox files are not visible to other users until you "commit" those changes back to the cvs depot. Please see "If You Don't Have a Sandbox" below for more information on how to create a sandbox. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit (please see the section entitled Update Your Existing Working Directory ("Sandbox")). Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. Additional Info --------------- Use the Apache cvs gui to see the diffs between file revisions. For example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V7.0 - August 28, 2012) ---------------------------------------------------------- NEW FEATURES: + Support for the linux-avx architecture was added. To build avx binaries for NetDRMS, you must have properly edited your config.local file. Please see config.local.template for an example. + Support for shadow tables was added (FULL JSOC RELEASE ONLY - NOT NetDRMS). A shadow table is a database table that acts as a cache for most database queries that DRMS uses when DRMS dataseries are queried. It allows DRMS to bypass doing expensive database operations. It is helpful for series that large and on which queries take a long time to complete. + The limbfit code now builds with icc only. + Added a json parser for C code (jsmn). + Added the module rawingest to be used for ingesting raw SUMS files. + Added the module jsoc_export_clone. + Added code to track "old" modules. DRMS will not let old modules run. This new code logs instances where a module was rejected. + Added support to libdsds.so for DSDS records that have no data files. + Added support to drms_count_records() for record-set queries that contain comma-separated sub-queries. + The slony-log parser (parse_slon_logs.pl) now supports UPDATE statements. If a record is updated in the original series table, that update will be propagated to the remote sites subscribed to the series. + Added a new script, addkeys.pl, that will add keywords to existing series, even if the series is currently being replicated. + Added the timestamp to all log entries written by jsoc_fetch. + Added support for cmdparams_get_mask64() - a function to retrieve 64-bit masks. + Added support to DRMS perl library for creating read-only and write-only pipes. + Added new SUMS API function - SUMLIB_PavailOn. + Several improvements were made to ingest_dsds_to_drms. + The global helioseismology code now supports CALVER64. + The global helioseimsology code now tracks source file versions. + Add support for IRIS to a bunch of code. + The limbfit code was re-organized to store many small data files into one larger TAS file. + Support for CALVER64 was added to the sharp code. + Updated the harp pipelines with changes from Mike Turmon. + Added a new module, m2m3harp. + Added a new module, aia_fixCROTA2. + Added support to the script that checks out files from the JSOC repository for a JSOC root directory that has a name other than "JSOC". + Added a script, findsessionrecs.pl, that will find all records in all series created within a time window. This is used for examining records that were created during a hardware crash. + Added a script, prodbuild.pl, that simplifies updating and building in the production CVS tree. + Added ring-diagram documentation (apps/datavg): added writing of verbose output to log segment if present, and recording of input recnum's; added logging of CalVer64 values used; added support for specification of tmid in date_time format; Documentation at http://hmi.stanford.edu/teams/rings/mod_datavg.html. + Added ring-diagram functionality (apps/gentargs): implemented Stonyhurst longitude reporting option for grids timed* Documentation at http://hmi.stanford.edu/teams/rings/mod_datavg.html. + Added ring-diagram functionality (apps/mtrack): added recording of recnums of mapped images to log; Added recording of calling params info to comment or history key; Added support for acceptance and/or rejection of records with certain values of CalVer64 (or other equivalent key); Added logging of image rejection cause summaries and CalVer64 values used; Documentation at http://hmi.stanford.edu/teams/rings/mod_datavg.html. + Added ring-diagram functionality (apps/rdcover): added pangle selection options as for datavg; Documentation at http://hmi.stanford.edu/teams/rings/mod_datavg.html. + Added time-distance functionality (apps/invert_td_hr): added option for selection of keywords to propagate, with standard list, including WCS keys for first two dimensions. + Added time-distance functionality (apps/travel_times): added option for selection of keywords to propagate from input. + Added time-distance functionality (scripts/tdsynop): changed default from test to non-test; Use hmi.TDKernels for inversion kernels; require version 1100; Use Stonyhurst rather than Carrington longitudes for targets in traveltimes (fixes problem due to rounding of slotted prime keys). DEFECTS FIXED: + In serverdefs.h: do not use Stanford-specific values as defaults, otherwise for localized builds, which may be remote sites, Stanford-specific values would be used. + Remove the requirement that a production-user table exists in the DRMS. + The "dontwait" flag that causes SUMS to asynchronously fetch storage units was disabled. The flag appears in multiple functions at multiple levels in the DRMS code. SUMS does not support asynchronous fetching. + Added code to disallow further SUMS access after DRMS times-out waiting for a SUM_get() call to resolve. + The implementation for DRMS cursory queries must strip off a SQL limit statement from an existing SQL query. In doing so, a parser was improperly searching for the limit statement, and finding a keyword named "LO_LIMIT". The code now properly finds the limit statement. + DRMS now rejects sunums of -1 when sending a list of sunums to SUMS via the SUM_infoEx() call. + DRMS now rejects the writing of a slice to a FITS file if the scaling of the slice does not match the existing scaling of the file. + A deadlock was discovered and fixed in the module-termination code. The SUMS thread was attempting to lock the server after drms_server_commit() had already done so. + There was a bug in the slony-log parser (parse_slon_logs.pl) in which it miscalculated the last-parsed log number. This was fixed. + Made all jsoc_export_as_fits arguments optional (except that either reqid or rsquery must be provided). + The export manager code (jsoc_export_manage) was improperly sending email notifications (messages announcing the completion of an export request) to email addresses formed by appending "j1.stanford.edu" to all file names in /home/jsoc. This was fixed by checking for a failure in a show_info call that wasn't being checked before. + The code that determines the current time in jsoc_fetch was fixed. + jsoc_fetch had been confusing the source and target records when exporting a linked segment. + Removed a redundant definition of drms_sprint_rec_query(). + Made fixes for bugs in jsoc_info. + Fixed issues with the exports of TAS files. + Fixed issues with font sizes in jsoc_export_as_movie. + Fixed issues with the cmdparams_get_intXX-function family. + Fixed error-reporting issues in the fitsrw_getfptr() function and in code that runs when errors occur when closing FITS files. + Fixed a buffer-overflow in libdsds.so. + Removed modify_series from our releases. - it is currently non-functional. + Fixed a bug in show_info in which the code was ignoring an error returned by drms_open_recordset(). + In show_info, removed repetitive and unneeded calls to SUM_get(). + A series filter was being prematurely freed in show_series. + Spurious error messages were removed from ingest_dsds_to_drms. + Fixed validation errors in exportdata.html, the export web page. + Fixed issues with a reset button in exportdata.html, the export web page. + Fixed issues with case-sensitivity when searching for words in exportdata.html, the export web page. + Updated the help messages in exportdata.html, the export web page. + Fixed memory leaks in the global helioseismology code. + Fixed very large memory leak in the sharp code. + We made two fixes to large numbers of records in the HARP series. + Fixed a bug in the code that maintains run logs. + Make the vfisv_filter() and invert_() declarations conform with the C99 standard. + Ring-diagram bug fixes (apps/mtrack): fixed bug in bscale-bzero overrides. + Ring-diagram bug fixes (apps/pspec3): fixed bug in test for minimum values out of scaling range (zero); Documentation at http://hmi.stanford.edu/teams/rings/mod_datavg.html. + Ring-diagram bug fixes (scripts/avg120): fixed qmask argument bug; previously was only using default value of 80000000 rather than desired value of 80004000; set pkey to T_REC instead of default T_OBS; Documentation at http://hmi.stanford.edu/teams/rings/pipe_avg120.html etc. + Ring-diagram bug fixes (scripts/rdday): improved/fixed record accounting when reprocessing or restarting; Documentation at http://hmi.stanford.edu/teams/rings/pipe_avg120.html etc. + Ring-diagram bug fixes (scripts/runrdsyn): fixed bug involved with continuation over CR boundary; added notification on last submission.