Release Notes JSOC V4.3 26May2008 ----------------------- -------- A release is a set of files, each having a specific version. And a release typcially has a version number because over time you have newer and newer releases of the same product. For example, a hypothetical 1.3 release may contain fileA#1.8, fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. JSOC releases are similarly versioned and contain a set of such files. JSOC release code is guaranteed to compile on cluster nodes (eg., n00, n02). The resulting binaries have been minimally tested. At the time of the creation of the release, the release versions of each file will be the most recent. But as time passes, newer versions of some files will be made, and there is no guarantee that these changes will not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute properly). There are several ways to use this release. If you wish to simply use pre-built binaries, you can simply use the production binaries, which are located at /home/production/cvs/JSOC. Every time a release is created, the binaries in this location get updated. Only the production user can update these binaries. So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example. If instead you want to work with stable source files, then you must have a sandbox, which is a local copy (in your home directory) of the files in the cvs depot. You would probably want to work with a sandbox if you plan on making eventual changes to the depot files. Changes you make to your sandbox files are not visible to other users until you "commit" those changes back to the cvs depot. Please see "If You Don't Have a Sandbox" below for more information on how to create a sandbox. There is also a "working" release which resides in in /home/jsoc/cvs/JSOC. New files may be placed here and existing files may be edited for common use before the next official release. Each time a release gets created, the source and binaries of the working release get updated. WARNING: the files you see here may not be stable since by the time you see them, another user may have edited them. Only the production release is guaranteed to be stable and unchanged between releases. Obtaining the Release --------------------- To update your working directory to this release, or to check-out this release anew, please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that users may have modified files since the release was created, so use of the scripts documented in the web page may result in a working directory whose content is not identical to the release. If updating, you can supply the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the latest release. This will ensure that your working directory has the exact, latest release versions of the files (eg., jsoc_sync.csh -R). If checking-out, you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command to achieve the analogous result, but for a fresh checkout. WARNING: if you use the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update your sources thereafter. Use of "-R" or "-r Ver_LATEST" will result in a cvs "sticky flag" being set. jsoc_update.pl and jsoc_sync.pl clear this sticky flag. Additional Info --------------- If you are unfamiliar with the use of cvs see the file: JSOC/CM/working_with_sandbox.txt. There's a linux4 cvs gui at xim:/usr/bin/lincvs Also on our jsoc web page: http://jsoc.stanford.edu/cvs/JSOC/ Use the Apache cvs gui to see the diffs. For example, go to http://jsoc.stanford.edu/cvs/JSOC/base/drms/ and click on the name in the File column and then click on "diffs to previous #" to see the diffs. Changes since previous release (V4.2 - April 8, 2008) ----------------------------------------------------- * Addition of Perl interface for SUMSAPI. * The JSOC_MACHINE variable is now set in the configure script (some users don't have JSOC_MACHINE set by the standard means). * Add make support for openMP and libgsl.a. * Some fixes so that mac (partially) builds. * Add make support for C modules that need to link against Fortran object files and libraries. * Fix a few inconsistencies where drms_array.c was rounding floating point numbers to integer values. Before the fix, rounding was inconsistent - sometimes truncation, sometimes rounding up, rounding down, or rounding away from zero if magnitude is x.5 or greater, etc. etc. Afterward, rounding is implemented with Linux's round() function. * Implemented bzero/bscale plan (developed by Phl, Rick, Karen, and Art) * Fix problems with CFITSIO char data type, which was being treated as unsigned char, but DRMS uses signed char. * Implement segment-list specifiers (you can now add a list of comma-separated segment names to a record-set query, and the returned record-set will contain those segments only). * Added support for compression of FITS files. * Added drms_keyword_snprintfval() - a function that prints formatted keyword values. * Bug fix in drms_setkey() function - for slotted TIME keyword values, DRMS confused string representations with the internal double representation. * Track down and remove an 'order by' statement from the select statement that selects the keyword information to be placed into the template record. Did this because re-ordering means that DRMS is out of sync with psql, and so that the order in which the .jsd specifies keywords matches what appears in the keyword HContainer_t. * Changed the default print format of _index keywords to %lld (from %d) because _index keywords are long long. * Implemented generic SLOT slotted-keywords. * The format field of TIME keywords now means 'precision' and the unit field means 'time zone'. * Add a version column to the *.drms_series tables. This allows code to conditionally execute, depending on the version of the .jsd used to create the series. * Use SDO_EPOCH, not SOHO_EPOCH, as the macro that returns the internal time in the _SDO_to_DRMS_time() call. The latter includes a function call of sscan_time(). Also, make _SDO_to_DRMS_time() a static inline function. * drms_record.c: Fixed size computation for recnum in drms_query_string; added recnum to allowed keywords in keylist for drms_keylist_memsize(). fixed a memory problem. * added drms_keylist_memsize() to estimate memory size of a keylist. it is used in drms_query_string() to generate limit for a given keylist query * Make the RequestID of the jsoc export be a long long, not a string. Move the seglist specification from a parameter to jsoc_export and drms_recordset_mapexport() to a segment-list specification that is appeneded to the end of the recordset query. * Grant delete privilege to sumsadmin when creating new series * Changed DRMS_LOG_RETENTION to 10 and used it in drms_storageunit.c. It was not used anywhere before. * _step keywords that use named epochs (like 'TSEQ_EPOCH') can now be of data type time, in addition to type string * Add the TSEQ_EPOCH epoch define. Can be used from within .jsd file * libdsds.so: Fix a bug in the conversion of SDS_LONG to DRMS_TYPE_LONGLONG. The bug was that on 32-bit machines, SDS_LONG was a 32-bit number, not a 64-bit number, so it was interpreted incorrecly. * Several SUMS changes, but CVS commit comments were cryptic. * sum_open.c: change time out to 3600 sec; expand timeout handling; default dsname to ; * sum_rpc.h: add new drives and make sure rpc prog # is hex and not decimal. * SUMLIB_NC_PaRequest_AP.pgc: set limit to 50000 * Updates to extcvscomm.pl (tool that organizes and prints cvs commit comments): Update this to print out ranges - greater than or equal to a date, less than or equal to a date, both greater than or equal to a date and less than or equal to a second date. * ingest_dsds_a.c: Added drms_free_array of input data array. * Add new program - Get next JSOC RequestID - this returns the next number (a unique number) in a sequence of numbers. Used for generating unique Request IDs, which are used to ID export requests. * Rearrange lookdata elements for compactness * SOURCE_ENV_FOR_HK_DAYFILE_DECODE - Checked in file used to set environment variable settings like project name and data type name for data series and setting pointers to hk configuration files and hk JSOC version map files. * decode_dayfile.c: Added change to process Signed byte value into a short variable. View update difference below * Added C program for decoding housekeeping dayfiles and writing hk keyword names and values to DRMS. When run program with the -p flag write report to standard output on the keyword name and values. * Implementation of module to ingest orbit position and velocity vectors into sdo.fds_orbit_vectors. * ingest_lev0.c: different ds for hmi and aia and make fsn 30 bits * write_hk_to_drms.c:Added lines to process SIGNED BYTE values using short variable instead of char variable because non-valid chars cause database to not load data. * ddf.pl: decode day file script is updated to run as cron job or at command line on JSOC production account on d00; Sets values to process dayfiles for either hmi,aia or sdo dayfiles for today in a cron job or at command line. * df_apid_list_day_files: Used by ingest_hsb_dayfile.pl script to know which apid to process. * gdfdrms.c: get dayfiles from DRMS * gdfdrms.pl: Gets dayfiles from DRMS & sends to decode_dayfile C executable. Used to gather Level 0 high speed bus,LMSAL,and SDO formatted. * ingest_LM_dayfile.pl: Script used to load into hmi.hk_dayfile or aia.hk_dayfile DRMS data series; Updated code for parsing apid from LMSAL formatted dayfile. * ingest_hsb_dayfile.pl: script which can ingest housekeeping dayfiles to DRMS. * Add jennifer and rock to email list that receives error messages during download of FDS files * o2helio: fix leaked input array memory; fix elapsed-time calculation; fixed crashes due to freeing strings owned by cmdparams; change name of input/output params to in/out; add cmd-line param to specify input/output seg names. * arithtool.c: When copying keyword values from one series to another, don't copy _index values (which get copied when you copy the corresponding slotted keyword values).