Return to release.notes CVS log

Up to [Development] / JSOC / CM / V4.4

Release notes for the Version 4.4 JSOC release

                       Release Notes JSOC V4.4         9Jun2008
                       -----------------------         --------

A release is a set of files, each having a specific version.  And a release typcially
has a version number because over time you have newer and newer releases of the 
same product.  For example, a hypothetical 1.3 release may contain fileA#1.8, 
fileB#1.2, fileC#2.2 and a 1.4 release may contain fileA#2.5, fileB#2.1, fileC#2.9. 
JSOC releases are similarly versioned and contain a set of such files.  JSOC release
code is guaranteed to compile on cluster nodes (eg., n00, n02).  The resulting binaries
have been minimally tested.  At the time of the creation of the release, the
release versions of each file will be the most recent.  But as time passes, newer versions 
of some files will be made, and there is no guarantee that these changes will
not destabilize JSOC (ie., they may cause JSOC to no longer compile or execute
properly).  

There are several ways to use this release.  If you wish to simply use pre-built
binaries, you can simply use the production binaries, which are located at 
/home/production/cvs/JSOC.  Every time a release is created, the binaries in
this location get updated.  Only the production user can update these binaries.
So, you could run /home/production/cvs/JSOC/bin/linux_x86_64/show_keys, for example.
If instead you want to work with stable source files, then you must have a sandbox,
which is a local copy (in your home directory) of the files in the cvs depot.  
You would probably want to work with a sandbox if you plan on making eventual 
changes to the depot files.  Changes you make to your sandbox files are not visible 
to other users until you "commit" those changes back to the cvs depot.  Please see
"If You Don't Have a Sandbox" below for more information on how to create a sandbox.  
There is also a "working" release which resides in in /home/jsoc/cvs/JSOC.  New 
files may be placed here and existing files may be edited for common use before the 
next official release.  Each time a release gets created, the source and binaries of 
the working release get updated.  WARNING: the files you see here may not be stable 
since by the time you see them, another user may have edited them. Only the production 
release is guaranteed to be stable and unchanged between releases.

Obtaining the Release
---------------------
To update your working directory to this release, or to check-out this release anew, 
please visit http://jsoc.stanford.edu/jsocwiki/CvsInit. Please keep in mind that
users may have modified files since the release was created, so use of the 
scripts documented in the web page may result in a working directory whose
content is not identical to the release.  If updating, you can supply 
the flag "-R" to the jsoc_update.pl and jsoc_sync.pl scripts to download the
latest release.  This will ensure that your working directory has the exact, latest
release versions of the files (eg., jsoc_sync.csh -R). If checking-out, 
you can supply the argument "-r Ver_LATEST" to the "cvs checkout" command
to achieve the analogous result, but for a fresh checkout.  WARNING: if you use 
the "-R" or "-r" flags, please use only jsoc_update.pl or jsoc_sync.pl to update 
your sources thereafter.  Use of "-R" or "-r Ver_LATEST" will result in a cvs
"sticky flag" being set.  jsoc_update.pl and jsoc_sync.pl clear this sticky flag.

Additional Info
---------------
If you are unfamiliar with the use of cvs see the file:
JSOC/CM/working_with_sandbox.txt.

There's a linux4 cvs gui at xim:/usr/bin/lincvs
Also on our jsoc web page:

http://jsoc.stanford.edu/cvs/JSOC/

Use the Apache cvs gui to see the diffs. For example, go to
http://jsoc.stanford.edu/cvs/JSOC/base/drms/
and click on the name in the File column and then click on
"diffs to previous #" to see the diffs.

Changes since previous release (V4.3 - May 26, 2008)
-----------------------------------------------------
New Features:

* TS_SLOT implementation.  In this incarnation of slotted keywords,
there is a new keyword, *_round, which determines where in slot 0 the
epoch resides.  If *_round is X seconds, then the epoch is X/2 seconds
after the beginning of slot 0.  The allows the user to refer to the
first observtion of multiple series of differing cadence by the same
date string.
* When parsing time strings within record-set queries, if no time zone
is specified, or if an invalid time zone is specified, the time zone
in the keyword->info->unit field is assumed (unit is specified in the
series' jsd).  Also, support time strings that have the format
YYYY.MM.DD_<time zone>.
* Add code to verify that keyword format fields in the .jsd are
compatible with the data type of the keyword.  If an incompatibility
is detected, a warning is printed, but the module will continue to run
to completion.
* Add drms_run script (from old jsoc source tree) to new JSOC source
tree (drms_run was not copied over during the migration for unknown
reasons).
* In jsoc_info, added units to series_struct keyword table.
* In show_info, added "-i" flag (input query?) to cause record query to be 
printed for each record.
* In lookdata.html, add sections 7 and 8 for exporting data and minor fixes sections 1-6.
* In level0 housekeeping configuration file, changed the location to
which dayfiles are written to
/tmp21/production/lev0_60d/hk_hsb_dayfile.
* In level0 housekeeping configuration file, added new environment
variables to handle writing to HK By APID data series for AIA, HMI,
and SDO data series.
* In Housekeeping decoding code, updated SDO_to_DRMS_time function to
a static function and made this match Art's function in
hmi_time_setting.c. Removed SDO_to_DRMS_time function from
decode_hk_vcdu.c
* The FDS cron script (dlfds.pl) now calls the orbit vector module
(extract_fds_statev) after downloding and ingesting the FDS data
files.
* Don't add new data to fds_orbit_vectors series if the new data are
equivalent to the existing data.  Add code to create
fds_orbit_ingesthist series if it doesn't exist.
* In Level0 image decoding,  allow compid = 128 (N,K,R=16,0,0 raw mode).
* In level 0 ingestion code, print out status on system() error.
* In write_hk_to_drms.c, added code to write to different HK by APID
data series for HMI,AIA, and SDO.
* In write_hk_to_drms.h, added defines for ranges of APIDs for HMI,AIA
and SDO packets.These values are used by code to determine which HK by
APID data series to write hk keywords to.
* aia.lev0.jsd: Draft of aia level 0 data series to switch to after 60 day test.
* hmi.lev0.jsd: Draft of jsd version to switch to after 60 day test.
* In df_apid_ds_list_for_egsefm, added values to tell
ingest_dayfile.pl script which dayfile data series to write dayfiles
from egsefm.
* In df_apid_ds_list_for_hsb, updated values for production. Used to
tell ingest_dayfile.pl script which dayfile data series to write.
* In df_apid_ds_list_for_moc, checked in mapping file which tells
ingest_dayfile script which dayfile data series to save dayfile.
* In df_apid_list_day_file_egsefm, list of apid to tell
ingest_dayfile.pl script which to ingest into DRMS dayfile series for
egsefm dayfiles.
* In df_apid_list_day_file_moc, list of apids tells ingest_dayfile.pl
script which dayfiles to load to drms dayfile data series.
* In gdfdrms.pl, ingest_LM_dayfile.pl, ingest_hsb_dayfile.pl, added
$ENV{'SUMSERVER'}="d02.Stanford.EDU".
* In ingest_dayfile.pl: Updated script to ingest xml file and dayfile
in data series. Currently does this for xml files from moc production
server.
* In ingest_dayfile.pl, updated script to write to DATE value in
dayfile data series from string value(2008.05.29) to time value(i.e.,
2008.05.29_00:00:00.000_TAI)
* In ingest_dayfile.pl, merge changes from ingest_LM_dayfile.pl and
ingest_hsb_dayfile.pl into ingest_dayfile.pl. 
* Finalize FDS download scripts - put JSOC_DBNAME and JSOC_DBUSER into
env var set code into top-level script, add seriesname parameter to
fdsIngest.pl.
* Added dlfds.pl, the new FDS cron script that reads a config file
that specifies 'development' or 'ground' or 'live'.
* Add jsd for sdo_ground.fds, and download config file for sdo ground moc products.
* Allow the fds ingest script to accept a series name parameter, and make the default sdo.fds
* Makefile and CVS modules file changes to accommodate new "cookbook"
project which contains example modules.

Bug Fixes
* Remove memory leaks in drms_names.c
* If a user specifies a TS_EQ keyword query that has a duration and
that duration is less than one slot-width, then round up to one
slot-width and print a warning.  If the duration is not a multiple of
the slot-width, then print a warning saying the duration is rounded.
* Add libfitsrw.a to makefile rules for the meta-libraries.
* Fix an error in the record-set query parsing code that invalidated
queries that contained more than one sql query.
* In modules that ingests DSDS series, fixed sign error in calculation of MJD.
* In save_packet_to_dayfile.c, added fix in check_dfd_file() function
where the file descriptor was never closed with closedir() and added
closedir(dir_p) to load_dfd_node().
* In aia.lev0_60d.jsd, with new updates to short keywords, the
create_series failed because of - in short keyword names.  Fix missing
comma on keyword line.
* Add the script path prefix to the FDS download specification file,
and remove the newline that gets added to the JSOC_MACHINE env
variable value.

Miscellaneous SUMS changes:
* dont' set pgport for datacapture machines
* add pgport setenv
* add msg "Tape %s not in T50"
* add msg Check tape_svc log
* don't set pgport for datacapture machines
* This has been replace by jsoc_sum db
* define SUMSERVER on d02 for everybody
* add getmsgimmed() for SUM_close() to clean up fd
* remove OFFSITEHOST
* add NC_PaRequest_AP_60d()
* dont' scp md5 file to other machine. will be on local dcs machine only.
* limit 2000
* new limit and no order by group_id
* initial
* take out pgport stuff. now set in sum_svc.c
* add CLOSED != -2 for unassigned tape too
* add != -2 to exclude cleaning tapes
* change ds list window size

Miscellaneous Data Capture changes:
* update processing of .parc file
* add -f to mv
* change TOTAL_TLM_VCDU to TOTAL_TLM_IM_PDU
* retry on .qac failure too
* remove file afte success on retry
* more msgs on retry
* add retry and output stdout and stderr to file
* use /usr/bin/scp
* don't copy .qac file if scp of .tlm file fails
* add printout at end of file
* fix vcdu count
* update for summary only

Miscellaneous Level 0 changes:
* add test0
* use sigalrmflg and execute code in main loop and not signal handler
* In level 0 ingestion code, update series names.
* In level 0 ingestion code, fix set_HMI_mech_values() call.
* In level 0 ingestion code, different ds for hmi and aia and make fsn 30 bits.