CVS Initialization

All of the software for the JSOC-SDP is maintained in the "cvs" software configuration management system. In order to make new code using the DRMS you will need to establish a cvs work area, load the JSOC software tree into it, configure it for the machine you are using for code development, and build the libraries and programs. Then you will be able to add your own pipeline modules or other tools.

What is CVS anyway? It's a code manager that allows check-in and check-out of programs from the same directory structure. In a nutshell, there is one official set of source-code files that resides in one or more file directories in a remote CVS server. This set is called the "repository". A developer who wishes to edit these files must first "checkout" (download) a COPY of the files to a "working directory" on their local work machine. The developer makes changes, then "commits" (uploads) the files back to the CVS server. Commiting is not the same thing as copying, since that would overwrite changes made by other developers who made changes between the checkout and commit of a file by another developer. Instead, CVS will MERGE changes back to the repository, i.e., it will determine exactly what code has changed and insert that change into the respository file. This works as long as two developers do not make conflicting changes (e.g., changing the same line of code in different ways). If a conflict occurs, the developer who is attempting to commit a change that conflicts with a previous change needs to manually edit the file in their working directory (ensuring that both their change and the previous changes are both contained within the working-directory file), and then commit the file. To learn more about CVS, read: [http://en.wikipedia.org/wiki/Concurrent_Versions_System a CVS Wiki]

The base JSOC system is revised often. The instructions for the most recent release should be consulted to make sure the following is still correct. See e.g. release

Create a CVS Working Directory

This step only needs to be done once. The normal place to put your JSOC work area is in a "cvs" subdirectory in your home directory. Under that directory put the "JSOC" tree. E.g.:

% cd
% mkdir cvs
% mkdir cvs/JSOC

Many users find it convenient to make a link to the jsoc tree. E.g.:

% cd
% ln -s cvs/JSOC .

Set Your Environment

This step also needs to be done but once. To use CVS and JSOC, you must set four environment variables:

• CVSROOT - Path to the CVS repository that contains CVS modules used by JSOC.
• CVS_RSH - Specifies the program used by the CVS client (the code that runs on your machine) to access the CVS repository.
• JSOCROOT - Path to the local JSOC tree (the place on your working machine where you want your copy of the respository files, AKA "working directory" in CVS lingo.). Setting this variable is REQUIRED for full functionality, but without it, the base system (see NetDRMS below) will still function.
• JSOC_MACHINE - Specifies the architecture of the machine that will be used to build and run JSOC programs.

In greater detail, a Stanford user should do the following. Set the CVSROOT environment variable to contain ":ext:sunroom.stanford.edu:/home/cvsuser/cvsroot". Set the CVS_RSH environment variable to contain "ssh". Set the JSOCROOT environment variable to contain "$HOME/cvs/JSOC". Set the JSOC_MACHINE environment variable to contain a string that correctly identifies your machine architecture (one of "linux_ia32", "linux_ia64", or "linux_x86_64"). It is best let a script do this for you (see below). It is best to put all these commands in a script that gets called from one of your initialization scripts (e.g., .login, .csh). Your script should look similar to the following:

setenv CVSROOT :ext:sunroom.stanford.edu:/home/cvsuser/cvsroot
setenv CVS_RSH ssh
setenv JSOCROOT $HOME/Dev/cvs/JSOC

set OS = `uname -s`
switch("$OS")
  case "Linux*":
    set CPU = `uname -m`
    breaksw
  default:
    set CPU = `uname -p`
    breaksw
endsw

if ( ! $?JSOC_MACHINE ) then
  if ( $OS == "Linux" ) then
    switch("$CPU")
    case "i686"
    case "i386"
    case "ia32"
      set JSOC_MACHINE = "linux_ia32"
      breaksw
    case "ia64"
      set JSOC_MACHINE = "linux_ia64"
      breaksw
    case "x86_64"
    case "em64t"
      set JSOC_MACHINE = "linux_x86_64"
      breaksw
    default:
      set JSOC_MACHINE = "custom"
      breaksw
    endsw
  else
     set JSOC_MACHINE = "custom"
  endif
endif

setenv JSOC_MACHINE $JSOC_MACHINE

Load the JSOC source code tree

The JSOC CVS respository is divided into two sets of files, one for each of two types of user. A Stanford user requires source code for the base DRMS system. This system provides low-level database access to data series. It also provides utility programs that perform common database-access tasks (e.g., querying the database to find information on a particular data series). In addition, a Stanford user requires access to source code that is specific to Stanford projects (e.g., parsing telemetry packets and ingesting the extracted data into data series). The second user requires the base DRMS system only. This user is not interested in building or running Stanford-specific code, but instead will be developing or using their own programs. Each set of files is identified by a CVS "module" - "JSOC" for the Stanford user, and "DRMS" for the non-Stanford user.

Each type of user must "check out" the appropriate set of JSOC files (i.e., download the files from the remote CVS server to their working directory on their local machine). This need be done only once (barring major changes to the JSOC CVS repository). Thereafter, the user can "update" their local source tree (see below). To check out the Stanford-specific set of files, a Stanford user does the following:

% cd $JSOCROOT/..
% cvs checkout -AP JSOC

To check out the base DRMS system, a user does the following:

cd $JSOCROOT/..
cvs checkout -AP DRMS

Note: If CVS asks you for a password or fails to checkout the code, make sure you have created a public and private ssh key under your ~/.ssh directory.

Update Existing Working Directory ("Sandbox")

To update the source code of these modules:

cd to your JSOC tree, which may be $JSOCROOT
jsoc_sync.csh

To update the source code in $JSOCROOT and build the default binaries:

cd $JSOCROOT
jsoc_update.csh

When others have made changes to the cvs managed software that you want to use, or when there has been a major release (you will be notified by email) you should update your set of JSOC code.

The easy way to update the JSOC code is to run the jsoc_update.csh script as follows:

% cd ~/cvs/jsoc
% jsoc_update.csh

Now you are ready to go. You may stop reading here is all you want to do is run JSOC programs.

A new make system is in place after Ver_3-5-PreRelease. If you have a cvs working directory of or before that, you'll need to run 'make clean' before 'cvs update' in order to clean up the .c files generated from .pgc files in src/base/libsum_pg.

Pipeline module source code is built in the src/pipeline directory. Look there for examples.

The jsoc_update.csh script will update all programs on all standard architectures. If it asks for passwords you will need to ask our sys-admin (action@sun) to fix your ~/.ssh files. You may want to change the order in which the builds are done by editting the order in the last few lines of this script in case your default machine type is other than the last one done in the default script.

If you do not want to use the script you can update manually with the following:

% cd ~/cvs/jsoc
% cvs update -APd
% ./configure
% make -j 4
...
%

Note: If the cvs update shows a file with a conflict, i.e. "C filename" then check with the CM or resolve the conflict.

Note: If the make file fails because it doesn't find the postgres 'ecpg' preprocessor,

• then you'll need to install the postgresql-devel package. E.g. "sudo yum install postgresql-devel"

Note: JSOC compilation assumes the following RPMs are installed:

• postgresql
• postgresql-devel
• postgresql-libs
• openssl-devel

Altering default 'make' behavior

Running 'make' will perform a 'default make' which generates a subset of the possible binaries. All compiler warnings will be suppressed and the binaries will be optimized for speed and contain no debug information. To read how to tune this behavior [http://jsoc.stanford.edu/alterdefmake.html click here].