JSOC File Structure

Background

On October 1, 2007, a newly reorganized JSOC code tree became effective. The new code tree differs from the original tree's structure. The original tree and the new tree coexist in CVS (new is in the "JSOC" module, old is in the "jsoc" module), but moving forward, changes to the old tree should cease with future changes being constrained to the new tree (with some exceptions). The old tree is becoming read-only

As of October 1, all developers should start using the new JSOC CVS tree, and discontinue using the old one. Production users who run binaries, and are not changing source code, should continue to do what they are currently doing. If you have any questions about what to do please talk to Art Amezcua.

If you are currently modifying files in the old tree, finish up there, submit those changes, and let Art know so he can duplicate those changes in the new tree. After that, work only in the new tree (one or two of you will continue to work in the old tree on occasion, but everybody else should work in the new tree).

To move over to the new tree, you have to do a couple of things that basically tell CVS you want to use the JSOC module (all caps) instead of the jsoc module (all lower, the one you've been using heretofore). These are changes you'll need to make in one of your startup scripts (like ~/.cshrc). The script(s) that you modify is the same one the you originally modified to set up your system for accessing the original jsoc source tree.

Change the JSOCROOT environment variable to contain $HOME/cvs/JSOC (note, upper case JSOC, not lower case).
Add $JSOCROOT/scripts to your path variable. It might already be defined. This will allow you to access certain drms/jsoc scripts from any directory, and it will also give you access to the drms/jsoc manpages.
Modify your cvsstatus alias to point to $JSOCROOT/base/util/scripts/cvsstatus.pl: alias cvsstatus "$JSOCROOT/base/util/scripts/cvsstatus.pl"
Optional: Add a link named ~/JSOC to ~/cvs/JSOC: cd ~; ln -s ~/cvs/JSOC JSOC

To check out the new tree:

cd ~/cvs
cvs checkout JSOC
cd $JSOCROOT
./configure
make

Alternatively, after checking out JSOC, you can run jsoc_update.pl from $JSOCROOT.

General Tree Structure

In general, code is divided into projects, with each project's source code and data occupying a single subdirectory of $JSOCROOT. The base system's code and data reside in $JSOCROOT/base, and the institution-specific projects reside in $JSOCROOT/proj. Inside each project's directory are typically located up to six subdirectories: apps, data, doc, libs, scripts, test. Inside each of these directories, files can be grouped into arbitrary subdirectories, or they may simply all reside directly in the directory if desired. For example, a large project might have several scripts having to do with downloading data from a server, and it might also have several scripts needed for ingesting data into the database. It might, therefore, be logical to create one directory for 'download' scripts, and one directory for 'ingest' scripts. In general, apps contains files that will produce, via the make system, executables. data contains binary and text files that these apps read for input. libs contains libraries against which the apps link. scripts contains script code that does not need to be compiled, such as perl and csh scripts. test contains files, executable or otherwise, that are needed for regression testing of the apps and libs within the parent directory.

New JSOC Tree View

At the top-most level, the directory structure looks like this after running 'cvs checkout JSOC' and 'configure':

In greater detail, the structure of $JSOCROOT/base looks like:

And, in greater detail, the structure of $JSOCROOT/proj looks like:

Optional Directories

Additional and optional directories include the following. These directories can be created as needed:

CVS/JSOC/{base|proj}/.../apps/data
CVS/JSOC/{base|proj}/.../apps/test
CVS/JSOC/{base|proj}/.../libs/<libname>/data
CVS/JSOC/{base|proj}/.../libs/<libname>/test

These directories contain JSOC-specific data files (such as tables), jsd files, and test programs. If they exist, they appear as nodes of "apps" and "libs" subtrees.

Generated Directories (not part of the CVS repository)

Several directories shown in the above tree view are created as part of the configuration and binary-build process. 'make' creates the following directories to contain all generated binary files:

CVS/JSOC/_linux_ia32
CVS/JSOC/_linux_x86_64

Binary output resides at various nodes within these two directories. 'make' also creates these directories:

CVS/JSOC/bin
CVS/JSOC/lib

These directories contain links to the executables and libraries built by 'make' and which reside at various nodes within the output tree. It is important to avoid name conflicts between binaries and between libraries. Conflicts are resolved by the last link created silently overwriting all others previously created with the same name.

The configure script creates these directories.

CVS/JSOC/jsds
CVS/JSOC/man
CVS/JSOC/scripts
CVS/JSOC/base/include

They contain links to jsd files, man pages, scripts, and base header files that live at various nodes within the source tree. It is important to avoid name conflicts. Conflicts are resolved by the FIRST same-named link "winning". The configure script outputs an error message in the event of a conflict.

Library Dependencies

There should only be five types of dependencies of binaries on libraries:

Any binary whose source resides in the path base/<a subpath>/apps or base/<a subpath>/libs can depend on any library whose source resides in base/<a subpath>/libs. It can also depend on any library whose source resides within a 'libs' directory off of a parent directory. So, base/drms/apps/drms_server.c can depend on the library source in base/libs/misc (base/libs is a parent of base/drms/apps/drms_server.c).
Any binary whose source resides in the path base/drms can depend on any library whose source resides in base/sums.
Any binary whose source resides in the path base/util can depend on any library whose source resides in base/drms.
Any binary whose source resides in the path proj/<a subpath>/apps or proj/<a subpath>/libs can depend on any library whose source resides in proj/<a subpath>/libs. It can also depend on any library whose source resides within a 'libs' directory off of a parent directory.
Any binary whose source resides in the path proj can depend on any library whose source resides in base.

Using CVS to Checkout the Base Files

To facilitate checking out just base code, a the CVS admin file "modules" has been modified to add two CVS modules: DRMS and JSOC. Type the following to checkout only the base (ie., DRMS) part of JSOC:

cd $JSOCROOT/..
cvs checkout DRMS

Type the following to checkout all JSOC files:

cvs checkout JSOC

To update the source code of these modules:

cd to your JSOC tree, which may be $JSOCROOT
jsoc_sync.pl

To update the source code in $JSOCROOT and build the default binaries:

cd $JSOCROOT
jsoc_update.pl