All JSOC Modules Common Interface
[Section 1. Programs, DRMS Modules, Shell Scripts]

Common main program used by all DRMS modules. More...

Collaboration diagram for All JSOC Modules Common Interface:

Synopsis:

<module> [GEN_FLAGS] [SESSION_ARGUMENTS] [MODULE_ARGUMENTS]

Execute the named module from a shell. When a module is executed from the command line, all argument values listed in the module are supplied from (in order of precedence): values on the command line; values in a named parameter file; or the default values specified in the module source, if any. If an argument cannot be otherwise evaluated, a notification message is issued and the driver program exits. If all the argument values exist, a DRMS session is opened or a connection is made to an existing DRMS server as appropriate, and the remaineder of the module code is executed. At the end of module execution, the driver disconnects from DRMS with or without commits to the database depending on the exit status of the module. No commits are ever made when a module aborts or is killed on a signal.

Modules are expected to run as functions (or subroutines) called by a main driver program that parses the argument list from the command line and environment, and sets up the drms->env environment for communication with the DRMS database via the DRMS API. Different driver programs can be made available, depending on the environment. Two such programs (in the form of libraries) are provided as part of the base system, jsoc_main.a and jsoc_main_sock.a (see jsoc_main.h). The former provides a direct connection to DRMS, for use by self-contained modules. The latter provides communication via a socket; it can be used for multiple modules connecting to a common server, as in a script. When multiple clients connect to a common server, commits to the database will only occur at the successful close of the server. Modules linked against jsoc_main.a are given their basename for the executable, while those linked against jsoc_main_sock have "_sock" appended to the basename. For example, a module named myprogram.c would produce executables named myprogram and myprogram_sock. (Additional drivers to produce, for example, CGI scripts such as myprogram_cgi can also be provided.)

Socket-connect vs direct-connect modules

In socket connect mode, a module talks to drms_server - Manages a database session and passes database requests to database from client modules program via a socket. Concurrent access is protected by locks drms_lock_server() and drms_unlock_server(). The following command codes employ locking: DRMS_ROLLBACK, DRMS_COMMIT, DRMS_NEWSERIES, DRMS_DROPSERIES, DRMS_NEWSLOTS, DRMS_SLOT_SETSTATE, DRMS_GETUNIT. This is a very high level locking that gives the correct results but not necessarily the max amount of parallelism. For example, we could lock at a lower level inside drms_su_getdirs(). Direct-connect is most handy when there is only a single client module. In that case, the socket presents a mere overhead. We thus fuse drms_server - Manages a database session and passes database requests to database from client modules program with the client module. The object file for both socket-connect and direct-connect are the same. They simply link to different libraries to produce the two executables with and without the _sock suffix.

Note:

drms_server_begin_transaction() and drms_server_end_transaction() can only be used in direct-connect mode. Why? Because we don't want to deal with concurrency issues when beginning/ending a transaction.

A socket-connect module looks for DRMSSESSION which identifies the host and port of a running drms_server - Manages a database session and passes database requests to database from client modules program. In case DRMSSESSION information is absent, the module would attempt to fork a drms_server - Manages a database session and passes database requests to database from client modules program and connect itself to it via a socket. When the module finishes executing the DoIt() function, it sends either SIGTERM upon failure, or SIGUSR1 upon success, to the drms_server - Manages a database session and passes database requests to database from client modules process and waits until the latter returns before exiting.

The -H option prints out the list of required arguments and their default or set values, then quits. (The argument --help has the same effect.) The -V option prints the full argument list before execution of the module; it also sets the drms_env->verbose element and may force printing of some diagnostic messages by the module driver. The -n flag is reserved by jsoc_main.a but is not currently used.

The -L flag forces logging of all module messages to stdout and stderr to a DRMS session log. The -Q option suppresses messages from the module to stdout and stderr, although the messages are still sent to the DRMS session log. Without session logging turned on, the -Q flag has no effect on logging. The -A flag forces archiving of all segments (including the logging) in SUMS to tape, if that is an option. If it is not set, then archiving is turned off. It is not clear how the presence or absence of this flag interacts with the archiving options defined for the data series.

All flags (except of course -H) are available to be passed down to the module. White space between the '=' sign and the argument value is optional. Flags can be concatenated as shown below.

GEN_FLAGS (can be grouped together):

-A: Force archiving of storage units during the current DRMS session. This overrides the value of series' archive flag.
-H|--help: Show jsoc_main usage information.
-L: Run jsoc_main driver with logging. Stdout and stderr are tee-ed to files in SU directory.
-V: Run jsoc_main driver in verbose mode.
-Q: Run jsoc_main driver in quiet mode (no terminal output).
-n: A nop; the module cannot define a flag named n.
--jsocmodver: print the JSOC module version number, then quit

SESSION_ARGUMENTS:

To specify an argument that affects properties of the DRMS session, use param=value, where param is one of the following.

• DRMS_ARCHIVE Sets the archive status for all storage units created during the DRMS session started by module. value can be either -1 (don't archive data, and when a storage unit's retention expires, delete all DRMS records whose data resides within this storage unit), 0 (don't archive, but do not delete any DRMS records), or 1 (archive data when the storage unit's retention expires).
• DRMS_RETENTION For all existing storage units accessed during the module's session, the value of this argument will be used update the retention time. The value is a positive 15-bit number. If the current retention time is greater than the time specified with this value, then this argument will have no effect. But if the current retention time is less than the time specified by this value, then SUMS will ensure that the SUs touched will have a retention time AT LEAST as large as this argument's value. Setting this argument will never reduce the retention time of any existing storage unit. This argument cannot be specified as an environment variable.
• DRMS_NEWSURETENTION The value of this argument will be the retention time for all new storage units created. The value is a positive 15-bit number. It will override the JSD retention time value specified for the series being operated on.
• DRMS_QUERY_MEM Sets the memory maximum for a database query.
• DRMS_DBTIMEOUT The value is the number of milliseconds any underlying database query is allowed to run, after which the database will issue an error, causing the module to terminate with an error, rolling back any database changes. The database time-out feature is initiated each time a transaction is started (in drms_server_begin_transaction()).
• JSOC_DBHOST Specifies (overrides) the database host to connect to. Default is DBNAME
• JSOC_DBNAME Specifies (overrides) the database name to use. Default is DBNAME
• JSOC_DBUSER Specifies (overrides) the username used during database host connection. Default is USER.
• JSOC_DBPASSWD Specifies (overrides) the password for the username. Default is PASSWD.
• JSOC_SESSIONNS Specifies (overrides) the DRMS session namespace.

MODULE_ARGUMENTS:

These are module-specific; refer to DRMS Utilities for module-specific documentation.

Examples:

Example 1: To ensure that the retention of existing storage units is at least 10000 days:

 show_info -P hmi.m_45s[2011.12.10_00:00/10m] DRMS_RETENTION=-10000

Example 2: To reduce retention to 0 so that SUMS will delete the storage units from disk (the caller must be the series owner or a production user):

 show_info -P hmi.m_45s[2011.12.10_00:00/10m] DRMS_RETENTION=0

Example 3: To change the retention to 100 days (the caller must be the series owner or a production user):

 show_info -P hmi.m_45s[2011.12.10_00:00/10m] DRMS_RETENTION=100