module
[DRMS Utilities]

Detailed Description

Execute the named module from a shell. When a module is executed from the command line, all argument values listed in the module are supplied from (in order of precedence): values on the command line; values in a named parameter file; or the default values specified in the module source, if any. If an argument cannot be otherwise evaluated, a notification message is issued and the driver program exits. If all the argument values exist, a DRMS session is opened or a connection is made to an existing DRMS server as appropriate, and the remaineder of the module code is executed. At the end of module execution, the driver disconnects from DRMS with or without commits to the database depending on the exit status of the module. No commits are ever made when a module aborts or is killed on a signal.

Modules are expected to run as functions (or subroutines) called by a main driver program that parses the argument list from the command line and environment, and sets up the drms->env environment for communication with the DRMS database via the DRMS API. Different driver programs can be made available, depending on the environment. Two such programs (in the form of libraries) are provided as part of the base system, jsoc_main.a and jsoc_main_sock.a (see jsoc_main.h). The former provides a direct connection to DRMS, for use by self-contained modules. The latter provides communication via a socket; it can be used for multiple modules connecting to a common server, as in a script. When multiple clients connect to a common server, commits to the database will only occur at the successful close of the server. Modules linked against jsoc_main.a are given their basename for the executable, while those linked against jsoc_main_sock have "_sock" appended to the basename. For example, a module named myprogram.c would produce executables named myprogram and myprogram_sock. (Additional drivers to produce, for example, CGI scripts such as myprogram_cgi can also be provided.)

Socket-connect vs direct-connect modules
In socket connect mode, a module talks to drms_server program via a socket. Concurrent access is protected by locks drms_lock_server() and drms_unlock_server(). The following command codes employ locking: DRMS_ROLLBACK, DRMS_COMMIT, DRMS_NEWSERIES, DRMS_DROPSERIES, DRMS_NEWSLOTS, DRMS_SLOT_SETSTATE, DRMS_GETUNIT. This is a very high level locking that gives the correct results but not necessarily the max amount of parallelism. For example, we could lock at a lower level inside drms_su_getdirs(). Direct-connect is most handy when there is only a single client module. In that case, the socket presents a mere overhead. We thus fuse drms_server program with the client module. The object file for both socket-connect and direct-connect are the same. They simply link to different libraries to produce the two executables with and without the _sock suffix.
Note:
drms_server_begin_transaction() and drms_server_end_transaction() can only be used in direct-connect mode. Why? Because we don't want to deal with concurrency issues when beginning/ending a transaction.
A socket-connect module looks for DRMSSESSION which identifies the host and port of a running drms_server program. In case DRMSSESSION information is absent, the module would attempt to fork a drms_server program and connect itself to it via a socket. When the module finishes executing the DoIt() function, it sends either SIGTERM upon failure, or SIGUSR1 upon success, to the drms_server process and waits until the latter returns before exiting.

The -H option prints out the list of required arguments and their default or set values, then quits. (The argument --help has the same effect.) The -V option prints the full argument list before execution of the module; it also sets the drms_env->verbose element and may force printing of some diagnostic messages by the module driver. The -n flag is reserved by jsoc_main.a but is not currently used.

The -L flag forces logging of all module messages to stdout and stderr to a DRMS session log. The -Q option suppresses messages from the module to stdout and stderr, although the messages are still sent to the DRMS session log. Without session logging turned on, the -Q flag has no effect on logging. The -A flag forces archiving of all segments (including the logging) in SUMS to tape, if that is an option. If it is not set, then archiving is turned off. It is not clear how the presence or absence of this flag interacts with the archiving options defined for the data series.

All flags (except of course -H) are available to be passed down to the module. White space between the '=' sign and the argument value is optional. Flags can be concatenated as shown below.

Usage:
<module> [DRIVER_FLAGS] [SESSION_ARGUMENTS] [MODULE_ARGUMENTS]
DRIVER_FLAGS (can be grouped together):
-H|--help: Show jsoc_main usage information.
-L: Run jsoc_main driver with logging. Stdout and stderr are tee-ed to files in SU directory.
-V: Run jsoc_main driver in verbose mode.
-Q: Run jsoc_main driver in quiet mode (no terminal output).
-n: A nop; the module cannot define a flag named n.
SESSION_ARGUMENTS:
To specify an argument that affects properties of the DRMS session, use param=value, where param is one of the following.
MODULE_ARGUMENTS:
These are module-specific; refer to DRMS Utilities for module-specific documentation.


Variables

DRMS_Env_t * drms_env
 Global DRMS Environment handle.
CmdParams_t cmdparams
 Global DRMS-module structure representing the command-line arguments provided to the module executable.
ModuleArgs_t * gModArgs = module_args
 Global DRMS-module pointer that refers to module_args.
ModuleArgs_t module_args []
 Global DRMS-module structure representing the default command-line arguments for a DRMS module.
ModuleArgs_t * gModArgs
 Global DRMS-module pointer that refers to module_args.
char * module_name
 Global DRMS-module string providing the name of the module.

Variable Documentation

ModuleArgs_t module_args[]

Global DRMS-module structure representing the default command-line arguments for a DRMS module.

module_args, a global array of ModuleArgs_t structures, provides a standard mechanism for declaring the parameters expected to be used by a module along with their types and default values, if any. module_args must be declared in every module. The elements of the module_args array are parsed and compared with arguments supplied to the module from the command line or other invocation to produce a CmdParams_t structure (cmdparams) through which their values are available through the params_get suite of functions.

The module_args declarator requires at least one element, which must be of type ARG_END (which is 0, so an empty initializer as shown in the synopsis is acceptable). Any array elements following the ARG_END element are ignored.

Although the default value (and range, if applicable) is supplied as a character string, it will be interpreted according to the declared type of the argument. Each element of cmdparams, except those of type ARG_VOID, must have a name field. Arguments of type ARG_INT, ARG_FLOAT, ARG_DOUBLE, and ARG_STRING should be self-explanatory. Arguemts of type ARG_FLAG are expected to have single-character names and to be associated with logical binaries, with a default value of FALSE (0); they can be set on the command line via the -X construct (where X is the name of the element to be set to TRUE). ARG_TIME is a special case of ARG_DOUBLE, whose default or assigned values are interpreted by sscan_time (q.v.). ARG_VOID is reserved for use with undeclared arguments supplied on the command line; it should not be used for declared arguments in the module_args list.

The types ARG_INTS, ARG_FLOATS, and ARG_DOUBLES are used for parameters that can be arrays of arbitrary length. The values must be supplied as comma separated sets enclosed within matched delimiting pairs of either brackets [], braces {} or parentheses () (unless there is only one value in the array, in which case the delimiters are optional). The total number of elements in the array is returned as the added parameter name_nvals, and the value for the nth element (counting from 0) as name_n_value. For example, a @ module_args element declared as:

{ARG_FLOATS, "lat", "[0.0, 5.0, 10.0]", "", ""},

would return 3 for params_get_int (params, "lat_nvals") and the value 5.0 for params_get_float (params, "lat_1_value"). The number of array values supplied at run time need not match the number in the default; indeed there is no necessity of setting any default value at all, just as with other types of arguments.

ARG_NUME is a special type of argument representing an enumeration class. It makes use of the module_args->range field, which must be a comma-separated list of strings. The value returned is an integer coresponding to the order number of the range element matching the supplied value. For example, a module_args element declared as:

{ARG_NUME, "color", "green", "", "red, yellow, green, blue"},

would return 2 for params_get_int (params, "color"). A failure occurs if the value supplied does not match anything in the range; the type is designed especially for use with driver programs that can provide menus of options, such as CGI forms.

ARG_DATASET and ARG_DATASERIES are special cases of ARG_STRING reserved for names of DRMS dataset specifications or series names in an environment where the database can be queried for possible values; they are not currently treated differently from any other type of string argument.

ARG_NEWDATA does not appear to be implemented; ARG_NUMARGS is reserved for internal use by the Fortran interface and should not be used.

To summarize, ModuleArgs_t->type must have one of the following values:

ARG_INT parameter is to be interpreted as type int
ARG_FLOAT parameter is to be interpreted as type double
ARG_DOUBLE parameter is to be interpreted as type double
ARG_TIME parameter is to be interpreted as type double, with a conversion from standard date-time string formats to a standard reference epoch
ARG_STRING parameter is to be interpreted as type char*
ARG_FLAG the parameter is (ordinarily) a single-character named one which can take the value of 0 or 1. The default value, if present, should be 0; as the command-line flag specifier can only set its parameter values to 1; however, it is better to leave the default value empty, so that the cmdparams_exists function can be used in the code.
ARG_NUME the parameter value is string-compared with the members of the module_args->range list, and replaced with the string representation of the number corresponding to the order number of the (first) matching token in the list; its value is subsequently to be interpreted as type int. Basically equivalent to type enum
ARG_INTS (not yet implemented)
ARG_FLOATS (not yet implemented)
ARG_DOUBLE synonymous with ARG_FLOATS
ARG_VOID (not yet implemented)
ARG_END signals the end of the parsed argument list. Elements may follow in the declaration, but will be ignored. Since ARG_END is defined as 0, an empty (null) member serves the same purpose.

The module_args->description is intended to be used only by the front-end handler for documentation, such as when the command is invoked with a -H help flag, or in CGI web forms.

Bug:
Range inspection is limited to arguments of type ARG_NUME and ARG_FLOAT.
See also:
module cmdparams.h sscan_time

Definition at line 27 of file drms_log.c.

Generated on Tue Jul 15 19:31:53 2008 for JSOC by  doxygen 1.5.4