Differences between revisions 3 and 13 (spanning 10 versions)
Revision 3 as of 2012-09-27 00:07:17
Size: 666
Editor: DNab421146
Comment:
Revision 13 as of 2012-09-28 01:26:17
Size: 12322
Editor: arta-mbp
Comment:
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
A user controls configuration by editing one or more configuration files: There are several files that control configuration. Some files are not to edited by users:
Line 7: Line 7:
== configsdp.txt ==  * '''$JSOC/configure'''[[BR]]
 This is the master script that a user runs before they run make. It needs to be run once after source-code and make-configuration changes, but it does not need to be run when building on a new platform. The user does not edit this file for configuration-control purposes, and, in general, only the CM will edit this file (for maintenance reasons, or in response to user requests). This file contains JSOC-SDP-DRMS-specific, and NetDRMS-specific configuration. For example, there is no DSDS support for NetDRMS builds, and configure contains a line that defines a compiler flag that disables DSDS support for NetDRMS.[[BR]][[BR]]
 The documentation for this file resides as comments in the file itself.
Line 9: Line 11:
This  * '''$JSOC/gen_init.csh'''[[BR]]
 This script, available only in NetDRMS releases, parses the config.local configuration file (described below), and generates the $JSOCROOT/base/include/localization.h file. This header file contains parameters that ''override'' the default set of parameters, which are specific for JSOC-SDP DRMS builds.
Line 11: Line 14:
The contents of this file, and how to edit them, are described more fully in CvsInit.  * '''$JSOC/base/drms/apps/serverdefs.h'''[[BR]]
 This file contains JSOC-SDP-specific #defines used to control compilation. It contains parameters that specify the name of the database administrative user, the default name of the database to connect to, and many others that tailor DRMS and SUMS. The contents of this file are overridden by $JSOC/base/include/localization.h, if it is present.
Line 13: Line 17:
 * '''$JSOC/base/include/localization.h'''[[BR]]
 This header file is generated when configure is run by the user. It is #included in $JSOCROOT/base/drms/apps/serverdefs.h, where it then ''overrides'' the #defines present in that file.
Line 14: Line 20:
== config.local ==  * '''$JSOC/moreconfigure.pl''' [[BR]]
 This script first determines which compiler to use. It will first look for an acceptable version of icc to run. If there isn't one, then it looks for an acceptable version of gcc. It also looks for an acceptable version of ifort for FORTRAN compilation and linking, and if there is no such version, it looks for an acceptable version of gfortran. If there is no acceptable C compiler or FORTRAN compiler present, the script fails. This script writes the compiler-configuration make variables to $JSOC/localization/custom.mk, which is included into make_basic.mk, where it can override the values defined in make_basic.mk. [[BR]][[BR]]
 The user can set a build parameter in config.local, AUTOSELCOMP, that will disable this auto-selection of compilers. In that case, the choice of compilers is determined by make_basic.mk (search for the line containing the make variable assignment "COMPILER = icc"), which are overridable by the compiler make variables in the custom.mk file (which are derived from parameters in config.local). The compiler variables are ultimately overridable by environment variables ($JSOC_COMPILER and $JSOC_FCOMPILER).[[BR]][[BR]]
 This script also parses the configsdp.txt file's __LIBS__ section, which contains make variables which contain paths to third-party libraries used by the JSOC-SDP DRMS. It also writes these make variables into the custom.mk file, which is then included into make_basic.mk, where the variables will have effect.

 * '''$JSOC/customizemake.pl''' [[BR]]
 This script parses the __DEFS__ and __MAKE__ sections of config.local. In general, the variables in the __MAKE__ section specify the paths to third-party libraries, but this section could contain any desired make variable. For each line in these sections, it writes into custom.mk a key-value make variable assignment statement, where the key in config.local becomes the make-variable name in custom.mk, and the value in config.local becomes the make-variable value in custom.mk. custom.mk is then included into make_basic.mk, where the variables will have effect.

 * '''$JSOC/customizedefs.pl''' [[BR]]
 This script parses the __DEFS__ section of config.local to produce a header file, $JSOC/base/include/customizeddefs.h. Each line in this file contains key-value type #defines where the key is a config.local line's parameter name, and the #define value is the config.local line's parameter value. This script defines acceptable parameter names, and fails if config.local contains a parameter that is not in this list of acceptable parameters. customizeddefs.h is #included into serverdefs.h, where its #defines can override the JSOC-SDP DRMS default ones defined in serverdefs.h.

 * '''$JSOC/custom.mk''' [[BR]]
 This file contains make variables that customize the build process. This file is first created by the configure script, but other scripts (i.e., customizedefs.pl, moreconfigure.pl, and customizemake.pl) append additional make variables. custom.mk is included into make_basic.mk where the contained make variable are used and often override make existing make variables (defined in make_basic.mk).

 * '''$JSOC/configproj.pl''' [[BR]]
 This script takes as input a configuration file that describes the set of projects included in the current build (configsdp.txt is actually such a file, but users can override the use of configsdp.txt by providing a different configuration file as an argument to the $JSOC/configure script). This file has several sections. The __MAKE__ section contains make-variable assignments, target definitions, and rules. configproj.pl creates $JSOC/localization/make_basic.mk, and copies, ver batim, the contents of the xml file's __MAKE__ section to make_basic.mk. $JSOC/make_basic.mk includes $JSOC/localization/make_basic.mk so that when the make system builds projects, it can use these additional variables, targets, and rules that apply only to projects. The __PROJ__ section contains xml that describes the set of projects that make will build. configproj.pl uses this information to create $JSOC/localization/target.mk (which contains make rules to create the make-output project directories if they do not exist) and $JSOC/localization/Rules.mk (which contains rules needed to build the code in the project directories). $JSOC/localization/target.mk is included into $JSOC/target.mk, and $JSOC/localization/Rules.mk is included into $JSOC/Rules.mk. The __PROJCFG__ section contains a script that configproj.pl copies to $JSOC/localization/configure. When $JSOC/configure runs, it runs $JSOC/localization/configure. The script can contain anything - currently it is used for creating links from $JSOC/bin/<plat> to lev0 binaries.

Other files are available for the user to edit. These files allow the user to control configuration:

 * '''$JSOC/configsdp.txt'''[[BR]]
 This file contains information specific to the JSOC-SDP that controls the build of JSOC-SDP-specific projects (like the "rings" project). In the __MAKE__ section are JSOC-SDP-specific make variables, targets, and rules. In the __PROJ__ section, is information needed by make to create project-specific directories. In the __PROJCFG__ section are shell commands to be used for project-specific management (like creating links to specified executables). Finally, in the __LIBS__ section are make variables that specify the paths of the third-party libraries used by the JSOC-SDP DRMS.[[BR]][[BR]]The presence of this file in the $JSOCROOT directory indicates to the make system that the source-code is to be used to create DRMS/SUMS for the JSOC-SDP (i.e., a "full" make). If this file is not present, then the make system will create a NetDRMS when make is run.[[BR]][[BR]]
 The contents of this file, and how to edit them, are described more fully in CvsInit.

 * '''$JSOC/config.local'''[[BR]]
 This file contains various system-wide parameters that affect the build, including the machine hosting the SUMS server, the paths to the PostgreSQL and CFITSIO libraries, and the machine hosting the DRMS database, and many others. '''This file must be created by the user prior to running configure.''' It is not part of any release. Instead, the release provides a template file (config.local.template for NetDRMS releases, and config.local.sutemplate for JSOC-SDP DRMS releases). The user should copy the template, and edit is as desired. [[BR]][[BR]]
 The documentation for this file, which includes examples, resides in the config.local.template (NetDRMS) or config.local.sutemplate (JSOC-SDP DRMS) file.

Once the user has modified these configuration files as desired, configuration proceeds when the user runs the $JSOCROOT/configure script. This script first determines if the build to be performed is a JSOC-SDP DRMS build, or a NetDRMS build. If $JSOCROOT/configsdp.txt is present, then it will assume the former, otherwise it will assume the latter. If performing a NetDRMS build, then the script requires that config.local be present. For a JSOC-SDP DRMS build, the presence of this file is NOT required. If it is present, then the contents of that file will customize the JSOC-SDP DRMS build, but if it is absent, then a default build will be performed.

If building NetDRMS, configure first runs gen_init.csh. This will parse the config.local file to produce a NetDRMS-site-specific localization.h file, which then contains parameters, specified by the user, that override the parameters used by the JSOC-SDP. The script then runs moreconfigure.pl, which examines the C and FORTRAN compilers available and chooses the "best" ones that make will use. These choices are codified in the $(COMPLIER) and $(FCOMPILER) make variables, which are written to $JSOC/localization/custom.mk, which is included into make_basic.mk where the contained make-variables assignments will have effect. This auto-selection of compilers can be disabled by setting the AUTOSELCOMP parameter in config.local to 0, in which case the $(COMPILER) and $(FCOMPILER) make variables in make_basic.mk are used for compiler selection, which are overridable by the $JSOC_COMPILER and $JSOC_FCOMPILER environment variables. configure finally runs customizemake.pl, which parses the config.local file, generating make-variable assignment statements out of the key-value parameter lines in config.local. These make-variable assignment statements are also written to custom.mk.

If building the JSOC SDP DRMS, and there is no $JSOCROOT/config.local file present, then, as with the NetDRMS build, configure runs moreconfigure.pl, which chooses the best C and FORTRAN compilers that are available. As before, the appropriate $(COMPLIER) and $(FCOMPILER) make-variable assignments are written to $JSOC/localization/custom.mk, and included into $JSOCROOT/make_basic.mk. If a config.local does exist, then configure first runs customizedefs.pl. This produces a header file, $JSOC/base/include/customizeddefs.h, that contains C #defines for each of the lines in the __DEFS__ section of config.local. customizeddefs.h is #included into serverdefs.h, where its #defines can override the JSOC-SDP DRMS default ones defined in serverdefs.h. After that, the script, as before, runs moreconfigure.pl, which chooses the best C and FORTRAN compilers that are available. The script finally runs customizemake.pl, which parses the config.local file, as before, generating make-variable assignment statements out of the key-value parameter lines in config.local. These make-variable assignment statements are also written to custom.mk.

configure next runs $JSOC/configproj.pl. This script parses a "project-configuration" file that describes the set of projects included in the current build (configsdp.txt is actually such a file, but users can override the use of configsdp.txt by providing a different configuration file as an argument to the $JSOC/configure script). This script creates several files in $JSOC/localization that are needed by the make system when building the project directories.

Build Configuration

The source-code configure script, $JSOCROOT/configure, sets environment variables, make variables, and creates on-the-fly make rules that allow users to localize and/or customize the way the JSOC-SDP code is built. Depending on how the make system is configured with these variables and rules, the user can build NetDRMS or the DRMS used at the JSOC-SDP. And for both builds, the user can customize various details.

There are several files that control configuration. Some files are not to edited by users:

  • $JSOC/configureBR This is the master script that a user runs before they run make. It needs to be run once after source-code and make-configuration changes, but it does not need to be run when building on a new platform. The user does not edit this file for configuration-control purposes, and, in general, only the CM will edit this file (for maintenance reasons, or in response to user requests). This file contains JSOC-SDP-DRMS-specific, and NetDRMS-specific configuration. For example, there is no DSDS support for NetDRMS builds, and configure contains a line that defines a compiler flag that disables DSDS support for NetDRMS.BRBR The documentation for this file resides as comments in the file itself.

  • $JSOC/gen_init.cshBR This script, available only in NetDRMS releases, parses the config.local configuration file (described below), and generates the $JSOCROOT/base/include/localization.h file. This header file contains parameters that override the default set of parameters, which are specific for JSOC-SDP DRMS builds.

  • $JSOC/base/drms/apps/serverdefs.hBR This file contains JSOC-SDP-specific #defines used to control compilation. It contains parameters that specify the name of the database administrative user, the default name of the database to connect to, and many others that tailor DRMS and SUMS. The contents of this file are overridden by $JSOC/base/include/localization.h, if it is present.

  • $JSOC/base/include/localization.hBR This header file is generated when configure is run by the user. It is #included in $JSOCROOT/base/drms/apps/serverdefs.h, where it then overrides the #defines present in that file.

  • $JSOC/moreconfigure.pl BR This script first determines which compiler to use. It will first look for an acceptable version of icc to run. If there isn't one, then it looks for an acceptable version of gcc. It also looks for an acceptable version of ifort for FORTRAN compilation and linking, and if there is no such version, it looks for an acceptable version of gfortran. If there is no acceptable C compiler or FORTRAN compiler present, the script fails. This script writes the compiler-configuration make variables to $JSOC/localization/custom.mk, which is included into make_basic.mk, where it can override the values defined in make_basic.mk. BRBR The user can set a build parameter in config.local, AUTOSELCOMP, that will disable this auto-selection of compilers. In that case, the choice of compilers is determined by make_basic.mk (search for the line containing the make variable assignment "COMPILER = icc"), which are overridable by the compiler make variables in the custom.mk file (which are derived from parameters in config.local). The compiler variables are ultimately overridable by environment variables ($JSOC_COMPILER and $JSOC_FCOMPILER).BRBR This script also parses the configsdp.txt file's LIBS section, which contains make variables which contain paths to third-party libraries used by the JSOC-SDP DRMS. It also writes these make variables into the custom.mk file, which is then included into make_basic.mk, where the variables will have effect.

  • $JSOC/customizemake.pl BR This script parses the DEFS and MAKE sections of config.local. In general, the variables in the MAKE section specify the paths to third-party libraries, but this section could contain any desired make variable. For each line in these sections, it writes into custom.mk a key-value make variable assignment statement, where the key in config.local becomes the make-variable name in custom.mk, and the value in config.local becomes the make-variable value in custom.mk. custom.mk is then included into make_basic.mk, where the variables will have effect.

  • $JSOC/customizedefs.pl BR This script parses the DEFS section of config.local to produce a header file, $JSOC/base/include/customizeddefs.h. Each line in this file contains key-value type #defines where the key is a config.local line's parameter name, and the #define value is the config.local line's parameter value. This script defines acceptable parameter names, and fails if config.local contains a parameter that is not in this list of acceptable parameters. customizeddefs.h is #included into serverdefs.h, where its #defines can override the JSOC-SDP DRMS default ones defined in serverdefs.h.

  • $JSOC/custom.mk BR This file contains make variables that customize the build process. This file is first created by the configure script, but other scripts (i.e., customizedefs.pl, moreconfigure.pl, and customizemake.pl) append additional make variables. custom.mk is included into make_basic.mk where the contained make variable are used and often override make existing make variables (defined in make_basic.mk).

  • $JSOC/configproj.pl BR This script takes as input a configuration file that describes the set of projects included in the current build (configsdp.txt is actually such a file, but users can override the use of configsdp.txt by providing a different configuration file as an argument to the $JSOC/configure script). This file has several sections. The MAKE section contains make-variable assignments, target definitions, and rules. configproj.pl creates $JSOC/localization/make_basic.mk, and copies, ver batim, the contents of the xml file's MAKE section to make_basic.mk. $JSOC/make_basic.mk includes $JSOC/localization/make_basic.mk so that when the make system builds projects, it can use these additional variables, targets, and rules that apply only to projects. The PROJ section contains xml that describes the set of projects that make will build. configproj.pl uses this information to create $JSOC/localization/target.mk (which contains make rules to create the make-output project directories if they do not exist) and $JSOC/localization/Rules.mk (which contains rules needed to build the code in the project directories). $JSOC/localization/target.mk is included into $JSOC/target.mk, and $JSOC/localization/Rules.mk is included into $JSOC/Rules.mk. The PROJCFG section contains a script that configproj.pl copies to $JSOC/localization/configure. When $JSOC/configure runs, it runs $JSOC/localization/configure. The script can contain anything - currently it is used for creating links from $JSOC/bin/<plat> to lev0 binaries.

Other files are available for the user to edit. These files allow the user to control configuration:

  • $JSOC/configsdp.txtBR This file contains information specific to the JSOC-SDP that controls the build of JSOC-SDP-specific projects (like the "rings" project). In the MAKE section are JSOC-SDP-specific make variables, targets, and rules. In the PROJ section, is information needed by make to create project-specific directories. In the PROJCFG section are shell commands to be used for project-specific management (like creating links to specified executables). Finally, in the LIBS section are make variables that specify the paths of the third-party libraries used by the JSOC-SDP DRMS.BRBRThe presence of this file in the $JSOCROOT directory indicates to the make system that the source-code is to be used to create DRMS/SUMS for the JSOC-SDP (i.e., a "full" make). If this file is not present, then the make system will create a NetDRMS when make is run.BRBR The contents of this file, and how to edit them, are described more fully in CvsInit.

  • $JSOC/config.localBR This file contains various system-wide parameters that affect the build, including the machine hosting the SUMS server, the paths to the PostgreSQL and CFITSIO libraries, and the machine hosting the DRMS database, and many others. This file must be created by the user prior to running configure. It is not part of any release. Instead, the release provides a template file (config.local.template for NetDRMS releases, and config.local.sutemplate for JSOC-SDP DRMS releases). The user should copy the template, and edit is as desired. BRBR The documentation for this file, which includes examples, resides in the config.local.template (NetDRMS) or config.local.sutemplate (JSOC-SDP DRMS) file.

Once the user has modified these configuration files as desired, configuration proceeds when the user runs the $JSOCROOT/configure script. This script first determines if the build to be performed is a JSOC-SDP DRMS build, or a NetDRMS build. If $JSOCROOT/configsdp.txt is present, then it will assume the former, otherwise it will assume the latter. If performing a NetDRMS build, then the script requires that config.local be present. For a JSOC-SDP DRMS build, the presence of this file is NOT required. If it is present, then the contents of that file will customize the JSOC-SDP DRMS build, but if it is absent, then a default build will be performed.

If building NetDRMS, configure first runs gen_init.csh. This will parse the config.local file to produce a NetDRMS-site-specific localization.h file, which then contains parameters, specified by the user, that override the parameters used by the JSOC-SDP. The script then runs moreconfigure.pl, which examines the C and FORTRAN compilers available and chooses the "best" ones that make will use. These choices are codified in the $(COMPLIER) and $(FCOMPILER) make variables, which are written to $JSOC/localization/custom.mk, which is included into make_basic.mk where the contained make-variables assignments will have effect. This auto-selection of compilers can be disabled by setting the AUTOSELCOMP parameter in config.local to 0, in which case the $(COMPILER) and $(FCOMPILER) make variables in make_basic.mk are used for compiler selection, which are overridable by the $JSOC_COMPILER and $JSOC_FCOMPILER environment variables. configure finally runs customizemake.pl, which parses the config.local file, generating make-variable assignment statements out of the key-value parameter lines in config.local. These make-variable assignment statements are also written to custom.mk.

If building the JSOC SDP DRMS, and there is no $JSOCROOT/config.local file present, then, as with the NetDRMS build, configure runs moreconfigure.pl, which chooses the best C and FORTRAN compilers that are available. As before, the appropriate $(COMPLIER) and $(FCOMPILER) make-variable assignments are written to $JSOC/localization/custom.mk, and included into $JSOCROOT/make_basic.mk. If a config.local does exist, then configure first runs customizedefs.pl. This produces a header file, $JSOC/base/include/customizeddefs.h, that contains C #defines for each of the lines in the DEFS section of config.local. customizeddefs.h is #included into serverdefs.h, where its #defines can override the JSOC-SDP DRMS default ones defined in serverdefs.h. After that, the script, as before, runs moreconfigure.pl, which chooses the best C and FORTRAN compilers that are available. The script finally runs customizemake.pl, which parses the config.local file, as before, generating make-variable assignment statements out of the key-value parameter lines in config.local. These make-variable assignment statements are also written to custom.mk.

configure next runs $JSOC/configproj.pl. This script parses a "project-configuration" file that describes the set of projects included in the current build (configsdp.txt is actually such a file, but users can override the use of configsdp.txt by providing a different configuration file as an argument to the $JSOC/configure script). This script creates several files in $JSOC/localization that are needed by the make system when building the project directories.

JsocWiki: BuildConfiguration (last edited 2013-05-01 04:35:24 by localhost)