View source for SCONS for online

=How To Build Online Packages with the SCONS-Based Build System=

::Elliott Wolin, 17-Dec-2013


'''Introduction'''

SCONS is used in the online build system due to its superior scripting abilities and built-in knowledge of dependencies in different languages (n.b. Make is used in the offline but they are currently in the process of switching to scons as well).  It is based on Python, the main scripting language used in the online.  Package building is vastly simplified via a library of SCONS scripts that implement the Hall D directory and installation scheme.  

'''Note:  package developers should only need to make minor modifications to a few SCONS scripts to build their package (see SConscript customization section below)'''

According the GlueX policy, developers working on individual packages should work from their own personal accounts.  Deployment of packages into the production environment, and the directory structures used for deployment, are the concern of software system managers (via the hdsys account), and will not be discussed here.

SVN is used for code management for all packages  We use the same repository as the offline, but in a separate online area.  Code management will not be discussed here.



'''Creating a new package'''

To create a new package in your own directory execute the Python script:

 $ create_online_package myPackage

This script resides in /gluex/builds/someRelease/scripts where someRelease is one of the deployed releases (e.g. devel).  The script will set up a package root and directory structure that includes SCONS files needed for building the package:

 $ ls myPackage/
 doc/  java/ SConstruct  scripts/  src/

C and C++ code resides in the src directory, scripts in scripts, Java code in the java directory.  A package can contain many C/C++ executables (in src/binsrc and subdirs), but only one library (libmyPackage.so) will be created from the code in the src/libsrc directory.  Plugins reside in the plugins directory, each plugin in its own subdirectory.  Also, only one jar file (myPackage.jar) will be created from the code in the java directory.  Unused directories can be deleted:

 $ ls myPackage/src/
 binsrc/  libsrc/  plugins/  test/

Java packages should be organized as follows:

 $ tree myPackage/java
 java
 |-- org
 |   `-- jlab
 |       `-- halld
 |           `-- myPackage
 |               `-- HelloWorld.java
 `-- SConscript

Note that one SConstruct file appears in the package root dir, and one SConscript file in each of the subdirectories of src, in the scripts directory and in the java directory.  You may need to customize the SConscript files to define package dependencies, see below.

SCONS will only build using code in the directories listed above, all other directories will be ignored.

You can check this new package into the main code management repository if desired.  More common would be to check out and modify an existing package, which should have the structure above (although some parts may be missing if not relevant).

Note:  the package holding front-end readout lists, rol, is a bit different due to the unique requirements of our CODA DAQ system.  Normally you would check this package out instead of creating it from scratch.  The main difference is that readout lists come in sets, with the set name appearing in the src directory (rather than binsrc, libsrc, etc).



'''Building a package'''

SConstruct is the top-level SCONS file, and the package can be built by executing the "scons" command in the package root directory.  SCONS will descend the directory tree and build the libraries, executables, etc. that it comes across.  Hidden architecture-dependent subdirectories in the source areas hold intermediate build files:

 $ cd myPackage
 $ scons
 scons: Reading SConscript files ...
 
 Processing package: myPackage
   --> Building for architecture Linux_RHEL6-x86_64-gcc4.8.0
 
 scons: done reading SConscript files.
 scons: Building targets ...
 scons: done building targets.

Executing "scons install" will install the C and C++ code previously built into an architecture-dependent directory in the package root directory, with subdirectories for header files, libraries, etc:

 $ scons install
 scons: Reading SConscript files ...
 
 Processing package: myPackage
  --> Building for architecture Linux_RHEL6-x86_64-gcc4.8.0
 
 scons: done reading SConscript files.
  
 scons: Building targets ...
 scons: `install' is up to date.
 scons: done building targets.
 
 $ ls
 doc/  java/  Linux_RHEL6-x86_64-gcc4.8.0/  SConstruct  scripts/  src/
 
 $ ls Linux_RHEL6-x86_64-gcc4.8.0/
 bin/  include/	lib/  plugins/ 

If you execute:

 $ scons ginstall

("global install") it will install the release directory, instead.  The release directory is the one that the "packages" directory resides in, i.e. above all the individual packages.


Java code gets built somewhat differently (in accordance with Java standard practice):

 $ scons java

builds into the java/build directory:

 $ ls java/
 build/	org/  SConscript
 
 $ tree java/build/
 java/build/
 |-- classes
 |   `-- org
 |       `-- jlab
 |           `-- halld
 |               `-- myPackage
 |                   `-- HelloWorld.class
 `-- lib
     `-- myPackage.jar

Many command-line flags and environment variables can affect the build, type "scons -H" for a full listing.



'''External Packages and Customizing SConscript Files'''

Libraries or executables may depend on other packages, i.e. to compile and link you may need to find header files, other libraries, etc.  If so you must first define environment variables to tell SCONS where to find the external package root directories, then tell SCONS that you need them in one or more SConscript files:

This tells SCONS where to find the EVIO package:

 $ setenv EVIOROOT someDir     # just EVIO will work, too

You tell SCONS to you need to link with external packages in the SConscript files using the define_dependencies() function.  The SConscript file below tell SCONS that the executables in src/binsrc depend on the et, evio and codaobject packages:

 $ more src/binsrc/SConscript 
 
 #  SConscript for binsrc
 #
 #  to customize fill in package dependencies
 #
 #  ejw, 16-apr-2013
 
 
 Import('*')
 from halld_lib import define_dependencies, buildBin
 
 
 #  define package dependencies
 define_dependencies(binEnv,'et evio codaobject')
 
 
 #  build programs
 buildBin(binEnv)

Note that the second argument to define_dependencies() is a space-separated list of package names (lower case). Fill in the names of all other packages your package must be compiled and linked with.  If you miss one you'll get a compiler or link error which should make it obvious what you forgot to list.

'''Important:  This should be the only modification to the build system you will need to make!'''

Behind the scenes system programmers create package definition files that, given the package root directory defined by environment variables, tell SCONS where to find the header files for the package, where the libraries reside, what libraries should be loaded, etc.  Contact a member of the Online group if one of these must be created for your package (only required if you want your library to be deployed in the production environment).