Online builds 10-apr-2013

From GlueXWiki
Jump to: navigation, search

Release area and SVN file structures

  • Below is a proposed release directory structure.
  • Official release area on HDCC is /gluex/builds.
  • Release area only writeable from hdsys account, read-only from all other accounts.
  • Package directories are what gets checked into SVN (except jar and Linux dirs).
  • Packages can be checked in/out by anyone (but not from hdops account).
  • Anyone can build and install packages in their own area.
  • Only hdsys can release a package into official release area.
  • Using osrelease.pl (from offline) to define architecture/compiler directory names.
  • Notes:
    • Should SCONS dir be under packages? Should it be installed?
    • Need make_online_package script to set up dirs for new package.


-- /gluex/builds
   `-- devel
       |-- doc
       |-- jar
       |-- Linux_RHEL6-x86_64-gcc4.8.0
       |   |-- bin
       |   |-- include
       |   |-- lib
       |   `-- scripts
       |-- packages
       |   |-- aPackage
       |   |-- anotherPackage
       |   `-- cMsg2et
       |       |-- doc
       |       |-- java
       |           `-- org
       |               `-- jlab
       |                   `-- halld
       |       |-- jar
       |       |-- Linux_RHEL6-x86_64-gcc4.8.0
       |       |   |-- bin
       |       |   |-- include
       |       |   |-- lib
       |       |   `-- scripts
       |       `-- src
       |           |-- binsrc
       |           |-- examples
       |           |-- libsrc
       |           |   `-- cMsg2et -> .
       |           |-- scripts
       |           `-- test
       `-- SCONS


SCONS scripts

  • Need one SConstruct file in top directory of every package.
  • SConstruct should be the same for ALL packages (can be modified)
  • Need SConscript file in each libsrc,binsrc,scripts,test directory.
  • SConscript needed to define package dependencies (e.g. determine include paths for compiling).
  • Need one system-level Python script per package to define dependencies, where to find header files and libs, etc.
  • Define package root dirs via env vars $PACKAGE or $PACKAGEROOT (e.g. $EVIOROOT).
  • Note: always compiles with -g (takes a little extra room but no performance penalty).
  • Have not thought about Java yet (Ant or SCONS)


  • To build (there are many other options):
$ scons                 # build
$ scons install         # install to $INSTALL_DIR if defined, otherwise install locally
$ scons -c              # undo build
$ scons -c install      # undo install
$ scons debug=y         # link with -Og
# scons opt=3           # link with -O3


  • Parallel Builds
    • I have not considered parallel builds of the entire online software system, and am not sure if SCONS can take care of this.
    • May require a custom script that partially parallelizes the full build, or maybe each package needs an SConscript file instead of SConstruct, with a global SConstruct above.
    • Current goal is to get everything working at the package level so developers can start creating new packages.


SConstruct - one per package, same for all packages:

#  Generic SConstruct for Hall D online packages
#
#  ejw, 8-may-2013


from halld_lib import init_halld_env, build_halld


#  create standard environment based on command line variables
env = init_halld_env(Variables())


#  modify environment if desired


#  build everything based on environment
build_halld(env)


SConscript to build a library:

#  SConscript for libsrc
#
#  to customize fill in package dependencies
#
#  ejw, 16-apr-2013
 

Import('libEnv')
from halld_lib import define_dependencies, buildLib


#  define dependencies on other packages
define_dependencies(libEnv,'cmsg')



#  build the lib
buildLib(libEnv)


SConscript to build binaries:

#  SConscript for binsrc
#
#  to customize fill in package dependencies
#
#  ejw, 16-apr-2013


Import('binEnv')
from halld_lib import define_dependencies, buildBin


#  define dependencies on other packages
define_dependencies(binEnv,'codaobject cmsg evio et')


#  build programs
buildBin(binEnv)


System-level Python script to define e.g. codaObject dependencies:

import os

 def loadcodaobject(env) :
	OSENV = os.environ

        if(OSENV.has_key('CODAOBJECTROOT') and os.path.exists(OSENV['CODAOBJECTROOT'])==True):
                rootDir = OSENV['CODAOBJECTROOT']
        elif(OSENV.has_key('CODAOBJECT') and os.path.exists(OSENV['CODAOBJECT'])==True):
                rootDir = OSENV['CODAOBJECT']
        else:
                print "?Need to set CODAOBJECTROOT or CODAOBJECT env variable!"
                sys.exit(1)
 
        if env['SHOWENV'] == "1":
                print "Loading CODAOBJECT software from ", rootDir
 
 
#  include files
        codaobjectincs = []
        codaobjectincs.append(rootDir + '/include')
        env.AppendUnique(CPPPATH=codaobjectincs)
  
 
#  library directories
        codaobjectldir = []
        codaobjectldir.append(rootDir + '/lib')
        env.AppendUnique(LIBPATH = codaobjectldir)
 

#  libraries
        codaobjectlibs = []
        codaobjectlibs.append('codaObject')
        codaobjectlibs.append('pthread')
        codaobjectlibs.append('rt')
        codaobjectlibs.append('dl')
        env.AppendUnique(LIBS = codaobjectlibs)