How to Add Constants

From GlueXWiki
Jump to: navigation, search

General Issues

The calibration constant database used by GlueX is the CCDB software package. The CCDB provides a feature-rich package for managing calibration constants, with a rich support for history under the philosophy that constants and tables should always be added but never deleted. Two different database backends are supported. The master database is stored in a MySQL database, which is accessible both inside and outside JLab. Standalone SQLite files are also supported for a variety of uses, including development and running on batch farms. Directions for managing SQLite files can be found here.

To run the CCDB program with a standalone SQLite DB, the CCDB_CONNECTION environmental variable should be set like this: (Note: csh syntax is used on this page)

setenv CCDB_CONNECTION sqlite:////path/to/sqlite.ccdb

To connect to the main MySQL database, set CCDB_CONNECTION like:

setenv CCDB_CONNECTION mysql://ccdb_user@hallddb.jlab.org/ccdb

The general work flow for developing new constants is to first use a private copy of the SQLIte CCDB DB to add new tables or new constants and to develop the code that uses them and to test their effects. Once you have finalized and tested the constants, use the same procedures and commands to add them to the master MySQL DB. When using the CCDB command line program, the only difference between using an SQLite and MySQL DB is the value of CCDB_CONNECTION, as described above.

Note that changes to the master MySQL database can only be made from machines at JLab, for security reasons. CCDB has a variety of useful features, including keeping a history of the changes made to constants tables. To make sure that your changes are properly attributed, please make sure to set the CCDB_USER environment variable to your username, e.g.

setenv CCDB_USER sdobbs

The creation of database tables is the beyond the scope of this page. For now, the best documentation is the excellent online help, which can be seen with the command "ccdb help mktbl". Please exercise caution since by design CCDB tables should not be deleted, and make such changes in consultation with the Calibration and Software Coordinators.

Adding Constants

Constants tables are stored in a directory-like hierarchy, which can be inspected using the command "ccdb ls". CCDB tables consist of multiple rows and columns. Information about the definition of a particular table can be found using the command "ccdb info", for example, "ccdb info /FCAL/gains".

New versions of constants are added to the CCDB through the use of text files, which have one line per table row and as many values in the line as table columns. For example, a constants file corresponding to a table with 3 columns and two rows is:

0.9  0.8  0.7
0.7  0.8  0.6

An example of adding a new version of constants to the table "/FCAL/gains" where the new constants are stored in the file "new_gains_file" is:

ccdb add /FCAL/gains new_gains_file

The CCDB powerful options for storing multiple versions of constants, but an important example is that sometimes different version of constants are used for Monte Carlo simulations than real data. An example of storing a version of constants for use in Monte Carlo simulation is

ccdb add /FCAL/gains::mc new_gains_file

For more complicated usages, please see the CCDB documentation.

Digi-level Constants

An important example of constants are those used to convert the raw digitized hits off of the detector to the calibrated hits used for the reconstruction procedures. The mapping between detector coordinates and channel number can be compiicated, so a series of scripts has been developed to make this procedure easier. The scripts can currently be obtained using the following command

svn co https://halldsvn.jlab.org/repos/trunk/home/sdobbs/calibration_scripts/flatten_tables

To build a file that can be inserted into the CCDB, we want to use the script build_calib_table.py. To get more information on the script, use the command

./build_calib_table.py -help

This script takes as input a text file with the detector index followed by the constant value and turns it into the format needed for one of the digi-level tables in the CCDB.

For example, the BCAL channels are indexed by <module layer sector end>, so the input file to the script would look like

1 1 1 0 0.575
1 1 1 1 0.769
1 1 2 0 0.902
1 1 2 1 0.480
...

An example of how to run the script is:

./build_calib_table.py BCAL BCAL_vals.txt

This produces a text file that can be used in the "ccdb add" command described in the previous section.