Calibration Train

From GlueXWiki
Revision as of 16:57, 18 October 2016 by Sdobbs (Talk | contribs) (→‎Calibration run plan)

Jump to: navigation, search

Fall 2016 Processing

Job Structure

Currently, two passes are planned: One automated step, and one to produce outputs for calibration procedures that are still manual.

The calibrations/plugins that are run on each pass are:

  • Pass 1
    • RF_online, HLDetectorTiming, TOF_TDC_shift. To add: PS/Tagger timing?
  • Pass 2
    • Calibrations: BCAL_LEDonline, BCAL_point_time (?), CDC_TimeToDistance, FCALpedestals, HLDetectorTiming, imaging, PSC_TW, PS_timing, pedestals, pedestal_online(?), ST_Propagation_Time, st_tw_corr_auto, TAGH_timewalk, TAGM_TW
    • EVIO skims: FCAL pi0, BCAL pi0, BCAL-LED, FCAL-LED, random, sync
    • ROOT skims: TOF_calib
  • Incoming
    • CDC gain calibrations
    • BCAL LED monitoring

Calibration run plan

The two priority items are to recalibrate the TOF with the its new running conditions and to verify calibrations with the new fADC250 firmware. All calibrations should be checked, in any case.

  1. Standalone calibrations (no forward tracking)
    • ✓ RF time (Paul M.)
    • Pedestals (System owners)
    • BCAL Attenuation length/gain ratio (Mark D.)
    • CDC time to distance (Mike S.)
    • CDC gain (Naomi)
    • ✓ SC timewalk (Mahmoud)
    • TOF timing (Offsets/timewalks) (Beni)
    • ✓ TAGM timing (Alex B.)
    • ✓ TAGH timing (Nathan)
    • ✓ PS timing (Nathan)
    • ✓ Overall timing (rough) (Mike S./Sean)
  2. Full tracking calibrations
    • ✓ BCAL Effective velocities (George)
    • ✓ Overall timing (Mike S./Sean)
    • ✓ SC Propagation time (Mahmoud)
  3. ✓ BCAL/FCAL pi0 calibrations (Adesh/Will M.)

All calibrations except the pi0 calibration should take no more than 1-3 2h runs of data.

Frequency

  • Timing will be checked for each run. The known variations are:
    • TOF (run-to-run)
    • Tagger (~<day, corrected in run-to-run in the spring)
  • CDC gains vary with temperature and pressure, can be averaged over a 1-2 hour run.

All other calibrations have been seen (so far) to be stable on a ~several week timescale

To-dos

  1. Finish skim improvements
  2. Automate constant -> CCDB pipeline
  3. Implement tracking database

Spring 2016 Processing

Run Groups

SWIF workflow Run Range Total # Jobs Notes Total Skim Size (TB) FCAL Skim (TB) BCAL Skim (TB) PS Skim (TB)
GlueX-CalibRun-2016-02-25 10457 - 10529 first pass, abandoned
GlueX-CalibRun-2016-02-29 10531 - 10647 first pass, abandoned
GlueX-CalibRun-2016-03-04 10649 - 10724 first pass, abandoned
GlueX-CalibRun-2016-03-18 10331 - 10913 6570 second pass 23 8.1 0.94 14
GlueX-CalibRun-2016-04-04 11048 - 11145 2993 first pass 9.4 4.7 1.0 3.5
GlueX-CalibRun-2016-04-11 11150 - 11312 3573 first pass 9.7 6.2 1.4 2.1
GlueX-CalibRun-2016-04-27 11366 - 11668 10599 first pass 25 16 2.8 5.5
TOTAL 67 35 6 25

Job Structure

Currently 3 "passes" through the data are performed. The first pass tries to do as many calibrations as possible with one file of data. The second pass tries to do calibrations that need a larger data set. The final pass runs through a full run to use the full statistics in a run and to generate outputs for other calibration procedures that can't be done automatically yet (e.g. pi0 calibrations).

The plugins that are run on each pass are:

  • Pass 1
    • Step 1 - RF_online
    • Step 2 - HLDetectorTiming (coarse timing+ADC/TDC alignment),TOF_TDC_shift
    • Step 3 - HLDetectorTiming (track-based timing),BCAL_TDC_Timing (timewalks)
  • Pass 2
    • Step 1 - TAGH_timewalk,BCAL_attenlength_gainratio,BCAL_TDC_Timing (full)
    • Step 2 - st_tw_corr_auto
  • Pass 3
    • HLDetectorTiming,PSC_TW,BCAL_attenlength_gainratio,BCAL_gainmatrix,FCALgains,FCALpedestals,ST_Tresolution,ST_Propagation_Time,p2gamma_hists,imaging,pedestal_online,BCAL_LEDonline,PS_timing,TOF_calib,pi0fcalskim,pi0fcalskim,ps_skim

Output

The output of the calibration jobs can be found at this webpage or at /volatile/halld/home/gxproj3/calib_jobs . There is a subdirectory for each calibration launch. Each launch has a directory for each run that is processed. For each run, several types of outputs are kept

  • The results for the processing of each file are kept in a subdirectory with the same number as the file, in the format NNN
  • The summed results for each pass through the data are kept in ROOT files with names of the form" hd_calib_passN_RunRRRRRR.root"
  • The processed results for each pass (e.g. constants files and figures) are kept in subdirectories with names of the form "passN/"

The relevant directories are:

  • ROOT files & calibration constants: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/output
  • EVIO skims: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/skims

General Information

Instructions

The software from the calibration train can be obtained from 

git clone https://github.com/sdobbs/calibration_train


Job submission is controlled by the following configuration file: 

# data.config - example configuration file
jobname = GlueX-CalibRun-2016-04-27    # name of the SWIF workflow
# data/memory sizes in GB
mem_requested = 9                      # max vmem for jobs
disk_space = 30                        # max disk space for jobs
nthreads = 8                           # number of threads requested
# time in hours
time_limit = 36                        # max run time for jobs

# this file is used for debugging
ccdb_table_file = configs/calib_tables

Building a SWIF workflow is controlled by the job_manager.py command 

ifarm1102> ./job_manager.py 
usage: job_manager.py init [config_file]
       job_manager.py build [-z] [config_file] [run file]
       job_manager.py run [-L] [config_file]

Some descriptions of the sub-commands:

  • init - This takes the configuration file as an input, does the basic setup for the job: creates directories, saves configurations, builds CCDB SQLite file
  • build - This takes the configuration file and a file with one run number per line as an input, and creates the SWIF workflow
  • run - This just starts the SWIF workflow. Standard SWIF commands can be used to deal with the workflow from here.

Currently, there are two major commands to run

Calibration Challenge

Old Planning

Organization

  • The jobs will be submitted every Tuesday at noon, JLab time.
  • The jobs will be run from the gxproj3 account [parallel use with EventStore jobs]
  • The output of the jobs will be stored in ...

Run Ranges

The following runs will be processed:

RunPeriod-2015-03

  • 2931, 3079, 3179, 3180, 3183, 3185

Calibrations

Job Requirements

Each calibration process should include the following:

What is Being Run

The following plugins are currently being run:

  • RF_online (RF signal)
  • BCAL_TDC_Timing
  • HLDetectorTiming
  • PSC_TW

Working on adding:

Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Calibration_Train&oldid=77969"