Difference between revisions of "Calibration Train"

From GlueXWiki
Jump to: navigation, search
(→‎To-dos)
(→‎Procedures)
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
__TOC__
 
__TOC__
  
−
= Fall 2016 Processing =
+
= Processing Overview =
 +
 
 +
* [[Old Calibration Train]]
  
 
== Job Structure ==  
 
== Job Structure ==  
Line 9: Line 11:
 
The calibrations/plugins that are run on each pass are:
 
The calibrations/plugins that are run on each pass are:
 
* '''Pass 1'''
 
* '''Pass 1'''
−
** RF_online, HLDetectorTiming, TOF_TDC_shift.  To add: PS/Tagger timing?
+
** Run as many calibrations as possible on one file
 +
*# Pass 1: RF_online
 +
*# Pass 2: HLDetectorTiming,TOF_TDC_shift
 +
*# Pass 3: st_tw_corr_auto [don't commit]
 +
*# Pass 4: HLDetectorTiming, CDC_amp, BCAL_TDC_Timing [time offsets, need to update]
 
* '''Pass 2'''
 
* '''Pass 2'''
−
** Calibrations:  BCAL_LEDonline, BCAL_point_time (?), CDC_TimeToDistance, FCALpedestals, HLDetectorTiming, imaging, PSC_TW,  PS_timing, pedestals, pedestal_online(?), ST_Propagation_Time, st_tw_corr_auto, TAGH_timewalk, TAGM_TW
+
** Process/skim full run
 +
** Calibrations:  BCAL_attenlength_gainratio, BCAL_LEDonline, CDC_amp,CDC_TimeToDistance, FCALpedestals, FCALpulsepeak, FCAL_TimingOffsets, HLDetectorTiming, imaging, PSC_TW,  PS_timing, pedestals,ST_Propagation_Time
 
** EVIO skims: FCAL pi0, BCAL pi0, BCAL-LED, FCAL-LED, random, sync
 
** EVIO skims: FCAL pi0, BCAL pi0, BCAL-LED, FCAL-LED, random, sync
 
** ROOT skims: TOF_calib
 
** ROOT skims: TOF_calib
 +
** Other [Monitoring]: BCAL_LED, BCAL_inv_mass, imaging, p2pi_hists, p3pi_hists
 
* '''Incoming'''
 
* '''Incoming'''
−
** CDC gain calibrations
+
** Tagger/PS workflow
 
** BCAL LED monitoring
 
** BCAL LED monitoring
  
Line 23: Line 31:
  
 
# Standalone calibrations (no forward tracking)
 
# Standalone calibrations (no forward tracking)
−
#* RF time (Paul M.)
+
#* ✓ RF time (Paul M.)
 
#* Pedestals (System owners)
 
#* Pedestals (System owners)
 
#* BCAL Attenuation length/gain ratio (Mark D.)
 
#* BCAL Attenuation length/gain ratio (Mark D.)
 
#* CDC time to distance (Mike S.)
 
#* CDC time to distance (Mike S.)
 
#* CDC gain (Naomi)
 
#* CDC gain (Naomi)
−
#* SC timewalk (Mahmoud)
+
#* ✓ SC timewalk (Mahmoud)
 
#* TOF timing (Offsets/timewalks) (Beni)
 
#* TOF timing (Offsets/timewalks) (Beni)
−
#* TAGM timing (Alex B.)
+
#* ✓ TAGM timing (Alex B.)
−
#* TAGH timing (Nathan)
+
#* ✓ TAGH timing (Nathan)
−
#* PS timing (Nathan)
+
#* ✓ PS timing (Nathan)  
−
#* TPOL calibration? (Nathan?)
+
#* ✓ Overall timing (rough) (Mike S./Sean)
−
#* Overall timing (rough) (Mike S./Sean)
+
 
# Full tracking calibrations
 
# Full tracking calibrations
−
#* BCAL Effective velocities (George)
+
#* ✓ BCAL Effective velocities (George)
−
#* Overall timing (Mike S./Sean)
+
#* ✓ Overall timing (Mike S./Sean)
−
#* SC Propagation time (Mahmoud)
+
#* ✓ SC Propagation time (Mahmoud)
−
# BCAL/FCAL pi0 calibrations (Adesh/Will M.)
+
# ✓ BCAL/FCAL pi0 calibrations (Adesh/Will M.)
  
 
All calibrations except the pi0 calibration should take no more than 1-3 2h runs of data.
 
All calibrations except the pi0 calibration should take no more than 1-3 2h runs of data.
 +
 +
=== Frequency ===
 +
 +
* Timing will be checked for each run.  The known variations are:
 +
** TOF (run-to-run)
 +
** Tagger (~<day, corrected in run-to-run in the spring)
 +
* CDC gains vary with temperature and pressure, can be averaged over a 1-2 hour run.
 +
 +
All other calibrations have been seen (so far) to be stable on a ~several week timescale
  
 
=== To-dos ===
 
=== To-dos ===
  
−
# Finish skim improvements
+
# <strike>Finish skim improvements</strike>
 
# Automate constant -> CCDB pipeline
 
# Automate constant -> CCDB pipeline
 
# Implement tracking database
 
# Implement tracking database
  
−
= Spring 2016 Processing =
+
= Procedures =
  
−
== Run Groups ==
+
=== How to start a new run period ===
 +
 
 +
# Edit configuration file, e.g., $CALIBRATION_TRAIN/configs/data.config
 +
#* Change job name to the current run period and set resource limits
 +
# Edit $CALIBRATION_TRAIN/template/job_wrapper.sh
 +
#* Set run period and version number
 +
# Set up the files and directories
 +
#* e.g. python setup_run.py configs/data.config
 +
# Create workflows
 +
#* swif create -workflow GXCalib-2017-01-pass1
 +
#* swif create -workflow GXCalib-2017-01-pass2
 +
# Create SQLite CCDB
 +
#* $CCDB_HOME/scripts/mysql2sqlite/mysql2sqlite.sh -hhallddb.jlab.org -uccdb_user ccdb | sqlite3 ccdb.sqlite
 +
#* mv ccdb.sqlite somewhere
 +
# Launch pass1 jobs
 +
#* python run_jobs_p1.py 2017-01 run_lists/f17.test
 +
 
 +
= Calibration Outputs =
 +
 
 +
== RunPeriod-2017-01 ==
  
 
{| class="wikitable"
 
{| class="wikitable"
−
! SWIF workflow
 
 
! Run Range
 
! Run Range
 +
! Version tag
 
! Total # Jobs
 
! Total # Jobs
 
! Notes
 
! Notes
 
! Total Skim Size (TB)
 
! Total Skim Size (TB)
−
! FCAL Skim (TB)
+
! BCAL pi0 (GB)
−
! BCAL Skim (TB)
+
! BCAL LED (GB)
 +
! FCAL pi0 (GB)
 +
! FCAL LED (GB)
 
! PS Skim (TB)
 
! PS Skim (TB)
 +
! TOF Skim (TB)
 
|-
 
|-
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-02-25 GlueX-CalibRun-2016-02-25] || 10457 - 10529 || ||  first pass, abandoned
+
| 30274 - 30621
−
|-
+
| ver01
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-02-25 GlueX-CalibRun-2016-02-29] || 10531 - 10647 || || first pass, abandoned
+
| 13362
−
|-
+
|  
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-02-25 GlueX-CalibRun-2016-03-04] || 10649 - 10724 || || first pass, abandoned
+
| 11.2
−
|-
+
| 338
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-03-18/  GlueX-CalibRun-2016-03-18] || 10331 - 10913 || 6570 || second pass || 23 || 8.1 || 0.94 || 14
+
| 74
−
|-
+
| 862
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-04-04/  GlueX-CalibRun-2016-04-04] || 11048 - 11145 || 2993 || first pass || 9.4 || 4.7 || 1.0 || 3.5
+
| 147
−
|-
+
| 6.36
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-04-11/  GlueX-CalibRun-2016-04-11] || 11150 - 11312 || 3573 || first pass || 9.7 || 6.2 || 1.4 || 2.1
+
| 1.63
−
|-
+
−
| [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-04-27/  GlueX-CalibRun-2016-04-27] || 11366 - 11668 || 10599 || first pass || 25  || 16 || 2.8 || 5.5
+
 
|-
 
|-
−
| '''TOTAL''' || || || || '''67''' || '''35''' || '''6''' || '''25'''
+
| 30622 - 30959
 +
| ver02
 +
| 13783
 +
| looser BCAL pi0 cuts
 +
| 23.3
 +
| 2379
 +
| 131
 +
| 1703
 +
| 297
 +
| 12.7
 +
| 3.36
 +
|-
 +
| 30960 -
 +
| ver03
 +
| 14869
 +
| looser BCAL pi0 cuts
 +
| 13.4
 +
| 1368
 +
| 71
 +
| 1000
 +
| 142
 +
| 7.47
 +
| 1.99
 
|}
 
|}
  
−
== Job Structure ==
+
* Skim files can be found in the following directory: /cache/halld/RunPeriod-2017-01/calib/ver01
−
 
+
** BCAL-LED - BCAL LED triggered events
−
Currently 3 "passes" through the data are performed. The first pass tries to do as many calibrations as possible with one file of data.  The second pass tries to do calibrations that need a larger data set.  The final pass runs through a full run to use the full statistics in a run and to generate outputs for other calibration procedures that can't be done automatically yet (e.g. pi0 calibrations).
+
** BCAL_pi0 - BCAL pi0 candidates
−
 
+
** FCAL-LED - FCAL LED triggered events
−
The plugins that are run on each pass are:
+
** FCAL_pi0 - FCAL pi0 candidates
−
* '''Pass 1'''
+
** PS - PS triggered events
−
** Step 1 - RF_online
+
** random - random (out-of-time) triggered events
−
** Step 2 - HLDetectorTiming (coarse timing+ADC/TDC alignment),TOF_TDC_shift
+
** sync - TS sync events
−
** Step 3 - HLDetectorTiming (track-based timing),BCAL_TDC_Timing (timewalks)
+
** TOF - TOF calibration ROOT skim
−
* '''Pass 2'''
+
−
** Step 1 - TAGH_timewalk,BCAL_attenlength_gainratio,BCAL_TDC_Timing (full)
+
−
** Step 2 - st_tw_corr_auto
+
−
* '''Pass 3'''
+
−
** HLDetectorTiming,PSC_TW,BCAL_attenlength_gainratio,BCAL_gainmatrix,FCALgains,FCALpedestals,ST_Tresolution,ST_Propagation_Time,p2gamma_hists,imaging,pedestal_online,BCAL_LEDonline,PS_timing,TOF_calib,pi0fcalskim,pi0fcalskim,ps_skim
+
−
 
+
−
== Output ==
+
−
 
+
−
The output of the calibration jobs can be found at [https://halldweb.jlab.org/calib_challenge/GlueX-CalibRun-2016-02-25/ this webpage] or at /volatile/halld/home/gxproj3/calib_jobs .  There is a subdirectory for each calibration launch.  Each launch has a directory for each run that is processed.  For each run, several types of outputs are kept
+
−
* The results for the processing of each file are kept in a subdirectory with the same number as the file, in the format NNN
+
−
* The summed results for each pass through the data are kept in ROOT files with names of the form" hd_calib_passN_RunRRRRRR.root"
+
−
* The processed results for each pass (e.g. constants files and figures) are kept in subdirectories with names of the form "passN/"
+
−
 
+
−
The relevant directories are:
+
−
* ROOT files & calibration constants: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/output
+
−
* EVIO skims: /volatile/halld/home/gxproj3/calib_jobs/[workflow]/skims
+
−
 
+
−
= General Information =
+
−
 
+
−
== Instructions ==
+
−
 
+
−
The software from the calibration train can be obtained from
+
−
<pre>
+
−
git clone https://github.com/sdobbs/calibration_train
+
−
</pre>
+
−
 
+
−
 
+
−
Job submission is controlled by the following configuration file:
+
−
<pre>
+
−
# data.config - example configuration file
+
−
jobname = GlueX-CalibRun-2016-04-27    # name of the SWIF workflow
+
−
# data/memory sizes in GB
+
−
mem_requested = 9                      # max vmem for jobs
+
−
disk_space = 30                        # max disk space for jobs
+
−
nthreads = 8                          # number of threads requested
+
−
# time in hours
+
−
time_limit = 36                        # max run time for jobs
+
−
 
+
−
# this file is used for debugging
+
−
ccdb_table_file = configs/calib_tables
+
−
</pre>
+
−
 
+
−
Building a SWIF workflow is controlled by the job_manager.py command
+
−
<pre>
+
−
ifarm1102> ./job_manager.py
+
−
usage: job_manager.py init [config_file]
+
−
      job_manager.py build [-z] [config_file] [run file]
+
−
      job_manager.py run [-L] [config_file]
+
−
</pre>
+
−
Some descriptions of the sub-commands:
+
−
* '''init''' - This takes the configuration file as an input, does the basic setup for the job: creates directories, saves configurations, builds CCDB SQLite file
+
−
* '''build''' - This takes the configuration file and a file with one run number per line as an input, and creates the SWIF workflow
+
−
* '''run''' - This just starts the SWIF workflow.  Standard SWIF commands can be used to deal with the workflow from here.
+
−
 
+
−
Currently, there are two major commands to run
+
−
 
+
−
= Calibration Challenge =
+
−
 
+
−
* [[Calibration Challenge|Calibration Challenge 1]]
+
−
 
+
−
= Old Planning =
+
−
 
+
−
== Organization ==
+
−
 
+
−
* The jobs will be submitted every Tuesday at noon, JLab time.
+
−
* The jobs will be run from the gxproj3 account  [parallel use with EventStore jobs]
+
−
 
+
−
* The output of the jobs will be stored in ...
+
−
 
+
−
== Run Ranges ==
+
−
 
+
−
The following runs will be processed:
+
−
 
+
−
'''RunPeriod-2015-03'''
+
−
* 2931, 3079, 3179, 3180, 3183, 3185
+
−
 
+
−
== Calibrations ==
+
  
−
=== Job Requirements ===
+
<!--
 +
ver01:
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/hists
 +
total = 1673.9482654 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/BCAL-LED
 +
total = 74.0351243354 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/BCAL_pi0
 +
total = 338.330394119 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/FCAL-LED
 +
total = 146.729467098 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/FCAL_pi0
 +
total = 862.115063164 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/PS
 +
total = 6364.3464742 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/TOF
 +
total = 1636.12805751 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/random
 +
total = 128.437043864 GB
 +
/mss/halld/RunPeriod-2017-01/calib/ver01/sync
 +
total = 3.4401245974 GB
  
−
Each calibration process should include the following:
 
−
* Plugin stored in standard location
 
−
** https://halldsvn.jlab.org/repos/trunk/sim-recon/src/plugins/Calibration
 
−
* ROOT/other scripts stored in standard location
 
−
** https://halldsvn.jlab.org/repos/trunk/sim-recon/src/scripts/calibrations ?
 
−
** Individual git repos?
 
−
* Output to files
 
−
* QA routines
 
  
−
=== What is Being Run ===
+
ver02:
  
−
The following plugins are currently being run:
 
−
* RF_online (RF signal)
 
−
* BCAL_TDC_Timing 
 
−
* HLDetectorTiming 
 
−
* PSC_TW
 
  
−
Working on adding:
+
ver03:
−
* BCAL_attenlength_gainratio
+
-->
−
* BCAL gains - /work/halld/home/wmcginle/Gain_Calib
+
−
* TOF calibrations - https://halldsvn.jlab.org/repos/trunk/home/zihlmann/TOF_calib/
+

Latest revision as of 23:53, 16 January 2018

Processing Overview

Job Structure

Currently, two passes are planned: One automated step, and one to produce outputs for calibration procedures that are still manual.

The calibrations/plugins that are run on each pass are:

  • Pass 1
    • Run as many calibrations as possible on one file
    1. Pass 1: RF_online
    2. Pass 2: HLDetectorTiming,TOF_TDC_shift
    3. Pass 3: st_tw_corr_auto [don't commit]
    4. Pass 4: HLDetectorTiming, CDC_amp, BCAL_TDC_Timing [time offsets, need to update]
  • Pass 2
    • Process/skim full run
    • Calibrations: BCAL_attenlength_gainratio, BCAL_LEDonline, CDC_amp,CDC_TimeToDistance, FCALpedestals, FCALpulsepeak, FCAL_TimingOffsets, HLDetectorTiming, imaging, PSC_TW, PS_timing, pedestals,ST_Propagation_Time
    • EVIO skims: FCAL pi0, BCAL pi0, BCAL-LED, FCAL-LED, random, sync
    • ROOT skims: TOF_calib
    • Other [Monitoring]: BCAL_LED, BCAL_inv_mass, imaging, p2pi_hists, p3pi_hists
  • Incoming
    • Tagger/PS workflow
    • BCAL LED monitoring

Calibration run plan

The two priority items are to recalibrate the TOF with the its new running conditions and to verify calibrations with the new fADC250 firmware. All calibrations should be checked, in any case.

  1. Standalone calibrations (no forward tracking)
    • ✓ RF time (Paul M.)
    • Pedestals (System owners)
    • BCAL Attenuation length/gain ratio (Mark D.)
    • CDC time to distance (Mike S.)
    • CDC gain (Naomi)
    • ✓ SC timewalk (Mahmoud)
    • TOF timing (Offsets/timewalks) (Beni)
    • ✓ TAGM timing (Alex B.)
    • ✓ TAGH timing (Nathan)
    • ✓ PS timing (Nathan)
    • ✓ Overall timing (rough) (Mike S./Sean)
  2. Full tracking calibrations
    • ✓ BCAL Effective velocities (George)
    • ✓ Overall timing (Mike S./Sean)
    • ✓ SC Propagation time (Mahmoud)
  3. ✓ BCAL/FCAL pi0 calibrations (Adesh/Will M.)

All calibrations except the pi0 calibration should take no more than 1-3 2h runs of data.

Frequency

  • Timing will be checked for each run. The known variations are:
    • TOF (run-to-run)
    • Tagger (~<day, corrected in run-to-run in the spring)
  • CDC gains vary with temperature and pressure, can be averaged over a 1-2 hour run.

All other calibrations have been seen (so far) to be stable on a ~several week timescale

To-dos

  1. Finish skim improvements
  2. Automate constant -> CCDB pipeline
  3. Implement tracking database

Procedures

How to start a new run period

  1. Edit configuration file, e.g., $CALIBRATION_TRAIN/configs/data.config
    • Change job name to the current run period and set resource limits
  2. Edit $CALIBRATION_TRAIN/template/job_wrapper.sh
    • Set run period and version number
  3. Set up the files and directories
    • e.g. python setup_run.py configs/data.config
  4. Create workflows
    • swif create -workflow GXCalib-2017-01-pass1
    • swif create -workflow GXCalib-2017-01-pass2
  5. Create SQLite CCDB
    • $CCDB_HOME/scripts/mysql2sqlite/mysql2sqlite.sh -hhallddb.jlab.org -uccdb_user ccdb | sqlite3 ccdb.sqlite
    • mv ccdb.sqlite somewhere
  6. Launch pass1 jobs
    • python run_jobs_p1.py 2017-01 run_lists/f17.test

Calibration Outputs

RunPeriod-2017-01

Run Range Version tag Total # Jobs Notes Total Skim Size (TB) BCAL pi0 (GB) BCAL LED (GB) FCAL pi0 (GB) FCAL LED (GB) PS Skim (TB) TOF Skim (TB)
30274 - 30621 ver01 13362 11.2 338 74 862 147 6.36 1.63
30622 - 30959 ver02 13783 looser BCAL pi0 cuts 23.3 2379 131 1703 297 12.7 3.36
30960 - ver03 14869 looser BCAL pi0 cuts 13.4 1368 71 1000 142 7.47 1.99
  • Skim files can be found in the following directory: /cache/halld/RunPeriod-2017-01/calib/ver01
    • BCAL-LED - BCAL LED triggered events
    • BCAL_pi0 - BCAL pi0 candidates
    • FCAL-LED - FCAL LED triggered events
    • FCAL_pi0 - FCAL pi0 candidates
    • PS - PS triggered events
    • random - random (out-of-time) triggered events
    • sync - TS sync events
    • TOF - TOF calibration ROOT skim


Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Calibration_Train&oldid=84847"