Difference between revisions of "Offline Analysis Commissioning"

From Hall D Ops Wiki
Jump to: navigation, search
Line 1: Line 1:
 
__TOC__
 
__TOC__
 
==Introduction==
 
==Introduction==
This document describes the goals of the data monitoring that will be carried out during the fall 2014 commissioning run of Hall D.
+
* This document describes the goals & plans for the data monitoring that will be carried out during the fall 2014 commissioning run of Hall D.
  
== Raw Data ==
+
== Commissioning Tests ==
* These questions are for every detector system: TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
+
=== Raw Data ===
  
=== Commissioning Tests ===
 
 
* Are the detectors working?  
 
* Are the detectors working?  
 +
** TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
 
** Do all of the channels have hits?  
 
** Do all of the channels have hits?  
 
** What are the hit counts/rates per channel?
 
** What are the hit counts/rates per channel?
Line 14: Line 14:
 
* Can we reproduce the online histograms with offline data?
 
* Can we reproduce the online histograms with offline data?
  
=== Offline Monitoring Plan ===
+
=== Reconstruction Quality Monitoring ===
* Pull key data/histograms from online histograms, put data in database, make history & plots viewable on webpage
+
* Periodically test that we can reproduce online histograms using on-tape EVIO data.
+
 
+
=== Action Items ===
+
 
+
* David: Online data monitoring environment (RootSpy, hdview2, etc.)
+
** David is updating RootSpy and its documentation/instructions.
+
** When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
+
** Make sure online monitoring histograms are (at least temporarily) on the ifarm work disk somewhere for quick offline access.
+
 
+
* Detector groups:
+
** Writing the monitoring plugins for their systems.
+
** Determining which plots are the primary plots (most important for shift-takers) and which are diagnostic plots.
+
** Integrating their histograms into RootSpy.
+
 
+
* Paul (Mark/Simon?):
+
** Will setup and maintain software for offline monitoring.
+
 
+
* Kei:
+
** Will write and (periodically) submit jobs to the farm that run over small samples of the EVIO tape data and test re-making of the online monitoring histograms.
+
 
+
* Sean:
+
** Will build (sqlite) database for storing key data monitoring information (new entry for each EVIO file).
+
** Will write and launch job scripts that pull key data from the online monitoring histograms (after each run ends) and stores them in the database.
+
*** These scripts should also grab key plots for the webpage and save the png(s) to disk.
+
 
+
* Justin:
+
** Will build webpage(s) for viewing raw data for: Recent runs (last 24/48 hours, primary plots), past run history (trends of key data from database)
+
** Will work with Sean to write scripts to ping database and grab histogram png(s) to update the webpages.
+
 
+
== Reconstruction Quality Monitoring ==
+
  
=== Commissioning Tests ===
 
 
* What is the calibration quality of each system?
 
* What is the calibration quality of each system?
 +
** TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS
 
* Can we perform reconstruction for each system? (Tracks, showers, etc.)
 
* Can we perform reconstruction for each system? (Tracks, showers, etc.)
 
* Are there any regions of the detector where reconstruction is inefficient?  
 
* Are there any regions of the detector where reconstruction is inefficient?  
Line 54: Line 23:
 
* What is the quality of the particle ID?
 
* What is the quality of the particle ID?
  
=== Offline Monitoring Plan ===
+
=== Analysis Quality Monitoring ===
 +
 
 +
* Can we see &pi;<sup>0</sup> peaks
 +
* Can we see simple final states
 +
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup>
 +
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup> &pi;<sup>0</sup>
 +
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup> &eta;
 +
 
 +
== Offline Data Monitoring Plans ==
 +
 
 +
=== Raw Data ===
 +
* Pull key data/histograms from online histograms, put data in database, make history & plots viewable on webpage
 +
* Periodically test that we can reproduce online histograms using on-tape EVIO data.
 +
 
 +
=== Calibration/Reconstruction Quality ===
 
* Periodically submit jobs to monitor calibration & reconstruction quality of data on tape.  
 
* Periodically submit jobs to monitor calibration & reconstruction quality of data on tape.  
 
** Just a file or two from each run.  
 
** Just a file or two from each run.  
Line 61: Line 44:
 
* When ready to do full reconstruction, run calibration & reconstruction quality plugins on all files, save key data to database & make viewable on webpage.
 
* When ready to do full reconstruction, run calibration & reconstruction quality plugins on all files, save key data to database & make viewable on webpage.
  
=== Action Items ===
+
=== Analysis Quality ===
* Paul: Will integrate the monitoring_hists plugin (reconstruction) plots into RootSpy.
+
* Periodically submit analysis jobs to study &pi;<sup>0</sup>'s and simple final states, show results at meetings.
** Will run full reconstruction over a small % of online data.
+
 
 +
== Action Items ==
 +
 
 +
* David/Online: Online data monitoring environment (RootSpy, hdview2, etc.)
 +
** David is updating RootSpy and its documentation/instructions.
 +
** When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
 +
** Make sure online monitoring histograms are (at least temporarily) on the ifarm work disk somewhere for quick offline access.  
  
 
* Detector groups:
 
* Detector groups:
** Writing the calibration scripts/programs/plugins for their systems.  
+
** Writing the raw data monitoring plugins for their systems.  
** Determining which plots are the primary plots and which are diagnostic plots.  
+
** Determining which raw data plots are the primary plots (most important for shift-takers) and which are diagnostic plots.
 +
** Integrating their raw data histograms into RootSpy.
 +
** Writing the calibration scripts/programs/plugins for their systems, and updating the reconstruction software as needed.
 +
 
 +
* Paul (Mark/Simon?):
 +
** Will setup and maintain software for offline monitoring.
 +
** Will integrate the monitoring_hists plugin (reconstruction) plots into RootSpy.
  
 
* Kei:
 
* Kei:
** Will write and (periodically) submit jobs to the farm that run over a few tape files in each run and produce updated calibration & reconstruction histograms.  
+
** Will write and (periodically) submit jobs to the farm to:
 +
*** Test a subset of the data to see whether we can reproduce the online hisotgrams offline
 +
*** Produce updated calibration & reconstruction quality histograms (a few files per run)
 +
*** Periodically submit analysis jobs to study &pi;<sup>0</sup>'s and simple final states, and will show results at meetings.  
  
 
* Sean:  
 
* Sean:  
** Will store these data in the database.  
+
** Will build (sqlite) database for storing key data monitoring information (run meta info, and entries for each EVIO file).
 +
** Will write and launch scripts that pull key data from the online monitoring histograms (after each run ends) and stores them in the database.
 +
*** These scripts should also grab key plots for the webpage and save the png(s) to disk.  
  
 
* Justin:  
 
* Justin:  
** Will build webpage(s) for viewing reconstruction & calibration quality plots for each run, & trends for all runs.  
+
** Will build webpage(s) for viewing:
** Will work with Sean to write scripts to ping database and grab histogram png(s) to update the webpages.
+
*** Raw data for: Recent runs (last 24/48 hours, primary plots), past run history (trends of key data from database)
 +
*** Reconstruction & calibration quality plots for each run, & trends for all runs.  
 +
** Will work with Sean to write scripts to ping database for data and grab histogram png(s) to update the webpages.
  
== Analysis Quality Monitoring ==
 
 
=== Commissioning Tests ===
 
* Can we see &pi;<sup>0</sup> peaks
 
* Can we see simple final states
 
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup>
 
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup> &pi;<sup>0</sup>
 
** &gamma; p &rarr; p &pi;<sup>+</sup> &pi;<sup>-</sup> &eta;
 
 
=== Offline Monitoring Plan ===
 
* Kei:
 
** Periodically submit analysis jobs to study &pi;<sup>0</sup>'s and simple final states, show results at meetings.
 
  
 
<!--
 
<!--

Revision as of 22:13, 23 September 2014

Introduction

  • This document describes the goals & plans for the data monitoring that will be carried out during the fall 2014 commissioning run of Hall D.

Commissioning Tests

Raw Data

  • Are the detectors working?
    • TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
    • Do all of the channels have hits?
    • What are the hit counts/rates per channel?
    • Are the energies & times OK, garbage, or out of range? (by channel)
  • Can we read data from tape?
  • Can we reproduce the online histograms with offline data?

Reconstruction Quality Monitoring

  • What is the calibration quality of each system?
    • TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS
  • Can we perform reconstruction for each system? (Tracks, showers, etc.)
  • Are there any regions of the detector where reconstruction is inefficient?
  • Are tracks being properly matched to hits in the other detectors?
  • What is the quality of the particle ID?

Analysis Quality Monitoring

  • Can we see π0 peaks
  • Can we see simple final states
    • γ p → p π+ π-
    • γ p → p π+ π- π0
    • γ p → p π+ π- η

Offline Data Monitoring Plans

Raw Data

  • Pull key data/histograms from online histograms, put data in database, make history & plots viewable on webpage
  • Periodically test that we can reproduce online histograms using on-tape EVIO data.

Calibration/Reconstruction Quality

  • Periodically submit jobs to monitor calibration & reconstruction quality of data on tape.
    • Just a file or two from each run.
    • Submit either at some fixed-time interval (every 2 weeks?) or perhaps after big changes.
    • Save key data in database, make history & key plots viewable on webpage
  • When ready to do full reconstruction, run calibration & reconstruction quality plugins on all files, save key data to database & make viewable on webpage.

Analysis Quality

  • Periodically submit analysis jobs to study π0's and simple final states, show results at meetings.

Action Items

  • David/Online: Online data monitoring environment (RootSpy, hdview2, etc.)
    • David is updating RootSpy and its documentation/instructions.
    • When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
    • Make sure online monitoring histograms are (at least temporarily) on the ifarm work disk somewhere for quick offline access.
  • Detector groups:
    • Writing the raw data monitoring plugins for their systems.
    • Determining which raw data plots are the primary plots (most important for shift-takers) and which are diagnostic plots.
    • Integrating their raw data histograms into RootSpy.
    • Writing the calibration scripts/programs/plugins for their systems, and updating the reconstruction software as needed.
  • Paul (Mark/Simon?):
    • Will setup and maintain software for offline monitoring.
    • Will integrate the monitoring_hists plugin (reconstruction) plots into RootSpy.
  • Kei:
    • Will write and (periodically) submit jobs to the farm to:
      • Test a subset of the data to see whether we can reproduce the online hisotgrams offline
      • Produce updated calibration & reconstruction quality histograms (a few files per run)
      • Periodically submit analysis jobs to study π0's and simple final states, and will show results at meetings.
  • Sean:
    • Will build (sqlite) database for storing key data monitoring information (run meta info, and entries for each EVIO file).
    • Will write and launch scripts that pull key data from the online monitoring histograms (after each run ends) and stores them in the database.
      • These scripts should also grab key plots for the webpage and save the png(s) to disk.
  • Justin:
    • Will build webpage(s) for viewing:
      • Raw data for: Recent runs (last 24/48 hours, primary plots), past run history (trends of key data from database)
      • Reconstruction & calibration quality plots for each run, & trends for all runs.
    • Will work with Sean to write scripts to ping database for data and grab histogram png(s) to update the webpages.