Difference between revisions of "Offline Analysis Commissioning"

From Hall D Ops Wiki
Jump to: navigation, search
(→‎Offline Monitoring Plan)
Line 23: Line 23:
 
** David is updating RootSpy and its documentation/instructions.
 
** David is updating RootSpy and its documentation/instructions.
 
** When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.  
 
** When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.  
−
** Make sure online monitoring histograms are archived somewhere.  
+
** Make sure online monitoring histograms are (at least temporarily) on the ifarm work disk somewhere for quick offline access.  
  
 
* Detector groups:
 
* Detector groups:
Line 29: Line 29:
 
** Determining which plots are the primary plots (most important for shift-takers) and which are diagnostic plots.  
 
** Determining which plots are the primary plots (most important for shift-takers) and which are diagnostic plots.  
 
** Integrating their histograms into RootSpy.
 
** Integrating their histograms into RootSpy.
 +
 +
* Paul/Simon/Mark:
 +
** Will setup and maintain software for offline monitoring.
 +
 +
* Kei:
 +
** Will write and (periodically) submit jobs to the farm that run over small samples of the EVIO tape data and test re-making of the online monitoring histograms.
 +
 +
* Sean:
 +
** Will build (sqlite) database for storing key data monitoring information (new entry for each EVIO file).
 +
** Will write and launch job scripts that pull key data from the online monitoring histograms (after each run ends) and stores them in the database.
 +
*** These scripts should also grab key plots for the webpage and save the png(s) to disk.
 +
 +
* Justin:
 +
** Will build webpage(s) for viewing raw data for: Today's runs (primary plots), yesterday's runs (primary plots), past run history (trends from database)
 +
** Will work with Sean to write scripts to ping database and grab histogram png(s) to update the webpages.
  
 
== Reconstruction Quality Monitoring ==
 
== Reconstruction Quality Monitoring ==
Line 69: Line 84:
 
<!--
 
<!--
  
−
# (Kei): Will setup an automatic nightly script that submits jobs to the farm, runs over the EVIO data on tape with all of the monitoring plugins (raw, calibration, reconstruction, analysis), and saves the ROOT files to disk.
 
−
# (Kei): Will periodically run a script that submits jobs to the farm that run over the EVIO data and remake the track reconstruction and calibration quality plots.
 
  
 
* How do the online histograms change as a function of time (e.g. hits/event vs run number)?
 
* How do the online histograms change as a function of time (e.g. hits/event vs run number)?
Line 76: Line 89:
 
* Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.  
 
* Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.  
 
* Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).  
 
* Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).  
−
* Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.
+
 
−
** Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.
+
−
* Sean: Make cron-job scripts that read the data from the ROOT files and:
+
−
** Store appropriate data in an sqlite database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.
+
 
** Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. &theta;, etc.)
 
** Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. &theta;, etc.)
 
* Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
 
* Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:

Revision as of 19:32, 23 September 2014

Introduction

This document describes the goals of the data monitoring that will be carried out during the fall 2014 commissioning run of Hall D.

Raw Data

  • These questions are for every detector system: TAGH, TAGM, CDC, FDC, SC, TOF, BCAL, FCAL, PS (Pair Spectrometer)

Commissioning Tests

  • Are the detectors working?
    • Do all of the channels have hits?
    • What are the hit counts/rates per channel?
    • Are the energies & times OK, garbage, or out of range? (by channel)
  • Can we read data from tape?
  • Can we reproduce the online histograms with offline data?

Offline Monitoring Plan

  • Pull key data/histograms from online histograms, put data in database, make history & plots viewable on webpage
  • Periodically test that we can reproduce online histograms using on-tape EVIO data.

Action Items

  • David: Online data monitoring environment (RootSpy, hdview2, etc.)
    • David is updating RootSpy and its documentation/instructions.
    • When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
    • Make sure online monitoring histograms are (at least temporarily) on the ifarm work disk somewhere for quick offline access.
  • Detector groups:
    • Writing the monitoring plugins for their systems.
    • Determining which plots are the primary plots (most important for shift-takers) and which are diagnostic plots.
    • Integrating their histograms into RootSpy.
  • Paul/Simon/Mark:
    • Will setup and maintain software for offline monitoring.
  • Kei:
    • Will write and (periodically) submit jobs to the farm that run over small samples of the EVIO tape data and test re-making of the online monitoring histograms.
  • Sean:
    • Will build (sqlite) database for storing key data monitoring information (new entry for each EVIO file).
    • Will write and launch job scripts that pull key data from the online monitoring histograms (after each run ends) and stores them in the database.
      • These scripts should also grab key plots for the webpage and save the png(s) to disk.
  • Justin:
    • Will build webpage(s) for viewing raw data for: Today's runs (primary plots), yesterday's runs (primary plots), past run history (trends from database)
    • Will work with Sean to write scripts to ping database and grab histogram png(s) to update the webpages.

Reconstruction Quality Monitoring

Commissioning Tests

  • What is the calibration quality of each system?
  • Can we perform reconstruction for each system? (Tracks, showers, etc.)
  • Are there any regions of the detector where reconstruction is inefficient?
  • Are tracks being properly matched to hits in the other detectors?
  • What is the quality of the particle ID?

Offline Monitoring Plan

  • Periodically submit jobs to monitor calibration & reconstruction quality of data on tape.
    • Just a file or two from each run.
    • Submit either at some fixed-time interval (every 2 weeks?) or perhaps after big changes.
    • Save key data in database, make history & key plots viewable on webpage
  • When ready to do full reconstruction, run calibration & reconstruction quality plugins on all files, save key data to database & make viewable on webpage.

Action Items

  • Paul: Will integrate the monitoring_hists plugin (reconstruction) plots into RootSpy.
    • Will run full reconstruction over a small % of online data.
  • Detector groups:
    • Writing the calibration scripts/programs/plugins for their systems.
    • Determining which plots are the primary plots and which are diagnostic plots.

Analysis Quality Monitoring

Commissioning Tests

  • Can we see π0 peaks
  • Can we see simple final states
    • γ p → p π+ π-
    • γ p → p π+ π- π0
    • γ p → p π+ π- η

Offline Monitoring Plan

  • Periodically submit analysis jobs to study π0's and simple final states.
    • Make key plots viewable on webpage.


Retrieved from "https://halldweb.jlab.org/hdops/wiki/index.php?title=Offline_Analysis_Commissioning&oldid=1120"