September 19, 2014 Data Monitoring

From GlueXWiki
Jump to: navigation, search


  1. Discuss what we want to know, and what we want to do.
    1. Online monitoring.
    2. Offline monitoring.
  2. Assign tasks.

Online Data Monitoring

What we want to know

  • Are the detectors working?
    • Detector Systems: TAGH, TAGM, SC, CDC, FDC, TOF, BCAL, FCAL, PS (Pair Spectrometer)
    • Do all of the channels have hits?
    • What are the hit counts/rates per channel?
    • Are the energies & times OK, garbage, or out of range? (by channel)
  • Is the reconstruction working?
    • Run full reconstruction over a small % of data, view tracking/shower-reconstruction results online (hdview2, monitoring_hists plugins).


Action Items

  • Mike will point us to collected EVIO data (at least for the CDC) as it becomes available.
  • David(?) will make sure everything necesssary is viewable in hdview2.
  • David is updating RootSpy and the documentation.
    • When he's finished, he'll contact the different detector groups and remind them to write their online monitoring plugins.
  • We (Paul?, David?, Sean?) will stay in contact with the detector groups, and make sure that they:
    • Write the plugins
    • Are histogramming the data we want (see above).
    • Integrate them into RootSpy, and determine which are the primary plots, the ones that are the most important for shift-takers.
  • David will make sure that the RF time is included in the translation table.
  • Paul will make sure that the monitoring_hists plugin plots (for reconstruction quality) are integrated and viewable in RootSpy.

Offline Data Monitoring

What we want to know

  • Can we reproduce the online histograms offline:
    • Can we read data from tape?
    • Are all the channels present?
    • Are all the recorded values within the expected ranges?
  • How do the online histograms change as a function of time (e.g. hits/event vs run number)?
  • Calibration status/quality histograms (individual runs & a function of time)
  • Reconstruction status/quality histograms
    • Num tracks, showers, hits, etc.
    • Reconstructed track kinematics
    • Track / detector-hit matching
    • PID Quality
  • Analysis histograms
    • Can we see pi0 peaks
    • Can we see simple final states
      • γ p → p π+ π-
      • γ p → p π+ π- π0
      • γ p → p π+ π- η

Action Items - Now (With what is available)

  • Paul: Will contact David & Mark and find locations to host the websites, mysql database, and ROOT files.
  • Paul: Will contact David and make sure there is an easy way to access the configuration used for a given run (which detectors were connected/on/off, trigger configuration, did the DAQ crash or did the data get written to disk).
  • Paul, Sean: Make sure that the detector groups make plugins for doing the calibrations and viewing the quality/results.
  • Kei: Make cron-job scripts that automatically read EVIO data from tape, run the online/monitoring plugins, and save the ROOT files to disk.
    • Raw data (hit occupancies) need to only be read once per run, calibration/reconstruction data need to be rerun and updated periodically.
  • Sean: Make cron-job scripts that read the data from the ROOT files and:
    • Store appropriate data in an sqlite database (e.g. hit rates, # events, # reconstructed tracks/showers, calibration resolutions) as a function of run number.
    • Make png(?) files of important histograms on disk (hit occupancies, calibration plots, reconstructed track p vs. θ, etc.)
  • Kei/Justin: Make cron-job scripts that create a version-0, "just get something working" webpage for viewing:
    • Important histograms for each detector (png's created by Sean)
    • (w/ Sean's help for database access): Values in the mysql database as a function of run number.

Action Items - Near Future

  • Kei, Sean: As more detector plugins become available, expand the system for viewing their data.
  • Justin: Make a more sophisticated/elaborate series of scripts & webpages for viewing:
    • Today's & yesterday's online data
    • Primary plots from any given run (each detector system: hit occupancies, calibration quality, reconstruction counts)
    • Performance as a function of time (each detector system: hit occupancies, calibration quality, reconstruction counts)
    • Links to run configuration information

Example Links

Miscellaneous Useful Information

Commissioning Software

  • HDDS Commissioning Branch Link
  • sim-recon Commissioning Branch Link


  • How to: Link
    • If you are running over EVIO files created from simulated data, you need to run with the environment variable:

Running with B-field = 0

  • For the simulation, in the file, set:
  • For the reconstruction, set B=(0,0,0) everywhere by running with the command line argument:


  • Accessing Onsite Webpages from Offsite Link
  • RootSpy Website
  • RootSpy Source Code Link