GlueX Offline Meeting, August 19, 2015

From GlueXWiki
Jump to: navigation, search

GlueX Offline Software Meeting
Wednesday, August 19, 2015
1:30 pm EDT
JLab: CEBAF Center F326/327


  1. Announcements
    1. build_scripts repository moved to GitHub
    2. Offline Monitoring (Kei) [links from last week's calibration meeting]
  2. Review of minutes from August 5 (all)
  3. Software for Correcting for Pedestal Drifts (Elton)
  4. Data Challenge 3 update (Mark)
  5. Spring 2015 Commissioning Simulations (Sean)
  6. Geant4 Update (Richard, David)
  7. Hall D Package Manager Update (Nathan)
  8. Flash ADC Pulse Processing Emulation: who is doing what? (all)
  9. Who is responsible for deleting Git branches?
  10. Default for builds: with debug symbols or without?
  11. Action Item Review

Communication Information

Remote Connection


Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2015 on the JLab CUE. This directory is accessible from the web at .



  • FIU: Mahmoud Kamel
  • FSU: Brad Cannon, Aristeidis Tsaris
  • JLab: Alex Barnes, Amber Boehnlein, Mark Ito (chair), David Lawrence, Paul Mattione, Elton Smith, Nathan Sparks, Justin Stevens, Simon Taylor, Beni Zihlmann
  • NU: Sean Dobbs
  • UConn: Richard Jones


  1. The build_scripts repository moved to GitHub earlier this week. On a related policy note, we decided that if a repository is maintained by a single person, then we can skip the pull-request/merge-by-someone-else procedure and allow the maintainer to push directory to the repository.
  2. Kei Moriya posted links from last week's calibration meeting on Offline Monitoring (see below). Paul mentioned that the last launch went smoothly and Kei is feeding back suggestions for SWIF to Chris Larrieu and has written a GlueX-oriented SWIF wrapper, hdswif. The next launch will be next week.

Review of minutes from August 5

We went over the minutes from last time. There was no significant discussion.

Software for Correcting for Pedestal Drifts

Elton raised the issue of how to apply pedestal corrections that occur on a time scale smaller than a single run. There are several online and offline aspects to the problem and the method is not settled on either side. We discussed several aspects of the problem.

  • The pedestals have been observed to drift significantly on a scale of a few hours. We might need to change pedestal a values on a 10 minute time scale.
  • The CCDB is currently set up to handle constants on a run-by-run basis. The conceptual design for event-dependent constants has been worked out, but nothing has been coded.
  • JANA already has the hooks for obtaining event-dependent constants from the CCDB when/if that functionality comes along.
  • There is a potential problem with constants that apply to event ranges since with multiple threads not every thread will cross a given event boundary at the same time. This could be solved by using the "event barriers" David recently introduced in JANA. This mechanism requires that all threads synchronize on a designated point in the input data stream, each thread not going past it until all threads have caught up. Event barriers can be triggered without dependence on special event types in the data stream. The development of this mechanism was originally motivated by the idea of using data from EPICS events in the data stream in the reconstruction.
  • There may be a possible bias in pedestals since read-out is only done for hits; there will always be a hit nearby in time when obtaining pedestals.
  • Depending on the online scheme, we may obtain pedestals on an event-by-event basis, or average them over a range of events. In the former case, calibration constants are not needed; the pedestals are embedded in the event stream.

We did not decide on a definite action plan. The character of the online solution is still evolving and that will largely dictate the offline approach.

Data Challenge 3 update

Mark has produced about half of 5000 input files and written them to the tape library. The files are 10 GB and contain about 430 k events each. Each file takes about a day of CPU time to produce. These files have also been reconstructed in a practice run on the data. Processing is ongoing.

Spring 2015 Commissioning Simulations

Mark reported that all of the 30,600 jobs are complete, both event generation and reconstruction. The "raw" data (mcsmear output, HDDM format), and the REST files have been archived to the tape library and are now available on the cache disk under /cache/mss/halld/detcom/02/smeared and rest respectively.

Sean mentioned that Elton was interested in simulations using the latest version of the simulation and reconstruction (more recent than what we used). He also cautioned that the new start counter reconstruction does not have correspondingly new constants for simulated data yet. Mark will look into tagging a release with the latest code save the latest start counter commit.

Geant4 Update

Richard reported on his detailed comparison of tracks between Geants 3 and 4. The results from a sample of 2:

  1. track 1: perfect agreement for about 750 steps
  2. track 2: spectacular disagreement

He traced the discrepancy to a feature implemented in Geant 3 where the magnetic field was turned off on the calorimeters to speed up shower development. This, in fact, is where most of the simulation time is spent. He found a subtle bug having to do with how the magnetic field is inherited from mother volumes in Geant 4 which needs to be addressed.

He will look at the cost in Geant 4 of leaving the field on and report to the group. He was hopeful that the CPU time penalty would not be too bad and we can run with the field on the calorimeters. This mode would be important for charged particles punching through the calorimeter, of particular interest for the charge pion polarizability experiment.

He is also working to complete the class that allows each straw to declare its number.

Richard also needs to test the Geant4 multi-threaded tracking facility.

David will work on implementing sensitive detectors along the lines used for the CPP.

Hall D Package Manager Update

Nathan gave us an update on the current state of the project for facilitating builds of GlueX software. See his slides for details of his presentation. The major changes since his last report to the group are (from slide 10):

  • Name change to hdpm from HDBuildManager
  • hdpm command user-interface
  • Check dependencies within listed packages
  • Control build commands with "commands.txt"
  • Support for geant4 and amptools
  • Setup scripts for 64-bit Linux and Mac OS X

He urged us to give it a try. An extensive README is available on GitHub.

Flash ADC Pulse Processing Emulation: who is doing what?

David described some of the current issues with emulation of the firmware algorithm used in the Flash ADCs.

  1. The current code does not reproduce the results of the firmware exactly.
  2. Parameters for the algorithm are different for different detectors. There is not mechanism at present to retrieve these parameters at reconstruction time.
  3. There are several people working on the problem, including Mike Staib, Mark Dalton and Kei.

To solve the second issue, in addition to access to the parameters themselves, one requires access to the translation tables to identify the relevant detector.

David will be calling a meeting to bring interested parties together, discuss the needed infrastructure, and try to avoid duplication of effort.

Who is responsible for deleting Git branches?

The sim-recon Git repository is starting to accumulate a significant number of branches from code improvements pushed upstream. Most can be deleted, but we have not assigned responsibility for doing so within the group.

Mark proposed a policy where the developer that originally pushed the changes is responsible for deleting branches after merge. There was some discussion about whether having "person doing the merge does the delete" instead. That might be more efficient, but Mark argued that the person pushing the changes was in the best position to judge. In the end we decided to try Mark's proposal.

Default for builds: with debug symbols or without?

Nathan presented results of studies on performance of both the GCC and the Clang C++ compilers comparing performance with and without the production of debugging symbols. The symbols are not only important for running the debugger, but give line number information on crashes when binaries are run normally. Compilation is slowed significantly and the disk footprint is an order of magnitude bigger with the symbols.

The current default is to build with debugging symbols. Although Nathan proposed changing the default, the group endorsed David's original judgment that having symbols is worth the cost.

We also talked a bit about Clang. It seems to perform marginally better than GCC. One major advantage that people reported is that the error messages produced at compiler time are much easier to understand, no small advantage. Clang also seems to be available as a package in most common Linux distributions.

Action Item Review

We did not get to this; the room was emptying quickly at this point. ;-)