GlueX Offline Meeting, February 3, 2016

From GlueXWiki
Jump to: navigation, search

GlueX Offline Software Meeting
Wednesday, February 3, 2016
1:30 pm EST
JLab: CEBAF Center F326/327


  1. Announcements
    1. sim-recon-1.9.0
    2. Compiler version survey results
    3. Proposed ROOT version bump
    4. More volatile space
      • quota increased from 35 TB to 40 TB
      • reservation increased from 10 TB to 20 TB
  2. Review of minutes from January 20 (all)
  3. Offline Monitoring (Paul): Monitoring Plan
  4. Farm usage for upcoming run and one-node/multi-threaded jobs (discussion with Sandy)
  5. Collaboration Meeting
  6. Geant4 Update (Richard, David)
  7. Review of recent pull requests (all)
  8. Data Challenge 3 update (Mark)
  9. Future Commissioning Simulations (all)
  10. Action Item Review

Communication Information

Remote Connection


Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2016 on the JLab CUE. This directory is accessible from the web at .



  • CMU: Naomi Jarvis
  • FIU: Mahmoud Kamel
  • JLab: Amber Boehnlein, Jie Chen, Mark Ito (chair), David Lawrence, Paul Mattione, Curtis Meyer, Sandy Philpott, Eric Pooser, Nathan Sparks, Justin Stevens, Simon Taylor
  • NU: Sean Dobbs


  1. sim-recon-1.9.0 has been released and built on the JLab CUE.
    • Simon mentioned that HDDS should be upgraded to remove the piece of plastic attached to the nose of the start counter in the repository.
  2. Compiler version survey results. Mark will propose moving to GCC 4.9.2 at the Collaboration Meeting.
  3. Proposed ROOT version bump. Mark did not receive any comment on his email. Unless objections are heard, we will go to 5.34.34 for the next release.
  4. More volatile space our quota (maximum possible use) increased from 35 TB to 40 TB and our reservation (guaranteed space) increased from 10 TB to 20 TB.

Offline Monitoring

Paul took us through his plan for monitoring and reconstructing the data from the upcoming run, emphasizing work to be done during the run itself. The main tasks are monitoring recently taken data, monitoring launches of all runs, and an initial comprehensive reconstruction pass.

Farm usage for upcoming run and one-node/multi-threaded jobs

With Paul's presentation as background we discussed with Sandy and Jie how Computer Center resources will be deployed to reconstruct the data.

  • We will use the standard method (jmigrate) for transferring raw data to tape, postponing use the write-through cache until we have more experience using it. We decided to go this route even though the write-through cache would give us access to the raw data without having to fetch them from tape.
  • We will try to exploit the new one-node/one-multi-threaded-job nodes on the farm. If more are needed, the Computer Center can re-configure standard nodes to this purpose.
  • We will likely need a new gxproj account to run the three tasks in parallel.
  • We expect that the data this run will be denser on average than before because of relatively more data taken in pulse-summary mode (mode 7).
  • The scripts for handling the incoming raw data task need to be re-written to use SWIF.
  • For one-node/one-job jobs, Paul will start out using the NTHREADS=NCORES option where the job senses the number of effective cores on the machine. This will result in 48 threads with our current nodes.
  • We will continue to submit one job per file, even with the larger number of threads.
  • The Lustre outages are much reduced recently. It is hope that running with a lower percentage of the disk space devoted to user storage, BIOS upgrades, and a return to default ZFS parameters will keep the system out of trouble. If not there is an option to switch out raid controllers in favor of host bus cards[?].

Collaboration Meeting

We added Paul to the agenda to talk about Offline Monitoring. He intends to be brief.

Data Challenge 3

Mark is copying raw data onto the tape library. After that he will delete the files freeing up a lot of space on the volatile disk.

Future Commissioning Simulations

Mark submitted 90 more jobs in addition to the 10 pilot jobs run last week.

We realized that the HDDS change Simon mentioned earlier (removing the plastic test target) should be incorporated in this round of simulation. Also Mark forgot to use run number 10,000 for the initial jobs. We will have to re-tag and re-run. Simon is working on improvements to the energy loss correction for electrons and positrons but those are not ready yet. On a related issue, Paul is almost done with his overhaul of kinematic fitting in the analysis library.

Justin mentioned that the JEF group is interested in using the results of this simulation round to estimate background levels for eta signals.

Using the "Issues" Feature of GitHub

Sean has recently submitted several sim-recon "issues" on the GitHub site. This is a convenient place to report problems or suggestions in a place where they can be discussed, documented, and referenced. In particular a pull request can refer to the issue it resolves. He already has received feedback on a couple of the recent submissions.

HDPM: Binary Distribution for OSX Available

Nathan announced that binary distributions for El Capitan are now available via the HDPM "fetch-dist" command. See the documentation on the HDPM wiki for details.