GlueX Offline Meeting, July 20, 2016

From GlueXWiki
Jump to: navigation, search

GlueX Offline Software Meeting
Wednesday, July 20, 2016
1:30 pm EDT
JLab: CEBAF Center F326/327


  1. Announcements
    1. dc3 retired: gxproj2 account free now
    2. Analysis Launch (Paul)
    3. Other announcements?
  2. Review of minutes from the last meeting (all)
  3. simX.X (Sean)
  4. Changes to mcsmear in sim-recon (Sean)
  5. ROOT 6 upgrade? (Mark)
  6. Multi-threaded REST file i/o with parallel hddm (Richard)
  7. Review of recent pull requests (all)
  8. Review of recent discussion on the Gluex Software Help List.
  9. Action Item Review

Communication Information

Remote Connection


Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2016 on the JLab CUE. This directory is accessible from the web at .



  • CMU: Mike Staib
  • FIU: Mahmoud Kamel
  • FSU: Brad Cannon
  • JLab: Alexander Austregesilo, Amber Boehnlein, Mark Ito (chair), David Lawrence, Paul Mattione, Nathan Sparks, Justin Stevens, Simon Taylor
  • NU: Sean Dobbs
  • Regina: Tegan Beattie
  • UConn: Richard Jones

You can view a recording of this meeting on the BlueJeans site.


  1. dc3 retired. Mark has officially put Data Challenge 3 out of its misery. The gxproj2 account is free now.
  2. Analysis Launch: Paul reminded us that plugins for the analysis launch are due this Friday at 5 pm.
    • The topic sparked a discussion of how to manage the ever growing number of plugins in sim-recon. They have now significantly increase the build time and build build disk footprint. Also many perform important, central functions, some are very close to private code. Some kind of management scheme is called for. Note that this is in some ways the kind of problem we want to have; too much coding going on out there. Some ideas:
      1. Move plugins to a separate plugins-only Git repository. (Mark)
      2. Reduce the number of plugins that are built automatically. (Justin)
      3. Have finer-grained build targets for SCons so that parts of the build can be skipped depending on the needs of the user. (Sean)
      4. Break sim-recon into functionally distinct (though interdependent) parts: simulation, reconstruction, and analysis (Nathan)
        • Dependence on AmpTools might also need some consideration in such a scheme (Justin)

Review of minutes from the last meeting

We went over the minutes from the meeting on July 6.

  • After the Lustre upgrade, the performance of the system has been a lot steadier. There have not been any reported hangs of the ifarm since. Justin and Sascha Somov have reported irregularities, but those were cleaned up by IT without loss of data.
  • sim-recon version 2.2.0 has been released. It fixes a problem with the non-linear energy correction in the FCAL and is being used in sim1.1.
  • REST data from phase 2 of the data reconstruction launches has been distributed to institutions outside the Lab.
  • Paul reported that the I/O contention reported by Mike when reading HDDM-formatted data is not an issue with the upcoming Analysis Launch. This is probably because of the large number of plugins in the Launch configuration.

ROOT 6 Upgrade

It looks like we are not ready to pull the trigger on this yet. Richard reported a problem building ROOT 6 with GCC 4.9.2. Nathan will consult with him on tracking it down. There will be dislocations due to old macros and private code not working when the change is made, but it is something we will have to deal with sooner or later. Justin and Sean pointed out we need to make the switch well in advance of the Fall run if at all. We will revisit the issue in two weeks time.

Multi-threaded REST file i/o with parallel hddm

Richard has implemented multi-threaded input and output in HDDM. It helps when the time spent in compression/de-compression is a significant fraction of the processing load. Implementation in JANA appears to require some significant effort and those involved have judged it not of the highest priority at present. Richard noted that in the case of pure Python implementations, the benefit will obtain.


Sean reported that Mark has started and he is about 1/3 of the way through after a couple of days. We are running version 2.2.0 of sim-recon. [Added in press: the jobs had been running with the wrong version. The version was corrected (set to 2.2.0) and the jobs restarted Wednesday evening.] Data is being simulated with runs matching those in the real data with 1% of the number of read-data events. In addition to REST files, monitoring histograms, and a fraction of the unreconstructed hddm-smeared data, trees from efficiency plugins are being kept.

Sean looked at the histograms and they seem reasonable for the most part. There are some problems, however:

  • BCAL timing is off. This has been seen for a while.
  • Start Counter matching is terrible, 60-75% depending on how it is plotted. The timing may not be tuned in the MC variation of CCDB.

Alex A. asked why the code is not identical to that used in the reconstruction launches. Mark said that this issue had been discussed, but the reconstruction era code did not to the FCAL energy correction correctly for simulation, and the effort was not spent to cherry-pick the changes. Paul thought that that effort would have been wasted anyway; we will re-do reconstruction and simulation before too long anyway. We agreed that this is a reasonable attitude right now.

Mark went through the list of plugins dropped from sim1.1 for various reasons.

For details of the conditions see the [ sim1.1 Static Website.

Changes to mcsmear in sim-recon

Sean led us through his email describing the changes. Specifically he mentioned:

  • TOF and Start Counter smearing changes
  • Implementation of channel-by-channel efficiencies
  • Use of the RCDB interface to fetch relevant parameters directly.

Thresholds: CDC, FDC

Sean encouraged the detector groups to review the way threshold are applied in the Monte Carlo. There may be some further design needed now that we have real data to compare to. See the relevant sections on the mcsmear updates wiki page.