GlueX Offline Meeting, January 26, 2018

GlueX Offline Software Meeting
Friday, January 26, 2018
10:00 am EST
JLab: CEBAF Center A110
BlueJeans: 968 592 007

Agenda

1. Announcements
   1. New sim-recon release, version 2.21.0 (Mark)
   2. New version set 2.25 has new sim-recon (and more) (Mark)
   3. New version of CCDB: v1.06.06 (Mark)
   4. New release of rcdb: v0.02.01 (Mark)
2. Review of minutes from the last meeting (all)
3. New RCDB version with SQLite capability (all)
4. Release management thoughts
5. Review of recent pull requests (all)
6. Review of recent discussion on the GlueX Software Help List (all)
7. Action Item Review (all)

Communication Information

Remote Connection

• The BlueJeans meeting number is 968 592 007 .
• Join the Meeting via BlueJeans

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2018 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2018/ .

Minutes

Present:

• CMU : Naomi Jarvis
• FSU : Sean Dobbs
• JLab: : Alex Austregesilo, Thomas Britton, Eugene Chudakov, Mark Ito (chair), Beni Zihlmann

There is a recording of this meeting on the BlueJeans site. Use your JLab credentials to access it.

Announcements

1. New sim-recon release, version 2.21.0. This is the first release that has Simon's new track matching code.
2. New version of CCDB: v1.06.06. Contains all changes since last July. Does not include features from the 1.07 development branch.
3. New release of RCDB: v0.02.01. Contains changes to the Python routines. The C++ API is the same as in v0.02. On a related topic, the website is still running a very old version (pre-GitHub era). Mark did update the alias.py there however, giving us access to the "is_2018production" query.
4. New version set 2.25 has new sim-recon (and more). This version set includes the releases of sim-recon, CCDB, and RCDB mentioned above.

Review of Minutes from the Last Meeting

We went over the [[GlueX Offline Meeting, January 10, 2018#Minutes|minutes from January 10. We did review disk usage on cache, work, and volatile. There was no other significant discussion.

New RCDB version with SQLite capability

Mark went over his recent email message describing the changes needed to bring on the new RCDB version. This will restore access to the SQLite version of the database and thus enable running on the OSG. There are two issues he has encountered since sending the message:

1. HDGeant4, via inclusion of the level 1 trigger library of sim-recon, has a dependence on RCDB. Its makefile therefore needs modification to search the new SQLiteCpp library. This has been done; the results are on the sqlitecpp_standalone branch of the HDGeant4 Git repository.
2. The new SQLiteCpp library needs a version of the SQLite library greater than or equal to 3.7.x The default SQLite (provided via RPMs) on RHEL6 and CentOS6 is 3.6.x. To build on these platforms, a more recent version of SQLite needs to be installed. At JLab, there is an SQLite 3.13.0 available on the /apps partition. A solution using this installation is underway.

Mark's proposal from the email was:

I've not yet merged any of these changes onto any of the master branches. My proposal is to prepare public versions of RCDB, SQLiteCpp, and build_scripts [and also HDGeant4] at JLab, and then submit a pull request for the change to sim-recon. When that gets merged I can create a tagged version and build that with the appropriate version set (version.xml).

Collaborators that are uninterested in these changes can continue on with older version sets. However, after this change, any update of the master branch of sim-recon will require an update of RCDB and HDGeant and installation of SQLiteCpp at the same time.

At present, the only technical difficulty standing in the way of this proposal is the SQLite version problem on RHEL6 and CentOS6. These platforms are not heavily used at JLab for offline processing and the pieces for a solution at JLab are at hand. The real problem will come for folks running CentOS6 off-site; each system admin will have to come up with a local solution. Note that SQLite version 3.7.0 was released on July 21, 2010, over seven years ago, so there are likely solutions out there.

We came to a consensus on going ahead with implementation of the proposal.

Release Management Thoughts

As reflected in the title of the agenda item, Sean has been thinking. He has identified a need for more formal control of our version sets. For a given run period there may be multiple reconstruction passes and for each reconstruction pass, there may be multiple iterations of the simulation software. In additions there are changing version of the GlueX Root Analysis (GRA) package. The correct combination of versions for a particular task can get complicated. He outlined the situation in a slide.

We discussed various approaches.

• Thomas pointed out that part of the complication is because simulation and reconstruction are bundled together in the same package. If they were separate packages, version control would be simpler. Mark mentioned that having HDGeant4 in a separate repository goes a long way toward sim/recon independence, but other important components remain bundled, most importantly mcsmear.
• Alex remarked that GRA version control is often usefully left to the individual analyzer. Often the master branch is used. Mark pointed out that often collaborators are not interested in making this choice; they would rather have a compatible version set-up for them automatically.
• Mark suggested that a meta-version XML file could identify version sets appropriate for each task, e. g.,

  <activity type="reconstruction" version_set="recon_2018-01_ver03.xml"/>
  <activity type="simulation" version_set="recon_2018-01_ver03-sim.xml"/>
  <activity type="analysis" version_set="recon_2018-01_ver03-ana.xml"/>
  

The group was generally in favor of a more formal system. We decided to take the topic offline for now---more thinking.

Simulation with Conditions Changing Run-by-Run

Eugene asked about an issue that came up in another meeting. If detector conditions, for example dead channels, are changing from run to run, are we able to reflect that accurately in the simulation. There are likely correlations that make an "average efficiency" calculation with an "average detector" inaccurate. Thomas remarked that MCwrapper has a mechanism to simulate events with a population of events, generated with different configurations in proportion to that in the real data, using the CCDB's "mc" variation to supply the run-by-run conditions. Mark also mentioned that we first did this with the simX.X runs in the past.

TOF Anomaly in Monitoring Histograms

Beni reported a feature in the Plot Browser histograms for Spring 2017, in the :Recon. TOF 2" section. For Monitoring Launch versions 19 and above, there are no TOF identified protons below 1.0 GeV/c. He cannot reproduce the effect in his own code. Alex A. told us that this feature is not there with Simon's new track matching code. There is a problem, however, since the last reconstruction launch was done with the old track matching. This problem is likely present in the resulting REST files.

Review of Recent Pull Requests

We reviewed them. No significant comments.

Review of Recent Discussion on the GlueX Software Help List

We reviewed recent discussion. No significant comments.