GlueX Offline Meeting, August 5, 2015

From GlueXWiki
Revision as of 17:16, 6 August 2015 by Marki (Talk | contribs) (Slides)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

GlueX Offline Software Meeting
Wednesday, August 5, 2015
1:30 pm EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. The meaning of ifarm has changed (Mark)
      • Use "ifarm62" for CentOS 6.2, "ifarm65" for CentOS 6.5. "ifarm" is the same as "ifarm65" (it used to point to CentOS 6.2 nodes).
    2. Policy on CCDB Variations for Reconstructing Simulated Data (Mark)
    3. New releases hdds 3.3, sim-recon 1.4.0 (Mark)
    4. ccdb_changes email list (Mark)
    5. New Git Guide (Paul)
    6. Github for Mac (David)
    7. Git versions on the JLab CUE (Mark)
    8. Emulation mode changes (David)
  2. Review of minutes from July 22 (all)
  3. Offline Monitoring (Kei et al.): Talk at previous calibration meeting on July 29.
  4. Spring 2015 Commissioning Simulations (Sean)
  5. Geant4 Update (Richard, David)
  6. Overhaul ROOT TTree Format (Paul)
  7. Conversion from Subversion to Git, discussion (all)
  8. Proposed move to Git: build_scripts, gluex_install, jproj (Mark)
  9. Action Item Review

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2015 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .

Minutes

Present:

  • CMU: Curtis Meyer
  • FIU: Mahmoud Kamel
  • FSU: Aristeidis Tsaris
  • JLab: Alex Barnes, Mark Dalton, Mark Ito (chair), David Lawrence, Paul Mattione, Kei Moriya, Eric Pooser, Nathan Sparks, Justin Stevens, Simon Taylor, Beni Zihlmann
  • NU: Sean Dobbs

Announcements

  1. The meaning of ifarm has changed. Use "ifarm62" for CentOS 6.2, "ifarm65" for CentOS 6.5. "ifarm" is the same as "ifarm65" (it used to point to CentOS 6.2 nodes).
  2. Policy on CCDB Variations for Reconstructing Simulated Data. This new wiki page summarizes our policy on how to use CCDB variations with Monte Carlo data.
  3. New releases hdds 3.3, sim-recon 1.4.0. Note that the release notes are now being posted on GitHub.
  4. ccdb_changes email list. The new list sends out a daily digest of changes to the CCDB.
  5. New Git Guide Paul transcribed his notes from his personal climb of the Git learning curve.
  6. Github for Mac. David pointed us to a GitHub-distributed application for the Mac desktop.
  7. Git versions on the JLab CUE. Simon and Elton found some problems with specific versions of Git installed at JLab. Suggested combination: /apps/bin/git on ifarm65 or jlabl*.
  8. Emulation mode changes. David reviewed his recent email describing his proposed changes to control of Flash-ADC emulation, aka, creating summary data from raw waveforms in the offline environment.

Review of minutes from July 22

We went over the minutes. There was discussion on a few items:

  • We still need to work out details of doing an HDDM/EVIO merge so that we can mix in real background events with simulated physics events.
  • David mentioned that there are some changes needed in JANA to support use of ROOT 6.
  • Mark found out that the lack of notification to repository watchers when a direct push to GitHub repositories is a feature, not a bug. See this page on GitHub for details. Turns out only two email addresses can be added to the distribution list; one of them should be a group email address. We decided not to turn on the feature to avoid excessive traffic on halld-offline@jlab.org.

Offline Monitoring

Kei gave a detailed report at the previous calibration meeting. He continues to work with Chris Larrieu on SWIF features and is relying more and more on it for the launches.

Spring 2015 Commissioning Simulations

Sean reports that the creation of the HDDM raw data for the Spring 2015 Commissioning Simulations is practically complete. The data is available in the /volatile/halld/detcom_02/smeared directory. It still remains to do the reconstruction pass on the data.

Geant4 Update

  • Richard has completed the incorporation of the three types of event sources:

particle gun, coherent bremsstrahlung (now implemented as a C++ class), and HDDM. Note that genr8 and bggen both create HDDM-formatted events.

  • Secondary vertices can be stored and examined later.
  • The complete PDG particle list has been implemented.
  • Richard is suggesting using curated physics lists from the LHC experiments rather than struggling to assemble our own. He is following David's example and is using a hybrid of LHCb and FRITIOF.
  • Next is to do a detailed comparison of particle propagation between GEANT 3 and Geant4.
  • He is using the boost library for Python hooks and some special functions. This means that we will have to install this package on our machines to get a successful build.
  • The next major task to work on creating the detector hits from the sensitive detectors.
  • The code can be cloned from GitHub and built. The URL is https://github.com/rjones30/HDGeant4.git.

Overhaul of the ROOT TTree Format

Paul led us through his email outlining the need for changes to the tree format. He said,

It needs to be done, I'm going to it,..., if you have any comments by Friday let me know.

Mark asked if the primary motivation was to take advantage of multi-threaded processing with PROOF. Paul explained the situation is more complicated than that one issue. User code might be simpler in certain circumstances in the new scheme and multi-threaded processing would be an option not available now.

Conversion from Subversion to Git: discussion

We had an unstructured discussion about our experience so far with Git.

  • People seem to generally like it.
  • David is still finding some details and concepts that need to be understood to use Git effectively.
  • Simon will try the "git diff --cached" command to look at difference between staged files and repository.
  • David noted that there is a bit of blame sharing with pull requests being issued by one person and merged in by another, a new twist.
  • David is liking the local nature of Git and the ease in making sandboxes to play with changes. He also noted the ease in changing from branch to branch. I that context, he test drove the "git stash" command and it performed as expected.
  • Mark solicited comments on whether the workflow we are using is too protective against changes or too accommodating of them. There was no real comment, which he interpreted as "about right".
  • Justin mentioned that at the S&T review, he was asked how we protect against badly broken code getting checked into the repository. He told the committee that we have changed to Git and that gives us a measure of control.
  • David noted that the email volume from watching the repositories seems higher than with Subversion.
  • David thought that although we are merging pull requests without a lot of review right now, going forward, as more junior members of the collaboration make pull requests, those requests in particular might get more careful scrutiny, as is appropriate.
  • Sean proposed that we have builds triggered on pull requests. He mentioned that there are tools out there to do things like that.

Proposed move to Git: build_scripts, gluex_install, jproj

Mark I. proposed moving version control of some of the packages he has been maintaining from Subversion to Git. There were no objections, but the topic generated discussion on other related topics.

  • Mark I. is proposing to create three new repositories on GitHub. Justin thought we might want to have a "scripts" repository and have these projects in subdirectories of that repository, like they are currently in Subversion. Mark thought that the repositories should be more like single-purpose projects one repository per project. David noted that this scheme will lead to a lot of repositories.
  • Justin observed that since all of the repositories are under the Jefferson Lab organization at GitHub on a single level, it is very difficult to browse the list and figure out which ones one needs. The name "build_scripts" gives you no clue that the repository is relevant for GlueX work.
    • Curtis thought we might want to prepend "HD" or something similar to the names of all of our repositories.
    • Mark I. noted that if one goes to the "GlueX Team" page at GitHub, one can click on a link to get a list of all repositories that the team is allowed to write to.
    • Mark thought that the situation is like the DocDB in that you cannot simply browse the titles to find the document you need. Rather you need a reference into the title space to get the document you want.
  • David pointed out that to get everything needed to build a working system, there are several packages that one needs to get and that the list cannot be intuited or guessed at. One needs an HDPM or a gluex_install.
    • David further noted that gluex_install does not work on Mac. We discussed ways that Mark I. could do some testing on a Mac platform.
    • Paul noted that there is a concept called "submodules" in Git that might help.

We did not come to conclusions on these issues, but clearly we will have to return to them.

Action Items

  1. Work out details of doing an HDDM/EVIO merge. (Sean, David)
  2. Complete work on changes to be compatible with ROOT 6. (Beni, David)
  3. Overhaul of the ROOT TTree Format. (Paul)
  4. Come up with a policy on organizing Git repositories and repository names. (Mark and interested parties)