GlueX Software Meeting, February 2, 2021

From GlueXWiki
Revision as of 19:38, 30 March 2021 by Marki (Talk | contribs) (Standardized Tests)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

GlueX Software Meeting
Tuesday, February 2, 2021
3:00 pm EST
BlueJeans: 968 592 007

Agenda

  1. Announcements
    1. New version set: 4.35.0 (Mark)
    2. Tape Recovery News (Mark)
    3. Old packages backed up and deleted from group disk (Mark)
    4. HSF Training: Continuous Integration in GitHub, February 16-19 (Mark)
  2. Review of Minutes from the Last Software Meeting (all)
  3. Minutes from the Last HDGeant4 Meeting (all)
  4. Collaboration Meeting Planning
    1. Upcoming Meeting
    2. Last Meeting
  5. Reconstruction on the OSG (Richard)
  6. Review of recent issues and pull requests:
    1. halld_recon
    2. halld_sim
    3. CCDB
    4. RCDB
    5. MCwrapper
  7. Review of recent discussion on the GlueX Software Help List (all)
  8. Action Item Review (all)

Minutes

Present: Alex Austregesilo, Thomas Britton, Sean Dobbs, Mark Ito (chair), Igal Jaegle, Richard Jones, David Lawrence, Simon Taylor, Nilanga Wickramaarachchi

There is a recording of this meeting. Log into the BlueJeans site first to gain access (use your JLab credentials).

Announcements

  1. New version set: 4.35.0. This one has the latest release of AmpTools.
  2. Tape Recovery News. The latest from Bryan Hess from last week.
  3. Old packages backed up and deleted from group disk. Peter Pauli reported that halld_recon 4.5.0 needed to put back for simulations on the OSG. Mark did that this morning.
  4. HSF Training: Continuous Integration in GitHub, February 16-19. Please see the Indico page for details.

Review of Minutes from the Last Software Meeting

We went over the minutes from January 19 without significant comment.

Minutes from the Last HDGeant4 Meeting

We went over the minutes from January 26. Mark mentioned that he is close to incorporating Richard's Diracxx package into our set of standard software.

Collaboration Meeting Planning

We decided that it was premature to make a definite plan for software-related talks at the upcoming collaboration meeting. We will revisit the topic once the agenda is announced.

Reconstruction on the OSG

Richard gave us an update on progress toward doing large-scale reconstruction on the Open Science Grid (OSG). He described issues with running the jobs, which have largely been solved. In the process of comparing the results on the OSG with those from the JLab farm, he did not find agreement and traced it to non-reproducibility of the resulting reconstructed events from repeated runs on identical data.

Further investigation reveals that most of the problems are in the Analysis Library. He continues to pursue the painstaking exploration of the problem.

Please see Richard's slides for the details.

Standardized Tests

Sean brought up the need to step up our testing game. We discussed ideas to both augment our current procedures and develop new ones.

  • We do a build test and simple execution tests on new pull requests. The output of the execution tests are not examined.
  • We do a nightly build test of all GlueX software. There is no nightly execution test done.
  • Every three days we do a reconstruction test with the latest nightly build. There is no automated verification of the results, just an email with a link to selected histograms.
  • Twice a week we do a simulation test, from event generation to reconstruction and histogram production. There is no automatic verification of results, although a small number of quantities are plotted as function of calendar time. The test also suffers from low statistics.
  • There is no test of output reproducibility.

Sean proposed we add this to future software meeting agendas. Richard suggested starting a wiki page to collect ideas.

Action Item Review

  1. Ask the Computer Center about staging the start of jobs in time. (Mark I.)
  2. Make sure that the automatic tests of HDGeant4 pull requests have been fully implemented. (Mark I., Sean)
  3. Finish conversion of halld_recon to use JANA2. (Nathan)
  4. Release CCDB 2.0 (Dmitry, Mark I.)
  5. Start a wiki page to collect ideas on software testing. (Mark)