Difference between revisions of "GlueX Offline Meeting, October 28, 2015"

From GlueXWiki
Jump to: navigation, search
(Agenda)
m (Text replacement - "http://argus.phys.uregina.ca/cgi-bin/public" to "https://halldweb.jlab.org/doc-public")
 
(3 intermediate revisions by the same user not shown)
Line 7: Line 7:
  
 
# Announcements
 
# Announcements
## new Version Management System features: URL checking for Git, hash-specific check-outs, [http://argus.phys.uregina.ca/cgi-bin/public/DocDB/ShowDocument?docid=2793 document update]
+
## new Version Management System features: URL checking for Git, hash-specific check-outs, [https://halldweb.jlab.org/doc-public/DocDB/ShowDocument?docid=2793 document update]
 
## [http://www.jlab.org/Hall-D/software/HDSoftware_Documentation/ Doxygen] fixed
 
## [http://www.jlab.org/Hall-D/software/HDSoftware_Documentation/ Doxygen] fixed
 
## [https://mailman.jlab.org/pipermail/halld-offline/2015-October/002155.html New releases: sim-recon 1.6.0 and HDDS 3.4]
 
## [https://mailman.jlab.org/pipermail/halld-offline/2015-October/002155.html New releases: sim-recon 1.6.0 and HDDS 3.4]
Line 32: Line 32:
  
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2015</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2015</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .
 +
 +
==Minutes==
 +
 +
Present:
 +
* '''FSU''': Brad Cannon, Alyssa Henderson
 +
* '''FIU''': Mahmoud Kamel
 +
* '''JLab''': Amber Boehnlein,  Mark Ito (chair), Kei Moriya, Nathan Sparks, Justin Stevens, Beni Zihlmann
 +
* '''KC''': Paul Mattione
 +
* '''NU''': Sean Dobbs
 +
 +
There is a [https://bluejeans.com/s/8Kew/ recording of this meeting] on the BlueJeans site. <!-- for 10/28 -->
 +
===Announcements===
 +
 +
* '''new Version Management System features'''. Mark described the changes: URL checking for Git clones and hash-specific check-outs, See the [https://halldweb.jlab.org/doc-public/DocDB/ShowDocument?docid=2793 updated document] for details. There is also an [https://halldweb.jlab.org/docs/gluex_build/ HTML version] now.
 +
* '''Doxygen fixed''': Beni noticed that the [http://www.jlab.org/Hall-D/software/HDSoftware_Documentation/ Doxygen site] was out-of-date. The update had been broken with recent changes to the nightly builds. Mark has restored the nightly updates.
 +
* '''[https://mailman.jlab.org/pipermail/halld-offline/2015-October/002155.html New releases: sim-recon 1.6.0 and HDDS 3.4]'''. Mark reminded us about these releases.
 +
* '''[https://halldweb.jlab.org/wiki-private Private wiki] getting there'''. Mark flashed the new site, which still under construction. The "Private:" pages have been migrated, but have been deleted from the public site yet. Links from the private to the public wiki can be made with wiki links ([[]]) by prepending "public:" to the page name, e. g., "public:Meetings".
 +
* '''[https://halldweb.jlab.org/wiki/index.php?title=GlueX_Offline_FAQ&diff=70904&oldid=70249 Recent additions to the Offline FAQ]'''. Mark highlighted the latest additions.
 +
* '''Build on new Ubuntu release'''. Beni's announced that he has succeeded with a build of the GlueX software on Ubuntu 15.10 using the GCC 5.2 compiler. Some changes were required to CCDB and root2email.
 +
 +
===Review of minutes from October 14===
 +
 +
We went over [[GlueX Offline Meeting, October 14, 2015#Minutes|minutes from the October 14 meeting]].
 +
 +
'''New Work Disk'''. Last week we switched over to a new file server for the work disk. The new space is based on Lustre. There are a couple of issues.
 +
# SQLite files will not work from a Lustre partition.
 +
# Builds have been observed to be significantly slower on the new disk (factors of a few). Speculation is that this is due to poor performance for small files with Lustre.
 +
We decided to ask SciComp to allow us to retain the space we had been using on the old traditional file server (14 TB) and use that for these specialized purposes.
 +
 +
===Data Challenge 3 update===
 +
 +
Mark found that the reconstruction for [[Data Challenge 3|DC3]] was not working judged from the ROOT files. He has decided to start from the beginning, generating new input data, using the latest release.
 +
 +
Sean mentioned that we might want to double-check that reconstruction of simulated data, converted from HDDM to EVIO, still works with the latest version of the reconstruction.
 +
 +
===Offline Monitoring===
 +
 +
Kei gave the report. See [[Media:2015-10-27-offline_monitoring.pdf|his slides]] for the details. The bullets from his final slide:
 +
 +
* Due to new plugins, now require 8GB of RAM to run jobs - some pruning of unnecessary histograms would be welcome
 +
* There are some crashes/errors reported in the log files - will send out notices
 +
* Will redo studies of time/resources against #cores/threads shown at collaboration meeting with improvements in tracking
 +
* Next launch: November 6 (Fri)
 +
* SWIF has changed underlying job structure, still rewriting cross-analysis scripts
 +
* Will add instructions to wiki
 +
 +
Sean suggested contacting individuals about their histograms if we really want to reduce the memory footprint.
 +
 +
===Sim1===
 +
 +
Sean discussed planning for [[Sim1 Conditions|the next simulation run]].
 +
 +
* Mark has run the first few test jobs.
 +
* Sean asked for input on the final conditions.
 +
* Justin thought we should try to match conditions in the Spring.
 +
* Sean also wants to have substantial statistics for this effort.
 +
* Kei asked about the magnetic field setting. At the calibration we settled on 1300 A as a reasonable guess.
 +
* Mark created a "gluex_simulations" repository on GitHub to version control the settings for this simulation.
 +
* Sean added some scripted checks that the simulation settings are reasonable for this effort.
 +
* Sean mentioned that the plan was to use the standard SQLite CCDB file and lock down the version of the constants by specifying the timestamp.
 +
* Justin suggesting using a slightly lower bound on the energy range to ensure that lower energy background events that might sneak into the sample in real data are present in the simulated sample. We settled on 5 GeV at a reasonable guess, although this question could stand more study.
 +
 +
===Review of recent pull requests===
 +
 +
We went over [https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aopen+is%3Apr the list] going back about two weeks.
 +
 +
Sean commented on his change to throw an exception on CCDB errors so that the failure comes near the point of the error.
 +
 +
Mark asked that folks try to provide meaningful comments when merging pull request. Often the best description of proposed changes comes when the pull request is first composed; copy and pasting that is a good start. That original description does not appear to make it into the git repository by default; it seems to exist only on the GitHub site.

Latest revision as of 15:44, 24 February 2017

GlueX Offline Software Meeting
Wednesday, October 28, 2015
1:30 pm EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. new Version Management System features: URL checking for Git, hash-specific check-outs, document update
    2. Doxygen fixed
    3. New releases: sim-recon 1.6.0 and HDDS 3.4
    4. Private wiki getting there
    5. Recent additions to the Offline FAQ
  2. Review of minutes from October 14 (all)
  3. Offline Monitoring (Kei)
  4. Geant4 Update (Richard, David)
  5. Data Challenge 3 update (Mark)
  6. Future Commissioning Simulations (all)
  7. b1pi results review
  8. Review of recent pull requests
    • comments on merge
  9. Action Item Review

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2015 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .

Minutes

Present:

  • FSU: Brad Cannon, Alyssa Henderson
  • FIU: Mahmoud Kamel
  • JLab: Amber Boehnlein, Mark Ito (chair), Kei Moriya, Nathan Sparks, Justin Stevens, Beni Zihlmann
  • KC: Paul Mattione
  • NU: Sean Dobbs

There is a recording of this meeting on the BlueJeans site.

Announcements

  • new Version Management System features. Mark described the changes: URL checking for Git clones and hash-specific check-outs, See the updated document for details. There is also an HTML version now.
  • Doxygen fixed: Beni noticed that the Doxygen site was out-of-date. The update had been broken with recent changes to the nightly builds. Mark has restored the nightly updates.
  • New releases: sim-recon 1.6.0 and HDDS 3.4. Mark reminded us about these releases.
  • Private wiki getting there. Mark flashed the new site, which still under construction. The "Private:" pages have been migrated, but have been deleted from the public site yet. Links from the private to the public wiki can be made with wiki links ([[]]) by prepending "public:" to the page name, e. g., "public:Meetings".
  • Recent additions to the Offline FAQ. Mark highlighted the latest additions.
  • Build on new Ubuntu release. Beni's announced that he has succeeded with a build of the GlueX software on Ubuntu 15.10 using the GCC 5.2 compiler. Some changes were required to CCDB and root2email.

Review of minutes from October 14

We went over minutes from the October 14 meeting.

New Work Disk. Last week we switched over to a new file server for the work disk. The new space is based on Lustre. There are a couple of issues.

  1. SQLite files will not work from a Lustre partition.
  2. Builds have been observed to be significantly slower on the new disk (factors of a few). Speculation is that this is due to poor performance for small files with Lustre.

We decided to ask SciComp to allow us to retain the space we had been using on the old traditional file server (14 TB) and use that for these specialized purposes.

Data Challenge 3 update

Mark found that the reconstruction for DC3 was not working judged from the ROOT files. He has decided to start from the beginning, generating new input data, using the latest release.

Sean mentioned that we might want to double-check that reconstruction of simulated data, converted from HDDM to EVIO, still works with the latest version of the reconstruction.

Offline Monitoring

Kei gave the report. See his slides for the details. The bullets from his final slide:

  • Due to new plugins, now require 8GB of RAM to run jobs - some pruning of unnecessary histograms would be welcome
  • There are some crashes/errors reported in the log files - will send out notices
  • Will redo studies of time/resources against #cores/threads shown at collaboration meeting with improvements in tracking
  • Next launch: November 6 (Fri)
  • SWIF has changed underlying job structure, still rewriting cross-analysis scripts
  • Will add instructions to wiki

Sean suggested contacting individuals about their histograms if we really want to reduce the memory footprint.

Sim1

Sean discussed planning for the next simulation run.

  • Mark has run the first few test jobs.
  • Sean asked for input on the final conditions.
  • Justin thought we should try to match conditions in the Spring.
  • Sean also wants to have substantial statistics for this effort.
  • Kei asked about the magnetic field setting. At the calibration we settled on 1300 A as a reasonable guess.
  • Mark created a "gluex_simulations" repository on GitHub to version control the settings for this simulation.
  • Sean added some scripted checks that the simulation settings are reasonable for this effort.
  • Sean mentioned that the plan was to use the standard SQLite CCDB file and lock down the version of the constants by specifying the timestamp.
  • Justin suggesting using a slightly lower bound on the energy range to ensure that lower energy background events that might sneak into the sample in real data are present in the simulated sample. We settled on 5 GeV at a reasonable guess, although this question could stand more study.

Review of recent pull requests

We went over the list going back about two weeks.

Sean commented on his change to throw an exception on CCDB errors so that the failure comes near the point of the error.

Mark asked that folks try to provide meaningful comments when merging pull request. Often the best description of proposed changes comes when the pull request is first composed; copy and pasting that is a good start. That original description does not appear to make it into the git repository by default; it seems to exist only on the GitHub site.