Difference between revisions of "GlueX Offline Meeting, September 16, 2015"

From GlueXWiki
Jump to: navigation, search
(Agenda)
(Review of minutes from September 2]])
 
(3 intermediate revisions by 2 users not shown)
Line 10: Line 10:
 
## [[Scripts for Installing GlueX Software|gluex_install]] moved to GitHub (Mark)
 
## [[Scripts for Installing GlueX Software|gluex_install]] moved to GitHub (Mark)
 
## [https://mailman.jlab.org/pipermail/halld-offline/2015-September/002143.html ROOT TTree Format Overhaul] (Paul)
 
## [https://mailman.jlab.org/pipermail/halld-offline/2015-September/002143.html ROOT TTree Format Overhaul] (Paul)
## Offline Monitoring (Kei)
+
## [https://halldweb.jlab.org/wiki/images/1/15/2015-09-16-offline_monitoring.pdf Offline Monitoring] (Kei)
 
# Review of [[GlueX Offline Meeting, September 2, 2015#Minutes|minutes from September 2]] (all)
 
# Review of [[GlueX Offline Meeting, September 2, 2015#Minutes|minutes from September 2]] (all)
 
# [[GlueX-Collaboration-Oct-2015|Collaboration Meeting]] October 8-10, 2015 at Jefferson Lab
 
# [[GlueX-Collaboration-Oct-2015|Collaboration Meeting]] October 8-10, 2015 at Jefferson Lab
Line 31: Line 31:
  
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2015</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2015</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .
 +
 +
==Minutes==
 +
 +
Present:
 +
* '''CMU''': Mike Staib
 +
* '''FIU''': Mahmoud Kamel
 +
* '''JLab''': Amber Boehnlein, Brad Cannon, Mark Dalton, Sean Dobbs, Mark Ito (chair), Paul Mattione, Curtis Meyer, Eric Pooser, Nathan Sparks, Justin Stevens, Simon Taylor
 +
* '''UConn''': Richard Jones, James McIntyre, ?
 +
 +
There is a [https://bluejeans.com/s/8Ank/ recording of this meeting] on the BlueJeans site.
 +
 +
===Announcements===
 +
 +
# '''[https://mailman.jlab.org/pipermail/halld-offline/2015-September/002144.html "git_update" simple email list]'''. The list is open for subscriptions. You get a daily digest of changes to all of the GitHub repositories owned by the GlueX team.
 +
# '''[[Scripts for Installing GlueX Software|gluex_install]] moved to GitHub'''. We discussed this move at a previous meeting. The move is now complete. It has been added to the list of those owned by the GlueX team.
 +
# '''[https://mailman.jlab.org/pipermail/halld-offline/2015-September/002143.html ROOT TTree Format Overhaul]'''. Paul went through his email message. The major structural change is to go from combos at the top-level to having events at the top level and combos below that. This facilitates multi-threaded processing with ROOT.
 +
# '''[https://halldweb.jlab.org/wiki/images/1/15/2015-09-16-offline_monitoring.pdf Offline Monitoring]'''. Kei brought us up-to-date. See his slides for details. A few selected points from Launch 14:
 +
#* 78 jobs of 4314 failed. They have been resubmitted.
 +
#* The tape library seemed to stop for about a day starting Sunday morning.
 +
#* Integrated over the span of the launch, we got about 6% of the available CPU on the Farm.
 +
#* Next launch will be September 15.
 +
#* If people want, we can launch again on October 2.
 +
 +
===Review of minutes from September 2===
 +
 +
We went over [[GlueX Offline Meeting, September 2, 2015#Minutes|the minutes]].
 +
 +
* SciComp is exploring the use of Lustre-based file server for our work disk. A trial rsync is underway.
 +
* Mailman.jlab.org seems to be accessible from off-site now, after last night's maintenance period. This needs confirmation.
 +
 +
===Collaboration Meeting, October 8-10, 2015 at Jefferson Lab===
 +
 +
Mark filled in [[GlueX-Collaboration-Oct-2015|the agenda]] based on the discussion at the last meeting. We need to add a calibration talk from Sean to the existing line-up.
 +
 +
===Geant4 Update===
 +
 +
Richard gave us an update.
 +
 +
* He started implementing detector hits with the CDC.
 +
* Evolution of the code in hit processing has occurred over the years:
 +
** obsolete code encountered
 +
** decisions had to be made in going to the new implementation
 +
** for example, BCAL had three hit implementation
 +
*** consulted with Tegan
 +
*** will forego implementation of waveform generation
 +
*** brings truth hit scheme into alignment with other detectors
 +
** sources of constants needed a uniform approach:
 +
**# CCDB stuff will continue to be fetched from CCDB
 +
**# constants defined in the code: const static public member of the class
 +
**# FFREAD cards in control.in for constants that can be changed by the user
 +
*** He will limit himself to these forms. This means changing the code, especially for item (2).
 +
* Run number source:
 +
** in the simulation context, used to be a decoration
 +
** for external generators, run number well-defined
 +
** in control.in, the user-implemented RUNNO card redundant with GEANT-implemented RUNG card
 +
** Simulation (hdgeant) should decide on the detector conditions, not the event generator (e. g., bggen)
 +
** We need to think through how to handle how run number affects the calibration and what is written in the files.
 +
 +
===Data Challenge 3===
 +
 +
Mark has started converting [[Data Challenge 3|DC3]] to a SWIF-based project.
 +
 +
===Spring 2015 Commissioning Simulations===
 +
 +
Mark has started setting up DetCom 2.1 to use the latest release (sim-recon 1.5.1) to do [[Spring 2015 Commissioning Simulations|the simulations]].
 +
 +
===Fall 2015 Commissioning Simulations===
 +
 +
We will not need to produce simulations specifically for "Fall 2015" since we will not do any running with the solenoid this year. Sean pointed out that we do need to prepare simulations for future data when it comes in any case. Curtis thought that a magnet current of 1350 A is as good guess as any for what should be simulated.
 +
 +
===Auto-Build on Pull Request===
 +
 +
Sean described his exploration of using GitHub facilities to kick-off actions based on events in a repository, i. e., webhooks. In particular we are interested in doing a build of sim-recon upon creation of a pull request suing the proposed code.
 +
* to implement this, you give GitHub a URL to use for sending a request.
 +
* action can be triggered on a number of possible events
 +
* a JSON-formatted payload is made available for the server to act on the request
 +
* need to write CGI script to process requests
 +
* he wrote a working prototype in Python, only standard Python packages were used
 +
** input parameter: branch name specified in the build request
 +
* Mark will write the script to do actual the build based on the existing script for the nightly build.
 +
* Results of the build can be posted back to GitHub as a comment to the build request
 +
* allows changes to be tested before inclusion in the master branch
 +
* "It's a lot easier than SVN, it turns out" [note from secretary: sorry, could not resist]
 +
* We will meet with the computer Center to work out installation of python packages and discuss security issues
 +
* Nathan suggested adding tests to the automatic action, e. g., hdgeant, profiling, etc.
 +
* Nathan has explored cloud-based solutions to performing automatic test, using Linux containers (light-weight virtual machines). The container can be loaded with the code to be tested and shipped to the cloud for execution. Can get sim-recon down to 1 GB, compressed, with source code and dependencies.
 +
** He has been using Docker as the enabling technology
 +
** Sean pointed out that the OSG uses a file-system-over-HTTP approach to distributing user code. It would be interesting to compare that approach with Linux containers.
 +
 +
===Review of recent pull requests===
 +
 +
Did not get to [https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aopen+is%3Apr this].

Latest revision as of 12:36, 14 October 2015

GlueX Offline Software Meeting
Wednesday, September 16, 2015
1:30 pm EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. "git_update" simple email list (Mark)
    2. gluex_install moved to GitHub (Mark)
    3. ROOT TTree Format Overhaul (Paul)
    4. Offline Monitoring (Kei)
  2. Review of minutes from September 2 (all)
  3. Collaboration Meeting October 8-10, 2015 at Jefferson Lab
  4. Geant4 Update (Richard, David)
  5. Data Challenge 3 update (Mark)
  6. Spring 2015 Commissioning Simulations
  7. Fall 2015 Commissioning Simulations (all)
  8. Auto-Build on Pull Request (Sean)
  9. Review of recent pull requests
  10. Action Item Review

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2015 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2015/ .

Minutes

Present:

  • CMU: Mike Staib
  • FIU: Mahmoud Kamel
  • JLab: Amber Boehnlein, Brad Cannon, Mark Dalton, Sean Dobbs, Mark Ito (chair), Paul Mattione, Curtis Meyer, Eric Pooser, Nathan Sparks, Justin Stevens, Simon Taylor
  • UConn: Richard Jones, James McIntyre, ?

There is a recording of this meeting on the BlueJeans site.

Announcements

  1. "git_update" simple email list. The list is open for subscriptions. You get a daily digest of changes to all of the GitHub repositories owned by the GlueX team.
  2. gluex_install moved to GitHub. We discussed this move at a previous meeting. The move is now complete. It has been added to the list of those owned by the GlueX team.
  3. ROOT TTree Format Overhaul. Paul went through his email message. The major structural change is to go from combos at the top-level to having events at the top level and combos below that. This facilitates multi-threaded processing with ROOT.
  4. Offline Monitoring. Kei brought us up-to-date. See his slides for details. A few selected points from Launch 14:
    • 78 jobs of 4314 failed. They have been resubmitted.
    • The tape library seemed to stop for about a day starting Sunday morning.
    • Integrated over the span of the launch, we got about 6% of the available CPU on the Farm.
    • Next launch will be September 15.
    • If people want, we can launch again on October 2.

Review of minutes from September 2

We went over the minutes.

  • SciComp is exploring the use of Lustre-based file server for our work disk. A trial rsync is underway.
  • Mailman.jlab.org seems to be accessible from off-site now, after last night's maintenance period. This needs confirmation.

Collaboration Meeting, October 8-10, 2015 at Jefferson Lab

Mark filled in the agenda based on the discussion at the last meeting. We need to add a calibration talk from Sean to the existing line-up.

Geant4 Update

Richard gave us an update.

  • He started implementing detector hits with the CDC.
  • Evolution of the code in hit processing has occurred over the years:
    • obsolete code encountered
    • decisions had to be made in going to the new implementation
    • for example, BCAL had three hit implementation
      • consulted with Tegan
      • will forego implementation of waveform generation
      • brings truth hit scheme into alignment with other detectors
    • sources of constants needed a uniform approach:
      1. CCDB stuff will continue to be fetched from CCDB
      2. constants defined in the code: const static public member of the class
      3. FFREAD cards in control.in for constants that can be changed by the user
      • He will limit himself to these forms. This means changing the code, especially for item (2).
  • Run number source:
    • in the simulation context, used to be a decoration
    • for external generators, run number well-defined
    • in control.in, the user-implemented RUNNO card redundant with GEANT-implemented RUNG card
    • Simulation (hdgeant) should decide on the detector conditions, not the event generator (e. g., bggen)
    • We need to think through how to handle how run number affects the calibration and what is written in the files.

Data Challenge 3

Mark has started converting DC3 to a SWIF-based project.

Spring 2015 Commissioning Simulations

Mark has started setting up DetCom 2.1 to use the latest release (sim-recon 1.5.1) to do the simulations.

Fall 2015 Commissioning Simulations

We will not need to produce simulations specifically for "Fall 2015" since we will not do any running with the solenoid this year. Sean pointed out that we do need to prepare simulations for future data when it comes in any case. Curtis thought that a magnet current of 1350 A is as good guess as any for what should be simulated.

Auto-Build on Pull Request

Sean described his exploration of using GitHub facilities to kick-off actions based on events in a repository, i. e., webhooks. In particular we are interested in doing a build of sim-recon upon creation of a pull request suing the proposed code.

  • to implement this, you give GitHub a URL to use for sending a request.
  • action can be triggered on a number of possible events
  • a JSON-formatted payload is made available for the server to act on the request
  • need to write CGI script to process requests
  • he wrote a working prototype in Python, only standard Python packages were used
    • input parameter: branch name specified in the build request
  • Mark will write the script to do actual the build based on the existing script for the nightly build.
  • Results of the build can be posted back to GitHub as a comment to the build request
  • allows changes to be tested before inclusion in the master branch
  • "It's a lot easier than SVN, it turns out" [note from secretary: sorry, could not resist]
  • We will meet with the computer Center to work out installation of python packages and discuss security issues
  • Nathan suggested adding tests to the automatic action, e. g., hdgeant, profiling, etc.
  • Nathan has explored cloud-based solutions to performing automatic test, using Linux containers (light-weight virtual machines). The container can be loaded with the code to be tested and shipped to the cloud for execution. Can get sim-recon down to 1 GB, compressed, with source code and dependencies.
    • He has been using Docker as the enabling technology
    • Sean pointed out that the OSG uses a file-system-over-HTTP approach to distributing user code. It would be interesting to compare that approach with Linux containers.

Review of recent pull requests

Did not get to this.