Difference between revisions of "GlueX Offline Meeting, November 13, 2013"

From GlueXWiki
Jump to: navigation, search
(Agenda)
m (Text replacement - "http://argus.phys.uregina.ca/cgi-bin/private" to "https://halldweb.jlab.org/doc-private")
 
(6 intermediate revisions by the same user not shown)
Line 10: Line 10:
 
## New release: [https://halldsvn.jlab.org/repos/tags/sim-recon-2013-10-17/RELEASE sim-recon-2013-10-17] (Mark)
 
## New release: [https://halldsvn.jlab.org/repos/tags/sim-recon-2013-10-17/RELEASE sim-recon-2013-10-17] (Mark)
 
## New storage type at JLab: pass-through cache (Mark)
 
## New storage type at JLab: pass-through cache (Mark)
 +
## [https://www.dropbox.com/s/n88rlj9kjjo14bz/workflowReq.pdf Requirements document for Data Flow Tools] (Mark)
 
# Review of [[GlueX Offline Meeting, October 30, 2013#Minutes|minutes from the last meeting]]: all
 
# Review of [[GlueX Offline Meeting, October 30, 2013#Minutes|minutes from the last meeting]]: all
 
# [[2013 Software Review|Software Review Planning]]
 
# [[2013 Software Review|Software Review Planning]]
## [http://argus.phys.uregina.ca/cgi-bin/private/DocDB/ShowDocument?docid=2350 GlueX/Hall-D Computing]
+
## [https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2350 GlueX/Hall-D Computing]
# [https://halldweb1.jlab.org/talks/2013-4Q/2013-11-13-iu-minidatachallenge.pdf bggen failures] (Kei)
+
# [https://halldweb.jlab.org/talks/2013-4Q/2013-11-13-iu-minidatachallenge.pdf bggen failures] (Kei)
 
# [[Media:20131113_scons.pdf|An SCons System for GlueX]] (David)
 
# [[Media:20131113_scons.pdf|An SCons System for GlueX]] (David)
 
# Mantis Bug Tracker Review
 
# Mantis Bug Tracker Review
Line 44: Line 45:
 
==Slides==
 
==Slides==
  
Talks can be deposited in the directory <code>/group/halld/www/halldweb1/html/talks/2013-4Q</code> on the JLab CUE. This directory is accessible from the web at https://halldweb1.jlab.org/talks/2013-4Q/ .
+
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2013-4Q</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2013-4Q/ .
 +
 
 +
=Minutes=
 +
 
 +
Present:
 +
* '''CMU''': Will Levine, Curtis Meyer
 +
* '''IU''': Kei Moriya, Matt Shepherd
 +
* '''JLab''': Mark Dalton, Mark Ito (chair), David Lawrence, Yi Qiang, Dmitry Romanov, Sascha Somov, Nathan Sparks, Simon Taylor, Elliott Wolin, Beni Zihlmann
 +
* '''MIT''': Justin Stevens
 +
 
 +
==Announcements==
 +
 
 +
* The Hall D Data Management Plan has been posted on the wiki [[Media:Data Management Plan Hall D v2.pdf|here]].
 +
* There is a new release of sim-recon, [https://halldsvn.jlab.org/repos/tags/sim-recon-2013-10-17/RELEASE sim-recon-2013-10-17]. Justin pointed out that this release has the disease of missing tracks in the backward direction. The precise plan to get out of this situation will be discussed in the coming days, offline.
 +
* New storage type at JLab: pass-through cache
 +
** This is writable disk space that is automatically backed up to the tape library. As the disk fills up, the oldest files (by access date) will be deleted, but those files are not lost forever. The user can request that any file be restored from the tape library. This restoration may displace other long-unaccessed files but...you get the idea.
 +
* [https://www.dropbox.com/s/n88rlj9kjjo14bz/workflowReq.pdf Requirements document for Data Flow Tools]
 +
** The committee charged to come up with these requirements has done so. The next step is to write specifications from which coding can proceed. A working committee has been formed to do this: Jie Chen (chair), Johann Goetz, Vardan Guryjan, Mark Ito, Chris Larrieu, and Dennis Weygand. These requirements have been added as an appendix to the [https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2350 computing document] to be given to the Software Review II committee.
 +
 
 +
==Software Review Planning==
 +
 
 +
Mark I. gave the report. Most of this information is collected in the [[2013_Software_Review|Software Review Planning wiki page]].
 +
 
 +
* We have a [https://www.dropbox.com/s/utaaxfvtisefm3d/12GeV_Software_Review_2013_v3.pdf final agenda] from Rolf Ent.
 +
* We have a mostly complete [https://halldweb.jlab.org/doc-private/DocDB/ShowDocument?docid=2350 written document] for the reviewers.
 +
** I contains various appendices covering workflow tools, the data management plan, and an analysis plan the Curtis wrote recently.
 +
** The draft would benefit from review from the working groups, especially the section on calibrations for the various detectors.
 +
* Drafts of all talks exist.
 +
** The official agenda and talks will be host on the [https://www.jlab.org/indico/ JLab Indico site].
 +
* Last Monday we had a meeting with Rolf. Talks from all of the Halls were reviewed.
 +
 
 +
==Simulation and Reconstruction Job Failures==
 +
 
 +
Kei have a detailed look at studies he has done to isolate failures in our simulation and reconstruction. See [https://halldweb.jlab.org/talks/2013-4Q/2013-11-13-iu-minidatachallenge.pdf his slides] for details.
 +
 
 +
He classified failures of reconstruction into two classes:
 +
# stopped: job did not run to completion
 +
# unusable: output REST file either small or not readable
 +
 
 +
He tried a variety of scenarios: isolating smeared, un-reconstructed events, with and without the preceding events in the file, isolating generated events (bggen output). He tried reproducing results in various configuration. Failures of all kinds occur indicating more than one problem is afflicting us.
 +
 
 +
He provided links to some of the problem event samples and to scripts he used in his studies.
 +
 
 +
==An SCons System for GlueX==
 +
 
 +
David presented his recent development of an SCons-based build system for sim-recon, calld SBMS. See [[Media:20131113_scons.pdf|his slides]] for details.
 +
 
 +
This is replacement for the GNU Make system we have using up to now. SCons is a Python-based system and as such allows scripting within its framework more naturally than for Make. The system is checked in on the trunk and lives side-by-side with the makefiles. Either scheme can be used.
 +
 
 +
Some slide titles:
 +
* Build System Features
 +
* Make vs. SCons
 +
* Why switch?
 +
* Common Makefile/SConscript
 +
* Directory-level (dispatchers)
 +
* Times to compile and install all of sim-recon (and clean) on gluon4
 +
* How to use
 +
* Output
 +
* Telling scons to show you what it's doing
 +
* Change to installation directory Structure
 +
* Documentation https://halldweb.jlab.org/wiki/index.php/SCons_Build_System

Latest revision as of 17:00, 24 February 2017

GlueX Offline Software Meeting
Wednesday, November 13, 2013
1:30 pm EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. Data Management Plan posted (Mark)
    2. New release: sim-recon-2013-10-17 (Mark)
    3. New storage type at JLab: pass-through cache (Mark)
    4. Requirements document for Data Flow Tools (Mark)
  2. Review of minutes from the last meeting: all
  3. Software Review Planning
    1. GlueX/Hall-D Computing
  4. bggen failures (Kei)
  5. An SCons System for GlueX (David)
  6. Mantis Bug Tracker Review
  7. Review of recent repository activity: all

Communication Information

Video Conferencing

Desktop Sharing

You can view the computer desktop in the meeting room at JLab via the web.

  1. Go to http://esnet.readytalk.com
  2. In the "join a meeting" box enter the Hall D code: 1833622
  3. Fill in the participant registration form.

Telephone

To connect by telephone:

  1. dial:
  2. enter access code followed by the # sign: 1833622#

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2013-4Q on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2013-4Q/ .

Minutes

Present:

  • CMU: Will Levine, Curtis Meyer
  • IU: Kei Moriya, Matt Shepherd
  • JLab: Mark Dalton, Mark Ito (chair), David Lawrence, Yi Qiang, Dmitry Romanov, Sascha Somov, Nathan Sparks, Simon Taylor, Elliott Wolin, Beni Zihlmann
  • MIT: Justin Stevens

Announcements

  • The Hall D Data Management Plan has been posted on the wiki here.
  • There is a new release of sim-recon, sim-recon-2013-10-17. Justin pointed out that this release has the disease of missing tracks in the backward direction. The precise plan to get out of this situation will be discussed in the coming days, offline.
  • New storage type at JLab: pass-through cache
    • This is writable disk space that is automatically backed up to the tape library. As the disk fills up, the oldest files (by access date) will be deleted, but those files are not lost forever. The user can request that any file be restored from the tape library. This restoration may displace other long-unaccessed files but...you get the idea.
  • Requirements document for Data Flow Tools
    • The committee charged to come up with these requirements has done so. The next step is to write specifications from which coding can proceed. A working committee has been formed to do this: Jie Chen (chair), Johann Goetz, Vardan Guryjan, Mark Ito, Chris Larrieu, and Dennis Weygand. These requirements have been added as an appendix to the computing document to be given to the Software Review II committee.

Software Review Planning

Mark I. gave the report. Most of this information is collected in the Software Review Planning wiki page.

  • We have a final agenda from Rolf Ent.
  • We have a mostly complete written document for the reviewers.
    • I contains various appendices covering workflow tools, the data management plan, and an analysis plan the Curtis wrote recently.
    • The draft would benefit from review from the working groups, especially the section on calibrations for the various detectors.
  • Drafts of all talks exist.
  • Last Monday we had a meeting with Rolf. Talks from all of the Halls were reviewed.

Simulation and Reconstruction Job Failures

Kei have a detailed look at studies he has done to isolate failures in our simulation and reconstruction. See his slides for details.

He classified failures of reconstruction into two classes:

  1. stopped: job did not run to completion
  2. unusable: output REST file either small or not readable

He tried a variety of scenarios: isolating smeared, un-reconstructed events, with and without the preceding events in the file, isolating generated events (bggen output). He tried reproducing results in various configuration. Failures of all kinds occur indicating more than one problem is afflicting us.

He provided links to some of the problem event samples and to scripts he used in his studies.

An SCons System for GlueX

David presented his recent development of an SCons-based build system for sim-recon, calld SBMS. See his slides for details.

This is replacement for the GNU Make system we have using up to now. SCons is a Python-based system and as such allows scripting within its framework more naturally than for Make. The system is checked in on the trunk and lives side-by-side with the makefiles. Either scheme can be used.

Some slide titles:

  • Build System Features
  • Make vs. SCons
  • Why switch?
  • Common Makefile/SConscript
  • Directory-level (dispatchers)
  • Times to compile and install all of sim-recon (and clean) on gluon4
  • How to use
  • Output
  • Telling scons to show you what it's doing
  • Change to installation directory Structure
  • Documentation https://halldweb.jlab.org/wiki/index.php/SCons_Build_System