Difference between revisions of "GlueX Offline Meeting, March 22, 2017"

From GlueXWiki
Jump to: navigation, search
(add minutes)
 
(3 intermediate revisions by 2 users not shown)
Line 13: Line 13:
 
# Review of [[GlueX Offline Meeting, March 8, 2017#Minutes|minutes from the last meeting]] (all)
 
# Review of [[GlueX Offline Meeting, March 8, 2017#Minutes|minutes from the last meeting]] (all)
 
# Plugin linking change proposal (David)
 
# Plugin linking change proposal (David)
# RCDB/CCDB Plans (Dmitry)
+
# [https://docs.google.com/presentation/d/1VyREyJygfBNz9pra3-1FIM7YUNo3Q8EopmhhUvAcVW0/edit?usp=sharing RCDB/CCDB Plans (Dmitry)]
 
# [[Media:20170322_jana_updates.pdf|JANA 0.7.8]] (David)
 
# [[Media:20170322_jana_updates.pdf|JANA 0.7.8]] (David)
 +
# Merging events into a simulated data stream. (Richard)
 
# Review of [https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aopen+is%3Apr recent pull requests] (all)
 
# Review of [https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aopen+is%3Apr recent pull requests] (all)
 
#* hdds from ccdb
 
#* hdds from ccdb
Line 30: Line 31:
  
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2017</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2017/ .
 
Talks can be deposited in the directory <code>/group/halld/www/halldweb/html/talks/2017</code> on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2017/ .
 +
 +
== Minutes ==
 +
 +
Present:
 +
* '''CMU''': Naomi Jarvis, Curtis Meyer
 +
* '''FIU''': Mahmoud Kamel
 +
* '''Glasgow''': Peter Pauli
 +
* '''JLab''': Alex Austregesilo, Thomas Britton, Brad Cannon, Eugene Chudakov, Hovanes Egiyan, Sergey Furletov, Mark Ito (chair), David Lawrence, Paul Mattione, Dmitry Romanov, Nathan Sparks, Simon Taylor, Beni Zihlmann
 +
* '''NU''':  Sean Dobbs
 +
* '''UConn''': Richard Jones
 +
 +
There is a [https://bluejeans.com/s/AYd1J/ recording of this meeting] on the BlueJeans site.
 +
 +
=== Announcements ===
 +
 +
# '''New simple email list: duplicate_cache'''. See [https://mailman.jlab.org/pipermail/halld-offline/2017-February/002656.html Mark's email message]. Subscribers get a weekly list of files on the cache disk that are in conflict with the tape library. Action is required in that case.
 +
# '''CCDB 1.06.03'''. There is a [https://github.com/JeffersonLab/ccdb/releases/tag/v1.06.03 new sub-minor release of CCDB] that corrects one of the example Python example scripts.
 +
# '''Upcoming SWIF features'''. Sean gave us a heads up about new features that Chris Larrieu is working on. Sean had actually asked for them some time ago. They include:
 +
#* ability to change and modify tags
 +
#* specify jobs that need to run before the job of interest runs
 +
#* specify files that need to exist before running
 +
#* query a URI about whether it is safe to run or not.
 +
# '''MCwrapper changes and progress'''. Thomas reports a lot of changes recently to add versatility. It can now use HDGeant or HDGeant4, as specified by the user, and can generate events using genr8, genEtaRegge, genTwoPiAmps[?], and bggen. Event merging will be added now that that mechanism is available. The code and examples are [https://github.com/JeffersonLab/hd_utilities/tree/master/MCwrapper available on GitHub].
 +
# '''New release of HDPM''' is coming soon. Nathan will adding to the list of pre-built packages (including Geant4).
 +
 +
=== Review of minutes from the last meeting ===
 +
 +
We looked at the [[GlueX Offline Meeting, March 8, 2017#Minutes|minutes from March 8]].
 +
 +
There will be a meeting next week of people interested in the '''contribution of university resources to the OSG''', as Richard proposed. Contact Mark if you are interested.
 +
 +
=== Plugin linking change proposal ===
 +
 +
David went through [https://groups.google.com/forum/#!topic/gluex-software/QhiTV5q2cHg his post to the help list] proposing that we drop search of the standard sim-recon libraries when building plugins. There is considerable savings in disk footprint and build time by doing this. The required libraries would come in via the main binary, e.&nbsp;g., hd_root. The fear is that this scheme may not work on all operating systems.
 +
 +
Sean suggested we at least test is on CentOS 7 before unleashing it on the master branch of sim-recon. Nathan pointed out that the pull request alone will automatically kick-off a test on that platform including a runs with plug-ins. We agreed that David should go ahead and issue the pull request.
 +
 +
Mark then mentioned two other measures we could take to reduce the size of the builds and the time it takes to build them.
 +
# Build both debug and non-debug versions of the code, but only retain the non-debug version for older, non-default versions. That would result in a net savings of disk space.
 +
# Only build a subset of the plugins.
 +
Reaction to these proposals were mixed. Mark will go away and think.
 +
 +
=== RCDB/CCDB Plans ===
 +
 +
Dmitry [https://docs.google.com/presentation/d/1VyREyJygfBNz9pra3-1FIM7YUNo3Q8EopmhhUvAcVW0/edit?usp=sharing presented a list of items] that he has started working on now that his visit has begun.
 +
* RCDB performance had degraded. He found ways to restructure the SQL queries to vastly improve performance.
 +
* There is a large basket of CCDB issues to address. He identified those that will get the highest priority.
 +
** Sean commented that emphasis should be placed on the functional items rather than the presentational ones. Also he would like to see log filtering and a JavaScript API for the CCDB.
 +
 +
=== JANA 0.7.8 ===
 +
 +
David described [[Media:20170322_jana_updates.pdf|features in the new release of JANA]]. He highlighted two.
 +
# '''XML geometry in CCDB'''. This will allow run-dependent handling of the geometry as [[GlueX_Offline_Meeting,_December_7,_2016#Geometry_Issues_in_sim-recon|Sean has proposed]].
 +
# '''janaview plugin'''. New life breathed into an old feature. This plugin enables GUI-base browsing of the JANA data structures including graphical presentation of factory dependencies. You need to look at the slides to get the idea.
 +
 +
=== Merging events into a simulated data stream ===
 +
 +
Richard has released a new version of mcsmear that will combine real events with Monte Carlo events in user-controlled proportions. [https://mailman.jlab.org/pipermail/halld-offline/2017-March/002686.html His email] describes the new feature and how to access it.
 +
 +
He gave us further explanation on a few points:
 +
* If any of the real-event files runs out of events, then mcsmear will go back to the beginning of the file and re-use the events in the file, ring-buffer-like.
 +
* If there are collisions (hits in a particular channel from multiple sources) they are combined or kept separate according the the double-pulse resolution parameter stored in the CCDB for the detector in question.
 +
* Syntactically, hits are merged with an overloaded "+=" operator.
 +
* There is an option to insert a time-shift parameter so that all events from a real event source can be added at an arbitrary time offset with respect to the simulated event.
 +
* The mixing of multiple real-event streams is accomplished by using a stream of null events as the input simulated stream. Richard has provided a null event generator for this purpose.
 +
 +
=== Review of recent pull requests ===
 +
 +
We went over [https://github.com/JeffersonLab/sim-recon/pulls?q=is%3Aclosed+is%3Apr the list].
 +
 +
Sean [https://github.com/JeffersonLab/sim-recon/pull/767 fixed the problem with DBCALGeometry] that Naomi reported last time.
 +
 +
Richard noted that there are a lot of old [https://github.com/JeffersonLab/sim-recon/issues issues still open on the sim-recon GitHub site]. People are encouraged to review any issues they have opened in the past and close them if they have been taken care of.
 +
 +
=== New Build Need ===
 +
 +
With recent changes we probably need a new build with updated versions of ROOT, CCDB, JANA, HDDS, and sim-recon. Mark will work on it.

Latest revision as of 18:53, 22 March 2017

GlueX Offline Software Meeting
Wednesday, March 22, 2017
11:00 am EDT
JLab: CEBAF Center F326/327

Agenda

  1. Announcements
    1. New simple email list: duplicate_cache
    2. CCDB 1.06.03
    3. Upcoming SWIF features (Sean)
    4. MCwrapper changes and progress (Thomas)
  2. Review of minutes from the last meeting (all)
  3. Plugin linking change proposal (David)
  4. RCDB/CCDB Plans (Dmitry)
  5. JANA 0.7.8 (David)
  6. Merging events into a simulated data stream. (Richard)
  7. Review of recent pull requests (all)
    • hdds from ccdb
  8. Review of recent discussion on the Gluex Software Help List.
  9. Action Item Review

Communication Information

Remote Connection

Slides

Talks can be deposited in the directory /group/halld/www/halldweb/html/talks/2017 on the JLab CUE. This directory is accessible from the web at https://halldweb.jlab.org/talks/2017/ .

Minutes

Present:

  • CMU: Naomi Jarvis, Curtis Meyer
  • FIU: Mahmoud Kamel
  • Glasgow: Peter Pauli
  • JLab: Alex Austregesilo, Thomas Britton, Brad Cannon, Eugene Chudakov, Hovanes Egiyan, Sergey Furletov, Mark Ito (chair), David Lawrence, Paul Mattione, Dmitry Romanov, Nathan Sparks, Simon Taylor, Beni Zihlmann
  • NU: Sean Dobbs
  • UConn: Richard Jones

There is a recording of this meeting on the BlueJeans site.

Announcements

  1. New simple email list: duplicate_cache. See Mark's email message. Subscribers get a weekly list of files on the cache disk that are in conflict with the tape library. Action is required in that case.
  2. CCDB 1.06.03. There is a new sub-minor release of CCDB that corrects one of the example Python example scripts.
  3. Upcoming SWIF features. Sean gave us a heads up about new features that Chris Larrieu is working on. Sean had actually asked for them some time ago. They include:
    • ability to change and modify tags
    • specify jobs that need to run before the job of interest runs
    • specify files that need to exist before running
    • query a URI about whether it is safe to run or not.
  4. MCwrapper changes and progress. Thomas reports a lot of changes recently to add versatility. It can now use HDGeant or HDGeant4, as specified by the user, and can generate events using genr8, genEtaRegge, genTwoPiAmps[?], and bggen. Event merging will be added now that that mechanism is available. The code and examples are available on GitHub.
  5. New release of HDPM is coming soon. Nathan will adding to the list of pre-built packages (including Geant4).

Review of minutes from the last meeting

We looked at the minutes from March 8.

There will be a meeting next week of people interested in the contribution of university resources to the OSG, as Richard proposed. Contact Mark if you are interested.

Plugin linking change proposal

David went through his post to the help list proposing that we drop search of the standard sim-recon libraries when building plugins. There is considerable savings in disk footprint and build time by doing this. The required libraries would come in via the main binary, e. g., hd_root. The fear is that this scheme may not work on all operating systems.

Sean suggested we at least test is on CentOS 7 before unleashing it on the master branch of sim-recon. Nathan pointed out that the pull request alone will automatically kick-off a test on that platform including a runs with plug-ins. We agreed that David should go ahead and issue the pull request.

Mark then mentioned two other measures we could take to reduce the size of the builds and the time it takes to build them.

  1. Build both debug and non-debug versions of the code, but only retain the non-debug version for older, non-default versions. That would result in a net savings of disk space.
  2. Only build a subset of the plugins.

Reaction to these proposals were mixed. Mark will go away and think.

RCDB/CCDB Plans

Dmitry presented a list of items that he has started working on now that his visit has begun.

  • RCDB performance had degraded. He found ways to restructure the SQL queries to vastly improve performance.
  • There is a large basket of CCDB issues to address. He identified those that will get the highest priority.
    • Sean commented that emphasis should be placed on the functional items rather than the presentational ones. Also he would like to see log filtering and a JavaScript API for the CCDB.

JANA 0.7.8

David described features in the new release of JANA. He highlighted two.

  1. XML geometry in CCDB. This will allow run-dependent handling of the geometry as Sean has proposed.
  2. janaview plugin. New life breathed into an old feature. This plugin enables GUI-base browsing of the JANA data structures including graphical presentation of factory dependencies. You need to look at the slides to get the idea.

Merging events into a simulated data stream

Richard has released a new version of mcsmear that will combine real events with Monte Carlo events in user-controlled proportions. His email describes the new feature and how to access it.

He gave us further explanation on a few points:

  • If any of the real-event files runs out of events, then mcsmear will go back to the beginning of the file and re-use the events in the file, ring-buffer-like.
  • If there are collisions (hits in a particular channel from multiple sources) they are combined or kept separate according the the double-pulse resolution parameter stored in the CCDB for the detector in question.
  • Syntactically, hits are merged with an overloaded "+=" operator.
  • There is an option to insert a time-shift parameter so that all events from a real event source can be added at an arbitrary time offset with respect to the simulated event.
  • The mixing of multiple real-event streams is accomplished by using a stream of null events as the input simulated stream. Richard has provided a null event generator for this purpose.

Review of recent pull requests

We went over the list.

Sean fixed the problem with DBCALGeometry that Naomi reported last time.

Richard noted that there are a lot of old issues still open on the sim-recon GitHub site. People are encouraged to review any issues they have opened in the past and close them if they have been taken care of.

New Build Need

With recent changes we probably need a new build with updated versions of ROOT, CCDB, JANA, HDDS, and sim-recon. Mark will work on it.