GlueX Software Meeting, March 3, 2020
GlueX Software Meeting
Tuesday, March 3, 2020
3:30 pm EST
JLab: CEBAF Center A110
BlueJeans: 968 592 007
- Review of Minutes from the Last Software Meeting (all)
- recon.xml files matching between MC and data analysis launches
- Review of recent issues and pull requests:
- Review of recent discussion on the GlueX Software Help List (all)
- Action Item Review (all)
- FSU: Sean Dobbs
- JLab: Alex Austregesilo, Hovanes Egiyan, Mark Ito (chair), Igal Jaegle, David Lawrence, Keigo Mizutani, Andrew Schick, Beni Zihlmann
- ODU: Nilanga Wickramaarachchi
There is a recording of his meeting on the BlueJeans site. Use your JLab credentials to access it.
- /volatile/halld moving to a new Lustre system Files on the old partition will be deleted tomorrow.
- Deprecating DIST in favor of HALLD_VERSIONS for version.xml files. Currently existing version set files will remain in DIST until May 1.
Review of Minutes from the Last Software Meeting
We went over the minutes from February 18.
- The plan to delete some old CDC tables from the CCDB went slightly awry. We had to restore from backup and scuttle the plan.
- The issue on having the collaboration in general maintain data integrity on the RCDB has been submitted.
- Dakota Christian's issue with MC turned out to be sensitivity to the order of decays in genr8.
recon.xml files matching between MC and data analysis launches
We discuss an issue raised in this question to the software help group from Nacer Hamdi.
The fundamental complication raised is that to do reconstruction we rely on a particular version of halld_recon. Subsequent Monte Carlo simulation therefore need to use the same version of halld_recon to be consistent. On the other hand, analysis launches on REST data produced by the reconstruction launches produce ROOT trees using the analysis library (among other packages) also provided by halld_recon. So root tree creation for simulation, to be consistent, needs to use this second version of halld_recon. This was first discussed back in May of last year. The solution we settled on was to create a new XML files (version_correlations.xml) that would identify pairs of halld_recon versions (actually version set files) so that the appropriate versions could be deployed on Monte Carlo.
- Nacer's question pointed out that not all useful version set pairs have been recorded and built.
- Mark pointed out that the implementation currently in MCwrapper-bot to guide the user to a choice of consistent versions is complicated. It would be difficult to describe at a level that can be reproduced easily by non-experts.
- Mark further pointed out the the correlation XML files was a stop-gap solution. It was easy to implement and works, but was not necessarily the "best" solution.
- Mark's suggestion is that we go back and revisit the idea of breaking out the analysis libraries from halld_recon. Recall that from that discussion last year, this is not as easy as it might seem on the surface; there are in fact many libraries involved in the overlap between reconstruction and analysis, not just the analysis library.
- Alex was skeptical about whether further deconstruction of halld_recon would result in a simpler system since there would still be libraries used in both reconstruction and analysis and so the two-version problem would still obtain.
- Alex proposes that the choice of MCwrapper be based on the version set of the analysis launch that is being simulated. Then the choice of both halld_recon versions should be unambiguous.
- Sean pointed out that is all of the information about what versions were used for which launches is stored in a database, an interface should be able to pick out correct pairs of versions sets.
- Alex has spoken to Thomas Britton about implementing the "pick-an-analysis-launch" approach for MCwrapper-bot. It was alleged that Thomas has agreed to work on it (someday).
Review of recent issues and pull requests
We went over several of the issue pages. See the list above in the agenda.
In the course of the review, Sean raised the issue of access control for the CCDB. We have seen mistakes go into the default variation of the CCDB on the master replica on the server on hallddb.jlab.org. Most of the time these have been caught and corrected promptly, but there is an example where a non-optimal set of constants was used for production reconstruction. Approaches:
- Sean advocates creating a white list so that only certain users can write to the default variation on the master.
- [added in press] This was not in the original implementation because it would prevent users from creating private variations on the database server, private variation that could be used for testing and learning. To maintain that capability an custom access control system would have to be coded and managed.
- David floated the idea of having a the master on a separate server altogether. This server could be configured to require a password when connecting.
Mark agreed to talk to Dmitry Romanov again to review what hooks are already available and get his ideas on an approach.