Difference between revisions of "Goals for Online Data Challenge 2013"
From GlueXWiki
m |
|||
| Line 4: | Line 4: | ||
7/9/2013 D. Lawrence | 7/9/2013 D. Lawrence | ||
| − | The primary goal of the Online Data Challenge is to test the | + | The primary goal of the Online Data Challenge is to test the event monitoring/rejection |
and data flow piece that fits between the DAQ system (past the final event builder) and the tape silo. | and data flow piece that fits between the DAQ system (past the final event builder) and the tape silo. | ||
Independent DAQ group testing addresses the pieces upstream (front-end modules through event builder) while the | Independent DAQ group testing addresses the pieces upstream (front-end modules through event builder) while the | ||
| Line 22: | Line 22: | ||
#: | #: | ||
# Test data rates of EVIO formatted raw data files: | # Test data rates of EVIO formatted raw data files: | ||
| − | #* use bggen generated data files that have been passed through a L1 event filter and converted into EVIO format that | + | #* use bggen generated data files that have been passed through a L1 event filter and converted into EVIO format that represent what is expected for the real data. |
| − | #* | + | #* read events from a file and place them into ET system on a node designated as the Event Builder (EB) node. |
| − | #* | + | #* read events from EB node's ET system into L3 client processes running on farm nodes, write them to Event Recorder node (ER) running on the RAID disk. |
| + | #: | ||
| + | # Test prototype L3 rejection algorithm: | ||
| + | #* run rejection algorithm in farm nodes, mark accepted events based on algorithm results, reject some events. | ||
| + | #* create and view histograms that display L3 rejection results. | ||
#: | #: | ||
# Test data rate to tape silo: | # Test data rate to tape silo: | ||
#* Files written to RAID disk will be transported to tape silo while simultaneously writing to the RAID disk. | #* Files written to RAID disk will be transported to tape silo while simultaneously writing to the RAID disk. | ||
| − | |||
| − | |||
| − | |||
#: | #: | ||
# Monitor health of farm and DAQ system nodes (counting house only) via farm monitoring software such as Ganglia. | # Monitor health of farm and DAQ system nodes (counting house only) via farm monitoring software such as Ganglia. | ||
Revision as of 09:08, 23 July 2013
Goals for the GlueX online Data Challenge 2013
7/22/2013 revised E. Wolin
7/9/2013 D. Lawrence
The primary goal of the Online Data Challenge is to test the event monitoring/rejection and data flow piece that fits between the DAQ system (past the final event builder) and the tape silo. Independent DAQ group testing addresses the pieces upstream (front-end modules through event builder) while the offline Data Challenges of 2012 and 2013 address the pieces downstream.
The 2013 ODC will take place in the Hall-D counting house using whatever equipment is currently installed. Some of this is final versions while some is temporary to allow for software systems development.
The specific goals of the 2013 ODC are:
- Test the RootSpy system in the counting house environment:
- run monitoring processes on multiple nodes that read events from a common ET system and create root histograms
- view summed monitoring histograms on at least 4 separate monitors in the counting room simultaneously
- create archive (ROOT file) of summed monitoring histograms
- read in and compare archived histograms via overlay with "live" histograms
- Test data rates of EVIO formatted raw data files:
- use bggen generated data files that have been passed through a L1 event filter and converted into EVIO format that represent what is expected for the real data.
- read events from a file and place them into ET system on a node designated as the Event Builder (EB) node.
- read events from EB node's ET system into L3 client processes running on farm nodes, write them to Event Recorder node (ER) running on the RAID disk.
- Test prototype L3 rejection algorithm:
- run rejection algorithm in farm nodes, mark accepted events based on algorithm results, reject some events.
- create and view histograms that display L3 rejection results.
- Test data rate to tape silo:
- Files written to RAID disk will be transported to tape silo while simultaneously writing to the RAID disk.
- Monitor health of farm and DAQ system nodes (counting house only) via farm monitoring software such as Ganglia.
Additional Goals
These goals will depend on the success of the goals listed above and the state of the software development at the time of the ODC.
- Test deferred processing system for L3:
- Multiple nodes pulling events from EB node and writing to local files
- Separate process on L3 nodes detect files, read them in and write ~10% of them to ER node where they are aggregated into single a file
