Difference between revisions of "Offline Data Processing"
From GlueXWiki
(→Run Information) |
(→Monitoring) |
||
(3 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | == | + | == To Include == |
− | + | - Eventstore, grid, university sites, calibration validation, reconstruction validation, skim validation | |
− | + | ||
− | + | - Code versioning, config scripts, software version restoration, database access (web, batch scripts, JANA) | |
− | + | - Documentation, backwards compatibility | |
− | + | ||
+ | == Monitoring == | ||
+ | |||
+ | * Monitoring jobs will be run on new data as it is written to tape. These jobs will do several things: | ||
+ | |||
+ | * If calibration / reconstruction-improvement work is ongoing, monitoring jobs will also be rerun on tape data at regular intervals. This may be weekly or bi-weekly, and may run over all of the data for the given run period, or only a subset of it, as the situation demands. | ||
== Calibrations == | == Calibrations == | ||
− | + | == Run Information == | |
− | + | == Reconstruction == | |
− | + | REST-format reconstruction data will be saved to tape at JLab. If the disk footprint is small enough, all of the REST reconstruction results will be stored on the /halld/volatile/ disk. If not, then a 2-track skim of the data will be available. | |
− | + | == Run Classification (Good / Junk) == | |
− | == | + | == Physical Skimming == |
− | * | + | * Physical, EVIO-format skims will be made available to the calibration on the /halld/volatile/ disk as needed to assist with calibration studies. No other physical skims will be created. |
− | + | == Logical Skimming == | |
− | * | + | * Logical skims will be stored in EventStore. These will be created by running the |
− | + | == Analysis Trains == | |
− | + | === Daily, Yesterday's-Runs === | |
− | + | === Weekly, Subset-of-all-past-runs === | |
− | + | ||
− | + | === Monthly, All-Past-Data === | |
− | + | == Simulation == | |
− | == | + | === Channel-by-channel Phase-space === |
− | + | === bggen === | |
− | + | == Acceptance studies == |
Latest revision as of 17:55, 20 January 2015
Contents
To Include
- Eventstore, grid, university sites, calibration validation, reconstruction validation, skim validation
- Code versioning, config scripts, software version restoration, database access (web, batch scripts, JANA)
- Documentation, backwards compatibility
Monitoring
- Monitoring jobs will be run on new data as it is written to tape. These jobs will do several things:
- If calibration / reconstruction-improvement work is ongoing, monitoring jobs will also be rerun on tape data at regular intervals. This may be weekly or bi-weekly, and may run over all of the data for the given run period, or only a subset of it, as the situation demands.
Calibrations
Run Information
Reconstruction
REST-format reconstruction data will be saved to tape at JLab. If the disk footprint is small enough, all of the REST reconstruction results will be stored on the /halld/volatile/ disk. If not, then a 2-track skim of the data will be available.
Run Classification (Good / Junk)
Physical Skimming
- Physical, EVIO-format skims will be made available to the calibration on the /halld/volatile/ disk as needed to assist with calibration studies. No other physical skims will be created.
Logical Skimming
- Logical skims will be stored in EventStore. These will be created by running the