Difference between revisions of "Offline Monitoring Incoming Data"
From GlueXWiki
(→Offline) |
(→Offline) |
||
| Line 23: | Line 23: | ||
</pre> | </pre> | ||
| + | == Setup for a new run period == | ||
| + | |||
| + | # Edit the job config file, ~/monitoring/incoming/input.config, which is used to register jobs in hdswif. A typical config file will look this: | ||
| + | PROJECT gluex | ||
| + | TRACK reconstruction | ||
| + | OS centos65 | ||
| + | NCORES 24 # 24 = Entire node | ||
| + | DISK 40 | ||
| + | RAM 32 # 32 GB per node | ||
| + | TIMELIMIT 8 | ||
| + | JOBNAMEBASE offmon | ||
| + | RUNPERIOD 2016-02 | ||
| + | VERSION 01 | ||
| + | OUTPUT_TOPDIR /cache/halld/offline_monitoring/RunPeriod-[RUNPERIOD]/ver[VERSION] # Example of other variables included in variable | ||
| + | SCRIPTFILE /home/gxproj1/monitoring/incoming/script.sh # Must specify full path | ||
| + | ENVFILE /home/gxproj1/env_monitoring_incoming # Must specify full path | ||
| + | |||
| + | # Create a new swif workflow: | ||
| + | <pre> | ||
| + | ~/monitoring/hdswif/hdswif.py create [workflow] -c ~/monitoring/incoming/input.config | ||
| + | </pre> | ||
* If not already running, the cron job can be launched by running: | * If not already running, the cron job can be launched by running: | ||
<pre> | <pre> | ||
Revision as of 17:30, 3 February 2016
Saving Online Monitoring Data
The procedure for writing the data out is given in, e.g., Raid-to-Silo Transfer Strategy.
Once the DAQ writes out the data to the raid disk, cron jobs will copy the file to tape, and within ~20 min., we will have access to the file on tape at /mss/halld/$RUN_PERIOD/rawdata/RunXXXXXX.
All online monitoring plugins will be run as data is taken. They will be accessible within the counting house via RootSpy, and for each run and file, a ROOT file containing the histograms will be saved within a subdirectory for each run.
For immediate access to these files, the raid disk files may be accessed directly from the counting house, or the tape files will be available within ~20 min. of the file being written out.
Offline
- The scripts for processing the incoming data are stored in svn at:
https://halldsvn.jlab.org/repos/trunk/scripts/monitoring/incoming/
Setup for a new run period
- Edit the job config file, ~/monitoring/incoming/input.config, which is used to register jobs in hdswif. A typical config file will look this:
PROJECT gluex TRACK reconstruction OS centos65 NCORES 24 # 24 = Entire node DISK 40 RAM 32 # 32 GB per node TIMELIMIT 8 JOBNAMEBASE offmon RUNPERIOD 2016-02 VERSION 01 OUTPUT_TOPDIR /cache/halld/offline_monitoring/RunPeriod-[RUNPERIOD]/ver[VERSION] # Example of other variables included in variable SCRIPTFILE /home/gxproj1/monitoring/incoming/script.sh # Must specify full path ENVFILE /home/gxproj1/env_monitoring_incoming # Must specify full path
- Create a new swif workflow:
~/monitoring/hdswif/hdswif.py create [workflow] -c ~/monitoring/incoming/input.config
- If not already running, the cron job can be launched by running:
crontab cron_incoming
