Difference between revisions of "Offline Monitoring Archived Data"

From GlueXWiki
Jump to: navigation, search
(→‎Prepare the Launch)
(Undo revision 108850 by Aaustreg (talk))
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
= Setup the Software & Environment =
 
= Setup the Software & Environment =
  
−
'''1)''' Checkout (or svn update) the launch scripts if needed:
+
'''1)''' Update the version xml file to indicate which versions of the software will be used. For hdds & sim-recon, set the names to those of the tags that you will be about to create (in a later step below):
 
<pre>
 
<pre>
−
svn co https://halldsvn.jlab.org/repos/trunk/scripts/monitoring/launch
+
~/builds/version_monitoring_launch.xml
 
</pre>
 
</pre>
  
−
'''2)''' You will need only the following 3 files:
+
'''2)''' Source the environment.  It may be tempting, but do NOT put this in the auto-exec-on-login scripts. It is dangerous, in case you need to switch between environments. Just call:  
 
<pre>
 
<pre>
−
launch.py
+
bash
−
script.sh
+
source ~/env_monitoring_launch.sh
−
jobs_analysis_bggen.config
+
 
</pre>
 
</pre>
  
−
= Prepare the Launch =
+
'''3)''' Updating & building hdds:
 +
<pre>
 +
cd $HDDS_HOME
 +
git pull                # Get latest software
 +
scons -c install        # Clean out the old install: EXTREMELY IMPORTANT for cleaning out stale headers
 +
scons install -j4      # Rebuild and re-install with 4 threads
 +
</pre>
  
−
'''1)''' Update jobs_analysis_bggen.config. Be sure to update the location for the '''<span style="color:red">SCRIPTFILE</span>''', '''<span style="color:red">JANA_CONFIG</span>''', and '''<span style="color:red">ENVFILE</span>''' where appropriate. The xml file that was used for the analysis launch for data should be chosen:
+
'''4)''' Updating & building halld_recon:  
 
<pre>
 
<pre>
−
ENVFILE               analysis-2017_01-ver46.xml
+
cd $HALLD_RECON_HOME/src
 +
git pull               # Get latest software
 +
scons -c install        # Clean out the old install: EXTREMELY IMPORTANT for cleaning out stale headers
 +
scons install -j4      # Rebuild and re-install with 4 threads
 
</pre>
 
</pre>
  
−
'''2)''' Also update the paths for '''<span style="color:red">INDATA_TOPDIR</span>''', '''<span style="color:red">OUTDIR_LARGE</span>''' (for trees and histograms), and '''<span style="color:red">OUTDIR_SMALL</span>''' (for log files):
+
'''5)''' Create a new sqlite file containing the very latest calibration constants. Original documentation on creating sqlite files are [https://halldweb.jlab.org/wiki/index.php/SQLite-form_of_the_CCDB_database here].
 
<pre>
 
<pre>
−
INDATA_TOPDIR    /cache/halld/gluex_simulations/REQUESTED_MC/F2018_ver02_21_bggen_batch01_20211104064237pm/hddm
+
cd `dirname $SQLITE_PATH`
−
OUTDIR_LARGE    /volatile/halld/home/$USER/bggen
+
$CCDB_HOME/scripts/mysql2sqlite/mysql2sqlite.sh -hhallddb.jlab.org -uccdb_user ccdb | sqlite3 ccdb.sqlite
−
OUTDIR_SMALL    /farm_out/$USER/bggen
+
mv ccdb.sqlite ccdb_monitoring_launch.sqlite
 
</pre>
 
</pre>
  
−
'''3)''' Make a new jana_analysis.config with your channel:
+
'''6)''' Tag the software, where '''<span style="color:red">"<type>"</span>''' below is either '''<span style="color:red">"offmon"</span>''' for offline monitoring launches, or '''<span style="color:red">"recon"</span>''' for full reconstruction launches:  
 
<pre>
 
<pre>
−
PLUGINS monitoring_hists,ReactionFilter
+
cd $HALLD_HOME/src/
−
NTHREADS 16
+
git tag -a <type>-20YY_MM-verVV -m "Used for offline monitoring 20YY-MM verVV"
−
COMBO:MAX_NEUTRALS 15
+
git push origin <type>-20YY_MM-verVV
−
Reaction1  1_14__8_9_14                # pi+ pi- p
+
 
−
Reaction1:Flags B4
+
cd $HDDS_HOME/
 +
git tag -a <type>-20YY_MM-verVV -m "Used for offline monitoring 20YY-MM verVV"
 +
git push origin <type>-20YY_MM-verVV
 
</pre>
 
</pre>
  
 +
'''7)''' Checkout (or svn update) the launch scripts if needed:
 +
<pre>
 +
cd ~/
 +
svn co https://halldsvn.jlab.org/repos/trunk/scripts/monitoring/
 +
</pre>
 +
 +
= Starting a new run period =
 +
When a new run period is started, it is best to make sure all top-level directories are created with the right permissions. This can save headaches later on when a different gxprojN account is used for offline monitoring.
  
−
'''4)''' Create the SWIF workflow.  The workflow must match the workflow name in the job config file:
+
'''1)''' Create top-level directories
 
<pre>
 
<pre>
−
swif create -workflow <my_workflow>
+
/cache/halld/RunPeriod-20YY-MM/recon/
 +
/cache/halld/offline_monitoring/RunPeriod-20YY-MM/
 +
/work/halld2/recon/RunPeriod-20YY-MM/
 +
/work/halld/data_monitoring/RunPeriod-20YY-MM/
 +
/work/halld2/data_monitoring/RunPeriod-20YY-MM/
 +
/group/halld/data_monitoring/run_conditions/RunPeriod-20YY-MM/
 
</pre>
 
</pre>
  
 +
'''2)''' Make sure other gxprojN users can write in with '''<span style="color:red">chmod g+w</span>''' [dir name] (NB: use '''<span style="color:red">jcache chmod</span>''' for /cache).<br> Check that permissions match those from previous run periods.
  
−
'''5)''' Register jobs for the workflow:
+
'''3)''' ???Deprecated??? Since the publish-to-web scripts are webpage-update scripts, you need to have pre-existing, template versions of a few files for the new run period.  It also expects to find comment hooks that include the new run period name, so make sure those are edited in the new template files.
 +
 
 +
= Prepare and Do the Launch =
 +
 
 +
'''1)''' Update the appropriate job config file, depending on the type of launch (e.g. '''<span style="color:red">jobs_offmon.config</span>''' for monitoring launches, '''<span style="color:red">jobs_recon.config</span>''' for full reconstruction). Definitely be sure to update '''<span style="color:red">RUNPERIOD</span>''', '''<span style="color:red">VERSION</span>''', and '''<span style="color:red">BATCH</span>''' where appropriate (if jobs submitted in batches).
 
<pre>
 
<pre>
−
~/monitoring/launch/launch.py jobs_analysis_bggen.config <run_min> <run_max>
+
~/monitoring/launch/jobs_offmon.config
 
</pre>
 
</pre>
  
 +
'''2)''' Update the appropriate jana config file, depending on the type of launch (e.g. '''<span style="color:red">jana_offmon.config</span>''' for monitoring launches, '''<span style="color:red">jana_recon.config</span>''' for full reconstruction). This contains the command line arguments given to JANA. Definitely be sure to update '''<span style="color:red">REST:DATAVERSIONSTRING</span>''' and '''<span style="color:red">JANA_CALIB_CONTEXT</span>'''.
 +
<pre>
 +
~/monitoring/launch/jana_offmon.config
 +
</pre>
 +
 +
'''3)''' Create the SWIF workflow.  The workflow should have a name like '''<span style="color:red">"recon_2016-02_ver05"</span>''' for monitoring launches and '''<span style="color:red">"recon_2016-02_ver01_batch01"</span>''' for full reconstruction launches.  It should also match the workflow name in the job config file (e.g. jobs_offmon.config).
 +
<pre>
 +
swif create -workflow <my_workflow>
 +
</pre>
 +
 +
'''4)''' Backup the software versions & appropriate jana config script, '''<span style="color:red">using appropriate names</span>''' for the launch, '''<span style="color:red">e.g.</span>''':
 +
<pre>
 +
cp ~/builds/version_monitoring_launch.xml /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/version_recon_2016_02_ver01.xml
 +
cp ~/monitoring/launch/jana_recon.config /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/jana_recon_2016_02_ver01_batch01.config
 +
cp ~/monitoring/launch/jobs_recon.config /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/jobs_recon_2016_02_ver01_batch01.config
 +
</pre>
 +
 +
'''5)''' Register jobs for the workflow, where '''<span style="color:red"><job_config_file></span>''' is (e.g.) '''<span style="color:red">"~/monitoring/launch/jobs_offmon.config"</span>''':
 +
<pre>
 +
~/monitoring/launch/launch.py <job_config_file> <run_min> <run_max>
 +
</pre>
 +
You can optionally specify specific file numbers to use.  For example, to submit jobs for the first 5 files of each run:
 +
<pre>
 +
~/monitoring/launch/launch.py <job_config_file> <run_min> <run_max> -f '00[0-4]'
 +
</pre>
  
 
'''6)''' Run the workflow:
 
'''6)''' Run the workflow:

Latest revision as of 18:21, 12 November 2021

Setup the Software & Environment

1) Update the version xml file to indicate which versions of the software will be used. For hdds & sim-recon, set the names to those of the tags that you will be about to create (in a later step below): 

~/builds/version_monitoring_launch.xml

2) Source the environment. It may be tempting, but do NOT put this in the auto-exec-on-login scripts. It is dangerous, in case you need to switch between environments. Just call: 

bash
source ~/env_monitoring_launch.sh

3) Updating & building hdds: 

cd $HDDS_HOME
git pull                # Get latest software
scons -c install        # Clean out the old install: EXTREMELY IMPORTANT for cleaning out stale headers
scons install -j4       # Rebuild and re-install with 4 threads

4) Updating & building halld_recon: 

cd $HALLD_RECON_HOME/src
git pull                # Get latest software
scons -c install        # Clean out the old install: EXTREMELY IMPORTANT for cleaning out stale headers
scons install -j4       # Rebuild and re-install with 4 threads

5) Create a new sqlite file containing the very latest calibration constants. Original documentation on creating sqlite files are here. 

cd `dirname $SQLITE_PATH`
$CCDB_HOME/scripts/mysql2sqlite/mysql2sqlite.sh -hhallddb.jlab.org -uccdb_user ccdb | sqlite3 ccdb.sqlite
mv ccdb.sqlite ccdb_monitoring_launch.sqlite

6) Tag the software, where "<type>" below is either "offmon" for offline monitoring launches, or "recon" for full reconstruction launches: 

cd $HALLD_HOME/src/
git tag -a <type>-20YY_MM-verVV -m "Used for offline monitoring 20YY-MM verVV"
git push origin <type>-20YY_MM-verVV

cd $HDDS_HOME/
git tag -a <type>-20YY_MM-verVV -m "Used for offline monitoring 20YY-MM verVV"
git push origin <type>-20YY_MM-verVV

7) Checkout (or svn update) the launch scripts if needed: 

cd ~/
svn co https://halldsvn.jlab.org/repos/trunk/scripts/monitoring/

Starting a new run period

When a new run period is started, it is best to make sure all top-level directories are created with the right permissions. This can save headaches later on when a different gxprojN account is used for offline monitoring.

1) Create top-level directories 

/cache/halld/RunPeriod-20YY-MM/recon/
/cache/halld/offline_monitoring/RunPeriod-20YY-MM/
/work/halld2/recon/RunPeriod-20YY-MM/
/work/halld/data_monitoring/RunPeriod-20YY-MM/
/work/halld2/data_monitoring/RunPeriod-20YY-MM/
/group/halld/data_monitoring/run_conditions/RunPeriod-20YY-MM/

2) Make sure other gxprojN users can write in with chmod g+w [dir name] (NB: use jcache chmod for /cache).
Check that permissions match those from previous run periods.

3) ???Deprecated??? Since the publish-to-web scripts are webpage-update scripts, you need to have pre-existing, template versions of a few files for the new run period. It also expects to find comment hooks that include the new run period name, so make sure those are edited in the new template files.

Prepare and Do the Launch

1) Update the appropriate job config file, depending on the type of launch (e.g. jobs_offmon.config for monitoring launches, jobs_recon.config for full reconstruction). Definitely be sure to update RUNPERIOD, VERSION, and BATCH where appropriate (if jobs submitted in batches). 

~/monitoring/launch/jobs_offmon.config

2) Update the appropriate jana config file, depending on the type of launch (e.g. jana_offmon.config for monitoring launches, jana_recon.config for full reconstruction). This contains the command line arguments given to JANA. Definitely be sure to update REST:DATAVERSIONSTRING and JANA_CALIB_CONTEXT. 

~/monitoring/launch/jana_offmon.config

3) Create the SWIF workflow. The workflow should have a name like "recon_2016-02_ver05" for monitoring launches and "recon_2016-02_ver01_batch01" for full reconstruction launches. It should also match the workflow name in the job config file (e.g. jobs_offmon.config). 

swif create -workflow <my_workflow>

4) Backup the software versions & appropriate jana config script, using appropriate names for the launch, e.g.: 

cp ~/builds/version_monitoring_launch.xml /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/version_recon_2016_02_ver01.xml
cp ~/monitoring/launch/jana_recon.config /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/jana_recon_2016_02_ver01_batch01.config
cp ~/monitoring/launch/jobs_recon.config /group/halld/data_monitoring/run_conditions/RunPeriod-2016-02/jobs_recon_2016_02_ver01_batch01.config

5) Register jobs for the workflow, where <job_config_file> is (e.g.) "~/monitoring/launch/jobs_offmon.config": 

~/monitoring/launch/launch.py <job_config_file> <run_min> <run_max>

You can optionally specify specific file numbers to use. For example, to submit jobs for the first 5 files of each run: 

~/monitoring/launch/launch.py <job_config_file> <run_min> <run_max> -f '00[0-4]'

6) Run the workflow: 

swif run -workflow <my_workflow>
Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Offline_Monitoring_Archived_Data&oldid=108862"