Difference between revisions of "SWIF Analysis Jobs"

From GlueXWiki
Jump to: navigation, search
(GlueX Analysis Jobs)
Line 25: Line 25:
  
 
== GlueX Analysis Jobs ==
 
== GlueX Analysis Jobs ==
 +
 +
The scripts at:
 +
 +
* [https://halldsvn.jlab.org/repos/trunk/home/pmatt/jobs/analysis/ SWIF Analysis Jobs Scripts]
 +
 +
can be used to easily submit analysis jobs to the cluster using SWIF.  Just follow these steps:
 +
 +
1) Check out the scripts:
 +
 +
svn co https://halldsvn.jlab.org/repos/trunk/home/pmatt/jobs/analysis/
 +
 +
2) Then, in that folder, edit the "GLOBAL VARIABLES" section of the swif_gluex_analysis.py file to:
 +
 +
* Change the resource requirements for your jobs.
 +
* Change the source data information. This tells the script where to look for the input REST files for your jobs.
 +
* Change the output data location: This tells the script where to save your output files.
 +
* Change the script file path: This should the path to which you just checked out the script.sh file. 
 +
* Change the environment file: This is the file that the job script sources to setup your environment prior to running hd_root. 
 +
* Change the config file path: This is where the script will auto-generate the JANA config file used when running hd_root.  It contains the command-line arguments used when running hd_root.  It can be anywhere you like. 
 +
* Change the tree names: This should be a comma-separated list of names of the root tree files that are generated by your analysis that you want to save.  For example, if the output tree file will have the name "tree_klambda.root", you should have "klambda" in the list to save it. 
 +
* Change the config file contents: These are the command line arguments used when running hd_root. 
 +
 +
3) Create your SWIF workflow and add jobs to it by calling:
 +
 +
swif_gluex_analysis.py my_workflow_name min_run max_run
 +
 +
This will attempt to create the workflow each time you call it, so if the workflow already exists, it will give you an error message.  It can be ignored. 
 +
 +
4) Start your workflow (submit jobs) by calling:
 +
 +
swif run -workflow my_workflow_name -errorlimit none
 +
 +
5) Check the status of your workflow (jobs) with:
 +
 +
swif status my_workflow_name
 +
 +
6) Resubmit failed jobs, change resource requirements, etc. by following the instructions at:
 +
 +
* [https://scicomp.jlab.org/docs/swif-cli SWIF Command-Line Instructions]

Revision as of 19:31, 29 February 2016

Overview

This page outlines how you can use my scripts to submit analysis jobs for SWIF. It assumes that you already have a working knowledge of the JLab farm system. If not, more information can be found at:

SciComp Job Links

Main

Documentation

Job Tracking

GlueX Analysis Jobs

The scripts at:

can be used to easily submit analysis jobs to the cluster using SWIF. Just follow these steps:

1) Check out the scripts:

svn co https://halldsvn.jlab.org/repos/trunk/home/pmatt/jobs/analysis/

2) Then, in that folder, edit the "GLOBAL VARIABLES" section of the swif_gluex_analysis.py file to:

  • Change the resource requirements for your jobs.
  • Change the source data information. This tells the script where to look for the input REST files for your jobs.
  • Change the output data location: This tells the script where to save your output files.
  • Change the script file path: This should the path to which you just checked out the script.sh file.
  • Change the environment file: This is the file that the job script sources to setup your environment prior to running hd_root.
  • Change the config file path: This is where the script will auto-generate the JANA config file used when running hd_root. It contains the command-line arguments used when running hd_root. It can be anywhere you like.
  • Change the tree names: This should be a comma-separated list of names of the root tree files that are generated by your analysis that you want to save. For example, if the output tree file will have the name "tree_klambda.root", you should have "klambda" in the list to save it.
  • Change the config file contents: These are the command line arguments used when running hd_root.

3) Create your SWIF workflow and add jobs to it by calling:

swif_gluex_analysis.py my_workflow_name min_run max_run

This will attempt to create the workflow each time you call it, so if the workflow already exists, it will give you an error message. It can be ignored.

4) Start your workflow (submit jobs) by calling:

swif run -workflow my_workflow_name -errorlimit none

5) Check the status of your workflow (jobs) with:

swif status my_workflow_name

6) Resubmit failed jobs, change resource requirements, etc. by following the instructions at:

* SWIF Command-Line Instructions