SWIF Analysis Jobs

From GlueXWiki
Revision as of 05:17, 22 June 2017 by Gvasil (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Overview

This page outlines how you can use some simple, generic scripts to submit SWIF analysis jobs on REST data at JLab. It assumes that you already have a working knowledge of the JLab farm system. If not, you can probably wing it, but more information can be found at:

Submitting GlueX Analysis Jobs

1) Check out the scripts:

svn co https://halldsvn.jlab.org/repos/trunk/home/pmatt/jobs/analysis/

2) Then, in that folder, edit the "GLOBAL VARIABLES" section of the "swif_gluex_analysis.py" file. It's largely self-explanatory, but just in case, see the section below on Script Global Variables.

3) You do NOT need to edit any other portion of the python script file.

4) You do NOT need to edit the "script.sh" file, unless you are creating skim files and want to save them. Otherwise, it will automatically save all results (danarest REST files, ROOT histograms, ROOT trees) (stdout & stderr saved by the job itself).

5) Create your SWIF workflow and add jobs to it by calling:

swif_gluex_analysis.py my_workflow_name min_run max_run

This will attempt to create the workflow each time you call it, so if the workflow already exists, it will give you an error message. It can be ignored.

6) Start your workflow (submit jobs) by calling:

swif run -workflow my_workflow_name -errorlimit none

7) Check the status of your workflow (jobs) with:

swif status my_workflow_name

8) Resubmit failed jobs, change resource requirements, etc. by following the instructions at:

9) Check your job output files. They will go to the location you requested in "swif_gluex_analysis.py". The log files (stdout and stderr) will be there as well.

Script Global Variables

  • Change the resource requirements for your jobs. This should ideally be the minimum necessary: the less you require, the sooner your jobs will run. However, exceed these, and the job will fail.
  • Change the source data information. This tells the script where to look for the input REST files for your jobs.
  • Change the output data location: This tells the script where to save your output files.
  • Job execution settings:
    • Change the script file path: This should the path to which you just checked out the "script.sh" file.
    • Change the environment file: This is the file that the job script sources to setup your environment prior to running hd_root.
    • Change the config file path: This is where the script will auto-generate the JANA config file used when running hd_root. It contains the command-line arguments used when running hd_root. It can be anywhere you like.
    • Change the tree names: This should be a comma-separated list of names of the root tree files that are generated by your analysis that you want to save. For example, if the output tree file will have the name "tree_klambda.root", you should have "klambda" in the list to save it. In general, these are the same names as your plugins.
  • Change the JANA config file contents: These are the command line arguments used when running hd_root. You should especially change the plugins.

SciComp Job Links (For Reference)

Main

Documentation

Job Tracking