HOWTO get your jobs to run on the Grid

Introduction

What I'm going to outline below is how I got my jobs to run on the Grid and what I need to do it. I'll try to include documentation where I can and fixes for other OS's. My OSG client machine is a Debbian Lenny distro here at the University of Regina. It was a random machine I had available but the OSG software works fine on it. (It didn't on my Mandriva 2010 distro on my desktop.) This was a month's worth of trial and error fixing bugs and firewall issues with Richard Jones, but we have most of them worked out and everything has been running fine for the last week or so.

Step 1: Getting your Grid Certificate

The security for the grid is quite robust and as such, a signed certificate from a known signing authority is needed. As I am working in Canada, I used Westgrid to get my certificate from [ http://www.gridcanada.ca/ca/index.html Grid Canada] under a project already registered at the UofR. You will require a sponsor who will verify that you are part of their project if you are not the project leader. It took 2-3 weeks to get my certificate since people at Westgrid were on holidays at the time. Normally is should take a few days. REMEMBER THE PASSWORD YOU SUBMITTTED!

The DOE will also provide certificates. You can follow the instructions here. Other signing authorities are also available.

It is also possible to generate your own certificate, though I have no experience of this. Instructions can be found [ http://security.ncsa.uiuc.edu/research/grid-howtos/usefulopenssl.php here].

When my Westgrid account was finally setup, I was given a key pair (two files, the certificate and the private key for that certificate, cert.pem and key.pem). Keep these safe in a place no one can access them as they are not encrypted. For encryption security and use on the Grid I converted these into a PKCS12 file (usercred.p12) on my client machine using OpenSSL (OpenSSL must be installed on the client. This is generally a distro package i.e. "apt-get install openssl"). The following command will convert your certificate and key to a PKCS12 file:

 openssl pkcs12 -export -in cert.pem -inkey key.pem -out usercred.p12

You will be prompted for an export password. This is the password you provided when you applied for or created your certificate.

In your home directory on your client create a directory called ".globus" and move your usercred.p12 file there.

mkdir -p ~/.globus
mv usercred.p12 ~/.globus/.

Now to install the OSG client software.

Installing OSG Client software

I installed the software as root, though the OSG instructions imply that this can be done for a single user. I am also using the bash shell.

Change to super user. Create a directory where the software will be installed:

su
mkdir -p /usr/local/osg
cd /usr/local/osg

Pacman Install

To install the OSG Client software, you will require the installer Pacman. I followed the instruction on the Open Science Grid Pacman Install site. Be sure to follow the instruction for OSG 1.2 only. (The Pacman install did not work with my Mandriva distro which is why I switched to the Debian machine.) "bash$" indicates the shell prompt.

bash$ wget http://atlas.bu.edu/~youssef/pacman/sample_cache/tarballs/pacman-3.28.tar.gz
bash$ tar --no-same-owner -xzvf pacman-3.28.tar.gz
bash$ cd pacman-3.28

For sh and bash shells:

bash$ source setup.sh

For csh and tcsh shells:

tcsh$ source setup.csh

Installing the OSG client

I am following the OSG 1.2 instructions from here.

bash$ cd /usr/local/osg
bash$ pacman -get http://software.grid.iu.edu/osg-1.2:client
 Do you want to add [1] to [trusted.caches]? (y/n/yall):  yall

You maybe be prompted with other questions. Follow the example here.

Now you have to install the CA certificates.

bash$ . ./setup.sh
bash$ vdt-ca-manage setupca --location local --url osg

Now, everything should have setup fine. Let's check:

bash$ source /usr/local/osg/setup.sh
bash$ vdt-version

Everything but "CA Certificates" should have an "OK" beside it. I added the "source setup.sh" line to the bash profile so I would not have to manually source it every time.

There are instructions for testing the OSG client software here: Validate Clients.

Setting up the OSG client

You will require the following environment variable:

bash$ export GLOBUS_HOSTNAME=your.ip.add.ress 

You must define your client's IP address or the grid will report an error that it doesn't know where the client is.

Because there are firewall's involved we must also define some port ranges:

bash$ export GLOBUS_TCP_PORT_RANGE=49000,49150 bash$ export GLOBUS_TCP_SOURCE_RANGE= 49000,49150

These are the current ports set for the Gluex voms.