Submit a Job

----------------------------------------------------------------------------------------------------------

 1. Log into the User Interface

You can connect to the DZ e-Science GRID via the SSH protocol. For this purpose it is required that you have an SSH client installed.

In order to connect you need the hostname of the UI that you would like to connect to and your credentials (username, password).

To log in the User Interface with X window forwarding enabled please follow the guidelines below:

  • For Windows users we recommend using the: PuTTY, Cygwin and Xming server to enable X forwarding over ssh
  • For Linux users: Use the following command in a terminal window 
       ssh -X Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.
    where username should be replaced with your own username.

 

Transfer files to/from the User Interface

To transfer your Input files to the User Interface and vice versa you will need to use either scp/sftp protocols.

  • For Windows users we recommend using the winscp utility.
  • For Linux users: Use the scp commands from the command line (terminal)

 

  

2. How to install your certificate  

  

After you have got your certificate application approved, export the certificate as a PKCS12 file and copy this .p12 file to the grid UI. In order to use the certificate for grid submissions, it needs to be converted to a public - private key pair. First, create a directory under your home directory in the grid UI

mkdir .globus 

 Replace MyCertificate.p12 with your PKCS12 file.

openssl pkcs12 -nocerts -in MyCertificate.p12 -out .globus/userkey.pem
openssl pkcs12 -clcerts -nokeys -in MyCertificate.p12 -out .globus/usercert.pem

Set permissions right

chmod 600 .globus/usercert.pem
chmod 400 .globus/userkey.pem

3. Job Submission Procedure

Proxy creation

Before proceeding with job submission you must authenticate yourself. This is done by creating a short term proxy (valid for 12 hours). The command to create this proxy is the following one:

$ voms-proxy-init -voms <VO>

Where the <VO> should be replaced by the name of the Virtual Organization (in our case: dzgrid)

Show details about the proxy:

  $ voms-proxy-info --all

Delete the proxy 

  $ voms-proxy-destroy

To extend the default limit of 12 hours your next step is to use the myproxy service and store a renewable mid term proxy (by default valid for 1 week). The most way to do that is to use a long-term proxy provided by the following command:

  $ myproxy-init -d -n -s myproxy.grid.arn.dz

 To get information about a long-term proxy 

  $ myproxy-info -s myproxy.grid.arn.dz

To destry a long-term proxy before it expires: 

  $ myproxy-destry -s

Job Preparation

1. Setting up the Module Environment

Most applications, compilers and libraries rely on environment variables to function properly. These variables should be set properly.

Environment modules (modules) allow a user to load all the settings needed by a particular application on demand, and to unload them when they are no longer needed. Switching from one compiler to the other, between different releases of the same application, and from one MPI library to another can be done in a snap, using just one command — module.

Module commands:

module       
   avail                             list available software modules
   load <modulefile>       load available software     
   list                                list currently loaded modules
   switch <modulefile1> <modulefile2>  switch between two modules
   purge                           remove all modules

For example, if we want to use the openmpi (version 2.0.2) and the GCC (6.2) we need to issue the following commands.

module load gcc/6.3
module load openmpi
 
2. JDL File

To describe this job we need to create a JDL file that will contain the full description of the job we want to submit.For example the simple JDL file is as follow:

  1. Executable = "/bin/hostname";  
  2. Arguments = "-f";  
  3. StdOutput = "std.out";  
  4. StdError = "std.err";  
  5. OutputSandbox = {"std.err","std.out"};  

 

In the case, we have created our own executable, and we want to run a batch of system commands using a script file we need to include this file into the job submission procedure.Supposing that we have the following script named script.sh with the following content:

  1. #!/bin/bash  
  3. # Print Coordinated Universal Time  
  4. date --utc  
  5.   
  6. # Print the name of the host  
  7. hostname -f 

The JDL file will be altered as so:

  1. Executable = "script.sh";  
  2. InputSandbox = {"script.sh"};  
  3. StdOutput = "std.out";  
  4. StdError = "std.err";  
  5. OutputSandbox = {"std.err","std.out"}; 

Job Submission

To finally submit the job the following command should be used:

$ glite-wms-job-submit -a -o JobId job.jdl
 

Status check

Since the job has been submitted we can check its status by issuing the following command

$ glite-wms-job-status -i JobId
 

Job Cancellation

To cancel a submitted job one must use the following command.

$ glite-wms-job-cancel -i JobId
 

Output retrieval

When the status of the job changes to Done one can retrieve the output data (actually the files given in the OutputSandbox attribute at the time of submission) by issuing the following command:

$ glite-wms-job-output -i JobId --dir results


A new folder named results is created and within that folder a new folder containing the output files is stored.