Advanced Job

----------------------------------------------------------------------------------------------------------

 1. MPI Job Submission

In this section you will find how to prepare and submit your MPI jobs to the Grid infrastructure. Submission of executables compiled with mpi to the Grid. The supported flavors of mpi on the Grid are MPICH2 and OpenMPI.

Hello World example

The code below: hello.cpp is a Hello World C++ example using MPI.

  1. #include <iostream>  
  2. #include <mpi.h>  
  3.   
  4. usingnamespace std;  
  5.   
  6. int main (int argc, char* argv[])  
  7. {  
  8.   int rank,size;  
  9.   MPI_Init( &argc, &argv );  
  10.   MPI_Comm_rank( MPI_COMM_WORLD, &rank );  
  11.   MPI_Comm_size( MPI_COMM_WORLD, &size );  
  12.   printf( "Hello world! This is process %d out of %d\n", rank, size );  
  13.   MPI_Finalize();  
  14.   return 0;  
  15. }  
  16.   
  17. </mpi.h></iostream>  

In order to submit the job place into the same directory the script.sh script for Hello World example batch job

  1. #!/bin/sh  
  2.   
  3. # Setup for mpi-start.  
  4. MY_EXECUTABLE=`pwd`/$1  
  5. MPI_FLAVOR=$2  
  6.   
  7. # Convert flavor to lowercase for passing to mpi-start.    
  8. MPI_FLAVOR_LOWER=`echo$MPI_FLAVOR | tr '[:upper:]''[:lower:]'`  
  9.   
  10. # Pull out the correct paths for the requested flavor.    
  11. evalMPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH`  
  12.   
  13. # Ensure the prefix is correctly set. Don't rely on the defaults.    
  14. eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH  
  15. export I2G_${MPI_FLAVOR}_PREFIX   
  16.   
  17. exportI2G_MPI_APPLICATION=$MY_EXECUTABLE  
  18. exportI2G_MPI_APPLICATION_ARGS=  
  19. exportI2G_MPI_TYPE=$MPI_FLAVOR_LOWER  
  20. chmod +x $I2G_MPI_APPLICATION  
  21.   
  22. # If these are set then you will get more debugging information.  
  23. exportI2G_MPI_START_VERBOSE=1  
  24. #export I2G_MPI_START_DEBUG=1  
  25.   
  26. # Invoke mpi-start.  
  27. $I2G_MPI_START  

A job.jdl template JDL file for Hello World example file is provided below

  1. Executable = "run.sh";   
  2. Arguments = "hello MPICH2";  
  3. CpuNumber = 8;   
  4. StdOutput = "std.out";   
  5. StdError = "std.err";   
  6. InputSandbox = {"run.sh", "hello"};   
  7. OutputSandbox = {"std.err","std.out"};   
  8. Requirements =   
  9.    Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment)    
  10.    && Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment); 

 

  

2. Job Collection Submission

  

A Job Collection is, as the name implies, a collection of regular Grid batch jobs. Using this gLite functionality the jobs within the Collection will be submitted simultaneously but will execute (run) independently.

As an example we will consider a simple case of two jobs comprising the collection. The first job is described through the following JDL file:

  1. Executable = "/bin/hostname";  
  2. StdOutput = "std.out";  
  3. StdError = "std.err";  
  4. OutputSandbox = {"std.err","std.out"};  

This job will execute the /bin/hostname executable on the host that it will run on and the result of this command will be printed into the StdOutput, which is set to std.out. The second job is described through the following JDL file:

  1. Executable = "/bin/date";  
  2. Arguments = "--utc";  
  3. StdOutput = "std.out";  
  4. StdError = "std.err";  
  5. OutputSandbox = {"std.err","std.out"};  

This job will execute the /bin/date executable on the host that it will run on with the argument --utc. The result of this command will be printed into the StdOutput, which is set to std.out.

Copy the contents above into two separate JDL files (name them for example job1.jdl and job2.jdl) and place them under a single folder (name the folder jdls).

In order to submit the two jobs as a Job Collection you may use the following command.

glite-wms-job-submit -a -o id.txt --collection jdls

In the above command we use the -o id.txt flag to store the job identifier in a file named id.txt. This will help us locate the status of our job later.

Also, notice that the flag --collection is used to state that this command will submit all the jdl files that are inside the jdls folder. Since the job has been submitted you may check its status by issuing the following command

glite-wms-job-status -i id.txt

With the above command one can monitor the status of all the jobs that have been submitted. When the status of the jobs changes to done one can retrieve their output by issuing the following command:

glite-wms-job-output -i id.txt --dir ./results
 

3.Parametric Job Submission