DIRAC Software Installation

Adding a Software Installation Module

Introduction

One of the nice features of doing Workload Management with Pilots is that there are a number of functionalities that are required for the execution of your payloads that can be taken care of by the pilot. One of them is the installation of required software packages and definition of the proper environment to make use of this software.

DIRAC allows a user community to define specialized modules that take care of this functionality. They can be used for:

• Deploying software on some "shared" directory at the site
• Discovering already installed software
• Deploying software in the working directory of the pilot
• Preparing the environment for the execution of a given application using the installed software

The SoftwareInstallation module

The SoftwareInstallation module contains all the knowledge about the installation and usage of the software for the execution of payloads for a community of users using a particular DIRAC extension. At the current moment it does not exist a general SoftwareInstallation module in DIRAC. Keeping this knowledge in a single place helps to keep consistency among deployment, discovery and usage of the supported software.

This module will be used by the JobAgent, after matching a new payload and before launching is execution, to check the availability of the software for the execution of the payload. The description of the payload (the JDL) should include one option, SoftwareDistModule, that indicates the import path of the module to be installed. For instance:

SoftwareDistModule = "CTADIRAC.Core.Utilities.SoftwareInstallation";

This module should include a python class definition with the same name that should accept in its __init__ method a single python dictionary:

import DIRAC

class SoftwareInstallation:

  def __init__( self, argumentsDict ):
    """ Standard constructor
    """
    self.job = argumentsDict.get( 'Job', {} )
    self.ce = argumentsDict.get( 'CE', {} )

  def execute( self ):
    """
      Check if requested packages are available
      otherwise install them locally
    """
    pass

And instance of this class will be instantiated by the JobAgent with a dictionary of dictionaries including the description of the matched Job in a 'Job' dictionary and the description of the executing resource in a 'CE' dictionary. For other uses of this class, additional items can be added to this dictionary.

The __init__ method should:

• extract from 'Job' dictionary the requested software. For instance SoftwarePackages. The name is a convention within a given DIRAC extension.
• extract from 'CE' dictionary the Platform and the possible list of CompatiblePlatforms.
• determine the different areas for software installation (for instance if there is some shared area defined by some environmental variable: VO_VO_CTA_IN2P3_FR_SW_DIR; or some DIRAC configuration option: /LocalSite/SharedArea ).

The execute method accepts no argument and should provide the following functionality:

• check if there is any software required.
• check if the required software is already available in the known software areas.
• install in the local Area of the pilot the missing software.

It should return DIRAC.S_OK() if all required software is found or could be installed, and otherwise DIRAC.S_ERROR( errorMessage ).

Additional methods can be defined to allow payload modules to set the environment to use each particular software or for the software manager to do the installation in shared software areas.