WISDOM

Challenge

The WISDOM Production Environment (WISDOM PE) is a meta-middleware, developed by LPC Clermont-Ferrand, France. It aims at managing and monitoring jobs on the EGEE and OSG Grids. It provides workflow for grid job handling including automated job submission, status check and report, and error recovery. It supports several applications from the bioscience area (docking, profiling etc…) and handles millions of jobs and tasks (during the first execution 41 million, during the second 142 million docking have been solved, for example).

The WISDOM PE utilizes the pull model. It submits agent jobs to the Grid. These agent or pilot jobs can pull and execute tasks. The task is a certain unit of work to be executed. It can have application specific parameters (which application needs to be used to execute it, what are the inputs, where are the inputs available, etc.). A job stops when no more tasks are available or if it reaches the wall clock time.

The WISDOM PE has 4 main components, as it is illustrated on the figure below:

• Task Manager: The Task Manager stores the real units of work to be executed. It gets the work from the user through Web Services. Sometimes a task can be a node of a more complex Workflow.

• Job Manager: The Job manager submits the agent jobs to the worker nodes of the different Grids. If a job fails it can resubmit it. It communicates with the WISDOM Information System and decides, whether it is needed to submit more agents.

• Data Manager: Manages all data and metadata related to the tasks.

• WISDOM Information System: Tracks jobs and job statuses.

wisdom1

Implementation

The WISDOM Production Environment has completed millions of dockings in the past. The aim of the project is to reduce the computation time by submitting jobs to various Grid middleware. Thanks to the huge computation power of Desktop Grids, EDGeS could support WISDOM to extend the available resources.

A new layer, the EDGeS Submitter needed to be created between WISDOM and EDGeS to solve the problem of submitting jobs to a DG platform. This submitter communicates with the WISDOM Job and Task Managers, queries for tasks, and redistributes them to a Desktop Grid. From the WISDOM Job Manager’s point of view the Submitter behaves like a simple job, say, submitted to the EGEE. It can have several instances submitted by the Job Manager, can return its status, and can be killed. An important difference is that, when the Job Submitter submits the EDGeS Job, it first pulls a task from the task manager and submits that task as a Desktop Grid work unit.

In order to submit tasks to Desktop Grid remotely, the 3G-bridge technology has been utilized. It gives the opportunity to submit, monitor and delete DG jobs through Web Services. The figure below illustrates this above described structure for submitting jobs from WISDOM PR to a DG using the 3-G Bridge.

wisdom2

In order to submit WISDOM jobs to EDGeS, at least one of the WISDOM applications had to be ported to EDGeS. First AutoDock, the flagship application of WISDOM has been nominated. It is feasible for DG execution for several reasons:

• It uses Autodock 4 as a legacy application which can be executed in Windows
• It can be divided into smaller work packages which can be executed in 20 minutes in an average PC.
• It has small inputs (1 Protein ~ 30 MB; 1 ligand ~ 1 KB)

Moreover AutoDock4 has been already ported for DG by the EDGeS. Though WISDOM uses AutoDock4 in a different scenario with different inputs, the previous application could be reused for porting.

Acknowledgement

This application was ported to the Westminster Local Desktop Grid within the framework of the European EDGeS Project.