Depalma is based on the client-server architecture.  The master server when initialized starts remote servers on all the machines that will be running jobs.  These machines are responsible for handling the local jobs.  The master server is responsible for managing these servers.

The master server is also what a user will interact with.  This can be done through the XML-RPC server it runs.


Depalma is a distributed computing package written in pure python.  It is meant for coarse parallel porcessing applications.  This is not a replacement for PVM or MPI or other parallel processing toolkits.  It is useful for running several long running processes in parallel.  For example, factoring integers, or finding eigenvales of large matrices in parallel.  Anything that requires a few minutes and you need to repeat several times is a canidate.

The master\_server module controls the cluster.

\begin{classdesc}{master\_server}{conf\_file}
Constructor for the master\_server.  This will start the XML-RPC server which will be used for controlling the cluster.
\end{classdesc}

\section{master\_server Objects}

All of these must be called over xml-rpc.  Many of these are meant to be called by remote servers, not the user controlling the cluster.  All functions are listed here for completeness. 

\begin{funcdesc}{init\_server}{}
This must be called after the master\_server is created before the cluster can be used.  This will start all the remote servers and initialize all the data structures for handling jobs which are submitted.
\end{funcdesc}

\begin{funcdesc}{callback}{host, callback\_type}
This should not be called by the user.  It is used by remote servers to alert the master server of state changes.
\end{funcdesc}

\begin{funcdesc}{get\_config}{}
This should not be called by the user.  It is used by remote servers to fetch the configuration file for the cluster.
\end{funcdesc}
\begin{funcdesc}{get\_web\_data}{}
This should not be called by the user.  It is used by the web front end to fetch the cluster state for display.
\end{funcdesc}
\begin{funcdesc}{load\_code}{code, code\_params}
Before any code can be run the method must be registered on the server.
The load_code method takes an argument, and a few configuration parameters
and distributes the name of the job and its properties across the cluster.

This does not run any code on the cluster, but alerts each server in the cluster
that this is code which could be run in the future.

The parameters passed determine how you will interact with the server, and 
what will be returned by a completed job.

code_params is a dictionary with several predefined keys.

code_params = {'io'           : True,
               'pickup'       : True,
               'error_pickup' : True,
               'buf'          : 8192
               'label'        : "calender"
              }

io           : specifies whether stdin/stdout  will be accessed when running this code.
                (optional, default: False)

pickup       : if the stdout and stderr should be returned when the process returns.
                (optional, default: False)

error_pickup : if the stdout and stderr should be returned if the process returns
               a non-zero exit code.
                (optional, default: False)

buf          : the buffer size of the communication channels.
                0 means unbuffered.
                1 or "line" is line buffering
                and other integer creates buffers of that size.
                (optional, default: 8192)

label        : a user defined identifier for the code, can be used instead 
                of job_codes below
                (optional, default: None)

Once a piece of code has been loaded on all the cluster machines, 
a job_code will be returned.

This job code can be used to start jobs on the cluster of the type initiated above.
\end{funcdesc}

\begin{funcdesc}{remove\_code}{}
\end{funcdesc}
\begin{funcdesc}{exec\_code}{}

The exec_code method takes the job_code identifier, or optionally a user defined label, and the arguments to be passed to the function.  The arguments are optional.

This will return a pid.  This is a unique identifier for the job on the cluster.  At this point the job will enter the waiting queue.

\end{funcdesc}
\begin{funcdesc}{finished}{}
\end{funcdesc}
\begin{funcdesc}{pickup}{}
\end{funcdesc}
\begin{funcdesc}{pickup\_avail}{}
\end{funcdesc}
\begin{funcdesc}{stdin}{}
\end{funcdesc}
\begin{funcdesc}{stdout}{}
\end{funcdesc}
\begin{funcdesc}{stderr}{}
\end{funcdesc}
\begin{funcdesc}{kill}{}
\end{funcdesc}
\begin{funcdesc}{kill\_9}{}
\end{funcdesc}
\begin{funcdesc}{ps}{}
\end{funcdesc}
\begin{funcdesc}{shutdown}{}
\end{funcdesc}
\begin{funcdesc}{debug}{}
\end{funcdesc}