PBSConnector
The PBS connector allows offloading execution to High-Performance Computing (HPC) facilities orchestrated by the PBS queue manager. It extends the QueueManagerConnector, which inherits from the ConnectorWrapper interface, allowing users to offload jobs to local or remote PBS controllers using the stacked locations mechanism. The HPC facility is supposed to be constantly active, reducing the deployment phase to deploy the inner connector (e.g., to create an SSHConnection pointing to an HPC login node).
Warning
Note that in StreamFlow v0.1, the QueueManagerConnector directly inherited from the SSHConnector at the implementation level. Consequently, all the properties needed to open an SSH connection to the HPC login node (e.g., hostname, username, and sshKey) were defined directly in the QueueManagerConnector. This path is still supported by StreamFlow v0.2, but it is deprecated and will be removed in StreamFlow v0.3.
Interaction with the PBS scheduler happens through a Bash script with #PBS directives. Users can pass the path of a custom script to the connector using the file attribute of the PBSService configuration. This file is interpreted as a Jinja2 template and populated at runtime by the connector. Alternatively, users can pass PBS options directly from YAML using the other options of a PBSService object.
As an example, suppose to have a PBS template script called qsub.sh, with the following content:
#!/bin/bash
#PBS -l nodes=1
#PBS -q queue_name
#PBS -l mem=1gb
{{streamflow_command}}
A PBS deployment configuration which uses the qsub.sh file to spawn jobs can be written as follows:
deployments:
pbs-example:
type: pbs
config:
services:
example:
file: qsub.sh
Alternatively, the same behaviour can be recreated by directly passing options through the YAML configuration, as follows:
deployments:
pbs-example:
type: pbs
config:
services:
example:
destination: queue_name
resources:
mem: 1gb
nodes: 1
Being passed directly to the qsub command line, the YAML options have higher priority than the file-based ones.
Warning
Note that the file property in the upper configuration level, i.e., outside a service definition, is still supported in StreamFlow v0.2, but it is deprecated and will be removed in StreamFlow v0.3.
The unit of binding is the entire HPC facility. In contrast, the scheduling unit is a single job placement in the PBS queue. Users can limit the maximum number of concurrently placed jobs by setting the maxConcurrentJobs parameter.
properties |
||
checkHostKey |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Perform a strict validation of the host SSH keys (and return exception if key is not recognized as valid) |
|
type |
boolean |
|
default |
True |
|
dataTransferConnection |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Sometimes HPC clusters provide dedicated hostnames for large data transfers, which guarantee a higher efficiency for data movements |
|
type |
string |
|
file |
(Deprecated. Use services.) Path to a file containing a Jinja2 template, describing how the StreamFlow command should be executed in the remote environment |
|
type |
string |
|
hostname |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Hostname of the HPC facility |
|
type |
string |
|
maxConcurrentJobs |
Maximum number of jobs concurrently scheduled for execution on the Queue Manager |
|
type |
integer |
|
default |
1 |
|
maxConcurrentSessions |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Maximum number of concurrent session to open for a single SSH client connection |
|
type |
integer |
|
default |
10 |
|
maxConnections |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Maximum number of concurrent connection to open for a single SSH node |
|
type |
integer |
|
default |
1 |
|
passwordFile |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to a file containing the password to use for authentication |
|
type |
string |
|
pollingInterval |
Time interval (in seconds) between consecutive termination checks |
|
type |
integer |
|
default |
5 |
|
services |
Map containing named configurations of PBS submissions. Parameters can be either specified as #BSUB directives in a file or directly in YAML format. |
|
type |
object |
|
patternProperties |
||
^[a-z][a-zA-Z0-9._-]*$ |
||
sshKey |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to the SSH key needed to connect with Slurm environment |
|
type |
string |
|
sshKeyPassphraseFile |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to a file containing the passphrase protecting the SSH key |
|
type |
string |
|
transferBufferSize |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Buffer size allocated for local and remote data transfers |
|
type |
integer |
|
default |
64kiB |
|
tunnel |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) External SSH connection parameters for tunneling |
|
type |
||
username |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Username needed to connect with the SSH environment |
|
type |
string |
|
PBSService
This complex type represents a submission to the PBS queue manager.
properties |
||
account |
Defines the account string associated with the job |
|
type |
string |
|
additionalAttributes |
Specify additional job attributes |
|
type |
object |
|
begin |
Declares the time after which the job is eligible for execution |
|
type |
string |
|
checkpoint |
Defines the interval at which the job will be checkpointed |
|
type |
string |
|
destination |
Defines the destination of the job. The destination names a queue, a server, or a queue at a server |
|
type |
string |
|
exportAllVariables |
Declares that all environment variables in the qsub command’s environment are to be exported to the batch job |
|
type |
boolean |
|
file |
Path to a file containing a Jinja2 template, describing how the StreamFlow command should be executed in the remote environment |
|
type |
string |
|
jobName |
Declares a name for the job. The name specified may be up to and including 15 characters in length. It must consist of printable characters with the first character alphabetic |
|
type |
string |
|
mailOptions |
Defines the set of conditions under which the execution server will send a mail message about the job |
|
type |
string |
|
prefix |
Defines the prefix that declares a directive to the qsub command within the script file |
|
type |
string |
|
priority |
Defines the priority of the job. The priority argument must be a integer between -1024 and +1023 inclusive |
|
type |
integer |
|
rerunnable |
Declares whether the job is rerunable |
|
type |
boolean |
|
resources |
Defines the resources that are required by the job and establishes a limit to the amount of resource that can be consumed |
|
type |
object |
|
shellList |
Declares the shell that interprets the job script |
|
type |
string |
|
userList |
Defines the user name under which the job is to run on the execution system |
|
type |
string |
|
variableList |
Expands the list of environment variables that are exported to the job |
|
type |
string |
|