QueueManagerConnector
The QueueManagerConnector is an abstract connector that serves as a base class to implement High Performance Computing connectors, based on queue managers (e.g., Slurm, PBS, and Flux). It extends the ConnectorWrapper interface, allowing users to offload jobs to local or remote queue managers. Plus, it extends the BatchConnector interface. The underlying HPC facility is supposed to be constantly active, reducing the deployment phase to deploy the inner connector (e.g., to create an SSHConnection pointing to an HPC login node).
Warning
Note that in StreamFlow v0.1, the QueueManagerConnector directly inherited from the SSHConnector at the implementation level. Consequently, all the properties needed to open an SSH connection to the HPC login node (e.g., hostname, username, and sshKey) were defined directly in the QueueManagerConnector. This path is still supported by StreamFlow v0.2, but it is deprecated and will be removed in StreamFlow v0.3.
properties |
||
checkHostKey |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Perform a strict validation of the host SSH keys (and return exception if key is not recognized as valid) |
|
type |
boolean |
|
default |
True |
|
dataTransferConnection |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Sometimes HPC clusters provide dedicated hostnames for large data transfers, which guarantee a higher efficiency for data movements |
|
type |
string |
|
file |
(Deprecated. Use services.) Path to a file containing a Jinja2 template, describing how the StreamFlow command should be executed in the remote environment |
|
type |
string |
|
hostname |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Hostname of the HPC facility |
|
type |
string |
|
maxConcurrentJobs |
Maximum number of jobs concurrently scheduled for execution on the Queue Manager |
|
type |
integer |
|
default |
1 |
|
maxConcurrentSessions |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Maximum number of concurrent session to open for a single SSH client connection |
|
type |
integer |
|
default |
10 |
|
maxConnections |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Maximum number of concurrent connection to open for a single SSH node |
|
type |
integer |
|
default |
1 |
|
passwordFile |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to a file containing the password to use for authentication |
|
type |
string |
|
pollingInterval |
Time interval (in seconds) between consecutive termination checks |
|
type |
integer |
|
default |
5 |
|
sshKey |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to the SSH key needed to connect with Slurm environment |
|
type |
string |
|
sshKeyPassphraseFile |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Path to a file containing the passphrase protecting the SSH key |
|
type |
string |
|
transferBufferSize |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Buffer size allocated for local and remote data transfers |
|
type |
integer |
|
default |
64kiB |
|
tunnel |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) External SSH connection parameters for tunneling |
|
type |
||
username |
(Deprecated. Use the wraps directive to wrap a standalone SSH connector.) Username needed to connect with the SSH environment |
|
type |
string |
|