CWL Docker Requirement

The CWL standard supports a DockerRequirement feature to execute one or more workflow steps inside a Docker container. A CWL runner must then ensure that all input files are available inside the container and choose a specific Docker runner to deploy the container. For example, the following script invokes a Node.js command inside a Docker image called node:slim:

cwlVersion: v1.2
class: CommandLineTool
baseCommand: node
requirements:
  DockerRequirement:
    dockerPull: node:slim
inputs:
  src:
    type: File
    inputBinding:
      position: 1
outputs:
  example_out:
    type: stdout
stdout: output.txt

By default, StreamFlow automatically maps a step with the DockerRequirement option onto a Docker deployment with the specified image. This mapping is pretty much equivalent to the following streamflow.yml file:

version: v1.0
workflows:
  example:
    type: cwl
    config:
      file: processfile
      settings: jobfile
    bindings:
      - step: /
        target:
          deployment: docker-example

deployments:
  docker-example:
    type: docker
    config:
      image: node:slim

StreamFlow also supports the possibility to map a CWL DockerRequirement onto different types of connectors through the CWLDockerTranslator extension point. In particular, the docker section of a workflow configuration can bind each step or subworkflow to a specific translator type, making it possible to convert a pure CWL workflow with DockerRequirement features into a hybrid workflow. The available translator types are: docker, kubernetes, none and singularity.

As an example, the following streamflow.yml file runs the above CommandLineTool using a SingularityConnector instead of a DockerConnector to spawn the container:

version: v1.0
workflows:
  example:
    type: cwl
    config:
      file: processfile
      settings: jobfile
      docker:
        - step: /
          deployment:
            type: singularity
            config: {}

In detail, StreamFlow instantiates a SingularityCWLDockerTranslator passing the content of the config field directly to the constructor. The translator is then in charge of generating a SingularityConnector instance with the specified configuration for each CWL DockerRequirement configuration in the target subworkflow.