Building and Running Workflows

Workflows allow apps to be run on the platform. They also allow you to create a pipeline of apps that will hand off their outputs to other apps as they complete. You can configure a complicated workflow once and when you are done the workflow will be kept so you don't have to configure it again.

Apps in a workflow will always begin executing as soon as their inputs are satisfied and if possible they will run independently. For example, if the first 4 apps in a workflow each depend on the previous step, but the 5th app can run immediately, then running the workflow will cause both the 1st and 5th apps to start processing immediately, while the 2nd-4th apps will wait for the 1st app to finish. For more details, see the Connecting input data section.

Saving of workflows

Workflows are saved as they are edited. There is no need to explicitly save a workflow.

Adding a step

To add a step, click the Add a Step button. This will bring up the list of available apps and applets.

Adding input data

In a step you have added, the black box with the app name indicates the app run in that step. To the left of that black box are inputs that the app takes. You can assign objects to inputs by clicking on one of the input boxes. This will bring up a dialog box which gives you the option to add one or more pieces of data. This dialog box will only consider inputs that match the type required by the app. If the app requires an input to have type "Mappings" then only items in your project of type "Mappings" will be shown. This means that in some cases there may be no data available in your project which meets the proper type. In this case, you will need to either first import data of that type, or generate data of that type in an earlier step in the workflow and connect the output that matches the requirements. For information on connecting data from an earlier app in the pipeline, see the section Connecting input data.

App inputs are rendered in one of several ways:

  1. Required inputs are orange
  2. Optional inputs are grey
  3. Required inputs with a default propagated from ("linked to") another input are darker grey with a link icon
  1. Assigned inputs are grey with a gradient background

When you have satisfied all of the input requirements for an app, the text below the black app box will say runnable in green.

Note: some apps require more input than just the input boxes to the left (for example the URL Fetcher app requires you to enter a URL). For information on how to add these parameters, see the next section.

Configuring app parameters

Many apps have additional parameters that you can change. For example, the GATK UnifiedGenotyper app contains parameters to change the behavior of the app similar to using UnifiedGenotyper locally from the command line. To access these parameters, click on the black app box. This will bring up a box of parameters. In some cases, there will be a tab of advanced parameters.

Connecting input data

You can connect multiple apps together by dragging the output of one app to the input of another. Once you start to drag an output or input, valid places where that item to go will be highlighted. If a second app takes the output of a first app, the second app will automatically begin running once the first app has finished.

Connection of outputs occurs in a top-down direction. You may connect the output of a higher step to the input of a lower one, but not vice versa.

Rearranging Steps

You can rearrange the steps in a workflow by dragging steps up and down. Mouse over the background of a stage you would like to drag. Click and hold to drag the step up or down. This will move other apps up or down as needed to accomodate the new position of the app you are moving

Deleting a step

To delete a step, move your mouse over the blue area for the step you would like to delete. This will cause an Actions context menu to appear in the upper right corner. Click it and chose Remove to delete the step.

Running a workflow

Once all of the inputs for all of the apps in the workflow have been satisfied (indicated by the black app box showing a green runnable label instead of orange set inputs), you can run the workflow by clicking on the green Start Analysis button in the upper right of the workflow.

Renaming workflows

Upon completion, a folder will be created with the same name as the workflow in which all data produced by the workflow will be saved. For this reason, it makes sense to name workflows intelligibly. Rename a workflow while in edit mode by clicking on its title at the top of the page.

Editing workflows

To edit an existing workflow, click the checkbox next to the workflow (1) and then click the Edit icon on the right of the toolbar (2). Alternatively, you can click on the vertical ellipsis affordance on the far right and choose the edit option from the dropdown menu (3).

Last edited by pmurray, 2017-06-28 21:16:52

 Feedback