Building and Running Workflows

Workflows allow apps to be run on the platform. They also allow you to create a pipeline of apps that will hand off their outputs to other apps as they complete. You can configure a complicated workflow once and when you are done the workflow will be kept so you don't have to configure it again.

Apps in a workflow will always begin executing as soon as their inputs are satisfied and if possible they will run independently. For example, if the first 4 apps in a workflow each depend on the previous step, but the 5th app can run immediately, then running the workflow will cause both the 1st and 5th apps to start processing immediately, while the 2nd-4th apps will wait for the 1st app to finish. For more details, see the Connecting input data section.

Saving of workflows

Workflows are saved as they are edited. There is no need to explicitly save a workflow.

Adding a step

To add a step add a box which says Add a step. This will bring up a set of boxes which contains apps you have installed. If the app you need isn't there, exit the dialog and click on the Apps link in the upper right of the window to browse all apps available on DNAnexus.

Adding input data

In a step you have added, the black box with the app name indicates the app run in that step. To the left of that black box are inputs that the app takes. You can assign objects to inputs by clicking on one of the input boxes. This will bring up a dialog box which gives you the option to add one or more pieces of data. This dialog box will only consider inputs that match the type required by the app. If the app requires an input to have type "Mappings" then only items in your project of type "Mappings" will be shown. This means that in some cases there may be no data available in your project which meets the proper type. In this case, you will need to either first import data of that type, or generate data of that type in an earlier step in the workflow and connect the output that matches the requirements. For information on connecting data from an earlier app in the pipeline, see the section Connecting input data.

App inputs are rendered in one of several ways:

  1. Required inputs are orange
  2. Optional inputs are grey
  3. Required inputs with a default propagated from ("linked to") another input are darker grey with a link icon
  1. Assigned inputs are gray with a gradient background

When you have satisfied all the input requirements for an app, the black app box will say configured in gray.

Note: some apps require more input than just the input boxes to the left (for example the URL Fetcher app requires you to enter a URL). For information on how to add these parameters, see the next section.

Configuring app parameters

Many apps have additional parameters that you can change. For example, the GATK UnifiedGenotyper app contains parameters to change the behavior of the app similar to using UnifiedGenotyper locally from the command line. To access these parameters, click on the black app box. This will bring up a box of parameters. In some cases, there will be a tab of advanced parameters.

Connecting input data

You can connect multiple apps together by dragging the output of one app to the input of another. Once you start to drag an output or input, valid places where that item to go will be highlighted. If a second app takes the output of a first app, the second app will automatically begin running once the first app has finished.

Connection of outputs occurs in a top-down direction. You may connect the output of a higher step to the input of a lower one, but not vice versa.

Rearranging Steps

You can rearrange the steps in a workflow by dragging steps up and down. Mouse over the blue area of a stage you would like to drag. The blue area you are over will become lighter. Click and hold to drag the step up and down. This will move other apps up or down as needed to accomodate the new position of the app you are moving

Deleting a step

To delete a step, move your mouse over the blue area for the step you would like to delete. This will cause a black to appear in the upper right corner of the app you are hovering over. Click it and chose Remove to delete the step.

Running a workflow

Once all of the inputs for all of the apps in the workflow have been satisfied (indicated by the black app box showing a gray configured instead of an orange set inputs), you can run the workflow by clicking on the green Start Analysis button in the upper right of the workflow.

Renaming workflows

Upon completion, a folder will be created with the same name as the workflow in which all data produced by the workflow will be saved. For this reason, it makes sense to name workflows intelligibly. Rename worklows while editing workflows by clicking on their name in bar the top of the page.

Editing workflows

To edit an existing workflow, click select the item by either clicking on the workflow name in the Manage tab, or select the box next to the workflow (1) and then select the Edit option (2) in the menu above. Alternatively you can right-click on the workflow and choose the edit option from the dropdown menu that appears (3)

Last edited by Thanh-Ha Nguyen, 2014-10-06 18:53:34