Helpstrings of SDK Command-Line Utilities

Below is a list of some of the various command-line utilities available in the SDK and some brief documentation for their usage.

General purpose dx utilities


usage: dx [-h] [--version] command ...

DNAnexus Command-Line Client, API v1.0.0, client v0.276.0

dx is a command-line client for interacting with the DNAnexus platform.  You
can log in, navigate, upload, organize and share your data, launch analyses,
and more.  For a quick tour of what the tool can do, see

For a breakdown of dx commands by category, run "dx help".

dx exits with exit code 3 if invalid input is provided or an invalid operation
is requested, and exit code 1 if an internal error is encountered.  The latter
usually indicate bugs in dx; please report them at

optional arguments:
  -h, --help  show this help message and exit
  --env-help  Display help message for overriding environment
  --version   show program's version number and exit


usage: dx-app-wizard [-h] [--json-file JSON_FILE] [--language LANGUAGE]
                     [--template {basic,parallelized,scatter-process-gather}]

Create a source code directory for a DNAnexus app. You will be prompted for
various metadata for the app as well as for its input and output

positional arguments:
  name                  Name of your app

optional arguments:
  -h, --help            show this help message and exit
  --json-file JSON_FILE
                        Use the metadata and IO spec found in the given file
  --language LANGUAGE   Programming language of your app
  --template {basic,parallelized,scatter-process-gather}
                        Execution pattern of your app


usage: dx-build-asset [-h] [src_dir]

Builds a DNAnexus Asset.

positional arguments:
  src_dir     Asset source directory (default: current directory)

optional arguments:
  -h, --help  show this help message and exit


usage: dx-build-report-html [-h] [-r REMOTE] [--local LOCAL] [-w WIDTH]
                            [-g HEIGHT]
                            src [src ...]

Constructs and saves/uploads an HTML report from HTML and/or linked images

positional arguments:
  src                   Source image or HTML file(s)

optional arguments:
  -h, --help            show this help message and exit
  -r REMOTE, --remote REMOTE
                        Destination route. Can be: (1) a project ID, (2) a
                        path, with or without object name (e.g.
                        /PATH/REPORT_NAME), (3) project ID + path (e.g.
  --local LOCAL         Local file to save baked HTML to
  -w WIDTH, --width WIDTH
                        Width of the final report, in pixels
  -g HEIGHT, --height HEIGHT
                        Height of the final report, in pixels


usage: dx-docker [-h] [-q] {pull,run,add-to-applet,create-asset} ...

positional arguments:
    pull                Pulls a docker image for use in DNAnexus
    run                 Runs a docker image in a container
    add-to-applet       Adds a local Docker image to an applet
    create-asset        Caches a local Docker image as an asset in the
                        DNAnexus platform (subject to change)

optional arguments:
  -h, --help            show this help message and exit
  -q, --quiet           Suppress printing pull progress to stderr


usage: dx-fetch-bundled-depends [-h]

Downloads the contents of runSpec.bundledDepends of a job running in the
execution environment.

optional arguments:
  -h, --help  show this help message and exit


usage: dx-generate-dxapp [-h] [-m TARGET_MODULE] [-f TARGET_FUNCTION]
                         [-x TARGET_EXECUTABLE] [-s SUBCOMMAND]
                         [-o OUTPUT_FILE]
                         [-p OUTPUT_PARAMS [OUTPUT_PARAMS ...]]
                         [-r OUTPUT_PARAM_REGEXP] [-n {bash,python2.7}]
                         [-i INSTANCE_TYPE] [-t TIMEOUT]
                         [--distribution DISTRIBUTION] [--release RELEASE]

optional arguments:
  -h, --help            show this help message and exit
  -m TARGET_MODULE, --target-module TARGET_MODULE
                        The fully-qualified module that contains the target
                        The main function that is called by the target
                        executable. This should bewhere the ArgumentParser is
                        The name of the executable. This is used in the
                        dxapp.json runSpec.
  -s SUBCOMMAND, --subcommand SUBCOMMAND
                        Subcommand to pass to the target method, if required.
  -o OUTPUT_FILE, --output-file OUTPUT_FILE
                        The output dxapp.json file. If not specified, output
                        will go to stdout.
                        Names of output parameters (in case they can't be
                        Regular expression that identifies output parameter
  -n {bash,python2.7}, --interpreter {bash,python2.7}
                        Type of script that will wrap the executable.
  -i INSTANCE_TYPE, --instance-type INSTANCE_TYPE
                        AWS instance type to use.
  -t TIMEOUT, --timeout TIMEOUT
                        Max runtime of this app (in hours).
  --distribution DISTRIBUTION
                        Distribution to use for the machine image.
  --release RELEASE     Distribution release to use for the machine image.


usage: dx-jobutil-add-output [-h] [--class [CLASSNAME]] [--array] name value

Reads and modifies job_output.json in your home directory to be a JSON hash
with key *name* and value  *value*.

If --class is not provided or is set to "auto", auto-detection of the output
format will occur.  In particular, it will treat it as a number, hash, or
boolean if it can be successfully parsed as such.  If it is a string which
matches the pattern for a data object ID, it will encapsulate it in a DNAnexus
link hash; otherwise it is treated as a simple string.

Use --array to append to an array of values or prepend "array:" to the --class

To use the output of another job as part of your output, use --class=jobref
(which will throw an error if it is not formatted correctly), or use the
automatic parsing which will recognize anything starting with a job ID as a
job-based object reference.  You should format the value as follows:

  Format: <job ID>:<output field name>
  Example: dx-jobutil-add-output myoutput --class=jobref \

Analysis references can be specified similarly with --class=analysisref and
formatted as:

  Format: <analysis ID>:<stage ID>.<output field name>
          <analysis ID>:<exported output field name>
  Example: dx-jobutil-add-output myoutput --class=analysisref \

positional arguments:
  name                 Name of the output field name
  value                Value of the output field

optional arguments:
  -h, --help           show this help message and exit
  --class [CLASSNAME]  Class of output for formatting purposes, e.g. "int";
                       default "auto"
  --array              Output field is an array


usage: dx-jobutil-dxlink [-h] object

Creates a DNAnexus link from an object ID or "<project ID>:<object ID>"
string. The result is of the form {"$dnanexus_link": "<object ID>"} or
{"$dnanexus_link": {"project": <project ID>, "id": <object ID>}}, as

positional arguments:
  object      Data object ID or "<Project ID>:<Data object ID>" to package
              into a DNAnexus link

optional arguments:
  -h, --help  show this help message and exit


usage: dx-jobutil-new-job [-h] [-i INPUT] [-j INPUT_JSON] [-f FILENAME]
                          [--instance-type INSTANCE_TYPE_OR_MAPPING]
                          [--instance-type-help] [--extra-args EXTRA_ARGS]
                          [--property KEY=VALUE] [--tag TAG] [--name NAME]
                          [--depends-on [JOB_OR_OBJECT_ID [JOB_OR_OBJECT_ID ...]]]

Creates a new job to run the named function with the specified input. If
successful, prints the ID of the new job.

positional arguments:
  function              Name of the function to run

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        An input to be added using "<input
                        name>[:<class>]=<input value>" (provide "class" if
                        there is no input spec; it can be any job IO class,
                        e.g. "string", "array:string", or "array"; if "class"
                        is "array" or not specified, the value will be
                        attempted to be parsed as JSON and is otherwise
                        treated as a string)
  -j INPUT_JSON, --input-json INPUT_JSON
                        The full input JSON (keys=input field names,
                        values=input field values)
  -f FILENAME, --input-json-file FILENAME
                        Load input JSON from FILENAME ("-" to use stdin)
  --instance-type INSTANCE_TYPE_OR_MAPPING
                        Specify instance type(s) for jobs this executable will
                        run; see --instance-type-help for more details
  --instance-type-help  Print help for specifying instance types
  --extra-args EXTRA_ARGS
                        Arguments (in JSON format) to pass to the underlying
                        API method, overriding the default settings
  --property KEY=VALUE  Key-value pair to add as a property; repeat as
                        necessary, e.g. "--property key1=val1 --property
  --tag TAG             Tag for the resulting execution; repeat as necessary,
                        e.g. "--tag tag1 --tag tag2"
  --name NAME           Name for the new job (default is the current job name,
                        plus ":<function>")
  --depends-on [JOB_OR_OBJECT_ID [JOB_OR_OBJECT_ID ...]]
                        Job and/or data object IDs that must finish or close
                        before the new job should be run. WARNING: For proper
                        parsing, do not use this flag directly before the
                        *function* parameter.


usage: dx-jobutil-parse-link [-h] [--no-project] dxlink

Parse a dxlink JSON hash into an object ID or project:object-id tuple

positional arguments:
  dxlink        Link to parse

optional arguments:
  -h, --help    show this help message and exit
  --no-project  Ignore project ID in an extended dxlink - just print the
                object ID


usage: dx-jobutil-report-error [-h] message [{AppInternalError,AppError}]

Creates job_error.json in your home directory, a JSON file to include the
error type and message for the running job. There are two types of errors you
may report: 1) AppError (the default) for recognized actionable errors, and 2)
AppInternalError for unexpected application errors.

positional arguments:
  message               Error message for the job
                        Error type

optional arguments:
  -h, --help            show this help message and exit


usage: dx-log-stream [-h] [-l {critical,error,warning,info,debug}]
                     [-s {DX_APP,DX_APP_STREAM}]

Redirects stdin to a DNAnexus log socket in the execution environment.

Valid logging levels:

│ --source                │ --level        │ Appears as │
│ DX_APP_STREAM (default) │ info (default) │ STDOUT     │
│ DX_APP_STREAM (default) │ error          │ STDERR     │
│ DX_APP                  │ debug          │ DEBUG      │
│ DX_APP                  │ info (default) │ INFO       │
│ DX_APP                  │ warning        │ WARNING    │
│ DX_APP                  │ error          │ ERROR      │
│ DX_APP                  │ critical       │ CRITICAL   │

optional arguments:
  -h, --help            show this help message and exit
  -l {critical,error,warning,info,debug}, --level {critical,error,warning,info,debug}
                        Logging level to use
                        Source ID to use


usage: dx-notebook-reconnect [-h] [--port PORT] job_id

Reconnect to a notebook job

positional arguments:
  job_id       Job-id of the notebook job to reconnect to.

optional arguments:
  -h, --help   show this help message and exit
  --port PORT  Local port to use for connecting.


usage: dx-print-bash-vars [-h]

Parses $HOME/job_input.json and prints the bash variables that would be
available in the execution environment.

optional arguments:
  -h, --help  show this help message and exit


Usage: dx-verify-file [options] -r <remote_file1_id> -l <local_file1> [-r <remote_file2_id> -l <local_file2> ...]

Available options:
  -h [ --help ]            Produce a help message
  --version                Print the version
  -e [ --env ]             Print environment information
  -a [ --auth-token ] arg  Specify the authentication token
  -r [ --remote-file ] arg ID of the remote file
  -l [ --local-file ] arg  Local file path
  --read-threads arg (=1)  Number of parallel disk read threads
  --md5-threads arg (=1)   Number of parallel MD5 compute threads
  -v [ --verbose ]         Verbose logging

Utilities useful in writing bash apps and applets


usage: dx-download-all-inputs [-h] [--except EXCLUDE] [--parallel]

Note: this is a utility for use by bash apps running in the DNAnexus Platform.

Downloads all files that were supplied as inputs to the app.  By convention,
if an input parameter "FOO" has value

    {"$dnanexus_link": "file-xxxx"}

and filename INPUT.TXT, then the linked file will be downloaded into the path:


If an input is an array of files, then all files will be placed into numbered
subdirectories under a parent directory named for the input. For example, if
the input key is FOO, and the inputs are {A, B, C}.vcf then, the directory
structure will be:


Zero padding is used to ensure argument order. For example, if there are 12
input files {A, B, C, D, E, F, G, H, I, J, K, L}.txt, the directory structure
will be:


This allows using shell globbing (FOO/*/*.vcf) to get all the files in the
input order.

optional arguments:
  -h, --help        show this help message and exit
  --except EXCLUDE  Do not download the input with this name. (May be used
                    multiple times.)
  --parallel        Download the files in parallel
  --sequential      Download the files sequentially


usage: dx-upload-all-outputs [-h] [--except EXCLUDE] [--parallel]
                             [--sequential] [--clearJSON CLEARJSON]

Note: this is a utility for use by bash apps running in the DNAnexus Platform.

Uploads all files and subdirectories in the directory $HOME/out, as described
below. It also adds relevant entries into the job_output.json file.

By convention, only directories with names equal to output parameter names are
expected to be found in the output directory, and any file(s) found in those
subdirectories will be uploaded as the corresponding outputs.  For example, a
file with the path


will be uploaded, and the key "FOO" will be added to the job_output.json file
with value

    {"$dnanexus_link": "file-xxxx"}

where "file-xxxx" is the ID of the newly uploaded file. If multiple files are
found, they will be added as an array output (in unspecified order). If
subdirectories are found under $HOME/out/FOO, then they are uploaded in their
entirety to the workspace, and values are added to FOO in the job_output.json
file. For example, the path:


will be uploaded to /BAR/XXX.TXT.

optional arguments:
  -h, --help            show this help message and exit
  --except EXCLUDE      Do not upload the input with this name. (May be used
                        multiple times.)
  --parallel            Upload the files in parallel
  --sequential          Upload the files sequentially
                        Clears the output JSON file prior to starting upload.
  --wait-on-close       Wait for files to close, default is not to wait

Last edited by Kurt Jensen, 2019-02-08 06:00:51