BaseSpace Sequence Hub CLI

You can work with your BaseSpace Sequence Hub data using the command line interface (CLI). The BaseSpace Sequence Hub CLI supports scripting and programmatic access to BaseSpace Sequence Hub for automation, bulk operations, and other routine functions. It can be used independently or in conjunction with BaseMount.

The BaseSpace Sequence Hub CLI has been updated to support automation features in the latest BaseSpace Sequence Hub.

Install BaseSpace Sequence Hub CLI

The BaseSpace Sequence Hub CLI is supported on Linux, Mac, and Windows (32 and 64 bit). It carries a small memory footprint, but for safety it should be installed on a machine with at least 1GB RAM.

Install:

# Linux
$ wget "https://api.bintray.com/content/basespace/BaseSpaceCLI-EarlyAccess-BIN/latest/\$latest/amd64-linux/bs?bt_package=latest" -O $HOME/bin/bs
# Mac
$ wget "https://api.bintray.com/content/basespace/BaseSpaceCLI-EarlyAccess-BIN/latest/\$latest/amd64-osx/bs?bt_package=latest" -O $HOME/bin/bs
# Windows
$ wget  "https://api.bintray.com/content/basespace/BaseSpaceCLI-EarlyAccess-BIN/latest/\$latest/amd64-windows/bs.exe?bt_package=latest" -O bs.exe

Note that your operating system might require that you change the file permissions to make the downloaded binary executable:

$ chmod u+x $HOME/bin/bs

The copy command bscp is currently installed as a separate binary. Copy the file into a location in your $PATH with a bs- prefix to make it available via the bs cp command.

# Linux                                                              
$ wget https://bintray.com/basespace/BaseSpace-Copy-BIN/download_file?file_path=0.5.4.331%2Flinux%2Fbscp -O $HOME/bin/bs-cp                                                   
# Mac                                                                
$ wget https://bintray.com/basespace/BaseSpace-Copy-BIN/download_file?file_path=0.5.4.331%2Fosx%2Fbscp -O $HOME/bin/bs-cp                                                   
# Windows                                                            
$ wget https://bintray.com/basespace/BaseSpace-Copy-BIN/download_file?file_path=0.5.4.331%2Fwindows%2Fbscp.exe -O bs-cp.exe                                                           

Authenticate

$ bs auth

Command Structure

The BaseSpace Sequence Hub CLI is based on a pattern of a command prefix and a tree of subcommands. Just as in docker there is docker run, docker build, docker exec and in git there is git pull, git push,git commit, the the BaseSpace Sequence Hub CLI has bs list, bs create and bs authenticate. Some commands have subcommands, such as bs list appsession and bs create project.

Each subcommand has its own set of options which can be accessed with --help or -h, as in the following example

$ bs auth --help                                                   
Usage:                                                              
    bs [OPTIONS] authenticate [authenticate-OPTIONS]               

Application Options:                                                  
    -V, --version   Display version                                         

Help Options:                                                         
    -h, --help      Show this help message                                     

[authenticate command options]                                      
-c, --config= Output to this BaseSpaceCLI configuration               
--force Overwrite existing configuration file, if it already exists   
--api-server= BaseSpace API hostname (default:                         
https://api.basespace.illumina.com) [$BASESPACE_API_SERVER]      
--scopes= List of scopes to authenticate with (default: READ GLOBAL, CREATE GLOBAL)                                                        
--timeout= Time in seconds before giving up (default: 120)  

Command Summary

CLI commands can use the following patterns:

  • Standalone commands—Single commands, eg. auth. See Standalone Commands.

  • Actions on entities—Actions on entities, eg. list appsession, import dataset, get biosample. These can be used in reverse, for example create project is equivalent to project create. See Entity Reference.

  • Actions on sub-entities—Actions on sub-entities, eg export workflow threshold or set project property. See Sub-Entity Reference.

The following table summarizes the CLI functions and commands. For detailed descriptions of commands, see Command Reference.

|Function|Available Commands| |--- |--- | |List and explore entities|bs list, bs get, bs headers| |List and download files within entities|bs contents, bs download| |Kill running apps|bs kill appsession| |Delete entities|bs delete| |Create biosamples and upload fastq files for them|bs create biosample, bs upload dataset| |Create analysis and configure analysis workflows from apps|bs create workflow, bs workflow threshold ..., bs workflow dependency ...|

Some command combinations have specific options. Review the usage message for more information. For example:

$ bs list appsession --help                                          
...                                                                   
[list command options]                                              
--exec-status= Filter by execution status                             
--input-biosamples= Filter by Input BioSample                         
...                                                                    

Specify API server and Access Token

Most BaseSpace Sequence Hub CLI commands require that you specify an API server to contact and an access token to authenticate against BaseSpace Sequence Hub. These can be specified in the following ways.

  • Config—Use -c / --config to specify the config name to use. Configuration files are generated by bs auth and stored in $HOME/.basespace. The file format used here is shared with other BSSH command line tools such as BaseMount, so if you have a BaseMount config file you do not need to derive another one.
  • Command Line—Use --api-server / --access-token to specify the API server and access token on the command line. Warning: This method can store details in your command history and threaten the security of your token.
  • Environment variables—Provide access credentials by specifying the environment variables $BASESPACE_ACCESS_TOKEN and $BASESPACE_API_SERVER. These can also be loaded from a config using bs config load.

Command Reference

The following sections list the standalone commands and action commands grouped by entity, sub-entity, and action. For more information about how the commands work in practice, see the Worked Examples.

  • The Entities table lists the actions that can be applied to the entity, with some examples.
  • The Sub-Entities table lists the owning entity.
  • The Actions table lists the entities and sub-entities the action applies to, with some examples.

Standalone Commands

Authenticate

Authenticates against an instance of BaseSpace Sequence Hub, storing the API server used and the derived access token for use in future commands.

Example:

# by default, this will use the config name "default" and the API server https://api.basespace.illumina.com
$ bs auth                                                            
# authenticate against Frankfurt BaseSpace Sequence Hub instance and into config called "frankfurt"                                                         
$ bs auth -c frankfurt --api-server https://api.euc1.sh.basespace.illumina.com/   

Additional options:

  • --scopes, to specify the scopes you obtain for your token.

  • --timeout, the amount of time to wait before giving up

Whoami

Returns information about the current user and token.

Example:

$bs whoami
+----------------+-------------------------------------------------+
| Name           | BaseSpace User                                  |
| Id             | 1234567                                         |
| Email          | basespaceuser@illumina.com                      |
| DateCreated    | 2015-01-16 15:31:22 +0000 UTC                   |
| DateLastActive | 2017-06-01 12:59:24 +0000 UTC                   |
| Host           | https://api.basespace.illumina.com              |
| Scopes         | ["READ GLOBAL" "CREATE GLOBAL" "BROWSE GLOBAL"] |
+----------------+-------------------------------------------------+

Entity Reference

Entity

Description

Available Actions

Sub-entities

Other notes

Example

Application

BaseSpace app

Get
Header
List

A token with MANAGE APPLICATIONS scope is needed for some operations

$ bs list applications --category=workflow

AppResult

Output from app

Content
Delete
Get
Header
List
Property

# get metadata about appresult with ID 123
$ bs get appresult -i 123
# show files in appresult with ID 123
$ bs contents appresults -i 123

AppSession

Representing an app during or after it has executed

Delete
Get
Header
Kill
List
Property

# get all the appsessions for a particular biosample
$ bs list appsessions --input-biosample=LP1000123

Biosample

Data and metadata for a biological sample

Get
Header
Import
List
Property

See worked examples

Config

Configuration files stored in $HOME/.basespace

List Load

Not a BSSH entity, but used purely by BSSHCLI

# Show all configs
$ bs config list ... # print config in form to be imported # as environment variables $ bs config load default

Dataset

Output from app that captures typed output

Delete
Get
Header
Import
List
Property

Project

Collection of other BSSH entities

Create
Delete
Get
Header
List
Property

# create a project by name and with a description
$ bs create project -n MyProject -d "This is a new project"
# list all project in csv format
$ bs list projects -f csv

Run

Sequencing run, sometimes called a flow cell

Content
Delete
Get
Header
List
Property

# get metadata about a specific run
$ bs run get -i 123
# get all the files in a run
$ bs run contents -i 123

Workflow

A type of BSSH app that can be configured with auto-launch dependencies and QC thresholds

Create
Threshold
Dependency

See worked examples for more details

Sub-Entity Reference

Entity

Description

Owning Entity

Example

Dependency Workflow
# export in json format the dependencies for workflow with ID 123
$ bs workflow dependency export -i 123
See worked examples for more details.
Lab Requeues The lab requeues that have been requested for a biosample Biosample
Property Key/value properties on entities AppResult AppSession Biosample Dataset Project
# show all properties for project with ID 123
$ bs project properties all -i 123
# get property with name PName from dataset with ID ds.123
$ bs dataset properties get -i ds.123 --property-name "Pname"

Threshold Workflow
# export in csv format the QC thresholds for workflow with ID 123
$ bs workflow threshold export -i 123

Action Reference

Action

Description

Available Entities

Notes

Example

All Get all properties for an entity Property
# get all properties for run with ID 123
$ bs run property all -i 123

Clear Used to remove workflow settings Workflow dependency
Workflow threshold
# remove all the workflow dependencies set for workflow 123
$ bs workflow dependency clear -i 123

Content Show the contents of a BSSH entity AppResult
Dataset
Run
# show all bam files from appresult with ID 123
$ bs appresult contents -i 123 --extension=bam

Delete Move an entity to trash AppResult
AppSession
Dataset
Project
Run
Items are not deleted until the trash is emptied
# delete project with name MyProject
$ bs delete project --name MyProject
# delete dataset with ID ds.123
$ bs delete dataset --id ds.123

Export Download BSSH data AppResult
Dataset
# download a tgz file from appresult 123
$ bs appresult export --id 291294 --extension=gz -o outfile.gz

Get Application
AppResult
Biosample
Dataset
Project
Property
Run
# get metadata about project with ID 123
$ bs get project --id 123

Headers Show available headers for an entity Application
AppResult
Biosample
Dataset
Project
Run
Useful to choose outputs for `bs list`
# Show available fields for an appsession
$ bs appsession headers

Import Create a new entity within BSSH Biosample
Dataset
Used to upload fastq files with  bs import dataset
This is equivalent to `bs upload sample` in the older version of the CLI
Kill Kill a running appsession Appsession
$ bs kill appsession 123

List List all entities Application
AppResult
Biosample
Dataset
Project
Property
Run
Has options to customize output:
  • Format (-f) which can be csv, yaml, json
  • Ability to specify headers with -F
  • Count items with --count
  • Richer templated output with --template
  • # list projects in yaml format
    $ bs list projects -f yaml
    # list appsessions with custom headers selected
    $ bs list appsessions -F Id -F ExecutionStatus
    # get the number of appresults $ bs list appresults --count
    # get data sets with just their IDs $ bs list data sets --terse

    Load Allow config variables to be loaded into environment Config
    Set Set a property or other setting Property

    CLI for the new data model

    What's New

    The updated CLI features has been designed to be faster, portable, and easier to maintain. We have added additional commands to support new features released in BaseSpace 5.0. These features include:

    • Workflows—BaseSpace Sequence Hub now includes support for apps that can be launched automatically and have an automated QC step, called analysis workflows. The new CLI includes commands to create, list and modify these workflows.

    • Datasets—BaseSpace Sequence Hub apps have the ability to create datasets, a new kind of entity that stores data and metadata in way that's easier to work with and query. The new CLI includes features to list and filter datasets.

    • Biosamples—The biosample is a new entity that groups together metadata and FASTQ data for a biological sample, including attaching analysis workflows to a biosample. The new CLI allows the listing and creation of biosamples and their metadata, including the creation of analysis workflows and inspecting lab requeues.

    The new CLI includes the following additional enhancements.

    • Richer options for listing entities and customising output

    • New get command to provide more details on a specific entity

    • Support for deleting appresults, appsessions, datasets, projects and runs.

    • Ability to inspect files within BaseSpace Sequence Hub entities using the contents command

    • Support for downloading data using the export command

    • More flexible install process. The CLI can be installed by downloading a single binary, which enables you to install the CLI in an environment where you do not have sudo privileges. Using a single binary file also allows simpler distribution of updated versions.

    • Support for all Linux platforms in addition to Mac and Windows.

    What's Not Included

    The new CLI does not include the following feature from the old CLI.

    • App launching, including the importing mechanism (bs launch app and bs import app)

    How Can I Get Support for CLI?

    CLI continues to be in alpha status and support is only available through the BaseSpace Sequence Hub Google Group.

    How Can I Run Both Versions of the CLI?

    If you want to try out the new CLI but retain the old CLI (eg. for compatibility with existing scripts or because you need app launch) you can download the new CLI into a different name (eg. bs2) and run both side-by-side. As an example (Linux):

    $ wget "https://bintray.com/basespace/BaseSpaceCLI-EarlyAccess-BIN/download_file?file_path=latest%2F0.9.4.459%2Famd64-linux%2Fbs" -O bs2