/dx-toolkit

DNAnexus platform client libraries

Primary LanguagePythonApache License 2.0Apache-2.0

DNAnexus Platform SDK

Welcome to the dx-toolkit repository! This repository contains the DNAnexus API language bindings and utilities for interacting with the DNAnexus platform.

See https://documentation.dnanexus.com/ and http://autodoc.dnanexus.com/ for relevant documentation.

Installing the toolkit from source

First, see the section "Build dependencies" below and install the appropriate dependencies for your platform.

To build the toolkit, simply run make.

Then, initialize your environment by sourcing this file:

source dx-toolkit/environment

You will then be able to use dx (the DNAnexus Command Line Client) and other utilities, and you will be able to use DNAnexus API bindings in the supported languages.

Building inside docker

To avoid lengthy installation of dependencies on OS X and simultaneous installations of development versions of dx-toolkit on the system, you can build your dev-version of dx-toolkit in a separate docker container.

  1. Start python:3.9-bullseye in the interactive mode, mounting the repo you are working on (<local_path_to_repo>/dx-toolkit):

    # from root folder of dx-tollkit
docker run -v `pwd`:/dx-toolkit -w /dx-toolkit -it --rm --entrypoint=/bin/bash python:3.9-bullseye

  2. From the interactive shell install dx-toolkit.

    • A. Use local source: 
      python -m pip install src/python/ --upgrade
    • B. Use remote source: 
      python3 -m pip install --upgrade 'git+https://github.com/dnanexus/dx-toolkit.git@master#egg=dxpy&subdirectory=src/python'

  3. Log in, install dependencies(if needed) and use the container while developing. To rebuild, just save the work and run the step 2 again.

Supported languages

The Platform SDK contains API language bindings for the following platforms:

  • Python (requires Python 2.7)
  • C++
  • Java (requires Java 7 or higher)
  • R

Javascript support lives in a separate repo, dnanexus/dx-javascript-toolkit.git.

Build dependencies

The following packages are required to build the toolkit. You can avoid having to install them by either downloading a compiled release from https://documentation.dnanexus.com/downloads, or by building only a portion of the toolkit that doesn't require them.

Note: There is a known incompatibility (in compiling dxcpp) when using GCC 4.7 with Boost 1.49. Please either use the GCC 4.6 series, or Boost 1.50+.

Ubuntu 20.04

sudo apt install python-is-python3 python3-venv libssl-dev flex bison libffi-dev

Ubuntu 16.04

sudo apt install make python-setuptools python-pip python-virtualenv python-dev \
  gcc g++ cmake libboost-all-dev libcurl4-openssl-dev zlib1g-dev libbz2-dev flex bison \
  openssl libssl-dev autoconf git curl

If your locale is not configured properly, you might need to run following commands as well (i.e. when working from Cloud Workstation):

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"
sudo dpkg-reconfigure locales

in the dpkg-reconfigure command, select en_US.UTF-8 locale.

Ubuntu 14.04

sudo apt-get install make python-setuptools python-pip python-virtualenv python-dev \
  g++ cmake libboost1.55-all-dev libcurl4-openssl-dev zlib1g-dev libbz2-dev flex bison \
  autoconf curl

Ubuntu 12.04 (deprecated)

sudo apt-get install make python-setuptools python-pip python-dev \
  g++ cmake libboost1.48-all-dev libcurl4-openssl-dev zlib1g-dev libbz2-dev flex bison \
  autoconf curl
sudo pip install --upgrade virtualenv

Fedora

yum install gcc gcc-c++ automake bison flex python python-pip \
  python-virtualenv boost-devel boost-static cmake openssl-devel \
  libcurl-devel bzip2-devel curl

This package set was tested on Fedora 20, which has the following package versions (abbreviated list):

  • gcc 4.8.2
  • Python 2.7.5
  • python-pip 1.4.1
  • python-virtualenv 1.10.1
  • boost 1.54.0
  • cmake 2.8.12.1
  • openssl 1.0.1e
  • libcurl 7.32.0

CentOS/RHEL 5.x/6.x

Install Python 2.7. Python 2.7 is not available natively on CentOS/RHEL 5 or 6. You can use the script build/centos_install_python27.sh, which installs it into /usr/local/bin. (Run the script as root.)

Install boost 1.48 or higher (at least the thread and regex libraries). This version of boost is not available natively on CentOS/RHEL 5 or 6. You can use the script build/centos_install_boost.sh, which installs it into /usr/local/lib.

Then:

yum install cmake libcurl-devel
easy_install-2.7 pip
pip-2.7 install virtualenv

Notes:

  • Tested on CentOS 5.4 and CentOS 6.2.

OS X

Install the Command Line Tools for XCode. (Free registration required with Apple)

Install pip and virtualenv for Python:

easy_install-2.7 pip
pip-2.7 install virtualenv

Install the following packages from source or via Homebrew, Fink, or MacPorts:

  • CMake (sudo port install cmake or brew install cmake)
  • Boost >= 1.49 (sudo port install boost or brew install boost)
  • GCC >= 4.6

    • On MacPorts, install and select GCC with:

      sudo port install gcc47
sudo port select --set gcc mp-gcc47

    • On Homebrew, install and select an up-to-date version of GCC with:

      brew tap homebrew/versions
brew install gcc47
export CC=gcc-4.7
export CXX=g++-4.7
  • bison >= 2.7, autoconf, automake
    • On Homebrew: brew install bison autoconf automake
    • On MacPorts: sudo port install bison autoconf automake

Windows

Warning: Not all parts of the SDK are compatible with Windows. Install the following dependencies to build the Upload Agent:

To generate the .dll dependencies required for Windows, run make ua, then make pynsist_installer, it is also possible to pass DLL_DEPS_FOLDER=C:/folder/path/ as an argument to make

Upload Agent

See the Upload Agent Readme for Upload Agent build documentation.

Reporting Bugs

Please use GitHub to report bugs, post suggestions, or send us pull requests.