TraCeR - reconstruction of T cell receptor sequences from single-cell RNA-seq data.
This tool reconstructs the sequences of rearranged and expressed T cell receptor genes from single-cell RNA-seq data. It then uses the TCR sequences to identify cells that have the same receptor sequences and so derive from the same original clonally-expanded cell.
For more information on TraCeR, its validation and how it can be applied to investigate T cell populations during infection, see our paper in Nature Methods or the bioRxiv preprint that preceded it.
Please email questions / problems to mjt.stubbington [at] gmail.com.
Also, please note that I (Mike) created and developed TraCeR while I was at EMBL-EBI and subsequently at the Sanger Institute. Now that I have left Sanger, I will continue to maintain the code when possible as a personal project independent of any employment that I have
TraCeR is written in Python and so can just be downloaded, made executable (with chmod u+x tracer) and run or run with python tracer. Download the latest version and accompanying files from www.github.com/teichlab/tracer.
TraCeR relies on several additional tools and Python modules that you should install.
Note that TraCeR is compatible with both Python 2 and 3.
- Bowtie2 - required for alignment of reads to synthetic TCR genomes.
- Trinity - required for assembly of reads into TCR contigs. TraCeR now works with both version 1 and version 2 of Trinity. It should automatically detect the version that is installed or you can specify it in the config file.
- Please note that Trinity requires a working installation of Bowtie v1.
- IgBLAST - required for analysis of assembled contigs. (ftp://ftp.ncbi.nih.gov/blast/executables/igblast/release/).
- makeblastdb - optional but required if you want to use TraCeR's
buildmode to make your own references. - Software for quantification of TCR expression:
- Graphviz - Dot and Neato drawing programs required for visualisation of clonotype graphs. This is optional - see the
--no_networksoption tosummarise.
Downloading the executable files from ftp://ftp.ncbi.nih.gov/blast/executables/igblast/release/<version_number> is not sufficient for a working IgBlast installation. You must also download the internal_data directory (ftp://ftp.ncbi.nih.gov/blast/executables/igblast/release/internal_data) and optional_file directory (ftp://ftp.ncbi.nih.gov/blast/executables/igblast/release/optional_file/) and put them into the same directory as the igblast executable. This is also described in the igblast README file.
You should also ensure to set the $IGDATA environment variable to point to the location of the IgBlast executable. For example run export IGDATA=/<path_to_igblast>/igblast/1.4.0/bin.
To set up the python dependencies, use the requirements file:
pip install -r requirements.txt
It is highly recommended that numpy and biopython are first installed through your system's package manager or conda.
Note: Seaborn depends on the module statsmodels, which if updated through other packages may cause problems in Seaborn. If such issues arise, try to uninstall statsmodels and install again:
conda uninstall statsmodels --yes
conda install -c taugspurger statsmodels=0.8.0
The tracer module is then installed using:
python setup.py install
This will add the binary 'tracer' to your local bin folder, which can then be run from anywhere.
Note that installing TraCeR using this method requires you to specify the location of the originally downloaded files in your config file (see below).
If you would like to contribute to TraCeR, you can set up a development version with
python setup.py develop
Which will make TraCeR accessible in your python environment, and incorporate local updates to the code.
Once the prerequisites above are installed and working you're ready to tell TraCeR where to find them.
TraCeR uses a configuration file to point it to the locations of files that it needs and a couple of other options.
An example configuration file is included in the repository - tracer.conf.
TraCeR looks for the configuration file, in descending order of priority, from the following sources:
- The
-coption used at run time for any of the TraCeR modes - The environmental variable
TRACER_CONF, which can be set to point to a default location - The default global location of
~/.tracerrc - If all else fails, the provided example
tracer.confin the directory is used
Important: If you specify relative paths in the config file these will be used as relative to the main installation directory. For example, resources/Mmus/igblast_dbs will resolve to /<wherever you installed tracer>/tracer/resources/Mmus/igblast_dbs.
Tracer will look in your system's PATH for external tools. You can override this behaviour by editing your ~/.tracerrc.
Edit ~/.tracerrc (or a copy) so that the paths within the [tool_locations] section point to the executables for all of the required tools.
[tool_locations]
#paths to tools used by TraCeR for alignment, quantitation, etc
bowtie2_path = /path/to/bowtie2
bowtie2-build_path = /path/to/bowtie2-build
igblast_path = /path/to/igblastn
makeblastdb_path = /path/to/makeblastdb
kallisto_path = /path/to/kallisto
salmon_path = /path/to/salmon
trinity_path = /path/to/trinity
dot_path = /path/to/dot
neato_path = /path/to/neato
[trinity_options]
#line below specifies maximum memory for Trinity Jellyfish component. Set it appropriately for your environment.
max_jellyfish_memory = 1G
Trinity needs to know the maximum memory available to it for the Jellyfish component. Specify this here.
#uncomment the line below to explicitly specify Trinity version. Options are '1' or '2'
#trinity_version = 2
TraCeR will automatically detect the version of Trinity you have installed. You can also explicitly specify it here if you wish.
If you're running Trinity with read lengths that are shorter than 50 bases, you'll be restricted to using the Inchworm component of Trinity, which does draft contig assembly via greedy kmer extension. In your 'tracer.conf', uncomment the 'inchworm_only=True' to activate short read mode, and uncomment the 'trinity_kmer_length' as well. You can change the trinity_kmer_length value as needed; 17 is the shortest value, and the default length in Trinity is 25. Trinity v2 should be used in this mode.
#uncomment the line below if you've got a configuration file for Trinity to use a computing grid #
trinity_grid_conf = /path/to/trinity/grid.conf
Trinity can parallelise contig assembly by submitting jobs across a compute cluster. If you're running in such an environment you can specify an optional trinity config file here. See the Trinity documentation for more information.
igblast_seqtype = TCR
Type of sequence to be analysed. Since TraCeR currently only works with TCR sequences, there's no need to change this.
[base_transcriptomes]
Mmus = /path/to/kallisto/transcriptome_for_Mmus
Hsap = /path/to/kallisto/transcriptome_for_Hsap
Location of the transcriptome fasta file to which the specific TCR sequences will be appended from each cell. Can be downloaded from http://bio.math.berkeley.edu/kallisto/transcriptomes/ and many other places. This must be a plain-text fasta file so decompress it if necessary (files from the Kallisto link are gzipped).
[salmon_base_indices]
Mmus = /path/to/salmon/index_for_Mmus
Hsap = /path/to/salmon/index_for_Hsap
[kallisto_base_indices]
Mmus = /path/to/kallisto/index_for_Mmus
Hsap = /path/to/kallisto/index_for_Hsap
Location of Kallisto/Salmon indices built (exclusively) from corresponding [base_transcriptomes]. These indices are only needed when option --small_index is used in Assemble mode (see below).
[salmon_options]
libType = A
kmerLen = 31
- Description of the type of sequencing library from which the reads come (containing, e.g., the relative orientation of paired end reads). As of version 0.7.0, Salmon also has the ability to automatically infer (i.e. guess) the library type based on how the first few thousand reads map to the transcriptome. Set
libType = Afor automatic detection. - Salmon builds the quasi-mapping-based index, using an auxiliary k-mer hash over k-mers of length
kmerLen. While quasi-mapping will make used of arbitrarily long matches between the query and reference, the k size selected here will act as the minimum acceptable length for a valid match. The value forkmerLenmust be odd; its default and maximum value is 31.
See salmon documentation for more details.
[tracer_location]
#Path to where TraCeR was originally downloaded
tracer_path = /path/to/tracer
Location of the cloned TraCeR repository containing tracerlib, test_data, resources etc. Eg. /user/software/tracer.
Needed for localisation of resources and test_data if running TraCeR with the tracer binary. Note: this is the path to the directory and not the executable.
TraCeR comes with a small dataset in test_data/ (containing only TCRA or TCRB reads for a single cell) that you can use to test your installation and config file and confirm that all the prerequisites are working. Run it as:
tracer test -p <ncores> -c <config_file>
Note: The data used in the test are derived from mouse T cells so make sure that the config file points to the appropriate mouse resource files.
You can also pass the following two options to change the Graphviz output format or to prevent attempts to draw network graphs
-g/--graph_format : Output format for the clonotype networks. This is passed directly to Graphviz and so must be one of the options detailed at http://www.graphviz.org/doc/info/output.html.
--no_networks : Don't try to draw clonotype network graphs. This is useful if you don't have a working installation of Graphviz.
Running test will peform the assemble step using the small test dataset.
It will then perform summarise using the assemblies that are generated along with pre-calculated output for two other cells (in test_data/results), dumping the data in the output directory as specified.
Compare the output in test_data/results/filtered_TCR_summary with the expected results in test_data/expected_summary. There should be three cells, each with one productive alpha, one productive beta, one non-productive alpha and one non-productive beta. Cells 1 and 2 should be in a clonotype.
Tracer has three modes: assemble, summarise and build.
Assemble takes fastq files of paired-end RNA-seq reads from a single-cell and reconstructs TCR sequences.
Summarise takes a set of directories containing output from the assemble phase (each directory represents a single cell) and summarises TCR recovery rates as well as generating clonotype networks.
Build creates new combinatorial recombinomes for species other than the inbuilt Human and Mouse.
tracer assemble [options] <file_1> [<file_2>] <cell_name> <output_directory>
<file_1>: fastq file containing #1 mates from paired-end sequencing or all reads from single-end sequencing.<file_2>: fastq file containing #2 mates from paired-end sequencing. Do not use if your data are from single-end sequencing.<cell_name>: name of the cell. This is arbitrary text that will be used for all subsequent references to the cell in filenames/labels etc.<output_directory>: directory for output. Will be created if it doesn't exist. Cell-specific output will go into/<output_directory>/<cell_name>. This path should be the same for every cell that you want to summarise together.
-p/--ncores <int>: number of processor cores available. This is passed to Bowtie2, Trinity, and Kallisto or Salmon. Default=1.-c/--config_file <conf_file>: config file to use. Default =~/.tracerrc--resource_dir: the directory containing the resources required for alignment. By default this is theresourcesdirectory in this repository, but can be pointed to a user-built set of resources.-s/--species: Species from which the T cells were derived. Options areMmusorHsapfor mouse or human data. If you have defined new species using thebuildmode, you should specify the same name here. Default =Mmus.--receptor_name: If, for some reason, you've used a different receptor name when usingbuildthen you should also specify it here. Default =TCR--loci: The specific loci that you wish to assemble. TraCeR knows about alpha (A), beta (B), gamma (G) and delta (D) for mouse and human TCRs. Other locus names can be specified if you usebuild. By default, TraCeR will attempt to assemble alpha and beta sequences. To change this, pass a space-delimited list of locus names. For example, to only look for gamma/delta sequences use--loci G D. Important The way that this argument is parsed means that you must follow it with another command line option or the argument parser will get confused. For example:--loci G D --p 1. We know this isn't ideal.-r/--resume_with_existing_files: if this is set, TraCeR will look for existing output files and not re-run steps that already appear to have been completed. This saves time if TraCeR died partway through a step and you want to resume where it left off.-m/--seq_method: method by which to generate sequences for assessment of recombinant productivity. By default (-m imgt), TraCeR replaces all but the junctional sequence of each detected recombinant with the reference sequence from IMGT prior to assessing productivity of the sequence. This makes the assumption that sequence changes outside the junctional region are due to PCR/sequencing errors rather than being genuine polymorphisms. This is likely to be true for well-characterised mouse sequences but may be less so for human and other outbred populations. To determine productivity from only the assembled contig sequence for each recombinant use-m assembly.-q/--quant_method: Method used for expression quantification. Options are-q salmonand-q kallisto(default).--small_index: Use this option to speed up expression quantification. The location of an index that is built (exclusively) from the correspondingbase_transcriptomemust be specified in the configuration file (under[salmon_base_indices]or[kallisto_base_indices]). Since this index does not contain any TCR sequences, it has to be built only once for each species and can then be used for all cells. When option--small-indexis used, reads are first quantified with thisbase_index(this should lead to non-zero TPM numbers for all non-TCR transcripts present in the cell). After selecting all transcripts with non-zero TPM numbers, cell-specific TCR sequences (as constructed by Bowtie-Trinity-IgBlast) are appended to that list. Then a new Kallisto/Salmon index is built from the combined set (since this set contains only a subset of thebase_transcriptome, this is fast). Finally, reads are quantified with this small index.--single_end: use this option if your data are single-end reads. If this option is set you must specify fragment length and fragment sd as below.--fragment_length: Estimated average fragment length in the sequencing library. Used for Kallisto quantification. Required for single-end data. Can also be set for paired-end data if you don't want Kallisto to estimate it directly.--fragment_sd: Estimated standard deviation of average fragment length in the sequencing library. Used for Kallisto quantification. Required for single-end data. Can also be set for paired-end data if you don't want Kallisto to estimate it directly.--invariant_sequences: Custom invariant sequence file. Use the default example in 'resources/Mmus/invariant_seqs.csv'
For each cell, an /<output_directory>/<cell_name> directory will be created. This will contain the following subdirectories.
-
<output_directory>/<cell_name>/aligned_reads
This contains the output from Bowtie2 with the sequences of the reads that aligned to the synthetic genomes. -
<output_directory>/<cell_name>/Trinity_output
Contains fasta files for each locus where contigs could be assembled. Also two text files that log successful and unsuccessful assemblies. -
<output_directory>/<cell_name>/IgBLAST_output
Files with the output from IgBLAST for the contigs from each locus. -
<output_directory>/<cell_name>/unfiltered_TCR_seqs
Files describing the TCR sequences that were assembled prior to filtering by expression if necessary.unfiltered_TCRs.txt: text file containing TCR details. Begins with count of productive/total rearrangements detected for each locus. Then details of each detected recombinant.<cell_name>_TCRseqs.fa: fasta file containing full-length, reconstructed TCR sequences.<cell_name>.pkl: Python pickle file containing the internal representation of the cell and its recombinants as used by TraCeR. This is used in the summarisation steps.
-
<output_directory>/<cell_name>/expression_quantification
Contains Kallisto/Salmon output with expression quantification of the entire transcriptome including the reconstructed TCRs. When option--small_indexis used, this directory contains only the output of the quantification with the small index (built from reconstructed TCRs and only a subset of the base transcriptome; see above). -
<output_directory>/<cell_name>/filtered_TCR_seqs
Contains the same files as the unfiltered directory above but these recombinants have been filtered so that only the two most highly expressed from each locus are retained. This resolves biologically implausible situtations where more than two recombinants are detected for a locus. This directory contains the final output with high-confidence TCR assignments.
tracer summarise [options] <input_dir>
<input_dir>: directory containing subdirectories of each cell you want to summarise.<resource_dir>: the directory containing the resources required for alignment. By default this is theresourcesdirectory in this repository, but can be pointed to a user-built set of resources.
-c/--config_file <conf_file>: config file to use. Default =~/.tracerrc-u/--use_unfiltered: Set this flag to use unfiltered recombinants for summary and networks rather than the recombinants filtered by expression level.--resource_dir: the directory containing the resources required for alignment. By default this is theresourcesdirectory in this repository, but can be pointed to a user-built set of resources.-s/--species: Species from which the T cells were derived. Options areMmusorHsapfor mouse or human data. If you have defined new species using thebuildmode, you should specify the same name here. Default =Mmus.--receptor_name: If, for some reason, you've used a different receptor name when usingbuildthen you should also specify it here. Default =TCR--loci: The specific loci that you wish to summarise. TraCeR knows about alpha (A), beta (B), gamma (G) and delta (D) for mouse and human TCRs. Other locus names can be specified if you usebuild. By default, TraCeR will attempt to summarise alpha and beta sequences. To change this, pass a space-delimited list of locus names. For example, to only look for gamma/delta sequences use--loci G D. Important The way that this argument is parsed means that you must follow it with another command line option or the argument parser will get confused. For example:--loci G D --p 1. We know this isn't ideal. Default =A B-i/--keep_invariant: TraCeR attempts to identify invariant TCR cells by their characteristic TCRA gene segments. By default, these are removed before creation of clonotype networks. Setting this option retains the invariant cells in all stages.-g/--graph_format: Output format for the clonotype networks. This is passed directly to Graphviz and so must be one of the options detailed at http://www.graphviz.org/doc/info/output.html.--no_networks: Don't try to draw clonotype network graphs. This is useful if you don't have a working installation of Graphviz.
Output is written to <input_dir>/filtered_TCR_summary or <input_dir>/unfiltered_TCR_summary depending on whether the --use_unfiltered option was set.
The following output files are generated:
TCR_summary.txtSummary statistics describing successful TCR reconstruction rates and the numbers of cells with 0, 1, 2 or more recombinants for each locus.recombinants.txtList of TCR identifiers, lengths, productivities, and CDR3 sequences (nucleotide and amino acid) for each cell. Note: It's possible for non-productive rearrangements to still have a detectable CDR3 if a frameshift introduces a stop-codon after this region. In these cases, the CDR3 nucleotides and amino acids are still reported but are shown in lower-case.reconstructed_lengths_TCR[A|B].pdfandreconstructed_lengths_TCR[A|B].txtDistribution plots (and text files with underlying data) showing the lengths of the VDJ regions from assembled TCR contigs. Longer contigs give higher-confidence segment assignments. Text files are only generated if at least one TCR is found for a locus. Plots are only generated if at least two TCRs are found for a locus.clonotype_sizes.pdfandclonotype_sizes.txtDistribution of clonotype sizes as bar graph and text file.clonotype_network_[with|without]_identifiers.<graph_format>graphical representation of clonotype networks either with full recombinant identifiers or just lines indicating presence/absence of recombinants.clonotype_network_[with|without]_identifiers.dotfiles describing the clonotype networks in the Graphviz DOT language
tracer build <species> <receptor_name> <locus_name> <N_padding> <colour> <V_seqs> <J_seqs> <C_seqs> <D_seqs>
<species>: Species (e.g. Mmus).<receptor_name>: Name of receptor (e.g. TCR).<locus_name>: Name of locus (e.g. A)<N_padding>: Number of ambiguous N nucleotides between V and J<colour>: Colour for the productive recombinants (optional). Specify as HTML (e.g. E41A1C) or use "random"<V_seqs>: Fasta file containing V gene sequences<J_seqs>: Fasta file containing J gene sequences<C_seqs>: Fasta file containing single constant region sequence<D_seqs>: Fasta file containing D gene sequences (optional)
-f/--force_overwrite: Force overwrite of existing resources-o/--output_dir: Optional directory to write new resource files to. If not specified, the built resources will be placed in the default TraCeR resource directory.
TraCeR is also available as a standalone Docker image on DockerHub, with all of its dependencies installed and configured appropriately. Running TraCeR from the image is very similar to running it from a normal installation. You can pass all the appropriate arguments to the Docker commandwith the usual syntax as described above. Two differences from installing it yourself are that:
--small_indexis not supported- You don't need to worry about specifying a configuration file. This is included in the container.
To run the TraCeR Docker image, run the following command from within a directory that contains your input data:
docker run -it --rm -v $PWD:/scratch -w /scratch teichlab/tracer
followed by any arguments that you want to pass to TraCeR.
The -it flag ensures that you see all the information TraCeR prints to the screen during its run, while --rm deletes the individual container created based on the image for the purpose of the run once the analysis is complete, not littering your computer's drive. -v creates a volume, allowing the created container to see the contents of your current working directory, and the -w flag sets the container's working directory to the newly created volume.
For example, if you wanted to run the test analysis, you should clone this GitHub repository, navigate to its main directory so you can see the test_data folder, and call the following (you need to specify the -o test_data so that the results get written to the volume you created, ensuring you can see them after the analysis is finished):
docker run -it --rm -v $PWD:/scratch -w /scratch teichlab/tracer test -o test_data
If you wish to use tracer build, you will need to specify --output_dir /scratch, as otherwise the resulting resources will be saved in the default location of the container and subsequently get forgotten about when the build analysis completes, making them unuseable for any actual analyses you may want to perform. This will make the Docker container save the resulting resources in the volume you created, and you can use them for assemble/summarise by running the Dockerised TraCeR from the same directory as the one you used for the build and specifying --resource_dir /scratch.
You may need to explicitly tell Docker to increase the memory that it can use. Instructions for Windows and Mac. Something like 6 or 8 GB is likely to be ok.