Welcome to DrivR-Base! This repository contains scripts for extracting feature information from different databases for single nucleotide variants (SNVs). These features are designed to be inputs for machine learning models, aiding in the prediction of functional impacts of genetic variants in human genome sequencing. The repository is organised into separate sub-directories for different feature groups (FG_), each serving a unique purpose. Please note that these scripts are only set up to analyse autosomal variants in the GRCh38 genome format.
- Introduction and Overview
- Table of Contents
- Feature Descriptions and Sources
- Setting up Docker and Git
- Option 1: Building the DrivR-Base Docker Image from the Dockerfile
- Option 2: Pulling the DrivR-Base Docker Image
- Interacting with the Docker Container
- Testing the Docker Container
- Running DrivR-Base without Docker
- Data Input Structure
- Dependencies
- Package Installation
- R Environment Setup with renv
- Python Environment Setup with Anaconda
- Directory Structure
- Feature Description & Usage
- Contributing
- License
- Acknowledgments
- Contact
- References
Feature Group Label | Feature Group Description | Source |
---|---|---|
FG1 | 20 sequence conservation and uniqueness features from UCSC genome browser, including PhyloP and PhastCons scores | UCSC |
FG2 | 3 Variant Effect Predictor features, including amino acid prediction, consequences, and distances to transcripts | VEP Cache |
FG3 | 125 dinucleotide properties for wild-type and mutant nucleotide sequences | DiProGB |
FG4 | 5 DNA shape properties, including electrostatic potential and minor groove width | DNAshapeR |
FG5 | GC-content and CpG counts for various window sizes | Self-calculated |
FG6 | Kernel-based DNA sequence similarity scores for wild-type and mutant sequences | Self-calculated |
FG7 | scores for 13 different amino acid substitution matrices | Bio2mds |
FG8 | 533 amino acid properties for both wild-type and mutant amino acids | AAindex |
FG9 | Results for 10 different ENCODE assays, including transcription factor binding sites and histone modifications | ENCODE |
FG10 | AlphaFold structural conformation and atom properties at the predicted amino acid site | AlphaFold |
If you haven't already got Docker Desktop installed locally, then download it and create a Docker account. The drivrbase docker file has been adapted from the VEP docker file [1].
Ensure that git is also installed
git --version
If the above command does not return a git version, then please install git.
Clone the DrivR-Base GitHub repository, which also contains the Dockerfile.
git clone https://github.com/amyfrancis97/DrivR-Base.git
cd DrivR-Base
Once inside the DrivR-Base directory, build the Docker image from the Dockerfile:
docker build -t drivrbase .
Once you have Docker Desktop installed locally, and an account set up, then pull the Docker image using the following commands in your terminal:
docker login
docker pull amyfrancis2409/drivrbase:latest # This pulls the Docker image
docker tag amyfrancis2409/drivrbase:latest drivrbase # This renames the image
Once the Docker image has been pulled or built, you can interact with the data and scripts using the docker run command: Note: Please increase the CPU, memory, and disk limit in your Docker desktop preferences before interacting with the container.
docker run -it --name drivrbase-container drivrbase /bin/bash
The above command will take a while since it will automatically download the variant effect predictor GRCh38 cache. Please note that if you run into "disk full" errors at this stage, you may be required to increase your disk image sizes in you Docker Desktop preferences. In case the VEP cache fails to download for any reason, you can manually install the file once inside the interactive Docker container by running:
perl /opt/vep/src/ensembl-vep/INSTALL.pl -a cf -s homo_sapiens -y GRCh38 --CACHEDIR /data
Once interactive mode is running, initialise the conda environment:
conda init
Exit the interactive environment:
exit
Adjust the permissions of the files in the DrivR-Base directory:
find . -type d -exec chmod 777 {} +
find . -type f -exec chmod 777 {} +
Copy the contents of the repository into the container (this ensures that the scripts inside the container are up-to-date):
docker cp . drivrbase-container:/data
Start & run the interactive environment:
docker start drivrbase-container
docker exec -it drivrbase-container /bin/bash
Activate the conda environment:
conda activate DrivR-Base
To test the scripts, run with the example variants in the /example directory:
chmod +x run_scripts.sh
./run_scripts.sh
Upon execution, this will create a sub-directory named /features within the /example directory, and will output a file "all_features.bed" that will contain the results for the DrivR-Base example variants for all of the features.
The ENCODE features are excluded automatically, since the scripts take a lot of time to run. If you want to include these features in the output file, please specify this when executing the run_scripts.sh script:
chmod +x run_scripts.sh
./run_scripts.sh --run_encode true
Upon execution, the "run_scripts.sh" script should output a file named all_featured.bed which will be located in the "/example/features" directory. The rows of the files contain the variants, and the columns contain all of the feature values for 1704 features. For a full list of features and their descriptions, please refer to our DrivR-Base paper and supplementary material.
Replace the example "variants_with_driver_stat.bed" file with your own variants, ensuring that the name is kept the same. The script will automatically generate the "variants.bed" file from this file and execute the rest of the scripts to generate the features for you variant set.
The "variants_with_driver_stat.bed" should be in tab delimited format, and the columns should reflect those in the example file, where the driver status is either "0" or "1":
Chromosome | Position | Reference Allele | Alternate Allele | Driver Status |
---|---|---|---|---|
chr1 | 934881 | A | G | 1 |
If the driver status is not relevant to your variants, please fill this column with either "0" or "1".
Once you've replaced this file with your own variants, simply execute the "run_scripts.sh" as before:
chmod +x run_scripts.sh
./run_scripts.sh --run_encode true
Due to the complexities of the environments and the packages used in DrivR-Base, we recommend that you use the above Docker containers for running the scripts. However, If you'd like to use the scripts independently then please refer to the .README files in the relevant sub-directories for the features you'd like to extract.
All variant files must be presented in the BED format shown in the following table but should not contain any headers:
Chromosome | Start | End | Reference Allele | Alternate Allele |
---|---|---|---|---|
chr1 | 934881 | 934881 | A | G |
Importantly, the chromosomal position must exist in a string format with a prefix of "chr", and the positions must be in an integer format. Crucially, the chromosomal position must be in the GRCh38 reference genome format. All features are extracted and queried using these descriptors, if the variant is provided in the wrong reference genome, the feature information will be incorrect. An example variant file in the correct format can be found in the /example directory.
The scripts in this repository require a range of dependencies.
To install the required packages, use the appropriate package manager for your system:
-
On Ubuntu/Debian:
sudo apt-get install bedtools samtools bcftools tabix libhts-dev
-
On CentOS/RHEL:
sudo yum install bedtools samtools bcftools tabix htslib-devel
-
On macOS (using Homebrew):
brew install bedtools samtools bcftools tabix htslib
If Anaconda is not already installed on your system, you can download and install Anaconda from the official Anaconda website: https://www.anaconda.com/products/distribution
These commands will install all the necessary packages for your project. Make sure you have the appropriate package manager installed on your system.
If you haven't already, make sure you have Git installed on your system. You can download and install Git from the official website: https://git-scm.com/
To install the required Perl modules, follow these steps:
-
Install cpanminus (App::cpanminus):
curl -L https://cpanmin.us | perl - App::cpanminus
-
Set up local::lib:
cpanm --local-lib=~/perl5 local::lib eval $(perl -I ~/perl5/lib/perl5/ -Mlocal::lib)
-
Install the Perl modules:
cpanm Archive::Zip cpanm DBD::mysql cpanm DBI
These commands will install the required Perl modules for your project.
The R environment for this project has been set up using renv. The relevant packages and versions can be found in the renv.lock
file.
To activate the renv
environment before executing any R script, use the following command in the command prompt:
RENVCMD="renv::activate(\"$renv_dir/renv.lock\")"
Where $renv_dir is the directory in which the renv.lock file is located.
To set up the Anaconda environment for this project, follow these steps:
-
Once Anaconda is installed on your system, navigate to the /DrivR-Base directory in your project.
-
Run the conda_setup.sh script to build the Conda environment using the DrivR-Base.yml file:
./conda_setup.sh
After running the script, the Conda environment named "DrivR-Base" will be created, and the required packages will be installed. You can activate the Conda environment anytime you work on your project using:
conda activate DrivR-Base
DrivR-Base/
|-- FG1_conservation/
| |-- get_conservation.sh
| |-- check_formatting.py
| |-- config.sh
| |-- README.md
|
|-- FG2_vep/
| |-- get_vep.sh
| |-- reformat_vep_aa.py
| |-- reformat_vep_conseq.py
| |-- reformat_vep_distance.py
| |-- config.sh
| |-- README.md
|
|-- FG3_dinucleotide_properties/
| |-- dinucleotide_properties.R
| |-- dinucleotidePropertyTable.csv
| |-- config.R
| |-- README.md
|
|-- FG4_dna_shape/
| |-- dna_shape.R
| |-- config.R
| |-- README.md
|
|-- FG5_gc_CpG/
| |-- get_gc_CpG.py
| |-- config.py
| |-- README.md
|
|-- FG6_kernel/
| |-- get_kernel.py
| |-- config.py
| |-- README.md
|
|-- FG7_aa_substitution_matrices/
| |-- aa_subs_matrices.R
| |-- config.sh
| |-- README.md
|
|-- FG8_aa_properties
| |-- extract_aa_properties.R
| |-- AAindex.rda
| |-- config.R
| |-- config.sh
| |-- README.md
|
|-- FG9_encode/
| |-- get_encode.sh
| |-- downloadEncode.py
| |-- addAnnotations.py
| |-- reformat_encode.py
| |-- config.sh
| |-- README.md
|
|-- FG10_alpha_fold/
| |-- get_alpha_fold.py
| |-- README.md
For individual feature descriptions and usage, please see the documentation within the relevant sub-directory.
Contributions are welcome! Follow the guidelines in CONTRIBUTING.md.
This project is licensed under LICENSE.
This work was carried out in the UK Medical Research Council Integrative Epidemiology Unit (MC_UU_00032/03) and using the computational facilities of the Advanced Computing Research Centre, University of Bristol.
Special thanks to:
- Tom Gaunt
- Colin Campbell
This work was funded by Cancer Research UK [C18281/A30905].
For enquiries, contact us at amy.francis@bristol.ac.uk.