WARNING: Failed to compile plugin LoF: can't open!
Opened this issue · 11 comments
Hi,
I am trying to use loftee plugin (Grch38) as a part of VEP and I got the following error.
Could you please help?
Thanks in advance
Command used
docker run -t -i -v /home/vep_data/:/opt/vep/.vep ensemblorg/ensembl-vep ./vep --cache
--format vcf --tab --force_overwrite --dir_cache /opt/vep/.vep/
--input_file /opt/vep/.vep/input/pakistani.exomes.FREEZE_fixed_hg38_normalised_SSv5_w100.vcf.gz
--output_file /opt/vep/.vep/my_output.vcf --dir_plugins /opt/vep/.vep/Plugins
--plugin LoF,
loftee_path:/opt/vep/.vep/Plugins/loftee,
gerp_bigwig:/opt/vep/.vep/LoFtool/gerp_conservation_scores.homo_sapiens.GRCh38.bw,
human_ancestor_fa:/opt/vep/.vep/LoFtool/human_ancestor.fa.gz,
conservation_file:/opt/vep/.vep/LoFtool/loftee.sql --fields Consequence,Codons,
Amino_acids,Gene,SYMBOL,Feature,EXON,LoF,LoF_filter,LoF_flags
-v /home/.cpanm/work/1597802785.16488/DBD-SQLite-1.64/blib/lib/DBD/:/opt/vep/DBD
Error:
Species 'homo_sapiens' loaded from database 'homo_sapiens_core_100_38'
Species 'homo_sapiens' loaded from database 'homo_sapiens_cdna_100_38'
Species 'homo_sapiens' loaded from database 'homo_sapiens_otherfeatures_100_38'
Species 'homo_sapiens' loaded from database 'homo_sapiens_rnaseq_100_38'
homo_sapiens_variation_100_38 loaded
homo_sapiens_funcgen_100_38 loaded
No ancestral database found
No ontology database found
ensembl_taxonomy API not found - ignoring ncbi_taxonomy
Use of uninitialized value $ensembl_metadata_db in concatenation (.) or string at /opt/vep/src/ensembl-vep/Bio/EnsEMBL/Registry.pm line 2393.
ensembl_metadata API not found - ignoring
No production database or adaptor found
Smartmatch is experimental at /opt/vep/.vep/Plugins/de_novo_donor.pl line 175.
Smartmatch is experimental at /opt/vep/.vep/Plugins/de_novo_donor.pl line 214.
Smartmatch is experimental at /opt/vep/.vep/Plugins/splice_site_scan.pl line 191.
Smartmatch is experimental at /opt/vep/.vep/Plugins/splice_site_scan.pl line 194.
Smartmatch is experimental at /opt/vep/.vep/Plugins/splice_site_scan.pl line 238.
Smartmatch is experimental at /opt/vep/.vep/Plugins/splice_site_scan.pl line 241.
WARNING: Failed to compile plugin LoF: can't open!
Compilation failed in require at /opt/vep/.vep/Plugins/loftee_splice_utils.pl line 4.
Compilation failed in require at /opt/vep/.vep/Plugins/LoF.pm line 33.
Compilation failed in require at (eval 51) line 2.
BEGIN failed--compilation aborted at (eval 51) line 2.
2020-08-19 23:02:56 - Auto-detected FASTA file in cache directory
It seems that you are not pointing to the correct path for LoF.pm
You may need to export PERL5LIB=/folder/containing_LoF.pm:$PERL5LIB
@JIBINJOHNV Hey Jibin did you manage to resolve this issue ?
I used many approaches like passing PERL5LIB with path to the loftee folder, and then other internal folders too ... still no success ...
export PERL5LIB=/scratch/vep_plugins/loftee/maxEntScan:/scratch/vep_plugins/loftee/splice_data:/scratch/vep_plugins/loftee:/scratch/vep_plugins:$PERL5LIB
Installed the loftee and other plugins at new path and passed that path to vep via --dir_plugins ... still no success !!!
I hardcoded the path to the required file in loftee_splice_utils.pl ... still no success !!!
@dantaki @konradjk
is there any workaround to make loftee work with vep for above mentioned issue ? Am I missing something very basic ? Please help team !!! I'm using perl v5.22.2
dbNSFP module is detected and works with my installation of vep ...
Hi @konradjk ... I thank you for your kind response ....
here is the snippet of the command I'm running ... I added all the different path to $PERL5LIB to ensure that LoF.pm finds the splice_data and maxEntScan ... but I had started with only /home/vaibhash/scratch/vep_plugins/loftee
export PERL5LIB=/home/vaibhash/scratch/vep_plugins/loftee/maxEntScan:/home/vaibhash/scratch/vep_plugins/loftee/splice_data:/home/vaibhash/scratch/vep_plugins/loftee:/home/vaibhash/scratch/vep_plugins:$PERL5LIB
echo $PERL5LIB
for chr in {1..22}; do
echo $chr
file=$(find ../data -type f -iname "_c${chr}_b.gz")
for item in $file; do
echo $item
fn="${item##*/}"
~/vep/vep -i $item
--plugin LoF,loftee_path:/home/vaibhash/scratch/vep_plugins/loftee,
human_ancestor_fa:/home/vaibhash/vep_plugins/loftee_files/human_ancestor.fa.gz,
conservation_file:/home/vaibhash/vep_plugins/loftee_files/phylocsf_gerp.sql
--plugin dbNSFP,/home/vaibhash/scratch/vep_plugins/dbnsfp_files/dbNSFP4.1a_grch38.gz,Ensembl_transcriptid,Uniprot_acc,VEP_canonical,LRT_pred,SIFT_pred,MutationTaster_pred,Polyphen2_HDIV_pred,Polyphen2_HVAR_pred
-everything
--offline
--cache
--dir_cache /home/vaibhash/scratch/vep_cache
--dir_plugins /home/vaibhash/scratch/vep_plugins
(even when I pass --dir_plugins /home/vaibhash/scratch/vep_plugins/loftee here I get the same error )
--buffer_size 10000
-v
--force_overwrite
-o ./annotate/Filter_$fn.txt
done
done
still I get following error ...
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/de_novo_donor.pl line 175.
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/de_novo_donor.pl line 214.
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/splice_site_scan.pl line 191.
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/splice_site_scan.pl line 194.
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/splice_site_scan.pl line 238.
Smartmatch is experimental at /home/vaibhash/scratch/vep_plugins/loftee/splice_site_scan.pl line 241.
WARNING: Failed to compile plugin LoF: can't open!
Compilation failed in require at /home/vaibhash/scratch/vep_plugins/loftee/loftee_splice_utils.pl line 4.
Compilation failed in require at /home/vaibhash/scratch/vep_plugins/loftee/LoF.pm line 33.
Compilation failed in require at (eval 70) line 2.
BEGIN failed--compilation aborted at (eval 70) line 2.
I had started with plugins being installed in default location i.e. ~/.vep/Plugins ... but it didn't work ... so I switched to plugins and cache being installed in --dir_cache /home/vaibhash/scratch/vep_cache and --dir_plugins /home/vaibhash/scratch/vep_plugins respectively ...
I know that I have to do something that allows loftee_splice_utils.pl to find internal modules, I tried hard coding the path but but even that didn't work, is there some specific way of hardcoding path in perl that help me bypass this issue ? I thank you with deep gratitude for jumping in to help me through this issue ...
vep is installed in ~/vep folder ...
and I'm running the script from ~/scratch/scripts
below works, i.e. vep with only dbNSFP ..
( here I just replaced the loop with SLURM batch array after I verified on landing node that this works ...)
export PERL5LIB=/home/vaibhash/scratch/vep_plugins:$PERL5LIB
echo $PERL5LIB
chr=$SLURM_ARRAY_TASK_ID
file=$(find ../data -type f -iname "_c${chr}_b.gz")
for item in $file; do
echo $item
fn="${item##*/}"
~/vep/vep -i $item
--plugin dbNSFP,/home/vaibhash/scratch/vep_plugins/dbnsfp_files/dbNSFP4.1a_grch38.gz,Ensembl_transcriptid,Uniprot_acc,VEP_canonical,LRT_pred,SIFT_pred,MutationTaster_pred,Polyphen2_HDIV_pred,Polyphen2_HVAR_pred
-everything
--offline
--cache
--dir_cache /home/vaibhash/scratch/vep_cache
--dir_plugins /home/vaibhash/scratch/vep_plugins
--buffer_size 10000
-v
--force_overwrite
-o ./annotate/Filter_$fn.txt
done
Hmm, that is strange. It doesn't seem to be finding maxEntScan/score3.pl in the subfolder. I would think adding it to PERL5LIB would have helped, but maybe something about Perl loading things from subdirectories has changed? Can you confirm the files are there, and if they are, maybe move the score*.pl files up a directory and modify the loftee_splice_utils.pl removing the subdirectory to see if that helps?
Hey @konradjk , Thanks again for the kind response ... Yes I had confirmed that the files exists ...
I moved the files score3.pl and score5.pl from maxEntScan folder and also the model folder ... still no luck ...
I just noticed that vep has maxEntScan as one of the plugin and as I installed all the plugins may be somehow LoF.pm is first calling that package ... I will try again with installing only the two plugins I want to use and see if that helps ...
just to be sure ... now a days vep install the packages through INSTALL.pl ... when I do so I get LoF.pm created in plugins folder ... but as mentioned in info in this project I create a new folder ... in my case loftee and clone all the content to that folder ... and I remove the LoF.pm file created by the vep ( I did diff and its basically same as the LoF.pm I pull from this project ) and point the vep tool to use the plugin cloned ...
Huh, that is surprising but definitely a possible reason for the issue. Can you change modify the first few lines of loftee_splice_utils.pl from require "maxEntScan/score3.pl" to require "score3.pl" (and same for score5), after you've moved them into the loftee/ directory?
Hi @konradjk ..
below are the steps I have taken ...
I removed all the other plugins in vep - only LoF and dbNSFP ...
moved the score3.pl and score5.pl out of maxEntScan , removed maxEntScan folder altogether ...
still getting the same error ...
WARNING: Failed to compile plugin LoF: can't open!
Compilation failed in require at /home/vaibhash/vep_plugins/loftee/loftee_splice_utils.pl line 4.
Compilation failed in require at /home/vaibhash/vep_plugins/loftee/LoF.pm line 33.
Compilation failed in require at (eval 70) line 2.
BEGIN failed--compilation aborted at (eval 70) line 2.
also respected @konradjk ...
I added a print statement in score3.pl and its getting printed on the console ... so that means that ~/vep_plugins/loftee/loftee_splice_utils.pl is able to find the file but not able to compile it ...
IMHO, if that is the case then no one would be able to use the plugin though current approach ...
I will try pulling the latest maxEntScan and see if that helps ...
respected @konradjk ...
after few trials and debugging
I think I managed to resolve this issue ( at least this one ) ...
with my version of perl i.e. (v5.22.2)
we get "human_ancestor_fa:/home/vaibhash/scratch/vep_files/loftee_files/human_ancestor.fa.gz," as the input variable and thus we were failing to open the file ...
I had to split the above string twice, once for ':' and once for ',' to ensure that open doesn't fail on me ( so that I can extract just the /home/vaibhash/scratch/vep_files/loftee_files/human_ancestor.fa.gz"
attached screenshot the code that fixed this issue ...
but after that I started getting the null error ....
i guess I need to update the cache version to ensure that this works for me ...
I again thank you with deep gratitude for responding to messages ... I hope this to and fro helps some future users ...
ps : I might ask more question regarding the null error on appropriate issue placeholder... so that information is not lost ...
Thanks again !!!
I am using docker vep with my sample input dataset, but I am getting the same error in this issue (attached screenshot) and I could able to see the LofTool.pm file in the path and its showing Can't locate LofTool.pl.
The following command was used:
Error screenshot:
Please help me to fix this issue
Thank you
Krithika S
If anyone else finds this issue (as I did) I think the issue is caused by new line characters in the LOFTEE string. @webhash had the following code:
--plugin LoF, loftee_path:/opt/vep/.vep/Plugins/loftee, gerp_bigwig:/opt/vep/.vep/LoFtool/gerp_conservation_scores.homo_sapiens.GRCh38.bw, human_ancestor_fa:/opt/vep/.vep/LoFtool/human_ancestor.fa.gz
For me, code like this also doesn't work, and I get exactly the same error messages:
WARNING: Failed to compile plugin LoF: can't open! Compilation failed in require at /vep_plugins/loftee/loftee_splice_utils.pl line 4. Compilation failed in require at /vep_plugins/loftee/LoF.pm line 33. Compilation failed in require at (eval 71) line 2. BEGIN failed--compilation aborted at (eval 71) line 2.
However, when formatted like so, without any whitespace between arguments passed to LOFTEE:
--plugin LoF,loftee_path:/opt/vep/.vep/Plugins/loftee,gerp_bigwig:/opt/vep/.vep/LoFtool/gerp_conservation_scores.homo_sapiens.GRCh38.bw,human_ancestor_fa:/opt/vep/.vep/LoFtool/human_ancestor.fa.gz
The code runs without issue. The error seems to come from code that runs in score3.pl when it is imported using require "maxEntScan/score3.pl";. This code attempts to parse the arguments and crashes, which in turn causes the import to fail. My perl version is 5.22.0 for reference.