Error running mRibo

Question

Error running mRibo

Closed this issue 5 years ago · 7 comments

Hi,
I tried running mRibo for collaborators and ran into an issue.

When running the mRibo-build script on the practice data (or the actual data), I get the following error:

------------------------------------------------------------------------------------------------------------------------
-----                 Building dictionary from mRibo/mRibo/practice_files/annotations/practice.gff                 -----
------------------------------------------------------------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/local/bin/build_annotation_dictionary.v2.py", line 85, in <module>
    GFF_conversion = ribo_util.GFF_to_dict(paths_in, gff_settings)
  File "/usr/local/bin/ribo_util.py", line 924, in GFF_to_dict
    chr = GFFgen.next()
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 742, in parse
    target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 322, in parse_in_parts
    for results in self.parse_simple(gff_files, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 343, in parse_simple
    for results in self._gff_process(gff_files, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 637, in _gff_process
    for out in self._lines_to_out_info(line_gen, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 695, in _lines_to_out_info
    fasta_recs = self._parse_fasta(FakeHandle(line_iter))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 560, in _parse_fasta
    return list(SeqIO.parse(in_handle, "fasta"))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/Bio/SeqIO/FastaIO.py", line 186, in FastaIterator
    for title, sequence in SimpleFastaParser(handle):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/Bio/SeqIO/FastaIO.py", line 48, in SimpleFastaParser
    for line in handle:
TypeError: iteration over non-sequence

------------------------------------------------------------------------------------------------------------------------
-----             Succesfully built dictionary from mRibo/mRibo/practice_files/annotations/practice.gff            -----
------------------------------------------------------------------------------------------------------------------------

Do you have any idea what could cause this issue?
I assume that this is a biopython / BCBio-gff dependency issue. It would really help me if you could tell me which exact version of the dependencies you used to run mRibo. (as for example newer version of biopython do not support python2 anymore, "Biopython 1.76 was our final release to support Python 2.7")

Thank you for creating this tool wrapper and your help.

Answer 1 · 2020-06-19T17:30:10.000Z

Hi @RickGelhausen,

Thanks for using my tool. I'm not sure it is a dependency issue, but rather an issue with the gff. In any case, the version of bcbio_nextgen I'm using is v1.1.0

Could you send me the full command of mRibo-build you used?

Thanks,

Diego

Answer 2 · 2020-06-19T17:32:23.000Z

And for bcbio-gff it is v0.6.6

Answer 3 · 2020-06-22T11:09:05.000Z

Thanks for your reply.

I tested again using these bcbio versions.

The test-call I performed was:
mRibo-build -a mRibo/mRibo/practice_files/annotations/practice.gff -g mRibo/mRibo/practice_files/genomes/genome.fa -C gelhausr -M Hv -o test_results

using the versions above, I now receive another error:

Traceback (most recent call last):
  File "/usr/local/bin/build_annotation_dictionary.v2.py", line 85, in <module>
    GFF_conversion = ribo_util.GFF_to_dict(paths_in, gff_settings)
  File "/usr/local/bin/ribo_util.py", line 924, in GFF_to_dict
    chr = GFFgen.next()
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 746, in parse
    target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 322, in parse_in_parts
    for results in self.parse_simple(gff_files, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 343, in parse_simple
    for results in self._gff_process(gff_files, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 637, in _gff_process
    for out in self._lines_to_out_info(line_gen, limit_info, target_lines):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 699, in _lines_to_out_info
    fasta_recs = self._parse_fasta(FakeHandle(line_iter))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/BCBio/GFF/GFFParser.py", line 560, in _parse_fasta
    return list(SeqIO.parse(in_handle, "fasta"))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/Bio/SeqIO/FastaIO.py", line 186, in FastaIterator
    for title, sequence in SimpleFastaParser(handle):
  File "/home/ubuntu/.local/lib/python2.7/site-packages/Bio/SeqIO/FastaIO.py", line 48, in SimpleFastaParser
    for line in handle:
TypeError: instance has no next() method

The error before was caused using bcbio-gff==0.6.4.

Thanks,
Rick

Answer 4 · 2020-06-23T05:44:19.000Z

Hi Rick,

I was unable to replicate the same error you got running the same test-call. Instead I got the following error when I ran mRibo-build -a mRibo/mRibo/practice_files/annotations/practice.gff -g mRibo/mRibo/practice_files/genomes/genome.fa -C gelhausr -M Hv -o test_results

Traceback (most recent call last): File "/usr/local/bin/build_annotation_dictionary.v2.py", line 82, in <module> GFF_conversion = ribo_util.GFF_to_dict(paths_in, gff_settings) File "/usr/local/bin/ribo_util.py", line 910, in GFF_to_dict biotype_qual = gff_settings['biotype_qual'] KeyError: 'biotype_qual'

This is due to a problem in ribo_util.py, which I have now fixed and pushed. Hopefully that fixes your error as well. Please let me know if it works.

Best,
Diego

Answer 5 · 2020-06-23T09:20:14.000Z

Hi Diego,

I did a re-setup of everything regarding mRibo, but I still get the same issue than before.
As we are both using the same data and you are unable to replicate the issue, I would assume that I am missing a dependency or I have a wrong version of some dependency.

Maybe I have some python2 / 3 mix-up going on due to miniconda. I will have to check this in more detail. I tried to get the dependencies in python2.7. I hope this is correct, as I noticed that the ribo_* scripts are python2 scripts.
The TypeError: instance has no next() method seems to be a common python2 /3 compatibilaty issue.

Regarding the gff_settings['biotype_qual'] KeyError: 'biotype_qual', I also got that last week, but I forgot that I changed something in there a week ago, to see if it runs through afterwards. I should have mentioned this before, but I completely forgot about it. Thank you for fixing it.

Thanks for the help,
Rick

Answer 6 · 2020-06-24T17:55:09.000Z

Hi Rick,

Sorry I couldn't be more help, but I will continue to look into this. If you figure out how to fix it, please post it here so that I can implement it. Cross-talk between python 2/3 can be annoying, but I am planning to package this into conda which will hopefully help with this. Here are some alternative so that you can proceed:

You can set up a virtual environment and install mRibo and its dependencies in that environment: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
Are you doing the analysis in H. volcanii? If yes, here is the Hv_dictionary. I gave an uncompressed .csv version so you can see its contents.
If not in Hv and you are analyzing a different organism, if you send me the .gff and .fa of the organism I can make a dictionary for you.

Best,

Diego

Answer 7 · 2020-06-25T08:09:20.000Z

Hi Diego,

thank you for your hint with the python environments. I usally use conda environments, as I did in this case. Now that I set up everything in a seperate python environment it works. I am now able to create the practice dictionary without any issue.

I assume that it also works using conda environments, but I might have mixed up some dependencies.
It is nice to hear that you want to create a conda recipe for your tool. This will definitely solve any further dependency issues.

I will now try to use mRibo on a campylobacter jejuni dataset.
If I should run into any further problems along the way, I will let you know.

Thank you again for your offers and helpful tips,
Rick