Error running iscope -- Something went wrong running Menetools

Question

Error running iscope -- Something went wrong running Menetools

aassie opened this issue 5 years ago · 14 comments

Hi there,

I am trying to run Metage2metabo and I am stuck at the iscope step.

The only error I have is Something went wrong running Menetools.

I already had something similar at the pathway tool step where I had to run the python code within python for it to see what the error was but I have no knowledge of menetool.

Is there a way to get a more comprehensive error message to know what's wrong?

Thank you,

Adrien

Answer 1 · 2020-03-23T06:52:12.000Z

Hi Adrien,

Can you tell me more about the command you use? Is it just m2m iscope or do you use the whole workflow? My guess is that there is either a problem with you seeds file or your metabolic networks.

I would suggest you to try running the following command with your seeds file and one of your metabolic networks files:

menescope -s seeds.sbml -d metabolic_network.sbml

Can you send us the error message that comes with that command please?

Thank you,

Clémence

Answer 2 · 2020-03-23T13:16:51.000Z

Hello Clémence,

Thanks for the quick response, I was running the following command:

m2m iscope -n m2m_recon/sbml/ -s test/metage2metabo/test/metabolic_data/seeds_toy.sbml -o m2m_iscope

I ran your command:
menescope -s test/metage2metabo/test/metabolic_data/seeds_toy.sbml -d m2m_recon/sbml/BIGb0170.sbml
And got the following error:

Reading draft network from m2m_recon/sbml/BIGb0170.sbml
Reading seeds from test/metage2metabo/test/metabolic_data/seeds_toy.sbml

Checking draft network scope
Traceback (most recent call last):
  File "/Users/adrien/opt/anaconda3/envs/M2M/bin/menescope", line 8, in <module>
    sys.exit(main_menescope())
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/site-packages/menetools/__main__.py", line 12, in main_menescope
    cmd_menescope()
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/site-packages/menetools/menescope.py", line 30, in cmd_menescope
    run_menescope(draft_sbml,seeds_sbml)
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/site-packages/menetools/menescope.py", line 67, in run_menescope
    for pred in model:
TypeError: 'NoneType' object is not iterable

I used the seeds_toy.sbml downloaded from the m2m github to see how the things works before trying to make my own.

Concerning my sbml files:

If I run m2m recon -g ./ -o m2m_recon -c 6 --noorphan --clean

I get

Something went wrong running Pathway Tools
Could not run Pathway Tools

A few things to note, I used genebank downloaded from PATRIC and to get the thing running I had to:

Add a trailing space on the LOCUS line
Add a SOURCE and ORGANISM line

I still got an error after that, so I went around and managed to get my sbml files by running:

import logging
import argparse
import logging
import os
import pkg_resources
import re
import sys
import tarfile
import time
from shutil import copyfile,which

from metage2metabo.m2m_workflow import run_workflow, recon, iscope, cscope, addedvalue, mincom, instance_community
from metage2metabo import sbml_management, utils

def metacom_analysis(sbml_dir, out_dir, seeds, host_mn):
    """Run the metabolism community analysis part of m2m
    Args:
        sbml_dir (str): sbml input directory
        out_dir (str): results directory
        seeds (str): seeds file
        host_mn (str): metabolic network file for host
    """
    # INDIVIDUAL SCOPES
    union_targets_iscope = iscope(sbml_dir, seeds, out_dir)
    # COMMUNITY SCOPE
    instance_com, targets_cscope = cscope(sbml_dir, seeds, out_dir, host_mn)
    # ADDED VALUE
    newtargets = addedvalue(union_targets_iscope, targets_cscope)
    if len(newtargets) > 0:
        sbml_management.create_species_sbml(newtargets, out_dir + "/community_analysis/targets.sbml")
        logger.info("Target file created with the addedvalue targets in: " +
                    out_dir + "/community_analysis/targets.sbml")
        # Add these targets to the instance
        logger.info("Setting these " + str(len(newtargets)) + " as targets")
        instance_w_targets = add_targets_to_instance(
            instance_com, out_dir,
            newtargets)
        # MINCOM
        mincom(instance_w_targets, out_dir)
        # remove intermediate files
        os.unlink(instance_com)
        os.unlink(instance_w_targets)
    else:
        logger.info("No newly producible compounds, hence no community selection will be computed")
        os.unlink(instance_com)

        
recon("./","m2m_recon","True",1,2,6,"True")

Answer 3 · 2020-03-23T14:40:53.000Z

I have tried to alter some SBMLs of mine to get a similar error but I never get it... Would you mind sharing your SBML file? You can send it by email if you agree (clemence.frioux/inria.fr).
Otherwise what you can do is a pip freeze to show me the versions of all the dependencies you use. I'll create a similar environment and keep on doing tests.

Regarding the recon run, we are at the moment trying to improve the error messages of m2m to return more details, as it is currently the case with mpwt, the underlying dependency of recon. We never used gbk files from PATRIC. As far as I am concerned, the missing trailing space after LOCUS is all new to me. However, the ORGANISM line is mandatory yes. The documentation of mpwt about gbk requirements is more precise than the documentation of m2m. We will add a link to this documentation.

To ensure that there is no other hidden error during the reconstruction of metabolic networks, could you run the command

mpwt -f ./ --patho --dat --md --clean --cpu 6 -v

Answer 4 · 2020-03-23T15:24:58.000Z

output of pip freeze:
I did create a fresh conda environment for m2m

alabaster==0.7.12
applaunchservices==0.2.1
appnope==0.1.0
argcomplete==1.11.1
argh==0.26.2
Arpeggio==1.9.2
astroid==2.3.3
atomicwrites==1.3.0
attrs==19.3.0
autopep8==1.5
Babel==2.8.0
backcall==0.1.0
biopython==1.74
bleach==3.1.3
certifi==2019.11.28
cffi==1.14.0
chardet==3.0.4
cloudpickle==1.3.0
clyngor==0.3.30
clyngor-with-clingo==5.3.post1
cobra==0.14.0
cryptography==2.8
cycler==0.10.0
decorator==4.4.2
defusedxml==0.6.0
depinfo==1.5.3
diff-match-patch==20181111
docopt==0.6.2
docutils==0.16
entrypoints==0.3
flake8==3.7.9
future==0.18.2
gevent==1.4.0
gffutils==0.10.1
greenlet==0.4.15
grequests==0.4.0
idna==2.9
imagesize==1.2.0
importlib-metadata==1.5.0
intervaltree==3.0.2
ipykernel==5.2.0
ipython==7.13.0
ipython-genutils==0.2.0
isort==4.3.21
jedi==0.15.2
Jinja2==2.11.1
jsonschema==3.2.0
jupyter-client==6.1.0
jupyter-core==4.6.3
keyring==21.1.1
kiwisolver==1.1.0
lazy-object-proxy==1.4.3
lxml==4.3.4
MarkupSafe==1.1.1
matplotlib==3.1.1
mccabe==0.6.1
MeneTools==2.0.3
Metage2Metabo==1.1.6
Miscoto==2.0.6
mistune==0.8.4
more-itertools==8.2.0
mpmath==1.1.0
mpwt==0.5.3
nbconvert==5.6.1
nbformat==5.0.4
numpy==1.18.2
numpydoc==0.9.2
optlang==1.4.4
packaging==20.3
padmet==4.0
pandas==1.0.3
pandocfilters==1.4.2
parso==0.5.2
pathtools==0.1.2
pexpect==4.8.0
pickleshare==0.7.5
pipdeptree==0.13.2
pluggy==0.13.1
prompt-toolkit==3.0.4
psutil==5.7.0
ptyprocess==0.6.0
py==1.8.1
pycodestyle==2.5.0
pycparser==2.20
pydocstyle==5.0.2
pyfaidx==0.5.8
pyflakes==2.1.1
Pygments==2.6.1
pylint==2.4.4
pyOpenSSL==19.1.0
pyparsing==2.4.6
pyPEG2==2.15.2
PyQt5==5.12.3
PyQt5-sip==4.19.18
PyQtWebEngine==5.12.1
pyrsistent==0.15.7
PySocks==1.7.1
pytest==5.4.1
python-dateutil==2.8.1
python-jsonrpc-server==0.3.4
python-language-server==0.31.9
python-libsbml==5.18.0
pytz==2019.3
PyYAML==5.3.1
pyzmq==19.0.0
QDarkStyle==2.8
QtAwesome==0.7.0
qtconsole==4.7.1
QtPy==1.9.0
requests==2.23.0
rope==0.16.0
Rtree==0.9.4
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.0
scipy==1.3.0
seaborn==0.9.0
simplejson==3.17.0
six==1.14.0
snowballstemmer==2.0.0
sortedcontainers==2.1.0
Sphinx==2.4.4
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==1.0.3
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.4
spyder==4.1.1
spyder-kernels==1.9.0
swiglpk==4.65.1
sympy==1.5.1
tabulate==0.8.7
testpath==0.4.4
tornado==6.0.4
traitlets==4.3.3
typed-ast==1.4.1
ujson==1.35
urllib3==1.25.8
watchdog==0.10.2
wcwidth==0.1.8
webencodings==0.5.1
wrapt==1.12.1
wurlitzer==2.0.0
yapf==0.29.0
zipp==3.1.0

Answer 5 · 2020-03-23T15:33:59.000Z

I could not reproduce the menetools (menescope) error with your data.
Here is the environment that I use:

argcomplete==1.11.1
argh==0.26.2
Arpeggio==1.9.2
attrs==19.3.0
biopython==1.76
certifi==2019.11.28
chardet==3.0.4
clyngor==0.3.30
clyngor-with-clingo==5.3.post1
cobra==0.14.0
cycler==0.10.0
depinfo==1.5.3
docopt==0.6.2
future==0.18.2
gevent==1.4.0
gffutils==0.10.1
greenlet==0.4.15
grequests==0.4.0
idna==2.9
importlib-metadata==1.5.0
kiwisolver==1.1.0
lxml==4.3.4
matplotlib==3.1.1
MeneTools==2.0.3
Metage2Metabo==1.1.6
Miscoto==2.0.6
more-itertools==8.2.0
mpmath==1.1.0
mpwt==0.5.3
numpy==1.18.2
optlang==1.4.4
packaging==20.3
padmet==4.0
pandas==1.0.3
pipdeptree==0.13.2
pluggy==0.13.1
py==1.8.1
pyfaidx==0.5.8
pyparsing==2.4.6
pyPEG2==2.15.2
pytest==5.4.1
python-dateutil==2.8.1
python-libsbml==5.18.0
pytz==2019.3
requests==2.23.0
ruamel.yaml==0.16.10
ruamel.yaml.clib==0.2.0
scipy==1.3.0
seaborn==0.9.0
simplejson==3.17.0
six==1.14.0
swiglpk==4.65.1
sympy==1.5.1
tabulate==0.8.7
urllib3==1.25.8
wcwidth==0.1.9
zipp==3.1.0

I personally made these tests with python 3.7.6 both on MacOS and Ubuntu 18.04.

Could you try to run menescope -s test/metage2metabo/test/metabolic_data/seeds_toy.sbml -d m2m_recon/sbml/BIGb0170.sbml in an environment with the same packages?

Answer 6 · 2020-03-23T15:46:40.000Z

New environment, exactly the same as above, new error:

 menescope -s test/metage2metabo/test/metabolic_data/seeds_toy.sbml -d m2m_recon/sbml/BIGb0170.sbml
Reading draft network from m2m_recon/sbml/BIGb0170.sbml
Reading seeds from test/metage2metabo/test/metabolic_data/seeds_toy.sbml

Checking draft network scope
Traceback (most recent call last):
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/clyngor/parsing.py", line 197, in parse_clasp_output
    line = next(output)
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/adrien/opt/anaconda3/envs/M2MC/bin/menescope", line 8, in <module>
    sys.exit(main_menescope())
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/menetools/__main__.py", line 12, in main_menescope
    cmd_menescope()
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/menetools/menescope.py", line 30, in cmd_menescope
    run_menescope(draft_sbml,seeds_sbml)
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/menetools/menescope.py", line 65, in run_menescope
    model = query.get_scope(draftnet, seeds)
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/menetools/query.py", line 25, in get_scope
    for model in models.discard_quotes.by_arity:
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/clyngor/answers.py", line 205, in __iter__
    for answer_set, optimization, optimality, answer_number in self._answers:
  File "/Users/adrien/opt/anaconda3/envs/M2MC/lib/python3.7/site-packages/clyngor/solving.py", line 208, in _gen_answers
    for ptype, payload in parse_clasp_output(stdout, yield_stats=True):
RuntimeError: generator raised StopIteration

Answer 7 · 2020-03-23T16:34:18.000Z

Well at least this is a change!
I wonder whether you have a second clingo, the ASP solver, installed on your system. Could it be the case?
What is the version you get when you run clingo --version from your conda environment?

Answer 8 · 2020-03-23T16:39:50.000Z

hmm, I have an error!

clingo --version
Traceback (most recent call last):
  File "/Users/adrien/opt/anaconda3/envs/M2M/bin/clingo", line 8, in <module>
    sys.exit(run_clingo())
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/site-packages/clyngor_with_clingo/__init__.py", line 24, in run_clingo
    subprocess.call([fname] + sys.argv[1:])
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '/Users/adrien/opt/anaconda3/envs/M2M/lib/python3.6/site-packages/clyngor_with_clingo/bin/macos/clingo'

Let me see if I can fix that 🤔

Answer 9 · 2020-03-23T16:45:45.000Z

Just notice that you are in your old python 3.6 env for this command, not the 3.7 that we share. But in both envs, I think the error is related to clingo. I just don't get why you would get this. My only hypothesis is that you have a clingo in the PATH that is from another version than the one used in m2m. The clyngor-with-clingo python package should install the clingo solver of the same version as the package. In the case of my environment, it is 5.3.

Answer 10 · 2020-03-23T17:18:39.000Z

This is true, sorry. I have multiple tab open in my terminal. The same thing happened.

Turns out clingo was on read-only on my machine. I don't know how it happened. Now the command worked!

intersection of scope 163
union of scope 541
max metabolites in scope 404
min metabolites in scope 217
average number of metabolites in scope 311.33 (+/- 58.81)
--- Indiv scopes runtime 4.03 seconds ---

Thank you so much for your help Clémence!

Answer 11 · 2020-03-23T17:47:55.000Z

You are welcome, I'm glad to read that it works! So apparently in some configurations of python 3.6 clingo get installed in read only mode. This seems to be more of a clyngor issue.
It doesn't seem to be linked to conda as I did a test with conda and python 3.6.10 and could not reproduce the bug. I'll keep looking

Answer 12 · 2020-03-23T17:54:39.000Z

@aassie can you tell me the exact number of the 3.6 version of python you were using please? Thank you very much

Answer 13 · 2020-03-23T18:14:51.000Z

It happened in env with Python 3.6.10 and in another one with Python 3.7.6

Answer 14 · 2020-04-21T09:43:45.000Z

I close this issue as it is related to Clyngor and has an open issue there.