ValueError: too many values to unpack

Question

ValueError: too many values to unpack

Closed this issue 6 years ago · 11 comments

Hi Lyam,

I'm trying to run metaTOR from a dedicated conda environment with all required dependencies installed.

metaTOR.sh align starts running but exit with a Python error at the network creation step:
any idea on how to fix that?

Python 3.6.6

Mapping reads...
70604753 reads; of these:
  70604753 (100.00%) were unpaired; of these:
    65283672 (92.46%) aligned 0 times
    4041938 (5.72%) aligned exactly 1 time
    1279143 (1.81%) aligned >1 times
7.54% overall alignment rate
70604753 reads; of these:
  70604753 (100.00%) were unpaired; of these:
    64590673 (91.48%) aligned 0 times
    4480279 (6.35%) aligned exactly 1 time
    1533801 (2.17%) aligned >1 times
8.52% overall alignment rate
[bam_sort_core] merging from 1 files and 1 in-memory blocks...
[bam_sort_core] merging from 1 files and 1 in-memory blocks...
Alignment done.
Merging bam files...
Done.
Generating network from alignments...
Traceback (most recent call last):
  File "/mibi/users/jnesme/Bin/metaTOR/network.py", line 821, in <module>
    main()
  File "/mibi/users/jnesme/Bin/metaTOR/network.py", line 810, in main
    my_assembly, = reference_file
ValueError: too many values to unpack (expected 1)

Pyton 3.6.6 and all dependencies installed:

Requirement already up-to-date: numpy in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (1.15.3)
Requirement already up-to-date: scipy in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 2)) (1.1.0)
Requirement already up-to-date: matplotlib in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 3)) (3.0.0)
Requirement already up-to-date: biopython in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 4)) (1.72)
Requirement already up-to-date: pysam in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 5)) (0.15.1)
Requirement already up-to-date: seaborn in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from -r requirements.txt (line 6)) (0.9.0)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 3)) (0.10.0)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 3)) (1.0.1)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 3)) (2.2.2)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 3)) (2.7.3)
Requirement already satisfied, skipping upgrade: pandas>=0.15.2 in /usr/local/home/jnesme/.local/lib/python3.6/site-packages (from seaborn->-r requirements.txt (line 6)) (0.20.1)
Requirement already satisfied, skipping upgrade: six in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from cycler>=0.10->matplotlib->-r requirements.txt (line 3)) (1.11.0)
Requirement already satisfied, skipping upgrade: setuptools in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->-r requirements.txt (line 3)) (40.4.3)
Requirement already satisfied, skipping upgrade: pytz>=2011k in /mibi/users/jnesme/miniconda2/envs/metator_env/lib/python3.6/site-packages (from pandas>=0.15.2->seaborn->-r requirements.txt (line 6)) (2018.5)

Answer 1 · 2018-10-24T13:19:15.000Z

Hello, I modified the incriminating line (network.py:810):

my_assembly, =

with

my_assembly, _ =

did that fix your issue?

Answer 2 · 2018-10-24T14:58:05.000Z

Edit: Yes, fixed now and produced the network file =) Thanks a lot for the quick reply. - I'm putting that here although I should prolly start nother issue: the model HMM fetched from dl.pasteur are not available anymore at the URL specified when runnin meta3c.sh dependencies ` Fetching HMMs... --2018-10-24 16:55:33-- http://dl.pasteur.fr/fop/5eHgTGww/modele_HMM.tar.gz Resolving dl.pasteur.fr... 157.99.21.75 Connecting to dl.pasteur.fr|157.99.21.75|:80... connected. HTTP request sent, awaiting response... 666 Bad Request - modele_HMM.tar.gz is expired 2018-10-24 16:55:33 ERROR 666: Bad Request - modele_HMM.tar.gz is expired. tar (child): modele_HMM.tar.gz: Cannot open: No such file or directory tar (child): Error is not recoverable: exiting now `

Answer 3 · 2018-10-24T15:21:18.000Z

Thanks for the heads-up, I updated the link. Does it work better now?

Answer 4 · 2018-10-24T15:36:07.000Z

I could download it yes. However, the filename isn't the same and it decompress in a dir called "hmm_databases" where default hmm directory location expects "modele_HMM"

Answer 5 · 2018-10-24T16:06:59.000Z

I'll update the default hmm directory name. Does it work seamlessly if you just run

./meta3c.sh dependencies

though? And in the meantime you can rename the directory and proceed to the rest of the pipeline.

Answer 6 · 2018-10-24T16:36:50.000Z

that's what I've been doing but I'm waiting for the partitioning step to end though

I'll then clone the updated metaTOR git directory and try running it to see if all is working as intended. I've already resumed some runs by just copying the "output" directory and using the same project name with -p
I guess this would still work for annotation step ?

Answer 7 · 2018-10-24T17:11:03.000Z

hum, (with previous git version):

./meta3c.sh partition -p 5KB.metaTOR.1
ends with:
Resolving partitions... paste: /mibi/users/jnesme/Bin/metaTOR/output/5KB.metaTOR.1/partition/iteration/1.community : No such file or directory

but the file exists and is non empty ?:
head /mibi/users/jnesme/Bin/metaTOR/output/5KB.metaTOR.1/partition/iteration/1.community
0
1
2
3
4
5
6
7
8
9

about the current release:

meta3c.sh dependencies ends now with:

`
hmm_databases.tgz 100%[==============================================>] 48.45M 1.90MB/s in 29s

2018-10-24 19:15:18 (1.66 MB/s) - ‘hmm_databases.tgz’ saved [50804980/50804980]

hmm_databases/
hmm_databases/conj.hmm
hmm_databases/essential.hmm
hmm_databases/pog.hmm
hmm_databases/SGC.hmm
hmm_databases/VOG.hmm
mv: cannot move ‘hmm_databases’ to ‘’: No such file or directory
OK
`

also, louvain should be put in a directory named "tools" (and also run make in louvain).

I'm now running ./meta3c.sh partition with current version. Let's see what happens.
Edit: still ends with the same error

Answer 8 · 2018-10-26T13:58:15.000Z

Could you upload your config_current.sh file? I seem to have trouble reproducing your issues. Also did you check you had all write/read/etc. access in the directory in question?

Cheers

Answer 9 · 2018-10-26T15:02:15.000Z

here it is, with ".txt" extension added so i could upload here.
config_current.sh.txt

Answer 10 · 2018-10-29T08:33:12.000Z

Could you try again with this new commit? I simply unquoted the incriminating expression:

paste "$(printf "${partition_dir}/iteration/%s.community " $(seq 1 "$repet"))" >"${tmp_dir}"/"${project}"_iterations_"${repet}".txt

into:

paste $(printf "${partition_dir}/iteration/%s.community " $(seq 1 "$repet")) >"${tmp_dir}"/"${project}"_iterations_"${repet}".txt

Cheers

Answer 11 · 2018-11-28T12:44:49.000Z

With the last few weeks' overhaul, this bug is probably unlikely to resurface again so I'm going to close this for now.