the installation test of pipeline-umi-amplicon failed
dieunelderilus opened this issue · 11 comments
Hello- I have followed properly the instructions provided to install this pipeline-umi-amplicon. However, the installation test failed Command used:
snakemake -j 1 -pr --configfile config.yml
Screenshot of the error message:
Please any help?
~DD
Hi and thanks for reporting this.
Could you pleas try activating the conda environment and then running:
pip install numpy==1.19.5
and then try running:
snakemake -j 1 -pr --configfile config.yml
again?
Thanks,
Philipp
Hello Philip,
Thank you for your response. Here is is the output of the pip intall command:
(pipeline-umi-amplicon) [qlk5@biolinux pipeline-umi-amplicon]$ pip install numpy==1.19.5
Requirement already satisfied: numpy==1.19.5 in /scicomp/home- pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages (1.19.5)
Here is the output of of the snakemake -j 1 -pr --configfile config.yml command:
I have tried this in 2 different computers (cluster) but I still have the same error message at the end.
Any suggestions?
Thanks in advance!
~DD
HI again and sorry for not getting back to you earlier.
Could you please post the content of this file:
here?
Thanks,
Philipp
Hello Philip - Thank you for your response, here is the content of the file *smolecule.log file:
<[08:51:22 - Smolecule] Given one input file, subreads are assumed to be grouped by read.
[08:51:22 - Smolecule] Running spoa pre-medaka consensus for all reads.
[08:51:23 - Smolecule] Writing medaka input bam for 1 reads.
[08:51:23 - Smolecule] Running medaka consensus.
[08:51:23 - Predict] Setting tensorflow inter/intra-op threads to 1/1.
[08:51:23 - Predict] Processing region(s): 0:0-1206
[08:51:23 - Predict] Using model: /scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/data/r941_min_high_g360_model.hdf5.
[08:51:23 - Predict] Processing 1 long region(s) with batching.
[08:51:23 - ModelStore] filepath /scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/data/r941_min_high_g360_model.hdf5
[08:51:23 - ModelStore] ModelStore exception <class 'NotImplementedError'>
Traceback (most recent call last):
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/bin/medaka", line 11, in
sys.exit(main())
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/medaka.py", line 669, in main
args.func(args)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/smolecule.py", line 443, in main
medaka.prediction.predict(args)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/prediction.py", line 147, in predict
model = model_store.load_model(time_steps=args.chunk_len)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/datastore.py", line 77, in load_model
model = model_partial_function(time_steps=time_steps)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/medaka/models.py", line 146, in build_model
model.add(Bidirectional(gru, input_shape=input_shape))
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 517, in _method_wrapper
result = method(self, *args, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/sequential.py", line 208, in add
layer(x)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/wrappers.py", line 539, in call
return super(Bidirectional, self).call(inputs, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 951, in call
return self._functional_construction_call(inputs, args, kwargs,
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1090, in _functional_construction_call
outputs = self._keras_tensor_symbolic_call(
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 822, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 863, in _infer_output_signature
outputs = call_fn(inputs, *args, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/wrappers.py", line 652, in call
y = self.forward_layer(forward_inputs,
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 660, in call
return super(RNN, self).call(inputs, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1012, in call
outputs = call_fn(inputs, *args, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent_v2.py", line 439, in call
inputs, initial_state, _ = self._process_inputs(inputs, initial_state, None)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 859, in _process_inputs
initial_state = self.get_initial_state(inputs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 642, in get_initial_state
init_state = get_initial_state_fn(
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 1948, in get_initial_state
return _generate_zero_filled_state_for_cell(self, inputs, batch_size, dtype)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 2987, in _generate_zero_filled_state_for_cell
return _generate_zero_filled_state(batch_size, cell.state_size, dtype)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 3005, in _generate_zero_filled_state
return create_zeros(state_size)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/keras/layers/recurrent.py", line 3000, in create_zeros
return array_ops.zeros(init_state_size, dtype=dtype)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2819, in wrapped
tensor = fun(*args, **kwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2868, in zeros
output = _constant_if_small(zero, shape, dtype, name)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 2804, in _constant_if_small
if np.prod(shape) < 1000:
File "<array_function internals>", line 5, in prod
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 3030, in prod
return _wrapreduction(a, np.multiply, 'prod', axis, dtype, out,
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 87, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
File "/scicomp/home-pure/qlk5/Desktop/important_programs/miniconda3/envs/pipeline-umi-amplicon/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 852, in array
raise NotImplementedError(
NotImplementedError: Cannot convert a symbolic Tensor (bidirectional/forward_gru1/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported>
Hello, just wanted to say that I had that same failed test error. I ran pip install numpy==1.19.5 before rerunning the test and it successfully fixed the issue on my end.
Hello @KeelyDulmage - Unfortunately I have tried your suggestion but it doesn't work in my computer. I have numpy==1.19.5 installed and running in my working environment.
Thank you
~DD
Hi again.
Thanks for sending through the log file. The error you get is the same i got with the wrong numpy version. When changing to the one we discussed above that problem was resolved.
I tried a few things, but I was unable to reproduce your problem which makes it very difficult to pinpoint the cause of the problem.
Could I ask you to run:
conda list > conda_env.log
and send me the log file please?
Thank you,
Philipp
Hello Philip - Yes , this looks strange. Please find attached bellow the output of conda list > conda_env.log:
conda_env.log
Thank you,
~DD
Same for me, installing numpy 1.19.5 did not resolve the problem (see also #13 )
This is strange , they have closed this issue while it is still a serious problem for the user. I still don't know how to solve this.
Cheers,
~DD
Same for me, installing numpy 1.19.5 did not resolve the problem (see also #13 )
This is strange , they have closed this issue while it is still a serious problem for the user. I still don't know how to solve this.
Cheers,
~DD
Just opened a new issue, pip install numpy==1.19.5 generates a numerous of dependency issues. Original versions in the environmen.yml file were not given. Building the conda env based on this file, will give you the latest versions of the predefined tools. Some of these tools need numpy 1.24.4 ... .
Running the pipeline as it is, takes you as far as generating unpolished UMI consensuses and some stats... . Better than nothing, but I hope they will resolve the issues. (Or look for another tool which can handle UMI amplicons)