[BUG]: Nanopype report crashes with minimap2 but not with ngmlr

Question

[BUG]: Nanopype report crashes with minimap2 but not with ngmlr

Closed this issue 4 years ago · 4 comments

Prerequisites

Put an X between the brackets on this line if you have done all of the following:
- Read the contributions guide at: https://nanopype.readthedocs.io/en/latest/contribution
- Checked that your issue isn't already filed: https://github.com/giesselmann/nanopype/issues

Description

Hi,
I'm trying to obtain the report.pdf from my run, but it stops with an error in the report_alignments_stats checkpoint (see bellow). Before I run it with ngmlr and it worked, but now with minimap2 it's breaking.
I hope you can help me. Thanks in advance.
Miguel

P.s.: I submitted the issue before I finish writing it (I needed to edit)

Steps to Reproduce

Regular Nanopype run with ngmlr (WORKED):

snakemake --snakefile ngmlr/nanopype/Snakefile --directory ngmlr/nanopype/ --profile ngmlr/nanopype/profiles/slurm/ --use-singularity --singularity-args '-B /data/storage_data_raw/lr_HEL_experiment -B /data/reference' --printshellcmds --reason --local-cores 1 methylation/nanopolish/ngmlr/guppy/HEL.5x.GRCh38_p12_Release_96.bedGraph

Nanopype report for ngmlr (WORKED):

snakemake --snakefile ngmlr/nanopype/Snakefile --directory ngmlr/nanopype/ --profile ngmlr/nanopype/profiles/slurm/ --use-singularity --printshellcmds --reason --local-cores 1 report.pdf

Regular Nanopype run with minimap2 (WORKED):

snakemake --snakefile minimap2/nanopype/Snakefile --directory minimap2/nanopype/ --profile minimap2/nanopype/profiles/slurm/ --use-singularity --singularity-args '-B /data/storage_data_raw/lr_HEL_experiment -B /data/reference' --printshellcmds --reason --local-cores 1 methylation/nanopolish/minimap2/guppy/HEL.5x.GRCh38_p12_Release_96.bedGraph

Nanopype report for minimap2 (NOT WORKED):

snakemake --snakefile minimap2/nanopype/Snakefile --directory minimap2/nanopype/ --profile minimap2/nanopype/profiles/slurm/ --use-singularity --printshellcmds --reason --local-cores 1 report.pdf

Expected behavior:

Nanopype to produce the report.pdf file.

Actual behavior:

Got the following error:

[Sun Aug 16 10:11:17 2020]
localcheckpoint report_alignments_stats:
    input: alignments/minimap2/guppy/batches/HEL/20191120_FAL66483_FLO-MIN106D_SQK-RAD004_HEL_original_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191127_FAL66209_FLO-MIN106D_SQK-RAD004_HEL_original_2ndrun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191203_FAL66972_FLO-MIN106D_SQK-RAD004_HEL_purified_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20200122_FAL66375_FLO-MIN106D_SQK-RAD004_HEL_overload.GRCh38_p12_Release_96.hdf5
    output: report/plots/alignments/
    jobid: 3
    reason: Missing output files: report/plots/alignments/; Input files updated by another job: alignments/minimap2/guppy/batches/HEL/20191127_FAL66209_FLO-MIN106D_SQK-RAD004_HEL_original_2ndrun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191120_FAL66483_FLO-MIN106D_SQK-RAD004_HEL_original_1strun.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20200122_FAL66375_FLO-MIN106D_SQK-RAD004_HEL_overload.GRCh38_p12_Release_96.hdf5, alignments/minimap2/guppy/batches/HEL/20191203_FAL66972_FLO-MIN106D_SQK-RAD004_HEL_purified_1strun.GRCh38_p12_Release_96.hdf5
Downstream jobs will be updated after completion.

/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/base.py:2456: RuntimeWarning: '<' not supported between instances of 'tuple' and 'int', sort order is undefined for incomparable objects
  return Index.union(this, other, sort=sort).astype(object, copy=False)
[Sun Aug 16 10:12:26 2020]
Error in rule report_alignments_stats:
    jobid: 3
    output: report/plots/alignments/

RuleException:
TypeError in line 241 of /path/minimap2/nanopype/rules/report.smk:
object of type 'int' has no len()
  File "/path/minimap2/nanopype/rules/report.smk", line 241, in __rule_report_alignments_stats
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 287, in concat
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/reshape/concat.py", line 498, in get_result
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 2325, in reindex
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/site-packages/pandas/core/indexes/multi.py", line 501, in from_tuples
  File "pandas/_libs/lib.pyx", line 2455, in pandas._libs.lib.tuples_to_object_array
  File "/srv/nfs/raid5/software/conda/miniconda3/envs/nanopype_1.0.0/lib/python3.8/concurrent/futures/thread.py", line 57, in run
Removing output files of failed job report_alignments_stats since they might be corrupted:
report/plots/alignments/

Versions

1.0.0

Additional Information

Let me know if you need anything else.

Log Files

Put an X between the brackets on this line if you have attached log files from the observed behavior.
20200816_10_13_34_933237.nanopype.log
2020-08-16T093542.850487.snakemake.log

Answer 1 · 2020-08-17T18:13:51.000Z

I will have a look latest Friday, maybe earlier, but have to prepare a talk for Thursday.
Pay

Answer 2 · 2020-08-26T11:15:49.000Z

I can now reproduce the issue here on different data.
Also minimap2, also in the alignment report, same line.
keeping you posted.

Answer 3 · 2020-09-07T17:50:08.000Z

Hi Miguel,
sorry for the delay, but I think I fixed this issue. I seems to depend on the pandas version and not on minimap2/ngmlr. After having the same error message, I fixed the alignment report rule.
Could you in a free minute try the version from the development branch?
Please make sure to delete the report folder before and run

pip install -r requirements.txt --upgrade

to upgrade the nanopype virtual env.

Pay

Answer 4 · 2020-09-09T12:01:58.000Z

Hi Pay,
That solved the problem.
Thanks.
Miguel