Nextomics/NextPolish

Error when running test data

Opened this issue · 1 comments

I've loaded NextPolish onto a virtual Ubuntu 20.04.2 LTS system and the install seems to have completed without issue, but when I run the test data, I've encountered a traceback error which I'm struggling to resolve. Could you please advise as to what actions I need to take to fix this?

Test data output:

[INFO] 2021-07-26 11:17:53,511 start...
[INFO] 2021-07-26 11:17:53,511 logfile: pid2201.log.info
[WARNING] 2021-07-26 11:17:53,515 Re-write workdir
[WARNING] 2021-07-26 11:17:53,520 Delete task: 6 due to missing hifi_fofn.
[INFO] 2021-07-26 11:17:53,520 scheduled tasks:
[5, 1, 2]
[INFO] 2021-07-26 11:17:53,520 options:
[INFO] 2021-07-26 11:17:53,521 {'job_type': 'local', 'job_prefix': 'nextPolish', 'task': [5, 1, 2], 'rewrite': 1, 'clean
tmp': 0, 'rerun': 3, 'parallel_jobs': '2', 'multithread_jobs': '3', 'cluster_options': '', 'genome_size': 111129, 'workdir': '/home/sarahjkerr/NextPolish/test_data/./01_rundir', 'polish_options': '-p 3', 'sgs_use_duplicate_reads': 0, 'sgs_unpaired': '0', 'sgs_max_depth': '100', 'sgs_block_size': 5556450.0, 'sgs_rm_nread': 1, 'sgs_align_options': 'bwa mem -p -t 3', 'lgs_block_size': 5556450.0, 'lgs_min_read_len': '5k', 'lgs_max_read_len': '0', 'lgs_max_depth': '100', 'lgs_minimap2_options': '-x map-ont -t 3', 'lgs_read_type': 'ont', 'hifi_block_size': '500M', 'hifi_min_read_len': '1k', 'hifi_max_read_len': '0', 'hifi_max_depth': '100', 'hifi_minimap2_options': '-x map-pb', 'genome': '/home/sarahjkerr/NextPolish/test_data/./raw.genome.fasta', 'sgs_fofn': '/home/sarahjkerr/NextPolish/test_data/./sgs.fofn', 'lgs_fofn': '/home/sarahjkerr/NextPolish/test_data/./lgs.fofn', 'lgs_polish': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.lgs_polish', 'hifi_polish': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.hifi_polish', 'score_chain': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.score_chain', 'kmer_count': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.kmer_count', 'snp_phase': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.snp_phase', 'snp_valid': '/home/sarahjkerr/NextPolish/test_data/./01_rundir/%02d.snp_valid', 'align_threads': 3}
[INFO] 2021-07-26 11:17:53,521 step 0 and task 5 start:
[INFO] 2021-07-26 11:17:53,523 skip step: db_split
[INFO] 2021-07-26 11:17:53,523 skip step: align_genome
[INFO] 2021-07-26 11:17:53,524 skip step: merge_bam
[INFO] 2021-07-26 11:17:58,534 Total jobs: 2
[INFO] 2021-07-26 11:17:58,537 Submit jobID:[2202] jobCmd:[/home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome0/nextPolish.sh] in the local_cycle.
[INFO] 2021-07-26 11:17:59,043 Submit jobID:[2208] jobCmd:[/home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome1/nextPolish.sh] in the local_cycle.
[ERROR] 2021-07-26 11:18:05,054 polish_genome failed: please check the following logs:
[ERROR] 2021-07-26 11:18:05,055 /home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome0/nextPolish.sh.e
[ERROR] 2021-07-26 11:18:05,055 /home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome1/nextPolish.sh.e

nextPolish.sh.e:

hostname

  • hostname
    cd /home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome0
  • cd /home/sarahjkerr/NextPolish/test_data/01_rundir/00.lgs_polish/04.polish.ref.sh.work/polish_genome0
    time /home/sarahjkerr/miniconda3/bin/python /home/sarahjkerr/NextPolish/lib/nextpolish2.py -sp -p 3 -g /home/sarahjkerr>+ time /home/sarahjkerr/miniconda3/bin/python /home/sarahjkerr/NextPolish/lib/nextpolish2.py -sp -p 3 -g /home/sarahjke>[INFO] 2021-07-26 11:17:58,628 Corrected step options:
    [INFO] 2021-07-26 11:17:58,628 Namespace(alignment_identity_ratio=0.8, alignment_score_ratio=0.8, auto=True, bam_list='>[WARNING] 2021-07-26 11:17:58,639 Adjust -p from 3 to 0, -w from 5000000 to 5000000, logical CPUs:4, available RAM:~5G,>Traceback (most recent call last):
    File "/home/sarahjkerr/NextPolish/lib/nextpolish2.py", line 256, in
    main(args)
    File "/home/sarahjkerr/NextPolish/lib/nextpolish2.py", line 192, in main
    pool = Pool(args.process, initializer=start)
    File "/home/sarahjkerr/miniconda3/lib/python3.8/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
    File "/home/sarahjkerr/miniconda3/lib/python3.8/multiprocessing/pool.py", line 205, in init
    raise ValueError("Number of processes must be at least 1")
    ValueError: Number of processes must be at least 1
    Command exited with non-zero status 1
    0.05user 0.05system 0:00.10elapsed 98%CPU (0avgtext+0avgdata 15080maxresident)k
    0inputs+0outputs (0major+2379minor)pagefaults 0swaps

Operating system: Ubuntu 20.04.2
GCC: 9.3.0
Python version: 3.8.10
Nextpolish version: command nextPolish -v didn't work but it was downloaded today so I'm pretty sure it's 1.3.1

moold commented

Hi, the available memory of your system is not enough, maybe you need to try a system with more available memory.