jtamames/SqueezeMeta

stop in step10

Closed this issue · 8 comments

Dear sir,
I met problem in step10
Mapping with Bowtie2 (Langmead and Salzberg 2012, Nat Methods 9(4), 357-9)
Creating reference from contigs
Working with sample 1: A7
Getting raw reads
Aligning to reference with bowtie
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0011.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0007.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0009.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0010.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0008.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0005.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0002.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0003.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0006.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0004.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0001.bam" : File exists
[E::hts_open_format] Failed to open file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0000.bam" : File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0000.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0001.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0002.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0003.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0004.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0005.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0006.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0007.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0008.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0009.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0010.bam": File exists
samtools sort: failed to create temporary file "/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A7.bam.tmp.0011.bam": File exists
Error running samtools at /lustre/BIF/nobackup/sun091/miniforge3/envs/SqueezeMeta/SqueezeMeta/scripts/10.mapsamples.pl line 272, line 0.000000.
Stopping in STEP10 -> 10.mapsamples.pl. Program finished abnormally

If you don't know what went wrong or want further advice, please look for similar issues in https://github.com/jtamames/SqueezeMeta/issues
Feel free to open a new issue if you don't find the answer there. Please add a brief description of the problem and upload the /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/syslog file (zip it first)
Died at /lustre/BIF/nobackup/sun091/miniconda3/envs/SqueezeMeta/bin/SqueezeMeta.pl line 941.
Could you give me some suggestions?

syslog.zip

It could be this bug in samtools (samtools/samtools#1739) but it hasn't triggered for us so far so I am not sure.
It could also be something related to writing permissions... did you perhaps run out of disk quota?
Has it worked ok with the test dataset that we provide with the databases?

It could be this bug in samtools (samtools/samtools#1739) but it hasn't triggered for us so far so I am not sure. It could also be something related to writing permissions... did you perhaps run out of disk quota? Has it worked ok with the test dataset that we provide with the databases?

Thanks for your suggestions . I have run step10 successfully, but there is still some wrong with step14
[0 seconds]: STEP14 -> BINNING: 14.runbinning.pl
[E::bgzf_read] Read block operation failed with error 4 after 0 of 4 bytes
ERROR: fail to open index BAM file '/lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam/Q19.A8.bam'
Traceback (most recent call last):
File "/lustre/BIF/nobackup/sun091/miniforge3/envs/SqueezeMeta/SqueezeMeta/bin/CONCOCT-1.1.0/scripts/concoct_coverage_table.py", line 91, in
generate_input_table(args.bedfile, args.bamfiles, samplenames=samplenames)
File "/lustre/BIF/nobackup/sun091/miniforge3/envs/SqueezeMeta/SqueezeMeta/bin/CONCOCT-1.1.0/scripts/concoct_coverage_table.py", line 41, in generate_input_table
sys.stderr.write(out)
TypeError: write() argument must be str, not bytes
Bus error
sh: 1: cannot create /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/intermediate/binners/concoct/concoct_int/clustering_merged.csv: Directory nonexistent
wc: /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/intermediate/binners/concoct/: Is a directory
Could you give me some suggestions?

What did you change in order to make step 10 work?
Can you share the /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/results/10.Q19.mappingstat file here?

I did not receive the file. Please attach it in GitHub.

I see, mapping seems to have worked fine, but it didn't generate the sorted BAM files.
Can you post here the output of ls /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/data/bam?

Probably you will need to repeat step 10 to make sure the BAM files get generated properly with

10.mapsamples.pl /lustre/BIF/nobackup/sun091/metagenome/coassembly_per_accession/Q19/

Closing due to inactivity, feel free to reopen