Issue with combining fastq files
kazar4 opened this issue · 1 comments
kazar4 commented
Hello, so from what I can tell I have all of the preinstallation done correctly, but when I run
sudo make test
It ends up failing in the guppy step, however the weird part is it does successfully basecall. I will include a log below. The important part to look at are the bottom 8 lines, where basecalling is completed successfully but it fails after that (however im including the full log it to show that guppy and pomoxis are found by the check_file portion).
[22:43:19 - check_file] Checking file on kgallman: /usr/bin/guppy_basecaller
[22:43:19 - check_file] File is symlink, following link. File: /usr/bin/guppy_basecaller
[22:43:19 - check_file] Checking file on kgallman: /opt/ont/guppy/bin/guppy_basecaller
[22:43:19 - check_file] File exists! File: /opt/ont/guppy/bin/guppy_basecaller Size: 306759232
[22:43:19 - check_file] Checking file on kgallman: /home/kazengallman/git/pomoxis/venv/bin/activate
[22:43:19 - check_file] File exists! File: /home/kazengallman/git/pomoxis/venv/bin/activate Size: 2227
[22:43:19 - check_file] Checking file on kgallman: MinIonRun1/reads
[22:43:19 - check_file] File exists! File: MinIonRun1/reads Size: 12288
[22:43:19 - check_files] Finished checking that input files exist.
GPU status before
kgallman Sun May 16 22:43:20 2021 455.45.01
[0] GeForce GTX 1050 Ti | 49°C, 0 % | 232 / 4036 MB | gdm(35M) kazengallman(145M) kazengallman(32M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M)
[22:43:39 - pick_gpu] SGE_HGR_gpu was not set, setting GPU to 0 based on memory and utilization
Runnning on host kgallman GPU 0
ONT Guppy basecalling software version 4.5.4+66c1a7753
config file: /opt/ont/guppy/data/dna_r9.4.1_450bps_hac.cfg
model file: /opt/ont/guppy/data/template_r9.4.1_450bps_hac.jsn
input path: MinIonRun1/reads
save path: MinIonRun1/guppy
chunk size: 2000
chunks per runner: 512
minimum qscore: 9
records per file: 4000
num basecallers: 4
gpu device: cuda:0
kernel path:
runners per device: 4
Found 162 fast5 files to process.
Init time: 528 ms
0% 10 20 30 40 50 60 70 80 90 100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************
Caller time: 37127 ms, Samples called: 14237088, samples/s: 383470
Finishing up any open output files.
Basecalling completed successfully.
gpustat after
kgallman Sun May 16 22:44:20 2021 455.45.01
[0] GeForce GTX 1050 Ti | 56°C, 56 % | 227 / 4036 MB | gdm(35M) kazengallman(133M) kazengallman(38M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M) kazengallman(1M)
Combining the following fastq files into MinIonRun1/guppy/basecalls.fastq.gz
Here is the snakemake log before failing
sleep 5
set +u; source /home/kazengallman/git/pomoxis/venv/bin/activate; set -u;
echo "Combining the following fastq files into MinIonRun1/guppy/basecalls.fastq.gz" >> MinIonRun1/guppy.log
ls MinIonRun1/guppy/*.fastq >> MinIonRun1/guppy.log
cat MinIonRun1/guppy/*.fastq | bgzip -@ 4 -c >MinIonRun1/guppy/basecalls.fastq.gz 2>> MinIonRun1/guppy.log
rm MinIonRun1/guppy/*.fastq
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Removing output files of failed job basecall_guppy since they might be corrupted:
MinIonRun1/guppy/sequencing_summary.txt
Job failed, going on with independent jobs.
Exiting because a job execution failed. Look above for error message
Complete log: /home/kazengallman/Desktop/katuali-test/katuali/test/.snakemake/log/2021-05-16T224319.536460.snakemake.log
So overall I am just confused what is causing it to fail, since unless im mistaken it does not exactly show the command that it failed on (I am assuming the command that it broke on is after the rm MinIonRun1/guppy/*.fastq)