Error during AssignCellsToSamples

[Thapeachydude]

Hi,

I'm trying to demultiplex ≈ 10 samples in a pooled 10x experiment using WES / WGS data as reference. Following the documentation at demuxafy I stumpled upon dropulation.
I've run the bam annotation, but during AssignCellsToSamples I'm encountering an error, see below:

INFO	2023-12-20 01:27:44	SNPUMIBasePileupIterator	Processed    15,000,000 records.  Elapsed time: 00:00:47s.  Time for last 1,000,000:    3s.  Last read position: chr1:26,475,346.  Last read name: LH00289:6:22733LLT3:2:2228:0:4320748
INFO	2023-12-20 01:27:47	SNPUMIBasePileupIterator	Processed    16,000,000 records.  Elapsed time: 00:00:50s.  Time for last 1,000,000:    2s.  Last read position: chr1:27,914,141.  Last read name: LH00289:7:22733TLT3:8:2165:34268:2442
INFO	2023-12-20 01:27:50	SNPUMIBasePileupIterator	Processed    17,000,000 records.  Elapsed time: 00:00:53s.  Time for last 1,000,000:    2s.  Last read position: chr1:29,089,658.  Last read name: A00730:829:HWNMJDSX5:3:2541:31358:25175
[Wed Dec 20 01:27:53 UTC 2023] org.broadinstitute.dropseqrna.barnyard.digitalallelecounts.sampleassignment.AssignCellsToSamples done. Elapsed time: 3.04 minutes.
Runtime.totalMemory()=2685403136
Exception in thread "main" htsjdk.samtools.util.RuntimeIOException: java.nio.file.NoSuchFileException: /scratch/tmp.40939362.mpohly/mpohly/sortingcollection.10853389501593454727.tmp
	at htsjdk.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:267)
	at htsjdk.samtools.util.SortingCollection.add(SortingCollection.java:182)
	at org.broadinstitute.dropseqrna.utils.SortingIteratorFactory.create(SortingIteratorFactory.java:67)
	at org.broadinstitute.dropseqrna.utils.readiterators.SamRecordSortingIteratorFactory.create(SamRecordSortingIteratorFactory.java:57)
	at org.broadinstitute.dropseqrna.barnyard.digitalallelecounts.SNPUMIBasePileupIterator.<init>(SNPUMIBasePileupIterator.java:142)
	at org.broadinstitute.dropseqrna.barnyard.digitalallelecounts.sampleassignment.AssignCellsToSamples.prepareIterator(AssignCellsToSamples.java:916)
	at org.broadinstitute.dropseqrna.barnyard.digitalallelecounts.sampleassignment.AssignCellsToSamples.doWork(AssignCellsToSamples.java:270)
	at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:308)
	at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)
	at org.broadinstitute.dropseqrna.cmdline.DropSeqMain.main(DropSeqMain.java:42)
Caused by: java.nio.file.NoSuchFileException: /scratch/tmp.40939362.mpohly/mpohly/sortingcollection.10853389501593454727.tmp
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
	at java.base/java.nio.file.Files.createFile(Files.java:648)
	at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:137)
	at java.base/java.nio.file.TempFileHelper.createTempFile(TempFileHelper.java:160)
	at java.base/java.nio.file.Files.createTempFile(Files.java:868)
	at htsjdk.samtools.util.IOUtil.newTempPath(IOUtil.java:396)
	at htsjdk.samtools.util.SortingCollection.newTempFile(SortingCollection.java:278)
	at htsjdk.samtools.util.SortingCollection.spillToDisk(SortingCollection.java:249)
	... 9 more

I'm running this on a HPC with sufficient memory and using the singularity image provided by demuxafy. I'm aware that this is not necessarily the default way to install the tool, so if you have any suggestions please let me know.

Many thanks and best,
M

[alecw]

Does the directory /scratch/tmp.40939362.mpohly/mpohly exist on the host on which this process ran? Make sure you set TMP_DIR command-line option (and/or java.io.tmpdir Java property) to a directory that exists.

[drneavin]

I would agree with @alecw interpretation that it can't find the /scratch/tmp.40939362.mpohly/mpohly/sortingcollection.10853389501593454727.tmp file which I think is because you need to bind the /scratch/ directory for the singularity command. I'll add a comment about this to the Demuxafy docs.

[Thapeachydude]

Yes, that seemed to do the trick. Thanks!