Possible error due to accessing files in S3 bucket

[Pharmacogenetecist]

Description of the bug

I am using BAM files.
I think this error is arising from some kind of read/write permissions to S3. My S3 bucket is mounted via Goofys and I have no issues reading writing to it via my ec2 instance. When I copy the files to my local directory and run them it works fine.

When the files are in S3 I get the following error:
-[nf-core/hlatyping] Pipeline completed with errors-
Error executing process > 'NFCORE_HLATYPING:HLATYPING:SAMTOOLS_COLLATEFASTQ (CPM00017472_T1)' Caused by:
  Process NFCORE_HLATYPING:HLATYPING:SAMTOOLS_COLLATEFASTQ (CPM00017472_T1) terminated with an error exit status (1) Command executed:   samtools collate
      
      --threads 2
      -O
      CPM00017472-B-D_20210211_CES_Clinical3.0.bam
      . |   samtools fastq
      
      --threads 2
      -1 CPM00017472_T1_1.fq.gz
      -2 CPM00017472_T1_2.fq.gz
      -0 CPM00017472_T1_other.fq.gz
      -s CPM00017472_T1_singleton.fq.gz   cat <<-END_VERSIONS > versions.yml
  "NFCORE_HLATYPING:HLATYPING:SAMTOOLS_COLLATEFASTQ":
      samtools: $(echo $(samtools --version 2>&1) | sed 's/^.samtools //; s/Using.$//')
  END_VERSIONS Command exit status:
  1 Command output:
  (empty) Command error:
  [E::hts_open_format] Failed to open file "CPM00017472-B-D_20210211_CES_Clinical3.0.bam" : Permission denied
  samtools collate: Cannot open input file "CPM00017472-B-D_20210211_CES_Clinical3.0.bam": Permission denied
  Failed to read header for "-" Work dir:
  /home/ec2-user/DATA/NFLOW_HLA_PIPELINE/MISC_030923/work/e5/9f912ae15d1b78ecb897316983200b Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

Command used and terminal output

nextflow run nf-core/hlatyping --input samplesheet.csv --outdir in_myec2_instance --genome GRCh37 -profile docker

Relevant files

No response

System information

No response

[ggabernet]

Hi @Pharmacogenetecist , thanks I do not have experience with Goofys, but have you tried specifying the full paths to the files in the S3 bucket?
Otherwise you can maybe ask directly in the Nextfow slack, there's a channel for AWS issues, as it seems to be generic and not pipeline-related.

[Pharmacogenetecist]

Hi @Pharmacogenetecist , thanks I do not have experience with Goofys, but have you tried specifying the full paths to the files in the S3 bucket? Otherwise you can maybe ask directly in the Nextfow slack, there's a channel for AWS issues, as it seems to be generic and not pipeline-related.

Thanks, I ended up running the pipeline locally . Thanks