snakemake/snakemake-wrappers

STAR wrapper - No output/error if CPUs don't support SIMD

KuechlerO opened this issue · 7 comments

Snakemake version
The described bug is independent of the used Snakemake version.

Describe the bug
I've already reported the bug here: alexdobin/STAR#2030

But basically the with the wrapper installed STAR executable is a bash script that is optimizing for the usage of SIMD architecture platforms. Yet if no SIMD or the wrong type of SIMD (i.e. sse and sse2) is supported, the script terminates without any output.
Here's the script that gets set up in the wrapper environment:

#------ .snakemake/conda/<hash>/bin/STAR --------------

#!/bin/bash

SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
DIR=$SCRIPT_DIR
BASE=${DIR}/$(basename "$0")
CMDARGS="$@"

function test_and_run () {
        if grep -q "$1" /proc/cpuinfo && [ -x "${BASE}-$1" ]; then
                cmd="${BASE}-$1"
                shift
                # echo "${cmd}" "$@"
                "${cmd}" "$@"
                exit
        fi
}

for SIMD in avx2 avx sse4.1 ssse3 sse3 sse2 sse ; do test_and_run ${SIMD} "$@" ; done

# fallback to plain option

Logs
No logs are generated

Minimal example
lscpu | grep -E 'avx2|avx|sse4.1|ssse3|sse3|sse2|sse'
That reveals that my server supports sse and sse2.
Yet with ls .snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR* it is revealed that neither sse nor sse2 is supported in the wrapper's STAR version. Hence the wrapper terminates without any output nor error.

$ ls .snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR*

# output:
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-avx
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-avx2
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-avx
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-avx2
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-plain
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-sse3
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-sse4.1
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STARlong-ssse3
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-plain
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-sse3
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-sse4.1
.snakemake/conda/46ba9ce3db1a5a7036564fea65b30af6_/bin/STAR-ssse3

# Thus, no executable is selected

Can you run STAR on your system without the wrapper?

Can you run STAR on your system without the wrapper?

Yeah STAR is not the problem: If I download the precompiled version from Alex as described here, I can call STAR.
alexdobin/STAR#2030

The wrapper itself is also not the problem.
It's the Conda package, which is the problem.

The last line from that file doesn't seem to make it into the final STAR-executable script in the BioConda-package:
https://github.com/bioconda/bioconda-recipes/blob/master/recipes/star/patches/0003-create-simd-dispatch.patch

+$BASE-plain "$@" that line is missing in the conda environment set-up by the wrapper.

I'm not familiar with Bioconda recepies, so right now I'm not able to fix it/making a pull request.

Also

mamba install -c bioconda star
STAR --version

that works.

Pinging @pettyalex, since it seems the author of the bioconda star recipe.

Additional note:
I have 2 servers with CentOS7 installed. Both of them are showing the same behaviour as explained above.

My 3. server has RockyLinux installed. Here no problem with the wrapper is observable. (Though this server also support ssse3, avx and avx2)

From what I can see, it is an issue with the bioconda recipe, no?

From what I can see, it is an issue with the bioconda recipe, no?

Yes, I think so to.
In that case we can close the issue if you want to @fgvieira