NERSC/buildtest-nersc

[Bug]: superlu test failing from E4S Testsuite

Closed this issue · 3 comments

CDASH Build

https://my.cdash.org/test/63278701

Link to buildspec file

https://github.com/buildtesters/buildtest-nersc/blob/devel/buildspecs/e4s/E4S-Testsuite/perlmutter/22.05/superlu.yml

Please describe the issue?

see E4S-Project/testsuite#38

Relevant log output

superlu : b6f4pqc
Cleaning /global/cfs/cdirs/m3503/buildtest/runs/perlmutter_check/2022-09-28/perlmutter.slurm.regular/superlu/superlu_e4s_testsuite_22.05/a16c8446/stage/testsuite/validation_tests/superlu
---CLEANUP LOG---
Compiling /global/cfs/cdirs/m3503/buildtest/runs/perlmutter_check/2022-09-28/perlmutter.slurm.regular/superlu/superlu_e4s_testsuite_22.05/a16c8446/stage/testsuite/validation_tests/superlu
---COMPILE LOG---
Skipping load: Environment already setup
+ cc -g ./c_sample.c -I/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/include/ -L/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib64 -L/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib -L/lib -lsuperlu -lopenblas -lm -o c_sample
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(util.c.o): in function `ifill':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/util.c:392: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/util.c:392: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(memory.c.o): in function `intCalloc':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/memory.c:161: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(csnode_dfs.c.o): in function `csnode_dfs':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/csnode_dfs.c:111: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(cmemory.c.o): in function `complexCalloc':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/cmemory.c:695: undefined reference to `__cray_dset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(cutil.c.o): in function `cfill':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/cutil.c:398: undefined reference to `__cray_dset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(get_perm_c.c.o): in function `getata':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:106: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:140: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:173: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(get_perm_c.c.o): in function `at_plus_a':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:240: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:269: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(get_perm_c.c.o):/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/get_perm_c.c:306: more undefined references to `__cray_sset_detect' follow
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `colamd_set_defaults':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:960: undefined reference to `__cray_dset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `symamd':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:1022: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `colamd_set_defaults':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:960: undefined reference to `__cray_dset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `symamd':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:1100: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:1226: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `colamd':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:1357: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `colamd_set_defaults':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:960: undefined reference to `__cray_dset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(colamd.c.o): in function `init_scoring':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/colamd.c:1940: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(csp_blas2.c.o): in function `sp_cgemv':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/csp_blas2.c:518: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(sp_coletree.c.o): in function `mxCallocInt':
/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/sp_coletree.c:69: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/sp_coletree.c:69: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/sp_coletree.c:69: undefined reference to `__cray_sset_detect'
/usr/bin/ld: /global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/cce-13.0.2/superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/lib/libsuperlu.a(sp_coletree.c.o):/global/common/software/spackecp/perlmutter/e4s-22.05/spack/var/spack/stage/spack-stage-superlu-5.3.0-b6f4pqcphndaspnzvlpjog6wxjhib4ff/spack-src/SRC/sp_coletree.c:190: more undefined references to `__cray_sset_detect' follow
collect2: error: ld returned 1 exit status
Compile failed

@shahzebsiddiqui
This error occurs when the test targets the cce-built version of superlu instead of gcc. Since we are currently only targeting gcc builds (until otherwise specified) I have changed the settings file for cray to always specify %gcc which should address this and similar issues (E4S-Project/testsuite@67560a5). Are there supposed to be non-gcc builds of different products accessible in the default e4s module? I thought different modules for different compilers was standard.

i see now, in that case we should tailor our E4S Testsuite to only run on gcc specs, lets try running the perlmutter e4s tests and then see if the test pass, if not we may need to fix the test.

oh good this test is now passing https://my.cdash.org/test/67656611

Screen Shot 2022-11-22 at 11 57 27 AM