Nesvilab/FragPipe

Running fragpipe headless can't recognize gcc libraries

Closed this issue · 14 comments

klannieha commented

Hi,
I am trying to run FragPipe DIA_SpecLib_Quant headless on a HPC slurm system. I have loaded different C++ modules before running FragPipe, but the pipeline does not seem to recognize the C++ versions when DIANN is executed. Is there a specific gcc version that i should load? I have tried loading gcc versions 7, 8, 9, 11, and none of them seem to be recognized. Let met know!
slurm-4921246.txt

Best
Annie

fcyu commented

Hi Annie,

You can run ldd diann to show which std libraries are missing. Following is an example

$ ldd diann-1.8.1.8
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by ./diann-1.8.1.8)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtimsdata.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtimsdata.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtimsdata.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtimsdata.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libm.so.6: version `GLIBC_2.23' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libtorch_cpu.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libc10.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libc10.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libc10.so)
./diann-1.8.1.8: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /storage/yufe/tools/FragPipe-19.1/fragpipe/tools/diann/1.8.2_beta_8/linux/./libc10.so)
        linux-vdso.so.1 =>  (0x00007ffe9fdd6000)
....

Best,

Fengchao

klannieha commented

Thanks Fengchao,
I have no loaded a StdEnv and checked the libraries:

[annieha@gra-login2 fragpipe]$ ldd ~/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig
linux-vdso.so.1 (0x00007fff16b40000)
libdl.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libdl.so.2 (0x00002b102d6d1000)
libpthread.so.0 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libpthread.so.0 (0x00002b102d6d6000)
libtimsdata.so => /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so (0x00002b102d6f6000)
libtorch_cpu.so => /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so (0x00002b102e93e000)
libc10.so => /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so (0x00002b1047bd3000)
libstdc++.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libstdc++.so.6 (0x00002b1047c7b000)
libm.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libm.so.6 (0x00002b1047f1f000)
libgcc_s.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgcc_s.so.1 (0x00002b1048062000)
libc.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libc.so.6 (0x00002b104807d000)
/lib64/ld-linux-x86-64.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/ld-linux-x86-64.so.2 (0x00002b102c587000)
libgomp.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgomp.so.1 (0x00002b104823b000)
librt.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/librt.so.1 (0x00002b1048280000)
libgomp-52f2fd74.so.1 => /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libgomp-52f2fd74.so.1 (0x00002b104828a000)

But when I ran ~/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig it still gave me the same errors as above...

Annie

fcyu commented

Hi Annie,

Maybe your login node (gra-login2) and the node actually running FragPipe have different configuration. Your log file shows that there are std libraries missing:

/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig --lib /local/annieha.4921246.0/spectraRT.tsv --predict --threads 8 --strip-unknown-mods --mod TMT,229.1629 --predict-n-frag 100
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.11' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libm.so.6: version `GLIBC_2.23' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)
/home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)

Best,

Fengchao

klannieha commented

I have tried just running /home/annieha/bin/fragpipe/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8-orig on my login node after I checked with ldd and it still gave me the same error even when ldd showed me the libraries are loaded..

fcyu commented

You only ran ldd on your login node. You need to run ldd on the actually running node, I think. You probably need to talk to your IT guy regarding the cluster configuration.

Best,

Fengchao

klannieha commented

I will try that out, Thanks a lot!

d0minicO commented

I had the exact same issue, I am running FragPipe 19.1 on a linux HPC in headless mode. Runs work fine with MSBooster turned off. With MSBooster turned on, however, I get these missing library errors:

/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.8' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.22' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.8' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.8' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.11' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.22' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libm.so.6: version GLIBC_2.27' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libm.so.6: version GLIBC_2.23' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.21' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version CXXABI_1.3.9' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)
/home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.22' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so) /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8: /lib64/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so)

The solution that worked for me is to patch the elf header on diann-1.8.1.8 to add the rpath containing the necessary lib.so files. The files in question are actually already included with fragpipe. MSBooster runs just fine now.

I achieved this by running:
patchelf --add-rpath /home/dowens/fragpipe/tools/diann_so/ /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8

I suppose an alternative solution could be to transfer over the .so files that are located within fragpipe/tools/diann_so/ to the /fragpipe/tools/diann/1.8.2_beta_8/linux/ directory that contains the other .so files, that the executable IS finding okay. For some reason the executable is not searching fragpipe/tools/diann_so/ and is using default ones located elsewhere on the system which causes the issue. I haven't tested if this would work yet. But I have made 4 successful runs now after patching diann as I described above.

  • 👍2
guoci commented

We are currently using LD_PRELOAD :
https://github.com/Nesvilab/FragPipe/blob/master/MSFragger-GUI/src/com/dmtavt/fragpipe/cmd/CmdDiann.java#L247
But it seems for some reason, it doesn't work on some computers.
Can you show me the output of ldd /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8?

  • 👍1
  • 😕1
d0minicO commented

Sure, here it is:

ldd /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8
linux-vdso.so.1 (0x00007ffecbcb9000)
libdl.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libdl.so.2 (0x00007fa4b49ea000)
libpthread.so.0 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libpthread.so.0 (0x00007fa4b49ca000)
libtimsdata.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so (0x00007fa4b3782000)
libtorch_cpu.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so (0x00007fa49a4ed000)
libc10.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so (0x00007fa49a447000)
libstdc++.so.6 => /home/dowens/fragpipe/tools/diann_so/libstdc++.so.6 (0x00007fa49a07a000)
libm.so.6 => /home/dowens/fragpipe/tools/diann_so/libm.so.6 (0x00007fa499d39000)
libgcc_s.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgcc_s.so.1 (0x00007fa499d1e000)
libc.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libc.so.6 (0x00007fa499b64000)
/lib64/ld-linux-x86-64.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/ld-linux-x86-64.so.2 (0x00007fa4b5b1b000)
libgomp.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgomp.so.1 (0x00007fa499b1d000)
librt.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/librt.so.1 (0x00007fa499b13000)
libgomp-52f2fd74.so.1 => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libgomp-52f2fd74.so.1 (0x00007fa4998e0000)

  • 👍1
guoci commented

@d0minicO Did you run ldd after you did patchelf or before?

d0minicO commented

Hi that output is after patching. I didn't save the output of ldd from before patching... Here is the output from running ldd on a freshly unzipped never run file, in case it helps!

ldd /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8
linux-vdso.so.1 (0x00007ffc71653000)
libdl.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libdl.so.2 (0x00007f0e17950000)
libpthread.so.0 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libpthread.so.0 (0x00007f0e17930000)
libtimsdata.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so (0x00007f0e166e8000)
libtorch_cpu.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so (0x00007f0dfd453000)
libc10.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so (0x00007f0dfd3ad000)
libstdc++.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libstdc++.so.6 (0x00007f0dfd109000)
libm.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libm.so.6 (0x00007f0dfcfc4000)
libgcc_s.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgcc_s.so.1 (0x00007f0dfcfa9000)
libc.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libc.so.6 (0x00007f0dfcdef000)
/lib64/ld-linux-x86-64.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/ld-linux-x86-64.so.2 (0x00007f0e18a74000)
libgomp.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgomp.so.1 (0x00007f0dfcda8000)
librt.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/librt.so.1 (0x00007f0dfcd9e000)
libgomp-52f2fd74.so.1 => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libgomp-52f2fd74.so.1 (0x00007f0dfcb6b000)

  • 👍1
guoci commented

@d0minicO The error message shows diann is using
/lib64/libstdc++.so.6
but your ldd output shows diann is using
/cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libstdc++.so.6
Are you running ldd on the same machine as the one you are running FragPipe? It seems that 2 different filesystems are used.

d0minicO commented

@guoci Yes you are right, my apologies, I forgot that I need to run ldd on the compute node. The ldd results in the comments above were run on the login node! Here are the outputs of ldd from the compute node.

PATCHED:
ldd /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8
linux-vdso.so.1 (0x00007ffc50b33000)
libdl.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libdl.so.2 (0x00002b095b727000)
libpthread.so.0 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libpthread.so.0 (0x00002b095b72c000)
libtimsdata.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so (0x00002b095b74c000)
libtorch_cpu.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so (0x00002b095c994000)
libc10.so => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so (0x00002b0975c29000)
libstdc++.so.6 => /home/dowens/fragpipe/tools/diann_so/libstdc++.so.6 (0x00002b0975ccf000)
libm.so.6 => /home/dowens/fragpipe/tools/diann_so/libm.so.6 (0x00002b097609e000)
libgcc_s.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgcc_s.so.1 (0x00002b09763dd000)
libc.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libc.so.6 (0x00002b09763f8000)
/lib64/ld-linux-x86-64.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/ld-linux-x86-64.so.2 (0x00002b095a5d0000)
libgomp.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgomp.so.1 (0x00002b09765b6000)
librt.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/librt.so.1 (0x00002b09765f9000)
libgomp-52f2fd74.so.1 => /home/dowens/fragpipe/tools/diann/1.8.2_beta_8/linux/libgomp-52f2fd74.so.1 (0x00002b0976603000)

NOT PATCHED:
ldd /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/diann-1.8.1.8
linux-vdso.so.1 (0x00007ffcbf071000)
libdl.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libdl.so.2 (0x00002b949e2bb000)
libpthread.so.0 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libpthread.so.0 (0x00002b949e2c0000)
libtimsdata.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libtimsdata.so (0x00002b949e2e0000)
libtorch_cpu.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libtorch_cpu.so (0x00002b949f528000)
libc10.so => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libc10.so (0x00002b94b87bd000)
libstdc++.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libstdc++.so.6 (0x00002b94b8863000)
libm.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libm.so.6 (0x00002b94b8b09000)
libgcc_s.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgcc_s.so.1 (0x00002b94b8c4c000)
libc.so.6 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/libc.so.6 (0x00002b94b8c67000)
/lib64/ld-linux-x86-64.so.2 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/ld-linux-x86-64.so.2 (0x00002b949d171000)
libgomp.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/usr/lib/gcc/x86_64-pc-linux-gnu/11.3.0/libgomp.so.1 (0x00002b94b8e25000)
librt.so.1 => /cvmfs/soft.computecanada.ca/gentoo/2020/lib64/librt.so.1 (0x00002b94b8e68000)
libgomp-52f2fd74.so.1 => /home/dowens/fragpipe_new/fragpipe/tools/diann/1.8.2_beta_8/linux/libgomp-52f2fd74.so.1 (0x00002b94b8e72000)

  • 👍1
guoci commented

@d0minicO Thanks for the information. It seems that ldd cannot be relied upon to check for missing shared libraries. I will fix it in the next release.

  • 👍1