cylondata/cylon

build: Cylon Summit recipe is failing

Opened this issue · 10 comments

AymenFJA commented

Hello all,

We were trying to deploy Cylon on Summit following this recipe: https://github.com/cylondata/cylon/tree/main/summit (README). We tried two modes of installation on Summit:

  1. Following the recipe mentioned above and using compute nodes and the Cylon did not finish as it was failing with dependency installation.
  2. We switched to the login node, and we were able to install Cylon, and it was faster. We exported the necessary arrow and build paths. Yet we keep getting this error:
    (We ran a test from a compute node here using jsrun as recommended)
    image

and this
image

Any help or idea is appreciated it.

nirandaperera commented
okilic1 commented

Hi @nirandaperera.

Which library path are you talking I tried with CYLON/lib, but I don't think it is working

image
mstaylor commented

I have run across a similar issue on Amazon Linux and it is related to cmake not finding openMPI. Niranda may have another suggestion but perhaps try installing from a OS repo such as yum or apt or build openmpi from source. I'm doing the same in parallel :-)

nirandaperera commented

Hi @okilic1,
I think we are talking about two separate issues here. @AymenFJA's initial issue was related to running cylon after compilation. There, you need to add <cylon install dir>/lib path to the LD_LIBRARY_PATH.

I believe you are referring to building cylon. Do you have OpenMPI installed in your system? If not, you can either install it locally, or install it using conda. It should resolve the compilation problem.

laszewsk commented

Can documentation for this be improved. Do code changes need to be done?

arupcsedu commented

Is this issue producing again? It should be fixed after executing
export LD_LIBRARY_PATH=$BUILD_PATH/arrow/install/lib64:$BUILD_PATH/glog/install/lib64:$BUILD_PATH/lib64:$BUILD_PATH/lib:$LD_LIBRARY_PATH

This is already in the documentation

AymenFJA commented

@arupcsedu we tried this approach before opening the ticket, and it did not work. Did you test from your side recently with this approach on Summit? If you did, then happy to give it a try again. Maybe something has changed.

arupcsedu commented

@AymenFJA , if you have access to the summit, let's have a joint session to make cylon work. The steps by steps guidelines mentioned in the https://github.com/cylondata/cylon/tree/main/summit, was working before in summit.

AymenFJA commented

@arupcsedu Unfortunately I have no access.

laszewsk commented

@arupcsedu we all wait for approval.