Methods install scripts

[foolnotion]

Hi,

I wanted to point out some issues with the environment/install scripts:

Bash scripts

1. The install scripts should not require elevated privileges ("sudo"). anaconda/miniconda is normally installed in an unprivileged location (home folder), so it shouldn't be necessary to do anything as root. This seems to trip some things up in some cases.

2. Some rudimentary way to check if a script succeeded would be useful. I would also redirect the compile output to a log file.

failed=()

# install all methods
for install_file in $(ls *.sh) ; do
    echo "vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv"
    echo "Running $install_file"
    echo "////////////////////////////////////////////////////////////////////////////////"

    bash $install_file

    if [ $? -gt 0 ]
    then
        failed+=($install_file)
    fi

    echo "////////////////////////////////////////////////////////////////////////////////"
    echo "Finished $install_file"
    echo "^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^"
done
echo "failed: ${failed}"

3. Running an install script again should probably clean the old src folders.

Conda environment

• pkg-config needs to be added to environment.yml, otherwise build scripts relying on it will fail (e.g. GP-GOMEA)

• cxxopts is not necessary (operon only needs it for the command-line program, not for the python module)

• on my ubuntu machine (16.04) , ellyn installs but fails to run
  nevermind, it's not supposed to be called directly.

My conda info:

$ conda info

     active environment : srbench
    active env location : /home/bogdb/miniconda3/envs/srbench
            shell level : 2
       user config file : /home/bogdb/.condarc
 populated config files : 
          conda version : 4.10.1
    conda-build version : not installed
         python version : 3.9.1.final.0
       virtual packages : __linux=5.11.13=0
                          __glibc=2.23=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/bogdb/miniconda3  (writable)
      conda av data dir : /home/bogdb/miniconda3/etc/conda
  conda av metadata url : https://repo.anaconda.com/pkgs/main
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/bogdb/miniconda3/pkgs
                          /home/bogdb/.conda/pkgs
       envs directories : /home/bogdb/miniconda3/envs
                          /home/bogdb/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.1 requests/2.25.1 CPython/3.9.1 Linux/5.11.13 ubuntu/16.04.7 glibc/2.23
                UID:GID : 1001:1001
             netrc file : None
           offline mode : False

[lacava]

all sounds good, thanks.

I have to revisit, but sudo was necessary for the GP-GOMEA install. Do you have a suggestion for https://github.com/EpistasisLab/srbench/blob/master/experiment/methods/src/gpgomea_install.sh to get around sudo make?

[foolnotion]

I just tested again and had no issue installing without sudo, a simple make install works if the install destination is writable, mine is under ~/miniconda3/envs/srbench/lib/python3.9/site-packages/pyGPGOMEA so it does not need any special privileges.

[lacava]

I made most of the suggested changes in 727c933 . The github actions workflow passes fine with all sudos removed. Locally, for operon, I am getting this error:

[ 51%] Built target operon
[100%] Built target pyoperon
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /home/bill/anaconda3/envs/srbench/lib/python3.9/site-packages/operon
CMake Error at cmake_install.cmake:54 (file):
  file INSTALL cannot set permissions on
  "/home/bill/anaconda3/envs/srbench/lib/python3.9/site-packages/operon":
  Operation not permitted.

Other notes:

pkg-config needs to be added to environment.yml, otherwise build scripts relying on it will fail (e.g. GP-GOMEA)

I'm not sure about this since the build scripts are succeeding for GP-GOMEA, using miniconda and the current environment (e.g. https://github.com/EpistasisLab/srbench/runs/2374405169?check_suite_focus=true)

cxxopts is not necessary (operon only needs it for the command-line program, not for the python module)

removed in b217529

[foolnotion]

If you had previously installed operon with sudo make install, the operon folder inside site-packages will belong to root, therefore giving the permissions error. A sudo rm -rf /home/bill/anaconda3/envs/srbench/lib/python3.9/site-packages/operon will probably fix it.

Regarding pkg-config, the important thing is that pkg-config --cflags python3 returns the right thing, since it is needed in the GP-GOMEA build. Adding pkg-config to the environment, therefore enabling it to find all the other packages installed in the env, is the safest way to do it. If it works without it i'd hazard to guess it's only by some happy coincidence :).

[lacava]

two issues:

• ITEA (@folivetti) requires sudo within the curl " " | sh step installing haskell
• after adding pkg-config dependency I get an error with ITEA: https://github.com/EpistasisLab/srbench/runs/2384207661?check_suite_focus=true#step:7:171

[folivetti]

Regarding 1, installing Haskell stack will only require sudo if one of the requirements are not installed by default. From the top of my head it will require GMP and maybe ffilib.

The second problem is also related to the lack of sudo, it needs to have these libraries installed

GSL
BLAS
LAPACK

[folivetti]

I think you could add the following conda packages in the environment.yml file:

gmp, libffi, gsl, libblas, liblapack

and everything should work (including not requiring a sudo)

[lacava]

I believe all of this is resolved now, let me know if you have additional ideas for the install script!