ankane/lightgbm-ruby

Build native libs instead of installing a bunch of potentially incompatible or broken ones

Closed this issue · 4 comments

barracuda156 commented

@ankane Could you please take a look?

Currently the gem seems to simply install arbitrary pre-built libs:

/opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.arm64.dylib
/opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.dylib
/opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.so
/opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.dll

This is a problematic way:

  1. This will fail on every arch which does not have a pre-built lib.
  2. This will likely not work correctly on specific OS versions even on supported archs.
  3. This leaves the gem broken due to hardcoded dependencies.
  4. It is desirable not to install unnecessary stuff: there is no reason for someone to want a Windows dll on macOS or BSD, I believe.

Instead, if the library is built from source, that gonna fix all those issues.

P. S. To the point 3, pre-built dylib has an odd hardcoded path for libomp:

Could not open /opt/homebrew/opt/libomp/lib/libomp.dylib: Error opening or reading file (referenced from /opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.arm64.dylib)
Could not open /usr/local/opt/libomp/lib/libomp.dylib: Error opening or reading file (referenced from /opt/local/lib/ruby3.3/gems/3.3.0/gems/lightgbm-0.3.2/vendor/lib_lightgbm.dylib)
--->  Found 2 broken files, matching files to ports
--->  Found 1 broken port, determining rebuild order
You can always run 'port rev-upgrade' again to fix errors.
The following ports will be rebuilt: rb33-lightgbm @0.3.2

This is wrong for three reasons:

  1. libomp is a clang-specific library, so gonna fail with GCC.
  2. libomp is an LLVM-specific library, so may not be available with Apple clang.
  3. It uses an odd hardcoded path, so this will fail even when libomp is available, but the path differs.
ankane commented

Hi @barracuda156, I think the current approach is better for most users, but you can build LightGBM from source and set LightGBM.ffi_lib if needed.

  • 😕1
barracuda156 commented

@ankane Ok, I have made a port for a standalone LightGBM now, once PR merged, it will be in MacPorts.

Could you say how do I set the variable for Ruby, so that it does not install any bundled libs but instead uses an external one?

barracuda156 commented

@ankane Any update on this?