onekey-sec/unblob

Unable to install unblob with pip in Ubuntu 24.04

jrmoore opened this issue · 7 comments

Describe the bug
Installation via pip fails when it reaches lief. I believe the problem is there, because failure occurs during its build process, but I am filing this report because unblob installation doesn't seem to rely in lief being installed through pip and tries to build it unconditionally (pip install lief works, installs 0.14.1 at the time of writing).

To Reproduce
Steps to reproduce the behavior:

  1. Create a new venv and enter its environment: python3 -m venv test && source test/bin/activate
  2. [Install lief manually: pip install lief]
  3. Attempt to install unblob with pip: pip install unblob
  4. See error
Error log
$ pip install unblob

DEPRECATION: Loading egg at /home/james/BinAnalysis/lib/python3.12/site-packages/binwalk-2.4.1+3acfb0c-py3.12.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation.. Discussion can be found at https://github.com/pypa/pip/issues/12330
Collecting unblob
  Using cached unblob-24.6.10-py3-none-any.whl.metadata (1.3 kB)
Collecting arpy<3.0.0,>=2.3.0 (from unblob)
  Using cached arpy-2.3.0-py3-none-any.whl
Collecting attrs>=23.1.0 (from unblob)
  Using cached attrs-23.2.0-py3-none-any.whl.metadata (9.5 kB)
Collecting click<9.0.0,>=8.1.7 (from unblob)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting cryptography<43.0,>=41.0 (from unblob)
  Using cached cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.3 kB)
Collecting dissect.cstruct<3.0,>=2.0 (from unblob)
  Using cached dissect.cstruct-2.0-py3-none-any.whl.metadata (6.0 kB)
Collecting jefferson<0.5.0,>=0.4.5 (from unblob)
  Using cached jefferson-0.4.5-py3-none-any.whl.metadata (1.6 kB)
Collecting lark<2.0.0,>=1.1.8 (from unblob)
  Using cached lark-1.1.9-py3-none-any.whl.metadata (1.9 kB)
Collecting lief<0.13.0,>=0.12.3 (from unblob)
  Using cached lief-0.12.3.zip (16.2 MB)
  Preparing metadata (setup.py) ... done
Collecting lz4<5.0.0,>=4.3.2 (from unblob)
  Using cached lz4-4.3.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
Collecting plotext<6.0,>=4.2.0 (from unblob)
  Using cached plotext-5.2.8-py3-none-any.whl.metadata (5.3 kB)
Collecting pluggy<2.0.0,>=1.3.0 (from unblob)
  Using cached pluggy-1.5.0-py3-none-any.whl.metadata (4.8 kB)
Collecting pyfatfs<2.0.0,>=1.0.5 (from unblob)
  Using cached pyfatfs-1.1.0-py3-none-any.whl.metadata (7.8 kB)
Collecting pyperscan<0.4.0,>=0.3.0 (from unblob)
  Using cached pyperscan-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.3 kB)
Collecting python-magic<0.5.0,>=0.4.27 (from unblob)
  Using cached python_magic-0.4.27-py2.py3-none-any.whl.metadata (5.8 kB)
Collecting rarfile<5.0,>=4.1 (from unblob)
  Using cached rarfile-4.2-py3-none-any.whl.metadata (4.4 kB)
Collecting rich<14.0.0,>=13.3.5 (from unblob)
  Using cached rich-13.7.1-py3-none-any.whl.metadata (18 kB)
Collecting structlog>=24.1.0 (from unblob)
  Using cached structlog-24.2.0-py3-none-any.whl.metadata (7.9 kB)
Collecting treelib<2.0.0,>=1.7.0 (from unblob)
  Using cached treelib-1.7.0-py3-none-any.whl.metadata (1.3 kB)
Collecting ubi-reader<0.9.0,>=0.8.9 (from unblob)
  Using cached ubi_reader-0.8.9-py3-none-any.whl.metadata (6.1 kB)
Collecting unblob-native<0.2.0,>=0.1.1 (from unblob)
  Using cached unblob_native-0.1.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.6 kB)
Collecting cffi>=1.12 (from cryptography<43.0,>=41.0->unblob)
  Using cached cffi-1.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting cstruct<6.0,>=5.2 (from jefferson<0.5.0,>=0.4.5->unblob)
  Using cached cstruct-5.3-py2.py3-none-any.whl.metadata (12 kB)
Collecting lzallright<0.3.0,>=0.2.1 (from jefferson<0.5.0,>=0.4.5->unblob)
  Using cached lzallright-0.2.4-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (946 bytes)
Collecting fs~=2.4 (from pyfatfs<2.0.0,>=1.0.5->unblob)
  Using cached fs-2.4.16-py2.py3-none-any.whl.metadata (6.3 kB)
Collecting markdown-it-py>=2.2.0 (from rich<14.0.0,>=13.3.5->unblob)
  Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich<14.0.0,>=13.3.5->unblob)
  Using cached pygments-2.18.0-py3-none-any.whl.metadata (2.5 kB)
Requirement already satisfied: six in /home/james/BinAnalysis/lib/python3.12/site-packages (from treelib<2.0.0,>=1.7.0->unblob) (1.16.0)
Collecting pycparser (from cffi>=1.12->cryptography<43.0,>=41.0->unblob)
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Collecting appdirs~=1.4.3 (from fs~=2.4->pyfatfs<2.0.0,>=1.0.5->unblob)
  Using cached appdirs-1.4.4-py2.py3-none-any.whl.metadata (9.0 kB)
Requirement already satisfied: setuptools in /home/james/BinAnalysis/lib/python3.12/site-packages (from fs~=2.4->pyfatfs<2.0.0,>=1.0.5->unblob) (70.0.0)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich<14.0.0,>=13.3.5->unblob)
  Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Using cached unblob-24.6.10-py3-none-any.whl (132 kB)
Using cached attrs-23.2.0-py3-none-any.whl (60 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl (3.9 MB)
Using cached dissect.cstruct-2.0-py3-none-any.whl (30 kB)
Using cached jefferson-0.4.5-py3-none-any.whl (8.8 kB)
Using cached lark-1.1.9-py3-none-any.whl (111 kB)
Using cached lz4-4.3.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Using cached plotext-5.2.8-py3-none-any.whl (60 kB)
Using cached pluggy-1.5.0-py3-none-any.whl (20 kB)
Using cached pyfatfs-1.1.0-py3-none-any.whl (37 kB)
Using cached pyperscan-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.2 MB)
Using cached python_magic-0.4.27-py2.py3-none-any.whl (13 kB)
Using cached rarfile-4.2-py3-none-any.whl (29 kB)
Using cached rich-13.7.1-py3-none-any.whl (240 kB)
Using cached structlog-24.2.0-py3-none-any.whl (65 kB)
Using cached treelib-1.7.0-py3-none-any.whl (18 kB)
Using cached ubi_reader-0.8.9-py3-none-any.whl (70 kB)
Using cached unblob_native-0.1.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
Using cached cffi-1.16.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (477 kB)
Using cached cstruct-5.3-py2.py3-none-any.whl (30 kB)
Using cached fs-2.4.16-py2.py3-none-any.whl (135 kB)
Using cached lzallright-0.2.4-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB)
Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Using cached pygments-2.18.0-py3-none-any.whl (1.2 MB)
Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: lief
  Building wheel for lief (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [383 lines of output]
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/setup.py:10: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
        from pkg_resources import get_distribution
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/setup.py:20: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        assert (LooseVersion(setuptools.__version__) >= LooseVersion(MIN_SETUPTOOLS_VERSION)), "LIEF requires a setuptools version '{}' or higher (pip install setuptools --upgrade)".format(MIN_SETUPTOOLS_VERSION)
      0.12.3
      running bdist_wheel
      running build
      running build_ext
      1.11.1
      Platform: Linux
      Wheel library: lief
      cmake /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19 -DLIEF_FORCE_API_EXPORTS=ON -DLIEF_PYTHON_API=on -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build -DPYTHON_EXECUTABLE=/home/james/BinAnalysis/bin/python -DLIEF_PYTHON_API=on -DLIEF_LOGGING_DEBUG=off -DCMAKE_BUILD_TYPE=Release
      -- The C compiler identification is GNU 13.2.0
      -- The CXX compiler identification is GNU 13.2.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- LIEF 0.12.3
      -- Caching: ON
      CMake Warning at CMakeLists.txt:71 (message):
        Can't find cache solution
      
      
      -- Enable JSON support
      -- Enable Frozen (C++14 support)
      -- CMAKE_CXX_FLAGS_RELEASE:           -O3 -DNDEBUG
      -- CMAKE_CXX_FLAGS:
      -- CMAKE_CXX_LINK_EXECUTABLE:         <CMAKE_CXX_COMPILER> <FLAGS> <CMAKE_CXX_LINK_FLAGS> <LINK_FLAGS> <OBJECTS> -o <TARGET> <LINK_LIBRARIES>
      -- CMAKE_CXX_LINK_FLAGS:
      -- CMAKE_EXE_LINKER_FLAGS:
      -- CMAKE_EXE_LINKER_FLAGS_RELEASE:
      -- CMAKE_SHARED_LINKER_FLAGS:
      -- CMAKE_SHARED_LINKER_FLAGS_RELEASE:
      -- CMAKE_CXX_LINK_LIBRARY_FILE_FLAG:
      -- CMAKE_LINK_LIBRARY_FILE_FLAG:
      -- CMAKE_LINK_INTERFACE_LIBRARIES:
      -- CMAKE_CXX_IMPLICIT_LINK_LIBRARIES: stdc++;m;gcc_s;gcc;c;gcc_s;gcc
      -- CMAKE_SYSTEM_PROCESSOR:            x86_64
      -- Performing Test C_SUPPORTS_WALL
      -- Performing Test C_SUPPORTS_WALL - Success
      -- Performing Test CXX_SUPPORTS_WALL
      -- Performing Test CXX_SUPPORTS_WALL - Success
      -- Performing Test C_SUPPORTS_WEXTRA
      -- Performing Test C_SUPPORTS_WEXTRA - Success
      -- Performing Test CXX_SUPPORTS_WEXTRA
      -- Performing Test CXX_SUPPORTS_WEXTRA - Success
      -- Performing Test C_SUPPORTS_WPEDANTIC
      -- Performing Test C_SUPPORTS_WPEDANTIC - Success
      -- Performing Test CXX_SUPPORTS_WPEDANTIC
      -- Performing Test CXX_SUPPORTS_WPEDANTIC - Success
      -- Performing Test C_SUPPORTS_NO_STACK_PROTECTOR
      -- Performing Test C_SUPPORTS_NO_STACK_PROTECTOR - Success
      -- Performing Test CXX_SUPPORTS_NO_STACK_PROTECTOR
      -- Performing Test CXX_SUPPORTS_NO_STACK_PROTECTOR - Success
      -- Performing Test C_SUPPORTS_OMIT_FRAME_POINTER
      -- Performing Test C_SUPPORTS_OMIT_FRAME_POINTER - Success
      -- Performing Test CXX_SUPPORTS_OMIT_FRAME_POINTER
      -- Performing Test CXX_SUPPORTS_OMIT_FRAME_POINTER - Success
      -- Performing Test C_SUPPORTS_NO_STRICT_ALIASING
      -- Performing Test C_SUPPORTS_NO_STRICT_ALIASING - Success
      -- Performing Test CXX_SUPPORTS_NO_STRICT_ALIASING
      -- Performing Test CXX_SUPPORTS_NO_STRICT_ALIASING - Success
      -- Performing Test C_SUPPORTS_EXCEPTION
      -- Performing Test C_SUPPORTS_EXCEPTION - Success
      -- Performing Test CXX_SUPPORTS_EXCEPTION
      -- Performing Test CXX_SUPPORTS_EXCEPTION - Success
      -- Performing Test C_SUPPORTS_NO_EXPANSION_TO_DEFINED
      -- Performing Test C_SUPPORTS_NO_EXPANSION_TO_DEFINED - Success
      -- Performing Test CXX_SUPPORTS_NO_EXPANSION_TO_DEFINED
      -- Performing Test CXX_SUPPORTS_NO_EXPANSION_TO_DEFINED - Success
      -- Performing Test C_SUPPORTS_ERR_RET_TYPE
      -- Performing Test C_SUPPORTS_ERR_RET_TYPE - Success
      -- Performing Test CXX_SUPPORTS_ERR_RET_TYPE
      -- Performing Test CXX_SUPPORTS_ERR_RET_TYPE - Success
      -- Performing Test C_SUPPORTS_DIAGNOSTICS_COLOR
      -- Performing Test C_SUPPORTS_DIAGNOSTICS_COLOR - Success
      -- Performing Test CXX_SUPPORTS_DIAGNOSTICS_COLOR
      -- Performing Test CXX_SUPPORTS_DIAGNOSTICS_COLOR - Success
      -- Performing Test C_SUPPORTS_COLOR_DIAGNOSTICS
      -- Performing Test C_SUPPORTS_COLOR_DIAGNOSTICS - Failed
      -- Performing Test CXX_SUPPORTS_COLOR_DIAGNOSTICS
      -- Performing Test CXX_SUPPORTS_COLOR_DIAGNOSTICS - Failed
      -- Configuration Types:
      -- Build Types:         Release
      CMake Deprecation Warning at api/python/CMakeLists.txt:1 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      CMake Warning (dev) at cmake/FindPythonLibsNew.cmake:98 (find_package):
        Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
        are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
        the cmake_policy command to set the policy and suppress this warning.
      
      Call Stack (most recent call first):
        api/python/CMakeLists.txt:35 (find_package)
      This warning is for project developers.  Use -Wno-dev to suppress it.
      
      -- Found PythonInterp: /home/james/BinAnalysis/bin/python (found suitable version "3.12.3", minimum required is "3.6")
      -- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.12.so
      -- Python version:     3.12.3
      -- Python lib:         /usr/lib/x86_64-linux-gnu/libpython3.12.so
      -- Python include:     /usr/include/python3.12
      -- Python interpreter: /home/james/BinAnalysis/bin/python
      -- Performing Test C_SUPPORTS_NO_MACRO_REDEFINED
      -- Performing Test C_SUPPORTS_NO_MACRO_REDEFINED - Success
      -- Performing Test CXX_SUPPORTS_NO_MACRO_REDEFINED
      -- Performing Test CXX_SUPPORTS_NO_MACRO_REDEFINED - Success
      -- Performing Test C_SUPPORTS_NO_DEPRECATED_DECLARATIONS
      -- Performing Test C_SUPPORTS_NO_DEPRECATED_DECLARATIONS - Success
      -- Performing Test CXX_SUPPORTS_NO_DEPRECATED_DECLARATIONS
      -- Performing Test CXX_SUPPORTS_NO_DEPRECATED_DECLARATIONS - Success
      -- Performing Test HAS_ANSI
      -- Performing Test HAS_ANSI - Success
      -- Configuring done (2.0s)
      -- Generating done (0.1s)
      -- Build files have been written to: /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312
      Using 1 jobs
      [  0%] Creating directories for 'lief_pybind11_project'
      [  0%] Performing download step (verify and extract) for 'lief_pybind11_project'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//pybind11-2.10.1.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/pybind11-2.10.1.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/api/python/lief_pybind11_project-prefix/src/lief_pybind11_project'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  1%] No update step for 'lief_pybind11_project'
      [  1%] No patch step for 'lief_pybind11_project'
      [  1%] No configure step for 'lief_pybind11_project'
      [  2%] No build step for 'lief_pybind11_project'
      [  2%] No install step for 'lief_pybind11_project'
      [  2%] Completed 'lief_pybind11_project'
      [  2%] Built target lief_pybind11_project
      [  2%] Creating directories for 'lief_spdlog_project'
      [  2%] Performing download step (verify and extract) for 'lief_spdlog_project'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//spdlog-1.9.2.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/spdlog-1.9.2.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_spdlog_project-prefix/src/lief_spdlog_project'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  2%] No update step for 'lief_spdlog_project'
      [  2%] No patch step for 'lief_spdlog_project'
      [  2%] No configure step for 'lief_spdlog_project'
      [  2%] No build step for 'lief_spdlog_project'
      [  3%] No install step for 'lief_spdlog_project'
      [  3%] Completed 'lief_spdlog_project'
      [  3%] Built target lief_spdlog_project
      [  3%] Creating directories for 'lief_span'
      [  3%] Performing download step (verify and extract) for 'lief_span'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//tcb-span-d6c6e30.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/tcb-span-d6c6e30.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_span-prefix/src/lief_span'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  3%] No update step for 'lief_span'
      [  4%] No patch step for 'lief_span'
      [  4%] No configure step for 'lief_span'
      [  4%] No build step for 'lief_span'
      [  4%] No install step for 'lief_span'
      [  4%] Completed 'lief_span'
      [  4%] Built target lief_span
      [  4%] Creating directories for 'lief_libjson'
      [  5%] Performing download step (verify and extract) for 'lief_libjson'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//json-3.11.2.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/json-3.11.2.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_libjson-prefix/src/lief_libjson'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  5%] No update step for 'lief_libjson'
      [  5%] No patch step for 'lief_libjson'
      [  5%] No configure step for 'lief_libjson'
      [  5%] No build step for 'lief_libjson'
      [  5%] No install step for 'lief_libjson'
      [  5%] Completed 'lief_libjson'
      [  5%] Built target lief_libjson
      [  5%] Creating directories for 'lief_mbed_tls'
      [  5%] Performing download step (verify and extract) for 'lief_mbed_tls'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//mbedtls-3.2.1.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/mbedtls-3.2.1.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/mbed_tls/src/lief_mbed_tls'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  5%] No update step for 'lief_mbed_tls'
      [  5%] No patch step for 'lief_mbed_tls'
      [  6%] No configure step for 'lief_mbed_tls'
      [  6%] No build step for 'lief_mbed_tls'
      [  6%] No install step for 'lief_mbed_tls'
      [  6%] Completed 'lief_mbed_tls'
      [  6%] Built target lief_mbed_tls
      [  6%] Creating directories for 'lief_frozen'
      [  6%] Performing download step (verify and extract) for 'lief_frozen'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//frozen-e6ddc43.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/frozen-e6ddc43.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_frozen-prefix/src/lief_frozen'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  6%] No update step for 'lief_frozen'
      [  7%] No patch step for 'lief_frozen'
      [  7%] No configure step for 'lief_frozen'
      [  7%] No build step for 'lief_frozen'
      [  7%] No install step for 'lief_frozen'
      [  8%] Completed 'lief_frozen'
      [  8%] Built target lief_frozen
      [  9%] Creating directories for 'lief_leaf'
      [  9%] Performing download step (verify and extract) for 'lief_leaf'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//leaf-1.81.0-prerelease.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/leaf-1.81.0-prerelease.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_leaf-prefix/src/lief_leaf'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [  9%] No update step for 'lief_leaf'
      [  9%] No patch step for 'lief_leaf'
      [  9%] No configure step for 'lief_leaf'
      [  9%] No build step for 'lief_leaf'
      [  9%] No install step for 'lief_leaf'
      [  9%] Completed 'lief_leaf'
      [  9%] Built target lief_leaf
      [  9%] Creating directories for 'lief_utfcpp'
      [ 10%] Performing download step (verify and extract) for 'lief_utfcpp'
      -- verifying file...
           file='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party//utfcpp-3.2.1.zip'
      -- verifying file... done
      -- extracting...
           src='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/third-party/utfcpp-3.2.1.zip'
           dst='/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/build/temp.linux-x86_64-cpython-312/lief_utfcpp-prefix/src/lief_utfcpp'
      -- extracting... [tar xfz]
      -- extracting... [analysis]
      -- extracting... [rename]
      -- extracting... [clean up]
      -- extracting... done
      [ 10%] No update step for 'lief_utfcpp'
      [ 10%] No patch step for 'lief_utfcpp'
      [ 10%] No configure step for 'lief_utfcpp'
      [ 10%] No build step for 'lief_utfcpp'
      [ 10%] No install step for 'lief_utfcpp'
      [ 10%] Completed 'lief_utfcpp'
      [ 10%] Built target lief_utfcpp
      [ 10%] Generating internal/utfcpp/utf8.h
      [ 10%] Generating include/LIEF/third-party/internal/leaf.hpp
      [ 10%] Generating include/LIEF/third-party/internal/span.hpp
      [ 10%] Generating internal/nlohmann/json.hpp
      [ 10%] Building CXX object CMakeFiles/LIB_LIEF.dir/src/errors.cpp.o
      [ 10%] Building CXX object CMakeFiles/LIB_LIEF.dir/src/hash_stream.cpp.o
      [ 11%] Building CXX object CMakeFiles/LIB_LIEF.dir/src/logging.cpp.o
      [ 11%] Building CXX object CMakeFiles/LIB_LIEF.dir/src/exception.cpp.o
      In file included from /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/src/exception.cpp:19:
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:75:38: error: expected ‘)’ before ‘offset’
         75 |   explicit read_out_of_bound(uint64_t offset, uint64_t size);
            |                             ~        ^~~~~~~
            |                                      )
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:76:38: error: expected ‘)’ before ‘offset’
         76 |   explicit read_out_of_bound(uint64_t offset);
            |                             ~        ^~~~~~~
            |                                      )
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/src/exception.cpp:50:1: error: no declaration matches ‘LIEF::read_out_of_bound::read_out_of_bound(uint64_t, uint64_t)’
         50 | read_out_of_bound::read_out_of_bound(uint64_t offset, uint64_t size) : LIEF::exception("") {
            | ^~~~~~~~~~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note: candidates are: ‘LIEF::read_out_of_bound::read_out_of_bound(LIEF::read_out_of_bound&&)’
         71 | class LIEF_API read_out_of_bound : public exception {
            |                ^~~~~~~~~~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const LIEF::read_out_of_bound&)’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound()’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const LIEF::exception&) [inherited from LIEF::exception]’
         73 |   using exception::exception;
            |                    ^~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(std::string) [inherited from LIEF::exception]’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const char*) [inherited from LIEF::exception]’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note: ‘class LIEF::read_out_of_bound’ defined here
         71 | class LIEF_API read_out_of_bound : public exception {
            |                ^~~~~~~~~~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/src/exception.cpp:58:1: error: no declaration matches ‘LIEF::read_out_of_bound::read_out_of_bound(uint64_t)’
         58 | read_out_of_bound::read_out_of_bound(uint64_t offset) : LIEF::exception("") {
            | ^~~~~~~~~~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note: candidates are: ‘LIEF::read_out_of_bound::read_out_of_bound(LIEF::read_out_of_bound&&)’
         71 | class LIEF_API read_out_of_bound : public exception {
            |                ^~~~~~~~~~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const LIEF::read_out_of_bound&)’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound()’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const LIEF::exception&) [inherited from LIEF::exception]’
         73 |   using exception::exception;
            |                    ^~~~~~~~~
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(std::string) [inherited from LIEF::exception]’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:73:20: note:                 ‘LIEF::read_out_of_bound::read_out_of_bound(const char*) [inherited from LIEF::exception]’
      /tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/include/LIEF/exception.hpp:71:16: note: ‘class LIEF::read_out_of_bound’ defined here
         71 | class LIEF_API read_out_of_bound : public exception {
            |                ^~~~~~~~~~~~~~~~~
      make[3]: *** [CMakeFiles/LIB_LIEF.dir/build.make:138: CMakeFiles/LIB_LIEF.dir/src/exception.cpp.o] Error 1
      make[2]: *** [CMakeFiles/Makefile2:411: CMakeFiles/LIB_LIEF.dir/all] Error 2
      make[1]: *** [CMakeFiles/Makefile2:472: api/python/CMakeFiles/pyLIEF.dir/rule] Error 2
      make: *** [Makefile:306: pyLIEF] Error 2
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/setup.py", line 484, in <module>
          setup(
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/wheel/bdist_wheel.py", line 368, in run
          self.run_command("build")
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 132, in run
          self.run_command(cmd_name)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
          self.distribution.run_command(command)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/home/james/BinAnalysis/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/setup.py", line 98, in run
          self.build_extension(ext)
        File "/tmp/pip-install-kng2yvua/lief_cb7cf54b35ed47fd9de6c686468bbb19/setup.py", line 308, in build_extension
          subprocess.check_call(['make', '-j', str(jobs), targets['python_bindings']], cwd=self.build_temp, env=env)
        File "/usr/lib/python3.12/subprocess.py", line 413, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['make', '-j', '1', 'pyLIEF']' returned non-zero exit status 2.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for lief
  Running setup.py clean for lief
Failed to build lief
ERROR: Could not build wheels for lief, which is required to install pyproject.toml-based projects

Expected behavior
unblob to be installed.

Environment information (please complete the following information):

  • OS: Ubuntu 24.04
  • Software versions (Docker, Python, Nix, Poetry, if applicable): Python 3.12.3, GCC 13.2.0 (Ubuntu 13.2.0-23ubuntu4).

We recommend you execute and paste the results of those commands in this issue so we can get a sense of your environment:

  • Linux/Darwin kernel version with uname -avr:
    Linux noble 6.8.0-35-generic #⁠35-Ubuntu SMP PREEMPT_DYNAMIC Mon May 20 15:51:52 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  • Linux distribution with cat /etc/lsb-release:
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=24.04
    DISTRIB_CODENAME=noble
    DISTRIB_DESCRIPTION="Ubuntu 24.04 LTS"

Your system is missing pre-requisites to build the lief wheel locally. This should not happen since Python projects (usually) publish pre-built wheels.

Turns out that lief did not publish a pre-built wheel for its version 0.12.3 (the one we pinned in unblob) on Python version 3.12.

If you use Python3.10 or Python 3.11 it should work. In the meantime, I'll check if we can move to version 0.14.1 (for which there exists a pre-built wheel), or ask if lief maintainers can publish a pre-built wheel of 0.12.3 for Python 3.12.

@jrmoore work in progress at #877 if you want to check it out. You can pull the repo and run pip install . within unblob directory to check if your system still tries to build LIEF wheels.

Sorry for the delay, just checked out the update-lief branch and installing like you mentioned works (in the repo root, pip complained of missing pyproject.toml otherwise), thank you.

I figured you would be using a specific version of that dependency, but I had no idea where to look, Python is something I don't really know.

I'll close this ticket once the MR lands. It'll take some time since we need to update a package in Nix.

Lief is updated in #877