Add support to release linux aarch64 wheels
odidev opened this issue · 8 comments
Problem
On aarch64, pip install nnabla is giving the below the error-
ERROR: Could not find a version that satisfies the requirement nnabla (from versions: none)
ERROR: No matching distribution found for nnabla
Resolution
On aarch64, pip install nnabla should download the wheels from PyPI.
@TomonobuTsujikawa and Team Please let me know your interest in releasing aarch64 wheels. To start with can you please suggest the steps/CI you are using for building the wheel.
Thank you for trying it!
Yes, as you pointed out, aarch64 is not released now, so anyone cannot install by pip.
But, I think you may be able to build from our source code.
Here is the build instructions from code:
- For nnabla (cpu only): https://github.com/sony/nnabla/blob/master/doc/build/build.md
- For nnabla-ext-cuda (+cuda): https://github.com/sony/nnabla-ext-cuda/blob/master/doc/build/build.md
Of course, you may need to install additional packages and/or change codes.
I have tried to build the code from the source using the steps mentioned here and the wheel is created successfully for aarch64 but it is taking a lot of time rather than downloading from PyPI and unpacking it. Also, installation from the source code requires additional dependencies to be installed.
It will be a great help if you can release nnabla Linux aarch64 wheels on PyPI.
Thank you! It's good news!
We need environment preparation and have to pass tests to support aarch64 officially.
But please let us investigate if we can support it.
We took some time, but we have released aarch64 wheel package in v1.27.0.
https://pypi.org/project/nnabla/#files
https://github.com/sony/nnabla/releases/tag/v1.27.0
I think the installation environment is limited because glibc2.31 is required, so we will improve it in future release.
@ TomonobuTsujikawa I have tried to install the wheel for aarch64 but it is giving the below issue :
# pip3 install nnabla
ERROR: Could not find a version that satisfies the requirement nnabla (from versions: none)
ERROR: No matching distribution found for nnabla
Python version output -
# python3 --version
Python 3.9.13
# pip3 --version
pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.9)
It looks that the issue is in the name of the wheel created and uploaded on PyPI. I have tried to run auditwheel but getting the below issue:
auditwheel repair nnabla-1.28.0-cp39-cp39-manylinux_2_31_aarch64.whl
INFO:auditwheel.main_repair:Repairing nnabla-1.28.0-cp39-cp39-manylinux_2_31_aarch64.whl
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: cannot repair "nnabla-1.28.0-cp39-cp39-manylinux_2_31_aarch64.whl" to "manylinux_2_17_aarch64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.
Can you please look into the issue and help me here?
Thank @odidev for detail description, it's very helpful for us!
I think you need to update pip, because pip supports PEP600 (manylinux_x_yy) from v20.3.
So, please try to update pip by: pip3 install -U pip.
@TomonobuTsujikawa Thanks for the help. I have updated the pip version and now the wheel is getting installed but when I am importing it. It is giving the below error:
>>> import nnabla
2022-06-02 17:09:36,449 [nnabla][INFO]: Initializing CPU extension...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.9/dist-packages/nnabla/__init__.py", line 34, in <module>
from .parameter import (
File "/usr/local/lib/python3.9/dist-packages/nnabla/parameter.py", line 29, in <module>
import nnabla.utils.nnabla_pb2 as nnabla_pb2
File "/usr/local/lib/python3.9/dist-packages/nnabla/utils/nnabla_pb2.py", line 36, in <module>
_descriptor.FieldDescriptor(
File "/usr/local/lib/python3.9/dist-packages/google/protobuf/descriptor.py", line 560, in __new__
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
If I downgrade the protobuf version to 3.20.1 then import is working fine. I have tried the second solution also Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower) but the error is same. Can you suggest how we can import it without doing anything other than installing nnabla from PyPI?