cloudflare/stpyv8

MacOS ARM version

rouvenoberholzer opened this issue Β· 11 comments

Any chance you can provide a MacOS ARM (M1/M2) compiled package?

FYI we rely on the virtual environments provided by Github to build the Python wheels we distribute. I was already aware that ARM architecture is not currently supported by the Github-hosted runners [1] so I checked if it is possible to rely on self-hosted runners. Unfortunately it seems like it is still not possible [2].

The following processor architectures are supported for the self-hosted runner application.

x64 - Linux, macOS, Windows.
ARM64 - Linux only.
ARM32 - Linux only.

Hopefully the architecture will be available in the next future. When it'll happen we will work to add this platform to the ones we already cover.

[1] https://github.com/actions/runner-images
[2] https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners

As mentioned in my previous comment MacOS ARM (M1/M2) support is something we are interested in. Hopefully Github will provide a runner image for this platform sooner or later. Once it happens automating the build and release workflows for such platform too should be extremely quick and easy.

As support for arm64 macOS self-hosted runners is now available, would it be possible to look into this again?

The following processor architectures are supported for the self-hosted runner application.

x64 - Linux, macOS, Windows.
ARM64 - Linux, macOS, Windows (currently in beta).
ARM32 - Linux.

buffer commented

You’re right. By the way I would prefer having a ARM64 macOS Github-hosted runner in order not to have another self-hosted runner to maintain. Apparently they are actively working on it

github/roadmap#528

buffer commented

FYI I was able to successfully build a arm64 macOS-11 image using cibuildwheel [1]. The workflow is still far from perfect and requires some care but I just wanted you to be aware that something is moving

CC @Gobot1234

[1] https://github.com/cloudflare/stpyv8/actions/runs/5822747577

buffer commented

@rouvenoberholzer FYI I recently released STPyV8 to PyPI. This means you can now install it by simply running

pip install stpyv8

The (potentially) good news is that if you're using OSX with Python 3.11 or 3.12 you'll end up installing a universal2 wheel. Which means that in theory it should work both on x86-64 and ARM64. I have not a ARM64 OSX handy so I can not perform tests. I would appreciate if you could give a try and let me know about your experience. Thanks!

CC @srnnkls @Gobot1234

In [4]: import STPyV8

ImportError Traceback (most recent call last)
Cell In[4], line 1
----> 1 import STPyV8

File /opt/homebrew/Caskroom/miniconda/base/envs/py12/lib/python3.12/site-packages/STPyV8.py:12
9 import re
10 import collections.abc
---> 12 import _STPyV8
14 version = _STPyV8.JSEngine.version
16 all = ["ReadOnly",
17 "DontEnum",
18 "DontDelete",
(...)
34 "JSUnlocker",
35 "JSPlatform"]

ImportError: dlopen(/opt/homebrew/Caskroom/miniconda/base/envs/py12/lib/python3.12/site-packages/_STPyV8.cpython-312-darwin.so, 0x0002): symbol not found in flat namespace '__ZN2v82V87DisposeEv'

@buffer FYI.

buffer commented

@pangzi499 thanks for reporting it. Seems like something is not working as expected on ARM64. I will keep on working on it. Thanks.

The ARM version is not properly linked. Obviously v8 itself is not included in the build.

~/Downloads ᐅ lipo -thin arm64 -output STPyV8.arm64.so /Users/[REDACTED]/venv/lib/python3.12/site-packages/_STPyV8.cpython-312-darwin.so
~/Downloads ᐅ lipo -thin x86_64 -output STPyV8.x64.so /Users/[REDACTED]/venv/lib/python3.12/site-packages/_STPyV8.cpython-312-darwin.so
~/Downloads ᐅ ls -lah |grep STPy
-rwxr-xr-x@     1 user  staff   1.3M Mar 20 15:00 STPyV8.arm64.so
-rwxr-xr-x@     1 user  staff    55M Mar 20 15:00 STPyV8.x64.so
image

Seems like we finally have a OSX ARM64 wheel. Right now I created a wheel just for Python 3.12 but I think I will add support for other Python versions in the next hours. I tested the wheel locally and it seems to work fine. If you want to play with the wheel you can retrieve it from the Artifacts section at [1]. And I would appreciate if you can share your experience if you go for it. The plan is to start uploading OSX ARM64 wheel(s) to PyPI starting from the next STPyV8 release thus allowing to install through pip on this platform too.

[1] https://github.com/cloudflare/stpyv8/actions/runs/8427806853

Seems like we finally have a OSX ARM64 wheel. Right now I created a wheel just for Python 3.12 but I think I will add support for other Python versions in the next hours. I tested the wheel locally and it seems to work fine. If you want to play with the wheel you can retrieve it from the Artifacts section at [1]. And I would appreciate if you can share your experience if you go for it. The plan is to start uploading OSX ARM64 wheel(s) to PyPI starting from the next STPyV8 release thus allowing to install through pip on this platform too.

[1] https://github.com/cloudflare/stpyv8/actions/runs/8427806853

Thank you! LGTM!