Errors installing Python dependencies on ARM based macOS

[cosmin]

I'm trying to setup a development environment on an ARM based macOS (M1) and it's been much more exciting than I expected. It might be worth updating either the dependencies or the build instructions or both.

I started by following the usual Python requirements from resource/doc/python.md until I got to

pip install -r ./python/requirements.txt

fails with the default clang on macOS with an error building libsvm-official

clang: error: unsupported option '-fopenmp'

which makes sense as macOS llvm does not support OpenMP. Common alternatives are to use GCC instead (brew install gcc). However

 CC=gcc-13 CXX=g++-13 pip install -r ./python/requirements.txt

fails due to gcc-13: error: unrecognized command-line option '-iwithsysroot/System/Library/Frameworks/System.framework/PrivateHeaders'.

Another alternative is using llvm from homebrew (brew install llvm) which would support OpenMP, however

LLVM_CONFIG=/opt/homebrew/opt/llvm/bin/llvm-config pip install -r ./python/requirements.txt

fails because it's trying to compile a universal binary (-arch arm64 -arch x86_64) and the Homebrew llvm does not support universal builds, so it complains about trying to build x86_64 against libraries only compiled for arm64.

Tracked this down to using system python which is a universal binary, so I tried again with homebrew python3 which would be arm64 only.

brew install python
[...]
LLVM_CONFIG=/opt/homebrew/opt/llvm/bin/llvm-config pip install -r ./python/requirements.txt

this then fails to build PyWavelets due to

pywt/_extensions/_pywt.c:253:12: fatal error: 'longintrepr.h' file not found

It looks like this this may have changed around Python 3.11, presumably a new version of PyWavelets might fix this but it is pinned to 1.1.1, so I tried downgrading Python to 3.10 to see if that would fix it

brew install python@3.10
[...]
LLVM_CONFIG=/opt/homebrew/opt/llvm/bin/llvm-config pip install -r ./python/requirements.txt

but then ran into another error in PyWavelets

 pywt/_extensions/_pywt.c:32259:5: error: expression is not assignable
          ++Py_REFCNT(o);

at this point I decided to go back to Python 3.11 and try unpinning PyWavelets instead, and finally got everything to build. However now unit tests started to fail, some due to what seems to be newer numpy getting picked up (arrays to stack must be passed as a "sequence" type such as list or tuple) and some that look like floating point math differences ('88.030328 != 88.030463 within 4 places (0.00013500000000021828 difference)' and so on).

At a minimum though it might be good to unpin PyWavelets and change the build instructions for Mac to specify using Homebrew LLVM, and then chase up the test issues separately. Happy to send some pull requests.

[li-zhi]

Yes, PR please:)

[nilfm99]

Is there any remaining issue in your dev workflow after the fixes?

[cosmin]

no, everything seems to work after all these got merged