wangkuiyi/gotorch

Cannot build cgotorch on Macbook M1 (arm64) system

hop-along-polly opened this issue · 1 comments

hop-along-polly commented

I've run into a few issues trying to build the cgo bindings for libtorch on my M1 Macbook (arm64).

I've tried building the bindings after using go get to fetch the package as well as cloning the repo directly from Github and running the build script from the raw source code.

Attempt 1 (using go get)

  1. run go get github.com/wangkuiyi/gotorch
  2. run $GOPATH/pkg/mod/github.com/wangkuiyi/gotorch@v0.0.0-20201028015551-9afed2f3ad7b/cgotorch/build.sh

This will produce the following errors. There are a few things I noticed in the output which makes it seem like there's a lot wrong here.

  1. unzip cannot find libtorch-macos-1.6.0.zip, libtorch-macos-1.6.0.zip.zip or libtorch-macos-1.6.0.zip.ZIP.
  2. ln: libtorch permission denied
  3. make: *** No rule to make target macos/libtorch', needed by libcgotorch.so'. Stop.
  4. The various permission denied issues at the end of the output.
Building for macOS ...
unzip:  cannot find or open libtorch-macos-1.6.0.zip, libtorch-macos-1.6.0.zip.zip or libtorch-macos-1.6.0.zip.ZIP.
ln: libtorch: Permission denied
+ make CXX=clang++ LIB_SUFFIX=dylib 'INSTALL_NAME=-install_name @rpath/$@' LIBTORCH_DIR=macos/libtorch GLIBCXX_USE_CXX11_ABI=1 LOAD=all_load CUDA_FLAGS= -f Makefile -j
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c cuda.cc -o cuda.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c device.cc -o device.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c functional.cc -o functional.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c init.cc -o init.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c memory.cc -o memory.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c optim.cc -o optim.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c pickle.cc -o pickle.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c tensor.cc -o tensor.o
clang++ -std=c++14 \
	-I .. -I libtorch/include \
	-I libtorch/include/torch/csrc/api/include \
	-fPIC  \
	-D_GLIBCXX_USE_CXX11_ABI=1 \
	-O -c torch.cc -o torch.o
make: *** No rule to make target `macos/libtorch', needed by `libcgotorch.so'.  Stop.
make: *** Waiting for unfinished jobs....
error: unable to open output file 'torch.o': 'Permission denied'
1 error generated.
error: unable to open output file 'cuda.o': 'Permission denied'
1 error generated.
error: unable to open output file 'pickle.o': 'Permission denied'
1 error generated.
make: *** [torch.o] Error 1
make: *** [cuda.o] Error 1
error: unable to open output file 'memory.o': 'Permission denied'
1 error generated.
make: *** [pickle.o] Error 1
make: *** [memory.o] Error 1
error: unable to open output file 'functional.o': 'Permission denied'
1 error generated.
error: unable to open output file 'init.o': 'Permission denied'
1 error generated.
error: unable to open output file 'device.o': 'Permission denied'
1 error generated.
make: *** [functional.o] Error 1
error: unable to open output file 'tensor.o': 'Permission denied'
1 error generated.
error: unable to open output file 'optim.o': 'Permission denied'
1 error generated.
make: *** [init.o] Error 1
make: *** [device.o] Error 1
make: *** [tensor.o] Error 1
make: *** [optim.o] Error 1
+ popd

Attempt 2 (cloning repository)

  1. git clone git@github.com:wangkuiyi/gotorch.git
  2. cd gotorch
  3. ./cgotorch/build.sh

This will produce this output (truncated for brevity). This seems to be something isn't compatible with an M1 (arm64) architecture.

Building for macOS ...
+ make CXX=clang++ LIB_SUFFIX=dylib 'INSTALL_NAME=-install_name @rpath/$@' LIBTORCH_DIR=macos/libtorch GLIBCXX_USE_CXX11_ABI=1 LOAD=all_load CUDA_FLAGS= -f Makefile -j
clang++ 	-L libtorch/lib \
	cuda.o device.o functional.o init.o memory.o optim.o pickle.o tensor.o torch.o \
	-shared \
	-o libcgotorch.so -install_name @rpath/libcgotorch.so \
	-Wl,-rpath,libtorch/lib \
	-Wl,-all_load libtorch/lib/libc10.dylib \
	-lc10 -ltorch -ltorch_cpu
ld: warning: ignoring file libtorch/lib/libtorch.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file libtorch/lib/libc10.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
ld: warning: ignoring file libtorch/lib/libtorch_cpu.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture arm64:

A couple of follow up questions based on these attempts.

  1. Do I need to install Pytorch ahead of time pip install torch?
  2. Specifically for the first error do I need to use sudo for the build.sh command?
hop-along-polly commented

I found an issue that links to this repository. https://github.com/ljk53/pytorch-rpi

Is downloading a pre-built libtorch binary a good idea?