Operating system "OSX" and architecture "ARMv8" are not supported.
mafaul opened this issue · 14 comments
On Apple Mac OS X with M1 chip, I cannot run OpenCV.loadShared();
Error message: Operating system "OSX" and architecture "ARMv8" are not supported.
Native support would be great, but in the meantime just don't block it and let Rosetta handle it please.
Any resolution to this?
There is a fork by @brainstorm that adds support for OSX / ARMv8: https://github.com/brainstorm/opencv
I'm not sure about the state of that fork. It seems that building succeeded once, but they never actually published the artifacts.
I'm currently looking into that, and if I find a resolution, I will let you know.
Native support would be great, but in the meantime just don't block it and let Rosetta handle it please.
As far as I know, you can only load x86 libs using Rosetta if your whole process, i.e. your JVM is running on Rosetta, so that you would need the x86 version of Java (and of all native libraries that you use, not just OpenCV). If you run it like that, I would think that it should work with this version, as this line should detect x86_64.
It works for me :)
See https://github.com/CCHS-Melbourne/openpnp-kiosk and this issue comment for proof.
I wish @vonnieda and/or @markmaker would merge it in though, along with related PRs and the CI improvements. That would ease up redistribution massively and potentially have even android phones running OpenPnP in the near future :-S
Sorry @brainstorm - I was under the impression that there was still work to be done. Is there something waiting for merge that I can just merge?
My apologies @vonnieda, that might have come out wrong and I might have failed to update you on progress... I think that setting aside unforeseen integration issues that are hard to test unless it's tested/merged by third parties, it should be fairly ready to merge/test by you and/or @markmaker.
TL;DR: It might break on your hands, but fixes should not be super tricky (not publishing objects somewhere, some forgotten custom assumption about my system, etc...), most of the fixing/testing heavy lifting has been done at this point, IMHO.
Just FYI, I'm only merging stuff in the openpnp / openpnp repo. I don't know a thing about JNI etc. 🥴
@brainstorm Sorry if I'm being dense here - what specifically do you want me to merge? Is there a PR or something I've missed? Can you be quite specific, and maybe give me a link?
@brainstorm Sorry if I'm being dense here - what specifically do you want me to merge? Is there a PR or something I've missed? Can you be quite specific, and maybe give me a link?
Sure! At this point in time I guess that the remaining PR required to merge from what I'm seeing is the one that removes JNAerator:
openpnp/openpnp-capture-java#6
This PR introduces the patched aarch64 JNI binding fixes that I (sloppily for now) patch on my Dockerfile for things to work and that in my experience triggered the "ARMv8 not supported" error in the past:
https://github.com/CCHS-Melbourne/openpnp-kiosk/blob/master/openpnp/Dockerfile.jetson-nano#L41
It cannot be blindly merged because target paths and repos are changed (i.e brainstorm vs openpnp artifacts), so before merging you should repoint it to your deployment setup by introducing some minor commits/changes to that PR.
Also it might make sense to cherry-pick some lines from https://github.com/openpnp/opencv/pull/75/files if things fail. I closed that PR because GHA does not offer OSX/M1 (Apple silicon) runners, AFAICT, so it's hard to automatically ship binaries for that one at this point (unless somebody cross-compiles it properly or github introduces native M1 CI runners).
OTOH, binaries for aarch64/LINUX should be very easy to generate, which will cater to many embedded Linux+ARM platforms.
Hope that makes sense?
@brainstorm wrote:
I closed that PR because GHA does not offer OSX/M1 (Apple silicon) runners, AFAICT, so it's hard to automatically ship binaries for that one at this point (unless somebody cross-compiles it properly or github introduces native M1 CI runners).
Yes, there are still no OSX/M1 (Apple silicon) runners, but is this a problem? Cross-compiling should work starting from Xcode 12.2 on. I could not access your build logs or artifacts from last year, since they expired, so I don't know if crosscompilation just worked.
What I found out when I tried to cross-compile in my own fork (click to open/close)
Running without modifications
In my run it shows Xcode version 12.4 which should be fine. However, the artifacts from the job build_mac_linux_x64 (macos-10.15, 8) lack the ARMv8 code:
macos-10.15/lib/libopencv_java453.dylib: Mach-O 64-bit dynamically linked shared library x86_64
Running with CMAKE_OSX_ARCHITECTURES
I think the proper solution would be something like set (CMAKE_OSX_ARCHITECTURES arm64 x86_64) which I tried, but resulted in a lot of errors. For example, about half of the static libraries which are build have no symbols, e.g.:
2022-05-30T10:47:21.4162410Z [ 95%] Linking CXX static library ../../lib/libopencv_dnn.a
2022-05-30T10:47:21.5266310Z /Applications/Xcode_12.4.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: for architecture: x86_64 file: ../../lib/libopencv_dnn.a(tengine_graph_convolution.cpp.o) has no symbols
This affects all libraries I marked with "FAIL":
- liblibtiff.a <-- FAIL
- liblibprotobuf.a <-- FAIL
- libIlmImf.a <-- FAIL
- libopencv_core.a <-- FAIL
- libopencv_flann.a
- libopencv_ml.a <-- FAIL
- libopencv_imgproc.a
- libopencv_photo.a
- libopencv_imgcodecs.a <-- FAIL
- libopencv_features2d.a <-- FAIL
- libopencv_videoio.a
- libopencv_highgui.a
- libopencv_calib3d.a <-- FAIL
- libopencv_dnn.a <-- FAIL
- libopencv_stitching.a
- libopencv_objdetect.a
- libopencv_video.a
There are more (other) errors, but I think they are not relevant unless the "has no symbols" problem is solved.
@lenaschimmel I suspect that I closed the PR because if artifacts are not shipped via CI it becomes yet another manual maintenance burden that not many are interested to support.
Yes, the build succeeded for aarch64/linux IIRC, but yes, your OSX issues ring a bell too... I didn't poke/test that route too much because having an (expensive!) M1 connected to a PnP is not very cost-effective for my use cases, I prefer cheaper SBCs or even Android phones :-S
@lenaschimmel I've seen SO answers about ranlib on macos complaining too much about empty library components and needing a flag to silence it. Eg. this one.
However your linked log file has this
ld: warning: ignoring file ../../../3rdparty/ippicv/ippicv_mac/icv/lib/intel64/libippicv.a, building for macOS-arm64 but attempting to link with file built for unknown-x86_64
which seems like it might be a contributing factor.
Based on org.openpnp.opencv@4.5.1-2
Perhaps try this ?
-
Unzip m1.zip to Project Resources Root.
unzip -o -d ~/ProjectPath/src/main/resources/opencv/m1 m1.zip -
Add open-cv.jar path to Project pom.xml
<dependency>
<groupId>org</groupId>
<artifactId>opencv</artifactId>
<version>455</version>
<scope>system</scope>
<systemPath>${project.basedir}\src\main\resources\opencv\m1\opencv-455.jar</systemPath>
</dependency>
- Change Open-CV initialization code to this
static {
// Init Open CV
if (OSEnum.OSX.isCurrent() && ArchEnum.ARMv8.isCurrent()) {
// M1
String opencvLocation = "opencv/m1/libopencv_java455.dylib";
URL url = ClassLoader.getSystemResource(opencvLocation);
System.load(url.getPath());
} else {
// Other plateform
String javaVersionStr = System.getProperty("java.version");
Integer javaVersion = Integer.valueOf(javaVersionStr.length() > 1 ?
javaVersionStr.substring(0, 2) :
javaVersionStr.substring(0, 1));
if (javaVersion > 12) {
// In Java 12+ loadShared() is not available. Use loadLocally() instead
nu.pattern.OpenCV.loadLocally();
} else {
nu.pattern.OpenCV.loadShared();
}
}
}
public enum ArchEnum {
X86_32("i386", "i686", "x86"),
X86_64("amd64", "x86_64"),
ARMv7("arm"),
ARMv8("aarch64", "arm64");
private final Set<String> patterns;
private ArchEnum(final String... patterns) {
this.patterns = new HashSet<String>(Arrays.asList(patterns));
}
public boolean isCurrent() {
final String osArch = System.getProperty("os.arch");
return patterns.contains(osArch);
}
}
public enum OSEnum {
OSX("^[Mm]ac OS X$"),
LINUX("^[Ll]inux$"),
WINDOWS("^[Ww]indows.*");
private final Set<Pattern> patterns;
private OSEnum(final String... patterns) {
this.patterns = new HashSet<Pattern>();
for (final String pattern : patterns) {
this.patterns.add(Pattern.compile(pattern));
}
}
private boolean is(final String id) {
for (final Pattern pattern : patterns) {
if (pattern.matcher(id).matches()) {
return true;
}
}
return false;
}
/**
*
* @param
* @return
*/
public boolean isCurrent() {
final String osName = System.getProperty("os.name");
return is(osName);
}
}
Apple Silicon is now supported natively in v4.5.5-1. Please give it a try and let me know if you run into any issues.