Regression with Scala 2.13: cannot compile GraalVM native-image

Question

Regression with Scala 2.13: cannot compile GraalVM native-image

plokhotnyuk opened this issue 5 years ago · 24 comments

An example which successfully compiles to native binaries from Scala 2.11 & 2.12 uber-jars cannot be compiled from Scala 2.13 uber-jar due some internal changes in Scala collection implementation.

Error output from the native-image tool is:

Error: Unsupported features in 5 methods
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: java.lang.Object[].clone(). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
	at parsing scala.collection.immutable.VectorPointer.gotoPosWritable1(Vector.scala:1243)
Call path from entry point to scala.collection.immutable.VectorPointer.gotoPosWritable1(int, int, int, Object[]): 
	at scala.collection.immutable.VectorPointer.gotoPosWritable1(Vector.scala:1242)
	at scala.collection.immutable.VectorPointer.gotoPosWritable1$(Vector.scala:1241)
	at scala.collection.immutable.VectorBuilder.addAll(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:27)
	at scala.collection.IterableFactory.apply(Factory.scala:104)
	at scala.collection.IterableFactory.apply$(Factory.scala:104)
	at scala.collection.immutable.List$.apply(List.scala:618)
	at scala.collection.SeqFactory$Delegate.apply(Factory.scala:305)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01$.main(Example01.scala:18)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01.main(Example01.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:147)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: java.lang.Object[][].clone(). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
	at parsing scala.collection.immutable.VectorPointer.stabilize(Vector.scala:1197)
Call path from entry point to scala.collection.immutable.VectorPointer.stabilize(int): 
	at scala.collection.immutable.VectorPointer.stabilize(Vector.scala:1163)
	at scala.collection.immutable.VectorPointer.stabilize$(Vector.scala:1163)
	at scala.collection.immutable.VectorBuilder.addAll(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:27)
	at scala.collection.IterableFactory.apply(Factory.scala:104)
	at scala.collection.IterableFactory.apply$(Factory.scala:104)
	at scala.collection.immutable.List$.apply(List.scala:618)
	at scala.collection.SeqFactory$Delegate.apply(Factory.scala:305)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01$.main(Example01.scala:18)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01.main(Example01.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:147)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported type java.lang.invoke.MemberName is reachable: All methods from java.lang.invoke should have been replaced during image building.
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
Trace: 
	at parsing java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:861)
Call path from entry point to java.lang.invoke.MethodHandles$Lookup.findVirtual(Class, String, MethodType): 
	no path found from entry point to target method

Error: type is not available in this platform: com.oracle.svm.hosted.ClassValueFeature
Trace: 	object java.lang.Class[]
	object java.lang.invoke.MethodType
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet
	method java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean)
Call path from entry point to java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean): 
	no path found from entry point to target method

Error: type is not available in this platform: com.oracle.svm.hosted.substitute.ComputedValueField
Trace: 	object java.lang.invoke.MethodType
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet
	method java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean)
Call path from entry point to java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean): 
	no path found from entry point to target method

Steps to reproduce:

Install or make sure all those things are installed: Sbt, GraalVM CE/EE and its native-image tool
Clone the following repo: https://github.com/plokhotnyuk/jsoniter-scala
Checkout the switch-examples-to-scala-2.13 branch
Run the following commands:

cd jsoniter-scala-examples
sbt clean +assembly
/usr/lib/jvm/graalvm-ce-19/bin/native-image --no-server --no-fallback -H:UnsafeAutomaticSubstitutionsLogLevel=3 -jar target/scala-2.13/jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT.jar

Answer 1 · 2019-07-15T19:08:09.000Z

is this a bug in Scala?

Answer 2 · 2019-07-15T19:57:23.000Z

IMHO, yes.

Some of these errors appear due to releaseFence calls, as reporred here. Others also could be prevented in case of opening some kind of community build for tools and libs which going to support AOT compilation mode, starting from scalac and core libraries.

Answer 3 · 2019-07-15T19:59:17.000Z

are you saying that in order for this to work, we can't use releaseFence at all?

Answer 4 · 2019-07-15T20:09:18.000Z

There are implementation options to make Unsafe usage recognizable for the Graal compiler.

And for edge cases some cooperation with the GraalVM team will be required for sure.

I have managed to fix compilation error for releaseFence calls by the following commit: plokhotnyuk/jsoniter-scala@e089f06

Others calls to java.lang.Object[].clone() and java.lang.Object[][].clone() are still not substituted, and if they are not called in runtime then possible W/A is to use the --allow-incomplete-classpath option of the GraalVM native-image compiler.

Answer 5 · 2019-10-02T19:59:51.000Z

Tentatively milestoned for 2.13.2, since it should certainly at least be assessed further.

Answer 6 · 2019-10-03T08:39:04.000Z

Great that you managed to override the call to releaseFence. This (allowing other platforms to customize low level stuff) is why we provide that hook through scala.runtime.Statics. Maybe we can do something similar for the array clone methods?

Answer 7 · 2019-10-03T08:47:01.000Z

What's the problem with Array.clone()? They're not low-level at all; they're straightforward reflection-free user code:
https://github.com/scala/scala/blob/4ec48ff29755fc01fdde880dbb4548085420c601/src/library/scala/runtime/ScalaRunTime.scala#L94-L105

Answer 8 · 2019-10-03T08:52:30.000Z

I meant java.lang.Object[].clone()

Answer 9 · 2019-10-03T08:55:00.000Z

Anyway, that still seems a little too common for graal not to support natively? Quick google led to a similar, resolved issue: oracle/graal#877

Answer 10 · 2019-10-03T12:17:44.000Z

@adriaanm checked with GraalVM CE 19.2.0.1 and still got the same error...

now I'm waiting for the next release: 19.2.1 or 19.3.0 to check again...

Answer 11 · 2019-10-03T12:40:49.000Z

Thanks for checking. Looks like the bug I linked to was reported fixed on the Java 11 version, which would be the upcoming 19.3, IUC.

Answer 12 · 2019-11-20T21:23:53.000Z

GraalVM 19.3.0 fixed this for me.

Answer 13 · 2019-11-20T21:43:53.000Z

Sweet! Thanks for reporting :-)

Answer 14 · 2020-01-31T07:15:56.000Z

I have managed to fix compilation error for releaseFence calls by the following commit: plokhotnyuk/jsoniter-scala@e089f06

Btw, I've found that it isn't necessary to substitute the scala.runtime.Statics class, for instance with your Target_scala_runtime_Statics class.

It's simply necessary to initialize the underlying scala.runtime.Statics.VM class, by specifying --initialize-at-build-time=scala.runtime.Statics$VM. You can do that either with graalVMNativeImageOptions, if using sbt-native-packager, or with a src/main/resources/META-INF/native-image/org.scala-lang/scala-lang/native-image.properties file containing

Args = --initialize-at-build-time=scala.runtime.Statics$VM

I'm thinking we should consider adding that file ourselves in the library jar, so it's pre-configured for all native-image users.

Answer 15 · 2020-01-31T08:27:48.000Z

I'm thinking we should consider adding that file ourselves in the library jar, so it's pre-configured for all native-image users.

Should the config files for native-image be on a separate artifact instead of polluting the library jar?

Answer 16 · 2020-01-31T09:19:20.000Z

Yeah, that's the consideration: should Graal native-image support come out the box or rely on some other artifact?

In terms of pollution, creating a separate artifact likely pollutes much more than the 59 bytes that native-image.properties adds. 😃

Answer 17 · 2020-02-01T08:14:59.000Z

@dwijnand I've tried to build an image with your file in this branch and it doesn't work for me:

[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]    classlist:   3,006.85 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]        (cap):     611.87 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]        setup:   1,539.92 ms
warning: unknown anonymous info of class scala.collection.immutable.LazyList$State$Empty$, assuming class is not anonymous. To remove the warning report an issue to the library or language author. The issue is caused by scala.collection.immutable.LazyList$State$Empty$ which is not following the naming convention.
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]   (typeflow):   4,817.32 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]    (objects):   4,055.61 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]   (features):     149.16 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:24882]     analysis:   9,139.04 ms
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call: java.lang.invoke.LambdaForm$MH/1637474956.invoke_MT(Object, Object)
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The error is then reported at run time when the invoke is executed.
Detailed message:
Trace: 
	at parsing scala.runtime.Statics.releaseFence(Statics.java:148)
Call path from entry point to scala.runtime.Statics.releaseFence(): 
	at scala.runtime.Statics.releaseFence(Statics.java:148)
	at scala.collection.immutable.Vector$.from(Vector.scala:65)
	at scala.collection.immutable.Vector$.from(Vector.scala:27)
	at scala.collection.IterableFactory.apply(Factory.scala:103)
	at scala.collection.IterableFactory.apply$(Factory.scala:103)
	at scala.collection.immutable.List$.apply(List.scala:611)
	at scala.collection.SeqFactory$Delegate.apply(Factory.scala:304)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01$.main(Example01.scala:18)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01.main(Example01.scala)
	at com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:151)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:186)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)

Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception
Error: Image build request failed with exit status 1

Steps to reproduce:

$ sbt clean +assembly
$ /usr/lib/jvm/graalvm-ce-java8/bin/java -version
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build 1.8.0_232-b07)
OpenJDK 64-Bit GraalVM CE 19.3.0.2 (build 25.232-b07-jvmci-19.3-b06, mixed mode)
$ /usr/lib/jvm/graalvm-ce-java8/bin/native-image --no-server --no-fallback --allow-incomplete-classpath -jar target/scala-2.13/jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT.jar

Have I missed something?

Answer 18 · 2020-02-01T11:09:53.000Z

@plokhotnyuk I'm not sure, it worked for me:

$ native-image --no-server --no-fallback --allow-incomplete-classpath -jar target/scala-2.13/jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT.jar
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]    classlist:   2,055.92 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]        (cap):   1,925.99 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]        setup:   2,948.79 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]   (typeflow):   5,549.69 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]    (objects):   5,066.50 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]   (features):     344.68 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]     analysis:  11,164.62 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]     (clinit):     427.08 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]     universe:     712.27 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]      (parse):     537.21 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]     (inline):   1,287.31 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]    (compile):   5,211.85 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]      compile:   7,470.92 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]        image:     795.07 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]        write:     206.90 ms
[jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT:36803]      [total]:  25,544.13 ms
$ ./jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT
User(John,List(Device(1,HTC One X)))
{"name":"John","devices":[{"id":2,"model":"iPhone X"}]}

I'm running graalvm-ce-java11-19.3.1, maybe it's because of that?

$ java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment GraalVM CE 19.3.1 (build 11.0.6+9-jvmci-19.3-b07)
OpenJDK 64-Bit Server VM GraalVM CE 19.3.1 (build 11.0.6+9-jvmci-19.3-b07, mixed mode, sharing)

Answer 19 · 2020-02-01T13:02:23.000Z

@dwijnand yeap, your native-image.properties works fine with graalvm-ce-java11-19.3.1 but the substitution is still required for graalvm-ce-java8-19.3.1.

Answer 20 · 2020-08-07T07:58:10.000Z

In case anyone has hit on a similar issue with GraalVM 20.1.0, Scala 2.12.12 and scala.collection.immutable.VM, the following changes fixed the problem for me.

  // build.sbt
  lazy val myProject
    .in(file("my-project"))
    .settings(
+     libraryDependencies += "org.graalvm.nativeimage" % "svm" % "20.1.0" % "compile-internal",
    )
    .enablePlugins(GraalVMNativeImagePlugin)
+  // my-project/src/main/scala/myproject/internal/substites/Target_scala_collection_immutable_VM.java
+  package myproject.internal.substitutes;
+  
+  import com.oracle.svm.core.annotate.Substitute;
+  import com.oracle.svm.core.annotate.TargetClass;
+  
+  @TargetClass(className = "scala.collection.immutable.VM")
+  final class Target_scala_runtime_Statics {
+  
+      @Substitute
+      public static void releaseFence() {
+          UnsafeUtils.UNSAFE.storeFence();
+      }
+  }
+  // my-project/src/main/scala/myproject/internal/substites/UnsafeUtils.java
+  package myproject.internal.substitutes;+  
+  import java.lang.reflect.Field;+  
+  class UnsafeUtils {
+      static final sun.misc.Unsafe UNSAFE;+  
+      static {
+          try {
+              Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
+              field.setAccessible(true);
+              UNSAFE = (sun.misc.Unsafe) field.get(null);
+          } catch (Throwable ex) {
+              throw new ExceptionInInitializerError(ex);
+          }
+      }
+  }

Big thanks to @plokhotnyuk for finding this workaround, which I found in plokhotnyuk/jsoniter-scala@e089f06

Answer 21 · 2020-08-29T07:56:16.000Z

I published the workaround above as an independent library:

// Add this to build.sbt for the native-image project
libraryDependencies += "org.scalameta" %% "svm-subs" % "20.1.0" % "compile-internal"

The jar is 4kb and has no external dependencies besides scala-library. You can download the jar and add it manually to the classpath if you prefer https://repo1.maven.org/maven2/org/scalameta/svm-subs_2.13/19.3.2/svm-subs_2.13-19.3.2.jar

This workaround is only needed for 2.12.12+ and 2.13.3+

Answer 22 · 2020-08-29T08:27:15.000Z

@olafurpg Very neat, thanks!

Answer 23 · 2020-08-29T12:39:03.000Z

In case anyone is interested, I published a new plugin called sbt-native-image (https://github.com/scalameta/sbt-native-image) that automatically adds the correct svm-subs dependency and provides other nice features like automatic GraalVM installation

Answer 24 · 2021-02-11T05:55:06.000Z

Sweet! Thanks for reporting :-)

Hey everyone - are we sure this issue should be closed? My first experience with GraalVM and Scala hit upon it very quickly (using GraalVM 21.0.0). I was able to circumvent issues by using the sbt-native-image plugin, but I'd say that's working around the issue. Thanks.