polyglot-chat-app not able to run
Opened this issue · 12 comments
Deleted user commented
./gradlew run
> Task :run
__ __ _ _
| \/ (_) ___ _ __ ___ _ __ __ _ _ _| |_
| |\/| | |/ __| '__/ _ \| '_ \ / _` | | | | __|
| | | | | (__| | | (_) | | | | (_| | |_| | |_
|_| |_|_|\___|_| \___/|_| |_|\__,_|\__,_|\__|
09:33:19.221 [main] INFO io.micronaut.runtime.Micronaut - Startup completed in 3021ms. Server Running: http://localhost:12345
[To redirect Truffle log output to a file use one of the following options:
* '--log.file=<path>' if the option is passed using a guest language launcher.
* '-Dpolyglot.log.file=<path>' if the option is passed using the host Java launcher.
* Configure logging using the polyglot embedding API.]
[engine] WARNING: The polyglot engine uses a fallback runtime that does not support runtime compilation to native code.
Execution without runtime compilation will negatively impact the guest application performance.
The following cause was found: JVMCI is not enabled for this JVM. Enable JVMCI using -XX:+EnableJVMCI.
For more information see: https://www.graalvm.org/latest/reference-manual/embed-languages/.
To disable this warning use the '--engine.WarnInterpreterOnly=false' option or the '-Dpolyglot.engine.WarnInterpreterOnly=false' system property.
09:33:55.491 [default-nioEventLoopGroup-1-5] INFO websocket.chat.ChatWebSocket - * WebSocket: onOpen received for session f5DFQCHfnGbAFCzVthJEWQ== from 'bob' regarding 'chat'
09:34:09.597 [default-nioEventLoopGroup-1-5] INFO websocket.chat.ChatWebSocket - * WebSocket: onMessage received for session f5DFQCHfnGbAFCzVthJEWQ== from 'bob' regarding 'chat'
Exception in thread "Thread-3" LookupError:
**********************************************************************
Resource vader_lexicon not found.
Please use the NLTK Downloader to obtain the resource:
>>> import nltk
>>> nltk.download('vader_lexicon')
For more information see: https://www.nltk.org/data.html
Attempted to load sentiment/vader_lexicon.zip/vader_lexicon/vader_lexicon.txt
Searched in:
- '/Users/bhoran/nltk_data'
- '/Users/bhoran/Oracle/GraalVM/graalvm-demos/polyglot-chat-app/build/resources/main/venv/nltk_data'
- '/Users/bhoran/Oracle/GraalVM/graalvm-demos/polyglot-chat-app/build/resources/main/venv/share/nltk_data'
- '/Users/bhoran/Oracle/GraalVM/graalvm-demos/polyglot-chat-app/build/resources/main/venv/lib/nltk_data'
- '/usr/share/nltk_data'
- '/usr/local/share/nltk_data'
- '/usr/lib/nltk_data'
- '/usr/local/lib/nltk_data'
- ''
**********************************************************************
at <python> find(build/resources/main/venv/lib/python3.10/site-packages/nltk/data.py:471-583:16484-21267)
at <python> _open(Unknown)
at <python> load(Unknown)
at <python> __init__(Unknown)
at <python> <module>(Unknown)
at org.graalvm.polyglot.Context.eval(Context.java:402)
at websocket.chat.PolyglotContextFactories.loadScript(PolyglotContextFactories.java:101)
at websocket.chat.PolyglotContextFactories.lambda$createContext$0(PolyglotContextFactories.java:95)
at java.base/java.lang.Thread.run(Thread.java:1583)
<==========---> 80% EXECUTING [2m 26s]
> :run
olyagpl commented
@bhoran , what JDK did you run this demo with? I'm testing on Oracle GraalVM 21+35.1, macOS, and the exception is different:
16:50:15.726 [main] INFO io.micronaut.runtime.Micronaut - Startup completed in 429ms. Server Running: http://localhost:12345
Exception in thread "Thread-8" ModuleNotFoundError: No module named 'nltk'
at <python> <module>(Unknown)
at org.graalvm.polyglot.Context.eval(Context.java:402)
at websocket.chat.PolyglotContextFactories.loadScript(PolyglotContextFactories.java:101)
at websocket.chat.PolyglotContextFactories.lambda$createContext$0(PolyglotContextFactories.java:95)
at java.base/java.lang.Thread.run(Thread.java:1583)
16:50:56.592 [default-nioEventLoopGroup-1-5] INFO websocket.chat.ChatWebSocket - * WebSocket: onOpen received for session IQ2DUaHAZFXMNrXbtsK1xw== from 'bob' regarding 'chat'
Deleted user commented
Same root cause by the looks of it.
I was using OpenJDK
openjdk version "21.0.2" 2024-01-16
OpenJDK Runtime Environment (build 21.0.2+13-58)
OpenJDK 64-Bit Server VM (build 21.0.2+13-58, mixed mode, sharing)
olyagpl commented
In my test round, on Oracle GraalVM 21+35.1 for macOS, I could run on JVM and as a native image.
@timfel , could you please see this exception and warnings in the console during the runs? Anything we can do to eliminate those?
ogupalo@ogupalo-mac polyglot-chat-app % ./gradlew nativeCompile
> Task :nativeCompile
[native-image-plugin] GraalVM Toolchain detection is disabled
[native-image-plugin] GraalVM location read from environment variable: JAVA_HOME
[native-image-plugin] Native Image executable path: /Library/Java/JavaVirtualMachines/graalvm-jdk-21+35.1/Contents/Home/lib/svm/bin/native-image
Warning: Using a deprecated option --rerun-class-initialization-at-runtime= from 'META-INF/native-image/org.graalvm.shadowed.icu4j/native-image.properties' in 'file:///Users/ogupalo/.gradle/caches/modules-2/files-2.1/org.graalvm.shadowed/icu4j/23.1.1/42b9cb08bb875f018a836c679100cf2cc944e941/icu4j-23.1.1.jar'. Currently there is no replacement for this option. Try using --initialize-at-run-time or use the non-API option -H:ClassInitialization directly.
Warning: Invalid option --add-exports org.graalvm.nativeimage/org.graalvm.nativeimage.impl=org.graalvm.py provided by 'META-INF/native-image/org.graalvm.py/native-image.properties' in 'file:///Users/ogupalo/.gradle/caches/modules-2/files-2.1/org.graalvm.python/python-language/23.1.1/2b91a6368e8fe71ad6edeeaf46bb2a9abb0e5ea1/python-language-23.1.1.jar'. Specified target-module 'org.graalvm.py' is unknown.
...
BUILD SUCCESSFUL in 11m 24s
5 actionable tasks: 1 executed, 4 up-to-date
ogupalo@ogupalo-mac polyglot-chat-app % build/native/nativeCompile/websocket.chat
__ __ _ _
| \/ (_) ___ _ __ ___ _ __ __ _ _ _| |_
| |\/| | |/ __| '__/ _ \| '_ \ / _` | | | | __|
| | | | | (__| | | (_) | | | | (_| | |_| | |_
|_| |_|_|\___|_| \___/|_| |_|\__,_|\__,_|\__|
17:08:51.185 [main] INFO io.micronaut.runtime.Micronaut - Startup completed in 35ms. Server Running: http://localhost:12345
17:08:57.633 [default-nioEventLoopGroup-1-4] INFO websocket.chat.ChatWebSocket - * WebSocket: onOpen received for session kGly9q3e0xB5Xci4ifi/8A== from 'bob' regarding 'chat'
Exception in thread "Thread-10" ModuleNotFoundError: No module named 'nltk'
at <python> <module>(Unknown)
at org.graalvm.polyglot/org.graalvm.polyglot.Context.eval(Context.java:402)
at websocket.chat.PolyglotContextFactories.loadScript(PolyglotContextFactories.java:101)
at websocket.chat.PolyglotContextFactories.lambda$createContext$0(PolyglotContextFactories.java:95)
at java.base@21/java.lang.Thread.runWith(Thread.java:1596)
at java.base@21/java.lang.Thread.run(Thread.java:1583)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)
17:09:00.920 [default-nioEventLoopGroup-1-4] INFO websocket.chat.ChatWebSocket - * WebSocket: onMessage received for session kGly9q3e0xB5Xci4ifi/8A== from 'bob' regarding 'chat'
Another observation, step 2 "Copy the venv into the output resources directory" seems redundant. A native image runs the same way.
timfel commented
@bhoran seems like the README is just missing the step that it tells you in the output:
Resource vader_lexicon not found.
Please use the NLTK Downloader to obtain the resource:
>>> import nltk
>>> nltk.download('vader_lexicon')
Please try to do this and then run again. So basically
graalpy-23.1.1-linux-amd64/bin/graalpy -m venv src/main/resources/venv
src/main/resources/venv/bin/graalpy -m pip install nltk
needs to become
graalpy-23.1.1-linux-amd64/bin/graalpy -m venv src/main/resources/venv
src/main/resources/venv/bin/graalpy -m pip install nltk
src/main/resources/venv/bin/graalpy -c "import nltk; nltk.download('vader_lexicon')"
@olyagpl
You have not installed nltk at all it seems, or you didn't do the
Copy the venv into the output resources directory:
step. That step is required, the gradle build will not put it in the right place for you, and then the native image cannot load the required packages.
timfel commented
To summarise, just because the Micronaut app starts doesn't mean all the steps worked. I guess I need to add more checks to give more and better error messages.
olyagpl commented
@timfel,
Importing the nltk module with a separate command, src/main/resources/venv/bin/graalpy -c "import nltk; nltk.download('vader_lexicon')" fixed the problem running on a JVM, but native-image run fails with the same:
Produced artifacts:
/Users/ogupalo/graal-development/graalvm-demos/polyglot-chat-app/build/native/nativeCompile/resources (language_internal_resources)
/Users/ogupalo/graal-development/graalvm-demos/polyglot-chat-app/build/native/nativeCompile/websocket.chat (executable)
============================================================================================
Finished generating 'websocket.chat' in 11m 23s.
[native-image-plugin] Native Image written to: /Users/ogupalo/graal-development/graalvm-demos/polyglot-chat-app/build/native/nativeCompile
BUILD SUCCESSFUL in 11m 35s
5 actionable tasks: 3 executed, 2 up-to-date
ogupalo@ogupalo-mac polyglot-chat-app % cp -R src/main/resources/venv/ build/native/nativeCompile/resources/python/
ogupalo@ogupalo-mac polyglot-chat-app % build/native/nativeCompile/websocket.chat
__ __ _ _
| \/ (_) ___ _ __ ___ _ __ __ _ _ _| |_
| |\/| | |/ __| '__/ _ \| '_ \ / _` | | | | __|
| | | | | (__| | | (_) | | | | (_| | |_| | |_
|_| |_|_|\___|_| \___/|_| |_|\__,_|\__,_|\__|
12:20:51.554 [main] INFO io.micronaut.runtime.Micronaut - Startup completed in 37ms. Server Running: http://localhost:12345
12:20:54.316 [default-nioEventLoopGroup-1-4] INFO websocket.chat.ChatWebSocket - * WebSocket: onOpen received for session MJxrOTzjAtY6I7lb36NAcQ== from 'bob' regarding 'chat'
Exception in thread "Thread-10" ModuleNotFoundError: No module named 'nltk'
at <python> <module>(Unknown)
at org.graalvm.polyglot/org.graalvm.polyglot.Context.eval(Context.java:402)
at websocket.chat.PolyglotContextFactories.loadScript(PolyglotContextFactories.java:101)
at websocket.chat.PolyglotContextFactories.lambda$createContext$0(PolyglotContextFactories.java:95)
at java.base@21/java.lang.Thread.runWith(Thread.java:1596)
at java.base@21/java.lang.Thread.run(Thread.java:1583)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:833)
ng.Thread.run(Thread.java:1583)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)
12:21:42.677 [default-nioEventLoopGroup-1-4] INFO websocket.chat.ChatWebSocket - * WebSocket: onMessage received for session wgH1TfIlNyA9snuWEhK5cQ== from 'bob' regarding 'chat'
olyagpl commented
I've updated the example README.md in https://github.com/graalvm/graalvm-demos/pull/258/files#diff-bb620106d27f119fde8a27fd423fff11cfa0636260ca891f70af44ae1b6a4dddR21.
Deleted user commented
Did you
"Copy the venv into the output resources directory"?
olyagpl commented
Yes, I can see the resources copied and the nltk module:
@bhoran , can you try from your end?
timfel commented
@olyagpl that isn't correctly copied. You should have
nativeCompile/resources/python/venv/bin/ and nativeCompile/resources/python/venv/include/ and nativeCompile/resources/python/venv/lib
timfel commented
I suppose my README was a bit terse
2. Copy the venv into the output _resources_ directory:
cp -R src/main/resources/venv/ build/native/nativeCompile/resources/python/
By this I meant that you want the venv directory to end up inside build/native/nativeCompile/resources/python/