Problem with initialising SharedInterpreter twice causing pthread_mutex_lock+0x4
Loc300 opened this issue · 1 comments
I´m using Jep in my Java Gradle project and use it to predict with pytorch neural network.
My initialising looking like that
public class Test { private SharedInterpreter interp = new SharedInterpreter(); public Test() throws IOException { interp.eval("import sys"); interp.eval("sys.path.append('')"); interp.runScript("/Users/test.py"); } }
When I initialize the class for the very first time and execute my prediction, everything works so far.
The line for the prediction is pretty much Object result = interp.invoke("predict_test", params);
So i can run my predict and train functions as often as i like, for that first SharedInterpreter instance. After i´m done
with the training is use interp.close()
. To avoid the error here:
Exception in thread "main" jep.JepException: JEP THREAD WARNING: Unsafe reuse of thread main for another Python Interpreter.
Please close() the previous Interpreter to ensure stability.
So in my case after one succesful training, im creating a new instance of the Test
class. And when im using my prediction function again,
im getting the following error:
A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00007ff815e1f3e4, pid=3753, tid=0x0000000000002803
JRE version: OpenJDK Runtime Environment (8.0_302-b08) (build 1.8.0_302-b08)
Java VM: OpenJDK 64-Bit Server VM (25.302-b08 mixed mode bsd-amd64 compressed oops)
Problematic frame:
C [libsystem_pthread.dylib+0x13e4] pthread_mutex_lock+0x4
somtimes: C [Python+0x12d713] take_gil+0x30
When i dont create a new instance of the Test
class, the functions work very well.
Am i missing something?
Environment (please complete the following information):
Im Using an M1 Macbook Pro
Sonoma 14.4 (23E214)
Python 3.11
JEP==4.2.0
Python packages are mostly pytorch, numpy, logging and json
This is not the expected behavior but it can be very difficult to identify whether this is a problem with jep or with one of the libraries you are using. If possible I recommend running your test without any imports and verify jep works with the base python interpreter. If that works then it is likely this isn't a problem with jep but rather a problem with one of the libraries. If a library holds state information about an interpreter in native memory then it may not be possible to use the library in multiple interpreters, In this case you may need to maintain a single SharedInterpreter for use whenever you access the library.