jaydebeapi.connect calling ucanaccess in seperate thread blocks

[chappers73]

Hi,
I am trying to create a threaded Python app that can check our MS Access Database and also have possibility of shutting down on CTRL-C, this is working perfectly until I use the connect function and try to close.

ucanaccess_jars = [
                "/home/auto/UCanAccess-5.0.0-bin/ucanaccess-5.0.0.jar",
                "/home/auto/UCanAccess-5.0.0-bin/lib/commons-lang3-3.8.1.jar",
                "/home/auto/UCanAccess-5.0.0-bin/lib/commons-logging-1.2.jar",
                "/home/auto/UCanAccess-5.0.0-bin/lib/hsqldb-2.5.0.jar",
                "/home/auto/UCanAccess-5.0.0-bin/lib/jackcess-3.0.1.jar",
            ]
            classpath = ":".join(ucanaccess_jars)
            cnxn = jaydebeapi.connect(
                "net.ucanaccess.jdbc.UcanaccessDriver",
                f"jdbc:ucanaccess://{db_path};COLUMNORDER=DISPLAY",
                ["", ""],
                classpath
            )

            crsr = cnxn.cursor()

DO STUFF()

        # Close cursor
        crsr.close()
        # Close connection
        cnxn.close()

It connects perfectly, grabs the data correctly and then indicates it has closed without error, the issue I have and can't seem to resolve is up until this point I can terminate my Python Script with CTRL-C and detect the SIGINT and gracefully close the app. After the DB data collection has completed and the function is no longer required i.e. The Thread has finished, I cannot detect the SIGINT signal and the Python script just aborts when I press CTRL-C and is not detected in my main thread. I have tried every other thread and only when this one is called does this happen. I did see a link to checking the jpype.isJVMStarted() and tried this too. I have even moved this to the main thread to try to stop the JVM to see if this is the problem, I execute the command jpype.shutdownJVM() but still can’t detect the SIGINT.
What am I doing wrong?
I’m tempted to put this entire DB collection into another APP and then user a Client -> Server to get the information out of it and then when my main app terminates correctly I can then cleanup by killing the Server? Seems a bit drastic.

[Thrameos]

What version of JPype?

[chappers73]

JPype1 v1.2.1
JayDeBeApi v1.2.3

[Thrameos]

Java interfers with Python signal handles. JPype can be set to interactive or noninteractive mode in startJVM. It is not clear from the example which mode it is in. Try the interactive mode to see if that gives the behavior you want. Noninteractive mode will terminate the program when Java terminates. Interactive tries to transfer control back to Python but it can only break on io. This is a limitation of the jvm, and is not something that jaydebeapi can control as neither java nor python have a common signal handler api to provide finer grain controls. If interactive mode does not work you will likely need to use more than one process.

Hope it helps.

[chappers73]

Thrameos, many thanks for the assistance, I will look at the interactive mode and see if I can instantiate the JVM from the jaydebeapi with the mode change , greatly appreciated.

[chappers73]

Hi Thrameos, struggling to get the 'jpype.startJVM(*args, **kwargs)' **kwargs to accept the interrupt option, can you give me some help. It will work with the *args being the jpype.startJVM(jpype.getDefaultJVMPath(), *options)

options = []
options.append('-Djava.class.path=%s' % classpath)
#options.append("interrupt=True")

But when I implement the interrupt is crashes/ does not load correctly.
What am I doing wrong
Ref: docs

[Thrameos]

Kwargs is a dict. Args is a tuple. I usually use something like

jpype.startJVM(classpath="pathtojars", interrupt=False)

If you need to store it prior to calling then make a dict and use the dictionary deal operator.

kwargs = { classpath:"pathtojar", interrupt:False }
jpype.startJVM(**kwargs)

[chappers73]

@Thrameos, after a whole lot of tests adding the interrupt=False and this in the main thread

jpype.java.lang.Thread.detach()
time.sleep(0.3)
jpype.java.lang.Thread.attachAsDaemon()
time.sleep(0.3)
jpype.shutdownJVM()

I can actually now detect the CTRL-C (keyboard) and shut the system down gracefully, I am very grateful for your help this has been driving me crazy for nearly 2 weeks. No idea why the above works but without the detach & attach the shutdownJVM blocks even though I only have the main thread left running and the JVM process,
Again many thanks,

[Thrameos]

Sounds like you have a thread which is attached as something other than daemon. Daemon thread don't block shutdown but normal ones do (JVM rules). I am guessing that something in JayDeBeAPI must be doing an attach call which is wrong. I should likely deprecate the attach call as there really is no need to do attach unless you manually create a thread in Python that you want to live on as Java but that path is always dangerous as it will crash at shutdown.

Glad to hear that I was able to help.

[Thrameos]

The problem is line 201 in JayDeBeAPI. Using attachThreadToJVM() will cause the thread to hang shutdown. I am not sure why this line would be necessary though there must have been a case in which the thread was not attached at some point. JPype has automatically attached threads for a while and attaching using this call will prevent shutdown unless the thread were the main thread. Earlier versions of JPype did not execute the shutdown sequence properly so it would terminate rather than hang which hid this particular bug.