clj-python/libpython-clj

process hanging in embeded mode on exception

behrica opened this issue · 2 comments

As a FUP of #221 which provided a new pathway to see the exception,
but does not prevent the "hanging" in an other pathway.

The simplest scenario is that we have a hanging python process by using this clojure code crash.clj

(ns train
  (:require [libpython-clj2.python :refer [py.- py.] :as py]
            [libpython-clj2.python.ffi :as ffi]))

(def locked (ffi/lock-gil))
(py/import-module "not existing")

and load it in embedded mode using manual-gil via:

export JDK_JAVA_OPTIONS="-Dlibpython_clj.manual_gil=true"
python -c 'from clojurebridge import cljbridge;cljbridge.load_clojure_file(clj_file="crash.clj")'

The exception thrown by python on the last line in crash.clj is swallowed by either clojurebridge or libpython-clj and the process hangs forever.

I still have this open
behrica/clojurebridge#2

as I don't know where a potential fix needed to be.

Just to clarify.

There is an new API call which prevents the hanging seen above:

(py/with-manual-gil ...

But this API is by no mean mandatory, neither very prominently documented.
So users might still use the other API, as above, and get the experience of a hanging process.

I think we should throw an exception (and terminate the JVM) as well in the case above.