/expackage

ExamplePackage: Demonstrates Joblib/Loky Multiprocessing Issue in frozen App

Primary LanguagePython

expackage

ExamplePackage: demonstrates joblib/loky multiprocessing Issue in frozen executables

This repository should function as a minimal reproducible example for an issue that occurs with joblib/loky multiprocessing used in a frozen executable.

There are several branches that demonstrate different approaches, all leading to either a) a crash or b) a multiprocessing bomb

Branches:

  • pool use of multiprocessing.Pool
  • threadpool use of multiprocessing.ThreadPool
  • queuethread use of queue.Queue()and threading.Thread
  • contextpool use of multiprocessing.get_context("spawn").Pool()

Builds:

Download builds for each branch:

Logs (expackage.exe cli-mode):

  • pool.log

    • cluster function returns results
    • after "Press any key to exit...": OSError: [WinError 87] The parameter is incorrect

  • queuethread.log

    • multiprocessing bomb, failing with:
    joblib.externals.loky.process_executor.TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated.
...
ERROR: The process "16800" not found.
ERROR: The process "7964" not found.
ERROR: The process "13552" not found.
ERROR: The process "17616" not found.

  • threadpool.log

    • equal to queuethread

  • contextpool (existing pool object)

    • no error, but slow