Proposal: Multiple Backends

[clegaard]

Motivation

The current approach embedding Python some complications compared to launching a new python interpreter.

Some of these

• Python extension module requires dynamic loading
• Some standard libraries behave differently, due to their reliance on global variables
• Virtual Environments seems brittle

Note that while, the name embedded suggests that the entire interpreter is included in the FMU this is not the case, it still requires that Python libraries is present on the system.

Backend - System Interpreter

My impression is that starting the Python interpreter from as a separate process and communicate with this using IPC.
The main benefit of doing this is that the python interpreter is working exactly as if it was launched from the command line.
The overhead of the communication would potentially be mitigated by less locking on the global interpreter lock.

Other Backends

Also having multiple backends
Multiple implementations of Python exist, some of which have nice properties of being fully portable or more efficient than CPython.

Proposal

Add support for selecting different Python "backends" that may be selected during export.

pyfmu export --project MyFMU --output exported/MyFMU --backend interpreter

This would add the following information to the slave_configuration.json file

{"slave_class": "Generator", "slave_script": "generator.py", "backend": "interpreter"}

Pros and Cons

Pros:

• Change is transparent to user, no changes to Python code necessary
• Less brittle backends, we rely only on "python" being in the path
• Better virtual environment support

Cons:

• Implementation effort
• More possibilities, more complexity for the user

Implementation Strategy

TBD