arduino/arduino-cli

Python process left running if an installed platform uses a Python script-based pluggable discovery tool

rei-vilo opened this issue · 3 comments

rei-vilo commented

Describe the problem

Arduino CLI allows boards platforms to use custom "pluggable discovery" tools.

The "Raspberry Pi Pico/RP2040" boards platform uses a custom pluggable discovery. This discovery is a Python script, which is executed via the Python interpreter:

https://github.com/earlephilhower/arduino-pico/blob/e60858c327690222594a8b35bc65f68e52491c5c/platform.txt#L119-L123

# Allow Pico boards to be auto-discovered by the IDE
#discovery.rp2040.pattern={runtime.tools.pqt-python3.path}/python3 -I "{runtime.platform.path}/tools/pluggable_discovery.py"
discovery.rp2040.pattern={runtime.platform.path}/system/python3/python3 -I "{runtime.platform.path}/tools/pluggable_discovery.py"
#pluggable_discovery.rp2040.pattern="{runtime.tools.pqt-python3.path}/python3" -I "{runtime.platform.path}/tools/pluggable_discovery.py"
pluggable_discovery.rp2040.pattern="{runtime.platform.path}/system/python3/python3" -I "{runtime.platform.path}/tools/pluggable_discovery.py"

🐛 The Python interpreter process that is created by Arduino CLI is not terminated on exit. This results in an accumulation of such processes over time.

To reproduce

$ arduino-cli version

arduino-cli  Version: git-snapshot Commit: a39f9fdc Date: 2025-03-13T13:55:21Z

$ arduino-cli --additional-urls https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json core install rp2040:rp2040

[...]

$ ps --windows | grep 'python3'

$ ./arduino-cli board list

Port Protocol Type        Board Name FQBN Core
COM1 serial   Serial Port Unknown

$ ./arduino-cli board list

Port Protocol Type        Board Name FQBN Core
COM1 serial   Serial Port Unknown

$ ps --windows | grep 'python3'
    86316       0       0      20780  ?              0 07:25:01 C:\Users\per\AppData\Local\Arduino15\packages\rp2040\tools\pqt-python3\1.0.1-base-3a57aed-1\python3.exe
   115388       0       0      49852  ?              0 07:25:14 C:\Users\per\AppData\Local\Arduino15\packages\rp2040\tools\pqt-python3\1.0.1-base-3a57aed-1\python3.exe

🐛 A python3.exe process is left running after each command that causes Arduino CLI to invoke the rp2040:rp2040 platform's pluggable discovery tool.

Expected behavior

Clean exit of Arduino CLI with no zombie threads.

Arduino CLI version

a39f9fd

Operating system

Windows

Operating system version

11

Additional context

Discussion: https://forum.arduino.cc/t/closing-ide-2-3-2-leaves-python-32-bit-running/1283853

Issue checklist

  • I searched for previous reports in the issue tracker
  • I verified the problem still occurs when using the latest nightly build
  • My report contains all necessary details
earlephilhower commented

This may not be an issue related to the IDE so much as one related to the implementation of Windows Python Threading class. We use python3 in the arduino-pico core because it lets us do lots of complicated processing without needing to build 8 different versions of an executable every time a change needs to be made to the scripting. Just like I did on the ESP8266, I ship a stripped portable Python3 interpreter for Mac and Windows that's only used for the core and not installed in the system path/etc. Under Linux it's assumed part of any modern Linux install and just called.

The UF2 scanner I whipped up one morning after reading the spec since it didn't seem to be built-in (maybe the new IDE includes UF2 scanning already and this can be dropped?). It uses Python threading.Thread to implement the START_SYNC call and process add/removes for the RP2040/RP2350 UF2 mode.

I think there was an interaction with the input() call and the UF2 scanner thread causing the child scanner thread to never quit even if stdin was closed and the input() scanning call threw an exception.

I've tested earlephilhower/arduino-pico#2853 on Linux and Win11 and have not seen any dangling python3 processes in the Task Manager after running the arduino-cli commands above. However even with the older script I didn't always see dangling processes. So potentially there was a race condition somewhere in the Python internals with the original code.

@rei-vilo can you replace your copy of pluggable_discovery.py with the one in that PR and give it your own stress test and report back?

  • ❤️2
rei-vilo commented

@earlephilhower Thank you very much for your prompt answer and congratulations for the solution.

With pluggable_discovery.py from the PR, existing the Arduino IDE closes the Python tool gracefully.

I am thus closing the issue.

  • 🎉1
earlephilhower commented

Thanks for the report! Merged the PR in the core.

  • 👍1