sylikc/pyexiftool

Windows hang

Opened this issue · 8 comments

simonmcnair commented

I'm hoping to use this in Linux, so hopefully not an issue but makes dev in two different environments hard. Most of the time it just hangs at program exit, but sometimes I get an Exception as below:

Exception ignored in: <function ExifTool.__del__ at 0x0000022E9FDFA840>
Traceback (most recent call last):
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python311\Lib\site-packages\exiftool\exiftool.py", line 327, in __del__
    if self.running:
       ^^^^^^^^^^^^
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python311\Lib\site-packages\exiftool\exiftool.py", line 557, in running
    if self._process.poll() is not None:
       ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1234, in poll
    return self._internal_poll()
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Simon\AppData\Local\Programs\Python\Python311\Lib\subprocess.py", line 1530, in _internal_poll
    if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [WinError 6] The handle is invalid
simonmcnair commented 
INFO:__main__:Property 'executable': set to "C:\WINDOWS\exiftool.exe"
INFO:__main__:Property 'common_args': set to "['-G', '-n']"
INFO:__main__:Property 'config_file': set to "None"
INFO:__main__:Method 'execute': Command sent = [b'-ver', b'-echo4', b'=${status}=post886066']
DEBUG:__main__:ExifToolHelper.execute: IN  params = ('-ver',)
DEBUG:__main__:ExifToolHelper.execute: OUT stdout = "12.62
"
DEBUG:__main__:ExifToolHelper.execute: OUT stderr = ""
DEBUG:__main__:ExifToolHelper.execute: OUT status = 0
INFO:__main__:Method 'run': Exiftool version '12.62' (pid 22412) launched with args '['C:\\WINDOWS\\exiftool.exe', '-stay_open', 'True', '-@', '-', '-common_args', '-G', '-n']'
simonmcnair commented

It appears to me as if, at exit, exiftool isn't being killed ?

simonmcnair commented

interestingly, even if I call et.terminate it happens

sylikc commented

@simonmcnair this might be related to the hang which is a CPython bug I reported years ago python/cpython#87950 ... and you mentioned you're using it in Linux, but the output shows Windows?

sylikc commented

The CPython bug manifests itself when you initialize the ExifToolHelper and depend on the destructor to stop the process. The way to work around the bug is to destruct the object yourself, but deleting it.

sylikc commented

CPython 3.12 seems to fix this entire issue btw

roryai commented

I encountered this issue today on Windows 10, CPython 3.12.1. The same code (below) worked on my Mac. I haven't yet tried it on Linux.

My test was hanging in pycharm. When I stopped the test (using stop button in pycharm interface) the test passed. This indicates that exiftool was returning the correct result, and also hanging.

I think I understand why. I thought I'd let you know as a point of interest @sylikc

I have ExifToolHelper inside a helper class that is a singleton, so that I can use it anywhere without having to pass it down through the call chain. This was in an attempt to take advantage of the batch processing performance gains.

(I now realise that my understanding was flawed and I was indeed not getting the performance gains I had hoped for by using the singleton pattern. The exiftool process was terminating (on Mac) after each file it read. On Windows it was hanging on the first file).

This is the singleton wrapper code:

from exiftool import ExifToolHelper


class ExiftoolWrapperMeta(type):
    _instance = {}

    def __call__(cls, *args, **kwargs):
        if cls not in cls._instance:
            cls._instance[cls] = super(ExiftoolWrapperMeta, cls).__call__(*args, **kwargs)
        return cls._instance[cls]


class ExiftoolWrapper(metaclass=ExiftoolWrapperMeta):

    def exiftool(self):
        return ExifToolHelper()

I was able to get it working with my setup above by manually terminating the process. I modified ExifToolWrapper like so:

class ExiftoolWrapper(metaclass=ExiftoolWrapperMeta):


    def exiftool(self):
        self.et = ExifToolHelper()
        return self.et

    def terminate(self):
        self.et.terminate()

So it seems that it was perhaps relying on garbage collection to terminate the process on Windows? This doesn't explain why it worked on Mac though, as the singleton wrapper can't be deleted, wouldn't be garbage collected.

I'm not sure if this provides any insight but thought I'd share anyway.

On the off chance that you're interested in seeing how I used your tool (love it btw, thank you!) you can see this file here, which uses the ExifToolWrapper: https://github.com/roryai/transfer_wizard_redux/blob/485270a577b200826b34b50de3998ddd53859ff9/app/capture_time_identifier.py

I'm going to redesign it later this week so that I'm making proper use of the batch processing capabilities, and allowing the program to terminate itself on Windows gracefully.

sylikc commented

Hmm, that's odd that the singleton is terminating the exiftool process after each execute... anyways, Mac and Windows works slightly differently in terms of the "thread spawning at python interpreter shutdown". The only time the Windows hangs is in garbage collection when the script ends.

If you're having some other hangs when the script is running, there might be some other issue happening.

  • 👍1