AttributeError: 'Delegate' object has no attribute '_library' when trying to execute quant_decoder

Question

AttributeError: 'Delegate' object has no attribute '_library' when trying to execute quant_decoder

lupitia1 opened this issue 4 years ago · 12 comments

@ivelin I have gone through precisely this a while back and am trying to update the implementation currently. I will do my best to provide what I did.

Does that mean EdgeTPU knows how to resolve the CustomOp reference in the graph and execute it?

Not quite, at least not as far as I can tell. It appears to be provided by the EdgeTPU library.

Is there a way to help TFLite resolve the CustomOp reference in the graph or that's an EdgeTPU feature only?

I was previously unable to find a way.

Looks like one way to inform TFLite of custom ops is to rebuild it from source. However that requires the CustomOp implementation to be available at build time.

Correct. Fortunately, it is. As I mentioned above, it appears to be part of the Edge TPU library. I simply build this code into my binary. You can find the op here: https://github.com/google-coral/edgetpu/blob/master/src/cpp/posenet/posenet_decoder_op.cc

Related code that is likely to be needed as well is available there as well. I have successfully used this in CPU only on Linux. I am working on a build for Windows. I am not fond of the Bazel build system, particularly because for whatever reason tensorflow does not use up to date versions of Bazel and building on Windows is not easy. Frankly neither is building on Linux but there is at least the Docker image available there.

Hello jwoolston! I'm trying to do the same right now but I'm facing the next problem:

AttributeError: 'Delegate' object has no attribute '_library' (decoder) [ec2-user@ip-172-31-5-112 mobilenet]$ vim run_model.py (decoder) [ec2-user@ip-172-31-5-112 mobilenet]$ python run_model.py }Traceback (most recent call last): File "run_model.py", line 3, in <module> tpu= tflite.load_delegate('libedgetpu.so.1') File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 166, in load_delegate delegate = Delegate(library, options) File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 90, in __init__ self._library = ctypes.pydll.LoadLibrary(library) File "/usr/local/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary return self._dlltype(name) File "/usr/local/lib/python3.7/ctypes/__init__.py", line 364, in __init__ self._handle = _dlopen(self._name, mode) OSError: libedgetpu.so.1: cannot open shared object file: No such file or directory Exception ignored in: <function Delegate.__del__ at 0x7fa6194d70e0> Traceback (most recent call last): File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 125, in __del__ if self._library is not None: AttributeError: 'Delegate' object has no attribute '_library'

When executing this code:
`import tflite_runtime.interpreter as tflite

tpu= tflite.load_delegate('../posenet-tflite-convert/data/edgetpu/libedgetpu/direct/aarch64/libedgetpu.so.1')
#posenet = tflite.load_delegate('posenet_decoder.so')
interpreter = tflite.Interpreter(model_path='posenet_mobilenet_v1_075_481_641_quant_decoder.tflite')
interpreter.allocate_tensors()`

I used this docker to generate the .so libraries required:
https://github.com/muka/posenet-tflite-convert

But I'm not sure how I have to connect the libraries with my code or the tflite_runtime.

Please, could you help me to have a better vision of this problem or tell me more about the solution you used?

I'm working on Amazon Linux 2.

Greetings!

Originally posted by @lupitia1 in #36 (comment)

Answer 1 · 2020-12-21T16:10:58.000Z

AttributeError: 'Delegate' object has no attribute '_library'

That usually means that it can't find the .so file that you are trying to load.

Checkout how I did it here (I have the posenet_decoder.so already compiled):
https://github.com/google-coral/project-posenet/blob/ugrade-to-frogfish/pose_engine.py#L80

I'm working on Amazon Linux 2.

Is this a cloud instance? Because if so, then how are you planning to attach an edgetpu hardware to it?

Answer 2 · 2020-12-23T08:22:31.000Z

AttributeError: 'Delegate' object has no attribute '_library'

That usually means that it can't find the .so file that you are trying to load.

Checkout how I did it here (I have the posenet_decoder.so already compiled):
https://github.com/google-coral/project-posenet/blob/ugrade-to-frogfish/pose_engine.py#L80

I'm working on Amazon Linux 2.

Is this a cloud instance? Because if so, then how are you planning to attach an edgetpu hardware to it?

Hello Namburger! Thank you for your reply,

I'm trying to use this model: Posenet in CPU (it works perfect in my edgeTPU but Im required to make it work on the CPU also) and I'm having problems with the decoder cause it doesn't find the custom op "PosenetDecoderOp". I tried to add it to Tensorflow and compile it from source but I'm having different small "problems" and it is taking too long. I will write here when it finish and share if it works or not.

Greetings!

Answer 3 · 2020-12-26T23:53:32.000Z

@lupitia1 I see, if you are just using the CPU model, then just don't load the eddetpu delegate at all because you either dont have that installed or don't have an edgetpu HW

        posenet_decoder_delegate = load_delegate('path/to/posenet_decoder.so')
        self._interpreter = Interpreter(
            model_path, experimental_delegates=[posenet_decoder_delegate])
        self._interpreter.allocate_tensors()

Answer 4 · 2020-12-28T10:18:06.000Z

@lupitia1 I see, if you are just using the CPU model, then just don't load the eddetpu delegate at all because you either dont have that installed or don't have an edgetpu HW
        posenet_decoder_delegate = load_delegate('path/to/posenet_decoder.so')
        self._interpreter = Interpreter(
            model_path, experimental_delegates=[posenet_decoder_delegate])
        self._interpreter.allocate_tensors()

Hello again! I was trying to use the model Posenet quant _decoder in the cpu version (in the cloud instance) but I couldn't cause it ask me to use the posenet_decoder_op and this is why I tried loading the delegate but now that I tried the following code based in the one you suggested:

  import tflite_runtime.interpreter as tflite
  
  tpu= tflite.load_delegate('posenet_decoder.so')
  interpreter = tflite.Interpreter(model_path='posenet_mobilenet_v1_075_481_641_quant_decoder.tflite')
  interpreter.allocate_tensors()

I get this error:

  
  Traceback (most recent call last):
    File "run_model.py", line 3, in <module>
      tpu= tflite.load_delegate('posenet_decoder.so')
    File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 166, in load_delegate
      delegate = Delegate(library, options)
    File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 90, in __init__
      self._library = ctypes.pydll.LoadLibrary(library)
    File "/usr/local/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary
      return self._dlltype(name)
    File "/usr/local/lib/python3.7/ctypes/__init__.py", line 364, in __init__
      self._handle = _dlopen(self._name, mode)
  OSError: posenet_decoder.so: cannot open shared object file: No such file or directory
  Exception ignored in: <function Delegate.__del__ at 0x7f90b1ed60e0>
  Traceback (most recent call last):
    File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 125, in __del__
      if self._library is not None:
  AttributeError: 'Delegate' object has no attribute '_library'

I have copied the posenet_decoder.so file to the same folder and I think it should work cause all the forums and threads I have read seems to solve this in an easy way. I don't know what is happening.

PD: In another work around, I also compiled Tensorflow from source with custom operators (I extracted the operators from edgetpu repository) but it doesn't recognize them when I run the code.

Answer 5 · 2020-12-28T13:47:30.000Z

@lupitia1
Just checkout my upgrade-to-frogfish branch: https://github.com/google-coral/project-posenet/tree/ugrade-to-frogfish
take off the edgetpu delegate and it should works, it works for me so I believe it has to be a path issue. As a matter of fact,

AttributeError: 'Delegate' object has no attribute '_library'

means that python couldn't find the so.

Another thing you can do is to set thelibrary path:

export LD_LIBRARY_PATH=/path/to/dir/containing/library/

Answer 6 · 2020-12-28T16:23:58.000Z

Thank you again,
I tried the checkout to this branch and exporting as an environment variable. I'm sure the path is okay cause I printed it from Python file:
(decoder) [ec2-user@ip mobilenet]$ dir /home/ec2-user/project-posenet/posenet_lib/x86_64/ posenet_decoder.so

But when executing the code I refered before, I'm getting the same error:

python3 run_model.py
/home/ec2-user/project-posenet/posenet_lib/x86_64/posenet_decoder.so
Traceback (most recent call last):
  File "run_model.py", line 4, in <module>
    tpu= tflite.load_delegate(os.environ.get('LD_LIBRARY_PATH'))
  File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 166, in load_delegate
    delegate = Delegate(library, options)
  File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 90, in __init__
    self._library = ctypes.pydll.LoadLibrary(library)
  File "/usr/local/lib/python3.7/ctypes/__init__.py", line 442, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/lib/python3.7/ctypes/__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/ec2-user/project-posenet/posenet_lib/x86_64/posenet_decoder.so)
Exception ignored in: <function Delegate.__del__ at 0x7ff61ad18170>
Traceback (most recent call last):
  File "/home/ec2-user/decoder/lib/python3.7/site-packages/tflite_runtime/interpreter.py", line 125, in __del__
    if self._library is not None:
AttributeError: 'Delegate' object has no attribute '_library'

I really appreciate your help and if you need more information about this let me know and I will add it here. All I want to do is run a prediction in a CPU with the posenet model.

Answer 7 · 2020-12-28T16:31:18.000Z

Ohhh, I see why...

OSError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /home/ec2-user/project-posenet/posenet_lib/x86_64/posenet_decoder.so)

It is worse than a path issue, unfortunately :/
The posenet_decoder.so that I built requires glibc 2.27, I believe the OS that you have on the ec2 instance is just really old and doesn't have glibc2.27. You can check with:

ldd --version

Answer 8 · 2020-12-29T08:44:57.000Z

You are right :/

ldd --version
ldd (GNU libc) 2.26
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

I will see if it is possible to update or I will change to a newer OS. I will update here with the result.

Answer 9 · 2020-12-29T15:45:41.000Z

You are right :/

ldd --version
ldd (GNU libc) 2.26
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

I will see if it is possible to update or I will change to a newer OS. I will update here with the result.

I messed up the instance that has GLIBC 2.26 trying to update so I changed to another one with a new OS (Ubuntu 18.04 LTS) with GLIBC 2.27.

ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1.2) 2.27
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

I tried to run the same code and now it is looking for GLIBC 2.29:

Traceback (most recent call last):
  File "run_model.py", line 2, in <module>
    tpu= tflite.load_delegate('./posenet_lib/x86_64/posenet_decoder.so')
  File "/usr/local/lib/python3.6/dist-packages/tflite_runtime/interpreter.py", line 152, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.6/dist-packages/tflite_runtime/interpreter.py", line 81, in __init__
    self._library = ctypes.pydll.LoadLibrary(library)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by ./posenet_lib/x86_64/posenet_decoder.so)
Exception ignored in: <bound method Delegate.__del__ of <tflite_runtime.interpreter.Delegate object at 0x7f313df1fdd8>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tflite_runtime/interpreter.py", line 116, in __del__
    if self._library is not None:
AttributeError: 'Delegate' object has no attribute '_library'

After searching about it in Google I ended reading about cross compiling. Now I have doubts if this must be executed in a 32-bit architecture system. Is possible that my 64-bit architecture is causing this problem?

UPDATE:
I ran the same steps in Ubuntu 20.04 with GLIBC 2.29 and it does recognice the library. But now it is asking me again for the operator:

 python3 run_model.py
Traceback (most recent call last):
  File "run_model.py", line 4, in <module>
    interpreter.allocate_tensors()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 259, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Encountered unresolved custom op: PosenetDecoderOp.Node number 32 (PosenetDecoderOp) failed to prepare.

I think I saw a compiled "posenet_decoder_op.so" in the edgetpu repo but I'm not sure if this could work. I will try and update.

Answer 10 · 2020-12-29T17:26:13.000Z

UPDATE:
I ran the same steps in Ubuntu 20.04 with GLIBC 2.29 and it does recognice the library. But now it is asking me again for the operator:
 python3 run_model.py
Traceback (most recent call last):
  File "run_model.py", line 4, in <module>
    interpreter.allocate_tensors()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/tflite_runtime/interpreter.py", line 259, in allocate_tensors
    return self._interpreter.AllocateTensors()
RuntimeError: Encountered unresolved custom op: PosenetDecoderOp.Node number 32 (PosenetDecoderOp) failed to prepare.
I think I saw a compiled "posenet_decoder_op.so" in the edgetpu repo but I'm not sure if this could work. I will try and update.

UPDATE: Now it works! :D
I was missing a little detail, passing the loaded delegate when loading the model, this is my code:

import tflite_runtime.interpreter as tflite
posenet=tflite.load_delegate('./posenet_lib/x86_64/posenet_decoder.so')
interpreter = tflite.Interpreter(model_path='posenet_mobilenet_v1_075_481_641_quant_decoder.tflite',experimental_delegates=[posenet])
interpreter.allocate_tensors()
# Get input and output tensors.
input_details_decoder = interpreter.get_input_details()
print(input_details_decoder)
output_details_decoder = interpreter.get_output_details()

I used the compiled posenet_decoder.so you told me to use in Ubuntu 20.04 and it works perfectly. Thank you so much for all your help and support @Namburger.

Answer 11 · 2020-12-29T17:29:38.000Z

@lupitia1 awesome, glad to help!

Answer 12 · 2020-12-30T11:16:45.000Z

Hello! I don't know if I have to create a new Issue for this:

Just realized that I need to do this on Windows also to test some features of my project :(
I tried to do exactly the same I did on Linux. After installing the tflite_runtime on Windows 10 (64 bits arch) I guess the [posenet_decoder.so] is not prepared to run on Windows. Is there a way to make this work on Windows too? I saw your pull request "Added TODO for window/mac supports" but is not working for me in this particular case.