Error Handling in _infer_injected_bindings

[HyperCharlie]

I am getting a weird issue where the injector is trying to get bindings for the builtin functools.partial method. This is C method, and inspect.getfullargspec(callable) on line 1159 is failing with "TypeError: unsupported callable". I have no idea where functools.partial is entering my injection stack; it seems to be something from multiple monkey patching but I am struggling to figure it out. However, I think we can make Injector just a bit better and solve my issue (potentially) by changing line 1159 in init.py from spec = inspect.getfullargspec(callable) to
try: spec = inspect.getfullargspec(callable) except TypeError: return {}

[HyperCharlie]

I can't push code to create a PR, but if you grant me some permissions I have a branch ready to go.

[jstasiak]

I'm sorry, you included almost no information needed to understand the issue. Please provide:

• your Python version
• the version of Injector you're using
• a complete stack trace
• ideally some code that illustrates the issue

[HyperCharlie]

Sorry about that. Was banging my head against a wall and got a bit ahead of myself.

I've created a recreate repo here: https://github.com/csibbach/NewRelicRecreate. Drop a newrelic key into newrelic.ini and run docker-compose up and you can see a complete breakdown of the issue. We're using Python 3.6 with Injector 0.18.3 and Flask-Injector 0.12.0.

When you fire up that container, you'll see this:

Attaching to newrelic-recreate_newrelic-recreate_1
newrelic-recreate_1  | Traceback (most recent call last):
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/inspect.py", line 1126, in getfullargspec
newrelic-recreate_1  |     sigcls=Signature)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/inspect.py", line 2273, in _signature_from_callable
newrelic-recreate_1  |     skip_bound_arg=skip_bound_arg)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/inspect.py", line 2097, in _signature_from_builtin
newrelic-recreate_1  |     raise ValueError("no signature found for builtin {!r}".format(func))
newrelic-recreate_1  | ValueError: no signature found for builtin <FunctionWrapper at 0x7f79ff48c798 for functools.partial at 0x7f79ff554908>
newrelic-recreate_1  | 
newrelic-recreate_1  | The above exception was the direct cause of the following exception:
newrelic-recreate_1  | 
newrelic-recreate_1  | Traceback (most recent call last):
newrelic-recreate_1  |   File "main.py", line 80, in <module>
newrelic-recreate_1  |     app = create_app()
newrelic-recreate_1  |   File "main.py", line 70, in create_app
newrelic-recreate_1  |     modules=[MyModule()],
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 317, in __init__
newrelic-recreate_1  |     process_dict(container, injector)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 344, in process_dict
newrelic-recreate_1  |     d[key] = wrap_fun(value, injector)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/site-packages/flask_injector.py", line 78, in wrap_fun
newrelic-recreate_1  |     return wrap_fun(inject(fun), injector)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/site-packages/injector/__init__.py", line 1367, in inject
newrelic-recreate_1  |     bindings = _infer_injected_bindings(function, only_explicit_bindings=False)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/site-packages/injector/__init__.py", line 1174, in _infer_injected_bindings
newrelic-recreate_1  |     spec = inspect.getfullargspec(callable)
newrelic-recreate_1  |   File "/usr/local/lib/python3.6/inspect.py", line 1132, in getfullargspec
newrelic-recreate_1  |     raise TypeError('unsupported callable') from ex
newrelic-recreate_1  | TypeError: unsupported callable

The issue is caused by NewRelic monkey patching the source libraries. If you comment that out, the code works fine. Now, obviously, this is not at it's heart an issue with Injector. However, I strongly suspect that putting some error handling around getfullargspec, as suggested above, would remedy the issue without hurting injector. We did a test some time ago and found calling getfullargspec(partial) spells instant doom, regardless of NewRelic.

[jstasiak]

functools.partial should be fine (this is Python 3.7, 3.8 and 3.9 I have locally):

>>> import functools
>>> import inspect
>>> 
>>> 
>>> def asd(a, b):
...     return a + b
... 
>>> 
>>> 
>>> p = functools.partial(asd, 1)
>>> 
>>> p(2)
3
>>> inspect.getfullargspec(p)
FullArgSpec(args=['b'], varargs=None, varkw=None, defaults=None, kwonlyargs=[], kwonlydefaults=None, annotations={})

Thanks for the detailed report and the code to reproduce! I can't run it as it requires NewRelic, but it should be helpful just as well.

[jstasiak]

I'm quite sure NewRelic is not patching the actual functools.partial (that would be... interesting) but it instead provides a wrapper of a result of a functools.partial() call (that's returned somewhere from your multi-layer decorator) and that fails to go through Injector.

If my understanding is correct there's no better thing for Injector (or Flask-Injector) to do here than to give up and raise an exception. The alternative would be to swallow the error and fail to provide you your dependencies in test_route in your example.

[HyperCharlie]

You are probably right; my suggested fix would just result in swallowing the exception and not provide any dependencies. Given that I have fixed this in the real codebase by removing that decorator, I think we can close this out for now. Thanks for your support and time! This is a great package and I'm not sure where I'd be without it.

[jstasiak]

Thank you for the kind words! I may be interested enough to get to the bottom of this when I have some time as I'm not sure what exactly does NewRelic do here but it remains to be seen if this happens. :)