pyutils/line_profiler

AttributeError after upgrading to line-profiler 4.0.0 from 3.5.1

Mogost opened this issue · 15 comments

Mogost commented

I do not see anything in changelog that may cause this.

Traceback (most recent call last):
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/contrib/staticfiles/handlers.py", line 76, in __call__
    return self.application(environ, start_response)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
    response = self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/base.py", line 130, in get_response
    response = self._middleware_chain(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/exception.py", line 49, in inner
    response = response_for_exception(request, exc)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/exception.py", line 149, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django_extensions/management/technical_response.py", line 41, in null_technical_500_response
    raise exc_value
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/middleware.py", line 58, in __call__
    response = toolbar.process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/timer.py", line 65, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/headers.py", line 46, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  [Previous line repeated 1 more time]
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/template_profiler_panel/panels/template.py", line 250, in process_request
    response = super(TemplateProfilerPanel, self).process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/staticfiles.py", line 116, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/logging.py", line 95, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 206, in process_request
    self._unwrap_closure_and_profile(self.view_func)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 198, in _unwrap_closure_and_profile
    self._unwrap_closure_and_profile(value)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 198, in _unwrap_closure_and_profile
    self._unwrap_closure_and_profile(value)
  File "/Users/user/.pyenv/versions/v_ver/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 179, in _unwrap_closure_and_profile
    self.line_profiler.add_function(func)
  File "line_profiler/_line_profiler.pyx", line 196, in line_profiler._line_profiler.LineProfiler.add_function
  File "line_profiler/_line_profiler.pyx", line 219, in line_profiler._line_profiler.LineProfiler.add_function
AttributeError: 'method' object has no attribute '__code__'

Connected with mikekeda/django-debug-toolbar-line-profiler#9

Erotemic commented

@Theelx can you take a look at this?

Theelx commented

Ah, I'll do some debugging now.

Theelx commented

@Mogost Can you provide some sample code to trigger this issue? I don't currently have my computer with me (I'm on my Github app now) so it'd help if I could see the code that's causing the issue and think about a solution while I'm on the move.

Theelx commented

I'm on my computer now, and when I add the @profile decorator to a method in a class, or when I create an instance of the LineProfiler class and call LineProfiler.add_function on the method, it works fine on CPython 3.8.13 and 3.9.13 (in pyenv). I see you're using 3.8, so that's interesting.

Theelx commented

I managed to reproduce it with this:

class Test:
    @profile
    @classmethod
    def method(self):
        print('hi')
Test.method()

That gives me this error:

/home/theel/.pyenv/versions/3.8.13/lib/python3.8/site-packages/line_profiler/line_profiler.py:51: UserWarning: Could not extract a code object for the object <classmethod object at 0x7fc05fb4e550>
  self.add_function(func)
Wrote profile results to test.py.lprof
Timer unit: 1e-06 s

Traceback (most recent call last):
  File "/home/theel/.pyenv/versions/3.8.13/bin/kernprof", line 8, in <module>
    sys.exit(main())
  File "/home/theel/.pyenv/versions/3.8.13/lib/python3.8/site-packages/kernprof.py", line 264, in main
    execfile(script_file, ns, ns)
  File "/home/theel/.pyenv/versions/3.8.13/lib/python3.8/site-packages/kernprof.py", line 32, in execfile
    exec(compile(f.read(), filename, 'exec'), globals, locals)
  File "test.py", line 3, in <module>
    class Test:
  File "test.py", line 6, in Test
    def method(self):
  File "/home/theel/.pyenv/versions/3.8.13/lib/python3.8/site-packages/line_profiler/line_profiler.py", line 54, in __call__
    elif is_generator(func):
  File "/home/theel/.pyenv/versions/3.8.13/lib/python3.8/site-packages/line_profiler/line_profiler.py", line 39, in is_generator
    isgen = (f.__code__.co_flags & CO_GENERATOR) != 0
AttributeError: 'classmethod' object has no attribute '__code__'

Luckily, there's a short-term fix:

class Test:
    @classmethod
    def method(self):
        print('hi')

Test.method = profile(Test.method)
Test.method()

Applying the profile decorator manually seems to somehow turn Test.method from a boundmethod to a function, which gives it the __code__ attribute.

hi
Wrote profile results to test.py.lprof
Timer unit: 1e-06 s

Total time: 2.061e-05 s
File: test.py
Function: method at line 4

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     4                                               @classmethod
     5                                               def method(self):
     6         1         20.6     20.6    100.0          print('hi')

Edit: I'm still going to do my best to fix this issue, however since there's a short-term fix, I need to prioritize my classwork and studying (I'm in college!) Hopefully, I can get a fix within 2-3 days.

Mogost commented

I can't really provide an example because I use https://github.com/mikekeda/django-debug-toolbar-line-profiler/ which is in turn a plugin for https://github.com/jazzband/django-debug-toolbar/ on a big Django project.
All I have is an exception trace.

It is definitely worth paying attention to the plugin itself, which I use.

Theelx commented

@Mogost Can you try using the Test.method = profile(Test.method) trick that I mentioned above, and see if it works?

Mogost commented 
Traceback (most recent call last):
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/contrib/staticfiles/handlers.py", line 76, in __call__
    return self.application(environ, start_response)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
    response = self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/base.py", line 130, in get_response
    response = self._middleware_chain(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/exception.py", line 49, in inner
    response = response_for_exception(request, exc)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/exception.py", line 114, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/exception.py", line 149, in handle_uncaught_exception
    return debug.technical_500_response(request, *exc_info)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django_extensions/management/technical_response.py", line 41, in null_technical_500_response
    raise exc_value
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/middleware.py", line 58, in __call__
    response = toolbar.process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/timer.py", line 65, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/headers.py", line 46, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  [Previous line repeated 1 more time]
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/template_profiler_panel/panels/template.py", line 250, in process_request
    response = super(TemplateProfilerPanel, self).process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/staticfiles.py", line 116, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/logging.py", line 95, in process_request
    return super().process_request(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar/panels/__init__.py", line 206, in process_request
    return self.get_response(request)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 206, in process_request
    self._unwrap_closure_and_profile(self.view_func)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 198, in _unwrap_closure_and_profile
    self._unwrap_closure_and_profile(value)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 198, in _unwrap_closure_and_profile
    self._unwrap_closure_and_profile(value)
  File "/Users/user/.pyenv/versions/var/lib/python3.8/site-packages/debug_toolbar_line_profiler/panel.py", line 179, in _unwrap_closure_and_profile
    self.line_profiler.add_function(func)
  File "line_profiler/_line_profiler.pyx", line 196, in line_profiler._line_profiler.LineProfiler.add_function
  File "line_profiler/_line_profiler.pyx", line 221, in line_profiler._line_profiler.LineProfiler.add_function
AttributeError: 'method' object has no attribute '__code__'

The problem still persists in 4.0.1

Theelx commented

Does using the profile() trick work?

cliffckerr commented

I'm also having this issue, although it's hard to reproduce. It caused my GitHub Actions tests to fail:
https://github.com/sciris/sciris/actions/runs/3615173776/jobs/6092102445

../sciris/sc_profiling.py:370: in profile
    wrapper = lp(run)
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/line_profiler/line_profiler.py:53: in __call__
    self.add_function(func)
line_profiler/_line_profiler.pyx:196: in line_profiler._line_profiler.LineProfiler.add_function
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   AttributeError: 'method' object has no attribute '__code__'

line_profiler/_line_profiler.pyx:221: AttributeError

This use case wraps LineProfiler by calling LineProfiler.add_function():
https://github.com/sciris/sciris/blob/v2.0.4/sciris/sc_profiling.py#L365

What's weird is that I can't reproduce this bug locally -- I have a fresh conda environment with everything seemingly the same as on GitHub, but the test passes.

cliffckerr commented

I'm not very experienced reading Cython code, so this could be a red herring, but the change I saw that seems most likely to be related to this bug is the change in the old version of the code from

<object>py_frame.f_code

being used to initialize the LineTiming class, to

code = func.__code__

being used in the new _code_replace() instead.

Theelx commented

I'm looking into this now with your new information. Thanks for reporting this!

  • ❤️1
Theelx commented

@cliffckerr It should be fixed in #191. Can you confirm that https://github.com/Theelx/sciris/actions/runs/3616146614 fixes the error and looks good before I ask @Erotemic to merge my PR?

cliffckerr commented

Yes that looks great -- thank you so much for the fast response @Theelx !

Mogost commented

Fixed by #191