vitsalis/PyCG

PyCG fails when parsing files whose names are used in init file declarations

gdrosos opened this issue · 1 comments

Description

When PyCG processes a python package which includes a function/class declaration in the __init__.py, it fails to process the classes of a file which has the same name with the declaration, and is stored in the same directory.

Steps to Reproduce

  1. Declare in an __init__py file a function test()
  2. Create in the same directory a python file named test.py and declare a class within.
  3. Run: pycg __init__.py test.py

PyCG will yield the following error:

   ...
    File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/preprocessor.py", line 65, in analyze_submodule
    super().analyze_submodule(PreProcessor, modname,
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/base.py", line 485, in analyze_submodule
    visitor.analyze()
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/preprocessor.py", line 375, in analyze
    self.visit(ast.parse(self.contents, self.filename))
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/preprocessor.py", line 114, in visit_Module
    super().visit_Module(node)
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/base.py", line 61, in visit_Module
    self.generic_visit(node)
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 418, in generic_visit
    self.visit(item)
  File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ast.py", line 410, in visit
    return visitor(node)
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/preprocessor.py", line 368, in visit_ClassDef
    super().visit_ClassDef(node)
  File "/opt/homebrew/lib/python3.10/site-packages/pycg-0.0.6-py3.10.egg/pycg/processing/base.py", line 126, in visit_ClassDef
    self.scope_manager.get_scope(self.current_ns).reset_counters()
AttributeError: 'NoneType' object has no attribute 'reset_counters'

Root Cause

When PyCG visits a new module/class/function declaration it creates a new scope for the corresponding namespace.
Therefore, when pycg visits the test()method in the init file it creates a scope with the namespace modulepath.test

During the preprocessing phase, when PyCG visits the module test defined from test.py, it checks if the scope of the corresponding module namespace already exists in the scope manager:

root_sc = self.scope_manager.get_scope(self.modname)
if not root_sc:
# initialize module scopes
items = self.scope_manager.handle_module(self.modname,
self.filename, self.contents)
root_sc = self.scope_manager.get_scope(self.modname)

Since the namespace modulepath.test exists already due to the _init_.py file declaration, PyCG mistakenly percieves the test modules as a root module and does not initialize the scopes declared in the module correctly.

As a result, in the base processing stage occuring later, when PyCG visits a new class definition in module test,

self.scope_manager.get_scope(self.current_ns).reset_counters()

it resets the counters of the corresponding namespace (e.g. modulepath.test.Class1) declared in the scope manager, but since the scopes of the module test have not been initialized, the specific namespace and consequently the scope does not exists, resulting in an AttributeError, since NoneType object has no attribute reset_counters. The error occurs only in Class declarations and not in function declarations because currently PyCG checks if the specific function scope exists before resetting the counters:
if self.scope_manager.get_scope(self.current_ns):
self.scope_manager.get_scope(self.current_ns).reset_counters()

Propably this check throws the issue under the carpet.

Proposed Fix

In order to tacke this issue, ideally we should make PyCG able to distinguish the difference between a scope defined through the a declaration in an init file and a module declaration with the same name.
In order to implement this, we could implement a dictionary mapping each scope namespace to its type (e.g. module/function/class scope) and we could modify PyCG to match two scopes only when their type is the same.

Related to #50

Closing due to archival of repository.