moigagoo/cliar

Callable type annotations like `open` don't work when using `from __future__ import annotations`

AgarwalPragy opened this issue · 11 comments

AgarwalPragy commented

PEP 563 -- Postponed Evaluation of Annotations made it so that annotations are stored as string literals, instead of the actual objects.
The annotations must then be resolved at runtime. https://www.python.org/dev/peps/pep-0563/#resolving-type-hints-at-runtime


  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 163, in __init__
    self._register_commands(handlers)
  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 242, in _register_commands
    self._register_arg(command_parser, arg_name, arg_data)
  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 215, in _register_arg
    help=arg_data.help
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1358, in add_argument
    raise ValueError('%r is not callable' % (type_func,))
ValueError: 'open' is not callable
moigagoo commented

How do you get this error? I'm on Python 3.7.3 and everything is working fine. If you inspect arg.type after this line, you'll see that its type in the case of open is not string but a function.

AgarwalPragy commented

How do you get this error? I'm on Python 3.7.3 and everything is working fine. If you inspect arg.type after this line, you'll see that its type in the case of open is not string but a function.

Postponed evaluation of annotations isn't available yet in Python 3.7.3. It is a feature that will appear in Python 3.8 and will be default behavior in Python 4.0.

For now, one can enable (part of) the feature by doing from __future__ import annotations, which causes all obj.__annotations__ to be a dict from name to a string. The string must then be resolved at runtime to get the actual object back.

Example:

from __future__ import annotations
from typing import get_type_hints

class X:
	var: open

print(X.__annotations__['var'], type(X.__annotations__['var']))  # open <class 'str'>
hints = get_type_hints(X)
print(hints['var'], type(hints['var']))  # <built-in function open> <class 'builtin_function_or_method'>
moigagoo commented

Just added from __future__ import annotations to cliar.py and the tests still pass. What am I doing wrong?

AgarwalPragy commented

Just added from __future__ import annotations to cliar.py and the tests still pass. What am I doing wrong?

That's weird. It should not work.

Breaking example:

from __future__ import annotations
from cliar import Cliar
import sys

class Breaking(Cliar):
    def foo(self, some_file: open):
        print(some_file)

if __name__ == '__main__':
    print(sys.version)
    Breaking().parse()
/Users/pragy.a/Desktop/categorization/iab_tester/venv/bin/python /Users/pragy.a/Desktop/categorization/iab_tester/src/breaking.py
3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) 
[Clang 6.0 (clang-600.0.57)]
Traceback (most recent call last):
  File "/Users/pragy.a/Desktop/categorization/iab_tester/src/breaking.py", line 13, in <module>
    Breaking().parse()
  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 163, in __init__
    self._register_commands(handlers)
  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 242, in _register_commands
    self._register_arg(command_parser, arg_name, arg_data)
  File "/Users/pragy.a/Desktop/categorization/iab_tester/venv/lib/python3.7/site-packages/cliar/cliar.py", line 215, in _register_arg
    help=arg_data.help
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/argparse.py", line 1358, in add_argument
    raise ValueError('%r is not callable' % (type_func,))
ValueError: 'open' is not callable

Process finished with exit code 1

If you remove the future import, it works

from cliar import Cliar
import sys

class Breaking(Cliar):
    def foo(self, some_file: open):
        print(some_file)

if __name__ == '__main__':
    print(sys.version)
    Breaking().parse()
/Users/pragy.a/Desktop/categorization/iab_tester/venv/bin/python /Users/pragy.a/Desktop/categorization/iab_tester/src/breaking.py
3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 16:52:21) 
[Clang 6.0 (clang-600.0.57)]
usage: breaking.py [-h] {foo} ...

optional arguments:
  -h, --help  show this help message and exit

commands:
  {foo}       Available commands:
    foo

Process finished with exit code 0

Cliar Version: 1.2.3
OS: MacOS High Sierra (10.13.6)

moigagoo commented

Thanks for the detailed issue report. I was able to reproduce and fix the issue: f40abea

moigagoo commented

However, the fix works only in Python 3.7, so we're losing backward compatibility with 3.6 :-(

moigagoo commented

I'm still unsure if this is the right thing to do, at least at this point in time... AFAIU this change will not affect us until 4.0 is released, which is in the distant future.

If it's not the default behavior in 3.8, than things should continue to work as they are, right?

  • 👍1
moigagoo commented

Moved the fix to a separate PR, since I'm not entirely sure it should be applied right now: #6

AgarwalPragy commented

However, the fix works only in Python 3.7, so we're losing backward compatibility with 3.6 :-(

Oh, that's bad.

I'm still unsure if this is the right thing to do, at least at this point in time... AFAIU this change will not affect us until 4.0 is released, which is in the distant future.

If it's not the default behavior in 3.8, than things should continue to work as they are, right?

As long as someone can find this issue and refer to your fix, they can patch their local copies of Cliar to get it working, if they absolutely can't live without the future import. So this is good enough. Thanks :D

  • 👍1
moigagoo commented

OK nevermind, no compat break necessary. Merging and releasing Cliar 1.2.5 with the fix.