mila-iqia/blocks

inspect.getargspec incompatible with kwonlyargs and function annotations

Opened this issue · 2 comments

If inspect.getargspec function is used to inspect a function with with kwonlyargs or function annotations a ValueError is raised.

In python3 a inspect.getfullargspec function exists witch could be used as a replacement but both getargspec and getfullargspec are deprecated in python >= 3.5 so it might be the best to use signatur() witch nevertheless is a bit more complex.

(Note that function annotations refer to PEP3107 and not to the annotation of a function...
e.g def f(a: ParamAnnotation) -> ReturnAnnotation: pass )

Because this is only relevant for people witch really want to use such functions in a project in on of the 5 context in witch blocks uses getargspec (e.g. the recurrent annotation) I personally don't think this has a high priority for now ;)

Can you elaborate on PEP3107?

We are really trying to use the modern python even though we have to be compatible with python 2.

Does signatur() exist in legacy python and python 3.[1-4]? If so, we are happy to merge a pull request!

There are two features in python3(only) witch change the signature of a function.
PEP3107 allows you add annotations to parameters and return types when defining a function (with ....parametername: annotation... and ...) -> annotation: ... ) and PEP3102 allows keyword-only arguments.

The python2 compatible inspect.getargspec function cannot represent the argspec/signature without breaking existing code (the namedtuple would need more parameter breaking unpacking). That's why they decided, that a exception is raised if a function with one of those is inspected with getargspec.

The getfullargspec function is practically getargspec+missing parts for new functions, but was deprecated in python3.5, instead the signatur function should be used (witch was introduced in python 3.3), for the usages in this project the main benefit for signature (compared to getfullargspec) would be small mainly the non usage of a deprecated function.

Looking it up, there seems to be a backport for inspect.signature for python with 2.6,2.7,(3.2), 3.3+ support but I haven't used it (it's called funcsigs)

Note that other than the python2->3 switch the 3.x updates are backward compatilbe (maybe expect some security fixes and bug as feature uses) and therefore all 3.x relases on itself have a smaller support frame (~5y), e.g. there are no updates (not even security ones) for python <=3.2 so only supporting python2 & python3.3+ (and later on 3.4+ etc.) should be fine.

To elaborate on PEP3107 ;):
well like mentioned above it allows you to add parameter/return type annotations, they do not change the function behavior on itself (but it's signature) and can be used e.g. by @ annotations or other forms of meta-programming. It seems to used for mainly two thinks:

  • giving optional (ignorable) type hints, witch among others can improve IDE's auto-completion. Through it's possible to have some kind of @check_types annotation witch wraps the function and adds some actual type checks.
  • mark some parameter for some annotation

e.g. recurrent could be rewritten in python3:

@recurrent(sequences=['inputs'], contexts=[],
                  states=['state],  outputs=['state'])
def apply(self, inputs, state=None):
    ...

#given some impl. for SequenceItem,State,ParamRef
@py3recurrent
def apply(self, inputs: SequenceItem, state: State =None) -> PramRef("state"):
   ...

But then such features are python3 only, so as long as there is python2 support,
it would be enough to change the usage of getargspec "sometime" in the future.
(even if people use this feature already they will not have a problem as long as it's
not happen to be a function annotated with @recurrent or @application)