unpy

Unified Python

Transpiles .pyi stubs from Python 3.13 to 3.10

Important

This project is in the alpha stage: You probably shouldn't use it in production.

Installation

$ pip install unpy

Usage

$ unpy --help
Usage: unpy [OPTIONS] SOURCE [OUTPUT]

Arguments:
  SOURCE    Path to the input .pyi file or '-' to read from stdin.  [required]
  [OUTPUT]  Path to the output .pyi file. Defaults to stdout.

Options:
  --version                       Show the version and exit
  --diff                          Show the changes between the input and
                                  output in unified diff format
  --target [3.10|3.11|3.12|3.13]  The minimum Python version that should be
                                  supported.  [default: 3.10]
  --help                          Show this message and exit.

Examples

Some simple examples of Python 3.13 stubs that are backported to Python 3.10.

Imports

$ unpy --target 3.10 --diff examples/imports.pyi

+++ -
@@ -1,6 +1,4 @@
- from types import CapsuleType
- from typing import override
- from warnings import deprecated
+ from typing_extensions import CapsuleType, deprecated, override

  @deprecated("RTFM")
  class Spam:
      __pyx_capi__: dict[str, CapsuleType]
      @override
      def __hash__(self, /) -> int: ...

Note the alphabetical order of the generated imports.

Type Aliases

$ unpy --target 3.10 --diff examples/type_aliases.pyi

+++ -
@@ -1,7 +1,15 @@
  from collections.abc import Callable
+ from typing import ParamSpec, TypeAlias, TypeVar
+ from typing_extensions import TypeAliasType, TypeVarTuple, Unpack

- type Binary = bytes | bytearray | memoryview
- type Vector[R: float] = tuple[R, ...]
- type tciD[V, K] = dict[K, V]
- type Things[*Ts] = tuple[*Ts]
- type Callback[**Tss] = Callable[Tss, None]
+ _R = TypeVar("_R", bound=float)
+ _V = TypeVar("_V")
+ _K = TypeVar("_K")
+ _Ts = TypeVarTuple("_Ts")
+ _Tss = ParamSpec("_Tss")
+
+ Binary: TypeAlias = bytes | bytearray | memoryview
+ Vector: TypeAlias = tuple[_R, ...]
+ tciD = TypeAliasType("tciD", dict[_K, _V], type_params=(_V, _K))
+ Things: TypeAlias = tuple[Unpack[_Ts]]
+ Callback: TypeAlias = Callable[_Tss, None]

Note that TypeAlias cannot be used with tciD because the definition order of the type parameters (at the left-hand side) does not match the order in which they are accessed (at the right-hand side), and the backported TypeAliasType must be used instead.

Functions

$ unpy --target 3.10 --diff examples/functions.pyi

+++ -
@@ -1,6 +1,11 @@
+ _T = TypeVar("_T")
+ _S = TypeVar("_S", str, bytes)
+ _X = TypeVar("_X")
+ _Theta = ParamSpec("_Theta")
+ _Y = TypeVar("_Y")
  from collections.abc import Callable as Def
- from typing import Concatenate as Concat
+ from typing import Concatenate as Concat, ParamSpec, TypeVar

- def noop[T](x: T, /) -> T: ...
- def concat[S: (str, bytes)](left: S, right: S) -> S: ...
- def curry[X, **Theta, Y](f: Def[Concat[X, Theta], Y], /) -> Def[[X], Def[Theta, Y]]: ...
+ def noop(x: _T, /) -> _T: ...
+ def concat(left: _S, right: _S) -> _S: ...
+ def curry(f: Def[Concat[_X, _Theta], _Y], /) -> Def[[_X], Def[_Theta, _Y]]: ...

Generic classes and protocols

$ unpy --target 3.10 --diff examples/generics.pyi

+++ -
@@ -1,17 +1,25 @@
- from typing import Protocol, overload
+ from typing import Generic, Protocol, overload
+ from typing_extensions import TypeVar
+
+ _T_contra = TypeVar("_T_contra", contravariant=True)
+ _T_co = TypeVar("_T_co", covariant=True)
+ _T = TypeVar("_T", infer_variance=True)
+ _D = TypeVar("_D")
+ _NameT = TypeVar("_NameT", infer_variance=True, bound=str)
+ _QualNameT = TypeVar("_QualNameT", infer_variance=True, bound=str, default=_NameT)

  class Boring: ...

- class CanGetItem[T_contra, T_co](Protocol):
-     def __getitem__(self, k: T_contra, /) -> T_co: ...
+ class CanGetItem(Protocol[_T_contra, _T_co]):
+     def __getitem__(self, k: _T_contra, /) -> _T_co: ...

- class Stack[T]:
-     def push(self, value: T, /) -> None: ...
+ class Stack(Generic[_T, _D]):
+     def push(self, value: _T, /) -> None: ...
      @overload
-     def pop(self, /) -> T: ...
+     def pop(self, /) -> _T: ...
      @overload
-     def pop[D](self, default: D, /) -> T | D: ...
+     def pop(self, default: _D, /) -> _T | _D: ...

- class Named[NameT: str, QualNameT: str = NameT]:
-     __name__: NameT
-     __qualname__: QualNameT
+ class Named(Generic[_NameT, _QualNameT]):
+     __name__: _NameT
+     __qualname__: _QualNameT

Note how TypeVar is (only) imported from typing_extensions here, which wasn't the case in the previous example. This is a consequence of the infer_variance parameter, which has been added in Python 3.12.

Project goals

Here's the alpha version of a prototype of a rough sketch of some initial ideas for the potential goals of unpy:

Towards the past
- Get frustrated while stubbing scipy
- Transpile Python 3.13 .pyi stubs to Python 3.10 stubs
- Package-level analysis and conversion
- Tooling for stub-only project integration
- Use this in scipy-stubs
- Gradually introduce this into numpy
Towards the future
- Beyond Python: $\text{Unpy} \supset \text{Python}$
- Language support & tooling for all .py projects
Towards each other
- Unified typechecking: Fast, reasonable, and language-agnostic

Features

Tooling

Target Python versions
- 3.13
- 3.12
- 3.11
- 3.10
- 3.9
Language support
- .pyi
- .py
Conversion
- stdin => stdout
- module => module
- package => package
- project => project (including the pyproject.toml)
Configuration
- --diff: Unified diffs
- --target: Target Python version, defaults to 3.10
- Project-based config in pyproject.toml under [tools.unpy]
- ...
Integration
- File watcher
- Pre-commit
- LSP
- UV
- VSCode extension
- (based)mypy plugin
- Project build tools
- Configurable type-checker integration
- Configurable formatter integration, e.g. ruff format
Performance
- Limit conversion to changed files

Stub backporting

Python 3.13 => 3.12
- PEP 742
  - typing.TypeIs => typing_extensions.TypeIs
- PEP 705
  - typing.ReadOnly => typing_extensions.ReadOnly
- PEP 702
  - warnings.deprecated => typing_extensions.deprecated
- PEP 696
  - Backport PEP 695 type signatures with a default
  - typing.NoDefault => typing_extensions.NoDefault
- Exceptions
  - asyncio.QueueShutDown => builtins.Exception
  - pathlib.UnsupportedOperation => builtins.NotImplementedError
  - queue.ShutDown => builtins.Exception
  - re.PatternError => re.error
- Typing
  - types.CapsuleType => typing_extensions.CapsuleType
  - typing.{ClassVar,Final} => typing_extensions.{ClassVar,Final} when nested
Python 3.12 => 3.11
- PEP 698
  - typing.override => typing_extensions.override
- PEP 695
  - Backport type _ aliases
  - Backport generic functions
  - Backport generic classes and protocols
  - typing.TypeAliasType => typing_extensions.TypeAliasType
- PEP 688
  - collections.abc.Buffer => typing_extensions.Buffer
  - inspect.BufferFlags => int
Python 3.11 => 3.10
- PEP 681
  - typing.dataclass_transform => typing_extensions.dataclass_transform
- PEP 675
  - typing.LiteralString => typing_extensions.LiteralString
- PEP 673
  - typing.Self => typing_extensions.Self
- PEP 655
  - typing.[Not]Required => typing_extensions.[Not]Required
- PEP 654
  - ~~builtins.BaseExceptionGroup~~
  - ~~builtins.ExceptionGroup~~
- PEP 646
  - typing.TypeVarTuple => typing_extensions.TypeVarTuple
  - typing.Unpack => typing_extensions.Unpack
  - *Ts => typing_extensions.Unpack[Ts] with Ts: TypeVarTuple
- asyncio
  - ~~asyncio.TaskGroup~~
- enum
  - enum.ReprEnum => enum.Enum
  - enum.StrEnum => str & enum.Enum
- typing
  - typing.Any => typing_extensions.Any if subclassed (not recommended)
Generated TypeVars
- De-duplicate extracted typevar-likes with same name if equivalent
- Prefix the names of extracted typevar-likes with _
- Rename incompatible typevar-likes with the same name (#86)

Simplification and refactoring

Generic type parameters
- Convert default=Any with bound=T to default=T
- Remove bound=Any and bound=object
- Infer variance of PEP 695 type parameters (#44)
  - If never used, it's redundant (and bivariant) (#46)
  - If constraints are specified, it's invariant
  - If suffixed with _co/_contra, it's covariant/contravariant
  - If used as public instance attribute, it's invariant
  - If only used as return-type (excluding __init__ and __new__), or for read-only attributes, it's covariant
  - If only used as parameter-type, it's contravariant
  - Otherwise, assume it's invariant
Methods
- Default return types for specific "special method" (#55)
- Transform self method parameters to be positional-only
Typing operators
- type[S] | type[T] => type[S | T]
- Flatten & de-duplicate unions of literals
- Remove redundant union values, e.g. bool | int => int

Beyond Python

@sealed types (#42)
Unified type-ignore comments (#68)
Set-based Literal syntax (#76)
Reusable method signature definitions (#97, #98)
Type-mappings, a DRY alternative to @overload
Intersection types (as implemented in basedmypy)
Higher-kinded types (see python/typing#548)
Inline callable types (inspired by PEP 677)

jorenham/unpy