IntelliSense support for Pydantic models
tiangolo opened this issue · 15 comments
I would like to have completion while creating an instance of a Pydantic model (class).
It's relatively similar to dataclasses and attrs. It performs validation and documentation on data. And it's heavily used in FastAPI, and now also in spaCy and Thinc (among many other tools).
For example, with a model like:
from typing import List
from pydantic import BaseModel
class User(BaseModel):
id: int
name: str
tags: List[str] = []When creating a new instance like:
user = User()I would like to receive completion inside of User() for id, name, and tags, telling me that those are the arguments expected, their types, and default values. Just as if it was a normal Python class like:
class User:
def __init__(self, id: int, name: str, tags: List[str] = []):
self.id = id
self.name = name
self.tags = tags
Option 1: support in the language server
The obvious first idea would be to have first-class support in the language server and VS Code Python extension itself.
But I understand that as this is related to a quite specific library and might be out of scope for the language server itself.
Option 2: Extending Pydantic to help the language server
Does the language server read that data from somewhere in the objects?
I know Pydantic has some code to provide a "proper" __init__ method for its models, and there's a mypy plugin.
So, if it's something that could be added to Pydantic for the VS Code Python extension and the language server to be able to read it, I could try to add it to Pydantic.
Option 3: an external VS Code plugin
How would be the appropriate way to build a plug-in to support it that integrates well/doesn't conflict with the language server?
For example, there's a PyCharm plugin for Pydantic https://plugins.jetbrains.com/plugin/12861-pydantic.
But VS Code is currently my favorite editor, so I would like to have something similar here...
I could try something to support this, but I have no knowledge of C# so I wouldn't be able to help much in the language server itself, I know TypeScript, though, so maybe I could try a VS Code plugin, but I would like some pointers to make sure I don't build something that conflicts with the VS Code Python extension, etc.
Pyright (which is the engine that Pylance uses for static type evaluation) provides full support for the type annotation standards defined in the Python specification, including all related PEPs. Unfortunately, the Pydantic interface cannot be described using standard Python type annotations, so "Option 2" is not viable.
Adding full support for Pydantic would require custom code. We've established a principle that we won't add custom logic for libraries in the type checker. Doing so would be a slippery slope and would lead to an unmaintainable code base over time, so it's unlikely that we would go with your "Option 1".
Option 3 would require us to write a plugin mechanism for Pyright, something that we may eventually do but is not likely to happen any time soon. We're currently focused on improving the experience for libraries that can be described by the standard Python type annotations.
Is it feasible for you to switch from Pydantic to dataclass, which is now part of the Python standard? Pyright has full support for dataclass. Or perhaps the next major version of Pydantic could be built on top of dataclass?
Thanks for your response, and again, awesome job with Pyright! 🎉
So, Pydantic can't be easily re-implemented based on dataclasses because it has to do extra logic that standard dataclasses don't do, like data validation, transformation, and serialization (dealing with metaclasses and black magic).
In fact, Pydantic provides its own dataclasses that can be used instead of the standard dataclasses to add some of the Pydantic features (not all) to those classes.
FastAPI can't be built (re-implemented) on top of dataclasses because it depends on that data validation, serialization, and documentation (with automatic JSON Schema) that is provided by Pydantic.
Nevertheless, I understand that the focus right now is in the standard library and I see you have been doing a great job with it.
I guess the current best approach would probably be to create an external VS Code extension for Pydantic, but then I wish there was a way to re-use the information that Pyright has already extracted to not duplicate/fork that work. I actually don't even know if that would make sense, I haven't checked the code and what would be necessary.
Another wild idea (that might be off-topic here) is to try to use https://github.com/Instagram/LibCST as the base of a language server for Pydantic, just to support those Pydantic-specific features. But I imagine the amount of work to build a language server with that must be enormous, given that writing a simple mypy plugin (that has to somewhat deal with an AST) is already a massive amount of work.
Hi everybody, and thanks for the great work in Pyright, Pydantic and Pylance (and VS Code).
Didn't @erictraut exclude option 2 too quickly? While it can't support the full model of Pydantic (with validations), it could give type hints when calling __init__().
Assuming Pyright obtains type information by calling typing.get_type_hints() and since @tiangolo says that
Pydantic has some code to provide a "proper"
__init__method for its models
it would require that this __init__ method would have the proper __annotations__.
Furthermore, if Pydantic would store the extended information in a PEP-593 Annotated Type (now in Python 3.9) and according to Eric, Pyright will support all type-related PEPs, it could show such extended information at least in a __str__ form.
Pyright is a static type checker. It intentionally does not run any of the code it is checking, nor should it! get_type_hints is a runtime call.
Pyright does support PEP 593, although it interprets only the standard type information provided in Annotated types, not the extended type information, which is not specified as part of the spec. It would be possible to display extended Annotated type information (e.g. in hover text) without interpreting it. But those annotations would need to be specified in the source code in a static form, not dynamically built at runtime.
Hi @erictraut - my apologies - I'm still confused about the constraints of static analysis in this very dynamic language. Plugins with some common protocol seem to be desirable in the future.
Thanks again, and best regards
Martin
Not having pydantic model support is a pretty big show-stopper for any FastAPI users. In fact they even offer their own implementation of dataclasses, in from pydantic.dataclasses import dataclass - which completely breaks down when you're trying to use it with Pylance.
Looks like there are issues here and here already.
They have a dedicated mypy plugin, so if Pylance gets popular enough (and has good support for custom modules) maybe they can be convinced to create one for Pyright too?
(since pyright ain't gonna do it:)
Creating a custom extension model for Pyright is something we might eventually do, but it's not going to happen any time soon. And I'm hopeful that it won't be necessary in the long run. We are focusing efforts on improving the standard type system capabilities in Python and working with library authors to build interfaces (and provide stubs) that conform to those standards. That will eliminate the need for custom plugins over time and is a much better and scalable solution than requiring each smart editor, linter, and static type checker to know about non-standard special-case behaviors.
Oh, this does the trick:
if TYPE_CHECKING:
from dataclasses import dataclass
else:
from pydantic.dataclasses import dataclass as dataclassThis would be great to have, it's tough having to choose between a better dev experience (Pylance + stdlib dataclass), a better runtime experience (no autocomplete + Pydantic), and a sneaky hack (TYPE_CHECKING decorator).
For now Pydantic (and by extension this will hold for all FastAPI users) is this one big show stopper that keeps me with the charm that must not be named. Oh and a few refactorings but those could be hacked
For me this is the only issue with vscode right now.
The code below covers most cases for me.
if TYPE_CHECKING: # pragma: no cover
static_check_init_args = dataclasses.dataclass
else:
def static_check_init_args(cls):
return cls
Since this is the first result on many Google searches, I'll link this here:
Until pydantic/pydantic#2721 is merged, you can add a small snippet to pydantic to get completions. Instructions here
This is the repo for the old language server; the above would only apply to Pylance and pyright.
Thank you for the report. This project is no longer active. If you are using the Python language server in VSCode, the language server you are using is Pylance and please the http://github.com/microsoft/pylance-release to report issues.
If you are using Visual Studio, then, please use http://github.com/microsoft/PTVS to report issues.
thank you