[BUG] Indexed PydanticObjectId with Pydantic v2 does not work properly
RobertoBochet opened this issue · 5 comments
RobertoBochet commented
Describe the bug
Hi, since I upgraded Pydantic from v1 to v2, if Document model has a indexed field of type PydanticObjectId then Pydantic raises an exception on boot
ImportError while loading conftest '/path/to/api/tests/conftest.py'.
[...]
api/models/treatment_plan/__init__.py:1: in <module>
from .treatment_intake import DBTreatmentIntake
api/models/treatment_plan/treatment_intake.py:21: in <module>
class TreatmentIntake(IdBase, TreatmentIntakeBase):
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:178: in __new__
complete_model_class(
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_model_construction.py:468: in complete_model_class
schema = cls.__get_pydantic_core_schema__(cls, handler)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/main.py:557: in __get_pydantic_core_schema__
return __handler(__source)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py:82: in __call__
schema = self._handler(__source_type)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:448: in generate_schema
schema = self._generate_schema(obj)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:689: in _generate_schema
return self._model_schema(obj)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:519: in _model_schema
{k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:519: in <dictcomp>
{k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:853: in _generate_md_field_schema
common_field = self._common_field_schema(name, field_info, decorators)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:905: in _common_field_schema
schema = self._apply_annotations(
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:1610: in _apply_annotations
schema = get_inner_schema(source_type)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py:82: in __call__
schema = self._handler(__source_type)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:1572: in inner_handler
from_property = self._generate_schema_from_property(obj, obj)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:610: in _generate_schema_from_property
schema = self._unpack_refs_defs(schema)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py:567: in _unpack_refs_defs
schema = flatten_schema_defs(schema)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:517: in flatten_schema_defs
return _simplify_schema_references(schema, inline=False)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:432: in _simplify_schema_references
schema = walk_core_schema(schema, collect_refs)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:407: in walk_core_schema
return f(schema, _dispatch)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:430: in collect_refs
return recurse(s, collect_refs)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:183: in walk
return f(schema.copy(), self._walk)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:430: in collect_refs
return recurse(s, collect_refs)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:186: in _walk
schema = self._schema_type_to_method[schema['type']](schema, f)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:195: in _handle_other_schemas
schema['schema'] = self.walk(sub_schema, f) # type: ignore
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:183: in walk
return f(schema.copy(), self._walk)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:430: in collect_refs
return recurse(s, collect_refs)
/home/roberto/.cache/pypoetry/virtualenvs/api-ANNybeXa-py3.11/lib/python3.11/site-packages/pydantic/_internal/_core_utils.py:186: in _walk
schema = self._schema_type_to_method[schema['type']](schema, f)
E KeyError: 'PydanticObjectId'
To Reproduce
class DBBadge(Document):
patientId: Indexed(PydanticObjectId)python=="3.11.5"
pydantic=="2.3.0"
beanie=="1.22.1"
Additional context
For the moment I moved the indexes in the Settings class, this "solve" the problem
roman-right commented
Reproduced. Thank you for the catch. I'll fix this soon
RobertoBochet commented
thank you, this may be the occasion to also implement #638
MrEarle commented
I believe I found a similar issue with UUID and EmailStr. This is happening to me on PydanticV2 and Beanie 1.23
Here's an example:
from uuid import uuid4
from beanie import Document, Indexed
from pydantic import UUID4, EmailStr, Field
class UserDB(Document):
"""User document model for MongoDB."""
uuid: Indexed(UUID4, unique=True) # type: ignore
email: Indexed(EmailStr, unique=True) # type: ignoreIf I only have the uuid field, I get KeyError: 'Annotated'. On the other hand, if I only have the email field, I get KeyError: 'EmailStr'.
The error doesn't happen if I remove Indexed. Update: If I replace Document with BaseModel, the issue presists.
roman-right commented
Thank you! I'll pick it up on the next bug-fixing sessions
roman-right commented
Hi @RobertoBochet @MrEarle ,
It looks like both issues are fixed. The second one can not be fixed directly, but Annotation can be used instead
Please try:
class DocumentWithIndexedObjectId(Document):
pyid: Indexed(PydanticObjectId)
uuid: Annotated[UUID4, Indexed(unique=True)]
email: Annotated[EmailStr, Indexed(unique=True)]