vertexAI: response_schema can't parse TypedDicts in generation config
Opened this issue · 3 comments
Description of the bug:
If a TypedDict is supplied in the response_schema field, as shown in the examples of the doc here, we get the following error:
python3.10/site-packages/vertexai/generative_models/_generative_mode
ls.py", line 2200, in _fix_schema_dict_for_gapic_in_place
if "type" in schema_dict:
TypeError: argument of type '_TypedDictMeta' is not iterable
It appears this functionality is supported in the genai library but not vertex AI?
Actual vs expected behavior:
Expected behaviour:
When a typedDict is passed as the input for response_schema, we should receive structured outputs in that format based on the documentation provided above. However, for the following code:
class CodeOutput(TypedDict):
line_number: int
original_line: str
new_line: str
conventional_commit_message: str
description: str
And the following generation_config -
self._generation_config = GenerationConfig(
response_mime_type="application/json",
response_schema=CodeOutput,
)
self._model.generate_content(
content,
safety_settings=self._safety_config,
generation_config=self._generation_config,
stream=False,
)
I receive the error:
File "/Users/shourya.sharma/dev/scrapesmith/src/scrapesmith/client.py", line 122, in __init__
self._generation_config = self._setup_generation_config()
File "/Users/shourya.sharma/dev/scrapesmith/src/scrapesmith/client.py", line 142, in _setup_generation_config
return GenerationConfig(
File "/Users/shourya.sharma/Library/Caches/pypoetry/virtualenvs/scrapesmith-9e6PYqsH-py3.10/lib/python3.10/site-packages/vertexai/generative_models/_generative_mode
ls.py", line 1747, in __init__
raw_schema = FunctionDeclaration(
File "/Users/shourya.sharma/Library/Caches/pypoetry/virtualenvs/scrapesmith-9e6PYqsH-py3.10/lib/python3.10/site-packages/vertexai/generative_models/_generative_mode
ls.py", line 2161, in __init__
_fix_schema_dict_for_gapic_in_place(parameters)
File "/Users/shourya.sharma/Library/Caches/pypoetry/virtualenvs/scrapesmith-9e6PYqsH-py3.10/lib/python3.10/site-packages/vertexai/generative_models/_generative_mode
ls.py", line 2200, in _fix_schema_dict_for_gapic_in_place
if "type" in schema_dict:
TypeError: argument of type '_TypedDictMeta' is not iterable
Current dev environment:
- MacOS Darwin (M2 Macbook Pro)
- Python 3.10.14
- VertexAI version 1.71.1
Any other information you'd like to share?
No response
You seem to be referencing the Python SDK code for Vertex AI's Generative AI. If you're looking for guidance on structured output in Vertex AI, you can refer to this link
Thanks
Is there a reason the 2 APIs (Vertex AI vs GenAI) don't have parity or same featureset for gemini models?