Pydantic-based response schema not followed consistently, but dictionary-based schema is.

[suprateembanerjee]

Description of the bug:

I'm trying to generate slide deck content following a certain schema.

Pydantic Schema:

class SlideSchema(BaseModel):
    header: str
    subheader: str | None
    body: str | None
    image: str | None
    teachers_notes: str | None

Actual vs expected behavior:

The following code is highly inconsistent (2/10 success):

model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
                slides_prompt,
                generation_config={
                        'response_mime_type': 'application/json',
                        'response_schema': list[SlideSchema]})

However, the following code is highly consistent (10/10 success), although less readable and less structured:

model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
                slides_prompt,
                generation_config={
                        'response_mime_type': 'application/json',
                        'response_schema': {
                                'type': 'ARRAY',
                                'items': {
                                        'type': 'OBJECT',
                                        'properties': {
                                                'header': {'type': 'STRING', 'nullable': False},
                                                'subheader': {'type': 'STRING', 'nullable': True},
                                                'body': {'type': 'STRING', 'nullable': True},
                                                'image': {'type': 'STRING', 'nullable': True},
                                                'teachers_notes': {'type': 'STRING', 'nullable': True} },
                                        'required': ['header', 'subheader', 'body', 'image', 'teachers_notes']
                                        }
                                }
                        }

Any other information you'd like to share?

How do we incorporate the expected consistent behavior using Pydantic schema?

[manojssmk]

Hi @suprateembanerjee

What you mentioned is correct. The output is inconsistent when using the Pydantic model. Here’s the gist with the code.

Thanks