theroyallab/tabbyAPI

[BUG] Structured Outputs?

Closed this issue · 0 comments

OS

Linux

GPU Library

CUDA 12.x

Python version

3.11

Describe the bug

I have been trying to create json structured outputs. I have a prompt that contains the json format in the response_format field in the request object. This is the standard (old) json schema (not a pydantic class). It looks like this...

Solved. I used the pydantic method and it worked fine.