direct output for json format plan

[yananchen1989]

hi author.

have u ever considered making the planner llm to directly out the plan which can be json parsable. which means that using eval to parse it, rather than sending it to gpt to for json conversion.

if not, may i know the reason ? is few shots demonstration not good enough to make it ?

as u know, requesting gpt for this json conversion costs money and time. and make the pipeline not that smooth.

looking forward your opinions.

thanks

[hsaest]

Hi Yanan,

It’s fine to directly output the plan in JSON format as long as the model can produce it in standard JSON. We include this additional step because, at the time of our project’s development, GPT-4 did not consistently support JSON output (but support now) in the long context, and other open-source models also struggled to provide consistently parsable JSON.

We strongly encourage and are pleased to see follow-up work aimed at simplifying the process of obtaining results.

Best,
Jian

[yananchen1989]

make sense. thanks.