Add AzureChatOpenAI as an option for summary_quality and other scorers

Question

Add AzureChatOpenAI as an option for summary_quality and other scorers

Closed this issue a year ago · 3 comments

Currently, the summary_quality scorer (and others like qa_quality) are hardcoded to the regular ChatOpenAI, but many will need AzureChatOpenAI in order to integrate Arthur Bench with out workflow. Could support for the Azure version from langchain be added?

Answer 1 · 2023-08-30T00:38:18.000Z

hi @MakonDev thanks for submitting this feature request! Please let us know if @maccam912's PR does not fit your use case

Answer 2 · 2023-08-30T13:23:40.000Z

It does look that PR will fit my use case! That'd be great, thanks! Excited to try it out

Answer 3 · 2023-10-03T15:21:19.000Z

hi @MakonDev this was released in version 0.2.2. Please give it a try and let us know what you think! We'll be posting a community roadmap soon and would appreciate any further feedback as well