zio/zio-openai

Bug in Chat Completions Role generation

Closed this issue · 1 comments

When trying to use the Chat.createChatCompletion method, there is an issue with the generated code around Role for ChatCompletionRequestUserMessage. Specifically, ChatCompletionRequestUserMessage takes in a role: zio.openai.model.Role which only has Assistant defined. There is no definition for User.

This is also curious because ChatCompletionRequestSystemMessage takes in a role: zio.openai.model.ChatCompletionRequestSystemMessage.Role (instead of zio.openai.model.Role). I would think that this would imply that the user messages should be taking in a role: zio.openai.model.ChatCompletionRequestUserMessage.Role or that all of the message types should reference zio.openai.model.Role which would contain all Role options.

Either way, the current code is bugged as there is no way to set the role as User for chat completion methods as far as I can tell.

After further investigation, I found the issue with the code. The problem comes from the Model.unifyEnums method.

private def unifyEnums(
allEnums: List[TypeDefinition.Enum]
): (List[TypeDefinition.Enum], Map[TypeDefinition.Enum, TypeDefinition.Enum]) = {
val grouped = allEnums.groupBy(enum => (enum.directName, enum.values))
allEnums.foldLeft(
(List.empty[TypeDefinition.Enum], Map.empty[TypeDefinition.Enum, TypeDefinition.Enum])
) { case ((result, mapping), enum) =>
val group = grouped((enum.directName, enum.values))
if (
group.size == 1 ||
enum.directName == "CaseType0" || // We don't want to move these generated cases into top level
enum.directName == "CaseType1" ||
enum.directName == "CaseType2" ||
enum.directName == "CaseType3" ||
enum.directName == "CaseType4" ||
enum.directName == "CaseType5" ||
enum.directName == "CaseType6" ||
enum.directName == "CaseType7"
) {
// This is a unique enum
(enum :: result, mapping + (enum -> enum))
} else {
// This is a duplicate enum
result.find(other =>
other.directName == enum.directName && other.values == enum.values
) match {
case Some(existing) =>
// We already have a unified enum
(result, mapping + (enum -> existing))
case None =>
// This is the first duplicate enum
val unified = TypeDefinition.Enum(enum.directName, None, enum.values, enum.description)
(unified :: result, mapping + (enum -> unified))
}
}
}
}
:
This algorithm to detect enums which have the same direct name and same set of values and then unify them at the top level has a critical bug. In particular, it will fail if there are 2 different enum groups which have the same directName. For example, in the openai schema (https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml) we can see that there are 2 enums each for

role:
          type: string
          enum: ["user"]

and

role:
          type: string
          enum: ["assistant"]

This means that both will try to become unified, but since they have the same directName (role), one of them will be overwritten. In the current release 0.4.0, the user version of this Role enum was overwritten by the assistant version.

In order to fix, we should only unify enums which do not have this multiple group conflict. I'll write up a PR to fix.