OpenAPITools/openapi-generator

Improve handling of oneOf

jmini opened this issue ยท 39 comments

jmini commented

With OAS3 it is possible to use oneOf.
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.1.md#schema-object

See one example in composed-oneof.yaml from the test suite.

We should discuss how we want to handle this.

In my opinion (for java) the Schema containing only oneOf entries should be an interface, and all model classes corresponding to the schema mentioned in the oneOf should implement this interface.

What should DefaultCodegen expose?

Currently DefaultCodegen simply exposes a simple string returnType, so we don't know what the possible values are. Although overriding fromOperation would give access to Map<String, Schema> schemas but not sure if implementations should use schemas directly or expect a certain field on DefaultCodegen to assist in determing the possible values.

changing returnType would probably break a lot of templates, so maybe there is a way for a lang to opt-in to supporting `oneOf``

just some thoughts, no real preferences on this atm

jmini commented

Well I think that in Java, an interface should be created, because you have no way to express a Type Union: return type is ObjA or ObjB.

Yes we need to add this to the Codegen without breaking anything for existing templates.
I did not investigate yet, how this could be added.

For Java, what would the interface look like for composed-oneof.yaml from the test suite? (ObjA and ObjB have different properties, and in theory could.have no properties in common). Also is the realtype field going to be a requirement to make it work?

jmini commented

We have problem with names when schema are inlined (this should be addressed in #8).
Let call it CompObj.


The interface can be empty in my opinion. The idea is that in Java code, you need to check with instanceOf: With obj being a CompObj

if(obj instanceOf ObjA) {
...
} else if(obj instanceOf ObjB) {
...
}

In typescript you do not need CompObj you can work directly with Union Type: http://www.typescriptlang.org/docs/handbook/advanced-types.html#union-types


About the discriminator (realtype)... It is not mandatory to have it in ObjA and ObjB. I do not think that it should be defined in the interface, but it will be useful information for the serializer/deserializer.

ah, yes, an empty/marker interface + casting would work. The old-school way of doing tagged unions in langs that don't support them is to expose a struct with a discriminator/enum that identifies which field contains the data e.g.

(pseudo-code)
[struct]
{
   discriminator: Enum/Int
   objA: ObjA
   objB: ObjB
}

which is more static in that it avoids casting, but has downside of having to use the discriminator to get the right data. But I'm not involved enough with Java to know what the best practices for Java are.

The marker interface method may be a cleaner solution.

Also worth noting other tools like c# autorest I don't think support this currently.

jmini commented

I think we should published the necessary information in the Codegen layer, each template can implement in its own way (depending on the capabilities / language features)

I duplicated this as #475, and came to the same conclusion as @jmini in the previous comment. Codegen should expose as much information as possible to the generators, and they should use language features accordingly.

My company requires oneOf functionality.

Is there a bugfix at the actual Version? I am using Version 3.3.2 and i have the same problem with oneOf.

jmini commented

I think I will give the marker-interface approach a try in the Java-client generators:

For those schemas:

components:
    schemas:
        MainObj:
            type: object
            oneOf:
                - $ref: '#/components/schemas/ObjA'
                - $ref: '#/components/schemas/ObjB'
            discriminator:
                propertyName: realtype
                mapping:
                    a-type: '#/components/schemas/ObjA'
                    b-type: '#/components/schemas/ObjB'
        ObjA:
            type: object
            properties:
                realtype:
                    type: string
                message:
                    type: string
        ObjB:
            type: object
            properties:
                realtype:
                    type: string
                description:
                    type: string
                code:
                    type: integer
                    format: int32

I would generate:

  • A marker interface MainObj (=> right now with with the latest 4.0.0 SNAPSHOT version, there is nothing generated. This creates compile error cannot find symbol class MainObj because of the import import org.openapitools.client.model.MainObj which is present).
  • 2 types (same as currently generated) that both implements this interface (this is not the case right now):
    • ObjA
    • ObjB

@jmini sounds good to me ๐Ÿ‘

jmini commented

Interesting case in oneOf.yaml:

    fruit:
      title: fruit
      type: object
      properties:
        color:
          type: string
      oneOf:
        - $ref: '#/components/schemas/apple'
        - $ref: '#/components/schemas/banana'

It seems to be possible to add properties and oneOf in the same schema. I am not sure what the semantic is in this case, but we might need to support this as well.

The interface pattern that I have described here, only works for Schema with only oneOf (meaning without properties)

Do the OpenApi-Generator support OneOf / Any-Of combinations?

clojj commented

What is the status of "oneOf" issues ?
We'd like to generate from a spec which has several oneOf... not only as requestBody, but also as responseBody

See also #2121 for Python Client support

Interesting case in oneOf.yaml:

    fruit:
      title: fruit
      type: object
      properties:
        color:
          type: string
      oneOf:
        - $ref: '#/components/schemas/apple'
        - $ref: '#/components/schemas/banana'

It seems to be possible to add properties and oneOf in the same schema. I am not sure what the semantic is in this case, but we might need to support this as well.

The interface pattern that I have described here, only works for Schema with only oneOf (meaning without properties)

@jmini I think json schema in this case specifies that there is an implicit allOf combining all these, but I can't seem to find the place where I read this. The spec would be equivalent to:

fruit_color:
  type: object
  properties:
    color:
      type: string
fruit_w_type:
  type: object
  oneOf:
    - $ref: '#/components/schemas/apple'
    - $ref: '#/components/schemas/banana'
fruit:
  title: fruit
  type: object
  allOf:
    - $ref: '#/components/schemas/fruit_color'
    - $ref: '#/components/schemas/fruit_w_type'

Any news on this?

When I use oneOf in the Swagger.json specification file, I'll get these errors:

[ERROR] generated-code/spring/src/main/java/com/se/edm/model/Network.java:[312,29] cannot find symbol
[ERROR]   symbol:   class OneOfModbusSLNetworkParameterModbusTCPNetworkParameterZigbeeNetworkParameter
[ERROR]   location: class com.se.edm.model.Network

The same for generated c# client. Oneof property typed as "Oneof..." in output project instead of using object (as it was in openapi-generator-tool of 3.3.4 version)

Could you share what the status is?

I have a Swagger file containing oneOf.

bankAccount: oneOf: - $ref: "#/components/schemas/BankAccountWithIban" - $ref: "#/components/schemas/BankAccountWithoutIban"

It compiles to

public static final String JSON_PROPERTY_BANK_ACCOUNT = "bankAccount"; private OneOfBankAccountWithIbanBankAccountWithoutIban bankAccount = null;

but OneOfBankAccountWithIbanBankAccountWithoutIban is undefined

Thanks a lot!

oneOf just means that one of the 'subschema' should match. Those schema's can even be validation-only schemas, without any properties. 3GPP is using 'oneOf' to express that one of two properties in the 'parent' must be required.

Example:

components: schemas: Notification: type: object properties: externalId: type: string msisdn: type: string data: type: string required: - data oneOf: - required: [externalId] - required: [msisdn]

When i'm running swagger codegen on the above snippet, i'm getting an empty class (no properties).

For reference, the full 3gpp spec containing oneOf's: http://www.3gpp.org/ftp//Specs/archive/29_series/29.122/29122-f40.zip

Thanks. My problem is that the class OneOfBankAccountWithIbanBankAccountWithoutIban is used as a type, but not created anywhere, causing the compilation to break.

The obvious workaround (declare the empty class in the regular code) is available.

I'm amused, so far I see no code generator produces correct output for oneOf, right? ๐Ÿ˜‚

Now for real, is there any codegen producing correct output out there? I'ld like to know ๐Ÿ˜ƒ

@realvictorprm I don't know about the dynamic langs, but for the static langs I think the issue is that trying to encode different result schemas in the typesystem could involve a lot of boilerplate, and each language would have to solve this in it's own way - and I think template authors are reluctant to force a lot of boilerplate or special casing on their users.

Do you have any ideas to contribute on the approach in general?

I'm not even sure if everyone has the same goals with safety/strictness vs convenience.

For example, I think it was proposed for Java the return type would be Object and the user would be forced to downcast to the appropriate type, as one solution.

Also, are there changes needed in the core generator to support oneOf, or is it only work that needs to be done by template authors for this?

I tried GoGenerator on the following:

https://github.com/jdegre/5GC_APIs/blob/master/TS29509_Nausf_UEAuthentication.yaml

which uses 'oneof' as follows

...
        5gAuthData:
          oneOf:
            - $ref: '#/components/schemas/Av5gAka'
            - $ref: '#/components/schemas/EapPayload'
...

The code is getting generated but, while execution it is not able to resolved oneof parameter. Is there any workaround ?

Hey folks ๐Ÿ‘‹ I took a shot at implementing this for Java jackson clients in #4785 - I'd be glad for any feedback and/or additional testing of my code. Thanks!

The same issue with "spring" generator. Is it possible to implement the same behavior which @bkabrda implemented for client generator in #4785?

I did it for Spring and all JaxRS server generators in my fork. See mmalygin@3303da5
To enable the feature, set useOneOfInterfaces=true in additional properties.

@mmalygin this relates/overlaps with #5381. Do you plan on creating a PR for this work?

@bkabrda @jimschubert - Do you have a view of which implementation best aligns with the merged changes from #4785? I think it is imperative that we have alignment across all flavors of clients and server generators.

I have the same problem when generating code from a spec containing oneOf, for QT5/C++.

A file "OneOf..." header file is included, but not generated anywhere (at least with version 4.3.0).

As @amitinfo2k. I also tried the go generator with the Video Analytics Serving OpenAPI

https://github.com/intel/video-analytics-serving/blob/v0.3.0-alpha/vaserving/rest_api/video-analytics-serving.yaml

It uses oneOf as well

      properties:
        source:
          discriminator:
            propertyName: type
          oneOf:
          - $ref: '#/components/schemas/URISource'
          - $ref: '#/components/schemas/DeviceSource'
          type: object
        destination:
          discriminator:
            propertyName: type
          oneOf:
          - $ref: '#/components/schemas/KafkaDestination'
          - $ref: '#/components/schemas/MQTTDestination'
          - $ref: '#/components/schemas/FileDestination'
          type: object

I used the docker generator for it:

$ docker images | grep openapi-generator-cli
openapitools/openapi-generator-cli                        latest                                     c03abe67cb2d        3 hours ago         135MB

$ docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate -i https://raw.githubusercontent.com/intel/video-analytics-serving/v0.3.0-alpha/vaserving/rest_api/video-analytics-serving.yaml -g go -o /local/out/go

Then using it in a client throws errors:

$ go run client.go 
# openapi
../../go/src/openapi/model_pipeline_request.go:13:9: undefined: OneOfUriSourceDeviceSource
../../go/src/openapi/model_pipeline_request.go:14:14: undefined: OneOfKafkaDestinationMqttDestinationFileDestination

The generated go code look like this:

package openapi
// PipelineRequest struct for PipelineRequest
type PipelineRequest struct {
	Source OneOfUriSourceDeviceSource `json:"source,omitempty"`
	Destination OneOfKafkaDestinationMqttDestinationFileDestination `json:"destination,omitempty"`
	// Client specified values. Returned with results.
	Tags map[string]interface{} `json:"tags,omitempty"`
	// Pipeline specific parameters.
	Parameters map[string]interface{} `json:"parameters,omitempty"`
}

Is there any fix I can test for this?

Please try the latest go-experimental generator, which has better support for oneOf and anyOf.

Tried the go-experimental one:

docker run --rm -v ${PWD}:/local openapitools/openapi-generator-cli generate -i https://raw.githubusercontent.com/intel/video-analytics-serving/v0.3.0-alpha/vaserving/rest_api/video-analytics-serving.yaml -g go-experimental -o /local/out/go

Still have problems with the oneOf attr:

test@ubuntu1804-2:~/Downloads/hello-go$ go run client.go 
# _/home/test/Downloads/hello-go/openapi
openapi/model_pipeline_request.go:18:10: undefined: OneOfURISourceDeviceSource
openapi/model_pipeline_request.go:19:15: undefined: OneOfKafkaDestinationMQTTDestinationFileDestination
openapi/model_pipeline_request.go:44:39: undefined: OneOfURISourceDeviceSource
openapi/model_pipeline_request.go:46:11: undefined: OneOfURISourceDeviceSource
openapi/model_pipeline_request.go:54:43: undefined: OneOfURISourceDeviceSource
openapi/model_pipeline_request.go:71:39: undefined: OneOfURISourceDeviceSource
openapi/model_pipeline_request.go:76:44: undefined: OneOfKafkaDestinationMQTTDestinationFileDestination
openapi/model_pipeline_request.go:78:11: undefined: OneOfKafkaDestinationMQTTDestinationFileDestination
openapi/model_pipeline_request.go:86:48: undefined: OneOfKafkaDestinationMQTTDestinationFileDestination
openapi/model_pipeline_request.go:103:44: undefined: OneOfKafkaDestinationMQTTDestinationFileDestination
openapi/model_pipeline_request.go:78:11: too many errors

The generated code is similar to the previous one:

package openapi

import (
	"encoding/json"
)

// PipelineRequest struct for PipelineRequest
type PipelineRequest struct {
	Source *OneOfURISourceDeviceSource `json:"source,omitempty"`
	Destination *OneOfKafkaDestinationMQTTDestinationFileDestination `json:"destination,omitempty"`
	// Client specified values. Returned with results.
	Tags *map[string]interface{} `json:"tags,omitempty"`
	// Pipeline specific parameters.
	Parameters *map[string]interface{} `json:"parameters,omitempty"`
}
cljk commented

Is there not unit-test for retrofit2 generator using oneOf?

For me the generated code does not even compile...

My schema

        user_profile_properties:
          type: array
          items:
            oneOf:
              - $ref: '#/components/schemas/UserProfileTextProperty'
              - $ref: '#/components/schemas/UserProfileImageProperty'
              - $ref: '#/components/schemas/UserProfileSelectionProperty'
            discriminator:
              propertyName: type
              mapping:
                text: '#/components/schemas/UserProfileTextProperty'
                image: '#/components/schemas/UserProfileImageProperty'
                selection: '#/components/schemas/UserProfileSelectionProperty'

The compile-error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project project-api-client-retrofit2: Compilation failure: Compilation failure:
[ERROR] /C:/Users/me/workspaces/work/project/project-api-client/project-api-client-retrofit2/target/generated-sources/openapi/src/main/java/net/worke/project/restclient/model/SystemInformation.java:[30,39] cannot find
symbol
[ERROR]   symbol:   class OneOfUserProfileTextPropertyUserProfileImagePropertyUserProfileSelectionProperty
[ERROR]   location: package net.worke.project.restclient.model
[ERROR] /C:/Users/me/workspaces/work/project/project-api-client/project-api-client-retrofit2/target/generated-sources/openapi/src/main/java/net/worke/project/restclient/model/SystemInformation.java:[75,16] cannot find
symbol
[ERROR]   symbol:   class OneOfUserProfileTextPropertyUserProfileImagePropertyUserProfileSelectionProperty
[ERROR]   location: class net.worke.project.restclient.model.SystemInformation
[ERROR] /C:/Users/me/workspaces/work/project/project-api-client/project-api-client-retrofit2/target/generated-sources/openapi/src/main/java/net/worke/project/restclient/model/SystemInformation.java:[301,55] cannot find
 symbol

The generated model class SystemInformation is referencing a class OneOfUserProfileTextPropertyUserProfileImagePropertyUserProfileSelectionProperty which is not there.

  public static final String SERIALIZED_NAME_USER_PROFILE_PROPERTIES = "user_profile_properties";
  @SerializedName(SERIALIZED_NAME_USER_PROFILE_PROPERTIES)
  private List<OneOfUserProfileTextPropertyUserProfileImagePropertyUserProfileSelectionProperty> userProfileProperties = null;

There is only UserProfileImageProperty, UserProfileTextPropertyand UserProfileSelectionProperty.

I checked the behaviour also for type: object instead of type: array - same (comparable) error.

@cljk retrofit2 doesn't have oneOf/anyOf support. Of course we welcome contributions to support that.

Please try jersey2 instead which has better support for oneOf/anyOf.

cljk commented

@wing328
Perhaps my definition/usage of oneOf was not correct. Even after consuming the OpenAPI doc several times Iยดm not quite sure. I modified my schema a bit and replaced it with usage of allOfand it now even works in retrofit2.
Instead of defining my property user_profile_properties as oneOf I now defined a super type which has at least the discriminator as field. Then in the subtypes I referenced it with allOf. This leads to the generation of a super class and my sub classes.
Processing/parsing tested successfully so far in jersey and retrofit2 client adapters.

OLD

        user_profile_properties:
          type: array
          items:
            oneOf:
              - $ref: '#/components/schemas/UserProfileTextProperty'
              - $ref: '#/components/schemas/UserProfileImageProperty'
              - $ref: '#/components/schemas/UserProfileSelectionProperty'
            discriminator:
              propertyName: type
              mapping:
                text: '#/components/schemas/UserProfileTextProperty'
                image: '#/components/schemas/UserProfileImageProperty'
                selection: '#/components/schemas/UserProfileSelectionProperty

NEW

       user_profile_properties:
          type: array
          items:
            $ref: '#/components/schemas/UserProfileProperty'


    UserProfileProperty:
      type: object
      required:
        - type
      properties:
        type:
          type: string
          # enum: [text, image, selection]
        name:
          type: string
      discriminator:
        propertyName: type
        mapping:
          text: '#/components/schemas/UserProfileTextProperty'
          email: '#/components/schemas/UserProfileEmailProperty'

    UserProfileTextProperty:
      allOf:
        - $ref: '#/components/schemas/UserProfileProperty'
        - type: object
          properties:
            multiline:
              type: boolean
            required:
              type: boolean

    UserProfileEmailProperty:
      allOf:
        - $ref: '#/components/schemas/UserProfileProperty'
        - type: object
          properties:
            required:
              type: boolean

Spoiler: the discriminator as enum does not work...

Is there any way to make oneOf, anyOf, allOf etc. work in de ASP.NET Core server stub generator? We don't have full control of the OpenAPI document we have to auto-generate code for (with the only guarantee being that the document adheres to the 3.0 spec)

Recently the document we have to adhere to started using oneOf and anyOf.
Whenever the document contains oneOf/anyOf validators like this:

"responses": {
    "200": {
      "description": "Success",
      "content": {
        "application/json": {
          "schema": {
            "oneOf": [
                {
                  "$ref": "#/components/schemas/CustomerModel"
                },
                {
                  "$ref": "#/components/schemas/ProjectModel"
                }
            ]
          }
        }
      }
}

the attribute generated references a class named OneOfProjectModelCustomerModel which doesn't exist.

[ProducesResponseType(statusCode: 200, type: typeof(OneOfProjectModelCustomerModel))]

The same happens for models like this:

"TestDataModel": {
    "type": "object",
    "properties": {
        "testValue": {
            "oneOf": [
              { 
                "type": "string" 
              },
              {
                "$ref": "#/components/schemas/CustomerModel"
              }
            ],
            "nullable": true
          }
    }
}

Will generate the following uncompilable property:

[DataMember(Name="testValue", EmitDefaultValue=true)]
public OneOfstringCustomerModel TestValue{ get; set; }

Is there a way to "fix" this using the generator (since we can't change the OpenAPI doc) or would we have to change the generated code (example: by replacing these classes with generic .NET "object" references)?

I have a similar use of the generator for netcore as your first example. The difference is that in the path response I have a $ref then the $ref has the oneOf in it. Something like this:

        '200':
          description: Ok
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/VPublic'

and the schema like this:

VPublic:
     oneOf:
       - $ref: '#/components/schemas/VPublicSR'
       - $ref: '#/components/schemas/VPublicVR'
       - $ref: '#/components/schemas/VPublicIMVI'
       - $ref: '#/components/schemas/VPublicOSI'

this creates a DTO named VPublic that has a combination of all fields across the subschemas.

So I guess for you that won't be an answer since you can't manipulate the spec you use? I have no idea how to do this otherwise.

For your second example I haven't tried anything like that. But I read that in the new spec Openapi 3.1 they introduced the polymorphism when defining types as an array... ["string","null"] or something like that. Of course that is the spec the tooling is not there yet :)

For context here, oneOf can be combined with any of the other openapi keywords.
One can have oneOf anyOf and allOf. Or properties and oneOf
Or items and oneOf or a type constraint and OneOf etc. This issue's question is one specific common use case, not the general use case. Python supports all of the mentioned general cases.
One can see this working for a model that combines allOf/anyOf/oneOf, and the test of it here.