Evaluate OpenAPI tooling

[pvdbosch]

In GitLab by @pvdbosch on Sep 7, 2018, 16:25

1. Test senya.io, stoplight.io, apicurio or any other interesting tool to author swagger files.

2. Evaluate code generation maven plugins: openapi-generator vs swagger codegen.
   Recommend one?

Comparison (WIP):

Swagger codegen:

• specifying explicitely additionalProperties: true was required to let generated Java classes support additional properties not specified in the Swagger file (untested on openapi-generator)
  • even with this specified, the generated code didn't work properly
• when generating fat swagger, creates top level types for inlined (non-Venetian-blind) types. It shouldn't. The generated types don't follow styleguide naming convention. (untested on openapi-generator)
• generated code's formatting a bit better
• invalid escaping for regexp: @Pattern(regexp="\d{11}") => doesn't compile, should be @Pattern(regexp="\\d{11}") (tested with v3 of codegen)

openapi-generator:

• more actively developed
• unnecessary java imports, bad formatting in generated code - maybe fixable with some tuning
• additionalProperties: true has no effect
• 4.x use of allOf:
  • generate useless *AllOf classes for Swagger types inlined in an allOf composition. They are imported in the composite class but not used
  • no inheritance in generated code anymore when using allOf with a referenced type: e.g. InvalidParamProblem no longer extends Problem (see [BUG] [Java] inheritance is generated without discriminator field OpenAPITools/openapi-generator#5097 (comment))
  • workaround for both: only specify base class in allOf and specify properties on same level like this:
    MyType: { allOf: [$ref: '#/components/schemas/T'], properties: { x: {type: string}} }
• converts format: uri to java.net.URI as of 4.1.0
• In 4.3.1: inline object types become java.lang.Object instead of proper generated type. Workaround: make them explicit (non-inline) types

Other notes:

• specifying an array of types, while valid JSON Schema, doesn't seem to be supported in OpenAPI
• format: decimal isn't part of the OpenAPI standards, and isn't supported for code generation
• Custom typemappings may be needed in maven config, e.g.
  • to get byte array as return type (e.g. when returning image) for type: file instead of default java.io.File:
    <typeMappings><typeMapping>file=byte[]</typeMapping></typeMappings>
• all plugins: JAX-RS always returns a default http status code (200 for OK with body, 204 for OK without body). Only way to override is by generating javax.ws.rs.core.Response return types for all generated models.

3. Smals will share their extensions to the code generation plugin, adding proper support for additional properties to all generated classes.

[pvdbosch]

In GitLab by @pvdbosch on Oct 5, 2018, 12:01

Most authoring tools seem to have difficulties with definitions referenced from external YAML files.

Some feedback from our developers:

• senya.io: IntelliJ plugin.
  • Problems with external schemas in graphical view, though supported in text view (i.e. links can be followed)
  • marks some properties as faulty while they're valid OpenAPI 2.0: the 'examples' property in a Response Object, and a property defined without a type
• swagger editor: also problems with external references; (see this issue and this other issue.
• might be working if files are reachable by http instead file:///
• Only tested with v3. Issue mentions swagger editor v2 might support it better than v3.
• Stoplight is not free of charge, so didn't test
• apicur.io: only single file swagger. No switching between graphical and text representation. Still beta.

RepreZen is another paying alternative, but didn't try out yet.

[pvdbosch]

In GitLab by @pvdbosch on Nov 23, 2018, 16:21

Latest 0.20 version of Senya works fine now. Tested with socialStatus API, with external files, both in text as UI mode.

I reported the bug concerning examples (as mentioned in #28) in Swagger and it got fixed.
Senya follows closely the OpenAPI 2.0 spec, which means that you have to use the MIME type as key of each example, matching one of the 'produces' MIME types. This means you can only specify a single example per response status code. For the socialStatuses API, we used to specify multiple examples using some free text description as key, as the Swagger editor didn't verify that they were MIME types.
The OpenAPI 3.0 spec does allow specifying multiple examples.

[pvdbosch]

In GitLab by @pvdbosch on Dec 17, 2018, 11:54

KaiZen as eclipse plugin

SwaggerUI watcher https://github.com/moon0326/swagger-ui-watcher views in browser

can be added to list of tools in style guide

[pvdbosch]

In GitLab by @wsalembi on Jan 20, 2019, 12:09

Senya Editor for Intellij IDEA has become a paid plugin for https://senya.io/release-notes-1-0-0/ (48 euro/year)

[pvdbosch]

In GitLab by @pvdbosch on Jan 22, 2019, 10:15

updated the styleguide:

• split tools and references section, add some more description
• mention Senya is commercial
• add Zalando's IntelliJ plugin

[pvdbosch]

In GitLab by @pvdbosch on Jan 22, 2019, 11:53

changed title from evaluate {-tools to author swagger files-} to evaluate {+swagger tooling+}

[pvdbosch]

In GitLab by @pvdbosch on Jan 22, 2019, 12:11

Broadening scope of the issue to also include other tooling, like codegen.

For code generation from swagger, there are two plugins:

• swagger-codegen: https://github.com/swagger-api/swagger-codegen
• openapi-generator from OpenAPITools: https://github.com/OpenAPITools/openapi-generator (= fork of swagger-codegen)

Both still seem under active development, though openapi-generator seems to have more activity.
What are practical experiences with the plugins of the WG members? Do we recommend the openapi-generator one?

Some issues encountered by @VirginieHayot when using the swagger codegen:

• specifying explicitely additionalProperties: true was required to let generated Java classes support additional properties not specified in the Swagger file
• even with this specified, the generated code didn't work properly

Smals (@smals-wisa ) developed an extension to the plugin, adding proper support for additional properties to all generated classes. It would be interesting to other WG members if this extension could be shared.

[pvdbosch]

In GitLab by @pvdbosch on Feb 26, 2019, 13:50

Added WIP comparison between codegen plugin to issue description, everyone can contribute.

[pvdbosch]

In GitLab by @pvdbosch on Feb 26, 2019, 13:50

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Feb 27, 2019, 12:14

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Feb 27, 2019, 12:17

Another codegen issue encountered with InvalidParamProblem; arrays of types cause property to be ignored during code generation:

  InvalidParam:
    type: object
    properties:
      in:
        type: string
      name:
        type: string
      reason:
        type: string
      value:
        # allow all types, as type is mandatory. Known issue: property is ignored when using swagger codegen (see #25)
        type: [string, number, integer, boolean, array, object, 'null']

(from new problem-v1.yaml)

[pvdbosch]

In GitLab by @pvdbosch on Feb 27, 2019, 12:47

mentioned in commit 229c7e7

[pvdbosch]

In GitLab by @pvdbosch on Mar 11, 2019, 09:50

Omitting type on the value property; this works with code generation to get Object type.

Having no type is valid OpenAPI 2.0 even though KaiZen (and maybe some other editors) indicates it as an error.

[pvdbosch]

In GitLab by @pvdbosch on Mar 11, 2019, 10:05

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Mar 11, 2019, 10:08

added some more remarks on codegen plugin, after some testing.

The openapi-generator generates some weird warnings, but they can be ignored it seems.

[INFO] --- openapi-generator-maven-plugin:3.3.4:generate (Generate server stub model) @ ssh-citizen-web ---
[WARNING] Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[INFO] Environment variable JAVA_POST_PROCESS_FILE not defined so the Java code may not be properly formatted. To define it, try 'export JAVA_POST_PROCESS_FILE="/usr/local/bin/clang-format -i"' (Linux/Mac)
[INFO] NOTE: To enable file post-processing, 'enablePostProcessFile' must be set to `true` (--enable-post-process-file for CLI).
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one
[WARNING] Multiple schemas found, returning only the first one

For the additionalProperties, maybe it's sufficient to create a subclass of the generated class in most cases?

[pvdbosch]

In GitLab by @pvdbosch on Mar 13, 2019, 14:05

mentioned in commit 7abe48a

[pvdbosch]

In GitLab by @pvdbosch on Mar 13, 2019, 14:57

mentioned in commit 11c6187

[pvdbosch]

In GitLab by @pvdbosch on May 10, 2019, 09:26

mentioned in commit 9c3e545

[pvdbosch]

In GitLab by @pvdbosch on May 10, 2019, 09:26

mentioned in commit c373a75

[pvdbosch]

In GitLab by @pvdbosch on May 10, 2019, 09:26

mentioned in commit a9ffd2d

[pvdbosch]

In GitLab by @pvdbosch on Jun 20, 2019, 09:39

changed the description

[pvdbosch]

In GitLab by @ehealth-eh053 on Jun 24, 2019, 11:09

 responses:
        '201':
          description: Created
        default:
          description: Failure server response
          schema:
            $ref: "./specs/ehealth-api-problem.yaml#/definitions/Problem"

When using the openapi-generator-maven-plugin the generated source doesn't use the correct code for the default errors.

    @ApiResponses(value = { 
        @ApiResponse(code = 204, message = "Created"),
        @ApiResponse(code = 200, message = "Failure server response", response = Problem.class) })

[pvdbosch]

In GitLab by @jimmy.praet on Jul 15, 2019, 11:56

I have some issues with the openapi-generator-maven-plugin (3.3.4):

      parameters:
        - in: query
          name: types
          type: array
          items:
            type: string
            enum: # cannot use a $ref here in Swagger 2.0
              - CLASSIC
              - BIIP35
              - BIIP38
              - BIIP77
              - PCSA
              - MUNICIPALITY
              - PROVINCE

This generates the following:

    @GET
    @Produces({ "application/json;charset=utf-8" })
    @ApiOperation(value = "Retrieve the list of CLSM partners", notes = "", tags={  })
    @ApiResponses(value = { 
        @ApiResponse(code = 200, message = "successful operation", response = Partners.class) })
    Response getPartnerList(@QueryParam("active")    Boolean active,@QueryParam("types")  @DefaultValue("new ArrayList<>()")   List<String> types);

The @DefaultValue("new ArrayList<>()") is problematic. When I don't supply a value for the parameter, I get a List in my implementation method containing 1 string element "new ArrayList<>()".

I tried configuring a default in the swagger file, but this didn't make any difference.

[pvdbosch]

In GitLab by @pvdbosch on Aug 8, 2019, 12:22

Another pitfall encountered with generated code is that JAX-RS always returns a default http status code (200 for OK with body, 204 for OK without body).

The status codes in the Swagger ApiResponse annotations are ignored by JAX-RS; they're only used when a swagger file is generated from the code when doing code-first development.

To allow for other response codes, you can configure <returnResponse>true</returnResponse> in the maven plugin, which will then let all generated methods return a javax.ws.rs.core.Response on which status code and response body can be set.
You'll loose type-safety of return types, but there doesn't seem to be a nicer way in JAX-RS.

[pvdbosch]

In GitLab by @pvdbosch on Aug 23, 2019, 11:13

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Aug 23, 2019, 11:15

changed the description

[pvdbosch]

In GitLab by @pvdbosch on Aug 23, 2019, 11:15

Some results of tests with openapi generator 4.1.0 added to issue description:

• 4.0.3 and 4.1.0 generate useless *AllOf classes for Swagger types inlined in an allOf composition. They are imported in the composite class but not used
• 4.1.0: Java classes don't seem to use inheritance anymore when using allOf with a referenced type (not yet tested on 4.0.x): e.g. InvalidParamProblem no longer extends Problem
• converts format: uri to java.net.URI as of 4.1.0

[pvdbosch]

In GitLab by @pvdbosch on Aug 23, 2019, 12:12

changed the description

[aethy]

My findings on the inheritance vs composition issue in comment #25 (comment) :

For a "oneOf", the discriminator makes sense, to know which object you receive. But just to have some abstract superclass (or better, an interface) for some classes, it doesn't make sense. The generated classes now all have the chosen discriminator property (e.g. "classType"), but the value is not coded in the generated classes although it should be possible (it's just the name of the class, right?). They are however generated on the server (by the jax-rs impl.?). But I don't like that, it's not clean to have such a type property in the returned JSON.

On another note, with openapi-generator-maven-plugin 5.4.0, the plugin configuration below generates invalid classes:

              <instantiationTypes>map=TreeMap</instantiationTypes>
              <typeMappings>URI=String,uri=String</typeMappings>

You need to use this instead:

              <instantiationTypes>map=java.util.TreeMap</instantiationTypes>
              <typeMappings>java.net.URI=String,uri=String</typeMappings>

[jpraet]

A PR OpenAPITools/openapi-generator#14172 has been merged in openapi-generator that supports configuration of a REF_AS_PARENT_IN_ALLOF "normalizer", which should enable use of inheritance.

[pvdbosch]

@jpraet, that's interesting! We can check how to best apply this (mainly to Problem types).

From what I can see, we'll be able to choose between:

• configure the normalizer, which makes "x-parent: true" the default for $refs in an allOf. This leaves the schemas unchanged.
• adding an "x-parent: true" property explicitly to the schema object that should become parent class
• create an interface type and add "x-implements: " to the schema (already supported in current release). This option limits the schema to a specific programming language or usage

[jpraet]

Another observation with openapi-generator-maven-plugin: when using an enum as a query param, only UPPER_CASE values are accepted at runtime. This is fixed in 6.x: OpenAPITools/openapi-generator#7494

[jpraet]

openapi-generator-maven-plugin 6.3.0 is released.

Using allOf refs as parent can be enabled globally by adding <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true</openapiNormalizer> to the plugin configuration.

Alternatively, "x-parent: true" can be added to the schema objects to be used as parent class.

[pvdbosch]

PR #123 makes some changes to list of OpenAPI tooling:
Removed tooling with no releases last couple of years: Reprezen/Kaizen, swagger-ui-watcher, senya.io
Added IntelliJ bundled ultimate and 42Crunch plugins

[jpraet]

In openapi-generator 4.x, using allOf like below seems to get the desired result with inheritance:

    S:
      allOf:
        - $ref: '#/components/schemas/T'
      properties:
        x:
          type: string
        y:
          type: string

Still seems like a deprecated behavior that might change in openapi-generator 5.x. We'll change InvalidParamProblem to use this type of definition. It seems equivalent according to OpenAPI spec, but gives better results for code generation.

In the latest openapi-generator versions 6.5.0 and 6.6.0, this no longer generates inheritance, even with <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true</openapiNormalizer>.

But with

    S:
      allOf:
        - $ref: '#/components/schemas/T'
        - type: object
          properties:
            x:
              type: string
            y:
              type: string

it works.

[jpraet]

I've reported an issue for the openapi-generator-maven-plugin regression mentioned above: OpenAPITools/openapi-generator#16389

[pvdbosch]

New version 7.0.0 of the openapi-generator seems to work fine:

• useless allOf classes aren't generated anymore for inline types
• parent classes are used by setting <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true</openapiNormalizer> (first $ref is parent) or by adding x-parent: true on the parent class. The x-parent setting has precedence over the normalizer.

We could add x-parent: true to the belgif Problem type, but I think activating the REF_AS_PARENT_IN_ALLOF=true could be the default behavior what most developers want. Its behavior can still be overridden by x-parent, though it becomes more difficult when reusing external schemas.

[pvdbosch]

IntelliJ Ultimate 2023.2 switched the OpenAPI preview from SwaggerUI to redocly. It has a couple UX regressions IMO: overview of all operations is less at-a-glance, and schemas aren't listed separately from the operations.

[jpraet]

New version 7.0.0 of the openapi-generator seems to work fine:
* parent classes are used by setting <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true</openapiNormalizer> (first $ref is parent) or by adding x-parent: true on the parent class. The x-parent setting has precedence over the normalizer.

From my testing and with the feedback I got from OpenAPITools/openapi-generator#16389, you'll need to set <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true,REFACTOR_ALLOF_WITH_PROPERTIES_ONLY=true</openapiNormalizer>
when the openapi style of combining allOf with object properties is used, like

    S:
      allOf:
        - $ref: '#/components/schemas/T'
      type: object
      properties:
        x:
          type: string
        y:
          type: string

[jpraet]

IntelliJ Ultimate 2023.2 switched the OpenAPI preview from SwaggerUI to redocly. It has a couple UX regressions IMO: overview of all operations is less at-a-glance, and schemas aren't listed separately from the operations.

There's a button to switch to SwaggerUI, bit it unfortunately keeps opening the preview in Redocly by default.

Redocly also crashes on this:

  /health:
    $ref: cbss/common/v1/common-v1.yaml#/paths/~1health

I think because of Redocly/redoc#2337

Other sidenote: I also get "Unresolved reference" errors now in the openapi editor since the last IntelliJ IDE upgrade.

        default:
          $ref: belgif/problem/v1/problem-v1.yaml#/components/responses/ProblemResponse

It's happy when I add quotes

        default:
          $ref: "belgif/problem/v1/problem-v1.yaml#/components/responses/ProblemResponse"

[pvdbosch]

From my testing and with the feedback I got from OpenAPITools/openapi-generator#16389, you'll need to set <openapiNormalizer>REF_AS_PARENT_IN_ALLOF=true,REFACTOR_ALLOF_WITH_PROPERTIES_ONLY=true</openapiNormalizer> when the openapi style of combining allOf with object properties is used, like

Works for me without REFACTOR_ALLOF_WITH_PROPERTIES_ONLY, but I'd have to create a vanilla project and compare it with your configuration to make sure.

There's a button to switch to SwaggerUI

Thanks, hadn't seen that.

[jpraet]

As of openapi-generator 6.5, optional array properties no longer get initialized with an empty list (OpenAPITools/openapi-generator#15891). From the ticket, it seems unsure whether that change in behavior is either intentional, or a regression.

While we don't want implementation quirks in tooling inform our decisions on API guidelines, it does raise the question for me whether we should spend more thought on making array properties required vs optional.

For example: the items property of collection resources.
The https://www.belgif.be/specification/rest/api-guide/#rule-col-repres states:

In case the collection is empty, the items property MUST have an empty array as value.

So shouldn't the items property be marked required then in the example openapi spec?

rest-guide/guide/src/main/asciidoc/resources-collection.adoc

Lines 88 to 98 in 66606c8

	EmployersResponse:
	allOf:
	- $ref: common/v1/common-v1.yaml#/definitions/SelfLink
	- type: object
	properties:
	items:
	type: array
	items:
	$ref: "#/definitions/EmployerLink"
	total:
	type: integer

[pvdbosch]

I agree, I don't see any use case in which "items" array shouldn't present in a collection.
When the array property is marked as required, does openapi-generator initialize it to an empty list in version >= 6.5?

I think the new behavior of the openapi-generator is consistent; other types are also initialized as null when the property is absent.

We also consider null and absent JSON properties as equivalent, while an empty array is different.

Rule: Null values [jsn-null]

Properties with a null value SHOULD be stripped from the JSON message.

A notable exception to this rule are JSON Merge Patch requests, in which a null value indicates the removal of a JSON property.

Note that this rule doesn’t apply to empty values (e.g. empty strings "" or empty arrays []) as they are considered different from a null value.

[pvdbosch]

some more openapi-generator 7.0 findings:

• additionalProperties are sometimes now generated as "class X extends HashMap<String, Type>", but not for the spring generator where it uses @JsonAnyGetter and @JsonAnySetter methods - fix: #1466 additionalProperties works now in spring generator OpenAPITools/openapi-generator#11572 ; hopefully this gets applied in future releases to JAX-RS as well
• plugin generates some markdown files with documentation now

[jpraet]

When the array property is marked as required, does openapi-generator initialize it to an empty list in version >= 6.5?

Yes

[jpraet]

IntelliJ Ultimate 2023.2 switched the OpenAPI preview from SwaggerUI to redocly. It has a couple UX regressions IMO: overview of all operations is less at-a-glance, and schemas aren't listed separately from the operations.

There's a button to switch to SwaggerUI, bit it unfortunately keeps opening the preview in Redocly by default.

As of 2023.2.2, IntelliJ remembers your choice of preview tool.

[wsalembi]

In v7.0.0, model classes have a new constructor with required properties. If this constructor is a single arg, Spring boot/Jackson applies a mapping so that fields with the

-- EXPECTED

"municipality": {
  "name": "Antwerp"
} 

-- ACCEPTED
"municipality": "Antwerp"


-- because of model.Municipality(String name) {} constructor

-- EXPECTED

"country": {
  "countryNisCode": 150
} 

-- REFUSED with error "country is not of type int" 
"country": "Belgium"

-- because of model.Country(Integer countryNisCode) {} constructor

[pvdbosch]

to investigate: differences between typeMappings/schemaMappings/importMappings in openapi-generator

[pvdbosch]

I've created a couple GitHub Discussions to replace this issue, in order to keep the issue backlog clean and the discussions more focused:

• OpenAPI editors #145
• OpenAPI validators #146
• Code generators #147

Please continue to exchange your ideas and experiences in these discussions.