no-invalid-media-type-examples | not working as expected

[jeremyfiel]

Describe the bug

i've looked at the OAS schema and it uses either a ref or example object. We are using an example schema in our definition but the external file is not a valid example per the linter. I can't share the full schema for a repo of the issue but my external schema file is matching the expected response body schema, is there any reason why the linter says it should be type object? my external document is a valid uri

$ redocly lint --config ~/repos/.redocly.yaml openapi.json --format stylish

validating specs/openapi.json...
openapi.json:
  101:22  warning  no-invalid-media-type-examples  Example value must conform to the schema: type must be object.

openapi.json

"content": {
	"application/json": {
		"schema": {
			"$ref": "./schemas/response-schema_v01.json"
		},
		"examples": {
			"response": {
				"externalValue": "./examples/success-response_v01.json"
			}
		}
	}
}

Response body schema

{
	"$schema": "http://json-schema.org/draft-04/schema#",
	"title": "ActivityLogs",
	"description": "Activity Logs",
	"type": "object",
	"properties": {
		"ActivityLogs": {
			"type": "array",
			"items": {
				"type": "object",
				"additionalProperties": false,
				"properties": {
					"activityLog": {
						"$ref": "../../../../common/logEntryBaseType_v02.json"
					},
					"links": {
						"type": "array",
						"items": {
							"$ref": "../../../../common/linkType_v01.json"
						}
					}
				}
			}
		}
	},
	"additionalProperties": false
}

Example

{
	"ActivityLogs": [
		{ 
                      "activityLog": { ...

OAS meta schema

"examples": {
        "properties": {
          "example": true,
          "examples": {
            "type": "object",
            "additionalProperties": {
              "$ref": "#/$defs/example-or-reference"
            }
          }
        }
      },
"$defs": {
"example-or-reference": {
        "if": {
          "type": "object",
          "required": [
            "$ref"
          ]
        },
        "then": {
          "$ref": "#/$defs/reference"
        },
        "else": {
          "$ref": "#/$defs/example"
        }
      }
},
"example": {
        "$comment": "https://spec.openapis.org/oas/v3.1.0#example-object",
        "type": "object",
        "properties": {
          "summary": {
            "type": "string"
          },
          "description": {
            "type": "string"
          },
          "value": true,
          "externalValue": {
            "type": "string",
            "format": "uri"
          }
        },
        "not": {
          "required": [
            "value",
            "externalValue"
          ]
        },
        "$ref": "#/$defs/specification-extensions",
        "unevaluatedProperties": false
      }

[adamaltman]

Note for our team: I wonder if we have tests with externalValue and are resolving them?

[tatomyr]

@adamaltman we have tests for the no-example-value-and-externalValue rule, however we are not resolving externalValues at the moment.

[jeremyfiel]

@tatomyr does the tool compare examples with the response schema? from your comment, it sounds like this validation is not performed but the error provided is highlighting my example external schema file in VS Code.

[tatomyr]

@jeremyfiel do you mean the highlighted file is being referenced by externalValue field? We do not resolve such links yet.
Could it by any chance be referenced by $ref from another place?

[jeremyfiel]

Yes, I can show you on slack if you want to see

[tatomyr]

@jeremyfiel sure, feel free to ping me: [email protected]

[jeremyfiel]

i've tried to make a solid example of different locations and usage of examples in OAS3.0.x and OAS3.1.x based on discussion with Andrew in our call.

I'm asking for Redocly to support externalValue $ref resolution to validate examples against the request/response/example/header/parameter schema.

I may be able to provide some development effort from our internal tooling team to make this happen. If there is interest, we can arrange further discussion for how you would like the feature added.

redocly-examples-repo.md

[tatomyr]

Thanks for summing it up!

[jeremyfiel]

I found a workaround to externalValue by using $ref.

it's not super pretty, but it seems to work ok.

Because the examples collection expects example objects, creating the example object schema with summary, value satisfies the defined JSON Schema when the $ref is parsed

## external example file  ./examples/confirmMessage-example_v01.json
{
    "summary": "confirmMessage-example",
    "value": {
        "confirmMessageID": "391708e2-b82a-4e40-8079-29256022a85d",
        "processStatus": "success"
    }
}

## oas definition
"responses": {
	"400": {
		"description": "Bad Request",
		"headers": {
			...
		},
		"content": {
			"application/json": {
				"schema": {
					"$ref": "../../../common/shared/confirm-message-schema_v03.json"
				},
				"examples": {
					"confirmMessage": {
						"$ref": "./examples/confirmMessage-example_v01.json"
					}
				}
			}
		}
	}
                }

[lornajane]

We should be careful about assuming that we can resolve/parse externalValue the same way we do other references, the purpose of it is to allow examples that cannot easily be added in JSON or YAML formats (see https://spec.openapis.org/oas/v3.1.0#exampleObject). It should be expected to be a string though, not an object, I'm not sure if that's currently the case.

The workaround here, using $ref to point to the example in another file, I think is the best approach - @jeremyfiel does Redocly CLI correctly check the example against the schema when you do it this way?

(just noticed we should wish a happy birthday to this issue, I thought it had been open a while! 🎂 )

[jeremyfiel]

I thnk it's reasonable to expect the use of $ref over externalValue. You're right about the intention is to express non-json/yaml strings. Because it accepts a uri-reference I think checking for a filename extension prior to validation would be a reasonable compromise to enable this functionality.

It does check the example but it fails with schema id or key exists for reused schemas. There's another bug filed for this issue for windows users. (Myself)

[tatomyr]

Still to be implemented for the lint command.