[BUG] [csharp-netcore] AdditionalProperties results in model inheriting Dictionary causing serialisation issues

[timvandenhof]

Description

An API vendor has an API that has a schema that specifies both (explicit) properties and additional properties.
This causes a C# model to be generated that inherits from Dictionary<string, object> and a property of the same type.
It seems that the JSON de-serializer can not handle this situation, as the properties are left blank/empty while the dictionary (where the model inherited from) is filled with data.

This seems like sort of documented behavior of the Newtonsoft.Json serializer. At least for serializing, nothing is mentioned about deserializing.

.NET dictionaries (types that inherit from IDictionary) are converted to JSON objects. Note that only the dictionary name/values will be written to the JSON object when serializing, and properties on the JSON object will be added to the dictionary's name/values when deserializing. Additional members on the .NET dictionary are ignored during serialization.

When serializing a dictionary, the keys of the dictionary are converted to strings and used as the JSON object property names. The string written for a key can be customized by either overriding ToString() for the key type or by implementing a TypeConverter. A TypeConverter will also support converting a custom string back again when deserializing a dictionary.

JsonDictionaryAttribute has options on it to customize the JsonConverter, type name handling, and reference handling that are applied to collection items.

When deserializing, if a member is typed as the interface IDictionary<TKey, TValue> then it will be deserialized as a Dictionary<TKey, TValue>.

See: https://www.newtonsoft.com/json/help/html/SerializationGuide.htm

openapi-generator version

Open API 5.1.1
https://github.com/OpenAPITools/openapi-generator/releases/tag/v5.1.1

OpenAPI declaration file content or url

repro.yml:

openapi: 3.0.1
info:
  title: Repro minimal API
  version: 0.0.1
paths:
  '/somerequest':
    get:
      summary: Some request
      responses:
        '200':
          description: Operation successful
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/GetSomeSchema'
components:
  schemas:
    GetSomeSchema:
      required:
        - someKey
        - someValue
        - someList
      description: JSON schema for GetSomeSchema
      type: object
      additionalProperties: true
      properties:
        someKey:
          description: Some key
          example: 6dd581d9-055b-4744-aa43-c4abe2bdb123
          type: string
        someValue:
          description: Some value
          example: 120
          type: integer
        someList:
          description: List of complex items
          type: array
          items:
            $ref: '#/components/schemas/SomeListItem'
    SomeListItem:
      description: Some list item object
      type: object
      additionalProperties: true
      properties:
        someChildKey:
          description: List item key.
          example: '1'
          type: string
        someChildValue:
          description: List item value.
          example: 120
          type: integer

Generation Details

Input:

Powershell command:

java -jar openapi-generator-cli.jar generate `
	-i repro.yml`
	-g csharp-netcore `
	-o .\ `
	-c config.yml `
	--skip-validate-spec

config.yml:

apiTests: false
modelTests: false

additionalProperties:
  targetFramework: netstandard2.0
  netCoreProjectFile: true
  useDateTimeOffset: true

Output:

GetSomeSchema.cs:

/*
 * Repro minimal API
 *
 * No description provided (generated by Openapi Generator https://github.com/openapitools/openapi-generator)
 *
 * The version of the OpenAPI document: 0.0.1
 * Generated by: https://github.com/openapitools/openapi-generator.git
 */


using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using System.IO;
using System.Runtime.Serialization;
using System.Text;
using System.Text.RegularExpressions;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
using Newtonsoft.Json.Linq;
using System.ComponentModel.DataAnnotations;
using OpenAPIDateConverter = Tls.Clients.Gbo.Client.OpenAPIDateConverter;

namespace Tls.Clients.Gbo.Model
{
    /// <summary>
    /// JSON schema for GetSomeSchema
    /// </summary>
    [DataContract(Name = "GetSomeSchema")]
    public partial class GetSomeSchema : Dictionary<String, Object>, IEquatable<GetSomeSchema>, IValidatableObject
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="GetSomeSchema" /> class.
        /// </summary>
        [JsonConstructorAttribute]
        protected GetSomeSchema()
        {
            this.AdditionalProperties = new Dictionary<string, object>();
        }
        /// <summary>
        /// Initializes a new instance of the <see cref="GetSomeSchema" /> class.
        /// </summary>
        /// <param name="someKey">Some key (required).</param>
        /// <param name="someValue">Some value (required).</param>
        /// <param name="someList">List of complex items (required).</param>
        public GetSomeSchema(string someKey = default(string), int someValue = default(int), List<SomeListItem> someList = default(List<SomeListItem>)) : base()
        {
            // to ensure "someKey" is required (not null)
            this.SomeKey = someKey ?? throw new ArgumentNullException("someKey is a required property for GetSomeSchema and cannot be null");
            this.SomeValue = someValue;
            // to ensure "someList" is required (not null)
            this.SomeList = someList ?? throw new ArgumentNullException("someList is a required property for GetSomeSchema and cannot be null");
            this.AdditionalProperties = new Dictionary<string, object>();
        }

        /// <summary>
        /// Some key
        /// </summary>
        /// <value>Some key</value>
        [DataMember(Name = "someKey", IsRequired = true, EmitDefaultValue = false)]
        public string SomeKey { get; set; }

        /// <summary>
        /// Some value
        /// </summary>
        /// <value>Some value</value>
        [DataMember(Name = "someValue", IsRequired = true, EmitDefaultValue = false)]
        public int SomeValue { get; set; }

        /// <summary>
        /// List of complex items
        /// </summary>
        /// <value>List of complex items</value>
        [DataMember(Name = "someList", IsRequired = true, EmitDefaultValue = false)]
        public List<SomeListItem> SomeList { get; set; }

        /// <summary>
        /// Gets or Sets additional properties
        /// </summary>
        [JsonExtensionData]
        public IDictionary<string, object> AdditionalProperties { get; set; }

        /// <summary>
        /// Returns the string presentation of the object
        /// </summary>
        /// <returns>String presentation of the object</returns>
        public override string ToString()
        {
            var sb = new StringBuilder();
            sb.Append("class GetSomeSchema {\n");
            sb.Append("  ").Append(base.ToString().Replace("\n", "\n  ")).Append("\n");
            sb.Append("  SomeKey: ").Append(SomeKey).Append("\n");
            sb.Append("  SomeValue: ").Append(SomeValue).Append("\n");
            sb.Append("  SomeList: ").Append(SomeList).Append("\n");
            sb.Append("  AdditionalProperties: ").Append(AdditionalProperties).Append("\n");
            sb.Append("}\n");
            return sb.ToString();
        }

        /// <summary>
        /// Returns the JSON string presentation of the object
        /// </summary>
        /// <returns>JSON string presentation of the object</returns>
        public string ToJson()
        {
            return Newtonsoft.Json.JsonConvert.SerializeObject(this, Newtonsoft.Json.Formatting.Indented);
        }

        /// <summary>
        /// Returns true if objects are equal
        /// </summary>
        /// <param name="input">Object to be compared</param>
        /// <returns>Boolean</returns>
        public override bool Equals(object input)
        {
            return this.Equals(input as GetSomeSchema);
        }

        /// <summary>
        /// Returns true if GetSomeSchema instances are equal
        /// </summary>
        /// <param name="input">Instance of GetSomeSchema to be compared</param>
        /// <returns>Boolean</returns>
        public bool Equals(GetSomeSchema input)
        {
            if (input == null)
                return false;

            return base.Equals(input) && 
                (
                    this.SomeKey == input.SomeKey ||
                    (this.SomeKey != null &&
                    this.SomeKey.Equals(input.SomeKey))
                ) && base.Equals(input) && 
                (
                    this.SomeValue == input.SomeValue ||
                    this.SomeValue.Equals(input.SomeValue)
                ) && base.Equals(input) && 
                (
                    this.SomeList == input.SomeList ||
                    this.SomeList != null &&
                    input.SomeList != null &&
                    this.SomeList.SequenceEqual(input.SomeList)
                )
                && (this.AdditionalProperties.Count == input.AdditionalProperties.Count && !this.AdditionalProperties.Except(input.AdditionalProperties).Any());
        }

        /// <summary>
        /// Gets the hash code
        /// </summary>
        /// <returns>Hash code</returns>
        public override int GetHashCode()
        {
            unchecked // Overflow is fine, just wrap
            {
                int hashCode = base.GetHashCode();
                if (this.SomeKey != null)
                    hashCode = hashCode * 59 + this.SomeKey.GetHashCode();
                hashCode = hashCode * 59 + this.SomeValue.GetHashCode();
                if (this.SomeList != null)
                    hashCode = hashCode * 59 + this.SomeList.GetHashCode();
                if (this.AdditionalProperties != null)
                    hashCode = hashCode * 59 + this.AdditionalProperties.GetHashCode();
                return hashCode;
            }
        }

        /// <summary>
        /// To validate all properties of the instance
        /// </summary>
        /// <param name="validationContext">Validation context</param>
        /// <returns>Validation Result</returns>
        IEnumerable<System.ComponentModel.DataAnnotations.ValidationResult> IValidatableObject.Validate(ValidationContext validationContext)
        {
            yield break;
        }
    }

}

SomeListItem.cs

/*
 * Repro minimal API
 *
 * No description provided (generated by Openapi Generator https://github.com/openapitools/openapi-generator)
 *
 * The version of the OpenAPI document: 0.0.1
 * Generated by: https://github.com/openapitools/openapi-generator.git
 */


using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
using System.IO;
using System.Runtime.Serialization;
using System.Text;
using System.Text.RegularExpressions;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
using Newtonsoft.Json.Linq;
using System.ComponentModel.DataAnnotations;
using OpenAPIDateConverter = Tls.Clients.Gbo.Client.OpenAPIDateConverter;

namespace Tls.Clients.Gbo.Model
{
    /// <summary>
    /// Some list item object
    /// </summary>
    [DataContract(Name = "SomeListItem")]
    public partial class SomeListItem : Dictionary<String, Object>, IEquatable<SomeListItem>, IValidatableObject
    {
        /// <summary>
        /// Initializes a new instance of the <see cref="SomeListItem" /> class.
        /// </summary>
        /// <param name="someChildKey">List item key..</param>
        /// <param name="someChildValue">List item value..</param>
        public SomeListItem(string someChildKey = default(string), int someChildValue = default(int)) : base()
        {
            this.SomeChildKey = someChildKey;
            this.SomeChildValue = someChildValue;
            this.AdditionalProperties = new Dictionary<string, object>();
        }

        /// <summary>
        /// List item key.
        /// </summary>
        /// <value>List item key.</value>
        [DataMember(Name = "someChildKey", EmitDefaultValue = false)]
        public string SomeChildKey { get; set; }

        /// <summary>
        /// List item value.
        /// </summary>
        /// <value>List item value.</value>
        [DataMember(Name = "someChildValue", EmitDefaultValue = false)]
        public int SomeChildValue { get; set; }

        /// <summary>
        /// Gets or Sets additional properties
        /// </summary>
        [JsonExtensionData]
        public IDictionary<string, object> AdditionalProperties { get; set; }

        /// <summary>
        /// Returns the string presentation of the object
        /// </summary>
        /// <returns>String presentation of the object</returns>
        public override string ToString()
        {
            var sb = new StringBuilder();
            sb.Append("class SomeListItem {\n");
            sb.Append("  ").Append(base.ToString().Replace("\n", "\n  ")).Append("\n");
            sb.Append("  SomeChildKey: ").Append(SomeChildKey).Append("\n");
            sb.Append("  SomeChildValue: ").Append(SomeChildValue).Append("\n");
            sb.Append("  AdditionalProperties: ").Append(AdditionalProperties).Append("\n");
            sb.Append("}\n");
            return sb.ToString();
        }

        /// <summary>
        /// Returns the JSON string presentation of the object
        /// </summary>
        /// <returns>JSON string presentation of the object</returns>
        public string ToJson()
        {
            return Newtonsoft.Json.JsonConvert.SerializeObject(this, Newtonsoft.Json.Formatting.Indented);
        }

        /// <summary>
        /// Returns true if objects are equal
        /// </summary>
        /// <param name="input">Object to be compared</param>
        /// <returns>Boolean</returns>
        public override bool Equals(object input)
        {
            return this.Equals(input as SomeListItem);
        }

        /// <summary>
        /// Returns true if SomeListItem instances are equal
        /// </summary>
        /// <param name="input">Instance of SomeListItem to be compared</param>
        /// <returns>Boolean</returns>
        public bool Equals(SomeListItem input)
        {
            if (input == null)
                return false;

            return base.Equals(input) && 
                (
                    this.SomeChildKey == input.SomeChildKey ||
                    (this.SomeChildKey != null &&
                    this.SomeChildKey.Equals(input.SomeChildKey))
                ) && base.Equals(input) && 
                (
                    this.SomeChildValue == input.SomeChildValue ||
                    this.SomeChildValue.Equals(input.SomeChildValue)
                )
                && (this.AdditionalProperties.Count == input.AdditionalProperties.Count && !this.AdditionalProperties.Except(input.AdditionalProperties).Any());
        }

        /// <summary>
        /// Gets the hash code
        /// </summary>
        /// <returns>Hash code</returns>
        public override int GetHashCode()
        {
            unchecked // Overflow is fine, just wrap
            {
                int hashCode = base.GetHashCode();
                if (this.SomeChildKey != null)
                    hashCode = hashCode * 59 + this.SomeChildKey.GetHashCode();
                hashCode = hashCode * 59 + this.SomeChildValue.GetHashCode();
                if (this.AdditionalProperties != null)
                    hashCode = hashCode * 59 + this.AdditionalProperties.GetHashCode();
                return hashCode;
            }
        }

        /// <summary>
        /// To validate all properties of the instance
        /// </summary>
        /// <param name="validationContext">Validation context</param>
        /// <returns>Validation Result</returns>
        IEnumerable<System.ComponentModel.DataAnnotations.ValidationResult> IValidatableObject.Validate(ValidationContext validationContext)
        {
            yield break;
        }
    }

}

Steps to reproduce

Run the command (either in powershell or windows terminal).
Then check out the output file GetSomeSchema.cs and SomeListItem.
Notice that both objects inherit from Dictionary which is causing issues with the Newtonsoft.Json serializer.

I've not been able to create a full repro project, as this minimal repo is from a 1.4 megabyte API definition. If needed I can create a full repro project.

Related issues/PRs

https://stackoverflow.com/questions/31099396/serialize-class-that-inherits-dictionary-is-not-serializing-properties
https://stackoverflow.com/questions/14893614/how-to-serialize-a-dictionary-as-part-of-its-parent-object-using-json-net

Suggest a fix

It seems that what is allowed in the OpenApi definition, a schema with explicit properties and additional properties, is not properly generating C# schema objects that can be handled by the used (de)serializer.

This is probably resolved by not having the schema object/class inherit Dictionary<string, object> any more. The specific properties are already there, including a property of type Dictionary<string, object> to store all properties on.

Not sure if this only needs a (breaking?) change on the mustache file or a more advanced modification.
Think it is generated by this file: https://github.com/OpenAPITools/openapi-generator/blob/e9fa93688617d0cb602c5805c48d80750d570289/modules/openapi-generator/src/main/resources/csharp-netcore/modelGeneric.mustache

Bug Report Checklist

• Have you provided a full/minimal spec to reproduce the issue?
• Have you validated the input using an OpenAPI validator (example)?
• Have you tested with the latest master to confirm the issue still exists?
• Have you searched for related issues/PRs?
• What's the actual output vs expected output?
• [Optional] Sponsorship to speed up the bug fix or feature request (example)