System.Text.Json.Serialization.JsonStringEnumConverter handles duplicate enum members inconsistently

[hf-kklein]

Description

When there is an enum and multiple enum members have the same underlying integer value, then the serialization behaviour is not consistent.

In the below examples the members marked as obsolete are there for legacy compatability.
In case a JSON has to be deserialized, it should still be possible to use the obsolete members, but internally I want to use the non-obsolete ones.

Reproduction Steps

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace TestSerialization;

public enum MyEnumA
{
    [EnumMember(Value = "FOO")] FOO = 1,

    [EnumMember(Value = "BAR")] BAR = 2,

    [Obsolete("Use FOO instead")] [EnumMember(Value = "F_OO")]
    F_OO = 1,

    [Obsolete("Use BAR instead")] [EnumMember(Value = "B_AR")]
    B_AR = 2,
}

/// <summary>
/// This enum has the same members as MyEnumA but in a different order.
/// Still the order between {FOO/F_OO} and {BAR/B_AR} is the same: Foo comes before Bar.
/// </summary>
public enum MyEnumB
{
    [Obsolete("Use FOO instead")] [EnumMember(Value = "F_OO")]
    F_OO = 1,

    [Obsolete("Use BAR instead")] [EnumMember(Value = "B_AR")]
    B_AR = 2,

    [EnumMember(Value = "FOO")] FOO = 1,

    [EnumMember(Value = "BAR")] BAR = 2,
}

public class MyClass
{
    public MyEnumA EnumMemberAFoo { get; set; }
    public MyEnumA EnumMemberAF_oo { get; set; }
    public MyEnumA EnumMemberABar { get; set; }
    public MyEnumA EnumMemberAB_ar { get; set; }
    
    
    public MyEnumB EnumMemberBFoo { get; set; }
    public MyEnumB EnumMemberBF_oo { get; set; }
    public MyEnumB EnumMemberBBar { get; set; }
    public MyEnumB EnumMemberBB_ar { get; set; }
}

[TestClass]
public class TestJsonStringEnumSerialization
{
    [TestMethod]
    [DataRow("EnumMemberA")]
    [DataRow("EnumMemberB")]
    public void Enum_Serialization_Behaviour_Is_Consistent(string prefix)
    {
        var myInstance = new MyClass
        {
            EnumMemberAFoo = MyEnumA.FOO,
            EnumMemberAF_oo = MyEnumA.F_OO,
            EnumMemberABar = MyEnumA.BAR,
            EnumMemberAB_ar = MyEnumA.B_AR,
            EnumMemberBFoo = MyEnumB.FOO,
            EnumMemberBF_oo = MyEnumB.F_OO,
            EnumMemberBBar = MyEnumB.BAR,
            EnumMemberBB_ar = MyEnumB.B_AR
        };

        var jsonString = System.Text.Json.JsonSerializer.Serialize(myInstance, new System.Text.Json.JsonSerializerOptions
        {
            Converters = { new System.Text.Json.Serialization.JsonStringEnumConverter() },
            WriteIndented = true
        });
        /*
        jsonString looks like this:
        {
          "EnumMemberAFoo": "F_OO",
          "EnumMemberAF_oo": "F_OO",
          "EnumMemberABar": "BAR",
          "EnumMemberAB_ar": "BAR",
          "EnumMemberBFoo": "FOO",
          "EnumMemberBF_oo": "FOO",
          "EnumMemberBBar": "B_AR",
          "EnumMemberBB_ar": "B_AR"
        }
        */
        var untypedDict = System.Text.Json.JsonSerializer.Deserialize<Dictionary<string, string>>(jsonString);
        Assert.IsNotNull(untypedDict);

        bool anyMemberAIsSerializedWithUnderscore = untypedDict.Where(kvp => kvp.Key.StartsWith(prefix)).Any(kvp => kvp.Value.Contains('_'));
        bool serializationBehaviourIsSelfConsistent;
        if (anyMemberAIsSerializedWithUnderscore)
        {
            serializationBehaviourIsSelfConsistent = untypedDict.Where(kvp => kvp.Key.StartsWith(prefix)).All(kvp => kvp.Value.Contains('_'));
        }
        else
        {
            // in this example we never go through this code path, but it makes the intention clear
            serializationBehaviourIsSelfConsistent = !untypedDict.Where(kvp => kvp.Key.StartsWith(prefix)).Any(kvp => kvp.Value.Contains('_'));
        }

        Assert.IsTrue(serializationBehaviourIsSelfConsistent);
    }
}

Expected behavior

For MyEnumA both non-obsolete members occur first:

• FOO before F_OO
• BAR before B_AR

For MyEnumB both obsolete members occur first:

• F_OO before FOO
• B_AR before BAR

My expectations are:

• If any Property with type EnumMemberA is serialized as F_OO, then other properties with type EnumMemberA should also use B_AR instead of BAR because there's no reason for different behaviour.
• If any Property with type EnumMemberA is serialized as FOO, then other properties with type EnumMemberA should also use BAR instead of B_AR because there's no reason for different behaviour.

Actual behavior

The behaviour does not depend on the ordering of the members with same underlying integer but on something else.
The behaviour is not self-consistent.

Please be aware: I'm not asking for either FOO or F_OO to be used by the serializer. It's a problem of my code that it uses two possible serializations for the same integer value. I'm fine with that and both as the serializer cannot guess which one I prefer.

But I'm expecting consistency: No F_OO when at other places BAR occurs for properties of the same type.

Regression?

No response

Known Workarounds

I don't know what exactly I'm expecting. But at least the behaviour should be self-consistent.

Configuration

.NET8

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[elgonzo]

(My apologies, i deleted my previous comment as it was pointless due to me not having understood your problem. Sorry about that...)

You cannot rely on the order of the enum member in the enum declaration governing which name an enum value is being converted into. As per the documentation for Enum.ToString() (https://learn.microsoft.com/en-us/dotnet/api/system.enum.tostring):

If multiple enumeration members have the same underlying value and you attempt to retrieve the string representation of an enumeration member's name based on its underlying value, your code should not make any assumptions about which name the method will return.

I noticed that in a PR in which you linked to this issue report, you stated that Newtonsoft.Json works.
Please be aware that you cannot rely on such observations for behavior that is essentially undefined.

Because here is the kicker: It also works with STJ, for example in dotnetfiddle: http://dotnetfiddle.net/gzXM4o. Given that regarding STJ there are environments where the resolution of duplicate enum members matches your expectations and environments where they don't, it is reasonable to assume the same is true for resolution of duplicate enum members when Newtonsoft.Json is at play. And as it so happens, dotnetfiddle is also an environment where using Newtonsoft.Json - unlike STJ there - won't meet your expectations: http://dotnetfiddle.net/Z3KaCm You are basically right now deep in "It works here but not there" territory if you try relying on undefined behavior like this...

[hf-kklein]

Thanks @elgonzo ! At least I learned something :)