Removal of dependency on Newtonsoft

[brentschmaltz]

This PR removes the use of Newtonsoft from IdentityModel by using Utf8JsonReader/Writer
JsonWebTokens were previously populated with JsonDocument.Parse, which resulted Claims and Values being parsed from a JsonElement each time properties were accessed.
This PR introduces a new model of reading with a Utf8JsonReader and creating a collection of values that have been deserialized from Json. This will improve performance when accessing either a System.Claim or a C# property. There is more work to do as it will be helpful if the JsonElement is remembered when parsing as when creating a Claim, the serialized Json is needed in the claim.

A first cut at improving creation of JWT's when a reduced number of string -> Utf8bytes -> string + string -> Utf8bytes transformations were taking place. The goal is to use buffer pooling and write into the buffer (Span) once, perform all work (signature, compression, encryption) on the buffer.

Standardized roundtrip serialization was improved with the goal of providing a deterministic model for 'additional' data found on some objects.

[jennyf19]

    public void TryGetClaim()

getting this error locally:

str1 != str2, StringComparison: 'Ordinal'
http://www.w3.org/2001/XMLSchema#double
http://www.w3.org/2001/XMLSchema#integer32

---

Refers to: test/Microsoft.IdentityModel.JsonWebTokens.Tests/JsonWebTokenTests.cs:160 in 40be4d0. [](commit_id = 40be4d0, deletion_comment = False)

[jennyf19]

    public void GetClaim()

str1 != str2, StringComparison: 'Ordinal'
http://www.w3.org/2001/XMLSchema#double
http://www.w3.org/2001/XMLSchema#integer32

---

Refers to: test/Microsoft.IdentityModel.JsonWebTokens.Tests/JsonWebTokenTests.cs:134 in 40be4d0. [](commit_id = 40be4d0, deletion_comment = False)

[jmprieur]

One breaking change in Wilson7 that we might want to fix (but in preview4) is:

jsonWebToken.TryGetPayload(JwtClaimsTypes.Roles, out roles) 

used to accept that roles is a string[]. I now needs to be a IList<string>

@brentschmaltz:

• is it expected?
• do we agree that we would provide something in preview4? (not preview 3)?

[jennyf19]

One breaking change in Wilson7 that we might want to fix (but in preview4) is:

jsonWebToken.TryGetPayload(JwtClaimsTypes.Roles, out roles) 

used to accept that roles is a string[]. I now needs to be a IList<string>

@brentschmaltz:

• is it expected?
• do we agree that we would provide something in preview4? (not preview 3)?

Thoughts @kevinchalet ?

[kevinchalet]

Thoughts @kevinchalet ?

Thanks for tagging me, @jennyf19!

I'm assuming TryGetPayload actually means TryGetPayloadValue. If so, OpenIddict uses it in 3 places: twice with string and once with ImmutableDictionary<string, string[]>. Chances things are broken with string are low, but with ImmutableDictionary<string, string[]>, it's not unreasonable so I'll give it a try 😄

Was this PR included in the preview that shipped yesterday?

[jennyf19]

Thoughts @kevinchalet ?

Thanks for tagging me, @jennyf19!

I'm assuming TryGetPayload actually means TryGetPayloadValue. If so, OpenIddict uses it in 3 places: twice with string and once with ImmutableDictionary<string, string[]>. Chances things are broken with string are low, but with ImmutableDictionary<string, string[]>, it's not unreasonable so I'll give it a try 😄

Was this PR included in the preview that shipped yesterday?

Yes, this is in preview3, which is out on NuGet as of Sunday (8/20) night. Would appreciate your feedback on this one.

[kevinchalet]

Would appreciate your feedback on this one.

Sadly, I'm seeing important regressions when moving to preview3. I'll open dedicated tickets.

[kevinchalet]

Sadly, I'm seeing important regressions when moving to preview3. I'll open dedicated tickets.

Here's the first one: #2244

[kevinchalet]

And the second one: #2245.

[kevinchalet]

And the third one: #2246.

It seems the migration to System.Text.Json has caused a lot of side effects/regressions. More agressive E2E testing would be a good idea. Having such bugs in the final version would be absolutely terrible 😅

[jennyf19]

And the third one: #2246.

It seems the migration to System.Text.Json has caused a lot of side effects/regressions. More agressive E2E testing would be a good idea. Having such bugs in the final version would be absolutely terrible 😅

We agree. Appreciate the bugs and all the thorough testing. We will improve our coverage as well.