Support more collections in STJ source generator #55566

layomia · 2021-07-13T13:28:07Z

Fixes #53393. The collection trimming tests are not updated in this PR (#53437).

dotnet-issue-labeler · 2021-07-13T13:28:13Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

ghost · 2021-07-13T13:28:15Z

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

Fixes #53393. The collection trimming tests are not updated in this PR (#53437).

Author:	layomia
Assignees:	layomia
Labels:	`area-System.Text.Json`
Milestone:	6.0.0

src/libraries/System.Text.Json/Common/ReflectionExtensions.cs

src/libraries/System.Text.Json/gen/TypeGenerationSpec.cs

src/libraries/System.Text.Json/Common/ReflectionExtensions.cs

src/libraries/System.Text.Json/tests/Common/CollectionTests/CollectionTests.Specialized.Read.cs

...tem.Text.Json/tests/System.Text.Json.SourceGeneration.Tests/Serialization/CollectionTests.cs

src/libraries/System.Text.Json/gen/Reflection/TypeExtensions.cs

steveharter

Misc feedback provided (none that can't be addressed after this PR)

Looks good pending green CI.

This new collection support is important to get in for Preview 7.

src/libraries/System.Text.Json/gen/JsonSourceGenerator.Parser.cs

src/libraries/System.Text.Json/ref/System.Text.Json.cs

layomia · 2021-07-15T18:00:45Z

FYI @eerhardt

eiriktsarpalis · 2021-07-15T18:16:56Z

src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfo.cs

+
+                        if (converter.RequiresDynamicMemberAccessors)
+                        {
+                            converter.Initialize(Options, this);


does this account for the possibility that one converter instance is initialized by multiple JsonTypeInfo instances? I think that it's occassionally possible although not sure it applies to converters that override RequiresDynamicMemberAccessors.

Hmm I think it could happen occasionally due to threading, but I think even in that case each JsonTypeInfo construction would create a separate converter instance, and the first one to complete would win, discarding the duplicates. Each converter instance would only be for a specific type, so there are no issues of creating invalid dynamic accessors.

eiriktsarpalis · 2021-07-15T18:22:56Z

src/libraries/System.Text.Json/tests/Common/CollectionTests/CollectionTests.AsyncEnumerable.cs

        {
            JsonSerializerOptions options = new JsonSerializerOptions
            {
                DefaultBufferSize = bufferSize
            };

-            string expectedJson = JsonSerializer.Serialize(source);
+            string expectedJson = await JsonSerializerWrapperForString.SerializeWrapper(source);


Since this method call is only used to generate the expected JSON string, we don't really need to exercise any async APIs here.

eiriktsarpalis · 2021-07-15T18:23:19Z

src/libraries/System.Text.Json/tests/Common/CollectionTests/CollectionTests.AsyncEnumerable.cs

        {
            JsonSerializerOptions options = new JsonSerializerOptions
            {
                DefaultBufferSize = bufferSize
            };

-            string expectedJson = JsonSerializer.Serialize(new { Data = source });
+            string expectedJson = await JsonSerializerWrapperForString.SerializeWrapper(new { Data = source });


Ditto for this and all the other expectedJson values in this file.

eiriktsarpalis · 2021-07-15T18:23:35Z

src/libraries/System.Text.Json/tests/Common/CollectionTests/CollectionTests.AsyncEnumerable.cs

@@ -9,24 +9,25 @@
 using System.Threading.Tasks;
 using Xunit;

-namespace System.Text.Json.Tests.Serialization
+namespace System.Text.Json.Serialization.Tests


Thanks for fixing this!

eiriktsarpalis · 2021-07-15T18:40:52Z

...em.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.DeserializeAsyncEnumerable.cs

@@ -7,7 +7,7 @@
 using System.Threading;
 using System.Threading.Tasks;
 using Xunit;
-using Utf8MemoryStream = System.Text.Json.Tests.Serialization.CollectionTests.Utf8MemoryStream;


FWIW the System.Text.Json.Tests.Serialization namespace has crept up in quite a few other places as well.

layomia · 2021-07-15T18:41:41Z

Test failures are unrelated. Will address further feedback in a follow-up.

...rc/System/Text/Json/Serialization/Converters/Collection/IEnumerableWithAddMethodConverter.cs

layomia added the area-System.Text.Json label Jul 13, 2021

layomia added this to the 6.0.0 milestone Jul 13, 2021

layomia self-assigned this Jul 13, 2021

dotnet-issue-labeler bot added the new-api-needs-documentation label Jul 13, 2021

layomia mentioned this pull request Jul 13, 2021

Remove C# dynamic support from JsonNode #55430

Merged

layomia force-pushed the SrcGenCollections branch 5 times, most recently from 17ef405 to f78d7b9 Compare July 14, 2021 16:28

layomia marked this pull request as ready for review July 14, 2021 16:30

layomia requested review from eiriktsarpalis, jozkee and steveharter as code owners July 14, 2021 16:30

layomia force-pushed the SrcGenCollections branch from f78d7b9 to ab75b8d Compare July 14, 2021 17:35

steveharter reviewed Jul 14, 2021

View reviewed changes

src/libraries/System.Text.Json/Common/ReflectionExtensions.cs Show resolved Hide resolved

src/libraries/System.Text.Json/Common/ReflectionExtensions.cs Show resolved Hide resolved

steveharter reviewed Jul 14, 2021

View reviewed changes

src/libraries/System.Text.Json/gen/TypeGenerationSpec.cs Show resolved Hide resolved

steveharter reviewed Jul 14, 2021

View reviewed changes

src/libraries/System.Text.Json/gen/Reflection/TypeExtensions.cs Show resolved Hide resolved

steveharter reviewed Jul 14, 2021

View reviewed changes

src/libraries/System.Text.Json/gen/Reflection/TypeExtensions.cs Show resolved Hide resolved

steveharter approved these changes Jul 14, 2021

View reviewed changes

layomia force-pushed the SrcGenCollections branch 3 times, most recently from 6ce1542 to 6c05c74 Compare July 15, 2021 02:47

layomia closed this Jul 15, 2021

layomia reopened this Jul 15, 2021

layomia force-pushed the SrcGenCollections branch from 6c05c74 to 011c403 Compare July 15, 2021 03:08

layomia closed this Jul 15, 2021

layomia reopened this Jul 15, 2021

layomia added 3 commits July 14, 2021 20:34

Support more collections in STJ source generator

6a30849

Fix tests

b0f2019

Revert redundant MLC change and address feedback

01ffde9

layomia force-pushed the SrcGenCollections branch from d408b89 to 01ffde9 Compare July 15, 2021 03:35

eiriktsarpalis reviewed Jul 15, 2021

View reviewed changes

src/libraries/System.Text.Json/gen/JsonSourceGenerator.Parser.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jul 15, 2021

View reviewed changes

src/libraries/System.Text.Json/ref/System.Text.Json.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jul 15, 2021

View reviewed changes

src/libraries/System.Text.Json/ref/System.Text.Json.cs Show resolved Hide resolved

eiriktsarpalis reviewed Jul 15, 2021

View reviewed changes

layomia merged commit 0a5cf4e into dotnet:main Jul 15, 2021

eerhardt reviewed Jul 15, 2021

View reviewed changes

...rc/System/Text/Json/Serialization/Converters/Collection/IEnumerableWithAddMethodConverter.cs Show resolved Hide resolved

layomia mentioned this pull request Jul 16, 2021

Reimplement dynamic member access for collection converters correctly #55846

Merged

ManickaP mentioned this pull request Jul 20, 2021

[QUIC] Remove AppContext switch from S.N.Quic #56027

Merged

MaximLipnin mentioned this pull request Jul 21, 2021

[MacCatalyst] App crash in System.Text.Json.SourceGeneration.Tests #56085

Closed

layomia mentioned this pull request Jul 23, 2021

Using Queue, Queue<T>, Stack, and ConcurrentStack<T> as properties in classes deserialized by System.Text.Json does not work when trimmed #53205

Closed

ghost locked as resolved and limited conversation to collaborators Aug 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support more collections in STJ source generator #55566

Support more collections in STJ source generator #55566

layomia commented Jul 13, 2021

dotnet-issue-labeler bot commented Jul 13, 2021

ghost commented Jul 13, 2021

steveharter left a comment

layomia commented Jul 15, 2021

eiriktsarpalis Jul 15, 2021

layomia Jul 15, 2021

eiriktsarpalis Jul 15, 2021

eiriktsarpalis Jul 15, 2021

eiriktsarpalis Jul 15, 2021

eiriktsarpalis Jul 15, 2021

layomia commented Jul 15, 2021

Support more collections in STJ source generator #55566

Support more collections in STJ source generator #55566

Conversation

layomia commented Jul 13, 2021

dotnet-issue-labeler bot commented Jul 13, 2021

ghost commented Jul 13, 2021

steveharter left a comment

Choose a reason for hiding this comment

layomia commented Jul 15, 2021

eiriktsarpalis Jul 15, 2021

Choose a reason for hiding this comment

layomia Jul 15, 2021

Choose a reason for hiding this comment

eiriktsarpalis Jul 15, 2021

Choose a reason for hiding this comment

eiriktsarpalis Jul 15, 2021

Choose a reason for hiding this comment

eiriktsarpalis Jul 15, 2021

Choose a reason for hiding this comment

eiriktsarpalis Jul 15, 2021

Choose a reason for hiding this comment

layomia commented Jul 15, 2021