Replace Newtonsoft.Json with the new .Net JsonSerializer. #3414

azchohfi · 2020-08-07T01:16:35Z

Fixes #3060, #889, #2957

This completely removes Newtonsoft.Json in favor of the new DotNet JsonSerializer.
We need to test this thoroughly. If something was serialized with Newtonsoft.Json and deserialized with this, it needs to work.

PR Type

What kind of change does this PR introduce?

Feature
Refactoring (no functional changes, no api changes)

What is the current behavior?

We rely on Newtonsoft.Json for any Json serialization or deserialization.

What is the new behavior?

We depend on the new System.Text.Json package.

PR Checklist

Please check if your PR fulfills the following requirements:

Tested code with current supported SDKs
Pull Request has been submitted to the documentation repository instructions. Link:
Sample in sample app has been added / updated (for bug fixes / features)
- Icon has been created (if new sample) following the Thumbnail Style Guide and templates
Tests for the changes have been added (for bug fixes / features) (if applicable)
Header has been added to all new source files (run build/UpdateHeaders.bat)
Contains NO breaking changes (lets test thoroughly, but I believe it should be fine).

ghost · 2020-08-07T01:16:37Z

Thanks azchohfi for opening a Pull Request! The reviewers will test the PR and highlight if there is any conflict or changes required. If the PR is approved we will proceed to merge the pull request 🙌

azchohfi · 2020-08-07T01:48:01Z

Since $type in JSON is not recommended, maybe its time we bump the version of the json to version 2, and create a new format that supports this better. @michael-hawker ? We could also support the same file version and just create a custom converter.

azchohfi · 2020-08-08T00:28:03Z

I fixed it with a few custom json converters.

Sergio0694

Looks great, glad to see the toolkit switching to System.Text.Json 🚀🚀🚀
Just left a couple suggestions and questions, but overall I like this a lot!

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs

Microsoft.Toolkit.Services/Services/Twitter/TwitterCoordinatesConverter.cs

Microsoft.Toolkit.Uwp.Services/Properties/Microsoft.Toolkit.Uwp.Services.rd.xml

Microsoft.Toolkit.Uwp.UI.Controls/InfiniteCanvas/Drawables/InkDrawable.cs

Microsoft.Toolkit.Uwp.UI.Controls/InfiniteCanvas/Drawables/TextDrawable.cs

Microsoft.Toolkit.Uwp.UI.Controls/InfiniteCanvas/JsonConverters/IDrawableConverter.cs

…in the services project.

ghost · 2020-08-13T19:05:36Z

This PR has been marked as "needs attention 👋" and awaiting a response from the team.

…Converter.cs Co-authored-by: Sergio Pedri <[email protected]>

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs

…der.cs Co-authored-by: Sergio Pedri <[email protected]>

ghost · 2020-08-13T22:30:08Z

This PR has been marked as "needs attention 👋" and awaiting a response from the team.

Sergio0694

Looks great! 🎉

Microsoft.Toolkit.Services/Microsoft.Toolkit.Services.csproj

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs

Co-authored-by: Sergio Pedri <[email protected]>

Microsoft.Toolkit.Services/Microsoft.Toolkit.Services.csproj

Rosuavio

I looks good, but I encountered an issue. An exception was thrown when I tried to "Import and Load" json generated from saving the canvas while on 1b08bd6.

I think we should try to support json generated from old version of the InfanteCanvas, can we fix this?

Attached is the json, the max view image, and the exception.
Infinite Canvas.zip

azchohfi · 2020-09-15T17:36:44Z

Nice catch! I'll look it up today!

azchohfi · 2020-09-15T18:32:44Z

Very simple fix. I'll just create a regression test to keep this stable.

ghost · 2020-09-15T19:36:08Z

This PR has been marked as "needs attention 👋" and awaiting a response from the team.

Rosuavio

Can't get it to crash with any old json (no matter how hard I scribble). Good work!

michael-hawker · 2020-09-15T21:06:36Z

@azchohfi looks like the last bit is that the unit test needs some rd.xml or other fix to load:

Test method UnitTests.UI.Controls.Test_InfiniteCanvas_Regression.Test_InfiniteCanvas_LoadsV1File threw exception: 
System.TypeInitializationException: A type initializer threw an exception. To determine which type, inspect the InnerException's StackTrace property. ---> EETypeRva:0x000430B0(System.Reflection.MissingRuntimeArtifactException): Cannot retrieve a MethodInfo for this delegate because the method it targeted (System.Text.Json.ReflectionMemberAccessor.CreateStructPropertyGetter[TClass, TProperty](ReflectionMemberAccessor.GetPropertyByRef`2)) was not enabled for metadata using the Dynamic attribute. For more information, please visit http://go.microsoft.com/fwlink/?LinkID=616868

michael-hawker

Approving, as I think we're good once the CI is fixed.

@azchohfi I think this means the only thing we're using Json serialization for in the UWP library is the StorageHelper, eh? And then for Controls it's the Infinite Canvas?

azchohfi · 2020-09-15T21:13:45Z

I think that is it, yes.

@Sergio0694

## Fixes #3692 This introduces a new light-weight serializer which we can use within the toolkit for the `SystemInformation` class still. And then developers with more complex type needs can specify their own serializer for the `BaseObjectStorageHelper` classes. See the related issue for more details. Follow-on to #3637 and #3414 ## PR Type What kind of change does this PR introduce?   - Feature  - Refactoring (no functional changes, no api changes)     ## What is the current behavior? Using `DataContractJsonSerializer`, though we had used `Newtonsoft.Json` in 6.1. ## What is the new behavior? Use new `SystemSerializer` which passes values to/from the Windows `ApplicationDataContainer` APIs directly. Updates our internal usages of this to the new system. Requires developer using the `BaseObjectStorageHelper` to provide a serializer implementation. This of course does provide a 'default'/system information within the toolkit now compared to our original thought of providing none at all before the `SystemInformation` problem was identified. ## PR Checklist Please check if your PR fulfills the following requirements: - [x] Tested code with current [supported SDKs](../readme.md#supported) - [ ] Pull Request has been submitted to the documentation repository [instructions](..\contributing.md#docs). Link:  - [ ] Sample in sample app has been added / updated (for bug fixes / features) - [ ] Icon has been created (if new sample) following the [Thumbnail Style Guide and templates](https://github.com/windows-toolkit/WindowsCommunityToolkit-design-assets) - [ ] New major technical changes in the toolkit have or will be added to the [Wiki](https://github.com/windows-toolkit/WindowsCommunityToolkit/wiki) e.g. build changes, source generators, testing infrastructure, sample creation changes, etc... - [ ] Tests for the changes have been added (for bug fixes / features) (if applicable) - [ ] Header has been added to all new source files (run *build/UpdateHeaders.bat*) - [ ] Contains **NO** breaking changes  ## Other information I tested copying the system.dat file from the Settings folder of our 6.1 sample app to the development version with the changes, no issues with `SystemInformation` reading the old data. Updated/added unit tests to test the old json layer vs. reading from the new SystemSerializer as well as just testing a new `System.Text.Json` complex scenario. Added tests for exceptions. TODO: - [ ] Add more tests? - [x] Run XAML Islands SystemInformation test - [ ] Do we want `SystemSerializer` to handle a few more 'basic' (but non-primitive) types? E.g. @Sergio0694 has a [similar implementation for a different related system](https://github.com/Sergio0694/Brainf_ckSharp/blob/master/src/Brainf_ckSharp.Services.Uwp/SettingsService.cs) - [ ] ???

Replace Newtonsoft.Json with the new .Net JsonSerializer.

467b173

ghost requested review from michael-hawker and Kyaa-dost August 7, 2020 01:16

ghost added the feature request 📬 A request for new changes to improve functionality label Aug 7, 2020

Updated InfiniteCanvas to use new .Net JsonSerializer.

48b1f54

Merge branch 'master' into removeNewtonsoftJson

e552564

azchohfi marked this pull request as ready for review August 8, 2020 00:28

azchohfi requested a review from Sergio0694 August 8, 2020 01:06

Merge branch 'master' into removeNewtonsoftJson

f53739b

Sergio0694 requested changes Aug 12, 2020

View reviewed changes

ghost added the needs author feedback 📝 label Aug 12, 2020

Simplified many using statements by changing the lang version to C#8 …

b446d06

…in the services project.

ghost added needs attention 👋 and removed needs author feedback 📝 labels Aug 13, 2020

azchohfi and others added 3 commits August 13, 2020 12:05

Update Microsoft.Toolkit.Services/Services/Twitter/TwitterCoordinates…

043edd6

…Converter.cs Co-authored-by: Sergio Pedri <[email protected]>

Applied PR feedback.

b5542c0

Merge branch 'master' into removeNewtonsoftJson

5328178

azchohfi requested a review from Sergio0694 August 13, 2020 19:33

Small improvement, removing magic number.

df7dab1

Sergio0694 requested changes Aug 13, 2020

View reviewed changes

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs Outdated Show resolved Hide resolved

ghost added the needs author feedback 📝 label Aug 13, 2020

Update Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvi…

00b7241

…der.cs Co-authored-by: Sergio Pedri <[email protected]>

ghost removed the needs author feedback 📝 label Aug 13, 2020

azchohfi requested a review from Sergio0694 August 14, 2020 19:29

Sergio0694 approved these changes Aug 15, 2020

View reviewed changes

Microsoft.Toolkit.Services/Microsoft.Toolkit.Services.csproj Outdated Show resolved Hide resolved

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs Show resolved Hide resolved

Microsoft.Toolkit.Services/Services/LinkedIn/LinkedInDataProvider.cs Show resolved Hide resolved

Remove extra blank line

e898102

Co-authored-by: Sergio Pedri <[email protected]>

michael-hawker mentioned this pull request Aug 18, 2020

Win2d Path Mini Language Parser added. #3430

Closed

7 tasks

Merge branch 'master' into removeNewtonsoftJson

7dae4ea

michael-hawker self-assigned this Sep 1, 2020

michael-hawker reviewed Sep 1, 2020

View reviewed changes

Microsoft.Toolkit.Services/Microsoft.Toolkit.Services.csproj Show resolved Hide resolved

michael-hawker added reviewers wanted ✋ for-review 📖 To evaluate and validate the Issues or PR labels Sep 1, 2020

azchohfi and others added 2 commits September 1, 2020 11:00

Merge branch 'master' into removeNewtonsoftJson

15ea761

Merge branch 'master' into removeNewtonsoftJson

11daa4f

Rosuavio suggested changes Sep 15, 2020

View reviewed changes

ghost added the needs author feedback 📝 label Sep 15, 2020

Fix infinite canvas json parsing regression.

915fdc7

ghost removed the needs author feedback 📝 label Sep 15, 2020

Rosuavio approved these changes Sep 15, 2020

View reviewed changes

ghost removed the needs attention 👋 label Sep 15, 2020

michael-hawker approved these changes Sep 15, 2020

View reviewed changes

Fixed .Net Native compilation of System.Text.Json.

89dafd1

Rosuavio merged commit 0e2d49f into master Sep 16, 2020

delete-merged-branch bot deleted the removeNewtonsoftJson branch September 16, 2020 17:42

michael-hawker added this to the 7.0 milestone Sep 16, 2020

michael-hawker mentioned this pull request Jan 28, 2021

SystemSerializer update for BaseObjectStorageHelper #3702

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace Newtonsoft.Json with the new .Net JsonSerializer. #3414

Replace Newtonsoft.Json with the new .Net JsonSerializer. #3414

azchohfi commented Aug 7, 2020 •

edited

Loading

ghost commented Aug 7, 2020

azchohfi commented Aug 7, 2020

azchohfi commented Aug 8, 2020

Sergio0694 left a comment

ghost commented Aug 13, 2020

ghost commented Aug 13, 2020

Sergio0694 left a comment

Rosuavio left a comment •

edited

Loading

azchohfi commented Sep 15, 2020

azchohfi commented Sep 15, 2020

ghost commented Sep 15, 2020

Rosuavio left a comment •

edited

Loading

michael-hawker commented Sep 15, 2020

michael-hawker left a comment

azchohfi commented Sep 15, 2020

Replace Newtonsoft.Json with the new .Net JsonSerializer. #3414

Replace Newtonsoft.Json with the new .Net JsonSerializer. #3414

Conversation

azchohfi commented Aug 7, 2020 • edited Loading

Fixes #3060, #889, #2957

PR Type

What is the current behavior?

What is the new behavior?

PR Checklist

ghost commented Aug 7, 2020

azchohfi commented Aug 7, 2020

azchohfi commented Aug 8, 2020

Sergio0694 left a comment

Choose a reason for hiding this comment

ghost commented Aug 13, 2020

ghost commented Aug 13, 2020

Sergio0694 left a comment

Choose a reason for hiding this comment

Rosuavio left a comment • edited Loading

Choose a reason for hiding this comment

azchohfi commented Sep 15, 2020

azchohfi commented Sep 15, 2020

ghost commented Sep 15, 2020

Rosuavio left a comment • edited Loading

Choose a reason for hiding this comment

michael-hawker commented Sep 15, 2020

michael-hawker left a comment

Choose a reason for hiding this comment

azchohfi commented Sep 15, 2020

azchohfi commented Aug 7, 2020 •

edited

Loading

Rosuavio left a comment •

edited

Loading

Rosuavio left a comment •

edited

Loading