Initial design doc for JsonSerialization using Source Generators #16

kevinwkt · 2020-07-16T20:49:02Z

This is a live design doc I'll keep on updating while getting feedback from the community.

layomia · 2020-07-17T13:23:00Z

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

+
+## Motivation
+
+There are comprehensive [documents](https://github.com/dotnet/designs/pull/113) detailing the needs and benefits of generating JSON serializers at compile time. Some of these benefits are improved startup time, reduction in private memory usage, and the most obvious, faster runtime for serialization/deserialization. After discussing some approaches and pros/cons of some of them we decided to go ahead and implement this feature using [Roslyn Source Generators](https://github.com/dotnet/roslyn/blob/master/docs/features/source-generators.cookbook.md). This document will outline the roadmap for the initial experiment and highlight actionable items for the base prototype.


The most prominent benefits here are improved startup time and the reduction of private memory usage on the first call to the serializer. These gains are potentially much larger than improvements in throughput (de)serialization - recall these early numbers shared by @steveharter

Method Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated

SerializeDefault 810.0 ns 7.65 ns 6.79 ns 807.9 ns 800.9 ns 820.5 ns 0.0903 - - 584 B

SerializeCodeGen 690.5 ns 2.13 ns 1.99 ns 690.7 ns 687.0 ns 692.9 ns 0.0914 - - 584 B

SerializeDefaultCold 876,899.9 ns 6,848.59 ns 5,718.88 ns 878,034.8 ns 868,050.4 ns 888,299.3 ns 10.6383 3.5461 - 67851 B

SerializeCodeGenCold 93,078.4 ns 843.89 ns 658.85 ns 93,149.7 ns 91,375.6 ns 93,891.3 ns 6.2870 2.9586 0.3698 39460 B

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

layomia · 2020-07-17T14:48:04Z

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

+This phase consists of taking the discovered types and recursively codegenerating the serialization methods.
+
+#### Proposed Approach
+The expected code generation has been already been [tackled](https://github.com/dotnet/runtimelab/compare/master...steveharter:ApiAdds) by @steveharter focusing mainly on performance gains and extendibility to the current codebase. This approach increases performance drastically in 2 different ways. The first would be during the first-time/warm-up performance for both CPU and memory by avoiding costly reflection to build up a Type metadata cache mentioned here. The second would be throughput improvement by avoiding the initial metadata-dictionary lookup on calls to the serializer by generating ```CreateObjectFunc```, ```SerializeFunc``` and ```DeserializeFunc``` when creating its ```ClassInfo```. 


We should discuss what happens when members are not public, especially when users indicate that they should be (de)serialized e.g. with [JsonConstructor], [JsonInclude] attribute?

We've held back on allowing non-public properties, fields, and ctors, but do allow non-public setters/getters for public properties due to user demand (dotnet/runtime#29743)

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

layomia · 2020-07-17T15:00:41Z

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

+
+For the most part the source code generation approach mentioned above solves this problem since the serialize and deserialize functions would live within the type class and once the serializer is initialized, the serializer can be used calling ```JsonSerializer.Serialize<Type>()``` or ```SourceGeneratedType.Serialize()```. In order to continue with this approach, an extension of the type class would be needed where the users would have to explicitly declare their types as partial classes for it to be extended by the source generator while the types they don’t own would be entirely created. 
+
+Even if this initialization isn't performed, we can always fallback to the slower status quo ```JsonSerializer``` methods.


We discussed having a way to still look up the type's metadata within the context e.g. the context having a dictionary/if-else branches to return the right JsonClassInfo. I think this will be how reflection-based callers of the serializer e.g. ASP.NET can take advantage of this feature given that they may not have enough context to call the more specific overloads

AraHaan · 2021-02-03T07:28:55Z

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

+
+```c#
+ // (Base Case) Codegen (De)Serialization ExampleLoginClassInfo. 
+[JsonSerializable] 


Looks like a good start, but cant we also be able to pass the SerializerOptions we want the source generator to use to this attribute as well too? Like if we know the json contains a lot of things where setting specific options would work, but the default ones would not work at all.

Yes, the generated metadata context class provides a ctor that will accept a JsonSerializerOptions instance which will be honored at runtime. See this as an example.

AraHaan · 2021-02-03T07:30:55Z

src/libraries/System.Text.Json/docs/JsonSerializationSourceGeneration.md

+        /// <summary>
+        /// Takes type as an argument and uses it to create a facade JsonClassInfo as TargetNameClassInfo.
+        /// </summary>
+        public JsonSerializableAttribute(Type type) { }


I would like a copy of the ctor here that would optionally take in JsonSerializerOptions to use with the source generator so that the generated source would use those specific options instead of possibly the default (in cases where wanting to (de)serialize the json might fail or have issues without those options explicitly being set.

Alternatively the options could be exposed as a settable property to avoid adding more ctors to this too.

I believe #16 (comment) addresses the concern here.

layomia · 2021-03-18T19:22:11Z

Merging this doc now. Will follow up with the latest thinking.

…2769) Transition to GC Unsafe mode on every MONO_RT_EXTERNAL_ONLY function in reflection.c In particular, fix mono_reflection_type_from_name which is used in https://github.com/xamarin/xamarin-android/blob/681887ebdbd192ce7ce1cd02221d4939599ba762/src/monodroid/jni/embedded-assemblies.cc#L350 Fixes stack traces like ``` 05-14 08:06:12.848 31274 31274 F DEBUG : #00 pc 00000b99 [vdso] (__kernel_vsyscall+9) 05-14 08:06:12.848 31274 31274 F DEBUG : #1 pc 0005ad68 /apex/com.android.runtime/lib/bionic/libc.so (syscall+40) (BuildId: 6e3a0180fa6637b68c0d181c343e6806) 05-14 08:06:12.848 31274 31274 F DEBUG : #2 pc 00076511 /apex/com.android.runtime/lib/bionic/libc.so (abort+209) (BuildId: 6e3a0180fa6637b68c0d181c343e6806) 05-14 08:06:12.848 31274 31274 F DEBUG : #3 pc 0002afcd /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::mono_log_handler(char const*, char const*, char const*, int, void*)+141) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1) 05-14 08:06:12.848 31274 31274 F DEBUG : #4 pc 00112c5d /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (eglib_log_adapter+141) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #5 pc 00020fdf /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_logv+175) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #6 pc 0002113a /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (monoeg_g_log+42) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #7 pc 00128892 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_transition_do_blocking+258) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #8 pc 0012a406 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_unbalanced_with_info+134) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #9 pc 0012a27e /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_threads_enter_gc_safe_region_internal+46) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #10 pc 000799a7 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_loader_lock+71) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #11 pc 000447a1 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_create_from_typedef+129) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #12 pc 0003c073 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_get_checked+99) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #13 pc 0003cc0f /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked_aux+735) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #14 pc 00037989 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_class_from_name_checked+73) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #15 pc 000cc5f4 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_internal+132) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #16 pc 000c9bce /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_get_type_with_rootimage+126) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #17 pc 000ca204 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (_mono_reflection_get_type_from_info+292) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #18 pc 000ca06e /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name_checked+334) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #19 pc 000c9f01 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonosgen-2.0.so (mono_reflection_type_from_name+49) (BuildId: b67e93dd750dafdd6f65f408b021b6a3a74868ac) 05-14 08:06:12.849 31274 31274 F DEBUG : #20 pc 0001b40b /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(char const*)+427) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1) 05-14 08:06:12.849 31274 31274 F DEBUG : #21 pc 0001b551 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::EmbeddedAssemblies::typemap_java_to_managed(_MonoString*)+113) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1) 05-14 08:06:12.849 31274 31274 F DEBUG : #22 pc 000211a7 /data/app/~~rMrkpKmVPaBpM5jKb8fPAg==/com.microsoft.maui-JfRo8RWSDJaNtJuBa0y7_Q==/lib/x86/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::typemap_java_to_managed(_MonoString*)+39) (BuildId: 9726f32ad5f8fa5e7c5762baf2f6e3294da41cc1) ```

Scan method body instructions for all opcodes which take a token operand. Create static dependency nodes for all tokens found. Rewrite the method body when writing output to update all tokens in it. I improved the tests to validate member removal as well. Implementation notes: - Brings in ILReader and ILOpcode form the type system - For now implemented ThrowHelper stub since the type system code uses this. What's missing: Scanning exception regions and reporting dependencies for tokens in them Rewriting exception regions and updating tokens in them I added MichalStrehovsky/iltrim#17 to track that work.

…(#63598) * Fix native frame unwind in syscall on arm64 for VS4Mac crash report. Add arm64 version of StepWithCompactNoEncoding for syscall leaf node wrappers that have compact encoding of 0. Fix ReadCompactEncodingRegister so it actually decrements the addr. Change StepWithCompactEncodingArm64 to match what MacOS libunwind does for framed and frameless stepping. arm64 can have frames with the same SP (but different IPs). Increment SP for this condition so createdump's unwind loop doesn't break out on the "SP not increasing" check and the frames are added to the thread frame list in the correct order. Add getting the unwind info for tail called functions like this: __ZL14PROCEndProcessPvji: 36630: f6 57 bd a9 stp x22, x21, [sp, #-48]! 36634: f4 4f 01 a9 stp x20, x19, [sp, #16] 36638: fd 7b 02 a9 stp x29, x30, [sp, #32] 3663c: fd 83 00 91 add x29, sp, #32 ... 367ac: e9 01 80 52 mov w9, #15 367b0: 7f 3e 02 71 cmp w19, #143 367b4: 20 01 88 1a csel w0, w9, w8, eq 367b8: 2e 00 00 94 bl _PROCAbort _TerminateProcess: -> 367bc: 22 00 80 52 mov w2, #1 367c0: 9c ff ff 17 b __ZL14PROCEndProcessPvji The IP (367bc) returns the (incorrect) frameless encoding with nothing on the stack (uses an incorrect LR to unwind). To fix this get the unwind info for PC -1 which points to PROCEndProcess with the correct unwind info. This matches how lldb unwinds this frame. Always address module segment to IP lookup list instead of checking the module regions. Strip pointer authentication bits on PC/LR.

# Local heap optimizations on Arm64 1. When not required to zero the allocated space for local heap (for sizes up to 64 bytes) - do not emit zeroing sequence. Instead do stack probing and adjust stack pointer: ```diff - stp xzr, xzr, [sp,#-16]! - stp xzr, xzr, [sp,#-16]! - stp xzr, xzr, [sp,#-16]! - stp xzr, xzr, [sp,#-16]! + ldr wzr, [sp],#-64 ``` 2. For sizes less than one `PAGE_SIZE` use `ldr wzr, [sp], #-amount` that does probing at `[sp]` and allocates the space at the same time. This saves one instruction for such local heap allocations: ```diff - ldr wzr, [sp] - sub sp, sp, #208 + ldr wzr, [sp],#-208 ``` Use `ldp tmpReg, xzr, [sp], #-amount` when the offset not encodable by post-index variant of `ldr`: ```diff - ldr wzr, [sp] - sub sp, sp, #512 + ldp x0, xzr, [sp],#-512 ``` 3. Allow non-loop zeroing (i.e. unrolled sequence) for sizes up to 128 bytes (i.e. up to `LCLHEAP_UNROLL_LIMIT`). This frees up two internal integer registers for such cases: ```diff - mov w11, #128 - ;; bbWeight=0.50 PerfScore 0.25 -G_M44913_IG19: ; gcrefRegs=00F9 {x0 x3 x4 x5 x6 x7}, byrefRegs=0000 {}, byref, isz stp xzr, xzr, [sp,#-16]! - subs x11, x11, #16 - bne G_M44913_IG19 + stp xzr, xzr, [sp,#-112]! + stp xzr, xzr, [sp,#16] + stp xzr, xzr, [sp,#32] + stp xzr, xzr, [sp,#48] + stp xzr, xzr, [sp,#64] + stp xzr, xzr, [sp,#80] + stp xzr, xzr, [sp,#96] ``` 4. Do zeroing in ascending order of the effective address: ```diff - mov w7, #96 -G_M49279_IG13: stp xzr, xzr, [sp,#-16]! - subs x7, x7, #16 - bne G_M49279_IG13 + stp xzr, xzr, [sp,#-80]! + stp xzr, xzr, [sp,#16] + stp xzr, xzr, [sp,#32] + stp xzr, xzr, [sp,#48] + stp xzr, xzr, [sp,#64] ``` In the example, the zeroing is done at `[initialSp-16], [initialSp-96], [initialSp-80], [initialSp-64], [initialSp-48], [initialSp-32]` addresses. The idea here is to allow a CPU to detect the sequential `memset` to `0` pattern and switch into write streaming mode.

* Initial implementation for contract customization fix build errors Move converter rooting to DefaultJsonTypeInfoResolver so that it can be used standalone Fix ConfigurationList.IsReadOnly Minor refactorings (#1) * Makes the following changes: * Move singleton initialization for DefaultTypeInfoResolver behind a static property. * Consolidate JsonSerializerContext & IJsonTypeInfoResolver values to a single field. * Move reflection fallback logic away from JsonSerializerContext and into JsonSerializerOptions * Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializerOptions.cs * remove testing of removed field Simplify the JsonTypeInfo.CreateObject implemenetation (#2) * Simplify the JsonTypeInfo.CreateObject implemenetation * Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs * Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.cs Co-authored-by: Krzysztof Wicher <[email protected]> Co-authored-by: Krzysztof Wicher <[email protected]> Tests and fixes for JsonTypeInfoKind.None TypeInfo type mismatch tests Allow setting NumberHandling on JsonTypeInfoKind.None test resolver returning wrong type of options JsonTypeInfo/JsonPropertyInfo mutability tests rename test file Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver (#3) * Move default converter rooting responsibility behind DefaultJsonTypeInfoResolver * address feedback Add simple test for using JsonTypeInfo<T> with APIs directly taking it fix and tests for untyped/typed CreateObject uncomment test cases, remove todo More tests and tiny fixes Add a JsonTypeInfoResolver.Combine test for JsonSerializerContext (#4) * Fix JsonTypeInfoResolver.Combine for JsonSerializerContext * Break up failing test Fix simple scenarios for combining contexts (#6) * Fix simple scenarios for combining contexts * feedback JsonSerializerContext combine test with different camel casing Remove unneeded virtual calls & branching when accessing Get & Set delegates (#7) JsonPropertyInfo tests everything minus ShouldSerialize & NumberHandling Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs Update src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs throw InvalidOperationException rather than ArgumentNullException for source gen when PropertyInfo.Name is assigned through JsonPropertyInfoValues tests for duplicated property names and JsonPropertyInfo.NumberHandling Add tests for NumberHandling and failing tests for ShouldSerialize disable the failing test and add extra checks disable remainder of the failing ShouldSerialize tests, fix working one Fix ShouldSerialize and IgnoreCondition interop Add failing tests for CreateObject + parametrized constructors Fix CreateObject support for JsonConstructor types (#10) * Fix CreateObject support for JsonConstructor types * address feedback Make contexts more combinator friendly (#9) * Make contexts more combinator friendly * remove converter cache * redesign test to account for JsonConstructorAttribute * Combine unit tests * address feedback * Add acceptance tests for DataContract attributes & Specified pattern (#11) * Add private field serialization acceptance test (#13) * tests, PR feedback (#14) * PR feedback and extra tests * Shorten class name, remove incorrect check (not true for polimorphic cases) * Make parameter matching for custom properties map property Name with parameter (#16) * Test static initialization with JsonTypeInfo (#17) * Fix test failures and proper fix this time (#18) * Fix test failures and proper fix this time * reinstate ActiveIssueAttribute * PR feedback and adjust couple of tests which don't set TypeInfoResolver * fix IAsyncEnumerable tests * Lock JsonSerializerOptions in JsonTypeInfo.EnsureConfigured() Co-authored-by: Eirik Tsarpalis <[email protected]> Co-authored-by: Eirik Tsarpalis <[email protected]>

* Support Arm64 "constructed" constants in SuperPMI asm diffs SuperPMI asm diffs tries to ignore constants that can change between multiple replays, such as addresses that the replay engine must generate and not simply hand back from the collected data. Often, addresses have associated relocations generated during replay. SuperPMI can use these relocations to adjust the constants to allow two replays to match. However, there are cases on Arm64 where an address both doesn't report a relocation and is "constructed" using multiple `mov`/`movk` instructions. One case is the `allocPgoInstrumentationBySchema()` API which returns a pointer to a PGO data buffer. An address within this buffer is constructed via a sequence such as: ``` mov x0, #63408 movk x0, #23602, lsl dotnet#16 movk x0, dotnet#606, lsl dotnet#32 ``` When SuperPMI replays this API, it constructs a new buffer and returns that pointer, which is used to construct various actual addresses that are generated as "constructed" constants, shown above. This change "de-constructs" the constants and looks them up in the replay address map. If base and diff match the mapped constants, there is no asm diff. * Fix 32-bit build I don't think we fully support 64-bit replay on 32-bit host, but this fix at least makes it possible for this case. * Support more general mov/movk sequence Allow JIT1 and JIT2 to have a different sequence of mov/movk[/movk[/movk]] that map to the same address in the address map. That is, the replay constant might require a different set of instructions (e.g., if a `movk` is missing because its constant is zero).

Initial design doc for JsonSerialization using Source Generators

f555c5a

kevinwkt requested review from jozkee, layomia and steveharter as code owners July 16, 2020 20:49

layomia reviewed Jul 17, 2020

View reviewed changes

kevinwkt added 4 commits July 17, 2020 14:33

Some Api changes to doc

615e6cb

Update design doc with new high level programming model

f8eae8b

Code changes to the design document

68ebe31

Port changes from one pager doc to design doc

8b0ebd8

jkotas added the area-JsonCodeGen label Jul 25, 2020

kevinwkt mentioned this pull request Jul 30, 2020

Create a benchmarking project using source generation feature #39

Open

safern changed the base branch from JsonCodeGen to feature/JsonCodeGen September 11, 2020 21:56

AraHaan reviewed Feb 3, 2021

View reviewed changes

layomia self-assigned this Feb 16, 2021

dlewis-arcontech mentioned this pull request Feb 19, 2021

WKS::gc_heap::make_unused_array seg fault #711

Closed

layomia merged commit 6d9a932 into dotnet:feature/JsonCodeGen Mar 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial design doc for JsonSerialization using Source Generators #16

Initial design doc for JsonSerialization using Source Generators #16

kevinwkt commented Jul 16, 2020

layomia Jul 17, 2020

layomia Jul 17, 2020 •

edited

Loading

layomia Jul 17, 2020

AraHaan Feb 3, 2021

layomia Feb 16, 2021

AraHaan Feb 3, 2021 •

edited

Loading

layomia Feb 16, 2021 •

edited

Loading

layomia commented Mar 18, 2021


		## Motivation

		There are comprehensive [documents](https://github.com/dotnet/designs/pull/113) detailing the needs and benefits of generating JSON serializers at compile time. Some of these benefits are improved startup time, reduction in private memory usage, and the most obvious, faster runtime for serialization/deserialization. After discussing some approaches and pros/cons of some of them we decided to go ahead and implement this feature using [Roslyn Source Generators](https://github.com/dotnet/roslyn/blob/master/docs/features/source-generators.cookbook.md). This document will outline the roadmap for the initial experiment and highlight actionable items for the base prototype.

Method	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
SerializeDefault	810.0 ns	7.65 ns	6.79 ns	807.9 ns	800.9 ns	820.5 ns	0.0903	-	-	584 B
SerializeCodeGen	690.5 ns	2.13 ns	1.99 ns	690.7 ns	687.0 ns	692.9 ns	0.0914	-	-	584 B
SerializeDefaultCold	876,899.9 ns	6,848.59 ns	5,718.88 ns	878,034.8 ns	868,050.4 ns	888,299.3 ns	10.6383	3.5461	-	67851 B
SerializeCodeGenCold	93,078.4 ns	843.89 ns	658.85 ns	93,149.7 ns	91,375.6 ns	93,891.3 ns	6.2870	2.9586	0.3698	39460 B


		For the most part the source code generation approach mentioned above solves this problem since the serialize and deserialize functions would live within the type class and once the serializer is initialized, the serializer can be used calling ```JsonSerializer.Serialize<Type>()``` or ```SourceGeneratedType.Serialize()```. In order to continue with this approach, an extension of the type class would be needed where the users would have to explicitly declare their types as partial classes for it to be extended by the source generator while the types they don’t own would be entirely created.

		Even if this initialization isn't performed, we can always fallback to the slower status quo ```JsonSerializer``` methods.

Initial design doc for JsonSerialization using Source Generators #16

Initial design doc for JsonSerialization using Source Generators #16

Conversation

kevinwkt commented Jul 16, 2020

layomia Jul 17, 2020

Choose a reason for hiding this comment

layomia Jul 17, 2020 • edited Loading

Choose a reason for hiding this comment

layomia Jul 17, 2020

Choose a reason for hiding this comment

AraHaan Feb 3, 2021

Choose a reason for hiding this comment

layomia Feb 16, 2021

Choose a reason for hiding this comment

AraHaan Feb 3, 2021 • edited Loading

Choose a reason for hiding this comment

layomia Feb 16, 2021 • edited Loading

Choose a reason for hiding this comment

layomia commented Mar 18, 2021

layomia Jul 17, 2020 •

edited

Loading

AraHaan Feb 3, 2021 •

edited

Loading

layomia Feb 16, 2021 •

edited

Loading