Add benchmarks for Utf8JsonReader.Get*

[jozkee]

Given that dotnet/runtime#38056 touches several Utf8JsonReader.Get* methods, I wanted to make sure that performance was not being affected and wrote these benchmarks.

The results of these benchmarks can be used as a reference for future changes.

cc @layomia @steveharter

[jozkee]

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.900 (1903/May2019Update/19H1)
Intel Core i7-9750H CPU 2.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-preview.8.20326.4
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.20.32602, CoreFX 5.0.20.32602), X64 RyuJIT
  Job-FKMNJY : .NET Core 5.0.0 (CoreCLR 5.0.20.32602, CoreFX 5.0.20.32602), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  

Method	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
GetByte	1,310.5 ns	1.48 ns	1.16 ns	1,310.3 ns	1,309.0 ns	1,312.9 ns	-	-	-	-
GetSByte	1,435.0 ns	6.18 ns	5.48 ns	1,432.7 ns	1,428.1 ns	1,444.4 ns	-	-	-	-
GetInt16	1,389.3 ns	9.32 ns	8.71 ns	1,384.5 ns	1,380.6 ns	1,403.7 ns	-	-	-	-
GetInt32	773.0 ns	1.37 ns	1.15 ns	772.7 ns	771.5 ns	775.2 ns	-	-	-	-
GetInt64	847.9 ns	6.26 ns	5.86 ns	844.0 ns	842.9 ns	859.0 ns	-	-	-	-
GetUInt16	1,335.8 ns	4.94 ns	4.38 ns	1,334.0 ns	1,330.5 ns	1,347.5 ns	-	-	-	-
GetUInt32	725.3 ns	1.68 ns	1.41 ns	725.2 ns	723.3 ns	728.7 ns	-	-	-	-
GetUInt64	796.1 ns	1.75 ns	1.37 ns	795.8 ns	794.5 ns	799.4 ns	-	-	-	-
GetSingle	4,037.1 ns	12.23 ns	11.44 ns	4,033.4 ns	4,024.2 ns	4,061.9 ns	-	-	-	-
GetDouble	3,955.5 ns	11.35 ns	9.48 ns	3,951.9 ns	3,944.7 ns	3,976.2 ns	-	-	-	-
GetDecimal	4,679.8 ns	11.93 ns	9.31 ns	4,676.6 ns	4,671.5 ns	4,703.5 ns	-	-	-	-
GetDateTime	3,591.4 ns	29.84 ns	26.45 ns	3,577.8 ns	3,563.1 ns	3,628.6 ns	-	-	-	-
GetDateTimeOffset	5,465.1 ns	35.54 ns	33.25 ns	5,444.1 ns	5,437.8 ns	5,522.2 ns	-	-	-	-
GetGuid	5,286.8 ns	41.58 ns	38.89 ns	5,265.9 ns	5,246.8 ns	5,368.5 ns	-	-	-	-
GetString	3,255.8 ns	21.96 ns	20.54 ns	3,265.1 ns	3,210.1 ns	3,275.6 ns	1.7753	-	-	11200 B

[layomia]

Thanks.

[layomia]

Suggested change

	private static readonly byte[] _jsonDoubleBytes = Encoding.UTF8.GetBytes(double.MaxValue.ToString("F"));
	private static readonly byte[] _jsonDoubleBytes = Encoding.UTF8.GetBytes(double.MaxValue.ToString());

This "F" format representation for this value is quite long, and it shows in the results for double compared to the other number types.

[jozkee]

My point was to get the perf. result in the worst case scenario, why we should not aim to that?

[layomia]

If we had to pick one, I think it's more valuable to benchmark the most common scenarios. A 312 digit number would be a rare occurrence in a JSON payload.

[layomia]

I think this string is too long. I think a string like the "The quick brown fox jumps over the lazy dog" pangram would do here, and be closer in length to strings founds in typical payloads.

[DrewScoggins]

Hmm, I think it might be useful to have both here. A longer string is going to give us protection against any bad scaling that might be introduced that we could miss with a shorter string.

[jozkee]

We could add

[Params(true, false)]
public bool UseLargeValue;

So we can test both scenarios, on true we use the MaxValue and on false we can use default.

[adamsitnik]

@jozkee do you know what part of the total time is spent for the creation of Utf8JsonReader, calling Read and calling GetByte()? What is the typical usage of reader.GetX() methods?

I am asking because the creation of the reader and call to reader.Read might be more expensive than the call to GetX which we are trying to measure here. If this is true, we should reconsider rewriting the benchmark to something like:

public sbyte ReadAndGetSBytes()
{
    var reader = new Utf8JsonReader(_jsonSByteBytes); // _jsonSByteBytes should contain more than 1 SByte (100 should be enough)
    sbyte result = default;
    while (reader.Read())
    {
        result += reader.GetSByte();
    }
    return result;
}

this should allow for removal of UseLargeValue and hence reducing the number of new benchmarks and the time it takes to run them by half. Please see https://github.com/dotnet/performance/blob/master/docs/microbenchmark-design-guidelines.md#Test-Cases for more

[jozkee]

Good point, for GetInt32, ~80% time is spent in the first two lines:

[Benchmark]
public void CreateAndReadInt32()
{
    var reader = new Utf8JsonReader(_jsonInt32Bytes);
    reader.Read();
}

Method	UseLargeValue	Mean	Error	StdDev	Median	Min	Max	Gen 0	Gen 1	Gen 2	Allocated
CreateAndReadInt32	False	39.95 ns	0.548 ns	0.512 ns	39.81 ns	39.42 ns	41.11 ns	-	-	-	-
GetInt32	False	46.41 ns	0.302 ns	0.283 ns	46.47 ns	45.90 ns	46.81 ns	-	-	-	-
CreateAndReadInt32	True	49.73 ns	0.211 ns	0.187 ns	49.71 ns	49.48 ns	50.02 ns	-	-	-	-
GetInt32	True	62.31 ns	0.647 ns	0.573 ns	62.42 ns	60.74 ns	63.15 ns	-	-	-	-

[jozkee]

FWIW I added ReadToEnd* benchmarks in order to measure how much time is spent in traversing the 100 elements of the json.

[jozkee]

Another approach that we could take to avoid having the ReadtoEnd* methods could be just call reader.GetInt32() over the same token a hundred times, that could remove the overhead of reader.Read().

[adamsitnik]

@jozkee thank you for adding more benchmarks to the performance repo and keeping the performance in mind when working on the BCL! With the recent changes, the benchmarks look much better. Please see my new comments. As soon as they are addressed we are ready to merge. Thanks!

[adamsitnik]

LGTM, thank you @jozkee !